2024-09-18T13:48:44.884 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-18T13:48:44.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T13:48:45.039 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070 branch: wip-bharath11-testing-2024-09-18-1115-quincy description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/snaps-few-objects} email: skanta@redhat.com first_in_suite: false job_id: '7911070' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath11-testing-2024-09-18-1115-quincy ceph: conf: client: debug monc: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr2: false ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 paxos min: 10 paxos service trim min: 5 paxos trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(POOL_APP_NOT_ENABLED\) mon_bind_msgr2: false sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: enable_coprs: - ceph/el9 flavor: default sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath11-testing-2024-09-18-1115-quincy sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mon.d - mon.e - osd.0 - osd.1 - osd.2 - - mon.f - mon.g - mon.h - mon.i - mgr.x - osd.3 - osd.4 - osd.5 - client.0 seed: 4631 sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath11-testing-2024-09-18-1115-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSskIyXoDcfzvr8QaEoieurcwLDOdKDrlLq0XuF1nNGB4o8ZAg3sCTTPolWzycXOBvVtX9c4WsqoUCoGESjjGk= smithi084.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFU+1UwJLA6P6DbOnwn117EcqYfhQMkIVNSOfSOj3mb4kHq2jhp0KU/fTMD4tl82iwuGtyPiODyA9aWk5xnF2o0= tasks: - install: null - ceph: null - mon_thrash: revive_delay: 90 thrash_delay: 1 thrash_many: true - rados: clients: - client.0 objects: 50 op_weights: copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_09:13:00 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-18T13:48:45.039 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa; will attempt to use it 2024-09-18T13:48:45.040 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa/tasks 2024-09-18T13:48:45.040 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-18T13:48:45.042 INFO:teuthology.task.internal:Checking packages... 2024-09-18T13:48:45.063 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7061ea6ca50908d104e9f3979dc615e118d9e153' 2024-09-18T13:48:45.063 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-18T13:48:45.063 INFO:teuthology.packaging:ref: None 2024-09-18T13:48:45.063 INFO:teuthology.packaging:tag: None 2024-09-18T13:48:45.064 INFO:teuthology.packaging:branch: wip-bharath11-testing-2024-09-18-1115-quincy 2024-09-18T13:48:45.064 INFO:teuthology.packaging:sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T13:48:45.064 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath11-testing-2024-09-18-1115-quincy 2024-09-18T13:48:45.201 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1645.g7061ea6c 2024-09-18T13:48:45.202 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-18T13:48:45.209 INFO:teuthology.task.internal:no buildpackages task found 2024-09-18T13:48:45.209 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-18T13:48:45.215 INFO:teuthology.task.internal:Saving configuration 2024-09-18T13:48:45.230 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-18T13:48:45.237 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-18T13:48:45.261 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 13:43:41.283977', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSskIyXoDcfzvr8QaEoieurcwLDOdKDrlLq0XuF1nNGB4o8ZAg3sCTTPolWzycXOBvVtX9c4WsqoUCoGESjjGk='} 2024-09-18T13:48:45.285 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 13:43:41.285119', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFU+1UwJLA6P6DbOnwn117EcqYfhQMkIVNSOfSOj3mb4kHq2jhp0KU/fTMD4tl82iwuGtyPiODyA9aWk5xnF2o0='} 2024-09-18T13:48:45.285 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-18T13:48:45.292 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'] 2024-09-18T13:48:45.292 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2024-09-18T13:48:45.292 INFO:teuthology.run_tasks:Running task console_log... 2024-09-18T13:48:45.351 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7eff89c57ac0>, signals=[15]) 2024-09-18T13:48:45.351 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-18T13:48:45.358 INFO:teuthology.task.internal:Opening connections... 2024-09-18T13:48:45.358 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-09-18T13:48:45.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T13:48:45.433 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2024-09-18T13:48:45.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T13:48:45.510 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-18T13:48:45.517 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-09-18T13:48:45.535 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-09-18T13:48:45.535 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-09-18T13:48:45.593 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-09-18T13:48:45.593 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-09-18T13:48:45.593 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-09-18T13:48:45.593 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-09-18T13:48:45.593 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-09-18T13:48:45.593 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-09-18T13:48:45.593 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T13:48:45.594 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-09-18T13:48:45.594 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-09-18T13:48:45.594 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T13:48:45.594 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-09-18T13:48:45.594 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T13:48:45.594 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T13:48:45.594 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T13:48:45.594 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-09-18T13:48:45.610 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2024-09-18T13:48:45.629 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2024-09-18T13:48:45.629 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-09-18T13:48:45.687 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2024-09-18T13:48:45.687 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="9" 2024-09-18T13:48:45.687 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2024-09-18T13:48:45.687 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2024-09-18T13:48:45.687 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="9" 2024-09-18T13:48:45.687 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el9" 2024-09-18T13:48:45.688 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T13:48:45.688 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2024-09-18T13:48:45.688 INFO:teuthology.orchestra.run.smithi084.stdout:LOGO="fedora-logo-icon" 2024-09-18T13:48:45.688 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T13:48:45.688 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2024-09-18T13:48:45.688 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T13:48:45.688 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T13:48:45.688 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T13:48:45.688 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-09-18T13:48:45.710 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-18T13:48:45.726 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-18T13:48:45.813 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-18T13:48:45.813 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-09-18T13:48:45.816 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2024-09-18T13:48:45.833 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-18T13:48:45.838 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-18T13:48:45.838 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-09-18T13:48:45.873 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2024-09-18T13:48:45.976 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-18T13:48:46.016 INFO:teuthology.run_tasks:Running task kernel... 2024-09-18T13:48:46.048 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-18T13:48:46.048 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.f': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-18T13:48:46.048 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T13:48:46.049 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T13:48:46.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T13:48:46.066 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-09-18T13:48:46.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T13:48:46.067 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-09-18T13:48:46.124 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-503.el9.x86_64 2024-09-18T13:48:46.124 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-503.el9.x86_64 2024-09-18T13:48:46.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2024-09-18T13:48:46.126 INFO:teuthology.orchestra.run.smithi084.stdout:5.14.0-503.el9.x86_64 2024-09-18T13:48:46.126 INFO:teuthology.task.kernel:Running kernel on smithi084: 5.14.0-503.el9.x86_64 2024-09-18T13:48:46.126 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum install -y kernel 2024-09-18T13:48:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-18T13:48:49.312 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.3 MB 00:01 2024-09-18T13:48:50.098 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-18T13:48:52.555 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 9.5 MB/s | 20 MB 00:02 2024-09-18T13:48:54.607 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-18T13:48:56.349 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-18T13:48:57.778 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-18T13:48:58.113 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-18T13:48:59.563 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 19 kB 00:00 2024-09-18T13:49:00.759 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-18T13:49:03.484 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-18T13:49:05.466 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T13:49:05.466 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T13:49:05.556 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:49:05.560 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:49:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-09-18T13:49:05.560 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:49:05.560 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:49:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T13:49:05.560 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-18T13:49:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T13:49:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T13:49:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T13:49:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:49:05.561 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:49:05.561 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:49:05.561 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-09-18T13:49:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:49:05.561 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 79 M 2024-09-18T13:49:05.561 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 126 M 2024-09-18T13:49:05.562 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:49:05.935 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 284 kB/s | 36 kB 00:00 2024-09-18T13:49:06.352 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-18T13:49:06.364 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-18T13:49:06.653 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 40 MB/s | 29 MB 00:00 2024-09-18T13:49:07.111 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 27 MB/s | 35 MB 00:01 2024-09-18T13:49:07.111 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:49:07.112 INFO:teuthology.orchestra.run.smithi045.stdout:Total 51 MB/s | 79 MB 00:01 2024-09-18T13:49:07.826 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:49:08.207 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:49:08.207 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:49:08.254 INFO:teuthology.orchestra.run.smithi084.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T13:49:08.254 INFO:teuthology.orchestra.run.smithi084.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T13:49:08.361 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:49:08.367 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:49:08.367 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-09-18T13:49:08.368 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:49:08.368 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:49:08.368 INFO:teuthology.orchestra.run.smithi084.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T13:49:08.368 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-18T13:49:08.368 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T13:49:08.368 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T13:49:08.368 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T13:49:08.368 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:49:08.368 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:49:08.368 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:49:08.368 INFO:teuthology.orchestra.run.smithi084.stdout:Install 4 Packages 2024-09-18T13:49:08.368 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:49:08.369 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 79 M 2024-09-18T13:49:08.369 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 126 M 2024-09-18T13:49:08.369 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:49:08.799 INFO:teuthology.orchestra.run.smithi084.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 119 kB/s | 36 kB 00:00 2024-09-18T13:49:09.176 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:49:09.176 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:49:11.868 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:49:12.533 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T13:49:12.622 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T13:49:14.506 INFO:teuthology.orchestra.run.smithi084.stdout:(2/4): kernel-modules-core-5.14.0-508.el9.x86_6 5.0 MB/s | 29 MB 00:05 2024-09-18T13:49:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T13:49:15.507 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T13:49:19.780 INFO:teuthology.orchestra.run.smithi084.stdout:(3/4): kernel-core-5.14.0-508.el9.x86_64.rpm 1.4 MB/s | 16 MB 00:11 2024-09-18T13:49:20.632 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T13:49:20.674 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T13:49:21.697 INFO:teuthology.orchestra.run.smithi084.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 2.6 MB/s | 35 MB 00:13 2024-09-18T13:49:21.697 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:49:21.697 INFO:teuthology.orchestra.run.smithi084.stdout:Total 5.9 MB/s | 79 MB 00:13 2024-09-18T13:49:22.376 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:49:22.725 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:49:22.726 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:49:23.677 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:49:23.678 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:49:25.819 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T13:49:26.096 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:49:26.743 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T13:49:26.832 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T13:49:29.566 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T13:49:29.717 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T13:49:34.565 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T13:49:34.601 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T13:49:39.431 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T13:50:04.444 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T13:50:04.487 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T13:50:04.901 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T13:50:04.902 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T13:50:04.902 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T13:50:04.902 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T13:50:05.775 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T13:50:05.776 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:50:05.776 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:50:05.776 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T13:50:05.776 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T13:50:05.776 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T13:50:05.776 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T13:50:05.776 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:50:05.776 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:50:06.115 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T13:50:06.203 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T13:50:06.203 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T13:50:06.204 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T13:50:06.204 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T13:50:06.204 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T13:50:06.204 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-18T13:50:06.204 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T13:50:06.204 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T13:50:06.262 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T13:50:06.263 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T13:50:06.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-09-18T13:50:06.305 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-09-18T13:50:06.305 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 82.el9 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8274670 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-09-18T13:50:06.306 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T13:50:06.307 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-09-18T13:50:06.307 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-09-18T13:50:06.307 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:50:06.307 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T13:50:06.307 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T13:50:06.307 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T13:50:06.307 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-09-18T13:50:06.307 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:50:06.307 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T13:50:06.308 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T13:50:06.308 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 5.14.0-508.el9.x86_64 2024-09-18T13:50:06.308 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T13:50:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-09-18T13:50:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T13:50:08.418 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-09-18T13:50:08.421 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-18T13:50:08.436 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.iOwOWNYqOA 2024-09-18T13:50:08.436 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.iOwOWNYqOA 2024-09-18T13:50:08.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.iOwOWNYqOA 2024-09-18T13:50:08.650 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.iOwOWNYqOA is 6KB 2024-09-18T13:50:08.699 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.iOwOWNYqOA 2024-09-18T13:50:08.713 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T13:50:08.775 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T13:50:08.776 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T13:50:08.776 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T13:50:08.776 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T13:50:08.777 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T13:50:08.958 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-09-18T13:50:17.834 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T13:50:17.880 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T13:50:18.203 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T13:50:18.203 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T13:50:18.203 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T13:50:18.203 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T13:50:19.076 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T13:50:19.076 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:50:19.076 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:50:19.076 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T13:50:19.077 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T13:50:19.077 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T13:50:19.077 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T13:50:19.077 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:50:19.077 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:50:19.456 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T13:50:19.545 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T13:50:19.545 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T13:50:19.545 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T13:50:19.546 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T13:50:19.546 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T13:50:19.546 INFO:teuthology.task.kernel:Installing distro kernel on mon.f... 2024-09-18T13:50:19.546 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi084.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T13:50:19.546 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T13:50:19.605 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T13:50:19.605 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T13:50:19.606 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -qi grub2-tools 2024-09-18T13:50:19.637 INFO:teuthology.orchestra.run.smithi084.stdout:Name : grub2-tools 2024-09-18T13:50:19.637 INFO:teuthology.orchestra.run.smithi084.stdout:Epoch : 1 2024-09-18T13:50:19.637 INFO:teuthology.orchestra.run.smithi084.stdout:Version : 2.06 2024-09-18T13:50:19.637 INFO:teuthology.orchestra.run.smithi084.stdout:Release : 82.el9 2024-09-18T13:50:19.637 INFO:teuthology.orchestra.run.smithi084.stdout:Architecture: x86_64 2024-09-18T13:50:19.637 INFO:teuthology.orchestra.run.smithi084.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T13:50:19.637 INFO:teuthology.orchestra.run.smithi084.stdout:Group : Unspecified 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout:Size : 8274670 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout:License : GPLv3+ 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout:Packager : builder@centos.org 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout:Vendor : CentOS 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout:Summary : Support tools for GRUB. 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout:Description : 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:50:19.638 INFO:teuthology.orchestra.run.smithi084.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T13:50:19.639 INFO:teuthology.orchestra.run.smithi084.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T13:50:19.639 INFO:teuthology.orchestra.run.smithi084.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T13:50:19.639 INFO:teuthology.orchestra.run.smithi084.stdout:hardware devices. 2024-09-18T13:50:19.639 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:50:19.639 INFO:teuthology.orchestra.run.smithi084.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T13:50:19.640 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T13:50:19.640 INFO:teuthology.task.kernel:Updating grub on smithi084 to boot 5.14.0-508.el9.x86_64 2024-09-18T13:50:19.640 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T13:50:20.294 INFO:teuthology.orchestra.run.smithi084.stderr:Generating grub configuration file ... 2024-09-18T13:50:21.627 INFO:teuthology.orchestra.run.smithi084.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T13:50:21.656 INFO:teuthology.orchestra.run.smithi084.stderr:done 2024-09-18T13:50:21.658 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2024-09-18T13:50:21.674 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.1Wz2FTyQVD 2024-09-18T13:50:21.674 DEBUG:teuthology.orchestra.run.smithi084:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1Wz2FTyQVD 2024-09-18T13:50:21.741 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.1Wz2FTyQVD 2024-09-18T13:50:21.886 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.1Wz2FTyQVD is 6KB 2024-09-18T13:50:21.935 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.1Wz2FTyQVD 2024-09-18T13:50:21.950 DEBUG:teuthology.orchestra.run.smithi084:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T13:50:22.016 INFO:teuthology.orchestra.run.smithi084.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T13:50:22.017 INFO:teuthology.orchestra.run.smithi084.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T13:50:22.017 INFO:teuthology.orchestra.run.smithi084.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T13:50:22.017 INFO:teuthology.orchestra.run.smithi084.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T13:50:22.018 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T13:50:22.165 DEBUG:teuthology.orchestra.run.smithi084:> sudo shutdown -r now 2024-09-18T13:50:38.977 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-18T13:50:38.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-18T13:50:38.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T13:50:52.181 INFO:teuthology.task.kernel:Checking client mon.f for new kernel version... 2024-09-18T13:50:52.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-18T13:50:52.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T13:50:57.341 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-09-18T13:51:06.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-18T13:51:06.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T13:51:10.653 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-09-18T13:51:19.661 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-18T13:51:19.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T13:51:22.717 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-09-18T13:51:34.729 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-18T13:51:34.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T13:51:53.149 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-09-18T13:52:06.353 DEBUG:teuthology.orchestra.remote:timed out 2024-09-18T13:52:08.152 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-18T13:52:08.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T13:52:08.556 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-09-18T13:52:08.951 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-18T13:52:08.951 INFO:teuthology.task.kernel:Checking kernel version of mon.f, want "5.14.0-508.el9.x86_64"... 2024-09-18T13:52:08.951 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-09-18T13:52:08.966 INFO:teuthology.orchestra.run.smithi084.stdout:5.14.0-508.el9.x86_64 2024-09-18T13:52:08.966 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T13:52:08.966 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T13:52:08.966 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T13:52:09.967 INFO:teuthology.task.kernel:Enabling kdb on mon.f... 2024-09-18T13:52:09.967 DEBUG:teuthology.orchestra.run.smithi084:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T13:52:10.097 INFO:teuthology.orchestra.run.smithi084.stdout:ttyS1 2024-09-18T13:52:10.119 DEBUG:teuthology.parallel:result is None 2024-09-18T13:52:18.354 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-18T13:52:18.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T13:52:18.647 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-09-18T13:52:19.018 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-18T13:52:19.018 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-18T13:52:19.018 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-09-18T13:52:19.035 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-508.el9.x86_64 2024-09-18T13:52:19.035 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T13:52:19.035 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T13:52:19.035 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T13:52:20.036 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-18T13:52:20.037 DEBUG:teuthology.orchestra.run.smithi045:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T13:52:20.118 INFO:teuthology.orchestra.run.smithi045.stdout:ttyS1 2024-09-18T13:52:20.141 DEBUG:teuthology.parallel:result is None 2024-09-18T13:52:20.142 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-18T13:52:20.148 INFO:teuthology.task.internal:Creating test directory... 2024-09-18T13:52:20.149 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T13:52:20.151 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T13:52:20.170 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-18T13:52:20.177 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-18T13:52:20.187 INFO:teuthology.task.internal:Creating archive directory... 2024-09-18T13:52:20.187 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T13:52:20.210 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T13:52:20.243 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-18T13:52:20.249 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-18T13:52:20.249 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T13:52:20.279 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T13:52:20.311 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T13:52:20.313 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T13:52:20.322 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T13:52:20.325 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T13:52:20.327 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-18T13:52:20.334 INFO:teuthology.task.internal:Configuring sudo... 2024-09-18T13:52:20.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T13:52:20.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T13:52:20.396 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-18T13:52:20.406 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-18T13:52:20.406 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T13:52:20.438 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T13:52:20.454 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T13:52:20.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T13:52:20.593 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T13:52:20.671 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T13:52:20.719 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:52:20.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T13:52:20.786 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T13:52:20.824 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T13:52:20.880 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T13:52:20.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T13:52:21.007 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:52:21.007 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T13:52:21.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-09-18T13:52:21.076 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2024-09-18T13:52:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T13:52:21.172 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T13:52:21.592 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-18T13:52:21.600 INFO:teuthology.task.internal:Starting timer... 2024-09-18T13:52:21.600 INFO:teuthology.run_tasks:Running task pcp... 2024-09-18T13:52:21.611 INFO:teuthology.run_tasks:Running task selinux... 2024-09-18T13:52:21.620 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-09-18T13:52:21.684 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-09-18T13:52:21.686 DEBUG:teuthology.orchestra.run.smithi084:> sudo service auditd rotate 2024-09-18T13:52:21.730 INFO:teuthology.orchestra.run.smithi084.stdout:Rotating logs: 2024-09-18T13:52:21.732 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-18T13:52:21.732 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-09-18T13:52:21.752 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-09-18T13:52:21.752 DEBUG:teuthology.orchestra.run.smithi084:> /usr/sbin/getenforce 2024-09-18T13:52:21.772 INFO:teuthology.orchestra.run.smithi084.stdout:Permissive 2024-09-18T13:52:21.773 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi084.front.sepia.ceph.com': 'permissive'} 2024-09-18T13:52:21.773 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-18T13:52:21.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T13:52:21.822 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-18T13:52:21.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T13:52:21.849 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-18T13:52:21.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-09-18T13:52:21.899 DEBUG:teuthology.orchestra.run.smithi084:> sudo /usr/sbin/setenforce permissive 2024-09-18T13:52:21.933 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-18T13:52:21.942 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-18T13:52:21.953 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-18T13:52:21.954 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi084.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-18T13:56:07.956 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi084.front.sepia.ceph.com')] 2024-09-18T13:56:07.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-18T13:56:07.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T13:56:08.036 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-09-18T13:56:08.112 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-18T13:56:08.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-18T13:56:08.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T13:56:08.190 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-09-18T13:56:08.269 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-18T13:56:08.269 INFO:teuthology.run_tasks:Running task clock... 2024-09-18T13:56:08.280 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-18T13:56:08.280 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T13:56:08.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T13:56:08.283 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T13:56:08.283 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T13:56:08.311 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T13:56:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T13:56:08.340 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T13:56:08.351 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-09-18T13:56:08.355 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T13:56:08.363 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-09-18T13:56:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T13:56:08.379 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: ntpd: command not found 2024-09-18T13:56:08.391 INFO:teuthology.orchestra.run.smithi084.stdout:506 Cannot talk to daemon 2024-09-18T13:56:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T13:56:08.405 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T13:56:08.419 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T13:56:08.447 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-09-18T13:56:08.450 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T13:56:08.450 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-09-18T13:56:08.450 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T13:56:08.450 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T13:56:08.450 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T13:56:08.450 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T13:56:08.472 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-09-18T13:56:08.476 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T13:56:08.476 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-09-18T13:56:08.476 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T13:56:08.476 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T13:56:08.476 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T13:56:08.476 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T13:56:08.477 INFO:teuthology.run_tasks:Running task install... 2024-09-18T13:56:08.485 DEBUG:teuthology.task.install:project ceph 2024-09-18T13:56:08.485 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}} 2024-09-18T13:56:08.485 DEBUG:teuthology.task.install:config {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'} 2024-09-18T13:56:08.485 INFO:teuthology.task.install:Using flavor: default 2024-09-18T13:56:08.490 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-18T13:56:08.491 INFO:teuthology.task.install:extra packages: [] 2024-09-18T13:56:08.491 DEBUG:teuthology.orchestra.run.smithi045:> sudo dnf -y install 'dnf-command(copr)' 2024-09-18T13:56:08.491 DEBUG:teuthology.orchestra.run.smithi084:> sudo dnf -y install 'dnf-command(copr)' 2024-09-18T13:56:09.088 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:48 ago on Wed 18 Sep 2024 01:55:20 PM UTC. 2024-09-18T13:56:09.089 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:44 ago on Wed 18 Sep 2024 01:55:24 PM UTC. 2024-09-18T13:56:09.173 INFO:teuthology.orchestra.run.smithi084.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-18T13:56:09.178 INFO:teuthology.orchestra.run.smithi045.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-18T13:56:09.216 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:56:09.217 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-18T13:56:09.217 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:56:09.222 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:56:09.224 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-18T13:56:09.224 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:56:09.283 DEBUG:teuthology.orchestra.run.smithi045:> sudo dnf -y copr enable ceph/el9 2024-09-18T13:56:09.391 DEBUG:teuthology.orchestra.run.smithi084:> sudo dnf -y copr enable ceph/el9 2024-09-18T13:56:09.462 INFO:teuthology.orchestra.run.smithi045.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-18T13:56:09.462 INFO:teuthology.orchestra.run.smithi045.stderr:of the main distribution, and quality may vary. 2024-09-18T13:56:09.462 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:56:09.463 INFO:teuthology.orchestra.run.smithi045.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-18T13:56:09.463 INFO:teuthology.orchestra.run.smithi045.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-18T13:56:09.463 INFO:teuthology.orchestra.run.smithi045.stderr:, 2024-09-18T13:56:09.463 INFO:teuthology.orchestra.run.smithi045.stderr:and packages are not held to any quality or security level. 2024-09-18T13:56:09.463 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:56:09.463 INFO:teuthology.orchestra.run.smithi045.stderr:Please do not file bug reports about these packages in Fedora 2024-09-18T13:56:09.463 INFO:teuthology.orchestra.run.smithi045.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-18T13:56:09.517 INFO:teuthology.orchestra.run.smithi045.stdout:Repository successfully enabled. 2024-09-18T13:56:09.553 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'tag': None, 'wait_for_package': False} 2024-09-18T13:56:09.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T13:56:09.561 INFO:teuthology.orchestra.run.smithi084.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-18T13:56:09.561 INFO:teuthology.orchestra.run.smithi084.stderr:of the main distribution, and quality may vary. 2024-09-18T13:56:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:56:09.561 INFO:teuthology.orchestra.run.smithi084.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-18T13:56:09.561 INFO:teuthology.orchestra.run.smithi084.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-18T13:56:09.561 INFO:teuthology.orchestra.run.smithi084.stderr:, 2024-09-18T13:56:09.561 INFO:teuthology.orchestra.run.smithi084.stderr:and packages are not held to any quality or security level. 2024-09-18T13:56:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:56:09.561 INFO:teuthology.orchestra.run.smithi084.stderr:Please do not file bug reports about these packages in Fedora 2024-09-18T13:56:09.561 INFO:teuthology.orchestra.run.smithi084.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-18T13:56:09.612 INFO:teuthology.orchestra.run.smithi084.stdout:Repository successfully enabled. 2024-09-18T13:56:09.647 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'tag': None, 'wait_for_package': False} 2024-09-18T13:56:09.647 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T13:56:09.704 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/ 2024-09-18T13:56:09.705 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1645.g7061ea6c 2024-09-18T13:56:09.788 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/ 2024-09-18T13:56:09.789 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1645.g7061ea6c 2024-09-18T13:56:09.815 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T13:56:09.816 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:56:09.816 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T13:56:09.851 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-18T13:56:09.852 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7061ea6ca50908d104e9f3979dc615e118d9e153/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T13:56:09.901 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T13:56:09.901 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:56:09.901 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T13:56:09.918 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T13:56:09.926 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-18T13:56:09.926 DEBUG:teuthology.orchestra.run.smithi084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7061ea6ca50908d104e9f3979dc615e118d9e153/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T13:56:09.992 DEBUG:teuthology.orchestra.run.smithi084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T13:56:09.995 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T13:56:10.061 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-09-18T13:56:10.063 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-09-18T13:56:10.067 DEBUG:teuthology.orchestra.run.smithi084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T13:56:10.131 INFO:teuthology.orchestra.run.smithi084.stdout:check_obsoletes = 1 2024-09-18T13:56:10.133 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum clean all 2024-09-18T13:56:10.298 INFO:teuthology.orchestra.run.smithi045.stdout:45 files removed 2024-09-18T13:56:10.325 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw 2024-09-18T13:56:10.375 INFO:teuthology.orchestra.run.smithi084.stdout:45 files removed 2024-09-18T13:56:10.402 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-radosgw 2024-09-18T13:56:10.901 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 279 kB/s | 77 kB 00:00 2024-09-18T13:56:10.967 INFO:teuthology.orchestra.run.smithi084.stdout:ceph packages for x86_64 274 kB/s | 77 kB 00:00 2024-09-18T13:56:11.177 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 47 kB/s | 11 kB 00:00 2024-09-18T13:56:11.251 INFO:teuthology.orchestra.run.smithi084.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-18T13:56:11.444 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 7.8 kB/s | 1.8 kB 00:00 2024-09-18T13:56:11.526 INFO:teuthology.orchestra.run.smithi084.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-18T13:56:11.603 INFO:teuthology.orchestra.run.smithi045.stdout:Copr repo for el9 owned by ceph 63 kB/s | 7.6 kB 00:00 2024-09-18T13:56:11.686 INFO:teuthology.orchestra.run.smithi084.stdout:Copr repo for el9 owned by ceph 62 kB/s | 7.6 kB 00:00 2024-09-18T13:56:13.320 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.3 MB 00:01 2024-09-18T13:56:13.369 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.3 MB 00:01 2024-09-18T13:56:15.204 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-18T13:56:18.796 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 4.7 MB/s | 20 MB 00:04 2024-09-18T13:56:20.404 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.5 MB 00:01 2024-09-18T13:56:22.013 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 97 kB/s | 19 kB 00:00 2024-09-18T13:56:22.892 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-18T13:56:23.347 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-18T13:56:25.006 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 19 kB 00:00 2024-09-18T13:56:26.394 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-18T13:56:28.356 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-18T13:56:30.656 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:56:30.677 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:56:30.677 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-18T13:56:30.677 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:56:30.677 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:56:30.678 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 14 M 2024-09-18T13:56:30.678 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2024-09-18T13:56:30.678 INFO:teuthology.orchestra.run.smithi084.stdout: librados2 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 3.1 M 2024-09-18T13:56:30.678 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 2.9 M 2024-09-18T13:56:30.678 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-18T13:56:30.678 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 5.0 M 2024-09-18T13:56:30.678 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 22 M 2024-09-18T13:56:30.678 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 25 k 2024-09-18T13:56:30.678 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T13:56:30.678 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 637 k 2024-09-18T13:56:30.678 INFO:teuthology.orchestra.run.smithi084.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T13:56:30.678 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 458 k 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 4.9 M 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 45 k 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 120 k 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 156 k 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 320 k 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 296 k 2024-09-18T13:56:30.679 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 99 k 2024-09-18T13:56:30.680 INFO:teuthology.orchestra.run.smithi084.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T13:56:30.680 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:56:30.680 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:56:30.680 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:56:30.680 INFO:teuthology.orchestra.run.smithi084.stdout:Install 22 Packages 2024-09-18T13:56:30.680 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 2 Packages 2024-09-18T13:56:30.680 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:56:30.681 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 58 M 2024-09-18T13:56:30.681 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:56:31.248 INFO:teuthology.orchestra.run.smithi084.stdout:(1/24): ceph-base-17.2.7-1645.g7061ea6c.el9.x86 10 MB/s | 5.0 MB 00:00 2024-09-18T13:56:31.316 INFO:teuthology.orchestra.run.smithi084.stdout:(2/24): ceph-selinux-17.2.7-1645.g7061ea6c.el9. 378 kB/s | 25 kB 00:00 2024-09-18T13:56:31.416 INFO:teuthology.orchestra.run.smithi084.stdout:(3/24): libcephfs2-17.2.7-1645.g7061ea6c.el9.x8 6.2 MB/s | 637 kB 00:00 2024-09-18T13:56:31.524 INFO:teuthology.orchestra.run.smithi084.stdout:(4/24): libradosstriper1-17.2.7-1645.g7061ea6c. 4.1 MB/s | 458 kB 00:00 2024-09-18T13:56:31.716 INFO:teuthology.orchestra.run.smithi084.stdout:(5/24): ceph-radosgw-17.2.7-1645.g7061ea6c.el9. 15 MB/s | 14 MB 00:00 2024-09-18T13:56:31.766 INFO:teuthology.orchestra.run.smithi084.stdout:(6/24): python3-ceph-argparse-17.2.7-1645.g7061 911 kB/s | 45 kB 00:00 2024-09-18T13:56:31.850 INFO:teuthology.orchestra.run.smithi084.stdout:(7/24): librgw2-17.2.7-1645.g7061ea6c.el9.x86_6 15 MB/s | 4.9 MB 00:00 2024-09-18T13:56:31.875 INFO:teuthology.orchestra.run.smithi084.stdout:(8/24): python3-ceph-common-17.2.7-1645.g7061ea 1.1 MB/s | 120 kB 00:00 2024-09-18T13:56:31.900 INFO:teuthology.orchestra.run.smithi084.stdout:(9/24): python3-cephfs-17.2.7-1645.g7061ea6c.el 3.0 MB/s | 156 kB 00:00 2024-09-18T13:56:31.925 INFO:teuthology.orchestra.run.smithi084.stdout:(10/24): python3-rados-17.2.7-1645.g7061ea6c.el 6.2 MB/s | 320 kB 00:00 2024-09-18T13:56:31.951 INFO:teuthology.orchestra.run.smithi084.stdout:(11/24): python3-rbd-17.2.7-1645.g7061ea6c.el9. 5.8 MB/s | 296 kB 00:00 2024-09-18T13:56:31.976 INFO:teuthology.orchestra.run.smithi084.stdout:(12/24): python3-rgw-17.2.7-1645.g7061ea6c.el9. 1.9 MB/s | 99 kB 00:00 2024-09-18T13:56:32.050 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-09-18T13:56:32.176 INFO:teuthology.orchestra.run.smithi084.stdout:(13/24): ceph-common-17.2.7-1645.g7061ea6c.el9. 16 MB/s | 22 MB 00:01 2024-09-18T13:56:32.218 INFO:teuthology.orchestra.run.smithi084.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 599 kB/s | 160 kB 00:00 2024-09-18T13:56:32.252 INFO:teuthology.orchestra.run.smithi084.stdout:(15/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 164 kB/s | 45 kB 00:00 2024-09-18T13:56:32.327 INFO:teuthology.orchestra.run.smithi084.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 565 kB/s | 42 kB 00:00 2024-09-18T13:56:32.352 INFO:teuthology.orchestra.run.smithi084.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-18T13:56:32.386 INFO:teuthology.orchestra.run.smithi084.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-09-18T13:56:32.436 INFO:teuthology.orchestra.run.smithi084.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-09-18T13:56:32.461 INFO:teuthology.orchestra.run.smithi084.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-18T13:56:32.487 INFO:teuthology.orchestra.run.smithi084.stdout:(21/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.9 MB/s | 308 kB 00:00 2024-09-18T13:56:32.612 INFO:teuthology.orchestra.run.smithi084.stdout:(22/24): librados2-17.2.7-1645.g7061ea6c.el9.x8 21 MB/s | 3.1 MB 00:00 2024-09-18T13:56:32.662 INFO:teuthology.orchestra.run.smithi084.stdout:(23/24): thrift-0.15.0-2.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-09-18T13:56:32.737 INFO:teuthology.orchestra.run.smithi084.stdout:(24/24): librbd1-17.2.7-1645.g7061ea6c.el9.x86_ 12 MB/s | 2.9 MB 00:00 2024-09-18T13:56:32.742 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:56:32.742 INFO:teuthology.orchestra.run.smithi084.stdout:Total 28 MB/s | 58 MB 00:02 2024-09-18T13:56:32.804 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:56:32.836 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:56:32.836 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:56:33.500 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:56:33.500 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:56:34.446 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:56:34.448 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:56:34.448 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 14 M 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 3.1 M 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 2.9 M 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 5.0 M 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 22 M 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 25 k 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 637 k 2024-09-18T13:56:34.449 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T13:56:34.450 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T13:56:34.450 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T13:56:34.450 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 458 k 2024-09-18T13:56:34.450 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T13:56:34.450 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 4.9 M 2024-09-18T13:56:34.450 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T13:56:34.450 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T13:56:34.450 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 45 k 2024-09-18T13:56:34.450 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 120 k 2024-09-18T13:56:34.450 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 156 k 2024-09-18T13:56:34.450 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T13:56:34.450 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 320 k 2024-09-18T13:56:34.451 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 296 k 2024-09-18T13:56:34.451 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 99 k 2024-09-18T13:56:34.451 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T13:56:34.451 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:56:34.451 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:56:34.451 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:56:34.451 INFO:teuthology.orchestra.run.smithi045.stdout:Install 22 Packages 2024-09-18T13:56:34.451 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 2 Packages 2024-09-18T13:56:34.451 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:56:34.452 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 58 M 2024-09-18T13:56:34.452 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:56:34.806 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:56:35.160 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-18T13:56:35.264 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/26 2024-09-18T13:56:35.357 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/26 2024-09-18T13:56:35.435 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/26 2024-09-18T13:56:35.457 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/26 2024-09-18T13:56:35.546 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/26 2024-09-18T13:56:35.592 INFO:teuthology.orchestra.run.smithi045.stdout:(1/24): ceph-base-17.2.7-1645.g7061ea6c.el9.x86 9.6 MB/s | 5.0 MB 00:00 2024-09-18T13:56:35.610 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-18T13:56:35.637 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-18T13:56:35.651 INFO:teuthology.orchestra.run.smithi045.stdout:(2/24): ceph-selinux-17.2.7-1645.g7061ea6c.el9. 432 kB/s | 25 kB 00:00 2024-09-18T13:56:35.726 INFO:teuthology.orchestra.run.smithi045.stdout:(3/24): libcephfs2-17.2.7-1645.g7061ea6c.el9.x8 8.3 MB/s | 637 kB 00:00 2024-09-18T13:56:35.810 INFO:teuthology.orchestra.run.smithi045.stdout:(4/24): libradosstriper1-17.2.7-1645.g7061ea6c. 5.4 MB/s | 458 kB 00:00 2024-09-18T13:56:36.036 INFO:teuthology.orchestra.run.smithi045.stdout:(5/24): ceph-radosgw-17.2.7-1645.g7061ea6c.el9. 15 MB/s | 14 MB 00:00 2024-09-18T13:56:36.077 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-18T13:56:36.078 INFO:teuthology.orchestra.run.smithi045.stdout:(6/24): python3-ceph-argparse-17.2.7-1645.g7061 1.1 MB/s | 45 kB 00:00 2024-09-18T13:56:36.102 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 8/26 2024-09-18T13:56:36.148 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 8/26 2024-09-18T13:56:36.152 INFO:teuthology.orchestra.run.smithi045.stdout:(7/24): librgw2-17.2.7-1645.g7061ea6c.el9.x86_6 14 MB/s | 4.9 MB 00:00 2024-09-18T13:56:36.178 INFO:teuthology.orchestra.run.smithi045.stdout:(8/24): python3-ceph-common-17.2.7-1645.g7061ea 1.2 MB/s | 120 kB 00:00 2024-09-18T13:56:36.184 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-18T13:56:36.203 INFO:teuthology.orchestra.run.smithi045.stdout:(9/24): python3-cephfs-17.2.7-1645.g7061ea6c.el 3.0 MB/s | 156 kB 00:00 2024-09-18T13:56:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-argparse-2:17.2.7-1645.g7061ea6c.el9. 10/26 2024-09-18T13:56:36.228 INFO:teuthology.orchestra.run.smithi045.stdout:(10/24): python3-rados-17.2.7-1645.g7061ea6c.el 6.2 MB/s | 320 kB 00:00 2024-09-18T13:56:36.254 INFO:teuthology.orchestra.run.smithi045.stdout:(11/24): python3-rbd-17.2.7-1645.g7061ea6c.el9. 5.8 MB/s | 296 kB 00:00 2024-09-18T13:56:36.264 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 11/26 2024-09-18T13:56:36.279 INFO:teuthology.orchestra.run.smithi045.stdout:(12/24): python3-rgw-17.2.7-1645.g7061ea6c.el9. 1.9 MB/s | 99 kB 00:00 2024-09-18T13:56:36.326 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 12/26 2024-09-18T13:56:36.354 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_6 13/26 2024-09-18T13:56:36.420 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_6 13/26 2024-09-18T13:56:36.476 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-18T13:56:36.513 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-18T13:56:36.521 INFO:teuthology.orchestra.run.smithi045.stdout:(13/24): ceph-common-17.2.7-1645.g7061ea6c.el9. 16 MB/s | 22 MB 00:01 2024-09-18T13:56:36.554 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-18T13:56:36.848 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-18T13:56:36.891 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 18/26 2024-09-18T13:56:36.981 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 18/26 2024-09-18T13:56:37.063 INFO:teuthology.orchestra.run.smithi045.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 58 kB/s | 45 kB 00:00 2024-09-18T13:56:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 19/26 2024-09-18T13:56:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-common-2:17.2.7-1645.g7061ea6c.el9.x8 20/26 2024-09-18T13:56:37.155 INFO:teuthology.orchestra.run.smithi045.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 178 kB/s | 160 kB 00:00 2024-09-18T13:56:37.380 INFO:teuthology.orchestra.run.smithi045.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 771 kB/s | 662 kB 00:00 2024-09-18T13:56:37.439 INFO:teuthology.orchestra.run.smithi045.stdout:(17/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 5.2 MB/s | 308 kB 00:00 2024-09-18T13:56:37.464 INFO:teuthology.orchestra.run.smithi045.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-18T13:56:37.490 INFO:teuthology.orchestra.run.smithi045.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-18T13:56:37.582 INFO:teuthology.orchestra.run.smithi045.stdout:(20/24): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-18T13:56:37.832 INFO:teuthology.orchestra.run.smithi045.stdout:(21/24): librados2-17.2.7-1645.g7061ea6c.el9.x8 12 MB/s | 3.1 MB 00:00 2024-09-18T13:56:37.857 INFO:teuthology.orchestra.run.smithi045.stdout:(22/24): python3-prettytable-0.7.2-27.el9.noarc 60 kB/s | 42 kB 00:00 2024-09-18T13:56:37.882 INFO:teuthology.orchestra.run.smithi045.stdout:(23/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 357 kB/s | 292 kB 00:00 2024-09-18T13:56:37.974 INFO:teuthology.orchestra.run.smithi045.stdout:(24/24): librbd1-17.2.7-1645.g7061ea6c.el9.x86_ 21 MB/s | 2.9 MB 00:00 2024-09-18T13:56:37.979 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:56:37.979 INFO:teuthology.orchestra.run.smithi045.stdout:Total 16 MB/s | 58 MB 00:03 2024-09-18T13:56:38.047 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:56:38.086 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:56:38.086 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:56:38.746 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:56:38.746 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:56:39.577 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 21/26 2024-09-18T13:56:39.615 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 21/26 2024-09-18T13:56:40.004 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:56:40.141 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 21/26 2024-09-18T13:56:40.176 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 22/26 2024-09-18T13:56:40.267 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 22/26 2024-09-18T13:56:40.267 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T13:56:40.267 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T13:56:40.267 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:56:40.302 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 23/26 2024-09-18T13:56:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-18T13:56:40.468 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/26 2024-09-18T13:56:40.570 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/26 2024-09-18T13:56:40.650 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/26 2024-09-18T13:56:40.686 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/26 2024-09-18T13:56:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/26 2024-09-18T13:56:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-18T13:56:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-18T13:56:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-18T13:56:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 8/26 2024-09-18T13:56:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 8/26 2024-09-18T13:56:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-18T13:56:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:17.2.7-1645.g7061ea6c.el9. 10/26 2024-09-18T13:56:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 11/26 2024-09-18T13:56:41.630 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 12/26 2024-09-18T13:56:41.659 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_6 13/26 2024-09-18T13:56:41.708 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_6 13/26 2024-09-18T13:56:41.748 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-18T13:56:41.784 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-18T13:56:41.825 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-18T13:56:42.134 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-18T13:56:42.171 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 18/26 2024-09-18T13:56:42.262 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 18/26 2024-09-18T13:56:42.338 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 19/26 2024-09-18T13:56:42.374 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:17.2.7-1645.g7061ea6c.el9.x8 20/26 2024-09-18T13:56:44.929 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 21/26 2024-09-18T13:56:44.963 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 21/26 2024-09-18T13:56:45.518 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 21/26 2024-09-18T13:56:45.558 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 22/26 2024-09-18T13:56:45.677 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 22/26 2024-09-18T13:56:45.677 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T13:56:45.677 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T13:56:45.677 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:56:45.709 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 23/26 2024-09-18T13:56:50.104 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 23/26 2024-09-18T13:56:50.104 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /sys 2024-09-18T13:56:50.105 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /proc 2024-09-18T13:56:50.105 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /mnt 2024-09-18T13:56:50.105 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /var/tmp 2024-09-18T13:56:50.105 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /home 2024-09-18T13:56:50.105 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /root 2024-09-18T13:56:50.105 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /tmp 2024-09-18T13:56:50.105 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:56:51.517 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 24/26 2024-09-18T13:56:51.578 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 24/26 2024-09-18T13:56:51.578 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:56:51.579 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T13:56:51.579 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T13:56:51.579 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T13:56:51.579 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:56:51.580 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-18T13:56:51.669 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-18T13:56:51.669 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-18T13:56:52.950 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-18T13:56:52.950 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/26 2024-09-18T13:56:52.950 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/26 2024-09-18T13:56:52.950 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/26 2024-09-18T13:56:52.950 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/26 2024-09-18T13:56:52.950 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 5/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_6 6/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 7/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1645.g7061ea6c.el9. 8/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-common-2:17.2.7-1645.g7061ea6c.el9.x8 9/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 10/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 11/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 12/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 13/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-18T13:56:52.951 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-18T13:56:52.952 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-18T13:56:52.952 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-18T13:56:52.952 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-18T13:56:52.952 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-18T13:56:52.952 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 23/26 2024-09-18T13:56:52.952 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-18T13:56:52.952 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 25/26 2024-09-18T13:56:54.011 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-18T13:56:54.011 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:56:54.011 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2024-09-18T13:56:54.011 INFO:teuthology.orchestra.run.smithi084.stdout: librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.012 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.012 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:56:54.012 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.012 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.012 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.012 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.013 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T13:56:54.013 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.013 INFO:teuthology.orchestra.run.smithi084.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T13:56:54.013 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T13:56:54.013 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T13:56:54.014 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.014 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T13:56:54.014 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.014 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T13:56:54.014 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T13:56:54.014 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.015 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.015 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.015 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T13:56:54.015 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.015 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.015 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:54.016 INFO:teuthology.orchestra.run.smithi084.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T13:56:54.016 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:56:54.016 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:56:54.373 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-test 2024-09-18T13:56:54.956 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:26 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:56:55.100 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:56:55.101 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:56:55.101 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-18T13:56:55.101 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:56:55.101 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:56:55.101 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 48 M 2024-09-18T13:56:55.101 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-18T13:56:55.101 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 163 k 2024-09-18T13:56:55.101 INFO:teuthology.orchestra.run.smithi084.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T13:56:55.101 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T13:56:55.101 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:56:55.101 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:56:55.101 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:56:55.102 INFO:teuthology.orchestra.run.smithi084.stdout:Install 4 Packages 2024-09-18T13:56:55.102 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:56:55.102 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 48 M 2024-09-18T13:56:55.102 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 193 M 2024-09-18T13:56:55.102 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:56:55.357 INFO:teuthology.orchestra.run.smithi084.stdout:(1/4): libcephsqlite-17.2.7-1645.g7061ea6c.el9. 785 kB/s | 163 kB 00:00 2024-09-18T13:56:55.382 INFO:teuthology.orchestra.run.smithi084.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-18T13:56:55.516 INFO:teuthology.orchestra.run.smithi084.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 402 kB/s | 64 kB 00:00 2024-09-18T13:56:55.840 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 23/26 2024-09-18T13:56:55.840 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-09-18T13:56:55.840 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-09-18T13:56:55.840 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-09-18T13:56:55.840 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-09-18T13:56:55.840 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-09-18T13:56:55.840 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-09-18T13:56:55.841 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-09-18T13:56:55.841 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:56:56.674 INFO:teuthology.orchestra.run.smithi084.stdout:(4/4): ceph-test-17.2.7-1645.g7061ea6c.el9.x86_ 31 MB/s | 48 MB 00:01 2024-09-18T13:56:56.676 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:56:56.677 INFO:teuthology.orchestra.run.smithi084.stdout:Total 31 MB/s | 48 MB 00:01 2024-09-18T13:56:56.705 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:56:56.724 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:56:56.725 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:56:57.090 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:56:57.090 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:56:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 24/26 2024-09-18T13:56:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 24/26 2024-09-18T13:56:57.272 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:56:57.272 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T13:56:57.272 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T13:56:57.272 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T13:56:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:56:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-18T13:56:57.362 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-18T13:56:57.362 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-18T13:56:57.604 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:56:57.684 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-18T13:56:57.723 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-18T13:56:57.759 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/4 2024-09-18T13:56:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-18T13:56:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/26 2024-09-18T13:56:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/26 2024-09-18T13:56:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/26 2024-09-18T13:56:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/26 2024-09-18T13:56:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 5/26 2024-09-18T13:56:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_6 6/26 2024-09-18T13:56:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 7/26 2024-09-18T13:56:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1645.g7061ea6c.el9. 8/26 2024-09-18T13:56:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:17.2.7-1645.g7061ea6c.el9.x8 9/26 2024-09-18T13:56:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 10/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 11/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 12/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 13/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-18T13:56:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 23/26 2024-09-18T13:56:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-18T13:56:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 25/26 2024-09-18T13:56:59.419 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-18T13:56:59.420 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:56:59.420 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-09-18T13:56:59.420 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.420 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.420 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:56:59.420 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.420 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.420 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.420 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.420 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T13:56:59.421 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.421 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T13:56:59.421 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T13:56:59.421 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T13:56:59.421 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.421 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T13:56:59.421 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.421 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T13:56:59.421 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T13:56:59.421 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.421 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.421 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.422 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T13:56:59.422 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.422 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.422 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:56:59.422 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T13:56:59.422 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:56:59.422 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:56:59.737 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-test 2024-09-18T13:57:00.345 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:28 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:57:00.489 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:57:00.490 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:00.490 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-18T13:57:00.490 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:00.490 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:57:00.490 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 48 M 2024-09-18T13:57:00.490 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-18T13:57:00.491 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 163 k 2024-09-18T13:57:00.491 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T13:57:00.491 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T13:57:00.491 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:00.491 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:57:00.491 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:00.491 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-09-18T13:57:00.491 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:00.492 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 48 M 2024-09-18T13:57:00.492 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 193 M 2024-09-18T13:57:00.492 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:57:00.735 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): libcephsqlite-17.2.7-1645.g7061ea6c.el9. 738 kB/s | 163 kB 00:00 2024-09-18T13:57:01.302 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 386 kB/s | 304 kB 00:00 2024-09-18T13:57:01.352 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 104 kB/s | 64 kB 00:00 2024-09-18T13:57:02.038 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): ceph-test-17.2.7-1645.g7061ea6c.el9.x86_ 31 MB/s | 48 MB 00:01 2024-09-18T13:57:02.040 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:02.040 INFO:teuthology.orchestra.run.smithi045.stdout:Total 31 MB/s | 48 MB 00:01 2024-09-18T13:57:02.069 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:57:02.091 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:57:02.092 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:57:02.204 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/4 2024-09-18T13:57:02.281 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/4 2024-09-18T13:57:02.477 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:57:02.478 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:57:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:57:03.247 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-18T13:57:03.285 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-18T13:57:03.313 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/4 2024-09-18T13:57:04.560 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/4 2024-09-18T13:57:04.560 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/4 2024-09-18T13:57:04.561 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/4 2024-09-18T13:57:04.561 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-18T13:57:04.855 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-18T13:57:04.855 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:04.855 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:57:04.855 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:04.855 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:04.855 INFO:teuthology.orchestra.run.smithi084.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T13:57:04.855 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T13:57:04.856 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:04.856 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:05.147 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph 2024-09-18T13:57:05.731 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:37 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:05.867 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:05.869 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:05.869 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-18T13:57:05.869 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:05.870 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:57:05.870 INFO:teuthology.orchestra.run.smithi084.stdout: ceph x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 6.5 k 2024-09-18T13:57:05.870 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-18T13:57:05.870 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 2.1 M 2024-09-18T13:57:05.870 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 1.4 M 2024-09-18T13:57:05.870 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 240 k 2024-09-18T13:57:05.870 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 3.9 M 2024-09-18T13:57:05.870 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 15 M 2024-09-18T13:57:05.870 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T13:57:05.870 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T13:57:05.870 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T13:57:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T13:57:05.872 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:05.873 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:57:05.873 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:05.873 INFO:teuthology.orchestra.run.smithi084.stdout:Install 34 Packages 2024-09-18T13:57:05.873 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:05.873 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 27 M 2024-09-18T13:57:05.873 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 96 M 2024-09-18T13:57:05.874 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:57:06.332 INFO:teuthology.orchestra.run.smithi084.stdout:(1/34): ceph-17.2.7-1645.g7061ea6c.el9.x86_64.r 47 kB/s | 6.5 kB 00:00 2024-09-18T13:57:06.499 INFO:teuthology.orchestra.run.smithi084.stdout:(2/34): ceph-mgr-17.2.7-1645.g7061ea6c.el9.x86_ 4.6 MB/s | 1.4 MB 00:00 2024-09-18T13:57:06.541 INFO:teuthology.orchestra.run.smithi084.stdout:(3/34): ceph-mds-17.2.7-1645.g7061ea6c.el9.x86_ 5.9 MB/s | 2.1 MB 00:00 2024-09-18T13:57:06.616 INFO:teuthology.orchestra.run.smithi084.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1645.g7061 3.1 MB/s | 240 kB 00:00 2024-09-18T13:57:06.708 INFO:teuthology.orchestra.run.smithi084.stdout:(5/34): ceph-mon-17.2.7-1645.g7061ea6c.el9.x86_ 10 MB/s | 3.9 MB 00:00 2024-09-18T13:57:07.017 INFO:teuthology.orchestra.run.smithi084.stdout:(6/34): ceph-osd-17.2.7-1645.g7061ea6c.el9.x86_ 30 MB/s | 15 MB 00:00 2024-09-18T13:57:07.042 INFO:teuthology.orchestra.run.smithi084.stdout:(7/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 595 kB/s | 253 kB 00:00 2024-09-18T13:57:07.117 INFO:teuthology.orchestra.run.smithi084.stdout:(8/34): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-09-18T13:57:07.193 INFO:teuthology.orchestra.run.smithi084.stdout:(9/34): python3-requests-2.25.1-8.el9.noarch.rp 1.6 MB/s | 125 kB 00:00 2024-09-18T13:57:07.234 INFO:teuthology.orchestra.run.smithi084.stdout:(10/34): python3-ply-3.11-14.el9.noarch.rpm 490 kB/s | 106 kB 00:00 2024-09-18T13:57:07.301 INFO:teuthology.orchestra.run.smithi084.stdout:(11/34): python3-cryptography-36.0.1-4.el9.x86_ 2.1 MB/s | 1.2 MB 00:00 2024-09-18T13:57:07.335 INFO:teuthology.orchestra.run.smithi084.stdout:(12/34): python3-urllib3-1.26.5-6.el9.noarch.rp 1.5 MB/s | 215 kB 00:00 2024-09-18T13:57:07.477 INFO:teuthology.orchestra.run.smithi084.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 198 kB/s | 35 kB 00:00 2024-09-18T13:57:07.560 INFO:teuthology.orchestra.run.smithi084.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 528 kB/s | 172 kB 00:00 2024-09-18T13:57:07.586 INFO:teuthology.orchestra.run.smithi084.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 308 kB/s | 77 kB 00:00 2024-09-18T13:57:07.611 INFO:teuthology.orchestra.run.smithi084.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 381 kB/s | 51 kB 00:00 2024-09-18T13:57:07.636 INFO:teuthology.orchestra.run.smithi084.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 551 kB/s | 42 kB 00:00 2024-09-18T13:57:07.687 INFO:teuthology.orchestra.run.smithi084.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 579 kB/s | 43 kB 00:00 2024-09-18T13:57:07.712 INFO:teuthology.orchestra.run.smithi084.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 461 kB/s | 11 kB 00:00 2024-09-18T13:57:07.737 INFO:teuthology.orchestra.run.smithi084.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-18T13:57:07.762 INFO:teuthology.orchestra.run.smithi084.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 353 kB/s | 18 kB 00:00 2024-09-18T13:57:07.787 INFO:teuthology.orchestra.run.smithi084.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-09-18T13:57:07.813 INFO:teuthology.orchestra.run.smithi084.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-18T13:57:07.838 INFO:teuthology.orchestra.run.smithi084.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-18T13:57:07.863 INFO:teuthology.orchestra.run.smithi084.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-18T13:57:07.897 INFO:teuthology.orchestra.run.smithi084.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-18T13:57:07.922 INFO:teuthology.orchestra.run.smithi084.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-09-18T13:57:07.948 INFO:teuthology.orchestra.run.smithi084.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-18T13:57:07.973 INFO:teuthology.orchestra.run.smithi084.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-18T13:57:07.994 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/4 2024-09-18T13:57:07.998 INFO:teuthology.orchestra.run.smithi084.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-09-18T13:57:08.023 INFO:teuthology.orchestra.run.smithi084.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-18T13:57:08.049 INFO:teuthology.orchestra.run.smithi084.stdout:(32/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 8.3 MB/s | 427 kB 00:00 2024-09-18T13:57:08.074 INFO:teuthology.orchestra.run.smithi084.stdout:(33/34): python3-zc-lockfile-2.0-10.el9.noarch. 397 kB/s | 20 kB 00:00 2024-09-18T13:57:08.088 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/4 2024-09-18T13:57:08.126 INFO:teuthology.orchestra.run.smithi084.stdout:(34/34): python3-webob-1.8.8-2.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-09-18T13:57:08.131 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:08.131 INFO:teuthology.orchestra.run.smithi084.stdout:Total 12 MB/s | 27 MB 00:02 2024-09-18T13:57:08.265 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:57:08.304 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:57:08.304 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:57:08.728 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:57:08.729 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:57:09.959 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:57:10.044 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-18T13:57:10.076 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-18T13:57:10.121 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-18T13:57:10.221 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-18T13:57:10.299 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-18T13:57:10.340 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-18T13:57:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-18T13:57:10.405 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-18T13:57:10.427 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/4 2024-09-18T13:57:10.428 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/4 2024-09-18T13:57:10.428 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/4 2024-09-18T13:57:10.428 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-18T13:57:10.441 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-18T13:57:10.481 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-18T13:57:10.530 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-18T13:57:10.565 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-18T13:57:10.599 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-18T13:57:10.656 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-18T13:57:10.694 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-18T13:57:10.747 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-18T13:57:10.747 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:10.747 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:57:10.747 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:10.748 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:10.748 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T13:57:10.748 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T13:57:10.748 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:10.748 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:57:10.769 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-18T13:57:10.906 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-18T13:57:10.986 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-18T13:57:11.000 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph 2024-09-18T13:57:11.048 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-18T13:57:11.087 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-18T13:57:11.135 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-18T13:57:11.191 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-18T13:57:11.376 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-18T13:57:11.433 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-18T13:57:11.510 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-18T13:57:11.599 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-18T13:57:11.600 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:39 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:57:11.654 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-18T13:57:11.740 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:57:11.743 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:11.743 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-18T13:57:11.743 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:11.743 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:57:11.743 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 6.5 k 2024-09-18T13:57:11.743 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-18T13:57:11.743 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 2.1 M 2024-09-18T13:57:11.743 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 1.4 M 2024-09-18T13:57:11.743 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 240 k 2024-09-18T13:57:11.743 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 3.9 M 2024-09-18T13:57:11.744 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 15 M 2024-09-18T13:57:11.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T13:57:11.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T13:57:11.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T13:57:11.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T13:57:11.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T13:57:11.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T13:57:11.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T13:57:11.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections 2024-09-18T13:57:11.744 INFO:teuthology.orchestra.run.smithi045.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-18T13:57:11.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T13:57:11.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T13:57:11.745 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T13:57:11.746 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T13:57:11.746 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T13:57:11.746 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T13:57:11.746 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T13:57:11.746 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T13:57:11.746 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:11.746 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:57:11.746 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:11.746 INFO:teuthology.orchestra.run.smithi045.stdout:Install 34 Packages 2024-09-18T13:57:11.746 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:11.747 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 27 M 2024-09-18T13:57:11.747 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 96 M 2024-09-18T13:57:11.747 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:57:11.770 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-18T13:57:11.918 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1645.g7061ea6c.el9. 29/34 2024-09-18T13:57:11.949 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 30/34 2024-09-18T13:57:11.976 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 30/34 2024-09-18T13:57:11.976 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:57:11.976 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T13:57:11.976 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T13:57:11.976 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T13:57:11.976 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:12.005 INFO:teuthology.orchestra.run.smithi045.stdout:(1/34): ceph-17.2.7-1645.g7061ea6c.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-18T13:57:12.205 INFO:teuthology.orchestra.run.smithi045.stdout:(2/34): ceph-mds-17.2.7-1645.g7061ea6c.el9.x86_ 6.2 MB/s | 2.1 MB 00:00 2024-09-18T13:57:12.247 INFO:teuthology.orchestra.run.smithi045.stdout:(3/34): ceph-mgr-17.2.7-1645.g7061ea6c.el9.x86_ 3.8 MB/s | 1.4 MB 00:00 2024-09-18T13:57:12.339 INFO:teuthology.orchestra.run.smithi045.stdout:(4/34): ceph-mon-17.2.7-1645.g7061ea6c.el9.x86_ 12 MB/s | 3.9 MB 00:00 2024-09-18T13:57:12.364 INFO:teuthology.orchestra.run.smithi045.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1645.g7061 2.0 MB/s | 240 kB 00:00 2024-09-18T13:57:12.481 INFO:teuthology.orchestra.run.smithi045.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-09-18T13:57:12.515 INFO:teuthology.orchestra.run.smithi045.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-09-18T13:57:12.565 INFO:teuthology.orchestra.run.smithi045.stdout:(8/34): python3-cryptography-36.0.1-4.el9.x86_6 6.1 MB/s | 1.2 MB 00:00 2024-09-18T13:57:12.590 INFO:teuthology.orchestra.run.smithi045.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-09-18T13:57:12.616 INFO:teuthology.orchestra.run.smithi045.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-18T13:57:12.641 INFO:teuthology.orchestra.run.smithi045.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-18T13:57:12.666 INFO:teuthology.orchestra.run.smithi045.stdout:(12/34): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-18T13:57:12.683 INFO:teuthology.orchestra.run.smithi045.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 830 kB/s | 35 kB 00:00 2024-09-18T13:57:12.842 INFO:teuthology.orchestra.run.smithi045.stdout:(14/34): ceph-osd-17.2.7-1645.g7061ea6c.el9.x86 24 MB/s | 15 MB 00:00 2024-09-18T13:57:12.859 INFO:teuthology.orchestra.run.smithi045.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 291 kB/s | 51 kB 00:00 2024-09-18T13:57:12.884 INFO:teuthology.orchestra.run.smithi045.stdout:(16/34): python3-packaging-20.9-5.el9.noarch.rp 355 kB/s | 77 kB 00:00 2024-09-18T13:57:12.935 INFO:teuthology.orchestra.run.smithi045.stdout:(17/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 576 kB/s | 43 kB 00:00 2024-09-18T13:57:12.960 INFO:teuthology.orchestra.run.smithi045.stdout:(18/34): python3-toml-0.10.2-6.el9.noarch.rpm 355 kB/s | 42 kB 00:00 2024-09-18T13:57:12.994 INFO:teuthology.orchestra.run.smithi045.stdout:(19/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-18T13:57:13.019 INFO:teuthology.orchestra.run.smithi045.stdout:(20/34): python3-cherrypy-18.6.1-2.el9.noarch.r 4.3 MB/s | 358 kB 00:00 2024-09-18T13:57:13.044 INFO:teuthology.orchestra.run.smithi045.stdout:(21/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 127 kB/s | 11 kB 00:00 2024-09-18T13:57:13.069 INFO:teuthology.orchestra.run.smithi045.stdout:(22/34): python3-jaraco-classes-3.2.1-5.el9.noa 241 kB/s | 18 kB 00:00 2024-09-18T13:57:13.095 INFO:teuthology.orchestra.run.smithi045.stdout:(23/34): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-09-18T13:57:13.120 INFO:teuthology.orchestra.run.smithi045.stdout:(24/34): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-09-18T13:57:13.145 INFO:teuthology.orchestra.run.smithi045.stdout:(25/34): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-18T13:57:13.171 INFO:teuthology.orchestra.run.smithi045.stdout:(26/34): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-18T13:57:13.196 INFO:teuthology.orchestra.run.smithi045.stdout:(27/34): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-18T13:57:13.221 INFO:teuthology.orchestra.run.smithi045.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 329 kB/s | 16 kB 00:00 2024-09-18T13:57:13.247 INFO:teuthology.orchestra.run.smithi045.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-18T13:57:13.271 INFO:teuthology.orchestra.run.smithi045.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-09-18T13:57:13.283 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 31/34 2024-09-18T13:57:13.297 INFO:teuthology.orchestra.run.smithi045.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-18T13:57:13.311 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 31/34 2024-09-18T13:57:13.311 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:57:13.312 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T13:57:13.312 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T13:57:13.312 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T13:57:13.312 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:13.322 INFO:teuthology.orchestra.run.smithi045.stdout:(32/34): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-18T13:57:13.348 INFO:teuthology.orchestra.run.smithi045.stdout:(33/34): python3-zc-lockfile-2.0-10.el9.noarch. 395 kB/s | 20 kB 00:00 2024-09-18T13:57:13.373 INFO:teuthology.orchestra.run.smithi045.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-18T13:57:13.377 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:13.377 INFO:teuthology.orchestra.run.smithi045.stdout:Total 17 MB/s | 27 MB 00:01 2024-09-18T13:57:13.517 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:57:13.559 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:57:13.559 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:57:13.684 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 32/34 2024-09-18T13:57:13.708 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 32/34 2024-09-18T13:57:13.708 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:57:13.708 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T13:57:13.708 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T13:57:13.709 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T13:57:13.709 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:13.910 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 33/34 2024-09-18T13:57:13.934 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 33/34 2024-09-18T13:57:13.934 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:57:13.934 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T13:57:13.934 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T13:57:13.934 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T13:57:13.934 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:13.992 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:57:13.992 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:57:13.998 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 34/34 2024-09-18T13:57:15.386 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:57:15.443 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-18T13:57:15.475 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-18T13:57:15.508 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-18T13:57:15.605 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-18T13:57:15.634 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 34/34 2024-09-18T13:57:15.634 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/34 2024-09-18T13:57:15.634 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/34 2024-09-18T13:57:15.634 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/34 2024-09-18T13:57:15.634 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/34 2024-09-18T13:57:15.634 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 5/34 2024-09-18T13:57:15.634 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1645.g7061ea6c.el9. 6/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-18T13:57:15.635 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-18T13:57:15.636 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-18T13:57:15.636 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-18T13:57:15.636 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-18T13:57:15.636 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-18T13:57:15.636 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-18T13:57:15.637 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-18T13:57:15.637 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-18T13:57:15.637 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-18T13:57:15.637 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-18T13:57:15.637 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-18T13:57:15.637 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-18T13:57:15.637 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-18T13:57:15.638 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-18T13:57:15.690 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-18T13:57:15.737 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-18T13:57:15.780 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-18T13:57:15.809 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-18T13:57:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-18T13:57:15.877 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-18T13:57:15.918 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-18T13:57:15.961 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-18T13:57:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-18T13:57:16.040 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-18T13:57:16.086 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-18T13:57:16.168 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-18T13:57:16.442 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-18T13:57:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-18T13:57:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-18T13:57:16.622 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-18T13:57:16.622 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:16.622 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:57:16.622 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:16.622 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:16.623 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:16.623 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:16.623 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:16.623 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:16.623 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T13:57:16.623 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T13:57:16.623 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T13:57:16.623 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T13:57:16.623 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T13:57:16.623 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T13:57:16.623 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T13:57:16.623 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T13:57:16.624 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T13:57:16.624 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T13:57:16.624 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T13:57:16.624 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T13:57:16.624 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T13:57:16.624 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T13:57:16.624 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T13:57:16.624 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T13:57:16.624 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T13:57:16.624 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T13:57:16.625 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T13:57:16.625 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T13:57:16.625 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T13:57:16.625 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T13:57:16.625 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T13:57:16.625 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T13:57:16.625 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T13:57:16.625 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T13:57:16.625 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T13:57:16.625 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T13:57:16.625 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:16.625 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:16.638 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-18T13:57:16.698 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-18T13:57:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-18T13:57:16.967 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-18T13:57:17.018 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-18T13:57:17.034 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-base 2024-09-18T13:57:17.099 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-18T13:57:17.206 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-18T13:57:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-18T13:57:17.380 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-18T13:57:17.537 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1645.g7061ea6c.el9. 29/34 2024-09-18T13:57:17.560 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 30/34 2024-09-18T13:57:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 30/34 2024-09-18T13:57:17.587 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:57:17.587 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T13:57:17.587 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T13:57:17.587 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T13:57:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:17.622 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:49 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:17.710 INFO:teuthology.orchestra.run.smithi084.stdout:Package ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:57:17.751 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:17.753 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-18T13:57:17.753 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:17.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install cephadm 2024-09-18T13:57:18.397 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:50 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:18.531 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:18.532 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:18.532 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-18T13:57:18.532 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:18.532 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:57:18.532 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 83 k 2024-09-18T13:57:18.532 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:18.533 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:57:18.533 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:18.533 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-18T13:57:18.533 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:18.533 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 83 k 2024-09-18T13:57:18.533 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 375 k 2024-09-18T13:57:18.533 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:57:18.727 INFO:teuthology.orchestra.run.smithi084.stdout:cephadm-17.2.7-1645.g7061ea6c.el9.noarch.rpm 430 kB/s | 83 kB 00:00 2024-09-18T13:57:18.727 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:18.728 INFO:teuthology.orchestra.run.smithi084.stdout:Total 425 kB/s | 83 kB 00:00 2024-09-18T13:57:18.728 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:57:18.733 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:57:18.733 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:57:18.735 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:57:18.735 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:57:18.809 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:57:19.034 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 31/34 2024-09-18T13:57:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 31/34 2024-09-18T13:57:19.066 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:57:19.066 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T13:57:19.066 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T13:57:19.066 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T13:57:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:19.293 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:57:19.390 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:57:19.454 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 32/34 2024-09-18T13:57:19.482 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 32/34 2024-09-18T13:57:19.482 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:57:19.482 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T13:57:19.482 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T13:57:19.482 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T13:57:19.482 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 33/34 2024-09-18T13:57:19.722 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 33/34 2024-09-18T13:57:19.722 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:57:19.723 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T13:57:19.723 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T13:57:19.723 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T13:57:19.723 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:19.755 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:57:19.789 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 34/34 2024-09-18T13:57:20.004 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:57:20.004 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:20.004 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:57:20.004 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:20.004 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:20.005 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:20.216 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-immutable-object-cache 2024-09-18T13:57:20.813 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:52 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:20.947 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:20.947 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:20.948 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repo Size 2024-09-18T13:57:20.948 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:20.948 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:57:20.948 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 137 k 2024-09-18T13:57:20.948 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:20.948 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:57:20.948 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:20.948 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-18T13:57:20.948 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:20.948 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 137 k 2024-09-18T13:57:20.948 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 379 k 2024-09-18T13:57:20.948 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:57:21.149 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-immutable-object-cache-17.2.7-1645.g7061ea 686 kB/s | 137 kB 00:00 2024-09-18T13:57:21.149 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:21.150 INFO:teuthology.orchestra.run.smithi084.stdout:Total 680 kB/s | 137 kB 00:00 2024-09-18T13:57:21.150 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:57:21.157 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:57:21.157 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:57:21.213 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:57:21.213 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:57:21.364 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:57:21.441 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c. 1/1 2024-09-18T13:57:21.467 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c. 1/1 2024-09-18T13:57:21.467 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:57:21.467 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T13:57:21.467 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 34/34 2024-09-18T13:57:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/34 2024-09-18T13:57:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/34 2024-09-18T13:57:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/34 2024-09-18T13:57:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/34 2024-09-18T13:57:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 5/34 2024-09-18T13:57:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1645.g7061ea6c.el9. 6/34 2024-09-18T13:57:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-18T13:57:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-18T13:57:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-18T13:57:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-18T13:57:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-18T13:57:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-18T13:57:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-18T13:57:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-18T13:57:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-18T13:57:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-18T13:57:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-18T13:57:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-18T13:57:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-18T13:57:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-18T13:57:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-18T13:57:22.401 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c. 1/1 2024-09-18T13:57:22.401 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:22.401 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:57:22.401 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:22.401 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:22.402 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-18T13:57:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:22.550 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:57:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T13:57:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T13:57:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T13:57:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T13:57:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T13:57:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T13:57:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T13:57:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T13:57:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T13:57:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T13:57:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:22.553 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:57:22.619 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr 2024-09-18T13:57:22.903 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-base 2024-09-18T13:57:23.204 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:55 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:23.295 INFO:teuthology.orchestra.run.smithi084.stdout:Package ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:57:23.338 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:23.339 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-18T13:57:23.339 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:23.410 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr-dashboard 2024-09-18T13:57:23.510 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:51 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:57:23.603 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:57:23.647 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:57:23.649 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-18T13:57:23.649 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:57:23.714 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install cephadm 2024-09-18T13:57:23.982 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:55 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:24.112 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:24.113 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:24.113 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-18T13:57:24.113 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:24.113 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:57:24.113 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 3.4 M 2024-09-18T13:57:24.114 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-18T13:57:24.114 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 24 k 2024-09-18T13:57:24.114 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 15 k 2024-09-18T13:57:24.114 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-18T13:57:24.114 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T13:57:24.114 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T13:57:24.114 INFO:teuthology.orchestra.run.smithi084.stdout:Installing weak dependencies: 2024-09-18T13:57:24.115 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-18T13:57:24.115 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:24.115 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:57:24.115 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:24.115 INFO:teuthology.orchestra.run.smithi084.stdout:Install 7 Packages 2024-09-18T13:57:24.115 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:24.116 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 3.7 M 2024-09-18T13:57:24.117 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 80 M 2024-09-18T13:57:24.117 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:57:24.313 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:52 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:57:24.449 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:57:24.451 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:24.451 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-18T13:57:24.451 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:24.451 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:57:24.452 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 83 k 2024-09-18T13:57:24.452 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:24.452 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:57:24.452 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:24.452 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-18T13:57:24.453 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:24.453 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 83 k 2024-09-18T13:57:24.453 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 375 k 2024-09-18T13:57:24.453 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:57:24.454 INFO:teuthology.orchestra.run.smithi084.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1645.g706 168 kB/s | 24 kB 00:00 2024-09-18T13:57:24.469 INFO:teuthology.orchestra.run.smithi084.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1645.g7061 90 kB/s | 15 kB 00:00 2024-09-18T13:57:24.502 INFO:teuthology.orchestra.run.smithi084.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 181 kB/s | 9.0 kB 00:00 2024-09-18T13:57:24.527 INFO:teuthology.orchestra.run.smithi084.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-18T13:57:24.544 INFO:teuthology.orchestra.run.smithi084.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 542 kB/s | 41 kB 00:00 2024-09-18T13:57:24.578 INFO:teuthology.orchestra.run.smithi084.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-18T13:57:24.639 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm-17.2.7-1645.g7061ea6c.el9.noarch.rpm 442 kB/s | 83 kB 00:00 2024-09-18T13:57:24.640 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:24.640 INFO:teuthology.orchestra.run.smithi045.stdout:Total 438 kB/s | 83 kB 00:00 2024-09-18T13:57:24.640 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:57:24.645 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:57:24.645 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:57:24.647 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:57:24.647 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:57:24.686 INFO:teuthology.orchestra.run.smithi084.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1645.g7061ea6c 8.8 MB/s | 3.4 MB 00:00 2024-09-18T13:57:24.690 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:24.690 INFO:teuthology.orchestra.run.smithi084.stdout:Total 6.3 MB/s | 3.7 MB 00:00 2024-09-18T13:57:24.690 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:57:24.698 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:57:24.698 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:57:24.725 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:57:24.799 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:57:24.799 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:57:25.105 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:57:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-18T13:57:25.199 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:57:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-18T13:57:25.251 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-18T13:57:25.284 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-18T13:57:25.305 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:57:25.312 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1645.g7061ea6c.el9.n 5/7 2024-09-18T13:57:25.638 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:57:25.920 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:57:25.920 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:25.920 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:57:25.920 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:25.921 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:25.921 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:57:26.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-immutable-object-cache 2024-09-18T13:57:26.301 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1645.g7061ea6c.el9. 6/7 2024-09-18T13:57:26.341 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarc 7/7 2024-09-18T13:57:26.752 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:54 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:57:26.894 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:57:26.894 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-09-18T13:57:26.895 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:26.895 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:57:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 137 k 2024-09-18T13:57:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:26.895 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:57:26.895 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:26.895 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-18T13:57:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:26.895 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 137 k 2024-09-18T13:57:26.895 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 379 k 2024-09-18T13:57:26.895 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:57:27.103 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-immutable-object-cache-17.2.7-1645.g7061ea 661 kB/s | 137 kB 00:00 2024-09-18T13:57:27.104 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:27.104 INFO:teuthology.orchestra.run.smithi045.stdout:Total 656 kB/s | 137 kB 00:00 2024-09-18T13:57:27.104 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:57:27.112 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:57:27.112 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:57:27.172 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:57:27.172 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:57:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:57:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c. 1/1 2024-09-18T13:57:27.387 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarc 7/7 2024-09-18T13:57:27.387 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1645.g7061ea6c.el9. 1/7 2024-09-18T13:57:27.387 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarc 2/7 2024-09-18T13:57:27.387 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1645.g7061ea6c.el9.n 3/7 2024-09-18T13:57:27.387 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-18T13:57:27.387 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-18T13:57:27.387 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-18T13:57:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c. 1/1 2024-09-18T13:57:27.406 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:57:27.406 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T13:57:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:27.677 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-18T13:57:27.677 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:27.677 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:57:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-18T13:57:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-18T13:57:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T13:57:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T13:57:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:27.678 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:27.895 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-18T13:57:28.358 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c. 1/1 2024-09-18T13:57:28.358 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:28.359 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:57:28.359 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:28.359 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:28.359 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:57:28.469 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:00 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:28.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr 2024-09-18T13:57:28.600 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:28.602 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-09-18T13:57:28.602 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-18T13:57:28.602 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-09-18T13:57:28.602 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:57:28.602 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 7.4 M 2024-09-18T13:57:28.602 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-18T13:57:28.602 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T13:57:28.602 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T13:57:28.602 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T13:57:28.602 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T13:57:28.602 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T13:57:28.603 INFO:teuthology.orchestra.run.smithi084.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T13:57:28.603 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T13:57:28.603 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T13:57:28.603 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T13:57:28.603 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T13:57:28.603 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T13:57:28.603 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:28.603 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:57:28.603 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-09-18T13:57:28.603 INFO:teuthology.orchestra.run.smithi084.stdout:Install 12 Packages 2024-09-18T13:57:28.603 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:28.604 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 38 M 2024-09-18T13:57:28.604 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 212 M 2024-09-18T13:57:28.604 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:57:29.203 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:57 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:57:29.302 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:57:29.302 INFO:teuthology.orchestra.run.smithi084.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-18T13:57:29.344 INFO:teuthology.orchestra.run.smithi084.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-09-18T13:57:29.347 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:57:29.349 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-18T13:57:29.349 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:57:29.361 INFO:teuthology.orchestra.run.smithi084.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-09-18T13:57:29.394 INFO:teuthology.orchestra.run.smithi084.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 546 kB/s | 18 kB 00:00 2024-09-18T13:57:29.418 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-dashboard 2024-09-18T13:57:29.428 INFO:teuthology.orchestra.run.smithi084.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 1.1 MB/s | 38 kB 00:00 2024-09-18T13:57:29.553 INFO:teuthology.orchestra.run.smithi084.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-09-18T13:57:29.654 INFO:teuthology.orchestra.run.smithi084.stdout:(7/12): ceph-mgr-diskprediction-local-17.2.7-16 15 MB/s | 7.4 MB 00:00 2024-09-18T13:57:29.687 INFO:teuthology.orchestra.run.smithi084.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-18T13:57:29.763 INFO:teuthology.orchestra.run.smithi084.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 15 MB/s | 4.9 MB 00:00 2024-09-18T13:57:29.788 INFO:teuthology.orchestra.run.smithi084.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.4 MB/s | 250 kB 00:00 2024-09-18T13:57:30.016 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:58 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:57:30.105 INFO:teuthology.orchestra.run.smithi084.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 MB/s | 16 MB 00:00 2024-09-18T13:57:30.153 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:57:30.153 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 3.4 M 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 24 k 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 15 k 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-18T13:57:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:30.155 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:57:30.155 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:30.155 INFO:teuthology.orchestra.run.smithi045.stdout:Install 7 Packages 2024-09-18T13:57:30.155 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:30.155 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 3.7 M 2024-09-18T13:57:30.155 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 80 M 2024-09-18T13:57:30.156 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:57:30.472 INFO:teuthology.orchestra.run.smithi084.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.2 MB/s | 5.1 MB 00:00 2024-09-18T13:57:30.472 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:30.473 INFO:teuthology.orchestra.run.smithi084.stdout:Total 20 MB/s | 38 MB 00:01 2024-09-18T13:57:30.642 INFO:teuthology.orchestra.run.smithi045.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1645.g706 163 kB/s | 24 kB 00:00 2024-09-18T13:57:30.667 INFO:teuthology.orchestra.run.smithi045.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1645.g7061 84 kB/s | 15 kB 00:00 2024-09-18T13:57:30.709 INFO:teuthology.orchestra.run.smithi045.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 136 kB/s | 9.0 kB 00:00 2024-09-18T13:57:30.734 INFO:teuthology.orchestra.run.smithi045.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-18T13:57:30.759 INFO:teuthology.orchestra.run.smithi045.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-18T13:57:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-18T13:57:30.826 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:57:30.852 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:57:30.852 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:57:30.893 INFO:teuthology.orchestra.run.smithi045.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1645.g7061ea6c 8.4 MB/s | 3.4 MB 00:00 2024-09-18T13:57:30.897 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:30.897 INFO:teuthology.orchestra.run.smithi045.stdout:Total 4.9 MB/s | 3.7 MB 00:00 2024-09-18T13:57:30.897 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:57:30.907 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:57:30.907 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:57:31.000 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:57:31.001 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:57:31.316 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:57:31.358 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:57:31.359 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:57:31.386 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-18T13:57:31.421 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-18T13:57:31.469 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-18T13:57:31.499 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-18T13:57:31.529 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1645.g7061ea6c.el9.n 5/7 2024-09-18T13:57:32.125 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:57:32.200 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-18T13:57:32.244 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-18T13:57:32.344 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-18T13:57:32.379 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-18T13:57:32.563 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-18T13:57:32.583 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1645.g7061ea6c.el9. 6/7 2024-09-18T13:57:32.594 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-18T13:57:32.623 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarc 7/7 2024-09-18T13:57:32.679 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-18T13:57:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-18T13:57:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-18T13:57:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarc 7/7 2024-09-18T13:57:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1645.g7061ea6c.el9. 1/7 2024-09-18T13:57:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarc 2/7 2024-09-18T13:57:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1645.g7061ea6c.el9.n 3/7 2024-09-18T13:57:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-18T13:57:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-18T13:57:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-18T13:57:33.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-18T13:57:33.871 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:33.871 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:57:33.871 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:33.871 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:33.871 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:33.871 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-18T13:57:33.871 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-18T13:57:33.872 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T13:57:33.872 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T13:57:33.872 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:33.872 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:57:34.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-18T13:57:34.685 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:02 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:57:34.733 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T13:57:34.825 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:57:34.826 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-09-18T13:57:34.826 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-18T13:57:34.826 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-09-18T13:57:34.827 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:57:34.827 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 7.4 M 2024-09-18T13:57:34.827 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-18T13:57:34.827 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T13:57:34.827 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T13:57:34.827 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T13:57:34.827 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T13:57:34.827 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T13:57:34.827 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T13:57:34.827 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T13:57:34.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T13:57:34.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T13:57:34.828 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T13:57:34.828 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T13:57:34.828 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:34.828 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:57:34.828 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-09-18T13:57:34.828 INFO:teuthology.orchestra.run.smithi045.stdout:Install 12 Packages 2024-09-18T13:57:34.828 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:34.834 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 38 M 2024-09-18T13:57:34.834 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 212 M 2024-09-18T13:57:34.834 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:57:35.776 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-18T13:57:35.801 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061e 12/12 2024-09-18T13:57:35.844 INFO:teuthology.orchestra.run.smithi045.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 16 MB/s | 7.4 MB 00:00 2024-09-18T13:57:35.869 INFO:teuthology.orchestra.run.smithi045.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 399 kB/s | 191 kB 00:00 2024-09-18T13:57:35.977 INFO:teuthology.orchestra.run.smithi045.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 245 kB/s | 33 kB 00:00 2024-09-18T13:57:36.028 INFO:teuthology.orchestra.run.smithi045.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 364 kB/s | 18 kB 00:00 2024-09-18T13:57:36.053 INFO:teuthology.orchestra.run.smithi045.stdout:(5/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 801 kB 00:00 2024-09-18T13:57:36.078 INFO:teuthology.orchestra.run.smithi045.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-18T13:57:36.195 INFO:teuthology.orchestra.run.smithi045.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-18T13:57:36.288 INFO:teuthology.orchestra.run.smithi045.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.1 MB/s | 3.0 MB 00:00 2024-09-18T13:57:36.346 INFO:teuthology.orchestra.run.smithi045.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.2 MB/s | 250 kB 00:00 2024-09-18T13:57:36.530 INFO:teuthology.orchestra.run.smithi045.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 10 MB/s | 4.9 MB 00:00 2024-09-18T13:57:36.605 INFO:teuthology.orchestra.run.smithi045.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-18T13:57:37.097 INFO:teuthology.orchestra.run.smithi045.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-18T13:57:37.097 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:37.098 INFO:teuthology.orchestra.run.smithi045.stdout:Total 17 MB/s | 38 MB 00:02 2024-09-18T13:57:37.463 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:57:37.490 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:57:37.490 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:57:38.009 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:57:38.009 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:57:38.351 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061e 12/12 2024-09-18T13:57:38.351 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061e 1/12 2024-09-18T13:57:38.351 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-18T13:57:38.351 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-18T13:57:38.352 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-18T13:57:38.352 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-18T13:57:38.352 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-18T13:57:38.352 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-18T13:57:38.352 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-18T13:57:38.352 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-18T13:57:38.352 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T13:57:38.352 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-18T13:57:38.794 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:57:38.801 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-18T13:57:38.801 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:38.801 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:57:38.801 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:38.801 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T13:57:38.801 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T13:57:38.801 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T13:57:38.801 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T13:57:38.801 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T13:57:38.801 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T13:57:38.802 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T13:57:38.802 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T13:57:38.802 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T13:57:38.802 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T13:57:38.802 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T13:57:38.802 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:38.802 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:38.870 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-18T13:57:38.912 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-18T13:57:39.017 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-18T13:57:39.055 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-18T13:57:39.185 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr-rook 2024-09-18T13:57:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-18T13:57:39.278 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-18T13:57:39.357 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-18T13:57:39.785 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:11 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:39.928 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:39.929 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:39.929 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-18T13:57:39.929 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:39.929 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:57:39.929 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 47 k 2024-09-18T13:57:39.929 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-18T13:57:39.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T13:57:39.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T13:57:39.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:39.930 INFO:teuthology.orchestra.run.smithi084.stdout:Install 13 Packages 2024-09-18T13:57:39.931 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:39.931 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 2.2 M 2024-09-18T13:57:39.931 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 26 M 2024-09-18T13:57:39.931 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:57:40.061 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-18T13:57:40.109 INFO:teuthology.orchestra.run.smithi084.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 202 kB/s | 26 kB 00:00 2024-09-18T13:57:40.134 INFO:teuthology.orchestra.run.smithi084.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 126 kB/s | 19 kB 00:00 2024-09-18T13:57:40.159 INFO:teuthology.orchestra.run.smithi084.stdout:(3/13): ceph-mgr-rook-17.2.7-1645.g7061ea6c.el9 267 kB/s | 47 kB 00:00 2024-09-18T13:57:40.187 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-18T13:57:40.276 INFO:teuthology.orchestra.run.smithi084.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 222 kB 00:00 2024-09-18T13:57:40.301 INFO:teuthology.orchestra.run.smithi084.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 950 kB/s | 159 kB 00:00 2024-09-18T13:57:40.335 INFO:teuthology.orchestra.run.smithi084.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 918 kB/s | 54 kB 00:00 2024-09-18T13:57:40.369 INFO:teuthology.orchestra.run.smithi084.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 483 kB/s | 32 kB 00:00 2024-09-18T13:57:40.402 INFO:teuthology.orchestra.run.smithi084.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 215 kB/s | 14 kB 00:00 2024-09-18T13:57:40.428 INFO:teuthology.orchestra.run.smithi084.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.0 MB/s | 279 kB 00:00 2024-09-18T13:57:40.461 INFO:teuthology.orchestra.run.smithi084.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.4 MB/s | 220 kB 00:00 2024-09-18T13:57:40.486 INFO:teuthology.orchestra.run.smithi084.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-18T13:57:40.512 INFO:teuthology.orchestra.run.smithi084.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-18T13:57:40.545 INFO:teuthology.orchestra.run.smithi084.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-18T13:57:40.548 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:40.548 INFO:teuthology.orchestra.run.smithi084.stdout:Total 3.6 MB/s | 2.2 MB 00:00 2024-09-18T13:57:40.613 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:57:40.628 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:57:40.628 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:57:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:57:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:57:41.217 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:57:41.383 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-18T13:57:41.438 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-18T13:57:41.494 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-18T13:57:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T13:57:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-18T13:57:41.572 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-18T13:57:41.682 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-18T13:57:41.793 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-18T13:57:41.849 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-18T13:57:42.566 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-18T13:57:42.618 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-18T13:57:42.620 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-18T13:57:42.647 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061e 12/12 2024-09-18T13:57:42.662 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-18T13:57:42.716 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-18T13:57:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 13/13 2024-09-18T13:57:43.519 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 13/13 2024-09-18T13:57:43.519 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 1/13 2024-09-18T13:57:43.519 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-18T13:57:43.519 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-18T13:57:43.519 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-18T13:57:43.519 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-18T13:57:43.519 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-18T13:57:43.519 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-18T13:57:43.519 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-18T13:57:43.520 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-18T13:57:43.520 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-18T13:57:43.520 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-18T13:57:43.520 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-18T13:57:43.858 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-18T13:57:43.858 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:43.858 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:57:43.858 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:43.858 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T13:57:43.858 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T13:57:43.858 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T13:57:43.858 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T13:57:43.859 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T13:57:43.859 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T13:57:43.859 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T13:57:43.859 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T13:57:43.859 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T13:57:43.859 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T13:57:43.859 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T13:57:43.859 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T13:57:43.859 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:43.859 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:44.123 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr-cephadm 2024-09-18T13:57:44.698 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:16 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:44.832 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:44.832 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:44.832 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-18T13:57:44.832 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:44.832 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:57:44.833 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 124 k 2024-09-18T13:57:44.833 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-18T13:57:44.833 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T13:57:44.833 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T13:57:44.833 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T13:57:44.833 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T13:57:44.833 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T13:57:44.833 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:44.833 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:57:44.833 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:44.833 INFO:teuthology.orchestra.run.smithi084.stdout:Install 6 Packages 2024-09-18T13:57:44.833 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:44.834 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 7.0 M 2024-09-18T13:57:44.834 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 33 M 2024-09-18T13:57:44.834 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:57:45.278 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061e 12/12 2024-09-18T13:57:45.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061e 1/12 2024-09-18T13:57:45.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-18T13:57:45.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-18T13:57:45.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-18T13:57:45.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-18T13:57:45.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-18T13:57:45.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-18T13:57:45.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-18T13:57:45.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-18T13:57:45.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T13:57:45.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-18T13:57:45.280 INFO:teuthology.orchestra.run.smithi084.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-09-18T13:57:45.346 INFO:teuthology.orchestra.run.smithi084.stdout:(2/6): ceph-mgr-cephadm-17.2.7-1645.g7061ea6c.e 575 kB/s | 124 kB 00:00 2024-09-18T13:57:45.380 INFO:teuthology.orchestra.run.smithi084.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.4 MB/s | 548 kB 00:00 2024-09-18T13:57:45.413 INFO:teuthology.orchestra.run.smithi084.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.3 MB/s | 74 kB 00:00 2024-09-18T13:57:45.438 INFO:teuthology.orchestra.run.smithi084.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 628 kB/s | 58 kB 00:00 2024-09-18T13:57:45.522 INFO:teuthology.orchestra.run.smithi084.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-18T13:57:45.524 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:45.524 INFO:teuthology.orchestra.run.smithi084.stdout:Total 10 MB/s | 7.0 MB 00:00 2024-09-18T13:57:45.594 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:57:45.605 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:57:45.605 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:57:45.698 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-18T13:57:45.699 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:45.699 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:57:45.699 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:45.699 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T13:57:45.699 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T13:57:45.699 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T13:57:45.699 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T13:57:45.699 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T13:57:45.699 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T13:57:45.699 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T13:57:45.699 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T13:57:45.699 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T13:57:45.700 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T13:57:45.700 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T13:57:45.700 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:45.700 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:57:45.702 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:57:45.702 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:57:46.014 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:57:46.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-rook 2024-09-18T13:57:46.203 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-18T13:57:46.257 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-18T13:57:46.690 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:14 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:57:46.767 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-18T13:57:46.828 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:57:46.829 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:46.829 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 47 k 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T13:57:46.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T13:57:46.831 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T13:57:46.831 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T13:57:46.831 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T13:57:46.831 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:46.831 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:57:46.831 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:46.831 INFO:teuthology.orchestra.run.smithi045.stdout:Install 13 Packages 2024-09-18T13:57:46.831 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:46.832 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 2.2 M 2024-09-18T13:57:46.832 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 26 M 2024-09-18T13:57:46.832 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:57:46.850 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-18T13:57:46.907 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-18T13:57:46.936 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 6/6 2024-09-18T13:57:47.598 INFO:teuthology.orchestra.run.smithi045.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 168 kB/s | 19 kB 00:00 2024-09-18T13:57:47.623 INFO:teuthology.orchestra.run.smithi045.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 183 kB/s | 26 kB 00:00 2024-09-18T13:57:47.649 INFO:teuthology.orchestra.run.smithi045.stdout:(3/13): ceph-mgr-rook-17.2.7-1645.g7061ea6c.el9 288 kB/s | 47 kB 00:00 2024-09-18T13:57:47.699 INFO:teuthology.orchestra.run.smithi045.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-18T13:57:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 6/6 2024-09-18T13:57:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/6 2024-09-18T13:57:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-18T13:57:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-18T13:57:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-18T13:57:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-18T13:57:47.741 INFO:teuthology.orchestra.run.smithi045.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-18T13:57:47.775 INFO:teuthology.orchestra.run.smithi045.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 714 kB/s | 54 kB 00:00 2024-09-18T13:57:47.808 INFO:teuthology.orchestra.run.smithi045.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.7 MB/s | 279 kB 00:00 2024-09-18T13:57:47.833 INFO:teuthology.orchestra.run.smithi045.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 349 kB/s | 32 kB 00:00 2024-09-18T13:57:47.867 INFO:teuthology.orchestra.run.smithi045.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 156 kB/s | 14 kB 00:00 2024-09-18T13:57:47.901 INFO:teuthology.orchestra.run.smithi045.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.9 MB/s | 59 kB 00:00 2024-09-18T13:57:47.926 INFO:teuthology.orchestra.run.smithi045.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-09-18T13:57:47.951 INFO:teuthology.orchestra.run.smithi045.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.5 MB/s | 220 kB 00:00 2024-09-18T13:57:47.972 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-18T13:57:47.972 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:47.972 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:57:47.973 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:47.973 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T13:57:47.973 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T13:57:47.973 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T13:57:47.973 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T13:57:47.973 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T13:57:47.973 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:47.973 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:47.993 INFO:teuthology.orchestra.run.smithi045.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-18T13:57:47.996 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:47.996 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.9 MB/s | 2.2 MB 00:01 2024-09-18T13:57:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:57:48.087 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:57:48.087 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:57:48.186 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:57:48.188 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:57:48.227 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-fuse 2024-09-18T13:57:48.709 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:57:48.804 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:20 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:48.883 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-18T13:57:48.935 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:48.936 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:48.936 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-18T13:57:48.936 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:48.936 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:57:48.936 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 813 k 2024-09-18T13:57:48.936 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-18T13:57:48.936 INFO:teuthology.orchestra.run.smithi084.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T13:57:48.936 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:48.936 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:57:48.936 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:48.936 INFO:teuthology.orchestra.run.smithi084.stdout:Install 2 Packages 2024-09-18T13:57:48.936 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:48.937 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 893 k 2024-09-18T13:57:48.937 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 2.6 M 2024-09-18T13:57:48.937 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:57:48.954 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-18T13:57:49.019 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-18T13:57:49.054 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-18T13:57:49.098 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-18T13:57:49.204 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-18T13:57:49.323 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-18T13:57:49.332 INFO:teuthology.orchestra.run.smithi084.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 548 kB/s | 80 kB 00:00 2024-09-18T13:57:49.375 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-18T13:57:49.440 INFO:teuthology.orchestra.run.smithi084.stdout:(2/2): ceph-fuse-17.2.7-1645.g7061ea6c.el9.x86_ 3.1 MB/s | 813 kB 00:00 2024-09-18T13:57:49.440 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:49.441 INFO:teuthology.orchestra.run.smithi084.stdout:Total 1.7 MB/s | 893 kB 00:00 2024-09-18T13:57:49.452 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:57:49.475 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:57:49.475 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:57:49.555 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:57:49.555 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:57:49.741 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:57:49.883 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-18T13:57:49.948 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/2 2024-09-18T13:57:50.140 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-18T13:57:50.195 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-18T13:57:50.230 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-18T13:57:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-18T13:57:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 13/13 2024-09-18T13:57:50.615 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/2 2024-09-18T13:57:50.615 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/2 2024-09-18T13:57:50.901 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-18T13:57:50.901 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:50.901 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:57:50.901 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-18T13:57:50.901 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:50.901 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 13/13 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 1/13 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-18T13:57:51.158 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-18T13:57:51.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-volume 2024-09-18T13:57:51.514 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-18T13:57:51.514 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:51.514 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:57:51.514 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:51.515 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:57:51.763 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:23 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:51.789 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-cephadm 2024-09-18T13:57:51.898 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:51.899 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:51.899 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-18T13:57:51.899 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:51.899 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:57:51.899 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 270 k 2024-09-18T13:57:51.900 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:51.900 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:57:51.900 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:51.900 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-18T13:57:51.900 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:51.900 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 270 k 2024-09-18T13:57:51.900 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 1.3 M 2024-09-18T13:57:51.900 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:57:52.127 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-volume-17.2.7-1645.g7061ea6c.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-18T13:57:52.128 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:52.128 INFO:teuthology.orchestra.run.smithi084.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-18T13:57:52.129 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:57:52.136 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:57:52.137 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:57:52.164 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:57:52.164 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:57:52.375 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:57:52.392 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:20 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:57:52.445 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:57:52.461 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:57:52.461 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:57:52.462 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T13:57:52.462 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:52.530 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:57:52.531 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:52.531 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-18T13:57:52.531 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:52.531 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:57:52.531 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 124 k 2024-09-18T13:57:52.531 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-18T13:57:52.531 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T13:57:52.531 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T13:57:52.532 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T13:57:52.532 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T13:57:52.532 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T13:57:52.532 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:52.532 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:57:52.532 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:52.532 INFO:teuthology.orchestra.run.smithi045.stdout:Install 6 Packages 2024-09-18T13:57:52.532 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:52.533 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 7.0 M 2024-09-18T13:57:52.533 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 33 M 2024-09-18T13:57:52.533 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:57:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1645.g7061ea6c.e 639 kB/s | 124 kB 00:00 2024-09-18T13:57:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.0 MB/s | 548 kB 00:00 2024-09-18T13:57:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-18T13:57:52.974 INFO:teuthology.orchestra.run.smithi045.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-18T13:57:53.441 INFO:teuthology.orchestra.run.smithi045.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 319 kB/s | 248 kB 00:00 2024-09-18T13:57:53.463 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:57:53.463 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:53.463 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:57:53.463 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:53.463 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:53.463 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:53.758 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install librados-devel 2024-09-18T13:57:53.799 INFO:teuthology.orchestra.run.smithi045.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-18T13:57:53.802 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:53.802 INFO:teuthology.orchestra.run.smithi045.stdout:Total 5.5 MB/s | 7.0 MB 00:01 2024-09-18T13:57:53.878 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:57:53.891 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:57:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:57:53.997 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:57:53.998 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:57:54.326 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:57:54.352 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:26 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:54.485 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:54.485 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:54.485 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repo Size 2024-09-18T13:57:54.485 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:54.486 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:57:54.486 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 126 k 2024-09-18T13:57:54.486 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:54.486 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:57:54.486 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:54.486 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-18T13:57:54.486 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:54.486 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 126 k 2024-09-18T13:57:54.486 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 448 k 2024-09-18T13:57:54.486 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:57:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-18T13:57:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-18T13:57:54.681 INFO:teuthology.orchestra.run.smithi084.stdout:librados-devel-17.2.7-1645.g7061ea6c.el9.x86_64 649 kB/s | 126 kB 00:00 2024-09-18T13:57:54.681 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:54.682 INFO:teuthology.orchestra.run.smithi084.stdout:Total 644 kB/s | 126 kB 00:00 2024-09-18T13:57:54.682 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:57:54.687 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:57:54.687 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:57:54.751 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:57:54.751 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:57:54.897 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:57:55.003 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:57:55.116 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-18T13:57:55.208 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-18T13:57:55.287 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-18T13:57:55.313 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 6/6 2024-09-18T13:57:55.460 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:57:55.721 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:57:55.721 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:55.721 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:57:55.721 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:55.721 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:55.721 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:55.993 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install libcephfs2 2024-09-18T13:57:56.013 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 6/6 2024-09-18T13:57:56.013 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/6 2024-09-18T13:57:56.013 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-18T13:57:56.013 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-18T13:57:56.013 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-18T13:57:56.013 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-18T13:57:56.295 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-18T13:57:56.296 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:56.296 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:57:56.296 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:57:56.296 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T13:57:56.296 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T13:57:56.296 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T13:57:56.296 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T13:57:56.296 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T13:57:56.296 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:56.296 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:57:56.565 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-fuse 2024-09-18T13:57:56.589 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:28 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:56.680 INFO:teuthology.orchestra.run.smithi084.stdout:Package libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:57:56.725 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:56.726 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-18T13:57:56.726 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:56.820 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install libcephfs-devel 2024-09-18T13:57:57.175 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:25 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:57:57.313 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:57:57.313 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-18T13:57:57.313 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:57.313 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:57:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 813 k 2024-09-18T13:57:57.313 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-18T13:57:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T13:57:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:57.314 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:57:57.314 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:57:57.314 INFO:teuthology.orchestra.run.smithi045.stdout:Install 2 Packages 2024-09-18T13:57:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:57.314 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 893 k 2024-09-18T13:57:57.315 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 2.6 M 2024-09-18T13:57:57.315 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:57:57.397 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:29 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:57.526 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:57.526 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:57.526 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repo Size 2024-09-18T13:57:57.526 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:57.526 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:57:57.526 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 25 k 2024-09-18T13:57:57.526 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:57.527 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:57:57.527 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:57:57.527 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-18T13:57:57.527 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:57.527 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 25 k 2024-09-18T13:57:57.527 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 110 k 2024-09-18T13:57:57.527 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:57:57.585 INFO:teuthology.orchestra.run.smithi045.stdout:(1/2): ceph-fuse-17.2.7-1645.g7061ea6c.el9.x86_ 3.2 MB/s | 813 kB 00:00 2024-09-18T13:57:57.619 INFO:teuthology.orchestra.run.smithi045.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 284 kB/s | 80 kB 00:00 2024-09-18T13:57:57.619 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:57.619 INFO:teuthology.orchestra.run.smithi045.stdout:Total 2.9 MB/s | 893 kB 00:00 2024-09-18T13:57:57.632 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:57:57.655 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:57:57.655 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:57:57.676 INFO:teuthology.orchestra.run.smithi084.stdout:libcephfs-devel-17.2.7-1645.g7061ea6c.el9.x86_6 168 kB/s | 25 kB 00:00 2024-09-18T13:57:57.676 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:57:57.677 INFO:teuthology.orchestra.run.smithi084.stdout:Total 167 kB/s | 25 kB 00:00 2024-09-18T13:57:57.677 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:57:57.680 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:57:57.680 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:57:57.682 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:57:57.682 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:57:57.740 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:57:57.741 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:57:57.771 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:57:57.889 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:57:57.934 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:57:58.076 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-18T13:57:58.146 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/2 2024-09-18T13:57:58.269 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:57:58.511 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:57:58.511 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:58.511 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:57:58.512 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:57:58.512 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:57:58.512 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:58.684 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install librados2 2024-09-18T13:57:58.809 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/2 2024-09-18T13:57:58.809 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/2 2024-09-18T13:57:59.092 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-18T13:57:59.092 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:59.092 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:57:59.092 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-18T13:57:59.092 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:57:59.092 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:57:59.256 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:31 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:57:59.344 INFO:teuthology.orchestra.run.smithi084.stdout:Package librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:57:59.375 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-volume 2024-09-18T13:57:59.387 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:57:59.388 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-18T13:57:59.388 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:57:59.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install librbd1 2024-09-18T13:57:59.984 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:27 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:00.034 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:32 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:58:00.121 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:00.122 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:00.122 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-18T13:58:00.122 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:00.122 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:58:00.122 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 270 k 2024-09-18T13:58:00.122 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:00.123 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:58:00.123 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:00.123 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-18T13:58:00.123 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:00.123 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 270 k 2024-09-18T13:58:00.123 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 1.3 M 2024-09-18T13:58:00.123 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:58:00.123 INFO:teuthology.orchestra.run.smithi084.stdout:Package librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:58:00.164 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:58:00.166 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-18T13:58:00.166 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:58:00.238 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install python3-rados 2024-09-18T13:58:00.349 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-volume-17.2.7-1645.g7061ea6c.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-18T13:58:00.350 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:58:00.350 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-18T13:58:00.350 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:58:00.359 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:58:00.360 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:58:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:58:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:58:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:58:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:58:00.702 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:58:00.702 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:58:00.702 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T13:58:00.702 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:00.813 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:32 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:58:00.902 INFO:teuthology.orchestra.run.smithi084.stdout:Package python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:58:00.944 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:58:00.946 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-18T13:58:00.946 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:58:01.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install python3-rgw 2024-09-18T13:58:01.600 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:33 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:58:01.689 INFO:teuthology.orchestra.run.smithi084.stdout:Package python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:58:01.696 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T13:58:01.696 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:01.696 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:58:01.696 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T13:58:01.696 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:01.696 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:01.732 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:58:01.733 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-18T13:58:01.733 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:58:01.808 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install python3-cephfs 2024-09-18T13:58:01.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados-devel 2024-09-18T13:58:02.383 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:34 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:58:02.472 INFO:teuthology.orchestra.run.smithi084.stdout:Package python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:58:02.514 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:58:02.515 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-18T13:58:02.515 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:58:02.592 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install python3-rbd 2024-09-18T13:58:02.598 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:30 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:02.737 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 126 k 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 126 k 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 448 k 2024-09-18T13:58:02.738 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:58:02.939 INFO:teuthology.orchestra.run.smithi045.stdout:librados-devel-17.2.7-1645.g7061ea6c.el9.x86_64 630 kB/s | 126 kB 00:00 2024-09-18T13:58:02.939 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:58:02.940 INFO:teuthology.orchestra.run.smithi045.stdout:Total 626 kB/s | 126 kB 00:00 2024-09-18T13:58:02.940 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:58:02.945 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:58:02.946 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:58:03.013 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:58:03.013 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:58:03.167 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:35 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:58:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:58:03.256 INFO:teuthology.orchestra.run.smithi084.stdout:Package python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:58:03.281 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:03.298 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:58:03.299 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-18T13:58:03.299 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:58:03.375 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install rbd-fuse 2024-09-18T13:58:03.730 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:03.953 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:35 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:58:03.991 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:03.991 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:03.992 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:58:03.992 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:58:03.992 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:03.992 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:04.083 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:58:04.084 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:58:04.084 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-09-18T13:58:04.084 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:58:04.084 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:58:04.084 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 82 k 2024-09-18T13:58:04.084 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:04.084 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:58:04.084 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:58:04.084 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-18T13:58:04.084 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:04.085 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 82 k 2024-09-18T13:58:04.085 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 222 k 2024-09-18T13:58:04.085 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:58:04.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs2 2024-09-18T13:58:04.278 INFO:teuthology.orchestra.run.smithi084.stdout:rbd-fuse-17.2.7-1645.g7061ea6c.el9.x86_64.rpm 424 kB/s | 82 kB 00:00 2024-09-18T13:58:04.278 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:58:04.279 INFO:teuthology.orchestra.run.smithi084.stdout:Total 422 kB/s | 82 kB 00:00 2024-09-18T13:58:04.279 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:58:04.285 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:58:04.286 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:58:04.348 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:58:04.348 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:58:04.505 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:58:04.600 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:04.866 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:32 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:04.960 INFO:teuthology.orchestra.run.smithi045.stdout:Package libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:58:05.000 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:05.005 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:05.006 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-18T13:58:05.007 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:05.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs-devel 2024-09-18T13:58:05.244 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:05.244 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:05.244 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:58:05.244 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:58:05.244 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:05.244 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:58:05.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install rbd-mirror 2024-09-18T13:58:05.687 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:33 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:05.824 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:05.825 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-09-18T13:58:05.825 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:05.825 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:58:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 25 k 2024-09-18T13:58:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:05.825 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:58:05.826 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:05.826 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-18T13:58:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:05.826 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 25 k 2024-09-18T13:58:05.826 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 110 k 2024-09-18T13:58:05.826 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:58:05.976 INFO:teuthology.orchestra.run.smithi045.stdout:libcephfs-devel-17.2.7-1645.g7061ea6c.el9.x86_6 166 kB/s | 25 kB 00:00 2024-09-18T13:58:05.977 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:58:05.977 INFO:teuthology.orchestra.run.smithi045.stdout:Total 165 kB/s | 25 kB 00:00 2024-09-18T13:58:05.977 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:58:05.981 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:58:05.981 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:58:05.982 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:58:05.983 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:58:06.072 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:58:06.107 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:38 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:58:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:06.245 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:58:06.246 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:58:06.246 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repo Size 2024-09-18T13:58:06.246 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:58:06.246 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:58:06.246 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 2.9 M 2024-09-18T13:58:06.246 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:06.246 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:58:06.246 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:58:06.246 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-18T13:58:06.246 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:06.246 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 2.9 M 2024-09-18T13:58:06.246 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 12 M 2024-09-18T13:58:06.247 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:58:06.536 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:06.606 INFO:teuthology.orchestra.run.smithi084.stdout:rbd-mirror-17.2.7-1645.g7061ea6c.el9.x86_64.rpm 8.2 MB/s | 2.9 MB 00:00 2024-09-18T13:58:06.606 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:58:06.607 INFO:teuthology.orchestra.run.smithi084.stdout:Total 8.1 MB/s | 2.9 MB 00:00 2024-09-18T13:58:06.607 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:58:06.616 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:58:06.616 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:58:06.694 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:58:06.695 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:58:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:06.794 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:58:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:58:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:06.794 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:07.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados2 2024-09-18T13:58:07.106 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:58:07.182 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:07.206 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:07.206 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:58:07.207 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T13:58:07.207 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T13:58:07.207 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T13:58:07.207 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:07.599 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:35 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:07.692 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:58:07.737 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:07.738 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-18T13:58:07.738 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:07.818 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librbd1 2024-09-18T13:58:08.360 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:08.361 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:08.361 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:58:08.361 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:58:08.361 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:08.361 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:58:08.422 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:36 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:08.516 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:58:08.560 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:08.561 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-18T13:58:08.562 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:08.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rados 2024-09-18T13:58:08.648 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install rbd-nbd 2024-09-18T13:58:09.242 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:41 ago on Wed 18 Sep 2024 01:56:28 PM UTC. 2024-09-18T13:58:09.247 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:37 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:09.340 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:58:09.375 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-18T13:58:09.375 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:58:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-09-18T13:58:09.376 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:58:09.376 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-18T13:58:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 165 k 2024-09-18T13:58:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:09.376 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-18T13:58:09.376 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-18T13:58:09.376 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-18T13:58:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:09.376 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 165 k 2024-09-18T13:58:09.377 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 477 k 2024-09-18T13:58:09.377 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-18T13:58:09.385 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:09.386 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-18T13:58:09.386 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:09.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rgw 2024-09-18T13:58:09.584 INFO:teuthology.orchestra.run.smithi084.stdout:rbd-nbd-17.2.7-1645.g7061ea6c.el9.x86_64.rpm 796 kB/s | 165 kB 00:00 2024-09-18T13:58:09.585 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:58:09.585 INFO:teuthology.orchestra.run.smithi084.stdout:Total 788 kB/s | 165 kB 00:00 2024-09-18T13:58:09.585 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-18T13:58:09.593 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-18T13:58:09.593 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-18T13:58:09.659 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-18T13:58:09.659 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-18T13:58:09.819 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-18T13:58:09.924 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:10.072 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:38 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:10.165 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:58:10.210 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:10.211 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-18T13:58:10.211 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:10.301 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-cephfs 2024-09-18T13:58:10.377 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:10.634 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:10.634 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:10.634 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-18T13:58:10.635 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:58:10.635 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:10.635 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-18T13:58:10.889 DEBUG:teuthology.parallel:result is None 2024-09-18T13:58:10.907 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:38 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:10.999 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:58:11.045 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:11.046 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-18T13:58:11.046 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:11.126 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rbd 2024-09-18T13:58:11.728 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:39 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:11.821 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T13:58:11.866 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:11.867 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-18T13:58:11.867 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:11.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-fuse 2024-09-18T13:58:12.551 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:40 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:12.688 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:12.689 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-09-18T13:58:12.689 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:12.689 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:58:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 82 k 2024-09-18T13:58:12.689 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:12.689 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:58:12.689 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:12.689 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-18T13:58:12.690 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:12.690 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 82 k 2024-09-18T13:58:12.690 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 222 k 2024-09-18T13:58:12.690 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:58:12.880 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-fuse-17.2.7-1645.g7061ea6c.el9.x86_64.rpm 430 kB/s | 82 kB 00:00 2024-09-18T13:58:12.881 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:58:12.881 INFO:teuthology.orchestra.run.smithi045.stdout:Total 427 kB/s | 82 kB 00:00 2024-09-18T13:58:12.881 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:58:12.888 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:58:12.889 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:58:12.955 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:58:12.955 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:58:13.116 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:58:13.205 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:13.605 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:13.889 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:13.890 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:58:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:58:13.890 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:13.890 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:14.141 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-mirror 2024-09-18T13:58:14.751 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:42 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:14.890 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:14.891 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-09-18T13:58:14.891 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:14.891 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:58:14.891 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 2.9 M 2024-09-18T13:58:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:14.892 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:58:14.892 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:14.892 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-18T13:58:14.892 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:14.892 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 2.9 M 2024-09-18T13:58:14.892 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 12 M 2024-09-18T13:58:14.892 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:58:15.234 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-mirror-17.2.7-1645.g7061ea6c.el9.x86_64.rpm 8.6 MB/s | 2.9 MB 00:00 2024-09-18T13:58:15.234 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:58:15.235 INFO:teuthology.orchestra.run.smithi045.stdout:Total 8.6 MB/s | 2.9 MB 00:00 2024-09-18T13:58:15.235 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:58:15.245 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:58:15.245 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:58:15.328 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:58:15.328 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:58:15.766 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:58:15.827 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:15.853 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:15.854 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T13:58:15.854 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T13:58:15.854 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T13:58:15.854 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T13:58:15.854 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:16.985 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:16.985 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:16.985 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:58:16.985 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:58:16.985 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:16.985 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:17.239 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-nbd 2024-09-18T13:58:17.845 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:45 ago on Wed 18 Sep 2024 01:56:32 PM UTC. 2024-09-18T13:58:17.984 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-18T13:58:17.984 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:17.985 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-09-18T13:58:17.985 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:17.985 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-18T13:58:17.985 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 165 k 2024-09-18T13:58:17.985 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:17.985 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-18T13:58:17.985 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-18T13:58:17.985 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-18T13:58:17.985 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:17.985 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 165 k 2024-09-18T13:58:17.985 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 477 k 2024-09-18T13:58:17.985 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-18T13:58:18.197 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-nbd-17.2.7-1645.g7061ea6c.el9.x86_64.rpm 780 kB/s | 165 kB 00:00 2024-09-18T13:58:18.198 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-18T13:58:18.198 INFO:teuthology.orchestra.run.smithi045.stdout:Total 775 kB/s | 165 kB 00:00 2024-09-18T13:58:18.198 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-18T13:58:18.206 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-18T13:58:18.207 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-18T13:58:18.273 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-18T13:58:18.273 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-18T13:58:18.435 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-18T13:58:18.531 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:18.940 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T13:58:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:19.226 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-18T13:58:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T13:58:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:19.227 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-18T13:58:19.529 DEBUG:teuthology.parallel:result is None 2024-09-18T13:58:19.530 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T13:58:19.675 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T13:58:19.697 INFO:teuthology.orchestra.run.smithi045.stdout:17.2.7-1645.g7061ea6c.el9 2024-09-18T13:58:19.698 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1645.g7061ea6c.el9 2024-09-18T13:58:19.698 INFO:teuthology.task.install:The correct ceph version 17.2.7-1645.g7061ea6c is installed. 2024-09-18T13:58:19.700 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T13:58:19.848 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T13:58:19.877 INFO:teuthology.orchestra.run.smithi084.stdout:17.2.7-1645.g7061ea6c.el9 2024-09-18T13:58:19.877 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1645.g7061ea6c.el9 2024-09-18T13:58:19.877 INFO:teuthology.task.install:The correct ceph version 17.2.7-1645.g7061ea6c is installed. 2024-09-18T13:58:19.879 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-18T13:58:19.879 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:19.879 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T13:58:19.906 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:19.906 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T13:58:19.947 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-18T13:58:19.947 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:19.947 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T13:58:19.973 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T13:58:20.039 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:20.039 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T13:58:20.067 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T13:58:20.134 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-18T13:58:20.134 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:20.134 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T13:58:20.160 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T13:58:20.228 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:20.228 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T13:58:20.255 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T13:58:20.321 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-18T13:58:20.321 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:20.321 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T13:58:20.347 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T13:58:20.413 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:20.413 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T13:58:20.438 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T13:58:20.502 INFO:teuthology.run_tasks:Running task ceph... 2024-09-18T13:58:20.584 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-18T13:58:20.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /var/log/ceph 2024-09-18T13:58:20.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /var/log/ceph 2024-09-18T13:58:20.616 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-18T13:58:20.616 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T13:58:20.660 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T13:58:20.684 INFO:tasks.ceph:Creating extra log directories... 2024-09-18T13:58:20.685 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T13:58:20.727 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T13:58:20.761 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-18T13:58:20.761 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2024-09-18T13:58:20.761 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070', 'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'description': 'rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/snaps-few-objects}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7911070', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'mon_bind_msgr2': False, 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'], ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'seed': 4631, 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'targets': {'smithi045.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSskIyXoDcfzvr8QaEoieurcwLDOdKDrlLq0XuF1nNGB4o8ZAg3sCTTPolWzycXOBvVtX9c4WsqoUCoGESjjGk=', 'smithi084.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFU+1UwJLA6P6DbOnwn117EcqYfhQMkIVNSOfSOj3mb4kHq2jhp0KU/fTMD4tl82iwuGtyPiODyA9aWk5xnF2o0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'revive_delay': 90, 'thrash_delay': 1, 'thrash_many': True}}, {'rados': {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-18_09:13:00', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-18T13:58:20.762 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T13:58:20.793 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T13:58:20.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T13:58:20.850 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T13:58:20.874 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:20.874 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T13:58:20.930 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T13:58:20.930 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-09-18T13:58:20.986 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T13:58:20.986 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T13:58:20.986 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-18T13:58:20.986 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T13:58:20.986 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T13:58:20.986 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-18 13:58:16.119645653 +0000 2024-09-18T13:58:20.986 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-18 13:55:48.120943259 +0000 2024-09-18T13:58:20.986 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-18 13:55:48.120943259 +0000 2024-09-18T13:58:20.986 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-18 13:55:48.120943259 +0000 2024-09-18T13:58:20.987 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T13:58:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-18T13:58:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-18T13:58:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 8.9952e-05 s, 5.7 MB/s 2024-09-18T13:58:21.051 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T13:58:21.108 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-09-18T13:58:21.164 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T13:58:21.164 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T13:58:21.164 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-18T13:58:21.164 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T13:58:21.164 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T13:58:21.164 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-18 13:58:16.119645653 +0000 2024-09-18T13:58:21.164 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-18 13:55:48.437938565 +0000 2024-09-18T13:58:21.164 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-18 13:55:48.437938565 +0000 2024-09-18T13:58:21.165 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-18 13:55:48.437938565 +0000 2024-09-18T13:58:21.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T13:58:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-18T13:58:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-18T13:58:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.5288e-05 s, 5.4 MB/s 2024-09-18T13:58:21.229 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T13:58:21.286 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-09-18T13:58:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T13:58:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T13:58:21.342 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-18T13:58:21.342 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T13:58:21.342 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T13:58:21.342 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-18 13:58:16.119645653 +0000 2024-09-18T13:58:21.342 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-18 13:55:48.749933946 +0000 2024-09-18T13:58:21.342 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-18 13:55:48.749933946 +0000 2024-09-18T13:58:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-18 13:55:48.749933946 +0000 2024-09-18T13:58:21.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T13:58:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-18T13:58:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-18T13:58:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.7383e-05 s, 5.3 MB/s 2024-09-18T13:58:21.407 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T13:58:21.465 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-09-18T13:58:21.521 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T13:58:21.521 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T13:58:21.521 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-18T13:58:21.522 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T13:58:21.522 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T13:58:21.522 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-18 13:58:16.120645639 +0000 2024-09-18T13:58:21.522 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-18 13:55:49.052929459 +0000 2024-09-18T13:58:21.522 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-18 13:55:49.052929459 +0000 2024-09-18T13:58:21.522 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-18 13:55:49.052929459 +0000 2024-09-18T13:58:21.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T13:58:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-18T13:58:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-18T13:58:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 8.7091e-05 s, 5.9 MB/s 2024-09-18T13:58:21.587 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T13:58:21.644 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-18T13:58:21.645 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:21.645 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T13:58:21.664 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T13:58:21.664 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2024-09-18T13:58:21.722 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T13:58:21.722 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T13:58:21.722 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-18T13:58:21.722 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T13:58:21.722 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T13:58:21.722 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-09-18 13:58:07.466537030 +0000 2024-09-18T13:58:21.722 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-18 13:55:46.819957451 +0000 2024-09-18T13:58:21.722 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-18 13:55:46.819957451 +0000 2024-09-18T13:58:21.722 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-09-18 13:55:46.819957451 +0000 2024-09-18T13:58:21.723 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T13:58:21.788 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-09-18T13:58:21.788 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-09-18T13:58:21.788 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000217597 s, 2.4 MB/s 2024-09-18T13:58:21.789 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T13:58:21.847 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2024-09-18T13:58:21.904 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T13:58:21.904 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T13:58:21.904 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-18T13:58:21.904 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T13:58:21.904 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T13:58:21.904 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-09-18 13:58:07.466537030 +0000 2024-09-18T13:58:21.904 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-18 13:55:47.150951649 +0000 2024-09-18T13:58:21.904 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-18 13:55:47.150951649 +0000 2024-09-18T13:58:21.904 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-09-18 13:55:47.150951649 +0000 2024-09-18T13:58:21.905 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T13:58:21.969 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-09-18T13:58:21.969 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-09-18T13:58:21.969 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000220712 s, 2.3 MB/s 2024-09-18T13:58:21.970 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T13:58:22.027 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2024-09-18T13:58:22.083 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T13:58:22.083 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T13:58:22.083 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-18T13:58:22.083 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T13:58:22.084 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T13:58:22.084 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-09-18 13:58:07.466537030 +0000 2024-09-18T13:58:22.084 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-18 13:55:47.469946056 +0000 2024-09-18T13:58:22.084 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-18 13:55:47.469946056 +0000 2024-09-18T13:58:22.084 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-09-18 13:55:47.469946056 +0000 2024-09-18T13:58:22.084 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T13:58:22.147 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-09-18T13:58:22.147 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-09-18T13:58:22.147 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000176887 s, 2.9 MB/s 2024-09-18T13:58:22.149 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T13:58:22.205 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2024-09-18T13:58:22.261 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T13:58:22.261 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T13:58:22.261 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-18T13:58:22.261 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T13:58:22.261 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T13:58:22.261 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-09-18 13:58:07.467537012 +0000 2024-09-18T13:58:22.261 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-18 13:55:47.799940271 +0000 2024-09-18T13:58:22.262 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-18 13:55:47.799940271 +0000 2024-09-18T13:58:22.262 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-09-18 13:55:47.799940271 +0000 2024-09-18T13:58:22.262 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T13:58:22.325 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-09-18T13:58:22.325 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-09-18T13:58:22.325 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.00017364 s, 2.9 MB/s 2024-09-18T13:58:22.326 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T13:58:22.382 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-18T13:58:22.383 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi084.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-18T13:58:22.383 INFO:tasks.ceph:Generating config... 2024-09-18T13:58:22.384 INFO:tasks.ceph:[client] debug monc = 20 2024-09-18T13:58:22.384 INFO:tasks.ceph:[client] debug ms = 1 2024-09-18T13:58:22.384 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-18T13:58:22.384 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-18T13:58:22.384 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-09-18T13:58:22.384 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-18T13:58:22.385 INFO:tasks.ceph:[global] ms type = async 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-18T13:58:22.385 INFO:tasks.ceph:[mon] paxos min = 10 2024-09-18T13:58:22.386 INFO:tasks.ceph:[mon] paxos service trim min = 5 2024-09-18T13:58:22.386 INFO:tasks.ceph:[mon] paxos trim min = 10 2024-09-18T13:58:22.386 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-18T13:58:22.386 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-18T13:58:22.386 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2024-09-18T13:58:22.386 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-18T13:58:22.386 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-18T13:58:22.386 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-18T13:58:22.386 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-18T13:58:22.386 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-18T13:58:22.386 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-18T13:58:22.387 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-18T13:58:22.387 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-18T13:58:22.387 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-18T13:58:22.387 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-18T13:58:22.387 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-18T13:58:22.387 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-18T13:58:22.387 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-18T13:58:22.387 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-18T13:58:22.387 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-18T13:58:22.387 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-18T13:58:22.387 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-18T13:58:22.387 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-18T13:58:22.388 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-18T13:58:22.388 INFO:tasks.ceph:Setting up mon.a... 2024-09-18T13:58:22.388 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-18T13:58:22.468 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.keyring 2024-09-18T13:58:22.471 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-18T13:58:22.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T13:58:22.582 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.45:6789]'), ('mon.b', '[v1:172.21.15.45:6790]'), ('mon.c', '[v1:172.21.15.45:6791]'), ('mon.d', '[v1:172.21.15.45:6792]'), ('mon.e', '[v1:172.21.15.45:6793]'), ('mon.f', '[v1:172.21.15.84:6789]'), ('mon.g', '[v1:172.21.15.84:6790]'), ('mon.h', '[v1:172.21.15.84:6791]'), ('mon.i', '[v1:172.21.15.84:6792]')] 2024-09-18T13:58:22.582 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.45:6789],[v1:172.21.15.45:6790],[v1:172.21.15.45:6791],[v1:172.21.15.45:6792],[v1:172.21.15.45:6793],[v1:172.21.15.84:6789],[v1:172.21.15.84:6790],[v1:172.21.15.84:6791],[v1:172.21.15.84:6792]', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mon.d': {}, 'mon.e': {}, 'mon.f': {}, 'mon.g': {}, 'mon.h': {}, 'mon.i': {}} 2024-09-18T13:58:22.583 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:22.583 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T13:58:22.639 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.45:6789]' --addv b '[v1:172.21.15.45:6790]' --addv c '[v1:172.21.15.45:6791]' --addv d '[v1:172.21.15.45:6792]' --addv e '[v1:172.21.15.45:6793]' --addv f '[v1:172.21.15.84:6789]' --addv g '[v1:172.21.15.84:6790]' --addv h '[v1:172.21.15.84:6791]' --addv i '[v1:172.21.15.84:6792]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-18T13:58:22.742 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-18T13:58:22.742 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: generated fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:22.742 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = octopus 2024-09-18T13:58:22.742 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 0 2024-09-18T13:58:22.742 INFO:teuthology.orchestra.run.smithi045.stdout:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:22.742 INFO:teuthology.orchestra.run.smithi045.stdout:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:22.742 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:22.742 INFO:teuthology.orchestra.run.smithi045.stdout:min_mon_release 15 (octopus) 2024-09-18T13:58:22.742 INFO:teuthology.orchestra.run.smithi045.stdout:election_strategy: 3 2024-09-18T13:58:22.742 INFO:teuthology.orchestra.run.smithi045.stdout:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:22.743 INFO:teuthology.orchestra.run.smithi045.stdout:1: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:22.743 INFO:teuthology.orchestra.run.smithi045.stdout:2: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:22.743 INFO:teuthology.orchestra.run.smithi045.stdout:3: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:22.743 INFO:teuthology.orchestra.run.smithi045.stdout:4: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:22.743 INFO:teuthology.orchestra.run.smithi045.stdout:5: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:22.743 INFO:teuthology.orchestra.run.smithi045.stdout:6: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:22.743 INFO:teuthology.orchestra.run.smithi045.stdout:7: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:22.743 INFO:teuthology.orchestra.run.smithi045.stdout:8: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:22.743 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (9 monitors) 2024-09-18T13:58:22.744 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T13:58:22.760 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0f5ca4bb-86c8-4c54-b818-324eb902ab17... 2024-09-18T13:58:22.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T13:58:22.802 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T13:58:22.846 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-09-18T13:58:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-09-18T13:58:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T13:58:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-09-18T13:58:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-09-18T13:58:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-09-18T13:58:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-09-18T13:58:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-09-18T13:58:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-09-18T13:58:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-09-18T13:58:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T13:58:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-09-18T13:58:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T13:58:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "[v1:172.21.15.45:6789],[v1:172.21.15.45:6790],[v1:172.21.15.45:6791],[v1:172.21.15.45:6792],[v1:172.21.15.45:6793],[v1:172.21.15.84:6789],[v1:172.21.15.84:6790],[v1:172.21.15.84:6791],[v1:172.21.15.84:6792]" 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: mon client directed command retry = 5 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: mon election default strategy = 3 2024-09-18T13:58:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: ms bind msgr2 = False 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: ms inject socket failures = 5000 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: ms type = async 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-09-18T13:58:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-09-18T13:58:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: bdev async discard = True 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: bdev enable discard = True 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore allocator = stupid 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore zero block detection = True 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 20 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 20 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-09-18T13:58:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 10 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify cached snaps = True 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify missing on start = True 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock override recovery settings = True 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: osd op queue = debug_random 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: osd op queue cut off = debug_random 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-18T13:58:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-09-18T13:58:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-09-18T13:58:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-09-18T13:58:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-09-18T13:58:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-09-18T13:58:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-09-18T13:58:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.854 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-09-18T13:58:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-18T13:58:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-09-18T13:58:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-09-18T13:58:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T13:58:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: mon min osdmap epochs = 25 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: mon osdmap full prune interval = 2 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: mon osdmap full prune min = 15 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: mon osdmap full prune txsize = 2 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: mon scrub inject crc mismatch = 0.01 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: mon scrub inject missing keys = 0.05 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: mon scrub interval = 300 2024-09-18T13:58:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: paxos min = 10 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: paxos service trim min = 5 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: paxos trim min = 10 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: debug monc = 20 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-09-18T13:58:22.857 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-09-18T13:58:22.858 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.d] 2024-09-18T13:58:22.858 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.e] 2024-09-18T13:58:22.858 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.f] 2024-09-18T13:58:22.858 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.g] 2024-09-18T13:58:22.858 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.h] 2024-09-18T13:58:22.858 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.i] 2024-09-18T13:58:22.858 INFO:teuthology.orchestra.run.smithi084.stdout:[global] 2024-09-18T13:58:22.859 INFO:teuthology.orchestra.run.smithi084.stdout: chdir = "" 2024-09-18T13:58:22.859 INFO:teuthology.orchestra.run.smithi084.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T13:58:22.859 INFO:teuthology.orchestra.run.smithi084.stdout: auth supported = cephx 2024-09-18T13:58:22.859 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.859 INFO:teuthology.orchestra.run.smithi084.stdout: filestore xattr use omap = true 2024-09-18T13:58:22.859 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.859 INFO:teuthology.orchestra.run.smithi084.stdout: mon clock drift allowed = 1.000 2024-09-18T13:58:22.859 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.859 INFO:teuthology.orchestra.run.smithi084.stdout: osd crush chooseleaf type = 0 2024-09-18T13:58:22.859 INFO:teuthology.orchestra.run.smithi084.stdout: auth debug = true 2024-09-18T13:58:22.859 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.859 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on old message = true 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on bug = true 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: mon pg warn max object skew = 0 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: osd pool default size = 2 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd allow primary affinity = true 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd allow pg remap = true 2024-09-18T13:58:22.860 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on legacy crush tunables = false 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on crush straw calc version zero = false 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on no sortbitwise = false 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on osd down out interval zero = false 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on too few osds = false 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: mon_allow_pool_size_one = true 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: osd default data pool replay window = 5 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: mon allow pool delete = true 2024-09-18T13:58:22.861 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout: mon cluster log file level = debug 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout: debug asserts on shutdown = true 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout: mon health detail to clog = false 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout: mon host = "[v1:172.21.15.45:6789],[v1:172.21.15.45:6790],[v1:172.21.15.45:6791],[v1:172.21.15.45:6792],[v1:172.21.15.45:6793],[v1:172.21.15.84:6789],[v1:172.21.15.84:6790],[v1:172.21.15.84:6791],[v1:172.21.15.84:6792]" 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout: mon client directed command retry = 5 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout: mon election default strategy = 3 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout: ms bind msgr2 = False 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout: ms inject socket failures = 5000 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout: ms type = async 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout: fsid = 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout:[osd] 2024-09-18T13:58:22.862 INFO:teuthology.orchestra.run.smithi084.stdout: osd journal size = 100 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: osd scrub load threshold = 5.0 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: osd scrub max interval = 600 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: osd mclock profile = high_recovery_ops 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: osd recover clone overlap = true 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: osd recovery max chunk = 1048576 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug shutdown = true 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug op order = true 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug verify stray on activate = true 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: osd open classes on start = true 2024-09-18T13:58:22.863 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug pg log writeout = true 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: osd deep scrub update digest min age = 30 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: osd map max advance = 10 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: journal zero on create = true 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: filestore ondisk finisher threads = 3 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: filestore apply finisher threads = 3 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: bdev debug aio = true 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug misdirected ops = true 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: bdev async discard = True 2024-09-18T13:58:22.864 INFO:teuthology.orchestra.run.smithi084.stdout: bdev enable discard = True 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore allocator = stupid 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore block size = 96636764160 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore fsck on mount = True 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore zero block detection = True 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: debug bluefs = 20 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: debug bluestore = 20 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: debug osd = 20 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: debug rocksdb = 10 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd full ratio = 0.9 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd nearfull ratio = 0.8 2024-09-18T13:58:22.865 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug verify cached snaps = True 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug verify missing on start = True 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: osd failsafe full ratio = 0.95 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: osd mclock override recovery settings = True 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: osd objectstore = bluestore 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: osd op queue = debug_random 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: osd op queue cut off = debug_random 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr] 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: debug mgr = 20 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: debug mon = 20 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: debug auth = 20 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min pgs per osd = 4 2024-09-18T13:58:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min bytes per osd = 10 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: mgr/telemetry/nag = false 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout:[mon] 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: debug mon = 20 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: debug paxos = 20 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: debug auth = 20 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: mon data avail warn = 5 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: mon mgr mkfs grace = 240 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min pgs per osd = 4 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd reporter subtree level = osd 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd prime pg temp = true 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min bytes per osd = 10 2024-09-18T13:58:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: # 1m isn't quite enough 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: mon_down_mkfs_grace = 2m 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_filestore_osds = false 2024-09-18T13:58:22.868 INFO:teuthology.orchestra.run.smithi084.stdout: mon min osdmap epochs = 25 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: mon osdmap full prune interval = 2 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: mon osdmap full prune min = 15 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: mon osdmap full prune txsize = 2 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: mon scrub inject crc mismatch = 0.01 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: mon scrub inject missing keys = 0.05 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: mon scrub interval = 300 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: paxos min = 10 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: paxos service trim min = 5 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: paxos trim min = 10 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout:[client] 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: rgw cache enabled = true 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: rgw enable ops log = true 2024-09-18T13:58:22.869 INFO:teuthology.orchestra.run.smithi084.stdout: rgw enable usage log = true 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout: debug monc = 20 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.a] 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.b] 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.c] 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.d] 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.e] 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.f] 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.g] 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.h] 2024-09-18T13:58:22.870 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.i] 2024-09-18T13:58:22.872 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-18T13:58:22.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-18T13:58:22.962 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-18T13:58:22.962 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:22.962 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-18T13:58:22.978 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:22.978 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-18T13:58:23.034 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi045.front.sepia.ceph.com 2024-09-18T13:58:23.034 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:23.034 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T13:58:23.034 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T13:58:23.110 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:23.110 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T13:58:23.166 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi084.front.sepia.ceph.com 2024-09-18T13:58:23.166 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:23.166 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T13:58:23.166 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T13:58:23.205 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:23.205 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T13:58:23.262 INFO:tasks.ceph:Setting up mon nodes... 2024-09-18T13:58:23.262 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-18T13:58:23.263 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T13:58:23.357 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T13:58:23.359 INFO:tasks.ceph:Setting up mds nodes... 2024-09-18T13:58:23.359 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-18T13:58:23.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-18T13:58:23.397 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-18T13:58:23.413 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-18T13:58:23.413 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi084.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-18T13:58:23.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-18T13:58:23.439 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-18T13:58:23.439 INFO:tasks.ceph:role: osd.0 2024-09-18T13:58:23.439 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-18T13:58:23.439 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T13:58:23.506 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T13:58:23.507 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T13:58:23.507 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T13:58:23.507 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T13:58:23.507 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T13:58:23.508 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-18T13:58:23.508 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T13:58:23.508 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T13:58:23.508 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T13:58:23.508 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T13:58:23.538 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-18T13:58:23.540 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-18T13:58:23.540 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-18T13:58:23.610 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-18T13:58:23.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-18T13:58:23.748 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-18T13:58:23.748 INFO:tasks.ceph:role: osd.1 2024-09-18T13:58:23.748 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-18T13:58:23.748 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T13:58:23.814 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T13:58:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T13:58:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T13:58:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T13:58:23.814 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T13:58:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-18T13:58:23.814 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T13:58:23.814 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T13:58:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T13:58:23.814 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T13:58:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-18T13:58:23.847 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-18T13:58:23.848 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-18T13:58:23.917 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-18T13:58:23.987 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-18T13:58:24.053 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-18T13:58:24.053 INFO:tasks.ceph:role: osd.2 2024-09-18T13:58:24.053 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-18T13:58:24.053 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T13:58:24.118 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T13:58:24.119 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T13:58:24.119 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T13:58:24.119 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T13:58:24.119 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T13:58:24.119 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-18T13:58:24.119 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T13:58:24.119 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T13:58:24.119 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T13:58:24.119 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T13:58:24.151 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-18T13:58:24.153 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-18T13:58:24.153 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-18T13:58:24.222 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-18T13:58:24.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T13:58:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:24.399+0000 7ff094126440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-18T13:58:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:24.399+0000 7ff094126440 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-18T13:58:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:24.399+0000 7ff094126440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T13:58:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:24.399+0000 7ff094126440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T13:58:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:24.399+0000 7ff094126440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T13:58:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:24.400+0000 7ff094126440 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-18T13:58:24.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:24.925+0000 7ff094126440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:25.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:25.687+0000 7ff094126440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:26.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:26.455+0000 7ff094126440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:27.249 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-18T13:58:27.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T13:58:27.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:27.368+0000 7f9266357440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-18T13:58:27.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:27.368+0000 7f9266357440 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-18T13:58:27.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:27.369+0000 7f9266357440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T13:58:27.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:27.369+0000 7f9266357440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T13:58:27.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:27.369+0000 7f9266357440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T13:58:27.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:27.369+0000 7f9266357440 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-18T13:58:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:27.893+0000 7f9266357440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:28.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:28.658+0000 7f9266357440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:29.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:29.421+0000 7f9266357440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:30.218 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-18T13:58:30.249 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T13:58:30.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:30.336+0000 7f730f51d440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-18T13:58:30.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:30.336+0000 7f730f51d440 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-18T13:58:30.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:30.336+0000 7f730f51d440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T13:58:30.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:30.336+0000 7f730f51d440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T13:58:30.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:30.336+0000 7f730f51d440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T13:58:30.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:30.336+0000 7f730f51d440 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-18T13:58:30.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:30.860+0000 7f730f51d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:31.626+0000 7f730f51d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:32.388+0000 7f730f51d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:33.186 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-18T13:58:33.217 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-18T13:58:33.247 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-18T13:58:33.247 INFO:tasks.ceph:role: osd.3 2024-09-18T13:58:33.247 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-18T13:58:33.247 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T13:58:33.323 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T13:58:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T13:58:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T13:58:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T13:58:33.323 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T13:58:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2024-09-18T13:58:33.323 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T13:58:33.323 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T13:58:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T13:58:33.324 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T13:58:33.821 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2024-09-18T13:58:33.823 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2024-09-18T13:58:33.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-18T13:58:33.948 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-18T13:58:33.982 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-18T13:58:34.049 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-18T13:58:34.049 INFO:tasks.ceph:role: osd.4 2024-09-18T13:58:34.049 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-18T13:58:34.049 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T13:58:34.123 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T13:58:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T13:58:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T13:58:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T13:58:34.124 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T13:58:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2024-09-18T13:58:34.124 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T13:58:34.124 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T13:58:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T13:58:34.124 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T13:58:34.589 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2024-09-18T13:58:34.591 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2024-09-18T13:58:34.591 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-18T13:58:34.684 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-18T13:58:34.715 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-18T13:58:34.782 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-18T13:58:34.782 INFO:tasks.ceph:role: osd.5 2024-09-18T13:58:34.783 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-18T13:58:34.783 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T13:58:34.855 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T13:58:34.855 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T13:58:34.856 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T13:58:34.856 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T13:58:34.856 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T13:58:34.856 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2024-09-18T13:58:34.856 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T13:58:34.856 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T13:58:34.856 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T13:58:34.856 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T13:58:35.340 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2024-09-18T13:58:35.342 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2024-09-18T13:58:35.342 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-18T13:58:35.466 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-18T13:58:35.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T13:58:35.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:35.598+0000 7f717ea10440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-18T13:58:35.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:35.601+0000 7f717ea10440 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-18T13:58:35.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:35.601+0000 7f717ea10440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T13:58:35.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:35.601+0000 7f717ea10440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T13:58:35.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:35.601+0000 7f717ea10440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T13:58:35.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:35.601+0000 7f717ea10440 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-18T13:58:36.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:36.127+0000 7f717ea10440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:36.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:36.887+0000 7f717ea10440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:37.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:37.650+0000 7f717ea10440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:38.445 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-18T13:58:38.474 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T13:58:38.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:38.559+0000 7f05bd757440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-18T13:58:38.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:38.559+0000 7f05bd757440 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-18T13:58:38.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:38.559+0000 7f05bd757440 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T13:58:38.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:38.559+0000 7f05bd757440 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T13:58:38.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:38.559+0000 7f05bd757440 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T13:58:38.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:38.560+0000 7f05bd757440 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-18T13:58:39.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:39.087+0000 7f05bd757440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:39.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:39.850+0000 7f05bd757440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:40.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:40.619+0000 7f05bd757440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:41.416 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-18T13:58:41.445 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T13:58:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:41.536+0000 7f9675a0a440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-18T13:58:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:41.537+0000 7f9675a0a440 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-18T13:58:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:41.537+0000 7f9675a0a440 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T13:58:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:41.537+0000 7f9675a0a440 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T13:58:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:41.537+0000 7f9675a0a440 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T13:58:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:41.537+0000 7f9675a0a440 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-18T13:58:42.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:42.064+0000 7f9675a0a440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:42.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:42.829+0000 7f9675a0a440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:43.595+0000 7f9675a0a440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T13:58:44.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-18T13:58:44.422 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-18T13:58:44.422 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:44.422 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-18T13:58:44.454 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:44.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-18T13:58:44.526 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:44.526 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-18T13:58:44.597 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:44.597 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-18T13:58:44.623 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:44.624 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-18T13:58:44.689 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:44.689 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-18T13:58:44.755 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:44.755 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-18T13:58:44.821 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:44.821 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-18T13:58:44.876 INFO:tasks.ceph:Adding keys to all mons... 2024-09-18T13:58:44.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T13:58:44.878 DEBUG:teuthology.orchestra.run.smithi084:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T13:58:44.919 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.0] 2024-09-18T13:58:44.919 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCA3Opmj+zdFxAAAz8RKlG9/6YnLqP2gxEiGw== 2024-09-18T13:58:44.919 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.1] 2024-09-18T13:58:44.919 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCD3OpmJukIFhAAgpoUGnieCmmd+eenw4pUmQ== 2024-09-18T13:58:44.920 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.2] 2024-09-18T13:58:44.920 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCG3OpmG6AbFBAAOKZHGZoUpDaARJjTRIDuvQ== 2024-09-18T13:58:44.920 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.x] 2024-09-18T13:58:44.920 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQB/3OpmIPVMFRAARNZV3yg4gqD/ifoBDD3XTA== 2024-09-18T13:58:44.920 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.3] 2024-09-18T13:58:44.920 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCL3Opm7hq8IxAAM5RiTk0hFfQ1Qi/EJvqUdg== 2024-09-18T13:58:44.920 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.4] 2024-09-18T13:58:44.921 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCO3OpmzbNxIRAAhjqNMd+zc3YoY7RZX/CxPg== 2024-09-18T13:58:44.921 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.5] 2024-09-18T13:58:44.921 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCR3OpmdWcZIBAAP7StA3/Cm5lAUsY3JSFRGw== 2024-09-18T13:58:44.921 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-09-18T13:58:44.921 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQB/3Opm06K6FxAAQYn/bGUSaM4kLY5muh/97g== 2024-09-18T13:58:44.939 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.0] 2024-09-18T13:58:44.939 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCA3Opmj+zdFxAAAz8RKlG9/6YnLqP2gxEiGw== 2024-09-18T13:58:44.939 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.1] 2024-09-18T13:58:44.939 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCD3OpmJukIFhAAgpoUGnieCmmd+eenw4pUmQ== 2024-09-18T13:58:44.940 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.2] 2024-09-18T13:58:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCG3OpmG6AbFBAAOKZHGZoUpDaARJjTRIDuvQ== 2024-09-18T13:58:44.940 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr.x] 2024-09-18T13:58:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQB/3OpmIPVMFRAARNZV3yg4gqD/ifoBDD3XTA== 2024-09-18T13:58:44.940 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.3] 2024-09-18T13:58:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCL3Opm7hq8IxAAM5RiTk0hFfQ1Qi/EJvqUdg== 2024-09-18T13:58:44.940 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.4] 2024-09-18T13:58:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCO3OpmzbNxIRAAhjqNMd+zc3YoY7RZX/CxPg== 2024-09-18T13:58:44.940 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.5] 2024-09-18T13:58:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCR3OpmdWcZIBAAP7StA3/Cm5lAUsY3JSFRGw== 2024-09-18T13:58:44.940 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2024-09-18T13:58:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQB/3Opm06K6FxAAQYn/bGUSaM4kLY5muh/97g== 2024-09-18T13:58:44.942 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T13:58:44.965 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T13:58:45.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T13:58:45.029 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T13:58:45.078 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T13:58:45.080 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T13:58:45.127 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T13:58:45.129 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T13:58:45.175 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T13:58:45.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T13:58:45.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T13:58:45.225 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T13:58:45.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T13:58:45.274 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T13:58:45.320 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T13:58:45.322 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T13:58:45.370 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-18T13:58:45.371 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-18T13:58:45.396 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T13:58:45.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-18T13:58:45.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-18T13:58:45.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T13:58:45.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-18T13:58:45.698 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-18T13:58:45.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T13:58:45.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-18T13:58:45.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-d 2024-09-18T13:58:45.948 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i d --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T13:58:46.040 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-d 2024-09-18T13:58:46.066 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-e 2024-09-18T13:58:46.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i e --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T13:58:46.225 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-e 2024-09-18T13:58:46.251 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mon/ceph-f 2024-09-18T13:58:46.279 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i f --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T13:58:46.375 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-f 2024-09-18T13:58:46.408 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mon/ceph-g 2024-09-18T13:58:46.475 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i g --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T13:58:46.566 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-g 2024-09-18T13:58:46.590 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mon/ceph-h 2024-09-18T13:58:46.655 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i h --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T13:58:46.746 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-h 2024-09-18T13:58:46.770 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mon/ceph-i 2024-09-18T13:58:46.835 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i i --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T13:58:46.926 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-i 2024-09-18T13:58:46.950 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T13:58:46.953 DEBUG:teuthology.orchestra.run.smithi084:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T13:58:47.005 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-18T13:58:47.006 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-18T13:58:47.006 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-18T13:58:47.012 INFO:tasks.ceph.mon.a:Started 2024-09-18T13:58:47.012 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-18T13:58:47.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-18T13:58:47.014 INFO:tasks.ceph.mon.b:Started 2024-09-18T13:58:47.014 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T13:58:47.015 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T13:58:47.017 INFO:tasks.ceph.mon.c:Started 2024-09-18T13:58:47.017 INFO:tasks.ceph.mon.d:Restarting daemon 2024-09-18T13:58:47.017 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-09-18T13:58:47.019 INFO:tasks.ceph.mon.d:Started 2024-09-18T13:58:47.020 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-18T13:58:47.020 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-18T13:58:47.022 INFO:tasks.ceph.mon.e:Started 2024-09-18T13:58:47.022 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-18T13:58:47.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-18T13:58:47.048 INFO:tasks.ceph.mon.f:Started 2024-09-18T13:58:47.048 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-18T13:58:47.048 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-18T13:58:47.050 INFO:tasks.ceph.mon.g:Started 2024-09-18T13:58:47.050 INFO:tasks.ceph.mon.h:Restarting daemon 2024-09-18T13:58:47.050 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-09-18T13:58:47.052 INFO:tasks.ceph.mon.h:Started 2024-09-18T13:58:47.052 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-18T13:58:47.052 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-18T13:58:47.055 INFO:tasks.ceph.mon.i:Started 2024-09-18T13:58:47.056 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-18T13:58:47.056 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-18T13:58:47.057 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-18T13:58:47.059 INFO:tasks.ceph.mgr.x:Started 2024-09-18T13:58:47.059 INFO:tasks.ceph:Setting crush tunables to default 2024-09-18T13:58:47.059 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd crush tunables default 2024-09-18T13:58:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.192+0000 7f49d491d640 10 monclient: get_monmap_and_config 2024-09-18T13:58:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.192+0000 7f49d491d640 10 monclient: build_initial_monmap 2024-09-18T13:58:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.192+0000 7f49d491d640 10 monclient: monmap: 2024-09-18T13:58:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:58:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:47.193898+0000 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:47.193898+0000 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:47.194 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:47.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.193+0000 7f49d491d640 1 Processor -- start 2024-09-18T13:58:47.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.193+0000 7f49d491d640 1 -- start start 2024-09-18T13:58:47.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.193+0000 7f49d491d640 10 monclient: init 2024-09-18T13:58:47.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49d491d640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:47.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49d491d640 10 monclient: _add_conns ranks=[5,2,4,8,3,1,7,6,0] 2024-09-18T13:58:47.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49d491d640 10 monclient(hunting): picked mon.noname-h con 0x7f49d006e810 addr v1:172.21.15.84:6791/0 2024-09-18T13:58:47.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49d491d640 10 monclient(hunting): picked mon.noname-b con 0x7f49d00a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T13:58:47.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49d491d640 10 monclient(hunting): picked mon.noname-c con 0x7f49d00a1080 addr v1:172.21.15.45:6791/0 2024-09-18T13:58:47.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49d491d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d005ed70 con 0x7f49d00a0cb0 2024-09-18T13:58:47.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49d491d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d0097620 con 0x7f49d00a1080 2024-09-18T13:58:47.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49d491d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d0097340 con 0x7f49d006e810 2024-09-18T13:58:47.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49d491d640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:47.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49d491d640 10 monclient(hunting): authenticate will time out at 742.532658s 2024-09-18T13:58:47.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49ced9d640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f49d00a1080 0x7f49d013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:35256/0 (socket says 172.21.15.45:35256) 2024-09-18T13:58:47.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49ced9d640 1 -- 172.21.15.45:0/1427939422 learned_addr learned my addr 172.21.15.45:0/1427939422 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:58:47.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:47.194+0000 7f49cdd9b640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f49d00a0cb0 0x7f49d0137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:46700/0 (socket says 172.21.15.45:46700) 2024-09-18T13:58:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 10 monclient(hunting): tick 2024-09-18T13:58:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 1 monclient(hunting): continuing hunt 2024-09-18T13:58:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T13:58:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 1 -- 172.21.15.45:0/1427939422 >> v1:172.21.15.84:6791/0 conn(0x7f49d006e810 legacy=0x7f49d00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 1 -- 172.21.15.45:0/1427939422 >> v1:172.21.15.45:6791/0 conn(0x7f49d00a1080 legacy=0x7f49d013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 1 -- 172.21.15.45:0/1427939422 >> v1:172.21.15.45:6790/0 conn(0x7f49d00a0cb0 legacy=0x7f49d0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 10 monclient: _add_conns ranks=[7,0,6,8,1,3] 2024-09-18T13:58:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 10 monclient(hunting): picked mon.noname-i con 0x7f49b00037a0 addr v1:172.21.15.84:6792/0 2024-09-18T13:58:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 10 monclient(hunting): picked mon.noname-a con 0x7f49b0007420 addr v1:172.21.15.45:6789/0 2024-09-18T13:58:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 10 monclient(hunting): picked mon.noname-d con 0x7f49b000b180 addr v1:172.21.15.45:6792/0 2024-09-18T13:58:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 1 -- 172.21.15.45:0/1427939422 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d00a1660 con 0x7f49b0007420 2024-09-18T13:58:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 1 -- 172.21.15.45:0/1427939422 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b000ff90 con 0x7f49b000b180 2024-09-18T13:58:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 1 -- 172.21.15.45:0/1427939422 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b00111d0 con 0x7f49b00037a0 2024-09-18T13:58:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:50.195+0000 7f49bffff640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:52.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 1 -- 172.21.15.45:0/1427939422 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322462877 0 0) 0x7f49d00a1660 con 0x7f49b0007420 2024-09-18T13:58:52.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 10 monclient(hunting): my global_id is 4104 2024-09-18T13:58:52.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 1 -- 172.21.15.45:0/1427939422 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b80034e0 con 0x7f49b0007420 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 1 -- 172.21.15.45:0/1427939422 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398318455 0 0) 0x7f49b000ff90 con 0x7f49b000b180 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 10 monclient(hunting): my global_id is 4119 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 1 -- 172.21.15.45:0/1427939422 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b8004860 con 0x7f49b000b180 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 1 -- 172.21.15.45:0/1427939422 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3028928406 0 0) 0x7f49b80034e0 con 0x7f49b0007420 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 1 -- 172.21.15.45:0/1427939422 >> v1:172.21.15.84:6792/0 conn(0x7f49b00037a0 legacy=0x7f49b0005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 1 -- 172.21.15.45:0/1427939422 >> v1:172.21.15.45:6792/0 conn(0x7f49b000b180 legacy=0x7f49b000d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 10 monclient: _finish_hunting 0 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 1 monclient: found mon.noname-a 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 1 -- 172.21.15.45:0/1427939422 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b0012490 con 0x7f49b0007420 2024-09-18T13:58:52.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 10 monclient: _finish_auth 0 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 1 -- 172.21.15.45:0/1427939422 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49c4001830 con 0x7f49b0007420 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49cd59a640 10 monclient: dump: 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49ced9d640 1 -- 172.21.15.45:0/1427939422 reap_dead start 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49d491d640 5 monclient: authenticate success, global_id 4104 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.195+0000 7f49d491d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.196+0000 7f49cd59a640 1 -- 172.21.15.45:0/1427939422 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f49c4001990 con 0x7f49b0007420 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.196+0000 7f49cd59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.196+0000 7f49cd59a640 1 -- 172.21.15.45:0/1427939422 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49c4001c40 con 0x7f49b0007420 2024-09-18T13:58:52.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.196+0000 7f49cd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.196+0000 7f49cd59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.196+0000 7f49cd59a640 10 monclient: dump: 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.200 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.201 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.201 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.201 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.201 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:52.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.196+0000 7f49d491d640 10 monclient: get_monmap_and_config success 2024-09-18T13:58:52.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.196+0000 7f49d491d640 4 set_mon_vals no callback set 2024-09-18T13:58:52.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.196+0000 7f49d491d640 10 monclient: shutdown 2024-09-18T13:58:52.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.196+0000 7f49d491d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:52.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.196+0000 7f49d491d640 1 -- 172.21.15.45:0/1427939422 >> v1:172.21.15.45:6789/0 conn(0x7f49b0007420 legacy=0x7f49b00098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.197+0000 7f49d491d640 1 -- 172.21.15.45:0/1427939422 shutdown_connections 2024-09-18T13:58:52.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.197+0000 7f49d491d640 1 -- 172.21.15.45:0/1427939422 >> 172.21.15.45:0/1427939422 conn(0x7f49d00a5bb0 msgr2=0x7f49d00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.197+0000 7f49d491d640 1 -- 172.21.15.45:0/1427939422 shutdown_connections 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.197+0000 7f49d491d640 1 -- 172.21.15.45:0/1427939422 wait complete. 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.197+0000 7f49d491d640 10 monclient: build_initial_monmap 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.197+0000 7f49d491d640 10 monclient: monmap: 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 1 Processor -- start 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 1 -- start start 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 10 monclient: init 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 10 monclient: _add_conns ranks=[3,1,8,4,6,5,2,0,7] 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 10 monclient(hunting): picked mon.noname-g con 0x7f49b0007420 addr v1:172.21.15.84:6790/0 2024-09-18T13:58:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 10 monclient(hunting): picked mon.noname-f con 0x7f49d0135740 addr v1:172.21.15.84:6789/0 2024-09-18T13:58:52.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 10 monclient(hunting): picked mon.noname-e con 0x7f49d0135b30 addr v1:172.21.15.45:6793/0 2024-09-18T13:58:52.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d00a2270 con 0x7f49d0135740 2024-09-18T13:58:52.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d0142730 con 0x7f49b0007420 2024-09-18T13:58:52.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d0121270 con 0x7f49d0135b30 2024-09-18T13:58:52.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:52.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49d491d640 10 monclient(hunting): authenticate will time out at 747.536728s 2024-09-18T13:58:52.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49ced9d640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f49d0135b30 0x7f49d014a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:53630/0 (socket says 172.21.15.45:53630) 2024-09-18T13:58:52.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.198+0000 7f49ced9d640 1 -- 172.21.15.45:0/1505857808 learned_addr learned my addr 172.21.15.45:0/1505857808 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:58:52.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588494423 0 0) 0x7f49b00111d0 con 0x7f49b0007420 2024-09-18T13:58:52.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 10 monclient(hunting): my global_id is 4107 2024-09-18T13:58:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b0011440 con 0x7f49b0007420 2024-09-18T13:58:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461810762 0 0) 0x7f49d00a2270 con 0x7f49d0135740 2024-09-18T13:58:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 10 monclient(hunting): my global_id is 4105 2024-09-18T13:58:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b00111d0 con 0x7f49d0135740 2024-09-18T13:58:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2130971698 0 0) 0x7f49b0011440 con 0x7f49b0007420 2024-09-18T13:58:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49d00a2270 con 0x7f49b0007420 2024-09-18T13:58:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49c0001a60 con 0x7f49b0007420 2024-09-18T13:58:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 10 monclient(hunting): dump: 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.204+0000 7f49be7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.205+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 622713327 0 0) 0x7f49b00111d0 con 0x7f49d0135740 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.205+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49b0011440 con 0x7f49d0135740 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.205+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49c4001770 con 0x7f49d0135740 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.205+0000 7f49be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.205+0000 7f49be7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.205+0000 7f49be7fc640 10 monclient(hunting): dump: 2024-09-18T13:58:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.211 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.212 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:52.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.205+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537527233 0 0) 0x7f49d0121270 con 0x7f49d0135b30 2024-09-18T13:58:52.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.205+0000 7f49be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.205+0000 7f49be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.205+0000 7f49be7fc640 10 monclient(hunting): my global_id is 4112 2024-09-18T13:58:52.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.205+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b00111d0 con 0x7f49d0135b30 2024-09-18T13:58:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.207+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4282985700 0 0) 0x7f49b00111d0 con 0x7f49d0135b30 2024-09-18T13:58:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49d0121270 con 0x7f49d0135b30 2024-09-18T13:58:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49c8001ca0 con 0x7f49d0135b30 2024-09-18T13:58:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T13:58:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 10 monclient(hunting): dump: 2024-09-18T13:58:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1537688644 0 0) 0x7f49b0011440 con 0x7f49d0135740 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 >> v1:172.21.15.45:6793/0 conn(0x7f49d0135b30 legacy=0x7f49d014a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 >> v1:172.21.15.84:6790/0 conn(0x7f49b0007420 legacy=0x7f49d0141850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 1 monclient: found mon.f 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d009fa10 con 0x7f49d0135740 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 10 monclient: _finish_auth 0 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49d491d640 5 monclient: authenticate success, global_id 4105 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49d491d640 10 monclient: _renew_subs 2024-09-18T13:58:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49d491d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49d491d640 1 -- 172.21.15.45:0/1505857808 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d0121900 con 0x7f49d0135740 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49d491d640 10 monclient: _renew_subs 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49d491d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.208+0000 7f49d491d640 1 -- 172.21.15.45:0/1505857808 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d0120d30 con 0x7f49d0135740 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.209+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f49c4004a50 con 0x7f49d0135740 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.209+0000 7f49be7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.209+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49c4004f00 con 0x7f49d0135740 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.209+0000 7f49be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.209+0000 7f49ccd99640 4 set_mon_vals no callback set 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.209+0000 7f49be7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.209+0000 7f49be7fc640 10 monclient: dump: 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.209+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f49c40052e0 con 0x7f49d0135740 2024-09-18T13:58:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.209+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 3071103901 0 0) 0x7f49c4006160 con 0x7f49d0135740 2024-09-18T13:58:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.209+0000 7f49d491d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.210+0000 7f49d491d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.210+0000 7f49d491d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:58:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.210+0000 7f49d491d640 1 -- 172.21.15.45:0/1505857808 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b4005010 con 0x7f49d0135740 2024-09-18T13:58:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.216+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.1 v1:172.21.15.84:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72049 (unknown 1092875540 0 1010104650) 0x7f49c4005720 con 0x7f49d0135740 2024-09-18T13:58:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.216+0000 7f49be7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.216+0000 7f49be7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:58:52.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.289+0000 7f49d491d640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-18T13:58:52.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.289+0000 7f49d491d640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-18T13:58:52.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.289+0000 7f49d491d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:58:52.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.289+0000 7f49d491d640 1 -- 172.21.15.45:0/1505857808 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f49b40052c0 con 0x7f49d0135740 2024-09-18T13:58:52.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.292+0000 7f49be7fc640 1 -- 172.21.15.45:0/1505857808 <== mon.1 v1:172.21.15.84:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (unknown 2277712955 0 0) 0x7f49c4006370 con 0x7f49d0135740 2024-09-18T13:58:52.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.292+0000 7f49be7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-18T13:58:52.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.292+0000 7f49be7fc640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2024-09-18T13:58:52.293 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-09-18T13:58:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.293+0000 7f49d491d640 10 monclient: shutdown 2024-09-18T13:58:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.293+0000 7f49d491d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.293+0000 7f49d491d640 1 -- 172.21.15.45:0/1505857808 >> v1:172.21.15.84:6789/0 conn(0x7f49d0135740 legacy=0x7f49d0146ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.293+0000 7f49d491d640 1 -- 172.21.15.45:0/1505857808 shutdown_connections 2024-09-18T13:58:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.293+0000 7f49d491d640 1 -- 172.21.15.45:0/1505857808 >> 172.21.15.45:0/1505857808 conn(0x7f49d00a5bb0 msgr2=0x7f49d00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.293+0000 7f49d491d640 1 -- 172.21.15.45:0/1505857808 shutdown_connections 2024-09-18T13:58:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:52.293+0000 7f49d491d640 1 -- 172.21.15.45:0/1505857808 wait complete. 2024-09-18T13:58:52.304 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-18T13:58:52.304 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:52.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-18T13:58:52.307 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:52.307+0000 7f3f645c1ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T13:58:52.330 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:52.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-18T13:58:52.345 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:52.345+0000 7f3f645c1ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T13:58:52.394 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:52.394+0000 7f3f645c1ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T13:58:52.395 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-18T13:58:52.396 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-18T13:58:52.461 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:52.461 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-18T13:58:52.487 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:52.487 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-18T13:58:52.538 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:52.538+0000 7f3f645c1ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T13:58:52.553 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-18T13:58:52.553 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-18T13:58:52.618 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new 8c3a8909-07ce-4141-98dd-131951f18ae2 0 2024-09-18T13:58:52.650 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:52.649+0000 7f3f645c1ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T13:58:52.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.742+0000 7f026d694640 10 monclient: get_monmap_and_config 2024-09-18T13:58:52.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.742+0000 7f026d694640 10 monclient: build_initial_monmap 2024-09-18T13:58:52.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.742+0000 7f026d694640 10 monclient: monmap: 2024-09-18T13:58:52.744 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 0 2024-09-18T13:58:52.744 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.744 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:52.743644+0000 2024-09-18T13:58:52.744 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:52.743644+0000 2024-09-18T13:58:52.744 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:52.744 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.744 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:52.744 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:52.744 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:52.744 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:52.745 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:52.745 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:52.745 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:52.745 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:52.745 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:52.745 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.743+0000 7f026d694640 1 Processor -- start 2024-09-18T13:58:52.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.743+0000 7f026d694640 1 -- start start 2024-09-18T13:58:52.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.743+0000 7f026d694640 10 monclient: init 2024-09-18T13:58:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.743+0000 7f026d694640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.743+0000 7f026d694640 10 monclient: _add_conns ranks=[2,6,3,1,5,8,4,0,7] 2024-09-18T13:58:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 10 monclient(hunting): picked mon.noname-b con 0x7f026806e810 addr v1:172.21.15.45:6790/0 2024-09-18T13:58:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 10 monclient(hunting): picked mon.noname-d con 0x7f02680a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 10 monclient(hunting): picked mon.noname-g con 0x7f02680a1080 addr v1:172.21.15.84:6790/0 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026805ed70 con 0x7f026806e810 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0268097620 con 0x7f02680a1080 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0268097340 con 0x7f02680a0cb0 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 10 monclient(hunting): authenticate will time out at 728.268110s 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f02677fe640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7f02680a1080 0x7f026813af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.84:37346/0 (socket says 172.21.15.84:37346) 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f02677fe640 1 -- 172.21.15.84:0/3949045774 learned_addr learned my addr 172.21.15.84:0/3949045774 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385414384 0 0) 0x7f0268097620 con 0x7f02680a1080 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient(hunting): my global_id is 4125 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02500034e0 con 0x7f02680a1080 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076693570 0 0) 0x7f026805ed70 con 0x7f026806e810 2024-09-18T13:58:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient(hunting): my global_id is 4106 2024-09-18T13:58:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0268097620 con 0x7f026806e810 2024-09-18T13:58:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 560056317 0 0) 0x7f0268097340 con 0x7f02680a0cb0 2024-09-18T13:58:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient(hunting): my global_id is 4128 2024-09-18T13:58:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f026805ed70 con 0x7f02680a0cb0 2024-09-18T13:58:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3467033976 0 0) 0x7f02500034e0 con 0x7f02680a1080 2024-09-18T13:58:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 >> v1:172.21.15.45:6792/0 conn(0x7f02680a0cb0 legacy=0x7f0268137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 >> v1:172.21.15.45:6790/0 conn(0x7f026806e810 legacy=0x7f02680a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient: _finish_hunting 0 2024-09-18T13:58:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 monclient: found mon.noname-g 2024-09-18T13:58:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T13:58:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680a1660 con 0x7f02680a1080 2024-09-18T13:58:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient: _finish_auth 0 2024-09-18T13:58:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0260001bb0 con 0x7f02680a1080 2024-09-18T13:58:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 5 monclient: authenticate success, global_id 4125 2024-09-18T13:58:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient: dump: 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.757 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0260001d90 con 0x7f02680a1080 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 1 -- 172.21.15.84:0/3949045774 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0260004f20 con 0x7f02680a1080 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 10 monclient: get_monmap_and_config success 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f026d694640 4 set_mon_vals no callback set 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:58:52.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.744+0000 7f0265ffb640 10 monclient: dump: 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.759 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.745+0000 7f026d694640 10 monclient: shutdown 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.745+0000 7f026d694640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.745+0000 7f026d694640 1 -- 172.21.15.84:0/3949045774 >> v1:172.21.15.84:6790/0 conn(0x7f02680a1080 legacy=0x7f026813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.745+0000 7f026d694640 1 -- 172.21.15.84:0/3949045774 shutdown_connections 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.745+0000 7f026d694640 1 -- 172.21.15.84:0/3949045774 >> 172.21.15.84:0/3949045774 conn(0x7f02680a5bb0 msgr2=0x7f02680a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.745+0000 7f026d694640 1 -- 172.21.15.84:0/3949045774 shutdown_connections 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.745+0000 7f026d694640 1 -- 172.21.15.84:0/3949045774 wait complete. 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.746+0000 7f026d694640 10 monclient: build_initial_monmap 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.746+0000 7f026d694640 10 monclient: monmap: 2024-09-18T13:58:52.760 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 0 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 0.000000 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:created 0.000000 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 1 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:52.761 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:52.762 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:52.762 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.746+0000 7f026d694640 1 Processor -- start 2024-09-18T13:58:52.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.746+0000 7f026d694640 1 -- start start 2024-09-18T13:58:52.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.746+0000 7f026d694640 10 monclient: init 2024-09-18T13:58:52.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f026d694640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:52.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f026d694640 10 monclient: _add_conns ranks=[8,0,4,7,2,3,6,1,5] 2024-09-18T13:58:52.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f026d694640 10 monclient(hunting): picked mon.noname-e con 0x7f026806e810 addr v1:172.21.15.45:6793/0 2024-09-18T13:58:52.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f026d694640 10 monclient(hunting): picked mon.noname-a con 0x7f02680a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f026d694640 10 monclient(hunting): picked mon.noname-c con 0x7f02680a1080 addr v1:172.21.15.45:6791/0 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f026d694640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0268122ec0 con 0x7f02680a0cb0 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f026d694640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026813dcb0 con 0x7f02680a1080 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f026d694640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0268122900 con 0x7f026806e810 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f026d694640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f026d694640 10 monclient(hunting): authenticate will time out at 728.271179s 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f02667fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f02680a0cb0 0x7f02681467d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.84:39522/0 (socket says 172.21.15.84:39522) 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f02677fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f02680a1080 0x7f0268149f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.84:43106/0 (socket says 172.21.15.84:43106) 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f02667fc640 1 -- 172.21.15.84:0/638044475 learned_addr learned my addr 172.21.15.84:0/638044475 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f0266ffd640 1 --1- 172.21.15.84:0/638044475 >> v1:172.21.15.45:6793/0 conn(0x7f026806e810 0x7f026813d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.84:37500/0 (socket says 172.21.15.84:37500) 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125959639 0 0) 0x7f0268122ec0 con 0x7f02680a0cb0 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): my global_id is 4122 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02540034e0 con 0x7f02680a0cb0 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716741348 0 0) 0x7f026813dcb0 con 0x7f02680a1080 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): my global_id is 4117 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0268122ec0 con 0x7f02680a1080 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322513770 0 0) 0x7f0268122900 con 0x7f026806e810 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): my global_id is 4121 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f026813dcb0 con 0x7f026806e810 2024-09-18T13:58:52.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3918715972 0 0) 0x7f02540034e0 con 0x7f02680a0cb0 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0268122900 con 0x7f02680a0cb0 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f025c002bf0 con 0x7f02680a0cb0 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): dump: 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.765 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1301737989 0 0) 0x7f0268122ec0 con 0x7f02680a1080 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f02540034e0 con 0x7f02680a1080 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1479275526 0 0) 0x7f026813dcb0 con 0x7f026806e810 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0268122ec0 con 0x7f026806e810 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0260001b70 con 0x7f02680a1080 2024-09-18T13:58:52.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.747+0000 7f024effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 10 monclient(hunting): dump: 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0258001b30 con 0x7f026806e810 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 10 monclient(hunting): dump: 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3338436289 0 0) 0x7f0268122900 con 0x7f02680a0cb0 2024-09-18T13:58:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 >> v1:172.21.15.45:6793/0 conn(0x7f026806e810 legacy=0x7f026813d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 >> v1:172.21.15.45:6791/0 conn(0x7f02680a1080 legacy=0x7f0268149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 10 monclient: _finish_hunting 0 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 1 monclient: found mon.a 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02681203c0 con 0x7f02680a0cb0 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 10 monclient: _finish_auth 0 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f026d694640 5 monclient: authenticate success, global_id 4122 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f026d694640 10 monclient: _renew_subs 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f026d694640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f026d694640 1 -- 172.21.15.84:0/638044475 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0268122620 con 0x7f02680a0cb0 2024-09-18T13:58:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f026d694640 10 monclient: _renew_subs 2024-09-18T13:58:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f026d694640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:58:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f026d694640 1 -- 172.21.15.84:0/638044475 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0268121750 con 0x7f02680a0cb0 2024-09-18T13:58:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f025c001760 con 0x7f02680a0cb0 2024-09-18T13:58:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f025c001c50 con 0x7f02680a0cb0 2024-09-18T13:58:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f02657fa640 4 set_mon_vals no callback set 2024-09-18T13:58:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:58:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 10 monclient: dump: 2024-09-18T13:58:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.773 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f025c0020b0 con 0x7f02680a0cb0 2024-09-18T13:58:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.748+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (unknown 1930370504 0 0) 0x7f025c006220 con 0x7f02680a0cb0 2024-09-18T13:58:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.749+0000 7f026d694640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.749+0000 7f026d694640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.749+0000 7f026d694640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:58:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.749+0000 7f026d694640 1 -- 172.21.15.84:0/638044475 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0248005010 con 0x7f02680a0cb0 2024-09-18T13:58:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.751+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72049 (unknown 1092875540 0 1010104650) 0x7f025c005830 con 0x7f02680a0cb0 2024-09-18T13:58:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.751+0000 7f024effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.751+0000 7f024effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:58:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.819+0000 7f026d694640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "8c3a8909-07ce-4141-98dd-131951f18ae2", "id": 0}] 2024-09-18T13:58:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.819+0000 7f026d694640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "8c3a8909-07ce-4141-98dd-131951f18ae2", "id": 0}] 2024-09-18T13:58:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.819+0000 7f026d694640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:58:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.819+0000 7f026d694640 1 -- 172.21.15.84:0/638044475 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd new", "uuid": "8c3a8909-07ce-4141-98dd-131951f18ae2", "id": 0} v 0) v1 -- 0x7f0248005840 con 0x7f02680a0cb0 2024-09-18T13:58:52.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.823+0000 7f024effd640 1 -- 172.21.15.84:0/638044475 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8c3a8909-07ce-4141-98dd-131951f18ae2", "id": 0}]=0 v3) v1 ==== 112+0+2 (unknown 3008748538 0 732707515) 0x7f025c0064b0 con 0x7f02680a0cb0 2024-09-18T13:58:52.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.823+0000 7f024effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "8c3a8909-07ce-4141-98dd-131951f18ae2", "id": 0}] 2024-09-18T13:58:52.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.823+0000 7f024effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:58:52.824 INFO:teuthology.orchestra.run.smithi084.stdout:0 2024-09-18T13:58:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.824+0000 7f026d694640 10 monclient: shutdown 2024-09-18T13:58:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.824+0000 7f026d694640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.824+0000 7f026d694640 1 -- 172.21.15.84:0/638044475 >> v1:172.21.15.45:6789/0 conn(0x7f02680a0cb0 legacy=0x7f02681467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.824+0000 7f026d694640 1 -- 172.21.15.84:0/638044475 shutdown_connections 2024-09-18T13:58:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.824+0000 7f026d694640 1 -- 172.21.15.84:0/638044475 >> 172.21.15.84:0/638044475 conn(0x7f02680a5bb0 msgr2=0x7f02680a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.824+0000 7f026d694640 1 -- 172.21.15.84:0/638044475 shutdown_connections 2024-09-18T13:58:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.824+0000 7f026d694640 1 -- 172.21.15.84:0/638044475 wait complete. 2024-09-18T13:58:52.834 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new a12b3bc3-1e03-47dc-98c1-c8bd199551a2 1 2024-09-18T13:58:52.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.924+0000 7fbfab457640 10 monclient: get_monmap_and_config 2024-09-18T13:58:52.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.924+0000 7fbfab457640 10 monclient: build_initial_monmap 2024-09-18T13:58:52.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.924+0000 7fbfab457640 10 monclient: monmap: 2024-09-18T13:58:52.928 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 0 2024-09-18T13:58:52.928 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.928 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:52.925037+0000 2024-09-18T13:58:52.928 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:52.925037+0000 2024-09-18T13:58:52.928 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:52.928 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.928 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:52.928 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.924+0000 7fbfab457640 1 Processor -- start 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.924+0000 7fbfab457640 1 -- start start 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.924+0000 7fbfab457640 10 monclient: init 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbfab457640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbfab457640 10 monclient: _add_conns ranks=[7,2,4,0,6,8,5,3,1] 2024-09-18T13:58:52.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbfab457640 10 monclient(hunting): picked mon.noname-i con 0x7fbfa406e810 addr v1:172.21.15.84:6792/0 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbfab457640 10 monclient(hunting): picked mon.noname-b con 0x7fbfa40a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbfab457640 10 monclient(hunting): picked mon.noname-c con 0x7fbfa40a1080 addr v1:172.21.15.45:6791/0 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbfab457640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa405ed70 con 0x7fbfa40a0cb0 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbfab457640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa4097620 con 0x7fbfa40a1080 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbfab457640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa4097340 con 0x7fbfa406e810 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbfab457640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbfab457640 10 monclient(hunting): authenticate will time out at 728.449433s 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbfa91f3640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7fbfa406e810 0x7fbfa40a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.84:57894/0 (socket says 172.21.15.84:57894) 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbfa91f3640 1 -- 172.21.15.84:0/2940918043 learned_addr learned my addr 172.21.15.84:0/2940918043 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbf9bfff640 1 -- 172.21.15.84:0/2940918043 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339493955 0 0) 0x7fbfa4097340 con 0x7fbfa406e810 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbf9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbf9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbf9bfff640 10 monclient(hunting): my global_id is 4111 2024-09-18T13:58:52.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbf9bfff640 1 -- 172.21.15.84:0/2940918043 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf900034e0 con 0x7fbfa406e810 2024-09-18T13:58:52.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.925+0000 7fbf9bfff640 1 -- 172.21.15.84:0/2940918043 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2846648821 0 0) 0x7fbf900034e0 con 0x7fbfa406e810 2024-09-18T13:58:52.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 1 -- 172.21.15.84:0/2940918043 >> v1:172.21.15.45:6791/0 conn(0x7fbfa40a1080 legacy=0x7fbfa413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 1 -- 172.21.15.84:0/2940918043 >> v1:172.21.15.45:6790/0 conn(0x7fbfa40a0cb0 legacy=0x7fbfa4137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 10 monclient: _finish_hunting 0 2024-09-18T13:58:52.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 1 monclient: found mon.noname-i 2024-09-18T13:58:52.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:52.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T13:58:52.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 1 -- 172.21.15.84:0/2940918043 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa40a1660 con 0x7fbfa406e810 2024-09-18T13:58:52.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 10 monclient: _finish_auth 0 2024-09-18T13:58:52.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:52.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 1 -- 172.21.15.84:0/2940918043 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbf94001830 con 0x7fbfa406e810 2024-09-18T13:58:52.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbfab457640 5 monclient: authenticate success, global_id 4111 2024-09-18T13:58:52.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbfab457640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:52.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:58:52.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 10 monclient: dump: 2024-09-18T13:58:52.932 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:52.932 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.932 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.933 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbfab457640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 1 -- 172.21.15.84:0/2940918043 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbf94001990 con 0x7fbfa406e810 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 1 -- 172.21.15.84:0/2940918043 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbf94004860 con 0x7fbfa406e810 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbfab457640 10 monclient: get_monmap_and_config success 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbfab457640 4 set_mon_vals no callback set 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbf9bfff640 10 monclient: dump: 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.934 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbfab457640 10 monclient: shutdown 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbfab457640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:52.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbfab457640 1 -- 172.21.15.84:0/2940918043 >> v1:172.21.15.84:6792/0 conn(0x7fbfa406e810 legacy=0x7fbfa40a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbfab457640 1 -- 172.21.15.84:0/2940918043 shutdown_connections 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbfab457640 1 -- 172.21.15.84:0/2940918043 >> 172.21.15.84:0/2940918043 conn(0x7fbfa40a5bb0 msgr2=0x7fbfa40a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbfab457640 1 -- 172.21.15.84:0/2940918043 shutdown_connections 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.926+0000 7fbfab457640 1 -- 172.21.15.84:0/2940918043 wait complete. 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.927+0000 7fbfab457640 10 monclient: build_initial_monmap 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.927+0000 7fbfab457640 10 monclient: monmap: 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 0 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 0.000000 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:created 0.000000 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 1 2024-09-18T13:58:52.936 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:52.937 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:52.937 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:52.937 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:52.937 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:52.937 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:52.937 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:52.937 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:52.937 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:52.937 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.927+0000 7fbfab457640 1 Processor -- start 2024-09-18T13:58:52.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.927+0000 7fbfab457640 1 -- start start 2024-09-18T13:58:52.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.927+0000 7fbfab457640 10 monclient: init 2024-09-18T13:58:52.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbfab457640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:52.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbfab457640 10 monclient: _add_conns ranks=[7,1,2,6,5,4,3,0,8] 2024-09-18T13:58:52.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbfab457640 10 monclient(hunting): picked mon.noname-i con 0x7fbfa406e810 addr v1:172.21.15.84:6792/0 2024-09-18T13:58:52.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbfab457640 10 monclient(hunting): picked mon.noname-f con 0x7fbfa40a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T13:58:52.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbfab457640 10 monclient(hunting): picked mon.noname-b con 0x7fbfa40a1080 addr v1:172.21.15.45:6790/0 2024-09-18T13:58:52.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbfab457640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa409e560 con 0x7fbfa40a0cb0 2024-09-18T13:58:52.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbfab457640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa411d7d0 con 0x7fbfa40a1080 2024-09-18T13:58:52.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbfab457640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa40a2270 con 0x7fbfa406e810 2024-09-18T13:58:52.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbfab457640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:52.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbfab457640 10 monclient(hunting): authenticate will time out at 728.452369s 2024-09-18T13:58:52.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbfa91f3640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7fbfa406e810 0x7fbfa411c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.84:57902/0 (socket says 172.21.15.84:57902) 2024-09-18T13:58:52.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbfa91f3640 1 -- 172.21.15.84:0/2450202238 learned_addr learned my addr 172.21.15.84:0/2450202238 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826405039 0 0) 0x7fbfa40a2270 con 0x7fbfa406e810 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): my global_id is 4120 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf880034e0 con 0x7fbfa406e810 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643850176 0 0) 0x7fbfa409e560 con 0x7fbfa40a0cb0 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): my global_id is 4114 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfa40a2270 con 0x7fbfa40a0cb0 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760732952 0 0) 0x7fbfa411d7d0 con 0x7fbfa40a1080 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:52.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): my global_id is 4124 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfa409e560 con 0x7fbfa40a1080 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2220083746 0 0) 0x7fbf880034e0 con 0x7fbfa406e810 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbfa411d7d0 con 0x7fbfa406e810 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbf940016f0 con 0x7fbfa406e810 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): dump: 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.944 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2211516210 0 0) 0x7fbfa40a2270 con 0x7fbfa40a0cb0 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.928+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf880034e0 con 0x7fbfa40a0cb0 2024-09-18T13:58:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbfa00019e0 con 0x7fbfa40a0cb0 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient(hunting): dump: 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.946 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1045792265 0 0) 0x7fbfa409e560 con 0x7fbfa40a1080 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbfa40a2270 con 0x7fbfa40a1080 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbf9c002c50 con 0x7fbfa40a1080 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient(hunting): dump: 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.947 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1472114024 0 0) 0x7fbfa411d7d0 con 0x7fbfa406e810 2024-09-18T13:58:52.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 >> v1:172.21.15.45:6790/0 conn(0x7fbfa40a1080 legacy=0x7fbfa4149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 >> v1:172.21.15.84:6789/0 conn(0x7fbfa40a0cb0 legacy=0x7fbfa411c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient: _finish_hunting 0 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 monclient: found mon.i 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa411d030 con 0x7fbfa406e810 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient: _finish_auth 0 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbfab457640 5 monclient: authenticate success, global_id 4120 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbfab457640 10 monclient: _renew_subs 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbfab457640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbfab457640 1 -- 172.21.15.84:0/2450202238 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa409fa10 con 0x7fbfa406e810 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbfab457640 10 monclient: _renew_subs 2024-09-18T13:58:52.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbfab457640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbfab457640 1 -- 172.21.15.84:0/2450202238 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa4120490 con 0x7fbfa406e810 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbf94004dd0 con 0x7fbfa406e810 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbf94005280 con 0x7fbfa406e810 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf9b7fe640 4 set_mon_vals no callback set 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 10 monclient: dump: 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:52.950 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7fbf94005660 con 0x7fbfa406e810 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.929+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (unknown 1538704695 0 0) 0x7fbf940064c0 con 0x7fbfa406e810 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.930+0000 7fbfab457640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:52.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.930+0000 7fbfab457640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:52.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.930+0000 7fbfab457640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:52.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.930+0000 7fbfab457640 1 -- 172.21.15.84:0/2450202238 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf84005010 con 0x7fbfa406e810 2024-09-18T13:58:52.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.932+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.7 v1:172.21.15.84:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72049 (unknown 1092875540 0 1010104650) 0x7fbf94005a50 con 0x7fbfa406e810 2024-09-18T13:58:52.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.932+0000 7fbf997fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:52.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.932+0000 7fbf997fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:58:52.952 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:52.947+0000 7f3f645c1ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T13:58:52.997 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:52.996+0000 7f3f645c1ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T13:58:52.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.999+0000 7fbfab457640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "a12b3bc3-1e03-47dc-98c1-c8bd199551a2", "id": 1}] 2024-09-18T13:58:52.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.999+0000 7fbfab457640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "a12b3bc3-1e03-47dc-98c1-c8bd199551a2", "id": 1}] 2024-09-18T13:58:52.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.999+0000 7fbfab457640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:52.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:52.999+0000 7fbfab457640 1 -- 172.21.15.84:0/2450202238 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "osd new", "uuid": "a12b3bc3-1e03-47dc-98c1-c8bd199551a2", "id": 1} v 0) v1 -- 0x7fbf84005840 con 0x7fbfa406e810 2024-09-18T13:58:53.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.004+0000 7fbf997fa640 1 -- 172.21.15.84:0/2450202238 <== mon.7 v1:172.21.15.84:6792/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a12b3bc3-1e03-47dc-98c1-c8bd199551a2", "id": 1}]=0 v4) v1 ==== 112+0+2 (unknown 3848735632 0 940483276) 0x7fbf94006750 con 0x7fbfa406e810 2024-09-18T13:58:53.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.004+0000 7fbf997fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "a12b3bc3-1e03-47dc-98c1-c8bd199551a2", "id": 1}] 2024-09-18T13:58:53.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.004+0000 7fbf997fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:58:53.005 INFO:teuthology.orchestra.run.smithi084.stdout:1 2024-09-18T13:58:53.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.005+0000 7fbfab457640 10 monclient: shutdown 2024-09-18T13:58:53.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.005+0000 7fbfab457640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:53.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.005+0000 7fbfab457640 1 -- 172.21.15.84:0/2450202238 >> v1:172.21.15.84:6792/0 conn(0x7fbfa406e810 legacy=0x7fbfa411c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.005+0000 7fbfab457640 1 -- 172.21.15.84:0/2450202238 shutdown_connections 2024-09-18T13:58:53.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.005+0000 7fbfab457640 1 -- 172.21.15.84:0/2450202238 >> 172.21.15.84:0/2450202238 conn(0x7fbfa40a5bb0 msgr2=0x7fbfa409ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:53.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.005+0000 7fbfab457640 1 -- 172.21.15.84:0/2450202238 shutdown_connections 2024-09-18T13:58:53.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.005+0000 7fbfab457640 1 -- 172.21.15.84:0/2450202238 wait complete. 2024-09-18T13:58:53.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new 216b8b62-e41c-4bc5-9395-f2d56d402b1f 2 2024-09-18T13:58:53.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.108+0000 7f13d7e46640 10 monclient: get_monmap_and_config 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.108+0000 7f13d7e46640 10 monclient: build_initial_monmap 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.108+0000 7f13d7e46640 10 monclient: monmap: 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 0 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:53.109101+0000 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:53.109101+0000 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:53.110 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.108+0000 7f13d7e46640 1 Processor -- start 2024-09-18T13:58:53.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d7e46640 1 -- start start 2024-09-18T13:58:53.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d7e46640 10 monclient: init 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d7e46640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d7e46640 10 monclient: _add_conns ranks=[2,1,7,0,4,8,3,6,5] 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d7e46640 10 monclient(hunting): picked mon.noname-b con 0x7f13d006e3d0 addr v1:172.21.15.45:6790/0 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d7e46640 10 monclient(hunting): picked mon.noname-f con 0x7f13d0132700 addr v1:172.21.15.84:6789/0 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d7e46640 10 monclient(hunting): picked mon.noname-i con 0x7f13d01387b0 addr v1:172.21.15.84:6792/0 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d7e46640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d005ed70 con 0x7f13d0132700 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d7e46640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d00681f0 con 0x7f13d006e3d0 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d7e46640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d00693b0 con 0x7f13d01387b0 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d7e46640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d7e46640 10 monclient(hunting): authenticate will time out at 728.633600s 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d53e1640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f13d0132700 0x7f13d0132ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:51678/0 (socket says 172.21.15.84:51678) 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d53e1640 1 -- 172.21.15.84:0/181230175 learned_addr learned my addr 172.21.15.84:0/181230175 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d63e3640 1 --1- 172.21.15.84:0/181230175 >> v1:172.21.15.84:6792/0 conn(0x7f13d01387b0 0x7f13d013ab90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.84:57914/0 (socket says 172.21.15.84:57914) 2024-09-18T13:58:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153258750 0 0) 0x7f13d005ed70 con 0x7f13d0132700 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d4be0640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d4be0640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d4be0640 10 monclient(hunting): my global_id is 4123 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13bc0034e0 con 0x7f13d0132700 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.109+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309512106 0 0) 0x7f13d00693b0 con 0x7f13d01387b0 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient(hunting): my global_id is 4129 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d005ed70 con 0x7f13d01387b0 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967077911 0 0) 0x7f13d00681f0 con 0x7f13d006e3d0 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient(hunting): my global_id is 4133 2024-09-18T13:58:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d00693b0 con 0x7f13d006e3d0 2024-09-18T13:58:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1749356608 0 0) 0x7f13bc0034e0 con 0x7f13d0132700 2024-09-18T13:58:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 >> v1:172.21.15.84:6792/0 conn(0x7f13d01387b0 legacy=0x7f13d013ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 >> v1:172.21.15.45:6790/0 conn(0x7f13d006e3d0 legacy=0x7f13d006e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient: _finish_hunting 0 2024-09-18T13:58:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 1 monclient: found mon.noname-f 2024-09-18T13:58:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T13:58:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0069690 con 0x7f13d0132700 2024-09-18T13:58:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient: _finish_auth 0 2024-09-18T13:58:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d7e46640 5 monclient: authenticate success, global_id 4123 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f13cc001830 con 0x7f13d0132700 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d7e46640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient: dump: 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.123 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d7e46640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f13cc001a10 con 0x7f13d0132700 2024-09-18T13:58:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d7e46640 10 monclient: get_monmap_and_config success 2024-09-18T13:58:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 1 -- 172.21.15.84:0/181230175 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f13cc004ba0 con 0x7f13d0132700 2024-09-18T13:58:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d7e46640 4 set_mon_vals no callback set 2024-09-18T13:58:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:58:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d4be0640 10 monclient: dump: 2024-09-18T13:58:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.125 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d7e46640 10 monclient: shutdown 2024-09-18T13:58:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d7e46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.110+0000 7f13d7e46640 1 -- 172.21.15.84:0/181230175 >> v1:172.21.15.84:6789/0 conn(0x7f13d0132700 legacy=0x7f13d0132ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.111+0000 7f13d7e46640 1 -- 172.21.15.84:0/181230175 shutdown_connections 2024-09-18T13:58:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.111+0000 7f13d7e46640 1 -- 172.21.15.84:0/181230175 >> 172.21.15.84:0/181230175 conn(0x7f13d0067d00 msgr2=0x7f13d00a3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.111+0000 7f13d7e46640 1 -- 172.21.15.84:0/181230175 shutdown_connections 2024-09-18T13:58:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.111+0000 7f13d7e46640 1 -- 172.21.15.84:0/181230175 wait complete. 2024-09-18T13:58:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.111+0000 7f13d7e46640 10 monclient: build_initial_monmap 2024-09-18T13:58:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.111+0000 7f13d7e46640 10 monclient: monmap: 2024-09-18T13:58:53.129 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 0 2024-09-18T13:58:53.129 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:58:53.129 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 0.000000 2024-09-18T13:58:53.129 INFO:teuthology.orchestra.run.smithi084.stderr:created 0.000000 2024-09-18T13:58:53.129 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:53.129 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 1 2024-09-18T13:58:53.130 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:53.130 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:53.130 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:53.130 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:53.130 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:53.131 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:53.131 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:53.131 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:53.131 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:53.131 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.111+0000 7f13d7e46640 1 Processor -- start 2024-09-18T13:58:53.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d7e46640 1 -- start start 2024-09-18T13:58:53.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d7e46640 10 monclient: init 2024-09-18T13:58:53.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d7e46640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:53.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d7e46640 10 monclient: _add_conns ranks=[2,6,7,4,8,0,1,3,5] 2024-09-18T13:58:53.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d7e46640 10 monclient(hunting): picked mon.noname-b con 0x7f13d006e3d0 addr v1:172.21.15.45:6790/0 2024-09-18T13:58:53.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d7e46640 10 monclient(hunting): picked mon.noname-d con 0x7f13d0132700 addr v1:172.21.15.45:6792/0 2024-09-18T13:58:53.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d7e46640 10 monclient(hunting): picked mon.noname-i con 0x7f13d01387b0 addr v1:172.21.15.84:6792/0 2024-09-18T13:58:53.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d7e46640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d0068a30 con 0x7f13d006e3d0 2024-09-18T13:58:53.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d7e46640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d0068700 con 0x7f13d0132700 2024-09-18T13:58:53.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d7e46640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d011a510 con 0x7f13d01387b0 2024-09-18T13:58:53.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d7e46640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:53.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d7e46640 10 monclient(hunting): authenticate will time out at 728.636562s 2024-09-18T13:58:53.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d63e3640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f13d01387b0 0x7f13d0137ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.84:57926/0 (socket says 172.21.15.84:57926) 2024-09-18T13:58:53.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13d63e3640 1 -- 172.21.15.84:0/560765142 learned_addr learned my addr 172.21.15.84:0/560765142 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-18T13:58:53.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771828202 0 0) 0x7f13d011a510 con 0x7f13d01387b0 2024-09-18T13:58:53.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 10 monclient(hunting): my global_id is 4138 2024-09-18T13:58:53.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13b40034e0 con 0x7f13d01387b0 2024-09-18T13:58:53.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235026602 0 0) 0x7f13d0068a30 con 0x7f13d006e3d0 2024-09-18T13:58:53.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 10 monclient(hunting): my global_id is 4142 2024-09-18T13:58:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d011a510 con 0x7f13d006e3d0 2024-09-18T13:58:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488174964 0 0) 0x7f13d0068700 con 0x7f13d0132700 2024-09-18T13:58:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 10 monclient(hunting): my global_id is 4137 2024-09-18T13:58:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.112+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d0068a30 con 0x7f13d0132700 2024-09-18T13:58:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3070622271 0 0) 0x7f13b40034e0 con 0x7f13d01387b0 2024-09-18T13:58:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13d0068700 con 0x7f13d01387b0 2024-09-18T13:58:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f13c8001d80 con 0x7f13d01387b0 2024-09-18T13:58:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:58:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient(hunting): dump: 2024-09-18T13:58:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.137 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2662252116 0 0) 0x7f13d011a510 con 0x7f13d006e3d0 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13b40034e0 con 0x7f13d006e3d0 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f13c0001a60 con 0x7f13d006e3d0 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient(hunting): dump: 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3946747826 0 0) 0x7f13d0068a30 con 0x7f13d0132700 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13d011a510 con 0x7f13d0132700 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f13cc001770 con 0x7f13d0132700 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.139 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T13:58:53.140 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient(hunting): dump: 2024-09-18T13:58:53.140 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.140 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.140 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.140 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.140 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.140 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.140 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.140 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.140 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.140 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.140 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3702952280 0 0) 0x7f13d0068700 con 0x7f13d01387b0 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 >> v1:172.21.15.45:6792/0 conn(0x7f13d0132700 legacy=0x7f13d01377b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 >> v1:172.21.15.45:6790/0 conn(0x7f13d006e3d0 legacy=0x7f13d01370a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient: _finish_hunting 0 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 monclient: found mon.i 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d011bfc0 con 0x7f13d01387b0 2024-09-18T13:58:53.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient: _finish_auth 0 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13d7e46640 5 monclient: authenticate success, global_id 4138 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13d7e46640 10 monclient: _renew_subs 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13d7e46640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13d7e46640 1 -- 172.21.15.84:0/560765142 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d011b060 con 0x7f13d01387b0 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f13c8001fe0 con 0x7f13d01387b0 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f13c8005110 con 0x7f13d01387b0 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 10 monclient: dump: 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.142 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.143 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.143 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.143 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.143 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.143 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.143 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.143 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.143 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.143 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.143 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.143 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.144 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.144 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.144 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f13c8005530 con 0x7f13d01387b0 2024-09-18T13:58:53.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c7fff640 4 set_mon_vals no callback set 2024-09-18T13:58:53.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13d7e46640 10 monclient: _renew_subs 2024-09-18T13:58:53.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13d7e46640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13d7e46640 1 -- 172.21.15.84:0/560765142 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d01321e0 con 0x7f13d01387b0 2024-09-18T13:58:53.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.113+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (unknown 378776977 0 0) 0x7f13c8005a60 con 0x7f13d01387b0 2024-09-18T13:58:53.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.114+0000 7f13d7e46640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.114+0000 7f13d7e46640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.114+0000 7f13d7e46640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.114+0000 7f13d7e46640 1 -- 172.21.15.84:0/560765142 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d004c4a0 con 0x7f13d01387b0 2024-09-18T13:58:53.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.116+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.7 v1:172.21.15.84:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72049 (unknown 1092875540 0 1010104650) 0x7f13c8006120 con 0x7f13d01387b0 2024-09-18T13:58:53.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.116+0000 7f13c5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.116+0000 7f13c5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:58:53.145 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:53.136+0000 7f3f645c1ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T13:58:53.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.185+0000 7f13d7e46640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "216b8b62-e41c-4bc5-9395-f2d56d402b1f", "id": 2}] 2024-09-18T13:58:53.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.185+0000 7f13d7e46640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "216b8b62-e41c-4bc5-9395-f2d56d402b1f", "id": 2}] 2024-09-18T13:58:53.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.185+0000 7f13d7e46640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.185+0000 7f13d7e46640 1 -- 172.21.15.84:0/560765142 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "osd new", "uuid": "216b8b62-e41c-4bc5-9395-f2d56d402b1f", "id": 2} v 0) v1 -- 0x7f13d004b720 con 0x7f13d01387b0 2024-09-18T13:58:53.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.188+0000 7f13c5ffb640 1 -- 172.21.15.84:0/560765142 <== mon.7 v1:172.21.15.84:6792/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "216b8b62-e41c-4bc5-9395-f2d56d402b1f", "id": 2}]=0 v5) v1 ==== 112+0+2 (unknown 242909834 0 216599125) 0x7f13c80063c0 con 0x7f13d01387b0 2024-09-18T13:58:53.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.188+0000 7f13c5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "216b8b62-e41c-4bc5-9395-f2d56d402b1f", "id": 2}] 2024-09-18T13:58:53.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.188+0000 7f13c5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:58:53.189 INFO:teuthology.orchestra.run.smithi084.stdout:2 2024-09-18T13:58:53.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.189+0000 7f13d7e46640 10 monclient: shutdown 2024-09-18T13:58:53.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.189+0000 7f13d7e46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:53.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.189+0000 7f13d7e46640 1 -- 172.21.15.84:0/560765142 >> v1:172.21.15.84:6792/0 conn(0x7f13d01387b0 legacy=0x7f13d0137ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.189+0000 7f13d7e46640 1 -- 172.21.15.84:0/560765142 shutdown_connections 2024-09-18T13:58:53.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.189+0000 7f13d7e46640 1 -- 172.21.15.84:0/560765142 >> 172.21.15.84:0/560765142 conn(0x7f13d0067d00 msgr2=0x7f13d00a1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:53.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.190+0000 7f13d7e46640 1 -- 172.21.15.84:0/560765142 shutdown_connections 2024-09-18T13:58:53.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.190+0000 7f13d7e46640 1 -- 172.21.15.84:0/560765142 wait complete. 2024-09-18T13:58:53.204 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new c750b0e6-9143-489d-9be0-eb085178ff2a 3 2024-09-18T13:58:53.296 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.295+0000 7fd409d1d640 10 monclient: get_monmap_and_config 2024-09-18T13:58:53.296 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.295+0000 7fd409d1d640 10 monclient: build_initial_monmap 2024-09-18T13:58:53.296 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.296+0000 7fd409d1d640 10 monclient: monmap: 2024-09-18T13:58:53.296 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 0 2024-09-18T13:58:53.296 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:53.296824+0000 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:53.296824+0000 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:53.297 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:53.298 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.296+0000 7fd409d1d640 1 Processor -- start 2024-09-18T13:58:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.296+0000 7fd409d1d640 1 -- start start 2024-09-18T13:58:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.296+0000 7fd409d1d640 10 monclient: init 2024-09-18T13:58:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd409d1d640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd409d1d640 10 monclient: _add_conns ranks=[0,3,5,7,4,2,6,1,8] 2024-09-18T13:58:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd409d1d640 10 monclient(hunting): picked mon.noname-a con 0x7fd40409ddb0 addr v1:172.21.15.45:6789/0 2024-09-18T13:58:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd409d1d640 10 monclient(hunting): picked mon.noname-g con 0x7fd40409e180 addr v1:172.21.15.84:6790/0 2024-09-18T13:58:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd409d1d640 10 monclient(hunting): picked mon.noname-h con 0x7fd40409d410 addr v1:172.21.15.84:6791/0 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd409d1d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40405ed70 con 0x7fd40409ddb0 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd409d1d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40409d890 con 0x7fd40409e180 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd409d1d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40409e830 con 0x7fd40409d410 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd409d1d640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd409d1d640 10 monclient(hunting): authenticate will time out at 728.821444s 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd3fbfff640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7fd40409e180 0x7fd40409c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.84:37358/0 (socket says 172.21.15.84:37358) 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd3fbfff640 1 -- 172.21.15.84:0/437363248 learned_addr learned my addr 172.21.15.84:0/437363248 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd403fff640 1 --1- 172.21.15.84:0/437363248 >> v1:172.21.15.84:6791/0 conn(0x7fd40409d410 0x7fd40413b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.84:50714/0 (socket says 172.21.15.84:50714) 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630879074 0 0) 0x7fd40409d890 con 0x7fd40409e180 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd402ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd402ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd402ffd640 10 monclient(hunting): my global_id is 4134 2024-09-18T13:58:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3f00034e0 con 0x7fd40409e180 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984171823 0 0) 0x7fd40409e830 con 0x7fd40409d410 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd402ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd402ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd402ffd640 10 monclient(hunting): my global_id is 4109 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.297+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd40409d890 con 0x7fd40409d410 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816273286 0 0) 0x7fd40405ed70 con 0x7fd40409ddb0 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient(hunting): my global_id is 4131 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd40409e830 con 0x7fd40409ddb0 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1252939469 0 0) 0x7fd3f00034e0 con 0x7fd40409e180 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 >> v1:172.21.15.84:6791/0 conn(0x7fd40409d410 legacy=0x7fd40413b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 >> v1:172.21.15.45:6789/0 conn(0x7fd40409ddb0 legacy=0x7fd40409bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 1 monclient: found mon.noname-g 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40409af60 con 0x7fd40409e180 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient: _finish_auth 0 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd409d1d640 5 monclient: authenticate success, global_id 4134 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd409d1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd3ec001830 con 0x7fd40409e180 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:58:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient: dump: 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd409d1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd3ec001990 con 0x7fd40409e180 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 1 -- 172.21.15.84:0/437363248 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd3ec004ba0 con 0x7fd40409e180 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd409d1d640 10 monclient: get_monmap_and_config success 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd409d1d640 4 set_mon_vals no callback set 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd402ffd640 10 monclient: dump: 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.315 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd409d1d640 10 monclient: shutdown 2024-09-18T13:58:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd409d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd409d1d640 1 -- 172.21.15.84:0/437363248 >> v1:172.21.15.84:6790/0 conn(0x7fd40409e180 legacy=0x7fd40409c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd409d1d640 1 -- 172.21.15.84:0/437363248 shutdown_connections 2024-09-18T13:58:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.298+0000 7fd409d1d640 1 -- 172.21.15.84:0/437363248 >> 172.21.15.84:0/437363248 conn(0x7fd404068b40 msgr2=0x7fd4040a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.299+0000 7fd409d1d640 1 -- 172.21.15.84:0/437363248 shutdown_connections 2024-09-18T13:58:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.299+0000 7fd409d1d640 1 -- 172.21.15.84:0/437363248 wait complete. 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.299+0000 7fd409d1d640 10 monclient: build_initial_monmap 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.299+0000 7fd409d1d640 10 monclient: monmap: 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 0 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 0.000000 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:created 0.000000 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 1 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.299+0000 7fd409d1d640 1 Processor -- start 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.299+0000 7fd409d1d640 1 -- start start 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.299+0000 7fd409d1d640 10 monclient: init 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd409d1d640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd409d1d640 10 monclient: _add_conns ranks=[7,3,0,5,1,4,8,2,6] 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd409d1d640 10 monclient(hunting): picked mon.noname-i con 0x7fd40409d410 addr v1:172.21.15.84:6792/0 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd409d1d640 10 monclient(hunting): picked mon.noname-g con 0x7fd40409ddb0 addr v1:172.21.15.84:6790/0 2024-09-18T13:58:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd409d1d640 10 monclient(hunting): picked mon.noname-a con 0x7fd40409e180 addr v1:172.21.15.45:6789/0 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd409d1d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40409eb00 con 0x7fd40409e180 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd409d1d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40409cd70 con 0x7fd40409ddb0 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd409d1d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40413dbe0 con 0x7fd40409d410 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd409d1d640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd409d1d640 10 monclient(hunting): authenticate will time out at 728.824337s 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fbfff640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7fd40409ddb0 0x7fd4041466e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.84:37362/0 (socket says 172.21.15.84:37362) 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd4037fe640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7fd40409d410 0x7fd40413d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.84:57930/0 (socket says 172.21.15.84:57930) 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fbfff640 1 -- 172.21.15.84:0/2358875572 learned_addr learned my addr 172.21.15.84:0/2358875572 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899469354 0 0) 0x7fd40409cd70 con 0x7fd40409ddb0 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): my global_id is 4143 2024-09-18T13:58:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3e40034e0 con 0x7fd40409ddb0 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260345446 0 0) 0x7fd40413dbe0 con 0x7fd40409d410 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): my global_id is 4147 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd40409cd70 con 0x7fd40409d410 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206504359 0 0) 0x7fd40409eb00 con 0x7fd40409e180 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): my global_id is 4140 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd40413dbe0 con 0x7fd40409e180 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1363741294 0 0) 0x7fd40409cd70 con 0x7fd40409d410 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd40409eb00 con 0x7fd40409d410 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd3f4001aa0 con 0x7fd40409d410 2024-09-18T13:58:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:58:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): dump: 2024-09-18T13:58:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2696365339 0 0) 0x7fd3e40034e0 con 0x7fd40409ddb0 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd40409cd70 con 0x7fd40409ddb0 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd3ec001770 con 0x7fd40409ddb0 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T13:58:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 10 monclient(hunting): dump: 2024-09-18T13:58:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2958842794 0 0) 0x7fd40413dbe0 con 0x7fd40409e180 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.300+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd3e40034e0 con 0x7fd40409e180 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd3fc001d00 con 0x7fd40409e180 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 10 monclient(hunting): dump: 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2795267410 0 0) 0x7fd40409eb00 con 0x7fd40409d410 2024-09-18T13:58:53.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 >> v1:172.21.15.84:6790/0 conn(0x7fd40409ddb0 legacy=0x7fd4041466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 >> v1:172.21.15.45:6789/0 conn(0x7fd40409e180 legacy=0x7fd404149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 10 monclient: _finish_hunting 0 2024-09-18T13:58:53.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 1 monclient: found mon.i 2024-09-18T13:58:53.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:53.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd404122570 con 0x7fd40409d410 2024-09-18T13:58:53.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 10 monclient: _finish_auth 0 2024-09-18T13:58:53.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:53.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd409d1d640 5 monclient: authenticate success, global_id 4147 2024-09-18T13:58:53.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd409d1d640 10 monclient: _renew_subs 2024-09-18T13:58:53.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd409d1d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd409d1d640 1 -- 172.21.15.84:0/2358875572 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd404122850 con 0x7fd40409d410 2024-09-18T13:58:53.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd409d1d640 10 monclient: _renew_subs 2024-09-18T13:58:53.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd409d1d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd409d1d640 1 -- 172.21.15.84:0/2358875572 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4041217f0 con 0x7fd40409d410 2024-09-18T13:58:53.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd3f4002d70 con 0x7fd40409d410 2024-09-18T13:58:53.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:53.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd3f4002370 con 0x7fd40409d410 2024-09-18T13:58:53.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd4027fc640 4 set_mon_vals no callback set 2024-09-18T13:58:53.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:58:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 10 monclient: dump: 2024-09-18T13:58:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.328 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.328 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.328 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.328 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.328 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.328 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7fd3f4002810 con 0x7fd40409d410 2024-09-18T13:58:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.301+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (unknown 2811245281 0 0) 0x7fd3f4006210 con 0x7fd40409d410 2024-09-18T13:58:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.302+0000 7fd409d1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.302+0000 7fd409d1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.302+0000 7fd409d1d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.302+0000 7fd409d1d640 1 -- 172.21.15.84:0/2358875572 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e8005010 con 0x7fd40409d410 2024-09-18T13:58:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.303+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.7 v1:172.21.15.84:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72049 (unknown 1092875540 0 1010104650) 0x7fd3f40057d0 con 0x7fd40409d410 2024-09-18T13:58:53.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.303+0000 7fd3fb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.303+0000 7fd3fb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:58:53.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.371+0000 7fd409d1d640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "c750b0e6-9143-489d-9be0-eb085178ff2a", "id": 3}] 2024-09-18T13:58:53.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.371+0000 7fd409d1d640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "c750b0e6-9143-489d-9be0-eb085178ff2a", "id": 3}] 2024-09-18T13:58:53.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.371+0000 7fd409d1d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.371+0000 7fd409d1d640 1 -- 172.21.15.84:0/2358875572 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "osd new", "uuid": "c750b0e6-9143-489d-9be0-eb085178ff2a", "id": 3} v 0) v1 -- 0x7fd3e8005840 con 0x7fd40409d410 2024-09-18T13:58:53.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.375+0000 7fd3fb7fe640 1 -- 172.21.15.84:0/2358875572 <== mon.7 v1:172.21.15.84:6792/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c750b0e6-9143-489d-9be0-eb085178ff2a", "id": 3}]=0 v6) v1 ==== 112+0+2 (unknown 2172536985 0 525046306) 0x7fd3f4005d70 con 0x7fd40409d410 2024-09-18T13:58:53.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.375+0000 7fd3fb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "c750b0e6-9143-489d-9be0-eb085178ff2a", "id": 3}] 2024-09-18T13:58:53.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.375+0000 7fd3fb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:58:53.376 INFO:teuthology.orchestra.run.smithi084.stdout:3 2024-09-18T13:58:53.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.376+0000 7fd409d1d640 10 monclient: shutdown 2024-09-18T13:58:53.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.376+0000 7fd409d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:53.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.376+0000 7fd409d1d640 1 -- 172.21.15.84:0/2358875572 >> v1:172.21.15.84:6792/0 conn(0x7fd40409d410 legacy=0x7fd40413d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.376+0000 7fd409d1d640 1 -- 172.21.15.84:0/2358875572 shutdown_connections 2024-09-18T13:58:53.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.376+0000 7fd409d1d640 1 -- 172.21.15.84:0/2358875572 >> 172.21.15.84:0/2358875572 conn(0x7fd404068b40 msgr2=0x7fd404062fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:53.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.376+0000 7fd409d1d640 1 -- 172.21.15.84:0/2358875572 shutdown_connections 2024-09-18T13:58:53.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.376+0000 7fd409d1d640 1 -- 172.21.15.84:0/2358875572 wait complete. 2024-09-18T13:58:53.386 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new 9680908d-dc81-44a0-8603-93711579d42e 4 2024-09-18T13:58:53.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.478+0000 7f6a34a34640 10 monclient: get_monmap_and_config 2024-09-18T13:58:53.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.478+0000 7f6a34a34640 10 monclient: build_initial_monmap 2024-09-18T13:58:53.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.478+0000 7f6a34a34640 10 monclient: monmap: 2024-09-18T13:58:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 0 2024-09-18T13:58:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:53.479723+0000 2024-09-18T13:58:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:53.479723+0000 2024-09-18T13:58:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:53.481 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:53.481 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:53.481 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:53.481 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.479+0000 7f6a34a34640 1 Processor -- start 2024-09-18T13:58:53.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.479+0000 7f6a34a34640 1 -- start start 2024-09-18T13:58:53.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.479+0000 7f6a34a34640 10 monclient: init 2024-09-18T13:58:53.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:53.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 10 monclient: _add_conns ranks=[1,6,2,8,3,0,7,5,4] 2024-09-18T13:58:53.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 10 monclient(hunting): picked mon.noname-f con 0x7f6a3006e810 addr v1:172.21.15.84:6789/0 2024-09-18T13:58:53.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 10 monclient(hunting): picked mon.noname-d con 0x7f6a300a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T13:58:53.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 10 monclient(hunting): picked mon.noname-b con 0x7f6a300a1080 addr v1:172.21.15.45:6790/0 2024-09-18T13:58:53.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3005ed70 con 0x7f6a3006e810 2024-09-18T13:58:53.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a30097620 con 0x7f6a300a1080 2024-09-18T13:58:53.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a30097340 con 0x7f6a300a0cb0 2024-09-18T13:58:53.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:53.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 10 monclient(hunting): authenticate will time out at 729.004146s 2024-09-18T13:58:53.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2e59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6a3006e810 0x7f6a300a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:51686/0 (socket says 172.21.15.84:51686) 2024-09-18T13:58:53.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2e59c640 1 -- 172.21.15.84:0/1253186753 learned_addr learned my addr 172.21.15.84:0/1253186753 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-18T13:58:53.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 1 -- 172.21.15.84:0/1253186753 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344586 0 0) 0x7f6a3005ed70 con 0x7f6a3006e810 2024-09-18T13:58:53.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient(hunting): my global_id is 4132 2024-09-18T13:58:53.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 1 -- 172.21.15.84:0/1253186753 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a180034e0 con 0x7f6a3006e810 2024-09-18T13:58:53.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 1 -- 172.21.15.84:0/1253186753 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3802453361 0 0) 0x7f6a180034e0 con 0x7f6a3006e810 2024-09-18T13:58:53.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 1 -- 172.21.15.84:0/1253186753 >> v1:172.21.15.45:6792/0 conn(0x7f6a300a0cb0 legacy=0x7f6a30137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 1 -- 172.21.15.84:0/1253186753 >> v1:172.21.15.45:6790/0 conn(0x7f6a300a1080 legacy=0x7f6a3013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient: _finish_hunting 0 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 1 monclient: found mon.noname-f 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 1 -- 172.21.15.84:0/1253186753 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a300a1660 con 0x7f6a3006e810 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient: _finish_auth 0 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 5 monclient: authenticate success, global_id 4132 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 1 -- 172.21.15.84:0/1253186753 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6a24001830 con 0x7f6a3006e810 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T13:58:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient: dump: 2024-09-18T13:58:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.495 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.495 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.495 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.495 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:53.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 1 -- 172.21.15.84:0/1253186753 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6a24001990 con 0x7f6a3006e810 2024-09-18T13:58:53.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:53.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 10 monclient: get_monmap_and_config success 2024-09-18T13:58:53.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 1 -- 172.21.15.84:0/1253186753 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6a24004860 con 0x7f6a3006e810 2024-09-18T13:58:53.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a34a34640 4 set_mon_vals no callback set 2024-09-18T13:58:53.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.480+0000 7f6a2d59a640 10 monclient: dump: 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.496 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.481+0000 7f6a34a34640 10 monclient: shutdown 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.481+0000 7f6a34a34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.481+0000 7f6a34a34640 1 -- 172.21.15.84:0/1253186753 >> v1:172.21.15.84:6789/0 conn(0x7f6a3006e810 legacy=0x7f6a300a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.481+0000 7f6a34a34640 1 -- 172.21.15.84:0/1253186753 shutdown_connections 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.481+0000 7f6a34a34640 1 -- 172.21.15.84:0/1253186753 >> 172.21.15.84:0/1253186753 conn(0x7f6a300a5bb0 msgr2=0x7f6a300a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.481+0000 7f6a34a34640 1 -- 172.21.15.84:0/1253186753 shutdown_connections 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.481+0000 7f6a34a34640 1 -- 172.21.15.84:0/1253186753 wait complete. 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 10 monclient: build_initial_monmap 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 10 monclient: monmap: 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 0 2024-09-18T13:58:53.497 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 0.000000 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:created 0.000000 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 1 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:53.498 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 1 Processor -- start 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 1 -- start start 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 10 monclient: init 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 10 monclient: _add_conns ranks=[1,4,3,2,7,6,8,0,5] 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 10 monclient(hunting): picked mon.noname-f con 0x7f6a3006e810 addr v1:172.21.15.84:6789/0 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 10 monclient(hunting): picked mon.noname-c con 0x7f6a300a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 10 monclient(hunting): picked mon.noname-g con 0x7f6a300a1080 addr v1:172.21.15.84:6790/0 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a30122c80 con 0x7f6a3006e810 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3013da70 con 0x7f6a300a1080 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a301226c0 con 0x7f6a300a0cb0 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a34a34640 10 monclient(hunting): authenticate will time out at 729.006833s 2024-09-18T13:58:53.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a2e59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6a3006e810 0x7f6a3013d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:51702/0 (socket says 172.21.15.84:51702) 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.482+0000 7f6a2e59c640 1 -- 172.21.15.84:0/1010744596 learned_addr learned my addr 172.21.15.84:0/1010744596 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056527730 0 0) 0x7f6a30122c80 con 0x7f6a3006e810 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): my global_id is 4141 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a100034e0 con 0x7f6a3006e810 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189836599 0 0) 0x7f6a3013da70 con 0x7f6a300a1080 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): my global_id is 4152 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a30122c80 con 0x7f6a300a1080 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876946321 0 0) 0x7f6a301226c0 con 0x7f6a300a0cb0 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): my global_id is 4135 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a3013da70 con 0x7f6a300a0cb0 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3587583600 0 0) 0x7f6a30122c80 con 0x7f6a300a1080 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a301226c0 con 0x7f6a300a1080 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6a28001c20 con 0x7f6a300a1080 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): dump: 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.501 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4169628226 0 0) 0x7f6a100034e0 con 0x7f6a3006e810 2024-09-18T13:58:53.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a30122c80 con 0x7f6a3006e810 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4072191861 0 0) 0x7f6a3013da70 con 0x7f6a300a0cb0 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a100034e0 con 0x7f6a300a0cb0 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6a240016f0 con 0x7f6a3006e810 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): dump: 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.503 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6a200019e0 con 0x7f6a300a0cb0 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient(hunting): dump: 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.504 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2471596768 0 0) 0x7f6a301226c0 con 0x7f6a300a1080 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 >> v1:172.21.15.45:6791/0 conn(0x7f6a300a0cb0 legacy=0x7f6a30146590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 >> v1:172.21.15.84:6789/0 conn(0x7f6a3006e810 legacy=0x7f6a3013d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 monclient: found mon.g 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a301201d0 con 0x7f6a300a1080 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient: _finish_auth 0 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a34a34640 5 monclient: authenticate success, global_id 4152 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a34a34640 10 monclient: _renew_subs 2024-09-18T13:58:53.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a34a34640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a34a34640 1 -- 172.21.15.84:0/1010744596 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a301223e0 con 0x7f6a300a1080 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a34a34640 10 monclient: _renew_subs 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a34a34640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a34a34640 1 -- 172.21.15.84:0/1010744596 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a30121510 con 0x7f6a300a1080 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6a28001e80 con 0x7f6a300a1080 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6a28002430 con 0x7f6a300a1080 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a2cd99640 4 set_mon_vals no callback set 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 10 monclient: dump: 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.507 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.508 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f6a280028d0 con 0x7f6a300a1080 2024-09-18T13:58:53.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.483+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (unknown 3187008587 0 0) 0x7f6a28003cf0 con 0x7f6a300a1080 2024-09-18T13:58:53.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.484+0000 7f6a34a34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.484+0000 7f6a34a34640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.484+0000 7f6a34a34640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:58:53.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.484+0000 7f6a34a34640 1 -- 172.21.15.84:0/1010744596 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a14005010 con 0x7f6a300a1080 2024-09-18T13:58:53.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.486+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.3 v1:172.21.15.84:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72049 (unknown 1092875540 0 1010104650) 0x7f6a280032b0 con 0x7f6a300a1080 2024-09-18T13:58:53.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.486+0000 7f6a1e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.486+0000 7f6a1e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:58:53.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.554+0000 7f6a34a34640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "9680908d-dc81-44a0-8603-93711579d42e", "id": 4}] 2024-09-18T13:58:53.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.554+0000 7f6a34a34640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "9680908d-dc81-44a0-8603-93711579d42e", "id": 4}] 2024-09-18T13:58:53.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.554+0000 7f6a34a34640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:58:53.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.554+0000 7f6a34a34640 1 -- 172.21.15.84:0/1010744596 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "osd new", "uuid": "9680908d-dc81-44a0-8603-93711579d42e", "id": 4} v 0) v1 -- 0x7f6a14005840 con 0x7f6a300a1080 2024-09-18T13:58:53.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.558+0000 7f6a1e7fc640 1 -- 172.21.15.84:0/1010744596 <== mon.3 v1:172.21.15.84:6790/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9680908d-dc81-44a0-8603-93711579d42e", "id": 4}]=0 v7) v1 ==== 112+0+2 (unknown 1679982767 0 1697012583) 0x7f6a28003790 con 0x7f6a300a1080 2024-09-18T13:58:53.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.558+0000 7f6a1e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "9680908d-dc81-44a0-8603-93711579d42e", "id": 4}] 2024-09-18T13:58:53.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.558+0000 7f6a1e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:58:53.559 INFO:teuthology.orchestra.run.smithi084.stdout:4 2024-09-18T13:58:53.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.559+0000 7f6a34a34640 10 monclient: shutdown 2024-09-18T13:58:53.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.559+0000 7f6a34a34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:53.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.559+0000 7f6a34a34640 1 -- 172.21.15.84:0/1010744596 >> v1:172.21.15.84:6790/0 conn(0x7f6a300a1080 legacy=0x7f6a30149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.559+0000 7f6a34a34640 1 -- 172.21.15.84:0/1010744596 shutdown_connections 2024-09-18T13:58:53.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.559+0000 7f6a34a34640 1 -- 172.21.15.84:0/1010744596 >> 172.21.15.84:0/1010744596 conn(0x7f6a300a5bb0 msgr2=0x7f6a300a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:53.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.559+0000 7f6a34a34640 1 -- 172.21.15.84:0/1010744596 shutdown_connections 2024-09-18T13:58:53.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.559+0000 7f6a34a34640 1 -- 172.21.15.84:0/1010744596 wait complete. 2024-09-18T13:58:53.569 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new b3b63eef-c8fc-495f-960f-7436315119ab 5 2024-09-18T13:58:53.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.660+0000 7fbdd3ed4640 10 monclient: get_monmap_and_config 2024-09-18T13:58:53.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.660+0000 7fbdd3ed4640 10 monclient: build_initial_monmap 2024-09-18T13:58:53.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.660+0000 7fbdd3ed4640 10 monclient: monmap: 2024-09-18T13:58:53.662 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 0 2024-09-18T13:58:53.662 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.662 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:53.661794+0000 2024-09-18T13:58:53.662 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:53.661794+0000 2024-09-18T13:58:53.662 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:53.662 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.662 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:53.663 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:53.663 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:53.663 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:53.663 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:53.663 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:53.663 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:53.663 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:53.663 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:53.663 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.661+0000 7fbdd3ed4640 1 Processor -- start 2024-09-18T13:58:53.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.661+0000 7fbdd3ed4640 1 -- start start 2024-09-18T13:58:53.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.661+0000 7fbdd3ed4640 10 monclient: init 2024-09-18T13:58:53.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:53.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 10 monclient: _add_conns ranks=[3,6,2,7,0,8,4,5,1] 2024-09-18T13:58:53.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 10 monclient(hunting): picked mon.noname-g con 0x7fbdcc09c9c0 addr v1:172.21.15.84:6790/0 2024-09-18T13:58:53.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 10 monclient(hunting): picked mon.noname-d con 0x7fbdcc09aa10 addr v1:172.21.15.45:6792/0 2024-09-18T13:58:53.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 10 monclient(hunting): picked mon.noname-b con 0x7fbdcc1367f0 addr v1:172.21.15.45:6790/0 2024-09-18T13:58:53.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc05ed70 con 0x7fbdcc1367f0 2024-09-18T13:58:53.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc099a80 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc09dbb0 con 0x7fbdcc09aa10 2024-09-18T13:58:53.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:53.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 10 monclient(hunting): authenticate will time out at 729.186284s 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd1c70640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7fbdcc09c9c0 0x7fbdcc09cd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.84:37374/0 (socket says 172.21.15.84:37374) 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd1c70640 1 -- 172.21.15.84:0/3104998452 learned_addr learned my addr 172.21.15.84:0/3104998452 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523081463 0 0) 0x7fbdcc099a80 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient(hunting): my global_id is 4161 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb80034e0 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001548843 0 0) 0x7fbdcc09dbb0 con 0x7fbdcc09aa10 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient(hunting): my global_id is 4155 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdcc099a80 con 0x7fbdcc09aa10 2024-09-18T13:58:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 791046430 0 0) 0x7fbdcc05ed70 con 0x7fbdcc1367f0 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient(hunting): my global_id is 4160 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdcc09dbb0 con 0x7fbdcc1367f0 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 198016579 0 0) 0x7fbdb80034e0 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 >> v1:172.21.15.45:6792/0 conn(0x7fbdcc09aa10 legacy=0x7fbdcc09d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 >> v1:172.21.15.45:6790/0 conn(0x7fbdcc1367f0 legacy=0x7fbdcc138bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient: _finish_hunting 0 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 monclient: found mon.noname-g 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc099810 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient: _finish_auth 0 2024-09-18T13:58:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbdc8001830 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 5 monclient: authenticate success, global_id 4161 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient: dump: 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbdc8001a10 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 10 monclient: get_monmap_and_config success 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 1 -- 172.21.15.84:0/3104998452 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbdc8004ba0 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd3ed4640 4 set_mon_vals no callback set 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.662+0000 7fbdd0c6e640 10 monclient: dump: 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.663+0000 7fbdd3ed4640 10 monclient: shutdown 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.663+0000 7fbdd3ed4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.663+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/3104998452 >> v1:172.21.15.84:6790/0 conn(0x7fbdcc09c9c0 legacy=0x7fbdcc09cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.663+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/3104998452 shutdown_connections 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.663+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/3104998452 >> 172.21.15.84:0/3104998452 conn(0x7fbdcc0a2710 msgr2=0x7fbdcc09eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.663+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/3104998452 shutdown_connections 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.663+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/3104998452 wait complete. 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 10 monclient: build_initial_monmap 2024-09-18T13:58:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 10 monclient: monmap: 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 0 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 0.000000 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:created 0.000000 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 1 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:53.679 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:53.679 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 1 Processor -- start 2024-09-18T13:58:53.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 1 -- start start 2024-09-18T13:58:53.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 10 monclient: init 2024-09-18T13:58:53.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:53.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 10 monclient: _add_conns ranks=[4,7,0,1,2,8,6,5,3] 2024-09-18T13:58:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 10 monclient(hunting): picked mon.noname-c con 0x7fbdcc09aa10 addr v1:172.21.15.45:6791/0 2024-09-18T13:58:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 10 monclient(hunting): picked mon.noname-i con 0x7fbdcc09c9c0 addr v1:172.21.15.84:6792/0 2024-09-18T13:58:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 10 monclient(hunting): picked mon.noname-a con 0x7fbdcc1367f0 addr v1:172.21.15.45:6789/0 2024-09-18T13:58:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc1302d0 con 0x7fbdcc1367f0 2024-09-18T13:58:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc09c420 con 0x7fbdcc09aa10 2024-09-18T13:58:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc06ec60 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.664+0000 7fbdd3ed4640 10 monclient(hunting): authenticate will time out at 729.188952s 2024-09-18T13:58:53.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdd146f640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7fbdcc09c9c0 0x7fbdcc06d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.84:57944/0 (socket says 172.21.15.84:57944) 2024-09-18T13:58:53.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdd146f640 1 -- 172.21.15.84:0/1750723765 learned_addr learned my addr 172.21.15.84:0/1750723765 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-18T13:58:53.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219256743 0 0) 0x7fbdcc06ec60 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): my global_id is 4156 2024-09-18T13:58:53.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb00034e0 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609642650 0 0) 0x7fbdcc09c420 con 0x7fbdcc09aa10 2024-09-18T13:58:53.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): my global_id is 4144 2024-09-18T13:58:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdcc06ec60 con 0x7fbdcc09aa10 2024-09-18T13:58:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610868231 0 0) 0x7fbdcc1302d0 con 0x7fbdcc1367f0 2024-09-18T13:58:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): my global_id is 4149 2024-09-18T13:58:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdcc09c420 con 0x7fbdcc1367f0 2024-09-18T13:58:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1104695981 0 0) 0x7fbdb00034e0 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbdcc1302d0 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbdbc001be0 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:58:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): dump: 2024-09-18T13:58:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3367574702 0 0) 0x7fbdcc06ec60 con 0x7fbdcc09aa10 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbdb00034e0 con 0x7fbdcc09aa10 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3612510334 0 0) 0x7fbdcc1302d0 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 >> v1:172.21.15.45:6791/0 conn(0x7fbdcc09aa10 legacy=0x7fbdcc06cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 >> v1:172.21.15.45:6789/0 conn(0x7fbdcc1367f0 legacy=0x7fbdcc06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient: _finish_hunting 0 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 monclient: found mon.i 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc11a250 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient: _finish_auth 0 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdd3ed4640 5 monclient: authenticate success, global_id 4156 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdd3ed4640 10 monclient: _renew_subs 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdd3ed4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/1750723765 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdcc118720 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbdbc002170 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbdbc002660 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc1ffb640 10 monclient: dump: 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:epoch 1 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:election_strategy: 3 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.665+0000 7fbdc3fff640 4 set_mon_vals no callback set 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.666+0000 7fbdd3ed4640 10 monclient: _renew_subs 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.666+0000 7fbdd3ed4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.666+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7fbdbc002a40 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.666+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/1750723765 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdcc119490 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.666+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (unknown 1377650690 0 0) 0x7fbdbc005820 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.666+0000 7fbdd3ed4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.666+0000 7fbdd3ed4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.666+0000 7fbdd3ed4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.666+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/1750723765 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdcc04c4a0 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.668+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.7 v1:172.21.15.84:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72049 (unknown 1092875540 0 1010104650) 0x7fbdbc017980 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.668+0000 7fbdc1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.668+0000 7fbdc1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:58:53.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.736+0000 7fbdd3ed4640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "b3b63eef-c8fc-495f-960f-7436315119ab", "id": 5}] 2024-09-18T13:58:53.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.736+0000 7fbdd3ed4640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "b3b63eef-c8fc-495f-960f-7436315119ab", "id": 5}] 2024-09-18T13:58:53.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.736+0000 7fbdd3ed4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:58:53.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.736+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/1750723765 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "osd new", "uuid": "b3b63eef-c8fc-495f-960f-7436315119ab", "id": 5} v 0) v1 -- 0x7fbdcc12f930 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.739+0000 7fbdc1ffb640 1 -- 172.21.15.84:0/1750723765 <== mon.7 v1:172.21.15.84:6792/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b3b63eef-c8fc-495f-960f-7436315119ab", "id": 5}]=0 v8) v1 ==== 112+0+2 (unknown 3507134825 0 1988412176) 0x7fbdbc0060e0 con 0x7fbdcc09c9c0 2024-09-18T13:58:53.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.739+0000 7fbdc1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "b3b63eef-c8fc-495f-960f-7436315119ab", "id": 5}] 2024-09-18T13:58:53.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.739+0000 7fbdc1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:58:53.740 INFO:teuthology.orchestra.run.smithi084.stdout:5 2024-09-18T13:58:53.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.740+0000 7fbdd3ed4640 10 monclient: shutdown 2024-09-18T13:58:53.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.740+0000 7fbdd3ed4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:53.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.740+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/1750723765 >> v1:172.21.15.84:6792/0 conn(0x7fbdcc09c9c0 legacy=0x7fbdcc06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.740+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/1750723765 shutdown_connections 2024-09-18T13:58:53.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.740+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/1750723765 >> 172.21.15.84:0/1750723765 conn(0x7fbdcc0a2710 msgr2=0x7fbdcc0a01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:53.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.740+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/1750723765 shutdown_connections 2024-09-18T13:58:53.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-18T13:58:53.740+0000 7fbdd3ed4640 1 -- 172.21.15.84:0/1750723765 wait complete. 2024-09-18T13:58:53.750 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-18T13:58:53.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-18T13:58:53.754 INFO:tasks.ceph.osd.0:Started 2024-09-18T13:58:53.754 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-18T13:58:53.754 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-18T13:58:53.756 INFO:tasks.ceph.osd.1:Started 2024-09-18T13:58:53.756 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-18T13:58:53.757 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-18T13:58:53.759 INFO:tasks.ceph.osd.2:Started 2024-09-18T13:58:53.759 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-18T13:58:53.759 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-18T13:58:53.761 INFO:tasks.ceph.osd.3:Started 2024-09-18T13:58:53.761 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-18T13:58:53.761 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-18T13:58:53.763 INFO:tasks.ceph.osd.4:Started 2024-09-18T13:58:53.763 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-18T13:58:53.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-18T13:58:53.767 INFO:tasks.ceph.osd.5:Started 2024-09-18T13:58:53.769 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T13:58:53.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.881+0000 7fb63ac8b640 10 monclient: get_monmap_and_config 2024-09-18T13:58:53.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.881+0000 7fb63ac8b640 10 monclient: build_initial_monmap 2024-09-18T13:58:53.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.881+0000 7fb63ac8b640 10 monclient: monmap: 2024-09-18T13:58:53.882 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:53.882885+0000 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:53.882885+0000 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:53.883 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:53.884 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.882+0000 7fb63ac8b640 1 Processor -- start 2024-09-18T13:58:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.882+0000 7fb63ac8b640 1 -- start start 2024-09-18T13:58:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.882+0000 7fb63ac8b640 10 monclient: init 2024-09-18T13:58:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb63ac8b640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb63ac8b640 10 monclient: _add_conns ranks=[5,4,6,7,0,1,3,2,8] 2024-09-18T13:58:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb63ac8b640 10 monclient(hunting): picked mon.noname-h con 0x7fb63406e810 addr v1:172.21.15.84:6791/0 2024-09-18T13:58:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb63ac8b640 10 monclient(hunting): picked mon.noname-c con 0x7fb6340a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T13:58:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb63ac8b640 10 monclient(hunting): picked mon.noname-d con 0x7fb6340a1080 addr v1:172.21.15.45:6792/0 2024-09-18T13:58:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb63ac8b640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63405ed70 con 0x7fb6340a0cb0 2024-09-18T13:58:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb639228640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fb6340a1080 0x7fb63413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:50724/0 (socket says 172.21.15.45:50724) 2024-09-18T13:58:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb639228640 1 -- 172.21.15.45:0/1180596732 learned_addr learned my addr 172.21.15.45:0/1180596732 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:58:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb633fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fb6340a0cb0 0x7fb634137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36902/0 (socket says 172.21.15.45:36902) 2024-09-18T13:58:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb63ac8b640 1 -- 172.21.15.45:0/1180596732 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb634097620 con 0x7fb63406e810 2024-09-18T13:58:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb63ac8b640 1 -- 172.21.15.45:0/1180596732 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb634097340 con 0x7fb6340a1080 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb63ac8b640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.883+0000 7fb63ac8b640 10 monclient(hunting): authenticate will time out at 749.221890s 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087538859 0 0) 0x7fb63405ed70 con 0x7fb6340a0cb0 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient(hunting): my global_id is 4171 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6240034e0 con 0x7fb6340a0cb0 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719455107 0 0) 0x7fb634097340 con 0x7fb6340a1080 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient(hunting): my global_id is 4191 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb63405ed70 con 0x7fb6340a1080 2024-09-18T13:58:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916176357 0 0) 0x7fb634097620 con 0x7fb63406e810 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient(hunting): my global_id is 4136 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb634097340 con 0x7fb63406e810 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3101726820 0 0) 0x7fb634097340 con 0x7fb63406e810 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 >> v1:172.21.15.45:6792/0 conn(0x7fb6340a1080 legacy=0x7fb63413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 >> v1:172.21.15.45:6791/0 conn(0x7fb6340a0cb0 legacy=0x7fb634137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient: _finish_hunting 0 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 monclient: found mon.noname-h 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6340a1660 con 0x7fb63406e810 2024-09-18T13:58:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient: _finish_auth 0 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb6280016d0 con 0x7fb63406e810 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient: dump: 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb63ac8b640 5 monclient: authenticate success, global_id 4136 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb63ac8b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb628001830 con 0x7fb63406e810 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 1 -- 172.21.15.45:0/1180596732 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb628001d60 con 0x7fb63406e810 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb6337fe640 10 monclient: dump: 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.896 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb63ac8b640 10 monclient: get_monmap_and_config success 2024-09-18T13:58:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.884+0000 7fb63ac8b640 4 set_mon_vals no callback set 2024-09-18T13:58:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.885+0000 7fb63ac8b640 10 monclient: shutdown 2024-09-18T13:58:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.885+0000 7fb63ac8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.885+0000 7fb63ac8b640 1 -- 172.21.15.45:0/1180596732 >> v1:172.21.15.84:6791/0 conn(0x7fb63406e810 legacy=0x7fb6340a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.885+0000 7fb63ac8b640 1 -- 172.21.15.45:0/1180596732 shutdown_connections 2024-09-18T13:58:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.885+0000 7fb63ac8b640 1 -- 172.21.15.45:0/1180596732 >> 172.21.15.45:0/1180596732 conn(0x7fb6340a5bb0 msgr2=0x7fb6340a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.885+0000 7fb63ac8b640 1 -- 172.21.15.45:0/1180596732 shutdown_connections 2024-09-18T13:58:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.885+0000 7fb63ac8b640 1 -- 172.21.15.45:0/1180596732 wait complete. 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.885+0000 7fb63ac8b640 10 monclient: build_initial_monmap 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.885+0000 7fb63ac8b640 10 monclient: monmap: 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.885+0000 7fb63ac8b640 1 Processor -- start 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb63ac8b640 1 -- start start 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb63ac8b640 10 monclient: init 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb63ac8b640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb63ac8b640 10 monclient: _add_conns ranks=[1,3,8,4,0,6,5,7,2] 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb63ac8b640 10 monclient(hunting): picked mon.noname-f con 0x7fb63406e810 addr v1:172.21.15.84:6789/0 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb63ac8b640 10 monclient(hunting): picked mon.noname-g con 0x7fb6340a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb63ac8b640 10 monclient(hunting): picked mon.noname-e con 0x7fb6340a1080 addr v1:172.21.15.45:6793/0 2024-09-18T13:58:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb63ac8b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63409fa10 con 0x7fb63406e810 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb63ac8b640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63413db80 con 0x7fb6340a0cb0 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb63ac8b640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6340a2270 con 0x7fb6340a1080 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb63ac8b640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb639228640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fb6340a1080 0x7fb634149db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:53670/0 (socket says 172.21.15.45:53670) 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb639228640 1 -- 172.21.15.45:0/3337302664 learned_addr learned my addr 172.21.15.45:0/3337302664 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.886+0000 7fb63ac8b640 10 monclient(hunting): authenticate will time out at 749.224826s 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147914581 0 0) 0x7fb6340a2270 con 0x7fb6340a1080 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): my global_id is 4166 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6180034e0 con 0x7fb6340a1080 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378823036 0 0) 0x7fb63409fa10 con 0x7fb63406e810 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): my global_id is 4150 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6340a2270 con 0x7fb63406e810 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45635210 0 0) 0x7fb63413db80 con 0x7fb6340a0cb0 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): my global_id is 4197 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb63409fa10 con 0x7fb6340a0cb0 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1043840987 0 0) 0x7fb6180034e0 con 0x7fb6340a1080 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb63413db80 con 0x7fb6340a1080 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb62c001ca0 con 0x7fb6340a1080 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:58:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): dump: 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4022297177 0 0) 0x7fb6340a2270 con 0x7fb63406e810 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb6180034e0 con 0x7fb63406e810 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3863589562 0 0) 0x7fb63409fa10 con 0x7fb6340a0cb0 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb618003b60 con 0x7fb6340a0cb0 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb6200014e0 con 0x7fb6340a0cb0 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient(hunting): dump: 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:58:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.903 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1785134492 0 0) 0x7fb63413db80 con 0x7fb6340a1080 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 >> v1:172.21.15.84:6790/0 conn(0x7fb6340a0cb0 legacy=0x7fb634146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 >> v1:172.21.15.84:6789/0 conn(0x7fb63406e810 legacy=0x7fb63413d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient: _finish_hunting 0 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 monclient: found mon.e 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63413e990 con 0x7fb6340a1080 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient: _finish_auth 0 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb63ac8b640 5 monclient: authenticate success, global_id 4166 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb63ac8b640 10 monclient: _renew_subs 2024-09-18T13:58:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb63ac8b640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:58:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb63ac8b640 1 -- 172.21.15.45:0/3337302664 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb634122510 con 0x7fb6340a1080 2024-09-18T13:58:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb63ac8b640 10 monclient: _renew_subs 2024-09-18T13:58:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb63ac8b640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:58:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb63ac8b640 1 -- 172.21.15.45:0/3337302664 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb634120120 con 0x7fb6340a1080 2024-09-18T13:58:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb62c004be0 con 0x7fb6340a1080 2024-09-18T13:58:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb62c005190 con 0x7fb6340a1080 2024-09-18T13:58:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:58:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 10 monclient: dump: 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:53.907 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:58:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb632ffd640 4 set_mon_vals no callback set 2024-09-18T13:58:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7fb62c005570 con 0x7fb6340a1080 2024-09-18T13:58:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.887+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (unknown 3832247472 0 0) 0x7fb62c005820 con 0x7fb6340a1080 2024-09-18T13:58:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.888+0000 7fb63ac8b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.888+0000 7fb63ac8b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.888+0000 7fb63ac8b640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:58:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.888+0000 7fb63ac8b640 1 -- 172.21.15.45:0/3337302664 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63404c4a0 con 0x7fb6340a1080 2024-09-18T13:58:53.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.890+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72049 (unknown 1092875540 0 1010104650) 0x7fb62c0060a0 con 0x7fb6340a1080 2024-09-18T13:58:53.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.890+0000 7fb630ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:53.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.890+0000 7fb630ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:58:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.948+0000 7fb63ac8b640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:58:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.948+0000 7fb63ac8b640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:58:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.948+0000 7fb63ac8b640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:58:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.948+0000 7fb63ac8b640 1 -- 172.21.15.45:0/3337302664 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb63404b720 con 0x7fb6340a1080 2024-09-18T13:58:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.948+0000 7fb630ff9640 1 -- 172.21.15.45:0/3337302664 <== mon.8 v1:172.21.15.45:6793/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5560 (unknown 150273500 0 3599917525) 0x7fb62c0062c0 con 0x7fb6340a1080 2024-09-18T13:58:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.948+0000 7fb630ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:58:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.948+0000 7fb630ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:58:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:58:53.950 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","created":"2024-09-18T13:58:52.168851+0000","modified":"2024-09-18T13:58:53.737759+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T13:58:53.737759+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"8c3a8909-07ce-4141-98dd-131951f18ae2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a12b3bc3-1e03-47dc-98c1-c8bd199551a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"216b8b62-e41c-4bc5-9395-f2d56d402b1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c750b0e6-9143-489d-9be0-eb085178ff2a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9680908d-dc81-44a0-8603-93711579d42e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b3b63eef-c8fc-495f-960f-7436315119ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T13:58:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.949+0000 7fb63ac8b640 10 monclient: shutdown 2024-09-18T13:58:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.949+0000 7fb63ac8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.949+0000 7fb63ac8b640 1 -- 172.21.15.45:0/3337302664 >> v1:172.21.15.45:6793/0 conn(0x7fb6340a1080 legacy=0x7fb634149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.949+0000 7fb63ac8b640 1 -- 172.21.15.45:0/3337302664 shutdown_connections 2024-09-18T13:58:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.949+0000 7fb63ac8b640 1 -- 172.21.15.45:0/3337302664 >> 172.21.15.45:0/3337302664 conn(0x7fb6340a5bb0 msgr2=0x7fb6340a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.949+0000 7fb63ac8b640 1 -- 172.21.15.45:0/3337302664 shutdown_connections 2024-09-18T13:58:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:58:53.949+0000 7fb63ac8b640 1 -- 172.21.15.45:0/3337302664 wait complete. 2024-09-18T13:58:53.961 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-18T13:58:53.961 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-18T13:58:54.111 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:54.110+0000 7f3f645c1ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T13:58:54.163 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:54.162+0000 7f3f645c1ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T13:58:54.348 INFO:tasks.ceph.osd.4.smithi084.stderr:2024-09-18T13:58:54.347+0000 7f1e10d1d440 -1 Falling back to public interface 2024-09-18T13:58:54.349 INFO:tasks.ceph.osd.3.smithi084.stderr:2024-09-18T13:58:54.347+0000 7f23a9723440 -1 Falling back to public interface 2024-09-18T13:58:54.349 INFO:tasks.ceph.osd.3.smithi084.stderr:2024-09-18T13:58:54.347+0000 7f23a79fd640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6800/0: (98) Address already in use 2024-09-18T13:58:54.349 INFO:tasks.ceph.osd.3.smithi084.stderr:2024-09-18T13:58:54.347+0000 7f23a79fd640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2024-09-18T13:58:54.364 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-18T13:58:54.363+0000 7f0474f23440 -1 Falling back to public interface 2024-09-18T13:58:54.365 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-18T13:58:54.365+0000 7fbb05416440 -1 Falling back to public interface 2024-09-18T13:58:54.366 INFO:tasks.ceph.osd.5.smithi084.stderr:2024-09-18T13:58:54.365+0000 7fbb14a19440 -1 Falling back to public interface 2024-09-18T13:58:54.369 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-18T13:58:54.368+0000 7ff26512f440 -1 Falling back to public interface 2024-09-18T13:58:54.524 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:54.524+0000 7f3f645c1ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-18T13:58:54.565 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T13:58:54.605 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:54.604+0000 7f3f645c1ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T13:58:54.658 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.657+0000 7f6339d9d640 10 monclient: get_monmap_and_config 2024-09-18T13:58:54.658 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.657+0000 7f6339d9d640 10 monclient: build_initial_monmap 2024-09-18T13:58:54.658 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.657+0000 7f6339d9d640 10 monclient: monmap: 2024-09-18T13:58:54.658 INFO:teuthology.misc.health.smithi045.stderr:epoch 0 2024-09-18T13:58:54.658 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:54.659 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:54.658741+0000 2024-09-18T13:58:54.659 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:54.658741+0000 2024-09-18T13:58:54.659 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:54.659 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:54.659 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:54.659 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:54.659 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:54.659 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:54.659 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:54.659 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:54.660 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:54.660 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:54.660 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:54.660 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:58:54.668 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.658+0000 7f6339d9d640 1 Processor -- start 2024-09-18T13:58:54.668 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.658+0000 7f6339d9d640 1 -- start start 2024-09-18T13:58:54.668 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.658+0000 7f6339d9d640 10 monclient: init 2024-09-18T13:58:54.668 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:54.668 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 10 monclient: _add_conns ranks=[1,4,2,7,6,8,5,0,3] 2024-09-18T13:58:54.668 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 10 monclient(hunting): picked mon.noname-f con 0x7f633409f5d0 addr v1:172.21.15.84:6789/0 2024-09-18T13:58:54.668 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 10 monclient(hunting): picked mon.noname-c con 0x7f63340a2840 addr v1:172.21.15.45:6791/0 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 10 monclient(hunting): picked mon.noname-b con 0x7f63341329e0 addr v1:172.21.15.45:6790/0 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6334066630 con 0x7f633409f5d0 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340a00b0 con 0x7f63341329e0 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340a1af0 con 0x7f63340a2840 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 10 monclient(hunting): authenticate will time out at 749.997107s 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6332ffd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f63340a2840 0x7f6334137840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36916/0 (socket says 172.21.15.45:36916) 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6333fff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f63341329e0 0x7f633413af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:56922/0 (socket says 172.21.15.45:56922) 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6332ffd640 1 -- 172.21.15.45:0/3596955155 learned_addr learned my addr 172.21.15.45:0/3596955155 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859264776 0 0) 0x7f63340a00b0 con 0x7f63341329e0 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:54.669 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient(hunting): my global_id is 4187 2024-09-18T13:58:54.670 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63200034e0 con 0x7f63341329e0 2024-09-18T13:58:54.670 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379089394 0 0) 0x7f63340a1af0 con 0x7f63340a2840 2024-09-18T13:58:54.670 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:54.670 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:54.670 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient(hunting): my global_id is 4180 2024-09-18T13:58:54.670 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63340a00b0 con 0x7f63340a2840 2024-09-18T13:58:54.670 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723681832 0 0) 0x7f6334066630 con 0x7f633409f5d0 2024-09-18T13:58:54.670 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:54.670 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:54.670 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient(hunting): my global_id is 4159 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63340a1af0 con 0x7f633409f5d0 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 580410118 0 0) 0x7f63340a00b0 con 0x7f63340a2840 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 >> v1:172.21.15.45:6790/0 conn(0x7f63341329e0 legacy=0x7f633413af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 >> v1:172.21.15.84:6789/0 conn(0x7f633409f5d0 legacy=0x7f633409f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient: _finish_hunting 0 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 1 monclient: found mon.noname-c 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633409d800 con 0x7f63340a2840 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient: _finish_auth 0 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f631c001810 con 0x7f63340a2840 2024-09-18T13:58:54.671 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 5 monclient: authenticate success, global_id 4180 2024-09-18T13:58:54.672 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:54.672 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:54.672 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T13:58:54.672 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f63327fc640 10 monclient: dump: 2024-09-18T13:58:54.672 INFO:teuthology.misc.health.smithi045.stderr:epoch 1 2024-09-18T13:58:54.672 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:54.672 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:54.672 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:54.672 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:54.672 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.659+0000 7f6339d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f631c001970 con 0x7f63340a2840 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f63327fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:54.673 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f6339d9d640 10 monclient: get_monmap_and_config success 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f63327fc640 1 -- 172.21.15.45:0/3596955155 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f631c004ba0 con 0x7f63340a2840 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f6339d9d640 4 set_mon_vals no callback set 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f63327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f63327fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f63327fc640 10 monclient: dump: 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:epoch 1 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:54.674 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f6339d9d640 10 monclient: shutdown 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f6339d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f6339d9d640 1 -- 172.21.15.45:0/3596955155 >> v1:172.21.15.45:6791/0 conn(0x7f63340a2840 legacy=0x7f6334137840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f6339d9d640 1 -- 172.21.15.45:0/3596955155 shutdown_connections 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f6339d9d640 1 -- 172.21.15.45:0/3596955155 >> 172.21.15.45:0/3596955155 conn(0x7f63340a5c70 msgr2=0x7f63340a6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f6339d9d640 1 -- 172.21.15.45:0/3596955155 shutdown_connections 2024-09-18T13:58:54.675 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.660+0000 7f6339d9d640 1 -- 172.21.15.45:0/3596955155 wait complete. 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.661+0000 7f6339d9d640 10 monclient: build_initial_monmap 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.661+0000 7f6339d9d640 10 monclient: monmap: 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:epoch 0 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:last_changed 0.000000 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:created 0.000000 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 1 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:58:54.676 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.661+0000 7f6339d9d640 1 Processor -- start 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.661+0000 7f6339d9d640 1 -- start start 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.661+0000 7f6339d9d640 10 monclient: init 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f6339d9d640 10 monclient: _reopen_session rank -1 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f6339d9d640 10 monclient: _add_conns ranks=[0,5,1,2,4,8,3,7,6] 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f6339d9d640 10 monclient(hunting): picked mon.noname-a con 0x7f633409f5d0 addr v1:172.21.15.45:6789/0 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f6339d9d640 10 monclient(hunting): picked mon.noname-h con 0x7f63340a2840 addr v1:172.21.15.84:6791/0 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f6339d9d640 10 monclient(hunting): picked mon.noname-f con 0x7f63341329e0 addr v1:172.21.15.84:6789/0 2024-09-18T13:58:54.677 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f6339d9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6334142320 con 0x7f633409f5d0 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f6339d9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633411d4a0 con 0x7f63341329e0 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f6339d9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340a2020 con 0x7f63340a2840 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f6339d9d640 10 monclient(hunting): _renew_subs 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f6339d9d640 10 monclient(hunting): authenticate will time out at 750.000235s 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f63337fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f633409f5d0 0x7f6334131510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49986/0 (socket says 172.21.15.45:49986) 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f63337fe640 1 -- 172.21.15.45:0/2781936818 learned_addr learned my addr 172.21.15.45:0/2781936818 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797251661 0 0) 0x7f6334142320 con 0x7f633409f5d0 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): my global_id is 4167 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63140034e0 con 0x7f633409f5d0 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266179590 0 0) 0x7f633411d4a0 con 0x7f63341329e0 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:54.678 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): my global_id is 4168 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6334142320 con 0x7f63341329e0 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506302081 0 0) 0x7f63340a2020 con 0x7f63340a2840 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): my global_id is 4145 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f633411d4a0 con 0x7f63340a2840 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 226475481 0 0) 0x7f63140034e0 con 0x7f633409f5d0 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63340a2020 con 0x7f633409f5d0 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6324001c50 con 0x7f633409f5d0 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:58:54.679 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): dump: 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:epoch 1 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:54.680 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3338007680 0 0) 0x7f633411d4a0 con 0x7f63340a2840 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63140034e0 con 0x7f63340a2840 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f631c001750 con 0x7f63340a2840 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): dump: 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:epoch 1 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:54.681 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3982684339 0 0) 0x7f6334142320 con 0x7f63341329e0 2024-09-18T13:58:54.682 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f633411d4a0 con 0x7f63341329e0 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f632c001d40 con 0x7f63341329e0 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 10 monclient(hunting): dump: 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:epoch 1 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:54.683 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3781175132 0 0) 0x7f63340a2020 con 0x7f633409f5d0 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.662+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 >> v1:172.21.15.84:6791/0 conn(0x7f63340a2840 legacy=0x7f63341467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 >> v1:172.21.15.84:6789/0 conn(0x7f63341329e0 legacy=0x7f6334149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 10 monclient: _finish_hunting 0 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 1 monclient: found mon.a 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:58:54.684 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633411e630 con 0x7f633409f5d0 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 10 monclient: _finish_auth 0 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f6339d9d640 5 monclient: authenticate success, global_id 4167 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f6339d9d640 10 monclient: _renew_subs 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f6339d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f6339d9d640 1 -- 172.21.15.45:0/2781936818 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633409b940 con 0x7f633409f5d0 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f6339d9d640 10 monclient: _renew_subs 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f6339d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f6339d9d640 1 -- 172.21.15.45:0/2781936818 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f633411d740 con 0x7f633409f5d0 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f63240020c0 con 0x7f633409f5d0 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f63240025f0 con 0x7f633409f5d0 2024-09-18T13:58:54.685 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f6331ffb640 4 set_mon_vals no callback set 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 10 monclient: dump: 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:epoch 1 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:58:54.686 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f63240053f0 con 0x7f633409f5d0 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.663+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (unknown 3832247472 0 0) 0x7f63240056a0 con 0x7f633409f5d0 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.664+0000 7f6339d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.664+0000 7f6339d9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.664+0000 7f6339d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.664+0000 7f6339d9d640 1 -- 172.21.15.45:0/2781936818 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6318005010 con 0x7f633409f5d0 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.666+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72049 (unknown 1092875540 0 1010104650) 0x7f6324017980 con 0x7f633409f5d0 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.666+0000 7f632b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:58:54.687 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.666+0000 7f632b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:58:54.694 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:54.693+0000 7f3f645c1ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T13:58:54.724 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.723+0000 7f6339d9d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:58:54.724 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.723+0000 7f6339d9d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:58:54.724 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.723+0000 7f6339d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:58:54.724 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.723+0000 7f6339d9d640 1 -- 172.21.15.45:0/2781936818 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63180052c0 con 0x7f633409f5d0 2024-09-18T13:58:54.725 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.724+0000 7f632b7fe640 1 -- 172.21.15.45:0/2781936818 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5560 (unknown 150273500 0 3599917525) 0x7f63240075c0 con 0x7f633409f5d0 2024-09-18T13:58:54.725 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.724+0000 7f632b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:58:54.725 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.724+0000 7f632b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:58:54.725 INFO:teuthology.misc.health.smithi045.stdout: 2024-09-18T13:58:54.725 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":8,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","created":"2024-09-18T13:58:52.168851+0000","modified":"2024-09-18T13:58:53.737759+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T13:58:53.737759+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"8c3a8909-07ce-4141-98dd-131951f18ae2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a12b3bc3-1e03-47dc-98c1-c8bd199551a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"216b8b62-e41c-4bc5-9395-f2d56d402b1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c750b0e6-9143-489d-9be0-eb085178ff2a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9680908d-dc81-44a0-8603-93711579d42e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b3b63eef-c8fc-495f-960f-7436315119ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T13:58:54.726 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.724+0000 7f6339d9d640 10 monclient: shutdown 2024-09-18T13:58:54.726 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.724+0000 7f6339d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:58:54.726 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.724+0000 7f6339d9d640 1 -- 172.21.15.45:0/2781936818 >> v1:172.21.15.45:6789/0 conn(0x7f633409f5d0 legacy=0x7f6334131510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:58:54.726 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.725+0000 7f6339d9d640 1 -- 172.21.15.45:0/2781936818 shutdown_connections 2024-09-18T13:58:54.726 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.725+0000 7f6339d9d640 1 -- 172.21.15.45:0/2781936818 >> 172.21.15.45:0/2781936818 conn(0x7f63340a5c70 msgr2=0x7f6334135210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:58:54.726 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.725+0000 7f6339d9d640 1 -- 172.21.15.45:0/2781936818 shutdown_connections 2024-09-18T13:58:54.727 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:58:54.725+0000 7f6339d9d640 1 -- 172.21.15.45:0/2781936818 wait complete. 2024-09-18T13:58:54.734 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-18T13:58:54.738 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:54.737+0000 7f3f645c1ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T13:58:54.831 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:54.830+0000 7f3f645c1ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-18T13:58:54.883 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:54.883+0000 7f3f645c1ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T13:58:54.927 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:54.926+0000 7f3f645c1ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T13:58:55.017 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:55.017+0000 7f3f645c1ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T13:58:55.065 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:55.064+0000 7f3f645c1ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T13:58:55.114 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:55.114+0000 7f3f645c1ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T13:58:55.182 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:55.182+0000 7f3f645c1ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T13:58:55.226 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:55.226+0000 7f3f645c1ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T13:58:55.268 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:55.267+0000 7f3f645c1ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T13:58:55.879 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:55.878+0000 7f3f645c1ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T13:58:56.892 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:56.891+0000 7f3f48ac9640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-18T13:58:56.893 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:56.891+0000 7f3f48ac9640 -1 mgr.server handle_report got status from non-daemon mon.g 2024-09-18T13:58:56.893 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:56.892+0000 7f3f48ac9640 -1 mgr.server handle_report got status from non-daemon mon.e 2024-09-18T13:58:56.893 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:56.892+0000 7f3f48ac9640 -1 mgr.server handle_report got status from non-daemon mon.d 2024-09-18T13:58:56.893 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:56.892+0000 7f3f48ac9640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-18T13:58:56.894 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:56.893+0000 7f3f48ac9640 -1 mgr.server handle_report got status from non-daemon mon.f 2024-09-18T13:58:56.894 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:56.894+0000 7f3f48ac9640 -1 mgr.server handle_report got status from non-daemon mon.h 2024-09-18T13:58:56.894 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-18T13:58:56.894+0000 7f3f48ac9640 -1 mgr.server handle_report got status from non-daemon mon.i 2024-09-18T13:58:57.800 INFO:tasks.ceph.osd.3.smithi084.stderr:2024-09-18T13:58:57.800+0000 7f23a9723440 -1 osd.3 0 log_to_monitors true 2024-09-18T13:58:57.817 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-18T13:58:57.817+0000 7fbb05416440 -1 osd.0 0 log_to_monitors true 2024-09-18T13:58:57.830 INFO:tasks.ceph.osd.4.smithi084.stderr:2024-09-18T13:58:57.829+0000 7f1e10d1d440 -1 osd.4 0 log_to_monitors true 2024-09-18T13:58:57.870 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-18T13:58:57.870+0000 7ff26512f440 -1 osd.2 0 log_to_monitors true 2024-09-18T13:58:57.874 INFO:tasks.ceph.osd.5.smithi084.stderr:2024-09-18T13:58:57.874+0000 7fbb14a19440 -1 osd.5 0 log_to_monitors true 2024-09-18T13:58:57.896 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-18T13:58:57.896+0000 7f0474f23440 -1 osd.1 0 log_to_monitors true 2024-09-18T13:58:58.901 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-18T13:58:58.900+0000 7ff25fbf6640 -1 osd.2 0 waiting for initial osdmap 2024-09-18T13:58:58.901 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-18T13:58:58.900+0000 7fbb013f9640 -1 osd.0 0 waiting for initial osdmap 2024-09-18T13:58:58.904 INFO:tasks.ceph.osd.4.smithi084.stderr:2024-09-18T13:58:58.904+0000 7f1e0b7f6640 -1 osd.4 0 waiting for initial osdmap 2024-09-18T13:58:58.904 INFO:tasks.ceph.osd.5.smithi084.stderr:2024-09-18T13:58:58.904+0000 7fbb0f1f6640 -1 osd.5 0 waiting for initial osdmap 2024-09-18T13:58:58.904 INFO:tasks.ceph.osd.3.smithi084.stderr:2024-09-18T13:58:58.904+0000 7f23a41f6640 -1 osd.3 0 waiting for initial osdmap 2024-09-18T13:58:58.908 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-18T13:58:58.907+0000 7ff258bd7640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T13:58:58.908 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-18T13:58:58.907+0000 7fbaf8be8640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T13:58:58.908 INFO:tasks.ceph.osd.5.smithi084.stderr:2024-09-18T13:58:58.907+0000 7fbb081e8640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T13:58:58.909 INFO:tasks.ceph.osd.4.smithi084.stderr:2024-09-18T13:58:58.908+0000 7f1e047e8640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T13:58:58.909 INFO:tasks.ceph.osd.3.smithi084.stderr:2024-09-18T13:58:58.908+0000 7f239d1e8640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T13:58:59.913 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-18T13:58:59.912+0000 7f046f9f6640 -1 osd.1 0 waiting for initial osdmap 2024-09-18T13:58:59.918 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-18T13:58:59.917+0000 7f0468884640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T13:59:01.340 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T13:59:01.440 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.438+0000 7f8b598ef640 10 monclient: get_monmap_and_config 2024-09-18T13:59:01.440 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.438+0000 7f8b598ef640 10 monclient: build_initial_monmap 2024-09-18T13:59:01.440 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.438+0000 7f8b598ef640 10 monclient: monmap: 2024-09-18T13:59:01.440 INFO:teuthology.misc.health.smithi045.stderr:epoch 0 2024-09-18T13:59:01.440 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.440 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:59:01.440107+0000 2024-09-18T13:59:01.440 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:59:01.440107+0000 2024-09-18T13:59:01.440 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.438+0000 7f8b598ef640 1 Processor -- start 2024-09-18T13:59:01.441 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.438+0000 7f8b598ef640 1 -- start start 2024-09-18T13:59:01.442 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.438+0000 7f8b598ef640 10 monclient: init 2024-09-18T13:59:01.449 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b598ef640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:01.449 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b598ef640 10 monclient: _add_conns ranks=[3,2,6,4,7,8,5,1,0] 2024-09-18T13:59:01.449 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b598ef640 10 monclient(hunting): picked mon.noname-g con 0x7f8b5409f5d0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:01.449 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b598ef640 10 monclient(hunting): picked mon.noname-b con 0x7f8b540a2840 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:01.450 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b598ef640 10 monclient(hunting): picked mon.noname-d con 0x7f8b541329e0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:01.450 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b598ef640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b54066630 con 0x7f8b540a2840 2024-09-18T13:59:01.450 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b598ef640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b540a00b0 con 0x7f8b5409f5d0 2024-09-18T13:59:01.450 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b598ef640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b540a1af0 con 0x7f8b541329e0 2024-09-18T13:59:01.450 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b598ef640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:01.450 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b598ef640 10 monclient(hunting): authenticate will time out at 756.778379s 2024-09-18T13:59:01.450 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b537fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f8b541329e0 0x7f8b5413af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43088/0 (socket says 172.21.15.45:43088) 2024-09-18T13:59:01.450 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b537fe640 1 -- 172.21.15.45:0/1441279618 learned_addr learned my addr 172.21.15.45:0/1441279618 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:01.450 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b4a7fc640 1 --1- 172.21.15.45:0/1441279618 >> v1:172.21.15.45:6790/0 conn(0x7f8b540a2840 0x7f8b54137840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:50252/0 (socket says 172.21.15.45:50252) 2024-09-18T13:59:01.450 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422421850 0 0) 0x7f8b54066630 con 0x7f8b540a2840 2024-09-18T13:59:01.450 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b527fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:01.450 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b527fc640 10 monclient(hunting): my global_id is 4223 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b400034e0 con 0x7f8b540a2840 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594405342 0 0) 0x7f8b540a1af0 con 0x7f8b541329e0 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b527fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b527fc640 10 monclient(hunting): my global_id is 4227 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b54066630 con 0x7f8b541329e0 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.439+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644578925 0 0) 0x7f8b540a00b0 con 0x7f8b5409f5d0 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient(hunting): my global_id is 4251 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b540a1af0 con 0x7f8b5409f5d0 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1926872375 0 0) 0x7f8b400034e0 con 0x7f8b540a2840 2024-09-18T13:59:01.451 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 >> v1:172.21.15.45:6792/0 conn(0x7f8b541329e0 legacy=0x7f8b5413af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 >> v1:172.21.15.84:6790/0 conn(0x7f8b5409f5d0 legacy=0x7f8b5409f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 1 monclient: found mon.noname-b 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5409d800 con 0x7f8b540a2840 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b3c001830 con 0x7f8b540a2840 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b598ef640 5 monclient: authenticate success, global_id 4223 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:01.452 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient: dump: 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:epoch 1 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:01.453 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:01.454 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:01.454 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:01.454 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:59:01.454 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b598ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:01.454 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8b3c001a10 con 0x7f8b540a2840 2024-09-18T13:59:01.454 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:01.454 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b598ef640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:01.454 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 1 -- 172.21.15.45:0/1441279618 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b3c004ae0 con 0x7f8b540a2840 2024-09-18T13:59:01.454 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b598ef640 4 set_mon_vals no callback set 2024-09-18T13:59:01.454 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b527fc640 10 monclient: dump: 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:epoch 1 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:01.455 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:01.456 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:01.456 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:01.456 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:01.456 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:59:01.456 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b598ef640 10 monclient: shutdown 2024-09-18T13:59:01.456 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b598ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:01.456 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b598ef640 1 -- 172.21.15.45:0/1441279618 >> v1:172.21.15.45:6790/0 conn(0x7f8b540a2840 legacy=0x7f8b54137840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.456 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b598ef640 1 -- 172.21.15.45:0/1441279618 shutdown_connections 2024-09-18T13:59:01.456 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b598ef640 1 -- 172.21.15.45:0/1441279618 >> 172.21.15.45:0/1441279618 conn(0x7f8b540a5c70 msgr2=0x7f8b540a6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:01.456 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b598ef640 1 -- 172.21.15.45:0/1441279618 shutdown_connections 2024-09-18T13:59:01.456 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.440+0000 7f8b598ef640 1 -- 172.21.15.45:0/1441279618 wait complete. 2024-09-18T13:59:01.456 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.441+0000 7f8b598ef640 10 monclient: build_initial_monmap 2024-09-18T13:59:01.457 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.441+0000 7f8b598ef640 10 monclient: monmap: 2024-09-18T13:59:01.457 INFO:teuthology.misc.health.smithi045.stderr:epoch 0 2024-09-18T13:59:01.457 INFO:teuthology.misc.health.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:01.457 INFO:teuthology.misc.health.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:01.457 INFO:teuthology.misc.health.smithi045.stderr:created 0.000000 2024-09-18T13:59:01.457 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:01.457 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:01.457 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:01.458 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:01.458 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:01.458 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:01.458 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:01.458 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:01.458 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:01.458 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:01.458 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:01.458 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:59:01.458 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.441+0000 7f8b598ef640 1 Processor -- start 2024-09-18T13:59:01.458 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b598ef640 1 -- start start 2024-09-18T13:59:01.458 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b598ef640 10 monclient: init 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b598ef640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b598ef640 10 monclient: _add_conns ranks=[4,6,8,0,7,1,2,3,5] 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b598ef640 10 monclient(hunting): picked mon.noname-c con 0x7f8b5409f5d0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b598ef640 10 monclient(hunting): picked mon.noname-d con 0x7f8b540a2840 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b598ef640 10 monclient(hunting): picked mon.noname-e con 0x7f8b541329e0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b598ef640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b540a23c0 con 0x7f8b5409f5d0 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b598ef640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b540a2020 con 0x7f8b540a2840 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b598ef640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5409b8e0 con 0x7f8b541329e0 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b598ef640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b52ffd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f8b5409f5d0 0x7f8b54131560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36274/0 (socket says 172.21.15.45:36274) 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b52ffd640 1 -- 172.21.15.45:0/3432521719 learned_addr learned my addr 172.21.15.45:0/3432521719 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b598ef640 10 monclient(hunting): authenticate will time out at 756.781731s 2024-09-18T13:59:01.459 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529032087 0 0) 0x7f8b540a2020 con 0x7f8b540a2840 2024-09-18T13:59:01.460 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:01.460 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:01.460 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b4b7fe640 10 monclient(hunting): my global_id is 4236 2024-09-18T13:59:01.460 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.442+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b340034e0 con 0x7f8b540a2840 2024-09-18T13:59:01.460 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653342698 0 0) 0x7f8b5409b8e0 con 0x7f8b541329e0 2024-09-18T13:59:01.460 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:01.460 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:01.460 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): my global_id is 4202 2024-09-18T13:59:01.460 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b540a2020 con 0x7f8b541329e0 2024-09-18T13:59:01.460 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402745919 0 0) 0x7f8b540a23c0 con 0x7f8b5409f5d0 2024-09-18T13:59:01.460 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:01.460 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:01.461 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): my global_id is 4216 2024-09-18T13:59:01.461 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b5409b8e0 con 0x7f8b5409f5d0 2024-09-18T13:59:01.461 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2821022890 0 0) 0x7f8b340034e0 con 0x7f8b540a2840 2024-09-18T13:59:01.461 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b540a23c0 con 0x7f8b540a2840 2024-09-18T13:59:01.461 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b3c004b60 con 0x7f8b540a2840 2024-09-18T13:59:01.461 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:01.461 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T13:59:01.461 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): dump: 2024-09-18T13:59:01.461 INFO:teuthology.misc.health.smithi045.stderr:epoch 1 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:01.462 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1834396225 0 0) 0x7f8b540a2020 con 0x7f8b541329e0 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b340034e0 con 0x7f8b541329e0 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b4c001d00 con 0x7f8b541329e0 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): dump: 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr:epoch 1 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.463 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 892744108 0 0) 0x7f8b5409b8e0 con 0x7f8b5409f5d0 2024-09-18T13:59:01.464 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b540a2020 con 0x7f8b5409f5d0 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b44002ad0 con 0x7f8b5409f5d0 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient(hunting): dump: 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:epoch 1 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:01.465 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1011300949 0 0) 0x7f8b540a23c0 con 0x7f8b540a2840 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 >> v1:172.21.15.45:6793/0 conn(0x7f8b541329e0 legacy=0x7f8b54149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 >> v1:172.21.15.45:6791/0 conn(0x7f8b5409f5d0 legacy=0x7f8b54131560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient: _finish_hunting 0 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 monclient: found mon.d 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5411e680 con 0x7f8b540a2840 2024-09-18T13:59:01.466 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient: _finish_auth 0 2024-09-18T13:59:01.467 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:01.467 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b598ef640 5 monclient: authenticate success, global_id 4236 2024-09-18T13:59:01.467 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b598ef640 10 monclient: _renew_subs 2024-09-18T13:59:01.467 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b598ef640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:01.467 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b598ef640 1 -- 172.21.15.45:0/3432521719 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5411d720 con 0x7f8b540a2840 2024-09-18T13:59:01.467 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b598ef640 10 monclient: _renew_subs 2024-09-18T13:59:01.467 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b598ef640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:01.467 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b598ef640 1 -- 172.21.15.45:0/3432521719 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b54131e10 con 0x7f8b540a2840 2024-09-18T13:59:01.467 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8b3c001170 con 0x7f8b540a2840 2024-09-18T13:59:01.467 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:01.468 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b3c0050a0 con 0x7f8b540a2840 2024-09-18T13:59:01.468 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:01.468 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:01.468 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b51ffb640 4 set_mon_vals no callback set 2024-09-18T13:59:01.468 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.443+0000 7f8b4b7fe640 10 monclient: dump: 2024-09-18T13:59:01.468 INFO:teuthology.misc.health.smithi045.stderr:epoch 1 2024-09-18T13:59:01.468 INFO:teuthology.misc.health.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.468 INFO:teuthology.misc.health.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.468 INFO:teuthology.misc.health.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.468 INFO:teuthology.misc.health.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:01.468 INFO:teuthology.misc.health.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr: 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.444+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 4) v1 ==== 44590+0+0 (unknown 2422757912 0 0) 0x7f8b3c010040 con 0x7f8b540a2840 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.444+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(12..12 src has 1..12) v4 ==== 2763+0+0 (unknown 3797004601 0 0) 0x7f8b3c0466f0 con 0x7f8b540a2840 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.444+0000 7f8b598ef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:01.469 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.444+0000 7f8b598ef640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:01.470 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.444+0000 7f8b598ef640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:01.470 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.444+0000 7f8b598ef640 1 -- 172.21.15.45:0/3432521719 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b38005010 con 0x7f8b540a2840 2024-09-18T13:59:01.470 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.445+0000 7f8b52ffd640 10 monclient: get_auth_request con 0x7f8b3403a9e0 auth_method 0 2024-09-18T13:59:01.470 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.449+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f8b3c015120 con 0x7f8b540a2840 2024-09-18T13:59:01.470 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.449+0000 7f8b4b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:01.470 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.449+0000 7f8b4b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:01.614 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.613+0000 7f8b598ef640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:59:01.614 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.613+0000 7f8b598ef640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:59:01.614 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.613+0000 7f8b598ef640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:01.615 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.613+0000 7f8b598ef640 1 -- 172.21.15.45:0/3432521719 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b380052c0 con 0x7f8b540a2840 2024-09-18T13:59:01.615 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.613+0000 7f8b4b7fe640 1 -- 172.21.15.45:0/3432521719 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+6863 (unknown 4250787923 0 2962431392) 0x7f8b3c045930 con 0x7f8b540a2840 2024-09-18T13:59:01.615 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.613+0000 7f8b4b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:59:01.615 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.613+0000 7f8b4b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:01.616 INFO:teuthology.misc.health.smithi045.stdout: 2024-09-18T13:59:01.616 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":12,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","created":"2024-09-18T13:58:52.168851+0000","modified":"2024-09-18T13:59:00.910314+0000","last_up_change":"2024-09-18T13:59:00.910314+0000","last_in_change":"2024-09-18T13:58:53.737759+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"8c3a8909-07ce-4141-98dd-131951f18ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6804","nonce":32276}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6805","nonce":32276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6807","nonce":32276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6806","nonce":32276}]},"public_addr":"172.21.15.45:6804/32276","cluster_addr":"172.21.15.45:6805/32276","heartbeat_back_addr":"172.21.15.45:6807/32276","heartbeat_front_addr":"172.21.15.45:6806/32276","state":["exists","up"]},{"osd":1,"uuid":"a12b3bc3-1e03-47dc-98c1-c8bd199551a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6800","nonce":32277}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6801","nonce":32277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6803","nonce":32277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6802","nonce":32277}]},"public_addr":"172.21.15.45:6800/32277","cluster_addr":"172.21.15.45:6801/32277","heartbeat_back_addr":"172.21.15.45:6803/32277","heartbeat_front_addr":"172.21.15.45:6802/32277","state":["exists","up"]},{"osd":2,"uuid":"216b8b62-e41c-4bc5-9395-f2d56d402b1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6808","nonce":32278}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6809","nonce":32278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6811","nonce":32278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6810","nonce":32278}]},"public_addr":"172.21.15.45:6808/32278","cluster_addr":"172.21.15.45:6809/32278","heartbeat_back_addr":"172.21.15.45:6811/32278","heartbeat_front_addr":"172.21.15.45:6810/32278","state":["exists","up"]},{"osd":3,"uuid":"c750b0e6-9143-489d-9be0-eb085178ff2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6802","nonce":32024}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6804","nonce":32024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6807","nonce":32024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6806","nonce":32024}]},"public_addr":"172.21.15.84:6802/32024","cluster_addr":"172.21.15.84:6804/32024","heartbeat_back_addr":"172.21.15.84:6807/32024","heartbeat_front_addr":"172.21.15.84:6806/32024","state":["exists","up"]},{"osd":4,"uuid":"9680908d-dc81-44a0-8603-93711579d42e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6800","nonce":32025}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6801","nonce":32025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6805","nonce":32025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6803","nonce":32025}]},"public_addr":"172.21.15.84:6800/32025","cluster_addr":"172.21.15.84:6801/32025","heartbeat_back_addr":"172.21.15.84:6805/32025","heartbeat_front_addr":"172.21.15.84:6803/32025","state":["exists","up"]},{"osd":5,"uuid":"b3b63eef-c8fc-495f-960f-7436315119ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6808","nonce":32048}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6809","nonce":32048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6811","nonce":32048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6810","nonce":32048}]},"public_addr":"172.21.15.84:6808/32048","cluster_addr":"172.21.15.84:6809/32048","heartbeat_back_addr":"172.21.15.84:6811/32048","heartbeat_front_addr":"172.21.15.84:6810/32048","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.823226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.904155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.801736+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.835588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.910413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T13:59:01.617 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.615+0000 7f8b598ef640 1 -- 172.21.15.45:0/3432521719 >> v1:172.21.15.84:6812/31478 conn(0x7f8b3403a9e0 legacy=0x7f8b3403ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.617 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.615+0000 7f8b598ef640 10 monclient: shutdown 2024-09-18T13:59:01.617 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.615+0000 7f8b598ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:01.617 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.615+0000 7f8b598ef640 1 -- 172.21.15.45:0/3432521719 >> v1:172.21.15.45:6792/0 conn(0x7f8b540a2840 legacy=0x7f8b54146750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.617 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.615+0000 7f8b598ef640 1 -- 172.21.15.45:0/3432521719 shutdown_connections 2024-09-18T13:59:01.617 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.615+0000 7f8b598ef640 1 -- 172.21.15.45:0/3432521719 >> 172.21.15.45:0/3432521719 conn(0x7f8b540a5c70 msgr2=0x7f8b5409e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:01.617 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.615+0000 7f8b598ef640 1 -- 172.21.15.45:0/3432521719 shutdown_connections 2024-09-18T13:59:01.617 INFO:teuthology.misc.health.smithi045.stderr:2024-09-18T13:59:01.615+0000 7f8b598ef640 1 -- 172.21.15.45:0/3432521719 wait complete. 2024-09-18T13:59:01.626 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-18T13:59:01.626 INFO:tasks.ceph:Creating RBD pool 2024-09-18T13:59:01.626 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-18T13:59:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.719+0000 7f726279d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.719+0000 7f726279d640 10 monclient: build_initial_monmap 2024-09-18T13:59:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.719+0000 7f726279d640 10 monclient: monmap: 2024-09-18T13:59:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:01.721696+0000 2024-09-18T13:59:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:01.721696+0000 2024-09-18T13:59:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:01.722 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.720+0000 7f726279d640 1 Processor -- start 2024-09-18T13:59:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.720+0000 7f726279d640 1 -- start start 2024-09-18T13:59:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.720+0000 7f726279d640 10 monclient: init 2024-09-18T13:59:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.720+0000 7f726279d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.720+0000 7f726279d640 10 monclient: _add_conns ranks=[8,1,5,7,6,4,2,3,0] 2024-09-18T13:59:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.720+0000 7f726279d640 10 monclient(hunting): picked mon.noname-e con 0x7f725c09f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.720+0000 7f726279d640 10 monclient(hunting): picked mon.noname-f con 0x7f725c09fba0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f726279d640 10 monclient(hunting): picked mon.noname-h con 0x7f725c09ff70 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f726279d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c05ed70 con 0x7f725c09fba0 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f726279d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c09dfc0 con 0x7f725c09ff70 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f726279d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c09dce0 con 0x7f725c09f7d0 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f726279d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f726279d640 10 monclient(hunting): authenticate will time out at 757.060051s 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725bfff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f725c09f7d0 0x7f725c132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:56664/0 (socket says 172.21.15.45:56664) 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725bfff640 1 -- 172.21.15.45:0/1585510866 learned_addr learned my addr 172.21.15.45:0/1585510866 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120022306 0 0) 0x7f725c09dce0 con 0x7f725c09f7d0 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient(hunting): my global_id is 4211 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72480034e0 con 0x7f725c09f7d0 2024-09-18T13:59:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677878699 0 0) 0x7f725c05ed70 con 0x7f725c09fba0 2024-09-18T13:59:01.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:01.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:01.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient(hunting): my global_id is 4204 2024-09-18T13:59:01.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f725c09dce0 con 0x7f725c09fba0 2024-09-18T13:59:01.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003126991 0 0) 0x7f725c09dfc0 con 0x7f725c09ff70 2024-09-18T13:59:01.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:01.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:01.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient(hunting): my global_id is 4199 2024-09-18T13:59:01.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f725c05ed70 con 0x7f725c09ff70 2024-09-18T13:59:01.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1634115716 0 0) 0x7f72480034e0 con 0x7f725c09f7d0 2024-09-18T13:59:01.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 >> v1:172.21.15.84:6791/0 conn(0x7f725c09ff70 legacy=0x7f725c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 >> v1:172.21.15.84:6789/0 conn(0x7f725c09fba0 legacy=0x7f725c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 monclient: found mon.noname-e 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c0a1c00 con 0x7f725c09f7d0 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient: _finish_auth 0 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f726279d640 5 monclient: authenticate success, global_id 4211 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f724c001830 con 0x7f725c09f7d0 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient: dump: 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f726279d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f724c001a10 con 0x7f725c09f7d0 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 1 -- 172.21.15.45:0/1585510866 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f724c004ba0 con 0x7f725c09f7d0 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f726279d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f726279d640 4 set_mon_vals no callback set 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.721+0000 7f725affd640 10 monclient: dump: 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.722+0000 7f726279d640 10 monclient: shutdown 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.722+0000 7f726279d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.722+0000 7f726279d640 1 -- 172.21.15.45:0/1585510866 >> v1:172.21.15.45:6793/0 conn(0x7f725c09f7d0 legacy=0x7f725c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.722+0000 7f726279d640 1 -- 172.21.15.45:0/1585510866 shutdown_connections 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.722+0000 7f726279d640 1 -- 172.21.15.45:0/1585510866 >> 172.21.15.45:0/1585510866 conn(0x7f725c0630b0 msgr2=0x7f725c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.722+0000 7f726279d640 1 -- 172.21.15.45:0/1585510866 shutdown_connections 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.722+0000 7f726279d640 1 -- 172.21.15.45:0/1585510866 wait complete. 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.722+0000 7f726279d640 10 monclient: build_initial_monmap 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.722+0000 7f726279d640 10 monclient: monmap: 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:01.738 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 1 Processor -- start 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 1 -- start start 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 10 monclient: init 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 10 monclient: _add_conns ranks=[5,6,0,7,4,8,2,1,3] 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 10 monclient(hunting): picked mon.noname-h con 0x7f725c09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 10 monclient(hunting): picked mon.noname-d con 0x7f725c09fba0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 10 monclient(hunting): picked mon.noname-a con 0x7f725c09ff70 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c09d310 con 0x7f725c09ff70 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c0a2560 con 0x7f725c09f7d0 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c11d510 con 0x7f725c09fba0 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.723+0000 7f726279d640 10 monclient(hunting): authenticate will time out at 757.062950s 2024-09-18T13:59:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f725b7fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f725c09fba0 0x7f725c146d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43098/0 (socket says 172.21.15.45:43098) 2024-09-18T13:59:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f725b7fe640 1 -- 172.21.15.45:0/1651828531 learned_addr learned my addr 172.21.15.45:0/1651828531 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598201465 0 0) 0x7f725c11d510 con 0x7f725c09fba0 2024-09-18T13:59:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): my global_id is 4245 2024-09-18T13:59:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f723c0034e0 con 0x7f725c09fba0 2024-09-18T13:59:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266463900 0 0) 0x7f725c09d310 con 0x7f725c09ff70 2024-09-18T13:59:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): my global_id is 4221 2024-09-18T13:59:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f725c11d510 con 0x7f725c09ff70 2024-09-18T13:59:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344025818 0 0) 0x7f725c0a2560 con 0x7f725c09f7d0 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): my global_id is 4208 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f725c09d310 con 0x7f725c09f7d0 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4282727720 0 0) 0x7f723c0034e0 con 0x7f725c09fba0 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f725c0a2560 con 0x7f725c09fba0 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7244001be0 con 0x7f725c09fba0 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): dump: 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.741 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:01.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2865040365 0 0) 0x7f725c09d310 con 0x7f725c09f7d0 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f723c0034e0 con 0x7f725c09f7d0 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f724c004ae0 con 0x7f725c09f7d0 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): dump: 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:01.743 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1197636030 0 0) 0x7f725c11d510 con 0x7f725c09ff70 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f725c09d310 con 0x7f725c09ff70 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7254001c40 con 0x7f725c09ff70 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:01.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient(hunting): dump: 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:01.745 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:01.746 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:01.746 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:01.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1989224616 0 0) 0x7f725c0a2560 con 0x7f725c09fba0 2024-09-18T13:59:01.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 >> v1:172.21.15.84:6791/0 conn(0x7f725c09f7d0 legacy=0x7f725c1316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 >> v1:172.21.15.45:6789/0 conn(0x7f725c09ff70 legacy=0x7f725c14a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient: _finish_hunting 0 2024-09-18T13:59:01.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 monclient: found mon.d 2024-09-18T13:59:01.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:01.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:01.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c1428a0 con 0x7f725c09fba0 2024-09-18T13:59:01.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 10 monclient: _finish_auth 0 2024-09-18T13:59:01.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f7253fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:01.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f726279d640 5 monclient: authenticate success, global_id 4245 2024-09-18T13:59:01.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f726279d640 10 monclient: _renew_subs 2024-09-18T13:59:01.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f726279d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f726279d640 1 -- 172.21.15.45:0/1651828531 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c11d920 con 0x7f725c09fba0 2024-09-18T13:59:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f726279d640 10 monclient: _renew_subs 2024-09-18T13:59:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f726279d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.724+0000 7f726279d640 1 -- 172.21.15.45:0/1651828531 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c136ee0 con 0x7f725c09fba0 2024-09-18T13:59:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.725+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7244002070 con 0x7f725c09fba0 2024-09-18T13:59:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.725+0000 7f7253fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.725+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f72440025e0 con 0x7f725c09fba0 2024-09-18T13:59:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.725+0000 7f7253fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.725+0000 7f7253fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.725+0000 7f725a7fc640 4 set_mon_vals no callback set 2024-09-18T13:59:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.725+0000 7f7253fff640 10 monclient: dump: 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:01.751 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.725+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 4) v1 ==== 44590+0+0 (unknown 2422757912 0 0) 0x7f72440027c0 con 0x7f725c09fba0 2024-09-18T13:59:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.725+0000 7f726279d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.725+0000 7f726279d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.725+0000 7f726279d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.725+0000 7f726279d640 1 -- 172.21.15.45:0/1651828531 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7240005010 con 0x7f725c09fba0 2024-09-18T13:59:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.726+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(12..12 src has 1..12) v4 ==== 2763+0+0 (unknown 3797004601 0 0) 0x7f7244046780 con 0x7f725c09fba0 2024-09-18T13:59:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.726+0000 7f725bfff640 10 monclient: get_auth_request con 0x7f723c03a8d0 auth_method 0 2024-09-18T13:59:01.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.730+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f7244011570 con 0x7f725c09fba0 2024-09-18T13:59:01.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.730+0000 7f7253fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:01.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.730+0000 7f7253fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.893+0000 7f726279d640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-18T13:59:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.893+0000 7f726279d640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-18T13:59:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.893+0000 7f726279d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.893+0000 7f726279d640 1 -- 172.21.15.45:0/1651828531 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f7240005840 con 0x7f725c09fba0 2024-09-18T13:59:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.961+0000 7f7253fff640 1 -- 172.21.15.45:0/1651828531 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (unknown 2984082091 0 0) 0x7f7244014e50 con 0x7f725c09fba0 2024-09-18T13:59:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.961+0000 7f7253fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-18T13:59:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.961+0000 7f7253fff640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2024-09-18T13:59:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'rbd' created 2024-09-18T13:59:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.963+0000 7f726279d640 1 -- 172.21.15.45:0/1651828531 >> v1:172.21.15.84:6812/31478 conn(0x7f723c03a8d0 legacy=0x7f723c03cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.963+0000 7f726279d640 10 monclient: shutdown 2024-09-18T13:59:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.963+0000 7f726279d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.963+0000 7f726279d640 1 -- 172.21.15.45:0/1651828531 >> v1:172.21.15.45:6792/0 conn(0x7f725c09fba0 legacy=0x7f725c146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.963+0000 7f726279d640 1 -- 172.21.15.45:0/1651828531 shutdown_connections 2024-09-18T13:59:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.963+0000 7f726279d640 1 -- 172.21.15.45:0/1651828531 >> 172.21.15.45:0/1651828531 conn(0x7f725c0630b0 msgr2=0x7f725c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.963+0000 7f726279d640 1 -- 172.21.15.45:0/1651828531 shutdown_connections 2024-09-18T13:59:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:01.963+0000 7f726279d640 1 -- 172.21.15.45:0/1651828531 wait complete. 2024-09-18T13:59:01.975 DEBUG:teuthology.orchestra.run.smithi045:> rbd --cluster ceph pool init rbd 2024-09-18T13:59:02.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.117+0000 7f632a83eb80 10 monclient: get_monmap_and_config 2024-09-18T13:59:02.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.117+0000 7f632a83eb80 10 monclient: build_initial_monmap 2024-09-18T13:59:02.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.117+0000 7f632a83eb80 10 monclient: monmap: 2024-09-18T13:59:02.119 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:02.119 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:02.119824+0000 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:02.119824+0000 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:02.120 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.118+0000 7f632a83eb80 1 Processor -- start 2024-09-18T13:59:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.118+0000 7f632a83eb80 1 -- start start 2024-09-18T13:59:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.118+0000 7f632a83eb80 10 monclient: init 2024-09-18T13:59:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 10 monclient: _reopen_session rank -1 2024-09-18T13:59:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 10 monclient: _add_conns ranks=[6,3,2,8,4,7,1,5,0] 2024-09-18T13:59:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 10 monclient(hunting): picked mon.noname-d con 0x559ee5564fc0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 10 monclient(hunting): picked mon.noname-g con 0x559ee5565390 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 10 monclient(hunting): picked mon.noname-b con 0x559ee5565760 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x559ee54870e0 con 0x559ee5565760 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x559ee5460270 con 0x559ee5565390 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x559ee546d0e0 con 0x559ee5564fc0 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 10 monclient(hunting): _renew_subs 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 10 monclient(hunting): authenticate will time out at 757.458149s 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6323fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x559ee5564fc0 0x559ee55fada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43112/0 (socket says 172.21.15.45:43112) 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6323fff640 1 -- 172.21.15.45:0/1496127237 learned_addr learned my addr 172.21.15.45:0/1496127237 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889368149 0 0) 0x559ee54870e0 con 0x559ee5565760 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient(hunting): my global_id is 4232 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f630c001e10 con 0x559ee5565760 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407170288 0 0) 0x559ee546d0e0 con 0x559ee5564fc0 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient(hunting): my global_id is 4254 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x559ee54870e0 con 0x559ee5564fc0 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851232268 0 0) 0x559ee5460270 con 0x559ee5565390 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient(hunting): my global_id is 4260 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x559ee546d0e0 con 0x559ee5565390 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1509352007 0 0) 0x559ee54870e0 con 0x559ee5564fc0 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 >> v1:172.21.15.84:6790/0 conn(0x559ee5565390 legacy=0x559ee55f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 >> v1:172.21.15.45:6790/0 conn(0x559ee5565760 legacy=0x559ee55f9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 1 monclient: found mon.noname-d 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ee5460600 con 0x559ee5564fc0 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6314001810 con 0x559ee5564fc0 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 5 monclient: authenticate success, global_id 4254 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T13:59:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f6322ffd640 10 monclient: dump: 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.119+0000 7f632a83eb80 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6314001970 con 0x559ee5564fc0 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f6322ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f6322ffd640 1 -- 172.21.15.45:0/1496127237 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f63140034d0 con 0x559ee5564fc0 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f632a83eb80 10 monclient: get_monmap_and_config success 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f632a83eb80 4 set_mon_vals no callback set 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f6322ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f6322ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f6322ffd640 10 monclient: dump: 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f632a83eb80 10 monclient: shutdown 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f632a83eb80 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f632a83eb80 1 -- 172.21.15.45:0/1496127237 >> v1:172.21.15.45:6792/0 conn(0x559ee5564fc0 legacy=0x559ee55fada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f632a83eb80 1 -- 172.21.15.45:0/1496127237 shutdown_connections 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f632a83eb80 1 -- 172.21.15.45:0/1496127237 >> 172.21.15.45:0/1496127237 conn(0x559ee556b800 msgr2=0x559ee556bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f632a83eb80 1 -- 172.21.15.45:0/1496127237 shutdown_connections 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.120+0000 7f632a83eb80 1 -- 172.21.15.45:0/1496127237 wait complete. 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 10 monclient: build_initial_monmap 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 10 monclient: monmap: 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 1 Processor -- start 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 1 -- start start 2024-09-18T13:59:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 10 monclient: init 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 10 monclient: _reopen_session rank -1 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 10 monclient: _add_conns ranks=[2,0,1,7,4,5,8,6,3] 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 10 monclient(hunting): picked mon.noname-b con 0x559ee5564fc0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 10 monclient(hunting): picked mon.noname-a con 0x559ee5565390 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 10 monclient(hunting): picked mon.noname-f con 0x559ee5565760 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x559ee5460060 con 0x559ee5565390 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x559ee5548740 con 0x559ee5565760 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x559ee555ba50 con 0x559ee5564fc0 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 10 monclient(hunting): _renew_subs 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f632a83eb80 10 monclient(hunting): authenticate will time out at 757.460925s 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.121+0000 7f6323fff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x559ee5564fc0 0x559ee56086b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:50264/0 (socket says 172.21.15.45:50264) 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6323fff640 1 -- 172.21.15.45:0/1484628189 learned_addr learned my addr 172.21.15.45:0/1484628189 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599014664 0 0) 0x559ee555ba50 con 0x559ee5564fc0 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): my global_id is 4241 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6308001e10 con 0x559ee5564fc0 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549902815 0 0) 0x559ee5460060 con 0x559ee5565390 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): my global_id is 4230 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x559ee555ba50 con 0x559ee5565390 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3903098408 0 0) 0x559ee5548740 con 0x559ee5565760 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): my global_id is 4213 2024-09-18T13:59:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x559ee5460060 con 0x559ee5565760 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2076710065 0 0) 0x7f6308001e10 con 0x559ee5564fc0 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x559ee5548740 con 0x559ee5564fc0 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f63140030d0 con 0x559ee5564fc0 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): dump: 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 277295338 0 0) 0x559ee555ba50 con 0x559ee5565390 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6308001e10 con 0x559ee5565390 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6318001a60 con 0x559ee5565390 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): dump: 2024-09-18T13:59:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3585680132 0 0) 0x559ee5460060 con 0x559ee5565760 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x559ee555ba50 con 0x559ee5565760 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f631c001b60 con 0x559ee5565760 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient(hunting): dump: 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4004431275 0 0) 0x7f6308001e10 con 0x559ee5565390 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 >> v1:172.21.15.45:6790/0 conn(0x559ee5564fc0 legacy=0x559ee56086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 >> v1:172.21.15.84:6789/0 conn(0x559ee5565760 legacy=0x559ee5610a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient: _finish_hunting 0 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 monclient: found mon.a 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ee5548480 con 0x559ee5565390 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 10 monclient: _finish_auth 0 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f6313fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f632a83eb80 5 monclient: authenticate success, global_id 4230 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f632a83eb80 10 monclient: _renew_subs 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f632a83eb80 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x559ee55f61c0 con 0x559ee5565390 2024-09-18T13:59:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.122+0000 7f632a83eb80 10 monclient: _renew_subs 2024-09-18T13:59:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f632a83eb80 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x559ee55f6d30 con 0x559ee5565390 2024-09-18T13:59:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6318002c70 con 0x559ee5565390 2024-09-18T13:59:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f6313fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f63180022b0 con 0x559ee5565390 2024-09-18T13:59:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f6313fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f6313fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f6313fff640 10 monclient: dump: 2024-09-18T13:59:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f63227fc640 4 set_mon_vals no callback set 2024-09-18T13:59:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 4) v1 ==== 44590+0+0 (unknown 2422757912 0 0) 0x7f6318002410 con 0x559ee5565390 2024-09-18T13:59:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(13..13 src has 1..13) v4 ==== 3480+0+0 (unknown 3593444948 0 0) 0x7f6318043d60 con 0x559ee5565390 2024-09-18T13:59:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f632a83eb80 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-18T13:59:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f632a83eb80 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-18T13:59:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f632a83eb80 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.123+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) v1 -- 0x559ee5612140 con 0x559ee5565390 2024-09-18T13:59:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.124+0000 7f6323fff640 10 monclient: get_auth_request con 0x7f6308039320 auth_method 0 2024-09-18T13:59:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.963+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 141+0+0 (unknown 2078359144 0 0) 0x7f63180127d0 con 0x559ee5565390 2024-09-18T13:59:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.963+0000 7f6313fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-18T13:59:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.963+0000 7f6313fff640 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2024-09-18T13:59:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.963+0000 7f632a83eb80 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.963+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x559ee547e630 con 0x559ee5565390 2024-09-18T13:59:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.964+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (unknown 1566986600 0 0) 0x7f6318012570 con 0x559ee5565390 2024-09-18T13:59:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.964+0000 7f6313fff640 10 monclient: handle_get_version_reply finishing 1 version 14 2024-09-18T13:59:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.964+0000 7f63227fc640 10 monclient: _renew_subs 2024-09-18T13:59:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.964+0000 7f63227fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:02.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.964+0000 7f63227fc640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=14}) v3 -- 0x7f6300002a70 con 0x559ee5565390 2024-09-18T13:59:02.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.964+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 11 ==== osd_map(14..14 src has 1..14) v4 ==== 1020+0+0 (unknown 4100707964 0 0) 0x7f6318013100 con 0x559ee5565390 2024-09-18T13:59:02.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.964+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6800/32277 -- osd_op(unknown.0.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14) v8 -- 0x559ee56167d0 con 0x559ee5612cd0 2024-09-18T13:59:02.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.964+0000 7f6328b4e640 10 monclient: get_auth_request con 0x559ee5612cd0 auth_method 0 2024-09-18T13:59:02.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.981+0000 7f6328b4e640 1 -- 172.21.15.45:0/1484628189 <== osd.1 v1:172.21.15.45:6800/32277 1 ==== osd_op_reply(1 rbd_trash [create] v14'1 uv1 ondisk = 0) v8 ==== 153+0+0 (unknown 1318299302 0 0) 0x7f631c003310 con 0x559ee5612cd0 2024-09-18T13:59:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.984+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.84:6800/32025 -- osd_op(unknown.0.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e14) v8 -- 0x559ee5667a50 con 0x559ee5663e60 2024-09-18T13:59:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:02.984+0000 7f6323fff640 10 monclient: get_auth_request con 0x559ee5663e60 auth_method 0 2024-09-18T13:59:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:03.704+0000 7f6323fff640 1 -- 172.21.15.45:0/1484628189 <== osd.4 v1:172.21.15.84:6800/32025 1 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (unknown 588695289 0 0) 0x7f6314006960 con 0x559ee5663e60 2024-09-18T13:59:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:03.704+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.84:6800/32025 -- osd_op(unknown.0.0:3 1.2 1:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e14) v8 -- 0x559ee56628f0 con 0x559ee5663e60 2024-09-18T13:59:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:03.705+0000 7f6323fff640 1 -- 172.21.15.45:0/1484628189 <== osd.4 v1:172.21.15.84:6800/32025 2 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (unknown 575028469 0 0) 0x7f6314006960 con 0x559ee5663e60 2024-09-18T13:59:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:03.705+0000 7f6321ffb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:03.705+0000 7f6321ffb640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6789/0 -- pool_op(create unmanaged snap pool 1 tid 4 name v0) v4 -- 0x7f6304001ee0 con 0x559ee5565390 2024-09-18T13:59:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:04.027+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 12 ==== pool_op_reply(tid 4 (0) Success v15) v1 ==== 55+0+0 (unknown 1982643896 0 0) 0x7f6318042ce0 con 0x559ee5565390 2024-09-18T13:59:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:04.027+0000 7f6313fff640 10 monclient: _renew_subs 2024-09-18T13:59:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:04.027+0000 7f6313fff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:04.027+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=15}) v3 -- 0x7f6308001e10 con 0x559ee5565390 2024-09-18T13:59:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:04.027+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 13 ==== osd_map(15..15 src has 1..15) v4 ==== 988+0+0 (unknown 1606232554 0 0) 0x7f6318012a00 con 0x559ee5565390 2024-09-18T13:59:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:04.027+0000 7f63227fc640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.84:6800/32025 -- osd_op(unknown.0.0:5 1.2 1:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x7f63000055a0 con 0x559ee5663e60 2024-09-18T13:59:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:04.030+0000 7f6323fff640 1 -- 172.21.15.45:0/1484628189 <== osd.4 v1:172.21.15.84:6800/32025 3 ==== osd_op_reply(5 rbd_info [create,write 0~8] v15'1 uv1 ondisk = 0) v8 ==== 194+0+0 (unknown 453491686 0 0) 0x7f6314006960 con 0x559ee5663e60 2024-09-18T13:59:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:04.030+0000 7f6321ffb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:04.030+0000 7f6321ffb640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6789/0 -- pool_op(delete unmanaged snap pool 1 tid 6 name v15) v4 -- 0x7f6304003380 con 0x559ee5565390 2024-09-18T13:59:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:04.468+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 14 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f63180028a0 con 0x559ee5565390 2024-09-18T13:59:05.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.035+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 15 ==== pool_op_reply(tid 6 (0) Success v16) v1 ==== 43+0+0 (unknown 599251060 0 0) 0x7f6318019b60 con 0x559ee5565390 2024-09-18T13:59:05.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.035+0000 7f6313fff640 10 monclient: _renew_subs 2024-09-18T13:59:05.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.035+0000 7f6313fff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:05.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.035+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=16}) v3 -- 0x559ee5460060 con 0x559ee5565390 2024-09-18T13:59:05.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.035+0000 7f6313fff640 1 -- 172.21.15.45:0/1484628189 <== mon.0 v1:172.21.15.45:6789/0 16 ==== osd_map(16..16 src has 1..16) v4 ==== 646+0+0 (unknown 504086702 0 0) 0x7f6318019b60 con 0x559ee5565390 2024-09-18T13:59:05.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.036+0000 7f63227fc640 1 -- 172.21.15.45:0/1484628189 --> v1:172.21.15.84:6800/32025 -- osd_op(unknown.0.0:7 1.2 1:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) v8 -- 0x7f63000059b0 con 0x559ee5663e60 2024-09-18T13:59:05.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.040+0000 7f6323fff640 1 -- 172.21.15.45:0/1484628189 <== osd.4 v1:172.21.15.84:6800/32025 4 ==== osd_op_reply(7 rbd_info [write 0~19] v16'2 uv2 ondisk = 0) v8 ==== 152+0+0 (unknown 1471445399 0 0) 0x7f6314006960 con 0x559ee5663e60 2024-09-18T13:59:05.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.040+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 >> v1:172.21.15.45:6800/32277 conn(0x559ee5612cd0 legacy=0x559ee56150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.040+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 >> v1:172.21.15.84:6800/32025 conn(0x559ee5663e60 legacy=0x559ee56662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.040+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 >> v1:172.21.15.84:6812/31478 conn(0x7f6308039320 legacy=0x7f630803b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.040+0000 7f6328b4e640 1 -- 172.21.15.45:0/1484628189 reap_dead start 2024-09-18T13:59:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.040+0000 7f632a83eb80 10 monclient: shutdown 2024-09-18T13:59:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.040+0000 7f632a83eb80 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.040+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 >> v1:172.21.15.45:6789/0 conn(0x559ee5565390 legacy=0x559ee555e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.040+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 shutdown_connections 2024-09-18T13:59:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.040+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 >> 172.21.15.45:0/1484628189 conn(0x559ee556b800 msgr2=0x559ee55f8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.041+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 shutdown_connections 2024-09-18T13:59:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.041+0000 7f632a83eb80 1 -- 172.21.15.45:0/1484628189 wait complete. 2024-09-18T13:59:05.045 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-18T13:59:05.046 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-18T13:59:05.046 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-18T13:59:05.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-18T13:59:05.047 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-18T13:59:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.148+0000 7fcc7919d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.148+0000 7fcc7919d640 10 monclient: build_initial_monmap 2024-09-18T13:59:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.148+0000 7fcc7919d640 10 monclient: monmap: 2024-09-18T13:59:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:05.149457+0000 2024-09-18T13:59:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:05.149457+0000 2024-09-18T13:59:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:05.150 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.149+0000 7fcc7919d640 1 Processor -- start 2024-09-18T13:59:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.149+0000 7fcc7919d640 1 -- start start 2024-09-18T13:59:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.149+0000 7fcc7919d640 10 monclient: init 2024-09-18T13:59:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.150+0000 7fcc7919d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.150+0000 7fcc7919d640 10 monclient: _add_conns ranks=[1,0,3,2,4,8,6,7,5] 2024-09-18T13:59:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.150+0000 7fcc7919d640 10 monclient(hunting): picked mon.noname-f con 0x7fcc7406e810 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.150+0000 7fcc7919d640 10 monclient(hunting): picked mon.noname-a con 0x7fcc740a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.150+0000 7fcc7919d640 10 monclient(hunting): picked mon.noname-g con 0x7fcc740a1080 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.150+0000 7fcc7919d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7405ed70 con 0x7fcc740a0cb0 2024-09-18T13:59:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.150+0000 7fcc7919d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc74097620 con 0x7fcc7406e810 2024-09-18T13:59:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.150+0000 7fcc7919d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc74097340 con 0x7fcc740a1080 2024-09-18T13:59:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.150+0000 7fcc7919d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.150+0000 7fcc7919d640 10 monclient(hunting): authenticate will time out at 760.488564s 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.150+0000 7fcc6bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcc740a0cb0 0x7fcc74137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49338/0 (socket says 172.21.15.45:49338) 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.150+0000 7fcc6bfff640 1 -- 172.21.15.45:0/3721017152 learned_addr learned my addr 172.21.15.45:0/3721017152 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803959280 0 0) 0x7fcc74097620 con 0x7fcc7406e810 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient(hunting): my global_id is 4240 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc600034e0 con 0x7fcc7406e810 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293129549 0 0) 0x7fcc7405ed70 con 0x7fcc740a0cb0 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient(hunting): my global_id is 4257 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc74097620 con 0x7fcc740a0cb0 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540878218 0 0) 0x7fcc74097340 con 0x7fcc740a1080 2024-09-18T13:59:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient(hunting): my global_id is 4287 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc7405ed70 con 0x7fcc740a1080 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2315132608 0 0) 0x7fcc600034e0 con 0x7fcc7406e810 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 >> v1:172.21.15.84:6790/0 conn(0x7fcc740a1080 legacy=0x7fcc7413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 >> v1:172.21.15.45:6789/0 conn(0x7fcc740a0cb0 legacy=0x7fcc74137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient: _finish_hunting 0 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 monclient: found mon.noname-f 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc740a1660 con 0x7fcc7406e810 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient: _finish_auth 0 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcc64001830 con 0x7fcc7406e810 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient: dump: 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7919d640 5 monclient: authenticate success, global_id 4240 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7919d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcc64001a10 con 0x7fcc7406e810 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7919d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 1 -- 172.21.15.45:0/3721017152 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcc64002240 con 0x7fcc7406e810 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7919d640 4 set_mon_vals no callback set 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.151+0000 7fcc7259c640 10 monclient: dump: 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.152+0000 7fcc7919d640 10 monclient: shutdown 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.152+0000 7fcc7919d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.152+0000 7fcc7919d640 1 -- 172.21.15.45:0/3721017152 >> v1:172.21.15.84:6789/0 conn(0x7fcc7406e810 legacy=0x7fcc740a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.152+0000 7fcc7919d640 1 -- 172.21.15.45:0/3721017152 shutdown_connections 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.152+0000 7fcc7919d640 1 -- 172.21.15.45:0/3721017152 >> 172.21.15.45:0/3721017152 conn(0x7fcc740a5bb0 msgr2=0x7fcc740a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.152+0000 7fcc7919d640 1 -- 172.21.15.45:0/3721017152 shutdown_connections 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.152+0000 7fcc7919d640 1 -- 172.21.15.45:0/3721017152 wait complete. 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.152+0000 7fcc7919d640 10 monclient: build_initial_monmap 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.152+0000 7fcc7919d640 10 monclient: monmap: 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 1 Processor -- start 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 1 -- start start 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 10 monclient: init 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 10 monclient: _add_conns ranks=[4,8,1,0,5,2,3,6,7] 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 10 monclient(hunting): picked mon.noname-c con 0x7fcc7406e810 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 10 monclient(hunting): picked mon.noname-e con 0x7fcc740a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 10 monclient(hunting): picked mon.noname-f con 0x7fcc740a1080 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7409fa10 con 0x7fcc740a1080 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc740a2270 con 0x7fcc7406e810 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc741217c0 con 0x7fcc740a0cb0 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc7919d640 10 monclient(hunting): authenticate will time out at 760.491623s 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc6bfff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fcc740a0cb0 0x7fcc74146a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:56680/0 (socket says 172.21.15.45:56680) 2024-09-18T13:59:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc6bfff640 1 -- 172.21.15.45:0/3738699621 learned_addr learned my addr 172.21.15.45:0/3738699621 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc72d9d640 1 --1- 172.21.15.45:0/3738699621 >> v1:172.21.15.45:6791/0 conn(0x7fcc7406e810 0x7fcc741343c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36290/0 (socket says 172.21.15.45:36290) 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174031267 0 0) 0x7fcc741217c0 con 0x7fcc740a0cb0 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc6affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc6affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc6affd640 10 monclient(hunting): my global_id is 4238 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc540034e0 con 0x7fcc740a0cb0 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968009460 0 0) 0x7fcc740a2270 con 0x7fcc7406e810 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc6affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc6affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc6affd640 10 monclient(hunting): my global_id is 4252 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.153+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc741217c0 con 0x7fcc7406e810 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 135034621 0 0) 0x7fcc7409fa10 con 0x7fcc740a1080 2024-09-18T13:59:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): my global_id is 4249 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc740a2270 con 0x7fcc740a1080 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 751023990 0 0) 0x7fcc741217c0 con 0x7fcc7406e810 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcc7409fa10 con 0x7fcc7406e810 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2793432010 0 0) 0x7fcc540034e0 con 0x7fcc740a0cb0 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcc741217c0 con 0x7fcc740a0cb0 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcc64001ec0 con 0x7fcc7406e810 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): dump: 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.169 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcc5c001b20 con 0x7fcc740a0cb0 2024-09-18T13:59:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): dump: 2024-09-18T13:59:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 89440003 0 0) 0x7fcc740a2270 con 0x7fcc740a1080 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcc540034e0 con 0x7fcc740a1080 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcc6c001d00 con 0x7fcc740a1080 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient(hunting): dump: 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 734529528 0 0) 0x7fcc7409fa10 con 0x7fcc7406e810 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 >> v1:172.21.15.45:6793/0 conn(0x7fcc740a0cb0 legacy=0x7fcc74146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 >> v1:172.21.15.84:6789/0 conn(0x7fcc740a1080 legacy=0x7fcc7414a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 monclient: found mon.c 2024-09-18T13:59:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7411fa20 con 0x7fcc7406e810 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient: _finish_auth 0 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc7919d640 5 monclient: authenticate success, global_id 4252 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc7919d640 10 monclient: _renew_subs 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc7919d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc7919d640 1 -- 172.21.15.45:0/3738699621 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc741214e0 con 0x7fcc7406e810 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc7919d640 10 monclient: _renew_subs 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc7919d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc7919d640 1 -- 172.21.15.45:0/3738699621 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc74134ad0 con 0x7fcc7406e810 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcc64001860 con 0x7fcc7406e810 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcc64002880 con 0x7fcc7406e810 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc71d9b640 4 set_mon_vals no callback set 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.154+0000 7fcc6affd640 10 monclient: dump: 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.155+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fcc640029e0 con 0x7fcc7406e810 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.155+0000 7fcc7919d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.155+0000 7fcc7919d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.155+0000 7fcc7919d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.155+0000 7fcc7919d640 1 -- 172.21.15.45:0/3738699621 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc58005010 con 0x7fcc7406e810 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.155+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7fcc640469d0 con 0x7fcc7406e810 2024-09-18T13:59:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.155+0000 7fcc6bfff640 10 monclient: get_auth_request con 0x7fcc5403a8f0 auth_method 0 2024-09-18T13:59:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.160+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.4 v1:172.21.15.45:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fcc64014fb0 con 0x7fcc7406e810 2024-09-18T13:59:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.160+0000 7fcc6affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.160+0000 7fcc6affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:05.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.370+0000 7fcc7919d640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2024-09-18T13:59:05.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.370+0000 7fcc7919d640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2024-09-18T13:59:05.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.370+0000 7fcc7919d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:05.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.370+0000 7fcc7919d640 1 -- 172.21.15.45:0/3738699621 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fcc580052c0 con 0x7fcc7406e810 2024-09-18T13:59:05.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.372+0000 7fcc6affd640 1 -- 172.21.15.45:0/3738699621 <== mon.4 v1:172.21.15.45:6791/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+87162 (unknown 481731318 0 1697107866) 0x7fcc64014dd0 con 0x7fcc7406e810 2024-09-18T13:59:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.372+0000 7fcc6affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2024-09-18T13:59:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.372+0000 7fcc6affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:59:05.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.374+0000 7fcc7919d640 1 -- 172.21.15.45:0/3738699621 >> v1:172.21.15.84:6812/31478 conn(0x7fcc5403a8f0 legacy=0x7fcc5403cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.374+0000 7fcc7919d640 10 monclient: shutdown 2024-09-18T13:59:05.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.374+0000 7fcc7919d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:05.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.374+0000 7fcc7919d640 1 -- 172.21.15.45:0/3738699621 >> v1:172.21.15.45:6791/0 conn(0x7fcc7406e810 legacy=0x7fcc741343c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.374+0000 7fcc7919d640 1 -- 172.21.15.45:0/3738699621 shutdown_connections 2024-09-18T13:59:05.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.374+0000 7fcc7919d640 1 -- 172.21.15.45:0/3738699621 >> 172.21.15.45:0/3738699621 conn(0x7fcc740a5bb0 msgr2=0x7fcc740a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:05.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.374+0000 7fcc7919d640 1 -- 172.21.15.45:0/3738699621 shutdown_connections 2024-09-18T13:59:05.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.374+0000 7fcc7919d640 1 -- 172.21.15.45:0/3738699621 wait complete. 2024-09-18T13:59:05.385 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"flags":0,"active_gid":4116,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6812","nonce":31478}]},"active_addr":"172.21.15.84:6812/31478","active_change":"2024-09-18T13:58:55.881473+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":11381844}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":829772183}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":1467114352}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":3734631637}]}]} 2024-09-18T13:59:05.389 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-18T13:59:05.389 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T13:59:05.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T13:59:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.484+0000 7fbaddb5d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.484+0000 7fbaddb5d640 10 monclient: build_initial_monmap 2024-09-18T13:59:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.484+0000 7fbaddb5d640 10 monclient: monmap: 2024-09-18T13:59:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:05.485646+0000 2024-09-18T13:59:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:05.485646+0000 2024-09-18T13:59:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:05.486 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.485+0000 7fbaddb5d640 1 Processor -- start 2024-09-18T13:59:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.485+0000 7fbaddb5d640 1 -- start start 2024-09-18T13:59:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.485+0000 7fbaddb5d640 10 monclient: init 2024-09-18T13:59:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.485+0000 7fbaddb5d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.485+0000 7fbaddb5d640 10 monclient: _add_conns ranks=[1,0,5,6,3,4,2,7,8] 2024-09-18T13:59:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.485+0000 7fbaddb5d640 10 monclient(hunting): picked mon.noname-f con 0x7fbad809f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.485+0000 7fbaddb5d640 10 monclient(hunting): picked mon.noname-a con 0x7fbad809fba0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.485+0000 7fbaddb5d640 10 monclient(hunting): picked mon.noname-h con 0x7fbad809ff70 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.485+0000 7fbaddb5d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad805ed70 con 0x7fbad809fba0 2024-09-18T13:59:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbaddb5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad809dfc0 con 0x7fbad809f7d0 2024-09-18T13:59:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbaddb5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad809dce0 con 0x7fbad809ff70 2024-09-18T13:59:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbaddb5d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbaddb5d640 10 monclient(hunting): authenticate will time out at 760.824032s 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad6ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbad809fba0 0x7fbad8137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49348/0 (socket says 172.21.15.45:49348) 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad6ffd640 1 -- 172.21.15.45:0/2893941677 learned_addr learned my addr 172.21.15.45:0/2893941677 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311686110 0 0) 0x7fbad805ed70 con 0x7fbad809fba0 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient(hunting): my global_id is 4266 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac00034e0 con 0x7fbad809fba0 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373357985 0 0) 0x7fbad809dce0 con 0x7fbad809ff70 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient(hunting): my global_id is 4226 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbad805ed70 con 0x7fbad809ff70 2024-09-18T13:59:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460865273 0 0) 0x7fbad809dfc0 con 0x7fbad809f7d0 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient(hunting): my global_id is 4258 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbad809dce0 con 0x7fbad809f7d0 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2677790963 0 0) 0x7fbac00034e0 con 0x7fbad809fba0 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 >> v1:172.21.15.84:6791/0 conn(0x7fbad809ff70 legacy=0x7fbad813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 >> v1:172.21.15.84:6789/0 conn(0x7fbad809f7d0 legacy=0x7fbad8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 monclient: found mon.noname-a 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad80a1c00 con 0x7fbad809fba0 2024-09-18T13:59:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbacc001830 con 0x7fbad809fba0 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbaddb5d640 5 monclient: authenticate success, global_id 4266 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbaddb5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient: dump: 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbaddb5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbacc001a10 con 0x7fbad809fba0 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 1 -- 172.21.15.45:0/2893941677 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbacc004ba0 con 0x7fbad809fba0 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbaddb5d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbaddb5d640 4 set_mon_vals no callback set 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.486+0000 7fbad67fc640 10 monclient: dump: 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.487+0000 7fbaddb5d640 10 monclient: shutdown 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.487+0000 7fbaddb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.487+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2893941677 >> v1:172.21.15.45:6789/0 conn(0x7fbad809fba0 legacy=0x7fbad8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.487+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2893941677 shutdown_connections 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.487+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2893941677 >> 172.21.15.45:0/2893941677 conn(0x7fbad80630b0 msgr2=0x7fbad80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.487+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2893941677 shutdown_connections 2024-09-18T13:59:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.487+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2893941677 wait complete. 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.487+0000 7fbaddb5d640 10 monclient: build_initial_monmap 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.487+0000 7fbaddb5d640 10 monclient: monmap: 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 1 Processor -- start 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 1 -- start start 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 10 monclient: init 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 10 monclient: _add_conns ranks=[4,8,7,6,3,2,0,5,1] 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 10 monclient(hunting): picked mon.noname-c con 0x7fbad809f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 10 monclient(hunting): picked mon.noname-e con 0x7fbad809fba0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 10 monclient(hunting): picked mon.noname-i con 0x7fbad809ff70 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad8142840 con 0x7fbad809f7d0 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad8121cb0 con 0x7fbad809ff70 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad80a2560 con 0x7fbad809fba0 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbaddb5d640 10 monclient(hunting): authenticate will time out at 760.826840s 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbad77fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fbad809f7d0 0x7fbad8141950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36296/0 (socket says 172.21.15.45:36296) 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbad77fe640 1 -- 172.21.15.45:0/2616445129 learned_addr learned my addr 172.21.15.45:0/2616445129 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.488+0000 7fbad6ffd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fbad809fba0 0x7fbad8131880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:56682/0 (socket says 172.21.15.45:56682) 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620903955 0 0) 0x7fbad80a2560 con 0x7fbad809fba0 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): my global_id is 4247 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac40034e0 con 0x7fbad809fba0 2024-09-18T13:59:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011116151 0 0) 0x7fbad8142840 con 0x7fbad809f7d0 2024-09-18T13:59:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): my global_id is 4261 2024-09-18T13:59:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbad80a2560 con 0x7fbad809f7d0 2024-09-18T13:59:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267338901 0 0) 0x7fbad8121cb0 con 0x7fbad809ff70 2024-09-18T13:59:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): my global_id is 4228 2024-09-18T13:59:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbad8142840 con 0x7fbad809ff70 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2728373846 0 0) 0x7fbac40034e0 con 0x7fbad809fba0 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbad8121cb0 con 0x7fbad809fba0 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbacc004be0 con 0x7fbad809fba0 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): dump: 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1369226483 0 0) 0x7fbad80a2560 con 0x7fbad809f7d0 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbac40034e0 con 0x7fbad809f7d0 2024-09-18T13:59:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbac8002a70 con 0x7fbad809f7d0 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): dump: 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2929840414 0 0) 0x7fbad8142840 con 0x7fbad809ff70 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbad80a2560 con 0x7fbad809ff70 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbad0001cc0 con 0x7fbad809ff70 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient(hunting): dump: 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2125447171 0 0) 0x7fbad8121cb0 con 0x7fbad809fba0 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 >> v1:172.21.15.84:6792/0 conn(0x7fbad809ff70 legacy=0x7fbad8149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 >> v1:172.21.15.45:6791/0 conn(0x7fbad809f7d0 legacy=0x7fbad8141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient: _finish_hunting 0 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 monclient: found mon.e 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad81219d0 con 0x7fbad809fba0 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient: _finish_auth 0 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbaddb5d640 5 monclient: authenticate success, global_id 4247 2024-09-18T13:59:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbaddb5d640 10 monclient: _renew_subs 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbaddb5d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2616445129 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad8131f90 con 0x7fbad809fba0 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbaddb5d640 10 monclient: _renew_subs 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbaddb5d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2616445129 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad8120c10 con 0x7fbad809fba0 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbacc001840 con 0x7fbad809fba0 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbacc005160 con 0x7fbad809fba0 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbad5ffb640 4 set_mon_vals no callback set 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.489+0000 7fbabf7fe640 10 monclient: dump: 2024-09-18T13:59:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.512 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.490+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fbacc010130 con 0x7fbad809fba0 2024-09-18T13:59:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.490+0000 7fbaddb5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.490+0000 7fbaddb5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.490+0000 7fbaddb5d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.490+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2616445129 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab8005010 con 0x7fbad809fba0 2024-09-18T13:59:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.490+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7fbacc046b30 con 0x7fbad809fba0 2024-09-18T13:59:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.491+0000 7fbad77fe640 10 monclient: get_auth_request con 0x7fbac403a890 auth_method 0 2024-09-18T13:59:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.495+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fbacc011680 con 0x7fbad809fba0 2024-09-18T13:59:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.495+0000 7fbabf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.495+0000 7fbabf7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.660+0000 7fbaddb5d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:59:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.660+0000 7fbaddb5d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:59:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.660+0000 7fbaddb5d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.660+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2616445129 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbab80052c0 con 0x7fbad809fba0 2024-09-18T13:59:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.661+0000 7fbabf7fe640 1 -- 172.21.15.45:0/2616445129 <== mon.8 v1:172.21.15.45:6793/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+10060 (unknown 1054498076 0 3527204855) 0x7fbacc014fa0 con 0x7fbad809fba0 2024-09-18T13:59:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.661+0000 7fbabf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:59:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.661+0000 7fbabf7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:05.662 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:59:05.662 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":16,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","created":"2024-09-18T13:58:52.168851+0000","modified":"2024-09-18T13:59:05.031012+0000","last_up_change":"2024-09-18T13:59:00.910314+0000","last_in_change":"2024-09-18T13:58:53.737759+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T13:59:01.925362+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-18T13:59:01.954982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"8c3a8909-07ce-4141-98dd-131951f18ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6804","nonce":32276}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6805","nonce":32276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6807","nonce":32276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6806","nonce":32276}]},"public_addr":"172.21.15.45:6804/32276","cluster_addr":"172.21.15.45:6805/32276","heartbeat_back_addr":"172.21.15.45:6807/32276","heartbeat_front_addr":"172.21.15.45:6806/32276","state":["exists","up"]},{"osd":1,"uuid":"a12b3bc3-1e03-47dc-98c1-c8bd199551a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6800","nonce":32277}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6801","nonce":32277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6803","nonce":32277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6802","nonce":32277}]},"public_addr":"172.21.15.45:6800/32277","cluster_addr":"172.21.15.45:6801/32277","heartbeat_back_addr":"172.21.15.45:6803/32277","heartbeat_front_addr":"172.21.15.45:6802/32277","state":["exists","up"]},{"osd":2,"uuid":"216b8b62-e41c-4bc5-9395-f2d56d402b1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6808","nonce":32278}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6809","nonce":32278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6811","nonce":32278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6810","nonce":32278}]},"public_addr":"172.21.15.45:6808/32278","cluster_addr":"172.21.15.45:6809/32278","heartbeat_back_addr":"172.21.15.45:6811/32278","heartbeat_front_addr":"172.21.15.45:6810/32278","state":["exists","up"]},{"osd":3,"uuid":"c750b0e6-9143-489d-9be0-eb085178ff2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6802","nonce":32024}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6804","nonce":32024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6807","nonce":32024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6806","nonce":32024}]},"public_addr":"172.21.15.84:6802/32024","cluster_addr":"172.21.15.84:6804/32024","heartbeat_back_addr":"172.21.15.84:6807/32024","heartbeat_front_addr":"172.21.15.84:6806/32024","state":["exists","up"]},{"osd":4,"uuid":"9680908d-dc81-44a0-8603-93711579d42e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6800","nonce":32025}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6801","nonce":32025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6805","nonce":32025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6803","nonce":32025}]},"public_addr":"172.21.15.84:6800/32025","cluster_addr":"172.21.15.84:6801/32025","heartbeat_back_addr":"172.21.15.84:6805/32025","heartbeat_front_addr":"172.21.15.84:6803/32025","state":["exists","up"]},{"osd":5,"uuid":"b3b63eef-c8fc-495f-960f-7436315119ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6808","nonce":32048}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6809","nonce":32048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6811","nonce":32048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6810","nonce":32048}]},"public_addr":"172.21.15.84:6808/32048","cluster_addr":"172.21.15.84:6809/32048","heartbeat_back_addr":"172.21.15.84:6811/32048","heartbeat_front_addr":"172.21.15.84:6810/32048","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.823226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.883512+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.904155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.801736+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.835588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.910413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T13:59:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.663+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2616445129 >> v1:172.21.15.84:6812/31478 conn(0x7fbac403a890 legacy=0x7fbac403cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.663+0000 7fbaddb5d640 10 monclient: shutdown 2024-09-18T13:59:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.663+0000 7fbaddb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.663+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2616445129 >> v1:172.21.15.45:6793/0 conn(0x7fbad809fba0 legacy=0x7fbad8131880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.663+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2616445129 shutdown_connections 2024-09-18T13:59:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.663+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2616445129 >> 172.21.15.45:0/2616445129 conn(0x7fbad80630b0 msgr2=0x7fbad80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.663+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2616445129 shutdown_connections 2024-09-18T13:59:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.663+0000 7fbaddb5d640 1 -- 172.21.15.45:0/2616445129 wait complete. 2024-09-18T13:59:05.674 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T13:59:05.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T13:59:05.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.770+0000 7f1a0099d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:05.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.770+0000 7f1a0099d640 10 monclient: build_initial_monmap 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.770+0000 7f1a0099d640 10 monclient: monmap: 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:05.771123+0000 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:05.771123+0000 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:05.772 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.770+0000 7f1a0099d640 1 Processor -- start 2024-09-18T13:59:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.770+0000 7f1a0099d640 1 -- start start 2024-09-18T13:59:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.770+0000 7f1a0099d640 10 monclient: init 2024-09-18T13:59:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f1a0099d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f1a0099d640 10 monclient: _add_conns ranks=[2,1,4,5,6,0,8,3,7] 2024-09-18T13:59:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f1a0099d640 10 monclient(hunting): picked mon.noname-b con 0x7f19fc09f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f1a0099d640 10 monclient(hunting): picked mon.noname-f con 0x7f19fc09fba0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f1a0099d640 10 monclient(hunting): picked mon.noname-c con 0x7f19fc09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f1a0099d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19fc05ed70 con 0x7f19fc09fba0 2024-09-18T13:59:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f1a0099d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19fc09dfc0 con 0x7f19fc09f7d0 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f1a0099d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19fc09dce0 con 0x7f19fc09ff70 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f1a0099d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f1a0099d640 10 monclient(hunting): authenticate will time out at 761.109564s 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19fa59c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f19fc09f7d0 0x7f19fc132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:50274/0 (socket says 172.21.15.45:50274) 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19fad9d640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f19fc09ff70 0x7f19fc13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36310/0 (socket says 172.21.15.45:36310) 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19fa59c640 1 -- 172.21.15.45:0/1735304843 learned_addr learned my addr 172.21.15.45:0/1735304843 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916128987 0 0) 0x7f19fc09dce0 con 0x7f19fc09ff70 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 10 monclient(hunting): my global_id is 4270 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e40034e0 con 0x7f19fc09ff70 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945262683 0 0) 0x7f19fc09dfc0 con 0x7f19fc09f7d0 2024-09-18T13:59:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 10 monclient(hunting): my global_id is 4259 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19fc09dce0 con 0x7f19fc09f7d0 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182013723 0 0) 0x7f19fc05ed70 con 0x7f19fc09fba0 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 10 monclient(hunting): my global_id is 4267 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.771+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19fc09dfc0 con 0x7f19fc09fba0 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 771248868 0 0) 0x7f19fc09dce0 con 0x7f19fc09f7d0 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 >> v1:172.21.15.45:6791/0 conn(0x7f19fc09ff70 legacy=0x7f19fc13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 >> v1:172.21.15.84:6789/0 conn(0x7f19fc09fba0 legacy=0x7f19fc137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 10 monclient: _finish_hunting 0 2024-09-18T13:59:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 1 monclient: found mon.noname-b 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc0a1c00 con 0x7f19fc09f7d0 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 10 monclient: _finish_auth 0 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f19ec001790 con 0x7f19fc09f7d0 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 10 monclient: dump: 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.783 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.783 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.783 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.783 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.783 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.783 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.783 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.783 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.783 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.783 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.783 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.783 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f1a0099d640 5 monclient: authenticate success, global_id 4259 2024-09-18T13:59:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f1a0099d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f19ec0018f0 con 0x7f19fc09f7d0 2024-09-18T13:59:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 1 -- 172.21.15.45:0/1735304843 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f19ec004860 con 0x7f19fc09f7d0 2024-09-18T13:59:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f19f959a640 10 monclient: dump: 2024-09-18T13:59:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f1a0099d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f1a0099d640 4 set_mon_vals no callback set 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f1a0099d640 10 monclient: shutdown 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f1a0099d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f1a0099d640 1 -- 172.21.15.45:0/1735304843 >> v1:172.21.15.45:6790/0 conn(0x7f19fc09f7d0 legacy=0x7f19fc132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.772+0000 7f1a0099d640 1 -- 172.21.15.45:0/1735304843 shutdown_connections 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.773+0000 7f1a0099d640 1 -- 172.21.15.45:0/1735304843 >> 172.21.15.45:0/1735304843 conn(0x7f19fc0630b0 msgr2=0x7f19fc0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.773+0000 7f1a0099d640 1 -- 172.21.15.45:0/1735304843 shutdown_connections 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.773+0000 7f1a0099d640 1 -- 172.21.15.45:0/1735304843 wait complete. 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.773+0000 7f1a0099d640 10 monclient: build_initial_monmap 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.773+0000 7f1a0099d640 10 monclient: monmap: 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.773+0000 7f1a0099d640 1 Processor -- start 2024-09-18T13:59:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.773+0000 7f1a0099d640 1 -- start start 2024-09-18T13:59:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.773+0000 7f1a0099d640 10 monclient: init 2024-09-18T13:59:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f1a0099d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f1a0099d640 10 monclient: _add_conns ranks=[1,7,5,8,6,3,2,4,0] 2024-09-18T13:59:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f1a0099d640 10 monclient(hunting): picked mon.noname-f con 0x7f19fc09f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f1a0099d640 10 monclient(hunting): picked mon.noname-i con 0x7f19fc09fba0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f1a0099d640 10 monclient(hunting): picked mon.noname-h con 0x7f19fc09ff70 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f1a0099d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19fc0a2560 con 0x7f19fc09f7d0 2024-09-18T13:59:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f1a0099d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19fc142630 con 0x7f19fc09ff70 2024-09-18T13:59:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f1a0099d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19fc121920 con 0x7f19fc09fba0 2024-09-18T13:59:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f1a0099d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f1a0099d640 10 monclient(hunting): authenticate will time out at 761.112445s 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19fa59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f19fc09f7d0 0x7f19fc1317d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:34070/0 (socket says 172.21.15.45:34070) 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19fa59c640 1 -- 172.21.15.45:0/1481673548 learned_addr learned my addr 172.21.15.45:0/1481673548 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19f9d9b640 1 --1- 172.21.15.45:0/1481673548 >> v1:172.21.15.84:6792/0 conn(0x7f19fc09fba0 0x7f19fc146b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:34328/0 (socket says 172.21.15.45:34328) 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184359428 0 0) 0x7f19fc121920 con 0x7f19fc09fba0 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 10 monclient(hunting): my global_id is 4237 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19dc0034e0 con 0x7f19fc09fba0 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521998599 0 0) 0x7f19fc142630 con 0x7f19fc09ff70 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 10 monclient(hunting): my global_id is 4235 2024-09-18T13:59:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19fc121920 con 0x7f19fc09ff70 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403620701 0 0) 0x7f19fc0a2560 con 0x7f19fc09f7d0 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 10 monclient(hunting): my global_id is 4276 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.774+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19fc142630 con 0x7f19fc09f7d0 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2728436632 0 0) 0x7f19dc0034e0 con 0x7f19fc09fba0 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19fc0a2560 con 0x7f19fc09fba0 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f19f0001c50 con 0x7f19fc09fba0 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.791 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 981027232 0 0) 0x7f19fc121920 con 0x7f19fc09ff70 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19dc0034e0 con 0x7f19fc09ff70 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f19f4001c40 con 0x7f19fc09ff70 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2746503145 0 0) 0x7f19fc142630 con 0x7f19fc09f7d0 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19fc121920 con 0x7f19fc09f7d0 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f19ec0016d0 con 0x7f19fc09f7d0 2024-09-18T13:59:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 245238226 0 0) 0x7f19fc0a2560 con 0x7f19fc09fba0 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 >> v1:172.21.15.84:6791/0 conn(0x7f19fc09ff70 legacy=0x7f19fc14a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 >> v1:172.21.15.84:6789/0 conn(0x7f19fc09f7d0 legacy=0x7f19fc1317d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 monclient: found mon.i 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc121c00 con 0x7f19fc09fba0 2024-09-18T13:59:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f1a0099d640 5 monclient: authenticate success, global_id 4237 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f1a0099d640 10 monclient: _renew_subs 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f1a0099d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f1a0099d640 1 -- 172.21.15.45:0/1481673548 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19fc131ee0 con 0x7f19fc09fba0 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f1a0099d640 10 monclient: _renew_subs 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f1a0099d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f1a0099d640 1 -- 172.21.15.45:0/1481673548 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f19fc120b60 con 0x7f19fc09fba0 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f19f00020e0 con 0x7f19fc09fba0 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f19f0002650 con 0x7f19fc09fba0 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19f8d99640 4 set_mon_vals no callback set 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.775+0000 7f19ea7fc640 10 monclient: dump: 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:05.798 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:05.798 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:05.798 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:05.798 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:05.798 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:05.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.776+0000 7f1a0099d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:05.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.776+0000 7f1a0099d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:05.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.776+0000 7f1a0099d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:05.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.776+0000 7f1a0099d640 1 -- 172.21.15.45:0/1481673548 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e0005010 con 0x7f19fc09fba0 2024-09-18T13:59:05.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.776+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f19f000ffe0 con 0x7f19fc09fba0 2024-09-18T13:59:05.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.776+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f19f0046a80 con 0x7f19fc09fba0 2024-09-18T13:59:05.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.777+0000 7f19fa59c640 10 monclient: get_auth_request con 0x7f19dc03aa90 auth_method 0 2024-09-18T13:59:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.781+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.7 v1:172.21.15.84:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f19f0015360 con 0x7f19fc09fba0 2024-09-18T13:59:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.781+0000 7f19ea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.781+0000 7f19ea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.946+0000 7f1a0099d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:59:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.946+0000 7f1a0099d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:59:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.946+0000 7f1a0099d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.946+0000 7f1a0099d640 1 -- 172.21.15.45:0/1481673548 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19e00052c0 con 0x7f19fc09fba0 2024-09-18T13:59:05.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.947+0000 7f19ea7fc640 1 -- 172.21.15.45:0/1481673548 <== mon.7 v1:172.21.15.84:6792/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+10060 (unknown 1054498076 0 3527204855) 0x7f19f0014d60 con 0x7f19fc09fba0 2024-09-18T13:59:05.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.947+0000 7f19ea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:59:05.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.947+0000 7f19ea7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:05.949 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:59:05.949 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":16,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","created":"2024-09-18T13:58:52.168851+0000","modified":"2024-09-18T13:59:05.031012+0000","last_up_change":"2024-09-18T13:59:00.910314+0000","last_in_change":"2024-09-18T13:58:53.737759+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T13:59:01.925362+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-18T13:59:01.954982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"8c3a8909-07ce-4141-98dd-131951f18ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6804","nonce":32276}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6805","nonce":32276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6807","nonce":32276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6806","nonce":32276}]},"public_addr":"172.21.15.45:6804/32276","cluster_addr":"172.21.15.45:6805/32276","heartbeat_back_addr":"172.21.15.45:6807/32276","heartbeat_front_addr":"172.21.15.45:6806/32276","state":["exists","up"]},{"osd":1,"uuid":"a12b3bc3-1e03-47dc-98c1-c8bd199551a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6800","nonce":32277}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6801","nonce":32277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6803","nonce":32277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6802","nonce":32277}]},"public_addr":"172.21.15.45:6800/32277","cluster_addr":"172.21.15.45:6801/32277","heartbeat_back_addr":"172.21.15.45:6803/32277","heartbeat_front_addr":"172.21.15.45:6802/32277","state":["exists","up"]},{"osd":2,"uuid":"216b8b62-e41c-4bc5-9395-f2d56d402b1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6808","nonce":32278}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6809","nonce":32278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6811","nonce":32278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6810","nonce":32278}]},"public_addr":"172.21.15.45:6808/32278","cluster_addr":"172.21.15.45:6809/32278","heartbeat_back_addr":"172.21.15.45:6811/32278","heartbeat_front_addr":"172.21.15.45:6810/32278","state":["exists","up"]},{"osd":3,"uuid":"c750b0e6-9143-489d-9be0-eb085178ff2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6802","nonce":32024}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6804","nonce":32024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6807","nonce":32024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6806","nonce":32024}]},"public_addr":"172.21.15.84:6802/32024","cluster_addr":"172.21.15.84:6804/32024","heartbeat_back_addr":"172.21.15.84:6807/32024","heartbeat_front_addr":"172.21.15.84:6806/32024","state":["exists","up"]},{"osd":4,"uuid":"9680908d-dc81-44a0-8603-93711579d42e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6800","nonce":32025}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6801","nonce":32025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6805","nonce":32025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6803","nonce":32025}]},"public_addr":"172.21.15.84:6800/32025","cluster_addr":"172.21.15.84:6801/32025","heartbeat_back_addr":"172.21.15.84:6805/32025","heartbeat_front_addr":"172.21.15.84:6803/32025","state":["exists","up"]},{"osd":5,"uuid":"b3b63eef-c8fc-495f-960f-7436315119ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6808","nonce":32048}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6809","nonce":32048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6811","nonce":32048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6810","nonce":32048}]},"public_addr":"172.21.15.84:6808/32048","cluster_addr":"172.21.15.84:6809/32048","heartbeat_back_addr":"172.21.15.84:6811/32048","heartbeat_front_addr":"172.21.15.84:6810/32048","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.823226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.883512+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.904155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.801736+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.835588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.910413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T13:59:05.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.949+0000 7f1a0099d640 1 -- 172.21.15.45:0/1481673548 >> v1:172.21.15.84:6812/31478 conn(0x7f19dc03aa90 legacy=0x7f19dc03cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.949+0000 7f1a0099d640 10 monclient: shutdown 2024-09-18T13:59:05.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.949+0000 7f1a0099d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:05.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.949+0000 7f1a0099d640 1 -- 172.21.15.45:0/1481673548 >> v1:172.21.15.84:6792/0 conn(0x7f19fc09fba0 legacy=0x7f19fc146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:05.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.949+0000 7f1a0099d640 1 -- 172.21.15.45:0/1481673548 shutdown_connections 2024-09-18T13:59:05.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.949+0000 7f1a0099d640 1 -- 172.21.15.45:0/1481673548 >> 172.21.15.45:0/1481673548 conn(0x7f19fc0630b0 msgr2=0x7f19fc0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:05.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.949+0000 7f1a0099d640 1 -- 172.21.15.45:0/1481673548 shutdown_connections 2024-09-18T13:59:05.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:05.949+0000 7f1a0099d640 1 -- 172.21.15.45:0/1481673548 wait complete. 2024-09-18T13:59:05.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T13:59:05.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T13:59:05.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T13:59:05.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T13:59:05.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T13:59:05.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T13:59:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.081+0000 7f8a46554640 10 monclient: get_monmap_and_config 2024-09-18T13:59:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.081+0000 7f8a46554640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.081+0000 7f8a46554640 10 monclient: monmap: 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:06.082856+0000 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:06.082856+0000 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.084 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.084 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.084 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.082+0000 7f8a46554640 1 Processor -- start 2024-09-18T13:59:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.082+0000 7f8a46554640 1 -- start start 2024-09-18T13:59:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.082+0000 7f8a46554640 10 monclient: init 2024-09-18T13:59:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.083+0000 7f8a46554640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.083+0000 7f8a46554640 10 monclient: _add_conns ranks=[5,7,1,8,3,6,2,4,0] 2024-09-18T13:59:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.083+0000 7f8a46554640 10 monclient(hunting): picked mon.noname-h con 0x7f8a4006e810 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.083+0000 7f8a46554640 10 monclient(hunting): picked mon.noname-i con 0x7f8a400a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.083+0000 7f8a46554640 10 monclient(hunting): picked mon.noname-f con 0x7f8a400a1080 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.083+0000 7f8a46554640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4005ed70 con 0x7f8a400a1080 2024-09-18T13:59:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.083+0000 7f8a46554640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a40097620 con 0x7f8a4006e810 2024-09-18T13:59:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.083+0000 7f8a46554640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a40097340 con 0x7f8a400a0cb0 2024-09-18T13:59:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.083+0000 7f8a46554640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.083+0000 7f8a46554640 10 monclient(hunting): authenticate will time out at 761.421704s 2024-09-18T13:59:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.083+0000 7f8a3f7fe640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f8a400a0cb0 0x7f8a40137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:34330/0 (socket says 172.21.15.45:34330) 2024-09-18T13:59:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.083+0000 7f8a3f7fe640 1 -- 172.21.15.45:0/1152582613 learned_addr learned my addr 172.21.15.45:0/1152582613 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284124079 0 0) 0x7f8a4005ed70 con 0x7f8a400a1080 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient(hunting): my global_id is 4285 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a280034e0 con 0x7f8a400a1080 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950527913 0 0) 0x7f8a40097340 con 0x7f8a400a0cb0 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient(hunting): my global_id is 4246 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a4005ed70 con 0x7f8a400a0cb0 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180402722 0 0) 0x7f8a40097620 con 0x7f8a4006e810 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient(hunting): my global_id is 4244 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a40097340 con 0x7f8a4006e810 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1872014632 0 0) 0x7f8a280034e0 con 0x7f8a400a1080 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 >> v1:172.21.15.84:6792/0 conn(0x7f8a400a0cb0 legacy=0x7f8a40137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 >> v1:172.21.15.84:6791/0 conn(0x7f8a4006e810 legacy=0x7f8a400a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 1 monclient: found mon.noname-f 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a400a1660 con 0x7f8a400a1080 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8a38001ae0 con 0x7f8a400a1080 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a3effd640 10 monclient: dump: 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.098 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a46554640 5 monclient: authenticate success, global_id 4285 2024-09-18T13:59:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.084+0000 7f8a46554640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8a38001c40 con 0x7f8a400a1080 2024-09-18T13:59:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a3effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a3effd640 1 -- 172.21.15.45:0/1152582613 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8a38002170 con 0x7f8a400a1080 2024-09-18T13:59:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a3effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a3effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a3effd640 10 monclient: dump: 2024-09-18T13:59:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.101 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a46554640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a46554640 4 set_mon_vals no callback set 2024-09-18T13:59:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a46554640 10 monclient: shutdown 2024-09-18T13:59:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a46554640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a46554640 1 -- 172.21.15.45:0/1152582613 >> v1:172.21.15.84:6789/0 conn(0x7f8a400a1080 legacy=0x7f8a4013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a46554640 1 -- 172.21.15.45:0/1152582613 shutdown_connections 2024-09-18T13:59:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.085+0000 7f8a46554640 1 -- 172.21.15.45:0/1152582613 >> 172.21.15.45:0/1152582613 conn(0x7f8a400a5bb0 msgr2=0x7f8a400a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.086+0000 7f8a46554640 1 -- 172.21.15.45:0/1152582613 shutdown_connections 2024-09-18T13:59:06.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.086+0000 7f8a46554640 1 -- 172.21.15.45:0/1152582613 wait complete. 2024-09-18T13:59:06.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.086+0000 7f8a46554640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.086+0000 7f8a46554640 10 monclient: monmap: 2024-09-18T13:59:06.102 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.102 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:06.102 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:06.102 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:06.102 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.102 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:06.102 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.102 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.102 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.103 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.087+0000 7f8a46554640 1 Processor -- start 2024-09-18T13:59:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.087+0000 7f8a46554640 1 -- start start 2024-09-18T13:59:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.087+0000 7f8a46554640 10 monclient: init 2024-09-18T13:59:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.087+0000 7f8a46554640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.087+0000 7f8a46554640 10 monclient: _add_conns ranks=[3,6,4,1,5,7,8,2,0] 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.087+0000 7f8a46554640 10 monclient(hunting): picked mon.noname-g con 0x7f8a4006e810 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.087+0000 7f8a46554640 10 monclient(hunting): picked mon.noname-d con 0x7f8a400a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a3f7fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f8a400a0cb0 0x7f8a4014a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43124/0 (socket says 172.21.15.45:43124) 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a44af1640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f8a400a1080 0x7f8a4014e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36322/0 (socket says 172.21.15.45:36322) 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a3ffff640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7f8a4006e810 0x7f8a40141710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:50930/0 (socket says 172.21.15.45:50930) 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a3ffff640 1 -- 172.21.15.45:0/302897524 learned_addr learned my addr 172.21.15.45:0/302897524 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a46554640 10 monclient(hunting): picked mon.noname-c con 0x7f8a400a1080 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4006cc60 con 0x7f8a4006e810 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a400a2270 con 0x7f8a400a1080 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a40141e20 con 0x7f8a400a0cb0 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031704137 0 0) 0x7f8a4006cc60 con 0x7f8a4006e810 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a46554640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a46554640 10 monclient(hunting): authenticate will time out at 761.426584s 2024-09-18T13:59:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): my global_id is 4296 2024-09-18T13:59:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a2c0034e0 con 0x7f8a4006e810 2024-09-18T13:59:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442069302 0 0) 0x7f8a40141e20 con 0x7f8a400a0cb0 2024-09-18T13:59:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): my global_id is 4272 2024-09-18T13:59:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a4006cc60 con 0x7f8a400a0cb0 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1576870940 0 0) 0x7f8a2c0034e0 con 0x7f8a4006e810 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8a40141e20 con 0x7f8a4006e810 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8a30001ba0 con 0x7f8a4006e810 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): dump: 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383782512 0 0) 0x7f8a400a2270 con 0x7f8a400a1080 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 10 monclient(hunting): my global_id is 4279 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.088+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a2c0034e0 con 0x7f8a400a1080 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1062803117 0 0) 0x7f8a40141e20 con 0x7f8a4006e810 2024-09-18T13:59:06.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 >> v1:172.21.15.45:6792/0 conn(0x7f8a400a0cb0 legacy=0x7f8a4014a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 >> v1:172.21.15.45:6791/0 conn(0x7f8a400a1080 legacy=0x7f8a4014e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 1 monclient: found mon.g 2024-09-18T13:59:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4006db50 con 0x7f8a4006e810 2024-09-18T13:59:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a46554640 5 monclient: authenticate success, global_id 4296 2024-09-18T13:59:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a46554640 10 monclient: _renew_subs 2024-09-18T13:59:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a46554640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a40142c30 con 0x7f8a4006e810 2024-09-18T13:59:06.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8a30001fb0 con 0x7f8a4006e810 2024-09-18T13:59:06.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a46554640 10 monclient: _renew_subs 2024-09-18T13:59:06.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a46554640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:06.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4006de30 con 0x7f8a4006e810 2024-09-18T13:59:06.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8a30002560 con 0x7f8a4006e810 2024-09-18T13:59:06.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a27fff640 10 monclient: dump: 2024-09-18T13:59:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.111 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.089+0000 7f8a3e7fc640 4 set_mon_vals no callback set 2024-09-18T13:59:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.090+0000 7f8a46554640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.090+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.84:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a4004c4a0 con 0x7f8a4006e810 2024-09-18T13:59:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.090+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f8a3000ff30 con 0x7f8a4006e810 2024-09-18T13:59:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.091+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f8a300468c0 con 0x7f8a4006e810 2024-09-18T13:59:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.091+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.84:6800/32025 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a2c0449f0 con 0x7f8a2c040d50 2024-09-18T13:59:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.091+0000 7f8a3f7fe640 10 monclient: get_auth_request con 0x7f8a2c03a140 auth_method 0 2024-09-18T13:59:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.091+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== mon.3 v1:172.21.15.84:6790/0 9 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (unknown 3847591702 0 0) 0x7f8a30015140 con 0x7f8a4006e810 2024-09-18T13:59:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.091+0000 7f8a27fff640 10 monclient: handle_get_version_reply finishing 1 version 16 2024-09-18T13:59:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.091+0000 7f8a44af1640 10 monclient: get_auth_request con 0x7f8a2c040d50 auth_method 0 2024-09-18T13:59:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.092+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== osd.4 v1:172.21.15.84:6800/32025 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f8a2c0449f0 con 0x7f8a2c040d50 2024-09-18T13:59:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.095+0000 7f4ce719d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.095+0000 7f4ce719d640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.095+0000 7f4ce719d640 10 monclient: monmap: 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:06.096272+0000 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:06.096272+0000 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.096+0000 7f4ce719d640 1 Processor -- start 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.096+0000 7f4ce719d640 1 -- start start 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.096+0000 7f4ce719d640 10 monclient: init 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.096+0000 7f4ce719d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.096+0000 7f4ce719d640 10 monclient: _add_conns ranks=[1,3,4,0,6,8,5,7,2] 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.096+0000 7f4ce719d640 10 monclient(hunting): picked mon.noname-f con 0x7f4ce006e350 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.096+0000 7f4ce719d640 10 monclient(hunting): picked mon.noname-g con 0x7f4ce009f530 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.096+0000 7f4ce719d640 10 monclient(hunting): picked mon.noname-c con 0x7f4ce0136d60 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.096+0000 7f4ce719d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce005ed70 con 0x7f4ce006e350 2024-09-18T13:59:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4ce719d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce006c480 con 0x7f4ce009f530 2024-09-18T13:59:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4ce719d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce006e720 con 0x7f4ce0136d60 2024-09-18T13:59:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4ce573a640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f4ce0136d60 0x7f4ce0139140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36326/0 (socket says 172.21.15.45:36326) 2024-09-18T13:59:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4ce573a640 1 -- 172.21.15.45:0/3032208148 learned_addr learned my addr 172.21.15.45:0/3032208148 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4ce719d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4ce719d640 10 monclient(hunting): authenticate will time out at 761.435286s 2024-09-18T13:59:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035823076 0 0) 0x7f4ce005ed70 con 0x7f4ce006e350 2024-09-18T13:59:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 10 monclient(hunting): my global_id is 4294 2024-09-18T13:59:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cd00034e0 con 0x7f4ce006e350 2024-09-18T13:59:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228393970 0 0) 0x7f4ce006c480 con 0x7f4ce009f530 2024-09-18T13:59:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 10 monclient(hunting): my global_id is 4305 2024-09-18T13:59:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ce005ed70 con 0x7f4ce009f530 2024-09-18T13:59:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713035825 0 0) 0x7f4ce006e720 con 0x7f4ce0136d60 2024-09-18T13:59:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 10 monclient(hunting): my global_id is 4288 2024-09-18T13:59:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ce006c480 con 0x7f4ce0136d60 2024-09-18T13:59:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1779144248 0 0) 0x7f4cd00034e0 con 0x7f4ce006e350 2024-09-18T13:59:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 >> v1:172.21.15.45:6791/0 conn(0x7f4ce0136d60 legacy=0x7f4ce0139140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.097+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 >> v1:172.21.15.84:6790/0 conn(0x7f4ce009f530 legacy=0x7f4ce0135640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 1 monclient: found mon.noname-f 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce006d890 con 0x7f4ce006e350 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4cd4001780 con 0x7f4ce006e350 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4ce719d640 5 monclient: authenticate success, global_id 4294 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4ce719d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T13:59:06.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 10 monclient: dump: 2024-09-18T13:59:06.128 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.128 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.128 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.128 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.128 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.128 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.128 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.128 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.128 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.128 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.128 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.128 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.129 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.129 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.129 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.129 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4cd40019a0 con 0x7f4ce006e350 2024-09-18T13:59:06.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4ce719d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4cdf7fe640 1 -- 172.21.15.45:0/3032208148 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4cd4001f00 con 0x7f4ce006e350 2024-09-18T13:59:06.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4ce719d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:06.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.098+0000 7f4ce719d640 4 set_mon_vals no callback set 2024-09-18T13:59:06.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.099+0000 7f4cdf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.099+0000 7f4cdf7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:06.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.099+0000 7f4cdf7fe640 10 monclient: dump: 2024-09-18T13:59:06.130 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.130 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.130 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.130 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.130 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.130 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.130 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.099+0000 7f4ce719d640 10 monclient: shutdown 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.099+0000 7f4ce719d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.099+0000 7f4ce719d640 1 -- 172.21.15.45:0/3032208148 >> v1:172.21.15.84:6789/0 conn(0x7f4ce006e350 legacy=0x7f4ce009ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.099+0000 7f4ce719d640 1 -- 172.21.15.45:0/3032208148 shutdown_connections 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.099+0000 7f4ce719d640 1 -- 172.21.15.45:0/3032208148 >> 172.21.15.45:0/3032208148 conn(0x7f4ce00a44c0 msgr2=0x7f4ce00a0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.099+0000 7f4ce719d640 1 -- 172.21.15.45:0/3032208148 shutdown_connections 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.099+0000 7f4ce719d640 1 -- 172.21.15.45:0/3032208148 wait complete. 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.100+0000 7f4ce719d640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.100+0000 7f4ce719d640 10 monclient: monmap: 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.100+0000 7f4ce719d640 1 Processor -- start 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.100+0000 7f4ce719d640 1 -- start start 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.100+0000 7f4ce719d640 10 monclient: init 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4ce719d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4ce719d640 10 monclient: _add_conns ranks=[1,0,7,8,5,2,4,6,3] 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4ce719d640 10 monclient(hunting): picked mon.noname-f con 0x7f4ce006e350 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4ce719d640 10 monclient(hunting): picked mon.noname-a con 0x7f4ce009f530 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4ce719d640 10 monclient(hunting): picked mon.noname-i con 0x7f4ce0136d60 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4ce719d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce01344c0 con 0x7f4ce009f530 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4ce719d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce009dcc0 con 0x7f4ce006e350 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4ce719d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce006cb80 con 0x7f4ce0136d60 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4ce719d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4ce719d640 10 monclient(hunting): authenticate will time out at 761.439460s 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4cdffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4ce009f530 0x7f4ce0131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49356/0 (socket says 172.21.15.45:49356) 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4cdffff640 1 -- 172.21.15.45:0/2262310648 learned_addr learned my addr 172.21.15.45:0/2262310648 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4ce4f39640 1 --1- 172.21.15.45:0/2262310648 >> v1:172.21.15.84:6789/0 conn(0x7f4ce006e350 0x7f4ce0131350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:34104/0 (socket says 172.21.15.45:34104) 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277046468 0 0) 0x7f4ce009dcc0 con 0x7f4ce006e350 2024-09-18T13:59:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4cdcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4cdcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4cdcff9640 10 monclient(hunting): my global_id is 4303 2024-09-18T13:59:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cc40034e0 con 0x7f4ce006e350 2024-09-18T13:59:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.101+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137771997 0 0) 0x7f4ce006cb80 con 0x7f4ce0136d60 2024-09-18T13:59:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): my global_id is 4255 2024-09-18T13:59:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ce009dcc0 con 0x7f4ce0136d60 2024-09-18T13:59:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851115242 0 0) 0x7f4ce01344c0 con 0x7f4ce009f530 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): my global_id is 4275 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ce006cb80 con 0x7f4ce009f530 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 183869985 0 0) 0x7f4cc40034e0 con 0x7f4ce006e350 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4ce01344c0 con 0x7f4ce006e350 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4cd4001e50 con 0x7f4ce006e350 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): dump: 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3896770303 0 0) 0x7f4ce009dcc0 con 0x7f4ce0136d60 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4cc40034e0 con 0x7f4ce0136d60 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4cd8001d20 con 0x7f4ce0136d60 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): dump: 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3935738819 0 0) 0x7f4ce006cb80 con 0x7f4ce009f530 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4ce009dcc0 con 0x7f4ce009f530 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4ccc001c40 con 0x7f4ce009f530 2024-09-18T13:59:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient(hunting): dump: 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.141 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2286417136 0 0) 0x7f4ce01344c0 con 0x7f4ce006e350 2024-09-18T13:59:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 >> v1:172.21.15.84:6792/0 conn(0x7f4ce0136d60 legacy=0x7f4ce0132170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 >> v1:172.21.15.45:6789/0 conn(0x7f4ce009f530 legacy=0x7f4ce0131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 monclient: found mon.f 2024-09-18T13:59:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce01346f0 con 0x7f4ce006e350 2024-09-18T13:59:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4ce719d640 5 monclient: authenticate success, global_id 4303 2024-09-18T13:59:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4ce719d640 10 monclient: _renew_subs 2024-09-18T13:59:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4ce719d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4ce719d640 1 -- 172.21.15.45:0/2262310648 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce0134990 con 0x7f4ce006e350 2024-09-18T13:59:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4ce719d640 10 monclient: _renew_subs 2024-09-18T13:59:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4ce719d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4ce719d640 1 -- 172.21.15.45:0/2262310648 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce014f280 con 0x7f4ce006e350 2024-09-18T13:59:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.102+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4cd40010e0 con 0x7f4ce006e350 2024-09-18T13:59:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.103+0000 7f4cdcff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.103+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4cd4002660 con 0x7f4ce006e350 2024-09-18T13:59:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.103+0000 7f4cdcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.103+0000 7f4cdcff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.103+0000 7f4cdcff9640 10 monclient: dump: 2024-09-18T13:59:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.145 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.103+0000 7f4cdeffd640 4 set_mon_vals no callback set 2024-09-18T13:59:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.103+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4cd40027c0 con 0x7f4ce006e350 2024-09-18T13:59:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.104+0000 7f4ce719d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.104+0000 7f4ce719d640 1 -- 172.21.15.45:0/2262310648 --> v1:172.21.15.84:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4cc0000f20 con 0x7f4ce006e350 2024-09-18T13:59:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.104+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f4cd4046780 con 0x7f4ce006e350 2024-09-18T13:59:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.104+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 --> v1:172.21.15.45:6808/32278 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4cc40452e0 con 0x7f4cc4041640 2024-09-18T13:59:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.104+0000 7f4cdffff640 10 monclient: get_auth_request con 0x7f4cc403aa80 auth_method 0 2024-09-18T13:59:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.104+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== mon.1 v1:172.21.15.84:6789/0 9 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (unknown 3847591702 0 0) 0x7f4cd4046b50 con 0x7f4ce006e350 2024-09-18T13:59:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.104+0000 7f4cdcff9640 10 monclient: handle_get_version_reply finishing 1 version 16 2024-09-18T13:59:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.104+0000 7f4ce573a640 10 monclient: get_auth_request con 0x7f4cc4041640 auth_method 0 2024-09-18T13:59:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.106+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== osd.2 v1:172.21.15.45:6808/32278 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f4cc40452e0 con 0x7f4cc4041640 2024-09-18T13:59:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.109+0000 7fce60ce8640 10 monclient: get_monmap_and_config 2024-09-18T13:59:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.109+0000 7fce60ce8640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.109+0000 7fce60ce8640 10 monclient: monmap: 2024-09-18T13:59:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:06.110394+0000 2024-09-18T13:59:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:06.110394+0000 2024-09-18T13:59:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.148 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.109+0000 7f1573e4f640 10 monclient: get_monmap_and_config 2024-09-18T13:59:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.109+0000 7f1573e4f640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.109+0000 7f1573e4f640 10 monclient: monmap: 2024-09-18T13:59:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:06.110652+0000 2024-09-18T13:59:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:06.110652+0000 2024-09-18T13:59:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.151 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.110+0000 7fce60ce8640 1 Processor -- start 2024-09-18T13:59:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.110+0000 7fce60ce8640 1 -- start start 2024-09-18T13:59:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.110+0000 7fce60ce8640 10 monclient: init 2024-09-18T13:59:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.110+0000 7f1573e4f640 1 Processor -- start 2024-09-18T13:59:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.110+0000 7f1573e4f640 1 -- start start 2024-09-18T13:59:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.110+0000 7f1573e4f640 10 monclient: init 2024-09-18T13:59:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.111+0000 7fce60ce8640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.111+0000 7fce60ce8640 10 monclient: _add_conns ranks=[0,7,6,2,3,8,1,4,5] 2024-09-18T13:59:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.111+0000 7fce5a59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fce5c06e810 0x7fce5c0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49370/0 (socket says 172.21.15.45:49370) 2024-09-18T13:59:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.111+0000 7fce5a59c640 1 -- 172.21.15.45:0/3227698314 learned_addr learned my addr 172.21.15.45:0/3227698314 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.111+0000 7fce60ce8640 10 monclient(hunting): picked mon.noname-a con 0x7fce5c06e810 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.111+0000 7f1573e4f640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.111+0000 7f1573e4f640 10 monclient: _add_conns ranks=[2,5,4,1,3,0,6,7,8] 2024-09-18T13:59:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.111+0000 7fce60ce8640 10 monclient(hunting): picked mon.noname-i con 0x7fce5c0a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.111+0000 7f1573e4f640 10 monclient(hunting): picked mon.noname-b con 0x7f156c06e810 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.111+0000 7f1573e4f640 10 monclient(hunting): picked mon.noname-h con 0x7f156c0a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7f1571beb640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f156c06e810 0x7f156c0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:50282/0 (socket says 172.21.15.45:50282) 2024-09-18T13:59:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7f15723ec640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f156c0a1080 0x7f156c13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36334/0 (socket says 172.21.15.45:36334) 2024-09-18T13:59:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7f1571beb640 1 -- 172.21.15.45:0/1072638453 learned_addr learned my addr 172.21.15.45:0/1072638453 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7fce60ce8640 10 monclient(hunting): picked mon.noname-d con 0x7fce5c0a1080 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7fce60ce8640 1 -- 172.21.15.45:0/3227698314 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c05ed70 con 0x7fce5c06e810 2024-09-18T13:59:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7f1573e4f640 10 monclient(hunting): picked mon.noname-c con 0x7f156c0a1080 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7fce60ce8640 1 -- 172.21.15.45:0/3227698314 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c097620 con 0x7fce5c0a1080 2024-09-18T13:59:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7f1573e4f640 1 -- 172.21.15.45:0/1072638453 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c05ed70 con 0x7f156c06e810 2024-09-18T13:59:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7f1573e4f640 1 -- 172.21.15.45:0/1072638453 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c097620 con 0x7f156c0a1080 2024-09-18T13:59:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7f1573e4f640 1 -- 172.21.15.45:0/1072638453 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c097340 con 0x7f156c0a0cb0 2024-09-18T13:59:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7f1573e4f640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7f1573e4f640 10 monclient(hunting): authenticate will time out at 761.450730s 2024-09-18T13:59:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7fce60ce8640 1 -- 172.21.15.45:0/3227698314 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c097340 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7fce60ce8640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414667472 0 0) 0x7fce5c097620 con 0x7fce5c0a1080 2024-09-18T13:59:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7fce60ce8640 10 monclient(hunting): authenticate will time out at 761.450786s 2024-09-18T13:59:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7fce5959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7fce5959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7fce5959a640 10 monclient(hunting): my global_id is 4281 2024-09-18T13:59:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.112+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce440034e0 con 0x7fce5c0a1080 2024-09-18T13:59:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815308293 0 0) 0x7fce5c097340 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827673751 0 0) 0x7f156c05ed70 con 0x7f156c06e810 2024-09-18T13:59:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient(hunting): my global_id is 4268 2024-09-18T13:59:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15580034e0 con 0x7f156c06e810 2024-09-18T13:59:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7fce5959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201494983 0 0) 0x7f156c097340 con 0x7f156c0a0cb0 2024-09-18T13:59:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient(hunting): my global_id is 4253 2024-09-18T13:59:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f156c05ed70 con 0x7f156c0a0cb0 2024-09-18T13:59:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7fce5959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989405261 0 0) 0x7f156c097620 con 0x7f156c0a1080 2024-09-18T13:59:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient(hunting): my global_id is 4297 2024-09-18T13:59:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f156c097340 con 0x7f156c0a1080 2024-09-18T13:59:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7fce5959a640 10 monclient(hunting): my global_id is 4264 2024-09-18T13:59:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce5c097620 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263117201 0 0) 0x7fce5c05ed70 con 0x7fce5c06e810 2024-09-18T13:59:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7fce5959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7fce5959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 895379955 0 0) 0x7f156c05ed70 con 0x7f156c0a0cb0 2024-09-18T13:59:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 >> v1:172.21.15.45:6791/0 conn(0x7f156c0a1080 legacy=0x7f156c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 >> v1:172.21.15.45:6790/0 conn(0x7f156c06e810 legacy=0x7f156c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 1 monclient: found mon.noname-h 2024-09-18T13:59:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T13:59:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0a1660 con 0x7f156c0a0cb0 2024-09-18T13:59:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f155c001810 con 0x7f156c0a0cb0 2024-09-18T13:59:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T13:59:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1570be9640 10 monclient: dump: 2024-09-18T13:59:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.166 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1573e4f640 5 monclient: authenticate success, global_id 4253 2024-09-18T13:59:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7f1573e4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7fce5959a640 10 monclient(hunting): my global_id is 4284 2024-09-18T13:59:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce5c097340 con 0x7fce5c06e810 2024-09-18T13:59:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.113+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1118870453 0 0) 0x7fce440034e0 con 0x7fce5c0a1080 2024-09-18T13:59:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 >> v1:172.21.15.84:6792/0 conn(0x7fce5c0a0cb0 legacy=0x7fce5c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f155c001970 con 0x7f156c0a0cb0 2024-09-18T13:59:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 >> v1:172.21.15.45:6789/0 conn(0x7fce5c06e810 legacy=0x7fce5c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 1 monclient: found mon.noname-d 2024-09-18T13:59:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T13:59:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c0a1660 con 0x7fce5c0a1080 2024-09-18T13:59:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7f1570be9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7f1570be9640 1 -- 172.21.15.45:0/1072638453 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f155c001ea0 con 0x7f156c0a0cb0 2024-09-18T13:59:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce54001b30 con 0x7fce5c0a1080 2024-09-18T13:59:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce60ce8640 5 monclient: authenticate success, global_id 4281 2024-09-18T13:59:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce60ce8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T13:59:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 10 monclient: dump: 2024-09-18T13:59:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.172 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce60ce8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7f1570be9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce54001d50 con 0x7fce5c0a1080 2024-09-18T13:59:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce5959a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7f1570be9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:06.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce60ce8640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:06.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7fce60ce8640 4 set_mon_vals no callback set 2024-09-18T13:59:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7f1570be9640 10 monclient: dump: 2024-09-18T13:59:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.177 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7f1573e4f640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7f1573e4f640 4 set_mon_vals no callback set 2024-09-18T13:59:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7f1573e4f640 10 monclient: shutdown 2024-09-18T13:59:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7f1573e4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.114+0000 7f1573e4f640 1 -- 172.21.15.45:0/1072638453 >> v1:172.21.15.84:6791/0 conn(0x7f156c0a0cb0 legacy=0x7f156c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7f1573e4f640 1 -- 172.21.15.45:0/1072638453 shutdown_connections 2024-09-18T13:59:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7f1573e4f640 1 -- 172.21.15.45:0/1072638453 >> 172.21.15.45:0/1072638453 conn(0x7f156c0a5bb0 msgr2=0x7f156c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7f1573e4f640 1 -- 172.21.15.45:0/1072638453 shutdown_connections 2024-09-18T13:59:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7f1573e4f640 1 -- 172.21.15.45:0/1072638453 wait complete. 2024-09-18T13:59:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7fce5959a640 1 -- 172.21.15.45:0/3227698314 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce540025c0 con 0x7fce5c0a1080 2024-09-18T13:59:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7fce5959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7fce5959a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7fce5959a640 10 monclient: dump: 2024-09-18T13:59:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.181 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7fce60ce8640 10 monclient: shutdown 2024-09-18T13:59:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7fce60ce8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7fce60ce8640 1 -- 172.21.15.45:0/3227698314 >> v1:172.21.15.45:6792/0 conn(0x7fce5c0a1080 legacy=0x7fce5c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7fce60ce8640 1 -- 172.21.15.45:0/3227698314 shutdown_connections 2024-09-18T13:59:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7fce60ce8640 1 -- 172.21.15.45:0/3227698314 >> 172.21.15.45:0/3227698314 conn(0x7fce5c0a5bb0 msgr2=0x7fce5c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7fce60ce8640 1 -- 172.21.15.45:0/3227698314 shutdown_connections 2024-09-18T13:59:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.115+0000 7fce60ce8640 1 -- 172.21.15.45:0/3227698314 wait complete. 2024-09-18T13:59:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.116+0000 7f1573e4f640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.116+0000 7f1573e4f640 10 monclient: monmap: 2024-09-18T13:59:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.184 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.116+0000 7f1573e4f640 1 Processor -- start 2024-09-18T13:59:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.116+0000 7f1573e4f640 1 -- start start 2024-09-18T13:59:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.116+0000 7f1573e4f640 10 monclient: init 2024-09-18T13:59:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.116+0000 7fce60ce8640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.116+0000 7fce60ce8640 10 monclient: monmap: 2024-09-18T13:59:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.186 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.186 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.186 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.186 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.186 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.186 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.186 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.186 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.186 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f1573e4f640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f1573e4f640 10 monclient: _add_conns ranks=[8,7,0,1,4,3,6,2,5] 2024-09-18T13:59:06.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f1573e4f640 10 monclient(hunting): picked mon.noname-e con 0x7f156c06e810 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:06.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f1571beb640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f156c06e810 0x7f156c13d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:56698/0 (socket says 172.21.15.45:56698) 2024-09-18T13:59:06.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f1571beb640 1 -- 172.21.15.45:0/214362665 learned_addr learned my addr 172.21.15.45:0/214362665 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f1573e4f640 10 monclient(hunting): picked mon.noname-i con 0x7f156c0a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f1573e4f640 10 monclient(hunting): picked mon.noname-a con 0x7f156c0a1080 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:06.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 --> v1:172.21.15.84:6800/32025 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8a4006b8b0 con 0x7f8a2c040d50 2024-09-18T13:59:06.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c122e00 con 0x7f156c0a1080 2024-09-18T13:59:06.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c13dbf0 con 0x7f156c0a0cb0 2024-09-18T13:59:06.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c122840 con 0x7f156c06e810 2024-09-18T13:59:06.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7fdaedc4f640 10 monclient: get_monmap_and_config 2024-09-18T13:59:06.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f1573e4f640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f1573e4f640 10 monclient(hunting): authenticate will time out at 761.455820s 2024-09-18T13:59:06.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7fce60ce8640 1 Processor -- start 2024-09-18T13:59:06.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7fdaedc4f640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7f8a27fff640 1 -- 172.21.15.45:0/302897524 <== osd.4 v1:172.21.15.84:6800/32025 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1800444376) 0x7f8a4006b8b0 con 0x7f8a2c040d50 2024-09-18T13:59:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7fdaedc4f640 10 monclient: monmap: 2024-09-18T13:59:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:06.118728+0000 2024-09-18T13:59:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:06.118728+0000 2024-09-18T13:59:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.192 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591519351 0 0) 0x7f156c13dbf0 con 0x7f156c0a0cb0 2024-09-18T13:59:06.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): my global_id is 4273 2024-09-18T13:59:06.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15500034e0 con 0x7f156c0a0cb0 2024-09-18T13:59:06.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542455611 0 0) 0x7f156c122e00 con 0x7f156c0a1080 2024-09-18T13:59:06.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): my global_id is 4293 2024-09-18T13:59:06.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f156c13dbf0 con 0x7f156c0a1080 2024-09-18T13:59:06.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.117+0000 7fce60ce8640 1 -- start start 2024-09-18T13:59:06.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7fce60ce8640 10 monclient: init 2024-09-18T13:59:06.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241595311 0 0) 0x7f156c122840 con 0x7f156c06e810 2024-09-18T13:59:06.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): my global_id is 4256 2024-09-18T13:59:06.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f156c122e00 con 0x7f156c06e810 2024-09-18T13:59:06.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2677017753 0 0) 0x7f15500034e0 con 0x7f156c0a0cb0 2024-09-18T13:59:06.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f156c122840 con 0x7f156c0a0cb0 2024-09-18T13:59:06.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f155c001e20 con 0x7f156c0a0cb0 2024-09-18T13:59:06.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:59:06.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): dump: 2024-09-18T13:59:06.197 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.197 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.197 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.197 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.197 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.197 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.197 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.197 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.198 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2795175409 0 0) 0x7f156c13dbf0 con 0x7f156c0a1080 2024-09-18T13:59:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f15500034e0 con 0x7f156c0a1080 2024-09-18T13:59:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1564001c30 con 0x7f156c0a1080 2024-09-18T13:59:06.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:06.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fce60ce8640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fdaedc4f640 1 Processor -- start 2024-09-18T13:59:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.118+0000 7f1561ffb640 10 monclient(hunting): dump: 2024-09-18T13:59:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.201 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fce60ce8640 10 monclient: _add_conns ranks=[4,8,7,0,3,1,5,6,2] 2024-09-18T13:59:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fdaedc4f640 1 -- start start 2024-09-18T13:59:06.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fdaedc4f640 10 monclient: init 2024-09-18T13:59:06.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1451315846 0 0) 0x7f156c122840 con 0x7f156c0a0cb0 2024-09-18T13:59:06.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 >> v1:172.21.15.45:6793/0 conn(0x7f156c06e810 legacy=0x7f156c13d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 >> v1:172.21.15.45:6789/0 conn(0x7f156c0a1080 legacy=0x7f156c149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1561ffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1561ffb640 1 monclient: found mon.i 2024-09-18T13:59:06.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1561ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1561ffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:06.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c1203e0 con 0x7f156c0a0cb0 2024-09-18T13:59:06.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1561ffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fce5a59c640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fce5c06e810 0x7fce5c13d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36342/0 (socket says 172.21.15.45:36342) 2024-09-18T13:59:06.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1561ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1573e4f640 5 monclient: authenticate success, global_id 4273 2024-09-18T13:59:06.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1573e4f640 10 monclient: _renew_subs 2024-09-18T13:59:06.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1573e4f640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:06.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c122560 con 0x7f156c0a0cb0 2024-09-18T13:59:06.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f2c7f6ef640 10 monclient: get_monmap_and_config 2024-09-18T13:59:06.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f2c7f6ef640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 >> v1:172.21.15.84:6800/32025 conn(0x7f8a2c040d50 legacy=0x7f8a2c043180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f2c7f6ef640 10 monclient: monmap: 2024-09-18T13:59:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:06.120590+0000 2024-09-18T13:59:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:06.120590+0000 2024-09-18T13:59:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.207 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.207 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.207 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.207 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.207 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.207 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 >> v1:172.21.15.84:6812/31478 conn(0x7f8a2c03a140 legacy=0x7f8a2c03c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f8a46554640 10 monclient: shutdown 2024-09-18T13:59:06.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f8a46554640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 >> v1:172.21.15.84:6790/0 conn(0x7f8a4006e810 legacy=0x7f8a40141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f8a44af1640 1 -- 172.21.15.45:0/302897524 reap_dead start 2024-09-18T13:59:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1573e4f640 10 monclient: _renew_subs 2024-09-18T13:59:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1573e4f640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c121690 con 0x7f156c0a0cb0 2024-09-18T13:59:06.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 shutdown_connections 2024-09-18T13:59:06.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 >> 172.21.15.45:0/302897524 conn(0x7f8a400a5bb0 msgr2=0x7f8a400a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f155c001740 con 0x7f156c0a0cb0 2024-09-18T13:59:06.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7f1561ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7f1563fff640 4 set_mon_vals no callback set 2024-09-18T13:59:06.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fdaedc4f640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fdaedc4f640 10 monclient: _add_conns ranks=[1,6,7,5,8,4,3,0,2] 2024-09-18T13:59:06.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fdaedc4f640 10 monclient(hunting): picked mon.noname-f con 0x7fdae806e810 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f155c002750 con 0x7f156c0a0cb0 2024-09-18T13:59:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7f1561ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7f1561ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7f1561ffb640 10 monclient: dump: 2024-09-18T13:59:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.212 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 shutdown_connections 2024-09-18T13:59:06.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f155c00d720 con 0x7f156c0a0cb0 2024-09-18T13:59:06.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7f8a46554640 1 -- 172.21.15.45:0/302897524 wait complete. 2024-09-18T13:59:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fce60ce8640 10 monclient(hunting): picked mon.noname-c con 0x7fce5c06e810 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7f2c7f6ef640 1 Processor -- start 2024-09-18T13:59:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fdaedc4f640 10 monclient(hunting): picked mon.noname-d con 0x7fdae80a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fdaedc4f640 10 monclient(hunting): picked mon.noname-i con 0x7fdae80a1080 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7f1573e4f640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.84:6792/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f154c000f20 con 0x7f156c0a0cb0 2024-09-18T13:59:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fdaedc4f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae805ed70 con 0x7fdae806e810 2024-09-18T13:59:06.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fdaedc4f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae8097620 con 0x7fdae80a0cb0 2024-09-18T13:59:06.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fdadeffd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fdae80a0cb0 0x7fdae8137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43134/0 (socket says 172.21.15.45:43134) 2024-09-18T13:59:06.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f155c044210 con 0x7f156c0a0cb0 2024-09-18T13:59:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7f2c7f6ef640 1 -- start start 2024-09-18T13:59:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7f2c7f6ef640 10 monclient: init 2024-09-18T13:59:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fdadeffd640 1 -- 172.21.15.45:0/3887253421 learned_addr learned my addr 172.21.15.45:0/3887253421 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fdae77fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdae806e810 0x7fdae80a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:34112/0 (socket says 172.21.15.45:34112) 2024-09-18T13:59:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fdaedc4f640 1 -- 172.21.15.45:0/3887253421 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae8097340 con 0x7fdae80a1080 2024-09-18T13:59:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fdaedc4f640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.84:6802/32024 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1550044e80 con 0x7f15500411e0 2024-09-18T13:59:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fdaedc4f640 10 monclient(hunting): authenticate will time out at 761.459413s 2024-09-18T13:59:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7f1571beb640 10 monclient: get_auth_request con 0x7f155003a6c0 auth_method 0 2024-09-18T13:59:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fce60ce8640 10 monclient(hunting): picked mon.noname-e con 0x7fce5c0a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fce60ce8640 10 monclient(hunting): picked mon.noname-i con 0x7fce5c0a1080 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fce60ce8640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c122ed0 con 0x7fce5c06e810 2024-09-18T13:59:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fce60ce8640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c13dcc0 con 0x7fce5c0a1080 2024-09-18T13:59:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fce60ce8640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c122910 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fce60ce8640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.119+0000 7fce60ce8640 10 monclient(hunting): authenticate will time out at 761.457998s 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce59d9b640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fce5c0a0cb0 0x7fce5c1467e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:56702/0 (socket says 172.21.15.45:56702) 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce59d9b640 1 -- 172.21.15.45:0/4267318196 learned_addr learned my addr 172.21.15.45:0/4267318196 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020022202 0 0) 0x7fce5c122910 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce4a7fc640 10 monclient(hunting): my global_id is 4265 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce3c0034e0 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3214853359 0 0) 0x7fce3c0034e0 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce5c122910 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce50001d90 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce4a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce4a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce4a7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.222 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.120+0000 7fce4a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:06.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2693832109 0 0) 0x7fce5c122910 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 >> v1:172.21.15.84:6792/0 conn(0x7fce5c0a1080 legacy=0x7fce5c149f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T13:59:06.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7f15723ec640 10 monclient: get_auth_request con 0x7f15500411e0 auth_method 0 2024-09-18T13:59:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 >> v1:172.21.15.45:6791/0 conn(0x7fce5c06e810 legacy=0x7fce5c13d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 1 monclient: found mon.e 2024-09-18T13:59:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c120460 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce60ce8640 5 monclient: authenticate success, global_id 4265 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce60ce8640 10 monclient: _renew_subs 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce60ce8640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce60ce8640 1 -- 172.21.15.45:0/4267318196 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5c122630 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce60ce8640 10 monclient: _renew_subs 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce60ce8640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce60ce8640 1 -- 172.21.15.45:0/4267318196 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c121900 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce50002260 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce50002810 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.121+0000 7fce58d99640 4 set_mon_vals no callback set 2024-09-18T13:59:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fce4a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:06.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fce4a7fc640 10 monclient: dump: 2024-09-18T13:59:06.225 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.225 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.225 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.225 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.225 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.225 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.226 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.226 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.226 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.226 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.226 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.226 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.227 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.227 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.227 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.227 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c7f6ef640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c7f6ef640 10 monclient: _add_conns ranks=[5,8,3,7,4,2,0,1,6] 2024-09-18T13:59:06.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813077228 0 0) 0x7fdae8097340 con 0x7fdae80a1080 2024-09-18T13:59:06.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 10 monclient(hunting): my global_id is 4282 2024-09-18T13:59:06.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad40034e0 con 0x7fdae80a1080 2024-09-18T13:59:06.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443609630 0 0) 0x7fdae805ed70 con 0x7fdae806e810 2024-09-18T13:59:06.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 10 monclient(hunting): my global_id is 4312 2024-09-18T13:59:06.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdae8097340 con 0x7fdae806e810 2024-09-18T13:59:06.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c7f6ef640 10 monclient(hunting): picked mon.noname-h con 0x7f2c7809f230 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:06.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c7f6ef640 10 monclient(hunting): picked mon.noname-e con 0x7f2c780a15b0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:06.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c7cc8a640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f2c780a15b0 0x7f2c78136f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:56704/0 (socket says 172.21.15.45:56704) 2024-09-18T13:59:06.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c7f6ef640 10 monclient(hunting): picked mon.noname-g con 0x7f2c7813cba0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:06.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c7f6ef640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7805ed70 con 0x7f2c7813cba0 2024-09-18T13:59:06.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c7f6ef640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7806f1b0 con 0x7f2c7809f230 2024-09-18T13:59:06.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c7f6ef640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7809f600 con 0x7f2c780a15b0 2024-09-18T13:59:06.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c7f6ef640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c7f6ef640 10 monclient(hunting): authenticate will time out at 761.460415s 2024-09-18T13:59:06.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c7cc8a640 1 -- 172.21.15.45:0/3557093342 learned_addr learned my addr 172.21.15.45:0/3557093342 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c6ffff640 1 -- 172.21.15.45:0/3557093342 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207192646 0 0) 0x7f2c7809f600 con 0x7f2c780a15b0 2024-09-18T13:59:06.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c6ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c6ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c6ffff640 10 monclient(hunting): my global_id is 4274 2024-09-18T13:59:06.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18636660 0 0) 0x7fdae8097620 con 0x7fdae80a0cb0 2024-09-18T13:59:06.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 10 monclient(hunting): my global_id is 4290 2024-09-18T13:59:06.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdae805ed70 con 0x7fdae80a0cb0 2024-09-18T13:59:06.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== mon.7 v1:172.21.15.84:6792/0 9 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (unknown 3847591702 0 0) 0x7f155c012b80 con 0x7f156c0a0cb0 2024-09-18T13:59:06.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f1561ffb640 10 monclient: handle_get_version_reply finishing 1 version 16 2024-09-18T13:59:06.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c6ffff640 1 -- 172.21.15.45:0/3557093342 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c640034e0 con 0x7f2c780a15b0 2024-09-18T13:59:06.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c6ffff640 1 -- 172.21.15.45:0/3557093342 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342467204 0 0) 0x7f2c7805ed70 con 0x7f2c7813cba0 2024-09-18T13:59:06.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c6ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c6ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7f2c6ffff640 10 monclient(hunting): my global_id is 4314 2024-09-18T13:59:06.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 1 -- 172.21.15.45:0/3557093342 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c7809f600 con 0x7f2c7813cba0 2024-09-18T13:59:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1430417515 0 0) 0x7fdad40034e0 con 0x7fdae80a1080 2024-09-18T13:59:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 >> v1:172.21.15.45:6792/0 conn(0x7fdae80a0cb0 legacy=0x7fdae8137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 >> v1:172.21.15.84:6789/0 conn(0x7fdae806e810 legacy=0x7fdae80a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 1 monclient: found mon.noname-i 2024-09-18T13:59:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T13:59:06.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae80a1660 con 0x7fdae80a1080 2024-09-18T13:59:06.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.122+0000 7fdae6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdae0001830 con 0x7fdae80a1080 2024-09-18T13:59:06.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdae6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdae6ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:59:06.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fce60ce8640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:06.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdae6ffd640 10 monclient: dump: 2024-09-18T13:59:06.242 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.242 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.242 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.242 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.243 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.243 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.243 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.243 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.243 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.243 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.244 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.244 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.244 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.244 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.244 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.244 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fce60ce8640 1 -- 172.21.15.45:0/4267318196 --> v1:172.21.15.45:6793/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce40000f20 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdaedc4f640 5 monclient: authenticate success, global_id 4282 2024-09-18T13:59:06.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdaedc4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== osd.3 v1:172.21.15.84:6802/32024 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f1550044e80 con 0x7f15500411e0 2024-09-18T13:59:06.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 1 -- 172.21.15.45:0/3557093342 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3345775273 0 0) 0x7f2c640034e0 con 0x7f2c780a15b0 2024-09-18T13:59:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 1 -- 172.21.15.45:0/3557093342 >> v1:172.21.15.84:6791/0 conn(0x7f2c7809f230 legacy=0x7f2c78136820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 1 -- 172.21.15.45:0/3557093342 >> v1:172.21.15.84:6790/0 conn(0x7f2c7813cba0 legacy=0x7f2c7813ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 1 monclient: found mon.noname-e 2024-09-18T13:59:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T13:59:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 1 -- 172.21.15.45:0/3557093342 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7806ab60 con 0x7f2c780a15b0 2024-09-18T13:59:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 1 -- 172.21.15.45:0/3557093342 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2c68001830 con 0x7f2c780a15b0 2024-09-18T13:59:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:06.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c6ffff640 10 monclient: dump: 2024-09-18T13:59:06.251 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.251 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.251 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.251 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.251 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.251 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.252 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.252 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.252 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.252 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.252 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.252 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.253 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fce50010120 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c7f6ef640 5 monclient: authenticate success, global_id 4274 2024-09-18T13:59:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7f2c7f6ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7fce500469b0 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fce5a59c640 10 monclient: get_auth_request con 0x7fce3c039d40 auth_method 0 2024-09-18T13:59:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 --> v1:172.21.15.84:6808/32048 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce3c0445f0 con 0x7fce3c040950 2024-09-18T13:59:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (unknown 3847591702 0 0) 0x7fce50046d80 con 0x7fce5c0a0cb0 2024-09-18T13:59:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fce4a7fc640 10 monclient: handle_get_version_reply finishing 1 version 16 2024-09-18T13:59:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdae0001990 con 0x7fdae80a1080 2024-09-18T13:59:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdae6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdae6ffd640 1 -- 172.21.15.45:0/3887253421 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdae0001ec0 con 0x7fdae80a1080 2024-09-18T13:59:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdae6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdae6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdae6ffd640 10 monclient: dump: 2024-09-18T13:59:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.257 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdaedc4f640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.123+0000 7fdaedc4f640 4 set_mon_vals no callback set 2024-09-18T13:59:06.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7fce5ad9d640 10 monclient: get_auth_request con 0x7fce3c040950 auth_method 0 2024-09-18T13:59:06.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7f2c6ffff640 1 -- 172.21.15.45:0/3557093342 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2c68001990 con 0x7f2c780a15b0 2024-09-18T13:59:06.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7f2c6ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7f2c7f6ef640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7f2c7f6ef640 4 set_mon_vals no callback set 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7f2c6ffff640 1 -- 172.21.15.45:0/3557093342 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2c68001c00 con 0x7f2c780a15b0 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7f2c6ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7f2c6ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7f2c6ffff640 10 monclient: dump: 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.260 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.260 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.260 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.260 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.260 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.260 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.260 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.260 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7fdaedc4f640 10 monclient: shutdown 2024-09-18T13:59:06.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7fdaedc4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7fdaedc4f640 1 -- 172.21.15.45:0/3887253421 >> v1:172.21.15.84:6792/0 conn(0x7fdae80a1080 legacy=0x7fdae813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7fdaedc4f640 1 -- 172.21.15.45:0/3887253421 shutdown_connections 2024-09-18T13:59:06.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7fdaedc4f640 1 -- 172.21.15.45:0/3887253421 >> 172.21.15.45:0/3887253421 conn(0x7fdae80a5bb0 msgr2=0x7fdae80a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7fdaedc4f640 1 -- 172.21.15.45:0/3887253421 shutdown_connections 2024-09-18T13:59:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.124+0000 7fdaedc4f640 1 -- 172.21.15.45:0/3887253421 wait complete. 2024-09-18T13:59:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7f2c7f6ef640 10 monclient: shutdown 2024-09-18T13:59:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7f2c7f6ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/3557093342 >> v1:172.21.15.45:6793/0 conn(0x7f2c780a15b0 legacy=0x7f2c78136f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/3557093342 shutdown_connections 2024-09-18T13:59:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/3557093342 >> 172.21.15.45:0/3557093342 conn(0x7f2c780a5bb0 msgr2=0x7f2c780a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 <== osd.5 v1:172.21.15.84:6808/32048 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fce3c0445f0 con 0x7fce3c040950 2024-09-18T13:59:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/3557093342 shutdown_connections 2024-09-18T13:59:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/3557093342 wait complete. 2024-09-18T13:59:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7f4ce719d640 1 -- 172.21.15.45:0/2262310648 --> v1:172.21.15.45:6808/32278 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4cc0002b40 con 0x7f4cc4041640 2024-09-18T13:59:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7fdaedc4f640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7fdaedc4f640 10 monclient: monmap: 2024-09-18T13:59:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.265 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.265 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.265 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.265 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.265 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.265 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.265 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.265 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.265 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7f4cdcff9640 1 -- 172.21.15.45:0/2262310648 <== osd.2 v1:172.21.15.45:6808/32278 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1800444376) 0x7f4cc0002b40 con 0x7f4cc4041640 2024-09-18T13:59:06.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.125+0000 7f2c7f6ef640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f2c7f6ef640 10 monclient: monmap: 2024-09-18T13:59:06.266 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.266 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:06.266 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:06.266 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:06.266 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.267 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:06.267 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.267 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.267 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.267 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.267 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.267 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.267 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.267 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.267 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.267 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7fdaedc4f640 1 Processor -- start 2024-09-18T13:59:06.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7fdaedc4f640 1 -- start start 2024-09-18T13:59:06.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7fdaedc4f640 10 monclient: init 2024-09-18T13:59:06.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f2c7f6ef640 1 Processor -- start 2024-09-18T13:59:06.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f2c7f6ef640 1 -- start start 2024-09-18T13:59:06.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f4ce719d640 1 -- 172.21.15.45:0/2262310648 >> v1:172.21.15.45:6808/32278 conn(0x7f4cc4041640 legacy=0x7f4cc4043a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f2c7f6ef640 10 monclient: init 2024-09-18T13:59:06.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f4ce719d640 1 -- 172.21.15.45:0/2262310648 >> v1:172.21.15.84:6812/31478 conn(0x7f4cc403aa80 legacy=0x7f4cc403cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f4ce719d640 10 monclient: shutdown 2024-09-18T13:59:06.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f4ce719d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f4ce719d640 1 -- 172.21.15.45:0/2262310648 >> v1:172.21.15.84:6789/0 conn(0x7f4ce006e350 legacy=0x7f4ce0131350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f4ce573a640 1 -- 172.21.15.45:0/2262310648 reap_dead start 2024-09-18T13:59:06.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f4ce719d640 1 -- 172.21.15.45:0/2262310648 shutdown_connections 2024-09-18T13:59:06.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f4ce719d640 1 -- 172.21.15.45:0/2262310648 >> 172.21.15.45:0/2262310648 conn(0x7f4ce00a44c0 msgr2=0x7f4ce0135860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f4ce719d640 1 -- 172.21.15.45:0/2262310648 shutdown_connections 2024-09-18T13:59:06.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.126+0000 7f4ce719d640 1 -- 172.21.15.45:0/2262310648 wait complete. 2024-09-18T13:59:06.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdaedc4f640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c7f6ef640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c7f6ef640 10 monclient: _add_conns ranks=[3,2,7,4,6,1,8,0,5] 2024-09-18T13:59:06.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdaedc4f640 10 monclient: _add_conns ranks=[1,4,6,5,8,7,2,3,0] 2024-09-18T13:59:06.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdaedc4f640 10 monclient(hunting): picked mon.noname-f con 0x7fdae806e810 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:06.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c7f6ef640 10 monclient(hunting): picked mon.noname-g con 0x7f2c7809f230 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:06.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c7f6ef640 10 monclient(hunting): picked mon.noname-b con 0x7f2c780a15b0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:06.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c7f6ef640 10 monclient(hunting): picked mon.noname-i con 0x7f2c7813cba0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c7f6ef640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7806f460 con 0x7f2c780a15b0 2024-09-18T13:59:06.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c7f6ef640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c78138570 con 0x7f2c7809f230 2024-09-18T13:59:06.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c7f6ef640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7806a6a0 con 0x7f2c7813cba0 2024-09-18T13:59:06.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c7f6ef640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c7f6ef640 10 monclient(hunting): authenticate will time out at 761.465303s 2024-09-18T13:59:06.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c7cc8a640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f2c780a15b0 0x7f2c7806c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:50286/0 (socket says 172.21.15.45:50286) 2024-09-18T13:59:06.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c7cc8a640 1 -- 172.21.15.45:0/1535182254 learned_addr learned my addr 172.21.15.45:0/1535182254 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdae77fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdae806e810 0x7fdae81417e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:34122/0 (socket says 172.21.15.45:34122) 2024-09-18T13:59:06.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592846128 0 0) 0x7f2c78138570 con 0x7f2c7809f230 2024-09-18T13:59:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdaedc4f640 10 monclient(hunting): picked mon.noname-c con 0x7fdae80a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdaedc4f640 10 monclient(hunting): picked mon.noname-d con 0x7fdae80a1080 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdaedc4f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae809fa10 con 0x7fdae806e810 2024-09-18T13:59:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdaedc4f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae80a2270 con 0x7fdae80a0cb0 2024-09-18T13:59:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdaedc4f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae8122aa0 con 0x7fdae80a1080 2024-09-18T13:59:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdaedc4f640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdaedc4f640 10 monclient(hunting): authenticate will time out at 761.465778s 2024-09-18T13:59:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdadeffd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fdae80a0cb0 0x7fdae8135610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36346/0 (socket says 172.21.15.45:36346) 2024-09-18T13:59:06.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c6d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c6d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c6d7fa640 10 monclient(hunting): my global_id is 4323 2024-09-18T13:59:06.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c5c0034e0 con 0x7f2c7809f230 2024-09-18T13:59:06.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784563478 0 0) 0x7f2c7806f460 con 0x7f2c780a15b0 2024-09-18T13:59:06.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c6d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c6d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c6d7fa640 10 monclient(hunting): my global_id is 4277 2024-09-18T13:59:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c78138570 con 0x7f2c780a15b0 2024-09-18T13:59:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.127+0000 7fdadeffd640 1 -- 172.21.15.45:0/2213133442 learned_addr learned my addr 172.21.15.45:0/2213133442 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293050296 0 0) 0x7fdae809fa10 con 0x7fdae806e810 2024-09-18T13:59:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): my global_id is 4321 2024-09-18T13:59:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdac80034e0 con 0x7fdae806e810 2024-09-18T13:59:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021489515 0 0) 0x7fdae80a2270 con 0x7fdae80a0cb0 2024-09-18T13:59:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): my global_id is 4306 2024-09-18T13:59:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdae809fa10 con 0x7fdae80a0cb0 2024-09-18T13:59:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4140748464 0 0) 0x7f2c78138570 con 0x7f2c780a15b0 2024-09-18T13:59:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c7806f460 con 0x7f2c780a15b0 2024-09-18T13:59:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2c68001b00 con 0x7f2c780a15b0 2024-09-18T13:59:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797684967 0 0) 0x7fdae8122aa0 con 0x7fdae80a1080 2024-09-18T13:59:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): my global_id is 4299 2024-09-18T13:59:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdae80a2270 con 0x7fdae80a1080 2024-09-18T13:59:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7f2c6d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7f2c6d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7f2c6d7fa640 10 monclient(hunting): dump: 2024-09-18T13:59:06.281 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.281 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.281 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.281 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.281 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.281 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.281 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.281 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.281 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.281 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.282 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3450897700 0 0) 0x7fdac80034e0 con 0x7fdae806e810 2024-09-18T13:59:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdae8122aa0 con 0x7fdae806e810 2024-09-18T13:59:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdad00014c0 con 0x7fdae806e810 2024-09-18T13:59:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T13:59:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): dump: 2024-09-18T13:59:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.286 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7f2c6d7fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.128+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 999147187 0 0) 0x7fdae8122aa0 con 0x7fdae806e810 2024-09-18T13:59:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 >> v1:172.21.15.45:6792/0 conn(0x7fdae80a1080 legacy=0x7fdae8149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 >> v1:172.21.15.45:6791/0 conn(0x7fdae80a0cb0 legacy=0x7fdae8135610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 1 monclient: found mon.f 2024-09-18T13:59:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae8120c70 con 0x7fdae806e810 2024-09-18T13:59:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdaedc4f640 5 monclient: authenticate success, global_id 4321 2024-09-18T13:59:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdaedc4f640 10 monclient: _renew_subs 2024-09-18T13:59:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3483088892 0 0) 0x7f2c7806f460 con 0x7f2c780a15b0 2024-09-18T13:59:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 >> v1:172.21.15.84:6792/0 conn(0x7f2c7813cba0 legacy=0x7f2c7806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 >> v1:172.21.15.84:6790/0 conn(0x7f2c7809f230 legacy=0x7f2c7806bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdaedc4f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 1 monclient: found mon.b 2024-09-18T13:59:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7806e6a0 con 0x7f2c780a15b0 2024-09-18T13:59:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdaedc4f640 1 -- 172.21.15.45:0/2213133442 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae8122730 con 0x7fdae806e810 2024-09-18T13:59:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c7f6ef640 5 monclient: authenticate success, global_id 4277 2024-09-18T13:59:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c7f6ef640 10 monclient: _renew_subs 2024-09-18T13:59:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdad00016e0 con 0x7fdae806e810 2024-09-18T13:59:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c7f6ef640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/1535182254 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7806dfe0 con 0x7f2c780a15b0 2024-09-18T13:59:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c7f6ef640 10 monclient: _renew_subs 2024-09-18T13:59:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c7f6ef640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/1535182254 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c78138b30 con 0x7f2c780a15b0 2024-09-18T13:59:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2c68001c60 con 0x7f2c780a15b0 2024-09-18T13:59:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2c680050d0 con 0x7f2c780a15b0 2024-09-18T13:59:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6d7fa640 10 monclient: dump: 2024-09-18T13:59:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.297 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7f2c6f7fe640 4 set_mon_vals no callback set 2024-09-18T13:59:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdad0004d90 con 0x7fdae806e810 2024-09-18T13:59:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdadffff640 10 monclient: dump: 2024-09-18T13:59:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.299 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdae67fc640 4 set_mon_vals no callback set 2024-09-18T13:59:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdaedc4f640 10 monclient: _renew_subs 2024-09-18T13:59:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.129+0000 7fdaedc4f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.130+0000 7fdaedc4f640 1 -- 172.21.15.45:0/2213133442 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae8135d20 con 0x7fdae806e810 2024-09-18T13:59:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.130+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2c680100a0 con 0x7f2c780a15b0 2024-09-18T13:59:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.131+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f2c68046960 con 0x7f2c780a15b0 2024-09-18T13:59:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.131+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fdad000fd60 con 0x7fdae806e810 2024-09-18T13:59:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.131+0000 7f2c7d48b640 10 monclient: get_auth_request con 0x7f2c5c039fa0 auth_method 0 2024-09-18T13:59:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.131+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/1535182254 --> v1:172.21.15.45:6804/32276 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c58005330 con 0x7f2c580015d0 2024-09-18T13:59:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.131+0000 7f2c7dc8c640 10 monclient: get_auth_request con 0x7f2c580015d0 auth_method 0 2024-09-18T13:59:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.131+0000 7fdadeffd640 10 monclient: get_auth_request con 0x7fdac803a060 auth_method 0 2024-09-18T13:59:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.131+0000 7fdaedc4f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.131+0000 7fdaedc4f640 1 -- 172.21.15.45:0/2213133442 --> v1:172.21.15.84:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdae804c4a0 con 0x7fdae806e810 2024-09-18T13:59:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.132+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7fdad0014fd0 con 0x7fdae806e810 2024-09-18T13:59:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.132+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 --> v1:172.21.15.45:6800/32277 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdac8044820 con 0x7fdac8040ba0 2024-09-18T13:59:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.132+0000 7fdae7fff640 10 monclient: get_auth_request con 0x7fdac8040ba0 auth_method 0 2024-09-18T13:59:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.132+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== mon.1 v1:172.21.15.84:6789/0 9 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (unknown 3847591702 0 0) 0x7fdad0014e10 con 0x7fdae806e810 2024-09-18T13:59:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.132+0000 7fdadffff640 10 monclient: handle_get_version_reply finishing 1 version 16 2024-09-18T13:59:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.133+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 <== osd.0 v1:172.21.15.45:6804/32276 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f2c58005330 con 0x7f2c580015d0 2024-09-18T13:59:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.133+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== osd.1 v1:172.21.15.45:6800/32277 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fdac8044820 con 0x7fdac8040ba0 2024-09-18T13:59:06.305 INFO:teuthology.orchestra.run.smithi045.stdout:47244640259 2024-09-18T13:59:06.306 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T13:59:06.307 INFO:teuthology.orchestra.run.smithi045.stdout:47244640259 2024-09-18T13:59:06.308 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T13:59:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.143+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 --> v1:172.21.15.84:6802/32024 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f154c002b40 con 0x7f15500411e0 2024-09-18T13:59:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.143+0000 7f1561ffb640 1 -- 172.21.15.45:0/214362665 <== osd.3 v1:172.21.15.84:6802/32024 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1800444376) 0x7f154c002b40 con 0x7f15500411e0 2024-09-18T13:59:06.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.143+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 >> v1:172.21.15.84:6802/32024 conn(0x7f15500411e0 legacy=0x7f1550043610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.144+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 >> v1:172.21.15.84:6812/31478 conn(0x7f155003a6c0 legacy=0x7f155003cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.144+0000 7f1573e4f640 10 monclient: shutdown 2024-09-18T13:59:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.144+0000 7f1573e4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.144+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 >> v1:172.21.15.84:6792/0 conn(0x7f156c0a0cb0 legacy=0x7f156c146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.144+0000 7f15723ec640 1 -- 172.21.15.45:0/214362665 reap_dead start 2024-09-18T13:59:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.144+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 shutdown_connections 2024-09-18T13:59:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.144+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 >> 172.21.15.45:0/214362665 conn(0x7f156c0a5bb0 msgr2=0x7f156c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.144+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 shutdown_connections 2024-09-18T13:59:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.144+0000 7f1573e4f640 1 -- 172.21.15.45:0/214362665 wait complete. 2024-09-18T13:59:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.146+0000 7fce60ce8640 1 -- 172.21.15.45:0/4267318196 --> v1:172.21.15.84:6808/32048 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fce40002b40 con 0x7fce3c040950 2024-09-18T13:59:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.147+0000 7fce4a7fc640 1 -- 172.21.15.45:0/4267318196 <== osd.5 v1:172.21.15.84:6808/32048 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1800444376) 0x7fce40002b40 con 0x7fce3c040950 2024-09-18T13:59:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.147+0000 7fce60ce8640 1 -- 172.21.15.45:0/4267318196 >> v1:172.21.15.84:6808/32048 conn(0x7fce3c040950 legacy=0x7fce3c042d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.147+0000 7fce60ce8640 1 -- 172.21.15.45:0/4267318196 >> v1:172.21.15.84:6812/31478 conn(0x7fce3c039d40 legacy=0x7fce3c03c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.147+0000 7fce60ce8640 10 monclient: shutdown 2024-09-18T13:59:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.147+0000 7fce60ce8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.147+0000 7fce60ce8640 1 -- 172.21.15.45:0/4267318196 >> v1:172.21.15.45:6793/0 conn(0x7fce5c0a0cb0 legacy=0x7fce5c1467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.147+0000 7fce5ad9d640 1 -- 172.21.15.45:0/4267318196 reap_dead start 2024-09-18T13:59:06.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.147+0000 7fce60ce8640 1 -- 172.21.15.45:0/4267318196 shutdown_connections 2024-09-18T13:59:06.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.147+0000 7fce60ce8640 1 -- 172.21.15.45:0/4267318196 >> 172.21.15.45:0/4267318196 conn(0x7fce5c0a5bb0 msgr2=0x7fce5c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.148+0000 7fce60ce8640 1 -- 172.21.15.45:0/4267318196 shutdown_connections 2024-09-18T13:59:06.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.148+0000 7fce60ce8640 1 -- 172.21.15.45:0/4267318196 wait complete. 2024-09-18T13:59:06.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.152+0000 7fdaedc4f640 1 -- 172.21.15.45:0/2213133442 --> v1:172.21.15.45:6800/32277 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdae8121a80 con 0x7fdac8040ba0 2024-09-18T13:59:06.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.152+0000 7fdadffff640 1 -- 172.21.15.45:0/2213133442 <== osd.1 v1:172.21.15.45:6800/32277 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 839772338) 0x7fdae8121a80 con 0x7fdac8040ba0 2024-09-18T13:59:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.152+0000 7fdaedc4f640 1 -- 172.21.15.45:0/2213133442 >> v1:172.21.15.45:6800/32277 conn(0x7fdac8040ba0 legacy=0x7fdac8042fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.152+0000 7fdaedc4f640 1 -- 172.21.15.45:0/2213133442 >> v1:172.21.15.84:6812/31478 conn(0x7fdac803a060 legacy=0x7fdac803c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.152+0000 7fdaedc4f640 10 monclient: shutdown 2024-09-18T13:59:06.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.152+0000 7fdaedc4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.152+0000 7fdaedc4f640 1 -- 172.21.15.45:0/2213133442 >> v1:172.21.15.84:6789/0 conn(0x7fdae806e810 legacy=0x7fdae81417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.152+0000 7fdae7fff640 1 -- 172.21.15.45:0/2213133442 reap_dead start 2024-09-18T13:59:06.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.153+0000 7fdaedc4f640 1 -- 172.21.15.45:0/2213133442 shutdown_connections 2024-09-18T13:59:06.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.153+0000 7fdaedc4f640 1 -- 172.21.15.45:0/2213133442 >> 172.21.15.45:0/2213133442 conn(0x7fdae80a5bb0 msgr2=0x7fdae80a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.153+0000 7fdaedc4f640 1 -- 172.21.15.45:0/2213133442 shutdown_connections 2024-09-18T13:59:06.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.153+0000 7fdaedc4f640 1 -- 172.21.15.45:0/2213133442 wait complete. 2024-09-18T13:59:06.317 INFO:teuthology.orchestra.run.smithi045.stdout:47244640259 2024-09-18T13:59:06.318 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T13:59:06.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.159+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/1535182254 --> v1:172.21.15.45:6804/32276 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2c58006ef0 con 0x7f2c580015d0 2024-09-18T13:59:06.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.160+0000 7f2c6d7fa640 1 -- 172.21.15.45:0/1535182254 <== osd.0 v1:172.21.15.45:6804/32276 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1800444376) 0x7f2c58006ef0 con 0x7f2c580015d0 2024-09-18T13:59:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.160+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/1535182254 >> v1:172.21.15.45:6804/32276 conn(0x7f2c580015d0 legacy=0x7f2c58003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.160+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/1535182254 >> v1:172.21.15.84:6812/31478 conn(0x7f2c5c039fa0 legacy=0x7f2c5c03c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.160+0000 7f2c7f6ef640 10 monclient: shutdown 2024-09-18T13:59:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.160+0000 7f2c7f6ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.160+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/1535182254 >> v1:172.21.15.45:6790/0 conn(0x7f2c780a15b0 legacy=0x7f2c7806c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.160+0000 7f2c7dc8c640 1 -- 172.21.15.45:0/1535182254 reap_dead start 2024-09-18T13:59:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.160+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/1535182254 shutdown_connections 2024-09-18T13:59:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.160+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/1535182254 >> 172.21.15.45:0/1535182254 conn(0x7f2c780a5bb0 msgr2=0x7f2c780a39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.160+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/1535182254 shutdown_connections 2024-09-18T13:59:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.160+0000 7f2c7f6ef640 1 -- 172.21.15.45:0/1535182254 wait complete. 2024-09-18T13:59:06.321 INFO:teuthology.orchestra.run.smithi045.stdout:47244640259 2024-09-18T13:59:06.321 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T13:59:06.322 INFO:teuthology.orchestra.run.smithi045.stdout:51539607555 2024-09-18T13:59:06.322 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T13:59:06.323 INFO:teuthology.orchestra.run.smithi045.stdout:47244640259 2024-09-18T13:59:06.323 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T13:59:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.457+0000 7fd7e7f5d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.457+0000 7fd7e7f5d640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.458+0000 7fd7e7f5d640 10 monclient: monmap: 2024-09-18T13:59:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:06.458943+0000 2024-09-18T13:59:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:06.458943+0000 2024-09-18T13:59:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.460 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.458+0000 7f86f4834640 10 monclient: get_monmap_and_config 2024-09-18T13:59:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.458+0000 7f86f4834640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.458+0000 7f86f4834640 10 monclient: monmap: 2024-09-18T13:59:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:06.459243+0000 2024-09-18T13:59:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:06.459243+0000 2024-09-18T13:59:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.462 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.458+0000 7fd7e7f5d640 1 Processor -- start 2024-09-18T13:59:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.458+0000 7f86f4834640 1 Processor -- start 2024-09-18T13:59:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7fd7e7f5d640 1 -- start start 2024-09-18T13:59:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7f86f4834640 1 -- start start 2024-09-18T13:59:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7f86f4834640 10 monclient: init 2024-09-18T13:59:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7fd7e7f5d640 10 monclient: init 2024-09-18T13:59:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7f86f4834640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7fd7e7f5d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7fd7e7f5d640 10 monclient: _add_conns ranks=[6,7,3,8,5,4,0,2,1] 2024-09-18T13:59:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7f86f4834640 10 monclient: _add_conns ranks=[3,1,4,6,8,2,7,0,5] 2024-09-18T13:59:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7fd7e7f5d640 10 monclient(hunting): picked mon.noname-d con 0x7fd7e006e810 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7fd7e7f5d640 10 monclient(hunting): picked mon.noname-i con 0x7fd7e00a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7fd7e7f5d640 10 monclient(hunting): picked mon.noname-g con 0x7fd7e00a1080 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7fd7e7f5d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e005ed70 con 0x7fd7e00a1080 2024-09-18T13:59:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7fd7e7f5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e0097620 con 0x7fd7e006e810 2024-09-18T13:59:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e7f5d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e0097340 con 0x7fd7e00a0cb0 2024-09-18T13:59:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e7f5d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e7f5d640 10 monclient(hunting): authenticate will time out at 761.798091s 2024-09-18T13:59:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e54f8640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7fd7e00a0cb0 0x7fd7e0137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:34420/0 (socket says 172.21.15.45:34420) 2024-09-18T13:59:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7f86f4834640 10 monclient(hunting): picked mon.noname-g con 0x7f86f006e810 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7f86f4834640 10 monclient(hunting): picked mon.noname-f con 0x7f86f00a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7f86f4834640 10 monclient(hunting): picked mon.noname-c con 0x7f86f00a1080 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7f86f4834640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f005ed70 con 0x7f86f00a0cb0 2024-09-18T13:59:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.459+0000 7f86f4834640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f0097620 con 0x7f86f006e810 2024-09-18T13:59:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86f4834640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f0097340 con 0x7f86f00a1080 2024-09-18T13:59:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86f4834640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86f4834640 10 monclient(hunting): authenticate will time out at 761.798059s 2024-09-18T13:59:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ee59c640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7f86f006e810 0x7f86f00a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:50986/0 (socket says 172.21.15.45:50986) 2024-09-18T13:59:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ee59c640 1 -- 172.21.15.45:0/3096892013 learned_addr learned my addr 172.21.15.45:0/3096892013 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e54f8640 1 -- 172.21.15.45:0/3922401812 learned_addr learned my addr 172.21.15.45:0/3922401812 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e64fa640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7fd7e00a1080 0x7fd7e013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:50994/0 (socket says 172.21.15.45:50994) 2024-09-18T13:59:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280292629 0 0) 0x7f86f0097620 con 0x7f86f006e810 2024-09-18T13:59:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 10 monclient(hunting): my global_id is 4332 2024-09-18T13:59:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d80034e0 con 0x7f86f006e810 2024-09-18T13:59:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170544641 0 0) 0x7f86f005ed70 con 0x7f86f00a0cb0 2024-09-18T13:59:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 10 monclient(hunting): my global_id is 4330 2024-09-18T13:59:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f0097620 con 0x7f86f00a0cb0 2024-09-18T13:59:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769042885 0 0) 0x7fd7e005ed70 con 0x7fd7e00a1080 2024-09-18T13:59:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 10 monclient(hunting): my global_id is 4341 2024-09-18T13:59:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7cc0034e0 con 0x7fd7e00a1080 2024-09-18T13:59:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262362249 0 0) 0x7fd7e0097340 con 0x7fd7e00a0cb0 2024-09-18T13:59:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 10 monclient(hunting): my global_id is 4300 2024-09-18T13:59:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7e005ed70 con 0x7fd7e00a0cb0 2024-09-18T13:59:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706629605 0 0) 0x7fd7e0097620 con 0x7fd7e006e810 2024-09-18T13:59:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 10 monclient(hunting): my global_id is 4308 2024-09-18T13:59:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7cc003990 con 0x7fd7e006e810 2024-09-18T13:59:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899669122 0 0) 0x7f86f0097340 con 0x7f86f00a1080 2024-09-18T13:59:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.460+0000 7f86ed59a640 10 monclient(hunting): my global_id is 4315 2024-09-18T13:59:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f005ed70 con 0x7f86f00a1080 2024-09-18T13:59:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1861268429 0 0) 0x7fd7e005ed70 con 0x7fd7e00a0cb0 2024-09-18T13:59:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 >> v1:172.21.15.45:6792/0 conn(0x7fd7e006e810 legacy=0x7fd7e00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 >> v1:172.21.15.84:6790/0 conn(0x7fd7e00a1080 legacy=0x7fd7e013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 1 monclient: found mon.noname-i 2024-09-18T13:59:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T13:59:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e00a1660 con 0x7fd7e00a0cb0 2024-09-18T13:59:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd7d00016c0 con 0x7fd7e00a0cb0 2024-09-18T13:59:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:59:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 10 monclient: dump: 2024-09-18T13:59:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e7f5d640 5 monclient: authenticate success, global_id 4300 2024-09-18T13:59:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e7f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 531906202 0 0) 0x7f86d80034e0 con 0x7f86f006e810 2024-09-18T13:59:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 >> v1:172.21.15.45:6791/0 conn(0x7f86f00a1080 legacy=0x7f86f013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 >> v1:172.21.15.84:6789/0 conn(0x7f86f00a0cb0 legacy=0x7f86f0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 1 monclient: found mon.noname-g 2024-09-18T13:59:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T13:59:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f00a1660 con 0x7f86f006e810 2024-09-18T13:59:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd7d0001820 con 0x7fd7e00a0cb0 2024-09-18T13:59:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86e0001830 con 0x7f86f006e810 2024-09-18T13:59:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:59:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 1 -- 172.21.15.45:0/3922401812 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd7d0004860 con 0x7fd7e00a0cb0 2024-09-18T13:59:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e4cf7640 10 monclient: dump: 2024-09-18T13:59:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.479 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e7f5d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7fd7e7f5d640 4 set_mon_vals no callback set 2024-09-18T13:59:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 10 monclient: dump: 2024-09-18T13:59:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.482 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86f4834640 5 monclient: authenticate success, global_id 4332 2024-09-18T13:59:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86f4834640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f86e0001990 con 0x7f86f006e810 2024-09-18T13:59:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86ed59a640 1 -- 172.21.15.45:0/3096892013 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86e0004860 con 0x7f86f006e810 2024-09-18T13:59:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86f4834640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.461+0000 7f86f4834640 4 set_mon_vals no callback set 2024-09-18T13:59:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7fd7e7f5d640 10 monclient: shutdown 2024-09-18T13:59:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7fd7e7f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/3922401812 >> v1:172.21.15.84:6792/0 conn(0x7fd7e00a0cb0 legacy=0x7fd7e0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7f86ed59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7f86ed59a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7f86ed59a640 10 monclient: dump: 2024-09-18T13:59:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.484 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/3922401812 shutdown_connections 2024-09-18T13:59:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/3922401812 >> 172.21.15.45:0/3922401812 conn(0x7fd7e00a5bb0 msgr2=0x7fd7e00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7f86f4834640 10 monclient: shutdown 2024-09-18T13:59:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7f86f4834640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7f86f4834640 1 -- 172.21.15.45:0/3096892013 >> v1:172.21.15.84:6790/0 conn(0x7f86f006e810 legacy=0x7f86f00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/3922401812 shutdown_connections 2024-09-18T13:59:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7f86f4834640 1 -- 172.21.15.45:0/3096892013 shutdown_connections 2024-09-18T13:59:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7f86f4834640 1 -- 172.21.15.45:0/3096892013 >> 172.21.15.45:0/3096892013 conn(0x7f86f00a5bb0 msgr2=0x7f86f00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/3922401812 wait complete. 2024-09-18T13:59:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7f86f4834640 1 -- 172.21.15.45:0/3096892013 shutdown_connections 2024-09-18T13:59:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.462+0000 7f86f4834640 1 -- 172.21.15.45:0/3096892013 wait complete. 2024-09-18T13:59:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.463+0000 7fd7e7f5d640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.463+0000 7fd7e7f5d640 10 monclient: monmap: 2024-09-18T13:59:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.488 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.463+0000 7f86f4834640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.463+0000 7f86f4834640 10 monclient: monmap: 2024-09-18T13:59:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.490 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.464+0000 7fd7e7f5d640 1 Processor -- start 2024-09-18T13:59:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.464+0000 7fd7e7f5d640 1 -- start start 2024-09-18T13:59:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.464+0000 7fd7e7f5d640 10 monclient: init 2024-09-18T13:59:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.464+0000 7f86f4834640 1 Processor -- start 2024-09-18T13:59:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.464+0000 7f86f4834640 1 -- start start 2024-09-18T13:59:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.464+0000 7f86f4834640 10 monclient: init 2024-09-18T13:59:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7fd7e7f5d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7fd7e7f5d640 10 monclient: _add_conns ranks=[0,7,8,1,5,6,4,3,2] 2024-09-18T13:59:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7fd7e7f5d640 10 monclient(hunting): picked mon.noname-a con 0x7fd7e006e810 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7fd7e7f5d640 10 monclient(hunting): picked mon.noname-i con 0x7fd7e00a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7fd7e5cf9640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd7e006e810 0x7fd7e013d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49392/0 (socket says 172.21.15.45:49392) 2024-09-18T13:59:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7f86f4834640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7f86f4834640 10 monclient: _add_conns ranks=[7,6,1,2,8,0,5,4,3] 2024-09-18T13:59:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7fd7e5cf9640 1 -- 172.21.15.45:0/1862680686 learned_addr learned my addr 172.21.15.45:0/1862680686 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7f86f4834640 10 monclient(hunting): picked mon.noname-i con 0x7f86f006e810 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7f86f4834640 10 monclient(hunting): picked mon.noname-d con 0x7f86f00a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7fd7e7f5d640 10 monclient(hunting): picked mon.noname-e con 0x7fd7e00a1080 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7f86ee59c640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f86f006e810 0x7f86f013d510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:34434/0 (socket says 172.21.15.45:34434) 2024-09-18T13:59:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7f86ee59c640 1 -- 172.21.15.45:0/286083967 learned_addr learned my addr 172.21.15.45:0/286083967 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e0122ea0 con 0x7fd7e006e810 2024-09-18T13:59:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.465+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e013dc90 con 0x7fd7e00a0cb0 2024-09-18T13:59:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e01228e0 con 0x7fd7e00a1080 2024-09-18T13:59:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7e7f5d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7e7f5d640 10 monclient(hunting): authenticate will time out at 761.804209s 2024-09-18T13:59:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638441715 0 0) 0x7fd7e013dc90 con 0x7fd7e00a0cb0 2024-09-18T13:59:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7f86f4834640 10 monclient(hunting): picked mon.noname-f con 0x7f86f00a1080 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f009fa10 con 0x7f86f00a1080 2024-09-18T13:59:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): my global_id is 4309 2024-09-18T13:59:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7c40034e0 con 0x7fd7e00a0cb0 2024-09-18T13:59:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822118268 0 0) 0x7fd7e0122ea0 con 0x7fd7e006e810 2024-09-18T13:59:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): my global_id is 4302 2024-09-18T13:59:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7e013dc90 con 0x7fd7e006e810 2024-09-18T13:59:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348550117 0 0) 0x7fd7e01228e0 con 0x7fd7e00a1080 2024-09-18T13:59:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): my global_id is 4283 2024-09-18T13:59:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7e0122ea0 con 0x7fd7e00a1080 2024-09-18T13:59:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f013dc20 con 0x7f86f00a0cb0 2024-09-18T13:59:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f00a2270 con 0x7f86f006e810 2024-09-18T13:59:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7f86f4834640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1600682342 0 0) 0x7fd7e013dc90 con 0x7fd7e006e810 2024-09-18T13:59:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd7e01228e0 con 0x7fd7e006e810 2024-09-18T13:59:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7f86f4834640 10 monclient(hunting): authenticate will time out at 761.804448s 2024-09-18T13:59:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489885270 0 0) 0x7f86f00a2270 con 0x7f86f006e810 2024-09-18T13:59:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7f86de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7f86de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7f86de7fc640 10 monclient(hunting): my global_id is 4318 2024-09-18T13:59:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d00034e0 con 0x7f86f006e810 2024-09-18T13:59:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd7dc001c60 con 0x7fd7e006e810 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): dump: 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 833679421 0 0) 0x7fd7e0122ea0 con 0x7fd7e00a1080 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd7e013dc90 con 0x7fd7e00a1080 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.466+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3623837196 0 0) 0x7fd7e01228e0 con 0x7fd7e006e810 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 >> v1:172.21.15.45:6793/0 conn(0x7fd7e00a1080 legacy=0x7fd7e0149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 >> v1:172.21.15.84:6792/0 conn(0x7fd7e00a0cb0 legacy=0x7fd7e01467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 1 monclient: found mon.a 2024-09-18T13:59:06.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:06.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e0120480 con 0x7fd7e006e810 2024-09-18T13:59:06.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1803118172 0 0) 0x7f86d00034e0 con 0x7f86f006e810 2024-09-18T13:59:06.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86f00a2270 con 0x7f86f006e810 2024-09-18T13:59:06.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934815210 0 0) 0x7f86f013dc20 con 0x7f86f00a0cb0 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient(hunting): my global_id is 4317 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d00034e0 con 0x7f86f00a0cb0 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86e00016f0 con 0x7f86f006e810 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.504 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7e7f5d640 5 monclient: authenticate success, global_id 4302 2024-09-18T13:59:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7e7f5d640 10 monclient: _renew_subs 2024-09-18T13:59:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7e7f5d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e0122600 con 0x7fd7e006e810 2024-09-18T13:59:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd7dc004b10 con 0x7fd7e006e810 2024-09-18T13:59:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd7dc0017a0 con 0x7fd7e006e810 2024-09-18T13:59:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d5ffb640 10 monclient: dump: 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.507 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397183516 0 0) 0x7f86f009fa10 con 0x7f86f00a1080 2024-09-18T13:59:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient(hunting): my global_id is 4339 2024-09-18T13:59:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f013dc20 con 0x7f86f00a1080 2024-09-18T13:59:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 973873723 0 0) 0x7f86f00a2270 con 0x7f86f006e810 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 >> v1:172.21.15.45:6792/0 conn(0x7f86f00a0cb0 legacy=0x7f86f0146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 >> v1:172.21.15.84:6789/0 conn(0x7f86f00a1080 legacy=0x7f86f0149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 1 monclient: found mon.i 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f013ea30 con 0x7f86f006e810 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86f4834640 5 monclient: authenticate success, global_id 4318 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86f4834640 10 monclient: _renew_subs 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86f4834640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f01225b0 con 0x7f86f006e810 2024-09-18T13:59:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.468+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f86e0004d90 con 0x7f86f006e810 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.468+0000 7f86de7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.468+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86e0005240 con 0x7f86f006e810 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.468+0000 7f86de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.468+0000 7f86de7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.468+0000 7f86de7fc640 10 monclient: dump: 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.510 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.511 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.511 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.511 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.511 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.511 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.511 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.511 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.468+0000 7f86ecd99640 4 set_mon_vals no callback set 2024-09-18T13:59:06.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.467+0000 7fd7d7fff640 4 set_mon_vals no callback set 2024-09-18T13:59:06.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.468+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd7dc00fe30 con 0x7fd7e006e810 2024-09-18T13:59:06.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.468+0000 7f86f4834640 10 monclient: _renew_subs 2024-09-18T13:59:06.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.468+0000 7f86f4834640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:06.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.468+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f0120250 con 0x7f86f006e810 2024-09-18T13:59:06.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.469+0000 7fd7e7f5d640 10 monclient: _renew_subs 2024-09-18T13:59:06.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.469+0000 7fd7e7f5d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:06.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.469+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e0121730 con 0x7fd7e006e810 2024-09-18T13:59:06.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.469+0000 7fd7e54f8640 10 monclient: get_auth_request con 0x7fd7c403a130 auth_method 0 2024-09-18T13:59:06.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.469+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f86e0010210 con 0x7f86f006e810 2024-09-18T13:59:06.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.469+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7fd7dc015070 con 0x7fd7e006e810 2024-09-18T13:59:06.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.469+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f86e0046d90 con 0x7f86f006e810 2024-09-18T13:59:06.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.470+0000 7f86edd9b640 10 monclient: get_auth_request con 0x7f86d0039e20 auth_method 0 2024-09-18T13:59:06.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.470+0000 7f86f4834640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.470+0000 7f86f4834640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.470+0000 7f86f4834640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:06.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.470+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f004c4a0 con 0x7f86f006e810 2024-09-18T13:59:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.470+0000 7fd7e7f5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.470+0000 7fd7e7f5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.470+0000 7fd7e7f5d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.470+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e004c4a0 con 0x7fd7e006e810 2024-09-18T13:59:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.474+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fd7dc0152d0 con 0x7fd7e006e810 2024-09-18T13:59:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.474+0000 7fd7d5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.474+0000 7fd7d5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.474+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 <== mon.7 v1:172.21.15.84:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f86e0015640 con 0x7f86f006e810 2024-09-18T13:59:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.475+0000 7f86de7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.475+0000 7f86de7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:06.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.499+0000 7fe0a579d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:06.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.499+0000 7fe0a579d640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.499+0000 7fe0a579d640 10 monclient: monmap: 2024-09-18T13:59:06.517 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.517 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.517 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:06.500810+0000 2024-09-18T13:59:06.517 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:06.500810+0000 2024-09-18T13:59:06.517 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.518 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.500+0000 7fda1071d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:06.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.500+0000 7fda1071d640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.500+0000 7fda1071d640 10 monclient: monmap: 2024-09-18T13:59:06.519 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.519 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.519 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:06.501256+0000 2024-09-18T13:59:06.519 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:06.501256+0000 2024-09-18T13:59:06.519 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.519 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.519 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.519 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.520 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.520 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.520 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.520 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.520 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.520 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.520 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.520 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.500+0000 7fe0a579d640 1 Processor -- start 2024-09-18T13:59:06.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.500+0000 7fe0a579d640 1 -- start start 2024-09-18T13:59:06.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.500+0000 7fe0a579d640 10 monclient: init 2024-09-18T13:59:06.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fda1071d640 1 Processor -- start 2024-09-18T13:59:06.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fda1071d640 1 -- start start 2024-09-18T13:59:06.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fda1071d640 10 monclient: init 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fe0a579d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fe0a579d640 10 monclient: _add_conns ranks=[2,8,0,3,6,1,7,4,5] 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fe0a579d640 10 monclient(hunting): picked mon.noname-b con 0x7fe0a006e810 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fe0a579d640 10 monclient(hunting): picked mon.noname-e con 0x7fe0a00a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fe0a579d640 10 monclient(hunting): picked mon.noname-a con 0x7fe0a00a1080 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fe0a579d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a005ed70 con 0x7fe0a00a1080 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fe0a579d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a0097620 con 0x7fe0a006e810 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fe0a579d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a0097340 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fe0a579d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fe0a579d640 10 monclient(hunting): authenticate will time out at 761.839697s 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fe09e7fc640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fe0a00a0cb0 0x7fe0a0137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:56710/0 (socket says 172.21.15.45:56710) 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.501+0000 7fe09e7fc640 1 -- 172.21.15.45:0/2843144339 learned_addr learned my addr 172.21.15.45:0/2843144339 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557695619 0 0) 0x7fe0a0097620 con 0x7fe0a006e810 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient(hunting): my global_id is 4286 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0880034e0 con 0x7fe0a006e810 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3748226460 0 0) 0x7fe0a005ed70 con 0x7fe0a00a1080 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient(hunting): my global_id is 4311 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe088004860 con 0x7fe0a00a1080 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402044828 0 0) 0x7fe0a0097340 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient(hunting): my global_id is 4292 2024-09-18T13:59:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a005ed70 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fda1071d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fda1071d640 10 monclient: _add_conns ranks=[0,7,3,5,1,4,8,2,6] 2024-09-18T13:59:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fda1071d640 10 monclient(hunting): picked mon.noname-a con 0x7fda0806e810 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fda1071d640 10 monclient(hunting): picked mon.noname-i con 0x7fda081330e0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1678613314 0 0) 0x7fe088004860 con 0x7fe0a00a1080 2024-09-18T13:59:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 >> v1:172.21.15.45:6793/0 conn(0x7fe0a00a0cb0 legacy=0x7fe0a0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 >> v1:172.21.15.45:6790/0 conn(0x7fe0a006e810 legacy=0x7fe0a00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 monclient: found mon.noname-a 2024-09-18T13:59:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T13:59:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a00a1660 con 0x7fe0a00a1080 2024-09-18T13:59:06.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe0a579d640 5 monclient: authenticate success, global_id 4311 2024-09-18T13:59:06.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe0a579d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fda0e4b9640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fda0806e810 0x7fda081329d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49420/0 (socket says 172.21.15.45:49420) 2024-09-18T13:59:06.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe098001b90 con 0x7fe0a00a1080 2024-09-18T13:59:06.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fda0e4b9640 1 -- 172.21.15.45:0/974764276 learned_addr learned my addr 172.21.15.45:0/974764276 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:06.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient: dump: 2024-09-18T13:59:06.528 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.528 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.529 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.530 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe0a579d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fda1071d640 10 monclient(hunting): picked mon.noname-g con 0x7fda08096a20 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:06.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe098001d70 con 0x7fe0a00a1080 2024-09-18T13:59:06.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 1 -- 172.21.15.45:0/2843144339 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe0980022a0 con 0x7fe0a00a1080 2024-09-18T13:59:06.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:06.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe09dffb640 10 monclient: dump: 2024-09-18T13:59:06.531 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.531 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.531 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.531 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.531 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.531 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.531 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.532 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.532 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.532 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.532 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.532 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.532 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.532 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.532 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.532 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe0a579d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:06.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fe0a579d640 4 set_mon_vals no callback set 2024-09-18T13:59:06.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.502+0000 7fda1071d640 1 -- 172.21.15.45:0/974764276 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0805ed70 con 0x7fda0806e810 2024-09-18T13:59:06.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda1071d640 1 -- 172.21.15.45:0/974764276 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0809f4e0 con 0x7fda08096a20 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda1071d640 1 -- 172.21.15.45:0/974764276 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0806ebe0 con 0x7fda081330e0 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda1071d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda1071d640 10 monclient(hunting): authenticate will time out at 761.841288s 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804158778 0 0) 0x7fda0809f4e0 con 0x7fda08096a20 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient(hunting): my global_id is 4350 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f40034e0 con 0x7fda08096a20 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717210115 0 0) 0x7fda0806ebe0 con 0x7fda081330e0 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient(hunting): my global_id is 4327 2024-09-18T13:59:06.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda0809f4e0 con 0x7fda081330e0 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286493232 0 0) 0x7fda0805ed70 con 0x7fda0806e810 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient(hunting): my global_id is 4320 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda0806ebe0 con 0x7fda0806e810 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2313258007 0 0) 0x7fd9f40034e0 con 0x7fda08096a20 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 >> v1:172.21.15.84:6792/0 conn(0x7fda081330e0 legacy=0x7fda081377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 >> v1:172.21.15.45:6789/0 conn(0x7fda0806e810 legacy=0x7fda081329d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 monclient: found mon.noname-g 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T13:59:06.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda08096fd0 con 0x7fda08096a20 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fda00001bb0 con 0x7fda08096a20 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient: dump: 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda1071d640 5 monclient: authenticate success, global_id 4350 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda1071d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fda00001d90 con 0x7fda08096a20 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 1 -- 172.21.15.45:0/974764276 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fda000022c0 con 0x7fda08096a20 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda0d4b7640 10 monclient: dump: 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.540 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda1071d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fda1071d640 4 set_mon_vals no callback set 2024-09-18T13:59:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fe0a579d640 10 monclient: shutdown 2024-09-18T13:59:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fe0a579d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.503+0000 7fe0a579d640 1 -- 172.21.15.45:0/2843144339 >> v1:172.21.15.45:6789/0 conn(0x7fe0a00a1080 legacy=0x7fe0a013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.504+0000 7fe0a579d640 1 -- 172.21.15.45:0/2843144339 shutdown_connections 2024-09-18T13:59:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.504+0000 7fe0a579d640 1 -- 172.21.15.45:0/2843144339 >> 172.21.15.45:0/2843144339 conn(0x7fe0a00a5bb0 msgr2=0x7fe0a00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.504+0000 7fe0a579d640 1 -- 172.21.15.45:0/2843144339 shutdown_connections 2024-09-18T13:59:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.504+0000 7fe0a579d640 1 -- 172.21.15.45:0/2843144339 wait complete. 2024-09-18T13:59:06.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.504+0000 7fda1071d640 10 monclient: shutdown 2024-09-18T13:59:06.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.504+0000 7fda1071d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.504+0000 7fda1071d640 1 -- 172.21.15.45:0/974764276 >> v1:172.21.15.84:6790/0 conn(0x7fda08096a20 legacy=0x7fda0813af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.505+0000 7fda1071d640 1 -- 172.21.15.45:0/974764276 shutdown_connections 2024-09-18T13:59:06.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.505+0000 7fda1071d640 1 -- 172.21.15.45:0/974764276 >> 172.21.15.45:0/974764276 conn(0x7fda080a5c20 msgr2=0x7fda080a6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.505+0000 7fda1071d640 1 -- 172.21.15.45:0/974764276 shutdown_connections 2024-09-18T13:59:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.505+0000 7fda1071d640 1 -- 172.21.15.45:0/974764276 wait complete. 2024-09-18T13:59:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.505+0000 7fe0a579d640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.505+0000 7fe0a579d640 10 monclient: monmap: 2024-09-18T13:59:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.544 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.505+0000 7fda1071d640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.506+0000 7fe0a579d640 1 Processor -- start 2024-09-18T13:59:06.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.506+0000 7fda1071d640 10 monclient: monmap: 2024-09-18T13:59:06.545 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.545 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:06.545 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:06.545 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:06.545 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.546 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.506+0000 7fe0a579d640 1 -- start start 2024-09-18T13:59:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.506+0000 7fe0a579d640 10 monclient: init 2024-09-18T13:59:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.506+0000 7fda1071d640 1 Processor -- start 2024-09-18T13:59:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.506+0000 7fda1071d640 1 -- start start 2024-09-18T13:59:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.506+0000 7fdbb591d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.506+0000 7fdbb591d640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.506+0000 7fdbb591d640 10 monclient: monmap: 2024-09-18T13:59:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:06.507621+0000 2024-09-18T13:59:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:06.507621+0000 2024-09-18T13:59:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.549 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.549 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.549 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.549 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.549 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.549 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.549 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.549 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.506+0000 7fda1071d640 10 monclient: init 2024-09-18T13:59:06.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fe0a579d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fe0a579d640 10 monclient: _add_conns ranks=[3,1,4,7,2,5,8,0,6] 2024-09-18T13:59:06.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fe0a579d640 10 monclient(hunting): picked mon.noname-g con 0x7fe0a006e810 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:06.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fda1071d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fda1071d640 10 monclient: _add_conns ranks=[0,6,3,1,2,7,8,4,5] 2024-09-18T13:59:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fe09effd640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7fe0a006e810 0x7fe0a013d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:51010/0 (socket says 172.21.15.45:51010) 2024-09-18T13:59:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fdbb591d640 1 Processor -- start 2024-09-18T13:59:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fda1071d640 10 monclient(hunting): picked mon.noname-a con 0x7fda0806e810 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fda1071d640 10 monclient(hunting): picked mon.noname-d con 0x7fda08096a20 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fda1071d640 10 monclient(hunting): picked mon.noname-g con 0x7fda081330e0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fda1071d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0809f9f0 con 0x7fda0806e810 2024-09-18T13:59:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fda1071d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda080a2320 con 0x7fda081330e0 2024-09-18T13:59:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fda1071d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda081311b0 con 0x7fda08096a20 2024-09-18T13:59:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fda1071d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fda1071d640 10 monclient(hunting): authenticate will time out at 761.845376s 2024-09-18T13:59:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fda0e4b9640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fda0806e810 0x7fda08130410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49436/0 (socket says 172.21.15.45:49436) 2024-09-18T13:59:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fda0e4b9640 1 -- 172.21.15.45:0/2690182369 learned_addr learned my addr 172.21.15.45:0/2690182369 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fe09effd640 1 -- 172.21.15.45:0/536475590 learned_addr learned my addr 172.21.15.45:0/536475590 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fe0a579d640 10 monclient(hunting): picked mon.noname-f con 0x7fe0a00a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:06.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667026652 0 0) 0x7fda0809f9f0 con 0x7fda0806e810 2024-09-18T13:59:06.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fd9fe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fd9fe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fe0a579d640 10 monclient(hunting): picked mon.noname-c con 0x7fe0a00a1080 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fd9fe7fc640 10 monclient(hunting): my global_id is 4329 2024-09-18T13:59:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9ec0034e0 con 0x7fda0806e810 2024-09-18T13:59:06.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a0122d80 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a013db70 con 0x7fe0a006e810 2024-09-18T13:59:06.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184677445 0 0) 0x7fda080a2320 con 0x7fda081330e0 2024-09-18T13:59:06.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient(hunting): my global_id is 4359 2024-09-18T13:59:06.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda0809f9f0 con 0x7fda081330e0 2024-09-18T13:59:06.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2998252885 0 0) 0x7fd9ec0034e0 con 0x7fda0806e810 2024-09-18T13:59:06.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7f7075234640 10 monclient: get_monmap_and_config 2024-09-18T13:59:06.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7f7075234640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.507+0000 7f7075234640 10 monclient: monmap: 2024-09-18T13:59:06.556 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.556 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.556 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:06.508426+0000 2024-09-18T13:59:06.556 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:06.508426+0000 2024-09-18T13:59:06.556 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.556 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.557 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.557 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.557 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.557 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.557 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.557 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.557 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.557 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.557 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.557 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fda080a2320 con 0x7fda0806e810 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fda04001aa0 con 0x7fda0806e810 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.558 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.559 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.559 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.559 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.559 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.559 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.559 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.559 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.559 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:06.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 649773376 0 0) 0x7fe0a0122d80 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1726044159 0 0) 0x7fda0809f9f0 con 0x7fda081330e0 2024-09-18T13:59:06.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a01227c0 con 0x7fe0a00a1080 2024-09-18T13:59:06.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd9ec0034e0 con 0x7fda081330e0 2024-09-18T13:59:06.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe0a579d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fda00002240 con 0x7fda081330e0 2024-09-18T13:59:06.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe0a579d640 10 monclient(hunting): authenticate will time out at 761.846705s 2024-09-18T13:59:06.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 10 monclient(hunting): my global_id is 4348 2024-09-18T13:59:06.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7f7075234640 1 Processor -- start 2024-09-18T13:59:06.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe08c0034e0 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311758926 0 0) 0x7fe0a013db70 con 0x7fe0a006e810 2024-09-18T13:59:06.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 10 monclient(hunting): my global_id is 4368 2024-09-18T13:59:06.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a0122d80 con 0x7fe0a006e810 2024-09-18T13:59:06.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364337351 0 0) 0x7fe0a01227c0 con 0x7fe0a00a1080 2024-09-18T13:59:06.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 10 monclient(hunting): my global_id is 4324 2024-09-18T13:59:06.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a013db70 con 0x7fe0a00a1080 2024-09-18T13:59:06.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:06.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:06.564 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.564 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.564 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.564 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.564 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.564 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1373455302 0 0) 0x7fda080a2320 con 0x7fda0806e810 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 >> v1:172.21.15.45:6792/0 conn(0x7fda08096a20 legacy=0x7fda081467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 >> v1:172.21.15.84:6790/0 conn(0x7fda081330e0 legacy=0x7fda08149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7f7075234640 1 -- start start 2024-09-18T13:59:06.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7f7075234640 10 monclient: init 2024-09-18T13:59:06.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7f7075234640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7f7075234640 10 monclient: _add_conns ranks=[2,5,7,0,4,6,1,8,3] 2024-09-18T13:59:06.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7f7075234640 10 monclient(hunting): picked mon.noname-b con 0x7f707012e5a0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:06.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f7075234640 10 monclient(hunting): picked mon.noname-h con 0x7f707006a300 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:06.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1358149454 0 0) 0x7fe08c0034e0 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0a01227c0 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe094001ca0 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.567 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 772663745 0 0) 0x7fe0a013db70 con 0x7fe0a00a1080 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe08c0034e0 con 0x7fe0a00a1080 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe0980013a0 con 0x7fe0a00a1080 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:06.568 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.569 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 908615948 0 0) 0x7fe0a0122d80 con 0x7fe0a006e810 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0a013db70 con 0x7fe0a006e810 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe090001d90 con 0x7fe0a006e810 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.570 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3930602421 0 0) 0x7fe0a01227c0 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 >> v1:172.21.15.45:6791/0 conn(0x7fe0a00a1080 legacy=0x7fe0a0149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 >> v1:172.21.15.84:6790/0 conn(0x7fe0a006e810 legacy=0x7fe0a013d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 monclient: found mon.f 2024-09-18T13:59:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a0120280 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fe086ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe0a579d640 5 monclient: authenticate success, global_id 4348 2024-09-18T13:59:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe0a579d640 10 monclient: _renew_subs 2024-09-18T13:59:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe0a579d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a01224e0 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe0a579d640 10 monclient: _renew_subs 2024-09-18T13:59:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe0a579d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a0121610 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fdbb591d640 1 -- start start 2024-09-18T13:59:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fdbb591d640 10 monclient: init 2024-09-18T13:59:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fdbb591d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fdbb591d640 10 monclient: _add_conns ranks=[0,8,1,7,3,5,4,6,2] 2024-09-18T13:59:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fdbb591d640 10 monclient(hunting): picked mon.noname-a con 0x7fdbb00670e0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fdbb591d640 10 monclient(hunting): picked mon.noname-e con 0x7fdbb006dd80 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fdbb591d640 10 monclient(hunting): picked mon.noname-f con 0x7fdbb013cc30 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fdbb591d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb005ed70 con 0x7fdbb00670e0 2024-09-18T13:59:06.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fdbb591d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb0067cf0 con 0x7fdbb013cc30 2024-09-18T13:59:06.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fdbb591d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb009f1a0 con 0x7fdbb006dd80 2024-09-18T13:59:06.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f7075234640 10 monclient(hunting): picked mon.noname-i con 0x7f707006eb60 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f706e59c640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f707006a300 0x7f707006e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:35190/0 (socket says 172.21.15.45:35190) 2024-09-18T13:59:06.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f706e59c640 1 -- 172.21.15.45:0/2847233970 learned_addr learned my addr 172.21.15.45:0/2847233970 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 monclient: found mon.a 2024-09-18T13:59:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda08142370 con 0x7fda0806e810 2024-09-18T13:59:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fd9fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fda1071d640 5 monclient: authenticate success, global_id 4329 2024-09-18T13:59:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fda1071d640 10 monclient: _renew_subs 2024-09-18T13:59:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fda1071d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fda1071d640 1 -- 172.21.15.45:0/2690182369 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0811f600 con 0x7fda0806e810 2024-09-18T13:59:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fda1071d640 10 monclient: _renew_subs 2024-09-18T13:59:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fda1071d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.508+0000 7fda1071d640 1 -- 172.21.15.45:0/2690182369 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fda08133970 con 0x7fda0806e810 2024-09-18T13:59:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fdbb591d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.509+0000 7fdbb591d640 10 monclient(hunting): authenticate will time out at 761.847976s 2024-09-18T13:59:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fdbae7fc640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fdbb006dd80 0x7fdbb006e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:56716/0 (socket says 172.21.15.45:56716) 2024-09-18T13:59:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fdbae7fc640 1 -- 172.21.15.45:0/395633524 learned_addr learned my addr 172.21.15.45:0/395633524 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f7075234640 1 -- 172.21.15.45:0/2847233970 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707005ed70 con 0x7f707012e5a0 2024-09-18T13:59:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f7075234640 1 -- 172.21.15.45:0/2847233970 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707006b950 con 0x7f707006a300 2024-09-18T13:59:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f7075234640 1 -- 172.21.15.45:0/2847233970 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707006b670 con 0x7f707006eb60 2024-09-18T13:59:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f7075234640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f7075234640 10 monclient(hunting): authenticate will time out at 761.848697s 2024-09-18T13:59:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965809333 0 0) 0x7f707006b950 con 0x7f707006a300 2024-09-18T13:59:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f706dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f706dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f706dd9b640 10 monclient(hunting): my global_id is 4271 2024-09-18T13:59:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70580034e0 con 0x7f707006a300 2024-09-18T13:59:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938356545 0 0) 0x7f707006b670 con 0x7f707006eb60 2024-09-18T13:59:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient(hunting): my global_id is 4336 2024-09-18T13:59:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f707006b950 con 0x7f707006eb60 2024-09-18T13:59:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fdbadffb640 1 -- 172.21.15.45:0/395633524 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369013867 0 0) 0x7fdbb009f1a0 con 0x7fdbb006dd80 2024-09-18T13:59:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fdbadffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fdbadffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fdbadffb640 10 monclient(hunting): my global_id is 4301 2024-09-18T13:59:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fdbadffb640 1 -- 172.21.15.45:0/395633524 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb980034e0 con 0x7fdbb006dd80 2024-09-18T13:59:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 1 -- 172.21.15.45:0/395633524 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978884236 0 0) 0x7fdbb005ed70 con 0x7fdbb00670e0 2024-09-18T13:59:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 10 monclient(hunting): my global_id is 4338 2024-09-18T13:59:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 1 -- 172.21.15.45:0/395633524 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb009f1a0 con 0x7fdbb00670e0 2024-09-18T13:59:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 1 -- 172.21.15.45:0/395633524 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3675353863 0 0) 0x7fdb980034e0 con 0x7fdbb006dd80 2024-09-18T13:59:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 1 -- 172.21.15.45:0/395633524 >> v1:172.21.15.84:6789/0 conn(0x7fdbb013cc30 legacy=0x7fdbb013f010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-18T13:59:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 1 -- 172.21.15.45:0/395633524 >> v1:172.21.15.45:6789/0 conn(0x7fdbb00670e0 legacy=0x7fdbb006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 1 monclient: found mon.noname-e 2024-09-18T13:59:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 1 -- 172.21.15.45:0/395633524 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb00674b0 con 0x7fdbb006dd80 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 1 -- 172.21.15.45:0/395633524 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdba4001850 con 0x7fdbb006dd80 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbadffb640 10 monclient: dump: 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.582 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbb591d640 5 monclient: authenticate success, global_id 4301 2024-09-18T13:59:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fdbb591d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965419357 0 0) 0x7f707005ed70 con 0x7f707012e5a0 2024-09-18T13:59:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient(hunting): my global_id is 4295 2024-09-18T13:59:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f707006b670 con 0x7f707012e5a0 2024-09-18T13:59:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2573931248 0 0) 0x7f70580034e0 con 0x7f707006a300 2024-09-18T13:59:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 >> v1:172.21.15.84:6792/0 conn(0x7f707006eb60 legacy=0x7f70701366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 >> v1:172.21.15.45:6790/0 conn(0x7f707012e5a0 legacy=0x7f707012e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 1 monclient: found mon.noname-h 2024-09-18T13:59:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070137e10 con 0x7f707006a300 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f7075234640 5 monclient: authenticate success, global_id 4271 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f7075234640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7064001760 con 0x7f707006a300 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient: dump: 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f7075234640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7064001980 con 0x7f707006a300 2024-09-18T13:59:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f706dd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fda04001ce0 con 0x7fda0806e810 2024-09-18T13:59:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fd9fe7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fda04004d90 con 0x7fda0806e810 2024-09-18T13:59:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fd9fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fda0ccb6640 4 set_mon_vals no callback set 2024-09-18T13:59:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fd9fe7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fd9fe7fc640 10 monclient: dump: 2024-09-18T13:59:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.587 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fda0400fda0 con 0x7fda0806e810 2024-09-18T13:59:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7fda04046a40 con 0x7fda0806e810 2024-09-18T13:59:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fd9f3fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe094001250 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe086ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe094004e10 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe086ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe086ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe086ffd640 10 monclient: dump: 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.590 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.590 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.590 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.510+0000 7fe09d7fa640 4 set_mon_vals no callback set 2024-09-18T13:59:06.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fe0a579d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fe0a579d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fe0a579d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fe09400fde0 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe080005010 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fd9f3fff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fd9f3fff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:06.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fd9f3fff640 1 -- 172.21.15.45:0/2690182369 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e8005010 con 0x7fda0806e810 2024-09-18T13:59:06.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f7075234640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:06.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.511+0000 7f7075234640 4 set_mon_vals no callback set 2024-09-18T13:59:06.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fda0dcb8640 10 monclient: get_auth_request con 0x7fd9ec03a3f0 auth_method 0 2024-09-18T13:59:06.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7fe0940469a0 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fe09effd640 10 monclient: get_auth_request con 0x7fe08c03a9a0 auth_method 0 2024-09-18T13:59:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fdbadffb640 1 -- 172.21.15.45:0/395633524 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdba40019b0 con 0x7fdbb006dd80 2024-09-18T13:59:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fdbadffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fdbadffb640 1 -- 172.21.15.45:0/395633524 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdba4004ba0 con 0x7fdbb006dd80 2024-09-18T13:59:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fdbadffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fdbadffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fdbadffb640 10 monclient: dump: 2024-09-18T13:59:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fdbb591d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7fdbb591d640 4 set_mon_vals no callback set 2024-09-18T13:59:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7f706dd9b640 1 -- 172.21.15.45:0/2847233970 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7064004860 con 0x7f707006a300 2024-09-18T13:59:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7f706dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7f706dd9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7f706dd9b640 10 monclient: dump: 2024-09-18T13:59:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7f7075234640 10 monclient: shutdown 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7f7075234640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.512+0000 7f7075234640 1 -- 172.21.15.45:0/2847233970 >> v1:172.21.15.84:6791/0 conn(0x7f707006a300 legacy=0x7f707006e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.513+0000 7f7075234640 1 -- 172.21.15.45:0/2847233970 shutdown_connections 2024-09-18T13:59:06.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.513+0000 7f7075234640 1 -- 172.21.15.45:0/2847233970 >> 172.21.15.45:0/2847233970 conn(0x7f7070067d00 msgr2=0x7f70700a19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.513+0000 7f7075234640 1 -- 172.21.15.45:0/2847233970 shutdown_connections 2024-09-18T13:59:06.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.513+0000 7fdbb591d640 10 monclient: shutdown 2024-09-18T13:59:06.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.513+0000 7fdbb591d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.513+0000 7fdbb591d640 1 -- 172.21.15.45:0/395633524 >> v1:172.21.15.45:6793/0 conn(0x7fdbb006dd80 legacy=0x7fdbb006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.513+0000 7fdbb591d640 1 -- 172.21.15.45:0/395633524 shutdown_connections 2024-09-18T13:59:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.513+0000 7fdbb591d640 1 -- 172.21.15.45:0/395633524 >> 172.21.15.45:0/395633524 conn(0x7fdbb0068c40 msgr2=0x7fdbb00a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.513+0000 7f7075234640 1 -- 172.21.15.45:0/2847233970 wait complete. 2024-09-18T13:59:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.513+0000 7fdbb591d640 1 -- 172.21.15.45:0/395633524 shutdown_connections 2024-09-18T13:59:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.513+0000 7fdbb591d640 1 -- 172.21.15.45:0/395633524 wait complete. 2024-09-18T13:59:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.514+0000 7f7075234640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.514+0000 7f7075234640 10 monclient: monmap: 2024-09-18T13:59:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.599 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.514+0000 7f7075234640 1 Processor -- start 2024-09-18T13:59:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.514+0000 7fdbb591d640 10 monclient: build_initial_monmap 2024-09-18T13:59:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.514+0000 7f7075234640 1 -- start start 2024-09-18T13:59:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.514+0000 7f7075234640 10 monclient: init 2024-09-18T13:59:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.514+0000 7fdbb591d640 10 monclient: monmap: 2024-09-18T13:59:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7fdbb591d640 1 Processor -- start 2024-09-18T13:59:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7fdbb591d640 1 -- start start 2024-09-18T13:59:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7fdbb591d640 10 monclient: init 2024-09-18T13:59:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7f7075234640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7f7075234640 10 monclient: _add_conns ranks=[7,0,2,1,8,6,5,3,4] 2024-09-18T13:59:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7f7075234640 10 monclient(hunting): picked mon.noname-i con 0x7f707006a300 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7f7075234640 10 monclient(hunting): picked mon.noname-a con 0x7f707006eb60 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7f7075234640 10 monclient(hunting): picked mon.noname-b con 0x7f707012e5a0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7f7075234640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707006aa30 con 0x7f707006eb60 2024-09-18T13:59:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7f7075234640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707009bca0 con 0x7f707012e5a0 2024-09-18T13:59:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7f7075234640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707012d8d0 con 0x7f707006a300 2024-09-18T13:59:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7f7075234640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7f7075234640 10 monclient(hunting): authenticate will time out at 761.853555s 2024-09-18T13:59:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7f706ed9d640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f707006a300 0x7f7070136d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:34474/0 (socket says 172.21.15.45:34474) 2024-09-18T13:59:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7f706ed9d640 1 -- 172.21.15.45:0/1575420707 learned_addr learned my addr 172.21.15.45:0/1575420707 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7fdbb591d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:06.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.515+0000 7fdbb591d640 10 monclient: _add_conns ranks=[0,1,5,8,7,2,4,6,3] 2024-09-18T13:59:06.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94185513 0 0) 0x7f707009bca0 con 0x7f707012e5a0 2024-09-18T13:59:06.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdbb591d640 10 monclient(hunting): picked mon.noname-a con 0x7fdbb00670e0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:06.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdbb591d640 10 monclient(hunting): picked mon.noname-f con 0x7fdbb013cc30 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:06.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdbb591d640 10 monclient(hunting): picked mon.noname-h con 0x7fdbb009a660 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:06.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdbb591d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb009e7b0 con 0x7fdbb00670e0 2024-09-18T13:59:06.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdbaeffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdbb00670e0 0x7fdbb0097ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49464/0 (socket says 172.21.15.45:49464) 2024-09-18T13:59:06.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdbaeffd640 1 -- 172.21.15.45:0/1167343508 learned_addr learned my addr 172.21.15.45:0/1167343508 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:06.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdbb591d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb006eaf0 con 0x7fdbb013cc30 2024-09-18T13:59:06.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdbb591d640 1 -- 172.21.15.45:0/1167343508 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb009c210 con 0x7fdbb009a660 2024-09-18T13:59:06.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdbb591d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdbb591d640 10 monclient(hunting): authenticate will time out at 761.854210s 2024-09-18T13:59:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): my global_id is 4304 2024-09-18T13:59:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f705c0034e0 con 0x7f707012e5a0 2024-09-18T13:59:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192119754 0 0) 0x7f707012d8d0 con 0x7f707006a300 2024-09-18T13:59:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931680208 0 0) 0x7fdbb009c210 con 0x7fdbb009a660 2024-09-18T13:59:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 10 monclient(hunting): my global_id is 4280 2024-09-18T13:59:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb9c0034e0 con 0x7fdbb009a660 2024-09-18T13:59:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): my global_id is 4345 2024-09-18T13:59:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f707009bca0 con 0x7f707006a300 2024-09-18T13:59:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1233883701 0 0) 0x7f705c0034e0 con 0x7f707012e5a0 2024-09-18T13:59:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f707012d8d0 con 0x7f707012e5a0 2024-09-18T13:59:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 946605605 0 0) 0x7fdb9c0034e0 con 0x7fdbb009a660 2024-09-18T13:59:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdbb009c210 con 0x7fdbb009a660 2024-09-18T13:59:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdba8001c20 con 0x7fdbb009a660 2024-09-18T13:59:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T13:59:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f70680019e0 con 0x7f707012e5a0 2024-09-18T13:59:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.610 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2578386799 0 0) 0x7f707009bca0 con 0x7f707006a300 2024-09-18T13:59:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f705c0034e0 con 0x7f707006a300 2024-09-18T13:59:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f70600029d0 con 0x7f707006a300 2024-09-18T13:59:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470011090 0 0) 0x7fdbb006eaf0 con 0x7fdbb013cc30 2024-09-18T13:59:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7f7056ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.614 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.516+0000 7fdb96ffd640 10 monclient(hunting): my global_id is 4357 2024-09-18T13:59:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb9c0034e0 con 0x7fdbb013cc30 2024-09-18T13:59:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3796289253 0 0) 0x7f707012d8d0 con 0x7f707012e5a0 2024-09-18T13:59:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 >> v1:172.21.15.84:6792/0 conn(0x7f707006a300 legacy=0x7f7070136d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 >> v1:172.21.15.45:6789/0 conn(0x7f707006eb60 legacy=0x7f70701447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7056ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7056ffd640 1 monclient: found mon.b 2024-09-18T13:59:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7056ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7056ffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070140280 con 0x7f707012e5a0 2024-09-18T13:59:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7056ffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7056ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.1 v1:172.21.15.84:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fe094011380 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fe086ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fe086ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7075234640 5 monclient: authenticate success, global_id 4304 2024-09-18T13:59:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1143527701 0 0) 0x7fdbb009c210 con 0x7fdbb009a660 2024-09-18T13:59:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 >> v1:172.21.15.84:6789/0 conn(0x7fdbb013cc30 legacy=0x7fdbb00985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 >> v1:172.21.15.45:6789/0 conn(0x7fdbb00670e0 legacy=0x7fdbb0097ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdb96ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdb96ffd640 1 monclient: found mon.h 2024-09-18T13:59:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdb96ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdb96ffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T13:59:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb0099fa0 con 0x7fdbb009a660 2024-09-18T13:59:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdb96ffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdb96ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdbb591d640 5 monclient: authenticate success, global_id 4280 2024-09-18T13:59:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdbb591d640 10 monclient: _renew_subs 2024-09-18T13:59:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdbb591d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T13:59:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdbb591d640 1 -- 172.21.15.45:0/1167343508 --> v1:172.21.15.84:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb0099a90 con 0x7fdbb009a660 2024-09-18T13:59:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7075234640 10 monclient: _renew_subs 2024-09-18T13:59:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdbb591d640 10 monclient: _renew_subs 2024-09-18T13:59:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdbb591d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T13:59:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7fdbb591d640 1 -- 172.21.15.45:0/1167343508 --> v1:172.21.15.84:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb0098cc0 con 0x7fdbb009a660 2024-09-18T13:59:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7075234640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7075234640 1 -- 172.21.15.45:0/1575420707 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707012be10 con 0x7f707012e5a0 2024-09-18T13:59:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7075234640 10 monclient: _renew_subs 2024-09-18T13:59:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7075234640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.517+0000 7f7075234640 1 -- 172.21.15.45:0/1575420707 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f707012c9e0 con 0x7f707012e5a0 2024-09-18T13:59:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.518+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7068002340 con 0x7f707012e5a0 2024-09-18T13:59:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.518+0000 7f7056ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.518+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f70680027f0 con 0x7f707012e5a0 2024-09-18T13:59:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.518+0000 7f7056ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.518+0000 7f7056ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.518+0000 7f7056ffd640 10 monclient: dump: 2024-09-18T13:59:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.622 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.518+0000 7fdbb591d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7fdbb591d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7fdbb591d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T13:59:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.518+0000 7f706d59a640 4 set_mon_vals no callback set 2024-09-18T13:59:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7fdbb591d640 1 -- 172.21.15.45:0/1167343508 --> v1:172.21.15.84:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb004c4a0 con 0x7fdbb009a660 2024-09-18T13:59:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.518+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7068002950 con 0x7f707012e5a0 2024-09-18T13:59:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.518+0000 7f7075234640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f70680443b0 con 0x7f707012e5a0 2024-09-18T13:59:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 <== mon.5 v1:172.21.15.84:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdba8001d80 con 0x7fdbb009a660 2024-09-18T13:59:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7fdb96ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 <== mon.5 v1:172.21.15.84:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdba80022f0 con 0x7fdbb009a660 2024-09-18T13:59:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7fdb96ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7fdb96ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7fdb96ffd640 10 monclient: dump: 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:06.626 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7fdbad7fa640 4 set_mon_vals no callback set 2024-09-18T13:59:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7f7075234640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7f7075234640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7f7075234640 1 -- 172.21.15.45:0/1575420707 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7050005010 con 0x7f707012e5a0 2024-09-18T13:59:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.519+0000 7f706ed9d640 10 monclient: get_auth_request con 0x7f705c03e940 auth_method 0 2024-09-18T13:59:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.520+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fda040153a0 con 0x7fda0806e810 2024-09-18T13:59:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.520+0000 7fd9fe7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.520+0000 7fd9fe7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.520+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 <== mon.5 v1:172.21.15.84:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fdba8002510 con 0x7fdbb009a660 2024-09-18T13:59:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.520+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 <== mon.5 v1:172.21.15.84:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7fdba80442c0 con 0x7fdbb009a660 2024-09-18T13:59:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.521+0000 7fdbaeffd640 10 monclient: get_auth_request con 0x7fdb9c039f00 auth_method 0 2024-09-18T13:59:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.525+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f7068003f20 con 0x7f707012e5a0 2024-09-18T13:59:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.525+0000 7f7056ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.525+0000 7f7056ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.525+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 <== mon.5 v1:172.21.15.84:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fdba80027c0 con 0x7fdbb009a660 2024-09-18T13:59:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.525+0000 7fdb96ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.525+0000 7fdb96ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.657+0000 7fd7e7f5d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-18T13:59:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.657+0000 7fd7e7f5d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-18T13:59:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.657+0000 7fd7e7f5d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.657+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd7e004b720 con 0x7fd7e006e810 2024-09-18T13:59:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.657+0000 7fd7d5ffb640 1 -- 172.21.15.45:0/1862680686 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (unknown 1823589948 0 3327677263) 0x7fd7dc014cd0 con 0x7fd7e006e810 2024-09-18T13:59:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.657+0000 7fd7d5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-18T13:59:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.657+0000 7fd7d5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:06.658 INFO:teuthology.orchestra.run.smithi045.stdout:47244640257 2024-09-18T13:59:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.659+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 >> v1:172.21.15.84:6812/31478 conn(0x7fd7c403a130 legacy=0x7fd7c403c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.659+0000 7fd7e7f5d640 10 monclient: shutdown 2024-09-18T13:59:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.659+0000 7fd7e7f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.659+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 >> v1:172.21.15.45:6789/0 conn(0x7fd7e006e810 legacy=0x7fd7e013d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.659+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 shutdown_connections 2024-09-18T13:59:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.660+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 >> 172.21.15.45:0/1862680686 conn(0x7fd7e00a5bb0 msgr2=0x7fd7e00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.660+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 shutdown_connections 2024-09-18T13:59:06.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.660+0000 7fd7e7f5d640 1 -- 172.21.15.45:0/1862680686 wait complete. 2024-09-18T13:59:06.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640257 for osd.4 2024-09-18T13:59:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.691+0000 7fdbb591d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-18T13:59:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.691+0000 7fdbb591d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-18T13:59:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.691+0000 7fdbb591d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T13:59:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.691+0000 7fdbb591d640 1 -- 172.21.15.45:0/1167343508 --> v1:172.21.15.84:6791/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdbb004b720 con 0x7fdbb009a660 2024-09-18T13:59:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.692+0000 7fdb96ffd640 1 -- 172.21.15.45:0/1167343508 <== mon.5 v1:172.21.15.84:6791/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 3327677263) 0x7fdba80125d0 con 0x7fdbb009a660 2024-09-18T13:59:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.692+0000 7fdb96ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-18T13:59:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.692+0000 7fdb96ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:06.693 INFO:teuthology.orchestra.run.smithi045.stdout:47244640257 2024-09-18T13:59:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.694+0000 7fdbb591d640 1 -- 172.21.15.45:0/1167343508 >> v1:172.21.15.84:6812/31478 conn(0x7fdb9c039f00 legacy=0x7fdb9c03c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.694+0000 7fdbb591d640 10 monclient: shutdown 2024-09-18T13:59:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.694+0000 7fdbb591d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.694+0000 7fdbb591d640 1 -- 172.21.15.45:0/1167343508 >> v1:172.21.15.84:6791/0 conn(0x7fdbb009a660 legacy=0x7fdbb014e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.694+0000 7fdbb591d640 1 -- 172.21.15.45:0/1167343508 shutdown_connections 2024-09-18T13:59:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.694+0000 7fdbb591d640 1 -- 172.21.15.45:0/1167343508 >> 172.21.15.45:0/1167343508 conn(0x7fdbb0068c40 msgr2=0x7fdbb0138e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.694+0000 7fdbb591d640 1 -- 172.21.15.45:0/1167343508 shutdown_connections 2024-09-18T13:59:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.694+0000 7fdbb591d640 1 -- 172.21.15.45:0/1167343508 wait complete. 2024-09-18T13:59:06.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640257 for osd.2 2024-09-18T13:59:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.771+0000 7f86f4834640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-18T13:59:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.771+0000 7f86f4834640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-18T13:59:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.771+0000 7f86f4834640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.771+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f86f004b720 con 0x7f86f006e810 2024-09-18T13:59:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.772+0000 7f86de7fc640 1 -- 172.21.15.45:0/286083967 <== mon.7 v1:172.21.15.84:6792/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (unknown 383520633 0 3327677263) 0x7f86e0015040 con 0x7f86f006e810 2024-09-18T13:59:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.772+0000 7f86de7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-18T13:59:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.772+0000 7f86de7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:06.774 INFO:teuthology.orchestra.run.smithi045.stdout:47244640257 2024-09-18T13:59:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.774+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 >> v1:172.21.15.84:6812/31478 conn(0x7f86d0039e20 legacy=0x7f86d003c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.774+0000 7f86f4834640 10 monclient: shutdown 2024-09-18T13:59:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.774+0000 7f86f4834640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.774+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 >> v1:172.21.15.84:6792/0 conn(0x7f86f006e810 legacy=0x7f86f013d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.774+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 shutdown_connections 2024-09-18T13:59:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.774+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 >> 172.21.15.45:0/286083967 conn(0x7f86f00a5bb0 msgr2=0x7f86f00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.774+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 shutdown_connections 2024-09-18T13:59:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.774+0000 7f86f4834640 1 -- 172.21.15.45:0/286083967 wait complete. 2024-09-18T13:59:06.785 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640257 for osd.3 2024-09-18T13:59:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.807+0000 7fe0a579d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-18T13:59:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.807+0000 7fe0a579d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-18T13:59:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.807+0000 7fe0a579d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T13:59:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.807+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe0800052c0 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.807+0000 7fe086ffd640 1 -- 172.21.15.45:0/536475590 <== mon.1 v1:172.21.15.84:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 3327677263) 0x7fe094014c80 con 0x7fe0a00a0cb0 2024-09-18T13:59:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.807+0000 7fe086ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-18T13:59:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.807+0000 7fe086ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:06.809 INFO:teuthology.orchestra.run.smithi045.stdout:47244640257 2024-09-18T13:59:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7f7075234640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-18T13:59:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7f7075234640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-18T13:59:06.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7f7075234640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:06.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7f7075234640 1 -- 172.21.15.45:0/1575420707 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f70500052c0 con 0x7f707012e5a0 2024-09-18T13:59:06.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fd9f3fff640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-18T13:59:06.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fd9f3fff640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-18T13:59:06.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fd9f3fff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:06.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fd9f3fff640 1 -- 172.21.15.45:0/2690182369 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd9e80052c0 con 0x7fda0806e810 2024-09-18T13:59:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7f7056ffd640 1 -- 172.21.15.45:0/1575420707 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (unknown 2131975755 0 3327677263) 0x7f7068012630 con 0x7f707012e5a0 2024-09-18T13:59:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7f7056ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-18T13:59:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7f7056ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:06.811 INFO:teuthology.orchestra.run.smithi045.stdout:47244640257 2024-09-18T13:59:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fd9fe7fc640 1 -- 172.21.15.45:0/2690182369 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 727498649) 0x7fda04014da0 con 0x7fda0806e810 2024-09-18T13:59:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fd9fe7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-18T13:59:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fd9fe7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 >> v1:172.21.15.84:6812/31478 conn(0x7fe08c03a9a0 legacy=0x7fe08c03ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fe0a579d640 10 monclient: shutdown 2024-09-18T13:59:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fe0a579d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 >> v1:172.21.15.84:6789/0 conn(0x7fe0a00a0cb0 legacy=0x7fe0a0146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.812 INFO:teuthology.orchestra.run.smithi045.stdout:51539607554 2024-09-18T13:59:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 shutdown_connections 2024-09-18T13:59:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.809+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 >> 172.21.15.45:0/536475590 conn(0x7fe0a00a5bb0 msgr2=0x7fe0a00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.810+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 shutdown_connections 2024-09-18T13:59:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.810+0000 7fe0a579d640 1 -- 172.21.15.45:0/536475590 wait complete. 2024-09-18T13:59:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.811+0000 7f7075234640 1 -- 172.21.15.45:0/1575420707 >> v1:172.21.15.84:6812/31478 conn(0x7f705c03e940 legacy=0x7f705c040df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.811+0000 7f7075234640 10 monclient: shutdown 2024-09-18T13:59:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.811+0000 7f7075234640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.811+0000 7f7075234640 1 -- 172.21.15.45:0/1575420707 >> v1:172.21.15.45:6790/0 conn(0x7f707012e5a0 legacy=0x7f7070147ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.811+0000 7fd9f3fff640 1 -- 172.21.15.45:0/2690182369 >> v1:172.21.15.84:6812/31478 conn(0x7fd9ec03a3f0 legacy=0x7fd9ec03c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.811+0000 7f7075234640 1 -- 172.21.15.45:0/1575420707 shutdown_connections 2024-09-18T13:59:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.811+0000 7f7075234640 1 -- 172.21.15.45:0/1575420707 >> 172.21.15.45:0/1575420707 conn(0x7f7070067d00 msgr2=0x7f707009c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.811+0000 7fd9f3fff640 10 monclient: shutdown 2024-09-18T13:59:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.811+0000 7fd9f3fff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.811+0000 7fd9f3fff640 1 -- 172.21.15.45:0/2690182369 >> v1:172.21.15.45:6789/0 conn(0x7fda0806e810 legacy=0x7fda08130410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.812+0000 7f7075234640 1 -- 172.21.15.45:0/1575420707 shutdown_connections 2024-09-18T13:59:06.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.812+0000 7fd9f3fff640 1 -- 172.21.15.45:0/2690182369 shutdown_connections 2024-09-18T13:59:06.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.812+0000 7fd9f3fff640 1 -- 172.21.15.45:0/2690182369 >> 172.21.15.45:0/2690182369 conn(0x7fda080a5c20 msgr2=0x7fda08131810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:06.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.812+0000 7f7075234640 1 -- 172.21.15.45:0/1575420707 wait complete. 2024-09-18T13:59:06.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.812+0000 7fd9f3fff640 1 -- 172.21.15.45:0/2690182369 shutdown_connections 2024-09-18T13:59:06.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:06.812+0000 7fd9f3fff640 1 -- 172.21.15.45:0/2690182369 wait complete. 2024-09-18T13:59:06.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640257 for osd.0 2024-09-18T13:59:06.824 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640257 for osd.5 2024-09-18T13:59:06.825 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607555 got 51539607554 for osd.1 2024-09-18T13:59:07.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T13:59:07.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T13:59:07.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.780+0000 7f7ec6b9d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:07.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.780+0000 7f7ec6b9d640 10 monclient: build_initial_monmap 2024-09-18T13:59:07.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.780+0000 7f7ec6b9d640 10 monclient: monmap: 2024-09-18T13:59:07.781 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:07.781 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:07.781843+0000 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:07.781843+0000 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:07.782 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.781+0000 7f7ec6b9d640 1 Processor -- start 2024-09-18T13:59:07.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.781+0000 7f7ec6b9d640 1 -- start start 2024-09-18T13:59:07.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.781+0000 7f7ec6b9d640 10 monclient: init 2024-09-18T13:59:07.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ec6b9d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:07.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ec6b9d640 10 monclient: _add_conns ranks=[2,5,6,4,3,8,1,0,7] 2024-09-18T13:59:07.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ec6b9d640 10 monclient(hunting): picked mon.noname-b con 0x7f7ec006e810 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:07.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ec6b9d640 10 monclient(hunting): picked mon.noname-h con 0x7f7ec00a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:07.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ec6b9d640 10 monclient(hunting): picked mon.noname-d con 0x7f7ec00a1080 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:07.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ec6b9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec005ed70 con 0x7f7ec006e810 2024-09-18T13:59:07.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ec6b9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec0097620 con 0x7f7ec00a0cb0 2024-09-18T13:59:07.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ec6b9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec0097340 con 0x7f7ec00a1080 2024-09-18T13:59:07.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ec6b9d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:07.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ec6b9d640 10 monclient(hunting): authenticate will time out at 763.120328s 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ec4939640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f7ec006e810 0x7f7ec00a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51496/0 (socket says 172.21.15.45:51496) 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ec4939640 1 -- 172.21.15.45:0/2049035729 learned_addr learned my addr 172.21.15.45:0/2049035729 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469316722 0 0) 0x7f7ec005ed70 con 0x7f7ec006e810 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 10 monclient(hunting): my global_id is 4313 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eac0034e0 con 0x7f7ec006e810 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030838192 0 0) 0x7f7ec0097340 con 0x7f7ec00a1080 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 10 monclient(hunting): my global_id is 4335 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec005ed70 con 0x7f7ec00a1080 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3659768401 0 0) 0x7f7ec0097620 con 0x7f7ec00a0cb0 2024-09-18T13:59:07.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 10 monclient(hunting): my global_id is 4289 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec0097340 con 0x7f7ec00a0cb0 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.782+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2197493171 0 0) 0x7f7eac0034e0 con 0x7f7ec006e810 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 >> v1:172.21.15.45:6792/0 conn(0x7f7ec00a1080 legacy=0x7f7ec013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 >> v1:172.21.15.84:6791/0 conn(0x7f7ec00a0cb0 legacy=0x7f7ec0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 10 monclient: _finish_hunting 0 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 1 monclient: found mon.noname-b 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec00a1660 con 0x7f7ec006e810 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 10 monclient: _finish_auth 0 2024-09-18T13:59:07.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7ea8001760 con 0x7f7ec006e810 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ec6b9d640 5 monclient: authenticate success, global_id 4313 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ec6b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 10 monclient: dump: 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.789 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ec6b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7ea80018c0 con 0x7f7ec006e810 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 1 -- 172.21.15.45:0/2049035729 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7ea8004860 con 0x7f7ec006e810 2024-09-18T13:59:07.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ec6b9d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ec6b9d640 4 set_mon_vals no callback set 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ebf7fe640 10 monclient: dump: 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ec6b9d640 10 monclient: shutdown 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ec6b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/2049035729 >> v1:172.21.15.45:6790/0 conn(0x7f7ec006e810 legacy=0x7f7ec00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/2049035729 shutdown_connections 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/2049035729 >> 172.21.15.45:0/2049035729 conn(0x7f7ec00a5bb0 msgr2=0x7f7ec00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/2049035729 shutdown_connections 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.783+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/2049035729 wait complete. 2024-09-18T13:59:07.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.784+0000 7f7ec6b9d640 10 monclient: build_initial_monmap 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.784+0000 7f7ec6b9d640 10 monclient: monmap: 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:07.794 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:07.794 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:07.794 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.785+0000 7f7ec6b9d640 1 Processor -- start 2024-09-18T13:59:07.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.785+0000 7f7ec6b9d640 1 -- start start 2024-09-18T13:59:07.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.785+0000 7f7ec6b9d640 10 monclient: init 2024-09-18T13:59:07.794 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T13:59:07.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ec6b9d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:07.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ec6b9d640 10 monclient: _add_conns ranks=[5,7,4,8,1,0,2,3,6] 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ec6b9d640 10 monclient(hunting): picked mon.noname-h con 0x7f7ec006e810 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ec6b9d640 10 monclient(hunting): picked mon.noname-i con 0x7f7ec00a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ec6b9d640 10 monclient(hunting): picked mon.noname-c con 0x7f7ec00a1080 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ec6b9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec009fa10 con 0x7f7ec00a1080 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ec6b9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec013db20 con 0x7f7ec006e810 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ec6b9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec00a2270 con 0x7f7ec00a0cb0 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ec6b9d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ec6b9d640 10 monclient(hunting): authenticate will time out at 763.124186s 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ec513a640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f7ec00a1080 0x7f7ec0149d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:45832/0 (socket says 172.21.15.45:45832) 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ec513a640 1 -- 172.21.15.45:0/4007419154 learned_addr learned my addr 172.21.15.45:0/4007419154 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446370907 0 0) 0x7f7ec009fa10 con 0x7f7ec00a1080 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): my global_id is 4333 2024-09-18T13:59:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea00034e0 con 0x7f7ec00a1080 2024-09-18T13:59:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810430838 0 0) 0x7f7ec00a2270 con 0x7f7ec00a0cb0 2024-09-18T13:59:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): my global_id is 4354 2024-09-18T13:59:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec009fa10 con 0x7f7ec00a0cb0 2024-09-18T13:59:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825063710 0 0) 0x7f7ec013db20 con 0x7f7ec006e810 2024-09-18T13:59:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): my global_id is 4298 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec00a2270 con 0x7f7ec006e810 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2082227348 0 0) 0x7f7ea00034e0 con 0x7f7ec00a1080 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ec013db20 con 0x7f7ec00a1080 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7eb8001a80 con 0x7f7ec00a1080 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): dump: 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 357653298 0 0) 0x7f7ec00a2270 con 0x7f7ec006e810 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ea00034e0 con 0x7f7ec006e810 2024-09-18T13:59:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3715948658 0 0) 0x7f7ec009fa10 con 0x7f7ec00a0cb0 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.786+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ec00a2270 con 0x7f7ec00a0cb0 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7ea8001620 con 0x7f7ec006e810 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient(hunting): dump: 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7eb0001e60 con 0x7f7ec00a0cb0 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient(hunting): dump: 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.804 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3221535029 0 0) 0x7f7ec013db20 con 0x7f7ec00a1080 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 >> v1:172.21.15.84:6792/0 conn(0x7f7ec00a0cb0 legacy=0x7f7ec0146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 >> v1:172.21.15.84:6791/0 conn(0x7f7ec006e810 legacy=0x7f7ec013d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient: _finish_hunting 0 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 1 monclient: found mon.c 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec013e930 con 0x7f7ec00a1080 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient: _finish_auth 0 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ec6b9d640 5 monclient: authenticate success, global_id 4333 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ec6b9d640 10 monclient: _renew_subs 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ec6b9d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/4007419154 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec01224b0 con 0x7f7ec00a1080 2024-09-18T13:59:07.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ec6b9d640 10 monclient: _renew_subs 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ec6b9d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/4007419154 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec01200c0 con 0x7f7ec00a1080 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7eb8001ff0 con 0x7f7ec00a1080 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7eb8004db0 con 0x7f7ec00a1080 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebeffd640 4 set_mon_vals no callback set 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.787+0000 7f7ebcff9640 10 monclient: dump: 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.788+0000 7f7ec6b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.788+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7eb800fd80 con 0x7f7ec00a1080 2024-09-18T13:59:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.788+0000 7f7ec6b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.788+0000 7f7ec6b9d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.788+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/4007419154 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea4005010 con 0x7f7ec00a1080 2024-09-18T13:59:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.788+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f7eb8046890 con 0x7f7ec00a1080 2024-09-18T13:59:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.788+0000 7f7ec4939640 10 monclient: get_auth_request con 0x7f7ea003ab40 auth_method 0 2024-09-18T13:59:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.797+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.4 v1:172.21.15.45:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f7eb80112d0 con 0x7f7ec00a1080 2024-09-18T13:59:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.797+0000 7f7ebcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.797+0000 7f7ebcff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:07.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T13:59:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.823+0000 7fce8c846640 10 monclient: get_monmap_and_config 2024-09-18T13:59:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.823+0000 7fce8c846640 10 monclient: build_initial_monmap 2024-09-18T13:59:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.823+0000 7fce8c846640 10 monclient: monmap: 2024-09-18T13:59:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:07.824802+0000 2024-09-18T13:59:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:07.824802+0000 2024-09-18T13:59:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:07.825 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T13:59:07.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T13:59:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.824+0000 7fce8c846640 1 Processor -- start 2024-09-18T13:59:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.824+0000 7fce8c846640 1 -- start start 2024-09-18T13:59:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.824+0000 7fce8c846640 10 monclient: init 2024-09-18T13:59:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce8c846640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce8c846640 10 monclient: _add_conns ranks=[3,4,6,8,1,0,2,7,5] 2024-09-18T13:59:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce8c846640 10 monclient(hunting): picked mon.noname-g con 0x7fce8809ddb0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce8c846640 10 monclient(hunting): picked mon.noname-c con 0x7fce8809e180 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce8c846640 10 monclient(hunting): picked mon.noname-d con 0x7fce8809d410 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce8c846640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8805ed70 con 0x7fce8809ddb0 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce8c846640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8809d890 con 0x7fce8809e180 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce8c846640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8809e830 con 0x7fce8809d410 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce8c846640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce8c846640 10 monclient(hunting): authenticate will time out at 763.163430s 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce7ffff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fce8809e180 0x7fce8809c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:45834/0 (socket says 172.21.15.45:45834) 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce86d9d640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fce8809d410 0x7fce8813b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34732/0 (socket says 172.21.15.45:34732) 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce7ffff640 1 -- 172.21.15.45:0/1824047151 learned_addr learned my addr 172.21.15.45:0/1824047151 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853870488 0 0) 0x7fce8805ed70 con 0x7fce8809ddb0 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 10 monclient(hunting): my global_id is 4377 2024-09-18T13:59:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce740034e0 con 0x7fce8809ddb0 2024-09-18T13:59:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363824926 0 0) 0x7fce8809d890 con 0x7fce8809e180 2024-09-18T13:59:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 10 monclient(hunting): my global_id is 4342 2024-09-18T13:59:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8805ed70 con 0x7fce8809e180 2024-09-18T13:59:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215182284 0 0) 0x7fce8809e830 con 0x7fce8809d410 2024-09-18T13:59:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 10 monclient(hunting): my global_id is 4344 2024-09-18T13:59:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.825+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8809d890 con 0x7fce8809d410 2024-09-18T13:59:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 369714706 0 0) 0x7fce740034e0 con 0x7fce8809ddb0 2024-09-18T13:59:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 >> v1:172.21.15.45:6792/0 conn(0x7fce8809d410 legacy=0x7fce8813b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 >> v1:172.21.15.45:6791/0 conn(0x7fce8809e180 legacy=0x7fce8809c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 10 monclient: _finish_hunting 0 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 1 monclient: found mon.noname-g 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8809af60 con 0x7fce8809ddb0 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 10 monclient: _finish_auth 0 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce78001830 con 0x7fce8809ddb0 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 10 monclient: dump: 2024-09-18T13:59:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce8c846640 5 monclient: authenticate success, global_id 4377 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce8c846640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce78001990 con 0x7fce8809ddb0 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 1 -- 172.21.15.45:0/1824047151 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce78001f40 con 0x7fce8809ddb0 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce8c846640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce8c846640 4 set_mon_vals no callback set 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.826+0000 7fce85d9b640 10 monclient: dump: 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.827+0000 7fce8c846640 10 monclient: shutdown 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.827+0000 7fce8c846640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.827+0000 7fce8c846640 1 -- 172.21.15.45:0/1824047151 >> v1:172.21.15.84:6790/0 conn(0x7fce8809ddb0 legacy=0x7fce8809bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.827+0000 7fce8c846640 1 -- 172.21.15.45:0/1824047151 shutdown_connections 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.827+0000 7fce8c846640 1 -- 172.21.15.45:0/1824047151 >> 172.21.15.45:0/1824047151 conn(0x7fce88068b40 msgr2=0x7fce880a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.827+0000 7fce8c846640 1 -- 172.21.15.45:0/1824047151 shutdown_connections 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.827+0000 7fce8c846640 1 -- 172.21.15.45:0/1824047151 wait complete. 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.827+0000 7fce8c846640 10 monclient: build_initial_monmap 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.827+0000 7fce8c846640 10 monclient: monmap: 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.828+0000 7fce8c846640 1 Processor -- start 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.828+0000 7fce8c846640 1 -- start start 2024-09-18T13:59:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.828+0000 7fce8c846640 10 monclient: init 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce8c846640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce8c846640 10 monclient: _add_conns ranks=[7,3,4,5,8,1,0,6,2] 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce8c846640 10 monclient(hunting): picked mon.noname-i con 0x7fce8809d410 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce8c846640 10 monclient(hunting): picked mon.noname-g con 0x7fce8809ddb0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce8c846640 10 monclient(hunting): picked mon.noname-c con 0x7fce8809e180 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce8c846640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce881227f0 con 0x7fce8809ddb0 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce8c846640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8813e8e0 con 0x7fce8809e180 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce86d9d640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fce8809e180 0x7fce88149df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:45844/0 (socket says 172.21.15.45:45844) 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce86d9d640 1 -- 172.21.15.45:0/3118706358 learned_addr learned my addr 172.21.15.45:0/3118706358 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce8c846640 1 -- 172.21.15.45:0/3118706358 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8813db80 con 0x7fce8809d410 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce8c846640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce8c846640 10 monclient(hunting): authenticate will time out at 763.167625s 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857254202 0 0) 0x7fce8813e8e0 con 0x7fce8809e180 2024-09-18T13:59:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce7e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce7e7fc640 10 monclient(hunting): my global_id is 4351 2024-09-18T13:59:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce680034e0 con 0x7fce8809e180 2024-09-18T13:59:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246884195 0 0) 0x7fce8813db80 con 0x7fce8809d410 2024-09-18T13:59:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce7e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce7e7fc640 10 monclient(hunting): my global_id is 4363 2024-09-18T13:59:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.829+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8813e8e0 con 0x7fce8809d410 2024-09-18T13:59:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129374314 0 0) 0x7fce881227f0 con 0x7fce8809ddb0 2024-09-18T13:59:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient(hunting): my global_id is 4386 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8813db80 con 0x7fce8809ddb0 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3845267004 0 0) 0x7fce680034e0 con 0x7fce8809e180 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce881227f0 con 0x7fce8809e180 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce80001d10 con 0x7fce8809e180 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2151132070 0 0) 0x7fce8813e8e0 con 0x7fce8809d410 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce680034e0 con 0x7fce8809d410 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce78001e40 con 0x7fce8809d410 2024-09-18T13:59:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2755565339 0 0) 0x7fce881227f0 con 0x7fce8809e180 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 >> v1:172.21.15.84:6792/0 conn(0x7fce8809d410 legacy=0x7fce8813d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 >> v1:172.21.15.84:6790/0 conn(0x7fce8809ddb0 legacy=0x7fce881466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 monclient: found mon.c 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce88122510 con 0x7fce8809e180 2024-09-18T13:59:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce8c846640 5 monclient: authenticate success, global_id 4351 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce8c846640 10 monclient: _renew_subs 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce8c846640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce8c846640 1 -- 172.21.15.45:0/3118706358 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce88122ad0 con 0x7fce8809e180 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce8c846640 10 monclient: _renew_subs 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce8c846640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce8c846640 1 -- 172.21.15.45:0/3118706358 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fce881217e0 con 0x7fce8809e180 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce80004b20 con 0x7fce8809e180 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce80005090 con 0x7fce8809e180 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce7e7fc640 10 monclient: dump: 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.850 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.850 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.850 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.830+0000 7fce8559a640 4 set_mon_vals no callback set 2024-09-18T13:59:07.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.831+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fce800051f0 con 0x7fce8809e180 2024-09-18T13:59:07.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.831+0000 7fce8c846640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:07.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.831+0000 7fce8c846640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:07.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.831+0000 7fce8c846640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:07.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.831+0000 7fce8c846640 1 -- 172.21.15.45:0/3118706358 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8804c4a0 con 0x7fce8809e180 2024-09-18T13:59:07.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.832+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7fce80046b80 con 0x7fce8809e180 2024-09-18T13:59:07.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.832+0000 7fce8659c640 10 monclient: get_auth_request con 0x7fce6803a650 auth_method 0 2024-09-18T13:59:07.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.839+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.4 v1:172.21.15.45:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fce80005860 con 0x7fce8809e180 2024-09-18T13:59:07.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.839+0000 7fce7e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:07.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.839+0000 7fce7e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:07.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.911+0000 7f521855d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:07.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.911+0000 7f521855d640 10 monclient: build_initial_monmap 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.911+0000 7f521855d640 10 monclient: monmap: 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:07.912763+0000 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:07.912763+0000 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:07.914 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:07.914 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:07.914 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:07.914 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.912+0000 7f521855d640 1 Processor -- start 2024-09-18T13:59:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.912+0000 7f521855d640 1 -- start start 2024-09-18T13:59:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.912+0000 7f521855d640 10 monclient: init 2024-09-18T13:59:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.913+0000 7f521855d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.913+0000 7f521855d640 10 monclient: _add_conns ranks=[1,8,6,7,0,3,4,5,2] 2024-09-18T13:59:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.913+0000 7f521855d640 10 monclient(hunting): picked mon.noname-f con 0x7f521006e810 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.913+0000 7f521855d640 10 monclient(hunting): picked mon.noname-e con 0x7f52100a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.913+0000 7f521855d640 10 monclient(hunting): picked mon.noname-d con 0x7f52100a1080 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.913+0000 7f521855d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521005ed70 con 0x7f521006e810 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.913+0000 7f521855d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5210097620 con 0x7f52100a1080 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.913+0000 7f521855d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5210097340 con 0x7f52100a0cb0 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.913+0000 7f521855d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.913+0000 7f521855d640 10 monclient(hunting): authenticate will time out at 763.251494s 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.913+0000 7f5216afa640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f52100a1080 0x7f521013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34748/0 (socket says 172.21.15.45:34748) 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.913+0000 7f5216afa640 1 -- 172.21.15.45:0/317440682 learned_addr learned my addr 172.21.15.45:0/317440682 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 1 -- 172.21.15.45:0/317440682 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707994 0 0) 0x7f5210097620 con 0x7f52100a1080 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 10 monclient(hunting): my global_id is 4353 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 1 -- 172.21.15.45:0/317440682 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51fc0034e0 con 0x7f52100a1080 2024-09-18T13:59:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 1 -- 172.21.15.45:0/317440682 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501739448 0 0) 0x7f5210097340 con 0x7f52100a0cb0 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 10 monclient(hunting): my global_id is 4310 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 1 -- 172.21.15.45:0/317440682 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5210097620 con 0x7f52100a0cb0 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 1 -- 172.21.15.45:0/317440682 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2982685548 0 0) 0x7f51fc0034e0 con 0x7f52100a1080 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 1 -- 172.21.15.45:0/317440682 >> v1:172.21.15.45:6793/0 conn(0x7f52100a0cb0 legacy=0x7f5210137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 1 -- 172.21.15.45:0/317440682 >> v1:172.21.15.84:6789/0 conn(0x7f521006e810 legacy=0x7f52100a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 10 monclient: _finish_hunting 0 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 1 monclient: found mon.noname-d 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 1 -- 172.21.15.45:0/317440682 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52100a1660 con 0x7f52100a1080 2024-09-18T13:59:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 10 monclient: _finish_auth 0 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 1 -- 172.21.15.45:0/317440682 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5208001bb0 con 0x7f52100a1080 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f521855d640 5 monclient: authenticate success, global_id 4353 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f521855d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 10 monclient: dump: 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f521855d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.914+0000 7f52152f7640 1 -- 172.21.15.45:0/317440682 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5208001d10 con 0x7f52100a1080 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.915+0000 7f52152f7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.915+0000 7f52152f7640 1 -- 172.21.15.45:0/317440682 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5208004be0 con 0x7f52100a1080 2024-09-18T13:59:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.915+0000 7f52152f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.915+0000 7f52152f7640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.915+0000 7f52152f7640 10 monclient: dump: 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.931 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.915+0000 7f521855d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.915+0000 7f521855d640 4 set_mon_vals no callback set 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.915+0000 7f521855d640 10 monclient: shutdown 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.915+0000 7f521855d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.915+0000 7f521855d640 1 -- 172.21.15.45:0/317440682 >> v1:172.21.15.45:6792/0 conn(0x7f52100a1080 legacy=0x7f521013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.916+0000 7f521855d640 1 -- 172.21.15.45:0/317440682 shutdown_connections 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.916+0000 7f521855d640 1 -- 172.21.15.45:0/317440682 >> 172.21.15.45:0/317440682 conn(0x7f52100a5bb0 msgr2=0x7f52100a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.916+0000 7f521855d640 1 -- 172.21.15.45:0/317440682 shutdown_connections 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.916+0000 7f521855d640 1 -- 172.21.15.45:0/317440682 wait complete. 2024-09-18T13:59:07.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.917+0000 7f521855d640 10 monclient: build_initial_monmap 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.917+0000 7f521855d640 10 monclient: monmap: 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.917+0000 7f521855d640 1 Processor -- start 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.917+0000 7f521855d640 1 -- start start 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.917+0000 7f521855d640 10 monclient: init 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.918+0000 7f521855d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.918+0000 7f521855d640 10 monclient: _add_conns ranks=[7,2,3,8,1,4,0,6,5] 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.918+0000 7f521855d640 10 monclient(hunting): picked mon.noname-i con 0x7f521006e810 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.918+0000 7f521855d640 10 monclient(hunting): picked mon.noname-b con 0x7f52100a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.918+0000 7f5215af8640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f52100a0cb0 0x7f5210135a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51504/0 (socket says 172.21.15.45:51504) 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.918+0000 7f5215af8640 1 -- 172.21.15.45:0/2998896021 learned_addr learned my addr 172.21.15.45:0/2998896021 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.918+0000 7f521855d640 10 monclient(hunting): picked mon.noname-g con 0x7f52100a1080 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.918+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52100a2270 con 0x7f52100a0cb0 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52101469b0 con 0x7f52100a1080 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306461770 0 0) 0x7f52100a2270 con 0x7f52100a0cb0 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521006bf50 con 0x7f521006e810 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f521855d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f521855d640 10 monclient(hunting): authenticate will time out at 763.257614s 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 10 monclient(hunting): my global_id is 4322 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f40034e0 con 0x7f52100a0cb0 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4063640682 0 0) 0x7f52101469b0 con 0x7f52100a1080 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 10 monclient(hunting): my global_id is 4395 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52100a2270 con 0x7f52100a1080 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112788237 0 0) 0x7f521006bf50 con 0x7f521006e810 2024-09-18T13:59:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 10 monclient(hunting): my global_id is 4372 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.919+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52101469b0 con 0x7f521006e810 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2465713926 0 0) 0x7f52101469b0 con 0x7f521006e810 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51f4004d50 con 0x7f521006e810 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 58813741 0 0) 0x7f52100a2270 con 0x7f52100a1080 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51f4005200 con 0x7f52100a1080 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5208001a70 con 0x7f52100a1080 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 10 monclient(hunting): dump: 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 759415324 0 0) 0x7f51f40034e0 con 0x7f52100a0cb0 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f52100a2270 con 0x7f52100a0cb0 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1373512404 0 0) 0x7f51f4005200 con 0x7f52100a1080 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 >> v1:172.21.15.84:6792/0 conn(0x7f521006e810 legacy=0x7f5210145b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 >> v1:172.21.15.45:6790/0 conn(0x7f52100a0cb0 legacy=0x7f5210135a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 1 monclient: found mon.g 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521009fa10 con 0x7f52100a1080 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.920+0000 7f52067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f521855d640 5 monclient: authenticate success, global_id 4395 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f521855d640 10 monclient: _renew_subs 2024-09-18T13:59:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f521855d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5210136140 con 0x7f52100a1080 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f521855d640 10 monclient: _renew_subs 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f521855d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f521006cb20 con 0x7f52100a1080 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5208004dd0 con 0x7f52100a1080 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f52067fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5208005280 con 0x7f52100a1080 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f52067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f5214af6640 4 set_mon_vals no callback set 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f52067fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.921+0000 7f52067fc640 10 monclient: dump: 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.922+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f5208010250 con 0x7f52100a1080 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.922+0000 7f521855d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.922+0000 7f521855d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.922+0000 7f521855d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:07.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.922+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521004c4a0 con 0x7f52100a1080 2024-09-18T13:59:07.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.923+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f5208046ca0 con 0x7f52100a1080 2024-09-18T13:59:07.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.923+0000 7f52162f9640 10 monclient: get_auth_request con 0x7f51f403a7d0 auth_method 0 2024-09-18T13:59:07.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.927+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.3 v1:172.21.15.84:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f5208015600 con 0x7f52100a1080 2024-09-18T13:59:07.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.928+0000 7f52067fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:07.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.928+0000 7f52067fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.945+0000 7ff3bc660640 10 monclient: get_monmap_and_config 2024-09-18T13:59:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.945+0000 7ff3bc660640 10 monclient: build_initial_monmap 2024-09-18T13:59:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.945+0000 7ff3bc660640 10 monclient: monmap: 2024-09-18T13:59:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:07.946746+0000 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:07.946746+0000 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:07.947 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.946+0000 7ff3bc660640 1 Processor -- start 2024-09-18T13:59:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.946+0000 7ff3bc660640 1 -- start start 2024-09-18T13:59:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.946+0000 7ff3bc660640 10 monclient: init 2024-09-18T13:59:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.947+0000 7ff3bc660640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.947+0000 7ff3bc660640 10 monclient: _add_conns ranks=[4,8,7,2,3,5,1,6,0] 2024-09-18T13:59:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.947+0000 7ff3bc660640 10 monclient(hunting): picked mon.noname-c con 0x7ff3b4136da0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.947+0000 7ff3bc660640 10 monclient(hunting): picked mon.noname-e con 0x7ff3b40a15b0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.947+0000 7ff3bc660640 10 monclient(hunting): picked mon.noname-i con 0x7ff3b413cd90 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.947+0000 7ff3b9bfb640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7ff3b40a15b0 0x7ff3b4137170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51842/0 (socket says 172.21.15.45:51842) 2024-09-18T13:59:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.947+0000 7ff3b9bfb640 1 -- 172.21.15.45:0/1205224783 learned_addr learned my addr 172.21.15.45:0/1205224783 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.947+0000 7ff3bc660640 1 -- 172.21.15.45:0/1205224783 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b405ed70 con 0x7ff3b4136da0 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.947+0000 7ff3bc660640 1 -- 172.21.15.45:0/1205224783 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b40a1980 con 0x7ff3b413cd90 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3bc660640 1 -- 172.21.15.45:0/1205224783 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b406bb70 con 0x7ff3b40a15b0 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186087432 0 0) 0x7ff3b405ed70 con 0x7ff3b4136da0 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3bc660640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3bc660640 10 monclient(hunting): authenticate will time out at 763.286293s 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient(hunting): my global_id is 4360 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3a00034e0 con 0x7ff3b4136da0 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549864069 0 0) 0x7ff3b406bb70 con 0x7ff3b40a15b0 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient(hunting): my global_id is 4319 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3b405ed70 con 0x7ff3b40a15b0 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956853198 0 0) 0x7ff3b40a1980 con 0x7ff3b413cd90 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient(hunting): my global_id is 4381 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3b406bb70 con 0x7ff3b413cd90 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4215895270 0 0) 0x7ff3b406bb70 con 0x7ff3b413cd90 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 >> v1:172.21.15.45:6793/0 conn(0x7ff3b40a15b0 legacy=0x7ff3b4137170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 >> v1:172.21.15.45:6791/0 conn(0x7ff3b4136da0 legacy=0x7ff3b406ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient: _finish_hunting 0 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 monclient: found mon.noname-i 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b406be50 con 0x7ff3b413cd90 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient: _finish_auth 0 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3ac001a50 con 0x7ff3b413cd90 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient: dump: 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3bc660640 5 monclient: authenticate success, global_id 4381 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3bc660640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff3ac001bb0 con 0x7ff3b413cd90 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 1 -- 172.21.15.45:0/1205224783 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3ac002160 con 0x7ff3b413cd90 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.948+0000 7ff3b93fa640 10 monclient: dump: 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.949+0000 7ff3bc660640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.949+0000 7ff3bc660640 4 set_mon_vals no callback set 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.949+0000 7ff3bc660640 10 monclient: shutdown 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.949+0000 7ff3bc660640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.949+0000 7ff3bc660640 1 -- 172.21.15.45:0/1205224783 >> v1:172.21.15.84:6792/0 conn(0x7ff3b413cd90 legacy=0x7ff3b413f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.950+0000 7ff3bc660640 1 -- 172.21.15.45:0/1205224783 shutdown_connections 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.950+0000 7ff3bc660640 1 -- 172.21.15.45:0/1205224783 >> 172.21.15.45:0/1205224783 conn(0x7ff3b40a5bb0 msgr2=0x7ff3b40a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.950+0000 7ff3bc660640 1 -- 172.21.15.45:0/1205224783 shutdown_connections 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.950+0000 7ff3bc660640 1 -- 172.21.15.45:0/1205224783 wait complete. 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.950+0000 7ff3bc660640 10 monclient: build_initial_monmap 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.950+0000 7ff3bc660640 10 monclient: monmap: 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:07.959 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:07.959 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:07.959 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:07.959 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:07.959 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.951+0000 7ff3bc660640 1 Processor -- start 2024-09-18T13:59:07.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.951+0000 7ff3bc660640 1 -- start start 2024-09-18T13:59:07.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.951+0000 7ff3bc660640 10 monclient: init 2024-09-18T13:59:07.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.952+0000 7ff3bc660640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:07.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.952+0000 7ff3bc660640 10 monclient: _add_conns ranks=[2,7,0,8,5,3,4,6,1] 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.952+0000 7ff3bc660640 10 monclient(hunting): picked mon.noname-b con 0x7ff3b40a15b0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.952+0000 7ff3bc660640 10 monclient(hunting): picked mon.noname-i con 0x7ff3b4136da0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.952+0000 7ff3bc660640 10 monclient(hunting): picked mon.noname-a con 0x7ff3b413cd90 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.952+0000 7ff3ba3fc640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7ff3b40a15b0 0x7ff3b409f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51516/0 (socket says 172.21.15.45:51516) 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.952+0000 7ff3ba3fc640 1 -- 172.21.15.45:0/4219683737 learned_addr learned my addr 172.21.15.45:0/4219683737 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.952+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b409fc60 con 0x7ff3b413cd90 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.952+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b40979b0 con 0x7ff3b40a15b0 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014283489 0 0) 0x7ff3b40979b0 con 0x7ff3b40a15b0 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b413b610 con 0x7ff3b4136da0 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3bc660640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3bc660640 10 monclient(hunting): authenticate will time out at 763.291209s 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3aa7fc640 10 monclient(hunting): my global_id is 4331 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3980034e0 con 0x7ff3b40a15b0 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253265089 0 0) 0x7ff3b409fc60 con 0x7ff3b413cd90 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3aa7fc640 10 monclient(hunting): my global_id is 4347 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3b40979b0 con 0x7ff3b413cd90 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.953+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504306657 0 0) 0x7ff3b413b610 con 0x7ff3b4136da0 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient(hunting): my global_id is 4390 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3b409fc60 con 0x7ff3b4136da0 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1607338886 0 0) 0x7ff3980034e0 con 0x7ff3b40a15b0 2024-09-18T13:59:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3b413b610 con 0x7ff3b40a15b0 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2184496823 0 0) 0x7ff3b40979b0 con 0x7ff3b413cd90 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3980034e0 con 0x7ff3b413cd90 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3b0001be0 con 0x7ff3b40a15b0 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.968 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3ac001a40 con 0x7ff3b413cd90 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3275098685 0 0) 0x7ff3b413b610 con 0x7ff3b40a15b0 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 >> v1:172.21.15.84:6792/0 conn(0x7ff3b4136da0 legacy=0x7ff3b414a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 >> v1:172.21.15.45:6789/0 conn(0x7ff3b413cd90 legacy=0x7ff3b414d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 monclient: found mon.b 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b413b1f0 con 0x7ff3b40a15b0 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff3b0001d40 con 0x7ff3b40a15b0 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3aa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3b8bf9640 4 set_mon_vals no callback set 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3bc660640 5 monclient: authenticate success, global_id 4331 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3bc660640 10 monclient: _renew_subs 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3bc660640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.954+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b40973f0 con 0x7ff3b40a15b0 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.955+0000 7ff3bc660640 10 monclient: _renew_subs 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.955+0000 7ff3bc660640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.955+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b409e810 con 0x7ff3b40a15b0 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.955+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3b0002670 con 0x7ff3b40a15b0 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.955+0000 7ff3aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.955+0000 7ff3aa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.955+0000 7ff3aa7fc640 10 monclient: dump: 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.956+0000 7ff3bc660640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.956+0000 7ff3bc660640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.956+0000 7ff3bc660640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.956+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b404c4a0 con 0x7ff3b40a15b0 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.956+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff3b00027d0 con 0x7ff3b40a15b0 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.957+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7ff3b0046920 con 0x7ff3b40a15b0 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.957+0000 7ff3b9bfb640 10 monclient: get_auth_request con 0x7ff39803a6c0 auth_method 0 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.961+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7ff3b0014d20 con 0x7ff3b40a15b0 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.961+0000 7ff3aa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.961+0000 7ff3aa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:07.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.981+0000 7faac3f9d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.981+0000 7faac3f9d640 10 monclient: build_initial_monmap 2024-09-18T13:59:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.981+0000 7faac3f9d640 10 monclient: monmap: 2024-09-18T13:59:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:07.982270+0000 2024-09-18T13:59:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:07.982270+0000 2024-09-18T13:59:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:07.984 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:07.984 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:07.984 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:07.984 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:07.984 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:07.984 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:07.984 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.982+0000 7faac3f9d640 1 Processor -- start 2024-09-18T13:59:07.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.982+0000 7faac3f9d640 1 -- start start 2024-09-18T13:59:07.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.982+0000 7faac3f9d640 10 monclient: init 2024-09-18T13:59:07.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac3f9d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:07.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac3f9d640 10 monclient: _add_conns ranks=[1,5,0,8,6,2,3,4,7] 2024-09-18T13:59:07.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac3f9d640 10 monclient(hunting): picked mon.noname-f con 0x7faabc06e810 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:07.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac3f9d640 10 monclient(hunting): picked mon.noname-h con 0x7faabc0a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:07.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac3f9d640 10 monclient(hunting): picked mon.noname-a con 0x7faabc0a1080 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:07.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac3f9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc05ed70 con 0x7faabc0a1080 2024-09-18T13:59:07.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac3f9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc097620 con 0x7faabc06e810 2024-09-18T13:59:07.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac3f9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc097340 con 0x7faabc0a0cb0 2024-09-18T13:59:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac3f9d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac3f9d640 10 monclient(hunting): authenticate will time out at 763.321405s 2024-09-18T13:59:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac253a640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7faabc0a1080 0x7faabc13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36898/0 (socket says 172.21.15.45:36898) 2024-09-18T13:59:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac253a640 1 -- 172.21.15.45:0/638204478 learned_addr learned my addr 172.21.15.45:0/638204478 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208763261 0 0) 0x7faabc097340 con 0x7faabc0a0cb0 2024-09-18T13:59:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 10 monclient(hunting): my global_id is 4307 2024-09-18T13:59:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa80034e0 con 0x7faabc0a0cb0 2024-09-18T13:59:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274011828 0 0) 0x7faabc097620 con 0x7faabc06e810 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 10 monclient(hunting): my global_id is 4366 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faabc097340 con 0x7faabc06e810 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955675468 0 0) 0x7faabc05ed70 con 0x7faabc0a1080 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 10 monclient(hunting): my global_id is 4356 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.983+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa8003990 con 0x7faabc0a1080 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2223799933 0 0) 0x7faaa80034e0 con 0x7faabc0a0cb0 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 >> v1:172.21.15.84:6789/0 conn(0x7faabc06e810 legacy=0x7faabc0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 >> v1:172.21.15.45:6789/0 conn(0x7faabc0a1080 legacy=0x7faabc13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 10 monclient: _finish_hunting 0 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 1 monclient: found mon.noname-h 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc0a1660 con 0x7faabc0a0cb0 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 10 monclient: _finish_auth 0 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faaac001830 con 0x7faabc0a0cb0 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 10 monclient: dump: 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac3f9d640 5 monclient: authenticate success, global_id 4307 2024-09-18T13:59:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac3f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faaac001a10 con 0x7faabc0a0cb0 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 1 -- 172.21.15.45:0/638204478 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faaac002240 con 0x7faabc0a0cb0 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac0d37640 10 monclient: dump: 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac3f9d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.984+0000 7faac3f9d640 4 set_mon_vals no callback set 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.985+0000 7faac3f9d640 10 monclient: shutdown 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.985+0000 7faac3f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.985+0000 7faac3f9d640 1 -- 172.21.15.45:0/638204478 >> v1:172.21.15.84:6791/0 conn(0x7faabc0a0cb0 legacy=0x7faabc137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.985+0000 7faac3f9d640 1 -- 172.21.15.45:0/638204478 shutdown_connections 2024-09-18T13:59:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.985+0000 7faac3f9d640 1 -- 172.21.15.45:0/638204478 >> 172.21.15.45:0/638204478 conn(0x7faabc0a5bb0 msgr2=0x7faabc0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.985+0000 7faac3f9d640 1 -- 172.21.15.45:0/638204478 shutdown_connections 2024-09-18T13:59:08.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.985+0000 7faac3f9d640 1 -- 172.21.15.45:0/638204478 wait complete. 2024-09-18T13:59:08.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.985+0000 7faac3f9d640 10 monclient: build_initial_monmap 2024-09-18T13:59:08.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.985+0000 7faac3f9d640 10 monclient: monmap: 2024-09-18T13:59:08.000 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:08.000 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:08.000 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:08.000 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:08.000 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:08.000 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:08.000 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:08.000 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.986+0000 7faac3f9d640 1 Processor -- start 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.986+0000 7faac3f9d640 1 -- start start 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.986+0000 7faac3f9d640 10 monclient: init 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.986+0000 7faac3f9d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.986+0000 7faac3f9d640 10 monclient: _add_conns ranks=[5,3,7,8,2,4,6,1,0] 2024-09-18T13:59:08.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.986+0000 7faac3f9d640 10 monclient(hunting): picked mon.noname-h con 0x7faabc06e810 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.986+0000 7faac3f9d640 10 monclient(hunting): picked mon.noname-g con 0x7faabc0a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.986+0000 7faac3f9d640 10 monclient(hunting): picked mon.noname-i con 0x7faabc0a1080 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.986+0000 7faac3f9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc09fa10 con 0x7faabc0a0cb0 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.986+0000 7faac1d39640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7faabc06e810 0x7faabc0998b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:55954/0 (socket says 172.21.15.45:55954) 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.986+0000 7faac1d39640 1 -- 172.21.15.45:0/3633265442 learned_addr learned my addr 172.21.15.45:0/3633265442 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.986+0000 7faac3f9d640 1 -- 172.21.15.45:0/3633265442 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc0a2270 con 0x7faabc06e810 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faac3f9d640 1 -- 172.21.15.45:0/3633265442 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc09a6d0 con 0x7faabc0a1080 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faac3f9d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faac3f9d640 10 monclient(hunting): authenticate will time out at 763.325138s 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216865494 0 0) 0x7faabc0a2270 con 0x7faabc06e810 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): my global_id is 4316 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa00034e0 con 0x7faabc06e810 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661343301 0 0) 0x7faabc09a6d0 con 0x7faabc0a1080 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): my global_id is 4399 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faabc0a2270 con 0x7faabc0a1080 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1924484914 0 0) 0x7faaa00034e0 con 0x7faabc06e810 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faabc09a6d0 con 0x7faabc06e810 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faab8001c20 con 0x7faabc06e810 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): dump: 2024-09-18T13:59:08.003 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.004 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1323406401 0 0) 0x7faabc0a2270 con 0x7faabc0a1080 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faaa00034e0 con 0x7faabc0a1080 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faab4001d00 con 0x7faabc0a1080 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient(hunting): dump: 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3478314587 0 0) 0x7faabc09a6d0 con 0x7faabc06e810 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 >> v1:172.21.15.84:6792/0 conn(0x7faabc0a1080 legacy=0x7faabc1459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 >> v1:172.21.15.84:6790/0 conn(0x7faabc0a0cb0 legacy=0x7faabc099fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 monclient: found mon.h 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc09b910 con 0x7faabc06e810 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faab1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faac3f9d640 5 monclient: authenticate success, global_id 4316 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faac3f9d640 10 monclient: _renew_subs 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faac3f9d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.987+0000 7faac3f9d640 1 -- 172.21.15.45:0/3633265442 --> v1:172.21.15.84:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faabc09b400 con 0x7faabc06e810 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.988+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.5 v1:172.21.15.84:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faab8002110 con 0x7faabc06e810 2024-09-18T13:59:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.988+0000 7faab1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.988+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.5 v1:172.21.15.84:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faab8004f10 con 0x7faabc06e810 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.988+0000 7faab1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.988+0000 7faab1ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.988+0000 7faab1ffb640 10 monclient: dump: 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.988+0000 7faab3fff640 4 set_mon_vals no callback set 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.988+0000 7faac3f9d640 10 monclient: _renew_subs 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.988+0000 7faac3f9d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.988+0000 7faac3f9d640 1 -- 172.21.15.45:0/3633265442 --> v1:172.21.15.84:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc09bfd0 con 0x7faabc06e810 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.989+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.5 v1:172.21.15.84:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7faab800fee0 con 0x7faabc06e810 2024-09-18T13:59:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.989+0000 7faac3f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.989+0000 7faac3f9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.989+0000 7faac3f9d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T13:59:08.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.989+0000 7faac3f9d640 1 -- 172.21.15.45:0/3633265442 --> v1:172.21.15.84:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faabc04c4a0 con 0x7faabc06e810 2024-09-18T13:59:08.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.989+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.5 v1:172.21.15.84:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7faab8046aa0 con 0x7faabc06e810 2024-09-18T13:59:08.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.990+0000 7faac1538640 10 monclient: get_auth_request con 0x7faaa003a420 auth_method 0 2024-09-18T13:59:08.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.991+0000 7f7f98714640 10 monclient: get_monmap_and_config 2024-09-18T13:59:08.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.991+0000 7f7f98714640 10 monclient: build_initial_monmap 2024-09-18T13:59:08.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.991+0000 7f7f98714640 10 monclient: monmap: 2024-09-18T13:59:08.013 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:07.992660+0000 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:07.992660+0000 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.992+0000 7f7f98714640 1 Processor -- start 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.992+0000 7f7f98714640 1 -- start start 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.992+0000 7f7f98714640 10 monclient: init 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f98714640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f98714640 10 monclient: _add_conns ranks=[6,5,7,0,1,8,4,2,3] 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f98714640 10 monclient(hunting): picked mon.noname-d con 0x7f7f9006e350 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f98714640 10 monclient(hunting): picked mon.noname-h con 0x7f7f9009f530 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f98714640 10 monclient(hunting): picked mon.noname-i con 0x7f7f90136d60 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f98714640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9005ed70 con 0x7f7f9009f530 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f98714640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9006c480 con 0x7f7f9006e350 2024-09-18T13:59:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f98714640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9006e720 con 0x7f7f90136d60 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f98714640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f98714640 10 monclient(hunting): authenticate will time out at 763.331343s 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f964b0640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f7f9006e350 0x7f7f9009ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34764/0 (socket says 172.21.15.45:34764) 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f964b0640 1 -- 172.21.15.45:0/3430163335 learned_addr learned my addr 172.21.15.45:0/3430163335 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f954ae640 1 -- 172.21.15.45:0/3430163335 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030069821 0 0) 0x7f7f9006c480 con 0x7f7f9006e350 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f954ae640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f954ae640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f954ae640 10 monclient(hunting): my global_id is 4362 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.993+0000 7f7f954ae640 1 -- 172.21.15.45:0/3430163335 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f7c0034e0 con 0x7f7f9006e350 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 1 -- 172.21.15.45:0/3430163335 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233780362 0 0) 0x7f7f9006e720 con 0x7f7f90136d60 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient(hunting): my global_id is 4408 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 1 -- 172.21.15.45:0/3430163335 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f9006c480 con 0x7f7f90136d60 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 1 -- 172.21.15.45:0/3430163335 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2575154381 0 0) 0x7f7f7c0034e0 con 0x7f7f9006e350 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 1 -- 172.21.15.45:0/3430163335 >> v1:172.21.15.84:6792/0 conn(0x7f7f90136d60 legacy=0x7f7f90139140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 1 -- 172.21.15.45:0/3430163335 >> v1:172.21.15.84:6791/0 conn(0x7f7f9009f530 legacy=0x7f7f90135640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient: _finish_hunting 0 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 1 monclient: found mon.noname-d 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 1 -- 172.21.15.45:0/3430163335 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9006d890 con 0x7f7f9006e350 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient: _finish_auth 0 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 1 -- 172.21.15.45:0/3430163335 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f8c001760 con 0x7f7f9006e350 2024-09-18T13:59:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient: dump: 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f98714640 5 monclient: authenticate success, global_id 4362 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f98714640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 1 -- 172.21.15.45:0/3430163335 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7f8c0018c0 con 0x7f7f9006e350 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 1 -- 172.21.15.45:0/3430163335 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f8c004860 con 0x7f7f9006e350 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f954ae640 10 monclient: dump: 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.020 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.5 v1:172.21.15.84:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7faab80119d0 con 0x7faabc06e810 2024-09-18T13:59:08.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7faab1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7faab1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f98714640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.994+0000 7f7f98714640 4 set_mon_vals no callback set 2024-09-18T13:59:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.995+0000 7f7f98714640 10 monclient: shutdown 2024-09-18T13:59:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.995+0000 7f7f98714640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.995+0000 7f7f98714640 1 -- 172.21.15.45:0/3430163335 >> v1:172.21.15.45:6792/0 conn(0x7f7f9006e350 legacy=0x7f7f9009ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.995+0000 7f7f98714640 1 -- 172.21.15.45:0/3430163335 shutdown_connections 2024-09-18T13:59:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.995+0000 7f7f98714640 1 -- 172.21.15.45:0/3430163335 >> 172.21.15.45:0/3430163335 conn(0x7f7f900a44c0 msgr2=0x7f7f900a0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.995+0000 7f7f98714640 1 -- 172.21.15.45:0/3430163335 shutdown_connections 2024-09-18T13:59:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.995+0000 7f7f98714640 1 -- 172.21.15.45:0/3430163335 wait complete. 2024-09-18T13:59:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.996+0000 7f7f98714640 10 monclient: build_initial_monmap 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.996+0000 7f7f98714640 10 monclient: monmap: 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.996+0000 7f7f98714640 1 Processor -- start 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f98714640 1 -- start start 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f98714640 10 monclient: init 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f98714640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f98714640 10 monclient: _add_conns ranks=[3,1,8,2,6,5,0,7,4] 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f98714640 10 monclient(hunting): picked mon.noname-g con 0x7f7f9006e350 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f98714640 10 monclient(hunting): picked mon.noname-f con 0x7f7f9009f530 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f98714640 10 monclient(hunting): picked mon.noname-e con 0x7f7f90136d60 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f98714640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9006cb80 con 0x7f7f9009f530 2024-09-18T13:59:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f98714640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f90069c90 con 0x7f7f9006e350 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f98714640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9009dcc0 con 0x7f7f90136d60 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f98714640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f98714640 10 monclient(hunting): authenticate will time out at 763.335680s 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f96cb1640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f7f90136d60 0x7f7f9014eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51858/0 (socket says 172.21.15.45:51858) 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.997+0000 7f7f96cb1640 1 -- 172.21.15.45:0/1411860396 learned_addr learned my addr 172.21.15.45:0/1411860396 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986913217 0 0) 0x7f7f90069c90 con 0x7f7f9006e350 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): my global_id is 4413 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f740034e0 con 0x7f7f9006e350 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607599117 0 0) 0x7f7f9006cb80 con 0x7f7f9009f530 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): my global_id is 4375 2024-09-18T13:59:08.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f90069c90 con 0x7f7f9009f530 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685584591 0 0) 0x7f7f9009dcc0 con 0x7f7f90136d60 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): my global_id is 4328 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f9006cb80 con 0x7f7f90136d60 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3085112741 0 0) 0x7f7f740034e0 con 0x7f7f9006e350 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7f9009dcc0 con 0x7f7f9006e350 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f8c0016e0 con 0x7f7f9006e350 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): dump: 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.031 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.032 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1763547034 0 0) 0x7f7f90069c90 con 0x7f7f9009f530 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7f740034e0 con 0x7f7f9009f530 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f800019d0 con 0x7f7f9009f530 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): dump: 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1731755837 0 0) 0x7f7f9006cb80 con 0x7f7f90136d60 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7f90069c90 con 0x7f7f90136d60 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f88001e60 con 0x7f7f90136d60 2024-09-18T13:59:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient(hunting): dump: 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3949866816 0 0) 0x7f7f9009dcc0 con 0x7f7f9006e350 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 >> v1:172.21.15.45:6793/0 conn(0x7f7f90136d60 legacy=0x7f7f9014eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 >> v1:172.21.15.84:6789/0 conn(0x7f7f9009f530 legacy=0x7f7f9014b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 monclient: found mon.g 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90068b20 con 0x7f7f9006e350 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.998+0000 7f7f867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f98714640 5 monclient: authenticate success, global_id 4413 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f98714640 10 monclient: _renew_subs 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f98714640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f98714640 1 -- 172.21.15.45:0/1411860396 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f90068da0 con 0x7f7f9006e350 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f98714640 10 monclient: _renew_subs 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f98714640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f98714640 1 -- 172.21.15.45:0/1411860396 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f90097540 con 0x7f7f9006e350 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7f8c004a50 con 0x7f7f9006e350 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f867fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f8c004f40 con 0x7f7f9006e350 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f867fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f867fc640 10 monclient: dump: 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:07.999+0000 7f7f94cad640 4 set_mon_vals no callback set 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.000+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7f8c00ff10 con 0x7f7f9006e350 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.000+0000 7f7f98714640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.000+0000 7f7f98714640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.000+0000 7f7f98714640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.000+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f7f8c0468f0 con 0x7f7f9006e350 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.000+0000 7f7f98714640 1 -- 172.21.15.45:0/1411860396 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f78005010 con 0x7f7f9006e350 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.001+0000 7f7f95caf640 10 monclient: get_auth_request con 0x7f7f7403aa10 auth_method 0 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.005+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.3 v1:172.21.15.84:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f7f8c014e70 con 0x7f7f9006e350 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.005+0000 7f7f867fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.005+0000 7f7f867fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.089+0000 7f7ec6b9d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-18T13:59:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.089+0000 7f7ec6b9d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-18T13:59:08.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.089+0000 7f7ec6b9d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:08.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.089+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/4007419154 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7ea40052c0 con 0x7f7ec00a1080 2024-09-18T13:59:08.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.090+0000 7f7ebcff9640 1 -- 172.21.15.45:0/4007419154 <== mon.4 v1:172.21.15.45:6791/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (unknown 1823589948 0 847495621) 0x7f7eb8014bd0 con 0x7f7ec00a1080 2024-09-18T13:59:08.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.090+0000 7f7ebcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-18T13:59:08.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.090+0000 7f7ebcff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:08.093 INFO:teuthology.orchestra.run.smithi045.stdout:47244640259 2024-09-18T13:59:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.093+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/4007419154 >> v1:172.21.15.84:6812/31478 conn(0x7f7ea003ab40 legacy=0x7f7ea003cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.093+0000 7f7ec6b9d640 10 monclient: shutdown 2024-09-18T13:59:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.093+0000 7f7ec6b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.093+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/4007419154 >> v1:172.21.15.45:6791/0 conn(0x7f7ec00a1080 legacy=0x7f7ec0149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.093+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/4007419154 shutdown_connections 2024-09-18T13:59:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.093+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/4007419154 >> 172.21.15.45:0/4007419154 conn(0x7f7ec00a5bb0 msgr2=0x7f7ec00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.094+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/4007419154 shutdown_connections 2024-09-18T13:59:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.094+0000 7f7ec6b9d640 1 -- 172.21.15.45:0/4007419154 wait complete. 2024-09-18T13:59:08.109 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.4 2024-09-18T13:59:08.110 DEBUG:teuthology.parallel:result is None 2024-09-18T13:59:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.124+0000 7f521855d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-18T13:59:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.124+0000 7f521855d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-18T13:59:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.124+0000 7f521855d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.124+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5210146230 con 0x7f52100a1080 2024-09-18T13:59:08.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.125+0000 7f52067fc640 1 -- 172.21.15.45:0/2998896021 <== mon.3 v1:172.21.15.84:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (unknown 383520633 0 847495621) 0x7f5208015000 con 0x7f52100a1080 2024-09-18T13:59:08.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.125+0000 7f52067fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-18T13:59:08.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.125+0000 7f52067fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:08.126 INFO:teuthology.orchestra.run.smithi045.stdout:47244640259 2024-09-18T13:59:08.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.127+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 >> v1:172.21.15.84:6812/31478 conn(0x7f51f403a7d0 legacy=0x7f51f403cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.127+0000 7f521855d640 10 monclient: shutdown 2024-09-18T13:59:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.127+0000 7f521855d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.127+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 >> v1:172.21.15.84:6790/0 conn(0x7f52100a1080 legacy=0x7f521014de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.127+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 shutdown_connections 2024-09-18T13:59:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.127+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 >> 172.21.15.45:0/2998896021 conn(0x7f52100a5bb0 msgr2=0x7f52100a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.127+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 shutdown_connections 2024-09-18T13:59:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.127+0000 7f521855d640 1 -- 172.21.15.45:0/2998896021 wait complete. 2024-09-18T13:59:08.138 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.3 2024-09-18T13:59:08.138 DEBUG:teuthology.parallel:result is None 2024-09-18T13:59:08.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.165+0000 7fce8c846640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-18T13:59:08.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.165+0000 7fce8c846640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-18T13:59:08.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.165+0000 7fce8c846640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:08.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.165+0000 7fce8c846640 1 -- 172.21.15.45:0/3118706358 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fce8804b720 con 0x7fce8809e180 2024-09-18T13:59:08.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.165+0000 7fce7e7fc640 1 -- 172.21.15.45:0/3118706358 <== mon.4 v1:172.21.15.45:6791/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 847495621) 0x7fce80014e60 con 0x7fce8809e180 2024-09-18T13:59:08.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.165+0000 7fce7e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-18T13:59:08.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.165+0000 7fce7e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:08.166 INFO:teuthology.orchestra.run.smithi045.stdout:47244640259 2024-09-18T13:59:08.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.167+0000 7fce8c846640 1 -- 172.21.15.45:0/3118706358 >> v1:172.21.15.84:6812/31478 conn(0x7fce6803a650 legacy=0x7fce6803cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.167+0000 7fce8c846640 10 monclient: shutdown 2024-09-18T13:59:08.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.167+0000 7fce8c846640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:08.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.167+0000 7fce8c846640 1 -- 172.21.15.45:0/3118706358 >> v1:172.21.15.45:6791/0 conn(0x7fce8809e180 legacy=0x7fce88149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.167+0000 7fce8c846640 1 -- 172.21.15.45:0/3118706358 shutdown_connections 2024-09-18T13:59:08.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.167+0000 7fce8c846640 1 -- 172.21.15.45:0/3118706358 >> 172.21.15.45:0/3118706358 conn(0x7fce88068b40 msgr2=0x7fce88062fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.167+0000 7fce8c846640 1 -- 172.21.15.45:0/3118706358 shutdown_connections 2024-09-18T13:59:08.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.167+0000 7fce8c846640 1 -- 172.21.15.45:0/3118706358 wait complete. 2024-09-18T13:59:08.179 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.2 2024-09-18T13:59:08.179 DEBUG:teuthology.parallel:result is None 2024-09-18T13:59:08.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.179+0000 7f7f98714640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-18T13:59:08.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.179+0000 7f7f98714640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-18T13:59:08.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.179+0000 7f7f98714640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:08.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.179+0000 7f7f98714640 1 -- 172.21.15.45:0/1411860396 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7f780052c0 con 0x7f7f9006e350 2024-09-18T13:59:08.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.180+0000 7f7f867fc640 1 -- 172.21.15.45:0/1411860396 <== mon.3 v1:172.21.15.84:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 203001719) 0x7f7f8c014c90 con 0x7f7f9006e350 2024-09-18T13:59:08.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.180+0000 7f7f867fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-18T13:59:08.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.180+0000 7f7f867fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:08.181 INFO:teuthology.orchestra.run.smithi045.stdout:51539607556 2024-09-18T13:59:08.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.182+0000 7f7f98714640 1 -- 172.21.15.45:0/1411860396 >> v1:172.21.15.84:6812/31478 conn(0x7f7f7403aa10 legacy=0x7f7f7403cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.182+0000 7f7f98714640 10 monclient: shutdown 2024-09-18T13:59:08.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.182+0000 7f7f98714640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:08.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.182+0000 7f7f98714640 1 -- 172.21.15.45:0/1411860396 >> v1:172.21.15.84:6790/0 conn(0x7f7f9006e350 legacy=0x7f7f90146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.182+0000 7f7f98714640 1 -- 172.21.15.45:0/1411860396 shutdown_connections 2024-09-18T13:59:08.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.182+0000 7f7f98714640 1 -- 172.21.15.45:0/1411860396 >> 172.21.15.45:0/1411860396 conn(0x7f7f900a44c0 msgr2=0x7f7f9012d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.182+0000 7f7f98714640 1 -- 172.21.15.45:0/1411860396 shutdown_connections 2024-09-18T13:59:08.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.182+0000 7f7f98714640 1 -- 172.21.15.45:0/1411860396 wait complete. 2024-09-18T13:59:08.192 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607555 got 51539607556 for osd.1 2024-09-18T13:59:08.192 DEBUG:teuthology.parallel:result is None 2024-09-18T13:59:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.224+0000 7faac3f9d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-18T13:59:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.224+0000 7faac3f9d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-18T13:59:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.224+0000 7faac3f9d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T13:59:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.224+0000 7faac3f9d640 1 -- 172.21.15.45:0/3633265442 --> v1:172.21.15.84:6791/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faabc04b720 con 0x7faabc06e810 2024-09-18T13:59:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.225+0000 7faab1ffb640 1 -- 172.21.15.45:0/3633265442 <== mon.5 v1:172.21.15.84:6791/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 847495621) 0x7faab8014d80 con 0x7faabc06e810 2024-09-18T13:59:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.225+0000 7faab1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-18T13:59:08.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.225+0000 7faab1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:08.226 INFO:teuthology.orchestra.run.smithi045.stdout:47244640259 2024-09-18T13:59:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.227+0000 7faac3f9d640 1 -- 172.21.15.45:0/3633265442 >> v1:172.21.15.84:6812/31478 conn(0x7faaa003a420 legacy=0x7faaa003c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.227+0000 7faac3f9d640 10 monclient: shutdown 2024-09-18T13:59:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.227+0000 7faac3f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.227+0000 7faac3f9d640 1 -- 172.21.15.45:0/3633265442 >> v1:172.21.15.84:6791/0 conn(0x7faabc06e810 legacy=0x7faabc0998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.227+0000 7faac3f9d640 1 -- 172.21.15.45:0/3633265442 shutdown_connections 2024-09-18T13:59:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.227+0000 7faac3f9d640 1 -- 172.21.15.45:0/3633265442 >> 172.21.15.45:0/3633265442 conn(0x7faabc0a5bb0 msgr2=0x7faabc12f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.227+0000 7faac3f9d640 1 -- 172.21.15.45:0/3633265442 shutdown_connections 2024-09-18T13:59:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.227+0000 7faac3f9d640 1 -- 172.21.15.45:0/3633265442 wait complete. 2024-09-18T13:59:08.238 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.0 2024-09-18T13:59:08.238 DEBUG:teuthology.parallel:result is None 2024-09-18T13:59:08.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.238+0000 7ff3bc660640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-18T13:59:08.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.238+0000 7ff3bc660640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-18T13:59:08.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.238+0000 7ff3bc660640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:08.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.238+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff3b404b720 con 0x7ff3b40a15b0 2024-09-18T13:59:08.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.238+0000 7ff3aa7fc640 1 -- 172.21.15.45:0/4219683737 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (unknown 2131975755 0 847495621) 0x7ff3b0014d20 con 0x7ff3b40a15b0 2024-09-18T13:59:08.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.238+0000 7ff3aa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-18T13:59:08.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.238+0000 7ff3aa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:08.239 INFO:teuthology.orchestra.run.smithi045.stdout:47244640259 2024-09-18T13:59:08.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.240+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 >> v1:172.21.15.84:6812/31478 conn(0x7ff39803a6c0 legacy=0x7ff39803cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.240+0000 7ff3bc660640 10 monclient: shutdown 2024-09-18T13:59:08.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.240+0000 7ff3bc660640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:08.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.240+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 >> v1:172.21.15.45:6790/0 conn(0x7ff3b40a15b0 legacy=0x7ff3b409f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.240+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 shutdown_connections 2024-09-18T13:59:08.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.240+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 >> 172.21.15.45:0/4219683737 conn(0x7ff3b40a5bb0 msgr2=0x7ff3b4132be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.240+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 shutdown_connections 2024-09-18T13:59:08.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.240+0000 7ff3bc660640 1 -- 172.21.15.45:0/4219683737 wait complete. 2024-09-18T13:59:08.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.5 2024-09-18T13:59:08.251 DEBUG:teuthology.parallel:result is None 2024-09-18T13:59:08.251 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T13:59:08.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T13:59:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.386+0000 7fdd066c9640 10 monclient: get_monmap_and_config 2024-09-18T13:59:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.386+0000 7fdd066c9640 10 monclient: build_initial_monmap 2024-09-18T13:59:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.386+0000 7fdd066c9640 10 monclient: monmap: 2024-09-18T13:59:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:08.387834+0000 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:08.387834+0000 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:08.388 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.387+0000 7fdd066c9640 1 Processor -- start 2024-09-18T13:59:08.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.387+0000 7fdd066c9640 1 -- start start 2024-09-18T13:59:08.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.387+0000 7fdd066c9640 10 monclient: init 2024-09-18T13:59:08.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 10 monclient: _add_conns ranks=[0,5,4,3,2,1,6,7,8] 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 10 monclient(hunting): picked mon.noname-a con 0x7fdd0009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 10 monclient(hunting): picked mon.noname-h con 0x7fdd0009fba0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 10 monclient(hunting): picked mon.noname-c con 0x7fdd0009ff70 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0005ed70 con 0x7fdd0009f7d0 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0009dfc0 con 0x7fdd0009ff70 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0009dce0 con 0x7fdd0009fba0 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 10 monclient(hunting): authenticate will time out at 763.726203s 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdd0009f7d0 0x7fdd00132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36904/0 (socket says 172.21.15.45:36904) 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfffff640 1 -- 172.21.15.45:0/1406042358 learned_addr learned my addr 172.21.15.45:0/1406042358 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582480734 0 0) 0x7fdd0009dfc0 con 0x7fdd0009ff70 2024-09-18T13:59:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient(hunting): my global_id is 4369 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce80034e0 con 0x7fdd0009ff70 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161813410 0 0) 0x7fdd0005ed70 con 0x7fdd0009f7d0 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient(hunting): my global_id is 4365 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd0009dfc0 con 0x7fdd0009f7d0 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312373600 0 0) 0x7fdd0009dce0 con 0x7fdd0009fba0 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient(hunting): my global_id is 4325 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd0005ed70 con 0x7fdd0009fba0 2024-09-18T13:59:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3372927016 0 0) 0x7fdce80034e0 con 0x7fdd0009ff70 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 >> v1:172.21.15.84:6791/0 conn(0x7fdd0009fba0 legacy=0x7fdd00137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 >> v1:172.21.15.45:6789/0 conn(0x7fdd0009f7d0 legacy=0x7fdd00132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 monclient: found mon.noname-c 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd000a1c00 con 0x7fdd0009ff70 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdcf8001bb0 con 0x7fdd0009ff70 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 5 monclient: authenticate success, global_id 4369 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient: dump: 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdcf8001d90 con 0x7fdd0009ff70 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 1 -- 172.21.15.45:0/1406042358 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdcf8002340 con 0x7fdd0009ff70 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdd066c9640 4 set_mon_vals no callback set 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.388+0000 7fdcfeffd640 10 monclient: dump: 2024-09-18T13:59:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.389+0000 7fdd066c9640 10 monclient: shutdown 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.389+0000 7fdd066c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.389+0000 7fdd066c9640 1 -- 172.21.15.45:0/1406042358 >> v1:172.21.15.45:6791/0 conn(0x7fdd0009ff70 legacy=0x7fdd0013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.389+0000 7fdd066c9640 1 -- 172.21.15.45:0/1406042358 shutdown_connections 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.389+0000 7fdd066c9640 1 -- 172.21.15.45:0/1406042358 >> 172.21.15.45:0/1406042358 conn(0x7fdd000630b0 msgr2=0x7fdd000634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.389+0000 7fdd066c9640 1 -- 172.21.15.45:0/1406042358 shutdown_connections 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.389+0000 7fdd066c9640 1 -- 172.21.15.45:0/1406042358 wait complete. 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.389+0000 7fdd066c9640 10 monclient: build_initial_monmap 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.389+0000 7fdd066c9640 10 monclient: monmap: 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 1 Processor -- start 2024-09-18T13:59:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 1 -- start start 2024-09-18T13:59:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 10 monclient: init 2024-09-18T13:59:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 10 monclient: _add_conns ranks=[3,2,7,1,6,8,0,4,5] 2024-09-18T13:59:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 10 monclient(hunting): picked mon.noname-g con 0x7fdd0009f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 10 monclient(hunting): picked mon.noname-b con 0x7fdd0009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 10 monclient(hunting): picked mon.noname-i con 0x7fdd0009ff70 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd00121b90 con 0x7fdd0009fba0 2024-09-18T13:59:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd00141eb0 con 0x7fdd0009f7d0 2024-09-18T13:59:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0009d310 con 0x7fdd0009ff70 2024-09-18T13:59:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.390+0000 7fdd066c9640 10 monclient(hunting): authenticate will time out at 763.728995s 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdcff7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fdd0009fba0 0x7fdd00131a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51518/0 (socket says 172.21.15.45:51518) 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdcff7fe640 1 -- 172.21.15.45:0/3012538009 learned_addr learned my addr 172.21.15.45:0/3012538009 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672930572 0 0) 0x7fdd00121b90 con 0x7fdd0009fba0 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): my global_id is 4340 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcec0034e0 con 0x7fdd0009fba0 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944337868 0 0) 0x7fdd0009d310 con 0x7fdd0009ff70 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): my global_id is 4417 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd00121b90 con 0x7fdd0009ff70 2024-09-18T13:59:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3274996363 0 0) 0x7fdcec0034e0 con 0x7fdd0009fba0 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd0009d310 con 0x7fdd0009fba0 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdcf4002bf0 con 0x7fdd0009fba0 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): dump: 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150826639 0 0) 0x7fdd00141eb0 con 0x7fdd0009f7d0 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient(hunting): my global_id is 4422 2024-09-18T13:59:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcec0034e0 con 0x7fdd0009f7d0 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3724189571 0 0) 0x7fdd0009d310 con 0x7fdd0009fba0 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 >> v1:172.21.15.84:6792/0 conn(0x7fdd0009ff70 legacy=0x7fdd00149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 >> v1:172.21.15.84:6790/0 conn(0x7fdd0009f7d0 legacy=0x7fdd001417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient: _finish_hunting 0 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 monclient: found mon.b 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd000a2560 con 0x7fdd0009fba0 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient: _finish_auth 0 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdd066c9640 5 monclient: authenticate success, global_id 4340 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdd066c9640 10 monclient: _renew_subs 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdd066c9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdd066c9640 1 -- 172.21.15.45:0/3012538009 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd001207e0 con 0x7fdd0009fba0 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdd066c9640 10 monclient: _renew_subs 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdd066c9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdd066c9640 1 -- 172.21.15.45:0/3012538009 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd00120060 con 0x7fdd0009fba0 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdcf4001760 con 0x7fdd0009fba0 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdcf4001c50 con 0x7fdd0009fba0 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdcfe7fc640 4 set_mon_vals no callback set 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.391+0000 7fdce7fff640 10 monclient: dump: 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.392+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fdcf4001db0 con 0x7fdd0009fba0 2024-09-18T13:59:08.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.392+0000 7fdd066c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.392+0000 7fdd066c9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.392+0000 7fdd066c9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:08.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.392+0000 7fdd066c9640 1 -- 172.21.15.45:0/3012538009 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce0005010 con 0x7fdd0009fba0 2024-09-18T13:59:08.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.393+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7fdcf4046a60 con 0x7fdd0009fba0 2024-09-18T13:59:08.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.393+0000 7fdcfffff640 10 monclient: get_auth_request con 0x7fdcec039f50 auth_method 0 2024-09-18T13:59:08.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.397+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fdcf4015340 con 0x7fdd0009fba0 2024-09-18T13:59:08.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.397+0000 7fdce7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.397+0000 7fdce7fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:08.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.556+0000 7fdd066c9640 1 -- 172.21.15.45:0/3012538009 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdce0000b90 con 0x7fdcec039f50 2024-09-18T13:59:08.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.558+0000 7fdce7fff640 1 -- 172.21.15.45:0/3012538009 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40972 (unknown 2965378022 0 3986473521) 0x7fdce0000b90 con 0x7fdcec039f50 2024-09-18T13:59:08.560 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:59:08.560 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-18T13:59:08.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.560+0000 7fdd066c9640 1 -- 172.21.15.45:0/3012538009 >> v1:172.21.15.84:6812/31478 conn(0x7fdcec039f50 legacy=0x7fdcec03c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.560+0000 7fdd066c9640 10 monclient: shutdown 2024-09-18T13:59:08.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.560+0000 7fdd066c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:08.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.560+0000 7fdd066c9640 1 -- 172.21.15.45:0/3012538009 >> v1:172.21.15.45:6790/0 conn(0x7fdd0009fba0 legacy=0x7fdd00131a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.560+0000 7fdd066c9640 1 -- 172.21.15.45:0/3012538009 shutdown_connections 2024-09-18T13:59:08.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.560+0000 7fdd066c9640 1 -- 172.21.15.45:0/3012538009 >> 172.21.15.45:0/3012538009 conn(0x7fdd000630b0 msgr2=0x7fdd000a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.560+0000 7fdd066c9640 1 -- 172.21.15.45:0/3012538009 shutdown_connections 2024-09-18T13:59:08.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.560+0000 7fdd066c9640 1 -- 172.21.15.45:0/3012538009 wait complete. 2024-09-18T13:59:08.571 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-09-18T13:59:07.887850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1786248,"kb_used_data":1848,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":564444792,"statfs":{"total":579820584960,"available":577991467008,"internally_reserved":0,"allocated":1892352,"data_stored":1304944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.875626+0000","last_change":"2024-09-18T13:59:05.875874+0000","last_active":"2024-09-18T13:59:05.875626+0000","last_peered":"2024-09-18T13:59:05.875626+0000","last_clean":"2024-09-18T13:59:05.875626+0000","last_became_active":"2024-09-18T13:59:02.976936+0000","last_became_peered":"2024-09-18T13:59:02.976936+0000","last_unstale":"2024-09-18T13:59:05.875626+0000","last_undegraded":"2024-09-18T13:59:05.875626+0000","last_fullsized":"2024-09-18T13:59:05.875626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:20:37.890959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036210499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.035734+0000","last_change":"2024-09-18T13:59:05.035951+0000","last_active":"2024-09-18T13:59:05.035734+0000","last_peered":"2024-09-18T13:59:05.035734+0000","last_clean":"2024-09-18T13:59:05.035734+0000","last_became_active":"2024-09-18T13:59:03.705691+0000","last_became_peered":"2024-09-18T13:59:03.705691+0000","last_unstale":"2024-09-18T13:59:05.035734+0000","last_undegraded":"2024-09-18T13:59:05.035734+0000","last_fullsized":"2024-09-18T13:59:05.035734+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:45:16.896857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033710499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036257+0000","last_change":"2024-09-18T13:59:05.036455+0000","last_active":"2024-09-18T13:59:05.036257+0000","last_peered":"2024-09-18T13:59:05.036257+0000","last_clean":"2024-09-18T13:59:05.036257+0000","last_became_active":"2024-09-18T13:59:02.974240+0000","last_became_peered":"2024-09-18T13:59:02.974240+0000","last_unstale":"2024-09-18T13:59:05.036257+0000","last_undegraded":"2024-09-18T13:59:05.036257+0000","last_fullsized":"2024-09-18T13:59:05.036257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:13:40.312100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038595099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036428+0000","last_change":"2024-09-18T13:59:05.036936+0000","last_active":"2024-09-18T13:59:05.036428+0000","last_peered":"2024-09-18T13:59:05.036428+0000","last_clean":"2024-09-18T13:59:05.036428+0000","last_became_active":"2024-09-18T13:59:02.977186+0000","last_became_peered":"2024-09-18T13:59:02.977186+0000","last_unstale":"2024-09-18T13:59:05.036428+0000","last_undegraded":"2024-09-18T13:59:05.036428+0000","last_fullsized":"2024-09-18T13:59:05.036428+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:01:46.516709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065897000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"15'72","reported_seq":100,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-18T13:59:04.370201+0000","last_change":"2024-09-18T13:59:02.975019+0000","last_active":"2024-09-18T13:59:04.370201+0000","last_peered":"2024-09-18T13:59:04.370201+0000","last_clean":"2024-09-18T13:59:04.370201+0000","last_became_active":"2024-09-18T13:59:02.974164+0000","last_became_peered":"2024-09-18T13:59:02.974164+0000","last_unstale":"2024-09-18T13:59:04.370201+0000","last_undegraded":"2024-09-18T13:59:04.370201+0000","last_fullsized":"2024-09-18T13:59:04.370201+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T22:10:24.958774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036373+0000","last_change":"2024-09-18T13:59:05.036690+0000","last_active":"2024-09-18T13:59:05.036373+0000","last_peered":"2024-09-18T13:59:05.036373+0000","last_clean":"2024-09-18T13:59:05.036373+0000","last_became_active":"2024-09-18T13:59:02.977312+0000","last_became_peered":"2024-09-18T13:59:02.977312+0000","last_unstale":"2024-09-18T13:59:05.036373+0000","last_undegraded":"2024-09-18T13:59:05.036373+0000","last_fullsized":"2024-09-18T13:59:05.036373+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:03:06.377092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072047099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036440+0000","last_change":"2024-09-18T13:59:05.036440+0000","last_active":"2024-09-18T13:59:05.036440+0000","last_peered":"2024-09-18T13:59:05.036440+0000","last_clean":"2024-09-18T13:59:05.036440+0000","last_became_active":"2024-09-18T13:59:03.705659+0000","last_became_peered":"2024-09-18T13:59:03.705659+0000","last_unstale":"2024-09-18T13:59:05.036440+0000","last_undegraded":"2024-09-18T13:59:05.036440+0000","last_fullsized":"2024-09-18T13:59:05.036440+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:49:16.930535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037781799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.875655+0000","last_change":"2024-09-18T13:59:05.875836+0000","last_active":"2024-09-18T13:59:05.875655+0000","last_peered":"2024-09-18T13:59:05.875655+0000","last_clean":"2024-09-18T13:59:05.875655+0000","last_became_active":"2024-09-18T13:59:03.705710+0000","last_became_peered":"2024-09-18T13:59:03.705710+0000","last_unstale":"2024-09-18T13:59:05.875655+0000","last_undegraded":"2024-09-18T13:59:05.875655+0000","last_fullsized":"2024-09-18T13:59:05.875655+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:12:55.202686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029541399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.035691+0000","last_change":"2024-09-18T13:59:05.035941+0000","last_active":"2024-09-18T13:59:05.035691+0000","last_peered":"2024-09-18T13:59:05.035691+0000","last_clean":"2024-09-18T13:59:05.035691+0000","last_became_active":"2024-09-18T13:59:03.705607+0000","last_became_peered":"2024-09-18T13:59:03.705607+0000","last_unstale":"2024-09-18T13:59:05.035691+0000","last_undegraded":"2024-09-18T13:59:05.035691+0000","last_fullsized":"2024-09-18T13:59:05.035691+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:25:38.392196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000366475,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":571475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297484,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074356,"statfs":{"total":96636764160,"available":96332140544,"internally_reserved":0,"allocated":131072,"data_stored":38431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":42566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640259,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":571475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":2,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297484,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074356,"statfs":{"total":96636764160,"available":96332140544,"internally_reserved":0,"allocated":131072,"data_stored":38431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640259,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":42566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T13:59:08.573 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T13:59:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.667+0000 7fab0575d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.667+0000 7fab0575d640 10 monclient: build_initial_monmap 2024-09-18T13:59:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.668+0000 7fab0575d640 10 monclient: monmap: 2024-09-18T13:59:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:08.668925+0000 2024-09-18T13:59:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:08.668925+0000 2024-09-18T13:59:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:08.670 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.668+0000 7fab0575d640 1 Processor -- start 2024-09-18T13:59:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.668+0000 7fab0575d640 1 -- start start 2024-09-18T13:59:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.668+0000 7fab0575d640 10 monclient: init 2024-09-18T13:59:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7fab0575d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7fab0575d640 10 monclient: _add_conns ranks=[1,5,7,3,4,2,6,0,8] 2024-09-18T13:59:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7fab0575d640 10 monclient(hunting): picked mon.noname-f con 0x7fab0009f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7fab0575d640 10 monclient(hunting): picked mon.noname-h con 0x7fab0009fba0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7fab0575d640 10 monclient(hunting): picked mon.noname-i con 0x7fab0009ff70 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7fab0575d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0005ed70 con 0x7fab0009f7d0 2024-09-18T13:59:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7fab0575d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0009dfc0 con 0x7fab0009fba0 2024-09-18T13:59:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7fab0575d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0009dce0 con 0x7fab0009ff70 2024-09-18T13:59:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7fab0575d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7fab0575d640 10 monclient(hunting): authenticate will time out at 764.007127s 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faaff7fe640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7fab0009ff70 0x7fab0013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:58288/0 (socket says 172.21.15.45:58288) 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faaff7fe640 1 -- 172.21.15.45:0/1819178357 learned_addr learned my addr 172.21.15.45:0/1819178357 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafeffd640 1 --1- 172.21.15.45:0/1819178357 >> v1:172.21.15.84:6789/0 conn(0x7fab0009f7d0 0x7fab00132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:43090/0 (socket says 172.21.15.45:43090) 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340019299 0 0) 0x7fab0005ed70 con 0x7fab0009f7d0 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient(hunting): my global_id is 4384 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faae80034e0 con 0x7fab0009f7d0 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709669398 0 0) 0x7fab0009dce0 con 0x7fab0009ff70 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient(hunting): my global_id is 4426 2024-09-18T13:59:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab0005ed70 con 0x7fab0009ff70 2024-09-18T13:59:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065017757 0 0) 0x7fab0009dfc0 con 0x7fab0009fba0 2024-09-18T13:59:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient(hunting): my global_id is 4334 2024-09-18T13:59:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab0009dce0 con 0x7fab0009fba0 2024-09-18T13:59:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3291412924 0 0) 0x7faae80034e0 con 0x7fab0009f7d0 2024-09-18T13:59:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 >> v1:172.21.15.84:6792/0 conn(0x7fab0009ff70 legacy=0x7fab0013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 >> v1:172.21.15.84:6791/0 conn(0x7fab0009fba0 legacy=0x7fab00137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 1 monclient: found mon.noname-f 2024-09-18T13:59:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab000a1c00 con 0x7fab0009f7d0 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faaf0001830 con 0x7fab0009f7d0 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7faafdffb640 10 monclient: dump: 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7fab0575d640 5 monclient: authenticate success, global_id 4384 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.669+0000 7fab0575d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faaf0001a10 con 0x7fab0009f7d0 2024-09-18T13:59:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7faafdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7faafdffb640 1 -- 172.21.15.45:0/1819178357 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faaf0004ae0 con 0x7fab0009f7d0 2024-09-18T13:59:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7faafdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7faafdffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7faafdffb640 10 monclient: dump: 2024-09-18T13:59:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7fab0575d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7fab0575d640 4 set_mon_vals no callback set 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7fab0575d640 10 monclient: shutdown 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7fab0575d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7fab0575d640 1 -- 172.21.15.45:0/1819178357 >> v1:172.21.15.84:6789/0 conn(0x7fab0009f7d0 legacy=0x7fab00132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7fab0575d640 1 -- 172.21.15.45:0/1819178357 shutdown_connections 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7fab0575d640 1 -- 172.21.15.45:0/1819178357 >> 172.21.15.45:0/1819178357 conn(0x7fab000630b0 msgr2=0x7fab000634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7fab0575d640 1 -- 172.21.15.45:0/1819178357 shutdown_connections 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.670+0000 7fab0575d640 1 -- 172.21.15.45:0/1819178357 wait complete. 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.671+0000 7fab0575d640 10 monclient: build_initial_monmap 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.671+0000 7fab0575d640 10 monclient: monmap: 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:08.684 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.671+0000 7fab0575d640 1 Processor -- start 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.671+0000 7fab0575d640 1 -- start start 2024-09-18T13:59:08.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.671+0000 7fab0575d640 10 monclient: init 2024-09-18T13:59:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.671+0000 7fab0575d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.671+0000 7fab0575d640 10 monclient: _add_conns ranks=[3,2,7,5,0,8,1,6,4] 2024-09-18T13:59:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.671+0000 7fab0575d640 10 monclient(hunting): picked mon.noname-g con 0x7fab0009f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7fab0575d640 10 monclient(hunting): picked mon.noname-b con 0x7fab0009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7fab0575d640 10 monclient(hunting): picked mon.noname-i con 0x7fab0009ff70 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7fab0575d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0009b9e0 con 0x7fab0009fba0 2024-09-18T13:59:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7fab0575d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0011e8c0 con 0x7fab0009f7d0 2024-09-18T13:59:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7fab0575d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0011eba0 con 0x7fab0009ff70 2024-09-18T13:59:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7fab0575d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7fab0575d640 10 monclient(hunting): authenticate will time out at 764.010090s 2024-09-18T13:59:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faafe7fc640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fab0009fba0 0x7fab00146d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51528/0 (socket says 172.21.15.45:51528) 2024-09-18T13:59:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faafe7fc640 1 -- 172.21.15.45:0/605545134 learned_addr learned my addr 172.21.15.45:0/605545134 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269144965 0 0) 0x7fab0009b9e0 con 0x7fab0009fba0 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): my global_id is 4349 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaec0034e0 con 0x7fab0009fba0 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184396382 0 0) 0x7fab0011eba0 con 0x7fab0009ff70 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): my global_id is 4435 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab0009b9e0 con 0x7fab0009ff70 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315908874 0 0) 0x7fab0011e8c0 con 0x7fab0009f7d0 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): my global_id is 4431 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab0011eba0 con 0x7fab0009f7d0 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 590434712 0 0) 0x7faaec0034e0 con 0x7fab0009fba0 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fab0011e8c0 con 0x7fab0009fba0 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faaf4001ef0 con 0x7fab0009fba0 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1741039513 0 0) 0x7fab0009b9e0 con 0x7fab0009ff70 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faaec0034e0 con 0x7fab0009ff70 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faaf8001870 con 0x7fab0009ff70 2024-09-18T13:59:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3165749557 0 0) 0x7fab0011eba0 con 0x7fab0009f7d0 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fab0009b9e0 con 0x7fab0009f7d0 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faaf0004ae0 con 0x7fab0009f7d0 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 16486765 0 0) 0x7fab0011e8c0 con 0x7fab0009fba0 2024-09-18T13:59:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 >> v1:172.21.15.84:6792/0 conn(0x7fab0009ff70 legacy=0x7fab0014a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 >> v1:172.21.15.84:6790/0 conn(0x7fab0009f7d0 legacy=0x7fab001317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 monclient: found mon.b 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0011ee80 con 0x7fab0009fba0 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7faae6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7fab0575d640 5 monclient: authenticate success, global_id 4349 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7fab0575d640 10 monclient: _renew_subs 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7fab0575d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.672+0000 7fab0575d640 1 -- 172.21.15.45:0/605545134 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0011d960 con 0x7fab0009fba0 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7fab0575d640 10 monclient: _renew_subs 2024-09-18T13:59:08.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7fab0575d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7fab0575d640 1 -- 172.21.15.45:0/605545134 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fab00132050 con 0x7fab0009fba0 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faaf4002480 con 0x7fab0009fba0 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7faae6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faaf40029b0 con 0x7fab0009fba0 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7faae6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7faafd7fa640 4 set_mon_vals no callback set 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7faae6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7faae6ffd640 10 monclient: dump: 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.694 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7faaf4002b10 con 0x7fab0009fba0 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7fab0575d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7fab0575d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7fab0575d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:08.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.673+0000 7fab0575d640 1 -- 172.21.15.45:0/605545134 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae0005010 con 0x7fab0009fba0 2024-09-18T13:59:08.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.674+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7faaf4046d20 con 0x7fab0009fba0 2024-09-18T13:59:08.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.674+0000 7faafeffd640 10 monclient: get_auth_request con 0x7faaec03aa80 auth_method 0 2024-09-18T13:59:08.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.678+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7faaf40156f0 con 0x7fab0009fba0 2024-09-18T13:59:08.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.678+0000 7faae6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.678+0000 7faae6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:08.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.837+0000 7fab0575d640 1 -- 172.21.15.45:0/605545134 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faae0000b90 con 0x7faaec03aa80 2024-09-18T13:59:08.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.838+0000 7faae6ffd640 1 -- 172.21.15.45:0/605545134 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40972 (unknown 2965378022 0 3986473521) 0x7faae0000b90 con 0x7faaec03aa80 2024-09-18T13:59:08.839 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:59:08.840 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-18T13:59:08.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.840+0000 7fab0575d640 1 -- 172.21.15.45:0/605545134 >> v1:172.21.15.84:6812/31478 conn(0x7faaec03aa80 legacy=0x7faaec03cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.840+0000 7fab0575d640 10 monclient: shutdown 2024-09-18T13:59:08.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.840+0000 7fab0575d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:08.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.840+0000 7fab0575d640 1 -- 172.21.15.45:0/605545134 >> v1:172.21.15.45:6790/0 conn(0x7fab0009fba0 legacy=0x7fab00146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.840+0000 7fab0575d640 1 -- 172.21.15.45:0/605545134 shutdown_connections 2024-09-18T13:59:08.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.840+0000 7fab0575d640 1 -- 172.21.15.45:0/605545134 >> 172.21.15.45:0/605545134 conn(0x7fab000630b0 msgr2=0x7fab00135720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.840+0000 7fab0575d640 1 -- 172.21.15.45:0/605545134 shutdown_connections 2024-09-18T13:59:08.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.841+0000 7fab0575d640 1 -- 172.21.15.45:0/605545134 wait complete. 2024-09-18T13:59:08.851 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-09-18T13:59:07.887850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1786248,"kb_used_data":1848,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":564444792,"statfs":{"total":579820584960,"available":577991467008,"internally_reserved":0,"allocated":1892352,"data_stored":1304944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.875626+0000","last_change":"2024-09-18T13:59:05.875874+0000","last_active":"2024-09-18T13:59:05.875626+0000","last_peered":"2024-09-18T13:59:05.875626+0000","last_clean":"2024-09-18T13:59:05.875626+0000","last_became_active":"2024-09-18T13:59:02.976936+0000","last_became_peered":"2024-09-18T13:59:02.976936+0000","last_unstale":"2024-09-18T13:59:05.875626+0000","last_undegraded":"2024-09-18T13:59:05.875626+0000","last_fullsized":"2024-09-18T13:59:05.875626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:20:37.890959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036210499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.035734+0000","last_change":"2024-09-18T13:59:05.035951+0000","last_active":"2024-09-18T13:59:05.035734+0000","last_peered":"2024-09-18T13:59:05.035734+0000","last_clean":"2024-09-18T13:59:05.035734+0000","last_became_active":"2024-09-18T13:59:03.705691+0000","last_became_peered":"2024-09-18T13:59:03.705691+0000","last_unstale":"2024-09-18T13:59:05.035734+0000","last_undegraded":"2024-09-18T13:59:05.035734+0000","last_fullsized":"2024-09-18T13:59:05.035734+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:45:16.896857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033710499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036257+0000","last_change":"2024-09-18T13:59:05.036455+0000","last_active":"2024-09-18T13:59:05.036257+0000","last_peered":"2024-09-18T13:59:05.036257+0000","last_clean":"2024-09-18T13:59:05.036257+0000","last_became_active":"2024-09-18T13:59:02.974240+0000","last_became_peered":"2024-09-18T13:59:02.974240+0000","last_unstale":"2024-09-18T13:59:05.036257+0000","last_undegraded":"2024-09-18T13:59:05.036257+0000","last_fullsized":"2024-09-18T13:59:05.036257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:13:40.312100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038595099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036428+0000","last_change":"2024-09-18T13:59:05.036936+0000","last_active":"2024-09-18T13:59:05.036428+0000","last_peered":"2024-09-18T13:59:05.036428+0000","last_clean":"2024-09-18T13:59:05.036428+0000","last_became_active":"2024-09-18T13:59:02.977186+0000","last_became_peered":"2024-09-18T13:59:02.977186+0000","last_unstale":"2024-09-18T13:59:05.036428+0000","last_undegraded":"2024-09-18T13:59:05.036428+0000","last_fullsized":"2024-09-18T13:59:05.036428+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:01:46.516709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065897000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"15'72","reported_seq":100,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-18T13:59:04.370201+0000","last_change":"2024-09-18T13:59:02.975019+0000","last_active":"2024-09-18T13:59:04.370201+0000","last_peered":"2024-09-18T13:59:04.370201+0000","last_clean":"2024-09-18T13:59:04.370201+0000","last_became_active":"2024-09-18T13:59:02.974164+0000","last_became_peered":"2024-09-18T13:59:02.974164+0000","last_unstale":"2024-09-18T13:59:04.370201+0000","last_undegraded":"2024-09-18T13:59:04.370201+0000","last_fullsized":"2024-09-18T13:59:04.370201+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T22:10:24.958774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036373+0000","last_change":"2024-09-18T13:59:05.036690+0000","last_active":"2024-09-18T13:59:05.036373+0000","last_peered":"2024-09-18T13:59:05.036373+0000","last_clean":"2024-09-18T13:59:05.036373+0000","last_became_active":"2024-09-18T13:59:02.977312+0000","last_became_peered":"2024-09-18T13:59:02.977312+0000","last_unstale":"2024-09-18T13:59:05.036373+0000","last_undegraded":"2024-09-18T13:59:05.036373+0000","last_fullsized":"2024-09-18T13:59:05.036373+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:03:06.377092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072047099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036440+0000","last_change":"2024-09-18T13:59:05.036440+0000","last_active":"2024-09-18T13:59:05.036440+0000","last_peered":"2024-09-18T13:59:05.036440+0000","last_clean":"2024-09-18T13:59:05.036440+0000","last_became_active":"2024-09-18T13:59:03.705659+0000","last_became_peered":"2024-09-18T13:59:03.705659+0000","last_unstale":"2024-09-18T13:59:05.036440+0000","last_undegraded":"2024-09-18T13:59:05.036440+0000","last_fullsized":"2024-09-18T13:59:05.036440+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:49:16.930535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037781799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.875655+0000","last_change":"2024-09-18T13:59:05.875836+0000","last_active":"2024-09-18T13:59:05.875655+0000","last_peered":"2024-09-18T13:59:05.875655+0000","last_clean":"2024-09-18T13:59:05.875655+0000","last_became_active":"2024-09-18T13:59:03.705710+0000","last_became_peered":"2024-09-18T13:59:03.705710+0000","last_unstale":"2024-09-18T13:59:05.875655+0000","last_undegraded":"2024-09-18T13:59:05.875655+0000","last_fullsized":"2024-09-18T13:59:05.875655+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:12:55.202686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029541399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.035691+0000","last_change":"2024-09-18T13:59:05.035941+0000","last_active":"2024-09-18T13:59:05.035691+0000","last_peered":"2024-09-18T13:59:05.035691+0000","last_clean":"2024-09-18T13:59:05.035691+0000","last_became_active":"2024-09-18T13:59:03.705607+0000","last_became_peered":"2024-09-18T13:59:03.705607+0000","last_unstale":"2024-09-18T13:59:05.035691+0000","last_undegraded":"2024-09-18T13:59:05.035691+0000","last_fullsized":"2024-09-18T13:59:05.035691+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:25:38.392196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000366475,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":571475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297484,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074356,"statfs":{"total":96636764160,"available":96332140544,"internally_reserved":0,"allocated":131072,"data_stored":38431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":42566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640259,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":571475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":2,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297484,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074356,"statfs":{"total":96636764160,"available":96332140544,"internally_reserved":0,"allocated":131072,"data_stored":38431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640259,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":42566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T13:59:08.852 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T13:59:08.852 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-18T13:59:08.853 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-18T13:59:08.853 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-18T13:59:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.947+0000 7fdc28860640 10 monclient: get_monmap_and_config 2024-09-18T13:59:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.947+0000 7fdc28860640 10 monclient: build_initial_monmap 2024-09-18T13:59:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.947+0000 7fdc28860640 10 monclient: monmap: 2024-09-18T13:59:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:08.948559+0000 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:08.948559+0000 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:08.950 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 1 Processor -- start 2024-09-18T13:59:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 1 -- start start 2024-09-18T13:59:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 10 monclient: init 2024-09-18T13:59:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 10 monclient: _add_conns ranks=[1,7,3,6,2,0,8,4,5] 2024-09-18T13:59:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 10 monclient(hunting): picked mon.noname-f con 0x7fdc2409f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 10 monclient(hunting): picked mon.noname-i con 0x7fdc2409fba0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 10 monclient(hunting): picked mon.noname-g con 0x7fdc2409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2405ed70 con 0x7fdc2409f7d0 2024-09-18T13:59:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2409dfc0 con 0x7fdc2409ff70 2024-09-18T13:59:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2409dce0 con 0x7fdc2409fba0 2024-09-18T13:59:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.948+0000 7fdc28860640 10 monclient(hunting): authenticate will time out at 764.286764s 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc19d9b640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7fdc2409fba0 0x7fdc24137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:58316/0 (socket says 172.21.15.45:58316) 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc19d9b640 1 -- 172.21.15.45:0/3716296169 learned_addr learned my addr 172.21.15.45:0/3716296169 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473947987 0 0) 0x7fdc2405ed70 con 0x7fdc2409f7d0 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient(hunting): my global_id is 4393 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc100034e0 con 0x7fdc2409f7d0 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157220371 0 0) 0x7fdc2409dce0 con 0x7fdc2409fba0 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient(hunting): my global_id is 4444 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc2405ed70 con 0x7fdc2409fba0 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503206242 0 0) 0x7fdc2409dfc0 con 0x7fdc2409ff70 2024-09-18T13:59:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient(hunting): my global_id is 4440 2024-09-18T13:59:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc2409dce0 con 0x7fdc2409ff70 2024-09-18T13:59:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3461478257 0 0) 0x7fdc100034e0 con 0x7fdc2409f7d0 2024-09-18T13:59:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 >> v1:172.21.15.84:6792/0 conn(0x7fdc2409fba0 legacy=0x7fdc24137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 >> v1:172.21.15.84:6790/0 conn(0x7fdc2409ff70 legacy=0x7fdc2413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient: _finish_hunting 0 2024-09-18T13:59:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 monclient: found mon.noname-f 2024-09-18T13:59:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T13:59:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc240a1c00 con 0x7fdc2409f7d0 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient: _finish_auth 0 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc28860640 5 monclient: authenticate success, global_id 4393 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc28860640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdc14001830 con 0x7fdc2409f7d0 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient: dump: 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc28860640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdc14001a10 con 0x7fdc2409f7d0 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc28860640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc28860640 4 set_mon_vals no callback set 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 1 -- 172.21.15.45:0/3716296169 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdc14001fc0 con 0x7fdc2409f7d0 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.949+0000 7fdc21d9b640 10 monclient: dump: 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.950+0000 7fdc28860640 10 monclient: shutdown 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.950+0000 7fdc28860640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.950+0000 7fdc28860640 1 -- 172.21.15.45:0/3716296169 >> v1:172.21.15.84:6789/0 conn(0x7fdc2409f7d0 legacy=0x7fdc24132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.950+0000 7fdc28860640 1 -- 172.21.15.45:0/3716296169 shutdown_connections 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.950+0000 7fdc28860640 1 -- 172.21.15.45:0/3716296169 >> 172.21.15.45:0/3716296169 conn(0x7fdc240630b0 msgr2=0x7fdc240634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.950+0000 7fdc28860640 1 -- 172.21.15.45:0/3716296169 shutdown_connections 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.950+0000 7fdc28860640 1 -- 172.21.15.45:0/3716296169 wait complete. 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.950+0000 7fdc28860640 10 monclient: build_initial_monmap 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 10 monclient: monmap: 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 1 Processor -- start 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 1 -- start start 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 10 monclient: init 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 10 monclient: _add_conns ranks=[2,0,3,1,8,6,4,7,5] 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 10 monclient(hunting): picked mon.noname-b con 0x7fdc2409f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 10 monclient(hunting): picked mon.noname-a con 0x7fdc2409fba0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 10 monclient(hunting): picked mon.noname-g con 0x7fdc2409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2409d310 con 0x7fdc2409fba0 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc240a2560 con 0x7fdc2409f7d0 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2411d850 con 0x7fdc2409ff70 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc28860640 10 monclient(hunting): authenticate will time out at 764.289857s 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc2259c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fdc2409f7d0 0x7fdc24131690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51540/0 (socket says 172.21.15.45:51540) 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.951+0000 7fdc2259c640 1 -- 172.21.15.45:0/1799063263 learned_addr learned my addr 172.21.15.45:0/1799063263 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805970991 0 0) 0x7fdc2409d310 con 0x7fdc2409fba0 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): my global_id is 4374 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc040034e0 con 0x7fdc2409fba0 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384389076 0 0) 0x7fdc240a2560 con 0x7fdc2409f7d0 2024-09-18T13:59:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): my global_id is 4358 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc2409d310 con 0x7fdc2409f7d0 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996766513 0 0) 0x7fdc2411d850 con 0x7fdc2409ff70 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): my global_id is 4449 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc240a2560 con 0x7fdc2409ff70 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2464622760 0 0) 0x7fdc040034e0 con 0x7fdc2409fba0 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdc2411d850 con 0x7fdc2409fba0 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdc0c001ce0 con 0x7fdc2409fba0 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): dump: 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.970 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1276159820 0 0) 0x7fdc2409d310 con 0x7fdc2409f7d0 2024-09-18T13:59:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdc040034e0 con 0x7fdc2409f7d0 2024-09-18T13:59:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdc14001ec0 con 0x7fdc2409f7d0 2024-09-18T13:59:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): dump: 2024-09-18T13:59:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1149255003 0 0) 0x7fdc240a2560 con 0x7fdc2409ff70 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdc2409d310 con 0x7fdc2409ff70 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdc1c001c00 con 0x7fdc2409ff70 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient(hunting): dump: 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3678879085 0 0) 0x7fdc2411d850 con 0x7fdc2409fba0 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 >> v1:172.21.15.84:6790/0 conn(0x7fdc2409ff70 legacy=0x7fdc2414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 >> v1:172.21.15.45:6790/0 conn(0x7fdc2409f7d0 legacy=0x7fdc24131690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 monclient: found mon.a 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2411e7b0 con 0x7fdc2409fba0 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient: _finish_auth 0 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc28860640 5 monclient: authenticate success, global_id 4374 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc28860640 10 monclient: _renew_subs 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc28860640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc28860640 1 -- 172.21.15.45:0/1799063263 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2409b9b0 con 0x7fdc2409fba0 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc28860640 10 monclient: _renew_subs 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc28860640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc28860640 1 -- 172.21.15.45:0/1799063263 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc24136ee0 con 0x7fdc2409fba0 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdc0c0012b0 con 0x7fdc2409fba0 2024-09-18T13:59:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdc0c004dd0 con 0x7fdc2409fba0 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc2159a640 4 set_mon_vals no callback set 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.952+0000 7fdc1affd640 10 monclient: dump: 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.953+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fdc0c00fda0 con 0x7fdc2409fba0 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.953+0000 7fdc28860640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.953+0000 7fdc28860640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.953+0000 7fdc28860640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.953+0000 7fdc28860640 1 -- 172.21.15.45:0/1799063263 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc08005010 con 0x7fdc2409fba0 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.953+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7fdc0c046960 con 0x7fdc2409fba0 2024-09-18T13:59:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.954+0000 7fdc2259c640 10 monclient: get_auth_request con 0x7fdc0403aa00 auth_method 0 2024-09-18T13:59:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.958+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fdc0c0152a0 con 0x7fdc2409fba0 2024-09-18T13:59:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.958+0000 7fdc1affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:08.958+0000 7fdc1affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:09.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.159+0000 7fdc28860640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2024-09-18T13:59:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.159+0000 7fdc28860640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2024-09-18T13:59:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.159+0000 7fdc28860640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.159+0000 7fdc28860640 1 -- 172.21.15.45:0/1799063263 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdc08005840 con 0x7fdc2409fba0 2024-09-18T13:59:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.159+0000 7fdc1affd640 1 -- 172.21.15.45:0/1799063263 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7fdc0c014ca0 con 0x7fdc2409fba0 2024-09-18T13:59:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.159+0000 7fdc1affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2024-09-18T13:59:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.159+0000 7fdc1affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:09.160 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:59:09.160 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-18T13:59:09.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.161+0000 7fdc28860640 1 -- 172.21.15.45:0/1799063263 >> v1:172.21.15.84:6812/31478 conn(0x7fdc0403aa00 legacy=0x7fdc0403ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.161+0000 7fdc28860640 10 monclient: shutdown 2024-09-18T13:59:09.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.161+0000 7fdc28860640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:09.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.161+0000 7fdc28860640 1 -- 172.21.15.45:0/1799063263 >> v1:172.21.15.45:6789/0 conn(0x7fdc2409fba0 legacy=0x7fdc24146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.161+0000 7fdc28860640 1 -- 172.21.15.45:0/1799063263 shutdown_connections 2024-09-18T13:59:09.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.161+0000 7fdc28860640 1 -- 172.21.15.45:0/1799063263 >> 172.21.15.45:0/1799063263 conn(0x7fdc240630b0 msgr2=0x7fdc240a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:09.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.161+0000 7fdc28860640 1 -- 172.21.15.45:0/1799063263 shutdown_connections 2024-09-18T13:59:09.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.161+0000 7fdc28860640 1 -- 172.21.15.45:0/1799063263 wait complete. 2024-09-18T13:59:09.172 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-18T13:59:09.172 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-09-18T13:59:09.213 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-09-18T13:59:09.213 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T13:59:09.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.309+0000 7f6fb146f640 10 monclient: get_monmap_and_config 2024-09-18T13:59:09.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.309+0000 7f6fb146f640 10 monclient: build_initial_monmap 2024-09-18T13:59:09.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.309+0000 7f6fb146f640 10 monclient: monmap: 2024-09-18T13:59:09.324 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:09.324 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.324 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:09.310323+0000 2024-09-18T13:59:09.324 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:09.310323+0000 2024-09-18T13:59:09.325 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:09.325 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.325 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:09.325 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:09.325 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:09.325 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:09.326 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:09.326 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:09.326 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:09.326 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:09.326 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:09.326 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.309+0000 7f6fb146f640 1 Processor -- start 2024-09-18T13:59:09.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fb146f640 1 -- start start 2024-09-18T13:59:09.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fb146f640 10 monclient: init 2024-09-18T13:59:09.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fb146f640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:09.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fb146f640 10 monclient: _add_conns ranks=[5,8,2,3,0,7,6,4,1] 2024-09-18T13:59:09.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fb146f640 10 monclient(hunting): picked mon.noname-h con 0x7f6fac09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:09.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fb146f640 10 monclient(hunting): picked mon.noname-e con 0x7f6fac09fba0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:09.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fb146f640 10 monclient(hunting): picked mon.noname-b con 0x7f6fac09ff70 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:09.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fb146f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac05ed70 con 0x7f6fac09ff70 2024-09-18T13:59:09.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fb146f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac09dfc0 con 0x7f6fac09f7d0 2024-09-18T13:59:09.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fb146f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac09dce0 con 0x7f6fac09fba0 2024-09-18T13:59:09.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fb146f640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:09.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fb146f640 10 monclient(hunting): authenticate will time out at 764.648533s 2024-09-18T13:59:09.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fab7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f6fac09ff70 0x7f6fac13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51554/0 (socket says 172.21.15.45:51554) 2024-09-18T13:59:09.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6faa7fc640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f6fac09fba0 0x7f6fac137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51870/0 (socket says 172.21.15.45:51870) 2024-09-18T13:59:09.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fab7fe640 1 -- 172.21.15.45:0/362001584 learned_addr learned my addr 172.21.15.45:0/362001584 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:09.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776309309 0 0) 0x7f6fac05ed70 con 0x7f6fac09ff70 2024-09-18T13:59:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fa9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fa9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fa9ffb640 10 monclient(hunting): my global_id is 4367 2024-09-18T13:59:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f940034e0 con 0x7f6fac09ff70 2024-09-18T13:59:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897166300 0 0) 0x7f6fac09dce0 con 0x7f6fac09fba0 2024-09-18T13:59:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fa9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.310+0000 7f6fa9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient(hunting): my global_id is 4337 2024-09-18T13:59:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fac05ed70 con 0x7f6fac09fba0 2024-09-18T13:59:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707343437 0 0) 0x7f6fac09dfc0 con 0x7f6fac09f7d0 2024-09-18T13:59:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient(hunting): my global_id is 4343 2024-09-18T13:59:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fac09dce0 con 0x7f6fac09f7d0 2024-09-18T13:59:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 911501437 0 0) 0x7f6f940034e0 con 0x7f6fac09ff70 2024-09-18T13:59:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 >> v1:172.21.15.45:6793/0 conn(0x7f6fac09fba0 legacy=0x7f6fac137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 >> v1:172.21.15.84:6791/0 conn(0x7f6fac09f7d0 legacy=0x7f6fac132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 1 monclient: found mon.noname-b 2024-09-18T13:59:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T13:59:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac0a1c00 con 0x7f6fac09ff70 2024-09-18T13:59:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fa4001bb0 con 0x7f6fac09ff70 2024-09-18T13:59:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fb146f640 5 monclient: authenticate success, global_id 4367 2024-09-18T13:59:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fb146f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient: dump: 2024-09-18T13:59:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.341 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fb146f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6fa4001d10 con 0x7f6fac09ff70 2024-09-18T13:59:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 1 -- 172.21.15.45:0/362001584 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fa4004f20 con 0x7f6fac09ff70 2024-09-18T13:59:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fb146f640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fb146f640 4 set_mon_vals no callback set 2024-09-18T13:59:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fa9ffb640 10 monclient: dump: 2024-09-18T13:59:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.345 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fb146f640 10 monclient: shutdown 2024-09-18T13:59:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fb146f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fb146f640 1 -- 172.21.15.45:0/362001584 >> v1:172.21.15.45:6790/0 conn(0x7f6fac09ff70 legacy=0x7f6fac13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fb146f640 1 -- 172.21.15.45:0/362001584 shutdown_connections 2024-09-18T13:59:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fb146f640 1 -- 172.21.15.45:0/362001584 >> 172.21.15.45:0/362001584 conn(0x7f6fac0630b0 msgr2=0x7f6fac0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fb146f640 1 -- 172.21.15.45:0/362001584 shutdown_connections 2024-09-18T13:59:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.311+0000 7f6fb146f640 1 -- 172.21.15.45:0/362001584 wait complete. 2024-09-18T13:59:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.312+0000 7f6fb146f640 10 monclient: build_initial_monmap 2024-09-18T13:59:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.312+0000 7f6fb146f640 10 monclient: monmap: 2024-09-18T13:59:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:09.350 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.312+0000 7f6fb146f640 1 Processor -- start 2024-09-18T13:59:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.312+0000 7f6fb146f640 1 -- start start 2024-09-18T13:59:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.312+0000 7f6fb146f640 10 monclient: init 2024-09-18T13:59:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6fb146f640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6fb146f640 10 monclient: _add_conns ranks=[5,8,6,4,7,2,3,1,0] 2024-09-18T13:59:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6fb146f640 10 monclient(hunting): picked mon.noname-h con 0x7f6fac09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6fb146f640 10 monclient(hunting): picked mon.noname-e con 0x7f6fac09fba0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6fb146f640 10 monclient(hunting): picked mon.noname-d con 0x7f6fac09ff70 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6fb146f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac09d310 con 0x7f6fac09f7d0 2024-09-18T13:59:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6fb146f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac0a2560 con 0x7f6fac09ff70 2024-09-18T13:59:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6fb146f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac142730 con 0x7f6fac09fba0 2024-09-18T13:59:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6fb146f640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6fb146f640 10 monclient(hunting): authenticate will time out at 764.651349s 2024-09-18T13:59:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6faa7fc640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f6fac09fba0 0x7f6fac131a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51880/0 (socket says 172.21.15.45:51880) 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6faa7fc640 1 -- 172.21.15.45:0/2055956112 learned_addr learned my addr 172.21.15.45:0/2055956112 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207556347 0 0) 0x7f6fac142730 con 0x7f6fac09fba0 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 10 monclient(hunting): my global_id is 4346 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f980034e0 con 0x7f6fac09fba0 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698123981 0 0) 0x7f6fac0a2560 con 0x7f6fac09ff70 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 10 monclient(hunting): my global_id is 4371 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fac142730 con 0x7f6fac09ff70 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018886375 0 0) 0x7f6fac09d310 con 0x7f6fac09f7d0 2024-09-18T13:59:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 10 monclient(hunting): my global_id is 4352 2024-09-18T13:59:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fac0a2560 con 0x7f6fac09f7d0 2024-09-18T13:59:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1400190244 0 0) 0x7f6f980034e0 con 0x7f6fac09fba0 2024-09-18T13:59:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fac09d310 con 0x7f6fac09fba0 2024-09-18T13:59:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3392540813 0 0) 0x7f6fac142730 con 0x7f6fac09ff70 2024-09-18T13:59:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6f980034e0 con 0x7f6fac09ff70 2024-09-18T13:59:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.313+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fa0001530 con 0x7f6fac09fba0 2024-09-18T13:59:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.355 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fa4001af0 con 0x7f6fac09ff70 2024-09-18T13:59:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1756109513 0 0) 0x7f6fac0a2560 con 0x7f6fac09f7d0 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fac142730 con 0x7f6fac09f7d0 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6f9c001a20 con 0x7f6fac09f7d0 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 57614071 0 0) 0x7f6fac09d310 con 0x7f6fac09fba0 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 >> v1:172.21.15.45:6792/0 conn(0x7f6fac09ff70 legacy=0x7f6fac149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 >> v1:172.21.15.84:6791/0 conn(0x7f6fac09f7d0 legacy=0x7f6fac141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 1 monclient: found mon.e 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac141f50 con 0x7f6fac09fba0 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6fb146f640 5 monclient: authenticate success, global_id 4346 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6fb146f640 10 monclient: _renew_subs 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6fb146f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6fb146f640 1 -- 172.21.15.45:0/2055956112 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fac120c90 con 0x7f6fac09fba0 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6fb146f640 10 monclient: _renew_subs 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6fb146f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:09.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6fb146f640 1 -- 172.21.15.45:0/2055956112 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac1200c0 con 0x7f6fac09fba0 2024-09-18T13:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6fa0004a50 con 0x7f6fac09fba0 2024-09-18T13:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fa0004f00 con 0x7f6fac09fba0 2024-09-18T13:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6fa8ff9640 4 set_mon_vals no callback set 2024-09-18T13:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.314+0000 7f6f92ffd640 10 monclient: dump: 2024-09-18T13:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.362 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.315+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f6fa000fed0 con 0x7f6fac09fba0 2024-09-18T13:59:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.315+0000 7f6fb146f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.315+0000 7f6fb146f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.315+0000 7f6fb146f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.315+0000 7f6fb146f640 1 -- 172.21.15.45:0/2055956112 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8c005010 con 0x7f6fac09fba0 2024-09-18T13:59:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.315+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f6fa00469c0 con 0x7f6fac09fba0 2024-09-18T13:59:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.315+0000 7f6faaffd640 10 monclient: get_auth_request con 0x7f6f9803aa40 auth_method 0 2024-09-18T13:59:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.319+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f6fa0015300 con 0x7f6fac09fba0 2024-09-18T13:59:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.320+0000 7f6f92ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.320+0000 7f6f92ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:09.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.483+0000 7f6fb146f640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:59:09.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.483+0000 7f6fb146f640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:59:09.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.483+0000 7f6fb146f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:09.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.483+0000 7f6fb146f640 1 -- 172.21.15.45:0/2055956112 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f8c0052c0 con 0x7f6fac09fba0 2024-09-18T13:59:09.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.483+0000 7f6f92ffd640 1 -- 172.21.15.45:0/2055956112 <== mon.8 v1:172.21.15.45:6793/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+10060 (unknown 1054498076 0 3527204855) 0x7f6fa0014d00 con 0x7f6fac09fba0 2024-09-18T13:59:09.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.483+0000 7f6f92ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T13:59:09.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.483+0000 7f6f92ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:09.485 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:59:09.485 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":16,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","created":"2024-09-18T13:58:52.168851+0000","modified":"2024-09-18T13:59:05.031012+0000","last_up_change":"2024-09-18T13:59:00.910314+0000","last_in_change":"2024-09-18T13:58:53.737759+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T13:59:01.925362+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-18T13:59:01.954982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"8c3a8909-07ce-4141-98dd-131951f18ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6804","nonce":32276}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6805","nonce":32276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6807","nonce":32276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6806","nonce":32276}]},"public_addr":"172.21.15.45:6804/32276","cluster_addr":"172.21.15.45:6805/32276","heartbeat_back_addr":"172.21.15.45:6807/32276","heartbeat_front_addr":"172.21.15.45:6806/32276","state":["exists","up"]},{"osd":1,"uuid":"a12b3bc3-1e03-47dc-98c1-c8bd199551a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6800","nonce":32277}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6801","nonce":32277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6803","nonce":32277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6802","nonce":32277}]},"public_addr":"172.21.15.45:6800/32277","cluster_addr":"172.21.15.45:6801/32277","heartbeat_back_addr":"172.21.15.45:6803/32277","heartbeat_front_addr":"172.21.15.45:6802/32277","state":["exists","up"]},{"osd":2,"uuid":"216b8b62-e41c-4bc5-9395-f2d56d402b1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6808","nonce":32278}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6809","nonce":32278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6811","nonce":32278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6810","nonce":32278}]},"public_addr":"172.21.15.45:6808/32278","cluster_addr":"172.21.15.45:6809/32278","heartbeat_back_addr":"172.21.15.45:6811/32278","heartbeat_front_addr":"172.21.15.45:6810/32278","state":["exists","up"]},{"osd":3,"uuid":"c750b0e6-9143-489d-9be0-eb085178ff2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6802","nonce":32024}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6804","nonce":32024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6807","nonce":32024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6806","nonce":32024}]},"public_addr":"172.21.15.84:6802/32024","cluster_addr":"172.21.15.84:6804/32024","heartbeat_back_addr":"172.21.15.84:6807/32024","heartbeat_front_addr":"172.21.15.84:6806/32024","state":["exists","up"]},{"osd":4,"uuid":"9680908d-dc81-44a0-8603-93711579d42e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6800","nonce":32025}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6801","nonce":32025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6805","nonce":32025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6803","nonce":32025}]},"public_addr":"172.21.15.84:6800/32025","cluster_addr":"172.21.15.84:6801/32025","heartbeat_back_addr":"172.21.15.84:6805/32025","heartbeat_front_addr":"172.21.15.84:6803/32025","state":["exists","up"]},{"osd":5,"uuid":"b3b63eef-c8fc-495f-960f-7436315119ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6808","nonce":32048}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6809","nonce":32048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6811","nonce":32048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6810","nonce":32048}]},"public_addr":"172.21.15.84:6808/32048","cluster_addr":"172.21.15.84:6809/32048","heartbeat_back_addr":"172.21.15.84:6811/32048","heartbeat_front_addr":"172.21.15.84:6810/32048","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.823226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.883512+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.904155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.801736+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.835588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.910413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T13:59:09.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.485+0000 7f6fb146f640 1 -- 172.21.15.45:0/2055956112 >> v1:172.21.15.84:6812/31478 conn(0x7f6f9803aa40 legacy=0x7f6f9803cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.485+0000 7f6fb146f640 10 monclient: shutdown 2024-09-18T13:59:09.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.485+0000 7f6fb146f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:09.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.485+0000 7f6fb146f640 1 -- 172.21.15.45:0/2055956112 >> v1:172.21.15.45:6793/0 conn(0x7f6fac09fba0 legacy=0x7f6fac131a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.485+0000 7f6fb146f640 1 -- 172.21.15.45:0/2055956112 shutdown_connections 2024-09-18T13:59:09.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.485+0000 7f6fb146f640 1 -- 172.21.15.45:0/2055956112 >> 172.21.15.45:0/2055956112 conn(0x7f6fac0630b0 msgr2=0x7f6fac0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:09.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.485+0000 7f6fb146f640 1 -- 172.21.15.45:0/2055956112 shutdown_connections 2024-09-18T13:59:09.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.485+0000 7f6fb146f640 1 -- 172.21.15.45:0/2055956112 wait complete. 2024-09-18T13:59:09.496 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-18T13:59:01.925362+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-18T13:59:01.954982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-18T13:59:09.497 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-18T13:59:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.594+0000 7f1a1dad4640 10 monclient: get_monmap_and_config 2024-09-18T13:59:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.594+0000 7f1a1dad4640 10 monclient: build_initial_monmap 2024-09-18T13:59:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.594+0000 7f1a1dad4640 10 monclient: monmap: 2024-09-18T13:59:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:09.595905+0000 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:09.595905+0000 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.595+0000 7f1a1dad4640 1 Processor -- start 2024-09-18T13:59:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.595+0000 7f1a1dad4640 1 -- start start 2024-09-18T13:59:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.595+0000 7f1a1dad4640 10 monclient: init 2024-09-18T13:59:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a1dad4640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a1dad4640 10 monclient: _add_conns ranks=[2,4,3,7,0,5,6,8,1] 2024-09-18T13:59:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a1dad4640 10 monclient(hunting): picked mon.noname-b con 0x7f1a1809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a1dad4640 10 monclient(hunting): picked mon.noname-c con 0x7f1a1809fba0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a1dad4640 10 monclient(hunting): picked mon.noname-g con 0x7f1a1809ff70 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a1dad4640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1805ed70 con 0x7f1a1809f7d0 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a1dad4640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1809dfc0 con 0x7f1a1809ff70 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a1dad4640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1809dce0 con 0x7f1a1809fba0 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a1dad4640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a1dad4640 10 monclient(hunting): authenticate will time out at 764.934236s 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a16ffd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f1a1809fba0 0x7f1a18137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:45890/0 (socket says 172.21.15.45:45890) 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a177fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f1a1809f7d0 0x7f1a18132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51558/0 (socket says 172.21.15.45:51558) 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a16ffd640 1 -- 172.21.15.45:0/2638456541 learned_addr learned my addr 172.21.15.45:0/2638456541 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 1 -- 172.21.15.45:0/2638456541 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750204906 0 0) 0x7f1a1809dce0 con 0x7f1a1809fba0 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 10 monclient(hunting): my global_id is 4378 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 1 -- 172.21.15.45:0/2638456541 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a000034e0 con 0x7f1a1809fba0 2024-09-18T13:59:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 1 -- 172.21.15.45:0/2638456541 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003724008 0 0) 0x7f1a1805ed70 con 0x7f1a1809f7d0 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 10 monclient(hunting): my global_id is 4376 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 1 -- 172.21.15.45:0/2638456541 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a1809dce0 con 0x7f1a1809f7d0 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 1 -- 172.21.15.45:0/2638456541 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 475117678 0 0) 0x7f1a1809dce0 con 0x7f1a1809f7d0 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 1 -- 172.21.15.45:0/2638456541 >> v1:172.21.15.45:6791/0 conn(0x7f1a1809fba0 legacy=0x7f1a18137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 1 -- 172.21.15.45:0/2638456541 >> v1:172.21.15.84:6790/0 conn(0x7f1a1809ff70 legacy=0x7f1a1813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 1 monclient: found mon.noname-b 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 1 -- 172.21.15.45:0/2638456541 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a180a1c00 con 0x7f1a1809f7d0 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a1dad4640 5 monclient: authenticate success, global_id 4376 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 1 -- 172.21.15.45:0/2638456541 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1a08001810 con 0x7f1a1809f7d0 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a1dad4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.596+0000 7f1a167fc640 10 monclient: dump: 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a1dad4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a167fc640 1 -- 172.21.15.45:0/2638456541 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1a08001970 con 0x7f1a1809f7d0 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a167fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a1dad4640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a1dad4640 4 set_mon_vals no callback set 2024-09-18T13:59:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a167fc640 1 -- 172.21.15.45:0/2638456541 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1a08004ae0 con 0x7f1a1809f7d0 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a167fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a167fc640 10 monclient: dump: 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a1dad4640 10 monclient: shutdown 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a1dad4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a1dad4640 1 -- 172.21.15.45:0/2638456541 >> v1:172.21.15.45:6790/0 conn(0x7f1a1809f7d0 legacy=0x7f1a18132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a1dad4640 1 -- 172.21.15.45:0/2638456541 shutdown_connections 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a1dad4640 1 -- 172.21.15.45:0/2638456541 >> 172.21.15.45:0/2638456541 conn(0x7f1a180630b0 msgr2=0x7f1a180634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a1dad4640 1 -- 172.21.15.45:0/2638456541 shutdown_connections 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.597+0000 7f1a1dad4640 1 -- 172.21.15.45:0/2638456541 wait complete. 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.598+0000 7f1a1dad4640 10 monclient: build_initial_monmap 2024-09-18T13:59:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.598+0000 7f1a1dad4640 10 monclient: monmap: 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.598+0000 7f1a1dad4640 1 Processor -- start 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.598+0000 7f1a1dad4640 1 -- start start 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.598+0000 7f1a1dad4640 10 monclient: init 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 10 monclient: _add_conns ranks=[6,3,4,8,5,7,1,0,2] 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 10 monclient(hunting): picked mon.noname-d con 0x7f1a1809f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 10 monclient(hunting): picked mon.noname-g con 0x7f1a1809fba0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 10 monclient(hunting): picked mon.noname-c con 0x7f1a1809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1809d310 con 0x7f1a1809fba0 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a180a2560 con 0x7f1a1809ff70 2024-09-18T13:59:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1809b890 con 0x7f1a1809f7d0 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 10 monclient(hunting): authenticate will time out at 764.937158s 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a177fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f1a1809f7d0 0x7f1a180984e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34778/0 (socket says 172.21.15.45:34778) 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a17fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f1a1809ff70 0x7f1a18149c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:45900/0 (socket says 172.21.15.45:45900) 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a177fe640 1 -- 172.21.15.45:0/389603758 learned_addr learned my addr 172.21.15.45:0/389603758 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834597847 0 0) 0x7f1a180a2560 con 0x7f1a1809ff70 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): my global_id is 4387 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a040034e0 con 0x7f1a1809ff70 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097391524 0 0) 0x7f1a1809b890 con 0x7f1a1809f7d0 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): my global_id is 4380 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a180a2560 con 0x7f1a1809f7d0 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311710515 0 0) 0x7f1a1809d310 con 0x7f1a1809fba0 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): my global_id is 4467 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a1809b890 con 0x7f1a1809fba0 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2860704707 0 0) 0x7f1a040034e0 con 0x7f1a1809ff70 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1a1809d310 con 0x7f1a1809ff70 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1a10001ca0 con 0x7f1a1809ff70 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): dump: 2024-09-18T13:59:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2188597291 0 0) 0x7f1a180a2560 con 0x7f1a1809f7d0 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1a040034e0 con 0x7f1a1809f7d0 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1a08001410 con 0x7f1a1809f7d0 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): dump: 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.617 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2614814039 0 0) 0x7f1a1809b890 con 0x7f1a1809fba0 2024-09-18T13:59:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1a180a2560 con 0x7f1a1809fba0 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1a0c0015b0 con 0x7f1a1809fba0 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient(hunting): dump: 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1659736915 0 0) 0x7f1a1809d310 con 0x7f1a1809ff70 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 >> v1:172.21.15.45:6792/0 conn(0x7f1a1809f7d0 legacy=0x7f1a180984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 >> v1:172.21.15.84:6790/0 conn(0x7f1a1809fba0 legacy=0x7f1a18098bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient: _finish_hunting 0 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 monclient: found mon.c 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1812d2f0 con 0x7f1a1809ff70 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 10 monclient: _finish_auth 0 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f19ff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 5 monclient: authenticate success, global_id 4387 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 10 monclient: _renew_subs 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.599+0000 7f1a1dad4640 1 -- 172.21.15.45:0/389603758 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1809c5e0 con 0x7f1a1809ff70 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f1a1dad4640 10 monclient: _renew_subs 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f1a1dad4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f1a1dad4640 1 -- 172.21.15.45:0/389603758 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a180994a0 con 0x7f1a1809ff70 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1a10002260 con 0x7f1a1809ff70 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f19ff7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1a10002850 con 0x7f1a1809ff70 2024-09-18T13:59:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f19ff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f1a15ffb640 4 set_mon_vals no callback set 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f19ff7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f19ff7fe640 10 monclient: dump: 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.622 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1a100029b0 con 0x7f1a1809ff70 2024-09-18T13:59:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f1a1dad4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f1a1dad4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f1a1dad4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.600+0000 7f1a1dad4640 1 -- 172.21.15.45:0/389603758 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f8005010 con 0x7f1a1809ff70 2024-09-18T13:59:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.601+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f1a10046ad0 con 0x7f1a1809ff70 2024-09-18T13:59:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.601+0000 7f1a177fe640 10 monclient: get_auth_request con 0x7f1a0403aa90 auth_method 0 2024-09-18T13:59:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.605+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.4 v1:172.21.15.45:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f1a10015410 con 0x7f1a1809ff70 2024-09-18T13:59:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.605+0000 7f19ff7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.605+0000 7f19ff7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:09.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.768+0000 7f1a1dad4640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-18T13:59:09.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.768+0000 7f1a1dad4640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-18T13:59:09.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.768+0000 7f1a1dad4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:09.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.768+0000 7f1a1dad4640 1 -- 172.21.15.45:0/389603758 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f19f8005840 con 0x7f1a1809ff70 2024-09-18T13:59:09.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.768+0000 7f19ff7fe640 1 -- 172.21.15.45:0/389603758 <== mon.4 v1:172.21.15.45:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (unknown 116879340 0 265718887) 0x7f1a10014e10 con 0x7f1a1809ff70 2024-09-18T13:59:09.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.769+0000 7f19ff7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-18T13:59:09.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.769+0000 7f19ff7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:09.770 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2024-09-18T13:59:09.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.770+0000 7f1a1dad4640 1 -- 172.21.15.45:0/389603758 >> v1:172.21.15.84:6812/31478 conn(0x7f1a0403aa90 legacy=0x7f1a0403cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.770+0000 7f1a1dad4640 10 monclient: shutdown 2024-09-18T13:59:09.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.770+0000 7f1a1dad4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:09.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.770+0000 7f1a1dad4640 1 -- 172.21.15.45:0/389603758 >> v1:172.21.15.45:6791/0 conn(0x7f1a1809ff70 legacy=0x7f1a18149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.770+0000 7f1a1dad4640 1 -- 172.21.15.45:0/389603758 shutdown_connections 2024-09-18T13:59:09.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.770+0000 7f1a1dad4640 1 -- 172.21.15.45:0/389603758 >> 172.21.15.45:0/389603758 conn(0x7f1a180630b0 msgr2=0x7f1a18130c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:09.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.770+0000 7f1a1dad4640 1 -- 172.21.15.45:0/389603758 shutdown_connections 2024-09-18T13:59:09.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.770+0000 7f1a1dad4640 1 -- 172.21.15.45:0/389603758 wait complete. 2024-09-18T13:59:09.782 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-18T13:59:09.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.880+0000 7f7ac839d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:09.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.880+0000 7f7ac839d640 10 monclient: build_initial_monmap 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.880+0000 7f7ac839d640 10 monclient: monmap: 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:09.881131+0000 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:09.881131+0000 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:09.881 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:09.882 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:09.882 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:09.882 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:09.882 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.880+0000 7f7ac839d640 1 Processor -- start 2024-09-18T13:59:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.880+0000 7f7ac839d640 1 -- start start 2024-09-18T13:59:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.880+0000 7f7ac839d640 10 monclient: init 2024-09-18T13:59:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac839d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac839d640 10 monclient: _add_conns ranks=[4,5,1,7,8,3,2,6,0] 2024-09-18T13:59:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac839d640 10 monclient(hunting): picked mon.noname-c con 0x7f7ac006e810 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac839d640 10 monclient(hunting): picked mon.noname-h con 0x7f7ac00a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac839d640 10 monclient(hunting): picked mon.noname-f con 0x7f7ac00a1080 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac839d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac005ed70 con 0x7f7ac00a1080 2024-09-18T13:59:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac839d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac0097620 con 0x7f7ac006e810 2024-09-18T13:59:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac839d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac0097340 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac839d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac839d640 10 monclient(hunting): authenticate will time out at 765.219609s 2024-09-18T13:59:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac6139640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f7ac006e810 0x7f7ac00a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:45908/0 (socket says 172.21.15.45:45908) 2024-09-18T13:59:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac6139640 1 -- 172.21.15.45:0/964466117 learned_addr learned my addr 172.21.15.45:0/964466117 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246089652 0 0) 0x7f7ac0097620 con 0x7f7ac006e810 2024-09-18T13:59:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac5137640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac5137640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.881+0000 7f7ac5137640 10 monclient(hunting): my global_id is 4396 2024-09-18T13:59:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aac0034e0 con 0x7f7ac006e810 2024-09-18T13:59:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919420252 0 0) 0x7f7ac0097340 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient(hunting): my global_id is 4361 2024-09-18T13:59:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ac0097620 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724874219 0 0) 0x7f7ac005ed70 con 0x7f7ac00a1080 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient(hunting): my global_id is 4402 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ac0097340 con 0x7f7ac00a1080 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2495734541 0 0) 0x7f7aac0034e0 con 0x7f7ac006e810 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 >> v1:172.21.15.84:6791/0 conn(0x7f7ac00a0cb0 legacy=0x7f7ac0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 >> v1:172.21.15.84:6789/0 conn(0x7f7ac00a1080 legacy=0x7f7ac013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient: _finish_hunting 0 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 monclient: found mon.noname-c 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac00a1660 con 0x7f7ac006e810 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient: _finish_auth 0 2024-09-18T13:59:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7abc001830 con 0x7f7ac006e810 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient: dump: 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac839d640 5 monclient: authenticate success, global_id 4396 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac839d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7abc001a10 con 0x7f7ac006e810 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 1 -- 172.21.15.45:0/964466117 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7abc004ba0 con 0x7f7ac006e810 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac5137640 10 monclient: dump: 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.895 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac839d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac839d640 4 set_mon_vals no callback set 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac839d640 10 monclient: shutdown 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac839d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac839d640 1 -- 172.21.15.45:0/964466117 >> v1:172.21.15.45:6791/0 conn(0x7f7ac006e810 legacy=0x7f7ac00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac839d640 1 -- 172.21.15.45:0/964466117 shutdown_connections 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac839d640 1 -- 172.21.15.45:0/964466117 >> 172.21.15.45:0/964466117 conn(0x7f7ac00a5bb0 msgr2=0x7f7ac00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac839d640 1 -- 172.21.15.45:0/964466117 shutdown_connections 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.882+0000 7f7ac839d640 1 -- 172.21.15.45:0/964466117 wait complete. 2024-09-18T13:59:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.883+0000 7f7ac839d640 10 monclient: build_initial_monmap 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.883+0000 7f7ac839d640 10 monclient: monmap: 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:09.897 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.883+0000 7f7ac839d640 1 Processor -- start 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.883+0000 7f7ac839d640 1 -- start start 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.883+0000 7f7ac839d640 10 monclient: init 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ac839d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ac839d640 10 monclient: _add_conns ranks=[8,6,0,2,5,4,1,7,3] 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ac839d640 10 monclient(hunting): picked mon.noname-e con 0x7f7ac006e810 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ac839d640 10 monclient(hunting): picked mon.noname-d con 0x7f7ac00a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ac839d640 10 monclient(hunting): picked mon.noname-a con 0x7f7ac00a1080 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ac839d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac01361a0 con 0x7f7ac00a1080 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ac839d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac00a2270 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ac839d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac009fa10 con 0x7f7ac006e810 2024-09-18T13:59:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ac839d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ac839d640 10 monclient(hunting): authenticate will time out at 765.222323s 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ac6139640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f7ac006e810 0x7f7ac0135400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51892/0 (socket says 172.21.15.45:51892) 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ac6139640 1 -- 172.21.15.45:0/1177563390 learned_addr learned my addr 172.21.15.45:0/1177563390 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386016233 0 0) 0x7f7ac00a2270 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): my global_id is 4389 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aa40034e0 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773705487 0 0) 0x7f7ac009fa10 con 0x7f7ac006e810 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): my global_id is 4355 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ac00a2270 con 0x7f7ac006e810 2024-09-18T13:59:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378578583 0 0) 0x7f7ac01361a0 con 0x7f7ac00a1080 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): my global_id is 4383 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ac009fa10 con 0x7f7ac00a1080 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3034929157 0 0) 0x7f7aa40034e0 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ac01361a0 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7ab0002b10 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): dump: 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.900 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.884+0000 7f7ab67fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2590800142 0 0) 0x7f7ac00a2270 con 0x7f7ac006e810 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7aa40034e0 con 0x7f7ac006e810 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2784584109 0 0) 0x7f7ac009fa10 con 0x7f7ac00a1080 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ac00a2270 con 0x7f7ac00a1080 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7abc0017f0 con 0x7f7ac006e810 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient(hunting): dump: 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7ab8001d00 con 0x7f7ac00a1080 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient(hunting): dump: 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 267877946 0 0) 0x7f7ac01361a0 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 >> v1:172.21.15.45:6793/0 conn(0x7f7ac006e810 legacy=0x7f7ac0135400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 >> v1:172.21.15.45:6789/0 conn(0x7f7ac00a1080 legacy=0x7f7ac0149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 monclient: found mon.d 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac01209f0 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ac839d640 5 monclient: authenticate success, global_id 4389 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ac839d640 10 monclient: _renew_subs 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ac839d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ac839d640 1 -- 172.21.15.45:0/1177563390 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac0142400 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ac839d640 10 monclient: _renew_subs 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ac839d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ac839d640 1 -- 172.21.15.45:0/1177563390 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac0136a90 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7ab0002d70 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7ab00019e0 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ac4936640 4 set_mon_vals no callback set 2024-09-18T13:59:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.885+0000 7f7ab67fc640 10 monclient: dump: 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:09.908 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.886+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7ab0001bc0 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.886+0000 7f7ac839d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.886+0000 7f7ac839d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.886+0000 7f7ac839d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.886+0000 7f7ac839d640 1 -- 172.21.15.45:0/1177563390 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa8005010 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.886+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f7ab00440c0 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.886+0000 7f7ac6139640 10 monclient: get_auth_request con 0x7f7aa403aab0 auth_method 0 2024-09-18T13:59:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.891+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f7ab00125e0 con 0x7f7ac00a0cb0 2024-09-18T13:59:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.891+0000 7f7ab67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:09.891+0000 7f7ab67fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:10.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.057+0000 7f7ac839d640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-18T13:59:10.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.057+0000 7f7ac839d640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-18T13:59:10.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.057+0000 7f7ac839d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:10.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.057+0000 7f7ac839d640 1 -- 172.21.15.45:0/1177563390 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7aa8005840 con 0x7f7ac00a0cb0 2024-09-18T13:59:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.057+0000 7f7ab67fc640 1 -- 172.21.15.45:0/1177563390 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (unknown 3870207488 0 2170607528) 0x7f7ab0012400 con 0x7f7ac00a0cb0 2024-09-18T13:59:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.057+0000 7f7ab67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-18T13:59:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.057+0000 7f7ab67fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:10.059 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-09-18T13:59:10.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.060+0000 7f7ac839d640 1 -- 172.21.15.45:0/1177563390 >> v1:172.21.15.84:6812/31478 conn(0x7f7aa403aab0 legacy=0x7f7aa403cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.060+0000 7f7ac839d640 10 monclient: shutdown 2024-09-18T13:59:10.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.060+0000 7f7ac839d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:10.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.060+0000 7f7ac839d640 1 -- 172.21.15.45:0/1177563390 >> v1:172.21.15.45:6792/0 conn(0x7f7ac00a0cb0 legacy=0x7f7ac01468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.060+0000 7f7ac839d640 1 -- 172.21.15.45:0/1177563390 shutdown_connections 2024-09-18T13:59:10.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.060+0000 7f7ac839d640 1 -- 172.21.15.45:0/1177563390 >> 172.21.15.45:0/1177563390 conn(0x7f7ac00a5bb0 msgr2=0x7f7ac00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:10.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.060+0000 7f7ac839d640 1 -- 172.21.15.45:0/1177563390 shutdown_connections 2024-09-18T13:59:10.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.060+0000 7f7ac839d640 1 -- 172.21.15.45:0/1177563390 wait complete. 2024-09-18T13:59:10.073 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-09-18T13:59:10.073 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T13:59:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.170+0000 7f8720ca8640 10 monclient: get_monmap_and_config 2024-09-18T13:59:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.170+0000 7f8720ca8640 10 monclient: build_initial_monmap 2024-09-18T13:59:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.170+0000 7f8720ca8640 10 monclient: monmap: 2024-09-18T13:59:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:10.171435+0000 2024-09-18T13:59:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:10.171435+0000 2024-09-18T13:59:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.171 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:10.172 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:10.172 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 1 Processor -- start 2024-09-18T13:59:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 1 -- start start 2024-09-18T13:59:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 10 monclient: init 2024-09-18T13:59:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 10 monclient: _add_conns ranks=[2,1,0,4,3,7,5,8,6] 2024-09-18T13:59:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 10 monclient(hunting): picked mon.noname-b con 0x7f871c09f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 10 monclient(hunting): picked mon.noname-f con 0x7f871c09fba0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 10 monclient(hunting): picked mon.noname-a con 0x7f871c09ff70 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c05ed70 con 0x7f871c09ff70 2024-09-18T13:59:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c09dfc0 con 0x7f871c09fba0 2024-09-18T13:59:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c09dce0 con 0x7f871c09f7d0 2024-09-18T13:59:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:10.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f8720ca8640 10 monclient(hunting): authenticate will time out at 765.509815s 2024-09-18T13:59:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f871ad9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f871c09ff70 0x7f871c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36934/0 (socket says 172.21.15.45:36934) 2024-09-18T13:59:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f871a59c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f871c09f7d0 0x7f871c132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51570/0 (socket says 172.21.15.45:51570) 2024-09-18T13:59:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.171+0000 7f871ad9d640 1 -- 172.21.15.45:0/3141233828 learned_addr learned my addr 172.21.15.45:0/3141233828 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 1 -- 172.21.15.45:0/3141233828 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078694429 0 0) 0x7f871c05ed70 con 0x7f871c09ff70 2024-09-18T13:59:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient(hunting): my global_id is 4392 2024-09-18T13:59:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 1 -- 172.21.15.45:0/3141233828 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87080034e0 con 0x7f871c09ff70 2024-09-18T13:59:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 1 -- 172.21.15.45:0/3141233828 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458079726 0 0) 0x7f871c09dce0 con 0x7f871c09f7d0 2024-09-18T13:59:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient(hunting): my global_id is 4385 2024-09-18T13:59:10.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 1 -- 172.21.15.45:0/3141233828 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f871c05ed70 con 0x7f871c09f7d0 2024-09-18T13:59:10.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 1 -- 172.21.15.45:0/3141233828 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3749630601 0 0) 0x7f87080034e0 con 0x7f871c09ff70 2024-09-18T13:59:10.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 1 -- 172.21.15.45:0/3141233828 >> v1:172.21.15.45:6790/0 conn(0x7f871c09f7d0 legacy=0x7f871c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 1 -- 172.21.15.45:0/3141233828 >> v1:172.21.15.84:6789/0 conn(0x7f871c09fba0 legacy=0x7f871c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient: _finish_hunting 0 2024-09-18T13:59:10.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 1 monclient: found mon.noname-a 2024-09-18T13:59:10.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:10.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T13:59:10.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 1 -- 172.21.15.45:0/3141233828 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c0a1c00 con 0x7f871c09ff70 2024-09-18T13:59:10.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient: _finish_auth 0 2024-09-18T13:59:10.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:10.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 1 -- 172.21.15.45:0/3141233828 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8714001830 con 0x7f871c09ff70 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f8720ca8640 5 monclient: authenticate success, global_id 4392 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f8720ca8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient: dump: 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f8720ca8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 1 -- 172.21.15.45:0/3141233828 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8714001a10 con 0x7f871c09ff70 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 1 -- 172.21.15.45:0/3141233828 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8714004ba0 con 0x7f871c09ff70 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f8720ca8640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f8720ca8640 4 set_mon_vals no callback set 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.172+0000 7f871959a640 10 monclient: dump: 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.184 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.173+0000 7f8720ca8640 10 monclient: shutdown 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.173+0000 7f8720ca8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.173+0000 7f8720ca8640 1 -- 172.21.15.45:0/3141233828 >> v1:172.21.15.45:6789/0 conn(0x7f871c09ff70 legacy=0x7f871c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.173+0000 7f8720ca8640 1 -- 172.21.15.45:0/3141233828 shutdown_connections 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.173+0000 7f8720ca8640 1 -- 172.21.15.45:0/3141233828 >> 172.21.15.45:0/3141233828 conn(0x7f871c0630b0 msgr2=0x7f871c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.173+0000 7f8720ca8640 1 -- 172.21.15.45:0/3141233828 shutdown_connections 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.173+0000 7f8720ca8640 1 -- 172.21.15.45:0/3141233828 wait complete. 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.173+0000 7f8720ca8640 10 monclient: build_initial_monmap 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.173+0000 7f8720ca8640 10 monclient: monmap: 2024-09-18T13:59:10.185 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:10.186 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.173+0000 7f8720ca8640 1 Processor -- start 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f8720ca8640 1 -- start start 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f8720ca8640 10 monclient: init 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f8720ca8640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f8720ca8640 10 monclient: _add_conns ranks=[0,4,5,3,1,7,6,2,8] 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f8720ca8640 10 monclient(hunting): picked mon.noname-a con 0x7f871c09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f8720ca8640 10 monclient(hunting): picked mon.noname-c con 0x7f871c09fba0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f8720ca8640 10 monclient(hunting): picked mon.noname-h con 0x7f871c09ff70 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f8720ca8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c0a2560 con 0x7f871c09f7d0 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f8720ca8640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c142580 con 0x7f871c09fba0 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f8720ca8640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c141db0 con 0x7f871c09ff70 2024-09-18T13:59:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f8720ca8640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f8720ca8640 10 monclient(hunting): authenticate will time out at 765.512513s 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f871a59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f871c09f7d0 0x7f871c1416a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36948/0 (socket says 172.21.15.45:36948) 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f871a59c640 1 -- 172.21.15.45:0/921402815 learned_addr learned my addr 172.21.15.45:0/921402815 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221374668 0 0) 0x7f871c0a2560 con 0x7f871c09f7d0 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 10 monclient(hunting): my global_id is 4401 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc0034e0 con 0x7f871c09f7d0 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294426770 0 0) 0x7f8704001230 con 0x7f871c09fba0 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 10 monclient(hunting): my global_id is 4405 2024-09-18T13:59:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f871c0a2560 con 0x7f871c09fba0 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992803076 0 0) 0x7f871c141db0 con 0x7f871c09ff70 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 10 monclient(hunting): my global_id is 4370 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8704001230 con 0x7f871c09ff70 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.174+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3714755259 0 0) 0x7f86fc0034e0 con 0x7f871c09f7d0 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f871c141db0 con 0x7f871c09f7d0 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f870c001670 con 0x7f871c09f7d0 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient(hunting): dump: 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 273767297 0 0) 0x7f871c0a2560 con 0x7f871c09fba0 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86fc0034e0 con 0x7f871c09fba0 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8704001c70 con 0x7f871c09fba0 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient(hunting): dump: 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3671202421 0 0) 0x7f871c141db0 con 0x7f871c09f7d0 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 >> v1:172.21.15.84:6791/0 conn(0x7f871c09ff70 legacy=0x7f871c149a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 >> v1:172.21.15.45:6791/0 conn(0x7f871c09fba0 legacy=0x7f871c131870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 1 monclient: found mon.a 2024-09-18T13:59:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c1219c0 con 0x7f871c09f7d0 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f8720ca8640 5 monclient: authenticate success, global_id 4401 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f8720ca8640 10 monclient: _renew_subs 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f8720ca8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f8720ca8640 1 -- 172.21.15.45:0/921402815 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c121ca0 con 0x7f871c09f7d0 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f8720ca8640 10 monclient: _renew_subs 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f8720ca8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f8720ca8640 1 -- 172.21.15.45:0/921402815 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c132120 con 0x7f871c09f7d0 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f870c004dd0 con 0x7f871c09f7d0 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f870c005280 con 0x7f871c09f7d0 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f8718d99640 4 set_mon_vals no callback set 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.175+0000 7f87127fc640 10 monclient: dump: 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.176+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f870c010250 con 0x7f871c09f7d0 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.176+0000 7f8720ca8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.176+0000 7f8720ca8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.176+0000 7f8720ca8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.176+0000 7f8720ca8640 1 -- 172.21.15.45:0/921402815 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8700005010 con 0x7f871c09f7d0 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.176+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f870c046e70 con 0x7f871c09f7d0 2024-09-18T13:59:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.176+0000 7f8719d9b640 10 monclient: get_auth_request con 0x7f86fc03a570 auth_method 0 2024-09-18T13:59:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.180+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f870c011890 con 0x7f871c09f7d0 2024-09-18T13:59:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.180+0000 7f87127fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.180+0000 7f87127fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.340+0000 7f8720ca8640 1 -- 172.21.15.45:0/921402815 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8700000b90 con 0x7f86fc03a570 2024-09-18T13:59:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.341+0000 7f87127fc640 1 -- 172.21.15.45:0/921402815 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40972 (unknown 2965378022 0 416456791) 0x7f8700000b90 con 0x7f86fc03a570 2024-09-18T13:59:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:59:10.343 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-18T13:59:10.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.343+0000 7f8720ca8640 1 -- 172.21.15.45:0/921402815 >> v1:172.21.15.84:6812/31478 conn(0x7f86fc03a570 legacy=0x7f86fc03ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.343+0000 7f8720ca8640 10 monclient: shutdown 2024-09-18T13:59:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.343+0000 7f8720ca8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.343+0000 7f8720ca8640 1 -- 172.21.15.45:0/921402815 >> v1:172.21.15.45:6789/0 conn(0x7f871c09f7d0 legacy=0x7f871c1416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.344+0000 7f8720ca8640 1 -- 172.21.15.45:0/921402815 shutdown_connections 2024-09-18T13:59:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.344+0000 7f8720ca8640 1 -- 172.21.15.45:0/921402815 >> 172.21.15.45:0/921402815 conn(0x7f871c0630b0 msgr2=0x7f871c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.344+0000 7f8720ca8640 1 -- 172.21.15.45:0/921402815 shutdown_connections 2024-09-18T13:59:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.344+0000 7f8720ca8640 1 -- 172.21.15.45:0/921402815 wait complete. 2024-09-18T13:59:10.354 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-18T13:59:09.888198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1786248,"kb_used_data":1848,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":564444792,"statfs":{"total":579820584960,"available":577991467008,"internally_reserved":0,"allocated":1892352,"data_stored":1304944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000348"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.875626+0000","last_change":"2024-09-18T13:59:05.875874+0000","last_active":"2024-09-18T13:59:05.875626+0000","last_peered":"2024-09-18T13:59:05.875626+0000","last_clean":"2024-09-18T13:59:05.875626+0000","last_became_active":"2024-09-18T13:59:02.976936+0000","last_became_peered":"2024-09-18T13:59:02.976936+0000","last_unstale":"2024-09-18T13:59:05.875626+0000","last_undegraded":"2024-09-18T13:59:05.875626+0000","last_fullsized":"2024-09-18T13:59:05.875626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:20:37.890959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036210499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.035734+0000","last_change":"2024-09-18T13:59:05.035951+0000","last_active":"2024-09-18T13:59:05.035734+0000","last_peered":"2024-09-18T13:59:05.035734+0000","last_clean":"2024-09-18T13:59:05.035734+0000","last_became_active":"2024-09-18T13:59:03.705691+0000","last_became_peered":"2024-09-18T13:59:03.705691+0000","last_unstale":"2024-09-18T13:59:05.035734+0000","last_undegraded":"2024-09-18T13:59:05.035734+0000","last_fullsized":"2024-09-18T13:59:05.035734+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:45:16.896857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033710499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036257+0000","last_change":"2024-09-18T13:59:05.036455+0000","last_active":"2024-09-18T13:59:05.036257+0000","last_peered":"2024-09-18T13:59:05.036257+0000","last_clean":"2024-09-18T13:59:05.036257+0000","last_became_active":"2024-09-18T13:59:02.974240+0000","last_became_peered":"2024-09-18T13:59:02.974240+0000","last_unstale":"2024-09-18T13:59:05.036257+0000","last_undegraded":"2024-09-18T13:59:05.036257+0000","last_fullsized":"2024-09-18T13:59:05.036257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:13:40.312100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038595099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036428+0000","last_change":"2024-09-18T13:59:05.036936+0000","last_active":"2024-09-18T13:59:05.036428+0000","last_peered":"2024-09-18T13:59:05.036428+0000","last_clean":"2024-09-18T13:59:05.036428+0000","last_became_active":"2024-09-18T13:59:02.977186+0000","last_became_peered":"2024-09-18T13:59:02.977186+0000","last_unstale":"2024-09-18T13:59:05.036428+0000","last_undegraded":"2024-09-18T13:59:05.036428+0000","last_fullsized":"2024-09-18T13:59:05.036428+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:01:46.516709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065897000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"15'72","reported_seq":100,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-18T13:59:04.370201+0000","last_change":"2024-09-18T13:59:02.975019+0000","last_active":"2024-09-18T13:59:04.370201+0000","last_peered":"2024-09-18T13:59:04.370201+0000","last_clean":"2024-09-18T13:59:04.370201+0000","last_became_active":"2024-09-18T13:59:02.974164+0000","last_became_peered":"2024-09-18T13:59:02.974164+0000","last_unstale":"2024-09-18T13:59:04.370201+0000","last_undegraded":"2024-09-18T13:59:04.370201+0000","last_fullsized":"2024-09-18T13:59:04.370201+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T22:10:24.958774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036373+0000","last_change":"2024-09-18T13:59:05.036690+0000","last_active":"2024-09-18T13:59:05.036373+0000","last_peered":"2024-09-18T13:59:05.036373+0000","last_clean":"2024-09-18T13:59:05.036373+0000","last_became_active":"2024-09-18T13:59:02.977312+0000","last_became_peered":"2024-09-18T13:59:02.977312+0000","last_unstale":"2024-09-18T13:59:05.036373+0000","last_undegraded":"2024-09-18T13:59:05.036373+0000","last_fullsized":"2024-09-18T13:59:05.036373+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:03:06.377092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072047099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036440+0000","last_change":"2024-09-18T13:59:05.036440+0000","last_active":"2024-09-18T13:59:05.036440+0000","last_peered":"2024-09-18T13:59:05.036440+0000","last_clean":"2024-09-18T13:59:05.036440+0000","last_became_active":"2024-09-18T13:59:03.705659+0000","last_became_peered":"2024-09-18T13:59:03.705659+0000","last_unstale":"2024-09-18T13:59:05.036440+0000","last_undegraded":"2024-09-18T13:59:05.036440+0000","last_fullsized":"2024-09-18T13:59:05.036440+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:49:16.930535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037781799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.875655+0000","last_change":"2024-09-18T13:59:05.875836+0000","last_active":"2024-09-18T13:59:05.875655+0000","last_peered":"2024-09-18T13:59:05.875655+0000","last_clean":"2024-09-18T13:59:05.875655+0000","last_became_active":"2024-09-18T13:59:03.705710+0000","last_became_peered":"2024-09-18T13:59:03.705710+0000","last_unstale":"2024-09-18T13:59:05.875655+0000","last_undegraded":"2024-09-18T13:59:05.875655+0000","last_fullsized":"2024-09-18T13:59:05.875655+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:12:55.202686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029541399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.035691+0000","last_change":"2024-09-18T13:59:05.035941+0000","last_active":"2024-09-18T13:59:05.035691+0000","last_peered":"2024-09-18T13:59:05.035691+0000","last_clean":"2024-09-18T13:59:05.035691+0000","last_became_active":"2024-09-18T13:59:03.705607+0000","last_became_peered":"2024-09-18T13:59:03.705607+0000","last_unstale":"2024-09-18T13:59:05.035691+0000","last_undegraded":"2024-09-18T13:59:05.035691+0000","last_fullsized":"2024-09-18T13:59:05.035691+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:25:38.392196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000366475,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":571475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297484,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074356,"statfs":{"total":96636764160,"available":96332140544,"internally_reserved":0,"allocated":131072,"data_stored":38431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":42566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640259,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":571475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":2,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297484,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074356,"statfs":{"total":96636764160,"available":96332140544,"internally_reserved":0,"allocated":131072,"data_stored":38431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640259,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":42566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T13:59:10.355 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T13:59:10.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.451+0000 7f4c4f2c0640 10 monclient: get_monmap_and_config 2024-09-18T13:59:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.451+0000 7f4c4f2c0640 10 monclient: build_initial_monmap 2024-09-18T13:59:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.451+0000 7f4c4f2c0640 10 monclient: monmap: 2024-09-18T13:59:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:10.452271+0000 2024-09-18T13:59:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:10.452271+0000 2024-09-18T13:59:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:10.453 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.451+0000 7f4c4f2c0640 1 Processor -- start 2024-09-18T13:59:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4f2c0640 1 -- start start 2024-09-18T13:59:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4f2c0640 10 monclient: init 2024-09-18T13:59:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4f2c0640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4f2c0640 10 monclient: _add_conns ranks=[2,5,4,7,8,1,3,0,6] 2024-09-18T13:59:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4f2c0640 10 monclient(hunting): picked mon.noname-b con 0x7f4c4809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4f2c0640 10 monclient(hunting): picked mon.noname-h con 0x7f4c4809fba0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4f2c0640 10 monclient(hunting): picked mon.noname-c con 0x7f4c4809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4f2c0640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4805ed70 con 0x7f4c4809f7d0 2024-09-18T13:59:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4f2c0640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4809dfc0 con 0x7f4c4809ff70 2024-09-18T13:59:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4f2c0640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4809dce0 con 0x7f4c4809fba0 2024-09-18T13:59:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4f2c0640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4f2c0640 10 monclient(hunting): authenticate will time out at 765.790616s 2024-09-18T13:59:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4d85d640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f4c4809ff70 0x7f4c4813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:45932/0 (socket says 172.21.15.45:45932) 2024-09-18T13:59:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4d05c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f4c4809f7d0 0x7f4c48132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51576/0 (socket says 172.21.15.45:51576) 2024-09-18T13:59:10.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.452+0000 7f4c4d85d640 1 -- 172.21.15.45:0/289660991 learned_addr learned my addr 172.21.15.45:0/289660991 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:10.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014049071 0 0) 0x7f4c4809dfc0 con 0x7f4c4809ff70 2024-09-18T13:59:10.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient(hunting): my global_id is 4414 2024-09-18T13:59:10.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c340034e0 con 0x7f4c4809ff70 2024-09-18T13:59:10.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883435919 0 0) 0x7f4c4805ed70 con 0x7f4c4809f7d0 2024-09-18T13:59:10.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient(hunting): my global_id is 4394 2024-09-18T13:59:10.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c4809dfc0 con 0x7f4c4809f7d0 2024-09-18T13:59:10.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442601797 0 0) 0x7f4c4809dce0 con 0x7f4c4809fba0 2024-09-18T13:59:10.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient(hunting): my global_id is 4379 2024-09-18T13:59:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c4805ed70 con 0x7f4c4809fba0 2024-09-18T13:59:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3653715104 0 0) 0x7f4c340034e0 con 0x7f4c4809ff70 2024-09-18T13:59:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 >> v1:172.21.15.84:6791/0 conn(0x7f4c4809fba0 legacy=0x7f4c48137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 >> v1:172.21.15.45:6790/0 conn(0x7f4c4809f7d0 legacy=0x7f4c48132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient: _finish_hunting 0 2024-09-18T13:59:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 monclient: found mon.noname-c 2024-09-18T13:59:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T13:59:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c480a1c00 con 0x7f4c4809ff70 2024-09-18T13:59:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient: _finish_auth 0 2024-09-18T13:59:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c40001830 con 0x7f4c4809ff70 2024-09-18T13:59:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T13:59:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient: dump: 2024-09-18T13:59:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.468 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c4f2c0640 5 monclient: authenticate success, global_id 4414 2024-09-18T13:59:10.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c4f2c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:10.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4c40001990 con 0x7f4c4809ff70 2024-09-18T13:59:10.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:10.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 1 -- 172.21.15.45:0/289660991 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c40004ba0 con 0x7f4c4809ff70 2024-09-18T13:59:10.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c4f2c0640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:10.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c4f2c0640 4 set_mon_vals no callback set 2024-09-18T13:59:10.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:10.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c3ffff640 10 monclient: dump: 2024-09-18T13:59:10.469 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.471 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c4f2c0640 10 monclient: shutdown 2024-09-18T13:59:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c4f2c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/289660991 >> v1:172.21.15.45:6791/0 conn(0x7f4c4809ff70 legacy=0x7f4c4813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/289660991 shutdown_connections 2024-09-18T13:59:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.453+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/289660991 >> 172.21.15.45:0/289660991 conn(0x7f4c480630b0 msgr2=0x7f4c480634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.454+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/289660991 shutdown_connections 2024-09-18T13:59:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.454+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/289660991 wait complete. 2024-09-18T13:59:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.454+0000 7f4c4f2c0640 10 monclient: build_initial_monmap 2024-09-18T13:59:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.454+0000 7f4c4f2c0640 10 monclient: monmap: 2024-09-18T13:59:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:10.473 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.454+0000 7f4c4f2c0640 1 Processor -- start 2024-09-18T13:59:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.454+0000 7f4c4f2c0640 1 -- start start 2024-09-18T13:59:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.454+0000 7f4c4f2c0640 10 monclient: init 2024-09-18T13:59:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c4f2c0640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c4f2c0640 10 monclient: _add_conns ranks=[6,1,2,7,3,5,4,0,8] 2024-09-18T13:59:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c4f2c0640 10 monclient(hunting): picked mon.noname-d con 0x7f4c4809f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c4f2c0640 10 monclient(hunting): picked mon.noname-f con 0x7f4c4809fba0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c4f2c0640 10 monclient(hunting): picked mon.noname-b con 0x7f4c4809ff70 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c4f2c0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4809d310 con 0x7f4c4809fba0 2024-09-18T13:59:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c4f2c0640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c480a2560 con 0x7f4c4809ff70 2024-09-18T13:59:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c4f2c0640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c48121f60 con 0x7f4c4809f7d0 2024-09-18T13:59:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c4f2c0640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c4f2c0640 10 monclient(hunting): authenticate will time out at 765.793611s 2024-09-18T13:59:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c4d05c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f4c4809f7d0 0x7f4c481418b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34804/0 (socket says 172.21.15.45:34804) 2024-09-18T13:59:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c4d05c640 1 -- 172.21.15.45:0/4124622400 learned_addr learned my addr 172.21.15.45:0/4124622400 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379109618 0 0) 0x7f4c48121f60 con 0x7f4c4809f7d0 2024-09-18T13:59:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c3d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c3d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c3d7fa640 10 monclient(hunting): my global_id is 4398 2024-09-18T13:59:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c2c0034e0 con 0x7f4c4809f7d0 2024-09-18T13:59:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082414011 0 0) 0x7f4c480a2560 con 0x7f4c4809ff70 2024-09-18T13:59:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c3d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c3d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c3d7fa640 10 monclient(hunting): my global_id is 4403 2024-09-18T13:59:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c48121f60 con 0x7f4c4809ff70 2024-09-18T13:59:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.455+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222976317 0 0) 0x7f4c4809d310 con 0x7f4c4809fba0 2024-09-18T13:59:10.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): my global_id is 4420 2024-09-18T13:59:10.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c480a2560 con 0x7f4c4809fba0 2024-09-18T13:59:10.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1335772005 0 0) 0x7f4c48121f60 con 0x7f4c4809ff70 2024-09-18T13:59:10.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4c4809d310 con 0x7f4c4809ff70 2024-09-18T13:59:10.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3489003626 0 0) 0x7f4c2c0034e0 con 0x7f4c4809f7d0 2024-09-18T13:59:10.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4c48121f60 con 0x7f4c4809f7d0 2024-09-18T13:59:10.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c40001770 con 0x7f4c4809ff70 2024-09-18T13:59:10.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:10.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): dump: 2024-09-18T13:59:10.477 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.477 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.477 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.477 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.477 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.477 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.477 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.477 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.477 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.477 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.478 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c44001990 con 0x7f4c4809f7d0 2024-09-18T13:59:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): dump: 2024-09-18T13:59:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.480 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1714572760 0 0) 0x7f4c480a2560 con 0x7f4c4809fba0 2024-09-18T13:59:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4c2c0034e0 con 0x7f4c4809fba0 2024-09-18T13:59:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c38001aa0 con 0x7f4c4809fba0 2024-09-18T13:59:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient(hunting): dump: 2024-09-18T13:59:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.482 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.482 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.482 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.482 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.482 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3685320651 0 0) 0x7f4c4809d310 con 0x7f4c4809ff70 2024-09-18T13:59:10.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 >> v1:172.21.15.45:6792/0 conn(0x7f4c4809f7d0 legacy=0x7f4c481418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 >> v1:172.21.15.84:6789/0 conn(0x7f4c4809fba0 legacy=0x7f4c48131a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient: _finish_hunting 0 2024-09-18T13:59:10.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 monclient: found mon.b 2024-09-18T13:59:10.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:10.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:10.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c48142790 con 0x7f4c4809ff70 2024-09-18T13:59:10.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient: _finish_auth 0 2024-09-18T13:59:10.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:10.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c4f2c0640 5 monclient: authenticate success, global_id 4403 2024-09-18T13:59:10.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c4f2c0640 10 monclient: _renew_subs 2024-09-18T13:59:10.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c4f2c0640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:10.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/4124622400 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c48121c30 con 0x7f4c4809ff70 2024-09-18T13:59:10.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c4f2c0640 10 monclient: _renew_subs 2024-09-18T13:59:10.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c4f2c0640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:10.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/4124622400 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c48120110 con 0x7f4c4809ff70 2024-09-18T13:59:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4c40004ad0 con 0x7f4c4809ff70 2024-09-18T13:59:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c400010e0 con 0x7f4c4809ff70 2024-09-18T13:59:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3f7fe640 4 set_mon_vals no callback set 2024-09-18T13:59:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.456+0000 7f4c3d7fa640 10 monclient: dump: 2024-09-18T13:59:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.485 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.485 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.485 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.485 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.485 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.485 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.485 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.485 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.485 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.485 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.486 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.486 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.457+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4c40005080 con 0x7f4c4809ff70 2024-09-18T13:59:10.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.457+0000 7f4c4f2c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:10.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.457+0000 7f4c4f2c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:10.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.457+0000 7f4c4f2c0640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:10.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.457+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/4124622400 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c28005010 con 0x7f4c4809ff70 2024-09-18T13:59:10.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.457+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f4c40046670 con 0x7f4c4809ff70 2024-09-18T13:59:10.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.457+0000 7f4c4d05c640 10 monclient: get_auth_request con 0x7f4c2c03a9b0 auth_method 0 2024-09-18T13:59:10.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.461+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f4c40015080 con 0x7f4c4809ff70 2024-09-18T13:59:10.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.461+0000 7f4c3d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:10.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.461+0000 7f4c3d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:10.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.622+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/4124622400 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c28000b90 con 0x7f4c2c03a9b0 2024-09-18T13:59:10.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.624+0000 7f4c3d7fa640 1 -- 172.21.15.45:0/4124622400 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40972 (unknown 2965378022 0 416456791) 0x7f4c28000b90 con 0x7f4c2c03a9b0 2024-09-18T13:59:10.625 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T13:59:10.625 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-18T13:59:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.625+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/4124622400 >> v1:172.21.15.84:6812/31478 conn(0x7f4c2c03a9b0 legacy=0x7f4c2c03ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.625+0000 7f4c4f2c0640 10 monclient: shutdown 2024-09-18T13:59:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.625+0000 7f4c4f2c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.625+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/4124622400 >> v1:172.21.15.45:6790/0 conn(0x7f4c4809ff70 legacy=0x7f4c48149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.626+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/4124622400 shutdown_connections 2024-09-18T13:59:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.626+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/4124622400 >> 172.21.15.45:0/4124622400 conn(0x7f4c480630b0 msgr2=0x7f4c480a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.626+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/4124622400 shutdown_connections 2024-09-18T13:59:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.626+0000 7f4c4f2c0640 1 -- 172.21.15.45:0/4124622400 wait complete. 2024-09-18T13:59:10.636 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-18T13:59:09.888198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1786248,"kb_used_data":1848,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":564444792,"statfs":{"total":579820584960,"available":577991467008,"internally_reserved":0,"allocated":1892352,"data_stored":1304944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000348"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.875626+0000","last_change":"2024-09-18T13:59:05.875874+0000","last_active":"2024-09-18T13:59:05.875626+0000","last_peered":"2024-09-18T13:59:05.875626+0000","last_clean":"2024-09-18T13:59:05.875626+0000","last_became_active":"2024-09-18T13:59:02.976936+0000","last_became_peered":"2024-09-18T13:59:02.976936+0000","last_unstale":"2024-09-18T13:59:05.875626+0000","last_undegraded":"2024-09-18T13:59:05.875626+0000","last_fullsized":"2024-09-18T13:59:05.875626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:20:37.890959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036210499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.035734+0000","last_change":"2024-09-18T13:59:05.035951+0000","last_active":"2024-09-18T13:59:05.035734+0000","last_peered":"2024-09-18T13:59:05.035734+0000","last_clean":"2024-09-18T13:59:05.035734+0000","last_became_active":"2024-09-18T13:59:03.705691+0000","last_became_peered":"2024-09-18T13:59:03.705691+0000","last_unstale":"2024-09-18T13:59:05.035734+0000","last_undegraded":"2024-09-18T13:59:05.035734+0000","last_fullsized":"2024-09-18T13:59:05.035734+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:45:16.896857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033710499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036257+0000","last_change":"2024-09-18T13:59:05.036455+0000","last_active":"2024-09-18T13:59:05.036257+0000","last_peered":"2024-09-18T13:59:05.036257+0000","last_clean":"2024-09-18T13:59:05.036257+0000","last_became_active":"2024-09-18T13:59:02.974240+0000","last_became_peered":"2024-09-18T13:59:02.974240+0000","last_unstale":"2024-09-18T13:59:05.036257+0000","last_undegraded":"2024-09-18T13:59:05.036257+0000","last_fullsized":"2024-09-18T13:59:05.036257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:13:40.312100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038595099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036428+0000","last_change":"2024-09-18T13:59:05.036936+0000","last_active":"2024-09-18T13:59:05.036428+0000","last_peered":"2024-09-18T13:59:05.036428+0000","last_clean":"2024-09-18T13:59:05.036428+0000","last_became_active":"2024-09-18T13:59:02.977186+0000","last_became_peered":"2024-09-18T13:59:02.977186+0000","last_unstale":"2024-09-18T13:59:05.036428+0000","last_undegraded":"2024-09-18T13:59:05.036428+0000","last_fullsized":"2024-09-18T13:59:05.036428+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:01:46.516709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065897000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"15'72","reported_seq":100,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-18T13:59:04.370201+0000","last_change":"2024-09-18T13:59:02.975019+0000","last_active":"2024-09-18T13:59:04.370201+0000","last_peered":"2024-09-18T13:59:04.370201+0000","last_clean":"2024-09-18T13:59:04.370201+0000","last_became_active":"2024-09-18T13:59:02.974164+0000","last_became_peered":"2024-09-18T13:59:02.974164+0000","last_unstale":"2024-09-18T13:59:04.370201+0000","last_undegraded":"2024-09-18T13:59:04.370201+0000","last_fullsized":"2024-09-18T13:59:04.370201+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T22:10:24.958774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036373+0000","last_change":"2024-09-18T13:59:05.036690+0000","last_active":"2024-09-18T13:59:05.036373+0000","last_peered":"2024-09-18T13:59:05.036373+0000","last_clean":"2024-09-18T13:59:05.036373+0000","last_became_active":"2024-09-18T13:59:02.977312+0000","last_became_peered":"2024-09-18T13:59:02.977312+0000","last_unstale":"2024-09-18T13:59:05.036373+0000","last_undegraded":"2024-09-18T13:59:05.036373+0000","last_fullsized":"2024-09-18T13:59:05.036373+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:03:06.377092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072047099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.036440+0000","last_change":"2024-09-18T13:59:05.036440+0000","last_active":"2024-09-18T13:59:05.036440+0000","last_peered":"2024-09-18T13:59:05.036440+0000","last_clean":"2024-09-18T13:59:05.036440+0000","last_became_active":"2024-09-18T13:59:03.705659+0000","last_became_peered":"2024-09-18T13:59:03.705659+0000","last_unstale":"2024-09-18T13:59:05.036440+0000","last_undegraded":"2024-09-18T13:59:05.036440+0000","last_fullsized":"2024-09-18T13:59:05.036440+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:49:16.930535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037781799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.875655+0000","last_change":"2024-09-18T13:59:05.875836+0000","last_active":"2024-09-18T13:59:05.875655+0000","last_peered":"2024-09-18T13:59:05.875655+0000","last_clean":"2024-09-18T13:59:05.875655+0000","last_became_active":"2024-09-18T13:59:03.705710+0000","last_became_peered":"2024-09-18T13:59:03.705710+0000","last_unstale":"2024-09-18T13:59:05.875655+0000","last_undegraded":"2024-09-18T13:59:05.875655+0000","last_fullsized":"2024-09-18T13:59:05.875655+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:12:55.202686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029541399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-18T13:59:05.035691+0000","last_change":"2024-09-18T13:59:05.035941+0000","last_active":"2024-09-18T13:59:05.035691+0000","last_peered":"2024-09-18T13:59:05.035691+0000","last_clean":"2024-09-18T13:59:05.035691+0000","last_became_active":"2024-09-18T13:59:03.705607+0000","last_became_peered":"2024-09-18T13:59:03.705607+0000","last_unstale":"2024-09-18T13:59:05.035691+0000","last_undegraded":"2024-09-18T13:59:05.035691+0000","last_fullsized":"2024-09-18T13:59:05.035691+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:25:38.392196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000366475,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":571475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297484,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074356,"statfs":{"total":96636764160,"available":96332140544,"internally_reserved":0,"allocated":131072,"data_stored":38431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":42566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640259,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":571475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":2,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297484,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074356,"statfs":{"total":96636764160,"available":96332140544,"internally_reserved":0,"allocated":131072,"data_stored":38431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640259,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":42566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T13:59:10.637 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-09-18T13:59:10.638 DEBUG:tasks.mon_thrash:Yielding 2024-09-18T13:59:10.638 INFO:teuthology.run_tasks:Running task rados... 2024-09-18T13:59:10.649 INFO:tasks.rados:Beginning rados... 2024-09-18T13:59:10.649 INFO:tasks.rados:config is {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000} 2024-09-18T13:59:10.649 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'mon_bind_msgr2': False, 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}} 2024-09-18T13:59:10.649 INFO:tasks.rados:config is {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000} 2024-09-18T13:59:10.649 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-09-18T13:59:10.658 INFO:tasks.rados:joining rados 2024-09-18T13:59:10.658 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-09-18T13:59:10.658 INFO:tasks.mon_thrash.mon_thrasher:seed: 1726667950, revive delay: 90.0, thrash delay: 1.0 thrash many: True, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-09-18T13:59:10.658 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T13:59:10.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T13:59:10.659 INFO:tasks.rados:clients are ['client.0'] 2024-09-18T13:59:10.659 INFO:tasks.rados:starting run 0 out of 1 2024-09-18T13:59:10.659 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-18T13:59:10.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2024-09-18T13:59:10.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.754+0000 7f31496e8640 10 monclient: get_monmap_and_config 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.754+0000 7f31496e8640 10 monclient: build_initial_monmap 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.754+0000 7f31496e8640 10 monclient: monmap: 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:10.755721+0000 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:10.755721+0000 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:10.756 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:10.757 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:10.757 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:10.757 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:10.757 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:10.757 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 1 Processor -- start 2024-09-18T13:59:10.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 1 -- start start 2024-09-18T13:59:10.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 10 monclient: init 2024-09-18T13:59:10.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:10.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 10 monclient: _add_conns ranks=[7,0,8,1,3,2,4,5,6] 2024-09-18T13:59:10.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 10 monclient(hunting): picked mon.noname-i con 0x7f314406e810 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:10.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 10 monclient(hunting): picked mon.noname-a con 0x7f31440a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:10.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 10 monclient(hunting): picked mon.noname-e con 0x7f31440a1080 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314405ed70 con 0x7f31440a0cb0 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3144097620 con 0x7f314406e810 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3144097340 con 0x7f31440a1080 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.755+0000 7f31496e8640 10 monclient(hunting): authenticate will time out at 766.093901s 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f31427fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f31440a0cb0 0x7f3144137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36956/0 (socket says 172.21.15.45:36956) 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f31427fc640 1 -- 172.21.15.45:0/326857999 learned_addr learned my addr 172.21.15.45:0/326857999 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f31437fe640 1 --1- 172.21.15.45:0/326857999 >> v1:172.21.15.45:6793/0 conn(0x7f31440a1080 0x7f314413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51898/0 (socket says 172.21.15.45:51898) 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978766114 0 0) 0x7f3144097340 con 0x7f31440a1080 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient(hunting): my global_id is 4364 2024-09-18T13:59:10.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f312c0034e0 con 0x7f31440a1080 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186120879 0 0) 0x7f314405ed70 con 0x7f31440a0cb0 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient(hunting): my global_id is 4410 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3144097340 con 0x7f31440a0cb0 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792771654 0 0) 0x7f3144097620 con 0x7f314406e810 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient(hunting): my global_id is 4453 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f314405ed70 con 0x7f314406e810 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1881002939 0 0) 0x7f312c0034e0 con 0x7f31440a1080 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 >> v1:172.21.15.84:6792/0 conn(0x7f314406e810 legacy=0x7f31440a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 >> v1:172.21.15.45:6789/0 conn(0x7f31440a0cb0 legacy=0x7f3144137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 monclient: found mon.noname-e 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31440a1660 con 0x7f31440a1080 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f313c001bb0 con 0x7f31440a1080 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f31496e8640 5 monclient: authenticate success, global_id 4364 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f31496e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient: dump: 2024-09-18T13:59:10.766 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.767 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f31496e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f313c001d10 con 0x7f31440a1080 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 1 -- 172.21.15.45:0/326857999 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f313c004b20 con 0x7f31440a1080 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f31496e8640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f31496e8640 4 set_mon_vals no callback set 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.756+0000 7f3141ffb640 10 monclient: dump: 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.768 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.769 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.769 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.769 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.769 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.769 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.769 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.769 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.769 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.769 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.769 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.769 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.757+0000 7f31496e8640 10 monclient: shutdown 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.757+0000 7f31496e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.757+0000 7f31496e8640 1 -- 172.21.15.45:0/326857999 >> v1:172.21.15.45:6793/0 conn(0x7f31440a1080 legacy=0x7f314413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.757+0000 7f31496e8640 1 -- 172.21.15.45:0/326857999 shutdown_connections 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.757+0000 7f31496e8640 1 -- 172.21.15.45:0/326857999 >> 172.21.15.45:0/326857999 conn(0x7f31440a5bb0 msgr2=0x7f31440a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.757+0000 7f31496e8640 1 -- 172.21.15.45:0/326857999 shutdown_connections 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.757+0000 7f31496e8640 1 -- 172.21.15.45:0/326857999 wait complete. 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.758+0000 7f31496e8640 10 monclient: build_initial_monmap 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.758+0000 7f31496e8640 10 monclient: monmap: 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:10.770 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.758+0000 7f31496e8640 1 Processor -- start 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.758+0000 7f31496e8640 1 -- start start 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.758+0000 7f31496e8640 10 monclient: init 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f31496e8640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f31496e8640 10 monclient: _add_conns ranks=[3,0,4,5,6,1,8,7,2] 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f31496e8640 10 monclient(hunting): picked mon.noname-g con 0x7f314406e810 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f31496e8640 10 monclient(hunting): picked mon.noname-a con 0x7f31440a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f31496e8640 10 monclient(hunting): picked mon.noname-c con 0x7f31440a1080 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f31496e8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314409fa10 con 0x7f31440a0cb0 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f31496e8640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314413dbf0 con 0x7f314406e810 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f31496e8640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31440a2270 con 0x7f31440a1080 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f31496e8640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f31496e8640 10 monclient(hunting): authenticate will time out at 766.097493s 2024-09-18T13:59:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f31427fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f31440a0cb0 0x7f31441466f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36970/0 (socket says 172.21.15.45:36970) 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f31427fc640 1 -- 172.21.15.45:0/194058718 learned_addr learned my addr 172.21.15.45:0/194058718 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432627861 0 0) 0x7f314409fa10 con 0x7f31440a0cb0 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 10 monclient(hunting): my global_id is 4419 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31300034e0 con 0x7f31440a0cb0 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017217377 0 0) 0x7f31440a2270 con 0x7f31440a1080 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 10 monclient(hunting): my global_id is 4423 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f314409fa10 con 0x7f31440a1080 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491766171 0 0) 0x7f314413dbf0 con 0x7f314406e810 2024-09-18T13:59:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.759+0000 7f312affd640 10 monclient(hunting): my global_id is 4476 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31440a2270 con 0x7f314406e810 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3879791273 0 0) 0x7f31300034e0 con 0x7f31440a0cb0 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f314413dbf0 con 0x7f31440a0cb0 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3134001690 con 0x7f31440a0cb0 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient(hunting): dump: 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2516691109 0 0) 0x7f314409fa10 con 0x7f31440a1080 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f31300034e0 con 0x7f31440a1080 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f313c001a70 con 0x7f31440a1080 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient(hunting): dump: 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.776 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.777 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.777 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.777 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.777 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.777 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.777 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.777 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.777 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.777 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1688766969 0 0) 0x7f31440a2270 con 0x7f314406e810 2024-09-18T13:59:10.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f314409fa10 con 0x7f314406e810 2024-09-18T13:59:10.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3138001aa0 con 0x7f314406e810 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient(hunting): dump: 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.779 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3079571689 0 0) 0x7f314413dbf0 con 0x7f31440a0cb0 2024-09-18T13:59:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 >> v1:172.21.15.45:6791/0 conn(0x7f31440a1080 legacy=0x7f3144149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 >> v1:172.21.15.84:6790/0 conn(0x7f314406e810 legacy=0x7f314413d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7fd81571d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:10.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7fd81571d640 10 monclient: build_initial_monmap 2024-09-18T13:59:10.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:10.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 monclient: found mon.a 2024-09-18T13:59:10.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:10.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:10.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314413ea00 con 0x7f31440a0cb0 2024-09-18T13:59:10.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient: _finish_auth 0 2024-09-18T13:59:10.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:10.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f31496e8640 5 monclient: authenticate success, global_id 4419 2024-09-18T13:59:10.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f31496e8640 10 monclient: _renew_subs 2024-09-18T13:59:10.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7fd81571d640 10 monclient: monmap: 2024-09-18T13:59:10.781 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:10.781 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.781 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:10.761302+0000 2024-09-18T13:59:10.781 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:10.761302+0000 2024-09-18T13:59:10.781 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:10.781 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.781 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:10.782 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:10.782 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:10.782 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:10.782 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:10.782 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:10.782 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:10.782 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:10.782 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:10.782 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f31496e8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f31496e8640 1 -- 172.21.15.45:0/194058718 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3144122580 con 0x7f31440a0cb0 2024-09-18T13:59:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f31496e8640 10 monclient: _renew_subs 2024-09-18T13:59:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f31496e8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f31496e8640 1 -- 172.21.15.45:0/194058718 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3144120190 con 0x7f31440a0cb0 2024-09-18T13:59:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3134004a50 con 0x7f31440a0cb0 2024-09-18T13:59:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3134004f00 con 0x7f31440a0cb0 2024-09-18T13:59:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f31417fa640 4 set_mon_vals no callback set 2024-09-18T13:59:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.760+0000 7f312affd640 10 monclient: dump: 2024-09-18T13:59:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.785 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.785 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.785 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.785 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.785 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.785 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.785 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 1 Processor -- start 2024-09-18T13:59:10.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 1 -- start start 2024-09-18T13:59:10.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 10 monclient: init 2024-09-18T13:59:10.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f313400fed0 con 0x7f31440a0cb0 2024-09-18T13:59:10.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7f31496e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:10.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7f31496e8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:10.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7f31496e8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:10.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7f31496e8640 1 -- 172.21.15.45:0/194058718 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3124005010 con 0x7f31440a0cb0 2024-09-18T13:59:10.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7f3134046a60 con 0x7f31440a0cb0 2024-09-18T13:59:10.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:10.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 10 monclient: _add_conns ranks=[8,1,0,3,2,5,7,6,4] 2024-09-18T13:59:10.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 10 monclient(hunting): picked mon.noname-e con 0x7fd81006dbc0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:10.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 10 monclient(hunting): picked mon.noname-f con 0x7fd81006e9f0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:10.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 10 monclient(hunting): picked mon.noname-a con 0x7fd81006d3b0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:10.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81005ed70 con 0x7fd81006d3b0 2024-09-18T13:59:10.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81006c780 con 0x7fd81006e9f0 2024-09-18T13:59:10.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81006c410 con 0x7fd81006dbc0 2024-09-18T13:59:10.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:10.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd81571d640 10 monclient(hunting): authenticate will time out at 766.099901s 2024-09-18T13:59:10.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7f3142ffd640 10 monclient: get_auth_request con 0x7f313003aab0 auth_method 0 2024-09-18T13:59:10.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.761+0000 7fd80effd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fd81006dbc0 0x7fd81006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51906/0 (socket says 172.21.15.45:51906) 2024-09-18T13:59:10.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80effd640 1 -- 172.21.15.45:0/3829163466 learned_addr learned my addr 172.21.15.45:0/3829163466 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 1 -- 172.21.15.45:0/3829163466 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306471134 0 0) 0x7fd81006c410 con 0x7fd81006dbc0 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient(hunting): my global_id is 4373 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 1 -- 172.21.15.45:0/3829163466 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7f80034e0 con 0x7fd81006dbc0 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 1 -- 172.21.15.45:0/3829163466 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031749470 0 0) 0x7fd81006c780 con 0x7fd81006e9f0 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient(hunting): my global_id is 4429 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 1 -- 172.21.15.45:0/3829163466 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81006c410 con 0x7fd81006e9f0 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 1 -- 172.21.15.45:0/3829163466 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3894928303 0 0) 0x7fd7f80034e0 con 0x7fd81006dbc0 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 1 -- 172.21.15.45:0/3829163466 >> v1:172.21.15.84:6789/0 conn(0x7fd81006e9f0 legacy=0x7fd810134430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 1 -- 172.21.15.45:0/3829163466 >> v1:172.21.15.45:6789/0 conn(0x7fd81006d3b0 legacy=0x7fd810137b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 1 monclient: found mon.noname-e 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 1 -- 172.21.15.45:0/3829163466 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8100a2190 con 0x7fd81006dbc0 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 1 -- 172.21.15.45:0/3829163466 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd800001830 con 0x7fd81006dbc0 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd81571d640 5 monclient: authenticate success, global_id 4373 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd81571d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient: dump: 2024-09-18T13:59:10.796 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.797 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd81571d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 1 -- 172.21.15.45:0/3829163466 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd800001a10 con 0x7fd81006dbc0 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 1 -- 172.21.15.45:0/3829163466 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd800004ba0 con 0x7fd81006dbc0 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd80dffb640 10 monclient: dump: 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.798 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd81571d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.762+0000 7fd81571d640 4 set_mon_vals no callback set 2024-09-18T13:59:10.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.763+0000 7fd81571d640 10 monclient: shutdown 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.763+0000 7fd81571d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.763+0000 7fd81571d640 1 -- 172.21.15.45:0/3829163466 >> v1:172.21.15.45:6793/0 conn(0x7fd81006dbc0 legacy=0x7fd81006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.763+0000 7fd81571d640 1 -- 172.21.15.45:0/3829163466 shutdown_connections 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.763+0000 7fd81571d640 1 -- 172.21.15.45:0/3829163466 >> 172.21.15.45:0/3829163466 conn(0x7fd810068760 msgr2=0x7fd8100a29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.763+0000 7fd81571d640 1 -- 172.21.15.45:0/3829163466 shutdown_connections 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.763+0000 7fd81571d640 1 -- 172.21.15.45:0/3829163466 wait complete. 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.763+0000 7fd81571d640 10 monclient: build_initial_monmap 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.763+0000 7fd81571d640 10 monclient: monmap: 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:10.800 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:10.801 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:10.801 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:10.801 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:10.801 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:10.801 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:10.801 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:10.801 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:10.801 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:10.801 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:10.801 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:10.801 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 1 Processor -- start 2024-09-18T13:59:10.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 1 -- start start 2024-09-18T13:59:10.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 10 monclient: init 2024-09-18T13:59:10.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:10.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 10 monclient: _add_conns ranks=[5,2,4,1,8,6,7,0,3] 2024-09-18T13:59:10.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 10 monclient(hunting): picked mon.noname-h con 0x7fd81006d3b0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:10.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 10 monclient(hunting): picked mon.noname-b con 0x7fd81006dbc0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:10.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 10 monclient(hunting): picked mon.noname-c con 0x7fd81006e9f0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:10.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81006b580 con 0x7fd81006dbc0 2024-09-18T13:59:10.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81009ccb0 con 0x7fd81006e9f0 2024-09-18T13:59:10.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8101204f0 con 0x7fd81006d3b0 2024-09-18T13:59:10.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:10.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd81571d640 10 monclient(hunting): authenticate will time out at 766.102787s 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd80f7fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fd81006e9f0 0x7fd810147930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:45958/0 (socket says 172.21.15.45:45958) 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd80e7fc640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fd81006dbc0 0x7fd8101333d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51596/0 (socket says 172.21.15.45:51596) 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.764+0000 7fd80f7fe640 1 -- 172.21.15.45:0/2427301475 learned_addr learned my addr 172.21.15.45:0/2427301475 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966340744 0 0) 0x7fd81006b580 con 0x7fd81006dbc0 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): my global_id is 4412 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7fc003510 con 0x7fd81006dbc0 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871527932 0 0) 0x7fd81009ccb0 con 0x7fd81006e9f0 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): my global_id is 4432 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81006b580 con 0x7fd81006e9f0 2024-09-18T13:59:10.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172122826 0 0) 0x7fd8101204f0 con 0x7fd81006d3b0 2024-09-18T13:59:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): my global_id is 4388 2024-09-18T13:59:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81009ccb0 con 0x7fd81006d3b0 2024-09-18T13:59:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 240467231 0 0) 0x7fd7fc003510 con 0x7fd81006dbc0 2024-09-18T13:59:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd8101204f0 con 0x7fd81006dbc0 2024-09-18T13:59:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd804002ab0 con 0x7fd81006dbc0 2024-09-18T13:59:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.806 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1093547622 0 0) 0x7fd81009ccb0 con 0x7fd81006d3b0 2024-09-18T13:59:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd7fc003510 con 0x7fd81006d3b0 2024-09-18T13:59:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd8000017f0 con 0x7fd81006d3b0 2024-09-18T13:59:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.808 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 442058185 0 0) 0x7fd81006b580 con 0x7fd81006e9f0 2024-09-18T13:59:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd81009ccb0 con 0x7fd81006e9f0 2024-09-18T13:59:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd808001da0 con 0x7fd81006e9f0 2024-09-18T13:59:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.809 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2562023691 0 0) 0x7fd8101204f0 con 0x7fd81006dbc0 2024-09-18T13:59:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 >> v1:172.21.15.84:6791/0 conn(0x7fd81006d3b0 legacy=0x7fd81013f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 >> v1:172.21.15.45:6791/0 conn(0x7fd81006e9f0 legacy=0x7fd810147930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 monclient: found mon.b 2024-09-18T13:59:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81011e9c0 con 0x7fd81006dbc0 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd81571d640 5 monclient: authenticate success, global_id 4412 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd81571d640 10 monclient: _renew_subs 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd81571d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd81571d640 1 -- 172.21.15.45:0/2427301475 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd810140300 con 0x7fd81006dbc0 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd81571d640 10 monclient: _renew_subs 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd804002d70 con 0x7fd81006dbc0 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd81571d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd81571d640 1 -- 172.21.15.45:0/2427301475 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd810133c80 con 0x7fd81006dbc0 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd804001940 con 0x7fd81006dbc0 2024-09-18T13:59:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd7f6ffd640 10 monclient: dump: 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:10.813 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.765+0000 7fd80d7fa640 4 set_mon_vals no callback set 2024-09-18T13:59:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.766+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd804001aa0 con 0x7fd81006dbc0 2024-09-18T13:59:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.766+0000 7fd81571d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.766+0000 7fd81571d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.766+0000 7fd81571d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.766+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f3134011a10 con 0x7f31440a0cb0 2024-09-18T13:59:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.766+0000 7f312affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.766+0000 7f312affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.766+0000 7fd81571d640 1 -- 172.21.15.45:0/2427301475 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81004c4a0 con 0x7fd81006dbc0 2024-09-18T13:59:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.767+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 179967801 0 0) 0x7fd804046820 con 0x7fd81006dbc0 2024-09-18T13:59:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.767+0000 7fd80effd640 10 monclient: get_auth_request con 0x7fd7fc03abb0 auth_method 0 2024-09-18T13:59:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.771+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fd804001d50 con 0x7fd81006dbc0 2024-09-18T13:59:10.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.771+0000 7fd7f6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:10.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.771+0000 7fd7f6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:10.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.936+0000 7fd81571d640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2024-09-18T13:59:10.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.936+0000 7fd81571d640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2024-09-18T13:59:10.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.936+0000 7fd81571d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:10.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.936+0000 7fd81571d640 1 -- 172.21.15.45:0/2427301475 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} v 0) v1 -- 0x7fd81004b720 con 0x7fd81006dbc0 2024-09-18T13:59:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.993+0000 7f31496e8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T13:59:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.993+0000 7f31496e8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T13:59:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.993+0000 7f31496e8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.993+0000 7f31496e8640 1 -- 172.21.15.45:0/194058718 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3124005840 con 0x7f31440a0cb0 2024-09-18T13:59:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.993+0000 7f312affd640 1 -- 172.21.15.45:0/194058718 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2689 (unknown 254746657 0 4072662144) 0x7f3134014da0 con 0x7f31440a0cb0 2024-09-18T13:59:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.993+0000 7f312affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T13:59:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.993+0000 7f312affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:10.995 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":14,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":18,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T13:59:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.995+0000 7f31496e8640 1 -- 172.21.15.45:0/194058718 >> v1:172.21.15.84:6812/31478 conn(0x7f313003aab0 legacy=0x7f313003cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.995+0000 7f31496e8640 10 monclient: shutdown 2024-09-18T13:59:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.995+0000 7f31496e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.995+0000 7f31496e8640 1 -- 172.21.15.45:0/194058718 >> v1:172.21.15.45:6789/0 conn(0x7f31440a0cb0 legacy=0x7f31441466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.995+0000 7f31496e8640 1 -- 172.21.15.45:0/194058718 shutdown_connections 2024-09-18T13:59:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.995+0000 7f31496e8640 1 -- 172.21.15.45:0/194058718 >> 172.21.15.45:0/194058718 conn(0x7f31440a5bb0 msgr2=0x7f31440a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.995+0000 7f31496e8640 1 -- 172.21.15.45:0/194058718 shutdown_connections 2024-09-18T13:59:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:10.995+0000 7f31496e8640 1 -- 172.21.15.45:0/194058718 wait complete. 2024-09-18T13:59:11.006 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T13:59:11.006 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-18T13:59:11.006 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T13:59:11.006 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T13:59:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.071+0000 7fd7f6ffd640 1 -- 172.21.15.45:0/2427301475 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]=0 pool 'unique_pool_0' created v17) v1 ==== 130+0+0 (unknown 1233589309 0 0) 0x7fd804014bc0 con 0x7fd81006dbc0 2024-09-18T13:59:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.071+0000 7fd7f6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2024-09-18T13:59:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.071+0000 7fd7f6ffd640 10 monclient: _finish_command 2 = system:0 pool 'unique_pool_0' created 2024-09-18T13:59:11.073 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'unique_pool_0' created 2024-09-18T13:59:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.077+0000 7fd81571d640 1 -- 172.21.15.45:0/2427301475 >> v1:172.21.15.84:6812/31478 conn(0x7fd7fc03abb0 legacy=0x7fd7fc03d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.077+0000 7fd81571d640 10 monclient: shutdown 2024-09-18T13:59:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.077+0000 7fd81571d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.077+0000 7fd81571d640 1 -- 172.21.15.45:0/2427301475 >> v1:172.21.15.45:6790/0 conn(0x7fd81006dbc0 legacy=0x7fd8101333d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.077+0000 7fd81571d640 1 -- 172.21.15.45:0/2427301475 shutdown_connections 2024-09-18T13:59:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.077+0000 7fd81571d640 1 -- 172.21.15.45:0/2427301475 >> 172.21.15.45:0/2427301475 conn(0x7fd810068760 msgr2=0x7fd8100670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.077+0000 7fd81571d640 1 -- 172.21.15.45:0/2427301475 shutdown_connections 2024-09-18T13:59:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.077+0000 7fd81571d640 1 -- 172.21.15.45:0/2427301475 wait complete. 2024-09-18T13:59:11.093 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-18T13:59:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.114+0000 7f4a6099d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.114+0000 7f4a6099d640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.114+0000 7f4a6099d640 10 monclient: monmap: 2024-09-18T13:59:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:11.115591+0000 2024-09-18T13:59:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:11.115591+0000 2024-09-18T13:59:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.116 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.115+0000 7f4a6099d640 1 Processor -- start 2024-09-18T13:59:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.115+0000 7f4a6099d640 1 -- start start 2024-09-18T13:59:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.115+0000 7f4a6099d640 10 monclient: init 2024-09-18T13:59:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.115+0000 7f4a6099d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.115+0000 7f4a6099d640 10 monclient: _add_conns ranks=[2,4,0,7,6,8,5,1,3] 2024-09-18T13:59:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.115+0000 7f4a6099d640 10 monclient(hunting): picked mon.noname-b con 0x7f4a5c06e810 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.115+0000 7f4a6099d640 10 monclient(hunting): picked mon.noname-c con 0x7f4a5c0a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.115+0000 7f4a6099d640 10 monclient(hunting): picked mon.noname-a con 0x7f4a5c0a1080 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.115+0000 7f4a6099d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c05ed70 con 0x7f4a5c0a1080 2024-09-18T13:59:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a6099d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c097620 con 0x7f4a5c06e810 2024-09-18T13:59:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a6099d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c097340 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a6099d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a6099d640 10 monclient(hunting): authenticate will time out at 766.454031s 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a59d9b640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f4a5c0a0cb0 0x7f4a5c137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:45962/0 (socket says 172.21.15.45:45962) 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a59d9b640 1 -- 172.21.15.45:0/1420347002 learned_addr learned my addr 172.21.15.45:0/1420347002 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5ad9d640 1 --1- 172.21.15.45:0/1420347002 >> v1:172.21.15.45:6789/0 conn(0x7f4a5c0a1080 0x7f4a5c13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36990/0 (socket says 172.21.15.45:36990) 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724200436 0 0) 0x7f4a5c097340 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient(hunting): my global_id is 4441 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a440034e0 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887017658 0 0) 0x7f4a5c097620 con 0x7f4a5c06e810 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient(hunting): my global_id is 4421 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a5c097340 con 0x7f4a5c06e810 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016155149 0 0) 0x7f4a5c05ed70 con 0x7f4a5c0a1080 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient(hunting): my global_id is 4428 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a5c097620 con 0x7f4a5c0a1080 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3000818058 0 0) 0x7f4a5c097340 con 0x7f4a5c06e810 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 >> v1:172.21.15.45:6791/0 conn(0x7f4a5c0a0cb0 legacy=0x7f4a5c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 >> v1:172.21.15.45:6789/0 conn(0x7f4a5c0a1080 legacy=0x7f4a5c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 1 monclient: found mon.noname-b 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c0a1660 con 0x7f4a5c06e810 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a4c001810 con 0x7f4a5c06e810 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a5959a640 10 monclient: dump: 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a6099d640 5 monclient: authenticate success, global_id 4421 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.116+0000 7f4a6099d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.117+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4a4c0019f0 con 0x7f4a5c06e810 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.117+0000 7f4a5959a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.117+0000 7f4a5959a640 1 -- 172.21.15.45:0/1420347002 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a4c001f20 con 0x7f4a5c06e810 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.117+0000 7f4a6099d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.117+0000 7f4a6099d640 4 set_mon_vals no callback set 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.117+0000 7f4a5959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.117+0000 7f4a5959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.117+0000 7f4a5959a640 10 monclient: dump: 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.131 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.117+0000 7f4a6099d640 10 monclient: shutdown 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.117+0000 7f4a6099d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.117+0000 7f4a6099d640 1 -- 172.21.15.45:0/1420347002 >> v1:172.21.15.45:6790/0 conn(0x7f4a5c06e810 legacy=0x7f4a5c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.118+0000 7f4a6099d640 1 -- 172.21.15.45:0/1420347002 shutdown_connections 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.118+0000 7f4a6099d640 1 -- 172.21.15.45:0/1420347002 >> 172.21.15.45:0/1420347002 conn(0x7f4a5c0a5bb0 msgr2=0x7f4a5c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.118+0000 7f4a6099d640 1 -- 172.21.15.45:0/1420347002 shutdown_connections 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.118+0000 7f4a6099d640 1 -- 172.21.15.45:0/1420347002 wait complete. 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.118+0000 7f4a6099d640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.118+0000 7f4a6099d640 10 monclient: monmap: 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.118+0000 7f4a6099d640 1 Processor -- start 2024-09-18T13:59:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a6099d640 1 -- start start 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a6099d640 10 monclient: init 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a6099d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a6099d640 10 monclient: _add_conns ranks=[3,4,2,0,7,1,5,6,8] 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a6099d640 10 monclient(hunting): picked mon.noname-g con 0x7f4a5c06e810 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a6099d640 10 monclient(hunting): picked mon.noname-c con 0x7f4a5c0a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a6099d640 10 monclient(hunting): picked mon.noname-b con 0x7f4a5c0a1080 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a6099d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c0a2270 con 0x7f4a5c0a1080 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a6099d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c122730 con 0x7f4a5c06e810 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a6099d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c13da30 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a6099d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a6099d640 10 monclient(hunting): authenticate will time out at 766.457600s 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a59d9b640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f4a5c0a0cb0 0x7f4a5c146530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:45972/0 (socket says 172.21.15.45:45972) 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a59d9b640 1 -- 172.21.15.45:0/2785142181 learned_addr learned my addr 172.21.15.45:0/2785142181 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.119+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822957567 0 0) 0x7f4a5c13da30 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): my global_id is 4450 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a3c0034e0 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156128331 0 0) 0x7f4a5c0a2270 con 0x7f4a5c0a1080 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): my global_id is 4430 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a5c13da30 con 0x7f4a5c0a1080 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049780987 0 0) 0x7f4a5c122730 con 0x7f4a5c06e810 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): my global_id is 4485 2024-09-18T13:59:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a5c0a2270 con 0x7f4a5c06e810 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4292467501 0 0) 0x7f4a3c0034e0 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a5c122730 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a50001690 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3696025969 0 0) 0x7f4a5c13da30 con 0x7f4a5c0a1080 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a3c0034e0 con 0x7f4a5c0a1080 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a54001bb0 con 0x7f4a5c0a1080 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 27248928 0 0) 0x7f4a5c122730 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 >> v1:172.21.15.84:6790/0 conn(0x7f4a5c06e810 legacy=0x7f4a5c13d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 >> v1:172.21.15.45:6790/0 conn(0x7f4a5c0a1080 legacy=0x7f4a5c149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 monclient: found mon.c 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c13e6f0 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a6099d640 5 monclient: authenticate success, global_id 4450 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a6099d640 10 monclient: _renew_subs 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a6099d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a6099d640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5c1223c0 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a6099d640 10 monclient: _renew_subs 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a6099d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a6099d640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c13ded0 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4a50004a50 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a50004f00 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4bfff640 4 set_mon_vals no callback set 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.120+0000 7f4a4a7fc640 10 monclient: dump: 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.121+0000 7f4a6099d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.121+0000 7f4a6099d640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4a40006580 con 0x7f4a40002830 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.121+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4a5000fed0 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 3882+0+0 (unknown 594623498 0 0) 0x7f4a50046b50 con 0x7f4a5c0a0cb0 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962816699 0 0) 0x7f4a40006580 con 0x7f4a40002830 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 10 monclient(hunting): my global_id is 4437 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a5c122730 con 0x7f4a40002830 2024-09-18T13:59:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a5ad9d640 10 monclient: get_auth_request con 0x7f4a3c03a5f0 auth_method 0 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2053783998 0 0) 0x7f4a5c122730 con 0x7f4a40002830 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a40006580 con 0x7f4a40002830 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a4c001020 con 0x7f4a40002830 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 10 monclient: dump: 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.144 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2640706143 0 0) 0x7f4a40006580 con 0x7f4a40002830 2024-09-18T13:59:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.122+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a4000a0a0 con 0x7f4a40002830 2024-09-18T13:59:11.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.123+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f4a4c0024e0 con 0x7f4a40002830 2024-09-18T13:59:11.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.123+0000 7f4a4a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.123+0000 7f4a4a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:11.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.123+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 >> v1:172.21.15.45:6789/0 conn(0x7f4a40002830 legacy=0x7f4a40004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.123+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 >> v1:172.21.15.45:6789/0 conn(0x7f4a40002830 legacy=0x7f4a40004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:11.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.135+0000 7f4a6099d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:11.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.135+0000 7f4a6099d640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4a40007540 con 0x7f4a400068f0 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.135+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252113566 0 0) 0x7f4a40007540 con 0x7f4a400068f0 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.135+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.135+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.135+0000 7f4a4a7fc640 10 monclient(hunting): my global_id is 4446 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.135+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a40006580 con 0x7f4a400068f0 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3356744961 0 0) 0x7f4a40006580 con 0x7f4a400068f0 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a40007540 con 0x7f4a400068f0 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a4c002d00 con 0x7f4a400068f0 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 10 monclient: dump: 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.148 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2834998243 0 0) 0x7f4a40007540 con 0x7f4a400068f0 2024-09-18T13:59:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4a400087a0 con 0x7f4a400068f0 2024-09-18T13:59:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 136984087) 0x7f4a4c004cf0 con 0x7f4a400068f0 2024-09-18T13:59:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 >> v1:172.21.15.45:6789/0 conn(0x7f4a400068f0 legacy=0x7f4a40006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.136+0000 7f4a4a7fc640 1 -- 172.21.15.45:0/2785142181 >> v1:172.21.15.45:6789/0 conn(0x7f4a400068f0 legacy=0x7f4a40006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:11.149 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:11.149 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 14, 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:11.150 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 18, 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.151 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.154 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:11.155 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:11.156 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.157 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.160 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.161 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.163 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:11.163 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:11.163 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.163 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.163 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.163 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:11.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:11.163 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.163 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:11.166 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:11.167 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.168 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.169 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:11.169 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.137+0000 7f4a3bfff640 1 -- 172.21.15.45:0/2785142181 >> v1:172.21.15.84:6812/31478 conn(0x7f4a3c03a5f0 legacy=0x7f4a3c03caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.137+0000 7f4a3bfff640 10 monclient: shutdown 2024-09-18T13:59:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.137+0000 7f4a3bfff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.137+0000 7f4a3bfff640 1 -- 172.21.15.45:0/2785142181 >> v1:172.21.15.45:6791/0 conn(0x7f4a5c0a0cb0 legacy=0x7f4a5c146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.137+0000 7f4a5ad9d640 1 -- 172.21.15.45:0/2785142181 reap_dead start 2024-09-18T13:59:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.137+0000 7f4a5ad9d640 1 -- 172.21.15.45:0/2785142181 reap_dead start 2024-09-18T13:59:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.137+0000 7f4a3bfff640 1 -- 172.21.15.45:0/2785142181 shutdown_connections 2024-09-18T13:59:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.137+0000 7f4a3bfff640 1 -- 172.21.15.45:0/2785142181 >> 172.21.15.45:0/2785142181 conn(0x7f4a5c0a5bb0 msgr2=0x7f4a5c09e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.137+0000 7f4a3bfff640 1 -- 172.21.15.45:0/2785142181 shutdown_connections 2024-09-18T13:59:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.137+0000 7f4a3bfff640 1 -- 172.21.15.45:0/2785142181 wait complete. 2024-09-18T13:59:11.171 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T13:59:11.171 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T13:59:11.171 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T13:59:11.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.206+0000 7fbc2c79d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:11.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.206+0000 7fbc2c79d640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.206+0000 7fbc2c79d640 10 monclient: monmap: 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:11.207618+0000 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:11.207618+0000 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.209 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.209 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.209 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.207+0000 7fbc2c79d640 1 Processor -- start 2024-09-18T13:59:11.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.207+0000 7fbc2c79d640 1 -- start start 2024-09-18T13:59:11.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.207+0000 7fbc2c79d640 10 monclient: init 2024-09-18T13:59:11.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.207+0000 7fbc2c79d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.207+0000 7fbc2c79d640 10 monclient: _add_conns ranks=[3,4,0,5,7,2,1,6,8] 2024-09-18T13:59:11.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.207+0000 7fbc2c79d640 10 monclient(hunting): picked mon.noname-g con 0x7fbc240a25c0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:11.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.207+0000 7fbc2c79d640 10 monclient(hunting): picked mon.noname-c con 0x7fbc2409ee00 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:11.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.207+0000 7fbc2c79d640 10 monclient(hunting): picked mon.noname-a con 0x7fbc2409f1d0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:11.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.207+0000 7fbc2c79d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2405ed70 con 0x7fbc2409f1d0 2024-09-18T13:59:11.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.207+0000 7fbc2c79d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc240a1a50 con 0x7fbc240a25c0 2024-09-18T13:59:11.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc2c79d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc240972b0 con 0x7fbc2409ee00 2024-09-18T13:59:11.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc2c79d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc2c79d640 10 monclient(hunting): authenticate will time out at 766.546022s 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29d38640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fbc2409ee00 0x7fbc24137710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:45986/0 (socket says 172.21.15.45:45986) 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29d38640 1 -- 172.21.15.45:0/4189580441 learned_addr learned my addr 172.21.15.45:0/4189580441 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc2ad3a640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbc2409f1d0 0x7fbc2413ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37012/0 (socket says 172.21.15.45:37012) 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316620469 0 0) 0x7fbc2405ed70 con 0x7fbc2409f1d0 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient(hunting): my global_id is 4455 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc100034e0 con 0x7fbc2409f1d0 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673470075 0 0) 0x7fbc240972b0 con 0x7fbc2409ee00 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient(hunting): my global_id is 4459 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc2405ed70 con 0x7fbc2409ee00 2024-09-18T13:59:11.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475179201 0 0) 0x7fbc240a1a50 con 0x7fbc240a25c0 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient(hunting): my global_id is 4494 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc240972b0 con 0x7fbc240a25c0 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1200017746 0 0) 0x7fbc100034e0 con 0x7fbc2409f1d0 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 >> v1:172.21.15.45:6791/0 conn(0x7fbc2409ee00 legacy=0x7fbc24137710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 >> v1:172.21.15.84:6790/0 conn(0x7fbc240a25c0 legacy=0x7fbc241323e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 monclient: found mon.noname-a 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc24133290 con 0x7fbc2409f1d0 2024-09-18T13:59:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc1c001bb0 con 0x7fbc2409f1d0 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient: dump: 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc2c79d640 5 monclient: authenticate success, global_id 4455 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc2c79d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbc1c001d90 con 0x7fbc2409f1d0 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 1 -- 172.21.15.45:0/4189580441 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc1c004e60 con 0x7fbc2409f1d0 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc2c79d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc2c79d640 4 set_mon_vals no callback set 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.208+0000 7fbc29537640 10 monclient: dump: 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.209+0000 7fbc2c79d640 10 monclient: shutdown 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.209+0000 7fbc2c79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.209+0000 7fbc2c79d640 1 -- 172.21.15.45:0/4189580441 >> v1:172.21.15.45:6789/0 conn(0x7fbc2409f1d0 legacy=0x7fbc2413ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.209+0000 7fbc2c79d640 1 -- 172.21.15.45:0/4189580441 shutdown_connections 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.209+0000 7fbc2c79d640 1 -- 172.21.15.45:0/4189580441 >> 172.21.15.45:0/4189580441 conn(0x7fbc240698b0 msgr2=0x7fbc24063030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.209+0000 7fbc2c79d640 1 -- 172.21.15.45:0/4189580441 shutdown_connections 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.209+0000 7fbc2c79d640 1 -- 172.21.15.45:0/4189580441 wait complete. 2024-09-18T13:59:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.209+0000 7fbc2c79d640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.209+0000 7fbc2c79d640 10 monclient: monmap: 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 1 Processor -- start 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 1 -- start start 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 10 monclient: init 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 10 monclient: _add_conns ranks=[3,4,2,6,0,1,5,7,8] 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 10 monclient(hunting): picked mon.noname-g con 0x7fbc2409ee00 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 10 monclient(hunting): picked mon.noname-c con 0x7fbc2409f1d0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 10 monclient(hunting): picked mon.noname-b con 0x7fbc240a25c0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2409f910 con 0x7fbc240a25c0 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc240a2190 con 0x7fbc2409ee00 2024-09-18T13:59:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc24141e30 con 0x7fbc2409f1d0 2024-09-18T13:59:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc2c79d640 10 monclient(hunting): authenticate will time out at 766.548849s 2024-09-18T13:59:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc29d38640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fbc2409f1d0 0x7fbc241316d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:46002/0 (socket says 172.21.15.45:46002) 2024-09-18T13:59:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.210+0000 7fbc29d38640 1 -- 172.21.15.45:0/1920127930 learned_addr learned my addr 172.21.15.45:0/1920127930 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931009821 0 0) 0x7fbc2409f910 con 0x7fbc240a25c0 2024-09-18T13:59:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): my global_id is 4439 2024-09-18T13:59:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc080034e0 con 0x7fbc240a25c0 2024-09-18T13:59:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575324085 0 0) 0x7fbc240a2190 con 0x7fbc2409ee00 2024-09-18T13:59:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): my global_id is 4503 2024-09-18T13:59:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc2409f910 con 0x7fbc2409ee00 2024-09-18T13:59:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584438290 0 0) 0x7fbc24141e30 con 0x7fbc2409f1d0 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): my global_id is 4468 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc240a2190 con 0x7fbc2409f1d0 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1824380742 0 0) 0x7fbc2409f910 con 0x7fbc2409ee00 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbc24141e30 con 0x7fbc2409ee00 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc200018c0 con 0x7fbc2409ee00 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1528301548 0 0) 0x7fbc240a2190 con 0x7fbc2409f1d0 2024-09-18T13:59:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbc2409f910 con 0x7fbc2409f1d0 2024-09-18T13:59:11.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 393599080 0 0) 0x7fbc080034e0 con 0x7fbc240a25c0 2024-09-18T13:59:11.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbc240a2190 con 0x7fbc240a25c0 2024-09-18T13:59:11.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc14001a20 con 0x7fbc2409f1d0 2024-09-18T13:59:11.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:11.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:11.228 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc1c004e60 con 0x7fbc240a25c0 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 728459179 0 0) 0x7fbc24141e30 con 0x7fbc2409ee00 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 >> v1:172.21.15.45:6791/0 conn(0x7fbc2409f1d0 legacy=0x7fbc241316d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 >> v1:172.21.15.45:6790/0 conn(0x7fbc240a25c0 legacy=0x7fbc24149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 monclient: found mon.g 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc24142080 con 0x7fbc2409ee00 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc1a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc2c79d640 5 monclient: authenticate success, global_id 4503 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc2c79d640 10 monclient: _renew_subs 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc2c79d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc2c79d640 1 -- 172.21.15.45:0/1920127930 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc24121820 con 0x7fbc2409ee00 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc2c79d640 10 monclient: _renew_subs 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc2c79d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.211+0000 7fbc2c79d640 1 -- 172.21.15.45:0/1920127930 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc24131de0 con 0x7fbc2409ee00 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.212+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbc20002da0 con 0x7fbc2409ee00 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.212+0000 7fbc1a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.212+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc20001de0 con 0x7fbc2409ee00 2024-09-18T13:59:11.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.212+0000 7fbc1a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.212+0000 7fbc28d36640 4 set_mon_vals no callback set 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.212+0000 7fbc1a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.212+0000 7fbc1a7fc640 10 monclient: dump: 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.233 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.234 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.212+0000 7fbc2c79d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.212+0000 7fbc2c79d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.212+0000 7fbc2c79d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.212+0000 7fbc2c79d640 1 -- 172.21.15.45:0/1920127930 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0c005010 con 0x7fbc2409ee00 2024-09-18T13:59:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.212+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fbc20001f40 con 0x7fbc2409ee00 2024-09-18T13:59:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.213+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 3882+0+0 (unknown 594623498 0 0) 0x7fbc20043fe0 con 0x7fbc2409ee00 2024-09-18T13:59:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.213+0000 7fbc29d38640 10 monclient: get_auth_request con 0x7fbc0803ac70 auth_method 0 2024-09-18T13:59:11.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.217+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.3 v1:172.21.15.84:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fbc20002690 con 0x7fbc2409ee00 2024-09-18T13:59:11.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.217+0000 7fbc1a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.217+0000 7fbc1a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.304+0000 7fce5bd9d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.304+0000 7fce5bd9d640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.304+0000 7fce5bd9d640 10 monclient: monmap: 2024-09-18T13:59:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:11.305241+0000 2024-09-18T13:59:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:11.305241+0000 2024-09-18T13:59:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.304+0000 7fce5bd9d640 1 Processor -- start 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce5bd9d640 1 -- start start 2024-09-18T13:59:11.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce5bd9d640 10 monclient: init 2024-09-18T13:59:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce5bd9d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce5bd9d640 10 monclient: _add_conns ranks=[8,6,3,2,1,7,0,5,4] 2024-09-18T13:59:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce5bd9d640 10 monclient(hunting): picked mon.noname-e con 0x7fce5406e810 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce5bd9d640 10 monclient(hunting): picked mon.noname-d con 0x7fce540a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce5bd9d640 10 monclient(hunting): picked mon.noname-g con 0x7fce540a1080 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce5bd9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5405ed70 con 0x7fce540a1080 2024-09-18T13:59:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce5bd9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce54097620 con 0x7fce540a0cb0 2024-09-18T13:59:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce5bd9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce54097340 con 0x7fce5406e810 2024-09-18T13:59:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce5bd9d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce5bd9d640 10 monclient(hunting): authenticate will time out at 766.643637s 2024-09-18T13:59:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce59b39640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fce5406e810 0x7fce540a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51918/0 (socket says 172.21.15.45:51918) 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce59b39640 1 -- 172.21.15.45:0/3972981479 learned_addr learned my addr 172.21.15.45:0/3972981479 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.305+0000 7fce59338640 1 --1- 172.21.15.45:0/3972981479 >> v1:172.21.15.45:6792/0 conn(0x7fce540a0cb0 0x7fce54137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34806/0 (socket says 172.21.15.45:34806) 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2991544435 0 0) 0x7fce5405ed70 con 0x7fce540a1080 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient(hunting): my global_id is 4512 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce400034e0 con 0x7fce540a1080 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994705765 0 0) 0x7fce54097340 con 0x7fce5406e810 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient(hunting): my global_id is 4382 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce5405ed70 con 0x7fce5406e810 2024-09-18T13:59:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483177777 0 0) 0x7fce54097620 con 0x7fce540a0cb0 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient(hunting): my global_id is 4407 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce54097340 con 0x7fce540a0cb0 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 483590616 0 0) 0x7fce400034e0 con 0x7fce540a1080 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 >> v1:172.21.15.45:6793/0 conn(0x7fce5406e810 legacy=0x7fce540a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 >> v1:172.21.15.45:6792/0 conn(0x7fce540a0cb0 legacy=0x7fce54137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 monclient: found mon.noname-g 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce540a1660 con 0x7fce540a1080 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce5bd9d640 5 monclient: authenticate success, global_id 4512 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce4c001bb0 con 0x7fce540a1080 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient: dump: 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce5bd9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce4c001d90 con 0x7fce540a1080 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 1 -- 172.21.15.45:0/3972981479 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce4c0025c0 con 0x7fce540a1080 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce58b37640 10 monclient: dump: 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce5bd9d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.306+0000 7fce5bd9d640 4 set_mon_vals no callback set 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.307+0000 7fce5bd9d640 10 monclient: shutdown 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.307+0000 7fce5bd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.307+0000 7fce5bd9d640 1 -- 172.21.15.45:0/3972981479 >> v1:172.21.15.84:6790/0 conn(0x7fce540a1080 legacy=0x7fce5413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.307+0000 7fce5bd9d640 1 -- 172.21.15.45:0/3972981479 shutdown_connections 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.307+0000 7fce5bd9d640 1 -- 172.21.15.45:0/3972981479 >> 172.21.15.45:0/3972981479 conn(0x7fce540a5bb0 msgr2=0x7fce540a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.307+0000 7fce5bd9d640 1 -- 172.21.15.45:0/3972981479 shutdown_connections 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.307+0000 7fce5bd9d640 1 -- 172.21.15.45:0/3972981479 wait complete. 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.308+0000 7fce5bd9d640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.308+0000 7fce5bd9d640 10 monclient: monmap: 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.308+0000 7fce5bd9d640 1 Processor -- start 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.308+0000 7fce5bd9d640 1 -- start start 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.308+0000 7fce5bd9d640 10 monclient: init 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce5bd9d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce5bd9d640 10 monclient: _add_conns ranks=[3,5,2,6,0,4,8,1,7] 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce5bd9d640 10 monclient(hunting): picked mon.noname-g con 0x7fce5406e810 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce5bd9d640 10 monclient(hunting): picked mon.noname-h con 0x7fce540a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce5bd9d640 10 monclient(hunting): picked mon.noname-b con 0x7fce540a1080 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce5bd9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce54122eb0 con 0x7fce540a1080 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce5bd9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5413dca0 con 0x7fce5406e810 2024-09-18T13:59:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce5bd9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce541228f0 con 0x7fce540a0cb0 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce5bd9d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce5bd9d640 10 monclient(hunting): authenticate will time out at 766.647482s 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce5a33a640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fce540a1080 0x7fce54149ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51632/0 (socket says 172.21.15.45:51632) 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce5a33a640 1 -- 172.21.15.45:0/960850274 learned_addr learned my addr 172.21.15.45:0/960850274 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596616291 0 0) 0x7fce54122eb0 con 0x7fce540a1080 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce49ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce49ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce49ffb640 10 monclient(hunting): my global_id is 4448 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.309+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce380034e0 con 0x7fce540a1080 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376347035 0 0) 0x7fce541228f0 con 0x7fce540a0cb0 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient(hunting): my global_id is 4397 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce54122eb0 con 0x7fce540a0cb0 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315036119 0 0) 0x7fce5413dca0 con 0x7fce5406e810 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient(hunting): my global_id is 4521 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce541228f0 con 0x7fce5406e810 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2126463116 0 0) 0x7fce380034e0 con 0x7fce540a1080 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce5413dca0 con 0x7fce540a1080 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce4c002240 con 0x7fce540a1080 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient(hunting): dump: 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.325 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 281459576 0 0) 0x7fce5413dca0 con 0x7fce540a1080 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 >> v1:172.21.15.84:6791/0 conn(0x7fce540a0cb0 legacy=0x7fce541467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 >> v1:172.21.15.84:6790/0 conn(0x7fce5406e810 legacy=0x7fce5413d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 monclient: found mon.b 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce54120400 con 0x7fce540a1080 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce5bd9d640 5 monclient: authenticate success, global_id 4448 2024-09-18T13:59:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce5bd9d640 10 monclient: _renew_subs 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce5bd9d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce5bd9d640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce54122610 con 0x7fce540a1080 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce4c002690 con 0x7fce540a1080 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce4c002c00 con 0x7fce540a1080 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce49ffb640 10 monclient: dump: 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce5bd9d640 10 monclient: _renew_subs 2024-09-18T13:59:11.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce4bfff640 4 set_mon_vals no callback set 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce5bd9d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.310+0000 7fce5bd9d640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fce54121740 con 0x7fce540a1080 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.311+0000 7fce5bd9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.311+0000 7fce5bd9d640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fce5404c4a0 con 0x7fce5413e840 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.311+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fce4c0105d0 con 0x7fce540a1080 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.311+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 3882+0+0 (unknown 594623498 0 0) 0x7fce4c046f10 con 0x7fce540a1080 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.311+0000 7fce59338640 10 monclient: get_auth_request con 0x7fce38039f40 auth_method 0 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.311+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459710542 0 0) 0x7fce5404c4a0 con 0x7fce5413e840 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.311+0000 7fce49ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.311+0000 7fce49ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.311+0000 7fce49ffb640 10 monclient(hunting): my global_id is 4457 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.311+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce5413dca0 con 0x7fce5413e840 2024-09-18T13:59:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2939911359 0 0) 0x7fce5413dca0 con 0x7fce5413e840 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce5404c4a0 con 0x7fce5413e840 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce50004ae0 con 0x7fce5413e840 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 10 monclient: dump: 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 66676319 0 0) 0x7fce5404c4a0 con 0x7fce5413e840 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce5404b720 con 0x7fce5413e840 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fce50004f00 con 0x7fce5413e840 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 >> v1:172.21.15.45:6790/0 conn(0x7fce5413e840 legacy=0x7fce5409bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.312+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 >> v1:172.21.15.45:6790/0 conn(0x7fce5413e840 legacy=0x7fce5409bcb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce5bd9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce5bd9d640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fce5413e290 con 0x7fce5413dec0 2024-09-18T13:59:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395564254 0 0) 0x7fce5413e290 con 0x7fce5413dec0 2024-09-18T13:59:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 10 monclient(hunting): my global_id is 4466 2024-09-18T13:59:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce5404c4a0 con 0x7fce5413dec0 2024-09-18T13:59:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2068938812 0 0) 0x7fce5404c4a0 con 0x7fce5413dec0 2024-09-18T13:59:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce5413e290 con 0x7fce5413dec0 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce50005780 con 0x7fce5413dec0 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 10 monclient: dump: 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2630173446 0 0) 0x7fce5413e290 con 0x7fce5413dec0 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.318+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fce54062dd0 con 0x7fce5413dec0 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7559 (unknown 0 0 2490841491) 0x7fce50006a50 con 0x7fce5413dec0 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce49ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce49ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 >> v1:172.21.15.45:6790/0 conn(0x7fce5413dec0 legacy=0x7fce5409c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce49ffb640 1 -- 172.21.15.45:0/960850274 >> v1:172.21.15.45:6790/0 conn(0x7fce5413dec0 legacy=0x7fce5409c3c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:11.336 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:11.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:11.336 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:11.336 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T13:59:11.336 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 14, 2024-09-18T13:59:11.336 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:11.336 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:11.336 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:11.336 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 19, 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.337 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.338 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:11.339 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.340 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:11.341 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.342 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.345 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:11.346 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.347 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.348 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:11.350 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.351 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.351 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.351 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:11.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:11.351 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.351 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.351 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:11.352 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.352 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.352 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.352 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:11.352 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:11.352 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.352 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.352 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.352 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.353 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.354 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.356 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:11.356 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:11.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce5bd9d640 1 -- 172.21.15.45:0/960850274 >> v1:172.21.15.84:6812/31478 conn(0x7fce38039f40 legacy=0x7fce3803c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce5bd9d640 10 monclient: shutdown 2024-09-18T13:59:11.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce5bd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce5bd9d640 1 -- 172.21.15.45:0/960850274 >> v1:172.21.15.45:6790/0 conn(0x7fce540a1080 legacy=0x7fce54149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce5a33a640 1 -- 172.21.15.45:0/960850274 reap_dead start 2024-09-18T13:59:11.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce5a33a640 1 -- 172.21.15.45:0/960850274 reap_dead start 2024-09-18T13:59:11.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce5bd9d640 1 -- 172.21.15.45:0/960850274 shutdown_connections 2024-09-18T13:59:11.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce5bd9d640 1 -- 172.21.15.45:0/960850274 >> 172.21.15.45:0/960850274 conn(0x7fce540a5bb0 msgr2=0x7fce540a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce5bd9d640 1 -- 172.21.15.45:0/960850274 shutdown_connections 2024-09-18T13:59:11.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.319+0000 7fce5bd9d640 1 -- 172.21.15.45:0/960850274 wait complete. 2024-09-18T13:59:11.358 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T13:59:11.358 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T13:59:11.358 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T13:59:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.389+0000 7fbc2c79d640 10 start_mon_command cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2024-09-18T13:59:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.389+0000 7fbc2c79d640 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2024-09-18T13:59:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.389+0000 7fbc2c79d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.389+0000 7fbc2c79d640 1 -- 172.21.15.45:0/1920127930 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbc0c0052c0 con 0x7fbc2409ee00 2024-09-18T13:59:11.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.485+0000 7fe26d68b640 10 monclient: get_monmap_and_config 2024-09-18T13:59:11.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.485+0000 7fe26d68b640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.485+0000 7fe26d68b640 10 monclient: monmap: 2024-09-18T13:59:11.486 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.486 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.486 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:11.486858+0000 2024-09-18T13:59:11.486 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:11.486858+0000 2024-09-18T13:59:11.487 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.487 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.487 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.487 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.487 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.487 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.487 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.487 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.487 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.487 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.487 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.487 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.486+0000 7fe26d68b640 1 Processor -- start 2024-09-18T13:59:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.486+0000 7fe26d68b640 1 -- start start 2024-09-18T13:59:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.486+0000 7fe26d68b640 10 monclient: init 2024-09-18T13:59:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.486+0000 7fe26d68b640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.486+0000 7fe26d68b640 10 monclient: _add_conns ranks=[7,0,8,3,1,4,6,5,2] 2024-09-18T13:59:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 10 monclient(hunting): picked mon.noname-i con 0x7fe26809f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 10 monclient(hunting): picked mon.noname-a con 0x7fe26809fba0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 10 monclient(hunting): picked mon.noname-e con 0x7fe26809ff70 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26805ed70 con 0x7fe26809fba0 2024-09-18T13:59:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26809dfc0 con 0x7fe26809f7d0 2024-09-18T13:59:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26809dce0 con 0x7fe26809ff70 2024-09-18T13:59:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 10 monclient(hunting): authenticate will time out at 766.825105s 2024-09-18T13:59:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe2667fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe26809fba0 0x7fe268137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37018/0 (socket says 172.21.15.45:37018) 2024-09-18T13:59:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe2667fc640 1 -- 172.21.15.45:0/168932873 learned_addr learned my addr 172.21.15.45:0/168932873 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897737871 0 0) 0x7fe26809dce0 con 0x7fe26809ff70 2024-09-18T13:59:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient(hunting): my global_id is 4391 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2500034e0 con 0x7fe26809ff70 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1064706936 0 0) 0x7fe26805ed70 con 0x7fe26809fba0 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient(hunting): my global_id is 4464 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe26809dce0 con 0x7fe26809fba0 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940363585 0 0) 0x7fe26809dfc0 con 0x7fe26809f7d0 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient(hunting): my global_id is 4462 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe26805ed70 con 0x7fe26809f7d0 2024-09-18T13:59:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2914897611 0 0) 0x7fe26809dce0 con 0x7fe26809fba0 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 >> v1:172.21.15.45:6793/0 conn(0x7fe26809ff70 legacy=0x7fe26813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 >> v1:172.21.15.84:6792/0 conn(0x7fe26809f7d0 legacy=0x7fe268132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 monclient: found mon.noname-a 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2680a1c00 con 0x7fe26809fba0 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 5 monclient: authenticate success, global_id 4464 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe25c001810 con 0x7fe26809fba0 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient: dump: 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe25c0019f0 con 0x7fe26809fba0 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 1 -- 172.21.15.45:0/168932873 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe25c004ba0 con 0x7fe26809fba0 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe26d68b640 4 set_mon_vals no callback set 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.487+0000 7fe265ffb640 10 monclient: dump: 2024-09-18T13:59:11.500 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.488+0000 7fe26d68b640 10 monclient: shutdown 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.488+0000 7fe26d68b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.488+0000 7fe26d68b640 1 -- 172.21.15.45:0/168932873 >> v1:172.21.15.45:6789/0 conn(0x7fe26809fba0 legacy=0x7fe268137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.488+0000 7fe26d68b640 1 -- 172.21.15.45:0/168932873 shutdown_connections 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.488+0000 7fe26d68b640 1 -- 172.21.15.45:0/168932873 >> 172.21.15.45:0/168932873 conn(0x7fe2680630b0 msgr2=0x7fe2680634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.488+0000 7fe26d68b640 1 -- 172.21.15.45:0/168932873 shutdown_connections 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.488+0000 7fe26d68b640 1 -- 172.21.15.45:0/168932873 wait complete. 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 10 monclient: monmap: 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 1 Processor -- start 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 1 -- start start 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 10 monclient: init 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 10 monclient: _add_conns ranks=[7,3,1,4,0,6,5,2,8] 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 10 monclient(hunting): picked mon.noname-i con 0x7fe26809f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 10 monclient(hunting): picked mon.noname-g con 0x7fe26809fba0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 10 monclient(hunting): picked mon.noname-f con 0x7fe26809ff70 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26809d310 con 0x7fe26809ff70 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2680a2560 con 0x7fe26809fba0 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2681220e0 con 0x7fe26809f7d0 2024-09-18T13:59:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.489+0000 7fe26d68b640 10 monclient(hunting): authenticate will time out at 766.827939s 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe266ffd640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7fe26809f7d0 0x7fe2681319d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:58348/0 (socket says 172.21.15.45:58348) 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe266ffd640 1 -- 172.21.15.45:0/1124301316 learned_addr learned my addr 172.21.15.45:0/1124301316 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe2667fc640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7fe26809fba0 0x7fe2681417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:54720/0 (socket says 172.21.15.45:54720) 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522186234 0 0) 0x7fe2681220e0 con 0x7fe26809f7d0 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): my global_id is 4471 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2540034e0 con 0x7fe26809f7d0 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795821416 0 0) 0x7fe26809d310 con 0x7fe26809ff70 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): my global_id is 4438 2024-09-18T13:59:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2681220e0 con 0x7fe26809ff70 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624263906 0 0) 0x7fe2680a2560 con 0x7fe26809fba0 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): my global_id is 4530 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe26809d310 con 0x7fe26809fba0 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3500355949 0 0) 0x7fe2540034e0 con 0x7fe26809f7d0 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe2680a2560 con 0x7fe26809f7d0 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe258002bf0 con 0x7fe26809f7d0 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): dump: 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.507 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1527542644 0 0) 0x7fe2681220e0 con 0x7fe26809ff70 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe2540034e0 con 0x7fe26809ff70 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3232199245 0 0) 0x7fe26809d310 con 0x7fe26809fba0 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe2681220e0 con 0x7fe26809fba0 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe260001b90 con 0x7fe26809ff70 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): dump: 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.509 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe25c001750 con 0x7fe26809fba0 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient(hunting): dump: 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1161687327 0 0) 0x7fe2680a2560 con 0x7fe26809f7d0 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 >> v1:172.21.15.84:6790/0 conn(0x7fe26809fba0 legacy=0x7fe2681417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 >> v1:172.21.15.84:6789/0 conn(0x7fe26809ff70 legacy=0x7fe268149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 monclient: found mon.i 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe268120710 con 0x7fe26809f7d0 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe24effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe26d68b640 5 monclient: authenticate success, global_id 4471 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe26d68b640 10 monclient: _renew_subs 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe26d68b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.490+0000 7fe26d68b640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2681320e0 con 0x7fe26809f7d0 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe26d68b640 10 monclient: _renew_subs 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe26d68b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe26d68b640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe268142050 con 0x7fe26809f7d0 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe258001760 con 0x7fe26809f7d0 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe24effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe258001c50 con 0x7fe26809f7d0 2024-09-18T13:59:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe24effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe24effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe2657fa640 4 set_mon_vals no callback set 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe24effd640 10 monclient: dump: 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe26d68b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe26d68b640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe248006580 con 0x7fe248002830 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.491+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fe258001e30 con 0x7fe26809f7d0 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 3882+0+0 (unknown 594623498 0 0) 0x7fe258046d30 con 0x7fe26809f7d0 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714282915 0 0) 0x7fe248006580 con 0x7fe248002830 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 10 monclient(hunting): my global_id is 4477 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2680a2560 con 0x7fe248002830 2024-09-18T13:59:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe2677fe640 10 monclient: get_auth_request con 0x7fe25403aa30 auth_method 0 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3253745661 0 0) 0x7fe2680a2560 con 0x7fe248002830 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe248006580 con 0x7fe248002830 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe25c004af0 con 0x7fe248002830 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 10 monclient: dump: 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.516 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 89998641 0 0) 0x7fe248006580 con 0x7fe248002830 2024-09-18T13:59:11.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe24800a0a0 con 0x7fe248002830 2024-09-18T13:59:11.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.492+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe25c0051a0 con 0x7fe248002830 2024-09-18T13:59:11.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.493+0000 7fe24effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.493+0000 7fe24effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:11.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.493+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 >> v1:172.21.15.45:6791/0 conn(0x7fe248002830 legacy=0x7fe248004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.493+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 >> v1:172.21.15.45:6791/0 conn(0x7fe248002830 legacy=0x7fe248004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:11.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.498+0000 7fe26d68b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:11.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.498+0000 7fe26d68b640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe248009660 con 0x7fe248009040 2024-09-18T13:59:11.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969722598 0 0) 0x7fe248009660 con 0x7fe248009040 2024-09-18T13:59:11.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 10 monclient(hunting): my global_id is 4486 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe248006580 con 0x7fe248009040 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1258028660 0 0) 0x7fe248006580 con 0x7fe248009040 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe248009660 con 0x7fe248009040 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe25c005a20 con 0x7fe248009040 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 10 monclient: dump: 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.520 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 100115030 0 0) 0x7fe248009660 con 0x7fe248009040 2024-09-18T13:59:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe248009920 con 0x7fe248009040 2024-09-18T13:59:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 1763265417) 0x7fe25c008190 con 0x7fe248009040 2024-09-18T13:59:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 >> v1:172.21.15.45:6791/0 conn(0x7fe248009040 legacy=0x7fe248008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.499+0000 7fe24effd640 1 -- 172.21.15.45:0/1124301316 >> v1:172.21.15.45:6791/0 conn(0x7fe248009040 legacy=0x7fe248008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:11.521 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:11.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:11.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:11.522 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:11.522 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 14, 2024-09-18T13:59:11.522 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:11.522 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:11.522 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:11.522 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:11.522 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:11.522 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T13:59:11.522 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:11.522 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:11.522 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:11.522 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 19, 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.523 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.524 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.525 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:11.525 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:11.525 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:11.525 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:11.525 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:11.525 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:11.525 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:11.525 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:11.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:11.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:11.525 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.526 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:11.527 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.528 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:11.531 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:11.532 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:11.534 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.535 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.536 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:11.538 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.539 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:11.540 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.541 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.541 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:11.541 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:11.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.500+0000 7fe26d68b640 1 -- 172.21.15.45:0/1124301316 >> v1:172.21.15.84:6812/31478 conn(0x7fe25403aa30 legacy=0x7fe25403cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.500+0000 7fe26d68b640 10 monclient: shutdown 2024-09-18T13:59:11.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.500+0000 7fe26d68b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.500+0000 7fe26d68b640 1 -- 172.21.15.45:0/1124301316 >> v1:172.21.15.84:6792/0 conn(0x7fe26809f7d0 legacy=0x7fe2681319d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.500+0000 7fe2677fe640 1 -- 172.21.15.45:0/1124301316 reap_dead start 2024-09-18T13:59:11.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.500+0000 7fe2677fe640 1 -- 172.21.15.45:0/1124301316 reap_dead start 2024-09-18T13:59:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.500+0000 7fe26d68b640 1 -- 172.21.15.45:0/1124301316 shutdown_connections 2024-09-18T13:59:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.500+0000 7fe26d68b640 1 -- 172.21.15.45:0/1124301316 >> 172.21.15.45:0/1124301316 conn(0x7fe2680630b0 msgr2=0x7fe2680a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.500+0000 7fe26d68b640 1 -- 172.21.15.45:0/1124301316 shutdown_connections 2024-09-18T13:59:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.500+0000 7fe26d68b640 1 -- 172.21.15.45:0/1124301316 wait complete. 2024-09-18T13:59:11.543 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T13:59:11.543 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T13:59:11.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T13:59:11.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.640+0000 7faa7eb9d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:11.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.640+0000 7faa7eb9d640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.640+0000 7faa7eb9d640 10 monclient: monmap: 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:11.641097+0000 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:11.641097+0000 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.642 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.642 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.640+0000 7faa7eb9d640 1 Processor -- start 2024-09-18T13:59:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.640+0000 7faa7eb9d640 1 -- start start 2024-09-18T13:59:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.640+0000 7faa7eb9d640 10 monclient: init 2024-09-18T13:59:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa7eb9d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa7eb9d640 10 monclient: _add_conns ranks=[6,8,7,0,3,5,4,1,2] 2024-09-18T13:59:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa7eb9d640 10 monclient(hunting): picked mon.noname-d con 0x7faa7806e810 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa7eb9d640 10 monclient(hunting): picked mon.noname-e con 0x7faa780a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa7eb9d640 10 monclient(hunting): picked mon.noname-i con 0x7faa780a1080 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa7eb9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7805ed70 con 0x7faa7806e810 2024-09-18T13:59:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa7eb9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa78097620 con 0x7faa780a1080 2024-09-18T13:59:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa7eb9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa78097340 con 0x7faa780a0cb0 2024-09-18T13:59:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa7eb9d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa7eb9d640 10 monclient(hunting): authenticate will time out at 766.979462s 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa7c939640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7faa7806e810 0x7faa780a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34812/0 (socket says 172.21.15.45:34812) 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa7c939640 1 -- 172.21.15.45:0/347569812 learned_addr learned my addr 172.21.15.45:0/347569812 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa777fe640 1 -- 172.21.15.45:0/347569812 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341100136 0 0) 0x7faa7805ed70 con 0x7faa7806e810 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa777fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa777fe640 10 monclient(hunting): my global_id is 4416 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa777fe640 1 -- 172.21.15.45:0/347569812 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa680034e0 con 0x7faa7806e810 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa777fe640 1 -- 172.21.15.45:0/347569812 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661616121 0 0) 0x7faa78097340 con 0x7faa780a0cb0 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa777fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa777fe640 10 monclient(hunting): my global_id is 4400 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.641+0000 7faa777fe640 1 -- 172.21.15.45:0/347569812 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa7805ed70 con 0x7faa780a0cb0 2024-09-18T13:59:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 1 -- 172.21.15.45:0/347569812 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3651118994 0 0) 0x7faa680034e0 con 0x7faa7806e810 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 1 -- 172.21.15.45:0/347569812 >> v1:172.21.15.45:6793/0 conn(0x7faa780a0cb0 legacy=0x7faa78137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 1 -- 172.21.15.45:0/347569812 >> v1:172.21.15.84:6792/0 conn(0x7faa780a1080 legacy=0x7faa7813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 1 monclient: found mon.noname-d 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 1 -- 172.21.15.45:0/347569812 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa780a1660 con 0x7faa7806e810 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 1 -- 172.21.15.45:0/347569812 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faa6c001830 con 0x7faa7806e810 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa7eb9d640 5 monclient: authenticate success, global_id 4416 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa7eb9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 10 monclient: dump: 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa7eb9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 1 -- 172.21.15.45:0/347569812 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faa6c001990 con 0x7faa7806e810 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 1 -- 172.21.15.45:0/347569812 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faa6c004860 con 0x7faa7806e810 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa7eb9d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa7eb9d640 4 set_mon_vals no callback set 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa777fe640 10 monclient: dump: 2024-09-18T13:59:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa7eb9d640 10 monclient: shutdown 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa7eb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa7eb9d640 1 -- 172.21.15.45:0/347569812 >> v1:172.21.15.45:6792/0 conn(0x7faa7806e810 legacy=0x7faa780a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa7eb9d640 1 -- 172.21.15.45:0/347569812 shutdown_connections 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa7eb9d640 1 -- 172.21.15.45:0/347569812 >> 172.21.15.45:0/347569812 conn(0x7faa780a5bb0 msgr2=0x7faa780a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa7eb9d640 1 -- 172.21.15.45:0/347569812 shutdown_connections 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.642+0000 7faa7eb9d640 1 -- 172.21.15.45:0/347569812 wait complete. 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.643+0000 7faa7eb9d640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.643+0000 7faa7eb9d640 10 monclient: monmap: 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.643+0000 7faa7eb9d640 1 Processor -- start 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.643+0000 7faa7eb9d640 1 -- start start 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.643+0000 7faa7eb9d640 10 monclient: init 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa7eb9d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa7eb9d640 10 monclient: _add_conns ranks=[3,5,8,7,0,2,6,1,4] 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa7eb9d640 10 monclient(hunting): picked mon.noname-g con 0x7faa7806e810 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa7eb9d640 10 monclient(hunting): picked mon.noname-h con 0x7faa780a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa7eb9d640 10 monclient(hunting): picked mon.noname-e con 0x7faa780a1080 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa7eb9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7811f2e0 con 0x7faa7806e810 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa7eb9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7809fa10 con 0x7faa780a0cb0 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa7eb9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa780a2270 con 0x7faa780a1080 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa7eb9d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa7eb9d640 10 monclient(hunting): authenticate will time out at 766.982201s 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa7d13a640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7faa780a1080 0x7faa78149920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51944/0 (socket says 172.21.15.45:51944) 2024-09-18T13:59:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa7d13a640 1 -- 172.21.15.45:0/3252817452 learned_addr learned my addr 172.21.15.45:0/3252817452 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473580607 0 0) 0x7faa780a2270 con 0x7faa780a1080 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): my global_id is 4409 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa5c0034e0 con 0x7faa780a1080 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087575182 0 0) 0x7faa7811f2e0 con 0x7faa7806e810 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): my global_id is 4539 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa780a2270 con 0x7faa7806e810 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936156095 0 0) 0x7faa7809fa10 con 0x7faa780a0cb0 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): my global_id is 4406 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa7811f2e0 con 0x7faa780a0cb0 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3300325586 0 0) 0x7faa5c0034e0 con 0x7faa780a1080 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faa7809fa10 con 0x7faa780a1080 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faa70001d40 con 0x7faa780a1080 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): dump: 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3147253117 0 0) 0x7faa7809fa10 con 0x7faa780a1080 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 >> v1:172.21.15.84:6791/0 conn(0x7faa780a0cb0 legacy=0x7faa7811dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 >> v1:172.21.15.84:6790/0 conn(0x7faa7806e810 legacy=0x7faa7811d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 monclient: found mon.e 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa78120820 con 0x7faa780a1080 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.644+0000 7faa74ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa7eb9d640 5 monclient: authenticate success, global_id 4409 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa7eb9d640 10 monclient: _renew_subs 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa7eb9d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa7eb9d640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7811ed60 con 0x7faa780a1080 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa7eb9d640 10 monclient: _renew_subs 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa7eb9d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa7eb9d640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7811f930 con 0x7faa780a1080 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faa70002380 con 0x7faa780a1080 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa74ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faa700028f0 con 0x7faa780a1080 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa74ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa74ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa74ff9640 10 monclient: dump: 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa76ffd640 4 set_mon_vals no callback set 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7faa70002ad0 con 0x7faa780a1080 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.645+0000 7faa7eb9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 3882+0+0 (unknown 594623498 0 0) 0x7faa70046d40 con 0x7faa780a1080 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa7eb9d640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7faa7804c4a0 con 0x7faa7811e310 2024-09-18T13:59:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa7c939640 10 monclient: get_auth_request con 0x7faa5c039f30 auth_method 0 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882265778 0 0) 0x7faa7804c4a0 con 0x7faa7811e310 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 10 monclient(hunting): my global_id is 4425 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa7809fa10 con 0x7faa7811e310 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1576179540 0 0) 0x7faa7809fa10 con 0x7faa7811e310 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faa7804c4a0 con 0x7faa7811e310 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faa64001840 con 0x7faa7811e310 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 10 monclient: dump: 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 368759338 0 0) 0x7faa7804c4a0 con 0x7faa7811e310 2024-09-18T13:59:11.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa7804b720 con 0x7faa7811e310 2024-09-18T13:59:11.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7faa64003b50 con 0x7faa7811e310 2024-09-18T13:59:11.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:11.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 >> v1:172.21.15.45:6792/0 conn(0x7faa7811e310 legacy=0x7faa7809b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.646+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 >> v1:172.21.15.45:6792/0 conn(0x7faa7811e310 legacy=0x7faa7809b590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.652+0000 7faa7eb9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.652+0000 7faa7eb9d640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7faa78062dd0 con 0x7faa7802a8d0 2024-09-18T13:59:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.652+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016033196 0 0) 0x7faa78062dd0 con 0x7faa7802a8d0 2024-09-18T13:59:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.652+0000 7faa74ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.652+0000 7faa74ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.652+0000 7faa74ff9640 10 monclient(hunting): my global_id is 4434 2024-09-18T13:59:11.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.652+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa7804c4a0 con 0x7faa7802a8d0 2024-09-18T13:59:11.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2991645715 0 0) 0x7faa7804c4a0 con 0x7faa7802a8d0 2024-09-18T13:59:11.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faa78062dd0 con 0x7faa7802a8d0 2024-09-18T13:59:11.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faa640022c0 con 0x7faa7802a8d0 2024-09-18T13:59:11.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:11.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 10 monclient: dump: 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.669 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.670 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.670 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1433432694 0 0) 0x7faa78062dd0 con 0x7faa7802a8d0 2024-09-18T13:59:11.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7faa780670e0 con 0x7faa7802a8d0 2024-09-18T13:59:11.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 3701911018) 0x7faa64003730 con 0x7faa7802a8d0 2024-09-18T13:59:11.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:11.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:11.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 >> v1:172.21.15.45:6792/0 conn(0x7faa7802a8d0 legacy=0x7faa7809bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa74ff9640 1 -- 172.21.15.45:0/3252817452 >> v1:172.21.15.45:6792/0 conn(0x7faa7802a8d0 legacy=0x7faa7809bca0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:11.670 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 14, 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:11.671 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:11.672 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.672 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 19, 2024-09-18T13:59:11.672 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:11.672 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:11.672 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:11.672 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.672 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.672 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.672 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.672 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.672 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.672 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.673 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:11.674 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.675 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:11.676 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.677 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:11.677 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:11.677 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.677 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.677 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.677 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:11.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:11.678 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:11.678 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.678 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.678 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:11.678 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:11.678 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.679 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:11.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.682 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:11.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.686 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.687 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:11.688 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.689 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.689 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:11.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.689 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:11.689 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:11.689 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:11.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.689 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.689 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:11.689 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:11.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa7eb9d640 1 -- 172.21.15.45:0/3252817452 >> v1:172.21.15.84:6812/31478 conn(0x7faa5c039f30 legacy=0x7faa5c03c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa7eb9d640 10 monclient: shutdown 2024-09-18T13:59:11.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa7eb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa7eb9d640 1 -- 172.21.15.45:0/3252817452 >> v1:172.21.15.45:6793/0 conn(0x7faa780a1080 legacy=0x7faa78149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa7d13a640 1 -- 172.21.15.45:0/3252817452 reap_dead start 2024-09-18T13:59:11.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.653+0000 7faa7d13a640 1 -- 172.21.15.45:0/3252817452 reap_dead start 2024-09-18T13:59:11.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.654+0000 7faa7eb9d640 1 -- 172.21.15.45:0/3252817452 shutdown_connections 2024-09-18T13:59:11.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.654+0000 7faa7eb9d640 1 -- 172.21.15.45:0/3252817452 >> 172.21.15.45:0/3252817452 conn(0x7faa780a5bb0 msgr2=0x7faa781334f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.654+0000 7faa7eb9d640 1 -- 172.21.15.45:0/3252817452 shutdown_connections 2024-09-18T13:59:11.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.654+0000 7faa7eb9d640 1 -- 172.21.15.45:0/3252817452 wait complete. 2024-09-18T13:59:11.691 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T13:59:11.692 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T13:59:11.692 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T13:59:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.788+0000 7fe2f0a68640 10 monclient: get_monmap_and_config 2024-09-18T13:59:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.788+0000 7fe2f0a68640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.788+0000 7fe2f0a68640 10 monclient: monmap: 2024-09-18T13:59:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:11.789309+0000 2024-09-18T13:59:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:11.789309+0000 2024-09-18T13:59:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.788+0000 7fe2f0a68640 1 Processor -- start 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2f0a68640 1 -- start start 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2f0a68640 10 monclient: init 2024-09-18T13:59:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2f0a68640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2f0a68640 10 monclient: _add_conns ranks=[0,2,1,7,6,3,8,4,5] 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2f0a68640 10 monclient(hunting): picked mon.noname-a con 0x7fe2ec09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2f0a68640 10 monclient(hunting): picked mon.noname-b con 0x7fe2ec09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2f0a68640 10 monclient(hunting): picked mon.noname-f con 0x7fe2ec09ff70 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2f0a68640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec05ed70 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2f0a68640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec09dfc0 con 0x7fe2ec09ff70 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2f0a68640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec09dce0 con 0x7fe2ec09fba0 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2f0a68640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2f0a68640 10 monclient(hunting): authenticate will time out at 767.127784s 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2ea59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe2ec09f7d0 0x7fe2ec132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37020/0 (socket says 172.21.15.45:37020) 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2ea59c640 1 -- 172.21.15.45:0/2756425031 learned_addr learned my addr 172.21.15.45:0/2756425031 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.789+0000 7fe2e9d9b640 1 --1- 172.21.15.45:0/2756425031 >> v1:172.21.15.45:6790/0 conn(0x7fe2ec09fba0 0x7fe2ec137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51670/0 (socket says 172.21.15.45:51670) 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153855073 0 0) 0x7fe2ec05ed70 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient(hunting): my global_id is 4473 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2d40034e0 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761324640 0 0) 0x7fe2ec09dce0 con 0x7fe2ec09fba0 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient(hunting): my global_id is 4475 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ec05ed70 con 0x7fe2ec09fba0 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867172981 0 0) 0x7fe2ec09dfc0 con 0x7fe2ec09ff70 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient(hunting): my global_id is 4447 2024-09-18T13:59:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ec09dce0 con 0x7fe2ec09ff70 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 540294717 0 0) 0x7fe2d40034e0 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 >> v1:172.21.15.45:6790/0 conn(0x7fe2ec09fba0 legacy=0x7fe2ec137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 >> v1:172.21.15.84:6789/0 conn(0x7fe2ec09ff70 legacy=0x7fe2ec13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 monclient: found mon.noname-a 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec0a1c00 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2f0a68640 5 monclient: authenticate success, global_id 4473 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe2dc001830 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2f0a68640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient: dump: 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2f0a68640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe2dc001a10 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 1 -- 172.21.15.45:0/2756425031 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe2dc004ba0 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2f0a68640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2f0a68640 4 set_mon_vals no callback set 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.790+0000 7fe2e959a640 10 monclient: dump: 2024-09-18T13:59:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.791+0000 7fe2f0a68640 10 monclient: shutdown 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.791+0000 7fe2f0a68640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.791+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2756425031 >> v1:172.21.15.45:6789/0 conn(0x7fe2ec09f7d0 legacy=0x7fe2ec132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.791+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2756425031 shutdown_connections 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.791+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2756425031 >> 172.21.15.45:0/2756425031 conn(0x7fe2ec0630b0 msgr2=0x7fe2ec0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.791+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2756425031 shutdown_connections 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.791+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2756425031 wait complete. 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.791+0000 7fe2f0a68640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.791+0000 7fe2f0a68640 10 monclient: monmap: 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.806 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.791+0000 7fe2f0a68640 1 Processor -- start 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2f0a68640 1 -- start start 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2f0a68640 10 monclient: init 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2f0a68640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2f0a68640 10 monclient: _add_conns ranks=[0,4,3,7,8,2,5,1,6] 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2f0a68640 10 monclient(hunting): picked mon.noname-a con 0x7fe2ec09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2f0a68640 10 monclient(hunting): picked mon.noname-c con 0x7fe2ec09fba0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2f0a68640 10 monclient(hunting): picked mon.noname-g con 0x7fe2ec09ff70 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2f0a68640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec11eae0 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2f0a68640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec11edc0 con 0x7fe2ec09ff70 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2f0a68640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec11e800 con 0x7fe2ec09fba0 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2f0a68640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2f0a68640 10 monclient(hunting): authenticate will time out at 767.130568s 2024-09-18T13:59:11.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2ea59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe2ec09f7d0 0x7fe2ec1316e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37034/0 (socket says 172.21.15.45:37034) 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2ea59c640 1 -- 172.21.15.45:0/2360420031 learned_addr learned my addr 172.21.15.45:0/2360420031 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155355934 0 0) 0x7fe2ec11eae0 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2da7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2da7fc640 10 monclient(hunting): my global_id is 4482 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2cc0034e0 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232188644 0 0) 0x7fe2ec11e800 con 0x7fe2ec09fba0 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2da7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2da7fc640 10 monclient(hunting): my global_id is 4495 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.792+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ec11eae0 con 0x7fe2ec09fba0 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2641417329 0 0) 0x7fe2ec11eae0 con 0x7fe2ec09fba0 2024-09-18T13:59:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe2ec11e800 con 0x7fe2ec09fba0 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1560641131 0 0) 0x7fe2cc0034e0 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe2ec11eae0 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe2e0001a60 con 0x7fe2ec09fba0 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618536530 0 0) 0x7fe2ec11edc0 con 0x7fe2ec09ff70 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient(hunting): my global_id is 4548 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2cc0034e0 con 0x7fe2ec09ff70 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe2dc001770 con 0x7fe2ec09f7d0 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 613786896 0 0) 0x7fe2ec11e800 con 0x7fe2ec09fba0 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 >> v1:172.21.15.84:6790/0 conn(0x7fe2ec09ff70 legacy=0x7fe2ec14a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 >> v1:172.21.15.45:6789/0 conn(0x7fe2ec09f7d0 legacy=0x7fe2ec1316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 monclient: found mon.c 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec131df0 con 0x7fe2ec09fba0 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2f0a68640 5 monclient: authenticate success, global_id 4495 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2f0a68640 10 monclient: _renew_subs 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2f0a68640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec11d4a0 con 0x7fe2ec09fba0 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2f0a68640 10 monclient: _renew_subs 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2f0a68640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec137040 con 0x7fe2ec09fba0 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe2e0001d80 con 0x7fe2ec09fba0 2024-09-18T13:59:11.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe2e0004e10 con 0x7fe2ec09fba0 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2e8d99640 4 set_mon_vals no callback set 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.793+0000 7fe2da7fc640 10 monclient: dump: 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2f0a68640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe2d0006580 con 0x7fe2d0002830 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fe2e000fe60 con 0x7fe2ec09fba0 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 3882+0+0 (unknown 594623498 0 0) 0x7fe2e0046a90 con 0x7fe2ec09fba0 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161234024 0 0) 0x7fe2d0006580 con 0x7fe2d0002830 2024-09-18T13:59:11.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 10 monclient(hunting): my global_id is 4418 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ec11e800 con 0x7fe2d0002830 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2ead9d640 10 monclient: get_auth_request con 0x7fe2cc03a610 auth_method 0 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 618567877 0 0) 0x7fe2ec11e800 con 0x7fe2d0002830 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe2d0006580 con 0x7fe2d0002830 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe2dc004c10 con 0x7fe2d0002830 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.794+0000 7fe2da7fc640 10 monclient: dump: 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.795+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3883653816 0 0) 0x7fe2d0006580 con 0x7fe2d0002830 2024-09-18T13:59:11.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.795+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2d000a0a0 con 0x7fe2d0002830 2024-09-18T13:59:11.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.795+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe2dc005510 con 0x7fe2d0002830 2024-09-18T13:59:11.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.795+0000 7fe2da7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.795+0000 7fe2da7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:11.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.795+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 >> v1:172.21.15.45:6793/0 conn(0x7fe2d0002830 legacy=0x7fe2d0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.795+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 >> v1:172.21.15.45:6793/0 conn(0x7fe2d0002830 legacy=0x7fe2d0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:11.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2f0a68640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:11.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe2d0007540 con 0x7fe2d00068f0 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988789565 0 0) 0x7fe2d0007540 con 0x7fe2d00068f0 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 10 monclient(hunting): my global_id is 4427 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2d0006580 con 0x7fe2d00068f0 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3728257277 0 0) 0x7fe2d0006580 con 0x7fe2d00068f0 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe2d0007540 con 0x7fe2d00068f0 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe2dc005d70 con 0x7fe2d00068f0 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 10 monclient: dump: 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1981810177 0 0) 0x7fe2d0007540 con 0x7fe2d00068f0 2024-09-18T13:59:11.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.801+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe2d00087a0 con 0x7fe2d00068f0 2024-09-18T13:59:11.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 3126554381) 0x7fe2dc006110 con 0x7fe2d00068f0 2024-09-18T13:59:11.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2da7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:11.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2da7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:11.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 >> v1:172.21.15.45:6793/0 conn(0x7fe2d00068f0 legacy=0x7fe2d0006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2da7fc640 1 -- 172.21.15.45:0/2360420031 >> v1:172.21.15.45:6793/0 conn(0x7fe2d00068f0 legacy=0x7fe2d0006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:11.822 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:11.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:11.822 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:11.822 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 14, 2024-09-18T13:59:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T13:59:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 19, 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.824 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:11.826 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.829 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:11.831 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:11.832 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.833 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.836 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.838 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.839 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.840 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:11.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.841 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:11.841 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:11.841 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:11.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.841 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.841 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:11.841 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:11.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2360420031 >> v1:172.21.15.84:6812/31478 conn(0x7fe2cc03a610 legacy=0x7fe2cc03cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2f0a68640 10 monclient: shutdown 2024-09-18T13:59:11.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2f0a68640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2360420031 >> v1:172.21.15.45:6791/0 conn(0x7fe2ec09fba0 legacy=0x7fe2ec146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2ead9d640 1 -- 172.21.15.45:0/2360420031 reap_dead start 2024-09-18T13:59:11.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2ead9d640 1 -- 172.21.15.45:0/2360420031 reap_dead start 2024-09-18T13:59:11.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2360420031 shutdown_connections 2024-09-18T13:59:11.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2360420031 >> 172.21.15.45:0/2360420031 conn(0x7fe2ec0630b0 msgr2=0x7fe2ec0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2360420031 shutdown_connections 2024-09-18T13:59:11.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.802+0000 7fe2f0a68640 1 -- 172.21.15.45:0/2360420031 wait complete. 2024-09-18T13:59:11.844 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T13:59:11.844 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T13:59:11.844 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T13:59:11.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.941+0000 7fb7fe49f640 10 monclient: get_monmap_and_config 2024-09-18T13:59:11.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.941+0000 7fb7fe49f640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.941+0000 7fb7fe49f640 10 monclient: monmap: 2024-09-18T13:59:11.942 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.942 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:11.942912+0000 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:11.942912+0000 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.943 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.942+0000 7fb7fe49f640 1 Processor -- start 2024-09-18T13:59:11.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.942+0000 7fb7fe49f640 1 -- start start 2024-09-18T13:59:11.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.942+0000 7fb7fe49f640 10 monclient: init 2024-09-18T13:59:11.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7fe49f640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7fe49f640 10 monclient: _add_conns ranks=[3,4,8,6,5,2,7,0,1] 2024-09-18T13:59:11.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7fe49f640 10 monclient(hunting): picked mon.noname-g con 0x7fb7f809f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:11.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7fe49f640 10 monclient(hunting): picked mon.noname-c con 0x7fb7f809fba0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:11.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7fe49f640 10 monclient(hunting): picked mon.noname-e con 0x7fb7f809ff70 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7fe49f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f805ed70 con 0x7fb7f809f7d0 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7fe49f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f809dfc0 con 0x7fb7f809fba0 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7fe49f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f809dce0 con 0x7fb7f809ff70 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7fe49f640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7fe49f640 10 monclient(hunting): authenticate will time out at 767.281387s 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f77fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fb7f809fba0 0x7fb7f8137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:46032/0 (socket says 172.21.15.45:46032) 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f77fe640 1 -- 172.21.15.45:0/2726160504 learned_addr learned my addr 172.21.15.45:0/2726160504 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7fca3c640 1 --1- 172.21.15.45:0/2726160504 >> v1:172.21.15.45:6793/0 conn(0x7fb7f809ff70 0x7fb7f813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51966/0 (socket says 172.21.15.45:51966) 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837747032 0 0) 0x7fb7f809dce0 con 0x7fb7f809ff70 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 10 monclient(hunting): my global_id is 4436 2024-09-18T13:59:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7e00034e0 con 0x7fb7f809ff70 2024-09-18T13:59:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136847340 0 0) 0x7fb7f805ed70 con 0x7fb7f809f7d0 2024-09-18T13:59:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 10 monclient(hunting): my global_id is 4557 2024-09-18T13:59:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f809dce0 con 0x7fb7f809f7d0 2024-09-18T13:59:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746206583 0 0) 0x7fb7f809dfc0 con 0x7fb7f809fba0 2024-09-18T13:59:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 10 monclient(hunting): my global_id is 4504 2024-09-18T13:59:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f805ed70 con 0x7fb7f809fba0 2024-09-18T13:59:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.943+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 507497366 0 0) 0x7fb7e00034e0 con 0x7fb7f809ff70 2024-09-18T13:59:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 >> v1:172.21.15.45:6791/0 conn(0x7fb7f809fba0 legacy=0x7fb7f8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 >> v1:172.21.15.84:6790/0 conn(0x7fb7f809f7d0 legacy=0x7fb7f8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 1 monclient: found mon.noname-e 2024-09-18T13:59:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T13:59:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f80a1c00 con 0x7fb7f809ff70 2024-09-18T13:59:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7fe49f640 5 monclient: authenticate success, global_id 4436 2024-09-18T13:59:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7fe49f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb7f0001bb0 con 0x7fb7f809ff70 2024-09-18T13:59:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 10 monclient: dump: 2024-09-18T13:59:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.958 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.958 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.958 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.958 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.958 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.958 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.958 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7fe49f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:11.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb7f0001d90 con 0x7fb7f809ff70 2024-09-18T13:59:11.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 1 -- 172.21.15.45:0/2726160504 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb7f0004e60 con 0x7fb7f809ff70 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7fe49f640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7fe49f640 4 set_mon_vals no callback set 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7f6ffd640 10 monclient: dump: 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7fe49f640 10 monclient: shutdown 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7fe49f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7fe49f640 1 -- 172.21.15.45:0/2726160504 >> v1:172.21.15.45:6793/0 conn(0x7fb7f809ff70 legacy=0x7fb7f813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7fe49f640 1 -- 172.21.15.45:0/2726160504 shutdown_connections 2024-09-18T13:59:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7fe49f640 1 -- 172.21.15.45:0/2726160504 >> 172.21.15.45:0/2726160504 conn(0x7fb7f80630b0 msgr2=0x7fb7f80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7fe49f640 1 -- 172.21.15.45:0/2726160504 shutdown_connections 2024-09-18T13:59:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.944+0000 7fb7fe49f640 1 -- 172.21.15.45:0/2726160504 wait complete. 2024-09-18T13:59:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.945+0000 7fb7fe49f640 10 monclient: build_initial_monmap 2024-09-18T13:59:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.945+0000 7fb7fe49f640 10 monclient: monmap: 2024-09-18T13:59:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.945+0000 7fb7fe49f640 1 Processor -- start 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.945+0000 7fb7fe49f640 1 -- start start 2024-09-18T13:59:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.945+0000 7fb7fe49f640 10 monclient: init 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7fe49f640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7fe49f640 10 monclient: _add_conns ranks=[2,7,0,8,4,6,5,3,1] 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7fe49f640 10 monclient(hunting): picked mon.noname-b con 0x7fb7f809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7fe49f640 10 monclient(hunting): picked mon.noname-i con 0x7fb7f809fba0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7fe49f640 10 monclient(hunting): picked mon.noname-a con 0x7fb7f809ff70 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7fe49f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f80a2560 con 0x7fb7f809ff70 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7fe49f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f8136ea0 con 0x7fb7f809f7d0 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7fe49f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f811ec90 con 0x7fb7f809fba0 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7fe49f640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7fe49f640 10 monclient(hunting): authenticate will time out at 767.284200s 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7f7fff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fb7f809f7d0 0x7fb7f81315b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51676/0 (socket says 172.21.15.45:51676) 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7f7fff640 1 -- 172.21.15.45:0/280406657 learned_addr learned my addr 172.21.15.45:0/280406657 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7fca3c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb7f809ff70 0x7fb7f814a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37042/0 (socket says 172.21.15.45:37042) 2024-09-18T13:59:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263226606 0 0) 0x7fb7f8136ea0 con 0x7fb7f809f7d0 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): my global_id is 4484 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7e40034e0 con 0x7fb7f809f7d0 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862795633 0 0) 0x7fb7f80a2560 con 0x7fb7f809ff70 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): my global_id is 4491 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f8136ea0 con 0x7fb7f809ff70 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847617065 0 0) 0x7fb7f811ec90 con 0x7fb7f809fba0 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): my global_id is 4489 2024-09-18T13:59:11.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f80a2560 con 0x7fb7f809fba0 2024-09-18T13:59:11.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2284926135 0 0) 0x7fb7e40034e0 con 0x7fb7f809f7d0 2024-09-18T13:59:11.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7f811ec90 con 0x7fb7f809f7d0 2024-09-18T13:59:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb7e8001d90 con 0x7fb7f809f7d0 2024-09-18T13:59:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): dump: 2024-09-18T13:59:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1302335136 0 0) 0x7fb7f8136ea0 con 0x7fb7f809ff70 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7e40034e0 con 0x7fb7f809ff70 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb7f0001b70 con 0x7fb7f809ff70 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 10 monclient(hunting): dump: 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.969 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.946+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3712087815 0 0) 0x7fb7f811ec90 con 0x7fb7f809f7d0 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 >> v1:172.21.15.84:6792/0 conn(0x7fb7f809fba0 legacy=0x7fb7f8146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 >> v1:172.21.15.45:6789/0 conn(0x7fb7f809ff70 legacy=0x7fb7f814a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 10 monclient: _finish_hunting 0 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 1 monclient: found mon.b 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f811d7e0 con 0x7fb7f809f7d0 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 10 monclient: _finish_auth 0 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7fe49f640 5 monclient: authenticate success, global_id 4484 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7fe49f640 10 monclient: _renew_subs 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7fe49f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7fe49f640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f811e6d0 con 0x7fb7f809f7d0 2024-09-18T13:59:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb7e8002220 con 0x7fb7f809f7d0 2024-09-18T13:59:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb7e80027d0 con 0x7fb7f809f7d0 2024-09-18T13:59:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7dffff640 10 monclient: dump: 2024-09-18T13:59:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7f67fc640 4 set_mon_vals no callback set 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7fe49f640 10 monclient: _renew_subs 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7fe49f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.947+0000 7fb7fe49f640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f809b7b0 con 0x7fb7f809f7d0 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7fe49f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7fe49f640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb7f804c4a0 con 0x7fb7f809ac50 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fb7e8010120 con 0x7fb7f809f7d0 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 3882+0+0 (unknown 594623498 0 0) 0x7fb7e8046af0 con 0x7fb7f809f7d0 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7fca3c640 10 monclient: get_auth_request con 0x7fb7e403a5b0 auth_method 0 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075320845 0 0) 0x7fb7f804c4a0 con 0x7fb7f809ac50 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 10 monclient(hunting): my global_id is 4456 2024-09-18T13:59:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f811ec90 con 0x7fb7f809ac50 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1289002141 0 0) 0x7fb7f811ec90 con 0x7fb7f809ac50 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7f804c4a0 con 0x7fb7f809ac50 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb7ec001930 con 0x7fb7f809ac50 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.948+0000 7fb7dffff640 10 monclient: dump: 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.949+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1782398330 0 0) 0x7fb7f804c4a0 con 0x7fb7f809ac50 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.949+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7f804b720 con 0x7fb7f809ac50 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.949+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb7ec001cf0 con 0x7fb7f809ac50 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.949+0000 7fb7dffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.949+0000 7fb7dffff640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:11.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.949+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 >> v1:172.21.15.84:6789/0 conn(0x7fb7f809ac50 legacy=0x7fb7f8096a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.949+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 >> v1:172.21.15.84:6789/0 conn(0x7fb7f809ac50 legacy=0x7fb7f8096a20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:11.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7fe49f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:11.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7fe49f640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb7f8071550 con 0x7fb7f802a8d0 2024-09-18T13:59:11.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813092657 0 0) 0x7fb7f8071550 con 0x7fb7f802a8d0 2024-09-18T13:59:11.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7dffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:11.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:11.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7dffff640 10 monclient(hunting): my global_id is 4465 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f804c4a0 con 0x7fb7f802a8d0 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3859962705 0 0) 0x7fb7f804c4a0 con 0x7fb7f802a8d0 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7f8071550 con 0x7fb7f802a8d0 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb7ec004e30 con 0x7fb7f802a8d0 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7dffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.955+0000 7fb7dffff640 10 monclient: dump: 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:11.979 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:11.980 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:11.980 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:11.980 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:11.980 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:11.980 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:11.980 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:11.980 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:11.980 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:11.980 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:11.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1465856793 0 0) 0x7fb7f8071550 con 0x7fb7f802a8d0 2024-09-18T13:59:11.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb7f809b020 con 0x7fb7f802a8d0 2024-09-18T13:59:11.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 388412858) 0x7fb7ec0050d0 con 0x7fb7f802a8d0 2024-09-18T13:59:11.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7dffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:11.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7dffff640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:11.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 >> v1:172.21.15.84:6789/0 conn(0x7fb7f802a8d0 legacy=0x7fb7f8097130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:11.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7dffff640 1 -- 172.21.15.45:0/280406657 >> v1:172.21.15.84:6789/0 conn(0x7fb7f802a8d0 legacy=0x7fb7f8097130 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:11.982 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:11.982 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:11.982 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:11.982 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:11.983 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 14, 2024-09-18T13:59:11.983 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:11.983 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:11.983 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:11.983 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:11.983 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:11.983 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T13:59:11.984 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:11.984 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:11.984 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:11.984 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:11.984 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.984 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 19, 2024-09-18T13:59:11.985 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:11.985 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:11.985 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:11.985 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.985 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.985 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.986 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.986 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.986 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.986 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.986 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.986 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.986 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.986 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:11.987 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:11.987 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.987 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.987 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.987 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.987 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.987 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.987 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.988 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.988 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.988 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.988 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:11.988 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:11.989 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:11.989 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:11.989 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:11.989 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:11.989 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:11.989 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:11.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:11.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:11.990 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:11.990 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:11.990 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:11.990 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:11.990 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:11.991 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:11.991 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:11.991 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:11.991 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:11.991 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:11.991 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:11.992 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:11.992 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:11.992 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:11.992 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:11.992 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:11.993 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:11.993 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:11.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.993 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:11.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.994 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:11.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:11.994 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.994 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.994 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:11.994 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.995 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.995 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.995 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:11.995 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:11.995 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.995 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.995 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.996 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:11.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:11.996 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:11.996 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:11.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:11.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:11.997 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:11.997 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:11.997 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:11.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:11.997 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:11.997 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:11.997 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:11.998 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:11.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:11.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:11.998 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:12.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:12.000 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.001 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:12.001 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.001 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.001 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.002 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:12.002 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:12.002 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.002 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.002 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.003 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:12.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:12.003 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.004 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:12.004 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.004 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.005 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:12.005 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:12.005 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.005 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.005 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.005 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:12.007 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.008 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.008 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.008 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:12.008 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:12.008 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.008 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.008 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.008 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:12.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.011 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.011 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:12.011 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:12.011 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.011 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.011 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.011 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:12.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:12.011 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.012 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.013 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.013 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.013 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:12.013 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:12.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.013 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:12.013 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:12.013 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:12.013 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.013 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.014 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:12.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.014 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:12.014 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:12.014 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:12.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.014 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.014 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:12.014 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7fe49f640 1 -- 172.21.15.45:0/280406657 >> v1:172.21.15.84:6812/31478 conn(0x7fb7e403a5b0 legacy=0x7fb7e403ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7fe49f640 10 monclient: shutdown 2024-09-18T13:59:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7fe49f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7fe49f640 1 -- 172.21.15.45:0/280406657 >> v1:172.21.15.45:6790/0 conn(0x7fb7f809f7d0 legacy=0x7fb7f81315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7fca3c640 1 -- 172.21.15.45:0/280406657 reap_dead start 2024-09-18T13:59:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7fca3c640 1 -- 172.21.15.45:0/280406657 reap_dead start 2024-09-18T13:59:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7fe49f640 1 -- 172.21.15.45:0/280406657 shutdown_connections 2024-09-18T13:59:12.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7fe49f640 1 -- 172.21.15.45:0/280406657 >> 172.21.15.45:0/280406657 conn(0x7fb7f80630b0 msgr2=0x7fb7f80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:12.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7fe49f640 1 -- 172.21.15.45:0/280406657 shutdown_connections 2024-09-18T13:59:12.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:11.956+0000 7fb7fe49f640 1 -- 172.21.15.45:0/280406657 wait complete. 2024-09-18T13:59:12.017 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T13:59:12.017 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T13:59:12.017 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T13:59:12.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.082+0000 7fbc1a7fc640 1 -- 172.21.15.45:0/1920127930 <== mon.3 v1:172.21.15.84:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]=0 enabled application 'rados' on pool 'unique_pool_0' v18) v1 ==== 197+0+0 (unknown 1606800723 0 0) 0x7fbc20012170 con 0x7fbc2409ee00 2024-09-18T13:59:12.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.082+0000 7fbc1a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2024-09-18T13:59:12.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.082+0000 7fbc1a7fc640 10 monclient: _finish_command 2 = system:0 enabled application 'rados' on pool 'unique_pool_0' 2024-09-18T13:59:12.083 INFO:teuthology.orchestra.run.smithi045.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-18T13:59:12.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.086+0000 7fbc2c79d640 1 -- 172.21.15.45:0/1920127930 >> v1:172.21.15.84:6812/31478 conn(0x7fbc0803ac70 legacy=0x7fbc0803d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.086+0000 7fbc2c79d640 10 monclient: shutdown 2024-09-18T13:59:12.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.086+0000 7fbc2c79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:12.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.086+0000 7fbc2c79d640 1 -- 172.21.15.45:0/1920127930 >> v1:172.21.15.84:6790/0 conn(0x7fbc2409ee00 legacy=0x7fbc24141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.086+0000 7fbc2c79d640 1 -- 172.21.15.45:0/1920127930 shutdown_connections 2024-09-18T13:59:12.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.086+0000 7fbc2c79d640 1 -- 172.21.15.45:0/1920127930 >> 172.21.15.45:0/1920127930 conn(0x7fbc240698b0 msgr2=0x7fbc240a0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:12.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.086+0000 7fbc2c79d640 1 -- 172.21.15.45:0/1920127930 shutdown_connections 2024-09-18T13:59:12.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.086+0000 7fbc2c79d640 1 -- 172.21.15.45:0/1920127930 wait complete. 2024-09-18T13:59:12.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.123+0000 7f7bbe91d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:12.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.123+0000 7f7bbe91d640 10 monclient: build_initial_monmap 2024-09-18T13:59:12.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.123+0000 7f7bbe91d640 10 monclient: monmap: 2024-09-18T13:59:12.124 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:12.124 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:12.124901+0000 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:12.124901+0000 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:12.125 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.124+0000 7f7bbe91d640 1 Processor -- start 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.124+0000 7f7bbe91d640 1 -- start start 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.124+0000 7f7bbe91d640 10 monclient: init 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bbe91d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bbe91d640 10 monclient: _add_conns ranks=[8,6,7,5,1,2,0,4,3] 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bbe91d640 10 monclient(hunting): picked mon.noname-e con 0x7f7bb809f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bbe91d640 10 monclient(hunting): picked mon.noname-d con 0x7f7bb809fba0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bbe91d640 10 monclient(hunting): picked mon.noname-i con 0x7f7bb809ff70 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bbe91d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb805ed70 con 0x7f7bb809fba0 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bbe91d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb809dfc0 con 0x7f7bb809ff70 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bbe91d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb809dce0 con 0x7f7bb809f7d0 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bbe91d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bbe91d640 10 monclient(hunting): authenticate will time out at 767.463532s 2024-09-18T13:59:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bb7fff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f7bb809f7d0 0x7f7bb8132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51968/0 (socket says 172.21.15.45:51968) 2024-09-18T13:59:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bb77fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f7bb809fba0 0x7f7bb8137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34834/0 (socket says 172.21.15.45:34834) 2024-09-18T13:59:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.125+0000 7f7bb7fff640 1 -- 172.21.15.45:0/3763792675 learned_addr learned my addr 172.21.15.45:0/3763792675 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 1 -- 172.21.15.45:0/3763792675 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090438821 0 0) 0x7f7bb809dce0 con 0x7f7bb809f7d0 2024-09-18T13:59:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient(hunting): my global_id is 4445 2024-09-18T13:59:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 1 -- 172.21.15.45:0/3763792675 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba00034e0 con 0x7f7bb809f7d0 2024-09-18T13:59:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 1 -- 172.21.15.45:0/3763792675 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309666255 0 0) 0x7f7bb805ed70 con 0x7f7bb809fba0 2024-09-18T13:59:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient(hunting): my global_id is 4443 2024-09-18T13:59:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 1 -- 172.21.15.45:0/3763792675 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb809dce0 con 0x7f7bb809fba0 2024-09-18T13:59:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 1 -- 172.21.15.45:0/3763792675 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1707999744 0 0) 0x7f7ba00034e0 con 0x7f7bb809f7d0 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 1 -- 172.21.15.45:0/3763792675 >> v1:172.21.15.84:6792/0 conn(0x7f7bb809ff70 legacy=0x7f7bb813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 1 -- 172.21.15.45:0/3763792675 >> v1:172.21.15.45:6792/0 conn(0x7f7bb809fba0 legacy=0x7f7bb8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 1 monclient: found mon.noname-e 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 1 -- 172.21.15.45:0/3763792675 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb80a1c00 con 0x7f7bb809f7d0 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 1 -- 172.21.15.45:0/3763792675 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7ba8001830 con 0x7f7bb809f7d0 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient: dump: 2024-09-18T13:59:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bbe91d640 5 monclient: authenticate success, global_id 4445 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bbe91d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 1 -- 172.21.15.45:0/3763792675 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7ba8001990 con 0x7f7bb809f7d0 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 1 -- 172.21.15.45:0/3763792675 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7ba8001c00 con 0x7f7bb809f7d0 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bb6ffd640 10 monclient: dump: 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bbe91d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.129+0000 7f7bbe91d640 4 set_mon_vals no callback set 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.130+0000 7f7bbe91d640 10 monclient: shutdown 2024-09-18T13:59:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.130+0000 7f7bbe91d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.130+0000 7f7bbe91d640 1 -- 172.21.15.45:0/3763792675 >> v1:172.21.15.45:6793/0 conn(0x7f7bb809f7d0 legacy=0x7f7bb8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.130+0000 7f7bbe91d640 1 -- 172.21.15.45:0/3763792675 shutdown_connections 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.130+0000 7f7bbe91d640 1 -- 172.21.15.45:0/3763792675 >> 172.21.15.45:0/3763792675 conn(0x7f7bb80630b0 msgr2=0x7f7bb80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.130+0000 7f7bbe91d640 1 -- 172.21.15.45:0/3763792675 shutdown_connections 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.130+0000 7f7bbe91d640 1 -- 172.21.15.45:0/3763792675 wait complete. 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.130+0000 7f7bbe91d640 10 monclient: build_initial_monmap 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.130+0000 7f7bbe91d640 10 monclient: monmap: 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:12.137 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 1 Processor -- start 2024-09-18T13:59:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 1 -- start start 2024-09-18T13:59:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 10 monclient: init 2024-09-18T13:59:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 10 monclient: _add_conns ranks=[7,5,4,8,0,1,3,2,6] 2024-09-18T13:59:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 10 monclient(hunting): picked mon.noname-i con 0x7f7bb809f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 10 monclient(hunting): picked mon.noname-h con 0x7f7bb809fba0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 10 monclient(hunting): picked mon.noname-c con 0x7f7bb809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb80a2560 con 0x7f7bb809ff70 2024-09-18T13:59:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb81426b0 con 0x7f7bb809fba0 2024-09-18T13:59:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb8121c10 con 0x7f7bb809f7d0 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbe91d640 10 monclient(hunting): authenticate will time out at 767.469901s 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.131+0000 7f7bbceba640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f7bb809ff70 0x7f7bb814a250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:46038/0 (socket says 172.21.15.45:46038) 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7bbceba640 1 -- 172.21.15.45:0/1672880801 learned_addr learned my addr 172.21.15.45:0/1672880801 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273578800 0 0) 0x7f7bb80a2560 con 0x7f7bb809ff70 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient(hunting): my global_id is 4513 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba40034e0 con 0x7f7bb809ff70 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1136425980 0 0) 0x7f7ba40034e0 con 0x7f7bb809ff70 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7bb80a2560 con 0x7f7bb809ff70 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7bb0001e40 con 0x7f7bb809ff70 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient(hunting): dump: 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 954126882 0 0) 0x7f7bb80a2560 con 0x7f7bb809ff70 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 >> v1:172.21.15.84:6792/0 conn(0x7f7bb809f7d0 legacy=0x7f7bb81317e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 >> v1:172.21.15.84:6791/0 conn(0x7f7bb809fba0 legacy=0x7f7bb8146b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient: _finish_hunting 0 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 1 monclient: found mon.c 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb8121930 con 0x7f7bb809ff70 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient: _finish_auth 0 2024-09-18T13:59:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7bbe91d640 5 monclient: authenticate success, global_id 4513 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7bbe91d640 10 monclient: _renew_subs 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7bbe91d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7bbe91d640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb811fe80 con 0x7f7bb809ff70 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7bbe91d640 10 monclient: _renew_subs 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7bbe91d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7bbe91d640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb8131ef0 con 0x7f7bb809ff70 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7bb0002630 con 0x7f7bb809ff70 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7bb0002b60 con 0x7f7bb809ff70 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7bb67fc640 4 set_mon_vals no callback set 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:12.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.132+0000 7f7b9ffff640 10 monclient: dump: 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.133+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7bb0002d40 con 0x7f7bb809ff70 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.133+0000 7f7bbe91d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.133+0000 7f7bbe91d640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7b98006580 con 0x7f7b98002830 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.133+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(19..19 src has 1..19) v4 ==== 3895+0+0 (unknown 2402260055 0 0) 0x7f7bb0046f60 con 0x7f7bb809ff70 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7bb77fe640 10 monclient: get_auth_request con 0x7f7ba4039d70 auth_method 0 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280651283 0 0) 0x7f7b98006580 con 0x7f7b98002830 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 10 monclient(hunting): my global_id is 4566 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb80a2560 con 0x7f7b98002830 2024-09-18T13:59:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1332527282 0 0) 0x7f7bb80a2560 con 0x7f7b98002830 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7b98006580 con 0x7f7b98002830 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7ba8001770 con 0x7f7b98002830 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 10 monclient: dump: 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3516954043 0 0) 0x7f7b98006580 con 0x7f7b98002830 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.134+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b9800a0a0 con 0x7f7b98002830 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.135+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7ba80047a0 con 0x7f7b98002830 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.135+0000 7f7b9ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.135+0000 7f7b9ffff640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.135+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 >> v1:172.21.15.84:6790/0 conn(0x7f7b98002830 legacy=0x7f7b98004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.135+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 >> v1:172.21.15.84:6790/0 conn(0x7f7b98002830 legacy=0x7f7b98004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7bbe91d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7bbe91d640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7b98009630 con 0x7f7b98009010 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185340916 0 0) 0x7f7b98009630 con 0x7f7b98009010 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 10 monclient(hunting): my global_id is 4575 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b98006580 con 0x7f7b98009010 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3507175251 0 0) 0x7f7b98006580 con 0x7f7b98009010 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7b98009630 con 0x7f7b98009010 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7ba8005060 con 0x7f7b98009010 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 10 monclient: dump: 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.152 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1549647668 0 0) 0x7f7b98009630 con 0x7f7b98009010 2024-09-18T13:59:12.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.141+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7b980098f0 con 0x7f7b98009010 2024-09-18T13:59:12.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7727 (unknown 0 0 3325218275) 0x7f7ba8007260 con 0x7f7b98009010 2024-09-18T13:59:12.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7b9ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:12.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7b9ffff640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:12.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 >> v1:172.21.15.84:6790/0 conn(0x7f7b98009010 legacy=0x7f7b98008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7b9ffff640 1 -- 172.21.15.45:0/1672880801 >> v1:172.21.15.84:6790/0 conn(0x7f7b98009010 legacy=0x7f7b98008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:12.154 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:12.154 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:12.154 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:12.154 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:12.154 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 14, 2024-09-18T13:59:12.154 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:12.154 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:12.154 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 19, 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:12.155 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:12.156 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:12.157 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:12.158 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:12.159 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.160 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:12.162 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:12.163 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:12.165 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.166 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.168 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.169 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:12.171 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:12.172 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T13:59:12.173 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.173 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.173 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T13:59:12.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.173 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:12.173 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:12.173 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:12.173 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.173 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.173 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:12.173 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:12.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7bbe91d640 1 -- 172.21.15.45:0/1672880801 >> v1:172.21.15.84:6812/31478 conn(0x7f7ba4039d70 legacy=0x7f7ba403c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7bbe91d640 10 monclient: shutdown 2024-09-18T13:59:12.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7bbe91d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:12.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7bbe91d640 1 -- 172.21.15.45:0/1672880801 >> v1:172.21.15.45:6791/0 conn(0x7f7bb809ff70 legacy=0x7f7bb814a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7bbceba640 1 -- 172.21.15.45:0/1672880801 reap_dead start 2024-09-18T13:59:12.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7bbceba640 1 -- 172.21.15.45:0/1672880801 reap_dead start 2024-09-18T13:59:12.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7bbe91d640 1 -- 172.21.15.45:0/1672880801 shutdown_connections 2024-09-18T13:59:12.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7bbe91d640 1 -- 172.21.15.45:0/1672880801 >> 172.21.15.45:0/1672880801 conn(0x7f7bb80630b0 msgr2=0x7f7bb80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:12.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7bbe91d640 1 -- 172.21.15.45:0/1672880801 shutdown_connections 2024-09-18T13:59:12.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.142+0000 7f7bbe91d640 1 -- 172.21.15.45:0/1672880801 wait complete. 2024-09-18T13:59:12.176 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T13:59:12.176 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T13:59:12.176 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T13:59:12.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.312+0000 7f1647694640 10 monclient: get_monmap_and_config 2024-09-18T13:59:12.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.312+0000 7f1647694640 10 monclient: build_initial_monmap 2024-09-18T13:59:12.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.312+0000 7f1647694640 10 monclient: monmap: 2024-09-18T13:59:12.313 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:12.313 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.313 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:12.313663+0000 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:12.313663+0000 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 1 Processor -- start 2024-09-18T13:59:12.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 1 -- start start 2024-09-18T13:59:12.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 10 monclient: init 2024-09-18T13:59:12.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:12.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 10 monclient: _add_conns ranks=[4,6,0,5,7,1,3,8,2] 2024-09-18T13:59:12.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 10 monclient(hunting): picked mon.noname-c con 0x7f16400a2610 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:12.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 10 monclient(hunting): picked mon.noname-d con 0x7f16401330b0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:12.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 10 monclient(hunting): picked mon.noname-a con 0x7f1640096a20 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:12.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164005ed70 con 0x7f1640096a20 2024-09-18T13:59:12.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16400a1740 con 0x7f16400a2610 2024-09-18T13:59:12.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16400a29e0 con 0x7f16401330b0 2024-09-18T13:59:12.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.313+0000 7f1647694640 10 monclient(hunting): authenticate will time out at 767.651956s 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1645430640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f16400a2610 0x7f16401329a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:46050/0 (socket says 172.21.15.45:46050) 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1645c31640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1640096a20 0x7f164013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37054/0 (socket says 172.21.15.45:37054) 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1645430640 1 -- 172.21.15.45:0/2026024768 learned_addr learned my addr 172.21.15.45:0/2026024768 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727826401 0 0) 0x7f164005ed70 con 0x7f1640096a20 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient(hunting): my global_id is 4500 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f162c0034e0 con 0x7f1640096a20 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873939569 0 0) 0x7f16400a1740 con 0x7f16400a2610 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient(hunting): my global_id is 4522 2024-09-18T13:59:12.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f164005ed70 con 0x7f16400a2610 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792640806 0 0) 0x7f16400a29e0 con 0x7f16401330b0 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient(hunting): my global_id is 4452 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16400a1740 con 0x7f16401330b0 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2702694147 0 0) 0x7f164005ed70 con 0x7f16400a2610 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 >> v1:172.21.15.45:6792/0 conn(0x7f16401330b0 legacy=0x7f1640137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 >> v1:172.21.15.45:6789/0 conn(0x7f1640096a20 legacy=0x7f164013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient: _finish_hunting 0 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 monclient: found mon.noname-c 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T13:59:12.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16400970a0 con 0x7f16400a2610 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient: _finish_auth 0 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f163c001790 con 0x7f16400a2610 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient: dump: 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1647694640 5 monclient: authenticate success, global_id 4522 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1647694640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f163c0018f0 con 0x7f16400a2610 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 1 -- 172.21.15.45:0/2026024768 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f163c004860 con 0x7f16400a2610 2024-09-18T13:59:12.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T13:59:12.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1637fff640 10 monclient: dump: 2024-09-18T13:59:12.327 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.327 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.327 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.327 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.327 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.327 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.327 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.327 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1647694640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.314+0000 7f1647694640 4 set_mon_vals no callback set 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.315+0000 7f1647694640 10 monclient: shutdown 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.315+0000 7f1647694640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.315+0000 7f1647694640 1 -- 172.21.15.45:0/2026024768 >> v1:172.21.15.45:6791/0 conn(0x7f16400a2610 legacy=0x7f16401329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.315+0000 7f1647694640 1 -- 172.21.15.45:0/2026024768 shutdown_connections 2024-09-18T13:59:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.315+0000 7f1647694640 1 -- 172.21.15.45:0/2026024768 >> 172.21.15.45:0/2026024768 conn(0x7f16400630f0 msgr2=0x7f16400634f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.315+0000 7f1647694640 1 -- 172.21.15.45:0/2026024768 shutdown_connections 2024-09-18T13:59:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.315+0000 7f1647694640 1 -- 172.21.15.45:0/2026024768 wait complete. 2024-09-18T13:59:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.315+0000 7f1647694640 10 monclient: build_initial_monmap 2024-09-18T13:59:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.315+0000 7f1647694640 10 monclient: monmap: 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:12.331 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 1 Processor -- start 2024-09-18T13:59:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 1 -- start start 2024-09-18T13:59:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 10 monclient: init 2024-09-18T13:59:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 10 monclient: _add_conns ranks=[2,6,7,8,3,0,1,4,5] 2024-09-18T13:59:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 10 monclient(hunting): picked mon.noname-b con 0x7f1640096a20 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 10 monclient(hunting): picked mon.noname-d con 0x7f16400a2610 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 10 monclient(hunting): picked mon.noname-i con 0x7f16401330b0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164011d610 con 0x7f1640096a20 2024-09-18T13:59:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16400a2350 con 0x7f16400a2610 2024-09-18T13:59:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164012f090 con 0x7f16401330b0 2024-09-18T13:59:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1647694640 10 monclient(hunting): authenticate will time out at 767.654954s 2024-09-18T13:59:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.316+0000 7f1644c2f640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f16400a2610 0x7f164009cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34854/0 (socket says 172.21.15.45:34854) 2024-09-18T13:59:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f1645430640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f1640096a20 0x7f164009c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:51708/0 (socket says 172.21.15.45:51708) 2024-09-18T13:59:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f1644c2f640 1 -- 172.21.15.45:0/714986394 learned_addr learned my addr 172.21.15.45:0/714986394 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345582873 0 0) 0x7f164011d610 con 0x7f1640096a20 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): my global_id is 4493 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16240034e0 con 0x7f1640096a20 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602452928 0 0) 0x7f16400a2350 con 0x7f16400a2610 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): my global_id is 4461 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f164011d610 con 0x7f16400a2610 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342252306 0 0) 0x7f164012f090 con 0x7f16401330b0 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): my global_id is 4498 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16400a2350 con 0x7f16401330b0 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 585000348 0 0) 0x7f16240034e0 con 0x7f1640096a20 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f164012f090 con 0x7f1640096a20 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f163c0016d0 con 0x7f1640096a20 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): dump: 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2291103575 0 0) 0x7f164011d610 con 0x7f16400a2610 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f16240034e0 con 0x7f16400a2610 2024-09-18T13:59:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1630001890 con 0x7f16400a2610 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient(hunting): dump: 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1419451489 0 0) 0x7f164012f090 con 0x7f1640096a20 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 >> v1:172.21.15.84:6792/0 conn(0x7f16401330b0 legacy=0x7f164009d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 >> v1:172.21.15.45:6792/0 conn(0x7f16400a2610 legacy=0x7f164009cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient: _finish_hunting 0 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 monclient: found mon.b 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164011d330 con 0x7f1640096a20 2024-09-18T13:59:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient: _finish_auth 0 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f1647694640 5 monclient: authenticate success, global_id 4493 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f1647694640 10 monclient: _renew_subs 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f1647694640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f1647694640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164011d050 con 0x7f1640096a20 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f1647694640 10 monclient: _renew_subs 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f1647694640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f1647694640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f164011c370 con 0x7f1640096a20 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f163c004860 con 0x7f1640096a20 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f163c0050d0 con 0x7f1640096a20 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.317+0000 7f16357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.318+0000 7f16357fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:12.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.318+0000 7f16377fe640 4 set_mon_vals no callback set 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.318+0000 7f16357fa640 10 monclient: dump: 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.339 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.318+0000 7f1647694640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.318+0000 7f1647694640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1620006580 con 0x7f1620002830 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.318+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f163c0100a0 con 0x7f1640096a20 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(19..19 src has 1..19) v4 ==== 3895+0+0 (unknown 2402260055 0 0) 0x7f163c046cd0 con 0x7f1640096a20 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f1645c31640 10 monclient: get_auth_request con 0x7f162403a530 auth_method 0 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281064488 0 0) 0x7f1620006580 con 0x7f1620002830 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 10 monclient(hunting): my global_id is 4415 2024-09-18T13:59:12.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f164012f090 con 0x7f1620002830 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 62354851 0 0) 0x7f164012f090 con 0x7f1620002830 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1620006580 con 0x7f1620002830 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1630004d50 con 0x7f1620002830 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 10 monclient: dump: 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.341 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1558562751 0 0) 0x7f1620006580 con 0x7f1620002830 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.319+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f162000a0a0 con 0x7f1620002830 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.320+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f16300050f0 con 0x7f1620002830 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.320+0000 7f16357fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:12.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.320+0000 7f16357fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:12.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.320+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 >> v1:172.21.15.84:6791/0 conn(0x7f1620002830 legacy=0x7f1620004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.320+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 >> v1:172.21.15.84:6791/0 conn(0x7f1620002830 legacy=0x7f1620004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:12.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f1647694640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:12.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f1647694640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1620009660 con 0x7f1620009040 2024-09-18T13:59:12.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183595765 0 0) 0x7f1620009660 con 0x7f1620009040 2024-09-18T13:59:12.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 10 monclient(hunting): my global_id is 4424 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1620006580 con 0x7f1620009040 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1442869132 0 0) 0x7f1620006580 con 0x7f1620009040 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1620009660 con 0x7f1620009040 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f16300058b0 con 0x7f1620009040 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 10 monclient: dump: 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.345 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3356980245 0 0) 0x7f1620009660 con 0x7f1620009040 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.326+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1620009920 con 0x7f1620009040 2024-09-18T13:59:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 2156519703) 0x7f1630006b80 con 0x7f1620009040 2024-09-18T13:59:12.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f16357fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:12.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f16357fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:12.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 >> v1:172.21.15.84:6791/0 conn(0x7f1620009040 legacy=0x7f1620008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f16357fa640 1 -- 172.21.15.45:0/714986394 >> v1:172.21.15.84:6791/0 conn(0x7f1620009040 legacy=0x7f1620008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:12.347 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:12.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:12.347 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:12.347 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:12.347 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 14, 2024-09-18T13:59:12.347 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 20, 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:12.348 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:12.349 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:12.350 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:12.351 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:12.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:12.353 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.354 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:12.357 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.358 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.360 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:12.361 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.364 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.365 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.366 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:12.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.366 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:12.366 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:12.366 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:12.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.366 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.366 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:12.366 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:12.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f1647694640 1 -- 172.21.15.45:0/714986394 >> v1:172.21.15.84:6812/31478 conn(0x7f162403a530 legacy=0x7f162403c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f1647694640 10 monclient: shutdown 2024-09-18T13:59:12.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f1647694640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:12.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f1647694640 1 -- 172.21.15.45:0/714986394 >> v1:172.21.15.45:6790/0 conn(0x7f1640096a20 legacy=0x7f164009c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f1645c31640 1 -- 172.21.15.45:0/714986394 reap_dead start 2024-09-18T13:59:12.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f1645c31640 1 -- 172.21.15.45:0/714986394 reap_dead start 2024-09-18T13:59:12.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f1647694640 1 -- 172.21.15.45:0/714986394 shutdown_connections 2024-09-18T13:59:12.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f1647694640 1 -- 172.21.15.45:0/714986394 >> 172.21.15.45:0/714986394 conn(0x7f16400630f0 msgr2=0x7f1640133b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:12.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f1647694640 1 -- 172.21.15.45:0/714986394 shutdown_connections 2024-09-18T13:59:12.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.327+0000 7f1647694640 1 -- 172.21.15.45:0/714986394 wait complete. 2024-09-18T13:59:12.369 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T13:59:12.369 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T13:59:12.369 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T13:59:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.465+0000 7fc0f4c0f640 10 monclient: get_monmap_and_config 2024-09-18T13:59:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.465+0000 7fc0f4c0f640 10 monclient: build_initial_monmap 2024-09-18T13:59:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.466+0000 7fc0f4c0f640 10 monclient: monmap: 2024-09-18T13:59:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:12.466936+0000 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:12.466936+0000 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:12.467 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.466+0000 7fc0f4c0f640 1 Processor -- start 2024-09-18T13:59:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.466+0000 7fc0f4c0f640 1 -- start start 2024-09-18T13:59:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.466+0000 7fc0f4c0f640 10 monclient: init 2024-09-18T13:59:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0f4c0f640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0f4c0f640 10 monclient: _add_conns ranks=[0,8,4,2,1,3,7,6,5] 2024-09-18T13:59:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0f4c0f640 10 monclient(hunting): picked mon.noname-a con 0x7fc0f006e810 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0f4c0f640 10 monclient(hunting): picked mon.noname-e con 0x7fc0f00a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0f4c0f640 10 monclient(hunting): picked mon.noname-c con 0x7fc0f00a1080 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0f4c0f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f005ed70 con 0x7fc0f006e810 2024-09-18T13:59:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0f4c0f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f0097620 con 0x7fc0f00a1080 2024-09-18T13:59:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0f4c0f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f0097340 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0f4c0f640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0f4c0f640 10 monclient(hunting): authenticate will time out at 767.805304s 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ee59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc0f006e810 0x7fc0f00a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37066/0 (socket says 172.21.15.45:37066) 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0edd9b640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fc0f00a0cb0 0x7fc0f0137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51984/0 (socket says 172.21.15.45:51984) 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ee59c640 1 -- 172.21.15.45:0/2220651140 learned_addr learned my addr 172.21.15.45:0/2220651140 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0eed9d640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fc0f00a1080 0x7fc0f013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:46062/0 (socket says 172.21.15.45:46062) 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697053728 0 0) 0x7fc0f0097340 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 10 monclient(hunting): my global_id is 4454 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0d80034e0 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027554118 0 0) 0x7fc0f005ed70 con 0x7fc0f006e810 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 10 monclient(hunting): my global_id is 4509 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0f0097340 con 0x7fc0f006e810 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656043680 0 0) 0x7fc0f0097620 con 0x7fc0f00a1080 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 10 monclient(hunting): my global_id is 4531 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0f005ed70 con 0x7fc0f00a1080 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.467+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4232947222 0 0) 0x7fc0f0097340 con 0x7fc0f006e810 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 >> v1:172.21.15.45:6793/0 conn(0x7fc0f00a0cb0 legacy=0x7fc0f0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 >> v1:172.21.15.45:6791/0 conn(0x7fc0f00a1080 legacy=0x7fc0f013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 10 monclient: _finish_hunting 0 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 1 monclient: found mon.noname-a 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T13:59:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f00a1660 con 0x7fc0f006e810 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 10 monclient: _finish_auth 0 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0e0001810 con 0x7fc0f006e810 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0f4c0f640 5 monclient: authenticate success, global_id 4509 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0f4c0f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 10 monclient: dump: 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0f4c0f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc0e0001970 con 0x7fc0f006e810 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 1 -- 172.21.15.45:0/2220651140 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0e0004860 con 0x7fc0f006e810 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0f4c0f640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0f4c0f640 4 set_mon_vals no callback set 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0ed59a640 10 monclient: dump: 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0f4c0f640 10 monclient: shutdown 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0f4c0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/2220651140 >> v1:172.21.15.45:6789/0 conn(0x7fc0f006e810 legacy=0x7fc0f00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/2220651140 shutdown_connections 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/2220651140 >> 172.21.15.45:0/2220651140 conn(0x7fc0f00a5bb0 msgr2=0x7fc0f00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/2220651140 shutdown_connections 2024-09-18T13:59:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.468+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/2220651140 wait complete. 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.469+0000 7fc0f4c0f640 10 monclient: build_initial_monmap 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.469+0000 7fc0f4c0f640 10 monclient: monmap: 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.469+0000 7fc0f4c0f640 1 Processor -- start 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.469+0000 7fc0f4c0f640 1 -- start start 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.469+0000 7fc0f4c0f640 10 monclient: init 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0f4c0f640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0f4c0f640 10 monclient: _add_conns ranks=[6,0,7,2,3,8,1,5,4] 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0f4c0f640 10 monclient(hunting): picked mon.noname-d con 0x7fc0f006e810 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0f4c0f640 10 monclient(hunting): picked mon.noname-a con 0x7fc0f00a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0f4c0f640 10 monclient(hunting): picked mon.noname-i con 0x7fc0f00a1080 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0f4c0f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f0122c20 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0f4c0f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f013da10 con 0x7fc0f006e810 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0f4c0f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f0122660 con 0x7fc0f00a1080 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0f4c0f640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0f4c0f640 10 monclient(hunting): authenticate will time out at 767.808200s 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0ee59c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fc0f006e810 0x7fc0f013d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34868/0 (socket says 172.21.15.45:34868) 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0ee59c640 1 -- 172.21.15.45:0/3984982064 learned_addr learned my addr 172.21.15.45:0/3984982064 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113649644 0 0) 0x7fc0f0122c20 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): my global_id is 4518 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0d00034e0 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296363105 0 0) 0x7fc0f013da10 con 0x7fc0f006e810 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): my global_id is 4470 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0f0122c20 con 0x7fc0f006e810 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3499511586 0 0) 0x7fc0d00034e0 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0f013da10 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0e40015d0 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1964099128 0 0) 0x7fc0f0122c20 con 0x7fc0f006e810 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0d00034e0 con 0x7fc0f006e810 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0e00016d0 con 0x7fc0f006e810 2024-09-18T13:59:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): dump: 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514038444 0 0) 0x7fc0f0122660 con 0x7fc0f00a1080 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient(hunting): my global_id is 4507 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0f0122c20 con 0x7fc0f00a1080 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1764226033 0 0) 0x7fc0f013da10 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 >> v1:172.21.15.84:6792/0 conn(0x7fc0f00a1080 legacy=0x7fc0f0149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 >> v1:172.21.15.45:6792/0 conn(0x7fc0f006e810 legacy=0x7fc0f013d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 monclient: found mon.a 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f0120170 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.470+0000 7fc0de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0f4c0f640 5 monclient: authenticate success, global_id 4518 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0f4c0f640 10 monclient: _renew_subs 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0f4c0f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f0122380 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0f4c0f640 10 monclient: _renew_subs 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0f4c0f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f01214b0 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc0e4004a50 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0de7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0e4004f00 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0de7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0de7fc640 10 monclient: dump: 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0ecd99640 4 set_mon_vals no callback set 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0f4c0f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc0d4006580 con 0x7fc0d4002830 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.471+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc0e4005060 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(19..19 src has 1..19) v4 ==== 3895+0+0 (unknown 2402260055 0 0) 0x7fc0e4046b20 con 0x7fc0f00a0cb0 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0eed9d640 10 monclient: get_auth_request con 0x7fc0d003a4e0 auth_method 0 2024-09-18T13:59:12.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557993159 0 0) 0x7fc0d4006580 con 0x7fc0d4002830 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 10 monclient(hunting): my global_id is 4516 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0f013da10 con 0x7fc0d4002830 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3868878737 0 0) 0x7fc0f013da10 con 0x7fc0d4002830 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0d4006580 con 0x7fc0d4002830 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0e00050d0 con 0x7fc0d4002830 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 10 monclient: dump: 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4236603737 0 0) 0x7fc0d4006580 con 0x7fc0d4002830 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.472+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0d400a0a0 con 0x7fc0d4002830 2024-09-18T13:59:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.473+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc0e00072f0 con 0x7fc0d4002830 2024-09-18T13:59:12.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.473+0000 7fc0de7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:12.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.473+0000 7fc0de7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:12.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.473+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 >> v1:172.21.15.84:6792/0 conn(0x7fc0d4002830 legacy=0x7fc0d4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.473+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 >> v1:172.21.15.84:6792/0 conn(0x7fc0d4002830 legacy=0x7fc0d4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.478+0000 7fc0f4c0f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.478+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc0d4009660 con 0x7fc0d4009040 2024-09-18T13:59:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714314684 0 0) 0x7fc0d4009660 con 0x7fc0d4009040 2024-09-18T13:59:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 10 monclient(hunting): my global_id is 4525 2024-09-18T13:59:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0d4006580 con 0x7fc0d4009040 2024-09-18T13:59:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 215819068 0 0) 0x7fc0d4006580 con 0x7fc0d4009040 2024-09-18T13:59:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0d4009660 con 0x7fc0d4009040 2024-09-18T13:59:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0e0005ad0 con 0x7fc0d4009040 2024-09-18T13:59:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 10 monclient: dump: 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:12.499 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3341393331 0 0) 0x7fc0d4009660 con 0x7fc0d4009040 2024-09-18T13:59:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.479+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc0d4009920 con 0x7fc0d4009040 2024-09-18T13:59:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 1222345859) 0x7fc0e0006f00 con 0x7fc0d4009040 2024-09-18T13:59:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0de7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0de7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 >> v1:172.21.15.84:6792/0 conn(0x7fc0d4009040 legacy=0x7fc0d4008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0de7fc640 1 -- 172.21.15.45:0/3984982064 >> v1:172.21.15.84:6792/0 conn(0x7fc0d4009040 legacy=0x7fc0d4008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 14, 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:12.500 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:12.501 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:12.501 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:12.501 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.501 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 20, 2024-09-18T13:59:12.501 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:12.501 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:12.501 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:12.501 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:12.501 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:12.501 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:12.501 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:12.501 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:12.502 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:12.503 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:12.504 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.505 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:12.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.507 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.509 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:12.510 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.511 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.513 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:12.514 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:12.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:12.516 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:12.516 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:12.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.516 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:12.516 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:12.516 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:12.516 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.516 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:12.517 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:12.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.518 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:12.518 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:12.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:12.518 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:12.518 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:12.518 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T13:59:12.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:12.518 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:12.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:12.518 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:12.518 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:12.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/3984982064 >> v1:172.21.15.84:6812/31478 conn(0x7fc0d003a4e0 legacy=0x7fc0d003c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0f4c0f640 10 monclient: shutdown 2024-09-18T13:59:12.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0f4c0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:12.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/3984982064 >> v1:172.21.15.45:6789/0 conn(0x7fc0f00a0cb0 legacy=0x7fc0f0146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:12.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0eed9d640 1 -- 172.21.15.45:0/3984982064 reap_dead start 2024-09-18T13:59:12.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0eed9d640 1 -- 172.21.15.45:0/3984982064 reap_dead start 2024-09-18T13:59:12.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/3984982064 shutdown_connections 2024-09-18T13:59:12.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/3984982064 >> 172.21.15.45:0/3984982064 conn(0x7fc0f00a5bb0 msgr2=0x7fc0f00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:12.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/3984982064 shutdown_connections 2024-09-18T13:59:12.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:12.480+0000 7fc0f4c0f640 1 -- 172.21.15.45:0/3984982064 wait complete. 2024-09-18T13:59:12.520 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T13:59:12.521 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-18T13:59:12.521 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['d', 'f', 'i'] 2024-09-18T13:59:12.521 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-18T13:59:12.521 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-18T13:59:12.521 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T13:59:12.521 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T13:59:12.532 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T13:59:12.532 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-18T13:59:12.532 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-18T13:59:12.532 INFO:tasks.mon_thrash:Sending STOP to mon f 2024-09-18T13:59:12.533 INFO:tasks.ceph.mon.f:Sent signal 19 2024-09-18T13:59:12.533 INFO:tasks.mon_thrash:Sending STOP to mon i 2024-09-18T13:59:12.533 INFO:tasks.ceph.mon.i:Sent signal 19 2024-09-18T13:59:12.533 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-18T13:59:13.099 DEBUG:teuthology.orchestra.run.smithi084:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0 2024-09-18T13:59:13.128 INFO:tasks.rados.rados.0.smithi084.stdout:adding op weight read -> 100 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout:adding op weight write -> 50 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout:adding op weight delete -> 50 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout:adding op weight snap_create -> 50 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout:adding op weight snap_remove -> 50 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout:adding op weight rollback -> 50 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout:adding op weight copy_from -> 50 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout:adding op weight write_excl -> 50 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout:ceph version 17.2.7-1645-g7061ea6c (7061ea6ca50908d104e9f3979dc615e118d9e153) quincy (stable) 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout:Configuration: 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout: Number of operations: 4000 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout: Number of objects: 50 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout: Max in flight operations: 16 2024-09-18T13:59:13.129 INFO:tasks.rados.rados.0.smithi084.stdout: Object size (in bytes): 4000000 2024-09-18T13:59:13.130 INFO:tasks.rados.rados.0.smithi084.stdout: Write stride min: 400000 2024-09-18T13:59:13.130 INFO:tasks.rados.rados.0.smithi084.stdout: Write stride max: 800000 2024-09-18T13:59:27.533 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-18T13:59:27.534 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-18T13:59:27.534 INFO:tasks.mon_thrash:Sending CONT to mon f 2024-09-18T13:59:27.535 INFO:tasks.ceph.mon.f:Sent signal 18 2024-09-18T13:59:27.535 INFO:tasks.mon_thrash:Sending CONT to mon i 2024-09-18T13:59:27.535 INFO:tasks.ceph.mon.i:Sent signal 18 2024-09-18T13:59:27.535 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-09-18T13:59:27.535 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T13:59:27.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.646+0000 7f2bc16ef640 10 monclient: get_monmap_and_config 2024-09-18T13:59:27.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.646+0000 7f2bc16ef640 10 monclient: build_initial_monmap 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.646+0000 7f2bc16ef640 10 monclient: monmap: 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:27.647124+0000 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:27.647124+0000 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:27.648 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.646+0000 7f2bc16ef640 1 Processor -- start 2024-09-18T13:59:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.646+0000 7f2bc16ef640 1 -- start start 2024-09-18T13:59:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.646+0000 7f2bc16ef640 10 monclient: init 2024-09-18T13:59:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.647+0000 7f2bc16ef640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.647+0000 7f2bc16ef640 10 monclient: _add_conns ranks=[1,5,2,6,3,4,0,8,7] 2024-09-18T13:59:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.647+0000 7f2bc16ef640 10 monclient(hunting): picked mon.noname-f con 0x7f2bbc06e810 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.647+0000 7f2bc16ef640 10 monclient(hunting): picked mon.noname-h con 0x7f2bbc0a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.647+0000 7f2bc16ef640 10 monclient(hunting): picked mon.noname-b con 0x7f2bbc0a1080 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.647+0000 7f2bc16ef640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc05ed70 con 0x7f2bbc06e810 2024-09-18T13:59:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.647+0000 7f2bc16ef640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc097620 con 0x7f2bbc0a1080 2024-09-18T13:59:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.647+0000 7f2bc16ef640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc097340 con 0x7f2bbc0a0cb0 2024-09-18T13:59:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.647+0000 7f2bc16ef640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.647+0000 7f2bc16ef640 10 monclient(hunting): authenticate will time out at 782.985407s 2024-09-18T13:59:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.647+0000 7f2bbb7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f2bbc0a1080 0x7f2bbc13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:45846/0 (socket says 172.21.15.45:45846) 2024-09-18T13:59:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:27.647+0000 7f2bbb7fe640 1 -- 172.21.15.45:0/1624358979 learned_addr learned my addr 172.21.15.45:0/1624358979 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:30.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 10 monclient(hunting): tick 2024-09-18T13:59:30.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 1 monclient(hunting): continuing hunt 2024-09-18T13:59:30.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T13:59:30.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 1 -- 172.21.15.45:0/1624358979 >> v1:172.21.15.84:6791/0 conn(0x7f2bbc0a0cb0 legacy=0x7f2bbc137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:30.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 1 -- 172.21.15.45:0/1624358979 >> v1:172.21.15.45:6790/0 conn(0x7f2bbc0a1080 legacy=0x7f2bbc13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:30.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 1 -- 172.21.15.45:0/1624358979 >> v1:172.21.15.84:6789/0 conn(0x7f2bbc06e810 legacy=0x7f2bbc0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:30.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 10 monclient: _add_conns ranks=[3,6,7,4,0,8] 2024-09-18T13:59:30.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f2ba80037a0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:30.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f2ba8007420 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f2ba800b1a0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 1 -- 172.21.15.45:0/1624358979 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc0a1660 con 0x7f2ba80037a0 2024-09-18T13:59:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 1 -- 172.21.15.45:0/1624358979 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba800ffb0 con 0x7f2ba8007420 2024-09-18T13:59:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 1 -- 172.21.15.45:0/1624358979 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba80111f0 con 0x7f2ba800b1a0 2024-09-18T13:59:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:30.647+0000 7f2bb8ff9640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.549+0000 7f2bb9ffb640 1 -- 172.21.15.45:0/1624358979 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504301016 0 0) 0x7f2bbc0a1660 con 0x7f2ba80037a0 2024-09-18T13:59:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.549+0000 7f2bb9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.549+0000 7f2bb9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.549+0000 7f2bb9ffb640 10 monclient(hunting): my global_id is 4584 2024-09-18T13:59:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.549+0000 7f2bb9ffb640 1 -- 172.21.15.45:0/1624358979 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba40034e0 con 0x7f2ba80037a0 2024-09-18T13:59:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 1 -- 172.21.15.45:0/1624358979 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4062318778 0 0) 0x7f2ba40034e0 con 0x7f2ba80037a0 2024-09-18T13:59:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 1 -- 172.21.15.45:0/1624358979 >> v1:172.21.15.84:6792/0 conn(0x7f2ba800b1a0 legacy=0x7f2ba800d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 1 -- 172.21.15.45:0/1624358979 >> v1:172.21.15.45:6792/0 conn(0x7f2ba8007420 legacy=0x7f2ba80098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 1 monclient: found mon.noname-g 2024-09-18T13:59:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T13:59:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 1 -- 172.21.15.45:0/1624358979 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba8012490 con 0x7f2ba80037a0 2024-09-18T13:59:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bbb7fe640 1 -- 172.21.15.45:0/1624358979 reap_dead start 2024-09-18T13:59:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 1 -- 172.21.15.45:0/1624358979 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2bac0017f0 con 0x7f2ba80037a0 2024-09-18T13:59:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:59:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 10 monclient: dump: 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bc16ef640 5 monclient: authenticate success, global_id 4584 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bc16ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 1 -- 172.21.15.45:0/1624358979 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2bac001a40 con 0x7f2ba80037a0 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bc16ef640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bc16ef640 4 set_mon_vals no callback set 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 1 -- 172.21.15.45:0/1624358979 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2bac001f70 con 0x7f2ba80037a0 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.550+0000 7f2bb9ffb640 10 monclient: dump: 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.556 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.551+0000 7f2bc16ef640 10 monclient: shutdown 2024-09-18T13:59:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.551+0000 7f2bc16ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.551+0000 7f2bc16ef640 1 -- 172.21.15.45:0/1624358979 >> v1:172.21.15.84:6790/0 conn(0x7f2ba80037a0 legacy=0x7f2ba8005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.551+0000 7f2bc16ef640 1 -- 172.21.15.45:0/1624358979 shutdown_connections 2024-09-18T13:59:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.551+0000 7f2bc16ef640 1 -- 172.21.15.45:0/1624358979 >> 172.21.15.45:0/1624358979 conn(0x7f2bbc0a5bb0 msgr2=0x7f2bbc0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.551+0000 7f2bc16ef640 1 -- 172.21.15.45:0/1624358979 shutdown_connections 2024-09-18T13:59:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.551+0000 7f2bc16ef640 1 -- 172.21.15.45:0/1624358979 wait complete. 2024-09-18T13:59:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.552+0000 7f2bc16ef640 10 monclient: build_initial_monmap 2024-09-18T13:59:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.552+0000 7f2bc16ef640 10 monclient: monmap: 2024-09-18T13:59:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 1 Processor -- start 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 1 -- start start 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 10 monclient: init 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 10 monclient: _add_conns ranks=[1,6,4,8,3,2,5,7,0] 2024-09-18T13:59:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 10 monclient(hunting): picked mon.noname-f con 0x7f2ba80037a0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:32.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 10 monclient(hunting): picked mon.noname-d con 0x7f2bbc11ff00 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:32.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 10 monclient(hunting): picked mon.noname-c con 0x7f2bbc147fd0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:32.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc0a2270 con 0x7f2ba80037a0 2024-09-18T13:59:32.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc1227e0 con 0x7f2bbc147fd0 2024-09-18T13:59:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc1202d0 con 0x7f2bbc11ff00 2024-09-18T13:59:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bc16ef640 10 monclient(hunting): authenticate will time out at 787.891944s 2024-09-18T13:59:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.553+0000 7f2bbb7fe640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f2bbc147fd0 legacy=0x7f2bbc14a3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T13:59:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.554+0000 7f2bba7fc640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f2bbc11ff00 0x7f2bbc1468b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:57994/0 (socket says 172.21.15.45:57994) 2024-09-18T13:59:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.554+0000 7f2bba7fc640 1 -- 172.21.15.45:0/234538937 learned_addr learned my addr 172.21.15.45:0/234538937 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.554+0000 7f2bbaffd640 1 --1- 172.21.15.45:0/234538937 >> v1:172.21.15.84:6789/0 conn(0x7f2ba80037a0 0x7f2bbc1353e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:44134/0 (socket says 172.21.15.45:44134) 2024-09-18T13:59:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.560+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754674844 0 0) 0x7f2bbc1202d0 con 0x7f2bbc11ff00 2024-09-18T13:59:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.560+0000 7f2ba2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.560+0000 7f2ba2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.560+0000 7f2ba2ffd640 10 monclient(hunting): my global_id is 4479 2024-09-18T13:59:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.560+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba8011460 con 0x7f2bbc11ff00 2024-09-18T13:59:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.560+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281603396 0 0) 0x7f2bbc0a2270 con 0x7f2ba80037a0 2024-09-18T13:59:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.560+0000 7f2ba2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.560+0000 7f2ba2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.560+0000 7f2ba2ffd640 10 monclient(hunting): my global_id is 4474 2024-09-18T13:59:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.560+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bbc1202d0 con 0x7f2ba80037a0 2024-09-18T13:59:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 738289349 0 0) 0x7f2ba8011460 con 0x7f2bbc11ff00 2024-09-18T13:59:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2bbc0a2270 con 0x7f2bbc11ff00 2024-09-18T13:59:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2bb0002ae0 con 0x7f2bbc11ff00 2024-09-18T13:59:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1190882327 0 0) 0x7f2bbc1202d0 con 0x7f2ba80037a0 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2ba8011460 con 0x7f2ba80037a0 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2bac001ef0 con 0x7f2ba80037a0 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient(hunting): dump: 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.569 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2953600589 0 0) 0x7f2bbc0a2270 con 0x7f2bbc11ff00 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 >> v1:172.21.15.45:6791/0 conn(0x7f2bbc147fd0 legacy=0x7f2bbc14a3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 >> v1:172.21.15.84:6789/0 conn(0x7f2ba80037a0 legacy=0x7f2bbc1353e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 monclient: found mon.d 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc142470 con 0x7f2bbc11ff00 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2bc16ef640 5 monclient: authenticate success, global_id 4479 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2bc16ef640 10 monclient: _renew_subs 2024-09-18T13:59:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2bc16ef640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2bc16ef640 1 -- 172.21.15.45:0/234538937 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc121610 con 0x7f2bbc11ff00 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2bc16ef640 10 monclient: _renew_subs 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2bc16ef640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2bc16ef640 1 -- 172.21.15.45:0/234538937 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc120a40 con 0x7f2bbc11ff00 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2bb0002d20 con 0x7f2bbc11ff00 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2bb0001780 con 0x7f2bbc11ff00 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2bb97fa640 4 set_mon_vals no callback set 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.561+0000 7f2ba2ffd640 10 monclient: dump: 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.571 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.572 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.562+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2bb0001960 con 0x7f2bbc11ff00 2024-09-18T13:59:32.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.562+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(19..19 src has 1..19) v4 ==== 3895+0+0 (unknown 2402260055 0 0) 0x7f2bb0046b20 con 0x7f2bbc11ff00 2024-09-18T13:59:32.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.562+0000 7f2bc16ef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:32.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.562+0000 7f2bc16ef640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:32.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.563+0000 7f2bc16ef640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:32.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.563+0000 7f2bc16ef640 1 -- 172.21.15.45:0/234538937 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c005010 con 0x7f2bbc11ff00 2024-09-18T13:59:32.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.563+0000 7f2bbaffd640 10 monclient: get_auth_request con 0x7f2ba803baa0 auth_method 0 2024-09-18T13:59:32.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.567+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f2bb0014e90 con 0x7f2bbc11ff00 2024-09-18T13:59:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.567+0000 7f2ba2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.567+0000 7f2ba2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:32.616 INFO:tasks.rados.rados.0.smithi084.stdout:1: write initial oid 1 2024-09-18T13:59:32.617 INFO:tasks.rados.rados.0.smithi084.stdout:1: seq_num 0 ranges {416807=475249,1542129=474678} 2024-09-18T13:59:32.621 INFO:tasks.rados.rados.0.smithi084.stdout:1: writing smithi08434663-1 from 416807 to 892056 tid 1 2024-09-18T13:59:32.628 INFO:tasks.rados.rados.0.smithi084.stdout:1: writing smithi08434663-1 from 1542129 to 2016807 tid 2 2024-09-18T13:59:32.628 INFO:tasks.rados.rados.0.smithi084.stdout:2: write initial oid 2 2024-09-18T13:59:32.628 INFO:tasks.rados.rados.0.smithi084.stdout:2: seq_num 1 ranges {416807=475249,1542129=474678} 2024-09-18T13:59:32.633 INFO:tasks.rados.rados.0.smithi084.stdout:2: writing smithi08434663-2 from 416807 to 892056 tid 1 2024-09-18T13:59:32.639 INFO:tasks.rados.rados.0.smithi084.stdout:2: writing smithi08434663-2 from 1542129 to 2016807 tid 2 2024-09-18T13:59:32.639 INFO:tasks.rados.rados.0.smithi084.stdout:3: write initial oid 3 2024-09-18T13:59:32.639 INFO:tasks.rados.rados.0.smithi084.stdout:3: seq_num 2 ranges {433614=550498,1600611=433003} 2024-09-18T13:59:32.644 INFO:tasks.rados.rados.0.smithi084.stdout:3: writing smithi08434663-3 from 433614 to 984112 tid 1 2024-09-18T13:59:32.651 INFO:tasks.rados.rados.0.smithi084.stdout:3: writing smithi08434663-3 from 1600611 to 2033614 tid 2 2024-09-18T13:59:32.651 INFO:tasks.rados.rados.0.smithi084.stdout:4: write initial oid 4 2024-09-18T13:59:32.651 INFO:tasks.rados.rados.0.smithi084.stdout:4: seq_num 3 ranges {450421=625747,1659093=391328} 2024-09-18T13:59:32.656 INFO:tasks.rados.rados.0.smithi084.stdout:4: writing smithi08434663-4 from 450421 to 1076168 tid 1 2024-09-18T13:59:32.662 INFO:tasks.rados.rados.0.smithi084.stdout:4: writing smithi08434663-4 from 1659093 to 2050421 tid 2 2024-09-18T13:59:32.662 INFO:tasks.rados.rados.0.smithi084.stdout:5: write initial oid 5 2024-09-18T13:59:32.662 INFO:tasks.rados.rados.0.smithi084.stdout:5: seq_num 4 ranges {467228=700996,1717575=349653} 2024-09-18T13:59:32.668 INFO:tasks.rados.rados.0.smithi084.stdout:5: writing smithi08434663-5 from 467228 to 1168224 tid 1 2024-09-18T13:59:32.673 INFO:tasks.rados.rados.0.smithi084.stdout:5: writing smithi08434663-5 from 1717575 to 2067228 tid 2 2024-09-18T13:59:32.674 INFO:tasks.rados.rados.0.smithi084.stdout:6: write initial oid 6 2024-09-18T13:59:32.674 INFO:tasks.rados.rados.0.smithi084.stdout:6: seq_num 5 ranges {484035=776245,2059704=24331} 2024-09-18T13:59:32.681 INFO:tasks.rados.rados.0.smithi084.stdout:6: writing smithi08434663-6 from 484035 to 1260280 tid 1 2024-09-18T13:59:32.685 INFO:tasks.rados.rados.0.smithi084.stdout:6: writing smithi08434663-6 from 2059704 to 2084035 tid 2 2024-09-18T13:59:32.686 INFO:tasks.rados.rados.0.smithi084.stdout:7: write initial oid 7 2024-09-18T13:59:32.686 INFO:tasks.rados.rados.0.smithi084.stdout:7: seq_num 6 ranges {500842=451494,1718186=382656} 2024-09-18T13:59:32.690 INFO:tasks.rados.rados.0.smithi084.stdout:7: writing smithi08434663-7 from 500842 to 952336 tid 1 2024-09-18T13:59:32.697 INFO:tasks.rados.rados.0.smithi084.stdout:7: writing smithi08434663-7 from 1718186 to 2100842 tid 2 2024-09-18T13:59:32.697 INFO:tasks.rados.rados.0.smithi084.stdout:8: write initial oid 8 2024-09-18T13:59:32.697 INFO:tasks.rados.rados.0.smithi084.stdout:8: seq_num 7 ranges {517649=526743,1776668=340981} 2024-09-18T13:59:32.702 INFO:tasks.rados.rados.0.smithi084.stdout:8: writing smithi08434663-8 from 517649 to 1044392 tid 1 2024-09-18T13:59:32.708 INFO:tasks.rados.rados.0.smithi084.stdout:8: writing smithi08434663-8 from 1776668 to 2117649 tid 2 2024-09-18T13:59:32.708 INFO:tasks.rados.rados.0.smithi084.stdout:9: write initial oid 9 2024-09-18T13:59:32.708 INFO:tasks.rados.rados.0.smithi084.stdout:9: seq_num 8 ranges {534456=718345,1951503=182953} 2024-09-18T13:59:32.715 INFO:tasks.rados.rados.0.smithi084.stdout:9: writing smithi08434663-9 from 534456 to 1252801 tid 1 2024-09-18T13:59:32.720 INFO:tasks.rados.rados.0.smithi084.stdout:9: writing smithi08434663-9 from 1951503 to 2134456 tid 2 2024-09-18T13:59:32.720 INFO:tasks.rados.rados.0.smithi084.stdout:10: write initial oid 10 2024-09-18T13:59:32.720 INFO:tasks.rados.rados.0.smithi084.stdout:10: seq_num 9 ranges {551263=793594,1893632=257631} 2024-09-18T13:59:32.727 INFO:tasks.rados.rados.0.smithi084.stdout:10: writing smithi08434663-10 from 551263 to 1344857 tid 1 2024-09-18T13:59:32.731 INFO:tasks.rados.rados.0.smithi084.stdout:10: writing smithi08434663-10 from 1893632 to 2151263 tid 2 2024-09-18T13:59:32.732 INFO:tasks.rados.rados.0.smithi084.stdout:11: write initial oid 11 2024-09-18T13:59:32.732 INFO:tasks.rados.rados.0.smithi084.stdout:11: seq_num 10 ranges {568070=468843,1552114=615956} 2024-09-18T13:59:32.737 INFO:tasks.rados.rados.0.smithi084.stdout:11: writing smithi08434663-11 from 568070 to 1036913 tid 1 2024-09-18T13:59:32.743 INFO:tasks.rados.rados.0.smithi084.stdout:11: writing smithi08434663-11 from 1552114 to 2168070 tid 2 2024-09-18T13:59:32.744 INFO:tasks.rados.rados.0.smithi084.stdout:12: write initial oid 12 2024-09-18T13:59:32.744 INFO:tasks.rados.rados.0.smithi084.stdout:12: seq_num 11 ranges {584877=544092,1610596=562003} 2024-09-18T13:59:32.750 INFO:tasks.rados.rados.0.smithi084.stdout:12: writing smithi08434663-12 from 584877 to 1128969 tid 1 2024-09-18T13:59:32.755 INFO:tasks.rados.rados.0.smithi084.stdout:12: writing smithi08434663-12 from 1610596 to 2172599 tid 2 2024-09-18T13:59:32.756 INFO:tasks.rados.rados.0.smithi084.stdout:13: write initial oid 13 2024-09-18T13:59:32.756 INFO:tasks.rados.rados.0.smithi084.stdout:13: seq_num 12 ranges {601684=619341,1669078=532606} 2024-09-18T13:59:32.762 INFO:tasks.rados.rados.0.smithi084.stdout:13: writing smithi08434663-13 from 601684 to 1221025 tid 1 2024-09-18T13:59:32.767 INFO:tasks.rados.rados.0.smithi084.stdout:13: writing smithi08434663-13 from 1669078 to 2201684 tid 2 2024-09-18T13:59:32.768 INFO:tasks.rados.rados.0.smithi084.stdout:14: write initial oid 14 2024-09-18T13:59:32.768 INFO:tasks.rados.rados.0.smithi084.stdout:14: seq_num 13 ranges {618491=694590,2011207=207284} 2024-09-18T13:59:32.774 INFO:tasks.rados.rados.0.smithi084.stdout:14: writing smithi08434663-14 from 618491 to 1313081 tid 1 2024-09-18T13:59:32.779 INFO:tasks.rados.rados.0.smithi084.stdout:14: writing smithi08434663-14 from 2011207 to 2218491 tid 2 2024-09-18T13:59:32.779 INFO:tasks.rados.rados.0.smithi084.stdout:15: write initial oid 15 2024-09-18T13:59:32.780 INFO:tasks.rados.rados.0.smithi084.stdout:15: seq_num 14 ranges {635298=769839,2069689=165609} 2024-09-18T13:59:32.787 INFO:tasks.rados.rados.0.smithi084.stdout:15: writing smithi08434663-15 from 635298 to 1405137 tid 1 2024-09-18T13:59:32.792 INFO:tasks.rados.rados.0.smithi084.stdout:15: writing smithi08434663-15 from 2069689 to 2235298 tid 2 2024-09-18T13:59:32.792 INFO:tasks.rados.rados.0.smithi084.stdout:16: write initial oid 16 2024-09-18T13:59:32.792 INFO:tasks.rados.rados.0.smithi084.stdout:16: seq_num 15 ranges {652105=445088,1728171=452988} 2024-09-18T13:59:32.797 INFO:tasks.rados.rados.0.smithi084.stdout:16: writing smithi08434663-16 from 652105 to 1097193 tid 1 2024-09-18T13:59:32.804 INFO:tasks.rados.rados.0.smithi084.stdout:16: writing smithi08434663-16 from 1728171 to 2181159 tid 2 2024-09-18T13:59:32.804 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:32.804 INFO:tasks.rados.rados.0.smithi084.stdout:1: finishing write tid 1 to smithi08434663-1 2024-09-18T13:59:32.805 INFO:tasks.rados.rados.0.smithi084.stdout:1: finishing write tid 2 to smithi08434663-1 2024-09-18T13:59:32.805 INFO:tasks.rados.rados.0.smithi084.stdout:1: finishing write tid 3 to smithi08434663-1 2024-09-18T13:59:32.805 INFO:tasks.rados.rados.0.smithi084.stdout:1: finishing write tid 4 to smithi08434663-1 2024-09-18T13:59:32.805 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T13:59:32.805 INFO:tasks.rados.rados.0.smithi084.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-09-18T13:59:32.806 INFO:tasks.rados.rados.0.smithi084.stdout:2: finishing write tid 1 to smithi08434663-2 2024-09-18T13:59:32.806 INFO:tasks.rados.rados.0.smithi084.stdout:2: finishing write tid 2 to smithi08434663-2 2024-09-18T13:59:32.806 INFO:tasks.rados.rados.0.smithi084.stdout:1: done (15 left) 2024-09-18T13:59:32.806 INFO:tasks.rados.rados.0.smithi084.stdout:17: write initial oid 17 2024-09-18T13:59:32.806 INFO:tasks.rados.rados.0.smithi084.stdout:17: seq_num 16 ranges {668912=636690,1903006=365906} 2024-09-18T13:59:32.810 INFO:tasks.rados.rados.0.smithi084.stdout:17: writing smithi08434663-17 from 668912 to 1305602 tid 1 2024-09-18T13:59:32.816 INFO:tasks.rados.rados.0.smithi084.stdout:17: writing smithi08434663-17 from 1903006 to 2268912 tid 2 2024-09-18T13:59:32.816 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:32.816 INFO:tasks.rados.rados.0.smithi084.stdout:2: finishing write tid 3 to smithi08434663-2 2024-09-18T13:59:32.816 INFO:tasks.rados.rados.0.smithi084.stdout:2: finishing write tid 4 to smithi08434663-2 2024-09-18T13:59:32.816 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T13:59:32.816 INFO:tasks.rados.rados.0.smithi084.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-09-18T13:59:32.817 INFO:tasks.rados.rados.0.smithi084.stdout:3: finishing write tid 1 to smithi08434663-3 2024-09-18T13:59:32.817 INFO:tasks.rados.rados.0.smithi084.stdout:3: finishing write tid 2 to smithi08434663-3 2024-09-18T13:59:32.817 INFO:tasks.rados.rados.0.smithi084.stdout:3: finishing write tid 3 to smithi08434663-3 2024-09-18T13:59:32.817 INFO:tasks.rados.rados.0.smithi084.stdout:3: finishing write tid 4 to smithi08434663-3 2024-09-18T13:59:32.817 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T13:59:32.817 INFO:tasks.rados.rados.0.smithi084.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-09-18T13:59:32.817 INFO:tasks.rados.rados.0.smithi084.stdout:4: finishing write tid 1 to smithi08434663-4 2024-09-18T13:59:32.817 INFO:tasks.rados.rados.0.smithi084.stdout:2: done (15 left) 2024-09-18T13:59:32.817 INFO:tasks.rados.rados.0.smithi084.stdout:3: done (14 left) 2024-09-18T13:59:32.817 INFO:tasks.rados.rados.0.smithi084.stdout:18: write initial oid 18 2024-09-18T13:59:32.817 INFO:tasks.rados.rados.0.smithi084.stdout:18: seq_num 17 ranges {685719=711939,1845135=440584} 2024-09-18T13:59:32.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.817+0000 7f2bc16ef640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T13:59:32.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.817+0000 7f2bc16ef640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T13:59:32.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.817+0000 7f2bc16ef640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:32.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.817+0000 7f2bc16ef640 1 -- 172.21.15.45:0/234538937 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2b9c005840 con 0x7f2bbc11ff00 2024-09-18T13:59:32.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.818+0000 7f2ba2ffd640 1 -- 172.21.15.45:0/234538937 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2682 (unknown 254746657 0 3988612391) 0x7f2bb0014cb0 con 0x7f2bbc11ff00 2024-09-18T13:59:32.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.818+0000 7f2ba2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T13:59:32.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.818+0000 7f2ba2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:32.819 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":32,"quorum":[0,1,2,3,5,6,7,8],"quorum_names":["a","f","b","g","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T13:59:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.820+0000 7f2bc16ef640 1 -- 172.21.15.45:0/234538937 >> v1:172.21.15.84:6812/31478 conn(0x7f2ba803baa0 legacy=0x7f2ba803df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.820+0000 7f2bc16ef640 10 monclient: shutdown 2024-09-18T13:59:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.820+0000 7f2bc16ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.820+0000 7f2bc16ef640 1 -- 172.21.15.45:0/234538937 >> v1:172.21.15.45:6792/0 conn(0x7f2bbc11ff00 legacy=0x7f2bbc1468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.820+0000 7f2bc16ef640 1 -- 172.21.15.45:0/234538937 shutdown_connections 2024-09-18T13:59:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.820+0000 7f2bc16ef640 1 -- 172.21.15.45:0/234538937 >> 172.21.15.45:0/234538937 conn(0x7f2bbc0a5bb0 msgr2=0x7f2bbc0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.820+0000 7f2bc16ef640 1 -- 172.21.15.45:0/234538937 shutdown_connections 2024-09-18T13:59:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.820+0000 7f2bc16ef640 1 -- 172.21.15.45:0/234538937 wait complete. 2024-09-18T13:59:32.824 INFO:tasks.rados.rados.0.smithi084.stdout:18: writing smithi08434663-18 from 685719 to 1397658 tid 1 2024-09-18T13:59:32.828 INFO:tasks.rados.rados.0.smithi084.stdout:18: writing smithi08434663-18 from 1845135 to 2285719 tid 2 2024-09-18T13:59:32.828 INFO:tasks.rados.rados.0.smithi084.stdout:19: write initial oid 19 2024-09-18T13:59:32.828 INFO:tasks.rados.rados.0.smithi084.stdout:19: seq_num 18 ranges {702526=787188,1903617=398909} 2024-09-18T13:59:32.832 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-09-18T13:59:32.832 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-09-18T13:59:32.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T13:59:32.836 INFO:tasks.rados.rados.0.smithi084.stdout:19: writing smithi08434663-19 from 702526 to 1489714 tid 1 2024-09-18T13:59:32.841 INFO:tasks.rados.rados.0.smithi084.stdout:19: writing smithi08434663-19 from 1903617 to 2302526 tid 2 2024-09-18T13:59:32.841 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:32.841 INFO:tasks.rados.rados.0.smithi084.stdout:4: finishing write tid 2 to smithi08434663-4 2024-09-18T13:59:32.841 INFO:tasks.rados.rados.0.smithi084.stdout:4: finishing write tid 3 to smithi08434663-4 2024-09-18T13:59:32.842 INFO:tasks.rados.rados.0.smithi084.stdout:4: finishing write tid 4 to smithi08434663-4 2024-09-18T13:59:32.842 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T13:59:32.842 INFO:tasks.rados.rados.0.smithi084.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-09-18T13:59:32.842 INFO:tasks.rados.rados.0.smithi084.stdout:4: done (15 left) 2024-09-18T13:59:32.842 INFO:tasks.rados.rados.0.smithi084.stdout:20: write initial oid 20 2024-09-18T13:59:32.842 INFO:tasks.rados.rados.0.smithi084.stdout:20: seq_num 19 ranges {719333=462437,1962099=357234} 2024-09-18T13:59:32.848 INFO:tasks.rados.rados.0.smithi084.stdout:20: writing smithi08434663-20 from 719333 to 1181770 tid 1 2024-09-18T13:59:32.854 INFO:tasks.rados.rados.0.smithi084.stdout:20: writing smithi08434663-20 from 1962099 to 2319333 tid 2 2024-09-18T13:59:32.854 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:32.854 INFO:tasks.rados.rados.0.smithi084.stdout:5: finishing write tid 1 to smithi08434663-5 2024-09-18T13:59:32.854 INFO:tasks.rados.rados.0.smithi084.stdout:5: finishing write tid 2 to smithi08434663-5 2024-09-18T13:59:32.854 INFO:tasks.rados.rados.0.smithi084.stdout:5: finishing write tid 3 to smithi08434663-5 2024-09-18T13:59:32.854 INFO:tasks.rados.rados.0.smithi084.stdout:5: finishing write tid 4 to smithi08434663-5 2024-09-18T13:59:32.854 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 3 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T13:59:32.855 INFO:tasks.rados.rados.0.smithi084.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-09-18T13:59:32.855 INFO:tasks.rados.rados.0.smithi084.stdout:6: finishing write tid 1 to smithi08434663-6 2024-09-18T13:59:32.855 INFO:tasks.rados.rados.0.smithi084.stdout:6: finishing write tid 2 to smithi08434663-6 2024-09-18T13:59:32.855 INFO:tasks.rados.rados.0.smithi084.stdout:5: done (15 left) 2024-09-18T13:59:32.855 INFO:tasks.rados.rados.0.smithi084.stdout:21: write initial oid 21 2024-09-18T13:59:32.855 INFO:tasks.rados.rados.0.smithi084.stdout:21: seq_num 20 ranges {736140=537686,2020581=315559} 2024-09-18T13:59:32.860 INFO:tasks.rados.rados.0.smithi084.stdout:21: writing smithi08434663-21 from 736140 to 1273826 tid 1 2024-09-18T13:59:32.867 INFO:tasks.rados.rados.0.smithi084.stdout:21: writing smithi08434663-21 from 2020581 to 2336140 tid 2 2024-09-18T13:59:32.867 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:32.867 INFO:tasks.rados.rados.0.smithi084.stdout:6: finishing write tid 3 to smithi08434663-6 2024-09-18T13:59:32.867 INFO:tasks.rados.rados.0.smithi084.stdout:6: finishing write tid 4 to smithi08434663-6 2024-09-18T13:59:32.867 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T13:59:32.867 INFO:tasks.rados.rados.0.smithi084.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-09-18T13:59:32.867 INFO:tasks.rados.rados.0.smithi084.stdout:7: finishing write tid 1 to smithi08434663-7 2024-09-18T13:59:32.867 INFO:tasks.rados.rados.0.smithi084.stdout:7: finishing write tid 2 to smithi08434663-7 2024-09-18T13:59:32.868 INFO:tasks.rados.rados.0.smithi084.stdout:7: finishing write tid 3 to smithi08434663-7 2024-09-18T13:59:32.868 INFO:tasks.rados.rados.0.smithi084.stdout:7: finishing write tid 4 to smithi08434663-7 2024-09-18T13:59:32.868 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 6 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T13:59:32.868 INFO:tasks.rados.rados.0.smithi084.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-09-18T13:59:32.868 INFO:tasks.rados.rados.0.smithi084.stdout:8: finishing write tid 1 to smithi08434663-8 2024-09-18T13:59:32.868 INFO:tasks.rados.rados.0.smithi084.stdout:8: finishing write tid 2 to smithi08434663-8 2024-09-18T13:59:32.868 INFO:tasks.rados.rados.0.smithi084.stdout:6: done (15 left) 2024-09-18T13:59:32.868 INFO:tasks.rados.rados.0.smithi084.stdout:7: done (14 left) 2024-09-18T13:59:32.868 INFO:tasks.rados.rados.0.smithi084.stdout:22: write initial oid 22 2024-09-18T13:59:32.868 INFO:tasks.rados.rados.0.smithi084.stdout:22: seq_num 21 ranges {752947=612935,1962710=390237} 2024-09-18T13:59:32.874 INFO:tasks.rados.rados.0.smithi084.stdout:22: writing smithi08434663-22 from 752947 to 1365882 tid 1 2024-09-18T13:59:32.879 INFO:tasks.rados.rados.0.smithi084.stdout:22: writing smithi08434663-22 from 1962710 to 2352947 tid 2 2024-09-18T13:59:32.879 INFO:tasks.rados.rados.0.smithi084.stdout:23: write initial oid 23 2024-09-18T13:59:32.879 INFO:tasks.rados.rados.0.smithi084.stdout:23: seq_num 22 ranges {769754=688184,2021192=348562} 2024-09-18T13:59:32.887 INFO:tasks.rados.rados.0.smithi084.stdout:23: writing smithi08434663-23 from 769754 to 1457938 tid 1 2024-09-18T13:59:32.893 INFO:tasks.rados.rados.0.smithi084.stdout:23: writing smithi08434663-23 from 2021192 to 2369754 tid 2 2024-09-18T13:59:32.893 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:32.893 INFO:tasks.rados.rados.0.smithi084.stdout:8: finishing write tid 3 to smithi08434663-8 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:8: finishing write tid 4 to smithi08434663-8 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 3 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:9: finishing write tid 1 to smithi08434663-9 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:9: finishing write tid 2 to smithi08434663-9 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:9: finishing write tid 3 to smithi08434663-9 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:9: finishing write tid 4 to smithi08434663-9 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 3 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:10: finishing write tid 1 to smithi08434663-10 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:10: finishing write tid 2 to smithi08434663-10 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:10: finishing write tid 3 to smithi08434663-10 2024-09-18T13:59:32.894 INFO:tasks.rados.rados.0.smithi084.stdout:10: finishing write tid 4 to smithi08434663-10 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 6 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:11: finishing write tid 1 to smithi08434663-11 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:11: finishing write tid 2 to smithi08434663-11 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:11: finishing write tid 3 to smithi08434663-11 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:11: finishing write tid 4 to smithi08434663-11 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 3 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:12: finishing write tid 1 to smithi08434663-12 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:12: finishing write tid 2 to smithi08434663-12 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:12: finishing write tid 3 to smithi08434663-12 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:8: done (15 left) 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:9: done (14 left) 2024-09-18T13:59:32.895 INFO:tasks.rados.rados.0.smithi084.stdout:10: done (13 left) 2024-09-18T13:59:32.896 INFO:tasks.rados.rados.0.smithi084.stdout:11: done (12 left) 2024-09-18T13:59:32.896 INFO:tasks.rados.rados.0.smithi084.stdout:24: write initial oid 24 2024-09-18T13:59:32.896 INFO:tasks.rados.rados.0.smithi084.stdout:24: seq_num 23 ranges {786561=479786,1796027=467664} 2024-09-18T13:59:32.899 INFO:tasks.rados.rados.0.smithi084.stdout:24: writing smithi08434663-24 from 786561 to 1266347 tid 1 2024-09-18T13:59:32.904 INFO:tasks.rados.rados.0.smithi084.stdout:24: writing smithi08434663-24 from 1796027 to 2263691 tid 2 2024-09-18T13:59:32.904 INFO:tasks.rados.rados.0.smithi084.stdout:25: write initial oid 25 2024-09-18T13:59:32.904 INFO:tasks.rados.rados.0.smithi084.stdout:25: seq_num 24 ranges {403368=555035,1454509=727675} 2024-09-18T13:59:32.909 INFO:tasks.rados.rados.0.smithi084.stdout:25: writing smithi08434663-25 from 403368 to 958403 tid 1 2024-09-18T13:59:32.916 INFO:tasks.rados.rados.0.smithi084.stdout:25: writing smithi08434663-25 from 1454509 to 2182184 tid 2 2024-09-18T13:59:32.916 INFO:tasks.rados.rados.0.smithi084.stdout:26: write initial oid 26 2024-09-18T13:59:32.916 INFO:tasks.rados.rados.0.smithi084.stdout:26: seq_num 25 ranges {420175=630284,1796638=471333} 2024-09-18T13:59:32.922 INFO:tasks.rados.rados.0.smithi084.stdout:26: writing smithi08434663-26 from 420175 to 1050459 tid 1 2024-09-18T13:59:32.929 INFO:tasks.rados.rados.0.smithi084.stdout:26: writing smithi08434663-26 from 1796638 to 2267971 tid 2 2024-09-18T13:59:32.929 INFO:tasks.rados.rados.0.smithi084.stdout:27: write initial oid 27 2024-09-18T13:59:32.929 INFO:tasks.rados.rados.0.smithi084.stdout:27: seq_num 26 ranges {436982=705533,1855120=581862} 2024-09-18T13:59:32.935 INFO:tasks.rados.rados.0.smithi084.stdout:27: writing smithi08434663-27 from 436982 to 1142515 tid 1 2024-09-18T13:59:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.936+0000 7f529eec0640 10 monclient: get_monmap_and_config 2024-09-18T13:59:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.936+0000 7f529eec0640 10 monclient: build_initial_monmap 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.936+0000 7f529eec0640 10 monclient: monmap: 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:32.937338+0000 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:32.937338+0000 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:32.938 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.946 INFO:tasks.rados.rados.0.smithi084.stdout:27: writing smithi08434663-27 from 1855120 to 2436982 tid 2 2024-09-18T13:59:32.947 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:32.947 INFO:tasks.rados.rados.0.smithi084.stdout:12: finishing write tid 4 to smithi08434663-12 2024-09-18T13:59:32.947 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T13:59:32.947 INFO:tasks.rados.rados.0.smithi084.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-09-18T13:59:32.947 INFO:tasks.rados.rados.0.smithi084.stdout:13: finishing write tid 1 to smithi08434663-13 2024-09-18T13:59:32.947 INFO:tasks.rados.rados.0.smithi084.stdout:13: finishing write tid 2 to smithi08434663-13 2024-09-18T13:59:32.947 INFO:tasks.rados.rados.0.smithi084.stdout:13: finishing write tid 3 to smithi08434663-13 2024-09-18T13:59:32.947 INFO:tasks.rados.rados.0.smithi084.stdout:13: finishing write tid 4 to smithi08434663-13 2024-09-18T13:59:32.947 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 3 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T13:59:32.947 INFO:tasks.rados.rados.0.smithi084.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-09-18T13:59:32.947 INFO:tasks.rados.rados.0.smithi084.stdout:14: finishing write tid 1 to smithi08434663-14 2024-09-18T13:59:32.947 INFO:tasks.rados.rados.0.smithi084.stdout:14: finishing write tid 2 to smithi08434663-14 2024-09-18T13:59:32.948 INFO:tasks.rados.rados.0.smithi084.stdout:14: finishing write tid 3 to smithi08434663-14 2024-09-18T13:59:32.948 INFO:tasks.rados.rados.0.smithi084.stdout:14: finishing write tid 4 to smithi08434663-14 2024-09-18T13:59:32.948 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 9 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T13:59:32.948 INFO:tasks.rados.rados.0.smithi084.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-09-18T13:59:32.948 INFO:tasks.rados.rados.0.smithi084.stdout:15: finishing write tid 1 to smithi08434663-15 2024-09-18T13:59:32.948 INFO:tasks.rados.rados.0.smithi084.stdout:12: done (15 left) 2024-09-18T13:59:32.948 INFO:tasks.rados.rados.0.smithi084.stdout:13: done (14 left) 2024-09-18T13:59:32.948 INFO:tasks.rados.rados.0.smithi084.stdout:14: done (13 left) 2024-09-18T13:59:32.948 INFO:tasks.rados.rados.0.smithi084.stdout:28: write initial oid 28 2024-09-18T13:59:32.948 INFO:tasks.rados.rados.0.smithi084.stdout:28: seq_num 27 ranges {453789=780782,1913602=475002} 2024-09-18T13:59:32.951 INFO:tasks.rados.rados.0.smithi084.stdout:28: writing smithi08434663-28 from 453789 to 1234571 tid 1 2024-09-18T13:59:32.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 1 Processor -- start 2024-09-18T13:59:32.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 1 -- start start 2024-09-18T13:59:32.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 10 monclient: init 2024-09-18T13:59:32.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 10 monclient: _add_conns ranks=[8,2,3,1,7,4,6,0,5] 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 10 monclient(hunting): picked mon.noname-e con 0x7f529809f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 10 monclient(hunting): picked mon.noname-b con 0x7f529809fba0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 10 monclient(hunting): picked mon.noname-g con 0x7f529809ff70 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529805ed70 con 0x7f529809fba0 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529809dfc0 con 0x7f529809ff70 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529809dce0 con 0x7f529809f7d0 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529eec0640 10 monclient(hunting): authenticate will time out at 788.275821s 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529cc5c640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f529809f7d0 0x7f5298132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:58702/0 (socket says 172.21.15.45:58702) 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f529cc5c640 1 -- 172.21.15.45:0/768360470 learned_addr learned my addr 172.21.15.45:0/768360470 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.937+0000 7f5297fff640 1 --1- 172.21.15.45:0/768360470 >> v1:172.21.15.45:6790/0 conn(0x7f529809fba0 0x7f5298137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:45866/0 (socket says 172.21.15.45:45866) 2024-09-18T13:59:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60225931 0 0) 0x7f529809dce0 con 0x7f529809f7d0 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient(hunting): my global_id is 4463 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52840034e0 con 0x7f529809f7d0 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82457483 0 0) 0x7f529805ed70 con 0x7f529809fba0 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient(hunting): my global_id is 4520 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f529809dce0 con 0x7f529809fba0 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531309110 0 0) 0x7f529809dfc0 con 0x7f529809ff70 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient(hunting): my global_id is 4593 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f529805ed70 con 0x7f529809ff70 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4226116041 0 0) 0x7f52840034e0 con 0x7f529809f7d0 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 >> v1:172.21.15.84:6790/0 conn(0x7f529809ff70 legacy=0x7f529813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 >> v1:172.21.15.45:6790/0 conn(0x7f529809fba0 legacy=0x7f5298137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient: _finish_hunting 0 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 monclient: found mon.noname-e 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52980a1c00 con 0x7f529809f7d0 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient: _finish_auth 0 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f529eec0640 5 monclient: authenticate success, global_id 4463 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5280001830 con 0x7f529809f7d0 2024-09-18T13:59:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient: dump: 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f529eec0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5280001990 con 0x7f529809f7d0 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 1 -- 172.21.15.45:0/768360470 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5280004ba0 con 0x7f529809f7d0 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f529eec0640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f529eec0640 4 set_mon_vals no callback set 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:32.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.938+0000 7f52977fe640 10 monclient: dump: 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.939+0000 7f529eec0640 10 monclient: shutdown 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.939+0000 7f529eec0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.939+0000 7f529eec0640 1 -- 172.21.15.45:0/768360470 >> v1:172.21.15.45:6793/0 conn(0x7f529809f7d0 legacy=0x7f5298132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.939+0000 7f529eec0640 1 -- 172.21.15.45:0/768360470 shutdown_connections 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.939+0000 7f529eec0640 1 -- 172.21.15.45:0/768360470 >> 172.21.15.45:0/768360470 conn(0x7f52980630b0 msgr2=0x7f52980634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.939+0000 7f529eec0640 1 -- 172.21.15.45:0/768360470 shutdown_connections 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.939+0000 7f529eec0640 1 -- 172.21.15.45:0/768360470 wait complete. 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.939+0000 7f529eec0640 10 monclient: build_initial_monmap 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.939+0000 7f529eec0640 10 monclient: monmap: 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:32.959 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:32.961 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.939+0000 7f529eec0640 1 Processor -- start 2024-09-18T13:59:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529eec0640 1 -- start start 2024-09-18T13:59:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529eec0640 10 monclient: init 2024-09-18T13:59:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529eec0640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529eec0640 10 monclient: _add_conns ranks=[2,7,0,3,4,8,5,1,6] 2024-09-18T13:59:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529eec0640 10 monclient(hunting): picked mon.noname-b con 0x7f529809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529eec0640 10 monclient(hunting): picked mon.noname-i con 0x7f529809fba0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529eec0640 10 monclient(hunting): picked mon.noname-a con 0x7f529809ff70 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529eec0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529809d310 con 0x7f529809ff70 2024-09-18T13:59:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529eec0640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52980a2560 con 0x7f529809f7d0 2024-09-18T13:59:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529eec0640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5298122480 con 0x7f529809fba0 2024-09-18T13:59:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529eec0640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529cc5c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f529809f7d0 0x7f5298141840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:45872/0 (socket says 172.21.15.45:45872) 2024-09-18T13:59:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529eec0640 10 monclient(hunting): authenticate will time out at 788.278763s 2024-09-18T13:59:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f529cc5c640 1 -- 172.21.15.45:0/1590580667 learned_addr learned my addr 172.21.15.45:0/1590580667 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.940+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064526392 0 0) 0x7f52980a2560 con 0x7f529809f7d0 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): my global_id is 4529 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52780034e0 con 0x7f529809f7d0 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429065200 0 0) 0x7f529809d310 con 0x7f529809ff70 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): my global_id is 4527 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52980a2560 con 0x7f529809ff70 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386648418 0 0) 0x7f5288001d00 con 0x7f529809fba0 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:32.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): my global_id is 4534 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f529809d310 con 0x7f529809fba0 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1060948806 0 0) 0x7f52780034e0 con 0x7f529809f7d0 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5288001d00 con 0x7f529809f7d0 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f52800047a0 con 0x7f529809f7d0 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): dump: 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.964 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.965 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3392284188 0 0) 0x7f529809d310 con 0x7f529809fba0 2024-09-18T13:59:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f52780034e0 con 0x7f529809fba0 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2146129375 0 0) 0x7f52980a2560 con 0x7f529809ff70 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f529809d310 con 0x7f529809ff70 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5288001f90 con 0x7f529809fba0 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): dump: 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f52900018c0 con 0x7f529809ff70 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient(hunting): dump: 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.969 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1704081844 0 0) 0x7f5288001d00 con 0x7f529809f7d0 2024-09-18T13:59:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 >> v1:172.21.15.84:6792/0 conn(0x7f529809fba0 legacy=0x7f5298131a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 >> v1:172.21.15.45:6789/0 conn(0x7f529809ff70 legacy=0x7f5298149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient: _finish_hunting 0 2024-09-18T13:59:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 monclient: found mon.b 2024-09-18T13:59:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5298121e70 con 0x7f529809f7d0 2024-09-18T13:59:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient: _finish_auth 0 2024-09-18T13:59:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f529eec0640 5 monclient: authenticate success, global_id 4529 2024-09-18T13:59:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f529eec0640 10 monclient: _renew_subs 2024-09-18T13:59:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f529eec0640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f529eec0640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5298121b70 con 0x7f529809f7d0 2024-09-18T13:59:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f529eec0640 10 monclient: _renew_subs 2024-09-18T13:59:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f529eec0640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f529eec0640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5298120c10 con 0x7f529809f7d0 2024-09-18T13:59:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5280004e60 con 0x7f529809f7d0 2024-09-18T13:59:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5280005310 con 0x7f529809f7d0 2024-09-18T13:59:32.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5296ffd640 4 set_mon_vals no callback set 2024-09-18T13:59:32.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:32.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.941+0000 7f5294ff9640 10 monclient: dump: 2024-09-18T13:59:32.971 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.971 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.971 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.971 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.971 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.971 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.971 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f52800102e0 con 0x7f529809f7d0 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f529eec0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f529eec0640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f527c006580 con 0x7f527c002830 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(20..20 src has 1..20) v4 ==== 3867+0+0 (unknown 541114214 0 0) 0x7f5280046c20 con 0x7f529809f7d0 2024-09-18T13:59:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78837632 0 0) 0x7f527c006580 con 0x7f527c002830 2024-09-18T13:59:32.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:32.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:32.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 10 monclient(hunting): my global_id is 4536 2024-09-18T13:59:32.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5288001d00 con 0x7f527c002830 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f529d45d640 10 monclient: get_auth_request con 0x7f527803aa50 auth_method 0 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1287756742 0 0) 0x7f5288001d00 con 0x7f527c002830 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f527c006580 con 0x7f527c002830 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5288005050 con 0x7f527c002830 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.942+0000 7f5294ff9640 10 monclient: dump: 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.974 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.975 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.975 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.975 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.975 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.975 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.975 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.975 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.975 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.975 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.975 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.943+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 311334606 0 0) 0x7f527c006580 con 0x7f527c002830 2024-09-18T13:59:32.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.943+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f527c00a0a0 con 0x7f527c002830 2024-09-18T13:59:32.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.943+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f52880053e0 con 0x7f527c002830 2024-09-18T13:59:32.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.943+0000 7f5294ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:32.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.943+0000 7f5294ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:32.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.943+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 >> v1:172.21.15.45:6789/0 conn(0x7f527c002830 legacy=0x7f527c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.943+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 >> v1:172.21.15.45:6789/0 conn(0x7f527c002830 legacy=0x7f527c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:32.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f529eec0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:32.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f529eec0640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f527c009610 con 0x7f527c008ff0 2024-09-18T13:59:32.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281524741 0 0) 0x7f527c009610 con 0x7f527c008ff0 2024-09-18T13:59:32.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:32.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:32.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 10 monclient(hunting): my global_id is 4545 2024-09-18T13:59:32.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f527c006580 con 0x7f527c008ff0 2024-09-18T13:59:32.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3871209380 0 0) 0x7f527c006580 con 0x7f527c008ff0 2024-09-18T13:59:32.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f527c009610 con 0x7f527c008ff0 2024-09-18T13:59:32.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5288005be0 con 0x7f527c008ff0 2024-09-18T13:59:32.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:32.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 10 monclient: dump: 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:32.979 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:32.979 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:32.979 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:32.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2616782282 0 0) 0x7f527c009610 con 0x7f527c008ff0 2024-09-18T13:59:32.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f527c0098d0 con 0x7f527c008ff0 2024-09-18T13:59:32.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.949+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7375 (unknown 0 0 3497517651) 0x7f52880083d0 con 0x7f527c008ff0 2024-09-18T13:59:32.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f5294ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:32.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f5294ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:32.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 >> v1:172.21.15.45:6789/0 conn(0x7f527c008ff0 legacy=0x7f527c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:32.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f5294ff9640 1 -- 172.21.15.45:0/1590580667 >> v1:172.21.15.45:6789/0 conn(0x7f527c008ff0 legacy=0x7f527c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:32.980 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 32, 2024-09-18T13:59:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T13:59:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:33.000 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f529eec0640 1 -- 172.21.15.45:0/1590580667 >> v1:172.21.15.84:6812/31478 conn(0x7f527803aa50 legacy=0x7f527803cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f529eec0640 10 monclient: shutdown 2024-09-18T13:59:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f529eec0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f529eec0640 1 -- 172.21.15.45:0/1590580667 >> v1:172.21.15.45:6790/0 conn(0x7f529809f7d0 legacy=0x7f5298141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f529d45d640 1 -- 172.21.15.45:0/1590580667 reap_dead start 2024-09-18T13:59:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f529d45d640 1 -- 172.21.15.45:0/1590580667 reap_dead start 2024-09-18T13:59:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f529eec0640 1 -- 172.21.15.45:0/1590580667 shutdown_connections 2024-09-18T13:59:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f529eec0640 1 -- 172.21.15.45:0/1590580667 >> 172.21.15.45:0/1590580667 conn(0x7f52980630b0 msgr2=0x7f52980a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f529eec0640 1 -- 172.21.15.45:0/1590580667 shutdown_connections 2024-09-18T13:59:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:32.950+0000 7f529eec0640 1 -- 172.21.15.45:0/1590580667 wait complete. 2024-09-18T13:59:33.004 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T13:59:33.004 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-09-18T13:59:33.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T13:59:33.007 INFO:tasks.rados.rados.0.smithi084.stdout:28: writing smithi08434663-28 from 1913602 to 2388604 tid 2 2024-09-18T13:59:33.007 INFO:tasks.rados.rados.0.smithi084.stdout:29: write initial oid 29 2024-09-18T13:59:33.007 INFO:tasks.rados.rados.0.smithi084.stdout:29: seq_num 28 ranges {470596=456031,1572084=618660} 2024-09-18T13:59:33.007 INFO:tasks.rados.rados.0.smithi084.stdout:29: writing smithi08434663-29 from 470596 to 926627 tid 1 2024-09-18T13:59:33.007 INFO:tasks.rados.rados.0.smithi084.stdout:29: writing smithi08434663-29 from 1572084 to 2190744 tid 2 2024-09-18T13:59:33.007 INFO:tasks.rados.rados.0.smithi084.stdout:30: write initial oid 30 2024-09-18T13:59:33.007 INFO:tasks.rados.rados.0.smithi084.stdout:30: seq_num 29 ranges {487403=531280,1514213=478671} 2024-09-18T13:59:33.008 INFO:tasks.rados.rados.0.smithi084.stdout:30: writing smithi08434663-30 from 487403 to 1018683 tid 1 2024-09-18T13:59:33.008 INFO:tasks.rados.rados.0.smithi084.stdout:30: writing smithi08434663-30 from 1514213 to 1992884 tid 2 2024-09-18T13:59:33.008 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.008 INFO:tasks.rados.rados.0.smithi084.stdout:15: finishing write tid 2 to smithi08434663-15 2024-09-18T13:59:33.008 INFO:tasks.rados.rados.0.smithi084.stdout:15: finishing write tid 3 to smithi08434663-15 2024-09-18T13:59:33.008 INFO:tasks.rados.rados.0.smithi084.stdout:15: finishing write tid 4 to smithi08434663-15 2024-09-18T13:59:33.008 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 6 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T13:59:33.008 INFO:tasks.rados.rados.0.smithi084.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-09-18T13:59:33.008 INFO:tasks.rados.rados.0.smithi084.stdout:16: finishing write tid 1 to smithi08434663-16 2024-09-18T13:59:33.008 INFO:tasks.rados.rados.0.smithi084.stdout:15: done (15 left) 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout:31: write initial oid 31 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout:31: seq_num 30 ranges {504210=606529,1572695=622329} 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout:31: writing smithi08434663-31 from 504210 to 1110739 tid 1 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout:31: writing smithi08434663-31 from 1572695 to 2195024 tid 2 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout:16: finishing write tid 2 to smithi08434663-16 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout:16: finishing write tid 3 to smithi08434663-16 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout:16: finishing write tid 4 to smithi08434663-16 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 9 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout:17: finishing write tid 1 to smithi08434663-17 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout:17: finishing write tid 2 to smithi08434663-17 2024-09-18T13:59:33.009 INFO:tasks.rados.rados.0.smithi084.stdout:17: finishing write tid 3 to smithi08434663-17 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:17: finishing write tid 4 to smithi08434663-17 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 6 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:18: finishing write tid 1 to smithi08434663-18 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:18: finishing write tid 2 to smithi08434663-18 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:16: done (15 left) 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:17: done (14 left) 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:32: write initial oid 32 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:32: seq_num 31 ranges {521017=798131,1747530=482340} 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:32: writing smithi08434663-32 from 521017 to 1319148 tid 1 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:32: writing smithi08434663-32 from 1747530 to 2229870 tid 2 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:33: write initial oid 33 2024-09-18T13:59:33.010 INFO:tasks.rados.rados.0.smithi084.stdout:33: seq_num 32 ranges {537824=473380,1806012=625998} 2024-09-18T13:59:33.011 INFO:tasks.rados.rados.0.smithi084.stdout:33: writing smithi08434663-33 from 537824 to 1011204 tid 1 2024-09-18T13:59:33.017 INFO:tasks.rados.rados.0.smithi084.stdout:33: writing smithi08434663-33 from 1806012 to 2432010 tid 2 2024-09-18T13:59:33.017 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.017 INFO:tasks.rados.rados.0.smithi084.stdout:18: finishing write tid 3 to smithi08434663-18 2024-09-18T13:59:33.017 INFO:tasks.rados.rados.0.smithi084.stdout:18: finishing write tid 4 to smithi08434663-18 2024-09-18T13:59:33.018 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 6 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T13:59:33.018 INFO:tasks.rados.rados.0.smithi084.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-09-18T13:59:33.018 INFO:tasks.rados.rados.0.smithi084.stdout:18: done (15 left) 2024-09-18T13:59:33.018 INFO:tasks.rados.rados.0.smithi084.stdout:34: write initial oid 34 2024-09-18T13:59:33.018 INFO:tasks.rados.rados.0.smithi084.stdout:34: seq_num 33 ranges {554631=548629,1748141=486009} 2024-09-18T13:59:33.023 INFO:tasks.rados.rados.0.smithi084.stdout:34: writing smithi08434663-34 from 554631 to 1103260 tid 1 2024-09-18T13:59:33.029 INFO:tasks.rados.rados.0.smithi084.stdout:34: writing smithi08434663-34 from 1748141 to 2234150 tid 2 2024-09-18T13:59:33.030 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.030 INFO:tasks.rados.rados.0.smithi084.stdout:19: finishing write tid 1 to smithi08434663-19 2024-09-18T13:59:33.030 INFO:tasks.rados.rados.0.smithi084.stdout:19: finishing write tid 2 to smithi08434663-19 2024-09-18T13:59:33.030 INFO:tasks.rados.rados.0.smithi084.stdout:19: finishing write tid 3 to smithi08434663-19 2024-09-18T13:59:33.030 INFO:tasks.rados.rados.0.smithi084.stdout:19: finishing write tid 4 to smithi08434663-19 2024-09-18T13:59:33.030 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 12 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T13:59:33.030 INFO:tasks.rados.rados.0.smithi084.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-09-18T13:59:33.030 INFO:tasks.rados.rados.0.smithi084.stdout:20: finishing write tid 1 to smithi08434663-20 2024-09-18T13:59:33.030 INFO:tasks.rados.rados.0.smithi084.stdout:19: done (15 left) 2024-09-18T13:59:33.030 INFO:tasks.rados.rados.0.smithi084.stdout:35: write initial oid 35 2024-09-18T13:59:33.030 INFO:tasks.rados.rados.0.smithi084.stdout:35: seq_num 34 ranges {571438=623878,1806623=629667} 2024-09-18T13:59:33.035 INFO:tasks.rados.rados.0.smithi084.stdout:35: writing smithi08434663-35 from 571438 to 1195316 tid 1 2024-09-18T13:59:33.042 INFO:tasks.rados.rados.0.smithi084.stdout:35: writing smithi08434663-35 from 1806623 to 2436290 tid 2 2024-09-18T13:59:33.042 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.042 INFO:tasks.rados.rados.0.smithi084.stdout:20: finishing write tid 2 to smithi08434663-20 2024-09-18T13:59:33.043 INFO:tasks.rados.rados.0.smithi084.stdout:20: finishing write tid 3 to smithi08434663-20 2024-09-18T13:59:33.043 INFO:tasks.rados.rados.0.smithi084.stdout:20: finishing write tid 4 to smithi08434663-20 2024-09-18T13:59:33.043 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 6 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T13:59:33.043 INFO:tasks.rados.rados.0.smithi084.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-09-18T13:59:33.043 INFO:tasks.rados.rados.0.smithi084.stdout:20: done (15 left) 2024-09-18T13:59:33.043 INFO:tasks.rados.rados.0.smithi084.stdout:36: write initial oid 36 2024-09-18T13:59:33.043 INFO:tasks.rados.rados.0.smithi084.stdout:36: seq_num 35 ranges {588245=699127,1865105=489678} 2024-09-18T13:59:33.049 INFO:tasks.rados.rados.0.smithi084.stdout:36: writing smithi08434663-36 from 588245 to 1287372 tid 1 2024-09-18T13:59:33.055 INFO:tasks.rados.rados.0.smithi084.stdout:36: writing smithi08434663-36 from 1865105 to 2354783 tid 2 2024-09-18T13:59:33.055 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.055 INFO:tasks.rados.rados.0.smithi084.stdout:21: finishing write tid 1 to smithi08434663-21 2024-09-18T13:59:33.055 INFO:tasks.rados.rados.0.smithi084.stdout:21: finishing write tid 2 to smithi08434663-21 2024-09-18T13:59:33.055 INFO:tasks.rados.rados.0.smithi084.stdout:21: finishing write tid 3 to smithi08434663-21 2024-09-18T13:59:33.056 INFO:tasks.rados.rados.0.smithi084.stdout:21: finishing write tid 4 to smithi08434663-21 2024-09-18T13:59:33.056 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 3 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T13:59:33.056 INFO:tasks.rados.rados.0.smithi084.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-09-18T13:59:33.056 INFO:tasks.rados.rados.0.smithi084.stdout:22: finishing write tid 1 to smithi08434663-22 2024-09-18T13:59:33.056 INFO:tasks.rados.rados.0.smithi084.stdout:21: done (15 left) 2024-09-18T13:59:33.056 INFO:tasks.rados.rados.0.smithi084.stdout:37: write initial oid 37 2024-09-18T13:59:33.056 INFO:tasks.rados.rados.0.smithi084.stdout:37: seq_num 36 ranges {605052=774376,1923587=633336} 2024-09-18T13:59:33.062 INFO:tasks.rados.rados.0.smithi084.stdout:37: writing smithi08434663-37 from 605052 to 1379428 tid 1 2024-09-18T13:59:33.069 INFO:tasks.rados.rados.0.smithi084.stdout:37: writing smithi08434663-37 from 1923587 to 2556923 tid 2 2024-09-18T13:59:33.070 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.070 INFO:tasks.rados.rados.0.smithi084.stdout:22: finishing write tid 2 to smithi08434663-22 2024-09-18T13:59:33.070 INFO:tasks.rados.rados.0.smithi084.stdout:22: finishing write tid 3 to smithi08434663-22 2024-09-18T13:59:33.070 INFO:tasks.rados.rados.0.smithi084.stdout:22: finishing write tid 4 to smithi08434663-22 2024-09-18T13:59:33.070 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 9 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T13:59:33.070 INFO:tasks.rados.rados.0.smithi084.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-09-18T13:59:33.070 INFO:tasks.rados.rados.0.smithi084.stdout:23: finishing write tid 1 to smithi08434663-23 2024-09-18T13:59:33.070 INFO:tasks.rados.rados.0.smithi084.stdout:23: finishing write tid 2 to smithi08434663-23 2024-09-18T13:59:33.070 INFO:tasks.rados.rados.0.smithi084.stdout:23: finishing write tid 3 to smithi08434663-23 2024-09-18T13:59:33.070 INFO:tasks.rados.rados.0.smithi084.stdout:23: finishing write tid 4 to smithi08434663-23 2024-09-18T13:59:33.070 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 9 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T13:59:33.070 INFO:tasks.rados.rados.0.smithi084.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-09-18T13:59:33.071 INFO:tasks.rados.rados.0.smithi084.stdout:22: done (15 left) 2024-09-18T13:59:33.071 INFO:tasks.rados.rados.0.smithi084.stdout:23: done (14 left) 2024-09-18T13:59:33.071 INFO:tasks.rados.rados.0.smithi084.stdout:38: write initial oid 38 2024-09-18T13:59:33.071 INFO:tasks.rados.rados.0.smithi084.stdout:38: seq_num 37 ranges {621859=449625,1865716=756143} 2024-09-18T13:59:33.075 INFO:tasks.rados.rados.0.smithi084.stdout:38: writing smithi08434663-38 from 621859 to 1071484 tid 1 2024-09-18T13:59:33.083 INFO:tasks.rados.rados.0.smithi084.stdout:38: writing smithi08434663-38 from 1865716 to 2621859 tid 2 2024-09-18T13:59:33.083 INFO:tasks.rados.rados.0.smithi084.stdout:39: write initial oid 39 2024-09-18T13:59:33.083 INFO:tasks.rados.rados.0.smithi084.stdout:39: seq_num 38 ranges {638666=524874,1924198=637005} 2024-09-18T13:59:33.089 INFO:tasks.rados.rados.0.smithi084.stdout:39: writing smithi08434663-39 from 638666 to 1163540 tid 1 2024-09-18T13:59:33.098 INFO:tasks.rados.rados.0.smithi084.stdout:39: writing smithi08434663-39 from 1924198 to 2561203 tid 2 2024-09-18T13:59:33.098 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.098 INFO:tasks.rados.rados.0.smithi084.stdout:24: finishing write tid 1 to smithi08434663-24 2024-09-18T13:59:33.098 INFO:tasks.rados.rados.0.smithi084.stdout:24: finishing write tid 2 to smithi08434663-24 2024-09-18T13:59:33.098 INFO:tasks.rados.rados.0.smithi084.stdout:24: finishing write tid 3 to smithi08434663-24 2024-09-18T13:59:33.098 INFO:tasks.rados.rados.0.smithi084.stdout:24: finishing write tid 4 to smithi08434663-24 2024-09-18T13:59:33.098 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 3 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T13:59:33.098 INFO:tasks.rados.rados.0.smithi084.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-09-18T13:59:33.098 INFO:tasks.rados.rados.0.smithi084.stdout:25: finishing write tid 1 to smithi08434663-25 2024-09-18T13:59:33.099 INFO:tasks.rados.rados.0.smithi084.stdout:25: finishing write tid 2 to smithi08434663-25 2024-09-18T13:59:33.099 INFO:tasks.rados.rados.0.smithi084.stdout:25: finishing write tid 3 to smithi08434663-25 2024-09-18T13:59:33.099 INFO:tasks.rados.rados.0.smithi084.stdout:25: finishing write tid 4 to smithi08434663-25 2024-09-18T13:59:33.099 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 6 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T13:59:33.099 INFO:tasks.rados.rados.0.smithi084.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-09-18T13:59:33.099 INFO:tasks.rados.rados.0.smithi084.stdout:24: done (15 left) 2024-09-18T13:59:33.099 INFO:tasks.rados.rados.0.smithi084.stdout:25: done (14 left) 2024-09-18T13:59:33.099 INFO:tasks.rados.rados.0.smithi084.stdout:40: write initial oid 40 2024-09-18T13:59:33.099 INFO:tasks.rados.rados.0.smithi084.stdout:40: seq_num 39 ranges {655473=716476,2099033=556440} 2024-09-18T13:59:33.104 INFO:tasks.rados.rados.0.smithi084.stdout:40: writing smithi08434663-40 from 655473 to 1371949 tid 1 2024-09-18T13:59:33.112 INFO:tasks.rados.rados.0.smithi084.stdout:40: writing smithi08434663-40 from 2099033 to 2655473 tid 2 2024-09-18T13:59:33.112 INFO:tasks.rados.rados.0.smithi084.stdout:41: write initial oid 41 2024-09-18T13:59:33.112 INFO:tasks.rados.rados.0.smithi084.stdout:41: seq_num 40 ranges {672280=791725,2157515=514765} 2024-09-18T13:59:33.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.114+0000 7f5522f5d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:33.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.114+0000 7f5522f5d640 10 monclient: build_initial_monmap 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.114+0000 7f5522f5d640 10 monclient: monmap: 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:33.115441+0000 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:33.115441+0000 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:33.116 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:33.116 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:33.116 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:33.116 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.123 INFO:tasks.rados.rados.0.smithi084.stdout:41: writing smithi08434663-41 from 672280 to 1464005 tid 1 2024-09-18T13:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 1 Processor -- start 2024-09-18T13:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 1 -- start start 2024-09-18T13:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 10 monclient: init 2024-09-18T13:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 10 monclient: _add_conns ranks=[5,4,7,8,2,0,1,3,6] 2024-09-18T13:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 10 monclient(hunting): picked mon.noname-h con 0x7f551c06e810 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 10 monclient(hunting): picked mon.noname-c con 0x7f551c0a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 10 monclient(hunting): picked mon.noname-i con 0x7f551c0a1080 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551c05ed70 con 0x7f551c0a0cb0 2024-09-18T13:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551c097620 con 0x7f551c06e810 2024-09-18T13:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551c097340 con 0x7f551c0a1080 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5522f5d640 10 monclient(hunting): authenticate will time out at 788.453803s 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f551bfff640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f551c0a0cb0 legacy=0x7f551c137810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f5520cf9640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f551c06e810 0x7f551c0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:38338/0 (socket says 172.21.15.45:38338) 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.115+0000 7f55214fa640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f551c0a1080 0x7f551c13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:41562/0 (socket says 172.21.15.45:41562) 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f5520cf9640 1 -- 172.21.15.45:0/2321954138 learned_addr learned my addr 172.21.15.45:0/2321954138 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 1 -- 172.21.15.45:0/2321954138 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962001639 0 0) 0x7f551c097340 con 0x7f551c0a1080 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient(hunting): my global_id is 4543 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 1 -- 172.21.15.45:0/2321954138 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f550c0034e0 con 0x7f551c0a1080 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 1 -- 172.21.15.45:0/2321954138 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538732018 0 0) 0x7f551c097620 con 0x7f551c06e810 2024-09-18T13:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient(hunting): my global_id is 4433 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 1 -- 172.21.15.45:0/2321954138 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f551c097340 con 0x7f551c06e810 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 1 -- 172.21.15.45:0/2321954138 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3376105393 0 0) 0x7f550c0034e0 con 0x7f551c0a1080 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 1 -- 172.21.15.45:0/2321954138 >> v1:172.21.15.84:6791/0 conn(0x7f551c06e810 legacy=0x7f551c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 1 -- 172.21.15.45:0/2321954138 >> v1:172.21.15.45:6791/0 conn(0x7f551c0a0cb0 legacy=0x7f551c137810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient: _finish_hunting 0 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 1 monclient: found mon.noname-i 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 1 -- 172.21.15.45:0/2321954138 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c0a1660 con 0x7f551c0a1080 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient: _finish_auth 0 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 1 -- 172.21.15.45:0/2321954138 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5514001bb0 con 0x7f551c0a1080 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient: dump: 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f5522f5d640 5 monclient: authenticate success, global_id 4543 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f5522f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 1 -- 172.21.15.45:0/2321954138 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5514001d10 con 0x7f551c0a1080 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f5522f5d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 1 -- 172.21.15.45:0/2321954138 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5514004b20 con 0x7f551c0a1080 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f5522f5d640 4 set_mon_vals no callback set 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.116+0000 7f551b7fe640 10 monclient: dump: 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.117+0000 7f5522f5d640 10 monclient: shutdown 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.117+0000 7f5522f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.117+0000 7f5522f5d640 1 -- 172.21.15.45:0/2321954138 >> v1:172.21.15.84:6792/0 conn(0x7f551c0a1080 legacy=0x7f551c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.117+0000 7f5522f5d640 1 -- 172.21.15.45:0/2321954138 shutdown_connections 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.117+0000 7f5522f5d640 1 -- 172.21.15.45:0/2321954138 >> 172.21.15.45:0/2321954138 conn(0x7f551c0a5bb0 msgr2=0x7f551c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.117+0000 7f5522f5d640 1 -- 172.21.15.45:0/2321954138 shutdown_connections 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.117+0000 7f5522f5d640 1 -- 172.21.15.45:0/2321954138 wait complete. 2024-09-18T13:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.117+0000 7f5522f5d640 10 monclient: build_initial_monmap 2024-09-18T13:59:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.117+0000 7f5522f5d640 10 monclient: monmap: 2024-09-18T13:59:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 1 Processor -- start 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 1 -- start start 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 10 monclient: init 2024-09-18T13:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 10 monclient: _add_conns ranks=[8,2,4,1,3,5,6,0,7] 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 10 monclient(hunting): picked mon.noname-e con 0x7f551c06e810 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 10 monclient(hunting): picked mon.noname-b con 0x7f551c0a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 10 monclient(hunting): picked mon.noname-c con 0x7f551c0a1080 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551c122cb0 con 0x7f551c0a0cb0 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551c13daa0 con 0x7f551c0a1080 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551c1226f0 con 0x7f551c06e810 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f55214fa640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f551c0a1080 legacy=0x7f551c149cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5522f5d640 10 monclient(hunting): authenticate will time out at 788.456876s 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5520cf9640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f551c06e810 0x7f551c13d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:58710/0 (socket says 172.21.15.45:58710) 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.118+0000 7f5520cf9640 1 -- 172.21.15.45:0/4267277940 learned_addr learned my addr 172.21.15.45:0/4267277940 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143549691 0 0) 0x7f551c1226f0 con 0x7f551c06e810 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): my global_id is 4472 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55000034e0 con 0x7f551c06e810 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832918798 0 0) 0x7f551c122cb0 con 0x7f551c0a0cb0 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): my global_id is 4538 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f551c1226f0 con 0x7f551c0a0cb0 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2812254163 0 0) 0x7f551c1226f0 con 0x7f551c0a0cb0 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f551c122cb0 con 0x7f551c0a0cb0 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f55080042b0 con 0x7f551c0a0cb0 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): dump: 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3070817094 0 0) 0x7f55000034e0 con 0x7f551c06e810 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f551c1226f0 con 0x7f551c06e810 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f55100015d0 con 0x7f551c06e810 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient(hunting): dump: 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 545761235 0 0) 0x7f551c122cb0 con 0x7f551c0a0cb0 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 >> v1:172.21.15.45:6793/0 conn(0x7f551c06e810 legacy=0x7f551c13d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 >> v1:172.21.15.45:6791/0 conn(0x7f551c0a1080 legacy=0x7f551c149cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient: _finish_hunting 0 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 monclient: found mon.b 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c120240 con 0x7f551c0a0cb0 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 10 monclient: _finish_auth 0 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5518ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5522f5d640 5 monclient: authenticate success, global_id 4538 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5522f5d640 10 monclient: _renew_subs 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5522f5d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5522f5d640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551c122410 con 0x7f551c0a0cb0 2024-09-18T13:59:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5522f5d640 10 monclient: _renew_subs 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5522f5d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.119+0000 7f5522f5d640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c121540 con 0x7f551c0a0cb0 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.120+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5508004840 con 0x7f551c0a0cb0 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.120+0000 7f5518ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.120+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5508004d30 con 0x7f551c0a0cb0 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.120+0000 7f5518ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.120+0000 7f551affd640 4 set_mon_vals no callback set 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.120+0000 7f5518ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.120+0000 7f5518ff9640 10 monclient: dump: 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.120+0000 7f5522f5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.120+0000 7f5522f5d640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f54fc006580 con 0x7f54fc002830 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.120+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f5508004f10 con 0x7f551c0a0cb0 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(20..20 src has 1..20) v4 ==== 3867+0+0 (unknown 541114214 0 0) 0x7f5508046aa0 con 0x7f551c0a0cb0 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907250404 0 0) 0x7f54fc006580 con 0x7f54fc002830 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f5518ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f5518ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f5518ff9640 10 monclient(hunting): my global_id is 4547 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f551c122cb0 con 0x7f54fc002830 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f55214fa640 10 monclient: get_auth_request con 0x7f550003a3d0 auth_method 0 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1765365948 0 0) 0x7f551c122cb0 con 0x7f54fc002830 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f54fc006580 con 0x7f54fc002830 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5510004ae0 con 0x7f54fc002830 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f5518ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f5518ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.121+0000 7f5518ff9640 10 monclient: dump: 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.143 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.122+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3258568854 0 0) 0x7f54fc006580 con 0x7f54fc002830 2024-09-18T13:59:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.122+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54fc00a0a0 con 0x7f54fc002830 2024-09-18T13:59:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.122+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5510005020 con 0x7f54fc002830 2024-09-18T13:59:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.122+0000 7f5518ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.122+0000 7f5518ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.122+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 >> v1:172.21.15.45:6790/0 conn(0x7f54fc002830 legacy=0x7f54fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.122+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 >> v1:172.21.15.45:6790/0 conn(0x7f54fc002830 legacy=0x7f54fc004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:33.145 INFO:tasks.rados.rados.0.smithi084.stdout:41: writing smithi08434663-41 from 2157515 to 2672280 tid 2 2024-09-18T13:59:33.145 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.145 INFO:tasks.rados.rados.0.smithi084.stdout:26: finishing write tid 1 to smithi08434663-26 2024-09-18T13:59:33.146 INFO:tasks.rados.rados.0.smithi084.stdout:26: finishing write tid 2 to smithi08434663-26 2024-09-18T13:59:33.146 INFO:tasks.rados.rados.0.smithi084.stdout:26: finishing write tid 3 to smithi08434663-26 2024-09-18T13:59:33.146 INFO:tasks.rados.rados.0.smithi084.stdout:26: finishing write tid 4 to smithi08434663-26 2024-09-18T13:59:33.146 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 9 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T13:59:33.146 INFO:tasks.rados.rados.0.smithi084.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-09-18T13:59:33.146 INFO:tasks.rados.rados.0.smithi084.stdout:27: finishing write tid 1 to smithi08434663-27 2024-09-18T13:59:33.146 INFO:tasks.rados.rados.0.smithi084.stdout:27: finishing write tid 2 to smithi08434663-27 2024-09-18T13:59:33.146 INFO:tasks.rados.rados.0.smithi084.stdout:27: finishing write tid 3 to smithi08434663-27 2024-09-18T13:59:33.146 INFO:tasks.rados.rados.0.smithi084.stdout:26: done (15 left) 2024-09-18T13:59:33.147 INFO:tasks.rados.rados.0.smithi084.stdout:42: write initial oid 42 2024-09-18T13:59:33.147 INFO:tasks.rados.rados.0.smithi084.stdout:42: seq_num 41 ranges {689087=466974,1699644=784332} 2024-09-18T13:59:33.147 INFO:tasks.rados.rados.0.smithi084.stdout:42: writing smithi08434663-42 from 689087 to 1156061 tid 1 2024-09-18T13:59:33.147 INFO:tasks.rados.rados.0.smithi084.stdout:42: writing smithi08434663-42 from 1699644 to 2483976 tid 2 2024-09-18T13:59:33.147 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.147 INFO:tasks.rados.rados.0.smithi084.stdout:27: finishing write tid 4 to smithi08434663-27 2024-09-18T13:59:33.147 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 12 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T13:59:33.147 INFO:tasks.rados.rados.0.smithi084.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-09-18T13:59:33.147 INFO:tasks.rados.rados.0.smithi084.stdout:28: finishing write tid 1 to smithi08434663-28 2024-09-18T13:59:33.147 INFO:tasks.rados.rados.0.smithi084.stdout:27: done (15 left) 2024-09-18T13:59:33.147 INFO:tasks.rados.rados.0.smithi084.stdout:43: write initial oid 43 2024-09-18T13:59:33.147 INFO:tasks.rados.rados.0.smithi084.stdout:43: seq_num 42 ranges {705894=542223,1758126=644343} 2024-09-18T13:59:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5522f5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5522f5d640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f54fc0074f0 con 0x7f54fc0068a0 2024-09-18T13:59:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225651288 0 0) 0x7f54fc0074f0 con 0x7f54fc0068a0 2024-09-18T13:59:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 10 monclient(hunting): my global_id is 4556 2024-09-18T13:59:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54fc006580 con 0x7f54fc0068a0 2024-09-18T13:59:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3419464526 0 0) 0x7f54fc006580 con 0x7f54fc0068a0 2024-09-18T13:59:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f54fc0074f0 con 0x7f54fc0068a0 2024-09-18T13:59:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f55100057e0 con 0x7f54fc0068a0 2024-09-18T13:59:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 10 monclient: dump: 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.150 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.150 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2152981783 0 0) 0x7f54fc0074f0 con 0x7f54fc0068a0 2024-09-18T13:59:33.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.129+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f54fc008750 con 0x7f54fc0068a0 2024-09-18T13:59:33.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7545 (unknown 0 0 200079645) 0x7f5510006c30 con 0x7f54fc0068a0 2024-09-18T13:59:33.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5518ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:33.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5518ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:33.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 >> v1:172.21.15.45:6790/0 conn(0x7f54fc0068a0 legacy=0x7f54fc006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5518ff9640 1 -- 172.21.15.45:0/4267277940 >> v1:172.21.15.45:6790/0 conn(0x7f54fc0068a0 legacy=0x7f54fc006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:33.150 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 32, 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T13:59:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:33.170 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5522f5d640 1 -- 172.21.15.45:0/4267277940 >> v1:172.21.15.84:6812/31478 conn(0x7f550003a3d0 legacy=0x7f550003c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5522f5d640 10 monclient: shutdown 2024-09-18T13:59:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5522f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5522f5d640 1 -- 172.21.15.45:0/4267277940 >> v1:172.21.15.45:6790/0 conn(0x7f551c0a0cb0 legacy=0x7f551c1465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f55214fa640 1 -- 172.21.15.45:0/4267277940 reap_dead start 2024-09-18T13:59:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f55214fa640 1 -- 172.21.15.45:0/4267277940 reap_dead start 2024-09-18T13:59:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5522f5d640 1 -- 172.21.15.45:0/4267277940 shutdown_connections 2024-09-18T13:59:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5522f5d640 1 -- 172.21.15.45:0/4267277940 >> 172.21.15.45:0/4267277940 conn(0x7f551c0a5bb0 msgr2=0x7f551c0a04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5522f5d640 1 -- 172.21.15.45:0/4267277940 shutdown_connections 2024-09-18T13:59:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.130+0000 7f5522f5d640 1 -- 172.21.15.45:0/4267277940 wait complete. 2024-09-18T13:59:33.174 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T13:59:33.174 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-09-18T13:59:33.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T13:59:33.175 INFO:tasks.rados.rados.0.smithi084.stdout:43: writing smithi08434663-43 from 705894 to 1248117 tid 1 2024-09-18T13:59:33.175 INFO:tasks.rados.rados.0.smithi084.stdout:43: writing smithi08434663-43 from 1758126 to 2402469 tid 2 2024-09-18T13:59:33.175 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.175 INFO:tasks.rados.rados.0.smithi084.stdout:28: finishing write tid 2 to smithi08434663-28 2024-09-18T13:59:33.175 INFO:tasks.rados.rados.0.smithi084.stdout:28: finishing write tid 3 to smithi08434663-28 2024-09-18T13:59:33.175 INFO:tasks.rados.rados.0.smithi084.stdout:28: finishing write tid 4 to smithi08434663-28 2024-09-18T13:59:33.175 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 15 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:29: finishing write tid 1 to smithi08434663-29 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:29: finishing write tid 2 to smithi08434663-29 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:29: finishing write tid 3 to smithi08434663-29 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:29: finishing write tid 4 to smithi08434663-29 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 9 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:30: finishing write tid 1 to smithi08434663-30 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:28: done (15 left) 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:29: done (14 left) 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:44: write initial oid 44 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:44: seq_num 43 ranges {722701=617472,1816608=788001} 2024-09-18T13:59:33.176 INFO:tasks.rados.rados.0.smithi084.stdout:44: writing smithi08434663-44 from 722701 to 1340173 tid 1 2024-09-18T13:59:33.177 INFO:tasks.rados.rados.0.smithi084.stdout:44: writing smithi08434663-44 from 1816608 to 2604609 tid 2 2024-09-18T13:59:33.177 INFO:tasks.rados.rados.0.smithi084.stdout:45: write initial oid 45 2024-09-18T13:59:33.177 INFO:tasks.rados.rados.0.smithi084.stdout:45: seq_num 44 ranges {739508=692721,1875090=648012} 2024-09-18T13:59:33.177 INFO:tasks.rados.rados.0.smithi084.stdout:45: writing smithi08434663-45 from 739508 to 1432229 tid 1 2024-09-18T13:59:33.180 INFO:tasks.rados.rados.0.smithi084.stdout:45: writing smithi08434663-45 from 1875090 to 2523102 tid 2 2024-09-18T13:59:33.180 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.180 INFO:tasks.rados.rados.0.smithi084.stdout:30: finishing write tid 2 to smithi08434663-30 2024-09-18T13:59:33.180 INFO:tasks.rados.rados.0.smithi084.stdout:30: finishing write tid 3 to smithi08434663-30 2024-09-18T13:59:33.180 INFO:tasks.rados.rados.0.smithi084.stdout:30: finishing write tid 4 to smithi08434663-30 2024-09-18T13:59:33.181 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 3 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T13:59:33.181 INFO:tasks.rados.rados.0.smithi084.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-09-18T13:59:33.181 INFO:tasks.rados.rados.0.smithi084.stdout:30: done (15 left) 2024-09-18T13:59:33.181 INFO:tasks.rados.rados.0.smithi084.stdout:46: write initial oid 46 2024-09-18T13:59:33.181 INFO:tasks.rados.rados.0.smithi084.stdout:46: seq_num 45 ranges {756315=767970,1933572=791670} 2024-09-18T13:59:33.188 INFO:tasks.rados.rados.0.smithi084.stdout:46: writing smithi08434663-46 from 756315 to 1524285 tid 1 2024-09-18T13:59:33.195 INFO:tasks.rados.rados.0.smithi084.stdout:46: writing smithi08434663-46 from 1933572 to 2725242 tid 2 2024-09-18T13:59:33.195 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.195 INFO:tasks.rados.rados.0.smithi084.stdout:31: finishing write tid 1 to smithi08434663-31 2024-09-18T13:59:33.195 INFO:tasks.rados.rados.0.smithi084.stdout:31: finishing write tid 2 to smithi08434663-31 2024-09-18T13:59:33.195 INFO:tasks.rados.rados.0.smithi084.stdout:31: finishing write tid 3 to smithi08434663-31 2024-09-18T13:59:33.195 INFO:tasks.rados.rados.0.smithi084.stdout:31: finishing write tid 4 to smithi08434663-31 2024-09-18T13:59:33.195 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 6 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T13:59:33.195 INFO:tasks.rados.rados.0.smithi084.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-09-18T13:59:33.195 INFO:tasks.rados.rados.0.smithi084.stdout:31: done (15 left) 2024-09-18T13:59:33.195 INFO:tasks.rados.rados.0.smithi084.stdout:47: write initial oid 47 2024-09-18T13:59:33.196 INFO:tasks.rados.rados.0.smithi084.stdout:47: seq_num 46 ranges {773122=559572,1992054=651681} 2024-09-18T13:59:33.202 INFO:tasks.rados.rados.0.smithi084.stdout:47: writing smithi08434663-47 from 773122 to 1332694 tid 1 2024-09-18T13:59:33.210 INFO:tasks.rados.rados.0.smithi084.stdout:47: writing smithi08434663-47 from 1992054 to 2643735 tid 2 2024-09-18T13:59:33.210 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:32: finishing write tid 1 to smithi08434663-32 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:32: finishing write tid 2 to smithi08434663-32 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:32: finishing write tid 3 to smithi08434663-32 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:32: finishing write tid 4 to smithi08434663-32 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 18 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:33: finishing write tid 1 to smithi08434663-33 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:33: finishing write tid 2 to smithi08434663-33 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:33: finishing write tid 3 to smithi08434663-33 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:33: finishing write tid 4 to smithi08434663-33 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 3 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:34: finishing write tid 1 to smithi08434663-34 2024-09-18T13:59:33.211 INFO:tasks.rados.rados.0.smithi084.stdout:34: finishing write tid 2 to smithi08434663-34 2024-09-18T13:59:33.212 INFO:tasks.rados.rados.0.smithi084.stdout:34: finishing write tid 3 to smithi08434663-34 2024-09-18T13:59:33.212 INFO:tasks.rados.rados.0.smithi084.stdout:34: finishing write tid 4 to smithi08434663-34 2024-09-18T13:59:33.212 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T13:59:33.212 INFO:tasks.rados.rados.0.smithi084.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-09-18T13:59:33.212 INFO:tasks.rados.rados.0.smithi084.stdout:35: finishing write tid 1 to smithi08434663-35 2024-09-18T13:59:33.212 INFO:tasks.rados.rados.0.smithi084.stdout:32: done (15 left) 2024-09-18T13:59:33.212 INFO:tasks.rados.rados.0.smithi084.stdout:33: done (14 left) 2024-09-18T13:59:33.212 INFO:tasks.rados.rados.0.smithi084.stdout:34: done (13 left) 2024-09-18T13:59:33.212 INFO:tasks.rados.rados.0.smithi084.stdout:48: write initial oid 48 2024-09-18T13:59:33.212 INFO:tasks.rados.rados.0.smithi084.stdout:48: seq_num 47 ranges {789929=634821,2050536=739393} 2024-09-18T13:59:33.217 INFO:tasks.rados.rados.0.smithi084.stdout:48: writing smithi08434663-48 from 789929 to 1424750 tid 1 2024-09-18T13:59:33.225 INFO:tasks.rados.rados.0.smithi084.stdout:48: writing smithi08434663-48 from 2050536 to 2789929 tid 2 2024-09-18T13:59:33.225 INFO:tasks.rados.rados.0.smithi084.stdout:49: write initial oid 49 2024-09-18T13:59:33.225 INFO:tasks.rados.rados.0.smithi084.stdout:49: seq_num 48 ranges {406736=710070,1709018=655350} 2024-09-18T13:59:33.231 INFO:tasks.rados.rados.0.smithi084.stdout:49: writing smithi08434663-49 from 406736 to 1116806 tid 1 2024-09-18T13:59:33.238 INFO:tasks.rados.rados.0.smithi084.stdout:49: writing smithi08434663-49 from 1709018 to 2364368 tid 2 2024-09-18T13:59:33.238 INFO:tasks.rados.rados.0.smithi084.stdout:50: write initial oid 50 2024-09-18T13:59:33.238 INFO:tasks.rados.rados.0.smithi084.stdout:50: seq_num 49 ranges {423543=785319,1767500=799008} 2024-09-18T13:59:33.245 INFO:tasks.rados.rados.0.smithi084.stdout:50: writing smithi08434663-50 from 423543 to 1208862 tid 1 2024-09-18T13:59:33.255 INFO:tasks.rados.rados.0.smithi084.stdout:50: writing smithi08434663-50 from 1767500 to 2566508 tid 2 2024-09-18T13:59:33.255 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 16 2024-09-18T13:59:33.255 INFO:tasks.rados.rados.0.smithi084.stdout:35: finishing write tid 2 to smithi08434663-35 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:35: finishing write tid 3 to smithi08434663-35 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:35: finishing write tid 4 to smithi08434663-35 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 12 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:36: finishing write tid 1 to smithi08434663-36 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:36: finishing write tid 2 to smithi08434663-36 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:36: finishing write tid 3 to smithi08434663-36 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:36: finishing write tid 4 to smithi08434663-36 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:37: finishing write tid 1 to smithi08434663-37 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:37: finishing write tid 2 to smithi08434663-37 2024-09-18T13:59:33.256 INFO:tasks.rados.rados.0.smithi084.stdout:37: finishing write tid 3 to smithi08434663-37 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:37: finishing write tid 4 to smithi08434663-37 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 9 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:38: finishing write tid 1 to smithi08434663-38 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:38: finishing write tid 2 to smithi08434663-38 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:38: finishing write tid 3 to smithi08434663-38 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:38: finishing write tid 4 to smithi08434663-38 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 3 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:39: finishing write tid 1 to smithi08434663-39 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:39: finishing write tid 2 to smithi08434663-39 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:39: finishing write tid 3 to smithi08434663-39 2024-09-18T13:59:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:39: finishing write tid 4 to smithi08434663-39 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 6 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:40: finishing write tid 1 to smithi08434663-40 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:40: finishing write tid 2 to smithi08434663-40 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:40: finishing write tid 3 to smithi08434663-40 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:40: finishing write tid 4 to smithi08434663-40 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 12 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:41: finishing write tid 1 to smithi08434663-41 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:41: finishing write tid 2 to smithi08434663-41 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:41: finishing write tid 3 to smithi08434663-41 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:41: finishing write tid 4 to smithi08434663-41 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 9 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T13:59:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:42: finishing write tid 1 to smithi08434663-42 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:42: finishing write tid 2 to smithi08434663-42 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:42: finishing write tid 3 to smithi08434663-42 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:42: finishing write tid 4 to smithi08434663-42 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:43: finishing write tid 1 to smithi08434663-43 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:35: done (15 left) 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:36: done (14 left) 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:37: done (13 left) 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:38: done (12 left) 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:39: done (11 left) 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:40: done (10 left) 2024-09-18T13:59:33.259 INFO:tasks.rados.rados.0.smithi084.stdout:41: done (9 left) 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:42: done (8 left) 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:51: snap_create 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:43: finishing write tid 2 to smithi08434663-43 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:43: finishing write tid 3 to smithi08434663-43 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:43: finishing write tid 4 to smithi08434663-43 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 6 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:44: finishing write tid 1 to smithi08434663-44 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:44: finishing write tid 2 to smithi08434663-44 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:44: finishing write tid 3 to smithi08434663-44 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:44: finishing write tid 4 to smithi08434663-44 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 6 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-09-18T13:59:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:45: finishing write tid 1 to smithi08434663-45 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:45: finishing write tid 2 to smithi08434663-45 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:45: finishing write tid 3 to smithi08434663-45 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:45: finishing write tid 4 to smithi08434663-45 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 15 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:46: finishing write tid 1 to smithi08434663-46 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:46: finishing write tid 2 to smithi08434663-46 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:46: finishing write tid 3 to smithi08434663-46 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:46: finishing write tid 4 to smithi08434663-46 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 15 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:47: finishing write tid 1 to smithi08434663-47 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:47: finishing write tid 2 to smithi08434663-47 2024-09-18T13:59:33.261 INFO:tasks.rados.rados.0.smithi084.stdout:47: finishing write tid 3 to smithi08434663-47 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:47: finishing write tid 4 to smithi08434663-47 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 9 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:48: finishing write tid 1 to smithi08434663-48 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:48: finishing write tid 2 to smithi08434663-48 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:48: finishing write tid 3 to smithi08434663-48 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:48: finishing write tid 4 to smithi08434663-48 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 9 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:49: finishing write tid 1 to smithi08434663-49 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:49: finishing write tid 2 to smithi08434663-49 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:49: finishing write tid 3 to smithi08434663-49 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:49: finishing write tid 4 to smithi08434663-49 2024-09-18T13:59:33.262 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 12 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T13:59:33.263 INFO:tasks.rados.rados.0.smithi084.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-09-18T13:59:33.263 INFO:tasks.rados.rados.0.smithi084.stdout:50: finishing write tid 1 to smithi08434663-50 2024-09-18T13:59:33.263 INFO:tasks.rados.rados.0.smithi084.stdout:50: finishing write tid 2 to smithi08434663-50 2024-09-18T13:59:33.263 INFO:tasks.rados.rados.0.smithi084.stdout:50: finishing write tid 3 to smithi08434663-50 2024-09-18T13:59:33.263 INFO:tasks.rados.rados.0.smithi084.stdout:50: finishing write tid 4 to smithi08434663-50 2024-09-18T13:59:33.264 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 21 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T13:59:33.264 INFO:tasks.rados.rados.0.smithi084.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-09-18T13:59:33.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.275+0000 7f42cf19d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.275+0000 7f42cf19d640 10 monclient: build_initial_monmap 2024-09-18T13:59:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.275+0000 7f42cf19d640 10 monclient: monmap: 2024-09-18T13:59:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:33.276458+0000 2024-09-18T13:59:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:33.276458+0000 2024-09-18T13:59:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:33.277 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 1 Processor -- start 2024-09-18T13:59:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 1 -- start start 2024-09-18T13:59:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 10 monclient: init 2024-09-18T13:59:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 10 monclient: _add_conns ranks=[7,8,5,6,0,2,1,4,3] 2024-09-18T13:59:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 10 monclient(hunting): picked mon.noname-i con 0x7f42c806e810 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 10 monclient(hunting): picked mon.noname-e con 0x7f42c80a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 10 monclient(hunting): picked mon.noname-h con 0x7f42c80a1080 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c805ed70 con 0x7f42c80a1080 2024-09-18T13:59:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c8097620 con 0x7f42c806e810 2024-09-18T13:59:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c8097340 con 0x7f42c80a0cb0 2024-09-18T13:59:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42cf19d640 10 monclient(hunting): authenticate will time out at 788.614802s 2024-09-18T13:59:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42bffff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f42c80a0cb0 0x7f42c8137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:58724/0 (socket says 172.21.15.45:58724) 2024-09-18T13:59:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.276+0000 7f42bffff640 1 -- 172.21.15.45:0/2355128138 learned_addr learned my addr 172.21.15.45:0/2355128138 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 1 -- 172.21.15.45:0/2355128138 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622633437 0 0) 0x7f42c8097340 con 0x7f42c80a0cb0 2024-09-18T13:59:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient(hunting): my global_id is 4481 2024-09-18T13:59:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 1 -- 172.21.15.45:0/2355128138 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42b00034e0 con 0x7f42c80a0cb0 2024-09-18T13:59:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 1 -- 172.21.15.45:0/2355128138 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1990071865 0 0) 0x7f42b00034e0 con 0x7f42c80a0cb0 2024-09-18T13:59:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 1 -- 172.21.15.45:0/2355128138 >> v1:172.21.15.84:6792/0 conn(0x7f42c806e810 legacy=0x7f42c80a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 1 -- 172.21.15.45:0/2355128138 >> v1:172.21.15.84:6791/0 conn(0x7f42c80a1080 legacy=0x7f42c813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient: _finish_hunting 0 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 1 monclient: found mon.noname-e 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 1 -- 172.21.15.45:0/2355128138 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c80a1660 con 0x7f42c80a0cb0 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient: _finish_auth 0 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 1 -- 172.21.15.45:0/2355128138 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f42b8001830 con 0x7f42c80a0cb0 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42cf19d640 5 monclient: authenticate success, global_id 4481 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42cf19d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient: dump: 2024-09-18T13:59:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42cf19d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 1 -- 172.21.15.45:0/2355128138 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f42b8001a10 con 0x7f42c80a0cb0 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 1 -- 172.21.15.45:0/2355128138 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f42b8004ba0 con 0x7f42c80a0cb0 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42cf19d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42cf19d640 4 set_mon_vals no callback set 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.277+0000 7f42bf7fe640 10 monclient: dump: 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.278+0000 7f42cf19d640 10 monclient: shutdown 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.278+0000 7f42cf19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.278+0000 7f42cf19d640 1 -- 172.21.15.45:0/2355128138 >> v1:172.21.15.45:6793/0 conn(0x7f42c80a0cb0 legacy=0x7f42c8137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.278+0000 7f42cf19d640 1 -- 172.21.15.45:0/2355128138 shutdown_connections 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.278+0000 7f42cf19d640 1 -- 172.21.15.45:0/2355128138 >> 172.21.15.45:0/2355128138 conn(0x7f42c80a5bb0 msgr2=0x7f42c80a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.278+0000 7f42cf19d640 1 -- 172.21.15.45:0/2355128138 shutdown_connections 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.278+0000 7f42cf19d640 1 -- 172.21.15.45:0/2355128138 wait complete. 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.278+0000 7f42cf19d640 10 monclient: build_initial_monmap 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.278+0000 7f42cf19d640 10 monclient: monmap: 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:33.291 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:33.292 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.278+0000 7f42cf19d640 1 Processor -- start 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cf19d640 1 -- start start 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cf19d640 10 monclient: init 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cf19d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cf19d640 10 monclient: _add_conns ranks=[8,7,4,3,6,1,2,5,0] 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cf19d640 10 monclient(hunting): picked mon.noname-e con 0x7f42c806e810 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cf19d640 10 monclient(hunting): picked mon.noname-i con 0x7f42c80a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cf19d640 10 monclient(hunting): picked mon.noname-c con 0x7f42c80a1080 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cf19d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c8122ec0 con 0x7f42c80a1080 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cf19d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c813dcb0 con 0x7f42c80a0cb0 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cd73a640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f42c80a1080 legacy=0x7f42c8149f00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cf19d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c8122900 con 0x7f42c806e810 2024-09-18T13:59:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cf19d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:33.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42cf19d640 10 monclient(hunting): authenticate will time out at 788.617608s 2024-09-18T13:59:33.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42ccf39640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f42c806e810 0x7f42c813d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:58738/0 (socket says 172.21.15.45:58738) 2024-09-18T13:59:33.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42ccf39640 1 -- 172.21.15.45:0/3714121712 learned_addr learned my addr 172.21.15.45:0/3714121712 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:33.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528440766 0 0) 0x7f42c8122900 con 0x7f42c806e810 2024-09-18T13:59:33.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42bcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42bcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42bcff9640 10 monclient(hunting): my global_id is 4490 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42b40034e0 con 0x7f42c806e810 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529917533 0 0) 0x7f42c813dcb0 con 0x7f42c80a0cb0 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42bcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42bcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42bcff9640 10 monclient(hunting): my global_id is 4561 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.279+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42c8122900 con 0x7f42c80a0cb0 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1891376880 0 0) 0x7f42b40034e0 con 0x7f42c806e810 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f42c813dcb0 con 0x7f42c806e810 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f42c4001be0 con 0x7f42c806e810 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient(hunting): dump: 2024-09-18T13:59:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2638192429 0 0) 0x7f42c8122900 con 0x7f42c80a0cb0 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f42b40034e0 con 0x7f42c80a0cb0 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f42b80017f0 con 0x7f42c80a0cb0 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient(hunting): dump: 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.297 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.298 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.298 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.298 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.298 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.298 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.298 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.298 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.298 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.299 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.300 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4024832815 0 0) 0x7f42c813dcb0 con 0x7f42c806e810 2024-09-18T13:59:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 >> v1:172.21.15.84:6792/0 conn(0x7f42c80a0cb0 legacy=0x7f42c81467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 >> v1:172.21.15.45:6791/0 conn(0x7f42c80a1080 legacy=0x7f42c8149f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T13:59:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient: _finish_hunting 0 2024-09-18T13:59:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 monclient: found mon.e 2024-09-18T13:59:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c8122620 con 0x7f42c806e810 2024-09-18T13:59:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient: _finish_auth 0 2024-09-18T13:59:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42cf19d640 5 monclient: authenticate success, global_id 4490 2024-09-18T13:59:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42cf19d640 10 monclient: _renew_subs 2024-09-18T13:59:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42cf19d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42cf19d640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c8122be0 con 0x7f42c806e810 2024-09-18T13:59:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42cf19d640 10 monclient: _renew_subs 2024-09-18T13:59:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42cf19d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T13:59:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42cf19d640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c8121750 con 0x7f42c806e810 2024-09-18T13:59:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f42c40021b0 con 0x7f42c806e810 2024-09-18T13:59:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f42c4002720 con 0x7f42c806e810 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42beffd640 4 set_mon_vals no callback set 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.280+0000 7f42bcff9640 10 monclient: dump: 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f42c4002880 con 0x7f42c806e810 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42cf19d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42cf19d640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f42a8006580 con 0x7f42a8002830 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(20..20 src has 1..20) v4 ==== 3867+0+0 (unknown 541114214 0 0) 0x7f42c4046ba0 con 0x7f42c806e810 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245294314 0 0) 0x7f42a8006580 con 0x7f42a8002830 2024-09-18T13:59:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 10 monclient(hunting): my global_id is 4488 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42c813dcb0 con 0x7f42a8002830 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42cd73a640 10 monclient: get_auth_request con 0x7f42b403a410 auth_method 0 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2181501443 0 0) 0x7f42c813dcb0 con 0x7f42a8002830 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f42a8006580 con 0x7f42a8002830 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f42b8004b10 con 0x7f42a8002830 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 10 monclient: dump: 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.304 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4140068845 0 0) 0x7f42a8006580 con 0x7f42a8002830 2024-09-18T13:59:33.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.281+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42a800a0a0 con 0x7f42a8002830 2024-09-18T13:59:33.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.282+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f42b8005250 con 0x7f42a8002830 2024-09-18T13:59:33.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.282+0000 7f42bcff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:33.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.282+0000 7f42bcff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:33.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.282+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 >> v1:172.21.15.45:6792/0 conn(0x7f42a8002830 legacy=0x7f42a8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.282+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 >> v1:172.21.15.45:6792/0 conn(0x7f42a8002830 legacy=0x7f42a8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.287+0000 7f42cf19d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.287+0000 7f42cf19d640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f42a8009640 con 0x7f42a8009020 2024-09-18T13:59:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584507317 0 0) 0x7f42a8009640 con 0x7f42a8009020 2024-09-18T13:59:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 10 monclient(hunting): my global_id is 4497 2024-09-18T13:59:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42a8006580 con 0x7f42a8009020 2024-09-18T13:59:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2883248522 0 0) 0x7f42a8006580 con 0x7f42a8009020 2024-09-18T13:59:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f42a8009640 con 0x7f42a8009020 2024-09-18T13:59:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f42b8005a10 con 0x7f42a8009020 2024-09-18T13:59:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 10 monclient: dump: 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.310 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 703987488 0 0) 0x7f42a8009640 con 0x7f42a8009020 2024-09-18T13:59:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f42a8009900 con 0x7f42a8009020 2024-09-18T13:59:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7375 (unknown 0 0 2940436662) 0x7f42b8007df0 con 0x7f42a8009020 2024-09-18T13:59:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 >> v1:172.21.15.45:6792/0 conn(0x7f42a8009020 legacy=0x7f42a8008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.288+0000 7f42bcff9640 1 -- 172.21.15.45:0/3714121712 >> v1:172.21.15.45:6792/0 conn(0x7f42a8009020 legacy=0x7f42a8008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:33.310 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 32, 2024-09-18T13:59:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.330 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:33.330 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.289+0000 7f42cf19d640 1 -- 172.21.15.45:0/3714121712 >> v1:172.21.15.84:6812/31478 conn(0x7f42b403a410 legacy=0x7f42b403c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.289+0000 7f42cf19d640 10 monclient: shutdown 2024-09-18T13:59:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.289+0000 7f42cf19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.289+0000 7f42cf19d640 1 -- 172.21.15.45:0/3714121712 >> v1:172.21.15.45:6793/0 conn(0x7f42c806e810 legacy=0x7f42c813d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.289+0000 7f42cd73a640 1 -- 172.21.15.45:0/3714121712 reap_dead start 2024-09-18T13:59:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.289+0000 7f42cd73a640 1 -- 172.21.15.45:0/3714121712 reap_dead start 2024-09-18T13:59:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.289+0000 7f42cf19d640 1 -- 172.21.15.45:0/3714121712 shutdown_connections 2024-09-18T13:59:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.289+0000 7f42cf19d640 1 -- 172.21.15.45:0/3714121712 >> 172.21.15.45:0/3714121712 conn(0x7f42c80a5bb0 msgr2=0x7f42c80a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.289+0000 7f42cf19d640 1 -- 172.21.15.45:0/3714121712 shutdown_connections 2024-09-18T13:59:33.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.289+0000 7f42cf19d640 1 -- 172.21.15.45:0/3714121712 wait complete. 2024-09-18T13:59:33.332 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T13:59:33.332 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-09-18T13:59:33.332 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T13:59:33.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.429+0000 7f4fae79d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:33.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.429+0000 7f4fae79d640 10 monclient: build_initial_monmap 2024-09-18T13:59:33.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.429+0000 7f4fae79d640 10 monclient: monmap: 2024-09-18T13:59:33.429 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:33.430009+0000 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:33.430009+0000 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:33.431 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.429+0000 7f4fae79d640 1 Processor -- start 2024-09-18T13:59:33.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.429+0000 7f4fae79d640 1 -- start start 2024-09-18T13:59:33.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.429+0000 7f4fae79d640 10 monclient: init 2024-09-18T13:59:33.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fae79d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:33.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fae79d640 10 monclient: _add_conns ranks=[5,0,8,7,4,2,1,3,6] 2024-09-18T13:59:33.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fae79d640 10 monclient(hunting): picked mon.noname-h con 0x7f4fa809f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:33.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fae79d640 10 monclient(hunting): picked mon.noname-a con 0x7f4fa809fba0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:33.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fae79d640 10 monclient(hunting): picked mon.noname-e con 0x7f4fa809ff70 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fae79d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa805ed70 con 0x7f4fa809fba0 2024-09-18T13:59:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fae79d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa809dfc0 con 0x7f4fa809f7d0 2024-09-18T13:59:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fae79d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa809dce0 con 0x7f4fa809ff70 2024-09-18T13:59:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fae79d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fae79d640 10 monclient(hunting): authenticate will time out at 788.768422s 2024-09-18T13:59:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa77fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4fa809fba0 0x7f4fa8137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59360/0 (socket says 172.21.15.45:59360) 2024-09-18T13:59:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4facd3a640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f4fa809ff70 0x7f4fa813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:58752/0 (socket says 172.21.15.45:58752) 2024-09-18T13:59:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa77fe640 1 -- 172.21.15.45:0/2685635368 learned_addr learned my addr 172.21.15.45:0/2685635368 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312602091 0 0) 0x7f4fa809dce0 con 0x7f4fa809ff70 2024-09-18T13:59:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 10 monclient(hunting): my global_id is 4499 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f900034e0 con 0x7f4fa809ff70 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78106446 0 0) 0x7f4fa805ed70 con 0x7f4fa809fba0 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 10 monclient(hunting): my global_id is 4554 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa809dce0 con 0x7f4fa809fba0 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149316821 0 0) 0x7f4fa809dfc0 con 0x7f4fa809f7d0 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 10 monclient(hunting): my global_id is 4451 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.430+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa805ed70 con 0x7f4fa809f7d0 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 171554937 0 0) 0x7f4f900034e0 con 0x7f4fa809ff70 2024-09-18T13:59:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 >> v1:172.21.15.84:6791/0 conn(0x7f4fa809f7d0 legacy=0x7f4fa8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 >> v1:172.21.15.45:6789/0 conn(0x7f4fa809fba0 legacy=0x7f4fa8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:33.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 1 monclient: found mon.noname-e 2024-09-18T13:59:33.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:33.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T13:59:33.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa80a1c00 con 0x7f4fa809ff70 2024-09-18T13:59:33.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 10 monclient: _finish_auth 0 2024-09-18T13:59:33.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:33.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4fa0001bb0 con 0x7f4fa809ff70 2024-09-18T13:59:33.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fae79d640 5 monclient: authenticate success, global_id 4499 2024-09-18T13:59:33.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fae79d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 10 monclient: dump: 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fae79d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4fa0001d90 con 0x7f4fa809ff70 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 1 -- 172.21.15.45:0/2685635368 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4fa0004e60 con 0x7f4fa809ff70 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fae79d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fae79d640 4 set_mon_vals no callback set 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fa6ffd640 10 monclient: dump: 2024-09-18T13:59:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.448 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.448 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.448 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.448 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.448 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.448 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.448 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.449 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fae79d640 10 monclient: shutdown 2024-09-18T13:59:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fae79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fae79d640 1 -- 172.21.15.45:0/2685635368 >> v1:172.21.15.45:6793/0 conn(0x7f4fa809ff70 legacy=0x7f4fa813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fae79d640 1 -- 172.21.15.45:0/2685635368 shutdown_connections 2024-09-18T13:59:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fae79d640 1 -- 172.21.15.45:0/2685635368 >> 172.21.15.45:0/2685635368 conn(0x7f4fa80630b0 msgr2=0x7f4fa80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fae79d640 1 -- 172.21.15.45:0/2685635368 shutdown_connections 2024-09-18T13:59:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.431+0000 7f4fae79d640 1 -- 172.21.15.45:0/2685635368 wait complete. 2024-09-18T13:59:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.432+0000 7f4fae79d640 10 monclient: build_initial_monmap 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.432+0000 7f4fae79d640 10 monclient: monmap: 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.432+0000 7f4fae79d640 1 Processor -- start 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.432+0000 7f4fae79d640 1 -- start start 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.432+0000 7f4fae79d640 10 monclient: init 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4fae79d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4fae79d640 10 monclient: _add_conns ranks=[3,6,2,8,7,4,1,5,0] 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4fae79d640 10 monclient(hunting): picked mon.noname-g con 0x7f4fa809f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4fae79d640 10 monclient(hunting): picked mon.noname-d con 0x7f4fa809fba0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4fae79d640 10 monclient(hunting): picked mon.noname-b con 0x7f4fa809ff70 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4fae79d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa81208b0 con 0x7f4fa809ff70 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4fae79d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa8121ee0 con 0x7f4fa809f7d0 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4fae79d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa809d310 con 0x7f4fa809fba0 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4fae79d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4fae79d640 10 monclient(hunting): authenticate will time out at 788.771426s 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4fa77fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f4fa809fba0 0x7f4fa8131ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:58016/0 (socket says 172.21.15.45:58016) 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4fa77fe640 1 -- 172.21.15.45:0/1724973299 learned_addr learned my addr 172.21.15.45:0/1724973299 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004184623 0 0) 0x7f4fa809d310 con 0x7f4fa809fba0 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 10 monclient(hunting): my global_id is 4506 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f940034e0 con 0x7f4fa809fba0 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549969738 0 0) 0x7f4fa81208b0 con 0x7f4fa809ff70 2024-09-18T13:59:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 10 monclient(hunting): my global_id is 4565 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa809d310 con 0x7f4fa809ff70 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362848688 0 0) 0x7f4fa8121ee0 con 0x7f4fa809f7d0 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 10 monclient(hunting): my global_id is 4602 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa81208b0 con 0x7f4fa809f7d0 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 121267033 0 0) 0x7f4f940034e0 con 0x7f4fa809fba0 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4fa8121ee0 con 0x7f4fa809fba0 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f9c001590 con 0x7f4fa809fba0 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.433+0000 7f4f8ffff640 10 monclient(hunting): dump: 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 585077612 0 0) 0x7f4fa81208b0 con 0x7f4fa809f7d0 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f940034e0 con 0x7f4fa809f7d0 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4066894664 0 0) 0x7f4fa809d310 con 0x7f4fa809ff70 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4fa81208b0 con 0x7f4fa809ff70 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f98001ae0 con 0x7f4fa809f7d0 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient(hunting): dump: 2024-09-18T13:59:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4fa0001af0 con 0x7f4fa809ff70 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient(hunting): dump: 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2003753766 0 0) 0x7f4fa8121ee0 con 0x7f4fa809fba0 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 >> v1:172.21.15.84:6790/0 conn(0x7f4fa809f7d0 legacy=0x7f4fa81418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 >> v1:172.21.15.45:6790/0 conn(0x7f4fa809ff70 legacy=0x7f4fa8149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient: _finish_hunting 0 2024-09-18T13:59:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 monclient: found mon.d 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa8142750 con 0x7f4fa809fba0 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient: _finish_auth 0 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4fae79d640 5 monclient: authenticate success, global_id 4506 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4fae79d640 10 monclient: _renew_subs 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4fae79d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4fae79d640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa8121c00 con 0x7f4fa809fba0 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4fae79d640 10 monclient: _renew_subs 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4fae79d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4fae79d640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa81200d0 con 0x7f4fa809fba0 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4f9c004a50 con 0x7f4fa809fba0 2024-09-18T13:59:33.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f9c004f00 con 0x7f4fa809fba0 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4fa67fc640 4 set_mon_vals no callback set 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.434+0000 7f4f8ffff640 10 monclient: dump: 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4f9c00fed0 con 0x7f4fa809fba0 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4fae79d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4fae79d640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4f88006580 con 0x7f4f88002830 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(20..20 src has 1..20) v4 ==== 3867+0+0 (unknown 541114214 0 0) 0x7f4f9c046bf0 con 0x7f4fa809fba0 2024-09-18T13:59:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713637927 0 0) 0x7f4f88006580 con 0x7f4f88002830 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 10 monclient(hunting): my global_id is 4508 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa8121ee0 con 0x7f4f88002830 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4facd3a640 10 monclient: get_auth_request con 0x7f4f9403aa00 auth_method 0 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2049027835 0 0) 0x7f4fa8121ee0 con 0x7f4f88002830 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f88006580 con 0x7f4f88002830 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f980023f0 con 0x7f4f88002830 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 10 monclient: dump: 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.462 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2422071672 0 0) 0x7f4f88006580 con 0x7f4f88002830 2024-09-18T13:59:33.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.435+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f8800a0a0 con 0x7f4f88002830 2024-09-18T13:59:33.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.436+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f4f980028d0 con 0x7f4f88002830 2024-09-18T13:59:33.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.436+0000 7f4f8ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:33.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.436+0000 7f4f8ffff640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:33.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.436+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 >> v1:172.21.15.45:6793/0 conn(0x7f4f88002830 legacy=0x7f4f88004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.436+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 >> v1:172.21.15.45:6793/0 conn(0x7f4f88002830 legacy=0x7f4f88004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.441+0000 7f4fae79d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.441+0000 7f4fae79d640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4f880074f0 con 0x7f4f880068a0 2024-09-18T13:59:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223206275 0 0) 0x7f4f880074f0 con 0x7f4f880068a0 2024-09-18T13:59:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 10 monclient(hunting): my global_id is 4517 2024-09-18T13:59:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f88006580 con 0x7f4f880068a0 2024-09-18T13:59:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3735353825 0 0) 0x7f4f88006580 con 0x7f4f880068a0 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f880074f0 con 0x7f4f880068a0 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f98003340 con 0x7f4f880068a0 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 10 monclient: dump: 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4041013297 0 0) 0x7f4f880074f0 con 0x7f4f880068a0 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4f88008750 con 0x7f4f880068a0 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7377 (unknown 0 0 2067792542) 0x7f4f980035e0 con 0x7f4f880068a0 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 >> v1:172.21.15.45:6793/0 conn(0x7f4f880068a0 legacy=0x7f4f88006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.442+0000 7f4f8ffff640 1 -- 172.21.15.45:0/1724973299 >> v1:172.21.15.45:6793/0 conn(0x7f4f880068a0 legacy=0x7f4f88006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:33.469 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T13:59:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 32, 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:33.488 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:33.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.443+0000 7f4fae79d640 1 -- 172.21.15.45:0/1724973299 >> v1:172.21.15.84:6812/31478 conn(0x7f4f9403aa00 legacy=0x7f4f9403ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.443+0000 7f4fae79d640 10 monclient: shutdown 2024-09-18T13:59:33.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.443+0000 7f4fae79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.443+0000 7f4fae79d640 1 -- 172.21.15.45:0/1724973299 >> v1:172.21.15.45:6792/0 conn(0x7f4fa809fba0 legacy=0x7f4fa8131ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.443+0000 7f4facd3a640 1 -- 172.21.15.45:0/1724973299 reap_dead start 2024-09-18T13:59:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.443+0000 7f4facd3a640 1 -- 172.21.15.45:0/1724973299 reap_dead start 2024-09-18T13:59:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.443+0000 7f4fae79d640 1 -- 172.21.15.45:0/1724973299 shutdown_connections 2024-09-18T13:59:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.443+0000 7f4fae79d640 1 -- 172.21.15.45:0/1724973299 >> 172.21.15.45:0/1724973299 conn(0x7f4fa80630b0 msgr2=0x7f4fa80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.443+0000 7f4fae79d640 1 -- 172.21.15.45:0/1724973299 shutdown_connections 2024-09-18T13:59:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.443+0000 7f4fae79d640 1 -- 172.21.15.45:0/1724973299 wait complete. 2024-09-18T13:59:33.490 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T13:59:33.490 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-09-18T13:59:33.491 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T13:59:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.588+0000 7f4cadd9d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.588+0000 7f4cadd9d640 10 monclient: build_initial_monmap 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.588+0000 7f4cadd9d640 10 monclient: monmap: 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:33.589152+0000 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:33.589152+0000 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:33.590 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:33.590 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:33.590 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:33.590 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.588+0000 7f4cadd9d640 1 Processor -- start 2024-09-18T13:59:33.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4cadd9d640 1 -- start start 2024-09-18T13:59:33.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4cadd9d640 10 monclient: init 2024-09-18T13:59:33.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4cadd9d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:33.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4cadd9d640 10 monclient: _add_conns ranks=[1,0,8,2,3,5,4,7,6] 2024-09-18T13:59:33.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4cadd9d640 10 monclient(hunting): picked mon.noname-f con 0x7f4ca809f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:33.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4cadd9d640 10 monclient(hunting): picked mon.noname-a con 0x7f4ca809fba0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4cadd9d640 10 monclient(hunting): picked mon.noname-e con 0x7f4ca809ff70 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4cadd9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca805ed70 con 0x7f4ca809fba0 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4cadd9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca809dfc0 con 0x7f4ca809f7d0 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4cadd9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca809dce0 con 0x7f4ca809ff70 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4cadd9d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4cadd9d640 10 monclient(hunting): authenticate will time out at 788.927530s 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca7fff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f4ca809ff70 0x7f4ca813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:58784/0 (socket says 172.21.15.45:58784) 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca6ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4ca809fba0 0x7f4ca8137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59374/0 (socket says 172.21.15.45:59374) 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca7fff640 1 -- 172.21.15.45:0/130820453 learned_addr learned my addr 172.21.15.45:0/130820453 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320299199 0 0) 0x7f4ca805ed70 con 0x7f4ca809fba0 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 10 monclient(hunting): my global_id is 4563 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c940034e0 con 0x7f4ca809fba0 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70758997 0 0) 0x7f4ca809dce0 con 0x7f4ca809ff70 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 10 monclient(hunting): my global_id is 4526 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ca805ed70 con 0x7f4ca809ff70 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266584104 0 0) 0x7f4ca809dfc0 con 0x7f4ca809f7d0 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 10 monclient(hunting): my global_id is 4483 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.589+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ca809dce0 con 0x7f4ca809f7d0 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1617717865 0 0) 0x7f4ca809dce0 con 0x7f4ca809f7d0 2024-09-18T13:59:33.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 >> v1:172.21.15.45:6793/0 conn(0x7f4ca809ff70 legacy=0x7f4ca813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 >> v1:172.21.15.45:6789/0 conn(0x7f4ca809fba0 legacy=0x7f4ca8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 1 monclient: found mon.noname-f 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca80a1c00 con 0x7f4ca809f7d0 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c98001ad0 con 0x7f4ca809f7d0 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 10 monclient: dump: 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.601 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4cadd9d640 5 monclient: authenticate success, global_id 4483 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4cadd9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4c98001cb0 con 0x7f4ca809f7d0 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 1 -- 172.21.15.45:0/130820453 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c980022a0 con 0x7f4ca809f7d0 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4cadd9d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4cadd9d640 4 set_mon_vals no callback set 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4ca67fc640 10 monclient: dump: 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.603 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.604 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4cadd9d640 10 monclient: shutdown 2024-09-18T13:59:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4cadd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4cadd9d640 1 -- 172.21.15.45:0/130820453 >> v1:172.21.15.84:6789/0 conn(0x7f4ca809f7d0 legacy=0x7f4ca8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4cadd9d640 1 -- 172.21.15.45:0/130820453 shutdown_connections 2024-09-18T13:59:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4cadd9d640 1 -- 172.21.15.45:0/130820453 >> 172.21.15.45:0/130820453 conn(0x7f4ca80630b0 msgr2=0x7f4ca80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4cadd9d640 1 -- 172.21.15.45:0/130820453 shutdown_connections 2024-09-18T13:59:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.590+0000 7f4cadd9d640 1 -- 172.21.15.45:0/130820453 wait complete. 2024-09-18T13:59:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.591+0000 7f4cadd9d640 10 monclient: build_initial_monmap 2024-09-18T13:59:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.591+0000 7f4cadd9d640 10 monclient: monmap: 2024-09-18T13:59:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:33.606 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.591+0000 7f4cadd9d640 1 Processor -- start 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.591+0000 7f4cadd9d640 1 -- start start 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.591+0000 7f4cadd9d640 10 monclient: init 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4cadd9d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4cadd9d640 10 monclient: _add_conns ranks=[0,7,4,6,1,5,3,8,2] 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4cadd9d640 10 monclient(hunting): picked mon.noname-a con 0x7f4ca809f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4cadd9d640 10 monclient(hunting): picked mon.noname-i con 0x7f4ca809fba0 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4cadd9d640 10 monclient(hunting): picked mon.noname-c con 0x7f4ca809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4cadd9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca80a2560 con 0x7f4ca809f7d0 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4cadd9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca8121ad0 con 0x7f4ca809ff70 2024-09-18T13:59:33.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4cadd9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca8142700 con 0x7f4ca809fba0 2024-09-18T13:59:33.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4cadd9d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:33.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4ca7fff640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f4ca809ff70 legacy=0x7f4ca814a260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T13:59:33.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4cadd9d640 10 monclient(hunting): authenticate will time out at 788.930500s 2024-09-18T13:59:33.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4ca77fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4ca809f7d0 0x7f4ca81316a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59384/0 (socket says 172.21.15.45:59384) 2024-09-18T13:59:33.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4ca77fe640 1 -- 172.21.15.45:0/4257841559 learned_addr learned my addr 172.21.15.45:0/4257841559 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:33.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252213183 0 0) 0x7f4ca80a2560 con 0x7f4ca809f7d0 2024-09-18T13:59:33.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4c9f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4c9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4c9f7fe640 10 monclient(hunting): my global_id is 4572 2024-09-18T13:59:33.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c880034e0 con 0x7f4ca809f7d0 2024-09-18T13:59:33.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402283514 0 0) 0x7f4ca8142700 con 0x7f4ca809fba0 2024-09-18T13:59:33.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4c9f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4c9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4c9f7fe640 10 monclient(hunting): my global_id is 4570 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ca80a2560 con 0x7f4ca809fba0 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3684655145 0 0) 0x7f4c880034e0 con 0x7f4ca809f7d0 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.592+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4ca8142700 con 0x7f4ca809f7d0 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c98002220 con 0x7f4ca809f7d0 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient(hunting): dump: 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.609 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 121719707 0 0) 0x7f4ca80a2560 con 0x7f4ca809fba0 2024-09-18T13:59:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4c880034e0 con 0x7f4ca809fba0 2024-09-18T13:59:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c900015b0 con 0x7f4ca809fba0 2024-09-18T13:59:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient(hunting): dump: 2024-09-18T13:59:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.611 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 439522564 0 0) 0x7f4ca8142700 con 0x7f4ca809f7d0 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 >> v1:172.21.15.84:6792/0 conn(0x7f4ca809fba0 legacy=0x7f4ca8146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 >> v1:172.21.15.45:6791/0 conn(0x7f4ca809ff70 legacy=0x7f4ca814a260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient: _finish_hunting 0 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 1 monclient: found mon.a 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca8132440 con 0x7f4ca809f7d0 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient: _finish_auth 0 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4cadd9d640 5 monclient: authenticate success, global_id 4572 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4cadd9d640 10 monclient: _renew_subs 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4cadd9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4cadd9d640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca81217f0 con 0x7f4ca809f7d0 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4cadd9d640 10 monclient: _renew_subs 2024-09-18T13:59:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4cadd9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4cadd9d640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca8120900 con 0x7f4ca809f7d0 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4c980028d0 con 0x7f4ca809f7d0 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c98002e00 con 0x7f4ca809f7d0 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4ca5ffb640 4 set_mon_vals no callback set 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.593+0000 7f4c9f7fe640 10 monclient: dump: 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4cadd9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4c98002f60 con 0x7f4ca809f7d0 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4cadd9d640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4c8c006580 con 0x7f4c8c002830 2024-09-18T13:59:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(20..20 src has 1..20) v4 ==== 3867+0+0 (unknown 541114214 0 0) 0x7f4c98047050 con 0x7f4ca809f7d0 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4ca7fff640 10 monclient: get_auth_request con 0x7f4c8803a330 auth_method 0 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998864005 0 0) 0x7f4c8c006580 con 0x7f4c8c002830 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 10 monclient(hunting): my global_id is 4492 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ca8142700 con 0x7f4c8c002830 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3022977363 0 0) 0x7f4ca8142700 con 0x7f4c8c002830 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4c8c006580 con 0x7f4c8c002830 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c90004d50 con 0x7f4c8c002830 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 10 monclient: dump: 2024-09-18T13:59:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.618 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3474613631 0 0) 0x7f4c8c006580 con 0x7f4c8c002830 2024-09-18T13:59:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.594+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c8c00a0a0 con 0x7f4c8c002830 2024-09-18T13:59:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.595+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f4c90007010 con 0x7f4c8c002830 2024-09-18T13:59:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.595+0000 7f4c9f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.595+0000 7f4c9f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.595+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 >> v1:172.21.15.84:6789/0 conn(0x7f4c8c002830 legacy=0x7f4c8c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.595+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 >> v1:172.21.15.84:6789/0 conn(0x7f4c8c002830 legacy=0x7f4c8c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:33.619 INFO:tasks.rados.rados.0.smithi084.stdout:43: done (8 left) 2024-09-18T13:59:33.620 INFO:tasks.rados.rados.0.smithi084.stdout:44: done (7 left) 2024-09-18T13:59:33.620 INFO:tasks.rados.rados.0.smithi084.stdout:45: done (6 left) 2024-09-18T13:59:33.620 INFO:tasks.rados.rados.0.smithi084.stdout:46: done (5 left) 2024-09-18T13:59:33.620 INFO:tasks.rados.rados.0.smithi084.stdout:47: done (4 left) 2024-09-18T13:59:33.620 INFO:tasks.rados.rados.0.smithi084.stdout:48: done (3 left) 2024-09-18T13:59:33.620 INFO:tasks.rados.rados.0.smithi084.stdout:49: done (2 left) 2024-09-18T13:59:33.620 INFO:tasks.rados.rados.0.smithi084.stdout:50: done (1 left) 2024-09-18T13:59:33.620 INFO:tasks.rados.rados.0.smithi084.stdout:51: done (0 left) 2024-09-18T13:59:33.620 INFO:tasks.rados.rados.0.smithi084.stdout:52: delete oid 40 current snap is 1 2024-09-18T13:59:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4cadd9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4cadd9d640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4c8c007540 con 0x7f4c8c0068f0 2024-09-18T13:59:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869023103 0 0) 0x7f4c8c007540 con 0x7f4c8c0068f0 2024-09-18T13:59:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 10 monclient(hunting): my global_id is 4501 2024-09-18T13:59:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c8c006580 con 0x7f4c8c0068f0 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 69221225 0 0) 0x7f4c8c006580 con 0x7f4c8c0068f0 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4c8c007540 con 0x7f4c8c0068f0 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4c90005910 con 0x7f4c8c0068f0 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 10 monclient: dump: 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 849667403 0 0) 0x7f4c8c007540 con 0x7f4c8c0068f0 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.601+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4c8c0087a0 con 0x7f4c8c0068f0 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7375 (unknown 0 0 2952624787) 0x7f4c90007240 con 0x7f4c8c0068f0 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4c9f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4c9f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 >> v1:172.21.15.84:6789/0 conn(0x7f4c8c0068f0 legacy=0x7f4c8c006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4c9f7fe640 1 -- 172.21.15.45:0/4257841559 >> v1:172.21.15.84:6789/0 conn(0x7f4c8c0068f0 legacy=0x7f4c8c006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:33.624 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 32, 2024-09-18T13:59:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:33.643 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4cadd9d640 1 -- 172.21.15.45:0/4257841559 >> v1:172.21.15.84:6812/31478 conn(0x7f4c8803a330 legacy=0x7f4c8803c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4cadd9d640 10 monclient: shutdown 2024-09-18T13:59:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4cadd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4cadd9d640 1 -- 172.21.15.45:0/4257841559 >> v1:172.21.15.45:6789/0 conn(0x7f4ca809f7d0 legacy=0x7f4ca81316a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4ca7fff640 1 -- 172.21.15.45:0/4257841559 reap_dead start 2024-09-18T13:59:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4ca7fff640 1 -- 172.21.15.45:0/4257841559 reap_dead start 2024-09-18T13:59:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4cadd9d640 1 -- 172.21.15.45:0/4257841559 shutdown_connections 2024-09-18T13:59:33.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4cadd9d640 1 -- 172.21.15.45:0/4257841559 >> 172.21.15.45:0/4257841559 conn(0x7f4ca80630b0 msgr2=0x7f4ca80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4cadd9d640 1 -- 172.21.15.45:0/4257841559 shutdown_connections 2024-09-18T13:59:33.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.602+0000 7f4cadd9d640 1 -- 172.21.15.45:0/4257841559 wait complete. 2024-09-18T13:59:33.646 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T13:59:33.646 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-09-18T13:59:33.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T13:59:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:52: done (0 left) 2024-09-18T13:59:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:53: read oid 10 snap 0 2024-09-18T13:59:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:53: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-18T13:59:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:54: read oid 32 snap 0 2024-09-18T13:59:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:54: expect (ObjNum 31 snap 0 seq_num 31) 2024-09-18T13:59:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:55: rollback oid 37 current snap is 1 2024-09-18T13:59:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 37 to 0 2024-09-18T13:59:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:56: delete oid 29 current snap is 1 2024-09-18T13:59:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:56: done (3 left) 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:57: copy_from oid 24 from oid 5 current snap is 1 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:58: read oid 28 snap -1 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:58: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:59: rollback oid 34 current snap is 1 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 34 to 0 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:60: delete oid 38 current snap is 1 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:59: finishing rollback tid 0 to smithi08434663-34 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:60: done (6 left) 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:61: read oid 30 snap -1 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:61: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:62: rollback oid 26 current snap is 1 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 26 to 0 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:63: rollback oid 33 current snap is 1 2024-09-18T13:59:33.650 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 33 to 0 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:59: finishing rollback tid 1 to smithi08434663-34 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:64: read oid 13 snap -1 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:64: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:65: read oid 14 snap -1 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:65: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:66: rollback oid 47 current snap is 1 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 47 to 0 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:67: write (excl) oid 2 current snap is 1 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:67: seq_num 53 ranges {490771=686315,1885075=406346} 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:67: writing smithi08434663-2 from 490771 to 1177086 tid 1 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:67: writing smithi08434663-2 from 1885075 to 2291421 tid 2 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:68: copy_from oid 39 from oid 1 current snap is 1 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:59: finishing rollback tid 2 to smithi08434663-34 2024-09-18T13:59:33.651 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 19 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T13:59:33.652 INFO:tasks.rados.rados.0.smithi084.stdout:54: expect (ObjNum 31 snap 0 seq_num 31) 2024-09-18T13:59:33.730 INFO:tasks.rados.rados.0.smithi084.stdout:66: finishing rollback tid 0 to smithi08434663-47 2024-09-18T13:59:33.730 INFO:tasks.rados.rados.0.smithi084.stdout:61: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-18T13:59:33.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.748+0000 7f104679d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:33.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.748+0000 7f104679d640 10 monclient: build_initial_monmap 2024-09-18T13:59:33.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.748+0000 7f104679d640 10 monclient: monmap: 2024-09-18T13:59:33.749 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:33.749 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.749 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:33.749582+0000 2024-09-18T13:59:33.749 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:33.749582+0000 2024-09-18T13:59:33.749 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:33.749 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:33.750 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.749+0000 7f104679d640 1 Processor -- start 2024-09-18T13:59:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.749+0000 7f104679d640 1 -- start start 2024-09-18T13:59:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.749+0000 7f104679d640 10 monclient: init 2024-09-18T13:59:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 10 monclient: _add_conns ranks=[0,3,2,1,6,7,5,4,8] 2024-09-18T13:59:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 10 monclient(hunting): picked mon.noname-a con 0x7f104009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 10 monclient(hunting): picked mon.noname-g con 0x7f104009fba0 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 10 monclient(hunting): picked mon.noname-b con 0x7f104009ff70 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104005ed70 con 0x7f104009f7d0 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104009dfc0 con 0x7f104009ff70 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104009dce0 con 0x7f104009fba0 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 10 monclient(hunting): authenticate will time out at 789.088140s 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103ffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f104009f7d0 0x7f1040132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59398/0 (socket says 172.21.15.45:59398) 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103ffff640 1 -- 172.21.15.45:0/1083262055 learned_addr learned my addr 172.21.15.45:0/1083262055 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619306885 0 0) 0x7f104005ed70 con 0x7f104009f7d0 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient(hunting): my global_id is 4581 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10280034e0 con 0x7f104009f7d0 2024-09-18T13:59:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020430335 0 0) 0x7f104009dfc0 con 0x7f104009ff70 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient(hunting): my global_id is 4574 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104005ed70 con 0x7f104009ff70 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402369400 0 0) 0x7f104009dce0 con 0x7f104009fba0 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient(hunting): my global_id is 4611 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104009dfc0 con 0x7f104009fba0 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 67244245 0 0) 0x7f10280034e0 con 0x7f104009f7d0 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 >> v1:172.21.15.84:6790/0 conn(0x7f104009fba0 legacy=0x7f1040137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 >> v1:172.21.15.45:6790/0 conn(0x7f104009ff70 legacy=0x7f104013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 monclient: found mon.noname-a 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10400a1c00 con 0x7f104009f7d0 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient: _finish_auth 0 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1030001830 con 0x7f104009f7d0 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 5 monclient: authenticate success, global_id 4581 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient: dump: 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1030001a10 con 0x7f104009f7d0 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 1 -- 172.21.15.45:0/1083262055 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1030004ba0 con 0x7f104009f7d0 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f104679d640 4 set_mon_vals no callback set 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.750+0000 7f103effd640 10 monclient: dump: 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.751+0000 7f104679d640 10 monclient: shutdown 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.751+0000 7f104679d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.751+0000 7f104679d640 1 -- 172.21.15.45:0/1083262055 >> v1:172.21.15.45:6789/0 conn(0x7f104009f7d0 legacy=0x7f1040132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.751+0000 7f104679d640 1 -- 172.21.15.45:0/1083262055 shutdown_connections 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.751+0000 7f104679d640 1 -- 172.21.15.45:0/1083262055 >> 172.21.15.45:0/1083262055 conn(0x7f10400630b0 msgr2=0x7f10400634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.751+0000 7f104679d640 1 -- 172.21.15.45:0/1083262055 shutdown_connections 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.751+0000 7f104679d640 1 -- 172.21.15.45:0/1083262055 wait complete. 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.751+0000 7f104679d640 10 monclient: build_initial_monmap 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.751+0000 7f104679d640 10 monclient: monmap: 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 1 Processor -- start 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 1 -- start start 2024-09-18T13:59:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 10 monclient: init 2024-09-18T13:59:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 10 monclient: _add_conns ranks=[5,1,0,7,3,2,4,8,6] 2024-09-18T13:59:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 10 monclient(hunting): picked mon.noname-h con 0x7f104009f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 10 monclient(hunting): picked mon.noname-f con 0x7f104009fba0 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 10 monclient(hunting): picked mon.noname-a con 0x7f104009ff70 addr v1:172.21.15.45:6789/0 2024-09-18T13:59:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10400a2560 con 0x7f104009ff70 2024-09-18T13:59:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104009d310 con 0x7f104009fba0 2024-09-18T13:59:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10401202a0 con 0x7f104009f7d0 2024-09-18T13:59:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f104679d640 10 monclient(hunting): authenticate will time out at 789.090894s 2024-09-18T13:59:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f1044d3a640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f104009ff70 0x7f1040149da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59412/0 (socket says 172.21.15.45:59412) 2024-09-18T13:59:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.752+0000 7f1044d3a640 1 -- 172.21.15.45:0/3383016261 learned_addr learned my addr 172.21.15.45:0/3383016261 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599037324 0 0) 0x7f10400a2560 con 0x7f104009ff70 2024-09-18T13:59:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): my global_id is 4590 2024-09-18T13:59:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f102c003510 con 0x7f104009ff70 2024-09-18T13:59:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447143103 0 0) 0x7f10401202a0 con 0x7f104009f7d0 2024-09-18T13:59:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): my global_id is 4460 2024-09-18T13:59:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10400a2560 con 0x7f104009f7d0 2024-09-18T13:59:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670161561 0 0) 0x7f104009d310 con 0x7f104009fba0 2024-09-18T13:59:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): my global_id is 4510 2024-09-18T13:59:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10401202a0 con 0x7f104009fba0 2024-09-18T13:59:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1806011480 0 0) 0x7f102c003510 con 0x7f104009ff70 2024-09-18T13:59:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f104009d310 con 0x7f104009ff70 2024-09-18T13:59:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1038001dc0 con 0x7f104009ff70 2024-09-18T13:59:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T13:59:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): dump: 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.771 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.771 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:33.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 890784568 0 0) 0x7f10400a2560 con 0x7f104009f7d0 2024-09-18T13:59:33.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f102c003510 con 0x7f104009f7d0 2024-09-18T13:59:33.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1030004ba0 con 0x7f104009f7d0 2024-09-18T13:59:33.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:33.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): dump: 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 240590909 0 0) 0x7f10401202a0 con 0x7f104009fba0 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10400a2560 con 0x7f104009fba0 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1034001a20 con 0x7f104009fba0 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient(hunting): dump: 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1532056611 0 0) 0x7f104009d310 con 0x7f104009ff70 2024-09-18T13:59:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 >> v1:172.21.15.84:6791/0 conn(0x7f104009f7d0 legacy=0x7f1040141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 >> v1:172.21.15.84:6789/0 conn(0x7f104009fba0 legacy=0x7f1040131c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient: _finish_hunting 0 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 monclient: found mon.a 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1040142860 con 0x7f104009ff70 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient: _finish_auth 0 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f104679d640 5 monclient: authenticate success, global_id 4590 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f104679d640 10 monclient: _renew_subs 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f104679d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f104679d640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1040120990 con 0x7f104009ff70 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f104679d640 10 monclient: _renew_subs 2024-09-18T13:59:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f104679d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f104679d640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1040120e70 con 0x7f104009ff70 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1038002310 con 0x7f104009ff70 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1038005050 con 0x7f104009ff70 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f103e7fc640 4 set_mon_vals no callback set 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.753+0000 7f1027fff640 10 monclient: dump: 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.776 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.754+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1038010020 con 0x7f104009ff70 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.754+0000 7f104679d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:33.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.754+0000 7f104679d640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1020006580 con 0x7f1020002830 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.754+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(21..21 src has 1..21) v4 ==== 3867+0+0 (unknown 1530407583 0 0) 0x7f1038046ce0 con 0x7f104009ff70 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.754+0000 7f103ffff640 10 monclient: get_auth_request con 0x7f102c03a9b0 auth_method 0 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167930147 0 0) 0x7f1020006580 con 0x7f1020002830 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 10 monclient(hunting): my global_id is 4620 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104009d310 con 0x7f1020002830 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3492601352 0 0) 0x7f104009d310 con 0x7f1020002830 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1020006580 con 0x7f1020002830 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1034002170 con 0x7f1020002830 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 10 monclient: dump: 2024-09-18T13:59:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.780 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2251385901 0 0) 0x7f1020006580 con 0x7f1020002830 2024-09-18T13:59:33.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f102000a0a0 con 0x7f1020002830 2024-09-18T13:59:33.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1034002670 con 0x7f1020002830 2024-09-18T13:59:33.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:33.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:33.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 >> v1:172.21.15.84:6790/0 conn(0x7f1020002830 legacy=0x7f1020004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.755+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 >> v1:172.21.15.84:6790/0 conn(0x7f1020002830 legacy=0x7f1020004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:33.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.761+0000 7f104679d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.761+0000 7f104679d640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f10200074f0 con 0x7f10200068a0 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.761+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431342953 0 0) 0x7f10200074f0 con 0x7f10200068a0 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.761+0000 7f1027fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.761+0000 7f1027fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.761+0000 7f1027fff640 10 monclient(hunting): my global_id is 4629 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.761+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1020006580 con 0x7f10200068a0 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 404107567 0 0) 0x7f1020006580 con 0x7f10200068a0 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10200074f0 con 0x7f10200068a0 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1034003000 con 0x7f10200068a0 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 10 monclient: dump: 2024-09-18T13:59:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.783 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2256063227 0 0) 0x7f10200074f0 con 0x7f10200068a0 2024-09-18T13:59:33.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1020008750 con 0x7f10200068a0 2024-09-18T13:59:33.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7715 (unknown 0 0 1500310337) 0x7f10340032a0 con 0x7f10200068a0 2024-09-18T13:59:33.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:33.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:33.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 >> v1:172.21.15.84:6790/0 conn(0x7f10200068a0 legacy=0x7f1020006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1027fff640 1 -- 172.21.15.45:0/3383016261 >> v1:172.21.15.84:6790/0 conn(0x7f10200068a0 legacy=0x7f1020006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:33.784 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:33.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:33.784 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 32, 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.785 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.786 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.787 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:33.788 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.789 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:33.793 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.794 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.797 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:33.798 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:33.799 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.801 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.802 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T13:59:33.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.804 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.804 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T13:59:33.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.804 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.804 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.804 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.804 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.804 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.804 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:33.804 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:33.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f104679d640 1 -- 172.21.15.45:0/3383016261 >> v1:172.21.15.84:6812/31478 conn(0x7f102c03a9b0 legacy=0x7f102c03ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f104679d640 10 monclient: shutdown 2024-09-18T13:59:33.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f104679d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f104679d640 1 -- 172.21.15.45:0/3383016261 >> v1:172.21.15.45:6789/0 conn(0x7f104009ff70 legacy=0x7f1040149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1044d3a640 1 -- 172.21.15.45:0/3383016261 reap_dead start 2024-09-18T13:59:33.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f1044d3a640 1 -- 172.21.15.45:0/3383016261 reap_dead start 2024-09-18T13:59:33.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f104679d640 1 -- 172.21.15.45:0/3383016261 shutdown_connections 2024-09-18T13:59:33.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.762+0000 7f104679d640 1 -- 172.21.15.45:0/3383016261 >> 172.21.15.45:0/3383016261 conn(0x7f10400630b0 msgr2=0x7f10400a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.763+0000 7f104679d640 1 -- 172.21.15.45:0/3383016261 shutdown_connections 2024-09-18T13:59:33.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.763+0000 7f104679d640 1 -- 172.21.15.45:0/3383016261 wait complete. 2024-09-18T13:59:33.807 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T13:59:33.807 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-09-18T13:59:33.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T13:59:33.812 INFO:tasks.rados.rados.0.smithi084.stdout:58: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-18T13:59:33.888 INFO:tasks.rados.rados.0.smithi084.stdout:66: finishing rollback tid 1 to smithi08434663-47 2024-09-18T13:59:33.888 INFO:tasks.rados.rados.0.smithi084.stdout:66: finishing rollback tid 2 to smithi08434663-47 2024-09-18T13:59:33.888 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 13 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T13:59:33.888 INFO:tasks.rados.rados.0.smithi084.stdout:53: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-18T13:59:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.904+0000 7fd229834640 10 monclient: get_monmap_and_config 2024-09-18T13:59:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.904+0000 7fd229834640 10 monclient: build_initial_monmap 2024-09-18T13:59:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.904+0000 7fd229834640 10 monclient: monmap: 2024-09-18T13:59:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:33.905113+0000 2024-09-18T13:59:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:33.905113+0000 2024-09-18T13:59:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:33.906 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.904+0000 7fd229834640 1 Processor -- start 2024-09-18T13:59:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.904+0000 7fd229834640 1 -- start start 2024-09-18T13:59:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.904+0000 7fd229834640 10 monclient: init 2024-09-18T13:59:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd229834640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd229834640 10 monclient: _add_conns ranks=[2,4,3,7,0,8,6,1,5] 2024-09-18T13:59:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd229834640 10 monclient(hunting): picked mon.noname-b con 0x7fd22406e810 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd229834640 10 monclient(hunting): picked mon.noname-c con 0x7fd2240a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd229834640 10 monclient(hunting): picked mon.noname-g con 0x7fd2240a1080 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd229834640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22405ed70 con 0x7fd22406e810 2024-09-18T13:59:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd229834640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd224097620 con 0x7fd2240a1080 2024-09-18T13:59:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd229834640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd224097340 con 0x7fd2240a0cb0 2024-09-18T13:59:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd229834640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd229834640 10 monclient(hunting): authenticate will time out at 789.243323s 2024-09-18T13:59:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd2227fc640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7fd2240a0cb0 legacy=0x7fd224137810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T13:59:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd222ffd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fd22406e810 0x7fd2240a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:45936/0 (socket says 172.21.15.45:45936) 2024-09-18T13:59:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd222ffd640 1 -- 172.21.15.45:0/3051401886 learned_addr learned my addr 172.21.15.45:0/3051401886 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 1 -- 172.21.15.45:0/3051401886 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336628423 0 0) 0x7fd22405ed70 con 0x7fd22406e810 2024-09-18T13:59:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 10 monclient(hunting): my global_id is 4583 2024-09-18T13:59:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 1 -- 172.21.15.45:0/3051401886 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2100034e0 con 0x7fd22406e810 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 1 -- 172.21.15.45:0/3051401886 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384701761 0 0) 0x7fd224097620 con 0x7fd2240a1080 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 10 monclient(hunting): my global_id is 4638 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 1 -- 172.21.15.45:0/3051401886 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22405ed70 con 0x7fd2240a1080 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 1 -- 172.21.15.45:0/3051401886 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 294580166 0 0) 0x7fd2100034e0 con 0x7fd22406e810 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 1 -- 172.21.15.45:0/3051401886 >> v1:172.21.15.45:6791/0 conn(0x7fd2240a0cb0 legacy=0x7fd224137810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 1 -- 172.21.15.45:0/3051401886 >> v1:172.21.15.84:6790/0 conn(0x7fd2240a1080 legacy=0x7fd22413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 10 monclient: _finish_hunting 0 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 1 monclient: found mon.noname-b 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 1 -- 172.21.15.45:0/3051401886 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2240a1660 con 0x7fd22406e810 2024-09-18T13:59:33.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 10 monclient: _finish_auth 0 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 1 -- 172.21.15.45:0/3051401886 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd214001830 con 0x7fd22406e810 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd221ffb640 10 monclient: dump: 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.916 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.917 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.917 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.917 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.917 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.917 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.917 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.917 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.917 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd229834640 5 monclient: authenticate success, global_id 4583 2024-09-18T13:59:33.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.905+0000 7fd229834640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:33.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd221ffb640 1 -- 172.21.15.45:0/3051401886 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd214001a10 con 0x7fd22406e810 2024-09-18T13:59:33.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd221ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:33.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd221ffb640 1 -- 172.21.15.45:0/3051401886 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd214004ba0 con 0x7fd22406e810 2024-09-18T13:59:33.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd229834640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:33.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd229834640 4 set_mon_vals no callback set 2024-09-18T13:59:33.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd221ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd221ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T13:59:33.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd221ffb640 10 monclient: dump: 2024-09-18T13:59:33.920 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.920 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.920 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.920 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.920 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.920 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.920 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.921 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd229834640 10 monclient: shutdown 2024-09-18T13:59:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd229834640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd229834640 1 -- 172.21.15.45:0/3051401886 >> v1:172.21.15.45:6790/0 conn(0x7fd22406e810 legacy=0x7fd2240a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd229834640 1 -- 172.21.15.45:0/3051401886 shutdown_connections 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd229834640 1 -- 172.21.15.45:0/3051401886 >> 172.21.15.45:0/3051401886 conn(0x7fd2240a5bb0 msgr2=0x7fd2240a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd229834640 1 -- 172.21.15.45:0/3051401886 shutdown_connections 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.906+0000 7fd229834640 1 -- 172.21.15.45:0/3051401886 wait complete. 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.907+0000 7fd229834640 10 monclient: build_initial_monmap 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.907+0000 7fd229834640 10 monclient: monmap: 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:33.922 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:33.923 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:33.923 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:33.923 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:33.923 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:33.923 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:33.923 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:33.923 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:33.923 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:33.923 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.907+0000 7fd229834640 1 Processor -- start 2024-09-18T13:59:33.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.907+0000 7fd229834640 1 -- start start 2024-09-18T13:59:33.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.907+0000 7fd229834640 10 monclient: init 2024-09-18T13:59:33.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.907+0000 7fd229834640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:33.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.907+0000 7fd229834640 10 monclient: _add_conns ranks=[1,6,5,4,8,3,0,2,7] 2024-09-18T13:59:33.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.907+0000 7fd229834640 10 monclient(hunting): picked mon.noname-f con 0x7fd22406e810 addr v1:172.21.15.84:6789/0 2024-09-18T13:59:33.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.907+0000 7fd229834640 10 monclient(hunting): picked mon.noname-d con 0x7fd2240a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T13:59:33.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.907+0000 7fd229834640 10 monclient(hunting): picked mon.noname-h con 0x7fd2240a1080 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:33.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.907+0000 7fd229834640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd224122cf0 con 0x7fd22406e810 2024-09-18T13:59:33.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.907+0000 7fd229834640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22413dae0 con 0x7fd2240a1080 2024-09-18T13:59:33.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd229834640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd224122730 con 0x7fd2240a0cb0 2024-09-18T13:59:33.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd229834640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:33.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd229834640 10 monclient(hunting): authenticate will time out at 789.246018s 2024-09-18T13:59:33.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd2227fc640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fd2240a0cb0 0x7fd224146600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:58024/0 (socket says 172.21.15.45:58024) 2024-09-18T13:59:33.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd2227fc640 1 -- 172.21.15.45:0/2422097190 learned_addr learned my addr 172.21.15.45:0/2422097190 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:33.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438472648 0 0) 0x7fd224122730 con 0x7fd2240a0cb0 2024-09-18T13:59:33.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): my global_id is 4515 2024-09-18T13:59:33.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2040034e0 con 0x7fd2240a0cb0 2024-09-18T13:59:33.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822399652 0 0) 0x7fd22413dae0 con 0x7fd2240a1080 2024-09-18T13:59:33.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): my global_id is 4469 2024-09-18T13:59:33.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd224122730 con 0x7fd2240a1080 2024-09-18T13:59:33.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197254539 0 0) 0x7fd224122cf0 con 0x7fd22406e810 2024-09-18T13:59:33.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): my global_id is 4519 2024-09-18T13:59:33.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22413dae0 con 0x7fd22406e810 2024-09-18T13:59:33.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1727842392 0 0) 0x7fd2040034e0 con 0x7fd2240a0cb0 2024-09-18T13:59:33.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd224122cf0 con 0x7fd2240a0cb0 2024-09-18T13:59:33.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd20c004340 con 0x7fd2240a0cb0 2024-09-18T13:59:33.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T13:59:33.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): dump: 2024-09-18T13:59:33.927 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.927 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.927 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.927 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.928 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.928 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.928 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.928 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.928 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.928 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.928 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.928 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.928 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.928 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.929 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.929 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:33.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 32177619 0 0) 0x7fd224122730 con 0x7fd2240a1080 2024-09-18T13:59:33.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2040034e0 con 0x7fd2240a1080 2024-09-18T13:59:33.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd21c001d00 con 0x7fd2240a1080 2024-09-18T13:59:33.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:33.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): dump: 2024-09-18T13:59:33.930 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.930 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.930 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.931 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.931 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.931 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.931 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.931 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.931 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.931 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.932 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.932 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.932 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.932 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.932 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.932 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1420512661 0 0) 0x7fd22413dae0 con 0x7fd22406e810 2024-09-18T13:59:33.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd224122730 con 0x7fd22406e810 2024-09-18T13:59:33.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd2140017f0 con 0x7fd22406e810 2024-09-18T13:59:33.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T13:59:33.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient(hunting): dump: 2024-09-18T13:59:33.933 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.933 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.933 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.933 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.933 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.934 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.934 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.934 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.934 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.934 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.934 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.934 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.934 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.935 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.935 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.935 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3288109478 0 0) 0x7fd224122cf0 con 0x7fd2240a0cb0 2024-09-18T13:59:33.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 >> v1:172.21.15.84:6791/0 conn(0x7fd2240a1080 legacy=0x7fd224149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 >> v1:172.21.15.84:6789/0 conn(0x7fd22406e810 legacy=0x7fd22413d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient: _finish_hunting 0 2024-09-18T13:59:33.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 monclient: found mon.d 2024-09-18T13:59:33.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2241201f0 con 0x7fd2240a0cb0 2024-09-18T13:59:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient: _finish_auth 0 2024-09-18T13:59:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd229834640 5 monclient: authenticate success, global_id 4515 2024-09-18T13:59:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd229834640 10 monclient: _renew_subs 2024-09-18T13:59:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd229834640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd229834640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd224122450 con 0x7fd2240a0cb0 2024-09-18T13:59:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd229834640 10 monclient: _renew_subs 2024-09-18T13:59:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd229834640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T13:59:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd229834640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd224121580 con 0x7fd2240a0cb0 2024-09-18T13:59:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd20c004520 con 0x7fd2240a0cb0 2024-09-18T13:59:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd21affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.908+0000 7fd2217fa640 4 set_mon_vals no callback set 2024-09-18T13:59:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.909+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd20c004a50 con 0x7fd2240a0cb0 2024-09-18T13:59:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.909+0000 7fd21affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.909+0000 7fd21affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T13:59:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.909+0000 7fd21affd640 10 monclient: dump: 2024-09-18T13:59:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.940 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.909+0000 7fd229834640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.909+0000 7fd229834640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd208006580 con 0x7fd208002830 2024-09-18T13:59:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.909+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd20c004c30 con 0x7fd2240a0cb0 2024-09-18T13:59:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(21..21 src has 1..21) v4 ==== 3867+0+0 (unknown 1530407583 0 0) 0x7fd20c046b60 con 0x7fd2240a0cb0 2024-09-18T13:59:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251642047 0 0) 0x7fd208006580 con 0x7fd208002830 2024-09-18T13:59:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 10 monclient(hunting): my global_id is 4478 2024-09-18T13:59:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd224122cf0 con 0x7fd208002830 2024-09-18T13:59:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd2237fe640 10 monclient: get_auth_request con 0x7fd20403a980 auth_method 0 2024-09-18T13:59:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1310067148 0 0) 0x7fd224122cf0 con 0x7fd208002830 2024-09-18T13:59:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd208006580 con 0x7fd208002830 2024-09-18T13:59:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd214004be0 con 0x7fd208002830 2024-09-18T13:59:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 10 monclient: dump: 2024-09-18T13:59:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.944 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1879069857 0 0) 0x7fd208006580 con 0x7fd208002830 2024-09-18T13:59:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd20800a0a0 con 0x7fd208002830 2024-09-18T13:59:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd214005210 con 0x7fd208002830 2024-09-18T13:59:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 >> v1:172.21.15.84:6791/0 conn(0x7fd208002830 legacy=0x7fd208004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.910+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 >> v1:172.21.15.84:6791/0 conn(0x7fd208002830 legacy=0x7fd208004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.916+0000 7fd229834640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.916+0000 7fd229834640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd208007540 con 0x7fd2080068f0 2024-09-18T13:59:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.916+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194210544 0 0) 0x7fd208007540 con 0x7fd2080068f0 2024-09-18T13:59:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 10 monclient(hunting): my global_id is 4487 2024-09-18T13:59:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd208006580 con 0x7fd2080068f0 2024-09-18T13:59:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 781137390 0 0) 0x7fd208006580 con 0x7fd2080068f0 2024-09-18T13:59:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd208007540 con 0x7fd2080068f0 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd214005970 con 0x7fd2080068f0 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 10 monclient: dump: 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1083895297 0 0) 0x7fd208007540 con 0x7fd2080068f0 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd2080087a0 con 0x7fd2080068f0 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7545 (unknown 0 0 3751586555) 0x7fd214005cd0 con 0x7fd2080068f0 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 >> v1:172.21.15.84:6791/0 conn(0x7fd2080068f0 legacy=0x7fd208006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd21affd640 1 -- 172.21.15.45:0/2422097190 >> v1:172.21.15.84:6791/0 conn(0x7fd2080068f0 legacy=0x7fd208006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:33.949 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 32, 2024-09-18T13:59:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:33.949 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.950 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:33.951 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:33.952 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:33.956 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.958 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.958 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:33.958 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:33.958 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.958 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.958 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.959 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:33.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:33.959 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.959 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.959 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:33.959 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.959 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.959 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:33.960 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:33.960 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.960 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.960 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.960 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:33.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:33.960 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.960 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.961 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:33.961 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.961 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.961 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.961 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:33.961 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:33.962 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.962 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.962 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.962 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:33.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:33.962 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.962 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.963 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:33.963 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.963 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.963 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.963 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:33.963 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:33.963 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.964 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.964 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.964 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:33.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:33.964 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.964 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:33.965 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.965 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:33.965 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:33.965 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.965 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.965 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.970 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:33.971 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:33.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:33.972 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:33.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:33.972 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:33.972 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:33.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd229834640 1 -- 172.21.15.45:0/2422097190 >> v1:172.21.15.84:6812/31478 conn(0x7fd20403a980 legacy=0x7fd20403ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd229834640 10 monclient: shutdown 2024-09-18T13:59:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd229834640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd229834640 1 -- 172.21.15.45:0/2422097190 >> v1:172.21.15.45:6792/0 conn(0x7fd2240a0cb0 legacy=0x7fd224146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.917+0000 7fd2237fe640 1 -- 172.21.15.45:0/2422097190 reap_dead start 2024-09-18T13:59:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.918+0000 7fd2237fe640 1 -- 172.21.15.45:0/2422097190 reap_dead start 2024-09-18T13:59:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.918+0000 7fd229834640 1 -- 172.21.15.45:0/2422097190 shutdown_connections 2024-09-18T13:59:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.918+0000 7fd229834640 1 -- 172.21.15.45:0/2422097190 >> 172.21.15.45:0/2422097190 conn(0x7fd2240a5bb0 msgr2=0x7fd2240a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.918+0000 7fd229834640 1 -- 172.21.15.45:0/2422097190 shutdown_connections 2024-09-18T13:59:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:33.918+0000 7fd229834640 1 -- 172.21.15.45:0/2422097190 wait complete. 2024-09-18T13:59:33.974 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T13:59:33.974 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-09-18T13:59:33.975 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T13:59:33.975 INFO:tasks.rados.rados.0.smithi084.stdout:62: finishing rollback tid 0 to smithi08434663-26 2024-09-18T13:59:33.975 INFO:tasks.rados.rados.0.smithi084.stdout:62: finishing rollback tid 1 to smithi08434663-26 2024-09-18T13:59:33.975 INFO:tasks.rados.rados.0.smithi084.stdout:62: finishing rollback tid 2 to smithi08434663-26 2024-09-18T13:59:33.975 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 16 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T13:59:33.976 INFO:tasks.rados.rados.0.smithi084.stdout:57: finishing copy_from racing read to smithi08434663-24 2024-09-18T13:59:33.976 INFO:tasks.rados.rados.0.smithi084.stdout:57: finishing copy_from to smithi08434663-24 2024-09-18T13:59:33.976 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 3 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T13:59:33.976 INFO:tasks.rados.rados.0.smithi084.stdout:55: finishing rollback tid 0 to smithi08434663-37 2024-09-18T13:59:33.976 INFO:tasks.rados.rados.0.smithi084.stdout:55: finishing rollback tid 1 to smithi08434663-37 2024-09-18T13:59:33.976 INFO:tasks.rados.rados.0.smithi084.stdout:55: finishing rollback tid 2 to smithi08434663-37 2024-09-18T13:59:33.976 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 16 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T13:59:33.976 INFO:tasks.rados.rados.0.smithi084.stdout:64: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-18T13:59:34.025 INFO:tasks.rados.rados.0.smithi084.stdout:65: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-18T13:59:34.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.072+0000 7f4f7611d640 10 monclient: get_monmap_and_config 2024-09-18T13:59:34.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.072+0000 7f4f7611d640 10 monclient: build_initial_monmap 2024-09-18T13:59:34.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.072+0000 7f4f7611d640 10 monclient: monmap: 2024-09-18T13:59:34.088 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:34.088 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:34.088 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:59:34.073776+0000 2024-09-18T13:59:34.089 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:59:34.073776+0000 2024-09-18T13:59:34.089 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:34.089 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:34.089 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:34.089 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:34.089 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:34.089 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:34.089 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:34.089 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:34.089 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:34.089 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.073+0000 7f4f7611d640 1 Processor -- start 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.073+0000 7f4f7611d640 1 -- start start 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.073+0000 7f4f7611d640 10 monclient: init 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 10 monclient: _add_conns ranks=[4,2,8,0,1,3,5,6,7] 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 10 monclient(hunting): picked mon.noname-c con 0x7f4f7006e810 addr v1:172.21.15.45:6791/0 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 10 monclient(hunting): picked mon.noname-b con 0x7f4f700a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 10 monclient(hunting): picked mon.noname-e con 0x7f4f700a1080 addr v1:172.21.15.45:6793/0 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7005ed70 con 0x7f4f700a0cb0 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f70097620 con 0x7f4f7006e810 2024-09-18T13:59:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f70097340 con 0x7f4f700a1080 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 10 monclient(hunting): authenticate will time out at 789.412108s 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6f7fe640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f4f7006e810 legacy=0x7f4f700a05a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6effd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f4f700a0cb0 0x7f4f70137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:45942/0 (socket says 172.21.15.45:45942) 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6effd640 1 -- 172.21.15.45:0/3202430198 learned_addr learned my addr 172.21.15.45:0/3202430198 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6ffff640 1 --1- 172.21.15.45:0/3202430198 >> v1:172.21.15.45:6793/0 conn(0x7f4f700a1080 0x7f4f7013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:58786/0 (socket says 172.21.15.45:58786) 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 1 -- 172.21.15.45:0/3202430198 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020530336 0 0) 0x7f4f70097340 con 0x7f4f700a1080 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient(hunting): my global_id is 4535 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 1 -- 172.21.15.45:0/3202430198 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f580034e0 con 0x7f4f700a1080 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 1 -- 172.21.15.45:0/3202430198 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995462789 0 0) 0x7f4f7005ed70 con 0x7f4f700a0cb0 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient(hunting): my global_id is 4592 2024-09-18T13:59:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 1 -- 172.21.15.45:0/3202430198 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f70097340 con 0x7f4f700a0cb0 2024-09-18T13:59:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 1 -- 172.21.15.45:0/3202430198 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2436091959 0 0) 0x7f4f580034e0 con 0x7f4f700a1080 2024-09-18T13:59:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 1 -- 172.21.15.45:0/3202430198 >> v1:172.21.15.45:6791/0 conn(0x7f4f7006e810 legacy=0x7f4f700a05a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T13:59:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 1 -- 172.21.15.45:0/3202430198 >> v1:172.21.15.45:6790/0 conn(0x7f4f700a0cb0 legacy=0x7f4f70137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient: _finish_hunting 0 2024-09-18T13:59:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 1 monclient: found mon.noname-e 2024-09-18T13:59:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T13:59:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 1 -- 172.21.15.45:0/3202430198 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f700a1660 con 0x7f4f700a1080 2024-09-18T13:59:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient: _finish_auth 0 2024-09-18T13:59:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 1 -- 172.21.15.45:0/3202430198 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f68001bb0 con 0x7f4f700a1080 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 5 monclient: authenticate success, global_id 4535 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient: dump: 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:34.094 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T13:59:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 1 -- 172.21.15.45:0/3202430198 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4f68001d90 con 0x7f4f700a1080 2024-09-18T13:59:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 10 monclient: get_monmap_and_config success 2024-09-18T13:59:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 1 -- 172.21.15.45:0/3202430198 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f68004e60 con 0x7f4f700a1080 2024-09-18T13:59:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f7611d640 4 set_mon_vals no callback set 2024-09-18T13:59:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T13:59:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.074+0000 7f4f6e7fc640 10 monclient: dump: 2024-09-18T13:59:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:34.097 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.075+0000 7f4f7611d640 10 monclient: shutdown 2024-09-18T13:59:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.075+0000 7f4f7611d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.075+0000 7f4f7611d640 1 -- 172.21.15.45:0/3202430198 >> v1:172.21.15.45:6793/0 conn(0x7f4f700a1080 legacy=0x7f4f7013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.075+0000 7f4f7611d640 1 -- 172.21.15.45:0/3202430198 shutdown_connections 2024-09-18T13:59:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.075+0000 7f4f7611d640 1 -- 172.21.15.45:0/3202430198 >> 172.21.15.45:0/3202430198 conn(0x7f4f700a5bb0 msgr2=0x7f4f700a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.075+0000 7f4f7611d640 1 -- 172.21.15.45:0/3202430198 shutdown_connections 2024-09-18T13:59:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.075+0000 7f4f7611d640 1 -- 172.21.15.45:0/3202430198 wait complete. 2024-09-18T13:59:34.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.075+0000 7f4f7611d640 10 monclient: build_initial_monmap 2024-09-18T13:59:34.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.075+0000 7f4f7611d640 10 monclient: monmap: 2024-09-18T13:59:34.098 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T13:59:34.098 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T13:59:34.098 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T13:59:34.098 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T13:59:34.098 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T13:59:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T13:59:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T13:59:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T13:59:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T13:59:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T13:59:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T13:59:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T13:59:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T13:59:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 1 Processor -- start 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 1 -- start start 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 10 monclient: init 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 10 monclient: _reopen_session rank -1 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 10 monclient: _add_conns ranks=[7,5,3,6,4,0,8,2,1] 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 10 monclient(hunting): picked mon.noname-i con 0x7f4f7006e810 addr v1:172.21.15.84:6792/0 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 10 monclient(hunting): picked mon.noname-h con 0x7f4f700a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 10 monclient(hunting): picked mon.noname-g con 0x7f4f700a1080 addr v1:172.21.15.84:6790/0 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7009fa10 con 0x7f4f700a1080 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f700a2270 con 0x7f4f700a0cb0 2024-09-18T13:59:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7011e1c0 con 0x7f4f7006e810 2024-09-18T13:59:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 10 monclient(hunting): _renew_subs 2024-09-18T13:59:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.076+0000 7f4f7611d640 10 monclient(hunting): authenticate will time out at 789.414863s 2024-09-18T13:59:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f6f7fe640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f4f7006e810 0x7f4f7009d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:41594/0 (socket says 172.21.15.45:41594) 2024-09-18T13:59:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f6effd640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f4f700a0cb0 0x7f4f7009df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:38396/0 (socket says 172.21.15.45:38396) 2024-09-18T13:59:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f6ffff640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7f4f700a1080 0x7f4f7009e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:35862/0 (socket says 172.21.15.45:35862) 2024-09-18T13:59:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f6f7fe640 1 -- 172.21.15.45:0/3722290268 learned_addr learned my addr 172.21.15.45:0/3722290268 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T13:59:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097066138 0 0) 0x7f4f7011e1c0 con 0x7f4f7006e810 2024-09-18T13:59:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): my global_id is 4579 2024-09-18T13:59:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f5c0034e0 con 0x7f4f7006e810 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926157008 0 0) 0x7f4f700a2270 con 0x7f4f700a0cb0 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): my global_id is 4496 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f7011e1c0 con 0x7f4f700a0cb0 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023605426 0 0) 0x7f4f7009fa10 con 0x7f4f700a1080 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): my global_id is 4647 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f700a2270 con 0x7f4f700a1080 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 320457078 0 0) 0x7f4f700a2270 con 0x7f4f700a1080 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f7009fa10 con 0x7f4f700a1080 2024-09-18T13:59:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 116808210 0 0) 0x7f4f5c0034e0 con 0x7f4f7006e810 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f700a2270 con 0x7f4f7006e810 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1459831421 0 0) 0x7f4f7011e1c0 con 0x7f4f700a0cb0 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f5c0034e0 con 0x7f4f700a0cb0 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f68001b70 con 0x7f4f700a1080 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): dump: 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f60004270 con 0x7f4f7006e810 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): dump: 2024-09-18T13:59:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:34.106 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:34.106 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:34.106 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:34.106 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:34.106 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:34.106 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:34.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f64001ba0 con 0x7f4f700a0cb0 2024-09-18T13:59:34.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:34.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient(hunting): dump: 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:34.108 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:34.108 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:34.108 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:34.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1121005402 0 0) 0x7f4f7009fa10 con 0x7f4f700a1080 2024-09-18T13:59:34.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 >> v1:172.21.15.84:6792/0 conn(0x7f4f7006e810 legacy=0x7f4f7009d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:34.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 >> v1:172.21.15.84:6791/0 conn(0x7f4f700a0cb0 legacy=0x7f4f7009df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:34.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient: _finish_hunting 0 2024-09-18T13:59:34.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 monclient: found mon.g 2024-09-18T13:59:34.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T13:59:34.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:34.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7011cff0 con 0x7f4f700a1080 2024-09-18T13:59:34.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient: _finish_auth 0 2024-09-18T13:59:34.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T13:59:34.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f7611d640 5 monclient: authenticate success, global_id 4647 2024-09-18T13:59:34.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f7611d640 10 monclient: _renew_subs 2024-09-18T13:59:34.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f7611d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:34.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f7611d640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7011c420 con 0x7f4f700a1080 2024-09-18T13:59:34.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f7611d640 10 monclient: _renew_subs 2024-09-18T13:59:34.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f7611d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T13:59:34.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f7611d640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f701300a0 con 0x7f4f700a1080 2024-09-18T13:59:34.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4f68001590 con 0x7f4f700a1080 2024-09-18T13:59:34.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T13:59:34.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f68005110 con 0x7f4f700a1080 2024-09-18T13:59:34.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:34.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.078+0000 7f4f577fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T13:59:34.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.077+0000 7f4f6dffb640 4 set_mon_vals no callback set 2024-09-18T13:59:34.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.078+0000 7f4f577fe640 10 monclient: dump: 2024-09-18T13:59:34.110 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.078+0000 7f4f7611d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.078+0000 7f4f7611d640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4f50006580 con 0x7f4f50002830 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.078+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4f680100e0 con 0x7f4f700a1080 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.078+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(21..21 src has 1..21) v4 ==== 3867+0+0 (unknown 1530407583 0 0) 0x7f4f68046e00 con 0x7f4f700a1080 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133916312 0 0) 0x7f4f50006580 con 0x7f4f50002830 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 10 monclient(hunting): my global_id is 4588 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f7009fa10 con 0x7f4f50002830 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f6effd640 10 monclient: get_auth_request con 0x7f4f5c03aba0 auth_method 0 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3909527971 0 0) 0x7f4f7009fa10 con 0x7f4f50002830 2024-09-18T13:59:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f50006580 con 0x7f4f50002830 2024-09-18T13:59:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f60004a00 con 0x7f4f50002830 2024-09-18T13:59:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 10 monclient: dump: 2024-09-18T13:59:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2326569244 0 0) 0x7f4f50006580 con 0x7f4f50002830 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f5000a0a0 con 0x7f4f50002830 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f4f60004f30 con 0x7f4f50002830 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T13:59:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 >> v1:172.21.15.84:6792/0 conn(0x7f4f50002830 legacy=0x7f4f50004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.079+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 >> v1:172.21.15.84:6792/0 conn(0x7f4f50002830 legacy=0x7f4f50004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.085+0000 7f4f7611d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T13:59:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.085+0000 7f4f7611d640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4f50009e70 con 0x7f4f500068f0 2024-09-18T13:59:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.085+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474722798 0 0) 0x7f4f50009e70 con 0x7f4f500068f0 2024-09-18T13:59:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.085+0000 7f4f577fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T13:59:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T13:59:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 10 monclient(hunting): my global_id is 4597 2024-09-18T13:59:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f50006580 con 0x7f4f500068f0 2024-09-18T13:59:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3218721703 0 0) 0x7f4f50006580 con 0x7f4f500068f0 2024-09-18T13:59:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f50009e70 con 0x7f4f500068f0 2024-09-18T13:59:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f60005490 con 0x7f4f500068f0 2024-09-18T13:59:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T13:59:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T13:59:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 10 monclient: dump: 2024-09-18T13:59:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T13:59:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T13:59:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T13:59:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T13:59:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T13:59:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T13:59:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T13:59:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T13:59:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T13:59:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T13:59:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T13:59:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T13:59:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T13:59:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T13:59:34.117 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T13:59:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2276780543 0 0) 0x7f4f50009e70 con 0x7f4f500068f0 2024-09-18T13:59:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4f50008750 con 0x7f4f500068f0 2024-09-18T13:59:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7375 (unknown 0 0 250646318) 0x7f4f60006880 con 0x7f4f500068f0 2024-09-18T13:59:34.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T13:59:34.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T13:59:34.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 >> v1:172.21.15.84:6792/0 conn(0x7f4f500068f0 legacy=0x7f4f50006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:34.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.086+0000 7f4f577fe640 1 -- 172.21.15.45:0/3722290268 >> v1:172.21.15.84:6792/0 conn(0x7f4f500068f0 legacy=0x7f4f50006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T13:59:34.119 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T13:59:34.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:34.119 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:34.120 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T13:59:34.120 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 32, 2024-09-18T13:59:34.120 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T13:59:34.120 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T13:59:34.120 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T13:59:34.121 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T13:59:34.121 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T13:59:34.121 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T13:59:34.121 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T13:59:34.121 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T13:59:34.121 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T13:59:34.121 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:34.121 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T13:59:34.121 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:34.122 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T13:59:34.122 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T13:59:34.122 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:34.122 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:34.122 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:34.122 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:34.122 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:34.123 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:34.123 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:34.123 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:34.123 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:34.123 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:34.123 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T13:59:34.123 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T13:59:34.123 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:34.123 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:34.123 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:34.123 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:34.123 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T13:59:34.124 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T13:59:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T13:59:34.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T13:59:34.127 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:34.127 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:34.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:34.127 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T13:59:34.127 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:34.127 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:34.127 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:34.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.127 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T13:59:34.127 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T13:59:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:34.129 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:34.129 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:34.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.129 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T13:59:34.129 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T13:59:34.129 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:34.129 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:34.129 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:34.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.129 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T13:59:34.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:34.131 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:34.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.134 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T13:59:34.134 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T13:59:34.134 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:34.134 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:34.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.134 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:34.134 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T13:59:34.134 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:34.134 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:34.134 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:34.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.134 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T13:59:34.135 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T13:59:34.135 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:34.135 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:34.135 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:34.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.136 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T13:59:34.136 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T13:59:34.136 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:34.136 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:34.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:34.137 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T13:59:34.137 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:34.137 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:34.137 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:34.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.137 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T13:59:34.138 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T13:59:34.138 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:34.138 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:34.138 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:34.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.138 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T13:59:34.138 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T13:59:34.138 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:34.138 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:34.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T13:59:34.140 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:34.141 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T13:59:34.142 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T13:59:34.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T13:59:34.142 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T13:59:34.142 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T13:59:34.142 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T13:59:34.142 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T13:59:34.142 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T13:59:34.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T13:59:34.142 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T13:59:34.142 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T13:59:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.087+0000 7f4f7611d640 1 -- 172.21.15.45:0/3722290268 >> v1:172.21.15.84:6812/31478 conn(0x7f4f5c03aba0 legacy=0x7f4f5c03d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.087+0000 7f4f7611d640 10 monclient: shutdown 2024-09-18T13:59:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.087+0000 7f4f7611d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T13:59:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.087+0000 7f4f7611d640 1 -- 172.21.15.45:0/3722290268 >> v1:172.21.15.84:6790/0 conn(0x7f4f700a1080 legacy=0x7f4f7009e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T13:59:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.087+0000 7f4f6ffff640 1 -- 172.21.15.45:0/3722290268 reap_dead start 2024-09-18T13:59:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.087+0000 7f4f6ffff640 1 -- 172.21.15.45:0/3722290268 reap_dead start 2024-09-18T13:59:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.087+0000 7f4f7611d640 1 -- 172.21.15.45:0/3722290268 shutdown_connections 2024-09-18T13:59:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.087+0000 7f4f7611d640 1 -- 172.21.15.45:0/3722290268 >> 172.21.15.45:0/3722290268 conn(0x7f4f700a5bb0 msgr2=0x7f4f70132c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T13:59:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.087+0000 7f4f7611d640 1 -- 172.21.15.45:0/3722290268 shutdown_connections 2024-09-18T13:59:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T13:59:34.087+0000 7f4f7611d640 1 -- 172.21.15.45:0/3722290268 wait complete. 2024-09-18T13:59:34.145 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T13:59:34.145 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-18T13:59:34.147 INFO:tasks.rados.rados.0.smithi084.stdout:67: finishing write tid 1 to smithi08434663-2 2024-09-18T13:59:34.147 INFO:tasks.rados.rados.0.smithi084.stdout:63: finishing rollback tid 0 to smithi08434663-33 2024-09-18T13:59:34.147 INFO:tasks.rados.rados.0.smithi084.stdout:63: finishing rollback tid 1 to smithi08434663-33 2024-09-18T13:59:34.147 INFO:tasks.rados.rados.0.smithi084.stdout:63: finishing rollback tid 2 to smithi08434663-33 2024-09-18T13:59:34.147 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 20 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T13:59:34.147 INFO:tasks.rados.rados.0.smithi084.stdout:68: finishing copy_from racing read to smithi08434663-39 2024-09-18T13:59:34.147 INFO:tasks.rados.rados.0.smithi084.stdout:67: finishing write tid 2 to smithi08434663-2 2024-09-18T13:59:34.147 INFO:tasks.rados.rados.0.smithi084.stdout:67: finishing write tid 3 to smithi08434663-2 2024-09-18T13:59:34.147 INFO:tasks.rados.rados.0.smithi084.stdout:67: finishing write tid 4 to smithi08434663-2 2024-09-18T13:59:34.147 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 25 (ObjNum 53 snap 1 seq_num 53) dirty exists 2024-09-18T13:59:34.147 INFO:tasks.rados.rados.0.smithi084.stdout:67: left oid 2 (ObjNum 53 snap 1 seq_num 53) 2024-09-18T13:59:34.147 INFO:tasks.rados.rados.0.smithi084.stdout:68: finishing copy_from to smithi08434663-39 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:53: done (13 left) 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:54: done (12 left) 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:55: done (11 left) 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:57: done (10 left) 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:58: done (9 left) 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:59: done (8 left) 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:61: done (7 left) 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:62: done (6 left) 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:63: done (5 left) 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:64: done (4 left) 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:65: done (3 left) 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:66: done (2 left) 2024-09-18T13:59:34.148 INFO:tasks.rados.rados.0.smithi084.stdout:67: done (1 left) 2024-09-18T13:59:34.149 INFO:tasks.rados.rados.0.smithi084.stdout:68: done (0 left) 2024-09-18T13:59:34.149 INFO:tasks.rados.rados.0.smithi084.stdout:69: read oid 15 snap -1 2024-09-18T13:59:34.149 INFO:tasks.rados.rados.0.smithi084.stdout:69: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-18T13:59:34.149 INFO:tasks.rados.rados.0.smithi084.stdout:70: write oid 19 current snap is 1 2024-09-18T13:59:34.149 INFO:tasks.rados.rados.0.smithi084.stdout:70: seq_num 54 ranges {507578=477917,1543557=550004,2833665=73913} 2024-09-18T13:59:34.149 INFO:tasks.rados.rados.0.smithi084.stdout:70: writing smithi08434663-19 from 507578 to 985495 tid 1 2024-09-18T13:59:34.149 INFO:tasks.rados.rados.0.smithi084.stdout:70: writing smithi08434663-19 from 1543557 to 2093561 tid 2 2024-09-18T13:59:34.149 INFO:tasks.rados.rados.0.smithi084.stdout:70: writing smithi08434663-19 from 2833665 to 2907578 tid 3 2024-09-18T13:59:34.149 INFO:tasks.rados.rados.0.smithi084.stdout:71: delete oid 4 current snap is 1 2024-09-18T13:59:34.149 INFO:tasks.rados.rados.0.smithi084.stdout:69: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-18T13:59:34.191 INFO:tasks.rados.rados.0.smithi084.stdout:70: finishing write tid 1 to smithi08434663-19 2024-09-18T13:59:34.191 INFO:tasks.rados.rados.0.smithi084.stdout:70: finishing write tid 2 to smithi08434663-19 2024-09-18T13:59:34.191 INFO:tasks.rados.rados.0.smithi084.stdout:70: finishing write tid 3 to smithi08434663-19 2024-09-18T13:59:34.191 INFO:tasks.rados.rados.0.smithi084.stdout:70: finishing write tid 4 to smithi08434663-19 2024-09-18T13:59:34.191 INFO:tasks.rados.rados.0.smithi084.stdout:70: finishing write tid 5 to smithi08434663-19 2024-09-18T13:59:34.191 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 30 (ObjNum 54 snap 1 seq_num 54) dirty exists 2024-09-18T13:59:34.191 INFO:tasks.rados.rados.0.smithi084.stdout:70: left oid 19 (ObjNum 54 snap 1 seq_num 54) 2024-09-18T13:59:34.191 INFO:tasks.rados.rados.0.smithi084.stdout:69: done (2 left) 2024-09-18T13:59:34.191 INFO:tasks.rados.rados.0.smithi084.stdout:70: done (1 left) 2024-09-18T13:59:34.191 INFO:tasks.rados.rados.0.smithi084.stdout:71: done (0 left) 2024-09-18T13:59:34.191 INFO:tasks.rados.rados.0.smithi084.stdout:72: write (excl) oid 6 current snap is 1 2024-09-18T13:59:34.191 INFO:tasks.rados.rados.0.smithi084.stdout:72: seq_num 56 ranges {541192=628415,1660521=553673,2888511=52681} 2024-09-18T13:59:34.197 INFO:tasks.rados.rados.0.smithi084.stdout:72: writing smithi08434663-6 from 541192 to 1169607 tid 1 2024-09-18T13:59:34.202 INFO:tasks.rados.rados.0.smithi084.stdout:72: writing smithi08434663-6 from 1660521 to 2214194 tid 2 2024-09-18T13:59:34.206 INFO:tasks.rados.rados.0.smithi084.stdout:72: writing smithi08434663-6 from 2888511 to 2941192 tid 3 2024-09-18T13:59:34.207 INFO:tasks.rados.rados.0.smithi084.stdout:73: read oid 28 snap -1 2024-09-18T13:59:34.207 INFO:tasks.rados.rados.0.smithi084.stdout:73: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-18T13:59:34.207 INFO:tasks.rados.rados.0.smithi084.stdout:74: read oid 38 snap 0 2024-09-18T13:59:34.207 INFO:tasks.rados.rados.0.smithi084.stdout:74: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T13:59:34.207 INFO:tasks.rados.rados.0.smithi084.stdout:75: snap_create 2024-09-18T13:59:34.207 INFO:tasks.rados.rados.0.smithi084.stdout:72: finishing write tid 1 to smithi08434663-6 2024-09-18T13:59:34.207 INFO:tasks.rados.rados.0.smithi084.stdout:72: finishing write tid 2 to smithi08434663-6 2024-09-18T13:59:34.208 INFO:tasks.rados.rados.0.smithi084.stdout:72: finishing write tid 3 to smithi08434663-6 2024-09-18T13:59:34.208 INFO:tasks.rados.rados.0.smithi084.stdout:72: finishing write tid 4 to smithi08434663-6 2024-09-18T13:59:34.208 INFO:tasks.rados.rados.0.smithi084.stdout:72: finishing write tid 5 to smithi08434663-6 2024-09-18T13:59:34.208 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 14 (ObjNum 56 snap 1 seq_num 56) dirty exists 2024-09-18T13:59:34.209 INFO:tasks.rados.rados.0.smithi084.stdout:72: left oid 6 (ObjNum 56 snap 1 seq_num 56) 2024-09-18T13:59:34.209 INFO:tasks.rados.rados.0.smithi084.stdout:73: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-18T13:59:34.285 INFO:tasks.rados.rados.0.smithi084.stdout:74: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T13:59:34.629 INFO:tasks.rados.rados.0.smithi084.stdout:72: done (3 left) 2024-09-18T13:59:34.629 INFO:tasks.rados.rados.0.smithi084.stdout:73: done (2 left) 2024-09-18T13:59:34.629 INFO:tasks.rados.rados.0.smithi084.stdout:74: done (1 left) 2024-09-18T13:59:34.629 INFO:tasks.rados.rados.0.smithi084.stdout:75: done (0 left) 2024-09-18T13:59:34.629 INFO:tasks.rados.rados.0.smithi084.stdout:76: delete oid 14 current snap is 2 2024-09-18T13:59:34.632 INFO:tasks.rados.rados.0.smithi084.stdout:76: done (0 left) 2024-09-18T13:59:34.632 INFO:tasks.rados.rados.0.smithi084.stdout:77: read oid 7 snap 1 2024-09-18T13:59:34.632 INFO:tasks.rados.rados.0.smithi084.stdout:77: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-18T13:59:34.632 INFO:tasks.rados.rados.0.smithi084.stdout:78: write oid 42 current snap is 2 2024-09-18T13:59:34.632 INFO:tasks.rados.rados.0.smithi084.stdout:78: seq_num 58 ranges {574806=778913,2061132=557342} 2024-09-18T13:59:34.639 INFO:tasks.rados.rados.0.smithi084.stdout:78: writing smithi08434663-42 from 574806 to 1353719 tid 1 2024-09-18T13:59:34.646 INFO:tasks.rados.rados.0.smithi084.stdout:78: writing smithi08434663-42 from 2061132 to 2618474 tid 2 2024-09-18T13:59:34.646 INFO:tasks.rados.rados.0.smithi084.stdout:79: read oid 6 snap -1 2024-09-18T13:59:34.647 INFO:tasks.rados.rados.0.smithi084.stdout:79: expect (ObjNum 56 snap 1 seq_num 56) 2024-09-18T13:59:34.647 INFO:tasks.rados.rados.0.smithi084.stdout:80: read oid 5 snap -1 2024-09-18T13:59:34.647 INFO:tasks.rados.rados.0.smithi084.stdout:80: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-18T13:59:34.647 INFO:tasks.rados.rados.0.smithi084.stdout:81: write oid 41 current snap is 2 2024-09-18T13:59:34.647 INFO:tasks.rados.rados.0.smithi084.stdout:81: seq_num 59 ranges {591613=454162,1719614=417353,2570780=420833} 2024-09-18T13:59:34.652 INFO:tasks.rados.rados.0.smithi084.stdout:81: writing smithi08434663-41 from 591613 to 1045775 tid 1 2024-09-18T13:59:34.658 INFO:tasks.rados.rados.0.smithi084.stdout:81: writing smithi08434663-41 from 1719614 to 2136967 tid 2 2024-09-18T13:59:34.662 INFO:tasks.rados.rados.0.smithi084.stdout:81: writing smithi08434663-41 from 2570780 to 2991613 tid 3 2024-09-18T13:59:34.662 INFO:tasks.rados.rados.0.smithi084.stdout:82: rollback oid 3 current snap is 2 2024-09-18T13:59:34.663 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 3 to 1 2024-09-18T13:59:34.663 INFO:tasks.rados.rados.0.smithi084.stdout:77: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-18T13:59:34.731 INFO:tasks.rados.rados.0.smithi084.stdout:78: finishing write tid 1 to smithi08434663-42 2024-09-18T13:59:34.731 INFO:tasks.rados.rados.0.smithi084.stdout:80: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-18T13:59:34.795 INFO:tasks.rados.rados.0.smithi084.stdout:78: finishing write tid 2 to smithi08434663-42 2024-09-18T13:59:34.795 INFO:tasks.rados.rados.0.smithi084.stdout:78: finishing write tid 3 to smithi08434663-42 2024-09-18T13:59:34.795 INFO:tasks.rados.rados.0.smithi084.stdout:78: finishing write tid 4 to smithi08434663-42 2024-09-18T13:59:34.796 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 25 (ObjNum 58 snap 2 seq_num 58) dirty exists 2024-09-18T13:59:34.796 INFO:tasks.rados.rados.0.smithi084.stdout:78: left oid 42 (ObjNum 58 snap 2 seq_num 58) 2024-09-18T13:59:34.796 INFO:tasks.rados.rados.0.smithi084.stdout:79: expect (ObjNum 56 snap 1 seq_num 56) 2024-09-18T13:59:34.924 INFO:tasks.rados.rados.0.smithi084.stdout:81: finishing write tid 1 to smithi08434663-41 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:81: finishing write tid 2 to smithi08434663-41 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:81: finishing write tid 3 to smithi08434663-41 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:81: finishing write tid 4 to smithi08434663-41 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:81: finishing write tid 5 to smithi08434663-41 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 30 (ObjNum 59 snap 2 seq_num 59) dirty exists 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:81: left oid 41 (ObjNum 59 snap 2 seq_num 59) 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:82: finishing rollback tid 0 to smithi08434663-3 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:82: finishing rollback tid 1 to smithi08434663-3 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:77: done (5 left) 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:78: done (4 left) 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:79: done (3 left) 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:80: done (2 left) 2024-09-18T13:59:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:81: done (1 left) 2024-09-18T13:59:34.926 INFO:tasks.rados.rados.0.smithi084.stdout:83: write (excl) oid 10 current snap is 2 2024-09-18T13:59:34.926 INFO:tasks.rados.rados.0.smithi084.stdout:82: finishing rollback tid 2 to smithi08434663-3 2024-09-18T13:59:34.926 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 13 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T13:59:34.926 INFO:tasks.rados.rados.0.smithi084.stdout:83: seq_num 60 ranges {608420=529411,1778096=561011,2881850=126570} 2024-09-18T13:59:34.929 INFO:tasks.rados.rados.0.smithi084.stdout:83: writing smithi08434663-10 from 608420 to 1137831 tid 1 2024-09-18T13:59:34.935 INFO:tasks.rados.rados.0.smithi084.stdout:83: writing smithi08434663-10 from 1778096 to 2339107 tid 2 2024-09-18T13:59:34.939 INFO:tasks.rados.rados.0.smithi084.stdout:83: writing smithi08434663-10 from 2881850 to 3008420 tid 3 2024-09-18T13:59:34.939 INFO:tasks.rados.rados.0.smithi084.stdout:82: done (1 left) 2024-09-18T13:59:34.940 INFO:tasks.rados.rados.0.smithi084.stdout:84: snap_create 2024-09-18T13:59:34.940 INFO:tasks.rados.rados.0.smithi084.stdout:83: finishing write tid 1 to smithi08434663-10 2024-09-18T13:59:34.941 INFO:tasks.rados.rados.0.smithi084.stdout:83: finishing write tid 2 to smithi08434663-10 2024-09-18T13:59:34.949 INFO:tasks.rados.rados.0.smithi084.stdout:83: finishing write tid 3 to smithi08434663-10 2024-09-18T13:59:34.949 INFO:tasks.rados.rados.0.smithi084.stdout:83: finishing write tid 4 to smithi08434663-10 2024-09-18T13:59:34.949 INFO:tasks.rados.rados.0.smithi084.stdout:83: finishing write tid 5 to smithi08434663-10 2024-09-18T13:59:34.950 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 21 (ObjNum 60 snap 2 seq_num 60) dirty exists 2024-09-18T13:59:34.950 INFO:tasks.rados.rados.0.smithi084.stdout:83: left oid 10 (ObjNum 60 snap 2 seq_num 60) 2024-09-18T13:59:35.634 INFO:tasks.rados.rados.0.smithi084.stdout:83: done (1 left) 2024-09-18T13:59:35.634 INFO:tasks.rados.rados.0.smithi084.stdout:84: done (0 left) 2024-09-18T13:59:35.634 INFO:tasks.rados.rados.0.smithi084.stdout:85: snap_create 2024-09-18T13:59:36.644 INFO:tasks.rados.rados.0.smithi084.stdout:85: done (0 left) 2024-09-18T13:59:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:86: snap_remove snap 0 2024-09-18T13:59:37.647 INFO:tasks.rados.rados.0.smithi084.stdout:86: done (0 left) 2024-09-18T13:59:37.647 INFO:tasks.rados.rados.0.smithi084.stdout:87: read oid 28 snap 3 2024-09-18T13:59:37.647 INFO:tasks.rados.rados.0.smithi084.stdout:87: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-18T13:59:37.647 INFO:tasks.rados.rados.0.smithi084.stdout:88: read oid 16 snap -1 2024-09-18T13:59:37.647 INFO:tasks.rados.rados.0.smithi084.stdout:88: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-18T13:59:37.647 INFO:tasks.rados.rados.0.smithi084.stdout:89: read oid 42 snap -1 2024-09-18T13:59:37.647 INFO:tasks.rados.rados.0.smithi084.stdout:89: expect (ObjNum 58 snap 2 seq_num 58) 2024-09-18T13:59:37.647 INFO:tasks.rados.rados.0.smithi084.stdout:90: rollback oid 27 current snap is 4 2024-09-18T13:59:37.647 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 27 to 1 2024-09-18T13:59:37.647 INFO:tasks.rados.rados.0.smithi084.stdout:91: rollback oid 6 current snap is 4 2024-09-18T13:59:37.648 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 3 2024-09-18T13:59:37.648 INFO:tasks.rados.rados.0.smithi084.stdout:92: delete oid 37 current snap is 4 2024-09-18T13:59:37.652 INFO:tasks.rados.rados.0.smithi084.stdout:88: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-18T13:59:37.733 INFO:tasks.rados.rados.0.smithi084.stdout:91: finishing rollback tid 0 to smithi08434663-6 2024-09-18T13:59:37.733 INFO:tasks.rados.rados.0.smithi084.stdout:91: finishing rollback tid 1 to smithi08434663-6 2024-09-18T13:59:37.733 INFO:tasks.rados.rados.0.smithi084.stdout:91: finishing rollback tid 2 to smithi08434663-6 2024-09-18T13:59:37.733 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 18 (ObjNum 56 snap 1 seq_num 56) dirty exists 2024-09-18T13:59:37.733 INFO:tasks.rados.rados.0.smithi084.stdout:87: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-18T13:59:37.808 INFO:tasks.rados.rados.0.smithi084.stdout:89: expect (ObjNum 58 snap 2 seq_num 58) 2024-09-18T13:59:37.941 INFO:tasks.rados.rados.0.smithi084.stdout:90: finishing rollback tid 0 to smithi08434663-27 2024-09-18T13:59:37.942 INFO:tasks.rados.rados.0.smithi084.stdout:90: finishing rollback tid 1 to smithi08434663-27 2024-09-18T13:59:37.942 INFO:tasks.rados.rados.0.smithi084.stdout:90: finishing rollback tid 2 to smithi08434663-27 2024-09-18T13:59:37.942 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 25 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T13:59:37.942 INFO:tasks.rados.rados.0.smithi084.stdout:87: done (5 left) 2024-09-18T13:59:37.942 INFO:tasks.rados.rados.0.smithi084.stdout:88: done (4 left) 2024-09-18T13:59:37.942 INFO:tasks.rados.rados.0.smithi084.stdout:89: done (3 left) 2024-09-18T13:59:37.942 INFO:tasks.rados.rados.0.smithi084.stdout:90: done (2 left) 2024-09-18T13:59:37.942 INFO:tasks.rados.rados.0.smithi084.stdout:91: done (1 left) 2024-09-18T13:59:37.942 INFO:tasks.rados.rados.0.smithi084.stdout:92: done (0 left) 2024-09-18T13:59:37.942 INFO:tasks.rados.rados.0.smithi084.stdout:93: read oid 9 snap -1 2024-09-18T13:59:37.943 INFO:tasks.rados.rados.0.smithi084.stdout:93: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T13:59:37.943 INFO:tasks.rados.rados.0.smithi084.stdout:94: read oid 11 snap -1 2024-09-18T13:59:37.943 INFO:tasks.rados.rados.0.smithi084.stdout:94: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-18T13:59:37.943 INFO:tasks.rados.rados.0.smithi084.stdout:95: write (excl) oid 38 current snap is 4 2024-09-18T13:59:37.943 INFO:tasks.rados.rados.0.smithi084.stdout:95: seq_num 62 ranges {642034=796262,1895060=564680} 2024-09-18T13:59:37.949 INFO:tasks.rados.rados.0.smithi084.stdout:95: writing smithi08434663-38 from 642034 to 1438296 tid 1 2024-09-18T13:59:37.955 INFO:tasks.rados.rados.0.smithi084.stdout:95: writing smithi08434663-38 from 1895060 to 2459740 tid 2 2024-09-18T13:59:37.955 INFO:tasks.rados.rados.0.smithi084.stdout:96: copy_from oid 29 from oid 25 current snap is 4 2024-09-18T13:59:37.955 INFO:tasks.rados.rados.0.smithi084.stdout:93: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T13:59:38.025 INFO:tasks.rados.rados.0.smithi084.stdout:94: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-18T13:59:38.092 INFO:tasks.rados.rados.0.smithi084.stdout:95: finishing write tid 1 to smithi08434663-38 2024-09-18T13:59:38.092 INFO:tasks.rados.rados.0.smithi084.stdout:95: finishing write tid 2 to smithi08434663-38 2024-09-18T13:59:38.092 INFO:tasks.rados.rados.0.smithi084.stdout:95: finishing write tid 3 to smithi08434663-38 2024-09-18T13:59:38.093 INFO:tasks.rados.rados.0.smithi084.stdout:95: finishing write tid 4 to smithi08434663-38 2024-09-18T13:59:38.093 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 13 (ObjNum 62 snap 4 seq_num 62) dirty exists 2024-09-18T13:59:38.093 INFO:tasks.rados.rados.0.smithi084.stdout:95: left oid 38 (ObjNum 62 snap 4 seq_num 62) 2024-09-18T13:59:38.093 INFO:tasks.rados.rados.0.smithi084.stdout:96: finishing copy_from racing read to smithi08434663-29 2024-09-18T13:59:38.093 INFO:tasks.rados.rados.0.smithi084.stdout:96: finishing copy_from to smithi08434663-29 2024-09-18T13:59:38.093 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 6 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T13:59:38.093 INFO:tasks.rados.rados.0.smithi084.stdout:93: done (3 left) 2024-09-18T13:59:38.093 INFO:tasks.rados.rados.0.smithi084.stdout:94: done (2 left) 2024-09-18T13:59:38.093 INFO:tasks.rados.rados.0.smithi084.stdout:95: done (1 left) 2024-09-18T13:59:38.093 INFO:tasks.rados.rados.0.smithi084.stdout:96: done (0 left) 2024-09-18T13:59:38.093 INFO:tasks.rados.rados.0.smithi084.stdout:97: write (excl) oid 11 current snap is 4 2024-09-18T13:59:38.093 INFO:tasks.rados.rados.0.smithi084.stdout:97: seq_num 63 ranges {658841=471511,1553542=708338,2964119=94722} 2024-09-18T13:59:38.098 INFO:tasks.rados.rados.0.smithi084.stdout:97: writing smithi08434663-11 from 658841 to 1130352 tid 1 2024-09-18T13:59:38.104 INFO:tasks.rados.rados.0.smithi084.stdout:97: writing smithi08434663-11 from 1553542 to 2261880 tid 2 2024-09-18T13:59:38.109 INFO:tasks.rados.rados.0.smithi084.stdout:97: writing smithi08434663-11 from 2964119 to 3058841 tid 3 2024-09-18T13:59:38.109 INFO:tasks.rados.rados.0.smithi084.stdout:98: write oid 35 current snap is 4 2024-09-18T13:59:38.109 INFO:tasks.rados.rados.0.smithi084.stdout:98: seq_num 64 ranges {675648=546760,2012024=568349} 2024-09-18T13:59:38.115 INFO:tasks.rados.rados.0.smithi084.stdout:98: writing smithi08434663-35 from 675648 to 1222408 tid 1 2024-09-18T13:59:38.123 INFO:tasks.rados.rados.0.smithi084.stdout:98: writing smithi08434663-35 from 2012024 to 2580373 tid 2 2024-09-18T13:59:38.123 INFO:tasks.rados.rados.0.smithi084.stdout:99: read oid 28 snap -1 2024-09-18T13:59:38.123 INFO:tasks.rados.rados.0.smithi084.stdout:99: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-18T13:59:38.124 INFO:tasks.rados.rados.0.smithi084.stdout:100: snap_create 2024-09-18T13:59:38.124 INFO:tasks.rados.rados.0.smithi084.stdout:97: finishing write tid 1 to smithi08434663-11 2024-09-18T13:59:38.124 INFO:tasks.rados.rados.0.smithi084.stdout:97: finishing write tid 2 to smithi08434663-11 2024-09-18T13:59:38.124 INFO:tasks.rados.rados.0.smithi084.stdout:97: finishing write tid 3 to smithi08434663-11 2024-09-18T13:59:38.124 INFO:tasks.rados.rados.0.smithi084.stdout:97: finishing write tid 4 to smithi08434663-11 2024-09-18T13:59:38.124 INFO:tasks.rados.rados.0.smithi084.stdout:97: finishing write tid 5 to smithi08434663-11 2024-09-18T13:59:38.124 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 24 (ObjNum 63 snap 4 seq_num 63) dirty exists 2024-09-18T13:59:38.124 INFO:tasks.rados.rados.0.smithi084.stdout:97: left oid 11 (ObjNum 63 snap 4 seq_num 63) 2024-09-18T13:59:38.124 INFO:tasks.rados.rados.0.smithi084.stdout:98: finishing write tid 1 to smithi08434663-35 2024-09-18T13:59:38.125 INFO:tasks.rados.rados.0.smithi084.stdout:99: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-18T13:59:38.202 INFO:tasks.rados.rados.0.smithi084.stdout:98: finishing write tid 2 to smithi08434663-35 2024-09-18T13:59:38.203 INFO:tasks.rados.rados.0.smithi084.stdout:98: finishing write tid 3 to smithi08434663-35 2024-09-18T13:59:38.203 INFO:tasks.rados.rados.0.smithi084.stdout:98: finishing write tid 4 to smithi08434663-35 2024-09-18T13:59:38.203 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 19 (ObjNum 64 snap 4 seq_num 64) dirty exists 2024-09-18T13:59:38.203 INFO:tasks.rados.rados.0.smithi084.stdout:98: left oid 35 (ObjNum 64 snap 4 seq_num 64) 2024-09-18T13:59:38.668 INFO:tasks.rados.rados.0.smithi084.stdout:97: done (3 left) 2024-09-18T13:59:38.668 INFO:tasks.rados.rados.0.smithi084.stdout:98: done (2 left) 2024-09-18T13:59:38.668 INFO:tasks.rados.rados.0.smithi084.stdout:99: done (1 left) 2024-09-18T13:59:38.668 INFO:tasks.rados.rados.0.smithi084.stdout:100: done (0 left) 2024-09-18T13:59:38.668 INFO:tasks.rados.rados.0.smithi084.stdout:101: read oid 29 snap -1 2024-09-18T13:59:38.668 INFO:tasks.rados.rados.0.smithi084.stdout:101: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-18T13:59:38.668 INFO:tasks.rados.rados.0.smithi084.stdout:102: read oid 21 snap -1 2024-09-18T13:59:38.668 INFO:tasks.rados.rados.0.smithi084.stdout:102: expect (ObjNum 20 snap 0 seq_num 20) 2024-09-18T13:59:38.668 INFO:tasks.rados.rados.0.smithi084.stdout:103: snap_remove snap 3 2024-09-18T13:59:39.672 INFO:tasks.rados.rados.0.smithi084.stdout:103: done (2 left) 2024-09-18T13:59:39.672 INFO:tasks.rados.rados.0.smithi084.stdout:104: delete oid 16 current snap is 5 2024-09-18T13:59:39.672 INFO:tasks.rados.rados.0.smithi084.stdout:102: expect (ObjNum 20 snap 0 seq_num 20) 2024-09-18T13:59:39.750 INFO:tasks.rados.rados.0.smithi084.stdout:101: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-18T13:59:39.824 INFO:tasks.rados.rados.0.smithi084.stdout:101: done (2 left) 2024-09-18T13:59:39.824 INFO:tasks.rados.rados.0.smithi084.stdout:102: done (1 left) 2024-09-18T13:59:39.824 INFO:tasks.rados.rados.0.smithi084.stdout:104: done (0 left) 2024-09-18T13:59:39.824 INFO:tasks.rados.rados.0.smithi084.stdout:105: snap_create 2024-09-18T13:59:40.677 INFO:tasks.rados.rados.0.smithi084.stdout:105: done (0 left) 2024-09-18T13:59:40.677 INFO:tasks.rados.rados.0.smithi084.stdout:106: snap_remove snap 5 2024-09-18T13:59:41.684 INFO:tasks.rados.rados.0.smithi084.stdout:106: done (0 left) 2024-09-18T13:59:41.684 INFO:tasks.rados.rados.0.smithi084.stdout:107: write oid 31 current snap is 6 2024-09-18T13:59:41.684 INFO:tasks.rados.rados.0.smithi084.stdout:107: seq_num 66 ranges {709262=697258,2012635=572018,3046388=62874} 2024-09-18T13:59:41.692 INFO:tasks.rados.rados.0.smithi084.stdout:107: writing smithi08434663-31 from 709262 to 1406520 tid 1 2024-09-18T13:59:41.698 INFO:tasks.rados.rados.0.smithi084.stdout:107: writing smithi08434663-31 from 2012635 to 2584653 tid 2 2024-09-18T13:59:41.701 INFO:tasks.rados.rados.0.smithi084.stdout:107: writing smithi08434663-31 from 3046388 to 3109262 tid 3 2024-09-18T13:59:41.701 INFO:tasks.rados.rados.0.smithi084.stdout:108: write (excl) oid 25 current snap is 6 2024-09-18T13:59:41.701 INFO:tasks.rados.rados.0.smithi084.stdout:107: finishing write tid 1 to smithi08434663-31 2024-09-18T13:59:41.702 INFO:tasks.rados.rados.0.smithi084.stdout:108: seq_num 67 ranges {726069=772507,2071117=715676} 2024-09-18T13:59:41.709 INFO:tasks.rados.rados.0.smithi084.stdout:108: writing smithi08434663-25 from 726069 to 1498576 tid 1 2024-09-18T13:59:41.720 INFO:tasks.rados.rados.0.smithi084.stdout:108: writing smithi08434663-25 from 2071117 to 2786793 tid 2 2024-09-18T13:59:41.720 INFO:tasks.rados.rados.0.smithi084.stdout:109: rollback oid 20 current snap is 6 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 20 to 2 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:107: finishing write tid 2 to smithi08434663-31 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:107: finishing write tid 3 to smithi08434663-31 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:107: finishing write tid 4 to smithi08434663-31 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:107: finishing write tid 5 to smithi08434663-31 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 15 (ObjNum 66 snap 6 seq_num 66) dirty exists 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:107: left oid 31 (ObjNum 66 snap 6 seq_num 66) 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:108: finishing write tid 1 to smithi08434663-25 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:107: done (2 left) 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:110: rollback oid 45 current snap is 6 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 45 to 4 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:111: delete oid 2 current snap is 6 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:111: done (3 left) 2024-09-18T13:59:41.721 INFO:tasks.rados.rados.0.smithi084.stdout:112: read oid 44 snap 1 2024-09-18T13:59:41.722 INFO:tasks.rados.rados.0.smithi084.stdout:112: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-18T13:59:41.722 INFO:tasks.rados.rados.0.smithi084.stdout:113: write oid 28 current snap is 6 2024-09-18T13:59:41.722 INFO:tasks.rados.rados.0.smithi084.stdout:113: seq_num 69 ranges {759683=639358,1904434=719345,3128657=31026} 2024-09-18T13:59:41.728 INFO:tasks.rados.rados.0.smithi084.stdout:113: writing smithi08434663-28 from 759683 to 1399041 tid 1 2024-09-18T13:59:41.735 INFO:tasks.rados.rados.0.smithi084.stdout:113: writing smithi08434663-28 from 1904434 to 2623779 tid 2 2024-09-18T13:59:41.738 INFO:tasks.rados.rados.0.smithi084.stdout:113: writing smithi08434663-28 from 3128657 to 3159683 tid 3 2024-09-18T13:59:41.739 INFO:tasks.rados.rados.0.smithi084.stdout:114: read oid 46 snap -1 2024-09-18T13:59:41.739 INFO:tasks.rados.rados.0.smithi084.stdout:114: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-18T13:59:41.739 INFO:tasks.rados.rados.0.smithi084.stdout:115: rollback oid 40 current snap is 6 2024-09-18T13:59:41.739 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 40 to 2 2024-09-18T13:59:41.739 INFO:tasks.rados.rados.0.smithi084.stdout:116: read oid 30 snap 1 2024-09-18T13:59:41.739 INFO:tasks.rados.rados.0.smithi084.stdout:116: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-18T13:59:41.739 INFO:tasks.rados.rados.0.smithi084.stdout:117: read oid 22 snap 4 2024-09-18T13:59:41.739 INFO:tasks.rados.rados.0.smithi084.stdout:117: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-18T13:59:41.739 INFO:tasks.rados.rados.0.smithi084.stdout:118: write (excl) oid 32 current snap is 6 2024-09-18T13:59:41.739 INFO:tasks.rados.rados.0.smithi084.stdout:118: seq_num 70 ranges {776490=714607,2246563=579356} 2024-09-18T13:59:41.747 INFO:tasks.rados.rados.0.smithi084.stdout:118: writing smithi08434663-32 from 776490 to 1491097 tid 1 2024-09-18T13:59:41.754 INFO:tasks.rados.rados.0.smithi084.stdout:118: writing smithi08434663-32 from 2246563 to 2825919 tid 2 2024-09-18T13:59:41.755 INFO:tasks.rados.rados.0.smithi084.stdout:119: write oid 24 current snap is 6 2024-09-18T13:59:41.755 INFO:tasks.rados.rados.0.smithi084.stdout:119: seq_num 71 ranges {793297=789856,2305045=723014} 2024-09-18T13:59:41.763 INFO:tasks.rados.rados.0.smithi084.stdout:119: writing smithi08434663-24 from 793297 to 1583153 tid 1 2024-09-18T13:59:41.771 INFO:tasks.rados.rados.0.smithi084.stdout:119: writing smithi08434663-24 from 2305045 to 3028059 tid 2 2024-09-18T13:59:41.771 INFO:tasks.rados.rados.0.smithi084.stdout:120: write (excl) oid 38 current snap is 6 2024-09-18T13:59:41.771 INFO:tasks.rados.rados.0.smithi084.stdout:109: finishing rollback tid 0 to smithi08434663-20 2024-09-18T13:59:41.771 INFO:tasks.rados.rados.0.smithi084.stdout:109: finishing rollback tid 1 to smithi08434663-20 2024-09-18T13:59:41.772 INFO:tasks.rados.rados.0.smithi084.stdout:110: finishing rollback tid 0 to smithi08434663-45 2024-09-18T13:59:41.772 INFO:tasks.rados.rados.0.smithi084.stdout:109: finishing rollback tid 2 to smithi08434663-20 2024-09-18T13:59:41.772 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 10 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T13:59:41.772 INFO:tasks.rados.rados.0.smithi084.stdout:110: finishing rollback tid 1 to smithi08434663-45 2024-09-18T13:59:41.772 INFO:tasks.rados.rados.0.smithi084.stdout:120: seq_num 72 ranges {410104=465105,1563527=583025,2810926=399178} 2024-09-18T13:59:41.776 INFO:tasks.rados.rados.0.smithi084.stdout:120: writing smithi08434663-38 from 410104 to 875209 tid 1 2024-09-18T13:59:41.782 INFO:tasks.rados.rados.0.smithi084.stdout:120: writing smithi08434663-38 from 1563527 to 2146552 tid 2 2024-09-18T13:59:41.788 INFO:tasks.rados.rados.0.smithi084.stdout:120: writing smithi08434663-38 from 2810926 to 3210104 tid 3 2024-09-18T13:59:41.788 INFO:tasks.rados.rados.0.smithi084.stdout:109: done (11 left) 2024-09-18T13:59:41.789 INFO:tasks.rados.rados.0.smithi084.stdout:121: write (excl) oid 35 current snap is 6 2024-09-18T13:59:41.789 INFO:tasks.rados.rados.0.smithi084.stdout:108: finishing write tid 2 to smithi08434663-25 2024-09-18T13:59:41.789 INFO:tasks.rados.rados.0.smithi084.stdout:110: finishing rollback tid 2 to smithi08434663-45 2024-09-18T13:59:41.789 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 42 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T13:59:41.789 INFO:tasks.rados.rados.0.smithi084.stdout:108: finishing write tid 3 to smithi08434663-25 2024-09-18T13:59:41.789 INFO:tasks.rados.rados.0.smithi084.stdout:121: seq_num 73 ranges {426911=540354,1622009=726683,3121996=104915} 2024-09-18T13:59:41.794 INFO:tasks.rados.rados.0.smithi084.stdout:121: writing smithi08434663-35 from 426911 to 967265 tid 1 2024-09-18T13:59:41.801 INFO:tasks.rados.rados.0.smithi084.stdout:121: writing smithi08434663-35 from 1622009 to 2348692 tid 2 2024-09-18T13:59:41.806 INFO:tasks.rados.rados.0.smithi084.stdout:121: writing smithi08434663-35 from 3121996 to 3226911 tid 3 2024-09-18T13:59:41.806 INFO:tasks.rados.rados.0.smithi084.stdout:110: done (11 left) 2024-09-18T13:59:41.806 INFO:tasks.rados.rados.0.smithi084.stdout:122: delete oid 26 current snap is 6 2024-09-18T13:59:41.806 INFO:tasks.rados.rados.0.smithi084.stdout:108: finishing write tid 4 to smithi08434663-25 2024-09-18T13:59:41.806 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 13 (ObjNum 67 snap 6 seq_num 67) dirty exists 2024-09-18T13:59:41.807 INFO:tasks.rados.rados.0.smithi084.stdout:108: left oid 25 (ObjNum 67 snap 6 seq_num 67) 2024-09-18T13:59:41.807 INFO:tasks.rados.rados.0.smithi084.stdout:112: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-18T13:59:41.891 INFO:tasks.rados.rados.0.smithi084.stdout:113: finishing write tid 1 to smithi08434663-28 2024-09-18T13:59:41.891 INFO:tasks.rados.rados.0.smithi084.stdout:113: finishing write tid 2 to smithi08434663-28 2024-09-18T13:59:41.891 INFO:tasks.rados.rados.0.smithi084.stdout:115: finishing rollback tid 1 to smithi08434663-40 2024-09-18T13:59:41.891 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 0 (ObjNum 33 snap 0 seq_num 2495498672) dirty dne 2024-09-18T13:59:41.891 INFO:tasks.rados.rados.0.smithi084.stdout:113: finishing write tid 3 to smithi08434663-28 2024-09-18T13:59:41.891 INFO:tasks.rados.rados.0.smithi084.stdout:116: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-18T13:59:41.972 INFO:tasks.rados.rados.0.smithi084.stdout:117: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-18T13:59:42.047 INFO:tasks.rados.rados.0.smithi084.stdout:114: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-18T13:59:42.131 INFO:tasks.rados.rados.0.smithi084.stdout:113: finishing write tid 4 to smithi08434663-28 2024-09-18T13:59:42.131 INFO:tasks.rados.rados.0.smithi084.stdout:113: finishing write tid 5 to smithi08434663-28 2024-09-18T13:59:42.131 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 45 (ObjNum 69 snap 6 seq_num 69) dirty exists 2024-09-18T13:59:42.131 INFO:tasks.rados.rados.0.smithi084.stdout:113: left oid 28 (ObjNum 69 snap 6 seq_num 69) 2024-09-18T13:59:42.131 INFO:tasks.rados.rados.0.smithi084.stdout:118: finishing write tid 1 to smithi08434663-32 2024-09-18T13:59:42.131 INFO:tasks.rados.rados.0.smithi084.stdout:118: finishing write tid 2 to smithi08434663-32 2024-09-18T13:59:42.131 INFO:tasks.rados.rados.0.smithi084.stdout:118: finishing write tid 3 to smithi08434663-32 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:118: finishing write tid 4 to smithi08434663-32 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 49 (ObjNum 70 snap 6 seq_num 70) dirty exists 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:118: left oid 32 (ObjNum 70 snap 6 seq_num 70) 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:119: finishing write tid 1 to smithi08434663-24 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:108: done (11 left) 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:112: done (10 left) 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:113: done (9 left) 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:114: done (8 left) 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:115: done (7 left) 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:116: done (6 left) 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:117: done (5 left) 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:118: done (4 left) 2024-09-18T13:59:42.132 INFO:tasks.rados.rados.0.smithi084.stdout:122: done (3 left) 2024-09-18T13:59:42.133 INFO:tasks.rados.rados.0.smithi084.stdout:123: snap_create 2024-09-18T13:59:42.133 INFO:tasks.rados.rados.0.smithi084.stdout:119: finishing write tid 2 to smithi08434663-24 2024-09-18T13:59:42.133 INFO:tasks.rados.rados.0.smithi084.stdout:119: finishing write tid 3 to smithi08434663-24 2024-09-18T13:59:42.133 INFO:tasks.rados.rados.0.smithi084.stdout:119: finishing write tid 4 to smithi08434663-24 2024-09-18T13:59:42.133 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 19 (ObjNum 71 snap 6 seq_num 71) dirty exists 2024-09-18T13:59:42.133 INFO:tasks.rados.rados.0.smithi084.stdout:119: left oid 24 (ObjNum 71 snap 6 seq_num 71) 2024-09-18T13:59:42.133 INFO:tasks.rados.rados.0.smithi084.stdout:120: finishing write tid 1 to smithi08434663-38 2024-09-18T13:59:42.133 INFO:tasks.rados.rados.0.smithi084.stdout:120: finishing write tid 2 to smithi08434663-38 2024-09-18T13:59:42.133 INFO:tasks.rados.rados.0.smithi084.stdout:120: finishing write tid 3 to smithi08434663-38 2024-09-18T13:59:42.133 INFO:tasks.rados.rados.0.smithi084.stdout:120: finishing write tid 4 to smithi08434663-38 2024-09-18T13:59:42.134 INFO:tasks.rados.rados.0.smithi084.stdout:120: finishing write tid 5 to smithi08434663-38 2024-09-18T13:59:42.134 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 18 (ObjNum 72 snap 6 seq_num 72) dirty exists 2024-09-18T13:59:42.134 INFO:tasks.rados.rados.0.smithi084.stdout:120: left oid 38 (ObjNum 72 snap 6 seq_num 72) 2024-09-18T13:59:42.134 INFO:tasks.rados.rados.0.smithi084.stdout:121: finishing write tid 1 to smithi08434663-35 2024-09-18T13:59:42.134 INFO:tasks.rados.rados.0.smithi084.stdout:121: finishing write tid 2 to smithi08434663-35 2024-09-18T13:59:42.134 INFO:tasks.rados.rados.0.smithi084.stdout:121: finishing write tid 3 to smithi08434663-35 2024-09-18T13:59:42.134 INFO:tasks.rados.rados.0.smithi084.stdout:121: finishing write tid 4 to smithi08434663-35 2024-09-18T13:59:42.134 INFO:tasks.rados.rados.0.smithi084.stdout:121: finishing write tid 5 to smithi08434663-35 2024-09-18T13:59:42.134 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 28 (ObjNum 73 snap 6 seq_num 73) dirty exists 2024-09-18T13:59:42.135 INFO:tasks.rados.rados.0.smithi084.stdout:121: left oid 35 (ObjNum 73 snap 6 seq_num 73) 2024-09-18T13:59:42.697 INFO:tasks.rados.rados.0.smithi084.stdout:119: done (3 left) 2024-09-18T13:59:42.697 INFO:tasks.rados.rados.0.smithi084.stdout:120: done (2 left) 2024-09-18T13:59:42.697 INFO:tasks.rados.rados.0.smithi084.stdout:121: done (1 left) 2024-09-18T13:59:42.697 INFO:tasks.rados.rados.0.smithi084.stdout:123: done (0 left) 2024-09-18T13:59:42.697 INFO:tasks.rados.rados.0.smithi084.stdout:124: write oid 19 current snap is 7 2024-09-18T13:59:42.697 INFO:tasks.rados.rados.0.smithi084.stdout:124: seq_num 75 ranges {460525=690852,1622620=730352,3060489=200036} 2024-09-18T13:59:42.703 INFO:tasks.rados.rados.0.smithi084.stdout:124: writing smithi08434663-19 from 460525 to 1151377 tid 1 2024-09-18T13:59:42.709 INFO:tasks.rados.rados.0.smithi084.stdout:124: writing smithi08434663-19 from 1622620 to 2352972 tid 2 2024-09-18T13:59:42.715 INFO:tasks.rados.rados.0.smithi084.stdout:124: writing smithi08434663-19 from 3060489 to 3260525 tid 3 2024-09-18T13:59:42.715 INFO:tasks.rados.rados.0.smithi084.stdout:125: read oid 33 snap 2 2024-09-18T13:59:42.715 INFO:tasks.rados.rados.0.smithi084.stdout:125: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-18T13:59:42.715 INFO:tasks.rados.rados.0.smithi084.stdout:126: delete oid 30 current snap is 7 2024-09-18T13:59:42.715 INFO:tasks.rados.rados.0.smithi084.stdout:124: finishing write tid 1 to smithi08434663-19 2024-09-18T13:59:42.715 INFO:tasks.rados.rados.0.smithi084.stdout:124: finishing write tid 2 to smithi08434663-19 2024-09-18T13:59:42.718 INFO:tasks.rados.rados.0.smithi084.stdout:124: finishing write tid 3 to smithi08434663-19 2024-09-18T13:59:42.719 INFO:tasks.rados.rados.0.smithi084.stdout:124: finishing write tid 4 to smithi08434663-19 2024-09-18T13:59:42.719 INFO:tasks.rados.rados.0.smithi084.stdout:124: finishing write tid 5 to smithi08434663-19 2024-09-18T13:59:42.719 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 54 (ObjNum 75 snap 7 seq_num 75) dirty exists 2024-09-18T13:59:42.719 INFO:tasks.rados.rados.0.smithi084.stdout:124: left oid 19 (ObjNum 75 snap 7 seq_num 75) 2024-09-18T13:59:42.721 INFO:tasks.rados.rados.0.smithi084.stdout:125: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-18T13:59:42.803 INFO:tasks.rados.rados.0.smithi084.stdout:124: done (2 left) 2024-09-18T13:59:42.803 INFO:tasks.rados.rados.0.smithi084.stdout:125: done (1 left) 2024-09-18T13:59:42.803 INFO:tasks.rados.rados.0.smithi084.stdout:126: done (0 left) 2024-09-18T13:59:42.803 INFO:tasks.rados.rados.0.smithi084.stdout:127: write oid 10 current snap is 7 2024-09-18T13:59:42.803 INFO:tasks.rados.rados.0.smithi084.stdout:127: seq_num 77 ranges {494139=557703,1455937=734021,2948041=346098} 2024-09-18T13:59:42.808 INFO:tasks.rados.rados.0.smithi084.stdout:127: writing smithi08434663-10 from 494139 to 1051842 tid 1 2024-09-18T13:59:42.815 INFO:tasks.rados.rados.0.smithi084.stdout:127: writing smithi08434663-10 from 1455937 to 2189958 tid 2 2024-09-18T13:59:42.821 INFO:tasks.rados.rados.0.smithi084.stdout:127: writing smithi08434663-10 from 2948041 to 3294139 tid 3 2024-09-18T13:59:42.821 INFO:tasks.rados.rados.0.smithi084.stdout:128: copy_from oid 35 from oid 26 current snap is 7 2024-09-18T13:59:42.821 INFO:tasks.rados.rados.0.smithi084.stdout:127: finishing write tid 1 to smithi08434663-10 2024-09-18T13:59:42.821 INFO:tasks.rados.rados.0.smithi084.stdout:127: finishing write tid 2 to smithi08434663-10 2024-09-18T13:59:42.821 INFO:tasks.rados.rados.0.smithi084.stdout:129: snap_remove snap 4 2024-09-18T13:59:43.697 INFO:tasks.rados.rados.0.smithi084.stdout:129: done (2 left) 2024-09-18T13:59:43.697 INFO:tasks.rados.rados.0.smithi084.stdout:130: copy_from oid 28 from oid 8 current snap is 7 2024-09-18T13:59:43.697 INFO:tasks.rados.rados.0.smithi084.stdout:131: read oid 2 snap 2 2024-09-18T13:59:43.697 INFO:tasks.rados.rados.0.smithi084.stdout:131: expect (ObjNum 53 snap 1 seq_num 53) 2024-09-18T13:59:43.698 INFO:tasks.rados.rados.0.smithi084.stdout:132: read oid 43 snap 6 2024-09-18T13:59:43.698 INFO:tasks.rados.rados.0.smithi084.stdout:132: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T13:59:43.698 INFO:tasks.rados.rados.0.smithi084.stdout:133: write oid 1 current snap is 7 2024-09-18T13:59:43.698 INFO:tasks.rados.rados.0.smithi084.stdout:133: seq_num 78 ranges {510946=632952,1798066=477679,2742758=457217} 2024-09-18T13:59:43.704 INFO:tasks.rados.rados.0.smithi084.stdout:133: writing smithi08434663-1 from 510946 to 1143898 tid 1 2024-09-18T13:59:43.710 INFO:tasks.rados.rados.0.smithi084.stdout:133: writing smithi08434663-1 from 1798066 to 2275745 tid 2 2024-09-18T13:59:43.717 INFO:tasks.rados.rados.0.smithi084.stdout:133: writing smithi08434663-1 from 2742758 to 3199975 tid 3 2024-09-18T13:59:43.717 INFO:tasks.rados.rados.0.smithi084.stdout:134: rollback oid 3 current snap is 7 2024-09-18T13:59:43.717 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 3 to 1 2024-09-18T13:59:43.717 INFO:tasks.rados.rados.0.smithi084.stdout:128: finishing copy_from to smithi08434663-35 2024-09-18T13:59:43.717 INFO:tasks.rados.rados.0.smithi084.stdout:128: got expected ENOENT (src dne) 2024-09-18T13:59:43.717 INFO:tasks.rados.rados.0.smithi084.stdout:128: finishing copy_from racing read to smithi08434663-35 2024-09-18T13:59:43.717 INFO:tasks.rados.rados.0.smithi084.stdout:127: finishing write tid 3 to smithi08434663-10 2024-09-18T13:59:43.717 INFO:tasks.rados.rados.0.smithi084.stdout:127: finishing write tid 4 to smithi08434663-10 2024-09-18T13:59:43.717 INFO:tasks.rados.rados.0.smithi084.stdout:127: finishing write tid 5 to smithi08434663-10 2024-09-18T13:59:43.717 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 38 (ObjNum 77 snap 7 seq_num 77) dirty exists 2024-09-18T13:59:43.718 INFO:tasks.rados.rados.0.smithi084.stdout:127: left oid 10 (ObjNum 77 snap 7 seq_num 77) 2024-09-18T13:59:43.718 INFO:tasks.rados.rados.0.smithi084.stdout:133: finishing write tid 1 to smithi08434663-1 2024-09-18T13:59:43.718 INFO:tasks.rados.rados.0.smithi084.stdout:132: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T13:59:43.805 INFO:tasks.rados.rados.0.smithi084.stdout:130: finishing copy_from racing read to smithi08434663-28 2024-09-18T13:59:43.806 INFO:tasks.rados.rados.0.smithi084.stdout:131: expect (ObjNum 53 snap 1 seq_num 53) 2024-09-18T13:59:43.931 INFO:tasks.rados.rados.0.smithi084.stdout:130: finishing copy_from to smithi08434663-28 2024-09-18T13:59:43.931 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 3 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T13:59:43.931 INFO:tasks.rados.rados.0.smithi084.stdout:133: finishing write tid 2 to smithi08434663-1 2024-09-18T13:59:43.931 INFO:tasks.rados.rados.0.smithi084.stdout:134: finishing rollback tid 0 to smithi08434663-3 2024-09-18T13:59:43.931 INFO:tasks.rados.rados.0.smithi084.stdout:134: finishing rollback tid 1 to smithi08434663-3 2024-09-18T13:59:43.931 INFO:tasks.rados.rados.0.smithi084.stdout:134: finishing rollback tid 2 to smithi08434663-3 2024-09-18T13:59:43.931 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 19 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T13:59:43.931 INFO:tasks.rados.rados.0.smithi084.stdout:133: finishing write tid 3 to smithi08434663-1 2024-09-18T13:59:43.931 INFO:tasks.rados.rados.0.smithi084.stdout:133: finishing write tid 4 to smithi08434663-1 2024-09-18T13:59:43.932 INFO:tasks.rados.rados.0.smithi084.stdout:133: finishing write tid 5 to smithi08434663-1 2024-09-18T13:59:43.932 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 47 (ObjNum 78 snap 7 seq_num 78) dirty exists 2024-09-18T13:59:43.932 INFO:tasks.rados.rados.0.smithi084.stdout:133: left oid 1 (ObjNum 78 snap 7 seq_num 78) 2024-09-18T13:59:43.932 INFO:tasks.rados.rados.0.smithi084.stdout:127: done (6 left) 2024-09-18T13:59:43.932 INFO:tasks.rados.rados.0.smithi084.stdout:128: done (5 left) 2024-09-18T13:59:43.932 INFO:tasks.rados.rados.0.smithi084.stdout:130: done (4 left) 2024-09-18T13:59:43.932 INFO:tasks.rados.rados.0.smithi084.stdout:131: done (3 left) 2024-09-18T13:59:43.932 INFO:tasks.rados.rados.0.smithi084.stdout:132: done (2 left) 2024-09-18T13:59:43.932 INFO:tasks.rados.rados.0.smithi084.stdout:133: done (1 left) 2024-09-18T13:59:43.932 INFO:tasks.rados.rados.0.smithi084.stdout:134: done (0 left) 2024-09-18T13:59:43.932 INFO:tasks.rados.rados.0.smithi084.stdout:135: snap_create 2024-09-18T13:59:44.710 INFO:tasks.rados.rados.0.smithi084.stdout:135: done (0 left) 2024-09-18T13:59:44.710 INFO:tasks.rados.rados.0.smithi084.stdout:136: delete oid 3 current snap is 8 2024-09-18T13:59:44.714 INFO:tasks.rados.rados.0.smithi084.stdout:136: done (0 left) 2024-09-18T13:59:44.714 INFO:tasks.rados.rados.0.smithi084.stdout:137: read oid 2 snap 7 2024-09-18T13:59:44.714 INFO:tasks.rados.rados.0.smithi084.stdout:137: expect deleted 2024-09-18T13:59:44.714 INFO:tasks.rados.rados.0.smithi084.stdout:138: snap_remove snap 1 2024-09-18T13:59:45.711 INFO:tasks.rados.rados.0.smithi084.stdout:138: done (1 left) 2024-09-18T13:59:45.711 INFO:tasks.rados.rados.0.smithi084.stdout:139: rollback oid 9 current snap is 8 2024-09-18T13:59:45.711 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 9 to 7 2024-09-18T13:59:45.711 INFO:tasks.rados.rados.0.smithi084.stdout:137: done (1 left) 2024-09-18T13:59:45.711 INFO:tasks.rados.rados.0.smithi084.stdout:140: read oid 14 snap -1 2024-09-18T13:59:45.711 INFO:tasks.rados.rados.0.smithi084.stdout:140: expect deleted 2024-09-18T13:59:45.711 INFO:tasks.rados.rados.0.smithi084.stdout:141: snap_create 2024-09-18T13:59:45.721 INFO:tasks.rados.rados.0.smithi084.stdout:139: finishing rollback tid 0 to smithi08434663-9 2024-09-18T13:59:45.723 INFO:tasks.rados.rados.0.smithi084.stdout:139: finishing rollback tid 1 to smithi08434663-9 2024-09-18T13:59:45.723 INFO:tasks.rados.rados.0.smithi084.stdout:139: finishing rollback tid 2 to smithi08434663-9 2024-09-18T13:59:45.723 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 36 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T13:59:46.728 INFO:tasks.rados.rados.0.smithi084.stdout:139: done (2 left) 2024-09-18T13:59:46.728 INFO:tasks.rados.rados.0.smithi084.stdout:140: done (1 left) 2024-09-18T13:59:46.729 INFO:tasks.rados.rados.0.smithi084.stdout:141: done (0 left) 2024-09-18T13:59:46.729 INFO:tasks.rados.rados.0.smithi084.stdout:142: snap_remove snap 2 2024-09-18T13:59:47.139 INFO:tasks.rados.rados.0.smithi084.stdout:142: done (0 left) 2024-09-18T13:59:47.139 INFO:tasks.rados.rados.0.smithi084.stdout:143: read oid 16 snap -1 2024-09-18T13:59:47.139 INFO:tasks.rados.rados.0.smithi084.stdout:143: expect deleted 2024-09-18T13:59:47.140 INFO:tasks.rados.rados.0.smithi084.stdout:144: delete oid 19 current snap is 9 2024-09-18T13:59:47.151 INFO:tasks.rados.rados.0.smithi084.stdout:143: done (1 left) 2024-09-18T13:59:47.151 INFO:tasks.rados.rados.0.smithi084.stdout:144: done (0 left) 2024-09-18T13:59:47.151 INFO:tasks.rados.rados.0.smithi084.stdout:145: read oid 34 snap -1 2024-09-18T13:59:47.151 INFO:tasks.rados.rados.0.smithi084.stdout:145: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T13:59:47.151 INFO:tasks.rados.rados.0.smithi084.stdout:146: delete oid 24 current snap is 9 2024-09-18T13:59:47.155 INFO:tasks.rados.rados.0.smithi084.stdout:146: done (1 left) 2024-09-18T13:59:47.155 INFO:tasks.rados.rados.0.smithi084.stdout:147: read oid 7 snap -1 2024-09-18T13:59:47.155 INFO:tasks.rados.rados.0.smithi084.stdout:147: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-18T13:59:47.155 INFO:tasks.rados.rados.0.smithi084.stdout:148: delete oid 49 current snap is 9 2024-09-18T13:59:47.161 INFO:tasks.rados.rados.0.smithi084.stdout:148: done (2 left) 2024-09-18T13:59:47.161 INFO:tasks.rados.rados.0.smithi084.stdout:149: copy_from oid 10 from oid 22 current snap is 9 2024-09-18T13:59:47.161 INFO:tasks.rados.rados.0.smithi084.stdout:150: read oid 19 snap -1 2024-09-18T13:59:47.161 INFO:tasks.rados.rados.0.smithi084.stdout:150: expect deleted 2024-09-18T13:59:47.161 INFO:tasks.rados.rados.0.smithi084.stdout:151: delete oid 21 current snap is 9 2024-09-18T13:59:47.161 INFO:tasks.rados.rados.0.smithi084.stdout:147: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-18T13:59:47.231 INFO:tasks.rados.rados.0.smithi084.stdout:147: done (4 left) 2024-09-18T13:59:47.231 INFO:tasks.rados.rados.0.smithi084.stdout:150: done (3 left) 2024-09-18T13:59:47.231 INFO:tasks.rados.rados.0.smithi084.stdout:151: done (2 left) 2024-09-18T13:59:47.231 INFO:tasks.rados.rados.0.smithi084.stdout:152: snap_remove snap 8 2024-09-18T13:59:48.153 INFO:tasks.rados.rados.0.smithi084.stdout:152: done (2 left) 2024-09-18T13:59:48.153 INFO:tasks.rados.rados.0.smithi084.stdout:153: rollback oid 8 current snap is 9 2024-09-18T13:59:48.153 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 8 to 7 2024-09-18T13:59:48.154 INFO:tasks.rados.rados.0.smithi084.stdout:145: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T13:59:48.236 INFO:tasks.rados.rados.0.smithi084.stdout:149: finishing copy_from racing read to smithi08434663-10 2024-09-18T13:59:48.237 INFO:tasks.rados.rados.0.smithi084.stdout:149: finishing copy_from to smithi08434663-10 2024-09-18T13:59:48.237 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 9 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T13:59:48.237 INFO:tasks.rados.rados.0.smithi084.stdout:153: finishing rollback tid 0 to smithi08434663-8 2024-09-18T13:59:48.237 INFO:tasks.rados.rados.0.smithi084.stdout:153: finishing rollback tid 1 to smithi08434663-8 2024-09-18T13:59:48.237 INFO:tasks.rados.rados.0.smithi084.stdout:153: finishing rollback tid 2 to smithi08434663-8 2024-09-18T13:59:48.237 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 20 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T13:59:48.237 INFO:tasks.rados.rados.0.smithi084.stdout:145: done (2 left) 2024-09-18T13:59:48.237 INFO:tasks.rados.rados.0.smithi084.stdout:149: done (1 left) 2024-09-18T13:59:48.237 INFO:tasks.rados.rados.0.smithi084.stdout:153: done (0 left) 2024-09-18T13:59:48.237 INFO:tasks.rados.rados.0.smithi084.stdout:154: copy_from oid 10 from oid 12 current snap is 9 2024-09-18T13:59:48.237 INFO:tasks.rados.rados.0.smithi084.stdout:155: snap_remove snap 7 2024-09-18T13:59:49.154 INFO:tasks.rados.rados.0.smithi084.stdout:155: done (1 left) 2024-09-18T13:59:49.154 INFO:tasks.rados.rados.0.smithi084.stdout:156: delete oid 15 current snap is 9 2024-09-18T13:59:49.155 INFO:tasks.rados.rados.0.smithi084.stdout:154: finishing copy_from racing read to smithi08434663-10 2024-09-18T13:59:49.155 INFO:tasks.rados.rados.0.smithi084.stdout:154: finishing copy_from to smithi08434663-10 2024-09-18T13:59:49.155 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T13:59:49.161 INFO:tasks.rados.rados.0.smithi084.stdout:154: done (1 left) 2024-09-18T13:59:49.161 INFO:tasks.rados.rados.0.smithi084.stdout:156: done (0 left) 2024-09-18T13:59:49.161 INFO:tasks.rados.rados.0.smithi084.stdout:157: rollback oid 45 current snap is 9 2024-09-18T13:59:49.162 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 45 to 6 2024-09-18T13:59:49.162 INFO:tasks.rados.rados.0.smithi084.stdout:158: read oid 46 snap -1 2024-09-18T13:59:49.162 INFO:tasks.rados.rados.0.smithi084.stdout:158: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-18T13:59:49.162 INFO:tasks.rados.rados.0.smithi084.stdout:159: copy_from oid 39 from oid 1 current snap is 9 2024-09-18T13:59:49.162 INFO:tasks.rados.rados.0.smithi084.stdout:160: write (excl) oid 37 current snap is 9 2024-09-18T13:59:49.162 INFO:tasks.rados.rados.0.smithi084.stdout:160: seq_num 85 ranges {628595=476048,1807440=632344,2934719=452474} 2024-09-18T13:59:49.167 INFO:tasks.rados.rados.0.smithi084.stdout:160: writing smithi08434663-37 from 628595 to 1104643 tid 1 2024-09-18T13:59:49.175 INFO:tasks.rados.rados.0.smithi084.stdout:160: writing smithi08434663-37 from 1807440 to 2439784 tid 2 2024-09-18T13:59:49.181 INFO:tasks.rados.rados.0.smithi084.stdout:160: writing smithi08434663-37 from 2934719 to 3387193 tid 3 2024-09-18T13:59:49.181 INFO:tasks.rados.rados.0.smithi084.stdout:161: read oid 32 snap -1 2024-09-18T13:59:49.181 INFO:tasks.rados.rados.0.smithi084.stdout:161: expect (ObjNum 70 snap 6 seq_num 70) 2024-09-18T13:59:49.181 INFO:tasks.rados.rados.0.smithi084.stdout:162: write (excl) oid 48 current snap is 9 2024-09-18T13:59:49.181 INFO:tasks.rados.rados.0.smithi084.stdout:157: finishing rollback tid 0 to smithi08434663-45 2024-09-18T13:59:49.181 INFO:tasks.rados.rados.0.smithi084.stdout:162: seq_num 86 ranges {645402=551297,1749569=492355,2845789=599613} 2024-09-18T13:59:49.187 INFO:tasks.rados.rados.0.smithi084.stdout:162: writing smithi08434663-48 from 645402 to 1196699 tid 1 2024-09-18T13:59:49.193 INFO:tasks.rados.rados.0.smithi084.stdout:162: writing smithi08434663-48 from 1749569 to 2241924 tid 2 2024-09-18T13:59:49.200 INFO:tasks.rados.rados.0.smithi084.stdout:162: writing smithi08434663-48 from 2845789 to 3445402 tid 3 2024-09-18T13:59:49.200 INFO:tasks.rados.rados.0.smithi084.stdout:163: rollback oid 23 current snap is 9 2024-09-18T13:59:49.200 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 23 to 6 2024-09-18T13:59:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:157: finishing rollback tid 1 to smithi08434663-45 2024-09-18T13:59:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:157: finishing rollback tid 2 to smithi08434663-45 2024-09-18T13:59:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 56 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T13:59:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:158: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-18T13:59:49.284 INFO:tasks.rados.rados.0.smithi084.stdout:160: finishing write tid 1 to smithi08434663-37 2024-09-18T13:59:49.284 INFO:tasks.rados.rados.0.smithi084.stdout:159: finishing copy_from racing read to smithi08434663-39 2024-09-18T13:59:49.284 INFO:tasks.rados.rados.0.smithi084.stdout:159: finishing copy_from to smithi08434663-39 2024-09-18T13:59:49.284 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 47 (ObjNum 78 snap 7 seq_num 78) dirty exists 2024-09-18T13:59:49.284 INFO:tasks.rados.rados.0.smithi084.stdout:160: finishing write tid 2 to smithi08434663-37 2024-09-18T13:59:49.285 INFO:tasks.rados.rados.0.smithi084.stdout:160: finishing write tid 3 to smithi08434663-37 2024-09-18T13:59:49.285 INFO:tasks.rados.rados.0.smithi084.stdout:161: expect (ObjNum 70 snap 6 seq_num 70) 2024-09-18T13:59:49.422 INFO:tasks.rados.rados.0.smithi084.stdout:160: finishing write tid 4 to smithi08434663-37 2024-09-18T13:59:49.422 INFO:tasks.rados.rados.0.smithi084.stdout:160: finishing write tid 5 to smithi08434663-37 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 42 (ObjNum 85 snap 9 seq_num 85) dirty exists 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:160: left oid 37 (ObjNum 85 snap 9 seq_num 85) 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:162: finishing write tid 1 to smithi08434663-48 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:162: finishing write tid 2 to smithi08434663-48 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:157: done (6 left) 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:158: done (5 left) 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:159: done (4 left) 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:160: done (3 left) 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:161: done (2 left) 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:164: read oid 27 snap -1 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:164: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:165: read oid 11 snap -1 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:165: expect (ObjNum 63 snap 4 seq_num 63) 2024-09-18T13:59:49.423 INFO:tasks.rados.rados.0.smithi084.stdout:166: copy_from oid 21 from oid 50 current snap is 9 2024-09-18T13:59:49.424 INFO:tasks.rados.rados.0.smithi084.stdout:162: finishing write tid 3 to smithi08434663-48 2024-09-18T13:59:49.424 INFO:tasks.rados.rados.0.smithi084.stdout:162: finishing write tid 4 to smithi08434663-48 2024-09-18T13:59:49.424 INFO:tasks.rados.rados.0.smithi084.stdout:163: finishing rollback tid 0 to smithi08434663-23 2024-09-18T13:59:49.424 INFO:tasks.rados.rados.0.smithi084.stdout:162: finishing write tid 5 to smithi08434663-48 2024-09-18T13:59:49.424 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 30 (ObjNum 86 snap 9 seq_num 86) dirty exists 2024-09-18T13:59:49.424 INFO:tasks.rados.rados.0.smithi084.stdout:162: left oid 48 (ObjNum 86 snap 9 seq_num 86) 2024-09-18T13:59:49.424 INFO:tasks.rados.rados.0.smithi084.stdout:163: finishing rollback tid 1 to smithi08434663-23 2024-09-18T13:59:49.424 INFO:tasks.rados.rados.0.smithi084.stdout:163: finishing rollback tid 2 to smithi08434663-23 2024-09-18T13:59:49.424 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 30 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T13:59:49.424 INFO:tasks.rados.rados.0.smithi084.stdout:162: done (4 left) 2024-09-18T13:59:49.424 INFO:tasks.rados.rados.0.smithi084.stdout:163: done (3 left) 2024-09-18T13:59:49.424 INFO:tasks.rados.rados.0.smithi084.stdout:167: snap_remove snap 6 2024-09-18T13:59:50.166 INFO:tasks.rados.rados.0.smithi084.stdout:164: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T13:59:50.246 INFO:tasks.rados.rados.0.smithi084.stdout:165: expect (ObjNum 63 snap 4 seq_num 63) 2024-09-18T13:59:50.380 INFO:tasks.rados.rados.0.smithi084.stdout:166: finishing copy_from racing read to smithi08434663-21 2024-09-18T13:59:50.380 INFO:tasks.rados.rados.0.smithi084.stdout:166: finishing copy_from to smithi08434663-21 2024-09-18T13:59:50.380 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 21 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T13:59:50.380 INFO:tasks.rados.rados.0.smithi084.stdout:164: done (3 left) 2024-09-18T13:59:50.380 INFO:tasks.rados.rados.0.smithi084.stdout:165: done (2 left) 2024-09-18T13:59:50.380 INFO:tasks.rados.rados.0.smithi084.stdout:166: done (1 left) 2024-09-18T13:59:50.380 INFO:tasks.rados.rados.0.smithi084.stdout:167: done (0 left) 2024-09-18T13:59:50.380 INFO:tasks.rados.rados.0.smithi084.stdout:168: delete oid 18 current snap is 9 2024-09-18T13:59:50.381 INFO:tasks.rados.rados.0.smithi084.stdout:168: done (0 left) 2024-09-18T13:59:50.382 INFO:tasks.rados.rados.0.smithi084.stdout:169: write (excl) oid 44 current snap is 9 2024-09-18T13:59:50.382 INFO:tasks.rados.rados.0.smithi084.stdout:169: seq_num 88 ranges {679016=701795,1866533=496024,2900635=402643} 2024-09-18T13:59:50.389 INFO:tasks.rados.rados.0.smithi084.stdout:169: writing smithi08434663-44 from 679016 to 1380811 tid 1 2024-09-18T13:59:50.395 INFO:tasks.rados.rados.0.smithi084.stdout:169: writing smithi08434663-44 from 1866533 to 2362557 tid 2 2024-09-18T13:59:50.400 INFO:tasks.rados.rados.0.smithi084.stdout:169: writing smithi08434663-44 from 2900635 to 3303278 tid 3 2024-09-18T13:59:50.400 INFO:tasks.rados.rados.0.smithi084.stdout:170: rollback oid 15 current snap is 9 2024-09-18T13:59:50.400 INFO:tasks.rados.rados.0.smithi084.stdout:170: done (1 left) 2024-09-18T13:59:50.401 INFO:tasks.rados.rados.0.smithi084.stdout:171: write (excl) oid 34 current snap is 9 2024-09-18T13:59:50.401 INFO:tasks.rados.rados.0.smithi084.stdout:169: finishing write tid 1 to smithi08434663-44 2024-09-18T13:59:50.401 INFO:tasks.rados.rados.0.smithi084.stdout:169: finishing write tid 2 to smithi08434663-44 2024-09-18T13:59:50.401 INFO:tasks.rados.rados.0.smithi084.stdout:171: seq_num 89 ranges {695823=777044,1925015=639682,3328058=167765} 2024-09-18T13:59:50.408 INFO:tasks.rados.rados.0.smithi084.stdout:171: writing smithi08434663-34 from 695823 to 1472867 tid 1 2024-09-18T13:59:50.414 INFO:tasks.rados.rados.0.smithi084.stdout:171: writing smithi08434663-34 from 1925015 to 2564697 tid 2 2024-09-18T13:59:50.420 INFO:tasks.rados.rados.0.smithi084.stdout:171: writing smithi08434663-34 from 3328058 to 3495823 tid 3 2024-09-18T13:59:50.420 INFO:tasks.rados.rados.0.smithi084.stdout:172: read oid 46 snap -1 2024-09-18T13:59:50.420 INFO:tasks.rados.rados.0.smithi084.stdout:172: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-18T13:59:50.420 INFO:tasks.rados.rados.0.smithi084.stdout:173: read oid 17 snap -1 2024-09-18T13:59:50.420 INFO:tasks.rados.rados.0.smithi084.stdout:173: expect (ObjNum 16 snap 0 seq_num 16) 2024-09-18T13:59:50.420 INFO:tasks.rados.rados.0.smithi084.stdout:174: copy_from oid 5 from oid 38 current snap is 9 2024-09-18T13:59:50.420 INFO:tasks.rados.rados.0.smithi084.stdout:169: finishing write tid 3 to smithi08434663-44 2024-09-18T13:59:50.420 INFO:tasks.rados.rados.0.smithi084.stdout:169: finishing write tid 4 to smithi08434663-44 2024-09-18T13:59:50.420 INFO:tasks.rados.rados.0.smithi084.stdout:169: finishing write tid 5 to smithi08434663-44 2024-09-18T13:59:50.420 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 10 (ObjNum 88 snap 9 seq_num 88) dirty exists 2024-09-18T13:59:50.420 INFO:tasks.rados.rados.0.smithi084.stdout:169: left oid 44 (ObjNum 88 snap 9 seq_num 88) 2024-09-18T13:59:50.420 INFO:tasks.rados.rados.0.smithi084.stdout:171: finishing write tid 1 to smithi08434663-34 2024-09-18T13:59:50.421 INFO:tasks.rados.rados.0.smithi084.stdout:171: finishing write tid 2 to smithi08434663-34 2024-09-18T13:59:50.421 INFO:tasks.rados.rados.0.smithi084.stdout:169: done (4 left) 2024-09-18T13:59:50.421 INFO:tasks.rados.rados.0.smithi084.stdout:175: copy_from oid 24 from oid 4 current snap is 9 2024-09-18T13:59:50.421 INFO:tasks.rados.rados.0.smithi084.stdout:176: delete oid 41 current snap is 9 2024-09-18T13:59:50.423 INFO:tasks.rados.rados.0.smithi084.stdout:175: finishing copy_from to smithi08434663-24 2024-09-18T13:59:50.423 INFO:tasks.rados.rados.0.smithi084.stdout:175: got expected ENOENT (src dne) 2024-09-18T13:59:50.423 INFO:tasks.rados.rados.0.smithi084.stdout:175: finishing copy_from racing read to smithi08434663-24 2024-09-18T13:59:50.424 INFO:tasks.rados.rados.0.smithi084.stdout:172: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-18T13:59:50.509 INFO:tasks.rados.rados.0.smithi084.stdout:171: finishing write tid 3 to smithi08434663-34 2024-09-18T13:59:50.509 INFO:tasks.rados.rados.0.smithi084.stdout:171: finishing write tid 4 to smithi08434663-34 2024-09-18T13:59:50.509 INFO:tasks.rados.rados.0.smithi084.stdout:171: finishing write tid 5 to smithi08434663-34 2024-09-18T13:59:50.509 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 62 (ObjNum 89 snap 9 seq_num 89) dirty exists 2024-09-18T13:59:50.510 INFO:tasks.rados.rados.0.smithi084.stdout:171: left oid 34 (ObjNum 89 snap 9 seq_num 89) 2024-09-18T13:59:50.510 INFO:tasks.rados.rados.0.smithi084.stdout:173: expect (ObjNum 16 snap 0 seq_num 16) 2024-09-18T13:59:50.582 INFO:tasks.rados.rados.0.smithi084.stdout:174: finishing copy_from racing read to smithi08434663-5 2024-09-18T13:59:50.583 INFO:tasks.rados.rados.0.smithi084.stdout:174: finishing copy_from to smithi08434663-5 2024-09-18T13:59:50.583 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 18 (ObjNum 72 snap 6 seq_num 72) dirty exists 2024-09-18T13:59:50.583 INFO:tasks.rados.rados.0.smithi084.stdout:171: done (5 left) 2024-09-18T13:59:50.583 INFO:tasks.rados.rados.0.smithi084.stdout:172: done (4 left) 2024-09-18T13:59:50.583 INFO:tasks.rados.rados.0.smithi084.stdout:173: done (3 left) 2024-09-18T13:59:50.583 INFO:tasks.rados.rados.0.smithi084.stdout:174: done (2 left) 2024-09-18T13:59:50.583 INFO:tasks.rados.rados.0.smithi084.stdout:175: done (1 left) 2024-09-18T13:59:50.583 INFO:tasks.rados.rados.0.smithi084.stdout:176: done (0 left) 2024-09-18T13:59:50.583 INFO:tasks.rados.rados.0.smithi084.stdout:177: write oid 39 current snap is 9 2024-09-18T13:59:50.583 INFO:tasks.rados.rados.0.smithi084.stdout:177: seq_num 91 ranges {729437=527542,1925626=643351,3266551=262886} 2024-09-18T13:59:50.589 INFO:tasks.rados.rados.0.smithi084.stdout:177: writing smithi08434663-39 from 729437 to 1256979 tid 1 2024-09-18T13:59:50.596 INFO:tasks.rados.rados.0.smithi084.stdout:177: writing smithi08434663-39 from 1925626 to 2568977 tid 2 2024-09-18T13:59:50.602 INFO:tasks.rados.rados.0.smithi084.stdout:177: writing smithi08434663-39 from 3266551 to 3529437 tid 3 2024-09-18T13:59:50.602 INFO:tasks.rados.rados.0.smithi084.stdout:178: snap_create 2024-09-18T13:59:50.602 INFO:tasks.rados.rados.0.smithi084.stdout:177: finishing write tid 1 to smithi08434663-39 2024-09-18T13:59:50.602 INFO:tasks.rados.rados.0.smithi084.stdout:177: finishing write tid 2 to smithi08434663-39 2024-09-18T13:59:50.603 INFO:tasks.rados.rados.0.smithi084.stdout:177: finishing write tid 3 to smithi08434663-39 2024-09-18T13:59:50.605 INFO:tasks.rados.rados.0.smithi084.stdout:177: finishing write tid 4 to smithi08434663-39 2024-09-18T13:59:50.605 INFO:tasks.rados.rados.0.smithi084.stdout:177: finishing write tid 5 to smithi08434663-39 2024-09-18T13:59:50.605 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 51 (ObjNum 91 snap 9 seq_num 91) dirty exists 2024-09-18T13:59:50.606 INFO:tasks.rados.rados.0.smithi084.stdout:177: left oid 39 (ObjNum 91 snap 9 seq_num 91) 2024-09-18T13:59:51.172 INFO:tasks.rados.rados.0.smithi084.stdout:177: done (1 left) 2024-09-18T13:59:51.172 INFO:tasks.rados.rados.0.smithi084.stdout:178: done (0 left) 2024-09-18T13:59:51.173 INFO:tasks.rados.rados.0.smithi084.stdout:179: copy_from oid 37 from oid 39 current snap is 10 2024-09-18T13:59:51.173 INFO:tasks.rados.rados.0.smithi084.stdout:180: write oid 3 current snap is 10 2024-09-18T13:59:51.173 INFO:tasks.rados.rados.0.smithi084.stdout:180: seq_num 92 ranges {746244=719144,2100461=503362,3126680=419564} 2024-09-18T13:59:51.181 INFO:tasks.rados.rados.0.smithi084.stdout:180: writing smithi08434663-3 from 746244 to 1465388 tid 1 2024-09-18T13:59:51.187 INFO:tasks.rados.rados.0.smithi084.stdout:180: writing smithi08434663-3 from 2100461 to 2603823 tid 2 2024-09-18T13:59:51.192 INFO:tasks.rados.rados.0.smithi084.stdout:180: writing smithi08434663-3 from 3126680 to 3546244 tid 3 2024-09-18T13:59:51.192 INFO:tasks.rados.rados.0.smithi084.stdout:181: write oid 7 current snap is 10 2024-09-18T13:59:51.192 INFO:tasks.rados.rados.0.smithi084.stdout:181: seq_num 93 ranges {763051=794393,2158943=647020,3437750=125301} 2024-09-18T13:59:51.200 INFO:tasks.rados.rados.0.smithi084.stdout:181: writing smithi08434663-7 from 763051 to 1557444 tid 1 2024-09-18T13:59:51.207 INFO:tasks.rados.rados.0.smithi084.stdout:181: writing smithi08434663-7 from 2158943 to 2805963 tid 2 2024-09-18T13:59:51.211 INFO:tasks.rados.rados.0.smithi084.stdout:181: writing smithi08434663-7 from 3437750 to 3563051 tid 3 2024-09-18T13:59:51.212 INFO:tasks.rados.rados.0.smithi084.stdout:182: copy_from oid 41 from oid 33 current snap is 10 2024-09-18T13:59:51.212 INFO:tasks.rados.rados.0.smithi084.stdout:179: finishing copy_from racing read to smithi08434663-37 2024-09-18T13:59:51.212 INFO:tasks.rados.rados.0.smithi084.stdout:180: finishing write tid 1 to smithi08434663-3 2024-09-18T13:59:51.212 INFO:tasks.rados.rados.0.smithi084.stdout:180: finishing write tid 2 to smithi08434663-3 2024-09-18T13:59:51.212 INFO:tasks.rados.rados.0.smithi084.stdout:179: finishing copy_from to smithi08434663-37 2024-09-18T13:59:51.212 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 51 (ObjNum 91 snap 9 seq_num 91) dirty exists 2024-09-18T13:59:51.212 INFO:tasks.rados.rados.0.smithi084.stdout:180: finishing write tid 3 to smithi08434663-3 2024-09-18T13:59:51.212 INFO:tasks.rados.rados.0.smithi084.stdout:180: finishing write tid 4 to smithi08434663-3 2024-09-18T13:59:51.212 INFO:tasks.rados.rados.0.smithi084.stdout:180: finishing write tid 5 to smithi08434663-3 2024-09-18T13:59:51.212 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 34 (ObjNum 92 snap 10 seq_num 92) dirty exists 2024-09-18T13:59:51.213 INFO:tasks.rados.rados.0.smithi084.stdout:180: left oid 3 (ObjNum 92 snap 10 seq_num 92) 2024-09-18T13:59:51.213 INFO:tasks.rados.rados.0.smithi084.stdout:181: finishing write tid 1 to smithi08434663-7 2024-09-18T13:59:51.213 INFO:tasks.rados.rados.0.smithi084.stdout:179: done (3 left) 2024-09-18T13:59:51.213 INFO:tasks.rados.rados.0.smithi084.stdout:180: done (2 left) 2024-09-18T13:59:51.213 INFO:tasks.rados.rados.0.smithi084.stdout:183: snap_remove snap 9 2024-09-18T13:59:52.179 INFO:tasks.rados.rados.0.smithi084.stdout:183: done (2 left) 2024-09-18T13:59:52.179 INFO:tasks.rados.rados.0.smithi084.stdout:184: rollback oid 35 current snap is 10 2024-09-18T13:59:52.179 INFO:tasks.rados.rados.0.smithi084.stdout:184: done (2 left) 2024-09-18T13:59:52.179 INFO:tasks.rados.rados.0.smithi084.stdout:185: read oid 23 snap -1 2024-09-18T13:59:52.179 INFO:tasks.rados.rados.0.smithi084.stdout:185: expect (ObjNum 22 snap 0 seq_num 22) 2024-09-18T13:59:52.179 INFO:tasks.rados.rados.0.smithi084.stdout:186: read oid 37 snap -1 2024-09-18T13:59:52.179 INFO:tasks.rados.rados.0.smithi084.stdout:186: expect (ObjNum 91 snap 9 seq_num 91) 2024-09-18T13:59:52.179 INFO:tasks.rados.rados.0.smithi084.stdout:187: write (excl) oid 4 current snap is 10 2024-09-18T13:59:52.179 INFO:tasks.rados.rados.0.smithi084.stdout:187: seq_num 94 ranges {779858=469642,1817425=507031,2781526=586628} 2024-09-18T13:59:52.185 INFO:tasks.rados.rados.0.smithi084.stdout:187: writing smithi08434663-4 from 779858 to 1249500 tid 1 2024-09-18T13:59:52.191 INFO:tasks.rados.rados.0.smithi084.stdout:187: writing smithi08434663-4 from 1817425 to 2324456 tid 2 2024-09-18T13:59:52.197 INFO:tasks.rados.rados.0.smithi084.stdout:187: writing smithi08434663-4 from 2781526 to 3368154 tid 3 2024-09-18T13:59:52.197 INFO:tasks.rados.rados.0.smithi084.stdout:188: read oid 31 snap -1 2024-09-18T13:59:52.197 INFO:tasks.rados.rados.0.smithi084.stdout:188: expect (ObjNum 66 snap 6 seq_num 66) 2024-09-18T13:59:52.197 INFO:tasks.rados.rados.0.smithi084.stdout:189: write oid 2 current snap is 10 2024-09-18T13:59:52.197 INFO:tasks.rados.rados.0.smithi084.stdout:181: finishing write tid 2 to smithi08434663-7 2024-09-18T13:59:52.198 INFO:tasks.rados.rados.0.smithi084.stdout:189: seq_num 95 ranges {796665=544891,1759554=650689,2976243=620422} 2024-09-18T13:59:52.204 INFO:tasks.rados.rados.0.smithi084.stdout:189: writing smithi08434663-2 from 796665 to 1341556 tid 1 2024-09-18T13:59:52.210 INFO:tasks.rados.rados.0.smithi084.stdout:189: writing smithi08434663-2 from 1759554 to 2410243 tid 2 2024-09-18T13:59:52.217 INFO:tasks.rados.rados.0.smithi084.stdout:189: writing smithi08434663-2 from 2976243 to 3596665 tid 3 2024-09-18T13:59:52.217 INFO:tasks.rados.rados.0.smithi084.stdout:190: snap_create 2024-09-18T13:59:52.217 INFO:tasks.rados.rados.0.smithi084.stdout:181: finishing write tid 3 to smithi08434663-7 2024-09-18T13:59:52.217 INFO:tasks.rados.rados.0.smithi084.stdout:181: finishing write tid 4 to smithi08434663-7 2024-09-18T13:59:52.217 INFO:tasks.rados.rados.0.smithi084.stdout:181: finishing write tid 5 to smithi08434663-7 2024-09-18T13:59:52.217 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 37 (ObjNum 93 snap 10 seq_num 93) dirty exists 2024-09-18T13:59:52.217 INFO:tasks.rados.rados.0.smithi084.stdout:181: left oid 7 (ObjNum 93 snap 10 seq_num 93) 2024-09-18T13:59:52.217 INFO:tasks.rados.rados.0.smithi084.stdout:182: finishing copy_from racing read to smithi08434663-41 2024-09-18T13:59:52.217 INFO:tasks.rados.rados.0.smithi084.stdout:182: finishing copy_from to smithi08434663-41 2024-09-18T13:59:52.217 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 20 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T13:59:52.217 INFO:tasks.rados.rados.0.smithi084.stdout:187: finishing write tid 1 to smithi08434663-4 2024-09-18T13:59:52.218 INFO:tasks.rados.rados.0.smithi084.stdout:185: expect (ObjNum 22 snap 0 seq_num 22) 2024-09-18T13:59:52.293 INFO:tasks.rados.rados.0.smithi084.stdout:187: finishing write tid 2 to smithi08434663-4 2024-09-18T13:59:52.293 INFO:tasks.rados.rados.0.smithi084.stdout:186: expect (ObjNum 91 snap 9 seq_num 91) 2024-09-18T13:59:52.490 INFO:tasks.rados.rados.0.smithi084.stdout:187: finishing write tid 3 to smithi08434663-4 2024-09-18T13:59:52.491 INFO:tasks.rados.rados.0.smithi084.stdout:187: finishing write tid 4 to smithi08434663-4 2024-09-18T13:59:52.491 INFO:tasks.rados.rados.0.smithi084.stdout:187: finishing write tid 5 to smithi08434663-4 2024-09-18T13:59:52.491 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 68 (ObjNum 94 snap 10 seq_num 94) dirty exists 2024-09-18T13:59:52.491 INFO:tasks.rados.rados.0.smithi084.stdout:187: left oid 4 (ObjNum 94 snap 10 seq_num 94) 2024-09-18T13:59:52.491 INFO:tasks.rados.rados.0.smithi084.stdout:188: expect (ObjNum 66 snap 6 seq_num 66) 2024-09-18T13:59:52.623 INFO:tasks.rados.rados.0.smithi084.stdout:189: finishing write tid 1 to smithi08434663-2 2024-09-18T13:59:52.623 INFO:tasks.rados.rados.0.smithi084.stdout:189: finishing write tid 2 to smithi08434663-2 2024-09-18T13:59:52.623 INFO:tasks.rados.rados.0.smithi084.stdout:189: finishing write tid 3 to smithi08434663-2 2024-09-18T13:59:52.624 INFO:tasks.rados.rados.0.smithi084.stdout:189: finishing write tid 4 to smithi08434663-2 2024-09-18T13:59:52.624 INFO:tasks.rados.rados.0.smithi084.stdout:189: finishing write tid 5 to smithi08434663-2 2024-09-18T13:59:52.624 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 96 (ObjNum 95 snap 10 seq_num 95) dirty exists 2024-09-18T13:59:52.624 INFO:tasks.rados.rados.0.smithi084.stdout:189: left oid 2 (ObjNum 95 snap 10 seq_num 95) 2024-09-18T13:59:53.179 INFO:tasks.rados.rados.0.smithi084.stdout:181: done (7 left) 2024-09-18T13:59:53.179 INFO:tasks.rados.rados.0.smithi084.stdout:182: done (6 left) 2024-09-18T13:59:53.179 INFO:tasks.rados.rados.0.smithi084.stdout:185: done (5 left) 2024-09-18T13:59:53.179 INFO:tasks.rados.rados.0.smithi084.stdout:186: done (4 left) 2024-09-18T13:59:53.179 INFO:tasks.rados.rados.0.smithi084.stdout:187: done (3 left) 2024-09-18T13:59:53.179 INFO:tasks.rados.rados.0.smithi084.stdout:188: done (2 left) 2024-09-18T13:59:53.179 INFO:tasks.rados.rados.0.smithi084.stdout:189: done (1 left) 2024-09-18T13:59:53.179 INFO:tasks.rados.rados.0.smithi084.stdout:190: done (0 left) 2024-09-18T13:59:53.180 INFO:tasks.rados.rados.0.smithi084.stdout:191: read oid 45 snap -1 2024-09-18T13:59:53.180 INFO:tasks.rados.rados.0.smithi084.stdout:191: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-18T13:59:53.180 INFO:tasks.rados.rados.0.smithi084.stdout:192: write oid 21 current snap is 11 2024-09-18T13:59:53.180 INFO:tasks.rados.rados.0.smithi084.stdout:192: seq_num 96 ranges {413472=620140,1818036=510700,3120019=493453} 2024-09-18T13:59:53.185 INFO:tasks.rados.rados.0.smithi084.stdout:192: writing smithi08434663-21 from 413472 to 1033612 tid 1 2024-09-18T13:59:53.192 INFO:tasks.rados.rados.0.smithi084.stdout:192: writing smithi08434663-21 from 1818036 to 2328736 tid 2 2024-09-18T13:59:53.199 INFO:tasks.rados.rados.0.smithi084.stdout:192: writing smithi08434663-21 from 3120019 to 3613472 tid 3 2024-09-18T13:59:53.199 INFO:tasks.rados.rados.0.smithi084.stdout:193: write oid 32 current snap is 11 2024-09-18T13:59:53.200 INFO:tasks.rados.rados.0.smithi084.stdout:193: seq_num 97 ranges {430279=695389,1876518=654358,3031089=536797} 2024-09-18T13:59:53.205 INFO:tasks.rados.rados.0.smithi084.stdout:193: writing smithi08434663-32 from 430279 to 1125668 tid 1 2024-09-18T13:59:53.213 INFO:tasks.rados.rados.0.smithi084.stdout:193: writing smithi08434663-32 from 1876518 to 2530876 tid 2 2024-09-18T13:59:53.219 INFO:tasks.rados.rados.0.smithi084.stdout:193: writing smithi08434663-32 from 3031089 to 3567886 tid 3 2024-09-18T13:59:53.219 INFO:tasks.rados.rados.0.smithi084.stdout:194: snap_create 2024-09-18T13:59:53.219 INFO:tasks.rados.rados.0.smithi084.stdout:191: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-18T13:59:53.305 INFO:tasks.rados.rados.0.smithi084.stdout:192: finishing write tid 1 to smithi08434663-21 2024-09-18T13:59:53.305 INFO:tasks.rados.rados.0.smithi084.stdout:192: finishing write tid 2 to smithi08434663-21 2024-09-18T13:59:53.305 INFO:tasks.rados.rados.0.smithi084.stdout:192: finishing write tid 3 to smithi08434663-21 2024-09-18T13:59:53.305 INFO:tasks.rados.rados.0.smithi084.stdout:192: finishing write tid 4 to smithi08434663-21 2024-09-18T13:59:53.305 INFO:tasks.rados.rados.0.smithi084.stdout:192: finishing write tid 5 to smithi08434663-21 2024-09-18T13:59:53.306 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 40 (ObjNum 96 snap 11 seq_num 96) dirty exists 2024-09-18T13:59:53.306 INFO:tasks.rados.rados.0.smithi084.stdout:192: left oid 21 (ObjNum 96 snap 11 seq_num 96) 2024-09-18T13:59:53.306 INFO:tasks.rados.rados.0.smithi084.stdout:193: finishing write tid 1 to smithi08434663-32 2024-09-18T13:59:53.306 INFO:tasks.rados.rados.0.smithi084.stdout:193: finishing write tid 2 to smithi08434663-32 2024-09-18T13:59:53.306 INFO:tasks.rados.rados.0.smithi084.stdout:193: finishing write tid 3 to smithi08434663-32 2024-09-18T13:59:53.306 INFO:tasks.rados.rados.0.smithi084.stdout:193: finishing write tid 4 to smithi08434663-32 2024-09-18T13:59:53.306 INFO:tasks.rados.rados.0.smithi084.stdout:193: finishing write tid 5 to smithi08434663-32 2024-09-18T13:59:53.306 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 101 (ObjNum 97 snap 11 seq_num 97) dirty exists 2024-09-18T13:59:53.306 INFO:tasks.rados.rados.0.smithi084.stdout:193: left oid 32 (ObjNum 97 snap 11 seq_num 97) 2024-09-18T13:59:54.189 INFO:tasks.rados.rados.0.smithi084.stdout:191: done (3 left) 2024-09-18T13:59:54.189 INFO:tasks.rados.rados.0.smithi084.stdout:192: done (2 left) 2024-09-18T13:59:54.189 INFO:tasks.rados.rados.0.smithi084.stdout:193: done (1 left) 2024-09-18T13:59:54.189 INFO:tasks.rados.rados.0.smithi084.stdout:194: done (0 left) 2024-09-18T13:59:54.189 INFO:tasks.rados.rados.0.smithi084.stdout:195: read oid 20 snap -1 2024-09-18T13:59:54.189 INFO:tasks.rados.rados.0.smithi084.stdout:195: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-18T13:59:54.189 INFO:tasks.rados.rados.0.smithi084.stdout:196: rollback oid 43 current snap is 12 2024-09-18T13:59:54.189 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 11 2024-09-18T13:59:54.190 INFO:tasks.rados.rados.0.smithi084.stdout:197: snap_create 2024-09-18T13:59:54.193 INFO:tasks.rados.rados.0.smithi084.stdout:196: finishing rollback tid 0 to smithi08434663-43 2024-09-18T13:59:54.193 INFO:tasks.rados.rados.0.smithi084.stdout:196: finishing rollback tid 1 to smithi08434663-43 2024-09-18T13:59:54.193 INFO:tasks.rados.rados.0.smithi084.stdout:196: finishing rollback tid 2 to smithi08434663-43 2024-09-18T13:59:54.193 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 24 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T13:59:54.197 INFO:tasks.rados.rados.0.smithi084.stdout:195: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-18T13:59:55.190 INFO:tasks.rados.rados.0.smithi084.stdout:195: done (2 left) 2024-09-18T13:59:55.190 INFO:tasks.rados.rados.0.smithi084.stdout:196: done (1 left) 2024-09-18T13:59:55.190 INFO:tasks.rados.rados.0.smithi084.stdout:197: done (0 left) 2024-09-18T13:59:55.190 INFO:tasks.rados.rados.0.smithi084.stdout:198: snap_remove snap 12 2024-09-18T13:59:56.208 INFO:tasks.rados.rados.0.smithi084.stdout:198: done (0 left) 2024-09-18T13:59:56.209 INFO:tasks.rados.rados.0.smithi084.stdout:199: snap_create 2024-09-18T13:59:57.141 INFO:tasks.rados.rados.0.smithi084.stdout:199: done (0 left) 2024-09-18T13:59:57.141 INFO:tasks.rados.rados.0.smithi084.stdout:200: snap_remove snap 10 2024-09-18T13:59:58.148 INFO:tasks.rados.rados.0.smithi084.stdout:200: done (0 left) 2024-09-18T13:59:58.264 INFO:tasks.rados.rados.0.smithi084.stdout:201: write oid 46 current snap is 14 2024-09-18T13:59:58.264 INFO:tasks.rados.rados.0.smithi084.stdout:201: seq_num 98 ranges {447086=770638,1935000=798016,3458512=188574} 2024-09-18T13:59:58.265 INFO:tasks.rados.rados.0.smithi084.stdout:201: writing smithi08434663-46 from 447086 to 1217724 tid 1 2024-09-18T13:59:58.266 INFO:tasks.rados.rados.0.smithi084.stdout:201: writing smithi08434663-46 from 1935000 to 2733016 tid 2 2024-09-18T13:59:58.266 INFO:tasks.rados.rados.0.smithi084.stdout:201: writing smithi08434663-46 from 3458512 to 3647086 tid 3 2024-09-18T13:59:58.266 INFO:tasks.rados.rados.0.smithi084.stdout:202: rollback oid 27 current snap is 14 2024-09-18T13:59:58.266 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 27 to 11 2024-09-18T13:59:58.266 INFO:tasks.rados.rados.0.smithi084.stdout:201: finishing write tid 1 to smithi08434663-46 2024-09-18T13:59:58.266 INFO:tasks.rados.rados.0.smithi084.stdout:201: finishing write tid 2 to smithi08434663-46 2024-09-18T13:59:58.266 INFO:tasks.rados.rados.0.smithi084.stdout:203: snap_remove snap 13 2024-09-18T13:59:59.149 INFO:tasks.rados.rados.0.smithi084.stdout:203: done (2 left) 2024-09-18T13:59:59.149 INFO:tasks.rados.rados.0.smithi084.stdout:204: snap_create 2024-09-18T13:59:59.149 INFO:tasks.rados.rados.0.smithi084.stdout:201: finishing write tid 3 to smithi08434663-46 2024-09-18T13:59:59.149 INFO:tasks.rados.rados.0.smithi084.stdout:201: finishing write tid 4 to smithi08434663-46 2024-09-18T13:59:59.150 INFO:tasks.rados.rados.0.smithi084.stdout:201: finishing write tid 5 to smithi08434663-46 2024-09-18T13:59:59.150 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 59 (ObjNum 98 snap 14 seq_num 98) dirty exists 2024-09-18T13:59:59.150 INFO:tasks.rados.rados.0.smithi084.stdout:201: left oid 46 (ObjNum 98 snap 14 seq_num 98) 2024-09-18T13:59:59.150 INFO:tasks.rados.rados.0.smithi084.stdout:202: finishing rollback tid 0 to smithi08434663-27 2024-09-18T13:59:59.150 INFO:tasks.rados.rados.0.smithi084.stdout:202: finishing rollback tid 1 to smithi08434663-27 2024-09-18T13:59:59.150 INFO:tasks.rados.rados.0.smithi084.stdout:202: finishing rollback tid 2 to smithi08434663-27 2024-09-18T13:59:59.150 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 78 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T14:00:00.162 INFO:tasks.rados.rados.0.smithi084.stdout:201: done (2 left) 2024-09-18T14:00:00.163 INFO:tasks.rados.rados.0.smithi084.stdout:202: done (1 left) 2024-09-18T14:00:00.163 INFO:tasks.rados.rados.0.smithi084.stdout:204: done (0 left) 2024-09-18T14:00:00.163 INFO:tasks.rados.rados.0.smithi084.stdout:205: snap_create 2024-09-18T14:00:01.165 INFO:tasks.rados.rados.0.smithi084.stdout:205: done (0 left) 2024-09-18T14:00:01.165 INFO:tasks.rados.rados.0.smithi084.stdout:206: copy_from oid 45 from oid 26 current snap is 16 2024-09-18T14:00:01.165 INFO:tasks.rados.rados.0.smithi084.stdout:207: delete oid 12 current snap is 16 2024-09-18T14:00:01.165 INFO:tasks.rados.rados.0.smithi084.stdout:206: finishing copy_from to smithi08434663-45 2024-09-18T14:00:01.165 INFO:tasks.rados.rados.0.smithi084.stdout:206: got expected ENOENT (src dne) 2024-09-18T14:00:01.165 INFO:tasks.rados.rados.0.smithi084.stdout:206: finishing copy_from racing read to smithi08434663-45 2024-09-18T14:00:01.168 INFO:tasks.rados.rados.0.smithi084.stdout:206: done (1 left) 2024-09-18T14:00:01.168 INFO:tasks.rados.rados.0.smithi084.stdout:207: done (0 left) 2024-09-18T14:00:01.168 INFO:tasks.rados.rados.0.smithi084.stdout:208: write (excl) oid 25 current snap is 16 2024-09-18T14:00:01.168 INFO:tasks.rados.rados.0.smithi084.stdout:208: seq_num 100 ranges {480700=637489,1651964=401685,2713358=770613} 2024-09-18T14:00:01.174 INFO:tasks.rados.rados.0.smithi084.stdout:208: writing smithi08434663-25 from 480700 to 1118189 tid 1 2024-09-18T14:00:01.179 INFO:tasks.rados.rados.0.smithi084.stdout:208: writing smithi08434663-25 from 1651964 to 2053649 tid 2 2024-09-18T14:00:01.187 INFO:tasks.rados.rados.0.smithi084.stdout:208: writing smithi08434663-25 from 2713358 to 3483971 tid 3 2024-09-18T14:00:01.187 INFO:tasks.rados.rados.0.smithi084.stdout:209: read oid 46 snap -1 2024-09-18T14:00:01.188 INFO:tasks.rados.rados.0.smithi084.stdout:209: expect (ObjNum 98 snap 14 seq_num 98) 2024-09-18T14:00:01.188 INFO:tasks.rados.rados.0.smithi084.stdout:210: delete oid 4 current snap is 16 2024-09-18T14:00:01.188 INFO:tasks.rados.rados.0.smithi084.stdout:208: finishing write tid 1 to smithi08434663-25 2024-09-18T14:00:01.188 INFO:tasks.rados.rados.0.smithi084.stdout:208: finishing write tid 2 to smithi08434663-25 2024-09-18T14:00:01.190 INFO:tasks.rados.rados.0.smithi084.stdout:210: done (2 left) 2024-09-18T14:00:01.191 INFO:tasks.rados.rados.0.smithi084.stdout:211: delete oid 20 current snap is 16 2024-09-18T14:00:01.191 INFO:tasks.rados.rados.0.smithi084.stdout:208: finishing write tid 3 to smithi08434663-25 2024-09-18T14:00:01.191 INFO:tasks.rados.rados.0.smithi084.stdout:208: finishing write tid 4 to smithi08434663-25 2024-09-18T14:00:01.192 INFO:tasks.rados.rados.0.smithi084.stdout:208: finishing write tid 5 to smithi08434663-25 2024-09-18T14:00:01.192 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 47 (ObjNum 100 snap 16 seq_num 100) dirty exists 2024-09-18T14:00:01.192 INFO:tasks.rados.rados.0.smithi084.stdout:208: left oid 25 (ObjNum 100 snap 16 seq_num 100) 2024-09-18T14:00:01.192 INFO:tasks.rados.rados.0.smithi084.stdout:208: done (2 left) 2024-09-18T14:00:01.193 INFO:tasks.rados.rados.0.smithi084.stdout:211: done (1 left) 2024-09-18T14:00:01.193 INFO:tasks.rados.rados.0.smithi084.stdout:212: read oid 4 snap -1 2024-09-18T14:00:01.193 INFO:tasks.rados.rados.0.smithi084.stdout:212: expect deleted 2024-09-18T14:00:01.193 INFO:tasks.rados.rados.0.smithi084.stdout:213: snap_create 2024-09-18T14:00:01.197 INFO:tasks.rados.rados.0.smithi084.stdout:209: expect (ObjNum 98 snap 14 seq_num 98) 2024-09-18T14:00:02.174 INFO:tasks.rados.rados.0.smithi084.stdout:209: done (2 left) 2024-09-18T14:00:02.174 INFO:tasks.rados.rados.0.smithi084.stdout:212: done (1 left) 2024-09-18T14:00:02.174 INFO:tasks.rados.rados.0.smithi084.stdout:213: done (0 left) 2024-09-18T14:00:02.174 INFO:tasks.rados.rados.0.smithi084.stdout:214: read oid 49 snap -1 2024-09-18T14:00:02.175 INFO:tasks.rados.rados.0.smithi084.stdout:214: expect deleted 2024-09-18T14:00:02.175 INFO:tasks.rados.rados.0.smithi084.stdout:215: snap_remove snap 14 2024-09-18T14:00:03.175 INFO:tasks.rados.rados.0.smithi084.stdout:215: done (1 left) 2024-09-18T14:00:03.175 INFO:tasks.rados.rados.0.smithi084.stdout:216: write (excl) oid 48 current snap is 17 2024-09-18T14:00:03.175 INFO:tasks.rados.rados.0.smithi084.stdout:216: seq_num 103 ranges {531121=463236,1711057=665365,3079274=651847} 2024-09-18T14:00:03.180 INFO:tasks.rados.rados.0.smithi084.stdout:216: writing smithi08434663-48 from 531121 to 994357 tid 1 2024-09-18T14:00:03.188 INFO:tasks.rados.rados.0.smithi084.stdout:216: writing smithi08434663-48 from 1711057 to 2376422 tid 2 2024-09-18T14:00:03.196 INFO:tasks.rados.rados.0.smithi084.stdout:216: writing smithi08434663-48 from 3079274 to 3731121 tid 3 2024-09-18T14:00:03.196 INFO:tasks.rados.rados.0.smithi084.stdout:217: snap_create 2024-09-18T14:00:03.196 INFO:tasks.rados.rados.0.smithi084.stdout:216: finishing write tid 1 to smithi08434663-48 2024-09-18T14:00:03.196 INFO:tasks.rados.rados.0.smithi084.stdout:216: finishing write tid 2 to smithi08434663-48 2024-09-18T14:00:03.199 INFO:tasks.rados.rados.0.smithi084.stdout:216: finishing write tid 3 to smithi08434663-48 2024-09-18T14:00:03.199 INFO:tasks.rados.rados.0.smithi084.stdout:216: finishing write tid 4 to smithi08434663-48 2024-09-18T14:00:03.199 INFO:tasks.rados.rados.0.smithi084.stdout:216: finishing write tid 5 to smithi08434663-48 2024-09-18T14:00:03.199 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 54 (ObjNum 103 snap 17 seq_num 103) dirty exists 2024-09-18T14:00:03.199 INFO:tasks.rados.rados.0.smithi084.stdout:216: left oid 48 (ObjNum 103 snap 17 seq_num 103) 2024-09-18T14:00:04.189 INFO:tasks.rados.rados.0.smithi084.stdout:214: done (2 left) 2024-09-18T14:00:04.189 INFO:tasks.rados.rados.0.smithi084.stdout:216: done (1 left) 2024-09-18T14:00:04.189 INFO:tasks.rados.rados.0.smithi084.stdout:217: done (0 left) 2024-09-18T14:00:04.189 INFO:tasks.rados.rados.0.smithi084.stdout:218: read oid 27 snap -1 2024-09-18T14:00:04.190 INFO:tasks.rados.rados.0.smithi084.stdout:218: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T14:00:04.190 INFO:tasks.rados.rados.0.smithi084.stdout:219: copy_from oid 26 from oid 49 current snap is 18 2024-09-18T14:00:04.190 INFO:tasks.rados.rados.0.smithi084.stdout:220: snap_remove snap 15 2024-09-18T14:00:05.191 INFO:tasks.rados.rados.0.smithi084.stdout:220: done (2 left) 2024-09-18T14:00:05.191 INFO:tasks.rados.rados.0.smithi084.stdout:221: copy_from oid 9 from oid 15 current snap is 18 2024-09-18T14:00:05.191 INFO:tasks.rados.rados.0.smithi084.stdout:218: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T14:00:05.268 INFO:tasks.rados.rados.0.smithi084.stdout:219: finishing copy_from to smithi08434663-26 2024-09-18T14:00:05.268 INFO:tasks.rados.rados.0.smithi084.stdout:219: got expected ENOENT (src dne) 2024-09-18T14:00:05.268 INFO:tasks.rados.rados.0.smithi084.stdout:219: finishing copy_from racing read to smithi08434663-26 2024-09-18T14:00:05.268 INFO:tasks.rados.rados.0.smithi084.stdout:221: finishing copy_from to smithi08434663-9 2024-09-18T14:00:05.268 INFO:tasks.rados.rados.0.smithi084.stdout:221: got expected ENOENT (src dne) 2024-09-18T14:00:05.268 INFO:tasks.rados.rados.0.smithi084.stdout:221: finishing copy_from racing read to smithi08434663-9 2024-09-18T14:00:05.268 INFO:tasks.rados.rados.0.smithi084.stdout:218: done (2 left) 2024-09-18T14:00:05.268 INFO:tasks.rados.rados.0.smithi084.stdout:219: done (1 left) 2024-09-18T14:00:05.268 INFO:tasks.rados.rados.0.smithi084.stdout:221: done (0 left) 2024-09-18T14:00:05.268 INFO:tasks.rados.rados.0.smithi084.stdout:222: delete oid 12 current snap is 18 2024-09-18T14:00:05.271 INFO:tasks.rados.rados.0.smithi084.stdout:222: done (0 left) 2024-09-18T14:00:05.271 INFO:tasks.rados.rados.0.smithi084.stdout:223: snap_remove snap 11 2024-09-18T14:00:06.200 INFO:tasks.rados.rados.0.smithi084.stdout:223: done (0 left) 2024-09-18T14:00:06.201 INFO:tasks.rados.rados.0.smithi084.stdout:224: snap_create 2024-09-18T14:00:07.142 INFO:tasks.rados.rados.0.smithi084.stdout:224: done (0 left) 2024-09-18T14:00:07.142 INFO:tasks.rados.rados.0.smithi084.stdout:225: write (excl) oid 28 current snap is 19 2024-09-18T14:00:07.143 INFO:tasks.rados.rados.0.smithi084.stdout:225: seq_num 105 ranges {564735=613734,1828021=669034,3134120=630615} 2024-09-18T14:00:07.148 INFO:tasks.rados.rados.0.smithi084.stdout:225: writing smithi08434663-28 from 564735 to 1178469 tid 1 2024-09-18T14:00:07.156 INFO:tasks.rados.rados.0.smithi084.stdout:225: writing smithi08434663-28 from 1828021 to 2497055 tid 2 2024-09-18T14:00:07.164 INFO:tasks.rados.rados.0.smithi084.stdout:225: writing smithi08434663-28 from 3134120 to 3764735 tid 3 2024-09-18T14:00:07.164 INFO:tasks.rados.rados.0.smithi084.stdout:226: write oid 12 current snap is 19 2024-09-18T14:00:07.165 INFO:tasks.rados.rados.0.smithi084.stdout:225: finishing write tid 1 to smithi08434663-28 2024-09-18T14:00:07.165 INFO:tasks.rados.rados.0.smithi084.stdout:226: seq_num 106 ranges {581542=688983,1886503=412692,2761543=670951} 2024-09-18T14:00:07.171 INFO:tasks.rados.rados.0.smithi084.stdout:226: writing smithi08434663-12 from 581542 to 1270525 tid 1 2024-09-18T14:00:07.177 INFO:tasks.rados.rados.0.smithi084.stdout:226: writing smithi08434663-12 from 1886503 to 2299195 tid 2 2024-09-18T14:00:07.183 INFO:tasks.rados.rados.0.smithi084.stdout:226: writing smithi08434663-12 from 2761543 to 3432494 tid 3 2024-09-18T14:00:07.183 INFO:tasks.rados.rados.0.smithi084.stdout:227: write oid 19 current snap is 19 2024-09-18T14:00:07.183 INFO:tasks.rados.rados.0.smithi084.stdout:225: finishing write tid 2 to smithi08434663-28 2024-09-18T14:00:07.183 INFO:tasks.rados.rados.0.smithi084.stdout:225: finishing write tid 3 to smithi08434663-28 2024-09-18T14:00:07.183 INFO:tasks.rados.rados.0.smithi084.stdout:225: finishing write tid 4 to smithi08434663-28 2024-09-18T14:00:07.183 INFO:tasks.rados.rados.0.smithi084.stdout:227: seq_num 107 ranges {598349=480585,1544985=672703,2905319=482223} 2024-09-18T14:00:07.189 INFO:tasks.rados.rados.0.smithi084.stdout:227: writing smithi08434663-19 from 598349 to 1078934 tid 1 2024-09-18T14:00:07.195 INFO:tasks.rados.rados.0.smithi084.stdout:227: writing smithi08434663-19 from 1544985 to 2217688 tid 2 2024-09-18T14:00:07.202 INFO:tasks.rados.rados.0.smithi084.stdout:227: writing smithi08434663-19 from 2905319 to 3387542 tid 3 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:228: write oid 8 current snap is 19 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:225: finishing write tid 5 to smithi08434663-28 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 117 (ObjNum 105 snap 19 seq_num 105) dirty exists 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:225: left oid 28 (ObjNum 105 snap 19 seq_num 105) 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:226: finishing write tid 1 to smithi08434663-12 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:226: finishing write tid 2 to smithi08434663-12 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:226: finishing write tid 3 to smithi08434663-12 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:226: finishing write tid 4 to smithi08434663-12 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:226: finishing write tid 5 to smithi08434663-12 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 121 (ObjNum 106 snap 19 seq_num 106) dirty exists 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:226: left oid 12 (ObjNum 106 snap 19 seq_num 106) 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:227: finishing write tid 1 to smithi08434663-19 2024-09-18T14:00:07.203 INFO:tasks.rados.rados.0.smithi084.stdout:227: finishing write tid 2 to smithi08434663-19 2024-09-18T14:00:07.204 INFO:tasks.rados.rados.0.smithi084.stdout:228: seq_num 108 ranges {615156=555834,1603467=416361,2816389=693495} 2024-09-18T14:00:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:228: writing smithi08434663-8 from 615156 to 1170990 tid 1 2024-09-18T14:00:07.213 INFO:tasks.rados.rados.0.smithi084.stdout:228: writing smithi08434663-8 from 1603467 to 2019828 tid 2 2024-09-18T14:00:07.221 INFO:tasks.rados.rados.0.smithi084.stdout:228: writing smithi08434663-8 from 2816389 to 3509884 tid 3 2024-09-18T14:00:07.221 INFO:tasks.rados.rados.0.smithi084.stdout:225: done (3 left) 2024-09-18T14:00:07.222 INFO:tasks.rados.rados.0.smithi084.stdout:226: done (2 left) 2024-09-18T14:00:07.222 INFO:tasks.rados.rados.0.smithi084.stdout:229: snap_create 2024-09-18T14:00:07.222 INFO:tasks.rados.rados.0.smithi084.stdout:227: finishing write tid 3 to smithi08434663-19 2024-09-18T14:00:07.222 INFO:tasks.rados.rados.0.smithi084.stdout:227: finishing write tid 4 to smithi08434663-19 2024-09-18T14:00:07.222 INFO:tasks.rados.rados.0.smithi084.stdout:227: finishing write tid 5 to smithi08434663-19 2024-09-18T14:00:07.222 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 125 (ObjNum 107 snap 19 seq_num 107) dirty exists 2024-09-18T14:00:07.222 INFO:tasks.rados.rados.0.smithi084.stdout:227: left oid 19 (ObjNum 107 snap 19 seq_num 107) 2024-09-18T14:00:07.222 INFO:tasks.rados.rados.0.smithi084.stdout:228: finishing write tid 1 to smithi08434663-8 2024-09-18T14:00:07.222 INFO:tasks.rados.rados.0.smithi084.stdout:228: finishing write tid 2 to smithi08434663-8 2024-09-18T14:00:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:228: finishing write tid 3 to smithi08434663-8 2024-09-18T14:00:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:228: finishing write tid 4 to smithi08434663-8 2024-09-18T14:00:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:228: finishing write tid 5 to smithi08434663-8 2024-09-18T14:00:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 37 (ObjNum 108 snap 19 seq_num 108) dirty exists 2024-09-18T14:00:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:228: left oid 8 (ObjNum 108 snap 19 seq_num 108) 2024-09-18T14:00:08.206 INFO:tasks.rados.rados.0.smithi084.stdout:227: done (2 left) 2024-09-18T14:00:08.206 INFO:tasks.rados.rados.0.smithi084.stdout:228: done (1 left) 2024-09-18T14:00:08.206 INFO:tasks.rados.rados.0.smithi084.stdout:229: done (0 left) 2024-09-18T14:00:08.207 INFO:tasks.rados.rados.0.smithi084.stdout:230: snap_create 2024-09-18T14:00:09.208 INFO:tasks.rados.rados.0.smithi084.stdout:230: done (0 left) 2024-09-18T14:00:09.208 INFO:tasks.rados.rados.0.smithi084.stdout:231: rollback oid 18 current snap is 21 2024-09-18T14:00:09.208 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 18 to 18 2024-09-18T14:00:09.209 INFO:tasks.rados.rados.0.smithi084.stdout:232: snap_remove snap 17 2024-09-18T14:00:10.221 INFO:tasks.rados.rados.0.smithi084.stdout:232: done (1 left) 2024-09-18T14:00:10.221 INFO:tasks.rados.rados.0.smithi084.stdout:233: copy_from oid 19 from oid 9 current snap is 21 2024-09-18T14:00:10.221 INFO:tasks.rados.rados.0.smithi084.stdout:231: finishing rollback tid 1 to smithi08434663-18 2024-09-18T14:00:10.221 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 0 (ObjNum 177 snap 0 seq_num 2495428144) dirty dne 2024-09-18T14:00:10.221 INFO:tasks.rados.rados.0.smithi084.stdout:231: done (1 left) 2024-09-18T14:00:10.221 INFO:tasks.rados.rados.0.smithi084.stdout:234: delete oid 11 current snap is 21 2024-09-18T14:00:10.225 INFO:tasks.rados.rados.0.smithi084.stdout:234: done (1 left) 2024-09-18T14:00:10.225 INFO:tasks.rados.rados.0.smithi084.stdout:235: write (excl) oid 35 current snap is 21 2024-09-18T14:00:10.226 INFO:tasks.rados.rados.0.smithi084.stdout:235: seq_num 110 ranges {648770=706332,2120431=420030,3271235=432392} 2024-09-18T14:00:10.233 INFO:tasks.rados.rados.0.smithi084.stdout:235: writing smithi08434663-35 from 648770 to 1355102 tid 1 2024-09-18T14:00:10.239 INFO:tasks.rados.rados.0.smithi084.stdout:235: writing smithi08434663-35 from 2120431 to 2540461 tid 2 2024-09-18T14:00:10.247 INFO:tasks.rados.rados.0.smithi084.stdout:235: writing smithi08434663-35 from 3271235 to 3703627 tid 3 2024-09-18T14:00:10.247 INFO:tasks.rados.rados.0.smithi084.stdout:236: rollback oid 33 current snap is 21 2024-09-18T14:00:10.247 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 33 to 18 2024-09-18T14:00:10.247 INFO:tasks.rados.rados.0.smithi084.stdout:233: finishing copy_from racing read to smithi08434663-19 2024-09-18T14:00:10.247 INFO:tasks.rados.rados.0.smithi084.stdout:233: finishing copy_from to smithi08434663-19 2024-09-18T14:00:10.247 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 36 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T14:00:10.247 INFO:tasks.rados.rados.0.smithi084.stdout:235: finishing write tid 1 to smithi08434663-35 2024-09-18T14:00:10.247 INFO:tasks.rados.rados.0.smithi084.stdout:235: finishing write tid 2 to smithi08434663-35 2024-09-18T14:00:10.247 INFO:tasks.rados.rados.0.smithi084.stdout:233: done (2 left) 2024-09-18T14:00:10.247 INFO:tasks.rados.rados.0.smithi084.stdout:237: copy_from oid 39 from oid 8 current snap is 21 2024-09-18T14:00:10.247 INFO:tasks.rados.rados.0.smithi084.stdout:238: copy_from oid 34 from oid 15 current snap is 21 2024-09-18T14:00:10.248 INFO:tasks.rados.rados.0.smithi084.stdout:239: rollback oid 26 current snap is 21 2024-09-18T14:00:10.248 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 26 to 20 2024-09-18T14:00:10.248 INFO:tasks.rados.rados.0.smithi084.stdout:240: snap_remove snap 19 2024-09-18T14:00:11.222 INFO:tasks.rados.rados.0.smithi084.stdout:239: finishing rollback tid 1 to smithi08434663-26 2024-09-18T14:00:11.222 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 0 (ObjNum 129 snap 0 seq_num 2495442992) dirty dne 2024-09-18T14:00:11.222 INFO:tasks.rados.rados.0.smithi084.stdout:238: finishing copy_from to smithi08434663-34 2024-09-18T14:00:11.222 INFO:tasks.rados.rados.0.smithi084.stdout:238: got expected ENOENT (src dne) 2024-09-18T14:00:11.222 INFO:tasks.rados.rados.0.smithi084.stdout:238: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:00:11.222 INFO:tasks.rados.rados.0.smithi084.stdout:235: finishing write tid 3 to smithi08434663-35 2024-09-18T14:00:11.222 INFO:tasks.rados.rados.0.smithi084.stdout:235: finishing write tid 4 to smithi08434663-35 2024-09-18T14:00:11.222 INFO:tasks.rados.rados.0.smithi084.stdout:235: finishing write tid 5 to smithi08434663-35 2024-09-18T14:00:11.222 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 68 (ObjNum 110 snap 21 seq_num 110) dirty exists 2024-09-18T14:00:11.222 INFO:tasks.rados.rados.0.smithi084.stdout:235: left oid 35 (ObjNum 110 snap 21 seq_num 110) 2024-09-18T14:00:11.223 INFO:tasks.rados.rados.0.smithi084.stdout:235: done (5 left) 2024-09-18T14:00:11.223 INFO:tasks.rados.rados.0.smithi084.stdout:238: done (4 left) 2024-09-18T14:00:11.223 INFO:tasks.rados.rados.0.smithi084.stdout:239: done (3 left) 2024-09-18T14:00:11.223 INFO:tasks.rados.rados.0.smithi084.stdout:240: done (2 left) 2024-09-18T14:00:11.223 INFO:tasks.rados.rados.0.smithi084.stdout:241: write oid 49 current snap is 21 2024-09-18T14:00:11.223 INFO:tasks.rados.rados.0.smithi084.stdout:241: seq_num 111 ranges {665577=781581,2062560=563688,3182305=643664} 2024-09-18T14:00:11.230 INFO:tasks.rados.rados.0.smithi084.stdout:241: writing smithi08434663-49 from 665577 to 1447158 tid 1 2024-09-18T14:00:11.237 INFO:tasks.rados.rados.0.smithi084.stdout:241: writing smithi08434663-49 from 2062560 to 2626248 tid 2 2024-09-18T14:00:11.244 INFO:tasks.rados.rados.0.smithi084.stdout:241: writing smithi08434663-49 from 3182305 to 3825969 tid 3 2024-09-18T14:00:11.244 INFO:tasks.rados.rados.0.smithi084.stdout:242: snap_remove snap 20 2024-09-18T14:00:12.236 INFO:tasks.rados.rados.0.smithi084.stdout:236: finishing rollback tid 0 to smithi08434663-33 2024-09-18T14:00:12.237 INFO:tasks.rados.rados.0.smithi084.stdout:236: finishing rollback tid 1 to smithi08434663-33 2024-09-18T14:00:12.237 INFO:tasks.rados.rados.0.smithi084.stdout:236: finishing rollback tid 2 to smithi08434663-33 2024-09-18T14:00:12.237 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 54 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T14:00:12.237 INFO:tasks.rados.rados.0.smithi084.stdout:237: finishing copy_from racing read to smithi08434663-39 2024-09-18T14:00:12.237 INFO:tasks.rados.rados.0.smithi084.stdout:237: finishing copy_from to smithi08434663-39 2024-09-18T14:00:12.237 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 37 (ObjNum 108 snap 19 seq_num 108) dirty exists 2024-09-18T14:00:12.237 INFO:tasks.rados.rados.0.smithi084.stdout:241: finishing write tid 1 to smithi08434663-49 2024-09-18T14:00:12.237 INFO:tasks.rados.rados.0.smithi084.stdout:241: finishing write tid 2 to smithi08434663-49 2024-09-18T14:00:12.237 INFO:tasks.rados.rados.0.smithi084.stdout:241: finishing write tid 3 to smithi08434663-49 2024-09-18T14:00:12.237 INFO:tasks.rados.rados.0.smithi084.stdout:241: finishing write tid 4 to smithi08434663-49 2024-09-18T14:00:12.237 INFO:tasks.rados.rados.0.smithi084.stdout:241: finishing write tid 5 to smithi08434663-49 2024-09-18T14:00:12.238 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 58 (ObjNum 111 snap 21 seq_num 111) dirty exists 2024-09-18T14:00:12.238 INFO:tasks.rados.rados.0.smithi084.stdout:241: left oid 49 (ObjNum 111 snap 21 seq_num 111) 2024-09-18T14:00:12.238 INFO:tasks.rados.rados.0.smithi084.stdout:236: done (3 left) 2024-09-18T14:00:12.238 INFO:tasks.rados.rados.0.smithi084.stdout:237: done (2 left) 2024-09-18T14:00:12.238 INFO:tasks.rados.rados.0.smithi084.stdout:241: done (1 left) 2024-09-18T14:00:12.238 INFO:tasks.rados.rados.0.smithi084.stdout:242: done (0 left) 2024-09-18T14:00:12.238 INFO:tasks.rados.rados.0.smithi084.stdout:243: delete oid 18 current snap is 21 2024-09-18T14:00:12.242 INFO:tasks.rados.rados.0.smithi084.stdout:243: done (0 left) 2024-09-18T14:00:12.242 INFO:tasks.rados.rados.0.smithi084.stdout:244: delete oid 46 current snap is 21 2024-09-18T14:00:12.244 INFO:tasks.rados.rados.0.smithi084.stdout:244: done (0 left) 2024-09-18T14:00:12.245 INFO:tasks.rados.rados.0.smithi084.stdout:245: read oid 29 snap -1 2024-09-18T14:00:12.245 INFO:tasks.rados.rados.0.smithi084.stdout:245: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-18T14:00:12.245 INFO:tasks.rados.rados.0.smithi084.stdout:246: snap_create 2024-09-18T14:00:12.252 INFO:tasks.rados.rados.0.smithi084.stdout:245: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-18T14:00:13.235 INFO:tasks.rados.rados.0.smithi084.stdout:245: done (1 left) 2024-09-18T14:00:13.235 INFO:tasks.rados.rados.0.smithi084.stdout:246: done (0 left) 2024-09-18T14:00:13.236 INFO:tasks.rados.rados.0.smithi084.stdout:247: snap_remove snap 21 2024-09-18T14:00:14.246 INFO:tasks.rados.rados.0.smithi084.stdout:247: done (0 left) 2024-09-18T14:00:14.246 INFO:tasks.rados.rados.0.smithi084.stdout:248: delete oid 7 current snap is 22 2024-09-18T14:00:14.250 INFO:tasks.rados.rados.0.smithi084.stdout:248: done (0 left) 2024-09-18T14:00:14.250 INFO:tasks.rados.rados.0.smithi084.stdout:249: read oid 25 snap 18 2024-09-18T14:00:14.250 INFO:tasks.rados.rados.0.smithi084.stdout:249: expect (ObjNum 100 snap 16 seq_num 100) 2024-09-18T14:00:14.250 INFO:tasks.rados.rados.0.smithi084.stdout:250: write oid 20 current snap is 22 2024-09-18T14:00:14.250 INFO:tasks.rados.rados.0.smithi084.stdout:250: seq_num 115 ranges {732805=798930,2296488=571026,3291997=405105} 2024-09-18T14:00:14.259 INFO:tasks.rados.rados.0.smithi084.stdout:250: writing smithi08434663-20 from 732805 to 1531735 tid 1 2024-09-18T14:00:14.266 INFO:tasks.rados.rados.0.smithi084.stdout:250: writing smithi08434663-20 from 2296488 to 2867514 tid 2 2024-09-18T14:00:14.271 INFO:tasks.rados.rados.0.smithi084.stdout:250: writing smithi08434663-20 from 3291997 to 3697102 tid 3 2024-09-18T14:00:14.271 INFO:tasks.rados.rados.0.smithi084.stdout:251: write oid 26 current snap is 22 2024-09-18T14:00:14.271 INFO:tasks.rados.rados.0.smithi084.stdout:251: seq_num 116 ranges {749612=474179,1954970=431037,2919420=616377} 2024-09-18T14:00:14.280 INFO:tasks.rados.rados.0.smithi084.stdout:251: writing smithi08434663-26 from 749612 to 1223791 tid 1 2024-09-18T14:00:14.284 INFO:tasks.rados.rados.0.smithi084.stdout:251: writing smithi08434663-26 from 1954970 to 2386007 tid 2 2024-09-18T14:00:14.291 INFO:tasks.rados.rados.0.smithi084.stdout:251: writing smithi08434663-26 from 2919420 to 3535797 tid 3 2024-09-18T14:00:14.292 INFO:tasks.rados.rados.0.smithi084.stdout:252: write (excl) oid 33 current snap is 22 2024-09-18T14:00:14.292 INFO:tasks.rados.rados.0.smithi084.stdout:252: seq_num 117 ranges {766419=549428,2013452=574695,3346843=427649} 2024-09-18T14:00:14.297 INFO:tasks.rados.rados.0.smithi084.stdout:252: writing smithi08434663-33 from 766419 to 1315847 tid 1 2024-09-18T14:00:14.304 INFO:tasks.rados.rados.0.smithi084.stdout:252: writing smithi08434663-33 from 2013452 to 2588147 tid 2 2024-09-18T14:00:14.311 INFO:tasks.rados.rados.0.smithi084.stdout:252: writing smithi08434663-33 from 3346843 to 3774492 tid 3 2024-09-18T14:00:14.311 INFO:tasks.rados.rados.0.smithi084.stdout:253: snap_remove snap 16 2024-09-18T14:00:15.246 INFO:tasks.rados.rados.0.smithi084.stdout:249: expect (ObjNum 100 snap 16 seq_num 100) 2024-09-18T14:00:15.434 INFO:tasks.rados.rados.0.smithi084.stdout:250: finishing write tid 1 to smithi08434663-20 2024-09-18T14:00:15.434 INFO:tasks.rados.rados.0.smithi084.stdout:250: finishing write tid 2 to smithi08434663-20 2024-09-18T14:00:15.434 INFO:tasks.rados.rados.0.smithi084.stdout:250: finishing write tid 3 to smithi08434663-20 2024-09-18T14:00:15.434 INFO:tasks.rados.rados.0.smithi084.stdout:249: done (4 left) 2024-09-18T14:00:15.434 INFO:tasks.rados.rados.0.smithi084.stdout:253: done (3 left) 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:254: delete oid 32 current snap is 22 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:250: finishing write tid 4 to smithi08434663-20 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:250: finishing write tid 5 to smithi08434663-20 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 43 (ObjNum 115 snap 22 seq_num 115) dirty exists 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:250: left oid 20 (ObjNum 115 snap 22 seq_num 115) 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:251: finishing write tid 1 to smithi08434663-26 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:251: finishing write tid 2 to smithi08434663-26 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:251: finishing write tid 3 to smithi08434663-26 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:251: finishing write tid 4 to smithi08434663-26 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:251: finishing write tid 5 to smithi08434663-26 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 86 (ObjNum 116 snap 22 seq_num 116) dirty exists 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:251: left oid 26 (ObjNum 116 snap 22 seq_num 116) 2024-09-18T14:00:15.435 INFO:tasks.rados.rados.0.smithi084.stdout:252: finishing write tid 1 to smithi08434663-33 2024-09-18T14:00:15.436 INFO:tasks.rados.rados.0.smithi084.stdout:252: finishing write tid 2 to smithi08434663-33 2024-09-18T14:00:15.436 INFO:tasks.rados.rados.0.smithi084.stdout:252: finishing write tid 3 to smithi08434663-33 2024-09-18T14:00:15.436 INFO:tasks.rados.rados.0.smithi084.stdout:252: finishing write tid 4 to smithi08434663-33 2024-09-18T14:00:15.436 INFO:tasks.rados.rados.0.smithi084.stdout:252: finishing write tid 5 to smithi08434663-33 2024-09-18T14:00:15.436 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 64 (ObjNum 117 snap 22 seq_num 117) dirty exists 2024-09-18T14:00:15.436 INFO:tasks.rados.rados.0.smithi084.stdout:252: left oid 33 (ObjNum 117 snap 22 seq_num 117) 2024-09-18T14:00:15.437 INFO:tasks.rados.rados.0.smithi084.stdout:250: done (3 left) 2024-09-18T14:00:15.437 INFO:tasks.rados.rados.0.smithi084.stdout:251: done (2 left) 2024-09-18T14:00:15.437 INFO:tasks.rados.rados.0.smithi084.stdout:252: done (1 left) 2024-09-18T14:00:15.437 INFO:tasks.rados.rados.0.smithi084.stdout:254: done (0 left) 2024-09-18T14:00:15.437 INFO:tasks.rados.rados.0.smithi084.stdout:255: delete oid 4 current snap is 22 2024-09-18T14:00:15.438 INFO:tasks.rados.rados.0.smithi084.stdout:255: done (0 left) 2024-09-18T14:00:15.438 INFO:tasks.rados.rados.0.smithi084.stdout:256: write (excl) oid 10 current snap is 22 2024-09-18T14:00:15.438 INFO:tasks.rados.rados.0.smithi084.stdout:256: seq_num 120 ranges {416840=775175,1672545=344295} 2024-09-18T14:00:15.444 INFO:tasks.rados.rados.0.smithi084.stdout:256: writing smithi08434663-10 from 416840 to 1192015 tid 1 2024-09-18T14:00:15.449 INFO:tasks.rados.rados.0.smithi084.stdout:256: writing smithi08434663-10 from 1672545 to 2016840 tid 2 2024-09-18T14:00:15.449 INFO:tasks.rados.rados.0.smithi084.stdout:257: snap_remove snap 18 2024-09-18T14:00:16.259 INFO:tasks.rados.rados.0.smithi084.stdout:257: done (1 left) 2024-09-18T14:00:16.259 INFO:tasks.rados.rados.0.smithi084.stdout:258: copy_from oid 22 from oid 42 current snap is 22 2024-09-18T14:00:16.259 INFO:tasks.rados.rados.0.smithi084.stdout:256: finishing write tid 1 to smithi08434663-10 2024-09-18T14:00:16.259 INFO:tasks.rados.rados.0.smithi084.stdout:256: finishing write tid 2 to smithi08434663-10 2024-09-18T14:00:16.259 INFO:tasks.rados.rados.0.smithi084.stdout:256: finishing write tid 3 to smithi08434663-10 2024-09-18T14:00:16.259 INFO:tasks.rados.rados.0.smithi084.stdout:256: finishing write tid 4 to smithi08434663-10 2024-09-18T14:00:16.260 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 90 (ObjNum 120 snap 22 seq_num 120) dirty exists 2024-09-18T14:00:16.260 INFO:tasks.rados.rados.0.smithi084.stdout:256: left oid 10 (ObjNum 120 snap 22 seq_num 120) 2024-09-18T14:00:16.260 INFO:tasks.rados.rados.0.smithi084.stdout:256: done (1 left) 2024-09-18T14:00:16.260 INFO:tasks.rados.rados.0.smithi084.stdout:259: write oid 11 current snap is 22 2024-09-18T14:00:16.260 INFO:tasks.rados.rados.0.smithi084.stdout:259: seq_num 121 ranges {433647=450424,1331027=582033} 2024-09-18T14:00:16.264 INFO:tasks.rados.rados.0.smithi084.stdout:259: writing smithi08434663-11 from 433647 to 884071 tid 1 2024-09-18T14:00:16.271 INFO:tasks.rados.rados.0.smithi084.stdout:259: writing smithi08434663-11 from 1331027 to 1913060 tid 2 2024-09-18T14:00:16.271 INFO:tasks.rados.rados.0.smithi084.stdout:260: rollback oid 21 current snap is 22 2024-09-18T14:00:16.271 INFO:tasks.rados.rados.0.smithi084.stdout:260: done (2 left) 2024-09-18T14:00:16.271 INFO:tasks.rados.rados.0.smithi084.stdout:261: write oid 13 current snap is 22 2024-09-18T14:00:16.271 INFO:tasks.rados.rados.0.smithi084.stdout:261: seq_num 122 ranges {450454=642026,1505862=544592} 2024-09-18T14:00:16.277 INFO:tasks.rados.rados.0.smithi084.stdout:261: writing smithi08434663-13 from 450454 to 1092480 tid 1 2024-09-18T14:00:16.283 INFO:tasks.rados.rados.0.smithi084.stdout:261: writing smithi08434663-13 from 1505862 to 2050454 tid 2 2024-09-18T14:00:16.283 INFO:tasks.rados.rados.0.smithi084.stdout:262: snap_create 2024-09-18T14:00:16.283 INFO:tasks.rados.rados.0.smithi084.stdout:258: finishing copy_from racing read to smithi08434663-22 2024-09-18T14:00:16.283 INFO:tasks.rados.rados.0.smithi084.stdout:259: finishing write tid 1 to smithi08434663-11 2024-09-18T14:00:16.283 INFO:tasks.rados.rados.0.smithi084.stdout:261: finishing write tid 1 to smithi08434663-13 2024-09-18T14:00:16.284 INFO:tasks.rados.rados.0.smithi084.stdout:259: finishing write tid 2 to smithi08434663-11 2024-09-18T14:00:16.284 INFO:tasks.rados.rados.0.smithi084.stdout:259: finishing write tid 3 to smithi08434663-11 2024-09-18T14:00:16.284 INFO:tasks.rados.rados.0.smithi084.stdout:259: finishing write tid 4 to smithi08434663-11 2024-09-18T14:00:16.284 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 61 (ObjNum 121 snap 22 seq_num 121) dirty exists 2024-09-18T14:00:16.284 INFO:tasks.rados.rados.0.smithi084.stdout:259: left oid 11 (ObjNum 121 snap 22 seq_num 121) 2024-09-18T14:00:16.284 INFO:tasks.rados.rados.0.smithi084.stdout:258: finishing copy_from to smithi08434663-22 2024-09-18T14:00:16.284 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 25 (ObjNum 58 snap 2 seq_num 58) dirty exists 2024-09-18T14:00:16.286 INFO:tasks.rados.rados.0.smithi084.stdout:261: finishing write tid 2 to smithi08434663-13 2024-09-18T14:00:16.286 INFO:tasks.rados.rados.0.smithi084.stdout:261: finishing write tid 3 to smithi08434663-13 2024-09-18T14:00:16.286 INFO:tasks.rados.rados.0.smithi084.stdout:261: finishing write tid 4 to smithi08434663-13 2024-09-18T14:00:16.286 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 24 (ObjNum 122 snap 22 seq_num 122) dirty exists 2024-09-18T14:00:16.286 INFO:tasks.rados.rados.0.smithi084.stdout:261: left oid 13 (ObjNum 122 snap 22 seq_num 122) 2024-09-18T14:00:17.144 INFO:tasks.rados.rados.0.smithi084.stdout:258: done (3 left) 2024-09-18T14:00:17.144 INFO:tasks.rados.rados.0.smithi084.stdout:259: done (2 left) 2024-09-18T14:00:17.144 INFO:tasks.rados.rados.0.smithi084.stdout:261: done (1 left) 2024-09-18T14:00:17.144 INFO:tasks.rados.rados.0.smithi084.stdout:262: done (0 left) 2024-09-18T14:00:17.144 INFO:tasks.rados.rados.0.smithi084.stdout:263: snap_create 2024-09-18T14:00:18.150 INFO:tasks.rados.rados.0.smithi084.stdout:263: done (0 left) 2024-09-18T14:00:18.150 INFO:tasks.rados.rados.0.smithi084.stdout:264: read oid 39 snap 22 2024-09-18T14:00:18.150 INFO:tasks.rados.rados.0.smithi084.stdout:264: expect (ObjNum 108 snap 19 seq_num 108) 2024-09-18T14:00:18.150 INFO:tasks.rados.rados.0.smithi084.stdout:265: snap_create 2024-09-18T14:00:18.161 INFO:tasks.rados.rados.0.smithi084.stdout:264: expect (ObjNum 108 snap 19 seq_num 108) 2024-09-18T14:00:19.151 INFO:tasks.rados.rados.0.smithi084.stdout:264: done (1 left) 2024-09-18T14:00:19.151 INFO:tasks.rados.rados.0.smithi084.stdout:265: done (0 left) 2024-09-18T14:00:19.152 INFO:tasks.rados.rados.0.smithi084.stdout:266: delete oid 15 current snap is 25 2024-09-18T14:00:19.153 INFO:tasks.rados.rados.0.smithi084.stdout:266: done (0 left) 2024-09-18T14:00:19.154 INFO:tasks.rados.rados.0.smithi084.stdout:267: read oid 47 snap 22 2024-09-18T14:00:19.154 INFO:tasks.rados.rados.0.smithi084.stdout:267: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T14:00:19.154 INFO:tasks.rados.rados.0.smithi084.stdout:268: write oid 15 current snap is 25 2024-09-18T14:00:19.154 INFO:tasks.rados.rados.0.smithi084.stdout:268: seq_num 124 ranges {484068=792524,1906473=177595} 2024-09-18T14:00:19.161 INFO:tasks.rados.rados.0.smithi084.stdout:268: writing smithi08434663-15 from 484068 to 1276592 tid 1 2024-09-18T14:00:19.166 INFO:tasks.rados.rados.0.smithi084.stdout:268: writing smithi08434663-15 from 1906473 to 2084068 tid 2 2024-09-18T14:00:19.166 INFO:tasks.rados.rados.0.smithi084.stdout:269: copy_from oid 23 from oid 49 current snap is 25 2024-09-18T14:00:19.166 INFO:tasks.rados.rados.0.smithi084.stdout:267: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T14:00:19.256 INFO:tasks.rados.rados.0.smithi084.stdout:268: finishing write tid 1 to smithi08434663-15 2024-09-18T14:00:19.256 INFO:tasks.rados.rados.0.smithi084.stdout:268: finishing write tid 2 to smithi08434663-15 2024-09-18T14:00:19.256 INFO:tasks.rados.rados.0.smithi084.stdout:268: finishing write tid 3 to smithi08434663-15 2024-09-18T14:00:19.256 INFO:tasks.rados.rados.0.smithi084.stdout:268: finishing write tid 4 to smithi08434663-15 2024-09-18T14:00:19.256 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 86 (ObjNum 124 snap 25 seq_num 124) dirty exists 2024-09-18T14:00:19.256 INFO:tasks.rados.rados.0.smithi084.stdout:268: left oid 15 (ObjNum 124 snap 25 seq_num 124) 2024-09-18T14:00:19.256 INFO:tasks.rados.rados.0.smithi084.stdout:269: finishing copy_from racing read to smithi08434663-23 2024-09-18T14:00:19.256 INFO:tasks.rados.rados.0.smithi084.stdout:269: finishing copy_from to smithi08434663-23 2024-09-18T14:00:19.257 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 58 (ObjNum 111 snap 21 seq_num 111) dirty exists 2024-09-18T14:00:19.257 INFO:tasks.rados.rados.0.smithi084.stdout:267: done (2 left) 2024-09-18T14:00:19.257 INFO:tasks.rados.rados.0.smithi084.stdout:268: done (1 left) 2024-09-18T14:00:19.257 INFO:tasks.rados.rados.0.smithi084.stdout:269: done (0 left) 2024-09-18T14:00:19.257 INFO:tasks.rados.rados.0.smithi084.stdout:270: copy_from oid 10 from oid 38 current snap is 25 2024-09-18T14:00:19.257 INFO:tasks.rados.rados.0.smithi084.stdout:271: write (excl) oid 13 current snap is 25 2024-09-18T14:00:19.257 INFO:tasks.rados.rados.0.smithi084.stdout:271: seq_num 125 ranges {500875=467773,1564955=535920} 2024-09-18T14:00:19.261 INFO:tasks.rados.rados.0.smithi084.stdout:271: writing smithi08434663-13 from 500875 to 968648 tid 1 2024-09-18T14:00:19.268 INFO:tasks.rados.rados.0.smithi084.stdout:271: writing smithi08434663-13 from 1564955 to 2100875 tid 2 2024-09-18T14:00:19.268 INFO:tasks.rados.rados.0.smithi084.stdout:271: finishing write tid 1 to smithi08434663-13 2024-09-18T14:00:19.268 INFO:tasks.rados.rados.0.smithi084.stdout:270: finishing copy_from racing read to smithi08434663-10 2024-09-18T14:00:19.268 INFO:tasks.rados.rados.0.smithi084.stdout:272: delete oid 43 current snap is 25 2024-09-18T14:00:19.270 INFO:tasks.rados.rados.0.smithi084.stdout:270: finishing copy_from to smithi08434663-10 2024-09-18T14:00:19.270 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 18 (ObjNum 72 snap 6 seq_num 72) dirty exists 2024-09-18T14:00:19.271 INFO:tasks.rados.rados.0.smithi084.stdout:270: done (2 left) 2024-09-18T14:00:19.272 INFO:tasks.rados.rados.0.smithi084.stdout:272: done (1 left) 2024-09-18T14:00:19.272 INFO:tasks.rados.rados.0.smithi084.stdout:273: write oid 22 current snap is 25 2024-09-18T14:00:19.272 INFO:tasks.rados.rados.0.smithi084.stdout:273: seq_num 127 ranges {534489=618271,1565566=568923} 2024-09-18T14:00:19.276 INFO:tasks.rados.rados.0.smithi084.stdout:273: writing smithi08434663-22 from 534489 to 1152760 tid 1 2024-09-18T14:00:19.282 INFO:tasks.rados.rados.0.smithi084.stdout:273: writing smithi08434663-22 from 1565566 to 2134489 tid 2 2024-09-18T14:00:19.282 INFO:tasks.rados.rados.0.smithi084.stdout:274: write oid 20 current snap is 25 2024-09-18T14:00:19.283 INFO:tasks.rados.rados.0.smithi084.stdout:274: seq_num 128 ranges {551296=693520,2024048=127248} 2024-09-18T14:00:19.289 INFO:tasks.rados.rados.0.smithi084.stdout:274: writing smithi08434663-20 from 551296 to 1244816 tid 1 2024-09-18T14:00:19.294 INFO:tasks.rados.rados.0.smithi084.stdout:274: writing smithi08434663-20 from 2024048 to 2151296 tid 2 2024-09-18T14:00:19.294 INFO:tasks.rados.rados.0.smithi084.stdout:275: read oid 31 snap -1 2024-09-18T14:00:19.294 INFO:tasks.rados.rados.0.smithi084.stdout:275: expect (ObjNum 66 snap 6 seq_num 66) 2024-09-18T14:00:19.294 INFO:tasks.rados.rados.0.smithi084.stdout:276: read oid 35 snap -1 2024-09-18T14:00:19.294 INFO:tasks.rados.rados.0.smithi084.stdout:276: expect (ObjNum 110 snap 21 seq_num 110) 2024-09-18T14:00:19.295 INFO:tasks.rados.rados.0.smithi084.stdout:277: snap_remove snap 24 2024-09-18T14:00:20.163 INFO:tasks.rados.rados.0.smithi084.stdout:277: done (5 left) 2024-09-18T14:00:20.163 INFO:tasks.rados.rados.0.smithi084.stdout:278: snap_create 2024-09-18T14:00:20.163 INFO:tasks.rados.rados.0.smithi084.stdout:271: finishing write tid 2 to smithi08434663-13 2024-09-18T14:00:20.163 INFO:tasks.rados.rados.0.smithi084.stdout:271: finishing write tid 3 to smithi08434663-13 2024-09-18T14:00:20.163 INFO:tasks.rados.rados.0.smithi084.stdout:271: finishing write tid 4 to smithi08434663-13 2024-09-18T14:00:20.163 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 28 (ObjNum 125 snap 25 seq_num 125) dirty exists 2024-09-18T14:00:20.163 INFO:tasks.rados.rados.0.smithi084.stdout:271: left oid 13 (ObjNum 125 snap 25 seq_num 125) 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:273: finishing write tid 1 to smithi08434663-22 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:273: finishing write tid 2 to smithi08434663-22 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:273: finishing write tid 3 to smithi08434663-22 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:273: finishing write tid 4 to smithi08434663-22 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 41 (ObjNum 127 snap 25 seq_num 127) dirty exists 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:273: left oid 22 (ObjNum 127 snap 25 seq_num 127) 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:274: finishing write tid 1 to smithi08434663-20 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:274: finishing write tid 2 to smithi08434663-20 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:274: finishing write tid 3 to smithi08434663-20 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:274: finishing write tid 4 to smithi08434663-20 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 51 (ObjNum 128 snap 25 seq_num 128) dirty exists 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:274: left oid 20 (ObjNum 128 snap 25 seq_num 128) 2024-09-18T14:00:20.164 INFO:tasks.rados.rados.0.smithi084.stdout:275: expect (ObjNum 66 snap 6 seq_num 66) 2024-09-18T14:00:20.295 INFO:tasks.rados.rados.0.smithi084.stdout:276: expect (ObjNum 110 snap 21 seq_num 110) 2024-09-18T14:00:21.164 INFO:tasks.rados.rados.0.smithi084.stdout:271: done (5 left) 2024-09-18T14:00:21.165 INFO:tasks.rados.rados.0.smithi084.stdout:273: done (4 left) 2024-09-18T14:00:21.165 INFO:tasks.rados.rados.0.smithi084.stdout:274: done (3 left) 2024-09-18T14:00:21.165 INFO:tasks.rados.rados.0.smithi084.stdout:275: done (2 left) 2024-09-18T14:00:21.165 INFO:tasks.rados.rados.0.smithi084.stdout:276: done (1 left) 2024-09-18T14:00:21.165 INFO:tasks.rados.rados.0.smithi084.stdout:278: done (0 left) 2024-09-18T14:00:21.165 INFO:tasks.rados.rados.0.smithi084.stdout:279: write oid 35 current snap is 26 2024-09-18T14:00:21.165 INFO:tasks.rados.rados.0.smithi084.stdout:279: seq_num 129 ranges {568103=768769,2082530=85573} 2024-09-18T14:00:21.172 INFO:tasks.rados.rados.0.smithi084.stdout:279: writing smithi08434663-35 from 568103 to 1336872 tid 1 2024-09-18T14:00:21.177 INFO:tasks.rados.rados.0.smithi084.stdout:279: writing smithi08434663-35 from 2082530 to 2168103 tid 2 2024-09-18T14:00:21.177 INFO:tasks.rados.rados.0.smithi084.stdout:280: read oid 37 snap 22 2024-09-18T14:00:21.178 INFO:tasks.rados.rados.0.smithi084.stdout:280: expect (ObjNum 91 snap 9 seq_num 91) 2024-09-18T14:00:21.178 INFO:tasks.rados.rados.0.smithi084.stdout:281: read oid 11 snap -1 2024-09-18T14:00:21.178 INFO:tasks.rados.rados.0.smithi084.stdout:281: expect (ObjNum 121 snap 22 seq_num 121) 2024-09-18T14:00:21.178 INFO:tasks.rados.rados.0.smithi084.stdout:282: write (excl) oid 31 current snap is 26 2024-09-18T14:00:21.178 INFO:tasks.rados.rados.0.smithi084.stdout:282: seq_num 130 ranges {584910=560371,1857365=327545} 2024-09-18T14:00:21.184 INFO:tasks.rados.rados.0.smithi084.stdout:282: writing smithi08434663-31 from 584910 to 1145281 tid 1 2024-09-18T14:00:21.191 INFO:tasks.rados.rados.0.smithi084.stdout:282: writing smithi08434663-31 from 1857365 to 2184910 tid 2 2024-09-18T14:00:21.191 INFO:tasks.rados.rados.0.smithi084.stdout:283: read oid 43 snap -1 2024-09-18T14:00:21.191 INFO:tasks.rados.rados.0.smithi084.stdout:283: expect deleted 2024-09-18T14:00:21.191 INFO:tasks.rados.rados.0.smithi084.stdout:284: write (excl) oid 16 current snap is 26 2024-09-18T14:00:21.191 INFO:tasks.rados.rados.0.smithi084.stdout:279: finishing write tid 1 to smithi08434663-35 2024-09-18T14:00:21.191 INFO:tasks.rados.rados.0.smithi084.stdout:284: seq_num 131 ranges {601717=635620,1799494=402223} 2024-09-18T14:00:21.197 INFO:tasks.rados.rados.0.smithi084.stdout:284: writing smithi08434663-16 from 601717 to 1237337 tid 1 2024-09-18T14:00:21.203 INFO:tasks.rados.rados.0.smithi084.stdout:284: writing smithi08434663-16 from 1799494 to 2201717 tid 2 2024-09-18T14:00:21.203 INFO:tasks.rados.rados.0.smithi084.stdout:285: snap_remove snap 25 2024-09-18T14:00:22.178 INFO:tasks.rados.rados.0.smithi084.stdout:285: done (6 left) 2024-09-18T14:00:22.178 INFO:tasks.rados.rados.0.smithi084.stdout:286: snap_create 2024-09-18T14:00:22.178 INFO:tasks.rados.rados.0.smithi084.stdout:279: finishing write tid 2 to smithi08434663-35 2024-09-18T14:00:22.178 INFO:tasks.rados.rados.0.smithi084.stdout:279: finishing write tid 3 to smithi08434663-35 2024-09-18T14:00:22.178 INFO:tasks.rados.rados.0.smithi084.stdout:279: finishing write tid 4 to smithi08434663-35 2024-09-18T14:00:22.178 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 90 (ObjNum 129 snap 26 seq_num 129) dirty exists 2024-09-18T14:00:22.178 INFO:tasks.rados.rados.0.smithi084.stdout:279: left oid 35 (ObjNum 129 snap 26 seq_num 129) 2024-09-18T14:00:22.178 INFO:tasks.rados.rados.0.smithi084.stdout:281: expect (ObjNum 121 snap 22 seq_num 121) 2024-09-18T14:00:22.242 INFO:tasks.rados.rados.0.smithi084.stdout:280: expect (ObjNum 91 snap 9 seq_num 91) 2024-09-18T14:00:22.441 INFO:tasks.rados.rados.0.smithi084.stdout:282: finishing write tid 1 to smithi08434663-31 2024-09-18T14:00:22.442 INFO:tasks.rados.rados.0.smithi084.stdout:282: finishing write tid 2 to smithi08434663-31 2024-09-18T14:00:22.442 INFO:tasks.rados.rados.0.smithi084.stdout:282: finishing write tid 3 to smithi08434663-31 2024-09-18T14:00:22.442 INFO:tasks.rados.rados.0.smithi084.stdout:282: finishing write tid 4 to smithi08434663-31 2024-09-18T14:00:22.442 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 43 (ObjNum 130 snap 26 seq_num 130) dirty exists 2024-09-18T14:00:22.442 INFO:tasks.rados.rados.0.smithi084.stdout:282: left oid 31 (ObjNum 130 snap 26 seq_num 130) 2024-09-18T14:00:22.442 INFO:tasks.rados.rados.0.smithi084.stdout:284: finishing write tid 1 to smithi08434663-16 2024-09-18T14:00:22.442 INFO:tasks.rados.rados.0.smithi084.stdout:284: finishing write tid 2 to smithi08434663-16 2024-09-18T14:00:22.442 INFO:tasks.rados.rados.0.smithi084.stdout:284: finishing write tid 3 to smithi08434663-16 2024-09-18T14:00:22.442 INFO:tasks.rados.rados.0.smithi084.stdout:284: finishing write tid 4 to smithi08434663-16 2024-09-18T14:00:22.442 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 93 (ObjNum 131 snap 26 seq_num 131) dirty exists 2024-09-18T14:00:22.443 INFO:tasks.rados.rados.0.smithi084.stdout:284: left oid 16 (ObjNum 131 snap 26 seq_num 131) 2024-09-18T14:00:23.177 INFO:tasks.rados.rados.0.smithi084.stdout:279: done (6 left) 2024-09-18T14:00:23.177 INFO:tasks.rados.rados.0.smithi084.stdout:280: done (5 left) 2024-09-18T14:00:23.178 INFO:tasks.rados.rados.0.smithi084.stdout:281: done (4 left) 2024-09-18T14:00:23.178 INFO:tasks.rados.rados.0.smithi084.stdout:282: done (3 left) 2024-09-18T14:00:23.178 INFO:tasks.rados.rados.0.smithi084.stdout:283: done (2 left) 2024-09-18T14:00:23.178 INFO:tasks.rados.rados.0.smithi084.stdout:284: done (1 left) 2024-09-18T14:00:23.178 INFO:tasks.rados.rados.0.smithi084.stdout:286: done (0 left) 2024-09-18T14:00:23.178 INFO:tasks.rados.rados.0.smithi084.stdout:287: read oid 24 snap -1 2024-09-18T14:00:23.178 INFO:tasks.rados.rados.0.smithi084.stdout:287: expect deleted 2024-09-18T14:00:23.178 INFO:tasks.rados.rados.0.smithi084.stdout:288: delete oid 1 current snap is 27 2024-09-18T14:00:23.185 INFO:tasks.rados.rados.0.smithi084.stdout:287: done (1 left) 2024-09-18T14:00:23.185 INFO:tasks.rados.rados.0.smithi084.stdout:288: done (0 left) 2024-09-18T14:00:23.185 INFO:tasks.rados.rados.0.smithi084.stdout:289: delete oid 31 current snap is 27 2024-09-18T14:00:23.189 INFO:tasks.rados.rados.0.smithi084.stdout:289: done (0 left) 2024-09-18T14:00:23.189 INFO:tasks.rados.rados.0.smithi084.stdout:290: snap_remove snap 26 2024-09-18T14:00:24.187 INFO:tasks.rados.rados.0.smithi084.stdout:290: done (0 left) 2024-09-18T14:00:24.187 INFO:tasks.rados.rados.0.smithi084.stdout:291: read oid 37 snap -1 2024-09-18T14:00:24.187 INFO:tasks.rados.rados.0.smithi084.stdout:291: expect (ObjNum 91 snap 9 seq_num 91) 2024-09-18T14:00:24.187 INFO:tasks.rados.rados.0.smithi084.stdout:292: read oid 7 snap 22 2024-09-18T14:00:24.187 INFO:tasks.rados.rados.0.smithi084.stdout:292: expect deleted 2024-09-18T14:00:24.187 INFO:tasks.rados.rados.0.smithi084.stdout:293: rollback oid 49 current snap is 27 2024-09-18T14:00:24.187 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 49 to 22 2024-09-18T14:00:24.187 INFO:tasks.rados.rados.0.smithi084.stdout:294: snap_create 2024-09-18T14:00:24.200 INFO:tasks.rados.rados.0.smithi084.stdout:291: expect (ObjNum 91 snap 9 seq_num 91) 2024-09-18T14:00:24.399 INFO:tasks.rados.rados.0.smithi084.stdout:293: finishing rollback tid 0 to smithi08434663-49 2024-09-18T14:00:24.399 INFO:tasks.rados.rados.0.smithi084.stdout:293: finishing rollback tid 1 to smithi08434663-49 2024-09-18T14:00:24.399 INFO:tasks.rados.rados.0.smithi084.stdout:293: finishing rollback tid 2 to smithi08434663-49 2024-09-18T14:00:24.399 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 72 (ObjNum 111 snap 21 seq_num 111) dirty exists 2024-09-18T14:00:25.188 INFO:tasks.rados.rados.0.smithi084.stdout:291: done (3 left) 2024-09-18T14:00:25.188 INFO:tasks.rados.rados.0.smithi084.stdout:292: done (2 left) 2024-09-18T14:00:25.188 INFO:tasks.rados.rados.0.smithi084.stdout:293: done (1 left) 2024-09-18T14:00:25.188 INFO:tasks.rados.rados.0.smithi084.stdout:294: done (0 left) 2024-09-18T14:00:25.188 INFO:tasks.rados.rados.0.smithi084.stdout:295: write oid 29 current snap is 28 2024-09-18T14:00:25.188 INFO:tasks.rados.rados.0.smithi084.stdout:295: seq_num 134 ranges {652138=461367,1574940=677198} 2024-09-18T14:00:25.194 INFO:tasks.rados.rados.0.smithi084.stdout:295: writing smithi08434663-29 from 652138 to 1113505 tid 1 2024-09-18T14:00:25.200 INFO:tasks.rados.rados.0.smithi084.stdout:295: writing smithi08434663-29 from 1574940 to 2252138 tid 2 2024-09-18T14:00:25.201 INFO:tasks.rados.rados.0.smithi084.stdout:296: snap_remove snap 22 2024-09-18T14:00:26.200 INFO:tasks.rados.rados.0.smithi084.stdout:296: done (1 left) 2024-09-18T14:00:26.200 INFO:tasks.rados.rados.0.smithi084.stdout:297: write (excl) oid 36 current snap is 28 2024-09-18T14:00:26.200 INFO:tasks.rados.rados.0.smithi084.stdout:295: finishing write tid 1 to smithi08434663-29 2024-09-18T14:00:26.200 INFO:tasks.rados.rados.0.smithi084.stdout:297: seq_num 135 ranges {668945=536616,1633422=491363} 2024-09-18T14:00:26.207 INFO:tasks.rados.rados.0.smithi084.stdout:297: writing smithi08434663-36 from 668945 to 1205561 tid 1 2024-09-18T14:00:26.212 INFO:tasks.rados.rados.0.smithi084.stdout:297: writing smithi08434663-36 from 1633422 to 2124785 tid 2 2024-09-18T14:00:26.212 INFO:tasks.rados.rados.0.smithi084.stdout:298: snap_create 2024-09-18T14:00:26.212 INFO:tasks.rados.rados.0.smithi084.stdout:295: finishing write tid 2 to smithi08434663-29 2024-09-18T14:00:26.212 INFO:tasks.rados.rados.0.smithi084.stdout:295: finishing write tid 3 to smithi08434663-29 2024-09-18T14:00:26.212 INFO:tasks.rados.rados.0.smithi084.stdout:295: finishing write tid 4 to smithi08434663-29 2024-09-18T14:00:26.212 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 34 (ObjNum 134 snap 28 seq_num 134) dirty exists 2024-09-18T14:00:26.212 INFO:tasks.rados.rados.0.smithi084.stdout:295: left oid 29 (ObjNum 134 snap 28 seq_num 134) 2024-09-18T14:00:26.212 INFO:tasks.rados.rados.0.smithi084.stdout:297: finishing write tid 1 to smithi08434663-36 2024-09-18T14:00:26.215 INFO:tasks.rados.rados.0.smithi084.stdout:297: finishing write tid 2 to smithi08434663-36 2024-09-18T14:00:26.215 INFO:tasks.rados.rados.0.smithi084.stdout:297: finishing write tid 3 to smithi08434663-36 2024-09-18T14:00:26.215 INFO:tasks.rados.rados.0.smithi084.stdout:297: finishing write tid 4 to smithi08434663-36 2024-09-18T14:00:26.215 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 78 (ObjNum 135 snap 28 seq_num 135) dirty exists 2024-09-18T14:00:26.215 INFO:tasks.rados.rados.0.smithi084.stdout:297: left oid 36 (ObjNum 135 snap 28 seq_num 135) 2024-09-18T14:00:27.146 INFO:tasks.rados.rados.0.smithi084.stdout:295: done (2 left) 2024-09-18T14:00:27.147 INFO:tasks.rados.rados.0.smithi084.stdout:297: done (1 left) 2024-09-18T14:00:27.147 INFO:tasks.rados.rados.0.smithi084.stdout:298: done (0 left) 2024-09-18T14:00:27.147 INFO:tasks.rados.rados.0.smithi084.stdout:299: write (excl) oid 40 current snap is 29 2024-09-18T14:00:27.147 INFO:tasks.rados.rados.0.smithi084.stdout:299: seq_num 136 ranges {685752=611865,1975551=310201} 2024-09-18T14:00:27.154 INFO:tasks.rados.rados.0.smithi084.stdout:299: writing smithi08434663-40 from 685752 to 1297617 tid 1 2024-09-18T14:00:27.160 INFO:tasks.rados.rados.0.smithi084.stdout:299: writing smithi08434663-40 from 1975551 to 2285752 tid 2 2024-09-18T14:00:27.160 INFO:tasks.rados.rados.0.smithi084.stdout:300: snap_remove snap 27 2024-09-18T14:00:28.163 INFO:tasks.rados.rados.0.smithi084.stdout:300: done (1 left) 2024-09-18T14:00:28.163 INFO:tasks.rados.rados.0.smithi084.stdout:301: delete oid 23 current snap is 29 2024-09-18T14:00:28.163 INFO:tasks.rados.rados.0.smithi084.stdout:299: finishing write tid 1 to smithi08434663-40 2024-09-18T14:00:28.163 INFO:tasks.rados.rados.0.smithi084.stdout:299: finishing write tid 2 to smithi08434663-40 2024-09-18T14:00:28.163 INFO:tasks.rados.rados.0.smithi084.stdout:299: finishing write tid 3 to smithi08434663-40 2024-09-18T14:00:28.163 INFO:tasks.rados.rados.0.smithi084.stdout:299: finishing write tid 4 to smithi08434663-40 2024-09-18T14:00:28.163 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 41 (ObjNum 136 snap 29 seq_num 136) dirty exists 2024-09-18T14:00:28.163 INFO:tasks.rados.rados.0.smithi084.stdout:299: left oid 40 (ObjNum 136 snap 29 seq_num 136) 2024-09-18T14:00:28.170 INFO:tasks.rados.rados.0.smithi084.stdout:299: done (1 left) 2024-09-18T14:00:28.170 INFO:tasks.rados.rados.0.smithi084.stdout:301: done (0 left) 2024-09-18T14:00:28.170 INFO:tasks.rados.rados.0.smithi084.stdout:302: read oid 43 snap -1 2024-09-18T14:00:28.170 INFO:tasks.rados.rados.0.smithi084.stdout:302: expect deleted 2024-09-18T14:00:28.171 INFO:tasks.rados.rados.0.smithi084.stdout:303: write (excl) oid 1 current snap is 29 2024-09-18T14:00:28.171 INFO:tasks.rados.rados.0.smithi084.stdout:303: seq_num 138 ranges {719366=478716,1808868=510498} 2024-09-18T14:00:28.177 INFO:tasks.rados.rados.0.smithi084.stdout:303: writing smithi08434663-1 from 719366 to 1198082 tid 1 2024-09-18T14:00:28.183 INFO:tasks.rados.rados.0.smithi084.stdout:303: writing smithi08434663-1 from 1808868 to 2319366 tid 2 2024-09-18T14:00:28.184 INFO:tasks.rados.rados.0.smithi084.stdout:304: read oid 31 snap -1 2024-09-18T14:00:28.184 INFO:tasks.rados.rados.0.smithi084.stdout:304: expect deleted 2024-09-18T14:00:28.184 INFO:tasks.rados.rados.0.smithi084.stdout:305: write oid 5 current snap is 29 2024-09-18T14:00:28.184 INFO:tasks.rados.rados.0.smithi084.stdout:305: seq_num 139 ranges {736173=553965,1867350=468823} 2024-09-18T14:00:28.190 INFO:tasks.rados.rados.0.smithi084.stdout:305: writing smithi08434663-5 from 736173 to 1290138 tid 1 2024-09-18T14:00:28.196 INFO:tasks.rados.rados.0.smithi084.stdout:305: writing smithi08434663-5 from 1867350 to 2336173 tid 2 2024-09-18T14:00:28.197 INFO:tasks.rados.rados.0.smithi084.stdout:306: read oid 34 snap -1 2024-09-18T14:00:28.197 INFO:tasks.rados.rados.0.smithi084.stdout:306: expect (ObjNum 89 snap 9 seq_num 89) 2024-09-18T14:00:28.197 INFO:tasks.rados.rados.0.smithi084.stdout:307: delete oid 2 current snap is 29 2024-09-18T14:00:28.197 INFO:tasks.rados.rados.0.smithi084.stdout:303: finishing write tid 1 to smithi08434663-1 2024-09-18T14:00:28.199 INFO:tasks.rados.rados.0.smithi084.stdout:303: finishing write tid 2 to smithi08434663-1 2024-09-18T14:00:28.199 INFO:tasks.rados.rados.0.smithi084.stdout:303: finishing write tid 3 to smithi08434663-1 2024-09-18T14:00:28.200 INFO:tasks.rados.rados.0.smithi084.stdout:303: finishing write tid 4 to smithi08434663-1 2024-09-18T14:00:28.200 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 107 (ObjNum 138 snap 29 seq_num 138) dirty exists 2024-09-18T14:00:28.200 INFO:tasks.rados.rados.0.smithi084.stdout:303: left oid 1 (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:00:28.200 INFO:tasks.rados.rados.0.smithi084.stdout:305: finishing write tid 1 to smithi08434663-5 2024-09-18T14:00:28.203 INFO:tasks.rados.rados.0.smithi084.stdout:302: done (5 left) 2024-09-18T14:00:28.203 INFO:tasks.rados.rados.0.smithi084.stdout:303: done (4 left) 2024-09-18T14:00:28.203 INFO:tasks.rados.rados.0.smithi084.stdout:304: done (3 left) 2024-09-18T14:00:28.203 INFO:tasks.rados.rados.0.smithi084.stdout:307: done (2 left) 2024-09-18T14:00:28.203 INFO:tasks.rados.rados.0.smithi084.stdout:308: read oid 21 snap -1 2024-09-18T14:00:28.203 INFO:tasks.rados.rados.0.smithi084.stdout:308: expect (ObjNum 96 snap 11 seq_num 96) 2024-09-18T14:00:28.203 INFO:tasks.rados.rados.0.smithi084.stdout:309: copy_from oid 43 from oid 36 current snap is 29 2024-09-18T14:00:28.203 INFO:tasks.rados.rados.0.smithi084.stdout:310: write oid 47 current snap is 29 2024-09-18T14:00:28.204 INFO:tasks.rados.rados.0.smithi084.stdout:310: seq_num 141 ranges {769787=704463,2267961=101826} 2024-09-18T14:00:28.212 INFO:tasks.rados.rados.0.smithi084.stdout:310: writing smithi08434663-47 from 769787 to 1474250 tid 1 2024-09-18T14:00:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:310: writing smithi08434663-47 from 2267961 to 2369787 tid 2 2024-09-18T14:00:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:311: copy_from oid 12 from oid 32 current snap is 29 2024-09-18T14:00:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:305: finishing write tid 2 to smithi08434663-5 2024-09-18T14:00:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:305: finishing write tid 3 to smithi08434663-5 2024-09-18T14:00:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:305: finishing write tid 4 to smithi08434663-5 2024-09-18T14:00:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 21 (ObjNum 139 snap 29 seq_num 139) dirty exists 2024-09-18T14:00:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:305: left oid 5 (ObjNum 139 snap 29 seq_num 139) 2024-09-18T14:00:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:306: expect (ObjNum 89 snap 9 seq_num 89) 2024-09-18T14:00:28.366 INFO:tasks.rados.rados.0.smithi084.stdout:308: expect (ObjNum 96 snap 11 seq_num 96) 2024-09-18T14:00:28.519 INFO:tasks.rados.rados.0.smithi084.stdout:309: finishing copy_from racing read to smithi08434663-43 2024-09-18T14:00:28.519 INFO:tasks.rados.rados.0.smithi084.stdout:309: finishing copy_from to smithi08434663-43 2024-09-18T14:00:28.519 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 78 (ObjNum 135 snap 28 seq_num 135) dirty exists 2024-09-18T14:00:28.519 INFO:tasks.rados.rados.0.smithi084.stdout:310: finishing write tid 1 to smithi08434663-47 2024-09-18T14:00:28.519 INFO:tasks.rados.rados.0.smithi084.stdout:311: finishing copy_from to smithi08434663-12 2024-09-18T14:00:28.519 INFO:tasks.rados.rados.0.smithi084.stdout:311: got expected ENOENT (src dne) 2024-09-18T14:00:28.519 INFO:tasks.rados.rados.0.smithi084.stdout:311: finishing copy_from racing read to smithi08434663-12 2024-09-18T14:00:28.519 INFO:tasks.rados.rados.0.smithi084.stdout:310: finishing write tid 2 to smithi08434663-47 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:310: finishing write tid 3 to smithi08434663-47 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:310: finishing write tid 4 to smithi08434663-47 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 65 (ObjNum 141 snap 29 seq_num 141) dirty exists 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:310: left oid 47 (ObjNum 141 snap 29 seq_num 141) 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:305: done (5 left) 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:306: done (4 left) 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:308: done (3 left) 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:309: done (2 left) 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:310: done (1 left) 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:311: done (0 left) 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:312: rollback oid 16 current snap is 29 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 16 to 28 2024-09-18T14:00:28.520 INFO:tasks.rados.rados.0.smithi084.stdout:313: delete oid 33 current snap is 29 2024-09-18T14:00:28.521 INFO:tasks.rados.rados.0.smithi084.stdout:313: done (1 left) 2024-09-18T14:00:28.521 INFO:tasks.rados.rados.0.smithi084.stdout:314: copy_from oid 27 from oid 48 current snap is 29 2024-09-18T14:00:28.521 INFO:tasks.rados.rados.0.smithi084.stdout:315: read oid 24 snap -1 2024-09-18T14:00:28.521 INFO:tasks.rados.rados.0.smithi084.stdout:315: expect deleted 2024-09-18T14:00:28.521 INFO:tasks.rados.rados.0.smithi084.stdout:316: snap_remove snap 23 2024-09-18T14:00:29.205 INFO:tasks.rados.rados.0.smithi084.stdout:316: done (3 left) 2024-09-18T14:00:29.205 INFO:tasks.rados.rados.0.smithi084.stdout:317: read oid 20 snap -1 2024-09-18T14:00:29.205 INFO:tasks.rados.rados.0.smithi084.stdout:317: expect (ObjNum 128 snap 25 seq_num 128) 2024-09-18T14:00:29.205 INFO:tasks.rados.rados.0.smithi084.stdout:318: read oid 50 snap -1 2024-09-18T14:00:29.205 INFO:tasks.rados.rados.0.smithi084.stdout:318: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-18T14:00:29.206 INFO:tasks.rados.rados.0.smithi084.stdout:319: snap_create 2024-09-18T14:00:29.206 INFO:tasks.rados.rados.0.smithi084.stdout:312: finishing rollback tid 0 to smithi08434663-16 2024-09-18T14:00:29.206 INFO:tasks.rados.rados.0.smithi084.stdout:312: finishing rollback tid 1 to smithi08434663-16 2024-09-18T14:00:29.206 INFO:tasks.rados.rados.0.smithi084.stdout:312: finishing rollback tid 2 to smithi08434663-16 2024-09-18T14:00:29.206 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 101 (ObjNum 131 snap 26 seq_num 131) dirty exists 2024-09-18T14:00:29.206 INFO:tasks.rados.rados.0.smithi084.stdout:314: finishing copy_from racing read to smithi08434663-27 2024-09-18T14:00:29.206 INFO:tasks.rados.rados.0.smithi084.stdout:314: finishing copy_from to smithi08434663-27 2024-09-18T14:00:29.206 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 54 (ObjNum 103 snap 17 seq_num 103) dirty exists 2024-09-18T14:00:29.213 INFO:tasks.rados.rados.0.smithi084.stdout:318: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-18T14:00:29.300 INFO:tasks.rados.rados.0.smithi084.stdout:317: expect (ObjNum 128 snap 25 seq_num 128) 2024-09-18T14:00:30.211 INFO:tasks.rados.rados.0.smithi084.stdout:312: done (5 left) 2024-09-18T14:00:30.211 INFO:tasks.rados.rados.0.smithi084.stdout:314: done (4 left) 2024-09-18T14:00:30.211 INFO:tasks.rados.rados.0.smithi084.stdout:315: done (3 left) 2024-09-18T14:00:30.212 INFO:tasks.rados.rados.0.smithi084.stdout:317: done (2 left) 2024-09-18T14:00:30.212 INFO:tasks.rados.rados.0.smithi084.stdout:318: done (1 left) 2024-09-18T14:00:30.212 INFO:tasks.rados.rados.0.smithi084.stdout:319: done (0 left) 2024-09-18T14:00:30.212 INFO:tasks.rados.rados.0.smithi084.stdout:320: write (excl) oid 13 current snap is 30 2024-09-18T14:00:30.212 INFO:tasks.rados.rados.0.smithi084.stdout:320: seq_num 143 ranges {403401=454961,1584925=506039} 2024-09-18T14:00:30.215 INFO:tasks.rados.rados.0.smithi084.stdout:320: writing smithi08434663-13 from 403401 to 858362 tid 1 2024-09-18T14:00:30.223 INFO:tasks.rados.rados.0.smithi084.stdout:320: writing smithi08434663-13 from 1584925 to 2090964 tid 2 2024-09-18T14:00:30.223 INFO:tasks.rados.rados.0.smithi084.stdout:321: read oid 24 snap -1 2024-09-18T14:00:30.223 INFO:tasks.rados.rados.0.smithi084.stdout:321: expect deleted 2024-09-18T14:00:30.223 INFO:tasks.rados.rados.0.smithi084.stdout:322: write (excl) oid 17 current snap is 30 2024-09-18T14:00:30.223 INFO:tasks.rados.rados.0.smithi084.stdout:322: seq_num 144 ranges {420208=530210,1527054=766050} 2024-09-18T14:00:30.227 INFO:tasks.rados.rados.0.smithi084.stdout:322: writing smithi08434663-17 from 420208 to 950418 tid 1 2024-09-18T14:00:30.234 INFO:tasks.rados.rados.0.smithi084.stdout:322: writing smithi08434663-17 from 1527054 to 2293104 tid 2 2024-09-18T14:00:30.235 INFO:tasks.rados.rados.0.smithi084.stdout:323: rollback oid 20 current snap is 30 2024-09-18T14:00:30.235 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 20 to 28 2024-09-18T14:00:30.235 INFO:tasks.rados.rados.0.smithi084.stdout:320: finishing write tid 1 to smithi08434663-13 2024-09-18T14:00:30.235 INFO:tasks.rados.rados.0.smithi084.stdout:320: finishing write tid 2 to smithi08434663-13 2024-09-18T14:00:30.235 INFO:tasks.rados.rados.0.smithi084.stdout:320: finishing write tid 3 to smithi08434663-13 2024-09-18T14:00:30.235 INFO:tasks.rados.rados.0.smithi084.stdout:320: finishing write tid 4 to smithi08434663-13 2024-09-18T14:00:30.236 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 51 (ObjNum 143 snap 30 seq_num 143) dirty exists 2024-09-18T14:00:30.236 INFO:tasks.rados.rados.0.smithi084.stdout:320: left oid 13 (ObjNum 143 snap 30 seq_num 143) 2024-09-18T14:00:30.236 INFO:tasks.rados.rados.0.smithi084.stdout:322: finishing write tid 1 to smithi08434663-17 2024-09-18T14:00:30.236 INFO:tasks.rados.rados.0.smithi084.stdout:320: done (3 left) 2024-09-18T14:00:30.236 INFO:tasks.rados.rados.0.smithi084.stdout:321: done (2 left) 2024-09-18T14:00:30.236 INFO:tasks.rados.rados.0.smithi084.stdout:324: delete oid 22 current snap is 30 2024-09-18T14:00:30.239 INFO:tasks.rados.rados.0.smithi084.stdout:323: finishing rollback tid 0 to smithi08434663-20 2024-09-18T14:00:30.239 INFO:tasks.rados.rados.0.smithi084.stdout:322: finishing write tid 2 to smithi08434663-17 2024-09-18T14:00:30.239 INFO:tasks.rados.rados.0.smithi084.stdout:323: finishing rollback tid 1 to smithi08434663-20 2024-09-18T14:00:30.240 INFO:tasks.rados.rados.0.smithi084.stdout:322: finishing write tid 3 to smithi08434663-17 2024-09-18T14:00:30.240 INFO:tasks.rados.rados.0.smithi084.stdout:323: finishing rollback tid 2 to smithi08434663-20 2024-09-18T14:00:30.240 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 61 (ObjNum 128 snap 25 seq_num 128) dirty exists 2024-09-18T14:00:30.240 INFO:tasks.rados.rados.0.smithi084.stdout:322: finishing write tid 4 to smithi08434663-17 2024-09-18T14:00:30.240 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 55 (ObjNum 144 snap 30 seq_num 144) dirty exists 2024-09-18T14:00:30.240 INFO:tasks.rados.rados.0.smithi084.stdout:322: left oid 17 (ObjNum 144 snap 30 seq_num 144) 2024-09-18T14:00:30.241 INFO:tasks.rados.rados.0.smithi084.stdout:322: done (2 left) 2024-09-18T14:00:30.241 INFO:tasks.rados.rados.0.smithi084.stdout:323: done (1 left) 2024-09-18T14:00:30.241 INFO:tasks.rados.rados.0.smithi084.stdout:324: done (0 left) 2024-09-18T14:00:30.241 INFO:tasks.rados.rados.0.smithi084.stdout:325: delete oid 34 current snap is 30 2024-09-18T14:00:30.243 INFO:tasks.rados.rados.0.smithi084.stdout:325: done (0 left) 2024-09-18T14:00:30.243 INFO:tasks.rados.rados.0.smithi084.stdout:326: write oid 15 current snap is 30 2024-09-18T14:00:30.243 INFO:tasks.rados.rados.0.smithi084.stdout:326: seq_num 147 ranges {470629=472310,1418853=513377} 2024-09-18T14:00:30.248 INFO:tasks.rados.rados.0.smithi084.stdout:326: writing smithi08434663-15 from 470629 to 942939 tid 1 2024-09-18T14:00:30.254 INFO:tasks.rados.rados.0.smithi084.stdout:326: writing smithi08434663-15 from 1418853 to 1932230 tid 2 2024-09-18T14:00:30.254 INFO:tasks.rados.rados.0.smithi084.stdout:327: write (excl) oid 39 current snap is 30 2024-09-18T14:00:30.254 INFO:tasks.rados.rados.0.smithi084.stdout:326: finishing write tid 1 to smithi08434663-15 2024-09-18T14:00:30.254 INFO:tasks.rados.rados.0.smithi084.stdout:327: seq_num 148 ranges {487436=547559,1760982=657035} 2024-09-18T14:00:30.259 INFO:tasks.rados.rados.0.smithi084.stdout:327: writing smithi08434663-39 from 487436 to 1034995 tid 1 2024-09-18T14:00:30.269 INFO:tasks.rados.rados.0.smithi084.stdout:327: writing smithi08434663-39 from 1760982 to 2418017 tid 2 2024-09-18T14:00:30.269 INFO:tasks.rados.rados.0.smithi084.stdout:328: snap_remove snap 28 2024-09-18T14:00:31.215 INFO:tasks.rados.rados.0.smithi084.stdout:328: done (2 left) 2024-09-18T14:00:31.215 INFO:tasks.rados.rados.0.smithi084.stdout:329: delete oid 50 current snap is 30 2024-09-18T14:00:31.215 INFO:tasks.rados.rados.0.smithi084.stdout:326: finishing write tid 2 to smithi08434663-15 2024-09-18T14:00:31.215 INFO:tasks.rados.rados.0.smithi084.stdout:326: finishing write tid 3 to smithi08434663-15 2024-09-18T14:00:31.216 INFO:tasks.rados.rados.0.smithi084.stdout:326: finishing write tid 4 to smithi08434663-15 2024-09-18T14:00:31.216 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 107 (ObjNum 147 snap 30 seq_num 147) dirty exists 2024-09-18T14:00:31.216 INFO:tasks.rados.rados.0.smithi084.stdout:326: left oid 15 (ObjNum 147 snap 30 seq_num 147) 2024-09-18T14:00:31.216 INFO:tasks.rados.rados.0.smithi084.stdout:327: finishing write tid 1 to smithi08434663-39 2024-09-18T14:00:31.216 INFO:tasks.rados.rados.0.smithi084.stdout:327: finishing write tid 2 to smithi08434663-39 2024-09-18T14:00:31.216 INFO:tasks.rados.rados.0.smithi084.stdout:327: finishing write tid 3 to smithi08434663-39 2024-09-18T14:00:31.216 INFO:tasks.rados.rados.0.smithi084.stdout:327: finishing write tid 4 to smithi08434663-39 2024-09-18T14:00:31.216 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 71 (ObjNum 148 snap 30 seq_num 148) dirty exists 2024-09-18T14:00:31.216 INFO:tasks.rados.rados.0.smithi084.stdout:327: left oid 39 (ObjNum 148 snap 30 seq_num 148) 2024-09-18T14:00:31.227 INFO:tasks.rados.rados.0.smithi084.stdout:326: done (2 left) 2024-09-18T14:00:31.227 INFO:tasks.rados.rados.0.smithi084.stdout:327: done (1 left) 2024-09-18T14:00:31.227 INFO:tasks.rados.rados.0.smithi084.stdout:329: done (0 left) 2024-09-18T14:00:31.227 INFO:tasks.rados.rados.0.smithi084.stdout:330: snap_create 2024-09-18T14:00:32.234 INFO:tasks.rados.rados.0.smithi084.stdout:330: done (0 left) 2024-09-18T14:00:32.234 INFO:tasks.rados.rados.0.smithi084.stdout:331: read oid 7 snap -1 2024-09-18T14:00:32.234 INFO:tasks.rados.rados.0.smithi084.stdout:331: expect deleted 2024-09-18T14:00:32.234 INFO:tasks.rados.rados.0.smithi084.stdout:332: delete oid 6 current snap is 31 2024-09-18T14:00:32.241 INFO:tasks.rados.rados.0.smithi084.stdout:331: done (1 left) 2024-09-18T14:00:32.241 INFO:tasks.rados.rados.0.smithi084.stdout:332: done (0 left) 2024-09-18T14:00:32.241 INFO:tasks.rados.rados.0.smithi084.stdout:333: snap_remove snap 30 2024-09-18T14:00:33.233 INFO:tasks.rados.rados.0.smithi084.stdout:333: done (0 left) 2024-09-18T14:00:33.233 INFO:tasks.rados.rados.0.smithi084.stdout:334: read oid 35 snap -1 2024-09-18T14:00:33.233 INFO:tasks.rados.rados.0.smithi084.stdout:334: expect (ObjNum 129 snap 26 seq_num 129) 2024-09-18T14:00:33.234 INFO:tasks.rados.rados.0.smithi084.stdout:335: snap_remove snap 29 2024-09-18T14:00:34.245 INFO:tasks.rados.rados.0.smithi084.stdout:335: done (1 left) 2024-09-18T14:00:34.245 INFO:tasks.rados.rados.0.smithi084.stdout:336: delete oid 32 current snap is 31 2024-09-18T14:00:34.246 INFO:tasks.rados.rados.0.smithi084.stdout:334: expect (ObjNum 129 snap 26 seq_num 129) 2024-09-18T14:00:34.364 INFO:tasks.rados.rados.0.smithi084.stdout:334: done (1 left) 2024-09-18T14:00:34.364 INFO:tasks.rados.rados.0.smithi084.stdout:336: done (0 left) 2024-09-18T14:00:34.365 INFO:tasks.rados.rados.0.smithi084.stdout:337: write (excl) oid 41 current snap is 31 2024-09-18T14:00:34.365 INFO:tasks.rados.rados.0.smithi084.stdout:337: seq_num 152 ranges {554664=448555,1478557=664373} 2024-09-18T14:00:34.369 INFO:tasks.rados.rados.0.smithi084.stdout:337: writing smithi08434663-41 from 554664 to 1003219 tid 1 2024-09-18T14:00:34.375 INFO:tasks.rados.rados.0.smithi084.stdout:337: writing smithi08434663-41 from 1478557 to 2142930 tid 2 2024-09-18T14:00:34.376 INFO:tasks.rados.rados.0.smithi084.stdout:338: snap_create 2024-09-18T14:00:34.376 INFO:tasks.rados.rados.0.smithi084.stdout:337: finishing write tid 1 to smithi08434663-41 2024-09-18T14:00:34.379 INFO:tasks.rados.rados.0.smithi084.stdout:337: finishing write tid 2 to smithi08434663-41 2024-09-18T14:00:34.379 INFO:tasks.rados.rados.0.smithi084.stdout:337: finishing write tid 3 to smithi08434663-41 2024-09-18T14:00:34.380 INFO:tasks.rados.rados.0.smithi084.stdout:337: finishing write tid 4 to smithi08434663-41 2024-09-18T14:00:34.380 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 86 (ObjNum 152 snap 31 seq_num 152) dirty exists 2024-09-18T14:00:34.380 INFO:tasks.rados.rados.0.smithi084.stdout:337: left oid 41 (ObjNum 152 snap 31 seq_num 152) 2024-09-18T14:00:35.252 INFO:tasks.rados.rados.0.smithi084.stdout:337: done (1 left) 2024-09-18T14:00:35.253 INFO:tasks.rados.rados.0.smithi084.stdout:338: done (0 left) 2024-09-18T14:00:35.253 INFO:tasks.rados.rados.0.smithi084.stdout:339: write oid 32 current snap is 32 2024-09-18T14:00:35.253 INFO:tasks.rados.rados.0.smithi084.stdout:339: seq_num 153 ranges {571471=640157,1653392=524384} 2024-09-18T14:00:35.259 INFO:tasks.rados.rados.0.smithi084.stdout:339: writing smithi08434663-32 from 571471 to 1211628 tid 1 2024-09-18T14:00:35.265 INFO:tasks.rados.rados.0.smithi084.stdout:339: writing smithi08434663-32 from 1653392 to 2177776 tid 2 2024-09-18T14:00:35.265 INFO:tasks.rados.rados.0.smithi084.stdout:340: write oid 48 current snap is 32 2024-09-18T14:00:35.265 INFO:tasks.rados.rados.0.smithi084.stdout:340: seq_num 154 ranges {588278=715406,1711874=668042} 2024-09-18T14:00:35.272 INFO:tasks.rados.rados.0.smithi084.stdout:340: writing smithi08434663-48 from 588278 to 1303684 tid 1 2024-09-18T14:00:35.278 INFO:tasks.rados.rados.0.smithi084.stdout:340: writing smithi08434663-48 from 1711874 to 2379916 tid 2 2024-09-18T14:00:35.278 INFO:tasks.rados.rados.0.smithi084.stdout:341: snap_remove snap 31 2024-09-18T14:00:36.258 INFO:tasks.rados.rados.0.smithi084.stdout:341: done (2 left) 2024-09-18T14:00:36.258 INFO:tasks.rados.rados.0.smithi084.stdout:342: rollback oid 31 current snap is 32 2024-09-18T14:00:36.258 INFO:tasks.rados.rados.0.smithi084.stdout:342: done (2 left) 2024-09-18T14:00:36.258 INFO:tasks.rados.rados.0.smithi084.stdout:343: delete oid 8 current snap is 32 2024-09-18T14:00:36.258 INFO:tasks.rados.rados.0.smithi084.stdout:339: finishing write tid 1 to smithi08434663-32 2024-09-18T14:00:36.258 INFO:tasks.rados.rados.0.smithi084.stdout:339: finishing write tid 2 to smithi08434663-32 2024-09-18T14:00:36.258 INFO:tasks.rados.rados.0.smithi084.stdout:339: finishing write tid 3 to smithi08434663-32 2024-09-18T14:00:36.258 INFO:tasks.rados.rados.0.smithi084.stdout:339: finishing write tid 4 to smithi08434663-32 2024-09-18T14:00:36.258 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 155 (ObjNum 153 snap 32 seq_num 153) dirty exists 2024-09-18T14:00:36.259 INFO:tasks.rados.rados.0.smithi084.stdout:339: left oid 32 (ObjNum 153 snap 32 seq_num 153) 2024-09-18T14:00:36.259 INFO:tasks.rados.rados.0.smithi084.stdout:340: finishing write tid 1 to smithi08434663-48 2024-09-18T14:00:36.259 INFO:tasks.rados.rados.0.smithi084.stdout:340: finishing write tid 2 to smithi08434663-48 2024-09-18T14:00:36.259 INFO:tasks.rados.rados.0.smithi084.stdout:340: finishing write tid 3 to smithi08434663-48 2024-09-18T14:00:36.259 INFO:tasks.rados.rados.0.smithi084.stdout:340: finishing write tid 4 to smithi08434663-48 2024-09-18T14:00:36.259 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 68 (ObjNum 154 snap 32 seq_num 154) dirty exists 2024-09-18T14:00:36.259 INFO:tasks.rados.rados.0.smithi084.stdout:340: left oid 48 (ObjNum 154 snap 32 seq_num 154) 2024-09-18T14:00:36.260 INFO:tasks.rados.rados.0.smithi084.stdout:339: done (2 left) 2024-09-18T14:00:36.260 INFO:tasks.rados.rados.0.smithi084.stdout:340: done (1 left) 2024-09-18T14:00:36.260 INFO:tasks.rados.rados.0.smithi084.stdout:343: done (0 left) 2024-09-18T14:00:36.260 INFO:tasks.rados.rados.0.smithi084.stdout:344: read oid 5 snap -1 2024-09-18T14:00:36.260 INFO:tasks.rados.rados.0.smithi084.stdout:344: expect (ObjNum 139 snap 29 seq_num 139) 2024-09-18T14:00:36.260 INFO:tasks.rados.rados.0.smithi084.stdout:345: snap_create 2024-09-18T14:00:36.265 INFO:tasks.rados.rados.0.smithi084.stdout:344: expect (ObjNum 139 snap 29 seq_num 139) 2024-09-18T14:00:37.148 INFO:tasks.rados.rados.0.smithi084.stdout:344: done (1 left) 2024-09-18T14:00:37.148 INFO:tasks.rados.rados.0.smithi084.stdout:345: done (0 left) 2024-09-18T14:00:37.148 INFO:tasks.rados.rados.0.smithi084.stdout:346: snap_create 2024-09-18T14:00:38.149 INFO:tasks.rados.rados.0.smithi084.stdout:346: done (0 left) 2024-09-18T14:00:38.150 INFO:tasks.rados.rados.0.smithi084.stdout:347: read oid 13 snap -1 2024-09-18T14:00:38.150 INFO:tasks.rados.rados.0.smithi084.stdout:347: expect (ObjNum 143 snap 30 seq_num 143) 2024-09-18T14:00:38.150 INFO:tasks.rados.rados.0.smithi084.stdout:348: write (excl) oid 11 current snap is 34 2024-09-18T14:00:38.150 INFO:tasks.rados.rados.0.smithi084.stdout:348: seq_num 156 ranges {621892=465904,1712485=671711} 2024-09-18T14:00:38.155 INFO:tasks.rados.rados.0.smithi084.stdout:348: writing smithi08434663-11 from 621892 to 1087796 tid 1 2024-09-18T14:00:38.163 INFO:tasks.rados.rados.0.smithi084.stdout:348: writing smithi08434663-11 from 1712485 to 2384196 tid 2 2024-09-18T14:00:38.163 INFO:tasks.rados.rados.0.smithi084.stdout:349: delete oid 7 current snap is 34 2024-09-18T14:00:38.163 INFO:tasks.rados.rados.0.smithi084.stdout:347: expect (ObjNum 143 snap 30 seq_num 143) 2024-09-18T14:00:38.314 INFO:tasks.rados.rados.0.smithi084.stdout:348: finishing write tid 1 to smithi08434663-11 2024-09-18T14:00:38.314 INFO:tasks.rados.rados.0.smithi084.stdout:348: finishing write tid 2 to smithi08434663-11 2024-09-18T14:00:38.314 INFO:tasks.rados.rados.0.smithi084.stdout:348: finishing write tid 3 to smithi08434663-11 2024-09-18T14:00:38.315 INFO:tasks.rados.rados.0.smithi084.stdout:348: finishing write tid 4 to smithi08434663-11 2024-09-18T14:00:38.315 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 81 (ObjNum 156 snap 34 seq_num 156) dirty exists 2024-09-18T14:00:38.315 INFO:tasks.rados.rados.0.smithi084.stdout:348: left oid 11 (ObjNum 156 snap 34 seq_num 156) 2024-09-18T14:00:38.315 INFO:tasks.rados.rados.0.smithi084.stdout:347: done (2 left) 2024-09-18T14:00:38.315 INFO:tasks.rados.rados.0.smithi084.stdout:348: done (1 left) 2024-09-18T14:00:38.315 INFO:tasks.rados.rados.0.smithi084.stdout:349: done (0 left) 2024-09-18T14:00:38.315 INFO:tasks.rados.rados.0.smithi084.stdout:350: snap_create 2024-09-18T14:00:39.264 INFO:tasks.rados.rados.0.smithi084.stdout:350: done (0 left) 2024-09-18T14:00:39.265 INFO:tasks.rados.rados.0.smithi084.stdout:351: write oid 32 current snap is 35 2024-09-18T14:00:39.265 INFO:tasks.rados.rados.0.smithi084.stdout:351: seq_num 158 ranges {655506=616402,1829449=675380} 2024-09-18T14:00:39.270 INFO:tasks.rados.rados.0.smithi084.stdout:351: writing smithi08434663-32 from 655506 to 1271908 tid 1 2024-09-18T14:00:39.277 INFO:tasks.rados.rados.0.smithi084.stdout:351: writing smithi08434663-32 from 1829449 to 2504829 tid 2 2024-09-18T14:00:39.278 INFO:tasks.rados.rados.0.smithi084.stdout:352: write (excl) oid 3 current snap is 35 2024-09-18T14:00:39.278 INFO:tasks.rados.rados.0.smithi084.stdout:351: finishing write tid 1 to smithi08434663-32 2024-09-18T14:00:39.278 INFO:tasks.rados.rados.0.smithi084.stdout:352: seq_num 159 ranges {672313=691651,1887931=419038} 2024-09-18T14:00:39.285 INFO:tasks.rados.rados.0.smithi084.stdout:352: writing smithi08434663-3 from 672313 to 1363964 tid 1 2024-09-18T14:00:39.290 INFO:tasks.rados.rados.0.smithi084.stdout:352: writing smithi08434663-3 from 1887931 to 2306969 tid 2 2024-09-18T14:00:39.291 INFO:tasks.rados.rados.0.smithi084.stdout:353: copy_from oid 46 from oid 24 current snap is 35 2024-09-18T14:00:39.291 INFO:tasks.rados.rados.0.smithi084.stdout:351: finishing write tid 2 to smithi08434663-32 2024-09-18T14:00:39.291 INFO:tasks.rados.rados.0.smithi084.stdout:351: finishing write tid 3 to smithi08434663-32 2024-09-18T14:00:39.291 INFO:tasks.rados.rados.0.smithi084.stdout:351: finishing write tid 4 to smithi08434663-32 2024-09-18T14:00:39.291 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 159 (ObjNum 158 snap 35 seq_num 158) dirty exists 2024-09-18T14:00:39.291 INFO:tasks.rados.rados.0.smithi084.stdout:351: left oid 32 (ObjNum 158 snap 35 seq_num 158) 2024-09-18T14:00:39.291 INFO:tasks.rados.rados.0.smithi084.stdout:351: done (2 left) 2024-09-18T14:00:39.291 INFO:tasks.rados.rados.0.smithi084.stdout:354: snap_create 2024-09-18T14:00:39.292 INFO:tasks.rados.rados.0.smithi084.stdout:352: finishing write tid 1 to smithi08434663-3 2024-09-18T14:00:39.293 INFO:tasks.rados.rados.0.smithi084.stdout:353: finishing copy_from to smithi08434663-46 2024-09-18T14:00:39.293 INFO:tasks.rados.rados.0.smithi084.stdout:353: got expected ENOENT (src dne) 2024-09-18T14:00:39.293 INFO:tasks.rados.rados.0.smithi084.stdout:353: finishing copy_from racing read to smithi08434663-46 2024-09-18T14:00:39.295 INFO:tasks.rados.rados.0.smithi084.stdout:352: finishing write tid 2 to smithi08434663-3 2024-09-18T14:00:39.296 INFO:tasks.rados.rados.0.smithi084.stdout:352: finishing write tid 3 to smithi08434663-3 2024-09-18T14:00:39.296 INFO:tasks.rados.rados.0.smithi084.stdout:352: finishing write tid 4 to smithi08434663-3 2024-09-18T14:00:39.296 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 57 (ObjNum 159 snap 35 seq_num 159) dirty exists 2024-09-18T14:00:39.296 INFO:tasks.rados.rados.0.smithi084.stdout:352: left oid 3 (ObjNum 159 snap 35 seq_num 159) 2024-09-18T14:00:40.269 INFO:tasks.rados.rados.0.smithi084.stdout:352: done (2 left) 2024-09-18T14:00:40.270 INFO:tasks.rados.rados.0.smithi084.stdout:353: done (1 left) 2024-09-18T14:00:40.270 INFO:tasks.rados.rados.0.smithi084.stdout:354: done (0 left) 2024-09-18T14:00:40.270 INFO:tasks.rados.rados.0.smithi084.stdout:355: rollback oid 14 current snap is 36 2024-09-18T14:00:40.270 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 14 to 34 2024-09-18T14:00:40.270 INFO:tasks.rados.rados.0.smithi084.stdout:356: write (excl) oid 5 current snap is 36 2024-09-18T14:00:40.270 INFO:tasks.rados.rados.0.smithi084.stdout:356: seq_num 160 ranges {689120=483253,1946413=679049} 2024-09-18T14:00:40.276 INFO:tasks.rados.rados.0.smithi084.stdout:356: writing smithi08434663-5 from 689120 to 1172373 tid 1 2024-09-18T14:00:40.284 INFO:tasks.rados.rados.0.smithi084.stdout:356: writing smithi08434663-5 from 1946413 to 2625462 tid 2 2024-09-18T14:00:40.285 INFO:tasks.rados.rados.0.smithi084.stdout:357: write oid 46 current snap is 36 2024-09-18T14:00:40.285 INFO:tasks.rados.rados.0.smithi084.stdout:355: finishing rollback tid 1 to smithi08434663-14 2024-09-18T14:00:40.285 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 0 (ObjNum 129 snap 0 seq_num 2493154064) dirty dne 2024-09-18T14:00:40.285 INFO:tasks.rados.rados.0.smithi084.stdout:356: finishing write tid 1 to smithi08434663-5 2024-09-18T14:00:40.285 INFO:tasks.rados.rados.0.smithi084.stdout:357: seq_num 161 ranges {705927=558502,2004895=422707} 2024-09-18T14:00:40.291 INFO:tasks.rados.rados.0.smithi084.stdout:357: writing smithi08434663-46 from 705927 to 1264429 tid 1 2024-09-18T14:00:40.297 INFO:tasks.rados.rados.0.smithi084.stdout:357: writing smithi08434663-46 from 2004895 to 2427602 tid 2 2024-09-18T14:00:40.298 INFO:tasks.rados.rados.0.smithi084.stdout:355: done (2 left) 2024-09-18T14:00:40.298 INFO:tasks.rados.rados.0.smithi084.stdout:358: write (excl) oid 17 current snap is 36 2024-09-18T14:00:40.298 INFO:tasks.rados.rados.0.smithi084.stdout:356: finishing write tid 2 to smithi08434663-5 2024-09-18T14:00:40.298 INFO:tasks.rados.rados.0.smithi084.stdout:356: finishing write tid 3 to smithi08434663-5 2024-09-18T14:00:40.298 INFO:tasks.rados.rados.0.smithi084.stdout:358: seq_num 162 ranges {722734=633751,2063377=659357} 2024-09-18T14:00:40.305 INFO:tasks.rados.rados.0.smithi084.stdout:358: writing smithi08434663-17 from 722734 to 1356485 tid 1 2024-09-18T14:00:40.313 INFO:tasks.rados.rados.0.smithi084.stdout:358: writing smithi08434663-17 from 2063377 to 2722734 tid 2 2024-09-18T14:00:40.313 INFO:tasks.rados.rados.0.smithi084.stdout:359: copy_from oid 30 from oid 20 current snap is 36 2024-09-18T14:00:40.313 INFO:tasks.rados.rados.0.smithi084.stdout:356: finishing write tid 4 to smithi08434663-5 2024-09-18T14:00:40.313 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 24 (ObjNum 160 snap 36 seq_num 160) dirty exists 2024-09-18T14:00:40.313 INFO:tasks.rados.rados.0.smithi084.stdout:356: left oid 5 (ObjNum 160 snap 36 seq_num 160) 2024-09-18T14:00:40.313 INFO:tasks.rados.rados.0.smithi084.stdout:357: finishing write tid 1 to smithi08434663-46 2024-09-18T14:00:40.313 INFO:tasks.rados.rados.0.smithi084.stdout:356: done (3 left) 2024-09-18T14:00:40.313 INFO:tasks.rados.rados.0.smithi084.stdout:360: read oid 3 snap -1 2024-09-18T14:00:40.314 INFO:tasks.rados.rados.0.smithi084.stdout:360: expect (ObjNum 159 snap 35 seq_num 159) 2024-09-18T14:00:40.314 INFO:tasks.rados.rados.0.smithi084.stdout:361: copy_from oid 18 from oid 10 current snap is 36 2024-09-18T14:00:40.314 INFO:tasks.rados.rados.0.smithi084.stdout:357: finishing write tid 2 to smithi08434663-46 2024-09-18T14:00:40.314 INFO:tasks.rados.rados.0.smithi084.stdout:357: finishing write tid 3 to smithi08434663-46 2024-09-18T14:00:40.314 INFO:tasks.rados.rados.0.smithi084.stdout:357: finishing write tid 4 to smithi08434663-46 2024-09-18T14:00:40.314 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 116 (ObjNum 161 snap 36 seq_num 161) dirty exists 2024-09-18T14:00:40.314 INFO:tasks.rados.rados.0.smithi084.stdout:357: left oid 46 (ObjNum 161 snap 36 seq_num 161) 2024-09-18T14:00:40.314 INFO:tasks.rados.rados.0.smithi084.stdout:358: finishing write tid 1 to smithi08434663-17 2024-09-18T14:00:40.314 INFO:tasks.rados.rados.0.smithi084.stdout:357: done (4 left) 2024-09-18T14:00:40.314 INFO:tasks.rados.rados.0.smithi084.stdout:362: write oid 50 current snap is 36 2024-09-18T14:00:40.314 INFO:tasks.rados.rados.0.smithi084.stdout:362: seq_num 163 ranges {739541=709000,2121859=426376} 2024-09-18T14:00:40.320 INFO:tasks.rados.rados.0.smithi084.stdout:362: writing smithi08434663-50 from 739541 to 1448541 tid 1 2024-09-18T14:00:40.327 INFO:tasks.rados.rados.0.smithi084.stdout:362: writing smithi08434663-50 from 2121859 to 2548235 tid 2 2024-09-18T14:00:40.327 INFO:tasks.rados.rados.0.smithi084.stdout:363: copy_from oid 25 from oid 48 current snap is 36 2024-09-18T14:00:40.327 INFO:tasks.rados.rados.0.smithi084.stdout:358: finishing write tid 2 to smithi08434663-17 2024-09-18T14:00:40.327 INFO:tasks.rados.rados.0.smithi084.stdout:358: finishing write tid 3 to smithi08434663-17 2024-09-18T14:00:40.327 INFO:tasks.rados.rados.0.smithi084.stdout:358: finishing write tid 4 to smithi08434663-17 2024-09-18T14:00:40.327 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 67 (ObjNum 162 snap 36 seq_num 162) dirty exists 2024-09-18T14:00:40.327 INFO:tasks.rados.rados.0.smithi084.stdout:358: left oid 17 (ObjNum 162 snap 36 seq_num 162) 2024-09-18T14:00:40.327 INFO:tasks.rados.rados.0.smithi084.stdout:359: finishing copy_from racing read to smithi08434663-30 2024-09-18T14:00:40.327 INFO:tasks.rados.rados.0.smithi084.stdout:360: expect (ObjNum 159 snap 35 seq_num 159) 2024-09-18T14:00:40.451 INFO:tasks.rados.rados.0.smithi084.stdout:359: finishing copy_from to smithi08434663-30 2024-09-18T14:00:40.451 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 61 (ObjNum 128 snap 25 seq_num 128) dirty exists 2024-09-18T14:00:40.451 INFO:tasks.rados.rados.0.smithi084.stdout:362: finishing write tid 1 to smithi08434663-50 2024-09-18T14:00:40.451 INFO:tasks.rados.rados.0.smithi084.stdout:362: finishing write tid 2 to smithi08434663-50 2024-09-18T14:00:40.451 INFO:tasks.rados.rados.0.smithi084.stdout:362: finishing write tid 3 to smithi08434663-50 2024-09-18T14:00:40.451 INFO:tasks.rados.rados.0.smithi084.stdout:362: finishing write tid 4 to smithi08434663-50 2024-09-18T14:00:40.451 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 162 (ObjNum 163 snap 36 seq_num 163) dirty exists 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:362: left oid 50 (ObjNum 163 snap 36 seq_num 163) 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:361: finishing copy_from racing read to smithi08434663-18 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:358: done (5 left) 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:359: done (4 left) 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:360: done (3 left) 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:362: done (2 left) 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:364: read oid 13 snap -1 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:364: expect (ObjNum 143 snap 30 seq_num 143) 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:365: read oid 32 snap -1 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:365: expect (ObjNum 158 snap 35 seq_num 158) 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:366: delete oid 6 current snap is 36 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:363: finishing copy_from racing read to smithi08434663-25 2024-09-18T14:00:40.452 INFO:tasks.rados.rados.0.smithi084.stdout:361: finishing copy_from to smithi08434663-18 2024-09-18T14:00:40.453 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 18 (ObjNum 72 snap 6 seq_num 72) dirty exists 2024-09-18T14:00:40.453 INFO:tasks.rados.rados.0.smithi084.stdout:363: finishing copy_from to smithi08434663-25 2024-09-18T14:00:40.453 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 68 (ObjNum 154 snap 32 seq_num 154) dirty exists 2024-09-18T14:00:40.453 INFO:tasks.rados.rados.0.smithi084.stdout:361: done (4 left) 2024-09-18T14:00:40.453 INFO:tasks.rados.rados.0.smithi084.stdout:363: done (3 left) 2024-09-18T14:00:40.453 INFO:tasks.rados.rados.0.smithi084.stdout:366: done (2 left) 2024-09-18T14:00:40.453 INFO:tasks.rados.rados.0.smithi084.stdout:367: snap_create 2024-09-18T14:00:40.455 INFO:tasks.rados.rados.0.smithi084.stdout:365: expect (ObjNum 158 snap 35 seq_num 158) 2024-09-18T14:00:40.577 INFO:tasks.rados.rados.0.smithi084.stdout:364: expect (ObjNum 143 snap 30 seq_num 143) 2024-09-18T14:00:41.275 INFO:tasks.rados.rados.0.smithi084.stdout:364: done (2 left) 2024-09-18T14:00:41.275 INFO:tasks.rados.rados.0.smithi084.stdout:365: done (1 left) 2024-09-18T14:00:41.275 INFO:tasks.rados.rados.0.smithi084.stdout:367: done (0 left) 2024-09-18T14:00:41.275 INFO:tasks.rados.rados.0.smithi084.stdout:368: read oid 44 snap -1 2024-09-18T14:00:41.275 INFO:tasks.rados.rados.0.smithi084.stdout:368: expect (ObjNum 88 snap 9 seq_num 88) 2024-09-18T14:00:41.275 INFO:tasks.rados.rados.0.smithi084.stdout:369: rollback oid 21 current snap is 37 2024-09-18T14:00:41.275 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 21 to 32 2024-09-18T14:00:41.275 INFO:tasks.rados.rados.0.smithi084.stdout:370: copy_from oid 30 from oid 19 current snap is 37 2024-09-18T14:00:41.275 INFO:tasks.rados.rados.0.smithi084.stdout:371: write oid 29 current snap is 37 2024-09-18T14:00:41.275 INFO:tasks.rados.rados.0.smithi084.stdout:371: seq_num 165 ranges {773155=459498,1722470=430045,2648676=124479} 2024-09-18T14:00:41.282 INFO:tasks.rados.rados.0.smithi084.stdout:371: writing smithi08434663-29 from 773155 to 1232653 tid 1 2024-09-18T14:00:41.287 INFO:tasks.rados.rados.0.smithi084.stdout:371: writing smithi08434663-29 from 1722470 to 2152515 tid 2 2024-09-18T14:00:41.291 INFO:tasks.rados.rados.0.smithi084.stdout:371: writing smithi08434663-29 from 2648676 to 2773155 tid 3 2024-09-18T14:00:41.291 INFO:tasks.rados.rados.0.smithi084.stdout:372: write (excl) oid 33 current snap is 37 2024-09-18T14:00:41.291 INFO:tasks.rados.rados.0.smithi084.stdout:372: seq_num 166 ranges {789962=534747,1780952=690056} 2024-09-18T14:00:41.298 INFO:tasks.rados.rados.0.smithi084.stdout:372: writing smithi08434663-33 from 789962 to 1324709 tid 1 2024-09-18T14:00:41.304 INFO:tasks.rados.rados.0.smithi084.stdout:372: writing smithi08434663-33 from 1780952 to 2471008 tid 2 2024-09-18T14:00:41.304 INFO:tasks.rados.rados.0.smithi084.stdout:373: delete oid 47 current snap is 37 2024-09-18T14:00:41.304 INFO:tasks.rados.rados.0.smithi084.stdout:369: finishing rollback tid 0 to smithi08434663-21 2024-09-18T14:00:41.304 INFO:tasks.rados.rados.0.smithi084.stdout:369: finishing rollback tid 1 to smithi08434663-21 2024-09-18T14:00:41.305 INFO:tasks.rados.rados.0.smithi084.stdout:368: expect (ObjNum 88 snap 9 seq_num 88) 2024-09-18T14:00:41.463 INFO:tasks.rados.rados.0.smithi084.stdout:369: finishing rollback tid 2 to smithi08434663-21 2024-09-18T14:00:41.463 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 76 (ObjNum 96 snap 11 seq_num 96) dirty exists 2024-09-18T14:00:41.463 INFO:tasks.rados.rados.0.smithi084.stdout:370: finishing copy_from racing read to smithi08434663-30 2024-09-18T14:00:41.463 INFO:tasks.rados.rados.0.smithi084.stdout:371: finishing write tid 1 to smithi08434663-29 2024-09-18T14:00:41.463 INFO:tasks.rados.rados.0.smithi084.stdout:370: finishing copy_from to smithi08434663-30 2024-09-18T14:00:41.463 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 36 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T14:00:41.463 INFO:tasks.rados.rados.0.smithi084.stdout:371: finishing write tid 2 to smithi08434663-29 2024-09-18T14:00:41.463 INFO:tasks.rados.rados.0.smithi084.stdout:368: done (5 left) 2024-09-18T14:00:41.463 INFO:tasks.rados.rados.0.smithi084.stdout:369: done (4 left) 2024-09-18T14:00:41.463 INFO:tasks.rados.rados.0.smithi084.stdout:370: done (3 left) 2024-09-18T14:00:41.463 INFO:tasks.rados.rados.0.smithi084.stdout:373: done (2 left) 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:374: rollback oid 26 current snap is 37 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 26 to 33 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:371: finishing write tid 3 to smithi08434663-29 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:371: finishing write tid 4 to smithi08434663-29 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:371: finishing write tid 5 to smithi08434663-29 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 72 (ObjNum 165 snap 37 seq_num 165) dirty exists 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:371: left oid 29 (ObjNum 165 snap 37 seq_num 165) 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:372: finishing write tid 1 to smithi08434663-33 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:372: finishing write tid 2 to smithi08434663-33 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:372: finishing write tid 3 to smithi08434663-33 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:372: finishing write tid 4 to smithi08434663-33 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 93 (ObjNum 166 snap 37 seq_num 166) dirty exists 2024-09-18T14:00:41.464 INFO:tasks.rados.rados.0.smithi084.stdout:372: left oid 33 (ObjNum 166 snap 37 seq_num 166) 2024-09-18T14:00:41.465 INFO:tasks.rados.rados.0.smithi084.stdout:371: done (2 left) 2024-09-18T14:00:41.465 INFO:tasks.rados.rados.0.smithi084.stdout:372: done (1 left) 2024-09-18T14:00:41.465 INFO:tasks.rados.rados.0.smithi084.stdout:375: snap_create 2024-09-18T14:00:41.465 INFO:tasks.rados.rados.0.smithi084.stdout:374: finishing rollback tid 0 to smithi08434663-26 2024-09-18T14:00:41.468 INFO:tasks.rados.rados.0.smithi084.stdout:374: finishing rollback tid 1 to smithi08434663-26 2024-09-18T14:00:41.468 INFO:tasks.rados.rados.0.smithi084.stdout:374: finishing rollback tid 2 to smithi08434663-26 2024-09-18T14:00:41.468 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 121 (ObjNum 116 snap 22 seq_num 116) dirty exists 2024-09-18T14:00:42.286 INFO:tasks.rados.rados.0.smithi084.stdout:374: done (1 left) 2024-09-18T14:00:42.287 INFO:tasks.rados.rados.0.smithi084.stdout:375: done (0 left) 2024-09-18T14:00:42.287 INFO:tasks.rados.rados.0.smithi084.stdout:376: delete oid 16 current snap is 38 2024-09-18T14:00:42.289 INFO:tasks.rados.rados.0.smithi084.stdout:376: done (0 left) 2024-09-18T14:00:42.290 INFO:tasks.rados.rados.0.smithi084.stdout:377: write oid 48 current snap is 38 2024-09-18T14:00:42.290 INFO:tasks.rados.rados.0.smithi084.stdout:377: seq_num 169 ranges {440383=476847,1556398=437383,2474721=365662} 2024-09-18T14:00:42.294 INFO:tasks.rados.rados.0.smithi084.stdout:377: writing smithi08434663-48 from 440383 to 917230 tid 1 2024-09-18T14:00:42.300 INFO:tasks.rados.rados.0.smithi084.stdout:377: writing smithi08434663-48 from 1556398 to 1993781 tid 2 2024-09-18T14:00:42.306 INFO:tasks.rados.rados.0.smithi084.stdout:377: writing smithi08434663-48 from 2474721 to 2840383 tid 3 2024-09-18T14:00:42.306 INFO:tasks.rados.rados.0.smithi084.stdout:378: copy_from oid 36 from oid 21 current snap is 38 2024-09-18T14:00:42.307 INFO:tasks.rados.rados.0.smithi084.stdout:377: finishing write tid 1 to smithi08434663-48 2024-09-18T14:00:42.307 INFO:tasks.rados.rados.0.smithi084.stdout:377: finishing write tid 2 to smithi08434663-48 2024-09-18T14:00:42.307 INFO:tasks.rados.rados.0.smithi084.stdout:379: delete oid 8 current snap is 38 2024-09-18T14:00:42.308 INFO:tasks.rados.rados.0.smithi084.stdout:379: done (2 left) 2024-09-18T14:00:42.308 INFO:tasks.rados.rados.0.smithi084.stdout:380: read oid 43 snap -1 2024-09-18T14:00:42.308 INFO:tasks.rados.rados.0.smithi084.stdout:380: expect (ObjNum 135 snap 28 seq_num 135) 2024-09-18T14:00:42.309 INFO:tasks.rados.rados.0.smithi084.stdout:381: write (excl) oid 23 current snap is 38 2024-09-18T14:00:42.309 INFO:tasks.rados.rados.0.smithi084.stdout:381: seq_num 171 ranges {473997=627345,1673362=441052,2529567=344430} 2024-09-18T14:00:42.314 INFO:tasks.rados.rados.0.smithi084.stdout:381: writing smithi08434663-23 from 473997 to 1101342 tid 1 2024-09-18T14:00:42.320 INFO:tasks.rados.rados.0.smithi084.stdout:381: writing smithi08434663-23 from 1673362 to 2114414 tid 2 2024-09-18T14:00:42.325 INFO:tasks.rados.rados.0.smithi084.stdout:381: writing smithi08434663-23 from 2529567 to 2873997 tid 3 2024-09-18T14:00:42.325 INFO:tasks.rados.rados.0.smithi084.stdout:382: copy_from oid 29 from oid 40 current snap is 38 2024-09-18T14:00:42.325 INFO:tasks.rados.rados.0.smithi084.stdout:377: finishing write tid 3 to smithi08434663-48 2024-09-18T14:00:42.325 INFO:tasks.rados.rados.0.smithi084.stdout:377: finishing write tid 4 to smithi08434663-48 2024-09-18T14:00:42.325 INFO:tasks.rados.rados.0.smithi084.stdout:377: finishing write tid 5 to smithi08434663-48 2024-09-18T14:00:42.325 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 81 (ObjNum 169 snap 38 seq_num 169) dirty exists 2024-09-18T14:00:42.325 INFO:tasks.rados.rados.0.smithi084.stdout:377: left oid 48 (ObjNum 169 snap 38 seq_num 169) 2024-09-18T14:00:42.325 INFO:tasks.rados.rados.0.smithi084.stdout:380: expect (ObjNum 135 snap 28 seq_num 135) 2024-09-18T14:00:42.429 INFO:tasks.rados.rados.0.smithi084.stdout:378: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:00:42.429 INFO:tasks.rados.rados.0.smithi084.stdout:381: finishing write tid 1 to smithi08434663-23 2024-09-18T14:00:42.429 INFO:tasks.rados.rados.0.smithi084.stdout:381: finishing write tid 2 to smithi08434663-23 2024-09-18T14:00:42.429 INFO:tasks.rados.rados.0.smithi084.stdout:378: finishing copy_from to smithi08434663-36 2024-09-18T14:00:42.430 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 76 (ObjNum 96 snap 11 seq_num 96) dirty exists 2024-09-18T14:00:42.430 INFO:tasks.rados.rados.0.smithi084.stdout:381: finishing write tid 3 to smithi08434663-23 2024-09-18T14:00:42.430 INFO:tasks.rados.rados.0.smithi084.stdout:381: finishing write tid 4 to smithi08434663-23 2024-09-18T14:00:42.430 INFO:tasks.rados.rados.0.smithi084.stdout:381: finishing write tid 5 to smithi08434663-23 2024-09-18T14:00:42.430 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 69 (ObjNum 171 snap 38 seq_num 171) dirty exists 2024-09-18T14:00:42.430 INFO:tasks.rados.rados.0.smithi084.stdout:381: left oid 23 (ObjNum 171 snap 38 seq_num 171) 2024-09-18T14:00:42.431 INFO:tasks.rados.rados.0.smithi084.stdout:377: done (4 left) 2024-09-18T14:00:42.431 INFO:tasks.rados.rados.0.smithi084.stdout:378: done (3 left) 2024-09-18T14:00:42.431 INFO:tasks.rados.rados.0.smithi084.stdout:380: done (2 left) 2024-09-18T14:00:42.431 INFO:tasks.rados.rados.0.smithi084.stdout:381: done (1 left) 2024-09-18T14:00:42.431 INFO:tasks.rados.rados.0.smithi084.stdout:383: copy_from oid 46 from oid 38 current snap is 38 2024-09-18T14:00:42.431 INFO:tasks.rados.rados.0.smithi084.stdout:382: finishing copy_from racing read to smithi08434663-29 2024-09-18T14:00:42.432 INFO:tasks.rados.rados.0.smithi084.stdout:382: finishing copy_from to smithi08434663-29 2024-09-18T14:00:42.432 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 41 (ObjNum 136 snap 29 seq_num 136) dirty exists 2024-09-18T14:00:42.432 INFO:tasks.rados.rados.0.smithi084.stdout:382: done (1 left) 2024-09-18T14:00:42.432 INFO:tasks.rados.rados.0.smithi084.stdout:384: snap_create 2024-09-18T14:00:42.438 INFO:tasks.rados.rados.0.smithi084.stdout:383: finishing copy_from racing read to smithi08434663-46 2024-09-18T14:00:42.440 INFO:tasks.rados.rados.0.smithi084.stdout:383: finishing copy_from to smithi08434663-46 2024-09-18T14:00:42.441 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 18 (ObjNum 72 snap 6 seq_num 72) dirty exists 2024-09-18T14:00:43.287 INFO:tasks.rados.rados.0.smithi084.stdout:383: done (1 left) 2024-09-18T14:00:43.287 INFO:tasks.rados.rados.0.smithi084.stdout:384: done (0 left) 2024-09-18T14:00:43.287 INFO:tasks.rados.rados.0.smithi084.stdout:385: copy_from oid 16 from oid 30 current snap is 39 2024-09-18T14:00:43.287 INFO:tasks.rados.rados.0.smithi084.stdout:386: copy_from oid 22 from oid 20 current snap is 39 2024-09-18T14:00:43.287 INFO:tasks.rados.rados.0.smithi084.stdout:387: snap_create 2024-09-18T14:00:43.294 INFO:tasks.rados.rados.0.smithi084.stdout:386: finishing copy_from racing read to smithi08434663-22 2024-09-18T14:00:43.294 INFO:tasks.rados.rados.0.smithi084.stdout:385: finishing copy_from racing read to smithi08434663-16 2024-09-18T14:00:43.297 INFO:tasks.rados.rados.0.smithi084.stdout:385: finishing copy_from to smithi08434663-16 2024-09-18T14:00:43.297 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 36 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T14:00:43.297 INFO:tasks.rados.rados.0.smithi084.stdout:386: finishing copy_from to smithi08434663-22 2024-09-18T14:00:43.297 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 61 (ObjNum 128 snap 25 seq_num 128) dirty exists 2024-09-18T14:00:44.294 INFO:tasks.rados.rados.0.smithi084.stdout:385: done (2 left) 2024-09-18T14:00:44.294 INFO:tasks.rados.rados.0.smithi084.stdout:386: done (1 left) 2024-09-18T14:00:44.294 INFO:tasks.rados.rados.0.smithi084.stdout:387: done (0 left) 2024-09-18T14:00:44.294 INFO:tasks.rados.rados.0.smithi084.stdout:388: delete oid 33 current snap is 40 2024-09-18T14:00:44.297 INFO:tasks.rados.rados.0.smithi084.stdout:388: done (0 left) 2024-09-18T14:00:44.297 INFO:tasks.rados.rados.0.smithi084.stdout:389: snap_create 2024-09-18T14:00:45.295 INFO:tasks.rados.rados.0.smithi084.stdout:389: done (0 left) 2024-09-18T14:00:45.295 INFO:tasks.rados.rados.0.smithi084.stdout:390: write oid 34 current snap is 41 2024-09-18T14:00:45.296 INFO:tasks.rados.rados.0.smithi084.stdout:390: seq_num 173 ranges {507611=777843,2073973=444721} 2024-09-18T14:00:45.302 INFO:tasks.rados.rados.0.smithi084.stdout:390: writing smithi08434663-34 from 507611 to 1285454 tid 1 2024-09-18T14:00:45.309 INFO:tasks.rados.rados.0.smithi084.stdout:390: writing smithi08434663-34 from 2073973 to 2518694 tid 2 2024-09-18T14:00:45.309 INFO:tasks.rados.rados.0.smithi084.stdout:391: copy_from oid 5 from oid 1 current snap is 41 2024-09-18T14:00:45.309 INFO:tasks.rados.rados.0.smithi084.stdout:390: finishing write tid 1 to smithi08434663-34 2024-09-18T14:00:45.309 INFO:tasks.rados.rados.0.smithi084.stdout:392: snap_remove snap 36 2024-09-18T14:00:46.306 INFO:tasks.rados.rados.0.smithi084.stdout:392: done (2 left) 2024-09-18T14:00:46.307 INFO:tasks.rados.rados.0.smithi084.stdout:393: read oid 26 snap 39 2024-09-18T14:00:46.307 INFO:tasks.rados.rados.0.smithi084.stdout:393: expect (ObjNum 116 snap 22 seq_num 116) 2024-09-18T14:00:46.307 INFO:tasks.rados.rados.0.smithi084.stdout:394: copy_from oid 19 from oid 21 current snap is 41 2024-09-18T14:00:46.307 INFO:tasks.rados.rados.0.smithi084.stdout:390: finishing write tid 2 to smithi08434663-34 2024-09-18T14:00:46.307 INFO:tasks.rados.rados.0.smithi084.stdout:390: finishing write tid 3 to smithi08434663-34 2024-09-18T14:00:46.307 INFO:tasks.rados.rados.0.smithi084.stdout:390: finishing write tid 4 to smithi08434663-34 2024-09-18T14:00:46.307 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 89 (ObjNum 173 snap 41 seq_num 173) dirty exists 2024-09-18T14:00:46.307 INFO:tasks.rados.rados.0.smithi084.stdout:390: left oid 34 (ObjNum 173 snap 41 seq_num 173) 2024-09-18T14:00:46.307 INFO:tasks.rados.rados.0.smithi084.stdout:391: finishing copy_from racing read to smithi08434663-5 2024-09-18T14:00:46.307 INFO:tasks.rados.rados.0.smithi084.stdout:391: finishing copy_from to smithi08434663-5 2024-09-18T14:00:46.307 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 107 (ObjNum 138 snap 29 seq_num 138) dirty exists 2024-09-18T14:00:46.308 INFO:tasks.rados.rados.0.smithi084.stdout:390: done (3 left) 2024-09-18T14:00:46.308 INFO:tasks.rados.rados.0.smithi084.stdout:391: done (2 left) 2024-09-18T14:00:46.308 INFO:tasks.rados.rados.0.smithi084.stdout:395: delete oid 44 current snap is 41 2024-09-18T14:00:46.312 INFO:tasks.rados.rados.0.smithi084.stdout:395: done (2 left) 2024-09-18T14:00:46.312 INFO:tasks.rados.rados.0.smithi084.stdout:396: delete oid 35 current snap is 41 2024-09-18T14:00:46.316 INFO:tasks.rados.rados.0.smithi084.stdout:396: done (2 left) 2024-09-18T14:00:46.316 INFO:tasks.rados.rados.0.smithi084.stdout:397: read oid 36 snap 35 2024-09-18T14:00:46.316 INFO:tasks.rados.rados.0.smithi084.stdout:397: expect (ObjNum 135 snap 28 seq_num 135) 2024-09-18T14:00:46.316 INFO:tasks.rados.rados.0.smithi084.stdout:398: rollback oid 34 current snap is 41 2024-09-18T14:00:46.316 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 34 to 39 2024-09-18T14:00:46.316 INFO:tasks.rados.rados.0.smithi084.stdout:399: read oid 35 snap -1 2024-09-18T14:00:46.316 INFO:tasks.rados.rados.0.smithi084.stdout:399: expect deleted 2024-09-18T14:00:46.316 INFO:tasks.rados.rados.0.smithi084.stdout:400: snap_remove snap 37 2024-09-18T14:00:47.150 INFO:tasks.rados.rados.0.smithi084.stdout:400: done (5 left) 2024-09-18T14:00:47.151 INFO:tasks.rados.rados.0.smithi084.stdout:401: delete oid 28 current snap is 41 2024-09-18T14:00:47.151 INFO:tasks.rados.rados.0.smithi084.stdout:398: finishing rollback tid 0 to smithi08434663-34 2024-09-18T14:00:47.151 INFO:tasks.rados.rados.0.smithi084.stdout:398: finishing rollback tid 1 to smithi08434663-34 2024-09-18T14:00:47.151 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 91 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:00:47.151 INFO:tasks.rados.rados.0.smithi084.stdout:393: expect (ObjNum 116 snap 22 seq_num 116) 2024-09-18T14:00:47.286 INFO:tasks.rados.rados.0.smithi084.stdout:394: finishing copy_from racing read to smithi08434663-19 2024-09-18T14:00:47.286 INFO:tasks.rados.rados.0.smithi084.stdout:397: expect (ObjNum 135 snap 28 seq_num 135) 2024-09-18T14:00:47.390 INFO:tasks.rados.rados.0.smithi084.stdout:394: finishing copy_from to smithi08434663-19 2024-09-18T14:00:47.390 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 76 (ObjNum 96 snap 11 seq_num 96) dirty exists 2024-09-18T14:00:47.390 INFO:tasks.rados.rados.0.smithi084.stdout:393: done (5 left) 2024-09-18T14:00:47.390 INFO:tasks.rados.rados.0.smithi084.stdout:394: done (4 left) 2024-09-18T14:00:47.390 INFO:tasks.rados.rados.0.smithi084.stdout:397: done (3 left) 2024-09-18T14:00:47.390 INFO:tasks.rados.rados.0.smithi084.stdout:398: done (2 left) 2024-09-18T14:00:47.391 INFO:tasks.rados.rados.0.smithi084.stdout:399: done (1 left) 2024-09-18T14:00:47.391 INFO:tasks.rados.rados.0.smithi084.stdout:401: done (0 left) 2024-09-18T14:00:47.391 INFO:tasks.rados.rados.0.smithi084.stdout:402: write (excl) oid 6 current snap is 41 2024-09-18T14:00:47.391 INFO:tasks.rados.rados.0.smithi084.stdout:402: seq_num 177 ranges {574839=795192,1907901=452059} 2024-09-18T14:00:47.397 INFO:tasks.rados.rados.0.smithi084.stdout:402: writing smithi08434663-6 from 574839 to 1370031 tid 1 2024-09-18T14:00:47.403 INFO:tasks.rados.rados.0.smithi084.stdout:402: writing smithi08434663-6 from 1907901 to 2359960 tid 2 2024-09-18T14:00:47.403 INFO:tasks.rados.rados.0.smithi084.stdout:403: read oid 14 snap 33 2024-09-18T14:00:47.403 INFO:tasks.rados.rados.0.smithi084.stdout:403: expect deleted 2024-09-18T14:00:47.403 INFO:tasks.rados.rados.0.smithi084.stdout:404: read oid 1 snap -1 2024-09-18T14:00:47.403 INFO:tasks.rados.rados.0.smithi084.stdout:404: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:00:47.403 INFO:tasks.rados.rados.0.smithi084.stdout:405: read oid 10 snap 32 2024-09-18T14:00:47.403 INFO:tasks.rados.rados.0.smithi084.stdout:405: expect (ObjNum 72 snap 6 seq_num 72) 2024-09-18T14:00:47.403 INFO:tasks.rados.rados.0.smithi084.stdout:406: write oid 19 current snap is 41 2024-09-18T14:00:47.403 INFO:tasks.rados.rados.0.smithi084.stdout:406: seq_num 178 ranges {591646=470441,1566383=595717,2888822=102824} 2024-09-18T14:00:47.409 INFO:tasks.rados.rados.0.smithi084.stdout:406: writing smithi08434663-19 from 591646 to 1062087 tid 1 2024-09-18T14:00:47.415 INFO:tasks.rados.rados.0.smithi084.stdout:406: writing smithi08434663-19 from 1566383 to 2162100 tid 2 2024-09-18T14:00:47.420 INFO:tasks.rados.rados.0.smithi084.stdout:406: writing smithi08434663-19 from 2888822 to 2991646 tid 3 2024-09-18T14:00:47.420 INFO:tasks.rados.rados.0.smithi084.stdout:407: write oid 44 current snap is 41 2024-09-18T14:00:47.420 INFO:tasks.rados.rados.0.smithi084.stdout:407: seq_num 179 ranges {608453=545690,1624865=455728,2632598=375855} 2024-09-18T14:00:47.426 INFO:tasks.rados.rados.0.smithi084.stdout:407: writing smithi08434663-44 from 608453 to 1154143 tid 1 2024-09-18T14:00:47.431 INFO:tasks.rados.rados.0.smithi084.stdout:407: writing smithi08434663-44 from 1624865 to 2080593 tid 2 2024-09-18T14:00:47.436 INFO:tasks.rados.rados.0.smithi084.stdout:407: writing smithi08434663-44 from 2632598 to 3008453 tid 3 2024-09-18T14:00:47.437 INFO:tasks.rados.rados.0.smithi084.stdout:408: snap_remove snap 34 2024-09-18T14:00:48.312 INFO:tasks.rados.rados.0.smithi084.stdout:408: done (6 left) 2024-09-18T14:00:48.312 INFO:tasks.rados.rados.0.smithi084.stdout:409: write (excl) oid 18 current snap is 41 2024-09-18T14:00:48.312 INFO:tasks.rados.rados.0.smithi084.stdout:409: seq_num 180 ranges {625260=620939,1683347=599386,2943668=81592} 2024-09-18T14:00:48.318 INFO:tasks.rados.rados.0.smithi084.stdout:409: writing smithi08434663-18 from 625260 to 1246199 tid 1 2024-09-18T14:00:48.325 INFO:tasks.rados.rados.0.smithi084.stdout:409: writing smithi08434663-18 from 1683347 to 2282733 tid 2 2024-09-18T14:00:48.329 INFO:tasks.rados.rados.0.smithi084.stdout:409: writing smithi08434663-18 from 2943668 to 3025260 tid 3 2024-09-18T14:00:48.330 INFO:tasks.rados.rados.0.smithi084.stdout:410: delete oid 35 current snap is 41 2024-09-18T14:00:48.330 INFO:tasks.rados.rados.0.smithi084.stdout:402: finishing write tid 1 to smithi08434663-6 2024-09-18T14:00:48.330 INFO:tasks.rados.rados.0.smithi084.stdout:402: finishing write tid 2 to smithi08434663-6 2024-09-18T14:00:48.330 INFO:tasks.rados.rados.0.smithi084.stdout:402: finishing write tid 3 to smithi08434663-6 2024-09-18T14:00:48.330 INFO:tasks.rados.rados.0.smithi084.stdout:402: finishing write tid 4 to smithi08434663-6 2024-09-18T14:00:48.330 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 72 (ObjNum 177 snap 41 seq_num 177) dirty exists 2024-09-18T14:00:48.330 INFO:tasks.rados.rados.0.smithi084.stdout:402: left oid 6 (ObjNum 177 snap 41 seq_num 177) 2024-09-18T14:00:48.330 INFO:tasks.rados.rados.0.smithi084.stdout:404: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:00:48.404 INFO:tasks.rados.rados.0.smithi084.stdout:406: finishing write tid 1 to smithi08434663-19 2024-09-18T14:00:48.405 INFO:tasks.rados.rados.0.smithi084.stdout:405: expect (ObjNum 72 snap 6 seq_num 72) 2024-09-18T14:00:48.534 INFO:tasks.rados.rados.0.smithi084.stdout:406: finishing write tid 2 to smithi08434663-19 2024-09-18T14:00:48.534 INFO:tasks.rados.rados.0.smithi084.stdout:406: finishing write tid 3 to smithi08434663-19 2024-09-18T14:00:48.534 INFO:tasks.rados.rados.0.smithi084.stdout:406: finishing write tid 4 to smithi08434663-19 2024-09-18T14:00:48.534 INFO:tasks.rados.rados.0.smithi084.stdout:406: finishing write tid 5 to smithi08434663-19 2024-09-18T14:00:48.534 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 172 (ObjNum 178 snap 41 seq_num 178) dirty exists 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:406: left oid 19 (ObjNum 178 snap 41 seq_num 178) 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:407: finishing write tid 1 to smithi08434663-44 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:402: done (7 left) 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:403: done (6 left) 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:404: done (5 left) 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:405: done (4 left) 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:406: done (3 left) 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:410: done (2 left) 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:411: snap_create 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:407: finishing write tid 2 to smithi08434663-44 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:407: finishing write tid 3 to smithi08434663-44 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:407: finishing write tid 4 to smithi08434663-44 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:407: finishing write tid 5 to smithi08434663-44 2024-09-18T14:00:48.535 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 112 (ObjNum 179 snap 41 seq_num 179) dirty exists 2024-09-18T14:00:48.536 INFO:tasks.rados.rados.0.smithi084.stdout:407: left oid 44 (ObjNum 179 snap 41 seq_num 179) 2024-09-18T14:00:48.536 INFO:tasks.rados.rados.0.smithi084.stdout:409: finishing write tid 1 to smithi08434663-18 2024-09-18T14:00:48.536 INFO:tasks.rados.rados.0.smithi084.stdout:409: finishing write tid 2 to smithi08434663-18 2024-09-18T14:00:48.536 INFO:tasks.rados.rados.0.smithi084.stdout:409: finishing write tid 3 to smithi08434663-18 2024-09-18T14:00:48.536 INFO:tasks.rados.rados.0.smithi084.stdout:409: finishing write tid 4 to smithi08434663-18 2024-09-18T14:00:48.536 INFO:tasks.rados.rados.0.smithi084.stdout:409: finishing write tid 5 to smithi08434663-18 2024-09-18T14:00:48.536 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 66 (ObjNum 180 snap 41 seq_num 180) dirty exists 2024-09-18T14:00:48.536 INFO:tasks.rados.rados.0.smithi084.stdout:409: left oid 18 (ObjNum 180 snap 41 seq_num 180) 2024-09-18T14:00:49.316 INFO:tasks.rados.rados.0.smithi084.stdout:407: done (2 left) 2024-09-18T14:00:49.316 INFO:tasks.rados.rados.0.smithi084.stdout:409: done (1 left) 2024-09-18T14:00:49.316 INFO:tasks.rados.rados.0.smithi084.stdout:411: done (0 left) 2024-09-18T14:00:49.317 INFO:tasks.rados.rados.0.smithi084.stdout:412: copy_from oid 24 from oid 32 current snap is 42 2024-09-18T14:00:49.317 INFO:tasks.rados.rados.0.smithi084.stdout:413: snap_remove snap 38 2024-09-18T14:00:50.325 INFO:tasks.rados.rados.0.smithi084.stdout:413: done (1 left) 2024-09-18T14:00:50.325 INFO:tasks.rados.rados.0.smithi084.stdout:414: rollback oid 36 current snap is 42 2024-09-18T14:00:50.326 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 36 to 40 2024-09-18T14:00:50.326 INFO:tasks.rados.rados.0.smithi084.stdout:412: finishing copy_from racing read to smithi08434663-24 2024-09-18T14:00:50.326 INFO:tasks.rados.rados.0.smithi084.stdout:412: finishing copy_from to smithi08434663-24 2024-09-18T14:00:50.326 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 159 (ObjNum 158 snap 35 seq_num 158) dirty exists 2024-09-18T14:00:50.326 INFO:tasks.rados.rados.0.smithi084.stdout:412: done (1 left) 2024-09-18T14:00:50.326 INFO:tasks.rados.rados.0.smithi084.stdout:415: copy_from oid 11 from oid 19 current snap is 42 2024-09-18T14:00:50.326 INFO:tasks.rados.rados.0.smithi084.stdout:416: snap_remove snap 32 2024-09-18T14:00:51.331 INFO:tasks.rados.rados.0.smithi084.stdout:416: done (2 left) 2024-09-18T14:00:51.331 INFO:tasks.rados.rados.0.smithi084.stdout:417: snap_remove snap 33 2024-09-18T14:00:52.339 INFO:tasks.rados.rados.0.smithi084.stdout:417: done (2 left) 2024-09-18T14:00:52.339 INFO:tasks.rados.rados.0.smithi084.stdout:418: write (excl) oid 40 current snap is 42 2024-09-18T14:00:52.339 INFO:tasks.rados.rados.0.smithi084.stdout:418: seq_num 182 ranges {658874=771437,2083958=603055} 2024-09-18T14:00:52.347 INFO:tasks.rados.rados.0.smithi084.stdout:418: writing smithi08434663-40 from 658874 to 1430311 tid 1 2024-09-18T14:00:52.355 INFO:tasks.rados.rados.0.smithi084.stdout:418: writing smithi08434663-40 from 2083958 to 2687013 tid 2 2024-09-18T14:00:52.355 INFO:tasks.rados.rados.0.smithi084.stdout:419: rollback oid 45 current snap is 42 2024-09-18T14:00:52.355 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 45 to 40 2024-09-18T14:00:52.355 INFO:tasks.rados.rados.0.smithi084.stdout:414: finishing rollback tid 0 to smithi08434663-36 2024-09-18T14:00:52.355 INFO:tasks.rados.rados.0.smithi084.stdout:414: finishing rollback tid 1 to smithi08434663-36 2024-09-18T14:00:52.356 INFO:tasks.rados.rados.0.smithi084.stdout:414: finishing rollback tid 2 to smithi08434663-36 2024-09-18T14:00:52.356 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 109 (ObjNum 96 snap 11 seq_num 96) dirty exists 2024-09-18T14:00:52.356 INFO:tasks.rados.rados.0.smithi084.stdout:415: finishing copy_from racing read to smithi08434663-11 2024-09-18T14:00:52.356 INFO:tasks.rados.rados.0.smithi084.stdout:415: finishing copy_from to smithi08434663-11 2024-09-18T14:00:52.356 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 172 (ObjNum 178 snap 41 seq_num 178) dirty exists 2024-09-18T14:00:52.356 INFO:tasks.rados.rados.0.smithi084.stdout:418: finishing write tid 1 to smithi08434663-40 2024-09-18T14:00:52.356 INFO:tasks.rados.rados.0.smithi084.stdout:414: done (3 left) 2024-09-18T14:00:52.356 INFO:tasks.rados.rados.0.smithi084.stdout:415: done (2 left) 2024-09-18T14:00:52.356 INFO:tasks.rados.rados.0.smithi084.stdout:420: write oid 46 current snap is 42 2024-09-18T14:00:52.356 INFO:tasks.rados.rados.0.smithi084.stdout:420: seq_num 183 ranges {675681=563039,1858793=746713,3025937=49744} 2024-09-18T14:00:52.361 INFO:tasks.rados.rados.0.smithi084.stdout:420: writing smithi08434663-46 from 675681 to 1238720 tid 1 2024-09-18T14:00:52.373 INFO:tasks.rados.rados.0.smithi084.stdout:420: writing smithi08434663-46 from 1858793 to 2605506 tid 2 2024-09-18T14:00:52.373 INFO:tasks.rados.rados.0.smithi084.stdout:420: writing smithi08434663-46 from 3025937 to 3075681 tid 3 2024-09-18T14:00:52.373 INFO:tasks.rados.rados.0.smithi084.stdout:421: read oid 44 snap -1 2024-09-18T14:00:52.374 INFO:tasks.rados.rados.0.smithi084.stdout:421: expect (ObjNum 179 snap 41 seq_num 179) 2024-09-18T14:00:52.374 INFO:tasks.rados.rados.0.smithi084.stdout:422: snap_remove snap 35 2024-09-18T14:00:53.339 INFO:tasks.rados.rados.0.smithi084.stdout:422: done (4 left) 2024-09-18T14:00:53.339 INFO:tasks.rados.rados.0.smithi084.stdout:423: write (excl) oid 7 current snap is 42 2024-09-18T14:00:53.339 INFO:tasks.rados.rados.0.smithi084.stdout:423: seq_num 184 ranges {692488=638288,1917275=606724} 2024-09-18T14:00:53.347 INFO:tasks.rados.rados.0.smithi084.stdout:423: writing smithi08434663-7 from 692488 to 1330776 tid 1 2024-09-18T14:00:53.354 INFO:tasks.rados.rados.0.smithi084.stdout:423: writing smithi08434663-7 from 1917275 to 2523999 tid 2 2024-09-18T14:00:53.354 INFO:tasks.rados.rados.0.smithi084.stdout:424: write oid 19 current snap is 42 2024-09-18T14:00:53.354 INFO:tasks.rados.rados.0.smithi084.stdout:419: finishing rollback tid 0 to smithi08434663-45 2024-09-18T14:00:53.354 INFO:tasks.rados.rados.0.smithi084.stdout:418: finishing write tid 2 to smithi08434663-40 2024-09-18T14:00:53.354 INFO:tasks.rados.rados.0.smithi084.stdout:419: finishing rollback tid 1 to smithi08434663-45 2024-09-18T14:00:53.355 INFO:tasks.rados.rados.0.smithi084.stdout:419: finishing rollback tid 2 to smithi08434663-45 2024-09-18T14:00:53.355 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 95 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T14:00:53.355 INFO:tasks.rados.rados.0.smithi084.stdout:418: finishing write tid 3 to smithi08434663-40 2024-09-18T14:00:53.355 INFO:tasks.rados.rados.0.smithi084.stdout:418: finishing write tid 4 to smithi08434663-40 2024-09-18T14:00:53.355 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 94 (ObjNum 182 snap 42 seq_num 182) dirty exists 2024-09-18T14:00:53.355 INFO:tasks.rados.rados.0.smithi084.stdout:418: left oid 40 (ObjNum 182 snap 42 seq_num 182) 2024-09-18T14:00:53.355 INFO:tasks.rados.rados.0.smithi084.stdout:420: finishing write tid 1 to smithi08434663-46 2024-09-18T14:00:53.355 INFO:tasks.rados.rados.0.smithi084.stdout:420: finishing write tid 2 to smithi08434663-46 2024-09-18T14:00:53.355 INFO:tasks.rados.rados.0.smithi084.stdout:420: finishing write tid 3 to smithi08434663-46 2024-09-18T14:00:53.355 INFO:tasks.rados.rados.0.smithi084.stdout:420: finishing write tid 4 to smithi08434663-46 2024-09-18T14:00:53.355 INFO:tasks.rados.rados.0.smithi084.stdout:424: seq_num 185 ranges {709295=713537,1859404=750382} 2024-09-18T14:00:53.361 INFO:tasks.rados.rados.0.smithi084.stdout:424: writing smithi08434663-19 from 709295 to 1422832 tid 1 2024-09-18T14:00:53.369 INFO:tasks.rados.rados.0.smithi084.stdout:424: writing smithi08434663-19 from 1859404 to 2609786 tid 2 2024-09-18T14:00:53.369 INFO:tasks.rados.rados.0.smithi084.stdout:418: done (5 left) 2024-09-18T14:00:53.369 INFO:tasks.rados.rados.0.smithi084.stdout:419: done (4 left) 2024-09-18T14:00:53.369 INFO:tasks.rados.rados.0.smithi084.stdout:425: rollback oid 6 current snap is 42 2024-09-18T14:00:53.369 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 39 2024-09-18T14:00:53.369 INFO:tasks.rados.rados.0.smithi084.stdout:420: finishing write tid 5 to smithi08434663-46 2024-09-18T14:00:53.369 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 153 (ObjNum 183 snap 42 seq_num 183) dirty exists 2024-09-18T14:00:53.370 INFO:tasks.rados.rados.0.smithi084.stdout:420: left oid 46 (ObjNum 183 snap 42 seq_num 183) 2024-09-18T14:00:53.370 INFO:tasks.rados.rados.0.smithi084.stdout:421: expect (ObjNum 179 snap 41 seq_num 179) 2024-09-18T14:00:53.466 INFO:tasks.rados.rados.0.smithi084.stdout:423: finishing write tid 1 to smithi08434663-7 2024-09-18T14:00:53.466 INFO:tasks.rados.rados.0.smithi084.stdout:423: finishing write tid 2 to smithi08434663-7 2024-09-18T14:00:53.466 INFO:tasks.rados.rados.0.smithi084.stdout:423: finishing write tid 3 to smithi08434663-7 2024-09-18T14:00:53.466 INFO:tasks.rados.rados.0.smithi084.stdout:423: finishing write tid 4 to smithi08434663-7 2024-09-18T14:00:53.466 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 75 (ObjNum 184 snap 42 seq_num 184) dirty exists 2024-09-18T14:00:53.466 INFO:tasks.rados.rados.0.smithi084.stdout:423: left oid 7 (ObjNum 184 snap 42 seq_num 184) 2024-09-18T14:00:53.466 INFO:tasks.rados.rados.0.smithi084.stdout:424: finishing write tid 1 to smithi08434663-19 2024-09-18T14:00:53.466 INFO:tasks.rados.rados.0.smithi084.stdout:425: finishing rollback tid 0 to smithi08434663-6 2024-09-18T14:00:53.466 INFO:tasks.rados.rados.0.smithi084.stdout:425: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:00:53.466 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 78 (ObjNum 129 snap 0 seq_num 2494528112) dirty dne 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:420: done (4 left) 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:421: done (3 left) 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:423: done (2 left) 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:425: done (1 left) 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:426: rollback oid 27 current snap is 42 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 27 to 39 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:424: finishing write tid 2 to smithi08434663-19 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:424: finishing write tid 3 to smithi08434663-19 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:424: finishing write tid 4 to smithi08434663-19 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 202 (ObjNum 185 snap 42 seq_num 185) dirty exists 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:424: left oid 19 (ObjNum 185 snap 42 seq_num 185) 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:424: done (1 left) 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:427: read oid 26 snap -1 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:427: expect (ObjNum 116 snap 22 seq_num 116) 2024-09-18T14:00:53.467 INFO:tasks.rados.rados.0.smithi084.stdout:428: read oid 30 snap -1 2024-09-18T14:00:53.468 INFO:tasks.rados.rados.0.smithi084.stdout:428: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:00:53.468 INFO:tasks.rados.rados.0.smithi084.stdout:429: read oid 32 snap -1 2024-09-18T14:00:53.468 INFO:tasks.rados.rados.0.smithi084.stdout:429: expect (ObjNum 158 snap 35 seq_num 158) 2024-09-18T14:00:53.468 INFO:tasks.rados.rados.0.smithi084.stdout:430: copy_from oid 44 from oid 43 current snap is 42 2024-09-18T14:00:53.468 INFO:tasks.rados.rados.0.smithi084.stdout:431: write (excl) oid 13 current snap is 42 2024-09-18T14:00:53.468 INFO:tasks.rados.rados.0.smithi084.stdout:431: seq_num 186 ranges {726102=788786,1917886=610393,3108206=17896} 2024-09-18T14:00:53.476 INFO:tasks.rados.rados.0.smithi084.stdout:431: writing smithi08434663-13 from 726102 to 1514888 tid 1 2024-09-18T14:00:53.482 INFO:tasks.rados.rados.0.smithi084.stdout:431: writing smithi08434663-13 from 1917886 to 2528279 tid 2 2024-09-18T14:00:53.486 INFO:tasks.rados.rados.0.smithi084.stdout:431: writing smithi08434663-13 from 3108206 to 3126102 tid 3 2024-09-18T14:00:53.486 INFO:tasks.rados.rados.0.smithi084.stdout:432: read oid 16 snap 41 2024-09-18T14:00:53.486 INFO:tasks.rados.rados.0.smithi084.stdout:432: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:00:53.487 INFO:tasks.rados.rados.0.smithi084.stdout:433: copy_from oid 20 from oid 4 current snap is 42 2024-09-18T14:00:53.487 INFO:tasks.rados.rados.0.smithi084.stdout:429: expect (ObjNum 158 snap 35 seq_num 158) 2024-09-18T14:00:53.609 INFO:tasks.rados.rados.0.smithi084.stdout:428: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:00:53.677 INFO:tasks.rados.rados.0.smithi084.stdout:427: expect (ObjNum 116 snap 22 seq_num 116) 2024-09-18T14:00:53.807 INFO:tasks.rados.rados.0.smithi084.stdout:426: finishing rollback tid 0 to smithi08434663-27 2024-09-18T14:00:53.807 INFO:tasks.rados.rados.0.smithi084.stdout:426: finishing rollback tid 1 to smithi08434663-27 2024-09-18T14:00:53.807 INFO:tasks.rados.rados.0.smithi084.stdout:426: finishing rollback tid 2 to smithi08434663-27 2024-09-18T14:00:53.807 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 133 (ObjNum 103 snap 17 seq_num 103) dirty exists 2024-09-18T14:00:53.807 INFO:tasks.rados.rados.0.smithi084.stdout:430: finishing copy_from racing read to smithi08434663-44 2024-09-18T14:00:53.807 INFO:tasks.rados.rados.0.smithi084.stdout:430: finishing copy_from to smithi08434663-44 2024-09-18T14:00:53.807 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 78 (ObjNum 135 snap 28 seq_num 135) dirty exists 2024-09-18T14:00:53.807 INFO:tasks.rados.rados.0.smithi084.stdout:431: finishing write tid 1 to smithi08434663-13 2024-09-18T14:00:53.807 INFO:tasks.rados.rados.0.smithi084.stdout:431: finishing write tid 2 to smithi08434663-13 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:426: done (7 left) 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:427: done (6 left) 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:428: done (5 left) 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:429: done (4 left) 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:430: done (3 left) 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:434: delete oid 50 current snap is 42 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:433: finishing copy_from to smithi08434663-20 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:433: got expected ENOENT (src dne) 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:433: finishing copy_from racing read to smithi08434663-20 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:431: finishing write tid 3 to smithi08434663-13 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:431: finishing write tid 4 to smithi08434663-13 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:431: finishing write tid 5 to smithi08434663-13 2024-09-18T14:00:53.808 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 105 (ObjNum 186 snap 42 seq_num 186) dirty exists 2024-09-18T14:00:53.809 INFO:tasks.rados.rados.0.smithi084.stdout:431: left oid 13 (ObjNum 186 snap 42 seq_num 186) 2024-09-18T14:00:53.809 INFO:tasks.rados.rados.0.smithi084.stdout:432: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:00:53.875 INFO:tasks.rados.rados.0.smithi084.stdout:431: done (3 left) 2024-09-18T14:00:53.875 INFO:tasks.rados.rados.0.smithi084.stdout:432: done (2 left) 2024-09-18T14:00:53.875 INFO:tasks.rados.rados.0.smithi084.stdout:433: done (1 left) 2024-09-18T14:00:53.875 INFO:tasks.rados.rados.0.smithi084.stdout:434: done (0 left) 2024-09-18T14:00:53.875 INFO:tasks.rados.rados.0.smithi084.stdout:435: delete oid 5 current snap is 42 2024-09-18T14:00:53.877 INFO:tasks.rados.rados.0.smithi084.stdout:435: done (0 left) 2024-09-18T14:00:53.877 INFO:tasks.rados.rados.0.smithi084.stdout:436: snap_remove snap 41 2024-09-18T14:00:54.354 INFO:tasks.rados.rados.0.smithi084.stdout:436: done (0 left) 2024-09-18T14:00:54.354 INFO:tasks.rados.rados.0.smithi084.stdout:437: rollback oid 44 current snap is 42 2024-09-18T14:00:54.355 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 44 to 39 2024-09-18T14:00:54.355 INFO:tasks.rados.rados.0.smithi084.stdout:438: copy_from oid 11 from oid 5 current snap is 42 2024-09-18T14:00:54.355 INFO:tasks.rados.rados.0.smithi084.stdout:439: read oid 39 snap 40 2024-09-18T14:00:54.355 INFO:tasks.rados.rados.0.smithi084.stdout:439: expect (ObjNum 148 snap 30 seq_num 148) 2024-09-18T14:00:54.355 INFO:tasks.rados.rados.0.smithi084.stdout:440: snap_create 2024-09-18T14:00:54.367 INFO:tasks.rados.rados.0.smithi084.stdout:438: finishing copy_from to smithi08434663-11 2024-09-18T14:00:54.367 INFO:tasks.rados.rados.0.smithi084.stdout:438: got expected ENOENT (src dne) 2024-09-18T14:00:54.367 INFO:tasks.rados.rados.0.smithi084.stdout:438: finishing copy_from racing read to smithi08434663-11 2024-09-18T14:00:54.368 INFO:tasks.rados.rados.0.smithi084.stdout:439: expect (ObjNum 148 snap 30 seq_num 148) 2024-09-18T14:00:54.509 INFO:tasks.rados.rados.0.smithi084.stdout:437: finishing rollback tid 0 to smithi08434663-44 2024-09-18T14:00:54.509 INFO:tasks.rados.rados.0.smithi084.stdout:437: finishing rollback tid 1 to smithi08434663-44 2024-09-18T14:00:54.509 INFO:tasks.rados.rados.0.smithi084.stdout:437: finishing rollback tid 2 to smithi08434663-44 2024-09-18T14:00:54.509 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 116 (ObjNum 88 snap 9 seq_num 88) dirty exists 2024-09-18T14:00:55.357 INFO:tasks.rados.rados.0.smithi084.stdout:437: done (3 left) 2024-09-18T14:00:55.357 INFO:tasks.rados.rados.0.smithi084.stdout:438: done (2 left) 2024-09-18T14:00:55.357 INFO:tasks.rados.rados.0.smithi084.stdout:439: done (1 left) 2024-09-18T14:00:55.357 INFO:tasks.rados.rados.0.smithi084.stdout:440: done (0 left) 2024-09-18T14:00:55.357 INFO:tasks.rados.rados.0.smithi084.stdout:441: read oid 47 snap -1 2024-09-18T14:00:55.357 INFO:tasks.rados.rados.0.smithi084.stdout:441: expect deleted 2024-09-18T14:00:55.357 INFO:tasks.rados.rados.0.smithi084.stdout:442: snap_remove snap 42 2024-09-18T14:00:56.366 INFO:tasks.rados.rados.0.smithi084.stdout:442: done (1 left) 2024-09-18T14:00:56.366 INFO:tasks.rados.rados.0.smithi084.stdout:443: snap_create 2024-09-18T14:00:57.153 INFO:tasks.rados.rados.0.smithi084.stdout:441: done (1 left) 2024-09-18T14:00:57.153 INFO:tasks.rados.rados.0.smithi084.stdout:443: done (0 left) 2024-09-18T14:00:57.153 INFO:tasks.rados.rados.0.smithi084.stdout:444: rollback oid 7 current snap is 44 2024-09-18T14:00:57.153 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 7 to 39 2024-09-18T14:00:57.154 INFO:tasks.rados.rados.0.smithi084.stdout:445: read oid 32 snap 40 2024-09-18T14:00:57.154 INFO:tasks.rados.rados.0.smithi084.stdout:445: expect (ObjNum 158 snap 35 seq_num 158) 2024-09-18T14:00:57.154 INFO:tasks.rados.rados.0.smithi084.stdout:446: read oid 46 snap -1 2024-09-18T14:00:57.154 INFO:tasks.rados.rados.0.smithi084.stdout:446: expect (ObjNum 183 snap 42 seq_num 183) 2024-09-18T14:00:57.154 INFO:tasks.rados.rados.0.smithi084.stdout:447: rollback oid 35 current snap is 44 2024-09-18T14:00:57.154 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 35 to 43 2024-09-18T14:00:57.154 INFO:tasks.rados.rados.0.smithi084.stdout:448: rollback oid 45 current snap is 44 2024-09-18T14:00:57.154 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 45 to 39 2024-09-18T14:00:57.154 INFO:tasks.rados.rados.0.smithi084.stdout:449: write oid 13 current snap is 44 2024-09-18T14:00:57.154 INFO:tasks.rados.rados.0.smithi084.stdout:449: seq_num 189 ranges {776523=614533,1976979=757720} 2024-09-18T14:00:57.161 INFO:tasks.rados.rados.0.smithi084.stdout:449: writing smithi08434663-13 from 776523 to 1391056 tid 1 2024-09-18T14:00:57.170 INFO:tasks.rados.rados.0.smithi084.stdout:449: writing smithi08434663-13 from 1976979 to 2734699 tid 2 2024-09-18T14:00:57.170 INFO:tasks.rados.rados.0.smithi084.stdout:450: read oid 37 snap -1 2024-09-18T14:00:57.171 INFO:tasks.rados.rados.0.smithi084.stdout:450: expect (ObjNum 91 snap 9 seq_num 91) 2024-09-18T14:00:57.171 INFO:tasks.rados.rados.0.smithi084.stdout:451: delete oid 20 current snap is 44 2024-09-18T14:00:57.171 INFO:tasks.rados.rados.0.smithi084.stdout:444: finishing rollback tid 0 to smithi08434663-7 2024-09-18T14:00:57.171 INFO:tasks.rados.rados.0.smithi084.stdout:444: finishing rollback tid 1 to smithi08434663-7 2024-09-18T14:00:57.171 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 83 (ObjNum 257 snap 0 seq_num 3269434832) dirty dne 2024-09-18T14:00:57.171 INFO:tasks.rados.rados.0.smithi084.stdout:445: expect (ObjNum 158 snap 35 seq_num 158) 2024-09-18T14:00:57.293 INFO:tasks.rados.rados.0.smithi084.stdout:448: finishing rollback tid 0 to smithi08434663-45 2024-09-18T14:00:57.293 INFO:tasks.rados.rados.0.smithi084.stdout:448: finishing rollback tid 1 to smithi08434663-45 2024-09-18T14:00:57.293 INFO:tasks.rados.rados.0.smithi084.stdout:448: finishing rollback tid 2 to smithi08434663-45 2024-09-18T14:00:57.293 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 103 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T14:00:57.293 INFO:tasks.rados.rados.0.smithi084.stdout:446: expect (ObjNum 183 snap 42 seq_num 183) 2024-09-18T14:00:57.450 INFO:tasks.rados.rados.0.smithi084.stdout:447: finishing rollback tid 1 to smithi08434663-35 2024-09-18T14:00:57.450 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 127 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-18T14:00:57.450 INFO:tasks.rados.rados.0.smithi084.stdout:444: done (7 left) 2024-09-18T14:00:57.450 INFO:tasks.rados.rados.0.smithi084.stdout:445: done (6 left) 2024-09-18T14:00:57.451 INFO:tasks.rados.rados.0.smithi084.stdout:446: done (5 left) 2024-09-18T14:00:57.451 INFO:tasks.rados.rados.0.smithi084.stdout:447: done (4 left) 2024-09-18T14:00:57.451 INFO:tasks.rados.rados.0.smithi084.stdout:448: done (3 left) 2024-09-18T14:00:57.451 INFO:tasks.rados.rados.0.smithi084.stdout:451: done (2 left) 2024-09-18T14:00:57.451 INFO:tasks.rados.rados.0.smithi084.stdout:452: snap_remove snap 39 2024-09-18T14:00:58.373 INFO:tasks.rados.rados.0.smithi084.stdout:452: done (2 left) 2024-09-18T14:00:58.373 INFO:tasks.rados.rados.0.smithi084.stdout:453: snap_remove snap 40 2024-09-18T14:00:59.374 INFO:tasks.rados.rados.0.smithi084.stdout:453: done (2 left) 2024-09-18T14:00:59.374 INFO:tasks.rados.rados.0.smithi084.stdout:454: snap_remove snap 43 2024-09-18T14:01:00.393 INFO:tasks.rados.rados.0.smithi084.stdout:454: done (2 left) 2024-09-18T14:01:00.477 INFO:tasks.rados.rados.0.smithi084.stdout:455: snap_create 2024-09-18T14:01:00.477 INFO:tasks.rados.rados.0.smithi084.stdout:449: finishing write tid 1 to smithi08434663-13 2024-09-18T14:01:00.477 INFO:tasks.rados.rados.0.smithi084.stdout:449: finishing write tid 2 to smithi08434663-13 2024-09-18T14:01:00.477 INFO:tasks.rados.rados.0.smithi084.stdout:449: finishing write tid 3 to smithi08434663-13 2024-09-18T14:01:00.477 INFO:tasks.rados.rados.0.smithi084.stdout:449: finishing write tid 4 to smithi08434663-13 2024-09-18T14:01:00.477 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 113 (ObjNum 189 snap 44 seq_num 189) dirty exists 2024-09-18T14:01:00.478 INFO:tasks.rados.rados.0.smithi084.stdout:449: left oid 13 (ObjNum 189 snap 44 seq_num 189) 2024-09-18T14:01:00.478 INFO:tasks.rados.rados.0.smithi084.stdout:450: expect (ObjNum 91 snap 9 seq_num 91) 2024-09-18T14:01:01.398 INFO:tasks.rados.rados.0.smithi084.stdout:449: done (2 left) 2024-09-18T14:01:01.398 INFO:tasks.rados.rados.0.smithi084.stdout:450: done (1 left) 2024-09-18T14:01:01.398 INFO:tasks.rados.rados.0.smithi084.stdout:455: done (0 left) 2024-09-18T14:01:01.398 INFO:tasks.rados.rados.0.smithi084.stdout:456: write oid 32 current snap is 45 2024-09-18T14:01:01.398 INFO:tasks.rados.rados.0.smithi084.stdout:456: seq_num 191 ranges {410137=481384,1410296=761389,2728968=481169} 2024-09-18T14:01:01.402 INFO:tasks.rados.rados.0.smithi084.stdout:456: writing smithi08434663-32 from 410137 to 891521 tid 1 2024-09-18T14:01:01.409 INFO:tasks.rados.rados.0.smithi084.stdout:456: writing smithi08434663-32 from 1410296 to 2171685 tid 2 2024-09-18T14:01:01.415 INFO:tasks.rados.rados.0.smithi084.stdout:456: writing smithi08434663-32 from 2728968 to 3210137 tid 3 2024-09-18T14:01:01.415 INFO:tasks.rados.rados.0.smithi084.stdout:457: read oid 7 snap -1 2024-09-18T14:01:01.416 INFO:tasks.rados.rados.0.smithi084.stdout:457: expect deleted 2024-09-18T14:01:01.416 INFO:tasks.rados.rados.0.smithi084.stdout:458: write oid 33 current snap is 45 2024-09-18T14:01:01.416 INFO:tasks.rados.rados.0.smithi084.stdout:458: seq_num 192 ranges {426944=556633,1468778=621400,2872744=354200} 2024-09-18T14:01:01.421 INFO:tasks.rados.rados.0.smithi084.stdout:458: writing smithi08434663-33 from 426944 to 983577 tid 1 2024-09-18T14:01:01.427 INFO:tasks.rados.rados.0.smithi084.stdout:458: writing smithi08434663-33 from 1468778 to 2090178 tid 2 2024-09-18T14:01:01.433 INFO:tasks.rados.rados.0.smithi084.stdout:458: writing smithi08434663-33 from 2872744 to 3226944 tid 3 2024-09-18T14:01:01.434 INFO:tasks.rados.rados.0.smithi084.stdout:459: read oid 36 snap -1 2024-09-18T14:01:01.434 INFO:tasks.rados.rados.0.smithi084.stdout:459: expect (ObjNum 96 snap 11 seq_num 96) 2024-09-18T14:01:01.434 INFO:tasks.rados.rados.0.smithi084.stdout:460: write oid 5 current snap is 45 2024-09-18T14:01:01.434 INFO:tasks.rados.rados.0.smithi084.stdout:456: finishing write tid 1 to smithi08434663-32 2024-09-18T14:01:01.434 INFO:tasks.rados.rados.0.smithi084.stdout:460: seq_num 193 ranges {443751=631882,1810907=765058,3067461=176290} 2024-09-18T14:01:01.439 INFO:tasks.rados.rados.0.smithi084.stdout:460: writing smithi08434663-5 from 443751 to 1075633 tid 1 2024-09-18T14:01:01.447 INFO:tasks.rados.rados.0.smithi084.stdout:460: writing smithi08434663-5 from 1810907 to 2575965 tid 2 2024-09-18T14:01:01.452 INFO:tasks.rados.rados.0.smithi084.stdout:460: writing smithi08434663-5 from 3067461 to 3243751 tid 3 2024-09-18T14:01:01.452 INFO:tasks.rados.rados.0.smithi084.stdout:461: snap_create 2024-09-18T14:01:01.452 INFO:tasks.rados.rados.0.smithi084.stdout:456: finishing write tid 2 to smithi08434663-32 2024-09-18T14:01:01.452 INFO:tasks.rados.rados.0.smithi084.stdout:456: finishing write tid 3 to smithi08434663-32 2024-09-18T14:01:01.452 INFO:tasks.rados.rados.0.smithi084.stdout:456: finishing write tid 4 to smithi08434663-32 2024-09-18T14:01:01.452 INFO:tasks.rados.rados.0.smithi084.stdout:456: finishing write tid 5 to smithi08434663-32 2024-09-18T14:01:01.452 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 225 (ObjNum 191 snap 45 seq_num 191) dirty exists 2024-09-18T14:01:01.453 INFO:tasks.rados.rados.0.smithi084.stdout:456: left oid 32 (ObjNum 191 snap 45 seq_num 191) 2024-09-18T14:01:01.453 INFO:tasks.rados.rados.0.smithi084.stdout:458: finishing write tid 1 to smithi08434663-33 2024-09-18T14:01:01.453 INFO:tasks.rados.rados.0.smithi084.stdout:458: finishing write tid 2 to smithi08434663-33 2024-09-18T14:01:01.453 INFO:tasks.rados.rados.0.smithi084.stdout:460: finishing write tid 1 to smithi08434663-5 2024-09-18T14:01:01.453 INFO:tasks.rados.rados.0.smithi084.stdout:459: expect (ObjNum 96 snap 11 seq_num 96) 2024-09-18T14:01:01.608 INFO:tasks.rados.rados.0.smithi084.stdout:458: finishing write tid 3 to smithi08434663-33 2024-09-18T14:01:01.608 INFO:tasks.rados.rados.0.smithi084.stdout:458: finishing write tid 4 to smithi08434663-33 2024-09-18T14:01:01.608 INFO:tasks.rados.rados.0.smithi084.stdout:458: finishing write tid 5 to smithi08434663-33 2024-09-18T14:01:01.608 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 129 (ObjNum 192 snap 45 seq_num 192) dirty exists 2024-09-18T14:01:01.608 INFO:tasks.rados.rados.0.smithi084.stdout:458: left oid 33 (ObjNum 192 snap 45 seq_num 192) 2024-09-18T14:01:01.608 INFO:tasks.rados.rados.0.smithi084.stdout:460: finishing write tid 2 to smithi08434663-5 2024-09-18T14:01:01.608 INFO:tasks.rados.rados.0.smithi084.stdout:460: finishing write tid 3 to smithi08434663-5 2024-09-18T14:01:01.609 INFO:tasks.rados.rados.0.smithi084.stdout:460: finishing write tid 4 to smithi08434663-5 2024-09-18T14:01:01.609 INFO:tasks.rados.rados.0.smithi084.stdout:460: finishing write tid 5 to smithi08434663-5 2024-09-18T14:01:01.609 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 120 (ObjNum 193 snap 45 seq_num 193) dirty exists 2024-09-18T14:01:01.609 INFO:tasks.rados.rados.0.smithi084.stdout:460: left oid 5 (ObjNum 193 snap 45 seq_num 193) 2024-09-18T14:01:02.154 INFO:tasks.rados.rados.0.smithi084.stdout:456: done (5 left) 2024-09-18T14:01:02.154 INFO:tasks.rados.rados.0.smithi084.stdout:457: done (4 left) 2024-09-18T14:01:02.154 INFO:tasks.rados.rados.0.smithi084.stdout:458: done (3 left) 2024-09-18T14:01:02.154 INFO:tasks.rados.rados.0.smithi084.stdout:459: done (2 left) 2024-09-18T14:01:02.154 INFO:tasks.rados.rados.0.smithi084.stdout:460: done (1 left) 2024-09-18T14:01:02.154 INFO:tasks.rados.rados.0.smithi084.stdout:461: done (0 left) 2024-09-18T14:01:02.154 INFO:tasks.rados.rados.0.smithi084.stdout:462: rollback oid 18 current snap is 46 2024-09-18T14:01:02.154 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 18 to 44 2024-09-18T14:01:02.154 INFO:tasks.rados.rados.0.smithi084.stdout:463: write oid 24 current snap is 46 2024-09-18T14:01:02.154 INFO:tasks.rados.rados.0.smithi084.stdout:463: seq_num 194 ranges {460558=707131,1869389=508716,3094884=165674} 2024-09-18T14:01:02.160 INFO:tasks.rados.rados.0.smithi084.stdout:463: writing smithi08434663-24 from 460558 to 1167689 tid 1 2024-09-18T14:01:02.167 INFO:tasks.rados.rados.0.smithi084.stdout:463: writing smithi08434663-24 from 1869389 to 2378105 tid 2 2024-09-18T14:01:02.172 INFO:tasks.rados.rados.0.smithi084.stdout:463: writing smithi08434663-24 from 3094884 to 3260558 tid 3 2024-09-18T14:01:02.173 INFO:tasks.rados.rados.0.smithi084.stdout:464: write oid 22 current snap is 46 2024-09-18T14:01:02.173 INFO:tasks.rados.rados.0.smithi084.stdout:462: finishing rollback tid 0 to smithi08434663-18 2024-09-18T14:01:02.173 INFO:tasks.rados.rados.0.smithi084.stdout:462: finishing rollback tid 1 to smithi08434663-18 2024-09-18T14:01:02.173 INFO:tasks.rados.rados.0.smithi084.stdout:462: finishing rollback tid 2 to smithi08434663-18 2024-09-18T14:01:02.173 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 80 (ObjNum 180 snap 41 seq_num 180) dirty exists 2024-09-18T14:01:02.173 INFO:tasks.rados.rados.0.smithi084.stdout:463: finishing write tid 1 to smithi08434663-24 2024-09-18T14:01:02.173 INFO:tasks.rados.rados.0.smithi084.stdout:463: finishing write tid 2 to smithi08434663-24 2024-09-18T14:01:02.173 INFO:tasks.rados.rados.0.smithi084.stdout:464: seq_num 195 ranges {477365=782380,1927871=768727,3122307=155058} 2024-09-18T14:01:02.179 INFO:tasks.rados.rados.0.smithi084.stdout:464: writing smithi08434663-22 from 477365 to 1259745 tid 1 2024-09-18T14:01:02.187 INFO:tasks.rados.rados.0.smithi084.stdout:464: writing smithi08434663-22 from 1927871 to 2696598 tid 2 2024-09-18T14:01:02.190 INFO:tasks.rados.rados.0.smithi084.stdout:464: writing smithi08434663-22 from 3122307 to 3277365 tid 3 2024-09-18T14:01:02.191 INFO:tasks.rados.rados.0.smithi084.stdout:462: done (2 left) 2024-09-18T14:01:02.191 INFO:tasks.rados.rados.0.smithi084.stdout:465: snap_remove snap 44 2024-09-18T14:01:03.160 INFO:tasks.rados.rados.0.smithi084.stdout:465: done (2 left) 2024-09-18T14:01:03.160 INFO:tasks.rados.rados.0.smithi084.stdout:466: write oid 46 current snap is 46 2024-09-18T14:01:03.161 INFO:tasks.rados.rados.0.smithi084.stdout:466: seq_num 196 ranges {494172=457629,1586353=512385,2749730=544442} 2024-09-18T14:01:03.166 INFO:tasks.rados.rados.0.smithi084.stdout:466: writing smithi08434663-46 from 494172 to 951801 tid 1 2024-09-18T14:01:03.173 INFO:tasks.rados.rados.0.smithi084.stdout:466: writing smithi08434663-46 from 1586353 to 2098738 tid 2 2024-09-18T14:01:03.181 INFO:tasks.rados.rados.0.smithi084.stdout:466: writing smithi08434663-46 from 2749730 to 3294172 tid 3 2024-09-18T14:01:03.181 INFO:tasks.rados.rados.0.smithi084.stdout:467: read oid 9 snap 45 2024-09-18T14:01:03.181 INFO:tasks.rados.rados.0.smithi084.stdout:467: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:01:03.181 INFO:tasks.rados.rados.0.smithi084.stdout:468: read oid 26 snap -1 2024-09-18T14:01:03.181 INFO:tasks.rados.rados.0.smithi084.stdout:468: expect (ObjNum 116 snap 22 seq_num 116) 2024-09-18T14:01:03.181 INFO:tasks.rados.rados.0.smithi084.stdout:469: copy_from oid 7 from oid 11 current snap is 46 2024-09-18T14:01:03.181 INFO:tasks.rados.rados.0.smithi084.stdout:463: finishing write tid 3 to smithi08434663-24 2024-09-18T14:01:03.181 INFO:tasks.rados.rados.0.smithi084.stdout:463: finishing write tid 4 to smithi08434663-24 2024-09-18T14:01:03.181 INFO:tasks.rados.rados.0.smithi084.stdout:463: finishing write tid 5 to smithi08434663-24 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 163 (ObjNum 194 snap 46 seq_num 194) dirty exists 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:463: left oid 24 (ObjNum 194 snap 46 seq_num 194) 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:464: finishing write tid 1 to smithi08434663-22 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:464: finishing write tid 2 to smithi08434663-22 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:463: done (5 left) 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:470: copy_from oid 21 from oid 30 current snap is 46 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:464: finishing write tid 3 to smithi08434663-22 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:464: finishing write tid 4 to smithi08434663-22 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:464: finishing write tid 5 to smithi08434663-22 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 85 (ObjNum 195 snap 46 seq_num 195) dirty exists 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:464: left oid 22 (ObjNum 195 snap 46 seq_num 195) 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:466: finishing write tid 1 to smithi08434663-46 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:466: finishing write tid 2 to smithi08434663-46 2024-09-18T14:01:03.182 INFO:tasks.rados.rados.0.smithi084.stdout:464: done (5 left) 2024-09-18T14:01:03.183 INFO:tasks.rados.rados.0.smithi084.stdout:471: read oid 42 snap 45 2024-09-18T14:01:03.183 INFO:tasks.rados.rados.0.smithi084.stdout:471: expect (ObjNum 58 snap 2 seq_num 58) 2024-09-18T14:01:03.183 INFO:tasks.rados.rados.0.smithi084.stdout:472: read oid 47 snap -1 2024-09-18T14:01:03.183 INFO:tasks.rados.rados.0.smithi084.stdout:472: expect deleted 2024-09-18T14:01:03.183 INFO:tasks.rados.rados.0.smithi084.stdout:473: read oid 6 snap 45 2024-09-18T14:01:03.183 INFO:tasks.rados.rados.0.smithi084.stdout:473: expect deleted 2024-09-18T14:01:03.183 INFO:tasks.rados.rados.0.smithi084.stdout:474: write oid 8 current snap is 46 2024-09-18T14:01:03.183 INFO:tasks.rados.rados.0.smithi084.stdout:474: seq_num 197 ranges {510979=532878,1528482=772396,3060800=250179} 2024-09-18T14:01:03.187 INFO:tasks.rados.rados.0.smithi084.stdout:474: writing smithi08434663-8 from 510979 to 1043857 tid 1 2024-09-18T14:01:03.194 INFO:tasks.rados.rados.0.smithi084.stdout:474: writing smithi08434663-8 from 1528482 to 2300878 tid 2 2024-09-18T14:01:03.200 INFO:tasks.rados.rados.0.smithi084.stdout:474: writing smithi08434663-8 from 3060800 to 3310979 tid 3 2024-09-18T14:01:03.200 INFO:tasks.rados.rados.0.smithi084.stdout:475: delete oid 48 current snap is 46 2024-09-18T14:01:03.200 INFO:tasks.rados.rados.0.smithi084.stdout:467: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:01:03.269 INFO:tasks.rados.rados.0.smithi084.stdout:466: finishing write tid 3 to smithi08434663-46 2024-09-18T14:01:03.270 INFO:tasks.rados.rados.0.smithi084.stdout:466: finishing write tid 4 to smithi08434663-46 2024-09-18T14:01:03.270 INFO:tasks.rados.rados.0.smithi084.stdout:466: finishing write tid 5 to smithi08434663-46 2024-09-18T14:01:03.270 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 173 (ObjNum 196 snap 46 seq_num 196) dirty exists 2024-09-18T14:01:03.270 INFO:tasks.rados.rados.0.smithi084.stdout:466: left oid 46 (ObjNum 196 snap 46 seq_num 196) 2024-09-18T14:01:03.270 INFO:tasks.rados.rados.0.smithi084.stdout:468: expect (ObjNum 116 snap 22 seq_num 116) 2024-09-18T14:01:03.397 INFO:tasks.rados.rados.0.smithi084.stdout:470: finishing copy_from racing read to smithi08434663-21 2024-09-18T14:01:03.397 INFO:tasks.rados.rados.0.smithi084.stdout:470: finishing copy_from to smithi08434663-21 2024-09-18T14:01:03.397 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 36 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T14:01:03.397 INFO:tasks.rados.rados.0.smithi084.stdout:471: expect (ObjNum 58 snap 2 seq_num 58) 2024-09-18T14:01:03.529 INFO:tasks.rados.rados.0.smithi084.stdout:474: finishing write tid 1 to smithi08434663-8 2024-09-18T14:01:03.529 INFO:tasks.rados.rados.0.smithi084.stdout:469: finishing copy_from racing read to smithi08434663-7 2024-09-18T14:01:03.529 INFO:tasks.rados.rados.0.smithi084.stdout:474: finishing write tid 2 to smithi08434663-8 2024-09-18T14:01:03.529 INFO:tasks.rados.rados.0.smithi084.stdout:469: finishing copy_from to smithi08434663-7 2024-09-18T14:01:03.529 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 172 (ObjNum 178 snap 41 seq_num 178) dirty exists 2024-09-18T14:01:03.529 INFO:tasks.rados.rados.0.smithi084.stdout:474: finishing write tid 3 to smithi08434663-8 2024-09-18T14:01:03.529 INFO:tasks.rados.rados.0.smithi084.stdout:474: finishing write tid 4 to smithi08434663-8 2024-09-18T14:01:03.529 INFO:tasks.rados.rados.0.smithi084.stdout:474: finishing write tid 5 to smithi08434663-8 2024-09-18T14:01:03.529 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 79 (ObjNum 197 snap 46 seq_num 197) dirty exists 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:474: left oid 8 (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:466: done (9 left) 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:467: done (8 left) 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:468: done (7 left) 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:469: done (6 left) 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:470: done (5 left) 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:471: done (4 left) 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:472: done (3 left) 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:473: done (2 left) 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:474: done (1 left) 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:475: done (0 left) 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:476: write (excl) oid 4 current snap is 46 2024-09-18T14:01:03.530 INFO:tasks.rados.rados.0.smithi084.stdout:476: seq_num 199 ranges {544593=799729,1761799=776065,2948352=396241} 2024-09-18T14:01:03.536 INFO:tasks.rados.rados.0.smithi084.stdout:476: writing smithi08434663-4 from 544593 to 1344322 tid 1 2024-09-18T14:01:03.542 INFO:tasks.rados.rados.0.smithi084.stdout:476: writing smithi08434663-4 from 1761799 to 2537864 tid 2 2024-09-18T14:01:03.547 INFO:tasks.rados.rados.0.smithi084.stdout:476: writing smithi08434663-4 from 2948352 to 3344593 tid 3 2024-09-18T14:01:03.547 INFO:tasks.rados.rados.0.smithi084.stdout:477: read oid 49 snap 45 2024-09-18T14:01:03.547 INFO:tasks.rados.rados.0.smithi084.stdout:477: expect (ObjNum 111 snap 21 seq_num 111) 2024-09-18T14:01:03.547 INFO:tasks.rados.rados.0.smithi084.stdout:478: rollback oid 18 current snap is 46 2024-09-18T14:01:03.547 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 18 to 45 2024-09-18T14:01:03.547 INFO:tasks.rados.rados.0.smithi084.stdout:476: finishing write tid 1 to smithi08434663-4 2024-09-18T14:01:03.548 INFO:tasks.rados.rados.0.smithi084.stdout:476: finishing write tid 2 to smithi08434663-4 2024-09-18T14:01:03.548 INFO:tasks.rados.rados.0.smithi084.stdout:479: rollback oid 10 current snap is 46 2024-09-18T14:01:03.548 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 10 to 45 2024-09-18T14:01:03.548 INFO:tasks.rados.rados.0.smithi084.stdout:480: copy_from oid 48 from oid 35 current snap is 46 2024-09-18T14:01:03.548 INFO:tasks.rados.rados.0.smithi084.stdout:481: rollback oid 11 current snap is 46 2024-09-18T14:01:03.548 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 11 to 45 2024-09-18T14:01:03.548 INFO:tasks.rados.rados.0.smithi084.stdout:482: write (excl) oid 5 current snap is 46 2024-09-18T14:01:03.548 INFO:tasks.rados.rados.0.smithi084.stdout:482: seq_num 200 ranges {561400=474978,1820281=519723,2859422=457579} 2024-09-18T14:01:03.553 INFO:tasks.rados.rados.0.smithi084.stdout:482: writing smithi08434663-5 from 561400 to 1036378 tid 1 2024-09-18T14:01:03.561 INFO:tasks.rados.rados.0.smithi084.stdout:482: writing smithi08434663-5 from 1820281 to 2340004 tid 2 2024-09-18T14:01:03.566 INFO:tasks.rados.rados.0.smithi084.stdout:482: writing smithi08434663-5 from 2859422 to 3317001 tid 3 2024-09-18T14:01:03.567 INFO:tasks.rados.rados.0.smithi084.stdout:483: write oid 25 current snap is 46 2024-09-18T14:01:03.567 INFO:tasks.rados.rados.0.smithi084.stdout:476: finishing write tid 3 to smithi08434663-4 2024-09-18T14:01:03.567 INFO:tasks.rados.rados.0.smithi084.stdout:476: finishing write tid 4 to smithi08434663-4 2024-09-18T14:01:03.567 INFO:tasks.rados.rados.0.smithi084.stdout:476: finishing write tid 5 to smithi08434663-4 2024-09-18T14:01:03.567 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 113 (ObjNum 199 snap 46 seq_num 199) dirty exists 2024-09-18T14:01:03.567 INFO:tasks.rados.rados.0.smithi084.stdout:476: left oid 4 (ObjNum 199 snap 46 seq_num 199) 2024-09-18T14:01:03.567 INFO:tasks.rados.rados.0.smithi084.stdout:480: finishing copy_from to smithi08434663-48 2024-09-18T14:01:03.567 INFO:tasks.rados.rados.0.smithi084.stdout:480: got expected ENOENT (src dne) 2024-09-18T14:01:03.567 INFO:tasks.rados.rados.0.smithi084.stdout:480: finishing copy_from racing read to smithi08434663-48 2024-09-18T14:01:03.567 INFO:tasks.rados.rados.0.smithi084.stdout:483: seq_num 201 ranges {578207=550227,1762410=779734,3286845=91362} 2024-09-18T14:01:03.572 INFO:tasks.rados.rados.0.smithi084.stdout:483: writing smithi08434663-25 from 578207 to 1128434 tid 1 2024-09-18T14:01:03.580 INFO:tasks.rados.rados.0.smithi084.stdout:483: writing smithi08434663-25 from 1762410 to 2542144 tid 2 2024-09-18T14:01:03.585 INFO:tasks.rados.rados.0.smithi084.stdout:483: writing smithi08434663-25 from 3286845 to 3378207 tid 3 2024-09-18T14:01:03.585 INFO:tasks.rados.rados.0.smithi084.stdout:476: done (7 left) 2024-09-18T14:01:03.585 INFO:tasks.rados.rados.0.smithi084.stdout:480: done (6 left) 2024-09-18T14:01:03.585 INFO:tasks.rados.rados.0.smithi084.stdout:484: rollback oid 22 current snap is 46 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 22 to 45 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:478: finishing rollback tid 0 to smithi08434663-18 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:478: finishing rollback tid 1 to smithi08434663-18 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:478: finishing rollback tid 2 to smithi08434663-18 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 92 (ObjNum 180 snap 41 seq_num 180) dirty exists 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:481: finishing rollback tid 0 to smithi08434663-11 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:479: finishing rollback tid 0 to smithi08434663-10 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:478: done (6 left) 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:485: read oid 23 snap -1 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:485: expect (ObjNum 171 snap 38 seq_num 171) 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:486: rollback oid 15 current snap is 46 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 15 to 45 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:481: finishing rollback tid 1 to smithi08434663-11 2024-09-18T14:01:03.586 INFO:tasks.rados.rados.0.smithi084.stdout:479: finishing rollback tid 1 to smithi08434663-10 2024-09-18T14:01:03.587 INFO:tasks.rados.rados.0.smithi084.stdout:479: finishing rollback tid 2 to smithi08434663-10 2024-09-18T14:01:03.587 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 145 (ObjNum 72 snap 6 seq_num 72) dirty exists 2024-09-18T14:01:03.587 INFO:tasks.rados.rados.0.smithi084.stdout:481: finishing rollback tid 2 to smithi08434663-11 2024-09-18T14:01:03.587 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 175 (ObjNum 178 snap 41 seq_num 178) dirty exists 2024-09-18T14:01:03.587 INFO:tasks.rados.rados.0.smithi084.stdout:477: expect (ObjNum 111 snap 21 seq_num 111) 2024-09-18T14:01:03.706 INFO:tasks.rados.rados.0.smithi084.stdout:482: finishing write tid 1 to smithi08434663-5 2024-09-18T14:01:03.706 INFO:tasks.rados.rados.0.smithi084.stdout:482: finishing write tid 2 to smithi08434663-5 2024-09-18T14:01:03.707 INFO:tasks.rados.rados.0.smithi084.stdout:482: finishing write tid 3 to smithi08434663-5 2024-09-18T14:01:03.707 INFO:tasks.rados.rados.0.smithi084.stdout:482: finishing write tid 4 to smithi08434663-5 2024-09-18T14:01:03.707 INFO:tasks.rados.rados.0.smithi084.stdout:482: finishing write tid 5 to smithi08434663-5 2024-09-18T14:01:03.707 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 124 (ObjNum 200 snap 46 seq_num 200) dirty exists 2024-09-18T14:01:03.707 INFO:tasks.rados.rados.0.smithi084.stdout:482: left oid 5 (ObjNum 200 snap 46 seq_num 200) 2024-09-18T14:01:03.707 INFO:tasks.rados.rados.0.smithi084.stdout:483: finishing write tid 1 to smithi08434663-25 2024-09-18T14:01:03.707 INFO:tasks.rados.rados.0.smithi084.stdout:483: finishing write tid 2 to smithi08434663-25 2024-09-18T14:01:03.707 INFO:tasks.rados.rados.0.smithi084.stdout:477: done (7 left) 2024-09-18T14:01:03.707 INFO:tasks.rados.rados.0.smithi084.stdout:479: done (6 left) 2024-09-18T14:01:03.708 INFO:tasks.rados.rados.0.smithi084.stdout:481: done (5 left) 2024-09-18T14:01:03.708 INFO:tasks.rados.rados.0.smithi084.stdout:482: done (4 left) 2024-09-18T14:01:03.708 INFO:tasks.rados.rados.0.smithi084.stdout:487: read oid 45 snap -1 2024-09-18T14:01:03.708 INFO:tasks.rados.rados.0.smithi084.stdout:487: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-18T14:01:03.708 INFO:tasks.rados.rados.0.smithi084.stdout:488: write oid 17 current snap is 46 2024-09-18T14:01:03.708 INFO:tasks.rados.rados.0.smithi084.stdout:483: finishing write tid 3 to smithi08434663-25 2024-09-18T14:01:03.708 INFO:tasks.rados.rados.0.smithi084.stdout:483: finishing write tid 4 to smithi08434663-25 2024-09-18T14:01:03.708 INFO:tasks.rados.rados.0.smithi084.stdout:483: finishing write tid 5 to smithi08434663-25 2024-09-18T14:01:03.708 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 120 (ObjNum 201 snap 46 seq_num 201) dirty exists 2024-09-18T14:01:03.708 INFO:tasks.rados.rados.0.smithi084.stdout:483: left oid 25 (ObjNum 201 snap 46 seq_num 201) 2024-09-18T14:01:03.709 INFO:tasks.rados.rados.0.smithi084.stdout:484: finishing rollback tid 0 to smithi08434663-22 2024-09-18T14:01:03.709 INFO:tasks.rados.rados.0.smithi084.stdout:484: finishing rollback tid 1 to smithi08434663-22 2024-09-18T14:01:03.709 INFO:tasks.rados.rados.0.smithi084.stdout:484: finishing rollback tid 2 to smithi08434663-22 2024-09-18T14:01:03.709 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 95 (ObjNum 128 snap 25 seq_num 128) dirty exists 2024-09-18T14:01:03.709 INFO:tasks.rados.rados.0.smithi084.stdout:486: finishing rollback tid 0 to smithi08434663-15 2024-09-18T14:01:03.709 INFO:tasks.rados.rados.0.smithi084.stdout:486: finishing rollback tid 1 to smithi08434663-15 2024-09-18T14:01:03.709 INFO:tasks.rados.rados.0.smithi084.stdout:486: finishing rollback tid 2 to smithi08434663-15 2024-09-18T14:01:03.709 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 177 (ObjNum 147 snap 30 seq_num 147) dirty exists 2024-09-18T14:01:03.709 INFO:tasks.rados.rados.0.smithi084.stdout:488: seq_num 202 ranges {595014=625476,1820892=523392,2797915=596476} 2024-09-18T14:01:03.713 INFO:tasks.rados.rados.0.smithi084.stdout:488: writing smithi08434663-17 from 595014 to 1220490 tid 1 2024-09-18T14:01:03.719 INFO:tasks.rados.rados.0.smithi084.stdout:488: writing smithi08434663-17 from 1820892 to 2344284 tid 2 2024-09-18T14:01:03.725 INFO:tasks.rados.rados.0.smithi084.stdout:488: writing smithi08434663-17 from 2797915 to 3394391 tid 3 2024-09-18T14:01:03.725 INFO:tasks.rados.rados.0.smithi084.stdout:483: done (5 left) 2024-09-18T14:01:03.725 INFO:tasks.rados.rados.0.smithi084.stdout:484: done (4 left) 2024-09-18T14:01:03.725 INFO:tasks.rados.rados.0.smithi084.stdout:486: done (3 left) 2024-09-18T14:01:03.725 INFO:tasks.rados.rados.0.smithi084.stdout:489: write oid 46 current snap is 46 2024-09-18T14:01:03.726 INFO:tasks.rados.rados.0.smithi084.stdout:489: seq_num 203 ranges {611821=700725,1879374=783403,3341691=70130} 2024-09-18T14:01:03.731 INFO:tasks.rados.rados.0.smithi084.stdout:489: writing smithi08434663-46 from 611821 to 1312546 tid 1 2024-09-18T14:01:03.739 INFO:tasks.rados.rados.0.smithi084.stdout:489: writing smithi08434663-46 from 1879374 to 2662777 tid 2 2024-09-18T14:01:03.743 INFO:tasks.rados.rados.0.smithi084.stdout:489: writing smithi08434663-46 from 3341691 to 3411821 tid 3 2024-09-18T14:01:03.743 INFO:tasks.rados.rados.0.smithi084.stdout:490: write (excl) oid 19 current snap is 46 2024-09-18T14:01:03.744 INFO:tasks.rados.rados.0.smithi084.stdout:490: seq_num 204 ranges {628628=775974,1937856=527061,3252761=175867} 2024-09-18T14:01:03.751 INFO:tasks.rados.rados.0.smithi084.stdout:490: writing smithi08434663-19 from 628628 to 1404602 tid 1 2024-09-18T14:01:03.757 INFO:tasks.rados.rados.0.smithi084.stdout:490: writing smithi08434663-19 from 1937856 to 2464917 tid 2 2024-09-18T14:01:03.763 INFO:tasks.rados.rados.0.smithi084.stdout:490: writing smithi08434663-19 from 3252761 to 3428628 tid 3 2024-09-18T14:01:03.763 INFO:tasks.rados.rados.0.smithi084.stdout:491: read oid 14 snap -1 2024-09-18T14:01:03.763 INFO:tasks.rados.rados.0.smithi084.stdout:491: expect deleted 2024-09-18T14:01:03.763 INFO:tasks.rados.rados.0.smithi084.stdout:492: read oid 25 snap 45 2024-09-18T14:01:03.764 INFO:tasks.rados.rados.0.smithi084.stdout:492: expect (ObjNum 154 snap 32 seq_num 154) 2024-09-18T14:01:03.764 INFO:tasks.rados.rados.0.smithi084.stdout:493: copy_from oid 10 from oid 27 current snap is 46 2024-09-18T14:01:03.764 INFO:tasks.rados.rados.0.smithi084.stdout:485: expect (ObjNum 171 snap 38 seq_num 171) 2024-09-18T14:01:03.855 INFO:tasks.rados.rados.0.smithi084.stdout:487: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-18T14:01:03.942 INFO:tasks.rados.rados.0.smithi084.stdout:488: finishing write tid 1 to smithi08434663-17 2024-09-18T14:01:03.942 INFO:tasks.rados.rados.0.smithi084.stdout:488: finishing write tid 2 to smithi08434663-17 2024-09-18T14:01:03.943 INFO:tasks.rados.rados.0.smithi084.stdout:488: finishing write tid 3 to smithi08434663-17 2024-09-18T14:01:03.943 INFO:tasks.rados.rados.0.smithi084.stdout:488: finishing write tid 4 to smithi08434663-17 2024-09-18T14:01:03.943 INFO:tasks.rados.rados.0.smithi084.stdout:488: finishing write tid 5 to smithi08434663-17 2024-09-18T14:01:03.943 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 128 (ObjNum 202 snap 46 seq_num 202) dirty exists 2024-09-18T14:01:03.943 INFO:tasks.rados.rados.0.smithi084.stdout:488: left oid 17 (ObjNum 202 snap 46 seq_num 202) 2024-09-18T14:01:03.943 INFO:tasks.rados.rados.0.smithi084.stdout:489: finishing write tid 1 to smithi08434663-46 2024-09-18T14:01:03.943 INFO:tasks.rados.rados.0.smithi084.stdout:489: finishing write tid 2 to smithi08434663-46 2024-09-18T14:01:03.943 INFO:tasks.rados.rados.0.smithi084.stdout:489: finishing write tid 3 to smithi08434663-46 2024-09-18T14:01:03.943 INFO:tasks.rados.rados.0.smithi084.stdout:489: finishing write tid 4 to smithi08434663-46 2024-09-18T14:01:03.943 INFO:tasks.rados.rados.0.smithi084.stdout:485: done (7 left) 2024-09-18T14:01:03.943 INFO:tasks.rados.rados.0.smithi084.stdout:487: done (6 left) 2024-09-18T14:01:03.943 INFO:tasks.rados.rados.0.smithi084.stdout:488: done (5 left) 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:494: rollback oid 6 current snap is 46 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 45 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:489: finishing write tid 5 to smithi08434663-46 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 181 (ObjNum 203 snap 46 seq_num 203) dirty exists 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:489: left oid 46 (ObjNum 203 snap 46 seq_num 203) 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:490: finishing write tid 1 to smithi08434663-19 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:490: finishing write tid 2 to smithi08434663-19 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:490: finishing write tid 3 to smithi08434663-19 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:490: finishing write tid 4 to smithi08434663-19 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:490: finishing write tid 5 to smithi08434663-19 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 232 (ObjNum 204 snap 46 seq_num 204) dirty exists 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:490: left oid 19 (ObjNum 204 snap 46 seq_num 204) 2024-09-18T14:01:03.944 INFO:tasks.rados.rados.0.smithi084.stdout:492: expect (ObjNum 154 snap 32 seq_num 154) 2024-09-18T14:01:04.083 INFO:tasks.rados.rados.0.smithi084.stdout:493: finishing copy_from racing read to smithi08434663-10 2024-09-18T14:01:04.083 INFO:tasks.rados.rados.0.smithi084.stdout:493: finishing copy_from to smithi08434663-10 2024-09-18T14:01:04.084 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 133 (ObjNum 103 snap 17 seq_num 103) dirty exists 2024-09-18T14:01:04.084 INFO:tasks.rados.rados.0.smithi084.stdout:494: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:01:04.084 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 0 (ObjNum -1025532688 snap 32643 seq_num 304) dirty dne 2024-09-18T14:01:04.084 INFO:tasks.rados.rados.0.smithi084.stdout:489: done (5 left) 2024-09-18T14:01:04.084 INFO:tasks.rados.rados.0.smithi084.stdout:490: done (4 left) 2024-09-18T14:01:04.084 INFO:tasks.rados.rados.0.smithi084.stdout:491: done (3 left) 2024-09-18T14:01:04.084 INFO:tasks.rados.rados.0.smithi084.stdout:492: done (2 left) 2024-09-18T14:01:04.084 INFO:tasks.rados.rados.0.smithi084.stdout:493: done (1 left) 2024-09-18T14:01:04.084 INFO:tasks.rados.rados.0.smithi084.stdout:494: done (0 left) 2024-09-18T14:01:04.084 INFO:tasks.rados.rados.0.smithi084.stdout:495: read oid 18 snap 45 2024-09-18T14:01:04.085 INFO:tasks.rados.rados.0.smithi084.stdout:495: expect (ObjNum 180 snap 41 seq_num 180) 2024-09-18T14:01:04.085 INFO:tasks.rados.rados.0.smithi084.stdout:496: read oid 45 snap -1 2024-09-18T14:01:04.085 INFO:tasks.rados.rados.0.smithi084.stdout:496: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-18T14:01:04.085 INFO:tasks.rados.rados.0.smithi084.stdout:497: write oid 24 current snap is 46 2024-09-18T14:01:04.085 INFO:tasks.rados.rados.0.smithi084.stdout:497: seq_num 205 ranges {645435=451223,1879985=787072,3280184=165251} 2024-09-18T14:01:04.089 INFO:tasks.rados.rados.0.smithi084.stdout:497: writing smithi08434663-24 from 645435 to 1096658 tid 1 2024-09-18T14:01:04.097 INFO:tasks.rados.rados.0.smithi084.stdout:497: writing smithi08434663-24 from 1879985 to 2667057 tid 2 2024-09-18T14:01:04.102 INFO:tasks.rados.rados.0.smithi084.stdout:497: writing smithi08434663-24 from 3280184 to 3445435 tid 3 2024-09-18T14:01:04.102 INFO:tasks.rados.rados.0.smithi084.stdout:498: copy_from oid 36 from oid 10 current snap is 46 2024-09-18T14:01:04.102 INFO:tasks.rados.rados.0.smithi084.stdout:496: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-18T14:01:04.146 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-18T14:01:04.146 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-18T14:01:04.146 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T14:01:04.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T14:01:04.150 INFO:tasks.ceph.mon.c:Started 2024-09-18T14:01:04.150 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-18T14:01:04.151 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-18T14:01:04.151 INFO:tasks.mon_thrash:Sending STOP to mon f 2024-09-18T14:01:04.151 INFO:tasks.ceph.mon.f:Sent signal 19 2024-09-18T14:01:04.152 INFO:tasks.mon_thrash:Sending STOP to mon i 2024-09-18T14:01:04.152 INFO:tasks.ceph.mon.i:Sent signal 19 2024-09-18T14:01:04.152 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-18T14:01:04.192 INFO:tasks.rados.rados.0.smithi084.stdout:497: finishing write tid 1 to smithi08434663-24 2024-09-18T14:01:04.192 INFO:tasks.rados.rados.0.smithi084.stdout:495: expect (ObjNum 180 snap 41 seq_num 180) 2024-09-18T14:01:04.363 INFO:tasks.rados.rados.0.smithi084.stdout:497: finishing write tid 2 to smithi08434663-24 2024-09-18T14:01:04.363 INFO:tasks.rados.rados.0.smithi084.stdout:497: finishing write tid 3 to smithi08434663-24 2024-09-18T14:01:04.363 INFO:tasks.rados.rados.0.smithi084.stdout:497: finishing write tid 4 to smithi08434663-24 2024-09-18T14:01:04.364 INFO:tasks.rados.rados.0.smithi084.stdout:497: finishing write tid 5 to smithi08434663-24 2024-09-18T14:01:04.364 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 167 (ObjNum 205 snap 46 seq_num 205) dirty exists 2024-09-18T14:01:04.364 INFO:tasks.rados.rados.0.smithi084.stdout:497: left oid 24 (ObjNum 205 snap 46 seq_num 205) 2024-09-18T14:01:04.364 INFO:tasks.rados.rados.0.smithi084.stdout:498: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:01:04.364 INFO:tasks.rados.rados.0.smithi084.stdout:498: finishing copy_from to smithi08434663-36 2024-09-18T14:01:04.364 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 133 (ObjNum 103 snap 17 seq_num 103) dirty exists 2024-09-18T14:01:04.364 INFO:tasks.rados.rados.0.smithi084.stdout:495: done (3 left) 2024-09-18T14:01:04.364 INFO:tasks.rados.rados.0.smithi084.stdout:496: done (2 left) 2024-09-18T14:01:04.364 INFO:tasks.rados.rados.0.smithi084.stdout:497: done (1 left) 2024-09-18T14:01:04.364 INFO:tasks.rados.rados.0.smithi084.stdout:498: done (0 left) 2024-09-18T14:01:04.364 INFO:tasks.rados.rados.0.smithi084.stdout:499: delete oid 37 current snap is 46 2024-09-18T14:01:04.366 INFO:tasks.rados.rados.0.smithi084.stdout:499: done (0 left) 2024-09-18T14:01:04.366 INFO:tasks.rados.rados.0.smithi084.stdout:500: snap_remove snap 45 2024-09-18T14:01:10.259 INFO:tasks.rados.rados.0.smithi084.stdout:500: done (0 left) 2024-09-18T14:01:10.259 INFO:tasks.rados.rados.0.smithi084.stdout:501: rollback oid 44 current snap is 46 2024-09-18T14:01:10.259 INFO:tasks.rados.rados.0.smithi084.stdout:501: done (0 left) 2024-09-18T14:01:10.259 INFO:tasks.rados.rados.0.smithi084.stdout:502: read oid 11 snap -1 2024-09-18T14:01:10.260 INFO:tasks.rados.rados.0.smithi084.stdout:502: expect (ObjNum 178 snap 41 seq_num 178) 2024-09-18T14:01:10.260 INFO:tasks.rados.rados.0.smithi084.stdout:503: read oid 44 snap -1 2024-09-18T14:01:10.260 INFO:tasks.rados.rados.0.smithi084.stdout:503: expect (ObjNum 88 snap 9 seq_num 88) 2024-09-18T14:01:10.260 INFO:tasks.rados.rados.0.smithi084.stdout:504: write (excl) oid 40 current snap is 46 2024-09-18T14:01:10.260 INFO:tasks.rados.rados.0.smithi084.stdout:504: seq_num 207 ranges {679049=718074,2113302=674388,3335030=144019} 2024-09-18T14:01:10.267 INFO:tasks.rados.rados.0.smithi084.stdout:504: writing smithi08434663-40 from 679049 to 1397123 tid 1 2024-09-18T14:01:10.276 INFO:tasks.rados.rados.0.smithi084.stdout:504: writing smithi08434663-40 from 2113302 to 2787690 tid 2 2024-09-18T14:01:10.280 INFO:tasks.rados.rados.0.smithi084.stdout:504: writing smithi08434663-40 from 3335030 to 3479049 tid 3 2024-09-18T14:01:10.280 INFO:tasks.rados.rados.0.smithi084.stdout:505: delete oid 17 current snap is 46 2024-09-18T14:01:10.280 INFO:tasks.rados.rados.0.smithi084.stdout:503: expect (ObjNum 88 snap 9 seq_num 88) 2024-09-18T14:01:10.438 INFO:tasks.rados.rados.0.smithi084.stdout:502: expect (ObjNum 178 snap 41 seq_num 178) 2024-09-18T14:01:10.608 INFO:tasks.rados.rados.0.smithi084.stdout:504: finishing write tid 1 to smithi08434663-40 2024-09-18T14:01:10.608 INFO:tasks.rados.rados.0.smithi084.stdout:504: finishing write tid 2 to smithi08434663-40 2024-09-18T14:01:10.608 INFO:tasks.rados.rados.0.smithi084.stdout:504: finishing write tid 3 to smithi08434663-40 2024-09-18T14:01:10.609 INFO:tasks.rados.rados.0.smithi084.stdout:504: finishing write tid 4 to smithi08434663-40 2024-09-18T14:01:10.609 INFO:tasks.rados.rados.0.smithi084.stdout:504: finishing write tid 5 to smithi08434663-40 2024-09-18T14:01:10.609 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 134 (ObjNum 207 snap 46 seq_num 207) dirty exists 2024-09-18T14:01:10.609 INFO:tasks.rados.rados.0.smithi084.stdout:504: left oid 40 (ObjNum 207 snap 46 seq_num 207) 2024-09-18T14:01:10.609 INFO:tasks.rados.rados.0.smithi084.stdout:502: done (3 left) 2024-09-18T14:01:10.609 INFO:tasks.rados.rados.0.smithi084.stdout:503: done (2 left) 2024-09-18T14:01:10.609 INFO:tasks.rados.rados.0.smithi084.stdout:504: done (1 left) 2024-09-18T14:01:10.609 INFO:tasks.rados.rados.0.smithi084.stdout:505: done (0 left) 2024-09-18T14:01:10.609 INFO:tasks.rados.rados.0.smithi084.stdout:506: snap_create 2024-09-18T14:01:11.265 INFO:tasks.rados.rados.0.smithi084.stdout:506: done (0 left) 2024-09-18T14:01:11.266 INFO:tasks.rados.rados.0.smithi084.stdout:507: write oid 42 current snap is 47 2024-09-18T14:01:11.266 INFO:tasks.rados.rados.0.smithi084.stdout:507: seq_num 209 ranges {712663=468572,1713913=678057,2873523=591733} 2024-09-18T14:01:11.272 INFO:tasks.rados.rados.0.smithi084.stdout:507: writing smithi08434663-42 from 712663 to 1181235 tid 1 2024-09-18T14:01:11.278 INFO:tasks.rados.rados.0.smithi084.stdout:507: writing smithi08434663-42 from 1713913 to 2391970 tid 2 2024-09-18T14:01:11.285 INFO:tasks.rados.rados.0.smithi084.stdout:507: writing smithi08434663-42 from 2873523 to 3465256 tid 3 2024-09-18T14:01:11.285 INFO:tasks.rados.rados.0.smithi084.stdout:508: write (excl) oid 34 current snap is 47 2024-09-18T14:01:11.285 INFO:tasks.rados.rados.0.smithi084.stdout:508: seq_num 210 ranges {729470=543821,1772395=538068,2900946=403005} 2024-09-18T14:01:11.291 INFO:tasks.rados.rados.0.smithi084.stdout:508: writing smithi08434663-34 from 729470 to 1273291 tid 1 2024-09-18T14:01:11.297 INFO:tasks.rados.rados.0.smithi084.stdout:508: writing smithi08434663-34 from 1772395 to 2310463 tid 2 2024-09-18T14:01:11.302 INFO:tasks.rados.rados.0.smithi084.stdout:508: writing smithi08434663-34 from 2900946 to 3303951 tid 3 2024-09-18T14:01:11.302 INFO:tasks.rados.rados.0.smithi084.stdout:509: snap_remove snap 46 2024-09-18T14:01:12.155 INFO:tasks.rados.rados.0.smithi084.stdout:509: done (2 left) 2024-09-18T14:01:12.155 INFO:tasks.rados.rados.0.smithi084.stdout:510: write oid 36 current snap is 47 2024-09-18T14:01:12.155 INFO:tasks.rados.rados.0.smithi084.stdout:510: seq_num 211 ranges {746277=619070,1830877=681726,2928369=617908} 2024-09-18T14:01:12.162 INFO:tasks.rados.rados.0.smithi084.stdout:510: writing smithi08434663-36 from 746277 to 1365347 tid 1 2024-09-18T14:01:12.169 INFO:tasks.rados.rados.0.smithi084.stdout:510: writing smithi08434663-36 from 1830877 to 2512603 tid 2 2024-09-18T14:01:12.176 INFO:tasks.rados.rados.0.smithi084.stdout:510: writing smithi08434663-36 from 2928369 to 3546277 tid 3 2024-09-18T14:01:12.176 INFO:tasks.rados.rados.0.smithi084.stdout:511: read oid 35 snap -1 2024-09-18T14:01:12.176 INFO:tasks.rados.rados.0.smithi084.stdout:511: expect deleted 2024-09-18T14:01:12.176 INFO:tasks.rados.rados.0.smithi084.stdout:512: rollback oid 19 current snap is 47 2024-09-18T14:01:12.176 INFO:tasks.rados.rados.0.smithi084.stdout:507: finishing write tid 1 to smithi08434663-42 2024-09-18T14:01:12.176 INFO:tasks.rados.rados.0.smithi084.stdout:507: finishing write tid 2 to smithi08434663-42 2024-09-18T14:01:12.176 INFO:tasks.rados.rados.0.smithi084.stdout:507: finishing write tid 3 to smithi08434663-42 2024-09-18T14:01:12.177 INFO:tasks.rados.rados.0.smithi084.stdout:507: finishing write tid 4 to smithi08434663-42 2024-09-18T14:01:12.177 INFO:tasks.rados.rados.0.smithi084.stdout:507: finishing write tid 5 to smithi08434663-42 2024-09-18T14:01:12.177 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 118 (ObjNum 209 snap 47 seq_num 209) dirty exists 2024-09-18T14:01:12.177 INFO:tasks.rados.rados.0.smithi084.stdout:507: left oid 42 (ObjNum 209 snap 47 seq_num 209) 2024-09-18T14:01:12.177 INFO:tasks.rados.rados.0.smithi084.stdout:508: finishing write tid 1 to smithi08434663-34 2024-09-18T14:01:12.177 INFO:tasks.rados.rados.0.smithi084.stdout:508: finishing write tid 2 to smithi08434663-34 2024-09-18T14:01:12.177 INFO:tasks.rados.rados.0.smithi084.stdout:507: done (4 left) 2024-09-18T14:01:12.177 INFO:tasks.rados.rados.0.smithi084.stdout:512: done (3 left) 2024-09-18T14:01:12.177 INFO:tasks.rados.rados.0.smithi084.stdout:513: snap_create 2024-09-18T14:01:12.177 INFO:tasks.rados.rados.0.smithi084.stdout:508: finishing write tid 3 to smithi08434663-34 2024-09-18T14:01:12.177 INFO:tasks.rados.rados.0.smithi084.stdout:508: finishing write tid 4 to smithi08434663-34 2024-09-18T14:01:12.177 INFO:tasks.rados.rados.0.smithi084.stdout:508: finishing write tid 5 to smithi08434663-34 2024-09-18T14:01:12.178 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 122 (ObjNum 210 snap 47 seq_num 210) dirty exists 2024-09-18T14:01:12.178 INFO:tasks.rados.rados.0.smithi084.stdout:508: left oid 34 (ObjNum 210 snap 47 seq_num 210) 2024-09-18T14:01:12.178 INFO:tasks.rados.rados.0.smithi084.stdout:510: finishing write tid 1 to smithi08434663-36 2024-09-18T14:01:12.178 INFO:tasks.rados.rados.0.smithi084.stdout:510: finishing write tid 2 to smithi08434663-36 2024-09-18T14:01:12.179 INFO:tasks.rados.rados.0.smithi084.stdout:510: finishing write tid 3 to smithi08434663-36 2024-09-18T14:01:12.179 INFO:tasks.rados.rados.0.smithi084.stdout:510: finishing write tid 4 to smithi08434663-36 2024-09-18T14:01:12.179 INFO:tasks.rados.rados.0.smithi084.stdout:510: finishing write tid 5 to smithi08434663-36 2024-09-18T14:01:12.180 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 141 (ObjNum 211 snap 47 seq_num 211) dirty exists 2024-09-18T14:01:12.180 INFO:tasks.rados.rados.0.smithi084.stdout:510: left oid 36 (ObjNum 211 snap 47 seq_num 211) 2024-09-18T14:01:13.161 INFO:tasks.rados.rados.0.smithi084.stdout:508: done (3 left) 2024-09-18T14:01:13.161 INFO:tasks.rados.rados.0.smithi084.stdout:510: done (2 left) 2024-09-18T14:01:13.162 INFO:tasks.rados.rados.0.smithi084.stdout:511: done (1 left) 2024-09-18T14:01:13.162 INFO:tasks.rados.rados.0.smithi084.stdout:513: done (0 left) 2024-09-18T14:01:13.162 INFO:tasks.rados.rados.0.smithi084.stdout:514: delete oid 10 current snap is 48 2024-09-18T14:01:13.169 INFO:tasks.rados.rados.0.smithi084.stdout:514: done (0 left) 2024-09-18T14:01:13.169 INFO:tasks.rados.rados.0.smithi084.stdout:515: snap_remove snap 47 2024-09-18T14:01:14.163 INFO:tasks.rados.rados.0.smithi084.stdout:515: done (0 left) 2024-09-18T14:01:14.163 INFO:tasks.rados.rados.0.smithi084.stdout:516: snap_create 2024-09-18T14:01:15.171 INFO:tasks.rados.rados.0.smithi084.stdout:516: done (0 left) 2024-09-18T14:01:15.171 INFO:tasks.rados.rados.0.smithi084.stdout:517: write oid 42 current snap is 49 2024-09-18T14:01:15.171 INFO:tasks.rados.rados.0.smithi084.stdout:517: seq_num 213 ranges {779891=485921,1947841=685395,3383215=196676} 2024-09-18T14:01:15.177 INFO:tasks.rados.rados.0.smithi084.stdout:517: writing smithi08434663-42 from 779891 to 1265812 tid 1 2024-09-18T14:01:15.185 INFO:tasks.rados.rados.0.smithi084.stdout:517: writing smithi08434663-42 from 1947841 to 2633236 tid 2 2024-09-18T14:01:15.191 INFO:tasks.rados.rados.0.smithi084.stdout:517: writing smithi08434663-42 from 3383215 to 3579891 tid 3 2024-09-18T14:01:15.191 INFO:tasks.rados.rados.0.smithi084.stdout:518: read oid 24 snap 48 2024-09-18T14:01:15.191 INFO:tasks.rados.rados.0.smithi084.stdout:518: expect (ObjNum 205 snap 46 seq_num 205) 2024-09-18T14:01:15.191 INFO:tasks.rados.rados.0.smithi084.stdout:519: delete oid 10 current snap is 49 2024-09-18T14:01:15.191 INFO:tasks.rados.rados.0.smithi084.stdout:517: finishing write tid 1 to smithi08434663-42 2024-09-18T14:01:15.191 INFO:tasks.rados.rados.0.smithi084.stdout:517: finishing write tid 2 to smithi08434663-42 2024-09-18T14:01:15.193 INFO:tasks.rados.rados.0.smithi084.stdout:517: finishing write tid 3 to smithi08434663-42 2024-09-18T14:01:15.193 INFO:tasks.rados.rados.0.smithi084.stdout:517: finishing write tid 4 to smithi08434663-42 2024-09-18T14:01:15.193 INFO:tasks.rados.rados.0.smithi084.stdout:517: finishing write tid 5 to smithi08434663-42 2024-09-18T14:01:15.194 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 129 (ObjNum 213 snap 49 seq_num 213) dirty exists 2024-09-18T14:01:15.194 INFO:tasks.rados.rados.0.smithi084.stdout:517: left oid 42 (ObjNum 213 snap 49 seq_num 213) 2024-09-18T14:01:15.204 INFO:tasks.rados.rados.0.smithi084.stdout:518: expect (ObjNum 205 snap 46 seq_num 205) 2024-09-18T14:01:15.426 INFO:tasks.rados.rados.0.smithi084.stdout:517: done (2 left) 2024-09-18T14:01:15.426 INFO:tasks.rados.rados.0.smithi084.stdout:518: done (1 left) 2024-09-18T14:01:15.426 INFO:tasks.rados.rados.0.smithi084.stdout:519: done (0 left) 2024-09-18T14:01:15.426 INFO:tasks.rados.rados.0.smithi084.stdout:520: read oid 12 snap 48 2024-09-18T14:01:15.426 INFO:tasks.rados.rados.0.smithi084.stdout:520: expect (ObjNum 106 snap 19 seq_num 106) 2024-09-18T14:01:15.426 INFO:tasks.rados.rados.0.smithi084.stdout:521: read oid 41 snap -1 2024-09-18T14:01:15.426 INFO:tasks.rados.rados.0.smithi084.stdout:521: expect (ObjNum 152 snap 31 seq_num 152) 2024-09-18T14:01:15.426 INFO:tasks.rados.rados.0.smithi084.stdout:522: snap_create 2024-09-18T14:01:15.429 INFO:tasks.rados.rados.0.smithi084.stdout:520: expect (ObjNum 106 snap 19 seq_num 106) 2024-09-18T14:01:15.549 INFO:tasks.rados.rados.0.smithi084.stdout:521: expect (ObjNum 152 snap 31 seq_num 152) 2024-09-18T14:01:16.185 INFO:tasks.rados.rados.0.smithi084.stdout:520: done (2 left) 2024-09-18T14:01:16.185 INFO:tasks.rados.rados.0.smithi084.stdout:521: done (1 left) 2024-09-18T14:01:16.185 INFO:tasks.rados.rados.0.smithi084.stdout:522: done (0 left) 2024-09-18T14:01:16.185 INFO:tasks.rados.rados.0.smithi084.stdout:523: copy_from oid 14 from oid 15 current snap is 50 2024-09-18T14:01:16.185 INFO:tasks.rados.rados.0.smithi084.stdout:524: write oid 39 current snap is 50 2024-09-18T14:01:16.186 INFO:tasks.rados.rados.0.smithi084.stdout:524: seq_num 215 ranges {413505=636419,1664805=689064,3038061=492071} 2024-09-18T14:01:16.191 INFO:tasks.rados.rados.0.smithi084.stdout:524: writing smithi08434663-39 from 413505 to 1049924 tid 1 2024-09-18T14:01:16.199 INFO:tasks.rados.rados.0.smithi084.stdout:524: writing smithi08434663-39 from 1664805 to 2353869 tid 2 2024-09-18T14:01:16.206 INFO:tasks.rados.rados.0.smithi084.stdout:524: writing smithi08434663-39 from 3038061 to 3530132 tid 3 2024-09-18T14:01:16.206 INFO:tasks.rados.rados.0.smithi084.stdout:525: write oid 33 current snap is 50 2024-09-18T14:01:16.206 INFO:tasks.rados.rados.0.smithi084.stdout:523: finishing copy_from racing read to smithi08434663-14 2024-09-18T14:01:16.206 INFO:tasks.rados.rados.0.smithi084.stdout:523: finishing copy_from to smithi08434663-14 2024-09-18T14:01:16.206 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 177 (ObjNum 147 snap 30 seq_num 147) dirty exists 2024-09-18T14:01:16.207 INFO:tasks.rados.rados.0.smithi084.stdout:524: finishing write tid 1 to smithi08434663-39 2024-09-18T14:01:16.207 INFO:tasks.rados.rados.0.smithi084.stdout:524: finishing write tid 2 to smithi08434663-39 2024-09-18T14:01:16.207 INFO:tasks.rados.rados.0.smithi084.stdout:525: seq_num 216 ranges {430312=711668,1723287=549075,2781837=703343} 2024-09-18T14:01:16.212 INFO:tasks.rados.rados.0.smithi084.stdout:525: writing smithi08434663-33 from 430312 to 1141980 tid 1 2024-09-18T14:01:16.218 INFO:tasks.rados.rados.0.smithi084.stdout:525: writing smithi08434663-33 from 1723287 to 2272362 tid 2 2024-09-18T14:01:16.226 INFO:tasks.rados.rados.0.smithi084.stdout:525: writing smithi08434663-33 from 2781837 to 3485180 tid 3 2024-09-18T14:01:16.226 INFO:tasks.rados.rados.0.smithi084.stdout:523: done (2 left) 2024-09-18T14:01:16.226 INFO:tasks.rados.rados.0.smithi084.stdout:526: read oid 44 snap -1 2024-09-18T14:01:16.226 INFO:tasks.rados.rados.0.smithi084.stdout:526: expect (ObjNum 88 snap 9 seq_num 88) 2024-09-18T14:01:16.226 INFO:tasks.rados.rados.0.smithi084.stdout:527: write oid 13 current snap is 50 2024-09-18T14:01:16.226 INFO:tasks.rados.rados.0.smithi084.stdout:524: finishing write tid 3 to smithi08434663-39 2024-09-18T14:01:16.226 INFO:tasks.rados.rados.0.smithi084.stdout:524: finishing write tid 4 to smithi08434663-39 2024-09-18T14:01:16.226 INFO:tasks.rados.rados.0.smithi084.stdout:527: seq_num 217 ranges {447119=786917,1665416=692733,2976554=514615} 2024-09-18T14:01:16.232 INFO:tasks.rados.rados.0.smithi084.stdout:527: writing smithi08434663-13 from 447119 to 1234036 tid 1 2024-09-18T14:01:16.238 INFO:tasks.rados.rados.0.smithi084.stdout:527: writing smithi08434663-13 from 1665416 to 2358149 tid 2 2024-09-18T14:01:16.245 INFO:tasks.rados.rados.0.smithi084.stdout:527: writing smithi08434663-13 from 2976554 to 3491169 tid 3 2024-09-18T14:01:16.245 INFO:tasks.rados.rados.0.smithi084.stdout:528: write oid 46 current snap is 50 2024-09-18T14:01:16.245 INFO:tasks.rados.rados.0.smithi084.stdout:524: finishing write tid 5 to smithi08434663-39 2024-09-18T14:01:16.245 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 179 (ObjNum 215 snap 50 seq_num 215) dirty exists 2024-09-18T14:01:16.245 INFO:tasks.rados.rados.0.smithi084.stdout:524: left oid 39 (ObjNum 215 snap 50 seq_num 215) 2024-09-18T14:01:16.245 INFO:tasks.rados.rados.0.smithi084.stdout:525: finishing write tid 1 to smithi08434663-33 2024-09-18T14:01:16.246 INFO:tasks.rados.rados.0.smithi084.stdout:525: finishing write tid 2 to smithi08434663-33 2024-09-18T14:01:16.246 INFO:tasks.rados.rados.0.smithi084.stdout:528: seq_num 218 ranges {463926=462166,1723898=436391,2603977=725887} 2024-09-18T14:01:16.250 INFO:tasks.rados.rados.0.smithi084.stdout:528: writing smithi08434663-46 from 463926 to 926092 tid 1 2024-09-18T14:01:16.257 INFO:tasks.rados.rados.0.smithi084.stdout:528: writing smithi08434663-46 from 1723898 to 2160289 tid 2 2024-09-18T14:01:16.263 INFO:tasks.rados.rados.0.smithi084.stdout:528: writing smithi08434663-46 from 2603977 to 3329864 tid 3 2024-09-18T14:01:16.263 INFO:tasks.rados.rados.0.smithi084.stdout:524: done (4 left) 2024-09-18T14:01:16.263 INFO:tasks.rados.rados.0.smithi084.stdout:529: write (excl) oid 4 current snap is 50 2024-09-18T14:01:16.263 INFO:tasks.rados.rados.0.smithi084.stdout:529: seq_num 219 ranges {480733=537415,1782380=696402,3031400=537159} 2024-09-18T14:01:16.269 INFO:tasks.rados.rados.0.smithi084.stdout:529: writing smithi08434663-4 from 480733 to 1018148 tid 1 2024-09-18T14:01:16.277 INFO:tasks.rados.rados.0.smithi084.stdout:529: writing smithi08434663-4 from 1782380 to 2478782 tid 2 2024-09-18T14:01:16.283 INFO:tasks.rados.rados.0.smithi084.stdout:529: writing smithi08434663-4 from 3031400 to 3568559 tid 3 2024-09-18T14:01:16.283 INFO:tasks.rados.rados.0.smithi084.stdout:530: rollback oid 6 current snap is 50 2024-09-18T14:01:16.283 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 49 2024-09-18T14:01:16.283 INFO:tasks.rados.rados.0.smithi084.stdout:525: finishing write tid 3 to smithi08434663-33 2024-09-18T14:01:16.283 INFO:tasks.rados.rados.0.smithi084.stdout:525: finishing write tid 4 to smithi08434663-33 2024-09-18T14:01:16.284 INFO:tasks.rados.rados.0.smithi084.stdout:525: finishing write tid 5 to smithi08434663-33 2024-09-18T14:01:16.284 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 146 (ObjNum 216 snap 50 seq_num 216) dirty exists 2024-09-18T14:01:16.284 INFO:tasks.rados.rados.0.smithi084.stdout:525: left oid 33 (ObjNum 216 snap 50 seq_num 216) 2024-09-18T14:01:16.284 INFO:tasks.rados.rados.0.smithi084.stdout:526: expect (ObjNum 88 snap 9 seq_num 88) 2024-09-18T14:01:16.441 INFO:tasks.rados.rados.0.smithi084.stdout:527: finishing write tid 1 to smithi08434663-13 2024-09-18T14:01:16.441 INFO:tasks.rados.rados.0.smithi084.stdout:527: finishing write tid 2 to smithi08434663-13 2024-09-18T14:01:16.441 INFO:tasks.rados.rados.0.smithi084.stdout:527: finishing write tid 3 to smithi08434663-13 2024-09-18T14:01:16.441 INFO:tasks.rados.rados.0.smithi084.stdout:527: finishing write tid 4 to smithi08434663-13 2024-09-18T14:01:16.441 INFO:tasks.rados.rados.0.smithi084.stdout:527: finishing write tid 5 to smithi08434663-13 2024-09-18T14:01:16.441 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 140 (ObjNum 217 snap 50 seq_num 217) dirty exists 2024-09-18T14:01:16.442 INFO:tasks.rados.rados.0.smithi084.stdout:527: left oid 13 (ObjNum 217 snap 50 seq_num 217) 2024-09-18T14:01:16.442 INFO:tasks.rados.rados.0.smithi084.stdout:528: finishing write tid 1 to smithi08434663-46 2024-09-18T14:01:16.442 INFO:tasks.rados.rados.0.smithi084.stdout:528: finishing write tid 2 to smithi08434663-46 2024-09-18T14:01:16.442 INFO:tasks.rados.rados.0.smithi084.stdout:525: done (5 left) 2024-09-18T14:01:16.442 INFO:tasks.rados.rados.0.smithi084.stdout:526: done (4 left) 2024-09-18T14:01:16.442 INFO:tasks.rados.rados.0.smithi084.stdout:527: done (3 left) 2024-09-18T14:01:16.442 INFO:tasks.rados.rados.0.smithi084.stdout:531: read oid 44 snap 48 2024-09-18T14:01:16.442 INFO:tasks.rados.rados.0.smithi084.stdout:531: expect (ObjNum 88 snap 9 seq_num 88) 2024-09-18T14:01:16.442 INFO:tasks.rados.rados.0.smithi084.stdout:532: snap_create 2024-09-18T14:01:16.442 INFO:tasks.rados.rados.0.smithi084.stdout:528: finishing write tid 3 to smithi08434663-46 2024-09-18T14:01:16.442 INFO:tasks.rados.rados.0.smithi084.stdout:528: finishing write tid 4 to smithi08434663-46 2024-09-18T14:01:16.443 INFO:tasks.rados.rados.0.smithi084.stdout:528: finishing write tid 5 to smithi08434663-46 2024-09-18T14:01:16.443 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 190 (ObjNum 218 snap 50 seq_num 218) dirty exists 2024-09-18T14:01:16.443 INFO:tasks.rados.rados.0.smithi084.stdout:528: left oid 46 (ObjNum 218 snap 50 seq_num 218) 2024-09-18T14:01:16.443 INFO:tasks.rados.rados.0.smithi084.stdout:529: finishing write tid 1 to smithi08434663-4 2024-09-18T14:01:16.443 INFO:tasks.rados.rados.0.smithi084.stdout:529: finishing write tid 2 to smithi08434663-4 2024-09-18T14:01:16.443 INFO:tasks.rados.rados.0.smithi084.stdout:530: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:01:16.443 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 0 (ObjNum 33 snap 0 seq_num 2495420544) dirty dne 2024-09-18T14:01:16.443 INFO:tasks.rados.rados.0.smithi084.stdout:529: finishing write tid 3 to smithi08434663-4 2024-09-18T14:01:16.443 INFO:tasks.rados.rados.0.smithi084.stdout:529: finishing write tid 4 to smithi08434663-4 2024-09-18T14:01:16.443 INFO:tasks.rados.rados.0.smithi084.stdout:529: finishing write tid 5 to smithi08434663-4 2024-09-18T14:01:16.443 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 134 (ObjNum 219 snap 50 seq_num 219) dirty exists 2024-09-18T14:01:16.443 INFO:tasks.rados.rados.0.smithi084.stdout:529: left oid 4 (ObjNum 219 snap 50 seq_num 219) 2024-09-18T14:01:16.444 INFO:tasks.rados.rados.0.smithi084.stdout:531: expect (ObjNum 88 snap 9 seq_num 88) 2024-09-18T14:01:17.156 INFO:tasks.rados.rados.0.smithi084.stdout:528: done (4 left) 2024-09-18T14:01:17.156 INFO:tasks.rados.rados.0.smithi084.stdout:529: done (3 left) 2024-09-18T14:01:17.156 INFO:tasks.rados.rados.0.smithi084.stdout:530: done (2 left) 2024-09-18T14:01:17.156 INFO:tasks.rados.rados.0.smithi084.stdout:531: done (1 left) 2024-09-18T14:01:17.157 INFO:tasks.rados.rados.0.smithi084.stdout:532: done (0 left) 2024-09-18T14:01:17.157 INFO:tasks.rados.rados.0.smithi084.stdout:533: snap_remove snap 48 2024-09-18T14:01:18.157 INFO:tasks.rados.rados.0.smithi084.stdout:533: done (0 left) 2024-09-18T14:01:18.157 INFO:tasks.rados.rados.0.smithi084.stdout:534: snap_create 2024-09-18T14:01:19.153 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-18T14:01:19.153 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-18T14:01:19.154 INFO:tasks.mon_thrash:Sending CONT to mon f 2024-09-18T14:01:19.154 INFO:tasks.ceph.mon.f:Sent signal 18 2024-09-18T14:01:19.154 INFO:tasks.mon_thrash:Sending CONT to mon i 2024-09-18T14:01:19.154 INFO:tasks.ceph.mon.i:Sent signal 18 2024-09-18T14:01:19.154 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:01:19.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:01:19.162 INFO:tasks.rados.rados.0.smithi084.stdout:534: done (0 left) 2024-09-18T14:01:19.163 INFO:tasks.rados.rados.0.smithi084.stdout:535: read oid 14 snap -1 2024-09-18T14:01:19.163 INFO:tasks.rados.rados.0.smithi084.stdout:535: expect (ObjNum 147 snap 30 seq_num 147) 2024-09-18T14:01:19.163 INFO:tasks.rados.rados.0.smithi084.stdout:536: copy_from oid 32 from oid 19 current snap is 52 2024-09-18T14:01:19.163 INFO:tasks.rados.rados.0.smithi084.stdout:537: rollback oid 41 current snap is 52 2024-09-18T14:01:19.163 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 41 to 51 2024-09-18T14:01:19.163 INFO:tasks.rados.rados.0.smithi084.stdout:538: copy_from oid 17 from oid 29 current snap is 52 2024-09-18T14:01:19.163 INFO:tasks.rados.rados.0.smithi084.stdout:539: write (excl) oid 10 current snap is 52 2024-09-18T14:01:19.163 INFO:tasks.rados.rados.0.smithi084.stdout:539: seq_num 220 ranges {497540=612664,1840862=440060,3058823=464784} 2024-09-18T14:01:19.169 INFO:tasks.rados.rados.0.smithi084.stdout:539: writing smithi08434663-10 from 497540 to 1110204 tid 1 2024-09-18T14:01:19.176 INFO:tasks.rados.rados.0.smithi084.stdout:539: writing smithi08434663-10 from 1840862 to 2280922 tid 2 2024-09-18T14:01:19.185 INFO:tasks.rados.rados.0.smithi084.stdout:539: writing smithi08434663-10 from 3058823 to 3523607 tid 3 2024-09-18T14:01:19.185 INFO:tasks.rados.rados.0.smithi084.stdout:540: write (excl) oid 48 current snap is 52 2024-09-18T14:01:19.185 INFO:tasks.rados.rados.0.smithi084.stdout:537: finishing rollback tid 0 to smithi08434663-41 2024-09-18T14:01:19.185 INFO:tasks.rados.rados.0.smithi084.stdout:540: seq_num 221 ranges {514347=404266,1499344=700071,2686246=676056} 2024-09-18T14:01:19.190 INFO:tasks.rados.rados.0.smithi084.stdout:540: writing smithi08434663-48 from 514347 to 918613 tid 1 2024-09-18T14:01:19.197 INFO:tasks.rados.rados.0.smithi084.stdout:540: writing smithi08434663-48 from 1499344 to 2199415 tid 2 2024-09-18T14:01:19.204 INFO:tasks.rados.rados.0.smithi084.stdout:540: writing smithi08434663-48 from 2686246 to 3362302 tid 3 2024-09-18T14:01:19.204 INFO:tasks.rados.rados.0.smithi084.stdout:541: read oid 11 snap -1 2024-09-18T14:01:19.204 INFO:tasks.rados.rados.0.smithi084.stdout:541: expect (ObjNum 178 snap 41 seq_num 178) 2024-09-18T14:01:19.204 INFO:tasks.rados.rados.0.smithi084.stdout:542: write (excl) oid 13 current snap is 52 2024-09-18T14:01:19.204 INFO:tasks.rados.rados.0.smithi084.stdout:537: finishing rollback tid 1 to smithi08434663-41 2024-09-18T14:01:19.204 INFO:tasks.rados.rados.0.smithi084.stdout:537: finishing rollback tid 2 to smithi08434663-41 2024-09-18T14:01:19.204 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 142 (ObjNum 152 snap 31 seq_num 152) dirty exists 2024-09-18T14:01:19.204 INFO:tasks.rados.rados.0.smithi084.stdout:538: finishing copy_from racing read to smithi08434663-17 2024-09-18T14:01:19.204 INFO:tasks.rados.rados.0.smithi084.stdout:535: expect (ObjNum 147 snap 30 seq_num 147) 2024-09-18T14:01:19.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 10 monclient: get_monmap_and_config 2024-09-18T14:01:19.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 10 monclient: build_initial_monmap 2024-09-18T14:01:19.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 10 monclient: monmap: 2024-09-18T14:01:19.270 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:19.270 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:19.270 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:19.270464+0000 2024-09-18T14:01:19.270 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:19.270464+0000 2024-09-18T14:01:19.270 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:19.270 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:19.271 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:19.271 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:19.271 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:19.271 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:19.271 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:19.271 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:19.271 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:19.271 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:19.271 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:19.271 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 1 Processor -- start 2024-09-18T14:01:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 1 -- start start 2024-09-18T14:01:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 10 monclient: init 2024-09-18T14:01:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 10 monclient: _add_conns ranks=[3,8,0,4,7,5,2,6,1] 2024-09-18T14:01:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 10 monclient(hunting): picked mon.noname-g con 0x7f5b1c09f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:01:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 10 monclient(hunting): picked mon.noname-e con 0x7f5b1c09fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 10 monclient(hunting): picked mon.noname-a con 0x7f5b1c09ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1c05ed70 con 0x7f5b1c09ff70 2024-09-18T14:01:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1c09dfc0 con 0x7f5b1c09f7d0 2024-09-18T14:01:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1c09dce0 con 0x7f5b1c09fba0 2024-09-18T14:01:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.269+0000 7f5b22a4f640 10 monclient(hunting): authenticate will time out at 894.608975s 2024-09-18T14:01:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.270+0000 7f5b20fec640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5b1c09ff70 0x7f5b1c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54834/0 (socket says 172.21.15.45:54834) 2024-09-18T14:01:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.270+0000 7f5b1b7fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f5b1c09fba0 0x7f5b1c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:46894/0 (socket says 172.21.15.45:46894) 2024-09-18T14:01:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:19.270+0000 7f5b20fec640 1 -- 172.21.15.45:0/975618253 learned_addr learned my addr 172.21.15.45:0/975618253 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:19.313 INFO:tasks.rados.rados.0.smithi084.stdout:538: finishing copy_from to smithi08434663-17 2024-09-18T14:01:19.313 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 41 (ObjNum 136 snap 29 seq_num 136) dirty exists 2024-09-18T14:01:19.313 INFO:tasks.rados.rados.0.smithi084.stdout:539: finishing write tid 1 to smithi08434663-10 2024-09-18T14:01:19.313 INFO:tasks.rados.rados.0.smithi084.stdout:539: finishing write tid 2 to smithi08434663-10 2024-09-18T14:01:19.314 INFO:tasks.rados.rados.0.smithi084.stdout:536: finishing copy_from racing read to smithi08434663-32 2024-09-18T14:01:19.314 INFO:tasks.rados.rados.0.smithi084.stdout:539: finishing write tid 3 to smithi08434663-10 2024-09-18T14:01:19.314 INFO:tasks.rados.rados.0.smithi084.stdout:539: finishing write tid 4 to smithi08434663-10 2024-09-18T14:01:19.314 INFO:tasks.rados.rados.0.smithi084.stdout:539: finishing write tid 5 to smithi08434663-10 2024-09-18T14:01:19.314 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 157 (ObjNum 220 snap 52 seq_num 220) dirty exists 2024-09-18T14:01:19.314 INFO:tasks.rados.rados.0.smithi084.stdout:539: left oid 10 (ObjNum 220 snap 52 seq_num 220) 2024-09-18T14:01:19.314 INFO:tasks.rados.rados.0.smithi084.stdout:536: finishing copy_from to smithi08434663-32 2024-09-18T14:01:19.314 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 232 (ObjNum 204 snap 46 seq_num 204) dirty exists 2024-09-18T14:01:19.314 INFO:tasks.rados.rados.0.smithi084.stdout:540: finishing write tid 1 to smithi08434663-48 2024-09-18T14:01:19.314 INFO:tasks.rados.rados.0.smithi084.stdout:542: seq_num 222 ranges {531154=479515,1557826=443729,2713669=487328,3679295=51859} 2024-09-18T14:01:19.318 INFO:tasks.rados.rados.0.smithi084.stdout:542: writing smithi08434663-13 from 531154 to 1010669 tid 1 2024-09-18T14:01:19.324 INFO:tasks.rados.rados.0.smithi084.stdout:542: writing smithi08434663-13 from 1557826 to 2001555 tid 2 2024-09-18T14:01:19.330 INFO:tasks.rados.rados.0.smithi084.stdout:542: writing smithi08434663-13 from 2713669 to 3200997 tid 3 2024-09-18T14:01:19.333 INFO:tasks.rados.rados.0.smithi084.stdout:542: writing smithi08434663-13 from 3679295 to 3731154 tid 4 2024-09-18T14:01:19.334 INFO:tasks.rados.rados.0.smithi084.stdout:535: done (7 left) 2024-09-18T14:01:19.334 INFO:tasks.rados.rados.0.smithi084.stdout:536: done (6 left) 2024-09-18T14:01:19.334 INFO:tasks.rados.rados.0.smithi084.stdout:537: done (5 left) 2024-09-18T14:01:19.334 INFO:tasks.rados.rados.0.smithi084.stdout:538: done (4 left) 2024-09-18T14:01:19.334 INFO:tasks.rados.rados.0.smithi084.stdout:539: done (3 left) 2024-09-18T14:01:19.334 INFO:tasks.rados.rados.0.smithi084.stdout:543: read oid 28 snap -1 2024-09-18T14:01:19.334 INFO:tasks.rados.rados.0.smithi084.stdout:543: expect deleted 2024-09-18T14:01:19.334 INFO:tasks.rados.rados.0.smithi084.stdout:544: copy_from oid 19 from oid 20 current snap is 52 2024-09-18T14:01:19.334 INFO:tasks.rados.rados.0.smithi084.stdout:540: finishing write tid 2 to smithi08434663-48 2024-09-18T14:01:19.334 INFO:tasks.rados.rados.0.smithi084.stdout:540: finishing write tid 3 to smithi08434663-48 2024-09-18T14:01:19.334 INFO:tasks.rados.rados.0.smithi084.stdout:541: expect (ObjNum 178 snap 41 seq_num 178) 2024-09-18T14:01:19.502 INFO:tasks.rados.rados.0.smithi084.stdout:540: finishing write tid 4 to smithi08434663-48 2024-09-18T14:01:19.502 INFO:tasks.rados.rados.0.smithi084.stdout:540: finishing write tid 5 to smithi08434663-48 2024-09-18T14:01:19.502 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 130 (ObjNum 221 snap 52 seq_num 221) dirty exists 2024-09-18T14:01:19.502 INFO:tasks.rados.rados.0.smithi084.stdout:540: left oid 48 (ObjNum 221 snap 52 seq_num 221) 2024-09-18T14:01:19.502 INFO:tasks.rados.rados.0.smithi084.stdout:542: finishing write tid 1 to smithi08434663-13 2024-09-18T14:01:19.502 INFO:tasks.rados.rados.0.smithi084.stdout:542: finishing write tid 2 to smithi08434663-13 2024-09-18T14:01:19.502 INFO:tasks.rados.rados.0.smithi084.stdout:542: finishing write tid 3 to smithi08434663-13 2024-09-18T14:01:19.502 INFO:tasks.rados.rados.0.smithi084.stdout:544: finishing copy_from to smithi08434663-19 2024-09-18T14:01:19.502 INFO:tasks.rados.rados.0.smithi084.stdout:544: got expected ENOENT (src dne) 2024-09-18T14:01:19.502 INFO:tasks.rados.rados.0.smithi084.stdout:544: finishing copy_from racing read to smithi08434663-19 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:542: finishing write tid 4 to smithi08434663-13 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:542: finishing write tid 5 to smithi08434663-13 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:542: finishing write tid 6 to smithi08434663-13 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 149 (ObjNum 222 snap 52 seq_num 222) dirty exists 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:542: left oid 13 (ObjNum 222 snap 52 seq_num 222) 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:540: done (4 left) 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:541: done (3 left) 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:542: done (2 left) 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:543: done (1 left) 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:544: done (0 left) 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:545: rollback oid 15 current snap is 52 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 15 to 50 2024-09-18T14:01:19.503 INFO:tasks.rados.rados.0.smithi084.stdout:546: snap_remove snap 49 2024-09-18T14:01:22.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 10 monclient(hunting): tick 2024-09-18T14:01:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 1 monclient(hunting): continuing hunt 2024-09-18T14:01:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:01:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 >> v1:172.21.15.45:6793/0 conn(0x7f5b1c09fba0 legacy=0x7f5b1c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 >> v1:172.21.15.84:6790/0 conn(0x7f5b1c09f7d0 legacy=0x7f5b1c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 >> v1:172.21.15.45:6789/0 conn(0x7f5b1c09ff70 legacy=0x7f5b1c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 10 monclient: _add_conns ranks=[1,6,5,2,4,7] 2024-09-18T14:01:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 10 monclient(hunting): picked mon.noname-f con 0x7f5b080037a0 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 10 monclient(hunting): picked mon.noname-d con 0x7f5b08007380 addr v1:172.21.15.45:6792/0 2024-09-18T14:01:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 10 monclient(hunting): picked mon.noname-h con 0x7f5b0800b100 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1c0a1c00 con 0x7f5b080037a0 2024-09-18T14:01:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0800ff80 con 0x7f5b0800b100 2024-09-18T14:01:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b080111c0 con 0x7f5b08007380 2024-09-18T14:01:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:22.271+0000 7f5b19ffb640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:25.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 10 monclient(hunting): tick 2024-09-18T14:01:25.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 1 monclient(hunting): continuing hunt 2024-09-18T14:01:25.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:01:25.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 >> v1:172.21.15.45:6792/0 conn(0x7f5b08007380 legacy=0x7f5b08009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:25.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 >> v1:172.21.15.84:6791/0 conn(0x7f5b0800b100 legacy=0x7f5b0800d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:25.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 >> v1:172.21.15.84:6789/0 conn(0x7f5b080037a0 legacy=0x7f5b08005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:25.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 10 monclient: _add_conns ranks=[2,4,7] 2024-09-18T14:01:25.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 10 monclient(hunting): picked mon.noname-b con 0x7f5b08010390 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:25.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 10 monclient(hunting): picked mon.noname-c con 0x7f5b08011350 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:25.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b20fec640 1 -- 172.21.15.45:0/975618253 reap_dead start 2024-09-18T14:01:25.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 10 monclient(hunting): picked mon.noname-i con 0x7f5b08017740 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:25.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b08012420 con 0x7f5b08010390 2024-09-18T14:01:25.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0800a000 con 0x7f5b08011350 2024-09-18T14:01:25.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0800a190 con 0x7f5b08017740 2024-09-18T14:01:25.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b19ffb640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:25.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:25.271+0000 7f5b20fec640 1 -- 172.21.15.45:0/975618253 reap_dead start 2024-09-18T14:01:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 10 monclient(hunting): tick 2024-09-18T14:01:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 1 monclient(hunting): continuing hunt 2024-09-18T14:01:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:01:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 >> v1:172.21.15.84:6792/0 conn(0x7f5b08017740 legacy=0x7f5b08019b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 >> v1:172.21.15.45:6791/0 conn(0x7f5b08011350 legacy=0x7f5b080117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 >> v1:172.21.15.45:6790/0 conn(0x7f5b08010390 legacy=0x7f5b080107c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 10 monclient: _add_conns ranks=[1,3,0,7,4,2,6,5,8] 2024-09-18T14:01:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 10 monclient(hunting): picked mon.noname-f con 0x7f5b0800a320 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 10 monclient(hunting): picked mon.noname-g con 0x7f5b08006310 addr v1:172.21.15.84:6790/0 2024-09-18T14:01:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 10 monclient(hunting): picked mon.noname-a con 0x7f5b080066e0 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b080098b0 con 0x7f5b080066e0 2024-09-18T14:01:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b08009bd0 con 0x7f5b0800a320 2024-09-18T14:01:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 1 -- 172.21.15.45:0/975618253 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b08009d60 con 0x7f5b08006310 2024-09-18T14:01:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:28.271+0000 7f5b19ffb640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:29.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 1 -- 172.21.15.45:0/975618253 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208563952 0 0) 0x7f5b080098b0 con 0x7f5b080066e0 2024-09-18T14:01:29.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:29.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:29.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 10 monclient(hunting): my global_id is 4599 2024-09-18T14:01:29.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 1 -- 172.21.15.45:0/975618253 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b040034e0 con 0x7f5b080066e0 2024-09-18T14:01:29.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 1 -- 172.21.15.45:0/975618253 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602053623 0 0) 0x7f5b08009d60 con 0x7f5b08006310 2024-09-18T14:01:29.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:29.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 10 monclient(hunting): my global_id is 4656 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 1 -- 172.21.15.45:0/975618253 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b080098b0 con 0x7f5b08006310 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 1 -- 172.21.15.45:0/975618253 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 478129972 0 0) 0x7f5b040034e0 con 0x7f5b080066e0 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 1 -- 172.21.15.45:0/975618253 >> v1:172.21.15.84:6790/0 conn(0x7f5b08006310 legacy=0x7f5b0800d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 1 -- 172.21.15.45:0/975618253 >> v1:172.21.15.84:6789/0 conn(0x7f5b0800a320 legacy=0x7f5b0800a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 10 monclient: _finish_hunting 0 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 1 monclient: found mon.noname-a 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.187+0000 7f5b1affd640 1 -- 172.21.15.45:0/975618253 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b08005c90 con 0x7f5b080066e0 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b1affd640 10 monclient: _finish_auth 0 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:29.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b1affd640 1 -- 172.21.15.45:0/975618253 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5b14001ab0 con 0x7f5b080066e0 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b22a4f640 5 monclient: authenticate success, global_id 4599 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b22a4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b20fec640 1 -- 172.21.15.45:0/975618253 reap_dead start 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b1affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b1affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b1affd640 10 monclient: dump: 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:29.191 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:29.192 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:29.192 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:29.192 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:29.192 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:29.192 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:29.192 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:29.192 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:29.192 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b22a4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b1affd640 1 -- 172.21.15.45:0/975618253 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5b14001c10 con 0x7f5b080066e0 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b1affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b1affd640 1 -- 172.21.15.45:0/975618253 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5b14004be0 con 0x7f5b080066e0 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b22a4f640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b22a4f640 4 set_mon_vals no callback set 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b1affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b1affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.188+0000 7f5b1affd640 10 monclient: dump: 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:29.194 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:29.194 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:29.194 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:29.194 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:29.194 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:29.194 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:29.194 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:29.194 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:29.194 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:29.194 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:29.194 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.189+0000 7f5b22a4f640 10 monclient: shutdown 2024-09-18T14:01:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.189+0000 7f5b22a4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.189+0000 7f5b22a4f640 1 -- 172.21.15.45:0/975618253 >> v1:172.21.15.45:6789/0 conn(0x7f5b080066e0 legacy=0x7f5b08006ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.189+0000 7f5b22a4f640 1 -- 172.21.15.45:0/975618253 shutdown_connections 2024-09-18T14:01:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.189+0000 7f5b22a4f640 1 -- 172.21.15.45:0/975618253 >> 172.21.15.45:0/975618253 conn(0x7f5b1c0630b0 msgr2=0x7f5b1c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.189+0000 7f5b22a4f640 1 -- 172.21.15.45:0/975618253 shutdown_connections 2024-09-18T14:01:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.189+0000 7f5b22a4f640 1 -- 172.21.15.45:0/975618253 wait complete. 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.190+0000 7f5b22a4f640 10 monclient: build_initial_monmap 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.190+0000 7f5b22a4f640 10 monclient: monmap: 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:29.201 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:29.201 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:29.201 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:29.201 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:29.201 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:29.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.190+0000 7f5b22a4f640 1 Processor -- start 2024-09-18T14:01:29.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b22a4f640 1 -- start start 2024-09-18T14:01:29.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b22a4f640 10 monclient: init 2024-09-18T14:01:29.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b22a4f640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:29.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b22a4f640 10 monclient: _add_conns ranks=[7,1,4,0,3,8,5,6,2] 2024-09-18T14:01:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b22a4f640 10 monclient(hunting): picked mon.noname-i con 0x7f5b080066e0 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b22a4f640 10 monclient(hunting): picked mon.noname-f con 0x7f5b1c09b1c0 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b22a4f640 10 monclient(hunting): picked mon.noname-c con 0x7f5b1c1483e0 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b22a4f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1c0a2560 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b22a4f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1c11eb20 con 0x7f5b1c1483e0 2024-09-18T14:01:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b22a4f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1c09b590 con 0x7f5b080066e0 2024-09-18T14:01:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b22a4f640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b22a4f640 10 monclient(hunting): authenticate will time out at 904.529847s 2024-09-18T14:01:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b20fec640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f5b1c1483e0 0x7f5b1c14a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36852/0 (socket says 172.21.15.45:36852) 2024-09-18T14:01:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.191+0000 7f5b20fec640 1 -- 172.21.15.45:0/3381323075 learned_addr learned my addr 172.21.15.45:0/3381323075 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.195+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443457110 0 0) 0x7f5b1c0a2560 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.195+0000 7f5b03fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.195+0000 7f5b03fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.195+0000 7f5b03fff640 10 monclient(hunting): my global_id is 4528 2024-09-18T14:01:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.195+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b08009a40 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.195+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859859832 0 0) 0x7f5b1c11eb20 con 0x7f5b1c1483e0 2024-09-18T14:01:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient(hunting): my global_id is 134104 2024-09-18T14:01:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b1c0a2560 con 0x7f5b1c1483e0 2024-09-18T14:01:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2607854281 0 0) 0x7f5b08009a40 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5b1c11eb20 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5b100019b0 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:01:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient(hunting): dump: 2024-09-18T14:01:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:29.205 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:29.205 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:29.205 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:29.205 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:29.205 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:29.205 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:29.205 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:29.206 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 871511704 0 0) 0x7f5b1c0a2560 con 0x7f5b1c1483e0 2024-09-18T14:01:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5b08009a40 con 0x7f5b1c1483e0 2024-09-18T14:01:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5b140020c0 con 0x7f5b1c1483e0 2024-09-18T14:01:29.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:29.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:29.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient(hunting): dump: 2024-09-18T14:01:29.207 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:29.207 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:29.207 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:29.207 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:29.207 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:29.208 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:29.208 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:29.208 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:29.208 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:29.208 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:29.208 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:29.208 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:29.208 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:29.208 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:29.208 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:29.208 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2740292782 0 0) 0x7f5b1c11eb20 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 >> v1:172.21.15.84:6792/0 conn(0x7f5b080066e0 legacy=0x7f5b1c131700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 >> v1:172.21.15.45:6791/0 conn(0x7f5b1c1483e0 legacy=0x7f5b1c14a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient: _finish_hunting 0 2024-09-18T14:01:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 monclient: found mon.f 2024-09-18T14:01:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:01:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c11d8e0 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient: _finish_auth 0 2024-09-18T14:01:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b22a4f640 5 monclient: authenticate success, global_id 4528 2024-09-18T14:01:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b22a4f640 10 monclient: _renew_subs 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b22a4f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b22a4f640 1 -- 172.21.15.45:0/3381323075 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c11e840 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b22a4f640 10 monclient: _renew_subs 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b22a4f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b22a4f640 1 -- 172.21.15.45:0/3381323075 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c136ea0 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5b10004860 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5b10004d90 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b1a7fc640 4 set_mon_vals no callback set 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.196+0000 7f5b03fff640 10 monclient: dump: 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:29.211 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:29.211 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:29.211 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:29.211 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:29.211 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:29.211 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:29.211 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:29.211 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:29.211 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:29.211 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:29.212 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:29.212 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:29.212 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:29.212 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:29.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.197+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f5b10004ef0 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.197+0000 7f5b22a4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:29.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.197+0000 7f5b22a4f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:29.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.197+0000 7f5b22a4f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:01:29.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.197+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(121..121 src has 1..121) v4 ==== 3927+0+0 (unknown 3424470482 0 0) 0x7f5b100469f0 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.197+0000 7f5b22a4f640 1 -- 172.21.15.45:0/3381323075 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5afc005010 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.197+0000 7f5b1bfff640 10 monclient: get_auth_request con 0x7f5b0803d320 auth_method 0 2024-09-18T14:01:29.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.202+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.1 v1:172.21.15.84:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f5b10015160 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.202+0000 7f5b03fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:29.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.202+0000 7f5b03fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:29.249 INFO:tasks.rados.rados.0.smithi084.stdout:546: done (1 left) 2024-09-18T14:01:29.249 INFO:tasks.rados.rados.0.smithi084.stdout:547: write oid 26 current snap is 52 2024-09-18T14:01:29.249 INFO:tasks.rados.rados.0.smithi084.stdout:547: seq_num 223 ranges {547961=554764,1616308=703740,2741092=698600} 2024-09-18T14:01:29.256 INFO:tasks.rados.rados.0.smithi084.stdout:547: writing smithi08434663-26 from 547961 to 1102725 tid 1 2024-09-18T14:01:29.264 INFO:tasks.rados.rados.0.smithi084.stdout:547: writing smithi08434663-26 from 1616308 to 2320048 tid 2 2024-09-18T14:01:29.269 INFO:tasks.rados.rados.0.smithi084.stdout:547: writing smithi08434663-26 from 2741092 to 3439692 tid 3 2024-09-18T14:01:29.269 INFO:tasks.rados.rados.0.smithi084.stdout:548: rollback oid 40 current snap is 52 2024-09-18T14:01:29.270 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 40 to 51 2024-09-18T14:01:29.270 INFO:tasks.rados.rados.0.smithi084.stdout:545: finishing rollback tid 0 to smithi08434663-15 2024-09-18T14:01:29.270 INFO:tasks.rados.rados.0.smithi084.stdout:545: finishing rollback tid 1 to smithi08434663-15 2024-09-18T14:01:29.270 INFO:tasks.rados.rados.0.smithi084.stdout:545: finishing rollback tid 2 to smithi08434663-15 2024-09-18T14:01:29.270 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 196 (ObjNum 147 snap 30 seq_num 147) dirty exists 2024-09-18T14:01:29.270 INFO:tasks.rados.rados.0.smithi084.stdout:547: finishing write tid 1 to smithi08434663-26 2024-09-18T14:01:29.270 INFO:tasks.rados.rados.0.smithi084.stdout:547: finishing write tid 2 to smithi08434663-26 2024-09-18T14:01:29.270 INFO:tasks.rados.rados.0.smithi084.stdout:545: done (2 left) 2024-09-18T14:01:29.270 INFO:tasks.rados.rados.0.smithi084.stdout:549: delete oid 45 current snap is 52 2024-09-18T14:01:29.273 INFO:tasks.rados.rados.0.smithi084.stdout:548: finishing rollback tid 0 to smithi08434663-40 2024-09-18T14:01:29.273 INFO:tasks.rados.rados.0.smithi084.stdout:549: done (2 left) 2024-09-18T14:01:29.273 INFO:tasks.rados.rados.0.smithi084.stdout:550: write oid 43 current snap is 52 2024-09-18T14:01:29.273 INFO:tasks.rados.rados.0.smithi084.stdout:550: seq_num 225 ranges {581575=705262,1733272=707409,3195938=437497} 2024-09-18T14:01:29.280 INFO:tasks.rados.rados.0.smithi084.stdout:550: writing smithi08434663-43 from 581575 to 1286837 tid 1 2024-09-18T14:01:29.286 INFO:tasks.rados.rados.0.smithi084.stdout:550: writing smithi08434663-43 from 1733272 to 2440681 tid 2 2024-09-18T14:01:29.293 INFO:tasks.rados.rados.0.smithi084.stdout:550: writing smithi08434663-43 from 3195938 to 3633435 tid 3 2024-09-18T14:01:29.293 INFO:tasks.rados.rados.0.smithi084.stdout:551: read oid 5 snap -1 2024-09-18T14:01:29.293 INFO:tasks.rados.rados.0.smithi084.stdout:551: expect (ObjNum 200 snap 46 seq_num 200) 2024-09-18T14:01:29.293 INFO:tasks.rados.rados.0.smithi084.stdout:552: snap_create 2024-09-18T14:01:29.293 INFO:tasks.rados.rados.0.smithi084.stdout:548: finishing rollback tid 1 to smithi08434663-40 2024-09-18T14:01:29.293 INFO:tasks.rados.rados.0.smithi084.stdout:548: finishing rollback tid 2 to smithi08434663-40 2024-09-18T14:01:29.293 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 155 (ObjNum 207 snap 46 seq_num 207) dirty exists 2024-09-18T14:01:29.293 INFO:tasks.rados.rados.0.smithi084.stdout:547: finishing write tid 3 to smithi08434663-26 2024-09-18T14:01:29.294 INFO:tasks.rados.rados.0.smithi084.stdout:547: finishing write tid 4 to smithi08434663-26 2024-09-18T14:01:29.294 INFO:tasks.rados.rados.0.smithi084.stdout:547: finishing write tid 5 to smithi08434663-26 2024-09-18T14:01:29.294 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 162 (ObjNum 223 snap 52 seq_num 223) dirty exists 2024-09-18T14:01:29.294 INFO:tasks.rados.rados.0.smithi084.stdout:547: left oid 26 (ObjNum 223 snap 52 seq_num 223) 2024-09-18T14:01:29.294 INFO:tasks.rados.rados.0.smithi084.stdout:550: finishing write tid 1 to smithi08434663-43 2024-09-18T14:01:29.294 INFO:tasks.rados.rados.0.smithi084.stdout:550: finishing write tid 2 to smithi08434663-43 2024-09-18T14:01:29.296 INFO:tasks.rados.rados.0.smithi084.stdout:550: finishing write tid 3 to smithi08434663-43 2024-09-18T14:01:29.296 INFO:tasks.rados.rados.0.smithi084.stdout:550: finishing write tid 4 to smithi08434663-43 2024-09-18T14:01:29.296 INFO:tasks.rados.rados.0.smithi084.stdout:550: finishing write tid 5 to smithi08434663-43 2024-09-18T14:01:29.296 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 82 (ObjNum 225 snap 52 seq_num 225) dirty exists 2024-09-18T14:01:29.296 INFO:tasks.rados.rados.0.smithi084.stdout:550: left oid 43 (ObjNum 225 snap 52 seq_num 225) 2024-09-18T14:01:29.298 INFO:tasks.rados.rados.0.smithi084.stdout:551: expect (ObjNum 200 snap 46 seq_num 200) 2024-09-18T14:01:29.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.448+0000 7f5b22a4f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:01:29.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.448+0000 7f5b22a4f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:01:29.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.448+0000 7f5b22a4f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:01:29.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.448+0000 7f5b22a4f640 1 -- 172.21.15.45:0/3381323075 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5afc005840 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.448+0000 7f5b03fff640 1 -- 172.21.15.45:0/3381323075 <== mon.1 v1:172.21.15.84:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2676 (unknown 254746657 0 2171097172) 0x7f5b10014b60 con 0x7f5b1c09b1c0 2024-09-18T14:01:29.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.448+0000 7f5b03fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:01:29.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.448+0000 7f5b03fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:29.449 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":40,"quorum":[0,1,2,3,4,5,8],"quorum_names":["a","f","b","g","c","h","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:01:29.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.450+0000 7f5b22a4f640 1 -- 172.21.15.45:0/3381323075 >> v1:172.21.15.84:6812/31478 conn(0x7f5b0803d320 legacy=0x7f5b0803f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:29.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.450+0000 7f5b22a4f640 10 monclient: shutdown 2024-09-18T14:01:29.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.450+0000 7f5b22a4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:29.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.450+0000 7f5b22a4f640 1 -- 172.21.15.45:0/3381323075 >> v1:172.21.15.84:6789/0 conn(0x7f5b1c09b1c0 legacy=0x7f5b1c146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:29.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.450+0000 7f5b22a4f640 1 -- 172.21.15.45:0/3381323075 shutdown_connections 2024-09-18T14:01:29.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.450+0000 7f5b22a4f640 1 -- 172.21.15.45:0/3381323075 >> 172.21.15.45:0/3381323075 conn(0x7f5b1c0630b0 msgr2=0x7f5b1c1357a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:29.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.451+0000 7f5b22a4f640 1 -- 172.21.15.45:0/3381323075 shutdown_connections 2024-09-18T14:01:29.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:29.451+0000 7f5b22a4f640 1 -- 172.21.15.45:0/3381323075 wait complete. 2024-09-18T14:01:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:547: done (4 left) 2024-09-18T14:01:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:548: done (3 left) 2024-09-18T14:01:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:550: done (2 left) 2024-09-18T14:01:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:551: done (1 left) 2024-09-18T14:01:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:552: done (0 left) 2024-09-18T14:01:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:553: read oid 49 snap -1 2024-09-18T14:01:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:553: expect (ObjNum 111 snap 21 seq_num 111) 2024-09-18T14:01:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:554: rollback oid 2 current snap is 53 2024-09-18T14:01:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 2 to 51 2024-09-18T14:01:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:555: rollback oid 44 current snap is 53 2024-09-18T14:01:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 44 to 50 2024-09-18T14:01:31.214 INFO:tasks.rados.rados.0.smithi084.stdout:556: copy_from oid 5 from oid 21 current snap is 53 2024-09-18T14:01:31.214 INFO:tasks.rados.rados.0.smithi084.stdout:557: write oid 3 current snap is 53 2024-09-18T14:01:31.214 INFO:tasks.rados.rados.0.smithi084.stdout:557: seq_num 226 ranges {598382=780511,2075401=451067,3107008=648769} 2024-09-18T14:01:31.220 INFO:tasks.rados.rados.0.smithi084.stdout:557: writing smithi08434663-3 from 598382 to 1378893 tid 1 2024-09-18T14:01:31.227 INFO:tasks.rados.rados.0.smithi084.stdout:557: writing smithi08434663-3 from 2075401 to 2526468 tid 2 2024-09-18T14:01:31.234 INFO:tasks.rados.rados.0.smithi084.stdout:557: writing smithi08434663-3 from 3107008 to 3755777 tid 3 2024-09-18T14:01:31.235 INFO:tasks.rados.rados.0.smithi084.stdout:558: read oid 24 snap -1 2024-09-18T14:01:31.235 INFO:tasks.rados.rados.0.smithi084.stdout:558: expect (ObjNum 205 snap 46 seq_num 205) 2024-09-18T14:01:31.235 INFO:tasks.rados.rados.0.smithi084.stdout:559: write (excl) oid 42 current snap is 53 2024-09-18T14:01:31.235 INFO:tasks.rados.rados.0.smithi084.stdout:554: finishing rollback tid 1 to smithi08434663-2 2024-09-18T14:01:31.235 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 0 (ObjNum 129 snap 0 seq_num 2495046432) dirty dne 2024-09-18T14:01:31.235 INFO:tasks.rados.rados.0.smithi084.stdout:555: finishing rollback tid 0 to smithi08434663-44 2024-09-18T14:01:31.235 INFO:tasks.rados.rados.0.smithi084.stdout:555: finishing rollback tid 1 to smithi08434663-44 2024-09-18T14:01:31.235 INFO:tasks.rados.rados.0.smithi084.stdout:555: finishing rollback tid 2 to smithi08434663-44 2024-09-18T14:01:31.235 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 127 (ObjNum 88 snap 9 seq_num 88) dirty exists 2024-09-18T14:01:31.235 INFO:tasks.rados.rados.0.smithi084.stdout:553: expect (ObjNum 111 snap 21 seq_num 111) 2024-09-18T14:01:31.357 INFO:tasks.rados.rados.0.smithi084.stdout:556: finishing copy_from racing read to smithi08434663-5 2024-09-18T14:01:31.357 INFO:tasks.rados.rados.0.smithi084.stdout:556: finishing copy_from to smithi08434663-5 2024-09-18T14:01:31.357 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 36 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T14:01:31.357 INFO:tasks.rados.rados.0.smithi084.stdout:557: finishing write tid 1 to smithi08434663-3 2024-09-18T14:01:31.357 INFO:tasks.rados.rados.0.smithi084.stdout:557: finishing write tid 2 to smithi08434663-3 2024-09-18T14:01:31.357 INFO:tasks.rados.rados.0.smithi084.stdout:559: seq_num 227 ranges {615189=455760,1733883=711078,3134431=460041} 2024-09-18T14:01:31.362 INFO:tasks.rados.rados.0.smithi084.stdout:559: writing smithi08434663-42 from 615189 to 1070949 tid 1 2024-09-18T14:01:31.369 INFO:tasks.rados.rados.0.smithi084.stdout:559: writing smithi08434663-42 from 1733883 to 2444961 tid 2 2024-09-18T14:01:31.376 INFO:tasks.rados.rados.0.smithi084.stdout:559: writing smithi08434663-42 from 3134431 to 3594472 tid 3 2024-09-18T14:01:31.376 INFO:tasks.rados.rados.0.smithi084.stdout:553: done (6 left) 2024-09-18T14:01:31.376 INFO:tasks.rados.rados.0.smithi084.stdout:554: done (5 left) 2024-09-18T14:01:31.376 INFO:tasks.rados.rados.0.smithi084.stdout:555: done (4 left) 2024-09-18T14:01:31.376 INFO:tasks.rados.rados.0.smithi084.stdout:556: done (3 left) 2024-09-18T14:01:31.376 INFO:tasks.rados.rados.0.smithi084.stdout:560: write (excl) oid 17 current snap is 53 2024-09-18T14:01:31.376 INFO:tasks.rados.rados.0.smithi084.stdout:560: seq_num 228 ranges {631996=531009,1792365=454736,2761854=671313} 2024-09-18T14:01:31.382 INFO:tasks.rados.rados.0.smithi084.stdout:560: writing smithi08434663-17 from 631996 to 1163005 tid 1 2024-09-18T14:01:31.387 INFO:tasks.rados.rados.0.smithi084.stdout:560: writing smithi08434663-17 from 1792365 to 2247101 tid 2 2024-09-18T14:01:31.394 INFO:tasks.rados.rados.0.smithi084.stdout:560: writing smithi08434663-17 from 2761854 to 3433167 tid 3 2024-09-18T14:01:31.394 INFO:tasks.rados.rados.0.smithi084.stdout:561: read oid 36 snap -1 2024-09-18T14:01:31.395 INFO:tasks.rados.rados.0.smithi084.stdout:561: expect (ObjNum 211 snap 47 seq_num 211) 2024-09-18T14:01:31.395 INFO:tasks.rados.rados.0.smithi084.stdout:562: rollback oid 6 current snap is 53 2024-09-18T14:01:31.395 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 50 2024-09-18T14:01:31.395 INFO:tasks.rados.rados.0.smithi084.stdout:557: finishing write tid 3 to smithi08434663-3 2024-09-18T14:01:31.395 INFO:tasks.rados.rados.0.smithi084.stdout:557: finishing write tid 4 to smithi08434663-3 2024-09-18T14:01:31.395 INFO:tasks.rados.rados.0.smithi084.stdout:557: finishing write tid 5 to smithi08434663-3 2024-09-18T14:01:31.395 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 104 (ObjNum 226 snap 53 seq_num 226) dirty exists 2024-09-18T14:01:31.395 INFO:tasks.rados.rados.0.smithi084.stdout:557: left oid 3 (ObjNum 226 snap 53 seq_num 226) 2024-09-18T14:01:31.395 INFO:tasks.rados.rados.0.smithi084.stdout:558: expect (ObjNum 205 snap 46 seq_num 205) 2024-09-18T14:01:31.618 INFO:tasks.rados.rados.0.smithi084.stdout:559: finishing write tid 1 to smithi08434663-42 2024-09-18T14:01:31.618 INFO:tasks.rados.rados.0.smithi084.stdout:559: finishing write tid 2 to smithi08434663-42 2024-09-18T14:01:31.618 INFO:tasks.rados.rados.0.smithi084.stdout:559: finishing write tid 3 to smithi08434663-42 2024-09-18T14:01:31.618 INFO:tasks.rados.rados.0.smithi084.stdout:559: finishing write tid 4 to smithi08434663-42 2024-09-18T14:01:31.618 INFO:tasks.rados.rados.0.smithi084.stdout:559: finishing write tid 5 to smithi08434663-42 2024-09-18T14:01:31.618 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 153 (ObjNum 227 snap 53 seq_num 227) dirty exists 2024-09-18T14:01:31.618 INFO:tasks.rados.rados.0.smithi084.stdout:559: left oid 42 (ObjNum 227 snap 53 seq_num 227) 2024-09-18T14:01:31.618 INFO:tasks.rados.rados.0.smithi084.stdout:560: finishing write tid 1 to smithi08434663-17 2024-09-18T14:01:31.618 INFO:tasks.rados.rados.0.smithi084.stdout:560: finishing write tid 2 to smithi08434663-17 2024-09-18T14:01:31.619 INFO:tasks.rados.rados.0.smithi084.stdout:557: done (5 left) 2024-09-18T14:01:31.619 INFO:tasks.rados.rados.0.smithi084.stdout:558: done (4 left) 2024-09-18T14:01:31.619 INFO:tasks.rados.rados.0.smithi084.stdout:559: done (3 left) 2024-09-18T14:01:31.619 INFO:tasks.rados.rados.0.smithi084.stdout:563: snap_create 2024-09-18T14:01:31.619 INFO:tasks.rados.rados.0.smithi084.stdout:562: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:01:31.619 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 0 (ObjNum 129 snap 0 seq_num 2495510368) dirty dne 2024-09-18T14:01:31.619 INFO:tasks.rados.rados.0.smithi084.stdout:560: finishing write tid 3 to smithi08434663-17 2024-09-18T14:01:31.619 INFO:tasks.rados.rados.0.smithi084.stdout:560: finishing write tid 4 to smithi08434663-17 2024-09-18T14:01:31.619 INFO:tasks.rados.rados.0.smithi084.stdout:560: finishing write tid 5 to smithi08434663-17 2024-09-18T14:01:31.619 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 160 (ObjNum 228 snap 53 seq_num 228) dirty exists 2024-09-18T14:01:31.619 INFO:tasks.rados.rados.0.smithi084.stdout:560: left oid 17 (ObjNum 228 snap 53 seq_num 228) 2024-09-18T14:01:31.619 INFO:tasks.rados.rados.0.smithi084.stdout:561: expect (ObjNum 211 snap 47 seq_num 211) 2024-09-18T14:01:32.159 INFO:tasks.rados.rados.0.smithi084.stdout:560: done (3 left) 2024-09-18T14:01:32.160 INFO:tasks.rados.rados.0.smithi084.stdout:561: done (2 left) 2024-09-18T14:01:32.160 INFO:tasks.rados.rados.0.smithi084.stdout:562: done (1 left) 2024-09-18T14:01:32.160 INFO:tasks.rados.rados.0.smithi084.stdout:563: done (0 left) 2024-09-18T14:01:32.160 INFO:tasks.rados.rados.0.smithi084.stdout:564: write oid 24 current snap is 54 2024-09-18T14:01:32.160 INFO:tasks.rados.rados.0.smithi084.stdout:564: seq_num 229 ranges {648803=722611,1967200=714747,3421983=426820} 2024-09-18T14:01:32.167 INFO:tasks.rados.rados.0.smithi084.stdout:564: writing smithi08434663-24 from 648803 to 1371414 tid 1 2024-09-18T14:01:32.174 INFO:tasks.rados.rados.0.smithi084.stdout:564: writing smithi08434663-24 from 1967200 to 2681947 tid 2 2024-09-18T14:01:32.181 INFO:tasks.rados.rados.0.smithi084.stdout:564: writing smithi08434663-24 from 3421983 to 3848803 tid 3 2024-09-18T14:01:32.181 INFO:tasks.rados.rados.0.smithi084.stdout:565: write oid 12 current snap is 54 2024-09-18T14:01:32.181 INFO:tasks.rados.rados.0.smithi084.stdout:565: seq_num 230 ranges {665610=797860,1909329=458405,2816700=410210} 2024-09-18T14:01:32.189 INFO:tasks.rados.rados.0.smithi084.stdout:565: writing smithi08434663-12 from 665610 to 1463470 tid 1 2024-09-18T14:01:32.194 INFO:tasks.rados.rados.0.smithi084.stdout:565: writing smithi08434663-12 from 1909329 to 2367734 tid 2 2024-09-18T14:01:32.199 INFO:tasks.rados.rados.0.smithi084.stdout:565: writing smithi08434663-12 from 2816700 to 3226910 tid 3 2024-09-18T14:01:32.199 INFO:tasks.rados.rados.0.smithi084.stdout:566: rollback oid 6 current snap is 54 2024-09-18T14:01:32.199 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 51 2024-09-18T14:01:32.200 INFO:tasks.rados.rados.0.smithi084.stdout:564: finishing write tid 1 to smithi08434663-24 2024-09-18T14:01:32.200 INFO:tasks.rados.rados.0.smithi084.stdout:564: finishing write tid 2 to smithi08434663-24 2024-09-18T14:01:32.200 INFO:tasks.rados.rados.0.smithi084.stdout:564: finishing write tid 3 to smithi08434663-24 2024-09-18T14:01:32.200 INFO:tasks.rados.rados.0.smithi084.stdout:564: finishing write tid 4 to smithi08434663-24 2024-09-18T14:01:32.200 INFO:tasks.rados.rados.0.smithi084.stdout:564: finishing write tid 5 to smithi08434663-24 2024-09-18T14:01:32.200 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 171 (ObjNum 229 snap 54 seq_num 229) dirty exists 2024-09-18T14:01:32.200 INFO:tasks.rados.rados.0.smithi084.stdout:564: left oid 24 (ObjNum 229 snap 54 seq_num 229) 2024-09-18T14:01:32.200 INFO:tasks.rados.rados.0.smithi084.stdout:565: finishing write tid 1 to smithi08434663-12 2024-09-18T14:01:32.200 INFO:tasks.rados.rados.0.smithi084.stdout:565: finishing write tid 2 to smithi08434663-12 2024-09-18T14:01:32.200 INFO:tasks.rados.rados.0.smithi084.stdout:564: done (2 left) 2024-09-18T14:01:32.200 INFO:tasks.rados.rados.0.smithi084.stdout:567: rollback oid 26 current snap is 54 2024-09-18T14:01:32.200 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 26 to 50 2024-09-18T14:01:32.201 INFO:tasks.rados.rados.0.smithi084.stdout:568: rollback oid 13 current snap is 54 2024-09-18T14:01:32.201 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 13 to 51 2024-09-18T14:01:32.201 INFO:tasks.rados.rados.0.smithi084.stdout:569: read oid 20 snap -1 2024-09-18T14:01:32.201 INFO:tasks.rados.rados.0.smithi084.stdout:569: expect deleted 2024-09-18T14:01:32.201 INFO:tasks.rados.rados.0.smithi084.stdout:570: write (excl) oid 10 current snap is 54 2024-09-18T14:01:32.201 INFO:tasks.rados.rados.0.smithi084.stdout:570: seq_num 231 ranges {682417=473109,1567811=602063,2844123=621482} 2024-09-18T14:01:32.205 INFO:tasks.rados.rados.0.smithi084.stdout:570: writing smithi08434663-10 from 682417 to 1155526 tid 1 2024-09-18T14:01:32.211 INFO:tasks.rados.rados.0.smithi084.stdout:570: writing smithi08434663-10 from 1567811 to 2169874 tid 2 2024-09-18T14:01:32.218 INFO:tasks.rados.rados.0.smithi084.stdout:570: writing smithi08434663-10 from 2844123 to 3465605 tid 3 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:571: delete oid 22 current snap is 54 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:566: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 0 (ObjNum 65 snap 0 seq_num 1568230242) dirty dne 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:565: finishing write tid 3 to smithi08434663-12 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:567: finishing rollback tid 0 to smithi08434663-26 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:565: finishing write tid 4 to smithi08434663-12 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:565: finishing write tid 5 to smithi08434663-12 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 244 (ObjNum 230 snap 54 seq_num 230) dirty exists 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:565: left oid 12 (ObjNum 230 snap 54 seq_num 230) 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:567: finishing rollback tid 1 to smithi08434663-26 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:567: finishing rollback tid 2 to smithi08434663-26 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 166 (ObjNum 116 snap 22 seq_num 116) dirty exists 2024-09-18T14:01:32.219 INFO:tasks.rados.rados.0.smithi084.stdout:568: finishing rollback tid 0 to smithi08434663-13 2024-09-18T14:01:32.220 INFO:tasks.rados.rados.0.smithi084.stdout:568: finishing rollback tid 1 to smithi08434663-13 2024-09-18T14:01:32.220 INFO:tasks.rados.rados.0.smithi084.stdout:568: finishing rollback tid 2 to smithi08434663-13 2024-09-18T14:01:32.220 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 164 (ObjNum 217 snap 50 seq_num 217) dirty exists 2024-09-18T14:01:32.220 INFO:tasks.rados.rados.0.smithi084.stdout:570: finishing write tid 1 to smithi08434663-10 2024-09-18T14:01:32.220 INFO:tasks.rados.rados.0.smithi084.stdout:570: finishing write tid 2 to smithi08434663-10 2024-09-18T14:01:32.224 INFO:tasks.rados.rados.0.smithi084.stdout:570: finishing write tid 3 to smithi08434663-10 2024-09-18T14:01:32.224 INFO:tasks.rados.rados.0.smithi084.stdout:570: finishing write tid 4 to smithi08434663-10 2024-09-18T14:01:32.224 INFO:tasks.rados.rados.0.smithi084.stdout:570: finishing write tid 5 to smithi08434663-10 2024-09-18T14:01:32.224 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 171 (ObjNum 231 snap 54 seq_num 231) dirty exists 2024-09-18T14:01:32.224 INFO:tasks.rados.rados.0.smithi084.stdout:570: left oid 10 (ObjNum 231 snap 54 seq_num 231) 2024-09-18T14:01:32.225 INFO:tasks.rados.rados.0.smithi084.stdout:565: done (6 left) 2024-09-18T14:01:32.225 INFO:tasks.rados.rados.0.smithi084.stdout:566: done (5 left) 2024-09-18T14:01:32.225 INFO:tasks.rados.rados.0.smithi084.stdout:567: done (4 left) 2024-09-18T14:01:32.225 INFO:tasks.rados.rados.0.smithi084.stdout:568: done (3 left) 2024-09-18T14:01:32.225 INFO:tasks.rados.rados.0.smithi084.stdout:569: done (2 left) 2024-09-18T14:01:32.225 INFO:tasks.rados.rados.0.smithi084.stdout:570: done (1 left) 2024-09-18T14:01:32.226 INFO:tasks.rados.rados.0.smithi084.stdout:571: done (0 left) 2024-09-18T14:01:32.226 INFO:tasks.rados.rados.0.smithi084.stdout:572: read oid 33 snap 53 2024-09-18T14:01:32.226 INFO:tasks.rados.rados.0.smithi084.stdout:572: expect (ObjNum 216 snap 50 seq_num 216) 2024-09-18T14:01:32.226 INFO:tasks.rados.rados.0.smithi084.stdout:573: snap_create 2024-09-18T14:01:32.236 INFO:tasks.rados.rados.0.smithi084.stdout:572: expect (ObjNum 216 snap 50 seq_num 216) 2024-09-18T14:01:32.462 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:01:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.564+0000 7f36f071d640 10 monclient: get_monmap_and_config 2024-09-18T14:01:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.564+0000 7f36f071d640 10 monclient: build_initial_monmap 2024-09-18T14:01:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.564+0000 7f36f071d640 10 monclient: monmap: 2024-09-18T14:01:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:32.565742+0000 2024-09-18T14:01:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:32.565742+0000 2024-09-18T14:01:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:32.566 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.565+0000 7f36f071d640 1 Processor -- start 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.565+0000 7f36f071d640 1 -- start start 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.565+0000 7f36f071d640 10 monclient: init 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36f071d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36f071d640 10 monclient: _add_conns ranks=[1,7,0,8,2,4,5,6,3] 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36f071d640 10 monclient(hunting): picked mon.noname-f con 0x7f36e809f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36f071d640 10 monclient(hunting): picked mon.noname-i con 0x7f36e809fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36f071d640 10 monclient(hunting): picked mon.noname-a con 0x7f36e809ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36f071d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e805ed70 con 0x7f36e809ff70 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36f071d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e809dfc0 con 0x7f36e809f7d0 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36f071d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e809dce0 con 0x7f36e809fba0 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36f071d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36f071d640 10 monclient(hunting): authenticate will time out at 907.904276s 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36eecba640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f36e809ff70 0x7f36e813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51368/0 (socket says 172.21.15.45:51368) 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36eecba640 1 -- 172.21.15.45:0/2157420149 learned_addr learned my addr 172.21.15.45:0/2157420149 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461825268 0 0) 0x7f36e805ed70 con 0x7f36e809ff70 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 10 monclient(hunting): my global_id is 4608 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d40034e0 con 0x7f36e809ff70 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471319274 0 0) 0x7f36e809dfc0 con 0x7f36e809f7d0 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 10 monclient(hunting): my global_id is 4537 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e805ed70 con 0x7f36e809f7d0 2024-09-18T14:01:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1357846166 0 0) 0x7f36e809dce0 con 0x7f36e809fba0 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 10 monclient(hunting): my global_id is 4606 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e809dfc0 con 0x7f36e809fba0 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 354122194 0 0) 0x7f36e805ed70 con 0x7f36e809f7d0 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 >> v1:172.21.15.84:6792/0 conn(0x7f36e809fba0 legacy=0x7f36e8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 >> v1:172.21.15.45:6789/0 conn(0x7f36e809ff70 legacy=0x7f36e813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 10 monclient: _finish_hunting 0 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 1 monclient: found mon.noname-f 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.566+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e80a1c00 con 0x7f36e809f7d0 2024-09-18T14:01:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36ed4b7640 10 monclient: _finish_auth 0 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36ed4b7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f36dc001810 con 0x7f36e809f7d0 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36f071d640 5 monclient: authenticate success, global_id 4537 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36ed4b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36ed4b7640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36ed4b7640 10 monclient: dump: 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36f071d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f36dc001970 con 0x7f36e809f7d0 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36ed4b7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36f071d640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36ed4b7640 1 -- 172.21.15.45:0/2157420149 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f36dc001f20 con 0x7f36e809f7d0 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36f071d640 4 set_mon_vals no callback set 2024-09-18T14:01:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36ed4b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36ed4b7640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36ed4b7640 10 monclient: dump: 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36f071d640 10 monclient: shutdown 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36f071d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36f071d640 1 -- 172.21.15.45:0/2157420149 >> v1:172.21.15.84:6789/0 conn(0x7f36e809f7d0 legacy=0x7f36e8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36f071d640 1 -- 172.21.15.45:0/2157420149 shutdown_connections 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36f071d640 1 -- 172.21.15.45:0/2157420149 >> 172.21.15.45:0/2157420149 conn(0x7f36e80630b0 msgr2=0x7f36e80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36f071d640 1 -- 172.21.15.45:0/2157420149 shutdown_connections 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.567+0000 7f36f071d640 1 -- 172.21.15.45:0/2157420149 wait complete. 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.568+0000 7f36f071d640 10 monclient: build_initial_monmap 2024-09-18T14:01:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.568+0000 7f36f071d640 10 monclient: monmap: 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:32.581 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.568+0000 7f36f071d640 1 Processor -- start 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.568+0000 7f36f071d640 1 -- start start 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.568+0000 7f36f071d640 10 monclient: init 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36f071d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36f071d640 10 monclient: _add_conns ranks=[2,4,0,5,8,6,1,7,3] 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36f071d640 10 monclient(hunting): picked mon.noname-b con 0x7f36e809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36f071d640 10 monclient(hunting): picked mon.noname-c con 0x7f36e809fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36f071d640 10 monclient(hunting): picked mon.noname-a con 0x7f36e809ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36f071d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e809d310 con 0x7f36e809ff70 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36f071d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e80a2560 con 0x7f36e809f7d0 2024-09-18T14:01:32.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36f071d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e8121ba0 con 0x7f36e809fba0 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36f071d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36f071d640 10 monclient(hunting): authenticate will time out at 907.907224s 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36ee4b9640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f36e809f7d0 0x7f36e81419d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43652/0 (socket says 172.21.15.45:43652) 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36ee4b9640 1 -- 172.21.15.45:0/2708705248 learned_addr learned my addr 172.21.15.45:0/2708705248 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387321152 0 0) 0x7f36e8121ba0 con 0x7f36e809fba0 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): my global_id is 134113 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36cc0034e0 con 0x7f36e809fba0 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602328347 0 0) 0x7f36e809d310 con 0x7f36e809ff70 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): my global_id is 4617 2024-09-18T14:01:32.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e8121ba0 con 0x7f36e809ff70 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105231276 0 0) 0x7f36e80a2560 con 0x7f36e809f7d0 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): my global_id is 4601 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e809d310 con 0x7f36e809f7d0 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1328847794 0 0) 0x7f36e809d310 con 0x7f36e809f7d0 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f36e80a2560 con 0x7f36e809f7d0 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f36dc001e20 con 0x7f36e809f7d0 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): dump: 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.584 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3514962746 0 0) 0x7f36cc0034e0 con 0x7f36e809fba0 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f36e809d310 con 0x7f36e809fba0 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 262051054 0 0) 0x7f36e8121ba0 con 0x7f36e809ff70 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f36cc0034e0 con 0x7f36e809ff70 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f36e40015e0 con 0x7f36e809fba0 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): dump: 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.586 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f36e0001d60 con 0x7f36e809ff70 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.569+0000 7f36da7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:32.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 10 monclient(hunting): dump: 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.588 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 34381078 0 0) 0x7f36e80a2560 con 0x7f36e809f7d0 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 >> v1:172.21.15.45:6791/0 conn(0x7f36e809fba0 legacy=0x7f36e81319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 >> v1:172.21.15.45:6789/0 conn(0x7f36e809ff70 legacy=0x7f36e8149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 1 monclient: found mon.b 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e81220d0 con 0x7f36e809f7d0 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 10 monclient: _finish_auth 0 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36f071d640 5 monclient: authenticate success, global_id 4601 2024-09-18T14:01:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36f071d640 10 monclient: _renew_subs 2024-09-18T14:01:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36f071d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36f071d640 1 -- 172.21.15.45:0/2708705248 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e8120050 con 0x7f36e809f7d0 2024-09-18T14:01:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36f071d640 10 monclient: _renew_subs 2024-09-18T14:01:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36f071d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36f071d640 1 -- 172.21.15.45:0/2708705248 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e81420e0 con 0x7f36e809f7d0 2024-09-18T14:01:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f36dc002550 con 0x7f36e809f7d0 2024-09-18T14:01:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f36dc002ac0 con 0x7f36e809f7d0 2024-09-18T14:01:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36eccb6640 4 set_mon_vals no callback set 2024-09-18T14:01:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.570+0000 7f36da7fc640 10 monclient: dump: 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.592 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.571+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f36dc002c20 con 0x7f36e809f7d0 2024-09-18T14:01:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.571+0000 7f36f071d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.571+0000 7f36f071d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.571+0000 7f36f071d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.571+0000 7f36f071d640 1 -- 172.21.15.45:0/2708705248 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d0005010 con 0x7f36e809f7d0 2024-09-18T14:01:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.571+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(124..124 src has 1..124) v4 ==== 3955+0+0 (unknown 1177902436 0 0) 0x7f36dc046b80 con 0x7f36e809f7d0 2024-09-18T14:01:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.571+0000 7f36edcb8640 10 monclient: get_auth_request con 0x7f36cc03a950 auth_method 0 2024-09-18T14:01:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.575+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f36dc0153d0 con 0x7f36e809f7d0 2024-09-18T14:01:32.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.575+0000 7f36da7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:32.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.575+0000 7f36da7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:32.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.803+0000 7f36f071d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:01:32.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.803+0000 7f36f071d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:01:32.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.803+0000 7f36f071d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:32.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.803+0000 7f36f071d640 1 -- 172.21.15.45:0/2708705248 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f36d0005840 con 0x7f36e809f7d0 2024-09-18T14:01:32.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.803+0000 7f36da7fc640 1 -- 172.21.15.45:0/2708705248 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 3047463235) 0x7f36dc014dd0 con 0x7f36e809f7d0 2024-09-18T14:01:32.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.803+0000 7f36da7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:01:32.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.803+0000 7f36da7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:32.804 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":50,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":2,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:01:32.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.805+0000 7f36f071d640 1 -- 172.21.15.45:0/2708705248 >> v1:172.21.15.84:6812/31478 conn(0x7f36cc03a950 legacy=0x7f36cc03ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.805+0000 7f36f071d640 10 monclient: shutdown 2024-09-18T14:01:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.805+0000 7f36f071d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.805+0000 7f36f071d640 1 -- 172.21.15.45:0/2708705248 >> v1:172.21.15.45:6790/0 conn(0x7f36e809f7d0 legacy=0x7f36e81419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.805+0000 7f36f071d640 1 -- 172.21.15.45:0/2708705248 shutdown_connections 2024-09-18T14:01:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.805+0000 7f36f071d640 1 -- 172.21.15.45:0/2708705248 >> 172.21.15.45:0/2708705248 conn(0x7f36e80630b0 msgr2=0x7f36e80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.805+0000 7f36f071d640 1 -- 172.21.15.45:0/2708705248 shutdown_connections 2024-09-18T14:01:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.805+0000 7f36f071d640 1 -- 172.21.15.45:0/2708705248 wait complete. 2024-09-18T14:01:32.815 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:01:32.815 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:01:32.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:01:32.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.912+0000 7fcc0159d640 10 monclient: get_monmap_and_config 2024-09-18T14:01:32.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.912+0000 7fcc0159d640 10 monclient: build_initial_monmap 2024-09-18T14:01:32.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.912+0000 7fcc0159d640 10 monclient: monmap: 2024-09-18T14:01:32.913 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:32.913 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:32.913571+0000 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:32.913571+0000 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:32.915 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 1 Processor -- start 2024-09-18T14:01:32.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 1 -- start start 2024-09-18T14:01:32.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 10 monclient: init 2024-09-18T14:01:32.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:32.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 10 monclient: _add_conns ranks=[6,4,5,0,3,7,8,2,1] 2024-09-18T14:01:32.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 10 monclient(hunting): picked mon.noname-d con 0x7fcbfc09f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:01:32.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 10 monclient(hunting): picked mon.noname-c con 0x7fcbfc09fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:32.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 10 monclient(hunting): picked mon.noname-h con 0x7fcbfc09ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:32.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbfc05ed70 con 0x7fcbfc09fba0 2024-09-18T14:01:32.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbfc09dfc0 con 0x7fcbfc09ff70 2024-09-18T14:01:32.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbfc09dce0 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcc0159d640 10 monclient(hunting): authenticate will time out at 908.251763s 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcbfa7fc640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fcbfc09fba0 0x7fcbfc137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36864/0 (socket says 172.21.15.45:36864) 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcbfaffd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fcbfc09f7d0 0x7fcbfc132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56038/0 (socket says 172.21.15.45:56038) 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.913+0000 7fcbfa7fc640 1 -- 172.21.15.45:0/2615936904 learned_addr learned my addr 172.21.15.45:0/2615936904 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672740435 0 0) 0x7fcbfc09dce0 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient(hunting): my global_id is 4524 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe80034e0 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426157385 0 0) 0x7fcbfc05ed70 con 0x7fcbfc09fba0 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient(hunting): my global_id is 134122 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbfc09dce0 con 0x7fcbfc09fba0 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958937340 0 0) 0x7fcbfc09dfc0 con 0x7fcbfc09ff70 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient(hunting): my global_id is 4505 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbfc05ed70 con 0x7fcbfc09ff70 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 910937868 0 0) 0x7fcbe80034e0 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 >> v1:172.21.15.84:6791/0 conn(0x7fcbfc09ff70 legacy=0x7fcbfc13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 >> v1:172.21.15.45:6791/0 conn(0x7fcbfc09fba0 legacy=0x7fcbfc137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 monclient: found mon.noname-d 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:01:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc0a1c00 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient: _finish_auth 0 2024-09-18T14:01:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcbec001830 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcc0159d640 5 monclient: authenticate success, global_id 4524 2024-09-18T14:01:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:01:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient: dump: 2024-09-18T14:01:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcc0159d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcbec001a10 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcc0159d640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 1 -- 172.21.15.45:0/2615936904 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcbec004ae0 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcc0159d640 4 set_mon_vals no callback set 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.914+0000 7fcbf9ffb640 10 monclient: dump: 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.929 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.915+0000 7fcc0159d640 10 monclient: shutdown 2024-09-18T14:01:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.915+0000 7fcc0159d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.915+0000 7fcc0159d640 1 -- 172.21.15.45:0/2615936904 >> v1:172.21.15.45:6792/0 conn(0x7fcbfc09f7d0 legacy=0x7fcbfc132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.915+0000 7fcc0159d640 1 -- 172.21.15.45:0/2615936904 shutdown_connections 2024-09-18T14:01:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.915+0000 7fcc0159d640 1 -- 172.21.15.45:0/2615936904 >> 172.21.15.45:0/2615936904 conn(0x7fcbfc0630b0 msgr2=0x7fcbfc0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.915+0000 7fcc0159d640 1 -- 172.21.15.45:0/2615936904 shutdown_connections 2024-09-18T14:01:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.915+0000 7fcc0159d640 1 -- 172.21.15.45:0/2615936904 wait complete. 2024-09-18T14:01:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.915+0000 7fcc0159d640 10 monclient: build_initial_monmap 2024-09-18T14:01:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.915+0000 7fcc0159d640 10 monclient: monmap: 2024-09-18T14:01:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:32.932 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 1 Processor -- start 2024-09-18T14:01:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 1 -- start start 2024-09-18T14:01:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 10 monclient: init 2024-09-18T14:01:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 10 monclient: _add_conns ranks=[1,5,7,0,6,4,3,2,8] 2024-09-18T14:01:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 10 monclient(hunting): picked mon.noname-f con 0x7fcbfc09f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 10 monclient(hunting): picked mon.noname-h con 0x7fcbfc09fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 10 monclient(hunting): picked mon.noname-i con 0x7fcbfc09ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbfc09d310 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbfc0a2560 con 0x7fcbfc09fba0 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbfc11d510 con 0x7fcbfc09ff70 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcc0159d640 10 monclient(hunting): authenticate will time out at 908.254724s 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcbfa7fc640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7fcbfc09fba0 0x7fcbfc146d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:47406/0 (socket says 172.21.15.45:47406) 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.916+0000 7fcbfa7fc640 1 -- 172.21.15.45:0/1835492659 learned_addr learned my addr 172.21.15.45:0/1835492659 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108610068 0 0) 0x7fcbfc0a2560 con 0x7fcbfc09fba0 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): my global_id is 4514 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbdc0034e0 con 0x7fcbfc09fba0 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737574691 0 0) 0x7fcbfc09d310 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): my global_id is 4546 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbfc0a2560 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657102682 0 0) 0x7fcbfc11d510 con 0x7fcbfc09ff70 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): my global_id is 4615 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbfc09d310 con 0x7fcbfc09ff70 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4177583164 0 0) 0x7fcbfc0a2560 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcbfc11d510 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcbec004b20 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:01:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): dump: 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3128931900 0 0) 0x7fcbdc0034e0 con 0x7fcbfc09fba0 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcbfc0a2560 con 0x7fcbfc09fba0 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcbe4001ba0 con 0x7fcbfc09fba0 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): dump: 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2703122908 0 0) 0x7fcbfc09d310 con 0x7fcbfc09ff70 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcbdc0034e0 con 0x7fcbfc09ff70 2024-09-18T14:01:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcbf4001d00 con 0x7fcbfc09ff70 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient(hunting): dump: 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1500626595 0 0) 0x7fcbfc11d510 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 >> v1:172.21.15.84:6792/0 conn(0x7fcbfc09ff70 legacy=0x7fcbfc14a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 >> v1:172.21.15.84:6791/0 conn(0x7fcbfc09fba0 legacy=0x7fcbfc146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 monclient: found mon.f 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc1428a0 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient: _finish_auth 0 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcc0159d640 5 monclient: authenticate success, global_id 4546 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcc0159d640 10 monclient: _renew_subs 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcc0159d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcc0159d640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbfc11d920 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcc0159d640 10 monclient: _renew_subs 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcc0159d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcc0159d640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbfc136ee0 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcbec001850 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcbec005120 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:01:32.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf97fa640 4 set_mon_vals no callback set 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.917+0000 7fcbf2ffd640 10 monclient: dump: 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.918+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fcbec0100f0 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.918+0000 7fcc0159d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.918+0000 7fcc0159d640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcbe0006580 con 0x7fcbe0002830 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.918+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(124..124 src has 1..124) v4 ==== 3955+0+0 (unknown 1177902436 0 0) 0x7fcbec046c00 con 0x7fcbfc09f7d0 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.918+0000 7fcbfb7fe640 10 monclient: get_auth_request con 0x7fcbdc03aa10 auth_method 0 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.918+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778934235 0 0) 0x7fcbe0006580 con 0x7fcbe0002830 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.918+0000 7fcbf2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.918+0000 7fcbf2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.918+0000 7fcbf2ffd640 10 monclient(hunting): my global_id is 4626 2024-09-18T14:01:32.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.918+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbfc11d510 con 0x7fcbe0002830 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 371549378 0 0) 0x7fcbfc11d510 con 0x7fcbe0002830 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcbe0006580 con 0x7fcbe0002830 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcbe4004be0 con 0x7fcbe0002830 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 10 monclient: dump: 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.943 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1235389886 0 0) 0x7fcbe0006580 con 0x7fcbe0002830 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcbe000a0a0 con 0x7fcbe0002830 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fcbe4005890 con 0x7fcbe0002830 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:32.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 >> v1:172.21.15.45:6789/0 conn(0x7fcbe0002830 legacy=0x7fcbe0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.919+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 >> v1:172.21.15.45:6789/0 conn(0x7fcbe0002830 legacy=0x7fcbe0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:32.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcc0159d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:32.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcc0159d640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcbe0009610 con 0x7fcbe0008ff0 2024-09-18T14:01:32.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906078873 0 0) 0x7fcbe0009610 con 0x7fcbe0008ff0 2024-09-18T14:01:32.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcbf2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:32.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcbf2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:32.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcbf2ffd640 10 monclient(hunting): my global_id is 4635 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe0006580 con 0x7fcbe0008ff0 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4222376720 0 0) 0x7fcbe0006580 con 0x7fcbe0008ff0 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcbe0009610 con 0x7fcbe0008ff0 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fcbe40056b0 con 0x7fcbe0008ff0 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcbf2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcbf2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.925+0000 7fcbf2ffd640 10 monclient: dump: 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:32.947 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4084393395 0 0) 0x7fcbe0009610 con 0x7fcbe0008ff0 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcbe00098d0 con 0x7fcbe0008ff0 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 57597172) 0x7fcbe4005d80 con 0x7fcbe0008ff0 2024-09-18T14:01:32.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcbf2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:32.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcbf2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:32.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 >> v1:172.21.15.45:6789/0 conn(0x7fcbe0008ff0 legacy=0x7fcbe0008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcbf2ffd640 1 -- 172.21.15.45:0/1835492659 >> v1:172.21.15.45:6789/0 conn(0x7fcbe0008ff0 legacy=0x7fcbe0008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:32.949 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 2, 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:01:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:32.968 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcc0159d640 1 -- 172.21.15.45:0/1835492659 >> v1:172.21.15.84:6812/31478 conn(0x7fcbdc03aa10 legacy=0x7fcbdc03cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcc0159d640 10 monclient: shutdown 2024-09-18T14:01:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcc0159d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcc0159d640 1 -- 172.21.15.45:0/1835492659 >> v1:172.21.15.84:6789/0 conn(0x7fcbfc09f7d0 legacy=0x7fcbfc1316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcbfb7fe640 1 -- 172.21.15.45:0/1835492659 reap_dead start 2024-09-18T14:01:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcbfb7fe640 1 -- 172.21.15.45:0/1835492659 reap_dead start 2024-09-18T14:01:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcc0159d640 1 -- 172.21.15.45:0/1835492659 shutdown_connections 2024-09-18T14:01:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcc0159d640 1 -- 172.21.15.45:0/1835492659 >> 172.21.15.45:0/1835492659 conn(0x7fcbfc0630b0 msgr2=0x7fcbfc0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcc0159d640 1 -- 172.21.15.45:0/1835492659 shutdown_connections 2024-09-18T14:01:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:32.926+0000 7fcc0159d640 1 -- 172.21.15.45:0/1835492659 wait complete. 2024-09-18T14:01:32.970 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:32.970 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:01:32.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:01:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.067+0000 7f2a5209f640 10 monclient: get_monmap_and_config 2024-09-18T14:01:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.067+0000 7f2a5209f640 10 monclient: build_initial_monmap 2024-09-18T14:01:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.067+0000 7f2a5209f640 10 monclient: monmap: 2024-09-18T14:01:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:33.068047+0000 2024-09-18T14:01:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:33.068047+0000 2024-09-18T14:01:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:33.069 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.067+0000 7f2a5209f640 1 Processor -- start 2024-09-18T14:01:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.067+0000 7f2a5209f640 1 -- start start 2024-09-18T14:01:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.067+0000 7f2a5209f640 10 monclient: init 2024-09-18T14:01:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a5209f640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a5209f640 10 monclient: _add_conns ranks=[8,6,4,5,2,1,0,3,7] 2024-09-18T14:01:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a5209f640 10 monclient(hunting): picked mon.noname-e con 0x7f2a4c06e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a5209f640 10 monclient(hunting): picked mon.noname-d con 0x7f2a4c0a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:01:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a5209f640 10 monclient(hunting): picked mon.noname-c con 0x7f2a4c0a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a5209f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c05ed70 con 0x7f2a4c0a1080 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a5209f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c097620 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a5209f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c097340 con 0x7f2a4c06e810 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a5209f640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a5209f640 10 monclient(hunting): authenticate will time out at 908.406548s 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4b7fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f2a4c06e810 0x7f2a4c0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:60936/0 (socket says 172.21.15.45:60936) 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4b7fe640 1 -- 172.21.15.45:0/2799980079 learned_addr learned my addr 172.21.15.45:0/2799980079 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4affd640 1 --1- 172.21.15.45:0/2799980079 >> v1:172.21.15.45:6792/0 conn(0x7f2a4c0a0cb0 0x7f2a4c137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56050/0 (socket says 172.21.15.45:56050) 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919742060 0 0) 0x7f2a4c097340 con 0x7f2a4c06e810 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 10 monclient(hunting): my global_id is 4544 2024-09-18T14:01:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a340034e0 con 0x7f2a4c06e810 2024-09-18T14:01:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666740659 0 0) 0x7f2a4c097620 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 10 monclient(hunting): my global_id is 4533 2024-09-18T14:01:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a4c097340 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575529302 0 0) 0x7f2a4c05ed70 con 0x7f2a4c0a1080 2024-09-18T14:01:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 10 monclient(hunting): my global_id is 134131 2024-09-18T14:01:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.068+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a4c097620 con 0x7f2a4c0a1080 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3624636226 0 0) 0x7f2a340034e0 con 0x7f2a4c06e810 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 >> v1:172.21.15.45:6792/0 conn(0x7f2a4c0a0cb0 legacy=0x7f2a4c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 >> v1:172.21.15.45:6791/0 conn(0x7f2a4c0a1080 legacy=0x7f2a4c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 1 monclient: found mon.noname-e 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c0a1660 con 0x7f2a4c06e810 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a3c001830 con 0x7f2a4c06e810 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 10 monclient: dump: 2024-09-18T14:01:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a5209f640 5 monclient: authenticate success, global_id 4544 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a5209f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2a3c001a10 con 0x7f2a4c06e810 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 1 -- 172.21.15.45:0/2799980079 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a3c004ba0 con 0x7f2a4c06e810 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a5209f640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a5209f640 4 set_mon_vals no callback set 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a4a7fc640 10 monclient: dump: 2024-09-18T14:01:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a5209f640 10 monclient: shutdown 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a5209f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a5209f640 1 -- 172.21.15.45:0/2799980079 >> v1:172.21.15.45:6793/0 conn(0x7f2a4c06e810 legacy=0x7f2a4c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a5209f640 1 -- 172.21.15.45:0/2799980079 shutdown_connections 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a5209f640 1 -- 172.21.15.45:0/2799980079 >> 172.21.15.45:0/2799980079 conn(0x7f2a4c0a5bb0 msgr2=0x7f2a4c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a5209f640 1 -- 172.21.15.45:0/2799980079 shutdown_connections 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.069+0000 7f2a5209f640 1 -- 172.21.15.45:0/2799980079 wait complete. 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.070+0000 7f2a5209f640 10 monclient: build_initial_monmap 2024-09-18T14:01:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.070+0000 7f2a5209f640 10 monclient: monmap: 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.070+0000 7f2a5209f640 1 Processor -- start 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.070+0000 7f2a5209f640 1 -- start start 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.070+0000 7f2a5209f640 10 monclient: init 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a5209f640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a5209f640 10 monclient: _add_conns ranks=[0,4,2,1,5,7,6,3,8] 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a5209f640 10 monclient(hunting): picked mon.noname-a con 0x7f2a4c06e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a5209f640 10 monclient(hunting): picked mon.noname-c con 0x7f2a4c0a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a5209f640 10 monclient(hunting): picked mon.noname-b con 0x7f2a4c0a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a5209f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c09fa10 con 0x7f2a4c06e810 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a5209f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c0a2270 con 0x7f2a4c0a1080 2024-09-18T14:01:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a5209f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c11c520 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a5209f640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a5209f640 10 monclient(hunting): authenticate will time out at 908.409354s 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a4affd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f2a4c0a0cb0 0x7f2a4c09bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36872/0 (socket says 172.21.15.45:36872) 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a4affd640 1 -- 172.21.15.45:0/951834194 learned_addr learned my addr 172.21.15.45:0/951834194 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985521838 0 0) 0x7f2a4c11c520 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 10 monclient(hunting): my global_id is 134140 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a380034e0 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350370883 0 0) 0x7f2a4c0a2270 con 0x7f2a4c0a1080 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 10 monclient(hunting): my global_id is 4610 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a4c11c520 con 0x7f2a4c0a1080 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086126141 0 0) 0x7f2a4c09fa10 con 0x7f2a4c06e810 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 10 monclient(hunting): my global_id is 4644 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a4c0a2270 con 0x7f2a4c06e810 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3559874498 0 0) 0x7f2a380034e0 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a4c09fa10 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a400015f0 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.071+0000 7f2a337fe640 10 monclient(hunting): dump: 2024-09-18T14:01:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3282463315 0 0) 0x7f2a4c11c520 con 0x7f2a4c0a1080 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a380034e0 con 0x7f2a4c0a1080 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3228944647 0 0) 0x7f2a4c0a2270 con 0x7f2a4c06e810 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a4c11c520 con 0x7f2a4c06e810 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a44001b10 con 0x7f2a4c0a1080 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient(hunting): dump: 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.092 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.092 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.092 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.092 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.092 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.092 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.092 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.092 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.092 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.092 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.092 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.092 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a3c0017f0 con 0x7f2a4c06e810 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient(hunting): dump: 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4039218133 0 0) 0x7f2a4c09fa10 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 >> v1:172.21.15.45:6790/0 conn(0x7f2a4c0a1080 legacy=0x7f2a4c09c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 >> v1:172.21.15.45:6789/0 conn(0x7f2a4c06e810 legacy=0x7f2a4c09b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient: _finish_hunting 0 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 monclient: found mon.c 2024-09-18T14:01:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c12dfa0 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient: _finish_auth 0 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a5209f640 5 monclient: authenticate success, global_id 134140 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a5209f640 10 monclient: _renew_subs 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a5209f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a5209f640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4c09d5b0 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a5209f640 10 monclient: _renew_subs 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a5209f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a5209f640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4c09d0a0 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2a40004a50 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a40004f00 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a49ffb640 4 set_mon_vals no callback set 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.072+0000 7f2a337fe640 10 monclient: dump: 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2a4000fed0 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a5209f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a5209f640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2a2c006580 con 0x7f2a2c002830 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(124..124 src has 1..124) v4 ==== 3955+0+0 (unknown 1177902436 0 0) 0x7f2a40046bb0 con 0x7f2a4c0a0cb0 2024-09-18T14:01:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884140424 0 0) 0x7f2a2c006580 con 0x7f2a2c002830 2024-09-18T14:01:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 10 monclient(hunting): my global_id is 4619 2024-09-18T14:01:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a4c09fa10 con 0x7f2a2c002830 2024-09-18T14:01:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a4bfff640 10 monclient: get_auth_request con 0x7f2a3803abd0 auth_method 0 2024-09-18T14:01:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 913662883 0 0) 0x7f2a4c09fa10 con 0x7f2a2c002830 2024-09-18T14:01:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a2c006580 con 0x7f2a2c002830 2024-09-18T14:01:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a3c001270 con 0x7f2a2c002830 2024-09-18T14:01:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 10 monclient: dump: 2024-09-18T14:01:33.098 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.099 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2416106727 0 0) 0x7f2a2c006580 con 0x7f2a2c002830 2024-09-18T14:01:33.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.073+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a2c00a0a0 con 0x7f2a2c002830 2024-09-18T14:01:33.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.074+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2a3c005160 con 0x7f2a2c002830 2024-09-18T14:01:33.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.074+0000 7f2a337fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:33.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.074+0000 7f2a337fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:33.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.074+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 >> v1:172.21.15.45:6790/0 conn(0x7f2a2c002830 legacy=0x7f2a2c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.074+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 >> v1:172.21.15.45:6790/0 conn(0x7f2a2c002830 legacy=0x7f2a2c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:33.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.079+0000 7f2a5209f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.079+0000 7f2a5209f640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2a2c009640 con 0x7f2a2c009020 2024-09-18T14:01:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815147866 0 0) 0x7f2a2c009640 con 0x7f2a2c009020 2024-09-18T14:01:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 10 monclient(hunting): my global_id is 4628 2024-09-18T14:01:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a2c006580 con 0x7f2a2c009020 2024-09-18T14:01:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2768494510 0 0) 0x7f2a2c006580 con 0x7f2a2c009020 2024-09-18T14:01:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a2c009640 con 0x7f2a2c009020 2024-09-18T14:01:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a3c005a60 con 0x7f2a2c009020 2024-09-18T14:01:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 10 monclient: dump: 2024-09-18T14:01:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.104 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3554612754 0 0) 0x7f2a2c009640 con 0x7f2a2c009020 2024-09-18T14:01:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2a2c009900 con 0x7f2a2c009020 2024-09-18T14:01:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 778004141) 0x7f2a3c008150 con 0x7f2a2c009020 2024-09-18T14:01:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 >> v1:172.21.15.45:6790/0 conn(0x7f2a2c009020 legacy=0x7f2a2c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.080+0000 7f2a337fe640 1 -- 172.21.15.45:0/951834194 >> v1:172.21.15.45:6790/0 conn(0x7f2a2c009020 legacy=0x7f2a2c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:33.105 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 2, 2024-09-18T14:01:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:01:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:01:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:33.125 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.081+0000 7f2a5209f640 1 -- 172.21.15.45:0/951834194 >> v1:172.21.15.84:6812/31478 conn(0x7f2a3803abd0 legacy=0x7f2a3803d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.081+0000 7f2a5209f640 10 monclient: shutdown 2024-09-18T14:01:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.081+0000 7f2a5209f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.081+0000 7f2a5209f640 1 -- 172.21.15.45:0/951834194 >> v1:172.21.15.45:6791/0 conn(0x7f2a4c0a0cb0 legacy=0x7f2a4c09bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.081+0000 7f2a4bfff640 1 -- 172.21.15.45:0/951834194 reap_dead start 2024-09-18T14:01:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.081+0000 7f2a4bfff640 1 -- 172.21.15.45:0/951834194 reap_dead start 2024-09-18T14:01:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.081+0000 7f2a5209f640 1 -- 172.21.15.45:0/951834194 shutdown_connections 2024-09-18T14:01:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.081+0000 7f2a5209f640 1 -- 172.21.15.45:0/951834194 >> 172.21.15.45:0/951834194 conn(0x7f2a4c0a5bb0 msgr2=0x7f2a4c133910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.081+0000 7f2a5209f640 1 -- 172.21.15.45:0/951834194 shutdown_connections 2024-09-18T14:01:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.081+0000 7f2a5209f640 1 -- 172.21.15.45:0/951834194 wait complete. 2024-09-18T14:01:33.128 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:33.128 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:01:33.128 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:01:33.221 INFO:tasks.rados.rados.0.smithi084.stdout:572: done (1 left) 2024-09-18T14:01:33.221 INFO:tasks.rados.rados.0.smithi084.stdout:573: done (0 left) 2024-09-18T14:01:33.221 INFO:tasks.rados.rados.0.smithi084.stdout:574: write oid 29 current snap is 55 2024-09-18T14:01:33.221 INFO:tasks.rados.rados.0.smithi084.stdout:574: seq_num 233 ranges {716031=623607,2084775=605732,3298969=617062} 2024-09-18T14:01:33.228 INFO:tasks.rados.rados.0.smithi084.stdout:574: writing smithi08434663-29 from 716031 to 1339638 tid 1 2024-09-18T14:01:33.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.229+0000 7f23c844f640 10 monclient: get_monmap_and_config 2024-09-18T14:01:33.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.229+0000 7f23c844f640 10 monclient: build_initial_monmap 2024-09-18T14:01:33.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.229+0000 7f23c844f640 10 monclient: monmap: 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:33.230387+0000 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:33.230387+0000 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:33.231 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:33.231 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.230+0000 7f23c844f640 1 Processor -- start 2024-09-18T14:01:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.230+0000 7f23c844f640 1 -- start start 2024-09-18T14:01:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.230+0000 7f23c844f640 10 monclient: init 2024-09-18T14:01:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.230+0000 7f23c844f640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.230+0000 7f23c844f640 10 monclient: _add_conns ranks=[7,0,5,1,3,2,6,8,4] 2024-09-18T14:01:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.230+0000 7f23c844f640 10 monclient(hunting): picked mon.noname-i con 0x7f23c006e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c844f640 10 monclient(hunting): picked mon.noname-a con 0x7f23c00a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c844f640 10 monclient(hunting): picked mon.noname-h con 0x7f23c00a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c844f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c005ed70 con 0x7f23c00a0cb0 2024-09-18T14:01:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c844f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c0097620 con 0x7f23c00a1080 2024-09-18T14:01:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c844f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c0097340 con 0x7f23c006e810 2024-09-18T14:01:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c844f640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c844f640 10 monclient(hunting): authenticate will time out at 908.569105s 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c69ec640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f23c00a1080 0x7f23c013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:47408/0 (socket says 172.21.15.45:47408) 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c69ec640 1 -- 172.21.15.45:0/1162799177 learned_addr learned my addr 172.21.15.45:0/1162799177 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 1 -- 172.21.15.45:0/1162799177 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059036610 0 0) 0x7f23c0097620 con 0x7f23c00a1080 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 10 monclient(hunting): my global_id is 4523 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 1 -- 172.21.15.45:0/1162799177 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23ac0034e0 con 0x7f23c00a1080 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 1 -- 172.21.15.45:0/1162799177 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886409662 0 0) 0x7f23c005ed70 con 0x7f23c00a0cb0 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 10 monclient(hunting): my global_id is 4653 2024-09-18T14:01:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 1 -- 172.21.15.45:0/1162799177 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c0097620 con 0x7f23c00a0cb0 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 1 -- 172.21.15.45:0/1162799177 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2278215660 0 0) 0x7f23ac0034e0 con 0x7f23c00a1080 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 1 -- 172.21.15.45:0/1162799177 >> v1:172.21.15.84:6792/0 conn(0x7f23c006e810 legacy=0x7f23c00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 1 -- 172.21.15.45:0/1162799177 >> v1:172.21.15.45:6789/0 conn(0x7f23c00a0cb0 legacy=0x7f23c0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 10 monclient: _finish_hunting 0 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 1 monclient: found mon.noname-h 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 1 -- 172.21.15.45:0/1162799177 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c00a1660 con 0x7f23c00a1080 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 10 monclient: _finish_auth 0 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 1 -- 172.21.15.45:0/1162799177 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f23b8001bb0 con 0x7f23c00a1080 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:01:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.231+0000 7f23c51e9640 10 monclient: dump: 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.232+0000 7f23c844f640 5 monclient: authenticate success, global_id 4523 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.232+0000 7f23c844f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.232+0000 7f23c51e9640 1 -- 172.21.15.45:0/1162799177 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f23b8001d90 con 0x7f23c00a1080 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.232+0000 7f23c51e9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.232+0000 7f23c51e9640 1 -- 172.21.15.45:0/1162799177 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f23b8004f20 con 0x7f23c00a1080 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.232+0000 7f23c51e9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.232+0000 7f23c51e9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.232+0000 7f23c51e9640 10 monclient: dump: 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.239 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.232+0000 7f23c844f640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.232+0000 7f23c844f640 4 set_mon_vals no callback set 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.233+0000 7f23c844f640 10 monclient: shutdown 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.233+0000 7f23c844f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.233+0000 7f23c844f640 1 -- 172.21.15.45:0/1162799177 >> v1:172.21.15.84:6791/0 conn(0x7f23c00a1080 legacy=0x7f23c013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.233+0000 7f23c844f640 1 -- 172.21.15.45:0/1162799177 shutdown_connections 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.233+0000 7f23c844f640 1 -- 172.21.15.45:0/1162799177 >> 172.21.15.45:0/1162799177 conn(0x7f23c00a5bb0 msgr2=0x7f23c00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.233+0000 7f23c844f640 1 -- 172.21.15.45:0/1162799177 shutdown_connections 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.233+0000 7f23c844f640 1 -- 172.21.15.45:0/1162799177 wait complete. 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.233+0000 7f23c844f640 10 monclient: build_initial_monmap 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.233+0000 7f23c844f640 10 monclient: monmap: 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 1 Processor -- start 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 1 -- start start 2024-09-18T14:01:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 10 monclient: init 2024-09-18T14:01:33.244 INFO:tasks.rados.rados.0.smithi084.stdout:574: writing smithi08434663-29 from 2084775 to 2690507 tid 2 2024-09-18T14:01:33.244 INFO:tasks.rados.rados.0.smithi084.stdout:574: writing smithi08434663-29 from 3298969 to 3916031 tid 3 2024-09-18T14:01:33.244 INFO:tasks.rados.rados.0.smithi084.stdout:575: write (excl) oid 40 current snap is 55 2024-09-18T14:01:33.244 INFO:tasks.rados.rados.0.smithi084.stdout:574: finishing write tid 1 to smithi08434663-29 2024-09-18T14:01:33.244 INFO:tasks.rados.rados.0.smithi084.stdout:574: finishing write tid 2 to smithi08434663-29 2024-09-18T14:01:33.244 INFO:tasks.rados.rados.0.smithi084.stdout:575: seq_num 234 ranges {732838=698856,2026904=465743,3210039=571651} 2024-09-18T14:01:33.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:33.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 10 monclient: _add_conns ranks=[4,7,1,0,6,3,5,2,8] 2024-09-18T14:01:33.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 10 monclient(hunting): picked mon.noname-c con 0x7f23c006e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:33.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 10 monclient(hunting): picked mon.noname-i con 0x7f23c00a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:33.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 10 monclient(hunting): picked mon.noname-f con 0x7f23c00a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:33.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c009fa10 con 0x7f23c00a1080 2024-09-18T14:01:33.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c013dbc0 con 0x7f23c006e810 2024-09-18T14:01:33.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c00a2270 con 0x7f23c00a0cb0 2024-09-18T14:01:33.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:33.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c844f640 10 monclient(hunting): authenticate will time out at 908.572876s 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c61eb640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f23c006e810 0x7f23c013d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36878/0 (socket says 172.21.15.45:36878) 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.234+0000 7f23c61eb640 1 -- 172.21.15.45:0/443949221 learned_addr learned my addr 172.21.15.45:0/443949221 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997615621 0 0) 0x7f23c013dbc0 con 0x7f23c006e810 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): my global_id is 134149 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23a40034e0 con 0x7f23c006e810 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853290560 0 0) 0x7f23c00a2270 con 0x7f23c00a0cb0 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): my global_id is 4624 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c013dbc0 con 0x7f23c00a0cb0 2024-09-18T14:01:33.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739119707 0 0) 0x7f23c009fa10 con 0x7f23c00a1080 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): my global_id is 4555 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c00a2270 con 0x7f23c00a1080 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3132823027 0 0) 0x7f23a40034e0 con 0x7f23c006e810 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f23c009fa10 con 0x7f23c006e810 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f23bc002b20 con 0x7f23c006e810 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): dump: 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.252 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.253 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 256800416 0 0) 0x7f23c013dbc0 con 0x7f23c00a0cb0 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f23a40034e0 con 0x7f23c00a0cb0 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f23b0001a60 con 0x7f23c00a0cb0 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): dump: 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.254 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.255 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.255 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.255 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.255 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.255 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.255 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.255 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.255 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.255 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4114403211 0 0) 0x7f23c00a2270 con 0x7f23c00a1080 2024-09-18T14:01:33.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f23c013dbc0 con 0x7f23c00a1080 2024-09-18T14:01:33.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f23b8001b70 con 0x7f23c00a1080 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient(hunting): dump: 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2184558426 0 0) 0x7f23c009fa10 con 0x7f23c006e810 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 >> v1:172.21.15.84:6792/0 conn(0x7f23c00a0cb0 legacy=0x7f23c01466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 >> v1:172.21.15.84:6789/0 conn(0x7f23c00a1080 legacy=0x7f23c0149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient: _finish_hunting 0 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 monclient: found mon.c 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:33.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c013e9d0 con 0x7f23c006e810 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient: _finish_auth 0 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23c844f640 5 monclient: authenticate success, global_id 134149 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23c844f640 10 monclient: _renew_subs 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23c844f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23c844f640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c0122550 con 0x7f23c006e810 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23c844f640 10 monclient: _renew_subs 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23c844f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23c844f640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c0120160 con 0x7f23c006e810 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f23bc001280 con 0x7f23c006e810 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f23bc001770 con 0x7f23c006e810 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23c49e8640 4 set_mon_vals no callback set 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.235+0000 7f23b67fc640 10 monclient: dump: 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.236+0000 7f23c844f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.236+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f23bc0018d0 con 0x7f23c006e810 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23c844f640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f23a8006580 con 0x7f23a8002830 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(125..125 src has 1..125) v4 ==== 3895+0+0 (unknown 3491418578 0 0) 0x7f23bc046b90 con 0x7f23c006e810 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23c69ec640 10 monclient: get_auth_request con 0x7f23a403aaa0 auth_method 0 2024-09-18T14:01:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926430969 0 0) 0x7f23a8006580 con 0x7f23a8002830 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 10 monclient(hunting): my global_id is 134158 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c009fa10 con 0x7f23a8002830 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 698423596 0 0) 0x7f23c009fa10 con 0x7f23a8002830 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f23a8006580 con 0x7f23a8002830 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f23b0002230 con 0x7f23a8002830 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 10 monclient: dump: 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.262 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.262 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.262 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.262 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.262 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.262 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.262 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.262 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.262 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.262 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.262 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.262 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2711847163 0 0) 0x7f23a8006580 con 0x7f23a8002830 2024-09-18T14:01:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.237+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23a800a0a0 con 0x7f23a8002830 2024-09-18T14:01:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.238+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f23b0002590 con 0x7f23a8002830 2024-09-18T14:01:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.238+0000 7f23b67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.238+0000 7f23b67fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.238+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 >> v1:172.21.15.45:6791/0 conn(0x7f23a8002830 legacy=0x7f23a8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.238+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 >> v1:172.21.15.45:6791/0 conn(0x7f23a8002830 legacy=0x7f23a8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.245+0000 7f23c844f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.245+0000 7f23c844f640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f23a8009660 con 0x7f23a8009040 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.245+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753280178 0 0) 0x7f23a8009660 con 0x7f23a8009040 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.245+0000 7f23b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.245+0000 7f23b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.245+0000 7f23b67fc640 10 monclient(hunting): my global_id is 134167 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.245+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23a8006580 con 0x7f23a8009040 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4293863049 0 0) 0x7f23a8006580 con 0x7f23a8009040 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f23a8009660 con 0x7f23a8009040 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f23b0002b70 con 0x7f23a8009040 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 10 monclient: dump: 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.266 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4065355482 0 0) 0x7f23a8009660 con 0x7f23a8009040 2024-09-18T14:01:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f23a8009920 con 0x7f23a8009040 2024-09-18T14:01:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3587152683) 0x7f23b00033c0 con 0x7f23a8009040 2024-09-18T14:01:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 >> v1:172.21.15.45:6791/0 conn(0x7f23a8009040 legacy=0x7f23a8008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.246+0000 7f23b67fc640 1 -- 172.21.15.45:0/443949221 >> v1:172.21.15.45:6791/0 conn(0x7f23a8009040 legacy=0x7f23a8008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:33.266 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:01:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:33.286 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:33.287 INFO:tasks.rados.rados.0.smithi084.stdout:575: writing smithi08434663-40 from 732838 to 1431694 tid 1 2024-09-18T14:01:33.288 INFO:tasks.rados.rados.0.smithi084.stdout:575: writing smithi08434663-40 from 2026904 to 2492647 tid 2 2024-09-18T14:01:33.288 INFO:tasks.rados.rados.0.smithi084.stdout:575: writing smithi08434663-40 from 3210039 to 3781690 tid 3 2024-09-18T14:01:33.288 INFO:tasks.rados.rados.0.smithi084.stdout:576: read oid 2 snap 52 2024-09-18T14:01:33.288 INFO:tasks.rados.rados.0.smithi084.stdout:576: expect deleted 2024-09-18T14:01:33.288 INFO:tasks.rados.rados.0.smithi084.stdout:577: snap_remove snap 53 2024-09-18T14:01:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.247+0000 7f23c844f640 1 -- 172.21.15.45:0/443949221 >> v1:172.21.15.84:6812/31478 conn(0x7f23a403aaa0 legacy=0x7f23a403cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.247+0000 7f23c844f640 10 monclient: shutdown 2024-09-18T14:01:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.247+0000 7f23c844f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.247+0000 7f23c844f640 1 -- 172.21.15.45:0/443949221 >> v1:172.21.15.45:6791/0 conn(0x7f23c006e810 legacy=0x7f23c013d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.247+0000 7f23c69ec640 1 -- 172.21.15.45:0/443949221 reap_dead start 2024-09-18T14:01:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.247+0000 7f23c69ec640 1 -- 172.21.15.45:0/443949221 reap_dead start 2024-09-18T14:01:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.247+0000 7f23c844f640 1 -- 172.21.15.45:0/443949221 shutdown_connections 2024-09-18T14:01:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.247+0000 7f23c844f640 1 -- 172.21.15.45:0/443949221 >> 172.21.15.45:0/443949221 conn(0x7f23c00a5bb0 msgr2=0x7f23c00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.247+0000 7f23c844f640 1 -- 172.21.15.45:0/443949221 shutdown_connections 2024-09-18T14:01:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.247+0000 7f23c844f640 1 -- 172.21.15.45:0/443949221 wait complete. 2024-09-18T14:01:33.289 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:33.289 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:01:33.290 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:01:33.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.386+0000 7f538a31d640 10 monclient: get_monmap_and_config 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.386+0000 7f538a31d640 10 monclient: build_initial_monmap 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.386+0000 7f538a31d640 10 monclient: monmap: 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:33.387247+0000 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:33.387247+0000 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:33.388 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.386+0000 7f538a31d640 1 Processor -- start 2024-09-18T14:01:33.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f538a31d640 1 -- start start 2024-09-18T14:01:33.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f538a31d640 10 monclient: init 2024-09-18T14:01:33.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f538a31d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:33.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f538a31d640 10 monclient: _add_conns ranks=[4,8,6,3,5,2,0,7,1] 2024-09-18T14:01:33.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f538a31d640 10 monclient(hunting): picked mon.noname-c con 0x7f538406e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f538a31d640 10 monclient(hunting): picked mon.noname-e con 0x7f53840a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f538a31d640 10 monclient(hunting): picked mon.noname-d con 0x7f53840a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f538a31d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538405ed70 con 0x7f538406e810 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f538a31d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5384097620 con 0x7f53840a1080 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f538a31d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5384097340 con 0x7f53840a0cb0 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f538a31d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f538a31d640 10 monclient(hunting): authenticate will time out at 908.725610s 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f5383fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f538406e810 0x7f53840a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36900/0 (socket says 172.21.15.45:36900) 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f5383fff640 1 -- 172.21.15.45:0/1918308044 learned_addr learned my addr 172.21.15.45:0/1918308044 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.387+0000 7f53837fe640 1 --1- 172.21.15.45:0/1918308044 >> v1:172.21.15.45:6793/0 conn(0x7f53840a0cb0 0x7f5384137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:60938/0 (socket says 172.21.15.45:60938) 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429216632 0 0) 0x7f538405ed70 con 0x7f538406e810 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient(hunting): my global_id is 134176 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f536c0034e0 con 0x7f538406e810 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524360356 0 0) 0x7f5384097620 con 0x7f53840a1080 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient(hunting): my global_id is 4542 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f538405ed70 con 0x7f53840a1080 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059771662 0 0) 0x7f5384097340 con 0x7f53840a0cb0 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient(hunting): my global_id is 4553 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5384097620 con 0x7f53840a0cb0 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2205633739 0 0) 0x7f536c0034e0 con 0x7f538406e810 2024-09-18T14:01:33.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 >> v1:172.21.15.45:6793/0 conn(0x7f53840a0cb0 legacy=0x7f5384137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 >> v1:172.21.15.45:6792/0 conn(0x7f53840a1080 legacy=0x7f538413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 monclient: found mon.noname-c 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53840a1660 con 0x7f538406e810 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient: _finish_auth 0 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5374001830 con 0x7f538406e810 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient: dump: 2024-09-18T14:01:33.398 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.399 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f538a31d640 5 monclient: authenticate success, global_id 134176 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f538a31d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5374001990 con 0x7f538406e810 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 1 -- 172.21.15.45:0/1918308044 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5374004860 con 0x7f538406e810 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f538a31d640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f538a31d640 4 set_mon_vals no callback set 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f5382ffd640 10 monclient: dump: 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f538a31d640 10 monclient: shutdown 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f538a31d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.388+0000 7f538a31d640 1 -- 172.21.15.45:0/1918308044 >> v1:172.21.15.45:6791/0 conn(0x7f538406e810 legacy=0x7f53840a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.389+0000 7f538a31d640 1 -- 172.21.15.45:0/1918308044 shutdown_connections 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.389+0000 7f538a31d640 1 -- 172.21.15.45:0/1918308044 >> 172.21.15.45:0/1918308044 conn(0x7f53840a5bb0 msgr2=0x7f53840a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.389+0000 7f538a31d640 1 -- 172.21.15.45:0/1918308044 shutdown_connections 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.389+0000 7f538a31d640 1 -- 172.21.15.45:0/1918308044 wait complete. 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.389+0000 7f538a31d640 10 monclient: build_initial_monmap 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.389+0000 7f538a31d640 10 monclient: monmap: 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:33.402 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.389+0000 7f538a31d640 1 Processor -- start 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f538a31d640 1 -- start start 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f538a31d640 10 monclient: init 2024-09-18T14:01:33.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f538a31d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f538a31d640 10 monclient: _add_conns ranks=[8,7,3,6,2,0,1,4,5] 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f538a31d640 10 monclient(hunting): picked mon.noname-e con 0x7f538406e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f538a31d640 10 monclient(hunting): picked mon.noname-i con 0x7f53840a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f538a31d640 10 monclient(hunting): picked mon.noname-g con 0x7f53840a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f538a31d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538409fa10 con 0x7f53840a1080 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f538a31d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53840a2270 con 0x7f53840a0cb0 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f538a31d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5384120e70 con 0x7f538406e810 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f538a31d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f538a31d640 10 monclient(hunting): authenticate will time out at 908.728523s 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f5383fff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f538406e810 0x7f5384135370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:60942/0 (socket says 172.21.15.45:60942) 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f5383fff640 1 -- 172.21.15.45:0/2237436346 learned_addr learned my addr 172.21.15.45:0/2237436346 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 54579062 0 0) 0x7f5384120e70 con 0x7f538406e810 2024-09-18T14:01:33.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f536bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f536bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f536bfff640 10 monclient(hunting): my global_id is 4562 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53700034e0 con 0x7f538406e810 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1370894110 0 0) 0x7f53700034e0 con 0x7f538406e810 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5384120e70 con 0x7f538406e810 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f53740016f0 con 0x7f538406e810 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.390+0000 7f536bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient(hunting): dump: 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.405 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.406 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.406 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.406 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.406 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.406 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.406 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.406 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.406 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.406 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.406 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.406 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703775797 0 0) 0x7f53840a2270 con 0x7f53840a0cb0 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient(hunting): my global_id is 4633 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53700034e0 con 0x7f53840a0cb0 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881828614 0 0) 0x7f538409fa10 con 0x7f53840a1080 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient(hunting): my global_id is 4665 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53840a2270 con 0x7f53840a1080 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 305047737 0 0) 0x7f5384120e70 con 0x7f538406e810 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 >> v1:172.21.15.84:6792/0 conn(0x7f53840a0cb0 legacy=0x7f53841468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 >> v1:172.21.15.84:6790/0 conn(0x7f53840a1080 legacy=0x7f5384149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:01:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 1 monclient: found mon.e 2024-09-18T14:01:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5384122a50 con 0x7f538406e810 2024-09-18T14:01:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient: _finish_auth 0 2024-09-18T14:01:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f538a31d640 5 monclient: authenticate success, global_id 4562 2024-09-18T14:01:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f538a31d640 10 monclient: _renew_subs 2024-09-18T14:01:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f538a31d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f538a31d640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5384122490 con 0x7f538406e810 2024-09-18T14:01:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f538a31d640 10 monclient: _renew_subs 2024-09-18T14:01:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f538a31d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f538a31d640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5384121530 con 0x7f538406e810 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5374004d90 con 0x7f538406e810 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5374005240 con 0x7f538406e810 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f53827fc640 4 set_mon_vals no callback set 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.391+0000 7f536bfff640 10 monclient: dump: 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.410 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.410 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.410 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.410 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.410 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f538a31d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f538a31d640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5364006580 con 0x7f5364002830 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f5374010210 con 0x7f538406e810 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(125..125 src has 1..125) v4 ==== 3895+0+0 (unknown 3491418578 0 0) 0x7f5374046e00 con 0x7f538406e810 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301894234 0 0) 0x7f5364006580 con 0x7f5364002830 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 10 monclient(hunting): my global_id is 4551 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5384120e70 con 0x7f5364002830 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f53888ba640 10 monclient: get_auth_request con 0x7f5370039f30 auth_method 0 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2431433612 0 0) 0x7f5384120e70 con 0x7f5364002830 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5364006580 con 0x7f5364002830 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f53780012d0 con 0x7f5364002830 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 10 monclient: dump: 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1003582220 0 0) 0x7f5364006580 con 0x7f5364002830 2024-09-18T14:01:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.392+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f536400a0a0 con 0x7f5364002830 2024-09-18T14:01:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.393+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f53780047a0 con 0x7f5364002830 2024-09-18T14:01:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.393+0000 7f536bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.393+0000 7f536bfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.393+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 >> v1:172.21.15.45:6792/0 conn(0x7f5364002830 legacy=0x7f5364004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.393+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 >> v1:172.21.15.45:6792/0 conn(0x7f5364002830 legacy=0x7f5364004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:33.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.398+0000 7f538a31d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:33.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.398+0000 7f538a31d640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5364007540 con 0x7f53640068f0 2024-09-18T14:01:33.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492258814 0 0) 0x7f5364007540 con 0x7f53640068f0 2024-09-18T14:01:33.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 10 monclient(hunting): my global_id is 4560 2024-09-18T14:01:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5364006580 con 0x7f53640068f0 2024-09-18T14:01:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2739539438 0 0) 0x7f5364006580 con 0x7f53640068f0 2024-09-18T14:01:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5364007540 con 0x7f53640068f0 2024-09-18T14:01:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5378004fc0 con 0x7f53640068f0 2024-09-18T14:01:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:01:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 10 monclient: dump: 2024-09-18T14:01:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.418 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2674220362 0 0) 0x7f5364007540 con 0x7f53640068f0 2024-09-18T14:01:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f53640087a0 con 0x7f53640068f0 2024-09-18T14:01:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3066648464) 0x7f5378005260 con 0x7f53640068f0 2024-09-18T14:01:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 >> v1:172.21.15.45:6792/0 conn(0x7f53640068f0 legacy=0x7f5364006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.399+0000 7f536bfff640 1 -- 172.21.15.45:0/2237436346 >> v1:172.21.15.45:6792/0 conn(0x7f53640068f0 legacy=0x7f5364006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:33.418 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:33.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.438 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:33.438 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.400+0000 7f538a31d640 1 -- 172.21.15.45:0/2237436346 >> v1:172.21.15.84:6812/31478 conn(0x7f5370039f30 legacy=0x7f537003c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.400+0000 7f538a31d640 10 monclient: shutdown 2024-09-18T14:01:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.400+0000 7f538a31d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.400+0000 7f538a31d640 1 -- 172.21.15.45:0/2237436346 >> v1:172.21.15.45:6793/0 conn(0x7f538406e810 legacy=0x7f5384135370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.400+0000 7f53888ba640 1 -- 172.21.15.45:0/2237436346 reap_dead start 2024-09-18T14:01:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.400+0000 7f53888ba640 1 -- 172.21.15.45:0/2237436346 reap_dead start 2024-09-18T14:01:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.400+0000 7f538a31d640 1 -- 172.21.15.45:0/2237436346 shutdown_connections 2024-09-18T14:01:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.400+0000 7f538a31d640 1 -- 172.21.15.45:0/2237436346 >> 172.21.15.45:0/2237436346 conn(0x7f53840a5bb0 msgr2=0x7f53840a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:33.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.400+0000 7f538a31d640 1 -- 172.21.15.45:0/2237436346 shutdown_connections 2024-09-18T14:01:33.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.400+0000 7f538a31d640 1 -- 172.21.15.45:0/2237436346 wait complete. 2024-09-18T14:01:33.440 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:33.440 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:01:33.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:01:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.536+0000 7f0e5bd5d640 10 monclient: get_monmap_and_config 2024-09-18T14:01:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.536+0000 7f0e5bd5d640 10 monclient: build_initial_monmap 2024-09-18T14:01:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.536+0000 7f0e5bd5d640 10 monclient: monmap: 2024-09-18T14:01:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:33.537819+0000 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:33.537819+0000 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:33.538 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.537+0000 7f0e5bd5d640 1 Processor -- start 2024-09-18T14:01:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.537+0000 7f0e5bd5d640 1 -- start start 2024-09-18T14:01:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.537+0000 7f0e5bd5d640 10 monclient: init 2024-09-18T14:01:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 10 monclient: _add_conns ranks=[4,2,8,5,7,6,3,0,1] 2024-09-18T14:01:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 10 monclient(hunting): picked mon.noname-c con 0x7f0e5406e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 10 monclient(hunting): picked mon.noname-b con 0x7f0e540a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 10 monclient(hunting): picked mon.noname-e con 0x7f0e540a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5405ed70 con 0x7f0e540a0cb0 2024-09-18T14:01:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e54097620 con 0x7f0e5406e810 2024-09-18T14:01:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e54097340 con 0x7f0e540a1080 2024-09-18T14:01:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 10 monclient(hunting): authenticate will time out at 908.876190s 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e592f8640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f0e540a0cb0 0x7f0e54137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43684/0 (socket says 172.21.15.45:43684) 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5a2fa640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f0e540a1080 0x7f0e5413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:60952/0 (socket says 172.21.15.45:60952) 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e592f8640 1 -- 172.21.15.45:0/2128951434 learned_addr learned my addr 172.21.15.45:0/2128951434 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484354343 0 0) 0x7f0e54097340 con 0x7f0e540a1080 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient(hunting): my global_id is 4571 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e400034e0 con 0x7f0e540a1080 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351933078 0 0) 0x7f0e5405ed70 con 0x7f0e540a0cb0 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient(hunting): my global_id is 4637 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e54097340 con 0x7f0e540a0cb0 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448765788 0 0) 0x7f0e54097620 con 0x7f0e5406e810 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient(hunting): my global_id is 134185 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5405ed70 con 0x7f0e5406e810 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2448172424 0 0) 0x7f0e400034e0 con 0x7f0e540a1080 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 >> v1:172.21.15.45:6791/0 conn(0x7f0e5406e810 legacy=0x7f0e540a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 >> v1:172.21.15.45:6790/0 conn(0x7f0e540a0cb0 legacy=0x7f0e54137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient: _finish_hunting 0 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 1 monclient: found mon.noname-e 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:01:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e540a1660 con 0x7f0e540a1080 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient: _finish_auth 0 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e4c001ae0 con 0x7f0e540a1080 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 5 monclient: authenticate success, global_id 4571 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e58af7640 10 monclient: dump: 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.538+0000 7f0e5bd5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0e4c001c40 con 0x7f0e540a1080 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e58af7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e58af7640 1 -- 172.21.15.45:0/2128951434 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e4c004b20 con 0x7f0e540a1080 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e5bd5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e5bd5d640 4 set_mon_vals no callback set 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e58af7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e58af7640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e58af7640 10 monclient: dump: 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e5bd5d640 10 monclient: shutdown 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e5bd5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/2128951434 >> v1:172.21.15.45:6793/0 conn(0x7f0e540a1080 legacy=0x7f0e5413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/2128951434 shutdown_connections 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/2128951434 >> 172.21.15.45:0/2128951434 conn(0x7f0e540a5bb0 msgr2=0x7f0e540a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/2128951434 shutdown_connections 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.539+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/2128951434 wait complete. 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 10 monclient: build_initial_monmap 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 10 monclient: monmap: 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 1 Processor -- start 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 1 -- start start 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 10 monclient: init 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 10 monclient: _add_conns ranks=[2,6,3,0,4,7,1,8,5] 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 10 monclient(hunting): picked mon.noname-b con 0x7f0e5406e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 10 monclient(hunting): picked mon.noname-d con 0x7f0e540a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 10 monclient(hunting): picked mon.noname-g con 0x7f0e540a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:01:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5409fa10 con 0x7f0e5406e810 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5413db20 con 0x7f0e540a1080 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e540a2270 con 0x7f0e540a0cb0 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.540+0000 7f0e5bd5d640 10 monclient(hunting): authenticate will time out at 908.878955s 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e59af9640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f0e5406e810 0x7f0e5413d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43698/0 (socket says 172.21.15.45:43698) 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e59af9640 1 -- 172.21.15.45:0/1961703706 learned_addr learned my addr 172.21.15.45:0/1961703706 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727134143 0 0) 0x7f0e5409fa10 con 0x7f0e5406e810 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): my global_id is 4646 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e380034e0 con 0x7f0e5406e810 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066197292 0 0) 0x7f0e540a2270 con 0x7f0e540a0cb0 2024-09-18T14:01:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): my global_id is 4569 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5409fa10 con 0x7f0e540a0cb0 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539660768 0 0) 0x7f0e5413db20 con 0x7f0e540a1080 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): my global_id is 4674 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e540a2270 con 0x7f0e540a1080 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1809407320 0 0) 0x7f0e380034e0 con 0x7f0e5406e810 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e5413db20 con 0x7f0e5406e810 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e48001b80 con 0x7f0e5406e810 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): dump: 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1663542922 0 0) 0x7f0e5409fa10 con 0x7f0e540a0cb0 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e380034e0 con 0x7f0e540a0cb0 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2507459123 0 0) 0x7f0e540a2270 con 0x7f0e540a1080 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e5409fa10 con 0x7f0e540a1080 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e50001550 con 0x7f0e540a0cb0 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): dump: 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e4c001aa0 con 0x7f0e540a1080 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient(hunting): dump: 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2266156913 0 0) 0x7f0e5413db20 con 0x7f0e5406e810 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 >> v1:172.21.15.45:6792/0 conn(0x7f0e540a0cb0 legacy=0x7f0e54146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 >> v1:172.21.15.84:6790/0 conn(0x7f0e540a1080 legacy=0x7f0e54149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 monclient: found mon.b 2024-09-18T14:01:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5413e930 con 0x7f0e5406e810 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient: _finish_auth 0 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e5bd5d640 5 monclient: authenticate success, global_id 4646 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e5bd5d640 10 monclient: _renew_subs 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e5bd5d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e541224b0 con 0x7f0e5406e810 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0e48001fd0 con 0x7f0e5406e810 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e45ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.542+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e48002540 con 0x7f0e5406e810 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.542+0000 7f0e45ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.542+0000 7f0e45ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.542+0000 7f0e45ffb640 10 monclient: dump: 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.542+0000 7f0e5bd5d640 10 monclient: _renew_subs 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.541+0000 7f0e47fff640 4 set_mon_vals no callback set 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.542+0000 7f0e5bd5d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.542+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e541200c0 con 0x7f0e5406e810 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.542+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f0e4800ff10 con 0x7f0e5406e810 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.542+0000 7f0e5bd5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.542+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0e5404c4a0 con 0x7f0e54121590 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.542+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(125..125 src has 1..125) v4 ==== 3895+0+0 (unknown 3491418578 0 0) 0x7f0e48046a60 con 0x7f0e5406e810 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457280489 0 0) 0x7f0e5404c4a0 con 0x7f0e54121590 2024-09-18T14:01:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 10 monclient(hunting): my global_id is 4580 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5413db20 con 0x7f0e54121590 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e5a2fa640 10 monclient: get_auth_request con 0x7f0e3803aba0 auth_method 0 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 711518401 0 0) 0x7f0e5413db20 con 0x7f0e54121590 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e5404c4a0 con 0x7f0e54121590 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e50001870 con 0x7f0e54121590 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 10 monclient: dump: 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.568 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1818694653 0 0) 0x7f0e5404c4a0 con 0x7f0e54121590 2024-09-18T14:01:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0e5404b720 con 0x7f0e54121590 2024-09-18T14:01:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0e50005040 con 0x7f0e54121590 2024-09-18T14:01:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 >> v1:172.21.15.45:6793/0 conn(0x7f0e54121590 legacy=0x7f0e5409baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.543+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 >> v1:172.21.15.45:6793/0 conn(0x7f0e54121590 legacy=0x7f0e5409baa0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.549+0000 7f0e5bd5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.549+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0e5413e130 con 0x7f0e5413dd60 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.549+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646078593 0 0) 0x7f0e5413e130 con 0x7f0e5413dd60 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.549+0000 7f0e45ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.549+0000 7f0e45ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.549+0000 7f0e45ffb640 10 monclient(hunting): my global_id is 4589 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.549+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5404c4a0 con 0x7f0e5413dd60 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.549+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2673626622 0 0) 0x7f0e5404c4a0 con 0x7f0e5413dd60 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.549+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e5413e130 con 0x7f0e5413dd60 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e500057a0 con 0x7f0e5413dd60 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e45ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e45ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e45ffb640 10 monclient: dump: 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.571 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.572 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 511350831 0 0) 0x7f0e5413e130 con 0x7f0e5413dd60 2024-09-18T14:01:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0e54120370 con 0x7f0e5413dd60 2024-09-18T14:01:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7388 (unknown 0 0 3662444727) 0x7f0e50005900 con 0x7f0e5413dd60 2024-09-18T14:01:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e45ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e45ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 >> v1:172.21.15.45:6793/0 conn(0x7f0e5413dd60 legacy=0x7f0e5409c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e45ffb640 1 -- 172.21.15.45:0/1961703706 >> v1:172.21.15.45:6793/0 conn(0x7f0e5413dd60 legacy=0x7f0e5409c1b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:33.572 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:33.591 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:33.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/1961703706 >> v1:172.21.15.84:6812/31478 conn(0x7f0e3803aba0 legacy=0x7f0e3803d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e5bd5d640 10 monclient: shutdown 2024-09-18T14:01:33.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e5bd5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:33.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/1961703706 >> v1:172.21.15.45:6790/0 conn(0x7f0e5406e810 legacy=0x7f0e5413d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e5a2fa640 1 -- 172.21.15.45:0/1961703706 reap_dead start 2024-09-18T14:01:33.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e5a2fa640 1 -- 172.21.15.45:0/1961703706 reap_dead start 2024-09-18T14:01:33.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/1961703706 shutdown_connections 2024-09-18T14:01:33.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/1961703706 >> 172.21.15.45:0/1961703706 conn(0x7f0e540a5bb0 msgr2=0x7f0e540a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:33.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/1961703706 shutdown_connections 2024-09-18T14:01:33.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.550+0000 7f0e5bd5d640 1 -- 172.21.15.45:0/1961703706 wait complete. 2024-09-18T14:01:33.594 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:33.594 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:01:33.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:01:33.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.690+0000 7fe46a79d640 10 monclient: get_monmap_and_config 2024-09-18T14:01:33.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.690+0000 7fe46a79d640 10 monclient: build_initial_monmap 2024-09-18T14:01:33.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.690+0000 7fe46a79d640 10 monclient: monmap: 2024-09-18T14:01:33.691 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:33.691 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.691 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:33.691434+0000 2024-09-18T14:01:33.691 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:33.691434+0000 2024-09-18T14:01:33.692 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:33.692 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.692 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:33.692 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:33.692 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:33.692 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:33.692 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:33.692 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:33.692 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:33.693 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:33.693 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:33.693 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 1 Processor -- start 2024-09-18T14:01:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 1 -- start start 2024-09-18T14:01:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 10 monclient: init 2024-09-18T14:01:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 10 monclient: _add_conns ranks=[3,2,8,6,7,0,5,4,1] 2024-09-18T14:01:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 10 monclient(hunting): picked mon.noname-g con 0x7fe46409f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:01:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 10 monclient(hunting): picked mon.noname-b con 0x7fe46409fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 10 monclient(hunting): picked mon.noname-e con 0x7fe46409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46405ed70 con 0x7fe46409fba0 2024-09-18T14:01:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46409dfc0 con 0x7fe46409f7d0 2024-09-18T14:01:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46409dce0 con 0x7fe46409ff70 2024-09-18T14:01:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe46a79d640 10 monclient(hunting): authenticate will time out at 909.029767s 2024-09-18T14:01:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe4637fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fe46409fba0 0x7fe464137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43700/0 (socket says 172.21.15.45:43700) 2024-09-18T14:01:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe4637fe640 1 -- 172.21.15.45:0/2920774943 learned_addr learned my addr 172.21.15.45:0/2920774943 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.691+0000 7fe468d3a640 1 --1- 172.21.15.45:0/2920774943 >> v1:172.21.15.45:6793/0 conn(0x7fe46409ff70 0x7fe46413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:60972/0 (socket says 172.21.15.45:60972) 2024-09-18T14:01:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147036715 0 0) 0x7fe46405ed70 con 0x7fe46409fba0 2024-09-18T14:01:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient(hunting): my global_id is 4655 2024-09-18T14:01:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe44c0034e0 con 0x7fe46409fba0 2024-09-18T14:01:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775484064 0 0) 0x7fe46409dce0 con 0x7fe46409ff70 2024-09-18T14:01:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient(hunting): my global_id is 4598 2024-09-18T14:01:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe46405ed70 con 0x7fe46409ff70 2024-09-18T14:01:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84703753 0 0) 0x7fe46409dfc0 con 0x7fe46409f7d0 2024-09-18T14:01:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient(hunting): my global_id is 4683 2024-09-18T14:01:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe46409dce0 con 0x7fe46409f7d0 2024-09-18T14:01:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2989561718 0 0) 0x7fe46405ed70 con 0x7fe46409ff70 2024-09-18T14:01:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 >> v1:172.21.15.84:6790/0 conn(0x7fe46409f7d0 legacy=0x7fe464132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 >> v1:172.21.15.45:6790/0 conn(0x7fe46409fba0 legacy=0x7fe464137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:01:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 monclient: found mon.noname-e 2024-09-18T14:01:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:01:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4640a1c00 con 0x7fe46409ff70 2024-09-18T14:01:33.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient: _finish_auth 0 2024-09-18T14:01:33.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:33.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe46a79d640 5 monclient: authenticate success, global_id 4598 2024-09-18T14:01:33.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe45c001b90 con 0x7fe46409ff70 2024-09-18T14:01:33.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe46a79d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:33.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:33.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient: dump: 2024-09-18T14:01:33.708 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.708 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.708 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.708 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.708 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.708 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.708 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.708 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.708 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.708 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.709 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe46a79d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe45c001cf0 con 0x7fe46409ff70 2024-09-18T14:01:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 1 -- 172.21.15.45:0/2920774943 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe45c004e60 con 0x7fe46409ff70 2024-09-18T14:01:33.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:33.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe462ffd640 10 monclient: dump: 2024-09-18T14:01:33.710 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.710 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.710 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.710 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.710 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.711 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.711 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.711 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.711 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.711 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.711 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.711 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.712 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.712 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.712 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.712 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe46a79d640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:33.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.692+0000 7fe46a79d640 4 set_mon_vals no callback set 2024-09-18T14:01:33.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.693+0000 7fe46a79d640 10 monclient: shutdown 2024-09-18T14:01:33.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.693+0000 7fe46a79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:33.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.693+0000 7fe46a79d640 1 -- 172.21.15.45:0/2920774943 >> v1:172.21.15.45:6793/0 conn(0x7fe46409ff70 legacy=0x7fe46413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.693+0000 7fe46a79d640 1 -- 172.21.15.45:0/2920774943 shutdown_connections 2024-09-18T14:01:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.693+0000 7fe46a79d640 1 -- 172.21.15.45:0/2920774943 >> 172.21.15.45:0/2920774943 conn(0x7fe4640630b0 msgr2=0x7fe4640634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.693+0000 7fe46a79d640 1 -- 172.21.15.45:0/2920774943 shutdown_connections 2024-09-18T14:01:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.693+0000 7fe46a79d640 1 -- 172.21.15.45:0/2920774943 wait complete. 2024-09-18T14:01:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.693+0000 7fe46a79d640 10 monclient: build_initial_monmap 2024-09-18T14:01:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.693+0000 7fe46a79d640 10 monclient: monmap: 2024-09-18T14:01:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:33.715 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.693+0000 7fe46a79d640 1 Processor -- start 2024-09-18T14:01:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe46a79d640 1 -- start start 2024-09-18T14:01:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe46a79d640 10 monclient: init 2024-09-18T14:01:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe46a79d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe46a79d640 10 monclient: _add_conns ranks=[0,1,8,4,2,5,6,7,3] 2024-09-18T14:01:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe46a79d640 10 monclient(hunting): picked mon.noname-a con 0x7fe46409f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe46a79d640 10 monclient(hunting): picked mon.noname-f con 0x7fe46409fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe46a79d640 10 monclient(hunting): picked mon.noname-e con 0x7fe46409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:33.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe46a79d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4641208b0 con 0x7fe46409f7d0 2024-09-18T14:01:33.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe46a79d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe464121ee0 con 0x7fe46409fba0 2024-09-18T14:01:33.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe46a79d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46409d310 con 0x7fe46409ff70 2024-09-18T14:01:33.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe46a79d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:33.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe46a79d640 10 monclient(hunting): authenticate will time out at 909.032667s 2024-09-18T14:01:33.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe463fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe46409f7d0 0x7fe4641418b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51416/0 (socket says 172.21.15.45:51416) 2024-09-18T14:01:33.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe468d3a640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fe46409ff70 0x7fe464149c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:60986/0 (socket says 172.21.15.45:60986) 2024-09-18T14:01:33.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe463fff640 1 -- 172.21.15.45:0/2484904128 learned_addr learned my addr 172.21.15.45:0/2484904128 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3324568731 0 0) 0x7fe46409d310 con 0x7fe46409ff70 2024-09-18T14:01:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe44bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe44bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe44bfff640 10 monclient(hunting): my global_id is 4607 2024-09-18T14:01:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4500034e0 con 0x7fe46409ff70 2024-09-18T14:01:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997617034 0 0) 0x7fe4641208b0 con 0x7fe46409f7d0 2024-09-18T14:01:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe44bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe44bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.694+0000 7fe44bfff640 10 monclient(hunting): my global_id is 4662 2024-09-18T14:01:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe46409d310 con 0x7fe46409f7d0 2024-09-18T14:01:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438779660 0 0) 0x7fe464121ee0 con 0x7fe46409fba0 2024-09-18T14:01:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): my global_id is 4564 2024-09-18T14:01:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4641208b0 con 0x7fe46409fba0 2024-09-18T14:01:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 5527110 0 0) 0x7fe4500034e0 con 0x7fe46409ff70 2024-09-18T14:01:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe464121ee0 con 0x7fe46409ff70 2024-09-18T14:01:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe45c001ad0 con 0x7fe46409ff70 2024-09-18T14:01:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): dump: 2024-09-18T14:01:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.719 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.719 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.719 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.719 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.719 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.719 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.719 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.719 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.719 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.719 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.719 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.719 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.720 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1292374243 0 0) 0x7fe46409d310 con 0x7fe46409f7d0 2024-09-18T14:01:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe4500034e0 con 0x7fe46409f7d0 2024-09-18T14:01:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe454001b20 con 0x7fe46409f7d0 2024-09-18T14:01:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): dump: 2024-09-18T14:01:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.722 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1338887991 0 0) 0x7fe4641208b0 con 0x7fe46409fba0 2024-09-18T14:01:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe46409d310 con 0x7fe46409fba0 2024-09-18T14:01:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe4580014e0 con 0x7fe46409fba0 2024-09-18T14:01:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:01:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient(hunting): dump: 2024-09-18T14:01:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.724 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2730605015 0 0) 0x7fe464121ee0 con 0x7fe46409ff70 2024-09-18T14:01:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 >> v1:172.21.15.84:6789/0 conn(0x7fe46409fba0 legacy=0x7fe464131ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 >> v1:172.21.15.45:6789/0 conn(0x7fe46409f7d0 legacy=0x7fe4641418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:01:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 monclient: found mon.e 2024-09-18T14:01:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe464142750 con 0x7fe46409ff70 2024-09-18T14:01:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient: _finish_auth 0 2024-09-18T14:01:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe46a79d640 5 monclient: authenticate success, global_id 4607 2024-09-18T14:01:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe46a79d640 10 monclient: _renew_subs 2024-09-18T14:01:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe46a79d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe46a79d640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe464121c00 con 0x7fe46409ff70 2024-09-18T14:01:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe46a79d640 10 monclient: _renew_subs 2024-09-18T14:01:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe46a79d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe46a79d640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4641200d0 con 0x7fe46409ff70 2024-09-18T14:01:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe45c004dd0 con 0x7fe46409ff70 2024-09-18T14:01:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe45c005340 con 0x7fe46409ff70 2024-09-18T14:01:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe4627fc640 4 set_mon_vals no callback set 2024-09-18T14:01:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.695+0000 7fe44bfff640 10 monclient: dump: 2024-09-18T14:01:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.727 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.696+0000 7fe46a79d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.696+0000 7fe46a79d640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe444006580 con 0x7fe444002830 2024-09-18T14:01:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.696+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fe45c010390 con 0x7fe46409ff70 2024-09-18T14:01:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.696+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(125..125 src has 1..125) v4 ==== 3895+0+0 (unknown 3491418578 0 0) 0x7fe45c046df0 con 0x7fe46409ff70 2024-09-18T14:01:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.696+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051874722 0 0) 0x7fe444006580 con 0x7fe444002830 2024-09-18T14:01:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.696+0000 7fe44bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.696+0000 7fe44bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.696+0000 7fe44bfff640 10 monclient(hunting): my global_id is 4573 2024-09-18T14:01:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.696+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe464121ee0 con 0x7fe444002830 2024-09-18T14:01:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.696+0000 7fe4637fe640 10 monclient: get_auth_request con 0x7fe45003a980 auth_method 0 2024-09-18T14:01:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1068454079 0 0) 0x7fe464121ee0 con 0x7fe444002830 2024-09-18T14:01:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe444006580 con 0x7fe444002830 2024-09-18T14:01:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe454002660 con 0x7fe444002830 2024-09-18T14:01:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:01:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 10 monclient: dump: 2024-09-18T14:01:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.730 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3684777488 0 0) 0x7fe444006580 con 0x7fe444002830 2024-09-18T14:01:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe44400a0a0 con 0x7fe444002830 2024-09-18T14:01:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe454002ad0 con 0x7fe444002830 2024-09-18T14:01:33.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:33.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:33.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 >> v1:172.21.15.84:6789/0 conn(0x7fe444002830 legacy=0x7fe444004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.697+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 >> v1:172.21.15.84:6789/0 conn(0x7fe444002830 legacy=0x7fe444004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:33.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.703+0000 7fe46a79d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:33.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.703+0000 7fe46a79d640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe4440095f0 con 0x7fe444008ff0 2024-09-18T14:01:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334120048 0 0) 0x7fe4440095f0 con 0x7fe444008ff0 2024-09-18T14:01:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 10 monclient(hunting): my global_id is 4582 2024-09-18T14:01:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe444006580 con 0x7fe444008ff0 2024-09-18T14:01:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1292649427 0 0) 0x7fe444006580 con 0x7fe444008ff0 2024-09-18T14:01:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe4440095f0 con 0x7fe444008ff0 2024-09-18T14:01:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe454002d60 con 0x7fe444008ff0 2024-09-18T14:01:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:01:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 10 monclient: dump: 2024-09-18T14:01:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.735 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.735 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.735 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.735 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3365006536 0 0) 0x7fe4440095f0 con 0x7fe444008ff0 2024-09-18T14:01:33.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe4440098b0 con 0x7fe444008ff0 2024-09-18T14:01:33.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 2492757208) 0x7fe454007cf0 con 0x7fe444008ff0 2024-09-18T14:01:33.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:33.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:33.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 >> v1:172.21.15.84:6789/0 conn(0x7fe444008ff0 legacy=0x7fe444008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.704+0000 7fe44bfff640 1 -- 172.21.15.45:0/2484904128 >> v1:172.21.15.84:6789/0 conn(0x7fe444008ff0 legacy=0x7fe444008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:33.736 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:33.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:33.736 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:33.736 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:33.736 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:33.736 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:33.736 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:33.737 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:33.737 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:33.737 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:33.737 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:33.737 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:33.737 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:33.737 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:33.737 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:33.737 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.737 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:01:33.738 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:33.738 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:33.738 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:33.738 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.738 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.738 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.738 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.738 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.738 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.739 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.739 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.739 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.739 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.739 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:33.739 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:33.739 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.739 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.739 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.740 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.740 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.740 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.740 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.740 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.740 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.740 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.740 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:33.740 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:33.741 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:33.741 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:33.741 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:33.741 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:33.741 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:33.741 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:33.741 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:33.741 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:33.741 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:33.741 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:33.741 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:33.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:33.742 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:33.742 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:33.742 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:33.742 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.742 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.742 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.742 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.742 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.742 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.743 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.743 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.743 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.743 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.743 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:33.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.743 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:33.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.743 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:33.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:33.744 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.744 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.744 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:33.744 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.744 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.744 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.745 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:33.745 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:33.745 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.745 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.745 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.745 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:33.745 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:33.745 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.745 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.746 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.746 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:33.746 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.746 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.746 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:33.746 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:33.746 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.746 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.746 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.747 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:33.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:33.747 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.747 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.747 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:33.747 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.748 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.748 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.748 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:33.748 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:33.748 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.748 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.748 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.748 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:33.749 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:33.749 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.749 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.749 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.749 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:33.749 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.749 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.749 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.749 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:33.750 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:33.750 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.750 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.750 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.750 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:33.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:33.750 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.750 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.751 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:33.751 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.751 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.751 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.751 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:33.751 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:33.751 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.751 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.751 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.752 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:33.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:33.752 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.752 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.752 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:33.752 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.753 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.753 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:33.753 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:33.753 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.753 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.753 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.753 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:33.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:33.754 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.754 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.754 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:33.754 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.754 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.755 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:33.755 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:33.755 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.755 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.755 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.755 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:33.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:33.755 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.755 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.756 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:33.756 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.756 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.756 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.756 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:33.756 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:33.756 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.756 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.757 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.757 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:33.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:33.757 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.757 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.757 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:33.757 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.758 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.758 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.758 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:33.758 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:33.758 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.758 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.758 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.758 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.758 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.758 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:33.759 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:33.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.759 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.759 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.759 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:33.759 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.759 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.759 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:33.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.759 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.759 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.759 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:33.760 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.760 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.760 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:33.760 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.705+0000 7fe46a79d640 1 -- 172.21.15.45:0/2484904128 >> v1:172.21.15.84:6812/31478 conn(0x7fe45003a980 legacy=0x7fe45003ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.705+0000 7fe46a79d640 10 monclient: shutdown 2024-09-18T14:01:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.705+0000 7fe46a79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.705+0000 7fe46a79d640 1 -- 172.21.15.45:0/2484904128 >> v1:172.21.15.45:6793/0 conn(0x7fe46409ff70 legacy=0x7fe464149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.705+0000 7fe468d3a640 1 -- 172.21.15.45:0/2484904128 reap_dead start 2024-09-18T14:01:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.705+0000 7fe468d3a640 1 -- 172.21.15.45:0/2484904128 reap_dead start 2024-09-18T14:01:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.705+0000 7fe46a79d640 1 -- 172.21.15.45:0/2484904128 shutdown_connections 2024-09-18T14:01:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.705+0000 7fe46a79d640 1 -- 172.21.15.45:0/2484904128 >> 172.21.15.45:0/2484904128 conn(0x7fe4640630b0 msgr2=0x7fe4640a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.705+0000 7fe46a79d640 1 -- 172.21.15.45:0/2484904128 shutdown_connections 2024-09-18T14:01:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.705+0000 7fe46a79d640 1 -- 172.21.15.45:0/2484904128 wait complete. 2024-09-18T14:01:33.766 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:33.766 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:01:33.767 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:01:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.863+0000 7f9d6935d640 10 monclient: get_monmap_and_config 2024-09-18T14:01:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.863+0000 7f9d6935d640 10 monclient: build_initial_monmap 2024-09-18T14:01:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.863+0000 7f9d6935d640 10 monclient: monmap: 2024-09-18T14:01:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:33.864212+0000 2024-09-18T14:01:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:33.864212+0000 2024-09-18T14:01:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:33.866 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.863+0000 7f9d6935d640 1 Processor -- start 2024-09-18T14:01:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.863+0000 7f9d6935d640 1 -- start start 2024-09-18T14:01:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.863+0000 7f9d6935d640 10 monclient: init 2024-09-18T14:01:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d6935d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d6935d640 10 monclient: _add_conns ranks=[5,2,3,6,4,7,8,1,0] 2024-09-18T14:01:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d6935d640 10 monclient(hunting): picked mon.noname-h con 0x7f9d6406e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d6935d640 10 monclient(hunting): picked mon.noname-b con 0x7f9d640a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d6935d640 10 monclient(hunting): picked mon.noname-g con 0x7f9d640a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:01:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d6935d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6405ed70 con 0x7f9d640a0cb0 2024-09-18T14:01:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d6935d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d64097620 con 0x7f9d640a1080 2024-09-18T14:01:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d6935d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d64097340 con 0x7f9d6406e810 2024-09-18T14:01:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d6935d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d6935d640 10 monclient(hunting): authenticate will time out at 909.202545s 2024-09-18T14:01:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d627fc640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f9d640a0cb0 0x7f9d64137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43716/0 (socket says 172.21.15.45:43716) 2024-09-18T14:01:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d627fc640 1 -- 172.21.15.45:0/3553957793 learned_addr learned my addr 172.21.15.45:0/3553957793 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d61ffb640 1 -- 172.21.15.45:0/3553957793 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472479466 0 0) 0x7f9d6405ed70 con 0x7f9d640a0cb0 2024-09-18T14:01:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d61ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d61ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d61ffb640 10 monclient(hunting): my global_id is 4664 2024-09-18T14:01:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.864+0000 7f9d61ffb640 1 -- 172.21.15.45:0/3553957793 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d500034e0 con 0x7f9d640a0cb0 2024-09-18T14:01:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 1 -- 172.21.15.45:0/3553957793 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1499015487 0 0) 0x7f9d500034e0 con 0x7f9d640a0cb0 2024-09-18T14:01:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 1 -- 172.21.15.45:0/3553957793 >> v1:172.21.15.84:6791/0 conn(0x7f9d6406e810 legacy=0x7f9d640a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 1 -- 172.21.15.45:0/3553957793 >> v1:172.21.15.84:6790/0 conn(0x7f9d640a1080 legacy=0x7f9d6413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:01:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 1 monclient: found mon.noname-b 2024-09-18T14:01:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:01:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 1 -- 172.21.15.45:0/3553957793 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d640a1660 con 0x7f9d640a0cb0 2024-09-18T14:01:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 10 monclient: _finish_auth 0 2024-09-18T14:01:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 1 -- 172.21.15.45:0/3553957793 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d4c001830 con 0x7f9d640a0cb0 2024-09-18T14:01:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d6935d640 5 monclient: authenticate success, global_id 4664 2024-09-18T14:01:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d6935d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:01:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 10 monclient: dump: 2024-09-18T14:01:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.883 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d6935d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 1 -- 172.21.15.45:0/3553957793 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9d4c001a10 con 0x7f9d640a0cb0 2024-09-18T14:01:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 1 -- 172.21.15.45:0/3553957793 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d4c004ba0 con 0x7f9d640a0cb0 2024-09-18T14:01:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d6935d640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d6935d640 4 set_mon_vals no callback set 2024-09-18T14:01:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d61ffb640 10 monclient: dump: 2024-09-18T14:01:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.885 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d6935d640 10 monclient: shutdown 2024-09-18T14:01:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d6935d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d6935d640 1 -- 172.21.15.45:0/3553957793 >> v1:172.21.15.45:6790/0 conn(0x7f9d640a0cb0 legacy=0x7f9d64137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d6935d640 1 -- 172.21.15.45:0/3553957793 shutdown_connections 2024-09-18T14:01:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d6935d640 1 -- 172.21.15.45:0/3553957793 >> 172.21.15.45:0/3553957793 conn(0x7f9d640a5bb0 msgr2=0x7f9d640a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d6935d640 1 -- 172.21.15.45:0/3553957793 shutdown_connections 2024-09-18T14:01:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.865+0000 7f9d6935d640 1 -- 172.21.15.45:0/3553957793 wait complete. 2024-09-18T14:01:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.866+0000 7f9d6935d640 10 monclient: build_initial_monmap 2024-09-18T14:01:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.866+0000 7f9d6935d640 10 monclient: monmap: 2024-09-18T14:01:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:33.887 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.866+0000 7f9d6935d640 1 Processor -- start 2024-09-18T14:01:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.866+0000 7f9d6935d640 1 -- start start 2024-09-18T14:01:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.866+0000 7f9d6935d640 10 monclient: init 2024-09-18T14:01:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d6935d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d6935d640 10 monclient: _add_conns ranks=[4,1,2,6,5,7,8,3,0] 2024-09-18T14:01:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d6935d640 10 monclient(hunting): picked mon.noname-c con 0x7f9d6406e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d6935d640 10 monclient(hunting): picked mon.noname-f con 0x7f9d640a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d6935d640 10 monclient(hunting): picked mon.noname-b con 0x7f9d640a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d6935d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6409fa10 con 0x7f9d640a0cb0 2024-09-18T14:01:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d6935d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d640a2270 con 0x7f9d640a1080 2024-09-18T14:01:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d6935d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d641207f0 con 0x7f9d6406e810 2024-09-18T14:01:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d6935d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d6935d640 10 monclient(hunting): authenticate will time out at 909.205372s 2024-09-18T14:01:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d637fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f9d640a1080 0x7f9d6411dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43722/0 (socket says 172.21.15.45:43722) 2024-09-18T14:01:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d637fe640 1 -- 172.21.15.45:0/519753993 learned_addr learned my addr 172.21.15.45:0/519753993 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d62ffd640 1 --1- 172.21.15.45:0/519753993 >> v1:172.21.15.45:6791/0 conn(0x7f9d6406e810 0x7f9d6411cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36928/0 (socket says 172.21.15.45:36928) 2024-09-18T14:01:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466748615 0 0) 0x7f9d640a2270 con 0x7f9d640a1080 2024-09-18T14:01:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 10 monclient(hunting): my global_id is 4673 2024-09-18T14:01:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d440034e0 con 0x7f9d640a1080 2024-09-18T14:01:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907030567 0 0) 0x7f9d641207f0 con 0x7f9d6406e810 2024-09-18T14:01:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 10 monclient(hunting): my global_id is 134194 2024-09-18T14:01:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d640a2270 con 0x7f9d6406e810 2024-09-18T14:01:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704142992 0 0) 0x7f9d6409fa10 con 0x7f9d640a0cb0 2024-09-18T14:01:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 10 monclient(hunting): my global_id is 4591 2024-09-18T14:01:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d641207f0 con 0x7f9d640a0cb0 2024-09-18T14:01:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4280557262 0 0) 0x7f9d640a2270 con 0x7f9d6406e810 2024-09-18T14:01:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d6409fa10 con 0x7f9d6406e810 2024-09-18T14:01:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2294878454 0 0) 0x7f9d440034e0 con 0x7f9d640a1080 2024-09-18T14:01:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.867+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d640a2270 con 0x7f9d640a1080 2024-09-18T14:01:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d54001a60 con 0x7f9d6406e810 2024-09-18T14:01:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:01:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient(hunting): dump: 2024-09-18T14:01:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.892 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4167196036 0 0) 0x7f9d641207f0 con 0x7f9d640a0cb0 2024-09-18T14:01:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d440034e0 con 0x7f9d640a0cb0 2024-09-18T14:01:33.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d5c001d40 con 0x7f9d640a1080 2024-09-18T14:01:33.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:33.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient(hunting): dump: 2024-09-18T14:01:33.893 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.893 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.893 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.893 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.893 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.893 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.893 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.894 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d4c0017f0 con 0x7f9d640a0cb0 2024-09-18T14:01:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:01:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient(hunting): dump: 2024-09-18T14:01:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.897 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 53542525 0 0) 0x7f9d6409fa10 con 0x7f9d6406e810 2024-09-18T14:01:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 >> v1:172.21.15.45:6790/0 conn(0x7f9d640a1080 legacy=0x7f9d6411dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 >> v1:172.21.15.84:6789/0 conn(0x7f9d640a0cb0 legacy=0x7f9d6411d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient: _finish_hunting 0 2024-09-18T14:01:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 1 monclient: found mon.c 2024-09-18T14:01:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d641325c0 con 0x7f9d6406e810 2024-09-18T14:01:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient: _finish_auth 0 2024-09-18T14:01:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d6935d640 5 monclient: authenticate success, global_id 134194 2024-09-18T14:01:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d6935d640 10 monclient: _renew_subs 2024-09-18T14:01:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d6935d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d6935d640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d64120510 con 0x7f9d6406e810 2024-09-18T14:01:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d6935d640 10 monclient: _renew_subs 2024-09-18T14:01:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d6935d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d6935d640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6411f620 con 0x7f9d6406e810 2024-09-18T14:01:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9d54001bc0 con 0x7f9d6406e810 2024-09-18T14:01:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:33.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d54001da0 con 0x7f9d6406e810 2024-09-18T14:01:33.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d617fa640 4 set_mon_vals no callback set 2024-09-18T14:01:33.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:33.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.868+0000 7f9d5affd640 10 monclient: dump: 2024-09-18T14:01:33.899 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.899 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.899 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.899 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.899 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.899 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.899 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.900 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.900 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.900 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.900 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.900 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.900 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.900 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.900 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.900 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d6935d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:33.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f9d540025f0 con 0x7f9d6406e810 2024-09-18T14:01:33.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d6935d640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9d48006580 con 0x7f9d48002830 2024-09-18T14:01:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(125..125 src has 1..125) v4 ==== 3895+0+0 (unknown 3491418578 0 0) 0x7f9d54046ad0 con 0x7f9d6406e810 2024-09-18T14:01:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d637fe640 10 monclient: get_auth_request con 0x7f9d4403a9f0 auth_method 0 2024-09-18T14:01:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42495680 0 0) 0x7f9d48006580 con 0x7f9d48002830 2024-09-18T14:01:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 10 monclient(hunting): my global_id is 4701 2024-09-18T14:01:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d6409fa10 con 0x7f9d48002830 2024-09-18T14:01:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 412543030 0 0) 0x7f9d6409fa10 con 0x7f9d48002830 2024-09-18T14:01:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d48006580 con 0x7f9d48002830 2024-09-18T14:01:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d4c0010a0 con 0x7f9d48002830 2024-09-18T14:01:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:01:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 10 monclient: dump: 2024-09-18T14:01:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.903 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 864931514 0 0) 0x7f9d48006580 con 0x7f9d48002830 2024-09-18T14:01:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.869+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d4800a0a0 con 0x7f9d48002830 2024-09-18T14:01:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.870+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9d4c005160 con 0x7f9d48002830 2024-09-18T14:01:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.870+0000 7f9d5affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.870+0000 7f9d5affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.870+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 >> v1:172.21.15.84:6790/0 conn(0x7f9d48002830 legacy=0x7f9d48004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.870+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 >> v1:172.21.15.84:6790/0 conn(0x7f9d48002830 legacy=0x7f9d48004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d6935d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d6935d640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9d48009660 con 0x7f9d48009040 2024-09-18T14:01:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489049172 0 0) 0x7f9d48009660 con 0x7f9d48009040 2024-09-18T14:01:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 10 monclient(hunting): my global_id is 4710 2024-09-18T14:01:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d48006580 con 0x7f9d48009040 2024-09-18T14:01:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 5961914 0 0) 0x7f9d48006580 con 0x7f9d48009040 2024-09-18T14:01:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d48009660 con 0x7f9d48009040 2024-09-18T14:01:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d4c005960 con 0x7f9d48009040 2024-09-18T14:01:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:01:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 10 monclient: dump: 2024-09-18T14:01:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:33.906 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2354195456 0 0) 0x7f9d48009660 con 0x7f9d48009040 2024-09-18T14:01:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.876+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9d48009920 con 0x7f9d48009040 2024-09-18T14:01:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7726 (unknown 0 0 2397916006) 0x7f9d4c008150 con 0x7f9d48009040 2024-09-18T14:01:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d5affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:33.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d5affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:33.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 >> v1:172.21.15.84:6790/0 conn(0x7f9d48009040 legacy=0x7f9d48008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d5affd640 1 -- 172.21.15.45:0/519753993 >> v1:172.21.15.84:6790/0 conn(0x7f9d48009040 legacy=0x7f9d48008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:33.907 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:33.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:33.908 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:33.908 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:33.908 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:33.908 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:33.908 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:33.908 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:33.908 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:33.908 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:33.908 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:33.908 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:33.908 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:33.909 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:33.909 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:33.909 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.909 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:01:33.909 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:33.909 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:33.909 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:33.909 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.909 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.909 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.909 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.911 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.911 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.911 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.911 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.911 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.911 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.911 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:33.911 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:33.911 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:33.911 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:33.912 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:33.912 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:33.912 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:33.912 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:33.912 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:33.912 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:33.912 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:33.913 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:33.913 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:33.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:33.913 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:33.913 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:33.913 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:33.913 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:33.913 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:33.913 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:33.913 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:33.914 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:33.914 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:33.914 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:33.914 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:33.914 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:33.914 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.914 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:33.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.914 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:33.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.914 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:33.915 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:33.915 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.915 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.915 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:33.915 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.915 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.915 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.916 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:33.916 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:33.916 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.916 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.916 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.916 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:33.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:33.916 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:33.917 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.917 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.917 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:33.918 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:33.918 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.918 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.918 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.918 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:33.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:33.918 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:33.919 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.919 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:33.919 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:33.919 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.919 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.920 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.920 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:33.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:33.920 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.920 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.921 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:33.921 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.921 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.921 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:33.921 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:33.921 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.921 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.921 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.922 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:33.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:33.922 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.922 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.922 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:33.922 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.922 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.922 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.923 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:33.923 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:33.923 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.923 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.923 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.923 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:33.923 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:33.923 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.923 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.924 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:33.924 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.924 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.924 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.924 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:33.924 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:33.924 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.925 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.925 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.925 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:33.925 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:33.925 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.925 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.926 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:33.926 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.926 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.926 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.926 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:33.926 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:33.926 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.927 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.927 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.927 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:33.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:33.927 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.927 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.928 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:33.928 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.928 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.928 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:33.928 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:33.928 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.928 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.929 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.929 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:33.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:33.929 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:33.929 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:33.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:33.929 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:33.930 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:33.930 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.930 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.930 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:33.930 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:33.930 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:33.930 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:33.930 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:33.930 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.931 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.931 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:33.931 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:33.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.931 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.931 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.931 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:33.931 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.931 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.932 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:01:33.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.932 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.932 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.932 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:33.932 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.932 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.932 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:33.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.932 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.932 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.933 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:01:33.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.933 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:33.933 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:01:33.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:33.933 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:33.933 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:33.933 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:33.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:33.933 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:33.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:33.934 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:33.934 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:33.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d6935d640 1 -- 172.21.15.45:0/519753993 >> v1:172.21.15.84:6812/31478 conn(0x7f9d4403a9f0 legacy=0x7f9d4403cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d6935d640 10 monclient: shutdown 2024-09-18T14:01:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d6935d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d6935d640 1 -- 172.21.15.45:0/519753993 >> v1:172.21.15.45:6791/0 conn(0x7f9d6406e810 legacy=0x7f9d6411cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d637fe640 1 -- 172.21.15.45:0/519753993 reap_dead start 2024-09-18T14:01:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d637fe640 1 -- 172.21.15.45:0/519753993 reap_dead start 2024-09-18T14:01:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d6935d640 1 -- 172.21.15.45:0/519753993 shutdown_connections 2024-09-18T14:01:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d6935d640 1 -- 172.21.15.45:0/519753993 >> 172.21.15.45:0/519753993 conn(0x7f9d640a5bb0 msgr2=0x7f9d64132d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d6935d640 1 -- 172.21.15.45:0/519753993 shutdown_connections 2024-09-18T14:01:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:33.877+0000 7f9d6935d640 1 -- 172.21.15.45:0/519753993 wait complete. 2024-09-18T14:01:33.937 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:33.937 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:01:33.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:01:34.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.034+0000 7fbc89417640 10 monclient: get_monmap_and_config 2024-09-18T14:01:34.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.034+0000 7fbc89417640 10 monclient: build_initial_monmap 2024-09-18T14:01:34.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.034+0000 7fbc89417640 10 monclient: monmap: 2024-09-18T14:01:34.035 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:34.035 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.035 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:34.035415+0000 2024-09-18T14:01:34.035 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:34.035415+0000 2024-09-18T14:01:34.035 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:34.035 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.035 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:34.035 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:34.035 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:34.036 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 1 Processor -- start 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 1 -- start start 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 10 monclient: init 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 10 monclient: _add_conns ranks=[2,4,3,1,7,6,0,5,8] 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 10 monclient(hunting): picked mon.noname-b con 0x7fbc8409f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 10 monclient(hunting): picked mon.noname-c con 0x7fbc8409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 10 monclient(hunting): picked mon.noname-g con 0x7fbc8409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8405ed70 con 0x7fbc8409f7d0 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8409dfc0 con 0x7fbc8409ff70 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8409dce0 con 0x7fbc8409fba0 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc89417640 10 monclient(hunting): authenticate will time out at 909.373622s 2024-09-18T14:01:34.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc82ffd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fbc8409f7d0 0x7fbc84132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43734/0 (socket says 172.21.15.45:43734) 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc82ffd640 1 -- 172.21.15.45:0/3424363345 learned_addr learned my addr 172.21.15.45:0/3424363345 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.035+0000 7fbc827fc640 1 --1- 172.21.15.45:0/3424363345 >> v1:172.21.15.45:6791/0 conn(0x7fbc8409fba0 0x7fbc84137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36934/0 (socket says 172.21.15.45:36934) 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153651879 0 0) 0x7fbc8409dce0 con 0x7fbc8409fba0 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient(hunting): my global_id is 134203 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc700034e0 con 0x7fbc8409fba0 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055592261 0 0) 0x7fbc8405ed70 con 0x7fbc8409f7d0 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient(hunting): my global_id is 4682 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc8409dce0 con 0x7fbc8409f7d0 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419448909 0 0) 0x7fbc8409dfc0 con 0x7fbc8409ff70 2024-09-18T14:01:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient(hunting): my global_id is 4719 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc8405ed70 con 0x7fbc8409ff70 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1558181013 0 0) 0x7fbc700034e0 con 0x7fbc8409fba0 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 >> v1:172.21.15.84:6790/0 conn(0x7fbc8409ff70 legacy=0x7fbc8413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 >> v1:172.21.15.45:6790/0 conn(0x7fbc8409f7d0 legacy=0x7fbc84132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 monclient: found mon.noname-c 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc840a1c00 con 0x7fbc8409fba0 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient: _finish_auth 0 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc6c001830 con 0x7fbc8409fba0 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc89417640 5 monclient: authenticate success, global_id 134203 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc89417640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient: dump: 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc89417640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbc6c001a10 con 0x7fbc8409fba0 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc89417640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 1 -- 172.21.15.45:0/3424363345 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc6c004ae0 con 0x7fbc8409fba0 2024-09-18T14:01:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc89417640 4 set_mon_vals no callback set 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc81ffb640 10 monclient: dump: 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc89417640 10 monclient: shutdown 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc89417640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.036+0000 7fbc89417640 1 -- 172.21.15.45:0/3424363345 >> v1:172.21.15.45:6791/0 conn(0x7fbc8409fba0 legacy=0x7fbc84137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.037+0000 7fbc89417640 1 -- 172.21.15.45:0/3424363345 shutdown_connections 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.037+0000 7fbc89417640 1 -- 172.21.15.45:0/3424363345 >> 172.21.15.45:0/3424363345 conn(0x7fbc840630b0 msgr2=0x7fbc840634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.037+0000 7fbc89417640 1 -- 172.21.15.45:0/3424363345 shutdown_connections 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.037+0000 7fbc89417640 1 -- 172.21.15.45:0/3424363345 wait complete. 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.037+0000 7fbc89417640 10 monclient: build_initial_monmap 2024-09-18T14:01:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.037+0000 7fbc89417640 10 monclient: monmap: 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 1 Processor -- start 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 1 -- start start 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 10 monclient: init 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 10 monclient: _add_conns ranks=[5,8,4,7,3,6,2,0,1] 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 10 monclient(hunting): picked mon.noname-h con 0x7fbc8409f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 10 monclient(hunting): picked mon.noname-e con 0x7fbc8409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 10 monclient(hunting): picked mon.noname-c con 0x7fbc8409ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8409d310 con 0x7fbc8409ff70 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc840a2560 con 0x7fbc8409f7d0 2024-09-18T14:01:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8409be50 con 0x7fbc8409fba0 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc89417640 10 monclient(hunting): authenticate will time out at 909.376632s 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc837fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fbc8409ff70 0x7fbc8414a470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36946/0 (socket says 172.21.15.45:36946) 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc837fe640 1 -- 172.21.15.45:0/1217073789 learned_addr learned my addr 172.21.15.45:0/1217073789 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983288718 0 0) 0x7fbc8409d310 con 0x7fbc8409ff70 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc7affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc7affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc7affd640 10 monclient(hunting): my global_id is 134212 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc640034e0 con 0x7fbc8409ff70 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491962660 0 0) 0x7fbc8409be50 con 0x7fbc8409fba0 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc7affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc7affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc7affd640 10 monclient(hunting): my global_id is 4616 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc8409d310 con 0x7fbc8409fba0 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.038+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518029362 0 0) 0x7fbc840a2560 con 0x7fbc8409f7d0 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): my global_id is 4541 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc8409be50 con 0x7fbc8409f7d0 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3839252058 0 0) 0x7fbc8409d310 con 0x7fbc8409fba0 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbc840a2560 con 0x7fbc8409fba0 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc6c004b20 con 0x7fbc8409fba0 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): dump: 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4259293713 0 0) 0x7fbc640034e0 con 0x7fbc8409ff70 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbc8409d310 con 0x7fbc8409ff70 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc7c002a50 con 0x7fbc8409ff70 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): dump: 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 556113735 0 0) 0x7fbc8409be50 con 0x7fbc8409f7d0 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbc640034e0 con 0x7fbc8409f7d0 2024-09-18T14:01:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc740018d0 con 0x7fbc8409f7d0 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient(hunting): dump: 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2074356213 0 0) 0x7fbc840a2560 con 0x7fbc8409fba0 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 >> v1:172.21.15.84:6791/0 conn(0x7fbc8409f7d0 legacy=0x7fbc84131720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 >> v1:172.21.15.45:6791/0 conn(0x7fbc8409ff70 legacy=0x7fbc8414a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient: _finish_hunting 0 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 monclient: found mon.e 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8409b940 con 0x7fbc8409fba0 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient: _finish_auth 0 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc89417640 5 monclient: authenticate success, global_id 4616 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc89417640 10 monclient: _renew_subs 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc89417640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc89417640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc84136ea0 con 0x7fbc8409fba0 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc89417640 10 monclient: _renew_subs 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc89417640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc89417640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8411d8e0 con 0x7fbc8409fba0 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbc6c0017e0 con 0x7fbc8409fba0 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc6c0050e0 con 0x7fbc8409fba0 2024-09-18T14:01:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc817fa640 4 set_mon_vals no callback set 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.039+0000 7fbc7affd640 10 monclient: dump: 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.040+0000 7fbc89417640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.040+0000 7fbc89417640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbc68006580 con 0x7fbc68002830 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.040+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fbc6c0100b0 con 0x7fbc8409fba0 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.040+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(125..125 src has 1..125) v4 ==== 3895+0+0 (unknown 3491418578 0 0) 0x7fbc6c046af0 con 0x7fbc8409fba0 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.040+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749247660 0 0) 0x7fbc68006580 con 0x7fbc68002830 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.040+0000 7fbc7affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.040+0000 7fbc7affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.040+0000 7fbc7affd640 10 monclient(hunting): my global_id is 4550 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.040+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc840a2560 con 0x7fbc68002830 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.040+0000 7fbc837fe640 10 monclient: get_auth_request con 0x7fbc6403a9e0 auth_method 0 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 954252387 0 0) 0x7fbc840a2560 con 0x7fbc68002830 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbc68006580 con 0x7fbc68002830 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc74004e60 con 0x7fbc68002830 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 10 monclient: dump: 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 340804158 0 0) 0x7fbc68006580 con 0x7fbc68002830 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbc6800a0a0 con 0x7fbc68002830 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fbc740070e0 con 0x7fbc68002830 2024-09-18T14:01:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 >> v1:172.21.15.84:6791/0 conn(0x7fbc68002830 legacy=0x7fbc68004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.041+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 >> v1:172.21.15.84:6791/0 conn(0x7fbc68002830 legacy=0x7fbc68004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc89417640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc89417640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbc68009610 con 0x7fbc68008ff0 2024-09-18T14:01:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443856956 0 0) 0x7fbc68009610 con 0x7fbc68008ff0 2024-09-18T14:01:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc7affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc7affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc7affd640 10 monclient(hunting): my global_id is 4559 2024-09-18T14:01:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc68006580 con 0x7fbc68008ff0 2024-09-18T14:01:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3955418581 0 0) 0x7fbc68006580 con 0x7fbc68008ff0 2024-09-18T14:01:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbc68009610 con 0x7fbc68008ff0 2024-09-18T14:01:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbc74005920 con 0x7fbc68008ff0 2024-09-18T14:01:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc7affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:01:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.047+0000 7fbc7affd640 10 monclient: dump: 2024-09-18T14:01:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.070 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2673315883 0 0) 0x7fbc68009610 con 0x7fbc68008ff0 2024-09-18T14:01:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbc680098d0 con 0x7fbc68008ff0 2024-09-18T14:01:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 3911711045) 0x7fbc74006cf0 con 0x7fbc68008ff0 2024-09-18T14:01:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc7affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc7affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 >> v1:172.21.15.84:6791/0 conn(0x7fbc68008ff0 legacy=0x7fbc68008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc7affd640 1 -- 172.21.15.45:0/1217073789 >> v1:172.21.15.84:6791/0 conn(0x7fbc68008ff0 legacy=0x7fbc68008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:34.070 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:34.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:34.071 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:34.071 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:34.071 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:34.071 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:34.071 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:34.071 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:34.071 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:34.071 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:34.071 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:34.071 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:34.071 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:34.072 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:34.073 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:34.073 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:34.073 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:34.073 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:34.073 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:34.073 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:34.073 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:34.073 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:34.073 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:34.073 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:34.073 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:34.073 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:34.074 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:34.075 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.077 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.079 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.081 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.082 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.083 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.084 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:34.086 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.087 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.089 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:34.090 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:34.090 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:34.090 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.090 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.090 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:34.090 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc89417640 1 -- 172.21.15.45:0/1217073789 >> v1:172.21.15.84:6812/31478 conn(0x7fbc6403a9e0 legacy=0x7fbc6403ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc89417640 10 monclient: shutdown 2024-09-18T14:01:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc89417640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc89417640 1 -- 172.21.15.45:0/1217073789 >> v1:172.21.15.45:6793/0 conn(0x7fbc8409fba0 legacy=0x7fbc84146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc837fe640 1 -- 172.21.15.45:0/1217073789 reap_dead start 2024-09-18T14:01:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc837fe640 1 -- 172.21.15.45:0/1217073789 reap_dead start 2024-09-18T14:01:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc89417640 1 -- 172.21.15.45:0/1217073789 shutdown_connections 2024-09-18T14:01:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc89417640 1 -- 172.21.15.45:0/1217073789 >> 172.21.15.45:0/1217073789 conn(0x7fbc840630b0 msgr2=0x7fbc840a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc89417640 1 -- 172.21.15.45:0/1217073789 shutdown_connections 2024-09-18T14:01:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.048+0000 7fbc89417640 1 -- 172.21.15.45:0/1217073789 wait complete. 2024-09-18T14:01:34.092 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:34.092 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:01:34.093 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:01:34.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.191+0000 7f512fc34640 10 monclient: get_monmap_and_config 2024-09-18T14:01:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.191+0000 7f512fc34640 10 monclient: build_initial_monmap 2024-09-18T14:01:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.191+0000 7f512fc34640 10 monclient: monmap: 2024-09-18T14:01:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:34.192185+0000 2024-09-18T14:01:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:34.192185+0000 2024-09-18T14:01:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:34.192 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:34.193 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:34.193 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.191+0000 7f512fc34640 1 Processor -- start 2024-09-18T14:01:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512fc34640 1 -- start start 2024-09-18T14:01:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512fc34640 10 monclient: init 2024-09-18T14:01:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512fc34640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512fc34640 10 monclient: _add_conns ranks=[4,3,6,2,5,0,7,8,1] 2024-09-18T14:01:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512fc34640 10 monclient(hunting): picked mon.noname-c con 0x7f512806e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512fc34640 10 monclient(hunting): picked mon.noname-g con 0x7f51280a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:01:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512fc34640 10 monclient(hunting): picked mon.noname-d con 0x7f51280a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:01:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512fc34640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512805ed70 con 0x7f51280a0cb0 2024-09-18T14:01:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512fc34640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5128097620 con 0x7f512806e810 2024-09-18T14:01:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512fc34640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5128097340 con 0x7f51280a1080 2024-09-18T14:01:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512fc34640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512fc34640 10 monclient(hunting): authenticate will time out at 909.530539s 2024-09-18T14:01:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512d9d0640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f512806e810 0x7f51280a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36958/0 (socket says 172.21.15.45:36958) 2024-09-18T14:01:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512d9d0640 1 -- 172.21.15.45:0/3970056963 learned_addr learned my addr 172.21.15.45:0/3970056963 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803497076 0 0) 0x7f5128097340 con 0x7f51280a1080 2024-09-18T14:01:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512c9ce640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512c9ce640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512c9ce640 10 monclient(hunting): my global_id is 4578 2024-09-18T14:01:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51140034e0 con 0x7f51280a1080 2024-09-18T14:01:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3068442987 0 0) 0x7f5128097620 con 0x7f512806e810 2024-09-18T14:01:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512c9ce640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512c9ce640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512c9ce640 10 monclient(hunting): my global_id is 134221 2024-09-18T14:01:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.192+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5128097340 con 0x7f512806e810 2024-09-18T14:01:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881041694 0 0) 0x7f512805ed70 con 0x7f51280a0cb0 2024-09-18T14:01:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient(hunting): my global_id is 4728 2024-09-18T14:01:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5128097620 con 0x7f51280a0cb0 2024-09-18T14:01:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1313395823 0 0) 0x7f51140034e0 con 0x7f51280a1080 2024-09-18T14:01:34.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 >> v1:172.21.15.45:6791/0 conn(0x7f512806e810 legacy=0x7f51280a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 >> v1:172.21.15.84:6790/0 conn(0x7f51280a0cb0 legacy=0x7f5128137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient: _finish_hunting 0 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 1 monclient: found mon.noname-d 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51280a1660 con 0x7f51280a1080 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient: _finish_auth 0 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512fc34640 5 monclient: authenticate success, global_id 4578 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5120001bb0 con 0x7f51280a1080 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:01:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient: dump: 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512fc34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5120001d10 con 0x7f51280a1080 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 1 -- 172.21.15.45:0/3970056963 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5120002240 con 0x7f51280a1080 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512c9ce640 10 monclient: dump: 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512fc34640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512fc34640 4 set_mon_vals no callback set 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512fc34640 10 monclient: shutdown 2024-09-18T14:01:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512fc34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512fc34640 1 -- 172.21.15.45:0/3970056963 >> v1:172.21.15.45:6792/0 conn(0x7f51280a1080 legacy=0x7f512813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512fc34640 1 -- 172.21.15.45:0/3970056963 shutdown_connections 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512fc34640 1 -- 172.21.15.45:0/3970056963 >> 172.21.15.45:0/3970056963 conn(0x7f51280a5bb0 msgr2=0x7f51280a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512fc34640 1 -- 172.21.15.45:0/3970056963 shutdown_connections 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.193+0000 7f512fc34640 1 -- 172.21.15.45:0/3970056963 wait complete. 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.194+0000 7f512fc34640 10 monclient: build_initial_monmap 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.194+0000 7f512fc34640 10 monclient: monmap: 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.194+0000 7f512fc34640 1 Processor -- start 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.194+0000 7f512fc34640 1 -- start start 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.194+0000 7f512fc34640 10 monclient: init 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512fc34640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512fc34640 10 monclient: _add_conns ranks=[8,6,4,1,2,5,0,7,3] 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512fc34640 10 monclient(hunting): picked mon.noname-e con 0x7f512806e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512fc34640 10 monclient(hunting): picked mon.noname-d con 0x7f51280a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512fc34640 10 monclient(hunting): picked mon.noname-c con 0x7f51280a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512fc34640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5128122dc0 con 0x7f51280a1080 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512fc34640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512813dbb0 con 0x7f51280a0cb0 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512fc34640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5128122800 con 0x7f512806e810 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512fc34640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512fc34640 10 monclient(hunting): authenticate will time out at 909.533269s 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512d1cf640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f51280a0cb0 0x7f51281466d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56128/0 (socket says 172.21.15.45:56128) 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512d1cf640 1 -- 172.21.15.45:0/1813917243 learned_addr learned my addr 172.21.15.45:0/1813917243 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512d9d0640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f512806e810 0x7f512813d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:60998/0 (socket says 172.21.15.45:60998) 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f512e1d1640 1 --1- 172.21.15.45:0/1813917243 >> v1:172.21.15.45:6791/0 conn(0x7f51280a1080 0x7f5128149e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36960/0 (socket says 172.21.15.45:36960) 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608299740 0 0) 0x7f5128122dc0 con 0x7f51280a1080 2024-09-18T14:01:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): my global_id is 134230 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f510c0034e0 con 0x7f51280a1080 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211305969 0 0) 0x7f512813dbb0 con 0x7f51280a0cb0 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): my global_id is 4587 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5128122dc0 con 0x7f51280a0cb0 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224094554 0 0) 0x7f5128122800 con 0x7f512806e810 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): my global_id is 4625 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f512813dbb0 con 0x7f512806e810 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3552514937 0 0) 0x7f510c0034e0 con 0x7f51280a1080 2024-09-18T14:01:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5128122800 con 0x7f51280a1080 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f51200021c0 con 0x7f51280a1080 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): dump: 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.213 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.213 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.213 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.213 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.213 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3198032395 0 0) 0x7f512813dbb0 con 0x7f512806e810 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f510c0034e0 con 0x7f512806e810 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3936432448 0 0) 0x7f5128122dc0 con 0x7f51280a0cb0 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.195+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f512813dbb0 con 0x7f51280a0cb0 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5124001480 con 0x7f512806e810 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient(hunting): dump: 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5118001aa0 con 0x7f51280a0cb0 2024-09-18T14:01:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:01:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient(hunting): dump: 2024-09-18T14:01:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.217 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 794263331 0 0) 0x7f5128122800 con 0x7f51280a1080 2024-09-18T14:01:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 >> v1:172.21.15.45:6793/0 conn(0x7f512806e810 legacy=0x7f512813d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 >> v1:172.21.15.45:6792/0 conn(0x7f51280a0cb0 legacy=0x7f51281466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient: _finish_hunting 0 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 1 monclient: found mon.c 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5128120310 con 0x7f51280a1080 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient: _finish_auth 0 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f512fc34640 5 monclient: authenticate success, global_id 134230 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f512fc34640 10 monclient: _renew_subs 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f512fc34640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f512fc34640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5128122520 con 0x7f51280a1080 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5120002650 con 0x7f51280a1080 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5120001490 con 0x7f51280a1080 2024-09-18T14:01:34.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511dffb640 10 monclient: dump: 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f512fc34640 10 monclient: _renew_subs 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f511ffff640 4 set_mon_vals no callback set 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f512fc34640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.196+0000 7f512fc34640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5128121650 con 0x7f51280a1080 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.197+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f5120010550 con 0x7f51280a1080 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.197+0000 7f512fc34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.197+0000 7f512fc34640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f512804c4a0 con 0x7f512813e750 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.197+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(125..125 src has 1..125) v4 ==== 3895+0+0 (unknown 3491418578 0 0) 0x7f5120046e40 con 0x7f51280a1080 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.197+0000 7f512d1cf640 10 monclient: get_auth_request con 0x7f510c03ab20 auth_method 0 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.197+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295025071 0 0) 0x7f512804c4a0 con 0x7f512813e750 2024-09-18T14:01:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.197+0000 7f511dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.197+0000 7f511dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.197+0000 7f511dffb640 10 monclient(hunting): my global_id is 4642 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.197+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5128122800 con 0x7f512813e750 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2602601635 0 0) 0x7f5128122800 con 0x7f512813e750 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f512804c4a0 con 0x7f512813e750 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5124004ae0 con 0x7f512813e750 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 10 monclient: dump: 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3339848137 0 0) 0x7f512804c4a0 con 0x7f512813e750 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f512804b720 con 0x7f512813e750 2024-09-18T14:01:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5124006fc0 con 0x7f512813e750 2024-09-18T14:01:34.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:34.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:34.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 >> v1:172.21.15.84:6792/0 conn(0x7f512813e750 legacy=0x7f512809bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.198+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 >> v1:172.21.15.84:6792/0 conn(0x7f512813e750 legacy=0x7f512809bbc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:34.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.204+0000 7f512fc34640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:34.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.204+0000 7f512fc34640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f512813e1a0 con 0x7f512813ddd0 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391085186 0 0) 0x7f512813e1a0 con 0x7f512813ddd0 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 10 monclient(hunting): my global_id is 4651 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f512804c4a0 con 0x7f512813ddd0 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 907010534 0 0) 0x7f512804c4a0 con 0x7f512813ddd0 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f512813e1a0 con 0x7f512813ddd0 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f51240055f0 con 0x7f512813ddd0 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 10 monclient: dump: 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2822459525 0 0) 0x7f512813e1a0 con 0x7f512813ddd0 2024-09-18T14:01:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5128062dd0 con 0x7f512813ddd0 2024-09-18T14:01:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 2011164325) 0x7f5124006a80 con 0x7f512813ddd0 2024-09-18T14:01:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 >> v1:172.21.15.84:6792/0 conn(0x7f512813ddd0 legacy=0x7f512809c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.205+0000 7f511dffb640 1 -- 172.21.15.45:0/1813917243 >> v1:172.21.15.84:6792/0 conn(0x7f512813ddd0 legacy=0x7f512809c2d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:34.227 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:34.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:34.227 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:34.227 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:34.227 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:34.228 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:34.228 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:34.228 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:34.228 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:34.228 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:34.228 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:34.228 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:34.228 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:34.228 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:34.228 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:34.228 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:34.228 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:34.229 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:34.230 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:34.230 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:34.230 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:34.230 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:34.230 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:34.230 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:34.230 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:34.230 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:34.230 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:34.230 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:34.230 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:34.231 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:34.232 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:34.233 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:34.233 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:34.233 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:34.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.233 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:34.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.233 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:34.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:34.233 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.233 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.234 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.234 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:34.234 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.234 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.234 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.234 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:34.234 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:34.234 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.234 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.237 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:34.237 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.237 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.237 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.237 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:34.237 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:34.237 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.237 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.238 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.238 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:34.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:34.238 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.238 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.238 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.238 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:34.238 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.238 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.239 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.239 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:34.239 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:34.239 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.239 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.239 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.240 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:34.241 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.242 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.244 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:34.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:34.244 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.244 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.244 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:34.244 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:34.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:34.247 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:34.248 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:34.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:34.248 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:34.248 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:34.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.206+0000 7f512fc34640 1 -- 172.21.15.45:0/1813917243 >> v1:172.21.15.84:6812/31478 conn(0x7f510c03ab20 legacy=0x7f510c03cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.206+0000 7f512fc34640 10 monclient: shutdown 2024-09-18T14:01:34.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.206+0000 7f512fc34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:34.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.206+0000 7f512fc34640 1 -- 172.21.15.45:0/1813917243 >> v1:172.21.15.45:6791/0 conn(0x7f51280a1080 legacy=0x7f5128149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.206+0000 7f512e1d1640 1 -- 172.21.15.45:0/1813917243 reap_dead start 2024-09-18T14:01:34.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.206+0000 7f512e1d1640 1 -- 172.21.15.45:0/1813917243 reap_dead start 2024-09-18T14:01:34.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.206+0000 7f512fc34640 1 -- 172.21.15.45:0/1813917243 shutdown_connections 2024-09-18T14:01:34.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.206+0000 7f512fc34640 1 -- 172.21.15.45:0/1813917243 >> 172.21.15.45:0/1813917243 conn(0x7f51280a5bb0 msgr2=0x7f51280a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:34.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.206+0000 7f512fc34640 1 -- 172.21.15.45:0/1813917243 shutdown_connections 2024-09-18T14:01:34.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.206+0000 7f512fc34640 1 -- 172.21.15.45:0/1813917243 wait complete. 2024-09-18T14:01:34.250 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:34.250 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-18T14:01:34.250 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-18T14:01:34.253 INFO:tasks.rados.rados.0.smithi084.stdout:574: finishing write tid 3 to smithi08434663-29 2024-09-18T14:01:34.253 INFO:tasks.rados.rados.0.smithi084.stdout:574: finishing write tid 4 to smithi08434663-29 2024-09-18T14:01:34.253 INFO:tasks.rados.rados.0.smithi084.stdout:574: finishing write tid 5 to smithi08434663-29 2024-09-18T14:01:34.253 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 169 (ObjNum 233 snap 55 seq_num 233) dirty exists 2024-09-18T14:01:34.253 INFO:tasks.rados.rados.0.smithi084.stdout:574: left oid 29 (ObjNum 233 snap 55 seq_num 233) 2024-09-18T14:01:34.253 INFO:tasks.rados.rados.0.smithi084.stdout:575: finishing write tid 1 to smithi08434663-40 2024-09-18T14:01:34.253 INFO:tasks.rados.rados.0.smithi084.stdout:575: finishing write tid 2 to smithi08434663-40 2024-09-18T14:01:34.254 INFO:tasks.rados.rados.0.smithi084.stdout:575: finishing write tid 3 to smithi08434663-40 2024-09-18T14:01:34.254 INFO:tasks.rados.rados.0.smithi084.stdout:575: finishing write tid 4 to smithi08434663-40 2024-09-18T14:01:34.254 INFO:tasks.rados.rados.0.smithi084.stdout:575: finishing write tid 5 to smithi08434663-40 2024-09-18T14:01:34.254 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 174 (ObjNum 234 snap 55 seq_num 234) dirty exists 2024-09-18T14:01:34.254 INFO:tasks.rados.rados.0.smithi084.stdout:575: left oid 40 (ObjNum 234 snap 55 seq_num 234) 2024-09-18T14:01:34.254 INFO:tasks.rados.rados.0.smithi084.stdout:574: done (3 left) 2024-09-18T14:01:34.254 INFO:tasks.rados.rados.0.smithi084.stdout:575: done (2 left) 2024-09-18T14:01:34.254 INFO:tasks.rados.rados.0.smithi084.stdout:576: done (1 left) 2024-09-18T14:01:34.254 INFO:tasks.rados.rados.0.smithi084.stdout:577: done (0 left) 2024-09-18T14:01:34.254 INFO:tasks.rados.rados.0.smithi084.stdout:578: read oid 24 snap -1 2024-09-18T14:01:34.255 INFO:tasks.rados.rados.0.smithi084.stdout:578: expect (ObjNum 229 snap 54 seq_num 229) 2024-09-18T14:01:34.255 INFO:tasks.rados.rados.0.smithi084.stdout:579: rollback oid 4 current snap is 55 2024-09-18T14:01:34.255 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 4 to 51 2024-09-18T14:01:34.255 INFO:tasks.rados.rados.0.smithi084.stdout:580: read oid 39 snap 50 2024-09-18T14:01:34.255 INFO:tasks.rados.rados.0.smithi084.stdout:580: expect (ObjNum 215 snap 50 seq_num 215) 2024-09-18T14:01:34.255 INFO:tasks.rados.rados.0.smithi084.stdout:581: read oid 11 snap -1 2024-09-18T14:01:34.255 INFO:tasks.rados.rados.0.smithi084.stdout:581: expect (ObjNum 178 snap 41 seq_num 178) 2024-09-18T14:01:34.255 INFO:tasks.rados.rados.0.smithi084.stdout:582: write oid 18 current snap is 55 2024-09-18T14:01:34.255 INFO:tasks.rados.rados.0.smithi084.stdout:582: seq_num 235 ranges {749645=774105,2085386=609401,3237462=712183} 2024-09-18T14:01:34.255 INFO:tasks.rados.rados.0.smithi084.stdout:582: writing smithi08434663-18 from 749645 to 1523750 tid 1 2024-09-18T14:01:34.255 INFO:tasks.rados.rados.0.smithi084.stdout:582: writing smithi08434663-18 from 2085386 to 2694787 tid 2 2024-09-18T14:01:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:582: writing smithi08434663-18 from 3237462 to 3949645 tid 3 2024-09-18T14:01:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:583: copy_from oid 17 from oid 34 current snap is 55 2024-09-18T14:01:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:579: finishing rollback tid 0 to smithi08434663-4 2024-09-18T14:01:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:579: finishing rollback tid 1 to smithi08434663-4 2024-09-18T14:01:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:579: finishing rollback tid 2 to smithi08434663-4 2024-09-18T14:01:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 157 (ObjNum 219 snap 50 seq_num 219) dirty exists 2024-09-18T14:01:34.258 INFO:tasks.rados.rados.0.smithi084.stdout:578: expect (ObjNum 229 snap 54 seq_num 229) 2024-09-18T14:01:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.352+0000 7f200280f640 10 monclient: get_monmap_and_config 2024-09-18T14:01:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.352+0000 7f200280f640 10 monclient: build_initial_monmap 2024-09-18T14:01:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.352+0000 7f200280f640 10 monclient: monmap: 2024-09-18T14:01:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:34.353593+0000 2024-09-18T14:01:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:34.353593+0000 2024-09-18T14:01:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:34.354 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.353+0000 7f200280f640 1 Processor -- start 2024-09-18T14:01:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.353+0000 7f200280f640 1 -- start start 2024-09-18T14:01:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.353+0000 7f200280f640 10 monclient: init 2024-09-18T14:01:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.353+0000 7f200280f640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.353+0000 7f200280f640 10 monclient: _add_conns ranks=[5,3,2,6,1,8,7,0,4] 2024-09-18T14:01:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 10 monclient(hunting): picked mon.noname-h con 0x7f1ffc09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 10 monclient(hunting): picked mon.noname-g con 0x7f1ffc09fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:01:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 10 monclient(hunting): picked mon.noname-b con 0x7f1ffc09ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc05ed70 con 0x7f1ffc09ff70 2024-09-18T14:01:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc09dfc0 con 0x7f1ffc09fba0 2024-09-18T14:01:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc09dce0 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 10 monclient(hunting): authenticate will time out at 909.692170s 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f2000dac640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f1ffc09ff70 0x7f1ffc13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43736/0 (socket says 172.21.15.45:43736) 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffbfff640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f1ffc09f7d0 0x7f1ffc132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:47452/0 (socket says 172.21.15.45:47452) 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffb7fe640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7f1ffc09fba0 0x7f1ffc137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:53190/0 (socket says 172.21.15.45:53190) 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f2000dac640 1 -- 172.21.15.45:0/4275572854 learned_addr learned my addr 172.21.15.45:0/4275572854 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034215745 0 0) 0x7f1ffc09dfc0 con 0x7f1ffc09fba0 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient(hunting): my global_id is 4737 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe40034e0 con 0x7f1ffc09fba0 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166476183 0 0) 0x7f1ffc09dce0 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient(hunting): my global_id is 4568 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ffc09dfc0 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532115954 0 0) 0x7f1ffc05ed70 con 0x7f1ffc09ff70 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient(hunting): my global_id is 4691 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ffc09dce0 con 0x7f1ffc09ff70 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 261286582 0 0) 0x7f1fe40034e0 con 0x7f1ffc09fba0 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 >> v1:172.21.15.84:6791/0 conn(0x7f1ffc09f7d0 legacy=0x7f1ffc132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 >> v1:172.21.15.45:6790/0 conn(0x7f1ffc09ff70 legacy=0x7f1ffc13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient: _finish_hunting 0 2024-09-18T14:01:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 monclient: found mon.noname-g 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc0a1c00 con 0x7f1ffc09fba0 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient: _finish_auth 0 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 5 monclient: authenticate success, global_id 4737 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ff0001760 con 0x7f1ffc09fba0 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient: dump: 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.367 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:34.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1ff00018c0 con 0x7f1ffc09fba0 2024-09-18T14:01:34.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:34.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 1 -- 172.21.15.45:0/4275572854 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ff00047a0 con 0x7f1ffc09fba0 2024-09-18T14:01:34.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:34.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f200280f640 4 set_mon_vals no callback set 2024-09-18T14:01:34.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.354+0000 7f1ffaffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:01:34.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.355+0000 7f1ffaffd640 10 monclient: dump: 2024-09-18T14:01:34.368 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.368 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.368 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.368 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.368 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.368 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.368 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.368 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.368 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.368 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.368 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.368 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.355+0000 7f200280f640 10 monclient: shutdown 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.355+0000 7f200280f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.355+0000 7f200280f640 1 -- 172.21.15.45:0/4275572854 >> v1:172.21.15.84:6790/0 conn(0x7f1ffc09fba0 legacy=0x7f1ffc137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.355+0000 7f200280f640 1 -- 172.21.15.45:0/4275572854 shutdown_connections 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.355+0000 7f200280f640 1 -- 172.21.15.45:0/4275572854 >> 172.21.15.45:0/4275572854 conn(0x7f1ffc0630b0 msgr2=0x7f1ffc0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.355+0000 7f200280f640 1 -- 172.21.15.45:0/4275572854 shutdown_connections 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.355+0000 7f200280f640 1 -- 172.21.15.45:0/4275572854 wait complete. 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.355+0000 7f200280f640 10 monclient: build_initial_monmap 2024-09-18T14:01:34.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.355+0000 7f200280f640 10 monclient: monmap: 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:34.370 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 1 Processor -- start 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 1 -- start start 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 10 monclient: init 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 10 monclient: _add_conns ranks=[0,3,4,1,8,6,2,7,5] 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 10 monclient(hunting): picked mon.noname-a con 0x7f1ffc09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 10 monclient(hunting): picked mon.noname-g con 0x7f1ffc09fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 10 monclient(hunting): picked mon.noname-c con 0x7f1ffc09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc09d310 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc0a2560 con 0x7f1ffc09fba0 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc1208f0 con 0x7f1ffc09ff70 2024-09-18T14:01:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:34.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f200280f640 10 monclient(hunting): authenticate will time out at 909.694886s 2024-09-18T14:01:34.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.356+0000 7f1ffbfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1ffc09f7d0 0x7f1ffc131700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51430/0 (socket says 172.21.15.45:51430) 2024-09-18T14:01:34.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1ffbfff640 1 -- 172.21.15.45:0/1343831927 learned_addr learned my addr 172.21.15.45:0/1343831927 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:34.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919745727 0 0) 0x7f1ffc09d310 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): my global_id is 4671 2024-09-18T14:01:34.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe80034e0 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842820640 0 0) 0x7f1ffc1208f0 con 0x7f1ffc09ff70 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): my global_id is 134239 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ffc09d310 con 0x7f1ffc09ff70 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289527577 0 0) 0x7f1ffc0a2560 con 0x7f1ffc09fba0 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): my global_id is 4746 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ffc1208f0 con 0x7f1ffc09fba0 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 216915968 0 0) 0x7f1fe80034e0 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ffc0a2560 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1fec0015b0 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): dump: 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2914461097 0 0) 0x7f1ffc09d310 con 0x7f1ffc09ff70 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1fe80034e0 con 0x7f1ffc09ff70 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ff4001bb0 con 0x7f1ffc09ff70 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient(hunting): dump: 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.376 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4273290407 0 0) 0x7f1ffc1208f0 con 0x7f1ffc09fba0 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ffc09d310 con 0x7f1ffc09fba0 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3667405768 0 0) 0x7f1ffc0a2560 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 >> v1:172.21.15.45:6791/0 conn(0x7f1ffc09ff70 legacy=0x7f1ffc14a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 >> v1:172.21.15.84:6790/0 conn(0x7f1ffc09fba0 legacy=0x7f1ffc146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient: _finish_hunting 0 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 monclient: found mon.a 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc121850 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient: _finish_auth 0 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f200280f640 5 monclient: authenticate success, global_id 4671 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f200280f640 10 monclient: _renew_subs 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f200280f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f200280f640 1 -- 172.21.15.45:0/1343831927 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ffc11fd20 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f200280f640 10 monclient: _renew_subs 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f200280f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f200280f640 1 -- 172.21.15.45:0/1343831927 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc120420 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1fec004a50 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1fec004f00 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1ffa7fc640 4 set_mon_vals no callback set 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.357+0000 7f1fe3fff640 10 monclient: dump: 2024-09-18T14:01:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:34.380 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:34.380 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:34.380 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:34.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.358+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1fec00fed0 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.358+0000 7f200280f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:34.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.358+0000 7f200280f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:34.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.358+0000 7f200280f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:01:34.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.358+0000 7f200280f640 1 -- 172.21.15.45:0/1343831927 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fdc005010 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.358+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(126..126 src has 1..126) v4 ==== 3939+0+0 (unknown 1077151785 0 0) 0x7f1fec046c40 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.359+0000 7f1ffb7fe640 10 monclient: get_auth_request con 0x7f1fe803a900 auth_method 0 2024-09-18T14:01:34.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.363+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f1fec0115f0 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.363+0000 7f1fe3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:34.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.363+0000 7f1fe3fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:34.542 INFO:tasks.rados.rados.0.smithi084.stdout:580: expect (ObjNum 215 snap 50 seq_num 215) 2024-09-18T14:01:34.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.589+0000 7f200280f640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-18T14:01:34.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.589+0000 7f200280f640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-18T14:01:34.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.589+0000 7f200280f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:01:34.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.589+0000 7f200280f640 1 -- 172.21.15.45:0/1343831927 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f1fdc005b90 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1fe3fff640 1 -- 172.21.15.45:0/1343831927 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f1fec014e00 con 0x7f1ffc09f7d0 2024-09-18T14:01:34.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1fe3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-18T14:01:34.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1fe3fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:34.594 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.594 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=19,config=2,health=11,kv=8,logm=60} crc {auth=2510180551,config=3140514009,health=1886805112,kv=594742435,logm=3740525177}) 2024-09-18T14:01:34.594 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=19,config=2,health=11,kv=8,logm=60} crc {auth=2567248742,config=3140514009,health=1886805112,kv=594742435,logm=2976173805}) 2024-09-18T14:01:34.594 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.594 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=19,config=2,health=11,kv=8,logm=60} crc {auth=2510180551,config=3140514009,health=1886805112,kv=594742435,logm=3740525177}) 2024-09-18T14:01:34.594 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=195090947,config=3140514009,health=2157093694,kv=594742435,logm=3535085428}) 2024-09-18T14:01:34.595 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.595 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=19,config=2,health=11,kv=8,logm=60} crc {auth=2510180551,config=3140514009,health=1886805112,kv=594742435,logm=3740525177}) 2024-09-18T14:01:34.595 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=19,config=2,health=11,kv=8,logm=60} crc {auth=4153921719,config=3140514009,health=1886805112,kv=594742435,logm=3455906861}) 2024-09-18T14:01:34.595 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.595 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=19,config=2,health=11,kv=8,logm=60} crc {auth=2510180551,config=3140514009,health=1886805112,kv=594742435,logm=3740525177}) 2024-09-18T14:01:34.595 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=195090947,config=3140514009,health=2157093694,kv=594742435,logm=304377780}) 2024-09-18T14:01:34.595 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.595 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=19,config=2,health=11,kv=8,logm=60} crc {auth=2510180551,config=3140514009,health=1886805112,kv=594742435,logm=3740525177}) 2024-09-18T14:01:34.595 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=17,config=2,health=12,kv=8,logm=61} crc {auth=2494196399,config=3140514009,health=2157093694,kv=4045555473,logm=1232341809}) 2024-09-18T14:01:34.595 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.596 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=19,config=2,health=11,kv=8,logm=60} crc {auth=2510180551,config=3140514009,health=1886805112,kv=594742435,logm=3740525177}) 2024-09-18T14:01:34.596 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=195090947,config=3140514009,health=2157093694,kv=594742435,logm=4022858898}) 2024-09-18T14:01:34.596 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.596 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=19,config=2,health=11,kv=8,logm=60} crc {auth=2510180551,config=3140514009,health=1886805112,kv=594742435,logm=3740525177}) 2024-09-18T14:01:34.596 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=19,config=2,health=12,kv=8,logm=59} crc {auth=2510180551,config=55734535,health=2157093694,kv=594742435,logm=2320965833}) 2024-09-18T14:01:34.596 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.596 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=19,config=2,health=11,kv=8,logm=60} crc {auth=2510180551,config=3140514009,health=1886805112,kv=594742435,logm=3740525177}) 2024-09-18T14:01:34.597 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=19,config=1,health=12,kv=8,logm=60} crc {auth=3933326300,config=1228700967,health=2157093694,kv=3514761565,logm=3925212495}) 2024-09-18T14:01:34.597 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.597 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1278697615}) 2024-09-18T14:01:34.597 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3517346348}) 2024-09-18T14:01:34.597 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.597 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1278697615}) 2024-09-18T14:01:34.597 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2585939849}) 2024-09-18T14:01:34.598 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.598 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1278697615}) 2024-09-18T14:01:34.598 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1065363589}) 2024-09-18T14:01:34.598 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.598 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1278697615}) 2024-09-18T14:01:34.598 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3259871582}) 2024-09-18T14:01:34.598 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.599 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1278697615}) 2024-09-18T14:01:34.599 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3151804712}) 2024-09-18T14:01:34.599 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.599 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1278697615}) 2024-09-18T14:01:34.599 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2993752643}) 2024-09-18T14:01:34.599 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.599 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1278697615}) 2024-09-18T14:01:34.599 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2884888909}) 2024-09-18T14:01:34.599 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.600 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1278697615}) 2024-09-18T14:01:34.600 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4052071347}) 2024-09-18T14:01:34.600 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.600 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3193440968}) 2024-09-18T14:01:34.600 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3835076072}) 2024-09-18T14:01:34.600 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.600 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3193440968}) 2024-09-18T14:01:34.600 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4237306103}) 2024-09-18T14:01:34.600 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.600 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3193440968}) 2024-09-18T14:01:34.600 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1549734114}) 2024-09-18T14:01:34.600 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.601 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3193440968}) 2024-09-18T14:01:34.601 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3557941372}) 2024-09-18T14:01:34.601 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.601 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3193440968}) 2024-09-18T14:01:34.601 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=440786348}) 2024-09-18T14:01:34.601 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.601 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3193440968}) 2024-09-18T14:01:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f200280f640 1 -- 172.21.15.45:0/1343831927 >> v1:172.21.15.84:6812/31478 conn(0x7f1fe803a900 legacy=0x7f1fe803cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f200280f640 10 monclient: shutdown 2024-09-18T14:01:34.602 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=204628843}) 2024-09-18T14:01:34.603 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f200280f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f200280f640 1 -- 172.21.15.45:0/1343831927 >> v1:172.21.15.45:6789/0 conn(0x7f1ffc09f7d0 legacy=0x7f1ffc131700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:34.604 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3193440968}) 2024-09-18T14:01:34.604 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4126814130}) 2024-09-18T14:01:34.604 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f200280f640 1 -- 172.21.15.45:0/1343831927 shutdown_connections 2024-09-18T14:01:34.606 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3193440968}) 2024-09-18T14:01:34.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f200280f640 1 -- 172.21.15.45:0/1343831927 >> 172.21.15.45:0/1343831927 conn(0x7f1ffc0630b0 msgr2=0x7f1ffc0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:34.606 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4015401792}) 2024-09-18T14:01:34.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f200280f640 1 -- 172.21.15.45:0/1343831927 shutdown_connections 2024-09-18T14:01:34.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:34.593+0000 7f200280f640 1 -- 172.21.15.45:0/1343831927 wait complete. 2024-09-18T14:01:34.626 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.626 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=985696137}) 2024-09-18T14:01:34.626 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3556229491}) 2024-09-18T14:01:34.626 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.627 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=985696137}) 2024-09-18T14:01:34.627 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2192442121}) 2024-09-18T14:01:34.627 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.627 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=985696137}) 2024-09-18T14:01:34.627 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=655439252}) 2024-09-18T14:01:34.627 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.627 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=985696137}) 2024-09-18T14:01:34.627 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3723856090}) 2024-09-18T14:01:34.627 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.627 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=985696137}) 2024-09-18T14:01:34.627 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3145123643}) 2024-09-18T14:01:34.627 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.628 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=985696137}) 2024-09-18T14:01:34.628 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3601568480}) 2024-09-18T14:01:34.628 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.628 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=985696137}) 2024-09-18T14:01:34.628 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3658040772}) 2024-09-18T14:01:34.628 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.628 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=985696137}) 2024-09-18T14:01:34.628 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=553840080}) 2024-09-18T14:01:34.628 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.629 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=247800375}) 2024-09-18T14:01:34.629 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3773041349}) 2024-09-18T14:01:34.629 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.629 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=247800375}) 2024-09-18T14:01:34.629 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2506384142}) 2024-09-18T14:01:34.629 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.629 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=247800375}) 2024-09-18T14:01:34.629 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1553224362}) 2024-09-18T14:01:34.629 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.629 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=247800375}) 2024-09-18T14:01:34.629 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2871728466}) 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=247800375}) 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2274625443}) 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=247800375}) 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=35708784}) 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=247800375}) 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3239358095}) 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=247800375}) 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1103419162}) 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.630 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=698901956}) 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2631685271}) 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=698901956}) 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=67831213}) 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=698901956}) 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=474517003}) 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=698901956}) 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3410708274}) 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=698901956}) 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1441388573}) 2024-09-18T14:01:34.631 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=698901956}) 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2748784840}) 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=698901956}) 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=467082842}) 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=698901956}) 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=890688911}) 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4177484936}) 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1597551953}) 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.632 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4177484936}) 2024-09-18T14:01:34.633 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3171686369}) 2024-09-18T14:01:34.633 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.633 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4177484936}) 2024-09-18T14:01:34.633 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2389625871}) 2024-09-18T14:01:34.633 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.633 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4177484936}) 2024-09-18T14:01:34.633 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3781353837}) 2024-09-18T14:01:34.633 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.633 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4177484936}) 2024-09-18T14:01:34.633 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2793873825}) 2024-09-18T14:01:34.633 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.634 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4177484936}) 2024-09-18T14:01:34.634 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1288472951}) 2024-09-18T14:01:34.634 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.634 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4177484936}) 2024-09-18T14:01:34.634 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2198459742}) 2024-09-18T14:01:34.634 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.634 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4177484936}) 2024-09-18T14:01:34.634 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3948184242}) 2024-09-18T14:01:34.634 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.634 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=5} crc {logm=2130221274,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1946418359,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3019501228,osd_snap=364816840,osdmap=4287124580}) 2024-09-18T14:01:34.634 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=17,osdmap=6} crc {logm=4007535083,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=671113429,mon_config_key=1810658514,monmap=455718034,osd_metadata=456487060,osd_pg_creating=3019501228,osd_snap=142393294,osdmap=1191756134}) 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=5} crc {logm=2130221274,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1946418359,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3019501228,osd_snap=364816840,osdmap=4287124580}) 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=43,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=3} crc {logm=1225784599,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=767367809,mon_config_key=1810658514,monmap=2358078507,osd_metadata=2800182873,osd_pg_creating=3019501228,osd_snap=602954864,osdmap=3250870012}) 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=5} crc {logm=2130221274,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1946418359,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3019501228,osd_snap=364816840,osdmap=4287124580}) 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=41,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=18,osdmap=6} crc {logm=2053734814,mdsmap=1531151339,mgr=1359932698,mgr_metadata=270970526,mgrstat=817972676,mon_config_key=1810658515,monmap=2358078507,osd_metadata=456487060,osd_pg_creating=3019501228,osd_snap=3672694959,osdmap=1191756134}) 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=5} crc {logm=2130221274,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1946418359,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3019501228,osd_snap=364816840,osdmap=4287124580}) 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=18,osdmap=5} crc {logm=1937852066,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=671113429,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_snap=3009989249,osdmap=2623044928}) 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=5} crc {logm=2130221274,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1946418359,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3019501228,osd_snap=364816840,osdmap=4287124580}) 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=41,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=3} crc {logm=4108564253,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=671113429,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3019501228,osd_snap=2793074774,osdmap=3250870012}) 2024-09-18T14:01:34.635 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.636 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=5} crc {logm=2130221274,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1946418359,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3019501228,osd_snap=364816840,osdmap=4287124580}) 2024-09-18T14:01:34.636 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=41,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=19,osdmap=2} crc {logm=529587033,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=671113429,mon_config_key=1810658514,monmap=455718034,osd_metadata=1675047786,osd_pg_creating=3019501228,osd_snap=273098119,osdmap=288130613}) 2024-09-18T14:01:34.636 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.636 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=5} crc {logm=2130221274,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1946418359,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3019501228,osd_snap=364816840,osdmap=4287124580}) 2024-09-18T14:01:34.636 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=43,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=3} crc {logm=2509523434,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1038468297,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3019501228,osd_snap=347401406,osdmap=2511937980}) 2024-09-18T14:01:34.636 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.636 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=5} crc {logm=2130221274,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1946418359,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3019501228,osd_snap=364816840,osdmap=4287124580}) 2024-09-18T14:01:34.636 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=43,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=3} crc {logm=1225784599,mdsmap=3140514009,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2075709738,mon_config_key=1810658514,monmap=3140514009,osd_metadata=2800182873,osd_pg_creating=3019501228,osd_snap=879150350,osdmap=3250870012}) 2024-09-18T14:01:34.636 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.636 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=659715155}) 2024-09-18T14:01:34.636 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=885354563}) 2024-09-18T14:01:34.636 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.637 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=659715155}) 2024-09-18T14:01:34.637 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1021484950}) 2024-09-18T14:01:34.638 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.638 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=659715155}) 2024-09-18T14:01:34.638 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1888703761}) 2024-09-18T14:01:34.638 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.638 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=659715155}) 2024-09-18T14:01:34.639 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2994136755}) 2024-09-18T14:01:34.639 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.639 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=659715155}) 2024-09-18T14:01:34.639 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2165447485}) 2024-09-18T14:01:34.639 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.639 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=659715155}) 2024-09-18T14:01:34.640 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1143135361}) 2024-09-18T14:01:34.640 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.640 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=659715155}) 2024-09-18T14:01:34.640 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=846655064}) 2024-09-18T14:01:34.640 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.641 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=659715155}) 2024-09-18T14:01:34.641 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3832732694}) 2024-09-18T14:01:34.641 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.641 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=88} crc {osdmap=1094997633}) 2024-09-18T14:01:34.641 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=88} crc {osdmap=2669714902}) 2024-09-18T14:01:34.641 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.642 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=88} crc {osdmap=1094997633}) 2024-09-18T14:01:34.642 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=91} crc {osdmap=1831379563}) 2024-09-18T14:01:34.642 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.643 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=88} crc {osdmap=1094997633}) 2024-09-18T14:01:34.643 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=87} crc {osdmap=1397954887}) 2024-09-18T14:01:34.644 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.644 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=88} crc {osdmap=1094997633}) 2024-09-18T14:01:34.644 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=88} crc {osdmap=1482749990}) 2024-09-18T14:01:34.644 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.644 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=88} crc {osdmap=1094997633}) 2024-09-18T14:01:34.644 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=91} crc {osdmap=1397267257}) 2024-09-18T14:01:34.644 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.644 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=88} crc {osdmap=1094997633}) 2024-09-18T14:01:34.645 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=87} crc {osdmap=3721198263}) 2024-09-18T14:01:34.645 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.645 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=88} crc {osdmap=1094997633}) 2024-09-18T14:01:34.645 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=86} crc {osdmap=1736714489}) 2024-09-18T14:01:34.645 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:01:34.645 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=88} crc {osdmap=1094997633}) 2024-09-18T14:01:34.645 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:01:34.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=90} crc {osdmap=703046669}) 2024-09-18T14:01:34.646 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-18T14:01:34.772 INFO:tasks.rados.rados.0.smithi084.stdout:581: expect (ObjNum 178 snap 41 seq_num 178) 2024-09-18T14:01:34.941 INFO:tasks.rados.rados.0.smithi084.stdout:582: finishing write tid 1 to smithi08434663-18 2024-09-18T14:01:34.941 INFO:tasks.rados.rados.0.smithi084.stdout:582: finishing write tid 2 to smithi08434663-18 2024-09-18T14:01:34.941 INFO:tasks.rados.rados.0.smithi084.stdout:582: finishing write tid 3 to smithi08434663-18 2024-09-18T14:01:34.941 INFO:tasks.rados.rados.0.smithi084.stdout:578: done (5 left) 2024-09-18T14:01:34.941 INFO:tasks.rados.rados.0.smithi084.stdout:579: done (4 left) 2024-09-18T14:01:34.941 INFO:tasks.rados.rados.0.smithi084.stdout:580: done (3 left) 2024-09-18T14:01:34.941 INFO:tasks.rados.rados.0.smithi084.stdout:581: done (2 left) 2024-09-18T14:01:34.941 INFO:tasks.rados.rados.0.smithi084.stdout:584: snap_create 2024-09-18T14:01:34.941 INFO:tasks.rados.rados.0.smithi084.stdout:582: finishing write tid 4 to smithi08434663-18 2024-09-18T14:01:34.941 INFO:tasks.rados.rados.0.smithi084.stdout:582: finishing write tid 5 to smithi08434663-18 2024-09-18T14:01:34.942 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 113 (ObjNum 235 snap 55 seq_num 235) dirty exists 2024-09-18T14:01:34.942 INFO:tasks.rados.rados.0.smithi084.stdout:582: left oid 18 (ObjNum 235 snap 55 seq_num 235) 2024-09-18T14:01:34.942 INFO:tasks.rados.rados.0.smithi084.stdout:583: finishing copy_from racing read to smithi08434663-17 2024-09-18T14:01:34.942 INFO:tasks.rados.rados.0.smithi084.stdout:583: finishing copy_from to smithi08434663-17 2024-09-18T14:01:34.942 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 122 (ObjNum 210 snap 47 seq_num 210) dirty exists 2024-09-18T14:01:35.248 INFO:tasks.rados.rados.0.smithi084.stdout:582: done (2 left) 2024-09-18T14:01:35.248 INFO:tasks.rados.rados.0.smithi084.stdout:583: done (1 left) 2024-09-18T14:01:35.248 INFO:tasks.rados.rados.0.smithi084.stdout:584: done (0 left) 2024-09-18T14:01:35.248 INFO:tasks.rados.rados.0.smithi084.stdout:585: write oid 23 current snap is 56 2024-09-18T14:01:35.248 INFO:tasks.rados.rados.0.smithi084.stdout:585: seq_num 236 ranges {766452=565707,1860221=469412,2981238=594195} 2024-09-18T14:01:35.255 INFO:tasks.rados.rados.0.smithi084.stdout:585: writing smithi08434663-23 from 766452 to 1332159 tid 1 2024-09-18T14:01:35.260 INFO:tasks.rados.rados.0.smithi084.stdout:585: writing smithi08434663-23 from 1860221 to 2329633 tid 2 2024-09-18T14:01:35.267 INFO:tasks.rados.rados.0.smithi084.stdout:585: writing smithi08434663-23 from 2981238 to 3575433 tid 3 2024-09-18T14:01:35.267 INFO:tasks.rados.rados.0.smithi084.stdout:586: write (excl) oid 15 current snap is 56 2024-09-18T14:01:35.267 INFO:tasks.rados.rados.0.smithi084.stdout:585: finishing write tid 1 to smithi08434663-23 2024-09-18T14:01:35.267 INFO:tasks.rados.rados.0.smithi084.stdout:585: finishing write tid 2 to smithi08434663-23 2024-09-18T14:01:35.267 INFO:tasks.rados.rados.0.smithi084.stdout:586: seq_num 237 ranges {783259=640956,1918703=613070,3008661=405467,3821939=161320} 2024-09-18T14:01:35.275 INFO:tasks.rados.rados.0.smithi084.stdout:586: writing smithi08434663-15 from 783259 to 1424215 tid 1 2024-09-18T14:01:35.280 INFO:tasks.rados.rados.0.smithi084.stdout:586: writing smithi08434663-15 from 1918703 to 2531773 tid 2 2024-09-18T14:01:35.285 INFO:tasks.rados.rados.0.smithi084.stdout:586: writing smithi08434663-15 from 3008661 to 3414128 tid 3 2024-09-18T14:01:35.289 INFO:tasks.rados.rados.0.smithi084.stdout:586: writing smithi08434663-15 from 3821939 to 3983259 tid 4 2024-09-18T14:01:35.289 INFO:tasks.rados.rados.0.smithi084.stdout:587: write (excl) oid 17 current snap is 56 2024-09-18T14:01:35.289 INFO:tasks.rados.rados.0.smithi084.stdout:585: finishing write tid 3 to smithi08434663-23 2024-09-18T14:01:35.289 INFO:tasks.rados.rados.0.smithi084.stdout:585: finishing write tid 4 to smithi08434663-23 2024-09-18T14:01:35.290 INFO:tasks.rados.rados.0.smithi084.stdout:585: finishing write tid 5 to smithi08434663-23 2024-09-18T14:01:35.290 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 176 (ObjNum 236 snap 56 seq_num 236) dirty exists 2024-09-18T14:01:35.290 INFO:tasks.rados.rados.0.smithi084.stdout:585: left oid 23 (ObjNum 236 snap 56 seq_num 236) 2024-09-18T14:01:35.290 INFO:tasks.rados.rados.0.smithi084.stdout:586: finishing write tid 1 to smithi08434663-15 2024-09-18T14:01:35.290 INFO:tasks.rados.rados.0.smithi084.stdout:586: finishing write tid 2 to smithi08434663-15 2024-09-18T14:01:35.290 INFO:tasks.rados.rados.0.smithi084.stdout:586: finishing write tid 3 to smithi08434663-15 2024-09-18T14:01:35.290 INFO:tasks.rados.rados.0.smithi084.stdout:587: seq_num 238 ranges {400066=716205,1860832=139234} 2024-09-18T14:01:35.295 INFO:tasks.rados.rados.0.smithi084.stdout:587: writing smithi08434663-17 from 400066 to 1116271 tid 1 2024-09-18T14:01:35.300 INFO:tasks.rados.rados.0.smithi084.stdout:587: writing smithi08434663-17 from 1860832 to 2000066 tid 2 2024-09-18T14:01:35.300 INFO:tasks.rados.rados.0.smithi084.stdout:585: done (2 left) 2024-09-18T14:01:35.300 INFO:tasks.rados.rados.0.smithi084.stdout:588: snap_create 2024-09-18T14:01:35.301 INFO:tasks.rados.rados.0.smithi084.stdout:586: finishing write tid 4 to smithi08434663-15 2024-09-18T14:01:35.301 INFO:tasks.rados.rados.0.smithi084.stdout:586: finishing write tid 5 to smithi08434663-15 2024-09-18T14:01:35.301 INFO:tasks.rados.rados.0.smithi084.stdout:586: finishing write tid 6 to smithi08434663-15 2024-09-18T14:01:35.301 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 206 (ObjNum 237 snap 56 seq_num 237) dirty exists 2024-09-18T14:01:35.301 INFO:tasks.rados.rados.0.smithi084.stdout:586: left oid 15 (ObjNum 237 snap 56 seq_num 237) 2024-09-18T14:01:35.301 INFO:tasks.rados.rados.0.smithi084.stdout:587: finishing write tid 1 to smithi08434663-17 2024-09-18T14:01:35.302 INFO:tasks.rados.rados.0.smithi084.stdout:587: finishing write tid 2 to smithi08434663-17 2024-09-18T14:01:35.302 INFO:tasks.rados.rados.0.smithi084.stdout:587: finishing write tid 3 to smithi08434663-17 2024-09-18T14:01:35.302 INFO:tasks.rados.rados.0.smithi084.stdout:587: finishing write tid 4 to smithi08434663-17 2024-09-18T14:01:35.302 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 186 (ObjNum 238 snap 56 seq_num 238) dirty exists 2024-09-18T14:01:35.302 INFO:tasks.rados.rados.0.smithi084.stdout:587: left oid 17 (ObjNum 238 snap 56 seq_num 238) 2024-09-18T14:01:35.646 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:01:35.647 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:01:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.753+0000 7f5afc66f640 10 monclient: get_monmap_and_config 2024-09-18T14:01:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.753+0000 7f5afc66f640 10 monclient: build_initial_monmap 2024-09-18T14:01:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.753+0000 7f5afc66f640 10 monclient: monmap: 2024-09-18T14:01:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:35.754486+0000 2024-09-18T14:01:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:35.754486+0000 2024-09-18T14:01:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:35.755 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 1 Processor -- start 2024-09-18T14:01:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 1 -- start start 2024-09-18T14:01:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 10 monclient: init 2024-09-18T14:01:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 10 monclient: _add_conns ranks=[3,7,8,1,6,5,2,4,0] 2024-09-18T14:01:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 10 monclient(hunting): picked mon.noname-g con 0x7f5af40a23c0 addr v1:172.21.15.84:6790/0 2024-09-18T14:01:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 10 monclient(hunting): picked mon.noname-i con 0x7f5af4132b80 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 10 monclient(hunting): picked mon.noname-e con 0x7f5af4132f50 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af405ed70 con 0x7f5af40a23c0 2024-09-18T14:01:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af4097270 con 0x7f5af4132b80 2024-09-18T14:01:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af409f4c0 con 0x7f5af4132f50 2024-09-18T14:01:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.754+0000 7f5afc66f640 10 monclient(hunting): authenticate will time out at 911.092917s 2024-09-18T14:01:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5afac0c640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f5af4132f50 0x7f5af413aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:32774/0 (socket says 172.21.15.45:32774) 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5afac0c640 1 -- 172.21.15.45:0/2443271527 learned_addr learned my addr 172.21.15.45:0/2443271527 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106316820 0 0) 0x7f5af409f4c0 con 0x7f5af4132f50 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient(hunting): my global_id is 4634 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ae00034e0 con 0x7f5af4132f50 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61559036 0 0) 0x7f5af405ed70 con 0x7f5af40a23c0 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient(hunting): my global_id is 4755 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af409f4c0 con 0x7f5af40a23c0 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995810514 0 0) 0x7f5af4097270 con 0x7f5af4132b80 2024-09-18T14:01:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient(hunting): my global_id is 4660 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af405ed70 con 0x7f5af4132b80 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2867260928 0 0) 0x7f5ae00034e0 con 0x7f5af4132f50 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 >> v1:172.21.15.84:6792/0 conn(0x7f5af4132b80 legacy=0x7f5af41377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 >> v1:172.21.15.84:6790/0 conn(0x7f5af40a23c0 legacy=0x7f5af4132470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient: _finish_hunting 0 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 monclient: found mon.noname-e 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40a1540 con 0x7f5af4132f50 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient: _finish_auth 0 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5aec001830 con 0x7f5af4132f50 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5afc66f640 5 monclient: authenticate success, global_id 4634 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5afc66f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient: dump: 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5afc66f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5aec001a10 con 0x7f5af4132f50 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 1 -- 172.21.15.45:0/2443271527 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5aec004ba0 con 0x7f5af4132f50 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5afc66f640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5afc66f640 4 set_mon_vals no callback set 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.755+0000 7f5af9409640 10 monclient: dump: 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:35.768 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:35.769 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:35.769 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:35.769 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:35.769 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:35.769 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:35.769 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:35.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.756+0000 7f5afc66f640 10 monclient: shutdown 2024-09-18T14:01:35.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.756+0000 7f5afc66f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:35.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.756+0000 7f5afc66f640 1 -- 172.21.15.45:0/2443271527 >> v1:172.21.15.45:6793/0 conn(0x7f5af4132f50 legacy=0x7f5af413aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:35.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.756+0000 7f5afc66f640 1 -- 172.21.15.45:0/2443271527 shutdown_connections 2024-09-18T14:01:35.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.756+0000 7f5afc66f640 1 -- 172.21.15.45:0/2443271527 >> 172.21.15.45:0/2443271527 conn(0x7f5af4069850 msgr2=0x7f5af4062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:35.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.756+0000 7f5afc66f640 1 -- 172.21.15.45:0/2443271527 shutdown_connections 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.756+0000 7f5afc66f640 1 -- 172.21.15.45:0/2443271527 wait complete. 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.756+0000 7f5afc66f640 10 monclient: build_initial_monmap 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.756+0000 7f5afc66f640 10 monclient: monmap: 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:35.770 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:35.771 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 1 Processor -- start 2024-09-18T14:01:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 1 -- start start 2024-09-18T14:01:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 10 monclient: init 2024-09-18T14:01:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 10 monclient: _add_conns ranks=[8,7,0,4,6,3,5,1,2] 2024-09-18T14:01:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 10 monclient(hunting): picked mon.noname-e con 0x7f5af40a23c0 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 10 monclient(hunting): picked mon.noname-i con 0x7f5af4132b80 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:35.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 10 monclient(hunting): picked mon.noname-a con 0x7f5af4132f50 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:35.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af409f940 con 0x7f5af4132f50 2024-09-18T14:01:35.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af40a21b0 con 0x7f5af4132b80 2024-09-18T14:01:35.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af41423b0 con 0x7f5af40a23c0 2024-09-18T14:01:35.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:35.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afc66f640 10 monclient(hunting): authenticate will time out at 911.095813s 2024-09-18T14:01:35.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afa40b640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f5af40a23c0 0x7f5af4131520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:32788/0 (socket says 172.21.15.45:32788) 2024-09-18T14:01:35.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.757+0000 7f5afa40b640 1 -- 172.21.15.45:0/3844664286 learned_addr learned my addr 172.21.15.45:0/3844664286 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:35.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879728281 0 0) 0x7f5af41423b0 con 0x7f5af40a23c0 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): my global_id is 4643 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ad80034e0 con 0x7f5af40a23c0 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876149579 0 0) 0x7f5af409f940 con 0x7f5af4132f50 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): my global_id is 4680 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af41423b0 con 0x7f5af4132f50 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657429653 0 0) 0x7f5af40a21b0 con 0x7f5af4132b80 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:35.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): my global_id is 4669 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af409f940 con 0x7f5af4132b80 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2344013413 0 0) 0x7f5ad80034e0 con 0x7f5af40a23c0 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5af40a21b0 con 0x7f5af40a23c0 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1342824065 0 0) 0x7f5af41423b0 con 0x7f5af4132f50 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ad80034e0 con 0x7f5af4132f50 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5af0001be0 con 0x7f5af40a23c0 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): dump: 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:35.774 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:35.775 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:35.775 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:35.775 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:35.775 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:35.775 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:35.775 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:35.775 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:35.775 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:35.775 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:35.776 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:35.776 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:35.776 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:35.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:35.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1749232562 0 0) 0x7f5af409f940 con 0x7f5af4132b80 2024-09-18T14:01:35.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5af41423b0 con 0x7f5af4132b80 2024-09-18T14:01:35.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5aec001770 con 0x7f5af4132f50 2024-09-18T14:01:35.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:35.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:35.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): dump: 2024-09-18T14:01:35.776 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:35.776 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:35.777 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5ae8001ca0 con 0x7f5af4132b80 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient(hunting): dump: 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:35.778 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 216313556 0 0) 0x7f5ad80034e0 con 0x7f5af4132f50 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 >> v1:172.21.15.45:6793/0 conn(0x7f5af40a23c0 legacy=0x7f5af4131520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 >> v1:172.21.15.84:6792/0 conn(0x7f5af4132b80 legacy=0x7f5af41468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient: _finish_hunting 0 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 monclient: found mon.a 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:35.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af4121670 con 0x7f5af4132f50 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient: _finish_auth 0 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5afc66f640 5 monclient: authenticate success, global_id 4680 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5afc66f640 10 monclient: _renew_subs 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5afc66f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5afc66f640 1 -- 172.21.15.45:0/3844664286 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af4121950 con 0x7f5af4132f50 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5afc66f640 10 monclient: _renew_subs 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5afc66f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5afc66f640 1 -- 172.21.15.45:0/3844664286 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af4133b30 con 0x7f5af4132f50 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5aec004a50 con 0x7f5af4132f50 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5aec004f00 con 0x7f5af4132f50 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5ae67fc640 10 monclient: dump: 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.758+0000 7f5af8c08640 4 set_mon_vals no callback set 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.759+0000 7f5afc66f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.759+0000 7f5afc66f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.759+0000 7f5afc66f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.759+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f5aec00fed0 con 0x7f5af4132f50 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.759+0000 7f5afc66f640 1 -- 172.21.15.45:0/3844664286 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5adc005010 con 0x7f5af4132f50 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.760+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(127..127 src has 1..127) v4 ==== 3911+0+0 (unknown 1916185066 0 0) 0x7f5aec046960 con 0x7f5af4132f50 2024-09-18T14:01:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.760+0000 7f5afa40b640 10 monclient: get_auth_request con 0x7f5ad803a9d0 auth_method 0 2024-09-18T14:01:35.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.764+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f5aec015270 con 0x7f5af4132f50 2024-09-18T14:01:35.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.764+0000 7f5ae67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:35.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.764+0000 7f5ae67fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.990+0000 7f5afc66f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:01:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.990+0000 7f5afc66f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:01:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.990+0000 7f5afc66f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:01:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.990+0000 7f5afc66f640 1 -- 172.21.15.45:0/3844664286 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5adc005840 con 0x7f5af4132f50 2024-09-18T14:01:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.991+0000 7f5ae67fc640 1 -- 172.21.15.45:0/3844664286 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 701381036) 0x7f5aec014c70 con 0x7f5af4132f50 2024-09-18T14:01:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.991+0000 7f5ae67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:01:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.991+0000 7f5ae67fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:35.992 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":50,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":5,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:01:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.992+0000 7f5afc66f640 1 -- 172.21.15.45:0/3844664286 >> v1:172.21.15.84:6812/31478 conn(0x7f5ad803a9d0 legacy=0x7f5ad803ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.992+0000 7f5afc66f640 10 monclient: shutdown 2024-09-18T14:01:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.992+0000 7f5afc66f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.992+0000 7f5afc66f640 1 -- 172.21.15.45:0/3844664286 >> v1:172.21.15.45:6789/0 conn(0x7f5af4132f50 legacy=0x7f5af4149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.992+0000 7f5afc66f640 1 -- 172.21.15.45:0/3844664286 shutdown_connections 2024-09-18T14:01:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.992+0000 7f5afc66f640 1 -- 172.21.15.45:0/3844664286 >> 172.21.15.45:0/3844664286 conn(0x7f5af4069850 msgr2=0x7f5af4099e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.993+0000 7f5afc66f640 1 -- 172.21.15.45:0/3844664286 shutdown_connections 2024-09-18T14:01:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:35.993+0000 7f5afc66f640 1 -- 172.21.15.45:0/3844664286 wait complete. 2024-09-18T14:01:36.003 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:01:36.003 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-18T14:01:36.003 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:01:36.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:01:36.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.100+0000 7f7dca8af640 10 monclient: get_monmap_and_config 2024-09-18T14:01:36.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.100+0000 7f7dca8af640 10 monclient: build_initial_monmap 2024-09-18T14:01:36.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.100+0000 7f7dca8af640 10 monclient: monmap: 2024-09-18T14:01:36.101 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:36.101 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.101 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:36.101512+0000 2024-09-18T14:01:36.101 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:36.101512+0000 2024-09-18T14:01:36.101 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:36.101 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.102 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:36.102 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:36.102 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:36.102 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:36.102 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:36.102 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:36.102 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:36.102 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:36.102 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:36.102 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.101+0000 7f7dca8af640 1 Processor -- start 2024-09-18T14:01:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.101+0000 7f7dca8af640 1 -- start start 2024-09-18T14:01:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.101+0000 7f7dca8af640 10 monclient: init 2024-09-18T14:01:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.101+0000 7f7dca8af640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.101+0000 7f7dca8af640 10 monclient: _add_conns ranks=[4,7,5,2,0,6,3,8,1] 2024-09-18T14:01:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.101+0000 7f7dca8af640 10 monclient(hunting): picked mon.noname-c con 0x7f7dc409f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.101+0000 7f7dca8af640 10 monclient(hunting): picked mon.noname-i con 0x7f7dc409fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.101+0000 7f7dca8af640 10 monclient(hunting): picked mon.noname-h con 0x7f7dc409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.101+0000 7f7dca8af640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc405ed70 con 0x7f7dc409f7d0 2024-09-18T14:01:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dca8af640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc409dfc0 con 0x7f7dc409ff70 2024-09-18T14:01:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dca8af640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc409dce0 con 0x7f7dc409fba0 2024-09-18T14:01:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dca8af640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dca8af640 10 monclient(hunting): authenticate will time out at 911.440050s 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc3fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f7dc409f7d0 0x7f7dc4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36972/0 (socket says 172.21.15.45:36972) 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc3fff640 1 -- 172.21.15.45:0/4268106274 learned_addr learned my addr 172.21.15.45:0/4268106274 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471396075 0 0) 0x7f7dc405ed70 con 0x7f7dc409f7d0 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient(hunting): my global_id is 134248 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dac0034e0 con 0x7f7dc409f7d0 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311564297 0 0) 0x7f7dc409dfc0 con 0x7f7dc409ff70 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient(hunting): my global_id is 4577 2024-09-18T14:01:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc405ed70 con 0x7f7dc409ff70 2024-09-18T14:01:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415644609 0 0) 0x7f7dc409dce0 con 0x7f7dc409fba0 2024-09-18T14:01:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient(hunting): my global_id is 4678 2024-09-18T14:01:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc409dfc0 con 0x7f7dc409fba0 2024-09-18T14:01:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3786376006 0 0) 0x7f7dac0034e0 con 0x7f7dc409f7d0 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 >> v1:172.21.15.84:6792/0 conn(0x7f7dc409fba0 legacy=0x7f7dc4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 >> v1:172.21.15.84:6791/0 conn(0x7f7dc409ff70 legacy=0x7f7dc413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 monclient: found mon.noname-c 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc40a1c00 con 0x7f7dc409f7d0 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient: _finish_auth 0 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dca8af640 5 monclient: authenticate success, global_id 134248 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7db4001830 con 0x7f7dc409f7d0 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:01:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient: dump: 2024-09-18T14:01:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.116 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dca8af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7db4001a10 con 0x7f7dc409f7d0 2024-09-18T14:01:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dca8af640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 1 -- 172.21.15.45:0/4268106274 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7db4004ba0 con 0x7f7dc409f7d0 2024-09-18T14:01:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dca8af640 4 set_mon_vals no callback set 2024-09-18T14:01:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.102+0000 7f7dc2ffd640 10 monclient: dump: 2024-09-18T14:01:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.120 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.103+0000 7f7dca8af640 10 monclient: shutdown 2024-09-18T14:01:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.103+0000 7f7dca8af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.103+0000 7f7dca8af640 1 -- 172.21.15.45:0/4268106274 >> v1:172.21.15.45:6791/0 conn(0x7f7dc409f7d0 legacy=0x7f7dc4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.103+0000 7f7dca8af640 1 -- 172.21.15.45:0/4268106274 shutdown_connections 2024-09-18T14:01:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.103+0000 7f7dca8af640 1 -- 172.21.15.45:0/4268106274 >> 172.21.15.45:0/4268106274 conn(0x7f7dc40630b0 msgr2=0x7f7dc40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.103+0000 7f7dca8af640 1 -- 172.21.15.45:0/4268106274 shutdown_connections 2024-09-18T14:01:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.103+0000 7f7dca8af640 1 -- 172.21.15.45:0/4268106274 wait complete. 2024-09-18T14:01:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.103+0000 7f7dca8af640 10 monclient: build_initial_monmap 2024-09-18T14:01:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.103+0000 7f7dca8af640 10 monclient: monmap: 2024-09-18T14:01:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:36.123 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 1 Processor -- start 2024-09-18T14:01:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 1 -- start start 2024-09-18T14:01:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 10 monclient: init 2024-09-18T14:01:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 10 monclient: _add_conns ranks=[4,6,5,8,0,3,1,7,2] 2024-09-18T14:01:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 10 monclient(hunting): picked mon.noname-c con 0x7f7dc409f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 10 monclient(hunting): picked mon.noname-d con 0x7f7dc409fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:01:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 10 monclient(hunting): picked mon.noname-h con 0x7f7dc409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc40a2560 con 0x7f7dc409f7d0 2024-09-18T14:01:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc409d310 con 0x7f7dc409ff70 2024-09-18T14:01:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc4142050 con 0x7f7dc409fba0 2024-09-18T14:01:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dca8af640 10 monclient(hunting): authenticate will time out at 911.442923s 2024-09-18T14:01:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dc3fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f7dc409f7d0 0x7f7dc4141940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36982/0 (socket says 172.21.15.45:36982) 2024-09-18T14:01:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.104+0000 7f7dc3fff640 1 -- 172.21.15.45:0/2279761779 learned_addr learned my addr 172.21.15.45:0/2279761779 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561300744 0 0) 0x7f7dc4142050 con 0x7f7dc409fba0 2024-09-18T14:01:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): my global_id is 4596 2024-09-18T14:01:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7db00034e0 con 0x7f7dc409fba0 2024-09-18T14:01:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373895481 0 0) 0x7f7dc40a2560 con 0x7f7dc409f7d0 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): my global_id is 134257 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc4142050 con 0x7f7dc409f7d0 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2681543374 0 0) 0x7f7db00034e0 con 0x7f7dc409fba0 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7dc40a2560 con 0x7f7dc409fba0 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7db8001a90 con 0x7f7dc409fba0 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): dump: 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.128 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165949124 0 0) 0x7f7dc409d310 con 0x7f7dc409ff70 2024-09-18T14:01:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): my global_id is 4586 2024-09-18T14:01:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7db00034e0 con 0x7f7dc409ff70 2024-09-18T14:01:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3523416947 0 0) 0x7f7dc4142050 con 0x7f7dc409f7d0 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7dc409d310 con 0x7f7dc409f7d0 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7db4004ae0 con 0x7f7dc409f7d0 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient(hunting): dump: 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.130 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2499279757 0 0) 0x7f7dc40a2560 con 0x7f7dc409fba0 2024-09-18T14:01:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 >> v1:172.21.15.84:6791/0 conn(0x7f7dc409ff70 legacy=0x7f7dc4149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 >> v1:172.21.15.45:6791/0 conn(0x7f7dc409f7d0 legacy=0x7f7dc4141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient: _finish_hunting 0 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 monclient: found mon.d 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc4122540 con 0x7f7dc409fba0 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient: _finish_auth 0 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dca8af640 5 monclient: authenticate success, global_id 4596 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dca8af640 10 monclient: _renew_subs 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dca8af640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dca8af640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc41208c0 con 0x7f7dc409fba0 2024-09-18T14:01:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dca8af640 10 monclient: _renew_subs 2024-09-18T14:01:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dca8af640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:01:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dca8af640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc4120170 con 0x7f7dc409fba0 2024-09-18T14:01:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7db8002d40 con 0x7f7dc409fba0 2024-09-18T14:01:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7db8002360 con 0x7f7dc409fba0 2024-09-18T14:01:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:01:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dc27fc640 4 set_mon_vals no callback set 2024-09-18T14:01:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.105+0000 7f7dabfff640 10 monclient: dump: 2024-09-18T14:01:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.134 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.106+0000 7f7dca8af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.106+0000 7f7dca8af640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7da4006580 con 0x7f7da4002830 2024-09-18T14:01:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.106+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7db80024c0 con 0x7f7dc409fba0 2024-09-18T14:01:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(127..127 src has 1..127) v4 ==== 3911+0+0 (unknown 1916185066 0 0) 0x7f7db8044260 con 0x7f7dc409fba0 2024-09-18T14:01:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339722745 0 0) 0x7f7da4006580 con 0x7f7da4002830 2024-09-18T14:01:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 10 monclient(hunting): my global_id is 4689 2024-09-18T14:01:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc40a2560 con 0x7f7da4002830 2024-09-18T14:01:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dc8e4c640 10 monclient: get_auth_request con 0x7f7db003a5d0 auth_method 0 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 347757033 0 0) 0x7f7dc40a2560 con 0x7f7da4002830 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7da4006580 con 0x7f7da4002830 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7db4001760 con 0x7f7da4002830 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 10 monclient: dump: 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.137 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2548689799 0 0) 0x7f7da4006580 con 0x7f7da4002830 2024-09-18T14:01:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7da400a0a0 con 0x7f7da4002830 2024-09-18T14:01:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7db4005320 con 0x7f7da4002830 2024-09-18T14:01:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 >> v1:172.21.15.45:6789/0 conn(0x7f7da4002830 legacy=0x7f7da4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.107+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 >> v1:172.21.15.45:6789/0 conn(0x7f7da4002830 legacy=0x7f7da4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.113+0000 7f7dca8af640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.113+0000 7f7dca8af640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7da4007540 con 0x7f7da40068f0 2024-09-18T14:01:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.113+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977813144 0 0) 0x7f7da4007540 con 0x7f7da40068f0 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.113+0000 7f7dabfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.113+0000 7f7dabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.113+0000 7f7dabfff640 10 monclient(hunting): my global_id is 4698 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.113+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7da4006580 con 0x7f7da40068f0 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.113+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2580223568 0 0) 0x7f7da4006580 con 0x7f7da40068f0 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.113+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7da4007540 con 0x7f7da40068f0 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7db4005c20 con 0x7f7da40068f0 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dabfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dabfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dabfff640 10 monclient: dump: 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4068454851 0 0) 0x7f7da4007540 con 0x7f7da40068f0 2024-09-18T14:01:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7da40087a0 con 0x7f7da40068f0 2024-09-18T14:01:36.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 407585982) 0x7f7db4008310 con 0x7f7da40068f0 2024-09-18T14:01:36.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dabfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:36.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dabfff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:36.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 >> v1:172.21.15.45:6789/0 conn(0x7f7da40068f0 legacy=0x7f7da4006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dabfff640 1 -- 172.21.15.45:0/2279761779 >> v1:172.21.15.45:6789/0 conn(0x7f7da40068f0 legacy=0x7f7da4006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:36.142 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:36.142 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:36.142 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:36.142 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:36.142 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 5, 2024-09-18T14:01:36.143 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.144 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.145 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.147 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.149 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:36.149 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:36.149 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.149 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.149 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.149 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:36.149 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:36.149 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.149 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:36.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:36.151 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.151 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.151 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.151 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:36.151 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.151 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.151 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.151 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:36.152 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.154 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.156 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.157 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:36.158 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.161 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:36.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.161 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.161 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.161 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:01:36.161 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.161 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.161 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:36.161 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:36.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dca8af640 1 -- 172.21.15.45:0/2279761779 >> v1:172.21.15.84:6812/31478 conn(0x7f7db003a5d0 legacy=0x7f7db003ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dca8af640 10 monclient: shutdown 2024-09-18T14:01:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dca8af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dca8af640 1 -- 172.21.15.45:0/2279761779 >> v1:172.21.15.45:6792/0 conn(0x7f7dc409fba0 legacy=0x7f7dc4131b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dc8e4c640 1 -- 172.21.15.45:0/2279761779 reap_dead start 2024-09-18T14:01:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dc8e4c640 1 -- 172.21.15.45:0/2279761779 reap_dead start 2024-09-18T14:01:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dca8af640 1 -- 172.21.15.45:0/2279761779 shutdown_connections 2024-09-18T14:01:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dca8af640 1 -- 172.21.15.45:0/2279761779 >> 172.21.15.45:0/2279761779 conn(0x7f7dc40630b0 msgr2=0x7f7dc40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dca8af640 1 -- 172.21.15.45:0/2279761779 shutdown_connections 2024-09-18T14:01:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.114+0000 7f7dca8af640 1 -- 172.21.15.45:0/2279761779 wait complete. 2024-09-18T14:01:36.163 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:36.164 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:01:36.164 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:01:36.253 INFO:tasks.rados.rados.0.smithi084.stdout:586: done (2 left) 2024-09-18T14:01:36.253 INFO:tasks.rados.rados.0.smithi084.stdout:587: done (1 left) 2024-09-18T14:01:36.253 INFO:tasks.rados.rados.0.smithi084.stdout:588: done (0 left) 2024-09-18T14:01:36.253 INFO:tasks.rados.rados.0.smithi084.stdout:589: write (excl) oid 49 current snap is 57 2024-09-18T14:01:36.253 INFO:tasks.rados.rados.0.smithi084.stdout:589: seq_num 239 ranges {416873=791454,1919314=97559} 2024-09-18T14:01:36.259 INFO:tasks.rados.rados.0.smithi084.stdout:589: writing smithi08434663-49 from 416873 to 1208327 tid 1 2024-09-18T14:01:36.263 INFO:tasks.rados.rados.0.smithi084.stdout:589: writing smithi08434663-49 from 1919314 to 2016873 tid 2 2024-09-18T14:01:36.264 INFO:tasks.rados.rados.0.smithi084.stdout:590: write oid 21 current snap is 57 2024-09-18T14:01:36.264 INFO:tasks.rados.rados.0.smithi084.stdout:590: seq_num 240 ranges {433680=466703,1577796=455884} 2024-09-18T14:01:36.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.265+0000 7f51f6ec9640 10 monclient: get_monmap_and_config 2024-09-18T14:01:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.265+0000 7f51f6ec9640 10 monclient: build_initial_monmap 2024-09-18T14:01:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.265+0000 7f51f6ec9640 10 monclient: monmap: 2024-09-18T14:01:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:36.266201+0000 2024-09-18T14:01:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:36.266201+0000 2024-09-18T14:01:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:36.267 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:36.267 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:36.267 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:36.267 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:36.267 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:36.267 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.274 INFO:tasks.rados.rados.0.smithi084.stdout:590: writing smithi08434663-21 from 433680 to 900383 tid 1 2024-09-18T14:01:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 1 Processor -- start 2024-09-18T14:01:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 1 -- start start 2024-09-18T14:01:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 10 monclient: init 2024-09-18T14:01:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 10 monclient: _add_conns ranks=[1,0,8,3,7,5,2,6,4] 2024-09-18T14:01:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 10 monclient(hunting): picked mon.noname-f con 0x7f51f009f3d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 10 monclient(hunting): picked mon.noname-a con 0x7f51f00979e0 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 10 monclient(hunting): picked mon.noname-e con 0x7f51f0138a20 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f005ed70 con 0x7f51f00979e0 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f0096d20 con 0x7f51f009f3d0 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f009f7a0 con 0x7f51f0138a20 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.266+0000 7f51f6ec9640 10 monclient(hunting): authenticate will time out at 911.604993s 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51f5466640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f51f0138a20 0x7f51f013ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:32800/0 (socket says 172.21.15.45:32800) 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51f5466640 1 -- 172.21.15.45:0/1925147739 learned_addr learned my addr 172.21.15.45:0/1925147739 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51effff640 1 --1- 172.21.15.45:0/1925147739 >> v1:172.21.15.45:6789/0 conn(0x7f51f00979e0 0x7f51f009d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51460/0 (socket says 172.21.15.45:51460) 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273578269 0 0) 0x7f51f005ed70 con 0x7f51f00979e0 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient(hunting): my global_id is 4707 2024-09-18T14:01:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51e00034e0 con 0x7f51f00979e0 2024-09-18T14:01:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1873200023 0 0) 0x7f51f009f7a0 con 0x7f51f0138a20 2024-09-18T14:01:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient(hunting): my global_id is 4652 2024-09-18T14:01:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f005ed70 con 0x7f51f0138a20 2024-09-18T14:01:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474419623 0 0) 0x7f51f0096d20 con 0x7f51f009f3d0 2024-09-18T14:01:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient(hunting): my global_id is 4600 2024-09-18T14:01:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f009f7a0 con 0x7f51f009f3d0 2024-09-18T14:01:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1923865555 0 0) 0x7f51e00034e0 con 0x7f51f00979e0 2024-09-18T14:01:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 >> v1:172.21.15.45:6793/0 conn(0x7f51f0138a20 legacy=0x7f51f013ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 >> v1:172.21.15.84:6789/0 conn(0x7f51f009f3d0 legacy=0x7f51f009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 monclient: found mon.noname-a 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0097000 con 0x7f51f00979e0 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient: _finish_auth 0 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f51dc001830 con 0x7f51f00979e0 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51f6ec9640 5 monclient: authenticate success, global_id 4707 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51f6ec9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient: dump: 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51f6ec9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f51dc001a10 con 0x7f51f00979e0 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 1 -- 172.21.15.45:0/1925147739 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f51dc004ae0 con 0x7f51f00979e0 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51f6ec9640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51f6ec9640 4 set_mon_vals no callback set 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.267+0000 7f51ef7fe640 10 monclient: dump: 2024-09-18T14:01:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.268+0000 7f51f6ec9640 10 monclient: shutdown 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.268+0000 7f51f6ec9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.268+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1925147739 >> v1:172.21.15.45:6789/0 conn(0x7f51f00979e0 legacy=0x7f51f009d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.268+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1925147739 shutdown_connections 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.268+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1925147739 >> 172.21.15.45:0/1925147739 conn(0x7f51f00a2710 msgr2=0x7f51f009eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.268+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1925147739 shutdown_connections 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.268+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1925147739 wait complete. 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.268+0000 7f51f6ec9640 10 monclient: build_initial_monmap 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.268+0000 7f51f6ec9640 10 monclient: monmap: 2024-09-18T14:01:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.269+0000 7f51f6ec9640 1 Processor -- start 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.269+0000 7f51f6ec9640 1 -- start start 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.269+0000 7f51f6ec9640 10 monclient: init 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51f6ec9640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51f6ec9640 10 monclient: _add_conns ranks=[6,1,5,7,4,8,3,0,2] 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51f6ec9640 10 monclient(hunting): picked mon.noname-d con 0x7f51f00979e0 addr v1:172.21.15.45:6792/0 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51f6ec9640 10 monclient(hunting): picked mon.noname-f con 0x7f51f009f3d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51f6ec9640 10 monclient(hunting): picked mon.noname-h con 0x7f51f0138a20 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51f6ec9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f011a340 con 0x7f51f009f3d0 2024-09-18T14:01:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51f6ec9640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f00983c0 con 0x7f51f0138a20 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51f6ec9640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f009e730 con 0x7f51f00979e0 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51f6ec9640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51f6ec9640 10 monclient(hunting): authenticate will time out at 911.608175s 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51f4c65640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f51f00979e0 0x7f51f01411f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56146/0 (socket says 172.21.15.45:56146) 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51f4c65640 1 -- 172.21.15.45:0/1786215618 learned_addr learned my addr 172.21.15.45:0/1786215618 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765883396 0 0) 0x7f51f009e730 con 0x7f51f00979e0 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): my global_id is 4605 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d40034e0 con 0x7f51f00979e0 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619970704 0 0) 0x7f51f0097000 con 0x7f51f009f3d0 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): my global_id is 4609 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f009e730 con 0x7f51f009f3d0 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669431442 0 0) 0x7f51f00983c0 con 0x7f51f0138a20 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): my global_id is 4595 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f0097000 con 0x7f51f0138a20 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1085092729 0 0) 0x7f51d40034e0 con 0x7f51f00979e0 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51f00983c0 con 0x7f51f00979e0 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f51e4001d50 con 0x7f51f00979e0 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): dump: 2024-09-18T14:01:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2069168293 0 0) 0x7f51f009e730 con 0x7f51f009f3d0 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51d40034e0 con 0x7f51f009f3d0 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f51dc001770 con 0x7f51f009f3d0 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 10 monclient(hunting): dump: 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.292 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.270+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2536063313 0 0) 0x7f51f0097000 con 0x7f51f0138a20 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51f009e730 con 0x7f51f0138a20 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f51e80018d0 con 0x7f51f0138a20 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 10 monclient(hunting): dump: 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1572138804 0 0) 0x7f51f00983c0 con 0x7f51f00979e0 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 >> v1:172.21.15.84:6791/0 conn(0x7f51f0138a20 legacy=0x7f51f0149580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 >> v1:172.21.15.84:6789/0 conn(0x7f51f009f3d0 legacy=0x7f51f0131d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 10 monclient: _finish_hunting 0 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 1 monclient: found mon.d 2024-09-18T14:01:36.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0132420 con 0x7f51f00979e0 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 10 monclient: _finish_auth 0 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51f6ec9640 5 monclient: authenticate success, global_id 4605 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51f6ec9640 10 monclient: _renew_subs 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51f6ec9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f011be00 con 0x7f51f00979e0 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51f6ec9640 10 monclient: _renew_subs 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51f6ec9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f0141fa0 con 0x7f51f00979e0 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f51e40021e0 con 0x7f51f00979e0 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f51e4002750 con 0x7f51f00979e0 2024-09-18T14:01:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51ecff9640 10 monclient: dump: 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.271+0000 7f51eeffd640 4 set_mon_vals no callback set 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51f6ec9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f51f004c4a0 con 0x7f51f011aea0 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f51e40100e0 con 0x7f51f00979e0 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 3911+0+0 (unknown 4286160998 0 0) 0x7f51e4046b00 con 0x7f51f00979e0 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148102211 0 0) 0x7f51f004c4a0 con 0x7f51f011aea0 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 10 monclient(hunting): my global_id is 4700 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f00983c0 con 0x7f51f011aea0 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51f5466640 10 monclient: get_auth_request con 0x7f51d403ab00 auth_method 0 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2152321151 0 0) 0x7f51f00983c0 con 0x7f51f011aea0 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51f004c4a0 con 0x7f51f011aea0 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f51dc004cd0 con 0x7f51f011aea0 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.272+0000 7f51ecff9640 10 monclient: dump: 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.273+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 411249588 0 0) 0x7f51f004c4a0 con 0x7f51f011aea0 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.273+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51f0068240 con 0x7f51f011aea0 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.273+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f51dc006fc0 con 0x7f51f011aea0 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.273+0000 7f51ecff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.273+0000 7f51ecff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:36.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.273+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 >> v1:172.21.15.45:6790/0 conn(0x7f51f011aea0 legacy=0x7f51f005e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.273+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 >> v1:172.21.15.45:6790/0 conn(0x7f51f011aea0 legacy=0x7f51f005e310 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:36.302 INFO:tasks.rados.rados.0.smithi084.stdout:590: writing smithi08434663-21 from 1577796 to 2033680 tid 2 2024-09-18T14:01:36.302 INFO:tasks.rados.rados.0.smithi084.stdout:591: copy_from oid 36 from oid 24 current snap is 57 2024-09-18T14:01:36.302 INFO:tasks.rados.rados.0.smithi084.stdout:589: finishing write tid 1 to smithi08434663-49 2024-09-18T14:01:36.302 INFO:tasks.rados.rados.0.smithi084.stdout:589: finishing write tid 2 to smithi08434663-49 2024-09-18T14:01:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:589: finishing write tid 3 to smithi08434663-49 2024-09-18T14:01:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:589: finishing write tid 4 to smithi08434663-49 2024-09-18T14:01:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 154 (ObjNum 239 snap 57 seq_num 239) dirty exists 2024-09-18T14:01:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:589: left oid 49 (ObjNum 239 snap 57 seq_num 239) 2024-09-18T14:01:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:590: finishing write tid 1 to smithi08434663-21 2024-09-18T14:01:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:589: done (2 left) 2024-09-18T14:01:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:592: write oid 39 current snap is 57 2024-09-18T14:01:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:592: seq_num 241 ranges {450487=541952,1636278=414209} 2024-09-18T14:01:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:592: writing smithi08434663-39 from 450487 to 992439 tid 1 2024-09-18T14:01:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:592: writing smithi08434663-39 from 1636278 to 2050487 tid 2 2024-09-18T14:01:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:593: snap_remove snap 50 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.281+0000 7f51f6ec9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.281+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f51f011c4e0 con 0x7f51f002aa40 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.281+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400859055 0 0) 0x7f51f011c4e0 con 0x7f51f002aa40 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.281+0000 7f51ecff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.281+0000 7f51ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.281+0000 7f51ecff9640 10 monclient(hunting): my global_id is 4709 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.281+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f004c4a0 con 0x7f51f002aa40 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1223745385 0 0) 0x7f51f004c4a0 con 0x7f51f002aa40 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51f011c4e0 con 0x7f51f002aa40 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f51dc004ea0 con 0x7f51f002aa40 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 10 monclient: dump: 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.306 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.306 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2965951740 0 0) 0x7f51f011c4e0 con 0x7f51f002aa40 2024-09-18T14:01:36.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f51f002ae10 con 0x7f51f002aa40 2024-09-18T14:01:36.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 628274098) 0x7f51dc006940 con 0x7f51f002aa40 2024-09-18T14:01:36.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:36.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:36.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 >> v1:172.21.15.45:6790/0 conn(0x7f51f002aa40 legacy=0x7f51f009a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.282+0000 7f51ecff9640 1 -- 172.21.15.45:0/1786215618 >> v1:172.21.15.45:6790/0 conn(0x7f51f002aa40 legacy=0x7f51f009a3d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:36.306 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:01:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:36.327 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:36.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.283+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1786215618 >> v1:172.21.15.84:6812/31478 conn(0x7f51d403ab00 legacy=0x7f51d403cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.283+0000 7f51f6ec9640 10 monclient: shutdown 2024-09-18T14:01:36.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.283+0000 7f51f6ec9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:36.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.283+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1786215618 >> v1:172.21.15.45:6792/0 conn(0x7f51f00979e0 legacy=0x7f51f01411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.283+0000 7f51f5466640 1 -- 172.21.15.45:0/1786215618 reap_dead start 2024-09-18T14:01:36.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.283+0000 7f51f5466640 1 -- 172.21.15.45:0/1786215618 reap_dead start 2024-09-18T14:01:36.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.283+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1786215618 shutdown_connections 2024-09-18T14:01:36.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.283+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1786215618 >> 172.21.15.45:0/1786215618 conn(0x7f51f00a2710 msgr2=0x7f51f0062fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:36.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.283+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1786215618 shutdown_connections 2024-09-18T14:01:36.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.283+0000 7f51f6ec9640 1 -- 172.21.15.45:0/1786215618 wait complete. 2024-09-18T14:01:36.330 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:36.330 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:01:36.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:01:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.427+0000 7f504299d640 10 monclient: get_monmap_and_config 2024-09-18T14:01:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.427+0000 7f504299d640 10 monclient: build_initial_monmap 2024-09-18T14:01:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.427+0000 7f504299d640 10 monclient: monmap: 2024-09-18T14:01:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:36.428139+0000 2024-09-18T14:01:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:36.428139+0000 2024-09-18T14:01:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:36.429 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.427+0000 7f504299d640 1 Processor -- start 2024-09-18T14:01:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.427+0000 7f504299d640 1 -- start start 2024-09-18T14:01:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f504299d640 10 monclient: init 2024-09-18T14:01:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f504299d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f504299d640 10 monclient: _add_conns ranks=[6,2,4,1,3,8,0,5,7] 2024-09-18T14:01:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f504299d640 10 monclient(hunting): picked mon.noname-d con 0x7f503c09f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:01:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f504299d640 10 monclient(hunting): picked mon.noname-b con 0x7f503c09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f504299d640 10 monclient(hunting): picked mon.noname-c con 0x7f503c09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f504299d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c05ed70 con 0x7f503c09fba0 2024-09-18T14:01:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f504299d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c09dfc0 con 0x7f503c09ff70 2024-09-18T14:01:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f504299d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c09dce0 con 0x7f503c09f7d0 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f504299d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f504299d640 10 monclient(hunting): authenticate will time out at 911.766521s 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f503b7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f503c09fba0 0x7f503c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43772/0 (socket says 172.21.15.45:43772) 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f5040f3a640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f503c09ff70 0x7f503c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36986/0 (socket says 172.21.15.45:36986) 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f503b7fe640 1 -- 172.21.15.45:0/406453220 learned_addr learned my addr 172.21.15.45:0/406453220 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f503bfff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f503c09f7d0 0x7f503c132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56160/0 (socket says 172.21.15.45:56160) 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151764751 0 0) 0x7f503c09dfc0 con 0x7f503c09ff70 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f503affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f503affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f503affd640 10 monclient(hunting): my global_id is 134266 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50240034e0 con 0x7f503c09ff70 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.428+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1264100089 0 0) 0x7f503c09dce0 con 0x7f503c09f7d0 2024-09-18T14:01:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient(hunting): my global_id is 4614 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f503c09dfc0 con 0x7f503c09f7d0 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878305505 0 0) 0x7f503c05ed70 con 0x7f503c09fba0 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient(hunting): my global_id is 4718 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f503c09dce0 con 0x7f503c09fba0 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1832270281 0 0) 0x7f50240034e0 con 0x7f503c09ff70 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 >> v1:172.21.15.45:6792/0 conn(0x7f503c09f7d0 legacy=0x7f503c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 >> v1:172.21.15.45:6790/0 conn(0x7f503c09fba0 legacy=0x7f503c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient: _finish_hunting 0 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 1 monclient: found mon.noname-c 2024-09-18T14:01:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c0a1c00 con 0x7f503c09ff70 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient: _finish_auth 0 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5034001ae0 con 0x7f503c09ff70 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient: dump: 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f504299d640 5 monclient: authenticate success, global_id 134266 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f504299d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5034001c40 con 0x7f503c09ff70 2024-09-18T14:01:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 1 -- 172.21.15.45:0/406453220 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5034004b20 con 0x7f503c09ff70 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f503affd640 10 monclient: dump: 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.444 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.444 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.444 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.444 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.444 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.444 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f504299d640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:36.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f504299d640 4 set_mon_vals no callback set 2024-09-18T14:01:36.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f504299d640 10 monclient: shutdown 2024-09-18T14:01:36.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f504299d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:36.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.429+0000 7f504299d640 1 -- 172.21.15.45:0/406453220 >> v1:172.21.15.45:6791/0 conn(0x7f503c09ff70 legacy=0x7f503c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.430+0000 7f504299d640 1 -- 172.21.15.45:0/406453220 shutdown_connections 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.430+0000 7f504299d640 1 -- 172.21.15.45:0/406453220 >> 172.21.15.45:0/406453220 conn(0x7f503c0630b0 msgr2=0x7f503c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.430+0000 7f504299d640 1 -- 172.21.15.45:0/406453220 shutdown_connections 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.430+0000 7f504299d640 1 -- 172.21.15.45:0/406453220 wait complete. 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.430+0000 7f504299d640 10 monclient: build_initial_monmap 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.430+0000 7f504299d640 10 monclient: monmap: 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.430+0000 7f504299d640 1 Processor -- start 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f504299d640 1 -- start start 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f504299d640 10 monclient: init 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f504299d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f504299d640 10 monclient: _add_conns ranks=[0,2,8,7,6,4,1,3,5] 2024-09-18T14:01:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f504299d640 10 monclient(hunting): picked mon.noname-a con 0x7f503c09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f504299d640 10 monclient(hunting): picked mon.noname-b con 0x7f503c09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f504299d640 10 monclient(hunting): picked mon.noname-e con 0x7f503c09ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f504299d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c09d380 con 0x7f503c09f7d0 2024-09-18T14:01:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f504299d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c11d610 con 0x7f503c09fba0 2024-09-18T14:01:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f504299d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c130480 con 0x7f503c09ff70 2024-09-18T14:01:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f504299d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f504299d640 10 monclient(hunting): authenticate will time out at 911.769490s 2024-09-18T14:01:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f503b7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f503c09fba0 0x7f503c142370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43788/0 (socket says 172.21.15.45:43788) 2024-09-18T14:01:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f503b7fe640 1 -- 172.21.15.45:0/4270255681 learned_addr learned my addr 172.21.15.45:0/4270255681 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638779398 0 0) 0x7f503c11d610 con 0x7f503c09fba0 2024-09-18T14:01:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 10 monclient(hunting): my global_id is 4727 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50280034e0 con 0x7f503c09fba0 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305742224 0 0) 0x7f503c130480 con 0x7f503c09ff70 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 10 monclient(hunting): my global_id is 4661 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f503c11d610 con 0x7f503c09ff70 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312825790 0 0) 0x7f502c001a50 con 0x7f503c09f7d0 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 10 monclient(hunting): my global_id is 4716 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.431+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f503c130480 con 0x7f503c09f7d0 2024-09-18T14:01:36.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 505244985 0 0) 0x7f50280034e0 con 0x7f503c09fba0 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f502c001a50 con 0x7f503c09fba0 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f50300016c0 con 0x7f503c09fba0 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient(hunting): dump: 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.449 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1637132812 0 0) 0x7f503c11d610 con 0x7f503c09ff70 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f50280034e0 con 0x7f503c09ff70 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4066444702 0 0) 0x7f503c130480 con 0x7f503c09f7d0 2024-09-18T14:01:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f503c11d610 con 0x7f503c09f7d0 2024-09-18T14:01:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5034001a20 con 0x7f503c09ff70 2024-09-18T14:01:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient(hunting): dump: 2024-09-18T14:01:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f502c001be0 con 0x7f503c09f7d0 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient(hunting): dump: 2024-09-18T14:01:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.453 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3543447457 0 0) 0x7f502c001a50 con 0x7f503c09fba0 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 >> v1:172.21.15.45:6793/0 conn(0x7f503c09ff70 legacy=0x7f503c149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 >> v1:172.21.15.45:6789/0 conn(0x7f503c09f7d0 legacy=0x7f503c141c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient: _finish_hunting 0 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 monclient: found mon.b 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c142a80 con 0x7f503c09fba0 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient: _finish_auth 0 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f504299d640 5 monclient: authenticate success, global_id 4727 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f504299d640 10 monclient: _renew_subs 2024-09-18T14:01:36.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f504299d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f504299d640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503c0a2560 con 0x7f503c09fba0 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f504299d640 10 monclient: _renew_subs 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f504299d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f504299d640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f503c11e330 con 0x7f503c09fba0 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5030001d20 con 0x7f503c09fba0 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5030004d90 con 0x7f503c09fba0 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f503a7fc640 4 set_mon_vals no callback set 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.432+0000 7f5023fff640 10 monclient: dump: 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.455 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.456 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f504299d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f504299d640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f501c006580 con 0x7f501c002830 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f503000fde0 con 0x7f503c09fba0 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 3911+0+0 (unknown 4286160998 0 0) 0x7f5030046b90 con 0x7f503c09fba0 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1833272515 0 0) 0x7f501c006580 con 0x7f501c002830 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 10 monclient(hunting): my global_id is 134275 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f502c001a50 con 0x7f501c002830 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2156605390 0 0) 0x7f502c001a50 con 0x7f501c002830 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f501c006580 con 0x7f501c002830 2024-09-18T14:01:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f502c004cd0 con 0x7f501c002830 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5040f3a640 10 monclient: get_auth_request con 0x7f502803aa70 auth_method 0 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.433+0000 7f5023fff640 10 monclient: dump: 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.434+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4237239683 0 0) 0x7f501c006580 con 0x7f501c002830 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.434+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f501c00a0a0 con 0x7f501c002830 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.434+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f502c0051a0 con 0x7f501c002830 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.434+0000 7f5023fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.434+0000 7f5023fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.434+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 >> v1:172.21.15.45:6791/0 conn(0x7f501c002830 legacy=0x7f501c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.434+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 >> v1:172.21.15.45:6791/0 conn(0x7f501c002830 legacy=0x7f501c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:36.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f504299d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:36.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f504299d640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f501c009610 con 0x7f501c008ff0 2024-09-18T14:01:36.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173812064 0 0) 0x7f501c009610 con 0x7f501c008ff0 2024-09-18T14:01:36.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 10 monclient(hunting): my global_id is 134284 2024-09-18T14:01:36.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f501c006580 con 0x7f501c008ff0 2024-09-18T14:01:36.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3528387528 0 0) 0x7f501c006580 con 0x7f501c008ff0 2024-09-18T14:01:36.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f501c009610 con 0x7f501c008ff0 2024-09-18T14:01:36.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f502c005a80 con 0x7f501c008ff0 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 10 monclient: dump: 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.462 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2264119514 0 0) 0x7f501c009610 con 0x7f501c008ff0 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.440+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f501c0098d0 con 0x7f501c008ff0 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 749016005) 0x7f502c005e20 con 0x7f501c008ff0 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f5023fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f5023fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 >> v1:172.21.15.45:6791/0 conn(0x7f501c008ff0 legacy=0x7f501c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f5023fff640 1 -- 172.21.15.45:0/4270255681 >> v1:172.21.15.45:6791/0 conn(0x7f501c008ff0 legacy=0x7f501c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:36.464 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:36.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:36.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:36.464 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:36.464 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:36.464 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:36.464 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:36.464 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:36.464 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:36.464 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:36.465 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:36.465 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:36.465 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:36.465 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:36.465 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:36.465 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.465 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:01:36.466 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:36.466 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:36.466 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:36.466 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.466 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.466 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.466 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.466 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.466 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.466 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.466 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.466 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.468 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:36.468 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:36.468 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:36.468 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:36.468 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:36.468 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:36.468 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:36.468 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:36.468 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:36.468 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:36.468 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:36.468 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:36.469 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:36.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:36.469 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:36.469 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:36.469 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:36.469 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.469 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.469 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.469 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.469 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.469 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.469 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.470 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:36.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:36.472 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.472 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.472 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:36.472 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.472 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.472 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.472 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:36.472 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:36.472 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.473 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.474 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.476 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:36.477 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:36.477 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.477 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.477 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.477 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:36.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:36.477 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.477 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.478 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.479 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.480 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.482 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.483 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:36.484 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.484 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.484 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:36.484 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f504299d640 1 -- 172.21.15.45:0/4270255681 >> v1:172.21.15.84:6812/31478 conn(0x7f502803aa70 legacy=0x7f502803cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f504299d640 10 monclient: shutdown 2024-09-18T14:01:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f504299d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f504299d640 1 -- 172.21.15.45:0/4270255681 >> v1:172.21.15.45:6790/0 conn(0x7f503c09fba0 legacy=0x7f503c142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f5040f3a640 1 -- 172.21.15.45:0/4270255681 reap_dead start 2024-09-18T14:01:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f5040f3a640 1 -- 172.21.15.45:0/4270255681 reap_dead start 2024-09-18T14:01:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f504299d640 1 -- 172.21.15.45:0/4270255681 shutdown_connections 2024-09-18T14:01:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f504299d640 1 -- 172.21.15.45:0/4270255681 >> 172.21.15.45:0/4270255681 conn(0x7f503c0630b0 msgr2=0x7f503c131e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f504299d640 1 -- 172.21.15.45:0/4270255681 shutdown_connections 2024-09-18T14:01:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.441+0000 7f504299d640 1 -- 172.21.15.45:0/4270255681 wait complete. 2024-09-18T14:01:36.486 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:36.486 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:01:36.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:01:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.583+0000 7f1542cc9640 10 monclient: get_monmap_and_config 2024-09-18T14:01:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.583+0000 7f1542cc9640 10 monclient: build_initial_monmap 2024-09-18T14:01:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.583+0000 7f1542cc9640 10 monclient: monmap: 2024-09-18T14:01:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:36.584031+0000 2024-09-18T14:01:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:36.584031+0000 2024-09-18T14:01:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:36.585 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.583+0000 7f1542cc9640 1 Processor -- start 2024-09-18T14:01:36.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.583+0000 7f1542cc9640 1 -- start start 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.583+0000 7f1542cc9640 10 monclient: init 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f1542cc9640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f1542cc9640 10 monclient: _add_conns ranks=[0,4,1,6,7,2,5,8,3] 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f1542cc9640 10 monclient(hunting): picked mon.noname-a con 0x7f153c06e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f1542cc9640 10 monclient(hunting): picked mon.noname-c con 0x7f153c0a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f1542cc9640 10 monclient(hunting): picked mon.noname-f con 0x7f153c0a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f1542cc9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c05ed70 con 0x7f153c06e810 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f1542cc9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c097620 con 0x7f153c0a1080 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f1542cc9640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c097340 con 0x7f153c0a0cb0 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f1542cc9640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f1542cc9640 10 monclient(hunting): authenticate will time out at 911.922430s 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f1540a65640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f153c06e810 0x7f153c0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51484/0 (socket says 172.21.15.45:51484) 2024-09-18T14:01:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f1540a65640 1 -- 172.21.15.45:0/1165773703 learned_addr learned my addr 172.21.15.45:0/1165773703 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884079815 0 0) 0x7f153c05ed70 con 0x7f153c06e810 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 10 monclient(hunting): my global_id is 4725 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15240034e0 con 0x7f153c06e810 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418361364 0 0) 0x7f153c097340 con 0x7f153c0a0cb0 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 10 monclient(hunting): my global_id is 134293 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f153c05ed70 con 0x7f153c0a0cb0 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3770508954 0 0) 0x7f153c097620 con 0x7f153c0a1080 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 10 monclient(hunting): my global_id is 4618 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.584+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f153c097340 con 0x7f153c0a1080 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 674045123 0 0) 0x7f15240034e0 con 0x7f153c06e810 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 >> v1:172.21.15.45:6791/0 conn(0x7f153c0a0cb0 legacy=0x7f153c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 >> v1:172.21.15.84:6789/0 conn(0x7f153c0a1080 legacy=0x7f153c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 1 monclient: found mon.noname-a 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c0a1660 con 0x7f153c06e810 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 10 monclient: _finish_auth 0 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f152c001830 con 0x7f153c06e810 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 10 monclient: dump: 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f1542cc9640 5 monclient: authenticate success, global_id 4725 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f1542cc9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f152c001a10 con 0x7f153c06e810 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f1542cc9640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f1542cc9640 4 set_mon_vals no callback set 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 1 -- 172.21.15.45:0/1165773703 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f152c004ba0 con 0x7f153c06e810 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f153b7fe640 10 monclient: dump: 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f1542cc9640 10 monclient: shutdown 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f1542cc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f1542cc9640 1 -- 172.21.15.45:0/1165773703 >> v1:172.21.15.45:6789/0 conn(0x7f153c06e810 legacy=0x7f153c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f1542cc9640 1 -- 172.21.15.45:0/1165773703 shutdown_connections 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f1542cc9640 1 -- 172.21.15.45:0/1165773703 >> 172.21.15.45:0/1165773703 conn(0x7f153c0a5bb0 msgr2=0x7f153c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f1542cc9640 1 -- 172.21.15.45:0/1165773703 shutdown_connections 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.585+0000 7f1542cc9640 1 -- 172.21.15.45:0/1165773703 wait complete. 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.586+0000 7f1542cc9640 10 monclient: build_initial_monmap 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.586+0000 7f1542cc9640 10 monclient: monmap: 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:36.601 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.586+0000 7f1542cc9640 1 Processor -- start 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.586+0000 7f1542cc9640 1 -- start start 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.586+0000 7f1542cc9640 10 monclient: init 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1542cc9640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1542cc9640 10 monclient: _add_conns ranks=[2,0,7,3,4,5,8,1,6] 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1542cc9640 10 monclient(hunting): picked mon.noname-b con 0x7f153c06e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1542cc9640 10 monclient(hunting): picked mon.noname-a con 0x7f153c0a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1542cc9640 10 monclient(hunting): picked mon.noname-i con 0x7f153c0a1080 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1542cc9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c1227b0 con 0x7f153c0a0cb0 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1542cc9640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c13e8a0 con 0x7f153c06e810 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1542cc9640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c13db40 con 0x7f153c0a1080 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1542cc9640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1542cc9640 10 monclient(hunting): authenticate will time out at 911.925301s 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1540a65640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f153c06e810 0x7f153c13d430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43792/0 (socket says 172.21.15.45:43792) 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1540a65640 1 -- 172.21.15.45:0/1896952759 learned_addr learned my addr 172.21.15.45:0/1896952759 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f153bfff640 1 --1- 172.21.15.45:0/1896952759 >> v1:172.21.15.45:6789/0 conn(0x7f153c0a0cb0 0x7f153c146680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51494/0 (socket says 172.21.15.45:51494) 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987289108 0 0) 0x7f153c1227b0 con 0x7f153c0a0cb0 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): my global_id is 4734 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15280034e0 con 0x7f153c0a0cb0 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676854915 0 0) 0x7f153c13e8a0 con 0x7f153c06e810 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): my global_id is 4736 2024-09-18T14:01:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f153c1227b0 con 0x7f153c06e810 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271741689 0 0) 0x7f153c13db40 con 0x7f153c0a1080 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): my global_id is 4687 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f153c13e8a0 con 0x7f153c0a1080 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2451096672 0 0) 0x7f15280034e0 con 0x7f153c0a0cb0 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f153c13db40 con 0x7f153c0a0cb0 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1530001d90 con 0x7f153c0a0cb0 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): dump: 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.605 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2862407992 0 0) 0x7f153c1227b0 con 0x7f153c06e810 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f15280034e0 con 0x7f153c06e810 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f152c0017f0 con 0x7f153c06e810 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): dump: 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2793425421 0 0) 0x7f153c13e8a0 con 0x7f153c0a1080 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f153c1227b0 con 0x7f153c0a1080 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1534001d80 con 0x7f153c0a1080 2024-09-18T14:01:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.587+0000 7f1538ff9640 10 monclient(hunting): dump: 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.609 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2005305801 0 0) 0x7f15280034e0 con 0x7f153c06e810 2024-09-18T14:01:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 >> v1:172.21.15.84:6792/0 conn(0x7f153c0a1080 legacy=0x7f153c149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 >> v1:172.21.15.45:6789/0 conn(0x7f153c0a0cb0 legacy=0x7f153c146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:01:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 1 monclient: found mon.b 2024-09-18T14:01:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c1224d0 con 0x7f153c06e810 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 10 monclient: _finish_auth 0 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1542cc9640 5 monclient: authenticate success, global_id 4736 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1542cc9640 10 monclient: _renew_subs 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1542cc9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1542cc9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c122a90 con 0x7f153c06e810 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1542cc9640 10 monclient: _renew_subs 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1542cc9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1542cc9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c1217a0 con 0x7f153c06e810 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f152c004a50 con 0x7f153c06e810 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f152c005160 con 0x7f153c06e810 2024-09-18T14:01:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f153affd640 4 set_mon_vals no callback set 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.588+0000 7f1538ff9640 10 monclient: dump: 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f152c010130 con 0x7f153c06e810 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1542cc9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1542cc9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f151c006580 con 0x7f151c002830 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 3911+0+0 (unknown 4286160998 0 0) 0x7f152c046ae0 con 0x7f153c06e810 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460807094 0 0) 0x7f151c006580 con 0x7f151c002830 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 10 monclient(hunting): my global_id is 4623 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15280034e0 con 0x7f151c002830 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1541266640 10 monclient: get_auth_request con 0x7f152803a9d0 auth_method 0 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2956876101 0 0) 0x7f15280034e0 con 0x7f151c002830 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f151c006580 con 0x7f151c002830 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1530002810 con 0x7f151c002830 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 10 monclient: dump: 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4129599791 0 0) 0x7f151c006580 con 0x7f151c002830 2024-09-18T14:01:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.589+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f151c00a0a0 con 0x7f151c002830 2024-09-18T14:01:36.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.590+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1530005310 con 0x7f151c002830 2024-09-18T14:01:36.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.590+0000 7f1538ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:36.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.590+0000 7f1538ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:36.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.590+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 >> v1:172.21.15.45:6792/0 conn(0x7f151c002830 legacy=0x7f151c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.590+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 >> v1:172.21.15.45:6792/0 conn(0x7f151c002830 legacy=0x7f151c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.595+0000 7f1542cc9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.595+0000 7f1542cc9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f151c009660 con 0x7f151c009040 2024-09-18T14:01:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464911891 0 0) 0x7f151c009660 con 0x7f151c009040 2024-09-18T14:01:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 10 monclient(hunting): my global_id is 4632 2024-09-18T14:01:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f151c006580 con 0x7f151c009040 2024-09-18T14:01:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 555282699 0 0) 0x7f151c006580 con 0x7f151c009040 2024-09-18T14:01:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f151c009660 con 0x7f151c009040 2024-09-18T14:01:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1530005870 con 0x7f151c009040 2024-09-18T14:01:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 10 monclient: dump: 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.619 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2291634284 0 0) 0x7f151c009660 con 0x7f151c009040 2024-09-18T14:01:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f151c009920 con 0x7f151c009040 2024-09-18T14:01:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 321255259) 0x7f1530007c50 con 0x7f151c009040 2024-09-18T14:01:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 >> v1:172.21.15.45:6792/0 conn(0x7f151c009040 legacy=0x7f151c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.596+0000 7f1538ff9640 1 -- 172.21.15.45:0/1896952759 >> v1:172.21.15.45:6792/0 conn(0x7f151c009040 legacy=0x7f151c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:36.619 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:36.641 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.597+0000 7f1542cc9640 1 -- 172.21.15.45:0/1896952759 >> v1:172.21.15.84:6812/31478 conn(0x7f152803a9d0 legacy=0x7f152803ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.597+0000 7f1542cc9640 10 monclient: shutdown 2024-09-18T14:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.597+0000 7f1542cc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.597+0000 7f1542cc9640 1 -- 172.21.15.45:0/1896952759 >> v1:172.21.15.45:6790/0 conn(0x7f153c06e810 legacy=0x7f153c13d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.597+0000 7f1541266640 1 -- 172.21.15.45:0/1896952759 reap_dead start 2024-09-18T14:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.597+0000 7f1541266640 1 -- 172.21.15.45:0/1896952759 reap_dead start 2024-09-18T14:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.597+0000 7f1542cc9640 1 -- 172.21.15.45:0/1896952759 shutdown_connections 2024-09-18T14:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.597+0000 7f1542cc9640 1 -- 172.21.15.45:0/1896952759 >> 172.21.15.45:0/1896952759 conn(0x7f153c0a5bb0 msgr2=0x7f153c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.597+0000 7f1542cc9640 1 -- 172.21.15.45:0/1896952759 shutdown_connections 2024-09-18T14:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.597+0000 7f1542cc9640 1 -- 172.21.15.45:0/1896952759 wait complete. 2024-09-18T14:01:36.644 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:36.644 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:01:36.644 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:01:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.741+0000 7f05ccf09640 10 monclient: get_monmap_and_config 2024-09-18T14:01:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.741+0000 7f05ccf09640 10 monclient: build_initial_monmap 2024-09-18T14:01:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.741+0000 7f05ccf09640 10 monclient: monmap: 2024-09-18T14:01:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:36.742522+0000 2024-09-18T14:01:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:36.742522+0000 2024-09-18T14:01:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:36.743 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 1 Processor -- start 2024-09-18T14:01:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 1 -- start start 2024-09-18T14:01:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 10 monclient: init 2024-09-18T14:01:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 10 monclient: _add_conns ranks=[0,5,2,7,4,8,1,3,6] 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 10 monclient(hunting): picked mon.noname-a con 0x7f05c809f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 10 monclient(hunting): picked mon.noname-h con 0x7f05c809fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 10 monclient(hunting): picked mon.noname-b con 0x7f05c809ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c805ed70 con 0x7f05c809f7d0 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c809dfc0 con 0x7f05c809ff70 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c809dce0 con 0x7f05c809fba0 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.742+0000 7f05ccf09640 10 monclient(hunting): authenticate will time out at 912.080969s 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c659c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f05c809f7d0 0x7f05c8132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51510/0 (socket says 172.21.15.45:51510) 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c659c640 1 -- 172.21.15.45:0/4286980596 learned_addr learned my addr 172.21.15.45:0/4286980596 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942029989 0 0) 0x7f05c805ed70 con 0x7f05c809f7d0 2024-09-18T14:01:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient(hunting): my global_id is 4743 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05b00034e0 con 0x7f05c809f7d0 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558741357 0 0) 0x7f05c809dfc0 con 0x7f05c809ff70 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient(hunting): my global_id is 4745 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05c805ed70 con 0x7f05c809ff70 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130893881 0 0) 0x7f05c809dce0 con 0x7f05c809fba0 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient(hunting): my global_id is 4604 2024-09-18T14:01:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05c809dfc0 con 0x7f05c809fba0 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 386551578 0 0) 0x7f05b00034e0 con 0x7f05c809f7d0 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 >> v1:172.21.15.84:6791/0 conn(0x7f05c809fba0 legacy=0x7f05c8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 >> v1:172.21.15.45:6790/0 conn(0x7f05c809ff70 legacy=0x7f05c813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient: _finish_hunting 0 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 monclient: found mon.noname-a 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c80a1c00 con 0x7f05c809f7d0 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient: _finish_auth 0 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05ccf09640 5 monclient: authenticate success, global_id 4743 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f05b8001830 con 0x7f05c809f7d0 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05ccf09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient: dump: 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05ccf09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f05b8001a10 con 0x7f05c809f7d0 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05ccf09640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 1 -- 172.21.15.45:0/4286980596 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f05b8004ba0 con 0x7f05c809f7d0 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05ccf09640 4 set_mon_vals no callback set 2024-09-18T14:01:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.743+0000 7f05c559a640 10 monclient: dump: 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.744+0000 7f05ccf09640 10 monclient: shutdown 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.744+0000 7f05ccf09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.744+0000 7f05ccf09640 1 -- 172.21.15.45:0/4286980596 >> v1:172.21.15.45:6789/0 conn(0x7f05c809f7d0 legacy=0x7f05c8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.744+0000 7f05ccf09640 1 -- 172.21.15.45:0/4286980596 shutdown_connections 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.744+0000 7f05ccf09640 1 -- 172.21.15.45:0/4286980596 >> 172.21.15.45:0/4286980596 conn(0x7f05c80630b0 msgr2=0x7f05c80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.744+0000 7f05ccf09640 1 -- 172.21.15.45:0/4286980596 shutdown_connections 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.744+0000 7f05ccf09640 1 -- 172.21.15.45:0/4286980596 wait complete. 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.744+0000 7f05ccf09640 10 monclient: build_initial_monmap 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.744+0000 7f05ccf09640 10 monclient: monmap: 2024-09-18T14:01:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:36.759 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 1 Processor -- start 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 1 -- start start 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 10 monclient: init 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 10 monclient: _add_conns ranks=[5,4,2,8,3,6,7,1,0] 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 10 monclient(hunting): picked mon.noname-h con 0x7f05c809f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 10 monclient(hunting): picked mon.noname-c con 0x7f05c809fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 10 monclient(hunting): picked mon.noname-b con 0x7f05c809ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c80a2560 con 0x7f05c809ff70 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c809d310 con 0x7f05c809fba0 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c81202a0 con 0x7f05c809f7d0 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05ccf09640 10 monclient(hunting): authenticate will time out at 912.083684s 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05c5d9b640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f05c809fba0 0x7f05c8131c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:37014/0 (socket says 172.21.15.45:37014) 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05c6d9d640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f05c809ff70 0x7f05c8149da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43816/0 (socket says 172.21.15.45:43816) 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.745+0000 7f05c5d9b640 1 -- 172.21.15.45:0/2995203719 learned_addr learned my addr 172.21.15.45:0/2995203719 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134999847 0 0) 0x7f05c80a2560 con 0x7f05c809ff70 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): my global_id is 4754 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05a8003510 con 0x7f05c809ff70 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062782972 0 0) 0x7f05c809d310 con 0x7f05c809fba0 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): my global_id is 134302 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05c80a2560 con 0x7f05c809fba0 2024-09-18T14:01:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6397983 0 0) 0x7f05c81202a0 con 0x7f05c809f7d0 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): my global_id is 4613 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05c809d310 con 0x7f05c809f7d0 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1492692138 0 0) 0x7f05a8003510 con 0x7f05c809ff70 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05c81202a0 con 0x7f05c809ff70 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f05c0001ce0 con 0x7f05c809ff70 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): dump: 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4010258999 0 0) 0x7f05c80a2560 con 0x7f05c809fba0 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05a8003510 con 0x7f05c809fba0 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 494730394 0 0) 0x7f05c809d310 con 0x7f05c809f7d0 2024-09-18T14:01:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05c80a2560 con 0x7f05c809f7d0 2024-09-18T14:01:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f05bc001bb0 con 0x7f05c809fba0 2024-09-18T14:01:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient(hunting): dump: 2024-09-18T14:01:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.765 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2920525751 0 0) 0x7f05c81202a0 con 0x7f05c809ff70 2024-09-18T14:01:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 >> v1:172.21.15.84:6791/0 conn(0x7f05c809f7d0 legacy=0x7f05c8141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 >> v1:172.21.15.45:6791/0 conn(0x7f05c809fba0 legacy=0x7f05c8131c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient: _finish_hunting 0 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 monclient: found mon.b 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c8142860 con 0x7f05c809ff70 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient: _finish_auth 0 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05ccf09640 5 monclient: authenticate success, global_id 4754 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05ccf09640 10 monclient: _renew_subs 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05ccf09640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05ccf09640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c8120990 con 0x7f05c809ff70 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05ccf09640 10 monclient: _renew_subs 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05ccf09640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05ccf09640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c8120e70 con 0x7f05c809ff70 2024-09-18T14:01:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f05c0002250 con 0x7f05c809ff70 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f05c0005090 con 0x7f05c809ff70 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05c4d99640 4 set_mon_vals no callback set 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.746+0000 7f05b67fc640 10 monclient: dump: 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.768 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.747+0000 7f05ccf09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.747+0000 7f05ccf09640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f05ac006580 con 0x7f05ac002830 2024-09-18T14:01:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.747+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f05c0010060 con 0x7f05c809ff70 2024-09-18T14:01:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.747+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 3911+0+0 (unknown 4286160998 0 0) 0x7f05c0046be0 con 0x7f05c809ff70 2024-09-18T14:01:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.747+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581812235 0 0) 0x7f05ac006580 con 0x7f05ac002830 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.747+0000 7f05b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.747+0000 7f05b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.747+0000 7f05b67fc640 10 monclient(hunting): my global_id is 4670 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.747+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05c81202a0 con 0x7f05ac002830 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05c5d9b640 10 monclient: get_auth_request con 0x7f05a803a8d0 auth_method 0 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1620288364 0 0) 0x7f05c81202a0 con 0x7f05ac002830 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05ac006580 con 0x7f05ac002830 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f05b80010b0 con 0x7f05ac002830 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 10 monclient: dump: 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.770 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 20048038 0 0) 0x7f05ac006580 con 0x7f05ac002830 2024-09-18T14:01:36.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05ac00a0a0 con 0x7f05ac002830 2024-09-18T14:01:36.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f05b8005160 con 0x7f05ac002830 2024-09-18T14:01:36.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:36.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:36.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 >> v1:172.21.15.45:6793/0 conn(0x7f05ac002830 legacy=0x7f05ac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.748+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 >> v1:172.21.15.45:6793/0 conn(0x7f05ac002830 legacy=0x7f05ac004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05ccf09640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05ccf09640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f05ac009660 con 0x7f05ac009040 2024-09-18T14:01:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235823028 0 0) 0x7f05ac009660 con 0x7f05ac009040 2024-09-18T14:01:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 10 monclient(hunting): my global_id is 4679 2024-09-18T14:01:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05ac006580 con 0x7f05ac009040 2024-09-18T14:01:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4136972320 0 0) 0x7f05ac006580 con 0x7f05ac009040 2024-09-18T14:01:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05ac009660 con 0x7f05ac009040 2024-09-18T14:01:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f05b80058c0 con 0x7f05ac009040 2024-09-18T14:01:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 10 monclient: dump: 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.775 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.775 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.775 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.775 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4131875557 0 0) 0x7f05ac009660 con 0x7f05ac009040 2024-09-18T14:01:36.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.754+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f05ac009920 con 0x7f05ac009040 2024-09-18T14:01:36.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7388 (unknown 0 0 2141247612) 0x7f05b8007cc0 con 0x7f05ac009040 2024-09-18T14:01:36.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05b67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:36.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05b67fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:36.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 >> v1:172.21.15.45:6793/0 conn(0x7f05ac009040 legacy=0x7f05ac008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05b67fc640 1 -- 172.21.15.45:0/2995203719 >> v1:172.21.15.45:6793/0 conn(0x7f05ac009040 legacy=0x7f05ac008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:36.776 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:01:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:36.796 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:36.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05ccf09640 1 -- 172.21.15.45:0/2995203719 >> v1:172.21.15.84:6812/31478 conn(0x7f05a803a8d0 legacy=0x7f05a803cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05ccf09640 10 monclient: shutdown 2024-09-18T14:01:36.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05ccf09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:36.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05ccf09640 1 -- 172.21.15.45:0/2995203719 >> v1:172.21.15.45:6790/0 conn(0x7f05c809ff70 legacy=0x7f05c8149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05c6d9d640 1 -- 172.21.15.45:0/2995203719 reap_dead start 2024-09-18T14:01:36.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05c6d9d640 1 -- 172.21.15.45:0/2995203719 reap_dead start 2024-09-18T14:01:36.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05ccf09640 1 -- 172.21.15.45:0/2995203719 shutdown_connections 2024-09-18T14:01:36.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05ccf09640 1 -- 172.21.15.45:0/2995203719 >> 172.21.15.45:0/2995203719 conn(0x7f05c80630b0 msgr2=0x7f05c80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:36.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05ccf09640 1 -- 172.21.15.45:0/2995203719 shutdown_connections 2024-09-18T14:01:36.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.755+0000 7f05ccf09640 1 -- 172.21.15.45:0/2995203719 wait complete. 2024-09-18T14:01:36.798 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:36.798 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:01:36.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:01:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.894+0000 7f752004f640 10 monclient: get_monmap_and_config 2024-09-18T14:01:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.894+0000 7f752004f640 10 monclient: build_initial_monmap 2024-09-18T14:01:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.894+0000 7f752004f640 10 monclient: monmap: 2024-09-18T14:01:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:36.895691+0000 2024-09-18T14:01:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:36.895691+0000 2024-09-18T14:01:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:36.896 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.895+0000 7f752004f640 1 Processor -- start 2024-09-18T14:01:36.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.895+0000 7f752004f640 1 -- start start 2024-09-18T14:01:36.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.895+0000 7f752004f640 10 monclient: init 2024-09-18T14:01:36.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:36.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 10 monclient: _add_conns ranks=[8,5,3,4,7,1,6,2,0] 2024-09-18T14:01:36.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 10 monclient(hunting): picked mon.noname-e con 0x7f751809f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:36.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 10 monclient(hunting): picked mon.noname-h con 0x7f751809fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 10 monclient(hunting): picked mon.noname-g con 0x7f751809ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751805ed70 con 0x7f751809ff70 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751809dfc0 con 0x7f751809fba0 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751809dce0 con 0x7f751809f7d0 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 10 monclient(hunting): authenticate will time out at 912.234244s 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751ddeb640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f751809f7d0 0x7f7518132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:32852/0 (socket says 172.21.15.45:32852) 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751ddeb640 1 -- 172.21.15.45:0/1845049626 learned_addr learned my addr 172.21.15.45:0/1845049626 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 1 -- 172.21.15.45:0/1845049626 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223181686 0 0) 0x7f751809dce0 con 0x7f751809f7d0 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient(hunting): my global_id is 4688 2024-09-18T14:01:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 1 -- 172.21.15.45:0/1845049626 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75040034e0 con 0x7f751809f7d0 2024-09-18T14:01:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 1 -- 172.21.15.45:0/1845049626 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3695172897 0 0) 0x7f751809dfc0 con 0x7f751809fba0 2024-09-18T14:01:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient(hunting): my global_id is 4622 2024-09-18T14:01:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 1 -- 172.21.15.45:0/1845049626 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f751809dce0 con 0x7f751809fba0 2024-09-18T14:01:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 1 -- 172.21.15.45:0/1845049626 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2306145835 0 0) 0x7f75040034e0 con 0x7f751809f7d0 2024-09-18T14:01:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 1 -- 172.21.15.45:0/1845049626 >> v1:172.21.15.84:6791/0 conn(0x7f751809fba0 legacy=0x7f7518137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 1 -- 172.21.15.45:0/1845049626 >> v1:172.21.15.84:6790/0 conn(0x7f751809ff70 legacy=0x7f751813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient: _finish_hunting 0 2024-09-18T14:01:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 1 monclient: found mon.noname-e 2024-09-18T14:01:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 1 -- 172.21.15.45:0/1845049626 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75180a1c00 con 0x7f751809f7d0 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient: _finish_auth 0 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 5 monclient: authenticate success, global_id 4688 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 1 -- 172.21.15.45:0/1845049626 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7510001830 con 0x7f751809f7d0 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient: dump: 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:36.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 1 -- 172.21.15.45:0/1845049626 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7510001990 con 0x7f751809f7d0 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 1 -- 172.21.15.45:0/1845049626 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7510004ba0 con 0x7f751809f7d0 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f752004f640 4 set_mon_vals no callback set 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.896+0000 7f751cde9640 10 monclient: dump: 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.911 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.897+0000 7f752004f640 10 monclient: shutdown 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.897+0000 7f752004f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.897+0000 7f752004f640 1 -- 172.21.15.45:0/1845049626 >> v1:172.21.15.45:6793/0 conn(0x7f751809f7d0 legacy=0x7f7518132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.897+0000 7f752004f640 1 -- 172.21.15.45:0/1845049626 shutdown_connections 2024-09-18T14:01:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.897+0000 7f752004f640 1 -- 172.21.15.45:0/1845049626 >> 172.21.15.45:0/1845049626 conn(0x7f75180630b0 msgr2=0x7f75180634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.897+0000 7f752004f640 1 -- 172.21.15.45:0/1845049626 shutdown_connections 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.897+0000 7f752004f640 1 -- 172.21.15.45:0/1845049626 wait complete. 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 10 monclient: build_initial_monmap 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 10 monclient: monmap: 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 1 Processor -- start 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 1 -- start start 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 10 monclient: init 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 10 monclient: _add_conns ranks=[2,7,4,0,3,5,1,6,8] 2024-09-18T14:01:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 10 monclient(hunting): picked mon.noname-b con 0x7f751809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 10 monclient(hunting): picked mon.noname-i con 0x7f751809fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 10 monclient(hunting): picked mon.noname-c con 0x7f751809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751809d310 con 0x7f751809f7d0 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7518121f40 con 0x7f751809ff70 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75180a2560 con 0x7f751809fba0 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f752004f640 10 monclient(hunting): authenticate will time out at 912.236880s 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f751ddeb640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f751809f7d0 0x7f7518141840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43820/0 (socket says 172.21.15.45:43820) 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.898+0000 7f751ddeb640 1 -- 172.21.15.45:0/2449994232 learned_addr learned my addr 172.21.15.45:0/2449994232 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456974244 0 0) 0x7f7518121f40 con 0x7f751809ff70 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): my global_id is 134311 2024-09-18T14:01:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74fc0034e0 con 0x7f751809ff70 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526073928 0 0) 0x7f751809d310 con 0x7f751809f7d0 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): my global_id is 4763 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7518121f40 con 0x7f751809f7d0 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639709262 0 0) 0x7f75180a2560 con 0x7f751809fba0 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): my global_id is 4696 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f751809d310 con 0x7f751809fba0 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2615446246 0 0) 0x7f74fc0034e0 con 0x7f751809ff70 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f75180a2560 con 0x7f751809ff70 2024-09-18T14:01:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7514002c20 con 0x7f751809ff70 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): dump: 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2813609282 0 0) 0x7f7518121f40 con 0x7f751809f7d0 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74fc0034e0 con 0x7f751809f7d0 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f75100017b0 con 0x7f751809f7d0 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): dump: 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3785903061 0 0) 0x7f751809d310 con 0x7f751809fba0 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7518121f40 con 0x7f751809fba0 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7508001aa0 con 0x7f751809fba0 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient(hunting): dump: 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.921 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2441167073 0 0) 0x7f75180a2560 con 0x7f751809ff70 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 >> v1:172.21.15.84:6792/0 conn(0x7f751809fba0 legacy=0x7f7518131a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 >> v1:172.21.15.45:6790/0 conn(0x7f751809f7d0 legacy=0x7f7518141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient: _finish_hunting 0 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 monclient: found mon.c 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75181200a0 con 0x7f751809ff70 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient: _finish_auth 0 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f752004f640 5 monclient: authenticate success, global_id 134311 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f752004f640 10 monclient: _renew_subs 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f752004f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f752004f640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7518121bd0 con 0x7f751809ff70 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7514001250 con 0x7f751809ff70 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7514001800 con 0x7f751809ff70 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.899+0000 7f750dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.900+0000 7f750dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.900+0000 7f750dffb640 10 monclient: dump: 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.900+0000 7f750ffff640 4 set_mon_vals no callback set 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.900+0000 7f752004f640 10 monclient: _renew_subs 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.900+0000 7f752004f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.900+0000 7f752004f640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7518141f50 con 0x7f751809ff70 2024-09-18T14:01:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.900+0000 7f752004f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.900+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f75140019e0 con 0x7f751809ff70 2024-09-18T14:01:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.900+0000 7f752004f640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f751804c4a0 con 0x7f7518099a40 2024-09-18T14:01:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 3911+0+0 (unknown 4286160998 0 0) 0x7f75140443a0 con 0x7f751809ff70 2024-09-18T14:01:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f751d5ea640 10 monclient: get_auth_request con 0x7f74fc03a900 auth_method 0 2024-09-18T14:01:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258407780 0 0) 0x7f751804c4a0 con 0x7f7518099a40 2024-09-18T14:01:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 10 monclient(hunting): my global_id is 4627 2024-09-18T14:01:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75180a2560 con 0x7f7518099a40 2024-09-18T14:01:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1083289730 0 0) 0x7f75180a2560 con 0x7f7518099a40 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f751804c4a0 con 0x7f7518099a40 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7510001250 con 0x7f7518099a40 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 10 monclient: dump: 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.927 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3790502725 0 0) 0x7f751804c4a0 con 0x7f7518099a40 2024-09-18T14:01:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.901+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f751804b720 con 0x7f7518099a40 2024-09-18T14:01:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.902+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7510007300 con 0x7f7518099a40 2024-09-18T14:01:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.902+0000 7f750dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:36.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.902+0000 7f750dffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:36.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.902+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 >> v1:172.21.15.84:6789/0 conn(0x7f7518099a40 legacy=0x7f7518099e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.902+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 >> v1:172.21.15.84:6789/0 conn(0x7f7518099a40 legacy=0x7f7518099e10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f752004f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f752004f640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7518071550 con 0x7f751809a520 2024-09-18T14:01:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153345563 0 0) 0x7f7518071550 con 0x7f751809a520 2024-09-18T14:01:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 10 monclient(hunting): my global_id is 4636 2024-09-18T14:01:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f751804c4a0 con 0x7f751809a520 2024-09-18T14:01:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1397083516 0 0) 0x7f751804c4a0 con 0x7f751809a520 2024-09-18T14:01:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7518071550 con 0x7f751809a520 2024-09-18T14:01:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7510005950 con 0x7f751809a520 2024-09-18T14:01:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:01:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 10 monclient: dump: 2024-09-18T14:01:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:36.932 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1495312787 0 0) 0x7f7518071550 con 0x7f751809a520 2024-09-18T14:01:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.908+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f751809d670 con 0x7f751809a520 2024-09-18T14:01:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 830002195) 0x7f751000a2f0 con 0x7f751809a520 2024-09-18T14:01:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f750dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f750dffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 >> v1:172.21.15.84:6789/0 conn(0x7f751809a520 legacy=0x7f751814f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f750dffb640 1 -- 172.21.15.45:0/2449994232 >> v1:172.21.15.84:6789/0 conn(0x7f751809a520 legacy=0x7f751814f2f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:36.933 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.934 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:36.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:36.953 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:36.953 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f752004f640 1 -- 172.21.15.45:0/2449994232 >> v1:172.21.15.84:6812/31478 conn(0x7f74fc03a900 legacy=0x7f74fc03cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f752004f640 10 monclient: shutdown 2024-09-18T14:01:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f752004f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f752004f640 1 -- 172.21.15.45:0/2449994232 >> v1:172.21.15.45:6791/0 conn(0x7f751809ff70 legacy=0x7f7518149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f751e5ec640 1 -- 172.21.15.45:0/2449994232 reap_dead start 2024-09-18T14:01:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f751e5ec640 1 -- 172.21.15.45:0/2449994232 reap_dead start 2024-09-18T14:01:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f752004f640 1 -- 172.21.15.45:0/2449994232 shutdown_connections 2024-09-18T14:01:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f752004f640 1 -- 172.21.15.45:0/2449994232 >> 172.21.15.45:0/2449994232 conn(0x7f75180630b0 msgr2=0x7f75180a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.909+0000 7f752004f640 1 -- 172.21.15.45:0/2449994232 shutdown_connections 2024-09-18T14:01:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:36.910+0000 7f752004f640 1 -- 172.21.15.45:0/2449994232 wait complete. 2024-09-18T14:01:36.955 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:36.955 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:01:36.955 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:01:37.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.052+0000 7efe35b09640 10 monclient: get_monmap_and_config 2024-09-18T14:01:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.052+0000 7efe35b09640 10 monclient: build_initial_monmap 2024-09-18T14:01:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.052+0000 7efe35b09640 10 monclient: monmap: 2024-09-18T14:01:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:37.053369+0000 2024-09-18T14:01:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:37.053369+0000 2024-09-18T14:01:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:37.055 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 1 Processor -- start 2024-09-18T14:01:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 1 -- start start 2024-09-18T14:01:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 10 monclient: init 2024-09-18T14:01:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 10 monclient: _add_conns ranks=[0,4,8,5,3,7,1,6,2] 2024-09-18T14:01:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 10 monclient(hunting): picked mon.noname-a con 0x7efe3009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 10 monclient(hunting): picked mon.noname-c con 0x7efe3009fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 10 monclient(hunting): picked mon.noname-e con 0x7efe3009ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3005ed70 con 0x7efe3009f7d0 2024-09-18T14:01:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3009dfc0 con 0x7efe3009fba0 2024-09-18T14:01:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3009dce0 con 0x7efe3009ff70 2024-09-18T14:01:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe35b09640 10 monclient(hunting): authenticate will time out at 912.391665s 2024-09-18T14:01:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe2effd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7efe3009fba0 0x7efe30137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:37024/0 (socket says 172.21.15.45:37024) 2024-09-18T14:01:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe2effd640 1 -- 172.21.15.45:0/2601558379 learned_addr learned my addr 172.21.15.45:0/2601558379 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.053+0000 7efe2ffff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7efe3009ff70 0x7efe3013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:32868/0 (socket says 172.21.15.45:32868) 2024-09-18T14:01:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915805484 0 0) 0x7efe3009dce0 con 0x7efe3009ff70 2024-09-18T14:01:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient(hunting): my global_id is 4697 2024-09-18T14:01:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe180034e0 con 0x7efe3009ff70 2024-09-18T14:01:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608259947 0 0) 0x7efe3005ed70 con 0x7efe3009f7d0 2024-09-18T14:01:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient(hunting): my global_id is 4752 2024-09-18T14:01:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe3009dce0 con 0x7efe3009f7d0 2024-09-18T14:01:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874106646 0 0) 0x7efe3009dfc0 con 0x7efe3009fba0 2024-09-18T14:01:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient(hunting): my global_id is 134320 2024-09-18T14:01:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe3005ed70 con 0x7efe3009fba0 2024-09-18T14:01:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1782595235 0 0) 0x7efe180034e0 con 0x7efe3009ff70 2024-09-18T14:01:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 >> v1:172.21.15.45:6791/0 conn(0x7efe3009fba0 legacy=0x7efe30137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 >> v1:172.21.15.45:6789/0 conn(0x7efe3009f7d0 legacy=0x7efe30132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:01:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 monclient: found mon.noname-e 2024-09-18T14:01:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:01:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300a1c00 con 0x7efe3009ff70 2024-09-18T14:01:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:01:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe35b09640 5 monclient: authenticate success, global_id 4697 2024-09-18T14:01:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe28001bb0 con 0x7efe3009ff70 2024-09-18T14:01:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient: dump: 2024-09-18T14:01:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.069 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe35b09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efe28001d90 con 0x7efe3009ff70 2024-09-18T14:01:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 1 -- 172.21.15.45:0/2601558379 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe28004e60 con 0x7efe3009ff70 2024-09-18T14:01:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe35b09640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe35b09640 4 set_mon_vals no callback set 2024-09-18T14:01:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.054+0000 7efe2e7fc640 10 monclient: dump: 2024-09-18T14:01:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.072 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.055+0000 7efe35b09640 10 monclient: shutdown 2024-09-18T14:01:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.055+0000 7efe35b09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.055+0000 7efe35b09640 1 -- 172.21.15.45:0/2601558379 >> v1:172.21.15.45:6793/0 conn(0x7efe3009ff70 legacy=0x7efe3013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.055+0000 7efe35b09640 1 -- 172.21.15.45:0/2601558379 shutdown_connections 2024-09-18T14:01:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.055+0000 7efe35b09640 1 -- 172.21.15.45:0/2601558379 >> 172.21.15.45:0/2601558379 conn(0x7efe300630b0 msgr2=0x7efe300634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.055+0000 7efe35b09640 1 -- 172.21.15.45:0/2601558379 shutdown_connections 2024-09-18T14:01:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.055+0000 7efe35b09640 1 -- 172.21.15.45:0/2601558379 wait complete. 2024-09-18T14:01:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.055+0000 7efe35b09640 10 monclient: build_initial_monmap 2024-09-18T14:01:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.055+0000 7efe35b09640 10 monclient: monmap: 2024-09-18T14:01:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:37.074 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 1 Processor -- start 2024-09-18T14:01:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 1 -- start start 2024-09-18T14:01:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 10 monclient: init 2024-09-18T14:01:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 10 monclient: _add_conns ranks=[8,2,5,4,0,1,6,7,3] 2024-09-18T14:01:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 10 monclient(hunting): picked mon.noname-e con 0x7efe3009f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 10 monclient(hunting): picked mon.noname-b con 0x7efe3009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 10 monclient(hunting): picked mon.noname-h con 0x7efe3009ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:01:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3009d310 con 0x7efe3009fba0 2024-09-18T14:01:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe300a2580 con 0x7efe3009ff70 2024-09-18T14:01:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe30120ad0 con 0x7efe3009f7d0 2024-09-18T14:01:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe35b09640 10 monclient(hunting): authenticate will time out at 912.394558s 2024-09-18T14:01:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe2f7fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7efe3009f7d0 0x7efe3011d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:32880/0 (socket says 172.21.15.45:32880) 2024-09-18T14:01:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe2f7fe640 1 -- 172.21.15.45:0/1374070496 learned_addr learned my addr 172.21.15.45:0/1374070496 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106482125 0 0) 0x7efe30120ad0 con 0x7efe3009f7d0 2024-09-18T14:01:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe177fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe177fe640 10 monclient(hunting): my global_id is 4706 2024-09-18T14:01:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe1c0034e0 con 0x7efe3009f7d0 2024-09-18T14:01:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745598279 0 0) 0x7efe3009d310 con 0x7efe3009fba0 2024-09-18T14:01:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe177fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe177fe640 10 monclient(hunting): my global_id is 4772 2024-09-18T14:01:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.056+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe30120ad0 con 0x7efe3009fba0 2024-09-18T14:01:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1782014470 0 0) 0x7efe1c0034e0 con 0x7efe3009f7d0 2024-09-18T14:01:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe3009d310 con 0x7efe3009f7d0 2024-09-18T14:01:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3225496262 0 0) 0x7efe30120ad0 con 0x7efe3009fba0 2024-09-18T14:01:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe1c0034e0 con 0x7efe3009fba0 2024-09-18T14:01:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe20001d90 con 0x7efe3009f7d0 2024-09-18T14:01:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient(hunting): dump: 2024-09-18T14:01:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.079 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe24001320 con 0x7efe3009fba0 2024-09-18T14:01:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:01:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient(hunting): dump: 2024-09-18T14:01:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.081 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812003678 0 0) 0x7efe300a2580 con 0x7efe3009ff70 2024-09-18T14:01:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient(hunting): my global_id is 4631 2024-09-18T14:01:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe30120ad0 con 0x7efe3009ff70 2024-09-18T14:01:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 158674726 0 0) 0x7efe3009d310 con 0x7efe3009f7d0 2024-09-18T14:01:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 >> v1:172.21.15.84:6791/0 conn(0x7efe3009ff70 legacy=0x7efe30149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 >> v1:172.21.15.45:6790/0 conn(0x7efe3009fba0 legacy=0x7efe3011d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient: _finish_hunting 0 2024-09-18T14:01:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 monclient: found mon.e 2024-09-18T14:01:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe30121760 con 0x7efe3009f7d0 2024-09-18T14:01:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient: _finish_auth 0 2024-09-18T14:01:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe35b09640 5 monclient: authenticate success, global_id 4706 2024-09-18T14:01:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe35b09640 10 monclient: _renew_subs 2024-09-18T14:01:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe35b09640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe35b09640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3011eed0 con 0x7efe3009f7d0 2024-09-18T14:01:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe35b09640 10 monclient: _renew_subs 2024-09-18T14:01:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe35b09640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe35b09640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efe301204f0 con 0x7efe3009f7d0 2024-09-18T14:01:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efe200021c0 con 0x7efe3009f7d0 2024-09-18T14:01:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe200027b0 con 0x7efe3009f7d0 2024-09-18T14:01:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe2dffb640 4 set_mon_vals no callback set 2024-09-18T14:01:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.057+0000 7efe177fe640 10 monclient: dump: 2024-09-18T14:01:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.085 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.058+0000 7efe35b09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.058+0000 7efe35b09640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7efe10006580 con 0x7efe10002830 2024-09-18T14:01:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.058+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7efe20010120 con 0x7efe3009f7d0 2024-09-18T14:01:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.058+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 3911+0+0 (unknown 4286160998 0 0) 0x7efe20046a50 con 0x7efe3009f7d0 2024-09-18T14:01:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.058+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826061029 0 0) 0x7efe10006580 con 0x7efe10002830 2024-09-18T14:01:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.058+0000 7efe177fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.058+0000 7efe177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.058+0000 7efe177fe640 10 monclient(hunting): my global_id is 4773 2024-09-18T14:01:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.058+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe3009d310 con 0x7efe10002830 2024-09-18T14:01:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.058+0000 7efe2ffff640 10 monclient: get_auth_request con 0x7efe1c03a580 auth_method 0 2024-09-18T14:01:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 726776152 0 0) 0x7efe3009d310 con 0x7efe10002830 2024-09-18T14:01:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe10006580 con 0x7efe10002830 2024-09-18T14:01:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe24004d90 con 0x7efe10002830 2024-09-18T14:01:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:01:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 10 monclient: dump: 2024-09-18T14:01:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.088 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2856720581 0 0) 0x7efe10006580 con 0x7efe10002830 2024-09-18T14:01:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe1000a0a0 con 0x7efe10002830 2024-09-18T14:01:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7efe240052c0 con 0x7efe10002830 2024-09-18T14:01:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 >> v1:172.21.15.84:6790/0 conn(0x7efe10002830 legacy=0x7efe10004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.059+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 >> v1:172.21.15.84:6790/0 conn(0x7efe10002830 legacy=0x7efe10004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.065+0000 7efe35b09640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.065+0000 7efe35b09640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7efe10009660 con 0x7efe10009040 2024-09-18T14:01:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.065+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297842732 0 0) 0x7efe10009660 con 0x7efe10009040 2024-09-18T14:01:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.065+0000 7efe177fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.065+0000 7efe177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.065+0000 7efe177fe640 10 monclient(hunting): my global_id is 4782 2024-09-18T14:01:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.065+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe10006580 con 0x7efe10009040 2024-09-18T14:01:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 635715360 0 0) 0x7efe10006580 con 0x7efe10009040 2024-09-18T14:01:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe10009660 con 0x7efe10009040 2024-09-18T14:01:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe240059a0 con 0x7efe10009040 2024-09-18T14:01:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:01:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 10 monclient: dump: 2024-09-18T14:01:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.093 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3282472923 0 0) 0x7efe10009660 con 0x7efe10009040 2024-09-18T14:01:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efe10009920 con 0x7efe10009040 2024-09-18T14:01:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7726 (unknown 0 0 2873981359) 0x7efe24006df0 con 0x7efe10009040 2024-09-18T14:01:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 >> v1:172.21.15.84:6790/0 conn(0x7efe10009040 legacy=0x7efe10008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.066+0000 7efe177fe640 1 -- 172.21.15.45:0/1374070496 >> v1:172.21.15.84:6790/0 conn(0x7efe10009040 legacy=0x7efe10008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:37.093 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:37.094 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:37.094 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:37.094 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:37.094 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:37.094 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:37.094 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:37.094 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:37.094 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:37.094 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:37.094 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:37.094 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:37.095 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:37.095 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:37.095 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:37.095 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.095 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:01:37.095 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:37.095 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:37.095 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:37.095 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:37.096 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:37.096 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:37.096 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:37.096 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:37.096 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:37.096 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:37.096 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:37.096 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:37.096 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.096 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:37.096 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:37.097 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:37.097 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:37.097 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:37.097 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:37.097 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:37.097 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:37.097 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:37.097 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:37.097 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:37.097 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.098 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:37.098 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:37.098 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:37.098 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:37.098 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:37.098 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:37.098 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:37.098 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:37.098 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:37.098 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:37.098 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:37.099 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:37.099 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:37.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:37.099 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:37.099 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:37.099 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:37.099 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:37.099 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:37.099 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:37.099 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:37.100 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:37.100 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:37.100 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:37.100 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:37.100 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:37.100 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.100 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:37.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.100 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:37.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.101 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:37.101 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:37.101 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.101 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.101 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.101 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:37.101 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.101 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.101 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.102 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:37.102 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:37.102 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.102 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.102 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.102 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:37.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:37.102 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.102 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.103 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:37.103 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.103 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.103 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.103 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:37.103 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:37.103 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.104 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.104 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.104 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:37.104 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:37.104 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.104 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.104 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.104 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:37.105 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.105 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.105 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:37.105 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:37.105 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.105 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.105 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.106 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:37.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:37.106 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.106 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.106 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:37.106 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.106 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.107 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.107 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:37.107 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:37.107 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.107 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.107 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.107 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:37.107 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:37.108 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.108 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.108 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.108 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:37.108 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.108 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.108 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.108 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:37.109 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:37.109 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.109 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.109 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.109 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:37.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:37.109 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.109 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.110 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:37.110 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.110 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.110 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.110 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:37.110 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:37.110 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.110 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.110 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.111 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:37.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:37.111 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.111 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.111 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:37.111 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.111 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.112 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.112 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:37.112 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:37.112 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.112 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.112 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.112 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:37.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.115 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:37.115 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.115 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.115 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.115 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:37.115 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:37.115 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.115 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.115 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.115 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.116 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.116 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:37.116 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:37.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.116 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:37.116 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:37.117 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:37.117 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.117 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.117 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:01:37.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.117 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:37.117 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:37.117 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:37.117 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.117 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.118 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:37.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.118 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:37.118 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:37.118 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:01:37.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.118 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.118 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:01:37.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.118 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:37.118 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:37.118 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:37.119 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.119 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.119 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:37.119 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:37.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.067+0000 7efe35b09640 1 -- 172.21.15.45:0/1374070496 >> v1:172.21.15.84:6812/31478 conn(0x7efe1c03a580 legacy=0x7efe1c03ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.067+0000 7efe35b09640 10 monclient: shutdown 2024-09-18T14:01:37.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.067+0000 7efe35b09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:37.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.067+0000 7efe35b09640 1 -- 172.21.15.45:0/1374070496 >> v1:172.21.15.45:6793/0 conn(0x7efe3009f7d0 legacy=0x7efe3011d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.067+0000 7efe2ffff640 1 -- 172.21.15.45:0/1374070496 reap_dead start 2024-09-18T14:01:37.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.067+0000 7efe2ffff640 1 -- 172.21.15.45:0/1374070496 reap_dead start 2024-09-18T14:01:37.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.067+0000 7efe35b09640 1 -- 172.21.15.45:0/1374070496 shutdown_connections 2024-09-18T14:01:37.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.067+0000 7efe35b09640 1 -- 172.21.15.45:0/1374070496 >> 172.21.15.45:0/1374070496 conn(0x7efe300630b0 msgr2=0x7efe300a45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:37.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.067+0000 7efe35b09640 1 -- 172.21.15.45:0/1374070496 shutdown_connections 2024-09-18T14:01:37.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.067+0000 7efe35b09640 1 -- 172.21.15.45:0/1374070496 wait complete. 2024-09-18T14:01:37.122 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:37.122 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:01:37.122 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:01:37.159 INFO:tasks.rados.rados.0.smithi084.stdout:593: done (3 left) 2024-09-18T14:01:37.159 INFO:tasks.rados.rados.0.smithi084.stdout:594: copy_from oid 37 from oid 1 current snap is 57 2024-09-18T14:01:37.159 INFO:tasks.rados.rados.0.smithi084.stdout:590: finishing write tid 2 to smithi08434663-21 2024-09-18T14:01:37.160 INFO:tasks.rados.rados.0.smithi084.stdout:590: finishing write tid 3 to smithi08434663-21 2024-09-18T14:01:37.160 INFO:tasks.rados.rados.0.smithi084.stdout:590: finishing write tid 4 to smithi08434663-21 2024-09-18T14:01:37.160 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 134 (ObjNum 240 snap 57 seq_num 240) dirty exists 2024-09-18T14:01:37.160 INFO:tasks.rados.rados.0.smithi084.stdout:590: left oid 21 (ObjNum 240 snap 57 seq_num 240) 2024-09-18T14:01:37.160 INFO:tasks.rados.rados.0.smithi084.stdout:591: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:01:37.160 INFO:tasks.rados.rados.0.smithi084.stdout:592: finishing write tid 1 to smithi08434663-39 2024-09-18T14:01:37.160 INFO:tasks.rados.rados.0.smithi084.stdout:591: finishing copy_from to smithi08434663-36 2024-09-18T14:01:37.160 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 171 (ObjNum 229 snap 54 seq_num 229) dirty exists 2024-09-18T14:01:37.160 INFO:tasks.rados.rados.0.smithi084.stdout:592: finishing write tid 2 to smithi08434663-39 2024-09-18T14:01:37.160 INFO:tasks.rados.rados.0.smithi084.stdout:592: finishing write tid 3 to smithi08434663-39 2024-09-18T14:01:37.160 INFO:tasks.rados.rados.0.smithi084.stdout:592: finishing write tid 4 to smithi08434663-39 2024-09-18T14:01:37.161 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 182 (ObjNum 241 snap 57 seq_num 241) dirty exists 2024-09-18T14:01:37.161 INFO:tasks.rados.rados.0.smithi084.stdout:592: left oid 39 (ObjNum 241 snap 57 seq_num 241) 2024-09-18T14:01:37.161 INFO:tasks.rados.rados.0.smithi084.stdout:590: done (3 left) 2024-09-18T14:01:37.161 INFO:tasks.rados.rados.0.smithi084.stdout:591: done (2 left) 2024-09-18T14:01:37.161 INFO:tasks.rados.rados.0.smithi084.stdout:592: done (1 left) 2024-09-18T14:01:37.161 INFO:tasks.rados.rados.0.smithi084.stdout:595: delete oid 20 current snap is 57 2024-09-18T14:01:37.165 INFO:tasks.rados.rados.0.smithi084.stdout:595: done (1 left) 2024-09-18T14:01:37.166 INFO:tasks.rados.rados.0.smithi084.stdout:596: snap_remove snap 52 2024-09-18T14:01:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.226+0000 7f773411d640 10 monclient: get_monmap_and_config 2024-09-18T14:01:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.226+0000 7f773411d640 10 monclient: build_initial_monmap 2024-09-18T14:01:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.226+0000 7f773411d640 10 monclient: monmap: 2024-09-18T14:01:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:37.227734+0000 2024-09-18T14:01:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:37.227734+0000 2024-09-18T14:01:37.228 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:37.229 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.229 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:37.229 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:37.229 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:37.229 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:37.229 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:37.229 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:37.229 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:37.229 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:37.230 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:37.230 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.227+0000 7f773411d640 1 Processor -- start 2024-09-18T14:01:37.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.227+0000 7f773411d640 1 -- start start 2024-09-18T14:01:37.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.227+0000 7f773411d640 10 monclient: init 2024-09-18T14:01:37.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.227+0000 7f773411d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:37.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 10 monclient: _add_conns ranks=[8,0,4,1,7,6,3,5,2] 2024-09-18T14:01:37.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 10 monclient(hunting): picked mon.noname-e con 0x7f772c06e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:37.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 10 monclient(hunting): picked mon.noname-a con 0x7f772c0a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:37.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 10 monclient(hunting): picked mon.noname-c con 0x7f772c0a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:37.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c05ed70 con 0x7f772c0a0cb0 2024-09-18T14:01:37.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c097620 con 0x7f772c0a1080 2024-09-18T14:01:37.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c097340 con 0x7f772c06e810 2024-09-18T14:01:37.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 10 monclient(hunting): authenticate will time out at 912.566106s 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7731eb9640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f772c06e810 0x7f772c0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:32886/0 (socket says 172.21.15.45:32886) 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7731eb9640 1 -- 172.21.15.45:0/837914232 learned_addr learned my addr 172.21.15.45:0/837914232 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f77316b8640 1 --1- 172.21.15.45:0/837914232 >> v1:172.21.15.45:6789/0 conn(0x7f772c0a0cb0 0x7f772c137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51526/0 (socket says 172.21.15.45:51526) 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306642657 0 0) 0x7f772c097620 con 0x7f772c0a1080 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient(hunting): my global_id is 134329 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77180034e0 con 0x7f772c0a1080 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3735596365 0 0) 0x7f772c097340 con 0x7f772c06e810 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient(hunting): my global_id is 4715 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772c097620 con 0x7f772c06e810 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242206522 0 0) 0x7f772c05ed70 con 0x7f772c0a0cb0 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient(hunting): my global_id is 4761 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772c097340 con 0x7f772c0a0cb0 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2560498925 0 0) 0x7f77180034e0 con 0x7f772c0a1080 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 >> v1:172.21.15.45:6793/0 conn(0x7f772c06e810 legacy=0x7f772c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 >> v1:172.21.15.45:6789/0 conn(0x7f772c0a0cb0 legacy=0x7f772c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient: _finish_hunting 0 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 monclient: found mon.noname-c 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:37.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c0a1660 con 0x7f772c0a1080 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient: _finish_auth 0 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 5 monclient: authenticate success, global_id 134329 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7724001bb0 con 0x7f772c0a1080 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient: dump: 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7724001d10 con 0x7f772c0a1080 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 1 -- 172.21.15.45:0/837914232 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7724002240 con 0x7f772c0a1080 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f773411d640 4 set_mon_vals no callback set 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.228+0000 7f7730eb7640 10 monclient: dump: 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.229+0000 7f773411d640 10 monclient: shutdown 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.229+0000 7f773411d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.229+0000 7f773411d640 1 -- 172.21.15.45:0/837914232 >> v1:172.21.15.45:6791/0 conn(0x7f772c0a1080 legacy=0x7f772c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.229+0000 7f773411d640 1 -- 172.21.15.45:0/837914232 shutdown_connections 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.229+0000 7f773411d640 1 -- 172.21.15.45:0/837914232 >> 172.21.15.45:0/837914232 conn(0x7f772c0a5bb0 msgr2=0x7f772c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:37.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.229+0000 7f773411d640 1 -- 172.21.15.45:0/837914232 shutdown_connections 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.229+0000 7f773411d640 1 -- 172.21.15.45:0/837914232 wait complete. 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 10 monclient: build_initial_monmap 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 10 monclient: monmap: 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:37.251 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 1 Processor -- start 2024-09-18T14:01:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 1 -- start start 2024-09-18T14:01:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 10 monclient: init 2024-09-18T14:01:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 10 monclient: _add_conns ranks=[7,0,8,5,3,4,2,6,1] 2024-09-18T14:01:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 10 monclient(hunting): picked mon.noname-i con 0x7f772c06e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 10 monclient(hunting): picked mon.noname-a con 0x7f772c0a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 10 monclient(hunting): picked mon.noname-e con 0x7f772c0a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c09fa10 con 0x7f772c0a0cb0 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c13da70 con 0x7f772c06e810 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c0a2270 con 0x7f772c0a1080 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f773411d640 10 monclient(hunting): authenticate will time out at 912.568925s 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f77316b8640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f772c0a0cb0 0x7f772c146570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51532/0 (socket says 172.21.15.45:51532) 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.230+0000 7f77316b8640 1 -- 172.21.15.45:0/1244404238 learned_addr learned my addr 172.21.15.45:0/1244404238 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449312851 0 0) 0x7f772c0a2270 con 0x7f772c0a1080 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): my global_id is 4724 2024-09-18T14:01:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77100034e0 con 0x7f772c0a1080 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880564767 0 0) 0x7f772c09fa10 con 0x7f772c0a0cb0 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): my global_id is 4770 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772c0a2270 con 0x7f772c0a0cb0 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250974983 0 0) 0x7f772c13da70 con 0x7f772c06e810 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): my global_id is 4705 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772c09fa10 con 0x7f772c06e810 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3941777895 0 0) 0x7f77100034e0 con 0x7f772c0a1080 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f772c13da70 con 0x7f772c0a1080 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f77240021c0 con 0x7f772c0a1080 2024-09-18T14:01:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): dump: 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1873700508 0 0) 0x7f772c0a2270 con 0x7f772c0a0cb0 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f77100034e0 con 0x7f772c0a0cb0 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f771c0012d0 con 0x7f772c0a0cb0 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): dump: 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3112967081 0 0) 0x7f772c09fa10 con 0x7f772c06e810 2024-09-18T14:01:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f772c0a2270 con 0x7f772c06e810 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7728001290 con 0x7f772c06e810 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient(hunting): dump: 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1951984086 0 0) 0x7f772c13da70 con 0x7f772c0a1080 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 >> v1:172.21.15.84:6792/0 conn(0x7f772c06e810 legacy=0x7f772c13d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 >> v1:172.21.15.45:6789/0 conn(0x7f772c0a0cb0 legacy=0x7f772c146570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 monclient: found mon.e 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c13e880 con 0x7f772c0a1080 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient: _finish_auth 0 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f773411d640 5 monclient: authenticate success, global_id 4724 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f773411d640 10 monclient: _renew_subs 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f773411d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f773411d640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772c122400 con 0x7f772c0a1080 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f77240025d0 con 0x7f772c0a1080 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7724002ac0 con 0x7f772c0a1080 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7721ffb640 10 monclient: dump: 2024-09-18T14:01:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.232+0000 7f773411d640 10 monclient: _renew_subs 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.231+0000 7f7723fff640 4 set_mon_vals no callback set 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.232+0000 7f773411d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.232+0000 7f773411d640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f772c120060 con 0x7f772c0a1080 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.232+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f772400db10 con 0x7f772c0a1080 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.232+0000 7f773411d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.232+0000 7f773411d640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f772c04c4a0 con 0x7f772c1214e0 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.232+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(129..129 src has 1..129) v4 ==== 3967+0+0 (unknown 339254258 0 0) 0x7f77240446a0 con 0x7f772c0a1080 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f77316b8640 10 monclient: get_auth_request con 0x7f771003a9d0 auth_method 0 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514608719 0 0) 0x7f772c04c4a0 con 0x7f772c1214e0 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 10 monclient(hunting): my global_id is 4640 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772c13da70 con 0x7f772c1214e0 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3547351983 0 0) 0x7f772c13da70 con 0x7f772c1214e0 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f772c04c4a0 con 0x7f772c1214e0 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7728004ae0 con 0x7f772c1214e0 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 10 monclient: dump: 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2056156464 0 0) 0x7f772c04c4a0 con 0x7f772c1214e0 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f772c04b720 con 0x7f772c1214e0 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7728006fc0 con 0x7f772c1214e0 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 >> v1:172.21.15.84:6791/0 conn(0x7f772c1214e0 legacy=0x7f772c09baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.233+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 >> v1:172.21.15.84:6791/0 conn(0x7f772c1214e0 legacy=0x7f772c09baa0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.239+0000 7f773411d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.239+0000 7f773411d640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f772c122ac0 con 0x7f772c13dcb0 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227283360 0 0) 0x7f772c122ac0 con 0x7f772c13dcb0 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 10 monclient(hunting): my global_id is 4649 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772c04c4a0 con 0x7f772c13dcb0 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 734055791 0 0) 0x7f772c04c4a0 con 0x7f772c13dcb0 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f772c122ac0 con 0x7f772c13dcb0 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f77280055b0 con 0x7f772c13dcb0 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:01:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 10 monclient: dump: 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.266 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.266 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.266 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4247815942 0 0) 0x7f772c122ac0 con 0x7f772c13dcb0 2024-09-18T14:01:37.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f772c13e080 con 0x7f772c13dcb0 2024-09-18T14:01:37.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 3792317530) 0x7f7728007250 con 0x7f772c13dcb0 2024-09-18T14:01:37.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:37.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:37.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 >> v1:172.21.15.84:6791/0 conn(0x7f772c13dcb0 legacy=0x7f772c09c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.240+0000 7f7721ffb640 1 -- 172.21.15.45:0/1244404238 >> v1:172.21.15.84:6791/0 conn(0x7f772c13dcb0 legacy=0x7f772c09c1b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:37.266 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:37.267 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 7, 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:37.268 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:37.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:37.271 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.272 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:37.278 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.278 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:37.278 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:37.278 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.279 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.280 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.280 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:37.280 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:37.280 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.280 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.280 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.280 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:37.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:37.280 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:37.281 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.281 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:37.281 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:37.282 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.284 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.285 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:37.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.241+0000 7f773411d640 1 -- 172.21.15.45:0/1244404238 >> v1:172.21.15.84:6812/31478 conn(0x7f771003a9d0 legacy=0x7f771003ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.241+0000 7f773411d640 10 monclient: shutdown 2024-09-18T14:01:37.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.241+0000 7f773411d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:37.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.241+0000 7f773411d640 1 -- 172.21.15.45:0/1244404238 >> v1:172.21.15.45:6793/0 conn(0x7f772c0a1080 legacy=0x7f772c149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.241+0000 7f77326ba640 1 -- 172.21.15.45:0/1244404238 reap_dead start 2024-09-18T14:01:37.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.241+0000 7f77326ba640 1 -- 172.21.15.45:0/1244404238 reap_dead start 2024-09-18T14:01:37.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.241+0000 7f773411d640 1 -- 172.21.15.45:0/1244404238 shutdown_connections 2024-09-18T14:01:37.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.241+0000 7f773411d640 1 -- 172.21.15.45:0/1244404238 >> 172.21.15.45:0/1244404238 conn(0x7f772c0a5bb0 msgr2=0x7f772c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:37.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.241+0000 7f773411d640 1 -- 172.21.15.45:0/1244404238 shutdown_connections 2024-09-18T14:01:37.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.241+0000 7f773411d640 1 -- 172.21.15.45:0/1244404238 wait complete. 2024-09-18T14:01:37.289 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:37.289 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:01:37.290 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:01:37.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.386+0000 7f7f30a34640 10 monclient: get_monmap_and_config 2024-09-18T14:01:37.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.386+0000 7f7f30a34640 10 monclient: build_initial_monmap 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.386+0000 7f7f30a34640 10 monclient: monmap: 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:37.387180+0000 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:37.387180+0000 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:37.388 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:37.388 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:37.388 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:37.388 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.386+0000 7f7f30a34640 1 Processor -- start 2024-09-18T14:01:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.386+0000 7f7f30a34640 1 -- start start 2024-09-18T14:01:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.386+0000 7f7f30a34640 10 monclient: init 2024-09-18T14:01:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f30a34640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f30a34640 10 monclient: _add_conns ranks=[8,4,1,5,6,0,7,3,2] 2024-09-18T14:01:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f30a34640 10 monclient(hunting): picked mon.noname-e con 0x7f7f2c06e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f30a34640 10 monclient(hunting): picked mon.noname-c con 0x7f7f2c0a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:01:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f30a34640 10 monclient(hunting): picked mon.noname-f con 0x7f7f2c0a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f30a34640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2c05ed70 con 0x7f7f2c0a1080 2024-09-18T14:01:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f30a34640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2c097620 con 0x7f7f2c0a0cb0 2024-09-18T14:01:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f30a34640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2c097340 con 0x7f7f2c06e810 2024-09-18T14:01:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f30a34640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f30a34640 10 monclient(hunting): authenticate will time out at 912.725506s 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f2a59c640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f7f2c06e810 0x7f7f2c0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:32912/0 (socket says 172.21.15.45:32912) 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f29d9b640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f7f2c0a0cb0 0x7f7f2c137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:37052/0 (socket says 172.21.15.45:37052) 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f2a59c640 1 -- 172.21.15.45:0/1745177559 learned_addr learned my addr 172.21.15.45:0/1745177559 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407233491 0 0) 0x7f7f2c097340 con 0x7f7f2c06e810 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f2959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f2959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f2959a640 10 monclient(hunting): my global_id is 4733 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f140034e0 con 0x7f7f2c06e810 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127365096 0 0) 0x7f7f2c097620 con 0x7f7f2c0a0cb0 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f2959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f2959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f2959a640 10 monclient(hunting): my global_id is 134338 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.387+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f2c097340 con 0x7f7f2c0a0cb0 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501912002 0 0) 0x7f7f2c05ed70 con 0x7f7f2c0a1080 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient(hunting): my global_id is 4645 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f2c097620 con 0x7f7f2c0a1080 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 920998950 0 0) 0x7f7f140034e0 con 0x7f7f2c06e810 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 >> v1:172.21.15.45:6791/0 conn(0x7f7f2c0a0cb0 legacy=0x7f7f2c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 >> v1:172.21.15.84:6789/0 conn(0x7f7f2c0a1080 legacy=0x7f7f2c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient: _finish_hunting 0 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 1 monclient: found mon.noname-e 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:01:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c0a1660 con 0x7f7f2c06e810 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient: _finish_auth 0 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f1c001830 con 0x7f7f2c06e810 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f30a34640 5 monclient: authenticate success, global_id 4733 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f30a34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient: dump: 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f30a34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7f1c001a10 con 0x7f7f2c06e810 2024-09-18T14:01:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 1 -- 172.21.15.45:0/1745177559 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f1c004ba0 con 0x7f7f2c06e810 2024-09-18T14:01:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f30a34640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f30a34640 4 set_mon_vals no callback set 2024-09-18T14:01:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f2959a640 10 monclient: dump: 2024-09-18T14:01:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f30a34640 10 monclient: shutdown 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f30a34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f30a34640 1 -- 172.21.15.45:0/1745177559 >> v1:172.21.15.45:6793/0 conn(0x7f7f2c06e810 legacy=0x7f7f2c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f30a34640 1 -- 172.21.15.45:0/1745177559 shutdown_connections 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f30a34640 1 -- 172.21.15.45:0/1745177559 >> 172.21.15.45:0/1745177559 conn(0x7f7f2c0a5bb0 msgr2=0x7f7f2c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f30a34640 1 -- 172.21.15.45:0/1745177559 shutdown_connections 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.388+0000 7f7f30a34640 1 -- 172.21.15.45:0/1745177559 wait complete. 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.389+0000 7f7f30a34640 10 monclient: build_initial_monmap 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.389+0000 7f7f30a34640 10 monclient: monmap: 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.389+0000 7f7f30a34640 1 Processor -- start 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.389+0000 7f7f30a34640 1 -- start start 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.389+0000 7f7f30a34640 10 monclient: init 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f30a34640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f30a34640 10 monclient: _add_conns ranks=[0,1,8,4,5,6,7,3,2] 2024-09-18T14:01:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f30a34640 10 monclient(hunting): picked mon.noname-a con 0x7f7f2c06e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f30a34640 10 monclient(hunting): picked mon.noname-f con 0x7f7f2c0a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f30a34640 10 monclient(hunting): picked mon.noname-e con 0x7f7f2c0a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f30a34640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2c122d20 con 0x7f7f2c06e810 2024-09-18T14:01:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f30a34640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2c13db10 con 0x7f7f2c0a0cb0 2024-09-18T14:01:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f30a34640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2c122760 con 0x7f7f2c0a1080 2024-09-18T14:01:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f30a34640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f30a34640 10 monclient(hunting): authenticate will time out at 912.728210s 2024-09-18T14:01:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f2a59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7f2c06e810 0x7f7f2c13d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51538/0 (socket says 172.21.15.45:51538) 2024-09-18T14:01:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f2a59c640 1 -- 172.21.15.45:0/89557255 learned_addr learned my addr 172.21.15.45:0/89557255 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f2ad9d640 1 --1- 172.21.15.45:0/89557255 >> v1:172.21.15.45:6793/0 conn(0x7f7f2c0a1080 0x7f7f2c149d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:32914/0 (socket says 172.21.15.45:32914) 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878096720 0 0) 0x7f7f2c122760 con 0x7f7f2c0a1080 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): my global_id is 4742 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f0c0034e0 con 0x7f7f2c0a1080 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966321806 0 0) 0x7f7f2c122d20 con 0x7f7f2c06e810 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): my global_id is 4779 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f2c122760 con 0x7f7f2c06e810 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7354522 0 0) 0x7f7f2c13db10 con 0x7f7f2c0a0cb0 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): my global_id is 4654 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f2c122d20 con 0x7f7f2c0a0cb0 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1985947897 0 0) 0x7f7f0c0034e0 con 0x7f7f2c0a1080 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7f2c13db10 con 0x7f7f2c0a1080 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f24002db0 con 0x7f7f2c0a1080 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): dump: 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 930519857 0 0) 0x7f7f2c122760 con 0x7f7f2c06e810 2024-09-18T14:01:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7f0c0034e0 con 0x7f7f2c06e810 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f1c001870 con 0x7f7f2c06e810 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): dump: 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3324726658 0 0) 0x7f7f2c122d20 con 0x7f7f2c0a0cb0 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7f2c122760 con 0x7f7f2c0a0cb0 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f200013a0 con 0x7f7f2c0a0cb0 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.390+0000 7f7f1a7fc640 10 monclient(hunting): dump: 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.412 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2935629586 0 0) 0x7f7f2c13db10 con 0x7f7f2c0a1080 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 >> v1:172.21.15.84:6789/0 conn(0x7f7f2c0a0cb0 legacy=0x7f7f2c146630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 >> v1:172.21.15.45:6789/0 conn(0x7f7f2c06e810 legacy=0x7f7f2c13d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 1 monclient: found mon.e 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c120220 con 0x7f7f2c0a1080 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f30a34640 5 monclient: authenticate success, global_id 4742 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f30a34640 10 monclient: _renew_subs 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f30a34640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f30a34640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2c122480 con 0x7f7f2c0a1080 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f30a34640 10 monclient: _renew_subs 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f30a34640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f30a34640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2c1215b0 con 0x7f7f2c0a1080 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7f24003070 con 0x7f7f2c0a1080 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f24001b80 con 0x7f7f2c0a1080 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f28d99640 4 set_mon_vals no callback set 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 10 monclient: dump: 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7f24001ce0 con 0x7f7f2c0a1080 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.391+0000 7f7f30a34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f30a34640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7f10006580 con 0x7f7f10002830 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(129..129 src has 1..129) v4 ==== 3967+0+0 (unknown 339254258 0 0) 0x7f7f24046d80 con 0x7f7f2c0a1080 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f2a59c640 1 -- 172.21.15.45:0/89557255 >> v1:172.21.15.84:6792/0 conn(0x7f7f10002830 legacy=0x7f7f10004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f2a59c640 1 -- 172.21.15.45:0/89557255 >> v1:172.21.15.84:6792/0 conn(0x7f7f10002830 legacy=0x7f7f10004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f2a59c640 1 --1- 172.21.15.45:0/89557255 >> v1:172.21.15.84:6792/0 conn(0x7f7f10002830 0x7f7f10004cc0 :-1 s=OPENED pgs=130 cs=1 l=1).handle_message read tag failed 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f2a59c640 1 --1- 172.21.15.45:0/89557255 >> v1:172.21.15.84:6792/0 conn(0x7f7f10002830 0x7f7f10004cc0 :-1 s=OPENED pgs=130 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f29d9b640 10 monclient: get_auth_request con 0x7f7f0c03a9a0 auth_method 0 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 >> v1:172.21.15.84:6792/0 conn(0x7f7f10002830 legacy=0x7f7f10004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 >> v1:172.21.15.84:6792/0 conn(0x7f7f10002830 legacy=0x7f7f10004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7f2c13db10 con 0x7f7f0c041790 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137640629 0 0) 0x7f7f2c13db10 con 0x7f7f0c041790 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f1a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f1a7fc640 10 monclient(hunting): my global_id is 4714 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f0c0034e0 con 0x7f7f0c041790 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.392+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 332585368 0 0) 0x7f7f0c0034e0 con 0x7f7f0c041790 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.393+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7f2c13db10 con 0x7f7f0c041790 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.393+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f1c005160 con 0x7f7f0c041790 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.393+0000 7f7f1a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.393+0000 7f7f1a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.393+0000 7f7f1a7fc640 10 monclient: dump: 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.393+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 400454450 0 0) 0x7f7f2c13db10 con 0x7f7f0c041790 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.393+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f0c046440 con 0x7f7f0c041790 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.393+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7f1c007320 con 0x7f7f0c041790 2024-09-18T14:01:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.393+0000 7f7f1a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:37.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.393+0000 7f7f1a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:37.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.393+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 >> v1:172.21.15.84:6792/0 conn(0x7f7f0c041790 legacy=0x7f7f0c043ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.393+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 >> v1:172.21.15.84:6792/0 conn(0x7f7f0c041790 legacy=0x7f7f0c043ba0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:37.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f30a34640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:01:37.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f30a34640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7f10007550 con 0x7f7f10006900 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663825712 0 0) 0x7f7f10007550 con 0x7f7f10006900 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f1a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f1a7fc640 10 monclient(hunting): my global_id is 4723 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f2c13db10 con 0x7f7f10006900 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2190042879 0 0) 0x7f7f2c13db10 con 0x7f7f10006900 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7f10007550 con 0x7f7f10006900 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7f1c005af0 con 0x7f7f10006900 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f1a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f1a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.399+0000 7f7f1a7fc640 10 monclient: dump: 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:37.422 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3534431701 0 0) 0x7f7f10007550 con 0x7f7f10006900 2024-09-18T14:01:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7f10008750 con 0x7f7f10006900 2024-09-18T14:01:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 344224028) 0x7f7f1c00a310 con 0x7f7f10006900 2024-09-18T14:01:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f1a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:01:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f1a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 >> v1:172.21.15.84:6792/0 conn(0x7f7f10006900 legacy=0x7f7f10006d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f1a7fc640 1 -- 172.21.15.45:0/89557255 >> v1:172.21.15.84:6792/0 conn(0x7f7f10006900 legacy=0x7f7f10006d80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:01:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f2ad9d640 1 -- 172.21.15.45:0/89557255 reap_dead start 2024-09-18T14:01:37.423 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:01:37.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:37.424 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:37.424 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:01:37.424 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 50, 2024-09-18T14:01:37.424 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:01:37.424 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:01:37.424 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:01:37.424 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:01:37.424 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:01:37.424 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:01:37.424 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:01:37.424 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 7, 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:37.425 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:37.426 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:01:37.427 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:01:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:01:37.429 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.430 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:01:37.432 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.434 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.434 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:01:37.434 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:01:37.434 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.434 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.434 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:01:37.435 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:01:37.436 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.436 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.436 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:01:37.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:01:37.437 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.437 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.437 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:01:37.437 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.438 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.438 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:01:37.438 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:01:37.438 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.438 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.438 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:01:37.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:01:37.439 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.439 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.439 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:01:37.439 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.439 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.439 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.439 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:01:37.439 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:01:37.439 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:01:37.444 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:01:37.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:01:37.444 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:01:37.444 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:01:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f30a34640 1 -- 172.21.15.45:0/89557255 >> v1:172.21.15.84:6812/31478 conn(0x7f7f0c03a9a0 legacy=0x7f7f0c03ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f30a34640 10 monclient: shutdown 2024-09-18T14:01:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f30a34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f30a34640 1 -- 172.21.15.45:0/89557255 >> v1:172.21.15.45:6793/0 conn(0x7f7f2c0a1080 legacy=0x7f7f2c149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f30a34640 1 -- 172.21.15.45:0/89557255 shutdown_connections 2024-09-18T14:01:37.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f30a34640 1 -- 172.21.15.45:0/89557255 >> 172.21.15.45:0/89557255 conn(0x7f7f2c0a5bb0 msgr2=0x7f7f2c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:37.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f30a34640 1 -- 172.21.15.45:0/89557255 shutdown_connections 2024-09-18T14:01:37.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:37.400+0000 7f7f30a34640 1 -- 172.21.15.45:0/89557255 wait complete. 2024-09-18T14:01:37.446 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:01:37.446 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['g', 'b', 'c'] 2024-09-18T14:01:37.446 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a', 'd'] 2024-09-18T14:01:37.446 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-18T14:01:37.446 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-18T14:01:37.447 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-18T14:01:37.447 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:01:37.461 INFO:tasks.ceph.mon.g:Stopped 2024-09-18T14:01:37.461 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-18T14:01:37.462 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-18T14:01:37.462 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-18T14:01:37.462 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:01:37.474 INFO:tasks.ceph.mon.b:Stopped 2024-09-18T14:01:37.474 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-18T14:01:37.474 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-18T14:01:37.474 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T14:01:37.474 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:01:37.525 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T14:01:37.525 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-09-18T14:01:37.525 INFO:tasks.ceph.mon.a:Sent signal 19 2024-09-18T14:01:37.526 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-18T14:01:37.526 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-18T14:01:37.526 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-18T14:01:52.527 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-09-18T14:01:52.529 INFO:tasks.ceph.mon.a:Sent signal 18 2024-09-18T14:01:52.529 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-18T14:01:52.529 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-18T14:01:52.530 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-09-18T14:01:52.530 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:01:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.637+0000 7f437ef9d640 10 monclient: get_monmap_and_config 2024-09-18T14:01:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.637+0000 7f437ef9d640 10 monclient: build_initial_monmap 2024-09-18T14:01:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.637+0000 7f437ef9d640 10 monclient: monmap: 2024-09-18T14:01:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:01:52.638710+0000 2024-09-18T14:01:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:01:52.638710+0000 2024-09-18T14:01:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:52.639 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.638+0000 7f437ef9d640 1 Processor -- start 2024-09-18T14:01:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.638+0000 7f437ef9d640 1 -- start start 2024-09-18T14:01:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.638+0000 7f437ef9d640 10 monclient: init 2024-09-18T14:01:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f437ef9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f437ef9d640 10 monclient: _add_conns ranks=[0,6,2,1,3,8,7,5,4] 2024-09-18T14:01:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f437ef9d640 10 monclient(hunting): picked mon.noname-a con 0x7f437809f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:01:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f437ef9d640 10 monclient(hunting): picked mon.noname-d con 0x7f437809fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:01:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f437ef9d640 10 monclient(hunting): picked mon.noname-b con 0x7f437809ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f437ef9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437805ed70 con 0x7f437809f7d0 2024-09-18T14:01:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f437ef9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437809dfc0 con 0x7f437809ff70 2024-09-18T14:01:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f437ef9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437809dce0 con 0x7f437809fba0 2024-09-18T14:01:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f437ef9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f437ef9d640 10 monclient(hunting): authenticate will time out at 927.977110s 2024-09-18T14:01:52.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f437d53a640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7f437809ff70 legacy=0x7f437813b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:01:52.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f436ffff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f437809fba0 0x7f4378137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:58926/0 (socket says 172.21.15.45:58926) 2024-09-18T14:01:52.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.639+0000 7f436ffff640 1 -- 172.21.15.45:0/2050747379 learned_addr learned my addr 172.21.15.45:0/2050747379 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:52.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:52.840+0000 7f437d53a640 1 -- 172.21.15.45:0/2050747379 >> v1:172.21.15.45:6790/0 conn(0x7f437809ff70 legacy=0x7f437813b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:01:53.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:53.241+0000 7f437d53a640 1 -- 172.21.15.45:0/2050747379 >> v1:172.21.15.45:6790/0 conn(0x7f437809ff70 legacy=0x7f437813b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:01:54.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:54.042+0000 7f437d53a640 1 -- 172.21.15.45:0/2050747379 >> v1:172.21.15.45:6790/0 conn(0x7f437809ff70 legacy=0x7f437813b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:01:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 10 monclient(hunting): tick 2024-09-18T14:01:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 1 monclient(hunting): continuing hunt 2024-09-18T14:01:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:01:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 1 -- 172.21.15.45:0/2050747379 >> v1:172.21.15.45:6792/0 conn(0x7f437809fba0 legacy=0x7f4378137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 1 -- 172.21.15.45:0/2050747379 >> v1:172.21.15.45:6790/0 conn(0x7f437809ff70 legacy=0x7f437813b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:01:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 1 -- 172.21.15.45:0/2050747379 >> v1:172.21.15.45:6789/0 conn(0x7f437809f7d0 legacy=0x7f4378132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 10 monclient: _add_conns ranks=[1,7,8,4,5,3] 2024-09-18T14:01:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 10 monclient(hunting): picked mon.noname-f con 0x7f4364003780 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 10 monclient(hunting): picked mon.noname-i con 0x7f4364007360 addr v1:172.21.15.84:6792/0 2024-09-18T14:01:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 10 monclient(hunting): picked mon.noname-e con 0x7f436400b0c0 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 1 -- 172.21.15.45:0/2050747379 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43780a1c00 con 0x7f4364003780 2024-09-18T14:01:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 1 -- 172.21.15.45:0/2050747379 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437809dfc0 con 0x7f4364007360 2024-09-18T14:01:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 1 -- 172.21.15.45:0/2050747379 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4364010f90 con 0x7f436400b0c0 2024-09-18T14:01:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:55.639+0000 7f436e7fc640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:57.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.266+0000 7f436f7fe640 1 -- 172.21.15.45:0/2050747379 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009610617 0 0) 0x7f43780a1c00 con 0x7f4364003780 2024-09-18T14:01:57.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.266+0000 7f436f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:57.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.266+0000 7f436f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.266+0000 7f436f7fe640 10 monclient(hunting): my global_id is 4663 2024-09-18T14:01:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.266+0000 7f436f7fe640 1 -- 172.21.15.45:0/2050747379 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43600034e0 con 0x7f4364003780 2024-09-18T14:01:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.266+0000 7f436f7fe640 1 -- 172.21.15.45:0/2050747379 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789823668 0 0) 0x7f437809dfc0 con 0x7f4364007360 2024-09-18T14:01:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.266+0000 7f436f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.266+0000 7f436f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.266+0000 7f436f7fe640 10 monclient(hunting): my global_id is 4732 2024-09-18T14:01:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.266+0000 7f436f7fe640 1 -- 172.21.15.45:0/2050747379 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43780a1c00 con 0x7f4364007360 2024-09-18T14:01:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 1 -- 172.21.15.45:0/2050747379 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2305583851 0 0) 0x7f43780a1c00 con 0x7f4364007360 2024-09-18T14:01:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 1 -- 172.21.15.45:0/2050747379 >> v1:172.21.15.45:6793/0 conn(0x7f436400b0c0 legacy=0x7f436400d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 1 -- 172.21.15.45:0/2050747379 >> v1:172.21.15.84:6789/0 conn(0x7f4364003780 legacy=0x7f4364005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:57.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f437d53a640 1 -- 172.21.15.45:0/2050747379 reap_dead start 2024-09-18T14:01:57.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:01:57.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 1 monclient: found mon.noname-i 2024-09-18T14:01:57.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:57.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T14:01:57.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 1 -- 172.21.15.45:0/2050747379 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43640121f0 con 0x7f4364007360 2024-09-18T14:01:57.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 10 monclient: _finish_auth 0 2024-09-18T14:01:57.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:57.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 1 -- 172.21.15.45:0/2050747379 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4368001770 con 0x7f4364007360 2024-09-18T14:01:57.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:57.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:01:57.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f436f7fe640 10 monclient: dump: 2024-09-18T14:01:57.274 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:57.274 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:57.274 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:57.274 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:57.274 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:57.274 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:57.274 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:57.275 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f437ef9d640 5 monclient: authenticate success, global_id 4732 2024-09-18T14:01:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.267+0000 7f437ef9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.268+0000 7f436f7fe640 1 -- 172.21.15.45:0/2050747379 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f43680018d0 con 0x7f4364007360 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.268+0000 7f436f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.268+0000 7f436f7fe640 1 -- 172.21.15.45:0/2050747379 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4368004860 con 0x7f4364007360 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.268+0000 7f437ef9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.268+0000 7f437ef9d640 4 set_mon_vals no callback set 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.268+0000 7f436f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.268+0000 7f436f7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.268+0000 7f436f7fe640 10 monclient: dump: 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.269+0000 7f437ef9d640 10 monclient: shutdown 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.269+0000 7f437ef9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:57.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.269+0000 7f437ef9d640 1 -- 172.21.15.45:0/2050747379 >> v1:172.21.15.84:6792/0 conn(0x7f4364007360 legacy=0x7f4364009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.269+0000 7f437ef9d640 1 -- 172.21.15.45:0/2050747379 shutdown_connections 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.269+0000 7f437ef9d640 1 -- 172.21.15.45:0/2050747379 >> 172.21.15.45:0/2050747379 conn(0x7f43780630b0 msgr2=0x7f43780634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.269+0000 7f437ef9d640 1 -- 172.21.15.45:0/2050747379 shutdown_connections 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.269+0000 7f437ef9d640 1 -- 172.21.15.45:0/2050747379 wait complete. 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.272+0000 7f437ef9d640 10 monclient: build_initial_monmap 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.272+0000 7f437ef9d640 10 monclient: monmap: 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:01:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:01:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:01:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:01:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:01:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:01:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:01:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:01:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:01:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:01:57.279 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:57.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.272+0000 7f437ef9d640 1 Processor -- start 2024-09-18T14:01:57.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.273+0000 7f437ef9d640 1 -- start start 2024-09-18T14:01:57.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.273+0000 7f437ef9d640 10 monclient: init 2024-09-18T14:01:57.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.274+0000 7f437ef9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:01:57.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.274+0000 7f437ef9d640 10 monclient: _add_conns ranks=[8,2,1,3,6,7,5,0,4] 2024-09-18T14:01:57.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.274+0000 7f437ef9d640 10 monclient(hunting): picked mon.noname-e con 0x7f4364007360 addr v1:172.21.15.45:6793/0 2024-09-18T14:01:57.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.274+0000 7f437ef9d640 10 monclient(hunting): picked mon.noname-b con 0x7f437809b610 addr v1:172.21.15.45:6790/0 2024-09-18T14:01:57.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.274+0000 7f437ef9d640 10 monclient(hunting): picked mon.noname-f con 0x7f4378144110 addr v1:172.21.15.84:6789/0 2024-09-18T14:01:57.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.274+0000 7f437ef9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43780a2560 con 0x7f4378144110 2024-09-18T14:01:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.274+0000 7f437ef9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437811da50 con 0x7f437809b610 2024-09-18T14:01:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.274+0000 7f437ef9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437809d310 con 0x7f4364007360 2024-09-18T14:01:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.274+0000 7f437ef9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:01:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.274+0000 7f437ef9d640 10 monclient(hunting): authenticate will time out at 932.612619s 2024-09-18T14:01:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.275+0000 7f436ffff640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7f437809b610 legacy=0x7f43781429f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:01:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.275+0000 7f437cd39640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f4364007360 0x7f43781315d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:52868/0 (socket says 172.21.15.45:52868) 2024-09-18T14:01:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.275+0000 7f437cd39640 1 -- 172.21.15.45:0/1012410574 learned_addr learned my addr 172.21.15.45:0/1012410574 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:01:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.276+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500658211 0 0) 0x7f437809d310 con 0x7f4364007360 2024-09-18T14:01:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.276+0000 7f436cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:01:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.276+0000 7f436cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:01:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.276+0000 7f436cff9640 10 monclient(hunting): my global_id is 4751 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.276+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4364003750 con 0x7f4364007360 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.277+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1447028020 0 0) 0x7f4364003750 con 0x7f4364007360 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.277+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f437809d310 con 0x7f4364007360 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.277+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4374001a90 con 0x7f4364007360 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.277+0000 7f436cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.277+0000 7f436cff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.277+0000 7f436cff9640 10 monclient(hunting): dump: 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.277+0000 7f436cff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3737361276 0 0) 0x7f437809d310 con 0x7f4364007360 2024-09-18T14:01:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 >> v1:172.21.15.45:6790/0 conn(0x7f437809b610 legacy=0x7f43781429f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 >> v1:172.21.15.84:6789/0 conn(0x7f4378144110 legacy=0x7f43781464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 10 monclient: _finish_hunting 0 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 1 monclient: found mon.e 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4378133be0 con 0x7f4364007360 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 10 monclient: _finish_auth 0 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f437ef9d640 5 monclient: authenticate success, global_id 4751 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f437ef9d640 10 monclient: _renew_subs 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f437ef9d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f437ef9d640 1 -- 172.21.15.45:0/1012410574 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437809b9e0 con 0x7f4364007360 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f437ef9d640 10 monclient: _renew_subs 2024-09-18T14:01:57.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f437ef9d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f437ef9d640 1 -- 172.21.15.45:0/1012410574 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f437811d6e0 con 0x7f4364007360 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f43740048b0 con 0x7f4364007360 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4374004d60 con 0x7f4364007360 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.280+0000 7f436e7fc640 4 set_mon_vals no callback set 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.281+0000 7f436cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.281+0000 7f436cff9640 10 monclient: dump: 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:01:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.281+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f437400fd30 con 0x7f4364007360 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.282+0000 7f437ef9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.282+0000 7f437ef9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:57.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.282+0000 7f437ef9d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:57.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.282+0000 7f437ef9d640 1 -- 172.21.15.45:0/1012410574 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4358005010 con 0x7f4364007360 2024-09-18T14:01:57.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.282+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(130..130 src has 1..130) v4 ==== 3939+0+0 (unknown 3552224666 0 0) 0x7f4374046ae0 con 0x7f4364007360 2024-09-18T14:01:57.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.282+0000 7f436ffff640 10 monclient: get_auth_request con 0x7f436403ad50 auth_method 0 2024-09-18T14:01:57.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.287+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f4374015240 con 0x7f4364007360 2024-09-18T14:01:57.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.287+0000 7f436cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:01:57.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.287+0000 7f436cff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:01:57.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.514+0000 7f437ef9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:01:57.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.514+0000 7f437ef9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:01:57.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.514+0000 7f437ef9d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:01:57.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.514+0000 7f437ef9d640 1 -- 172.21.15.45:0/1012410574 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4358005840 con 0x7f4364007360 2024-09-18T14:01:57.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.514+0000 7f436cff9640 1 -- 172.21.15.45:0/1012410574 <== mon.8 v1:172.21.15.45:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2664 (unknown 254746657 0 2575821732) 0x7f4374014c40 con 0x7f4364007360 2024-09-18T14:01:57.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.514+0000 7f436cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:01:57.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.514+0000 7f436cff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:01:57.515 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":56,"quorum":[0,1,5,7,8],"quorum_names":["a","f","h","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:01:57.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.516+0000 7f437ef9d640 1 -- 172.21.15.45:0/1012410574 >> v1:172.21.15.84:6812/31478 conn(0x7f436403ad50 legacy=0x7f436403d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:57.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.516+0000 7f437ef9d640 10 monclient: shutdown 2024-09-18T14:01:57.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.516+0000 7f437ef9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:01:57.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.516+0000 7f437ef9d640 1 -- 172.21.15.45:0/1012410574 >> v1:172.21.15.45:6793/0 conn(0x7f4364007360 legacy=0x7f43781315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:01:57.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.516+0000 7f437ef9d640 1 -- 172.21.15.45:0/1012410574 shutdown_connections 2024-09-18T14:01:57.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.516+0000 7f437ef9d640 1 -- 172.21.15.45:0/1012410574 >> 172.21.15.45:0/1012410574 conn(0x7f43780630b0 msgr2=0x7f437806c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:01:57.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.517+0000 7f437ef9d640 1 -- 172.21.15.45:0/1012410574 shutdown_connections 2024-09-18T14:01:57.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:01:57.517+0000 7f437ef9d640 1 -- 172.21.15.45:0/1012410574 wait complete. 2024-09-18T14:02:00.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:02:00.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.638+0000 7f75b6f14640 10 monclient: get_monmap_and_config 2024-09-18T14:02:00.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.638+0000 7f75b6f14640 10 monclient: build_initial_monmap 2024-09-18T14:02:00.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.638+0000 7f75b6f14640 10 monclient: monmap: 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:02:00.638986+0000 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:02:00.638986+0000 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:00.640 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:00.640 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:00.640 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.638+0000 7f75b6f14640 1 Processor -- start 2024-09-18T14:02:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.638+0000 7f75b6f14640 1 -- start start 2024-09-18T14:02:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.638+0000 7f75b6f14640 10 monclient: init 2024-09-18T14:02:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b6f14640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b6f14640 10 monclient: _add_conns ranks=[4,6,8,2,5,3,0,1,7] 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b6f14640 10 monclient(hunting): picked mon.noname-c con 0x7f75b009f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b6f14640 10 monclient(hunting): picked mon.noname-d con 0x7f75b009fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b6f14640 10 monclient(hunting): picked mon.noname-e con 0x7f75b009ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b6f14640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b005ed70 con 0x7f75b009f7d0 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b6f14640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b009dfc0 con 0x7f75b009fba0 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b6f14640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b009dce0 con 0x7f75b009ff70 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b6f14640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b6f14640 10 monclient(hunting): authenticate will time out at 935.977413s 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b4cb0640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f75b009f7d0 legacy=0x7f75b0132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b54b1640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f75b009ff70 0x7f75b013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:55588/0 (socket says 172.21.15.45:55588) 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75a7fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f75b009fba0 0x7f75b0137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:53872/0 (socket says 172.21.15.45:53872) 2024-09-18T14:02:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.639+0000 7f75b54b1640 1 -- 172.21.15.45:0/3466930626 learned_addr learned my addr 172.21.15.45:0/3466930626 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:00.840+0000 7f75b4cb0640 1 -- 172.21.15.45:0/3466930626 >> v1:172.21.15.45:6791/0 conn(0x7f75b009f7d0 legacy=0x7f75b0132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:02:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:01.241+0000 7f75b4cb0640 1 -- 172.21.15.45:0/3466930626 >> v1:172.21.15.45:6791/0 conn(0x7f75b009f7d0 legacy=0x7f75b0132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:02:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:02.042+0000 7f75b4cb0640 1 -- 172.21.15.45:0/3466930626 >> v1:172.21.15.45:6791/0 conn(0x7f75b009f7d0 legacy=0x7f75b0132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:02:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:02.531+0000 7f75a7fff640 1 -- 172.21.15.45:0/3466930626 >> v1:172.21.15.45:6792/0 conn(0x7f75b009fba0 legacy=0x7f75b0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-18T14:02:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:02.531+0000 7f75a7fff640 1 -- 172.21.15.45:0/3466930626 >> v1:172.21.15.45:6792/0 conn(0x7f75b009fba0 legacy=0x7f75b0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:02:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:02.531+0000 7f75a7fff640 1 --1- 172.21.15.45:0/3466930626 >> v1:172.21.15.45:6792/0 conn(0x7f75b009fba0 0x7f75b0137a60 :-1 s=OPENED pgs=142 cs=1 l=1).handle_message read tag failed 2024-09-18T14:02:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:02.531+0000 7f75a7fff640 1 --1- 172.21.15.45:0/3466930626 >> v1:172.21.15.45:6792/0 conn(0x7f75b009fba0 0x7f75b0137a60 :-1 s=OPENED pgs=142 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:02:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:02.531+0000 7f75a77fe640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.45:6792/0 2024-09-18T14:02:03.541 INFO:tasks.rados.rados.0.smithi084.stdout:596: done (1 left) 2024-09-18T14:02:03.541 INFO:tasks.rados.rados.0.smithi084.stdout:597: delete oid 50 current snap is 57 2024-09-18T14:02:03.541 INFO:tasks.rados.rados.0.smithi084.stdout:594: finishing copy_from racing read to smithi08434663-37 2024-09-18T14:02:03.541 INFO:tasks.rados.rados.0.smithi084.stdout:594: finishing copy_from to smithi08434663-37 2024-09-18T14:02:03.542 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 107 (ObjNum 138 snap 29 seq_num 138) dirty exists 2024-09-18T14:02:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.542+0000 7f75a77fe640 1 -- 172.21.15.45:0/3466930626 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3544442980 0 0) 0x7f75b009dce0 con 0x7f75b009ff70 2024-09-18T14:02:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.542+0000 7f75a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.542+0000 7f75a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.542+0000 7f75a77fe640 10 monclient(hunting): my global_id is 4760 2024-09-18T14:02:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.542+0000 7f75a77fe640 1 -- 172.21.15.45:0/3466930626 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75980034e0 con 0x7f75b009ff70 2024-09-18T14:02:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 1 -- 172.21.15.45:0/3466930626 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3677995249 0 0) 0x7f75980034e0 con 0x7f75b009ff70 2024-09-18T14:02:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 1 -- 172.21.15.45:0/3466930626 >> v1:172.21.15.45:6792/0 conn(0x7f75b009fba0 legacy=0x7f75b0137a60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 1 -- 172.21.15.45:0/3466930626 >> v1:172.21.15.45:6791/0 conn(0x7f75b009f7d0 legacy=0x7f75b0132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:02:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 10 monclient: _finish_hunting 0 2024-09-18T14:02:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 1 monclient: found mon.noname-e 2024-09-18T14:02:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:02:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 1 -- 172.21.15.45:0/3466930626 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b00a1c00 con 0x7f75b009ff70 2024-09-18T14:02:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 10 monclient: _finish_auth 0 2024-09-18T14:02:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 1 -- 172.21.15.45:0/3466930626 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f75a8001b50 con 0x7f75b009ff70 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75b6f14640 5 monclient: authenticate success, global_id 4760 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75b6f14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75a77fe640 10 monclient: dump: 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.543+0000 7f75b6f14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.544+0000 7f75a77fe640 1 -- 172.21.15.45:0/3466930626 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f75a8001cb0 con 0x7f75b009ff70 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.544+0000 7f75a77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.544+0000 7f75a77fe640 1 -- 172.21.15.45:0/3466930626 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f75a8004be0 con 0x7f75b009ff70 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.544+0000 7f75a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.544+0000 7f75a77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.544+0000 7f75a77fe640 10 monclient: dump: 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:03.548 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:03.549 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:03.549 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:03.549 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.544+0000 7f75b6f14640 10 monclient: get_monmap_and_config success 2024-09-18T14:02:03.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.544+0000 7f75b6f14640 4 set_mon_vals no callback set 2024-09-18T14:02:03.550 INFO:tasks.rados.rados.0.smithi084.stdout:594: done (1 left) 2024-09-18T14:02:03.550 INFO:tasks.rados.rados.0.smithi084.stdout:597: done (0 left) 2024-09-18T14:02:03.550 INFO:tasks.rados.rados.0.smithi084.stdout:598: delete oid 49 current snap is 57 2024-09-18T14:02:03.550 INFO:tasks.rados.rados.0.smithi084.stdout:598: done (0 left) 2024-09-18T14:02:03.550 INFO:tasks.rados.rados.0.smithi084.stdout:599: write (excl) oid 39 current snap is 57 2024-09-18T14:02:03.550 INFO:tasks.rados.rados.0.smithi084.stdout:599: seq_num 245 ranges {517715=559301,1870206=247509} 2024-09-18T14:02:03.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.545+0000 7f75b6f14640 10 monclient: shutdown 2024-09-18T14:02:03.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.545+0000 7f75b6f14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.545+0000 7f75b6f14640 1 -- 172.21.15.45:0/3466930626 >> v1:172.21.15.45:6793/0 conn(0x7f75b009ff70 legacy=0x7f75b013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.545+0000 7f75b6f14640 1 -- 172.21.15.45:0/3466930626 shutdown_connections 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.545+0000 7f75b6f14640 1 -- 172.21.15.45:0/3466930626 >> 172.21.15.45:0/3466930626 conn(0x7f75b00630b0 msgr2=0x7f75b00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.545+0000 7f75b6f14640 1 -- 172.21.15.45:0/3466930626 shutdown_connections 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.545+0000 7f75b6f14640 1 -- 172.21.15.45:0/3466930626 wait complete. 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.545+0000 7f75b6f14640 10 monclient: build_initial_monmap 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.545+0000 7f75b6f14640 10 monclient: monmap: 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:03.555 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.546+0000 7f75b6f14640 1 Processor -- start 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.546+0000 7f75b6f14640 1 -- start start 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.546+0000 7f75b6f14640 10 monclient: init 2024-09-18T14:02:03.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b6f14640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:03.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b6f14640 10 monclient: _add_conns ranks=[0,2,4,3,5,8,7,6,1] 2024-09-18T14:02:03.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b6f14640 10 monclient(hunting): picked mon.noname-a con 0x7f75b009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:02:03.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b6f14640 10 monclient(hunting): picked mon.noname-b con 0x7f75b009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:02:03.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b6f14640 10 monclient(hunting): picked mon.noname-c con 0x7f75b009ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:02:03.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b6f14640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b009d310 con 0x7f75b009f7d0 2024-09-18T14:02:03.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b6f14640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b00a2560 con 0x7f75b009fba0 2024-09-18T14:02:03.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b6f14640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b0142670 con 0x7f75b009ff70 2024-09-18T14:02:03.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b6f14640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:03.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b54b1640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f75b009ff70 legacy=0x7f75b014a250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:02:03.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75a7fff640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7f75b009fba0 legacy=0x7f75b0146b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:02:03.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b6f14640 10 monclient(hunting): authenticate will time out at 938.885190s 2024-09-18T14:02:03.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b4cb0640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f75b009f7d0 0x7f75b0131760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35664/0 (socket says 172.21.15.45:35664) 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.547+0000 7f75b4cb0640 1 -- 172.21.15.45:0/2475890401 learned_addr learned my addr 172.21.15.45:0/2475890401 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.551+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196838720 0 0) 0x7f75b009d310 con 0x7f75b009f7d0 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.551+0000 7f75a4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.551+0000 7f75a4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.551+0000 7f75a4ff9640 10 monclient(hunting): my global_id is 4788 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.551+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f759c0034e0 con 0x7f75b009f7d0 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2120314210 0 0) 0x7f759c0034e0 con 0x7f75b009f7d0 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f75b009d310 con 0x7f75b009f7d0 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f75ac004440 con 0x7f75b009f7d0 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 10 monclient(hunting): dump: 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:03.558 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:03.559 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3741577447 0 0) 0x7f75b009d310 con 0x7f75b009f7d0 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 >> v1:172.21.15.45:6791/0 conn(0x7f75b009ff70 legacy=0x7f75b014a250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 >> v1:172.21.15.45:6790/0 conn(0x7f75b009fba0 legacy=0x7f75b0146b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 1 monclient: found mon.a 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b01218b0 con 0x7f75b009f7d0 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 10 monclient: _finish_auth 0 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75b6f14640 5 monclient: authenticate success, global_id 4788 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75b6f14640 10 monclient: _renew_subs 2024-09-18T14:02:03.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75b6f14640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75b6f14640 1 -- 172.21.15.45:0/2475890401 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b011fdf0 con 0x7f75b009f7d0 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75b6f14640 10 monclient: _renew_subs 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75b6f14640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75b6f14640 1 -- 172.21.15.45:0/2475890401 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b01209c0 con 0x7f75b009f7d0 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f75ac0049d0 con 0x7f75b009f7d0 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f75ac004f40 con 0x7f75b009f7d0 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a6ffd640 4 set_mon_vals no callback set 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.552+0000 7f75a4ff9640 10 monclient: dump: 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:03.561 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.553+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f75ac0050a0 con 0x7f75b009f7d0 2024-09-18T14:02:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.553+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(130..130 src has 1..130) v4 ==== 3939+0+0 (unknown 3552224666 0 0) 0x7f75ac046c90 con 0x7f75b009f7d0 2024-09-18T14:02:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.553+0000 7f75a7fff640 10 monclient: get_auth_request con 0x7f759c039cb0 auth_method 0 2024-09-18T14:02:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.553+0000 7f75b6f14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.553+0000 7f75b6f14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.553+0000 7f75b6f14640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:02:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.553+0000 7f75b6f14640 1 -- 172.21.15.45:0/2475890401 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7590005010 con 0x7f75b009f7d0 2024-09-18T14:02:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.558+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f75ac015430 con 0x7f75b009f7d0 2024-09-18T14:02:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.558+0000 7f75a4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.558+0000 7f75a4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:02:03.564 INFO:tasks.rados.rados.0.smithi084.stdout:599: writing smithi08434663-39 from 517715 to 1077016 tid 1 2024-09-18T14:02:03.565 INFO:tasks.rados.rados.0.smithi084.stdout:599: writing smithi08434663-39 from 1870206 to 2117715 tid 2 2024-09-18T14:02:03.565 INFO:tasks.rados.rados.0.smithi084.stdout:600: write oid 9 current snap is 57 2024-09-18T14:02:03.565 INFO:tasks.rados.rados.0.smithi084.stdout:600: seq_num 246 ranges {534522=634550,1812335=322187} 2024-09-18T14:02:03.565 INFO:tasks.rados.rados.0.smithi084.stdout:600: writing smithi08434663-9 from 534522 to 1169072 tid 1 2024-09-18T14:02:03.570 INFO:tasks.rados.rados.0.smithi084.stdout:600: writing smithi08434663-9 from 1812335 to 2134522 tid 2 2024-09-18T14:02:03.570 INFO:tasks.rados.rados.0.smithi084.stdout:601: write oid 19 current snap is 57 2024-09-18T14:02:03.570 INFO:tasks.rados.rados.0.smithi084.stdout:599: finishing write tid 1 to smithi08434663-39 2024-09-18T14:02:03.570 INFO:tasks.rados.rados.0.smithi084.stdout:599: finishing write tid 2 to smithi08434663-39 2024-09-18T14:02:03.571 INFO:tasks.rados.rados.0.smithi084.stdout:599: finishing write tid 3 to smithi08434663-39 2024-09-18T14:02:03.571 INFO:tasks.rados.rados.0.smithi084.stdout:599: finishing write tid 4 to smithi08434663-39 2024-09-18T14:02:03.571 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 185 (ObjNum 245 snap 57 seq_num 245) dirty exists 2024-09-18T14:02:03.571 INFO:tasks.rados.rados.0.smithi084.stdout:599: left oid 39 (ObjNum 245 snap 57 seq_num 245) 2024-09-18T14:02:03.571 INFO:tasks.rados.rados.0.smithi084.stdout:600: finishing write tid 1 to smithi08434663-9 2024-09-18T14:02:03.572 INFO:tasks.rados.rados.0.smithi084.stdout:601: seq_num 247 ranges {551329=709799,1870817=280512} 2024-09-18T14:02:03.577 INFO:tasks.rados.rados.0.smithi084.stdout:601: writing smithi08434663-19 from 551329 to 1261128 tid 1 2024-09-18T14:02:03.582 INFO:tasks.rados.rados.0.smithi084.stdout:601: writing smithi08434663-19 from 1870817 to 2151329 tid 2 2024-09-18T14:02:03.582 INFO:tasks.rados.rados.0.smithi084.stdout:599: done (2 left) 2024-09-18T14:02:03.582 INFO:tasks.rados.rados.0.smithi084.stdout:602: delete oid 26 current snap is 57 2024-09-18T14:02:03.582 INFO:tasks.rados.rados.0.smithi084.stdout:600: finishing write tid 2 to smithi08434663-9 2024-09-18T14:02:03.582 INFO:tasks.rados.rados.0.smithi084.stdout:600: finishing write tid 3 to smithi08434663-9 2024-09-18T14:02:03.582 INFO:tasks.rados.rados.0.smithi084.stdout:600: finishing write tid 4 to smithi08434663-9 2024-09-18T14:02:03.582 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 214 (ObjNum 246 snap 57 seq_num 246) dirty exists 2024-09-18T14:02:03.583 INFO:tasks.rados.rados.0.smithi084.stdout:600: left oid 9 (ObjNum 246 snap 57 seq_num 246) 2024-09-18T14:02:03.584 INFO:tasks.rados.rados.0.smithi084.stdout:600: done (2 left) 2024-09-18T14:02:03.584 INFO:tasks.rados.rados.0.smithi084.stdout:602: done (1 left) 2024-09-18T14:02:03.584 INFO:tasks.rados.rados.0.smithi084.stdout:603: write oid 21 current snap is 57 2024-09-18T14:02:03.584 INFO:tasks.rados.rados.0.smithi084.stdout:603: seq_num 249 ranges {584943=460297,1587781=597162} 2024-09-18T14:02:03.590 INFO:tasks.rados.rados.0.smithi084.stdout:603: writing smithi08434663-21 from 584943 to 1045240 tid 1 2024-09-18T14:02:03.596 INFO:tasks.rados.rados.0.smithi084.stdout:603: writing smithi08434663-21 from 1587781 to 2184943 tid 2 2024-09-18T14:02:03.596 INFO:tasks.rados.rados.0.smithi084.stdout:604: snap_remove snap 54 2024-09-18T14:02:03.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.794+0000 7f75b6f14640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:02:03.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.794+0000 7f75b6f14640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:02:03.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.794+0000 7f75b6f14640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:02:03.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.794+0000 7f75b6f14640 1 -- 172.21.15.45:0/2475890401 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7590005840 con 0x7f75b009f7d0 2024-09-18T14:02:03.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.795+0000 7f75a4ff9640 1 -- 172.21.15.45:0/2475890401 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2670 (unknown 254746657 0 364774853) 0x7f75ac014e30 con 0x7f75b009f7d0 2024-09-18T14:02:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.795+0000 7f75a4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:02:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.795+0000 7f75a4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:02:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":58,"quorum":[0,1,5,6,7,8],"quorum_names":["a","f","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:02:03.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.796+0000 7f75b6f14640 1 -- 172.21.15.45:0/2475890401 >> v1:172.21.15.84:6812/31478 conn(0x7f759c039cb0 legacy=0x7f759c03c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:03.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.796+0000 7f75b6f14640 10 monclient: shutdown 2024-09-18T14:02:03.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.796+0000 7f75b6f14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:03.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.796+0000 7f75b6f14640 1 -- 172.21.15.45:0/2475890401 >> v1:172.21.15.45:6789/0 conn(0x7f75b009f7d0 legacy=0x7f75b0131760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.796+0000 7f75b6f14640 1 -- 172.21.15.45:0/2475890401 shutdown_connections 2024-09-18T14:02:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.796+0000 7f75b6f14640 1 -- 172.21.15.45:0/2475890401 >> 172.21.15.45:0/2475890401 conn(0x7f75b00630b0 msgr2=0x7f75b00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.797+0000 7f75b6f14640 1 -- 172.21.15.45:0/2475890401 shutdown_connections 2024-09-18T14:02:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.797+0000 7f75b6f14640 1 -- 172.21.15.45:0/2475890401 wait complete. 2024-09-18T14:02:03.807 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-09-18T14:02:03.807 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon a 2024-09-18T14:02:03.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:02:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.904+0000 7f864d494640 10 monclient: get_monmap_and_config 2024-09-18T14:02:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.904+0000 7f864d494640 10 monclient: build_initial_monmap 2024-09-18T14:02:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.904+0000 7f864d494640 10 monclient: monmap: 2024-09-18T14:02:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:02:03.905057+0000 2024-09-18T14:02:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:02:03.905057+0000 2024-09-18T14:02:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:03.906 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:03.906 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:03.906 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:03.906 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:03.906 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.904+0000 7f864d494640 1 Processor -- start 2024-09-18T14:02:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.904+0000 7f864d494640 1 -- start start 2024-09-18T14:02:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.904+0000 7f864d494640 10 monclient: init 2024-09-18T14:02:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f864d494640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f864d494640 10 monclient: _add_conns ranks=[4,1,8,5,3,0,2,6,7] 2024-09-18T14:02:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f864d494640 10 monclient(hunting): picked mon.noname-c con 0x7f864806e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:02:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f864d494640 10 monclient(hunting): picked mon.noname-f con 0x7f86480a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:02:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f864d494640 10 monclient(hunting): picked mon.noname-e con 0x7f86480a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:02:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f864d494640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864805ed70 con 0x7f86480a0cb0 2024-09-18T14:02:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8646ffd640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f864806e810 legacy=0x7f86480a05a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:02:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f864d494640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8648097620 con 0x7f864806e810 2024-09-18T14:02:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f864d494640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8648097340 con 0x7f86480a1080 2024-09-18T14:02:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f864d494640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f864d494640 10 monclient(hunting): authenticate will time out at 939.243413s 2024-09-18T14:02:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f86477fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f86480a1080 0x7f864813af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:55594/0 (socket says 172.21.15.45:55594) 2024-09-18T14:02:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f86477fe640 1 -- 172.21.15.45:0/4238210895 learned_addr learned my addr 172.21.15.45:0/4238210895 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 1 -- 172.21.15.45:0/4238210895 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604041937 0 0) 0x7f8648097340 con 0x7f86480a1080 2024-09-18T14:02:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 10 monclient(hunting): my global_id is 4769 2024-09-18T14:02:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 1 -- 172.21.15.45:0/4238210895 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86300034e0 con 0x7f86480a1080 2024-09-18T14:02:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 1 -- 172.21.15.45:0/4238210895 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1643932812 0 0) 0x7f86300034e0 con 0x7f86480a1080 2024-09-18T14:02:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 1 -- 172.21.15.45:0/4238210895 >> v1:172.21.15.45:6791/0 conn(0x7f864806e810 legacy=0x7f86480a05a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:02:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 1 -- 172.21.15.45:0/4238210895 >> v1:172.21.15.84:6789/0 conn(0x7f86480a0cb0 legacy=0x7f8648137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:02:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 1 monclient: found mon.noname-e 2024-09-18T14:02:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:02:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 1 -- 172.21.15.45:0/4238210895 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86480a1660 con 0x7f86480a1080 2024-09-18T14:02:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 10 monclient: _finish_auth 0 2024-09-18T14:02:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 1 -- 172.21.15.45:0/4238210895 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8640001830 con 0x7f86480a1080 2024-09-18T14:02:03.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:03.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:02:03.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f8645ffb640 10 monclient: dump: 2024-09-18T14:02:03.919 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:03.919 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:03.919 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.919 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.919 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:03.919 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f864d494640 5 monclient: authenticate success, global_id 4769 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.905+0000 7f864d494640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:02:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f8645ffb640 1 -- 172.21.15.45:0/4238210895 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8640001a10 con 0x7f86480a1080 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f8645ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f8645ffb640 1 -- 172.21.15.45:0/4238210895 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8640004ba0 con 0x7f86480a1080 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f864d494640 10 monclient: get_monmap_and_config success 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f864d494640 4 set_mon_vals no callback set 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f8645ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f8645ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f8645ffb640 10 monclient: dump: 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f864d494640 10 monclient: shutdown 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f864d494640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f864d494640 1 -- 172.21.15.45:0/4238210895 >> v1:172.21.15.45:6793/0 conn(0x7f86480a1080 legacy=0x7f864813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:03.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f864d494640 1 -- 172.21.15.45:0/4238210895 shutdown_connections 2024-09-18T14:02:03.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f864d494640 1 -- 172.21.15.45:0/4238210895 >> 172.21.15.45:0/4238210895 conn(0x7f86480a5bb0 msgr2=0x7f86480a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:03.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f864d494640 1 -- 172.21.15.45:0/4238210895 shutdown_connections 2024-09-18T14:02:03.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.906+0000 7f864d494640 1 -- 172.21.15.45:0/4238210895 wait complete. 2024-09-18T14:02:03.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.907+0000 7f864d494640 10 monclient: build_initial_monmap 2024-09-18T14:02:03.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.907+0000 7f864d494640 10 monclient: monmap: 2024-09-18T14:02:03.923 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:03.923 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:02:03.923 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:02:03.923 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:02:03.923 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:03.924 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:02:03.924 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:03.924 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:03.924 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:03.924 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:03.924 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:03.925 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:03.925 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:03.925 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:03.925 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:03.925 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.907+0000 7f864d494640 1 Processor -- start 2024-09-18T14:02:03.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.907+0000 7f864d494640 1 -- start start 2024-09-18T14:02:03.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.907+0000 7f864d494640 10 monclient: init 2024-09-18T14:02:03.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.907+0000 7f864d494640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:03.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.907+0000 7f864d494640 10 monclient: _add_conns ranks=[1,2,3,7,4,5,0,6,8] 2024-09-18T14:02:03.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.907+0000 7f864d494640 10 monclient(hunting): picked mon.noname-f con 0x7f864806e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:02:03.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.907+0000 7f864d494640 10 monclient(hunting): picked mon.noname-b con 0x7f86480a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:02:03.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.907+0000 7f864d494640 10 monclient(hunting): picked mon.noname-g con 0x7f86480a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:02:03.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.907+0000 7f864d494640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864809fa10 con 0x7f864806e810 2024-09-18T14:02:03.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f864d494640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86480a2270 con 0x7f86480a0cb0 2024-09-18T14:02:03.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f864d494640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864811f700 con 0x7f86480a1080 2024-09-18T14:02:03.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f864d494640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:03.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f864d494640 10 monclient(hunting): authenticate will time out at 939.246023s 2024-09-18T14:02:03.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f86467fc640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7f86480a0cb0 legacy=0x7f864811d750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:02:03.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f86477fe640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7f86480a1080 legacy=0x7f864811de60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:02:03.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f8646ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f864806e810 0x7f864811d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:47708/0 (socket says 172.21.15.45:47708) 2024-09-18T14:02:03.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f8646ffd640 1 -- 172.21.15.45:0/3821900868 learned_addr learned my addr 172.21.15.45:0/3821900868 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272939438 0 0) 0x7f864809fa10 con 0x7f864806e810 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 10 monclient(hunting): my global_id is 4681 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86340034e0 con 0x7f864806e810 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 405397700 0 0) 0x7f86340034e0 con 0x7f864806e810 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f864809fa10 con 0x7f864806e810 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86380046c0 con 0x7f864806e810 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 10 monclient(hunting): dump: 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:03.929 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 743846162 0 0) 0x7f864809fa10 con 0x7f864806e810 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 >> v1:172.21.15.84:6790/0 conn(0x7f86480a1080 legacy=0x7f864811de60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 >> v1:172.21.15.45:6790/0 conn(0x7f86480a0cb0 legacy=0x7f864811d750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 10 monclient: _finish_hunting 0 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 1 monclient: found mon.f 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8648120660 con 0x7f864806e810 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 10 monclient: _finish_auth 0 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f862effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f864d494640 5 monclient: authenticate success, global_id 4681 2024-09-18T14:02:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f864d494640 10 monclient: _renew_subs 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f864d494640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.908+0000 7f864d494640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864811eb30 con 0x7f864806e810 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f864d494640 10 monclient: _renew_subs 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f864d494640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f864d494640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8648132840 con 0x7f864806e810 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8638004d70 con 0x7f864806e810 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f862effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8638005220 con 0x7f864806e810 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f862effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f86457fa640 4 set_mon_vals no callback set 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f862effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f862effd640 10 monclient: dump: 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:03.932 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f864d494640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f864d494640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8628006580 con 0x7f8628002830 2024-09-18T14:02:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.909+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f86380101f0 con 0x7f864806e810 2024-09-18T14:02:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(130..130 src has 1..130) v4 ==== 3939+0+0 (unknown 3552224666 0 0) 0x7f8638046ee0 con 0x7f864806e810 2024-09-18T14:02:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780268482 0 0) 0x7f8628006580 con 0x7f8628002830 2024-09-18T14:02:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 10 monclient(hunting): my global_id is 4797 2024-09-18T14:02:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f864809fa10 con 0x7f8628002830 2024-09-18T14:02:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f86477fe640 10 monclient: get_auth_request con 0x7f8634039b90 auth_method 0 2024-09-18T14:02:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1919883205 0 0) 0x7f864809fa10 con 0x7f8628002830 2024-09-18T14:02:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8628006580 con 0x7f8628002830 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f863c0018e0 con 0x7f8628002830 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 10 monclient: dump: 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:03.935 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1046728468 0 0) 0x7f8628006580 con 0x7f8628002830 2024-09-18T14:02:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f862800a0a0 con 0x7f8628002830 2024-09-18T14:02:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.910+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f863c005780 con 0x7f8628002830 2024-09-18T14:02:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.911+0000 7f862effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.911+0000 7f862effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:02:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.911+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 >> v1:172.21.15.45:6789/0 conn(0x7f8628002830 legacy=0x7f8628004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:03.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.911+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 >> v1:172.21.15.45:6789/0 conn(0x7f8628002830 legacy=0x7f8628004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.916+0000 7f864d494640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:02:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.916+0000 7f864d494640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8628009640 con 0x7f8628009020 2024-09-18T14:02:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.916+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132373331 0 0) 0x7f8628009640 con 0x7f8628009020 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 10 monclient(hunting): my global_id is 4806 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8628006580 con 0x7f8628009020 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2832240500 0 0) 0x7f8628006580 con 0x7f8628009020 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8628009640 con 0x7f8628009020 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f863c0020f0 con 0x7f8628009020 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 10 monclient: dump: 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2942032795 0 0) 0x7f8628009640 con 0x7f8628009020 2024-09-18T14:02:03.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8628009900 con 0x7f8628009020 2024-09-18T14:02:03.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7353 (unknown 0 0 4003601369) 0x7f863c0024c0 con 0x7f8628009020 2024-09-18T14:02:03.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:02:03.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:02:03.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 >> v1:172.21.15.45:6789/0 conn(0x7f8628009020 legacy=0x7f8628008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:03.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f862effd640 1 -- 172.21.15.45:0/3821900868 >> v1:172.21.15.45:6789/0 conn(0x7f8628009020 legacy=0x7f8628008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:03.940 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:02:03.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:02:03.940 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:02:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:02:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 58, 2024-09-18T14:02:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:02:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:02:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:02:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:02:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:02:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:02:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:02:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:02:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:02:03.942 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:02:03.943 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:03.943 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:03.943 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:03.943 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:03.943 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:03.943 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:03.943 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:03.943 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:03.943 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:02:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:02:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:02:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:02:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:02:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:02:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:02:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:03.945 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:02:03.946 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:03.947 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:03.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.949 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:02:03.949 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:02:03.949 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:03.949 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:03.949 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:03.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.949 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:02:03.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:02:03.949 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:03.949 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:03.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.951 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:02:03.951 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:02:03.951 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:03.951 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:03.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.951 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:03.951 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:02:03.951 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:03.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:03.952 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:03.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.954 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:02:03.955 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:02:03.955 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:03.955 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:03.955 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:03.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.955 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:02:03.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:02:03.955 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:03.955 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:03.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:03.956 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:02:03.956 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:03.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:03.956 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:03.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.956 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:02:03.956 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:02:03.956 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:03.956 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:03.956 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:03.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.957 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:02:03.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:02:03.957 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:03.957 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:03.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:03.957 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:02:03.957 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:03.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:03.957 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:03.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:03.958 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.959 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:02:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:03.961 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:03.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:03.961 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:02:03.961 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:02:03.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f864d494640 1 -- 172.21.15.45:0/3821900868 >> v1:172.21.15.84:6812/31478 conn(0x7f8634039b90 legacy=0x7f863403c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:03.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f864d494640 10 monclient: shutdown 2024-09-18T14:02:03.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f864d494640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:03.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.917+0000 7f864d494640 1 -- 172.21.15.45:0/3821900868 >> v1:172.21.15.84:6789/0 conn(0x7f864806e810 legacy=0x7f864811d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:03.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.918+0000 7f86477fe640 1 -- 172.21.15.45:0/3821900868 reap_dead start 2024-09-18T14:02:03.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.918+0000 7f86477fe640 1 -- 172.21.15.45:0/3821900868 reap_dead start 2024-09-18T14:02:03.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.918+0000 7f864d494640 1 -- 172.21.15.45:0/3821900868 shutdown_connections 2024-09-18T14:02:03.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.918+0000 7f864d494640 1 -- 172.21.15.45:0/3821900868 >> 172.21.15.45:0/3821900868 conn(0x7f86480a5bb0 msgr2=0x7f8648132cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:03.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.918+0000 7f864d494640 1 -- 172.21.15.45:0/3821900868 shutdown_connections 2024-09-18T14:02:03.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:03.918+0000 7f864d494640 1 -- 172.21.15.45:0/3821900868 wait complete. 2024-09-18T14:02:03.963 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-18T14:02:03.963 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-09-18T14:02:03.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:02:04.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.060+0000 7f5313b5d640 10 monclient: get_monmap_and_config 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.060+0000 7f5313b5d640 10 monclient: build_initial_monmap 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.060+0000 7f5313b5d640 10 monclient: monmap: 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:02:04.061567+0000 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:02:04.061567+0000 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:04.063 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:04.063 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:04.063 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:04.063 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:04.063 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 1 Processor -- start 2024-09-18T14:02:04.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 1 -- start start 2024-09-18T14:02:04.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 10 monclient: init 2024-09-18T14:02:04.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:04.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 10 monclient: _add_conns ranks=[4,0,8,7,1,3,2,5,6] 2024-09-18T14:02:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 10 monclient(hunting): picked mon.noname-c con 0x7f530c09f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:02:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 10 monclient(hunting): picked mon.noname-a con 0x7f530c09fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:02:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 10 monclient(hunting): picked mon.noname-e con 0x7f530c09ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:02:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c05ed70 con 0x7f530c09fba0 2024-09-18T14:02:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c09dfc0 con 0x7f530c09f7d0 2024-09-18T14:02:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c09dce0 con 0x7f530c09ff70 2024-09-18T14:02:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f5313b5d640 10 monclient(hunting): authenticate will time out at 939.399912s 2024-09-18T14:02:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.061+0000 7f53118f9640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f530c09f7d0 legacy=0x7f530c132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:02:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53120fa640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f530c09ff70 0x7f530c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:55602/0 (socket says 172.21.15.45:55602) 2024-09-18T14:02:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53120fa640 1 -- 172.21.15.45:0/2907342505 learned_addr learned my addr 172.21.15.45:0/2907342505 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 1 -- 172.21.15.45:0/2907342505 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497509878 0 0) 0x7f530c09dce0 con 0x7f530c09ff70 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient(hunting): my global_id is 4778 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 1 -- 172.21.15.45:0/2907342505 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f80034e0 con 0x7f530c09ff70 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 1 -- 172.21.15.45:0/2907342505 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158108247 0 0) 0x7f530c05ed70 con 0x7f530c09fba0 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient(hunting): my global_id is 4815 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 1 -- 172.21.15.45:0/2907342505 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f530c09dce0 con 0x7f530c09fba0 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 1 -- 172.21.15.45:0/2907342505 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 820528251 0 0) 0x7f52f80034e0 con 0x7f530c09ff70 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 1 -- 172.21.15.45:0/2907342505 >> v1:172.21.15.45:6791/0 conn(0x7f530c09f7d0 legacy=0x7f530c132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:02:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 1 -- 172.21.15.45:0/2907342505 >> v1:172.21.15.45:6789/0 conn(0x7f530c09fba0 legacy=0x7f530c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient: _finish_hunting 0 2024-09-18T14:02:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 1 monclient: found mon.noname-e 2024-09-18T14:02:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:02:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 1 -- 172.21.15.45:0/2907342505 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0a1c00 con 0x7f530c09ff70 2024-09-18T14:02:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient: _finish_auth 0 2024-09-18T14:02:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 1 -- 172.21.15.45:0/2907342505 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5304001bb0 con 0x7f530c09ff70 2024-09-18T14:02:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f5313b5d640 5 monclient: authenticate success, global_id 4778 2024-09-18T14:02:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f5313b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:02:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient: dump: 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.075 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f5313b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:02:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 1 -- 172.21.15.45:0/2907342505 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5304001d90 con 0x7f530c09ff70 2024-09-18T14:02:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 1 -- 172.21.15.45:0/2907342505 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5304004f20 con 0x7f530c09ff70 2024-09-18T14:02:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f5313b5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:02:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f5313b5d640 4 set_mon_vals no callback set 2024-09-18T14:02:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:02:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.062+0000 7f53108f7640 10 monclient: dump: 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.063+0000 7f5313b5d640 10 monclient: shutdown 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.063+0000 7f5313b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.063+0000 7f5313b5d640 1 -- 172.21.15.45:0/2907342505 >> v1:172.21.15.45:6793/0 conn(0x7f530c09ff70 legacy=0x7f530c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.063+0000 7f5313b5d640 1 -- 172.21.15.45:0/2907342505 shutdown_connections 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.063+0000 7f5313b5d640 1 -- 172.21.15.45:0/2907342505 >> 172.21.15.45:0/2907342505 conn(0x7f530c0630b0 msgr2=0x7f530c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.063+0000 7f5313b5d640 1 -- 172.21.15.45:0/2907342505 shutdown_connections 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.063+0000 7f5313b5d640 1 -- 172.21.15.45:0/2907342505 wait complete. 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.063+0000 7f5313b5d640 10 monclient: build_initial_monmap 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.063+0000 7f5313b5d640 10 monclient: monmap: 2024-09-18T14:02:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 1 Processor -- start 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 1 -- start start 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 10 monclient: init 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 10 monclient: _add_conns ranks=[2,6,4,3,7,0,5,1,8] 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 10 monclient(hunting): picked mon.noname-b con 0x7f530c09f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 10 monclient(hunting): picked mon.noname-d con 0x7f530c09fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 10 monclient(hunting): picked mon.noname-c con 0x7f530c09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c0a2560 con 0x7f530c09f7d0 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f53118f9640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7f530c09f7d0 legacy=0x7f530c141bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:02:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c120730 con 0x7f530c09ff70 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c120140 con 0x7f530c09fba0 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5313b5d640 10 monclient(hunting): authenticate will time out at 939.402565s 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f53120fa640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f530c09ff70 legacy=0x7f530c149e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f53110f8640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f530c09fba0 0x7f530c142300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:53878/0 (socket says 172.21.15.45:53878) 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f53110f8640 1 -- 172.21.15.45:0/3625399226 learned_addr learned my addr 172.21.15.45:0/3625399226 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987452977 0 0) 0x7f530c120140 con 0x7f530c09fba0 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5301ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5301ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5301ffb640 10 monclient(hunting): my global_id is 4641 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.064+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f00034e0 con 0x7f530c09fba0 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 117704220 0 0) 0x7f52f00034e0 con 0x7f530c09fba0 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f530c120140 con 0x7f530c09fba0 2024-09-18T14:02:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f52fc001d90 con 0x7f530c09fba0 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 10 monclient(hunting): dump: 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.082 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.082 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3711997109 0 0) 0x7f530c120140 con 0x7f530c09fba0 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 >> v1:172.21.15.45:6791/0 conn(0x7f530c09ff70 legacy=0x7f530c149e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 >> v1:172.21.15.45:6790/0 conn(0x7f530c09f7d0 legacy=0x7f530c141bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 1 monclient: found mon.d 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:02:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c142a10 con 0x7f530c09fba0 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 10 monclient: _finish_auth 0 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5313b5d640 5 monclient: authenticate success, global_id 4641 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5313b5d640 10 monclient: _renew_subs 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5313b5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5313b5d640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530c120450 con 0x7f530c09fba0 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f52fc0021a0 con 0x7f530c09fba0 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f52fc002750 con 0x7f530c09fba0 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5301ffb640 10 monclient: dump: 2024-09-18T14:02:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5303fff640 4 set_mon_vals no callback set 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5313b5d640 10 monclient: _renew_subs 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5313b5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.065+0000 7f5313b5d640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f530c11f4f0 con 0x7f530c09fba0 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f52fc010120 con 0x7f530c09fba0 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5313b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5313b5d640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f530c04c4a0 con 0x7f530c02aa40 2024-09-18T14:02:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(130..130 src has 1..130) v4 ==== 3939+0+0 (unknown 3552224666 0 0) 0x7f52fc046b30 con 0x7f530c09fba0 2024-09-18T14:02:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696971250 0 0) 0x7f530c04c4a0 con 0x7f530c02aa40 2024-09-18T14:02:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 10 monclient(hunting): my global_id is 4650 2024-09-18T14:02:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f530c120140 con 0x7f530c02aa40 2024-09-18T14:02:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 789245769 0 0) 0x7f530c120140 con 0x7f530c02aa40 2024-09-18T14:02:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f530c04c4a0 con 0x7f530c02aa40 2024-09-18T14:02:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f53120fa640 10 monclient: get_auth_request con 0x7f52f0039c10 auth_method 0 2024-09-18T14:02:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f53080040b0 con 0x7f530c02aa40 2024-09-18T14:02:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:02:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 10 monclient: dump: 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.090 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1647021198 0 0) 0x7f530c04c4a0 con 0x7f530c02aa40 2024-09-18T14:02:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.066+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f530c02ae10 con 0x7f530c02aa40 2024-09-18T14:02:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.067+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5308006390 con 0x7f530c02aa40 2024-09-18T14:02:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.067+0000 7f5301ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.067+0000 7f5301ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:02:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.067+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 >> v1:172.21.15.45:6792/0 conn(0x7f530c02aa40 legacy=0x7f530c05e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.067+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 >> v1:172.21.15.45:6792/0 conn(0x7f530c02aa40 legacy=0x7f530c05e310 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:04.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.072+0000 7f5313b5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:02:04.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.072+0000 7f5313b5d640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f530c04b720 con 0x7f530c11df20 2024-09-18T14:02:04.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295048211 0 0) 0x7f530c04b720 con 0x7f530c11df20 2024-09-18T14:02:04.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 10 monclient(hunting): my global_id is 4659 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f530c04c4a0 con 0x7f530c11df20 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1938422583 0 0) 0x7f530c04c4a0 con 0x7f530c11df20 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f530c04b720 con 0x7f530c11df20 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5308004bb0 con 0x7f530c11df20 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 10 monclient: dump: 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1645634270 0 0) 0x7f530c04b720 con 0x7f530c11df20 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f530c1208f0 con 0x7f530c11df20 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7353 (unknown 0 0 4191886731) 0x7f5308006080 con 0x7f530c11df20 2024-09-18T14:02:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:02:04.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:02:04.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 >> v1:172.21.15.45:6792/0 conn(0x7f530c11df20 legacy=0x7f530c099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.073+0000 7f5301ffb640 1 -- 172.21.15.45:0/3625399226 >> v1:172.21.15.45:6792/0 conn(0x7f530c11df20 legacy=0x7f530c099a40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:04.095 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:02:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:02:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:02:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:02:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 58, 2024-09-18T14:02:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:02:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:02:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:02:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:02:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:02:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:02:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:02:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:02:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:02:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:02:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:02:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:02:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:02:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:02:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:02:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:02:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:02:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:02:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:02:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:02:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:02:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:02:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:02:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:02:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:02:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:02:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:02:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:02:04.114 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:02:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.074+0000 7f5313b5d640 1 -- 172.21.15.45:0/3625399226 >> v1:172.21.15.84:6812/31478 conn(0x7f52f0039c10 legacy=0x7f52f003c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.074+0000 7f5313b5d640 10 monclient: shutdown 2024-09-18T14:02:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.074+0000 7f5313b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.074+0000 7f5313b5d640 1 -- 172.21.15.45:0/3625399226 >> v1:172.21.15.45:6792/0 conn(0x7f530c09fba0 legacy=0x7f530c142300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.074+0000 7f53120fa640 1 -- 172.21.15.45:0/3625399226 reap_dead start 2024-09-18T14:02:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.074+0000 7f53120fa640 1 -- 172.21.15.45:0/3625399226 reap_dead start 2024-09-18T14:02:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.074+0000 7f5313b5d640 1 -- 172.21.15.45:0/3625399226 shutdown_connections 2024-09-18T14:02:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.074+0000 7f5313b5d640 1 -- 172.21.15.45:0/3625399226 >> 172.21.15.45:0/3625399226 conn(0x7f530c0630b0 msgr2=0x7f530c0a45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.074+0000 7f5313b5d640 1 -- 172.21.15.45:0/3625399226 shutdown_connections 2024-09-18T14:02:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.074+0000 7f5313b5d640 1 -- 172.21.15.45:0/3625399226 wait complete. 2024-09-18T14:02:04.117 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-18T14:02:04.117 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-09-18T14:02:04.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:02:04.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.214+0000 7fbd34954640 10 monclient: get_monmap_and_config 2024-09-18T14:02:04.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.214+0000 7fbd34954640 10 monclient: build_initial_monmap 2024-09-18T14:02:04.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.214+0000 7fbd34954640 10 monclient: monmap: 2024-09-18T14:02:04.215 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:04.215 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.215 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:02:04.215569+0000 2024-09-18T14:02:04.215 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:02:04.215569+0000 2024-09-18T14:02:04.216 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:04.216 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.216 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:04.216 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:04.216 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:04.216 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:04.216 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:04.216 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:04.216 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:04.216 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:04.216 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:04.216 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 1 Processor -- start 2024-09-18T14:02:04.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 1 -- start start 2024-09-18T14:02:04.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 10 monclient: init 2024-09-18T14:02:04.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:04.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 10 monclient: _add_conns ranks=[0,4,8,5,3,7,2,1,6] 2024-09-18T14:02:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 10 monclient(hunting): picked mon.noname-a con 0x7fbd3009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:02:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 10 monclient(hunting): picked mon.noname-c con 0x7fbd3009fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:02:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 10 monclient(hunting): picked mon.noname-e con 0x7fbd3009ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:02:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3005ed70 con 0x7fbd3009f7d0 2024-09-18T14:02:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3009dfc0 con 0x7fbd3009fba0 2024-09-18T14:02:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3009dce0 con 0x7fbd3009ff70 2024-09-18T14:02:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd2dd9b640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7fbd3009fba0 legacy=0x7fbd30137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:02:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.215+0000 7fbd34954640 10 monclient(hunting): authenticate will time out at 939.553907s 2024-09-18T14:02:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2e59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbd3009f7d0 0x7fbd30132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35708/0 (socket says 172.21.15.45:35708) 2024-09-18T14:02:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2e59c640 1 -- 172.21.15.45:0/425373238 learned_addr learned my addr 172.21.15.45:0/425373238 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2ed9d640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fbd3009ff70 0x7fbd3013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:55614/0 (socket says 172.21.15.45:55614) 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 1 -- 172.21.15.45:0/425373238 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834983869 0 0) 0x7fbd3005ed70 con 0x7fbd3009f7d0 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient(hunting): my global_id is 4824 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 1 -- 172.21.15.45:0/425373238 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd180034e0 con 0x7fbd3009f7d0 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 1 -- 172.21.15.45:0/425373238 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431388371 0 0) 0x7fbd3009dce0 con 0x7fbd3009ff70 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient(hunting): my global_id is 4787 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 1 -- 172.21.15.45:0/425373238 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd3005ed70 con 0x7fbd3009ff70 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 1 -- 172.21.15.45:0/425373238 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1984326058 0 0) 0x7fbd180034e0 con 0x7fbd3009f7d0 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 1 -- 172.21.15.45:0/425373238 >> v1:172.21.15.45:6793/0 conn(0x7fbd3009ff70 legacy=0x7fbd3013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 1 -- 172.21.15.45:0/425373238 >> v1:172.21.15.45:6791/0 conn(0x7fbd3009fba0 legacy=0x7fbd30137a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:02:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient: _finish_hunting 0 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 1 monclient: found mon.noname-a 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 1 -- 172.21.15.45:0/425373238 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd300a1c00 con 0x7fbd3009f7d0 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient: _finish_auth 0 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 1 -- 172.21.15.45:0/425373238 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd20001760 con 0x7fbd3009f7d0 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd34954640 5 monclient: authenticate success, global_id 4824 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd34954640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient: dump: 2024-09-18T14:02:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd34954640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 1 -- 172.21.15.45:0/425373238 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbd20001940 con 0x7fbd3009f7d0 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 1 -- 172.21.15.45:0/425373238 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd20004ba0 con 0x7fbd3009f7d0 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd2d59a640 10 monclient: dump: 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd34954640 10 monclient: get_monmap_and_config success 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.216+0000 7fbd34954640 4 set_mon_vals no callback set 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.217+0000 7fbd34954640 10 monclient: shutdown 2024-09-18T14:02:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.217+0000 7fbd34954640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.217+0000 7fbd34954640 1 -- 172.21.15.45:0/425373238 >> v1:172.21.15.45:6789/0 conn(0x7fbd3009f7d0 legacy=0x7fbd30132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.217+0000 7fbd34954640 1 -- 172.21.15.45:0/425373238 shutdown_connections 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.217+0000 7fbd34954640 1 -- 172.21.15.45:0/425373238 >> 172.21.15.45:0/425373238 conn(0x7fbd300630b0 msgr2=0x7fbd300634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.217+0000 7fbd34954640 1 -- 172.21.15.45:0/425373238 shutdown_connections 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.217+0000 7fbd34954640 1 -- 172.21.15.45:0/425373238 wait complete. 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.217+0000 7fbd34954640 10 monclient: build_initial_monmap 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.217+0000 7fbd34954640 10 monclient: monmap: 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 1 Processor -- start 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 1 -- start start 2024-09-18T14:02:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 10 monclient: init 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 10 monclient: _add_conns ranks=[6,5,8,4,1,7,2,3,0] 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 10 monclient(hunting): picked mon.noname-d con 0x7fbd3009f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 10 monclient(hunting): picked mon.noname-h con 0x7fbd3009fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 10 monclient(hunting): picked mon.noname-e con 0x7fbd3009ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd300a2560 con 0x7fbd3009fba0 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd30122010 con 0x7fbd3009f7d0 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd30121ca0 con 0x7fbd3009ff70 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd34954640 10 monclient(hunting): authenticate will time out at 939.556739s 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd2ed9d640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fbd3009ff70 0x7fbd30149cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:55616/0 (socket says 172.21.15.45:55616) 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd2ed9d640 1 -- 172.21.15.45:0/3713569103 learned_addr learned my addr 172.21.15.45:0/3713569103 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.218+0000 7fbd2e59c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fbd3009f7d0 0x7fbd30141950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:53904/0 (socket says 172.21.15.45:53904) 2024-09-18T14:02:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280976088 0 0) 0x7fbd30121ca0 con 0x7fbd3009ff70 2024-09-18T14:02:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): my global_id is 4796 2024-09-18T14:02:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd100034e0 con 0x7fbd3009ff70 2024-09-18T14:02:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588025860 0 0) 0x7fbd30122010 con 0x7fbd3009f7d0 2024-09-18T14:02:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): my global_id is 4668 2024-09-18T14:02:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd30121ca0 con 0x7fbd3009f7d0 2024-09-18T14:02:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1037833507 0 0) 0x7fbd30121ca0 con 0x7fbd3009f7d0 2024-09-18T14:02:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd30122010 con 0x7fbd3009f7d0 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd20004ba0 con 0x7fbd3009f7d0 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): dump: 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4147704611 0 0) 0x7fbd100034e0 con 0x7fbd3009ff70 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd30121ca0 con 0x7fbd3009ff70 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881632052 0 0) 0x7fbd300a2560 con 0x7fbd3009fba0 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): my global_id is 4658 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd100034e0 con 0x7fbd3009fba0 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd28001d70 con 0x7fbd3009ff70 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient(hunting): dump: 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 343757588 0 0) 0x7fbd30122010 con 0x7fbd3009f7d0 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 >> v1:172.21.15.45:6793/0 conn(0x7fbd3009ff70 legacy=0x7fbd30149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 >> v1:172.21.15.84:6791/0 conn(0x7fbd3009fba0 legacy=0x7fbd30131870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 monclient: found mon.d 2024-09-18T14:02:04.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd301219c0 con 0x7fbd3009f7d0 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd34954640 5 monclient: authenticate success, global_id 4668 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd34954640 10 monclient: _renew_subs 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd34954640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd34954640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd301420e0 con 0x7fbd3009f7d0 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd34954640 10 monclient: _renew_subs 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd34954640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd34954640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd30132120 con 0x7fbd3009f7d0 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbd20001730 con 0x7fbd3009f7d0 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd20005160 con 0x7fbd3009f7d0 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd2cd99640 4 set_mon_vals no callback set 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.219+0000 7fbd1e7fc640 10 monclient: dump: 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.220+0000 7fbd34954640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.220+0000 7fbd34954640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbd14006580 con 0x7fbd14002830 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.220+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fbd20010130 con 0x7fbd3009f7d0 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(130..130 src has 1..130) v4 ==== 3939+0+0 (unknown 3552224666 0 0) 0x7fbd20046c00 con 0x7fbd3009f7d0 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959927367 0 0) 0x7fbd14006580 con 0x7fbd14002830 2024-09-18T14:02:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 10 monclient(hunting): my global_id is 4805 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd30122010 con 0x7fbd14002830 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd2ed9d640 10 monclient: get_auth_request con 0x7fbd1003a600 auth_method 0 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1597239333 0 0) 0x7fbd30122010 con 0x7fbd14002830 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd14006580 con 0x7fbd14002830 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd24004100 con 0x7fbd14002830 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 10 monclient: dump: 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 172621992 0 0) 0x7fbd14006580 con 0x7fbd14002830 2024-09-18T14:02:04.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd1400a0a0 con 0x7fbd14002830 2024-09-18T14:02:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fbd24005670 con 0x7fbd14002830 2024-09-18T14:02:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:02:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 >> v1:172.21.15.45:6793/0 conn(0x7fbd14002830 legacy=0x7fbd14004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.221+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 >> v1:172.21.15.45:6793/0 conn(0x7fbd14002830 legacy=0x7fbd14004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.227+0000 7fbd34954640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.227+0000 7fbd34954640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbd14007540 con 0x7fbd140068f0 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.227+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276061768 0 0) 0x7fbd14007540 con 0x7fbd140068f0 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.227+0000 7fbd1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.227+0000 7fbd1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.227+0000 7fbd1e7fc640 10 monclient(hunting): my global_id is 4814 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.227+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd14006580 con 0x7fbd140068f0 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3425207902 0 0) 0x7fbd14006580 con 0x7fbd140068f0 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd14007540 con 0x7fbd140068f0 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd240047d0 con 0x7fbd140068f0 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 10 monclient: dump: 2024-09-18T14:02:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.249 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 296037062 0 0) 0x7fbd14007540 con 0x7fbd140068f0 2024-09-18T14:02:04.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbd140087a0 con 0x7fbd140068f0 2024-09-18T14:02:04.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7525 (unknown 0 0 208690652) 0x7fbd240058c0 con 0x7fbd140068f0 2024-09-18T14:02:04.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:02:04.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:02:04.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 >> v1:172.21.15.45:6793/0 conn(0x7fbd140068f0 legacy=0x7fbd14006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd1e7fc640 1 -- 172.21.15.45:0/3713569103 >> v1:172.21.15.45:6793/0 conn(0x7fbd140068f0 legacy=0x7fbd14006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:04.249 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:02:04.249 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 58, 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:02:04.250 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:02:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:02:04.252 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:02:04.253 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:02:04.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.255 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.256 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:02:04.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.258 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.259 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:02:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:02:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:02:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:02:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:02:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:02:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:02:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.262 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:02:04.262 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:02:04.262 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.262 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.262 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:02:04.262 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.262 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.262 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.263 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:02:04.263 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:02:04.263 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.263 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.263 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.263 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:02:04.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:02:04.263 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.263 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.265 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.266 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:02:04.267 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.268 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.269 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:02:04.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.269 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.269 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.269 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:02:04.269 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.269 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.269 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:02:04.269 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:02:04.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd34954640 1 -- 172.21.15.45:0/3713569103 >> v1:172.21.15.84:6812/31478 conn(0x7fbd1003a600 legacy=0x7fbd1003cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd34954640 10 monclient: shutdown 2024-09-18T14:02:04.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd34954640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:04.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd34954640 1 -- 172.21.15.45:0/3713569103 >> v1:172.21.15.45:6792/0 conn(0x7fbd3009f7d0 legacy=0x7fbd30141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd2ed9d640 1 -- 172.21.15.45:0/3713569103 reap_dead start 2024-09-18T14:02:04.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd2ed9d640 1 -- 172.21.15.45:0/3713569103 reap_dead start 2024-09-18T14:02:04.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd34954640 1 -- 172.21.15.45:0/3713569103 shutdown_connections 2024-09-18T14:02:04.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd34954640 1 -- 172.21.15.45:0/3713569103 >> 172.21.15.45:0/3713569103 conn(0x7fbd300630b0 msgr2=0x7fbd300a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:04.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.228+0000 7fbd34954640 1 -- 172.21.15.45:0/3713569103 shutdown_connections 2024-09-18T14:02:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.229+0000 7fbd34954640 1 -- 172.21.15.45:0/3713569103 wait complete. 2024-09-18T14:02:04.272 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-18T14:02:04.272 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon f 2024-09-18T14:02:04.272 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:02:04.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.368+0000 7f2d714af640 10 monclient: get_monmap_and_config 2024-09-18T14:02:04.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.368+0000 7f2d714af640 10 monclient: build_initial_monmap 2024-09-18T14:02:04.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.368+0000 7f2d714af640 10 monclient: monmap: 2024-09-18T14:02:04.369 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:04.369 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:02:04.369899+0000 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:02:04.369899+0000 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:04.370 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.369+0000 7f2d714af640 1 Processor -- start 2024-09-18T14:02:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.369+0000 7f2d714af640 1 -- start start 2024-09-18T14:02:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.369+0000 7f2d714af640 10 monclient: init 2024-09-18T14:02:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 10 monclient: _add_conns ranks=[2,7,8,5,3,0,4,6,1] 2024-09-18T14:02:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 10 monclient(hunting): picked mon.noname-b con 0x7f2d6c09f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:02:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 10 monclient(hunting): picked mon.noname-i con 0x7f2d6c09fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:02:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 10 monclient(hunting): picked mon.noname-e con 0x7f2d6c09ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:02:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c05ed70 con 0x7f2d6c09f7d0 2024-09-18T14:02:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c09dfc0 con 0x7f2d6c09fba0 2024-09-18T14:02:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c09dce0 con 0x7f2d6c09ff70 2024-09-18T14:02:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 10 monclient(hunting): authenticate will time out at 939.708226s 2024-09-18T14:02:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6affd640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7f2d6c09f7d0 legacy=0x7f2d6c132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:02:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6b7fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f2d6c09ff70 0x7f2d6c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:55638/0 (socket says 172.21.15.45:55638) 2024-09-18T14:02:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6b7fe640 1 -- 172.21.15.45:0/2434620158 learned_addr learned my addr 172.21.15.45:0/2434620158 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 1 -- 172.21.15.45:0/2434620158 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605013226 0 0) 0x7f2d6c09dce0 con 0x7f2d6c09ff70 2024-09-18T14:02:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 10 monclient(hunting): my global_id is 4823 2024-09-18T14:02:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 1 -- 172.21.15.45:0/2434620158 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d580034e0 con 0x7f2d6c09ff70 2024-09-18T14:02:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 1 -- 172.21.15.45:0/2434620158 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162138900 0 0) 0x7f2d6c09dfc0 con 0x7f2d6c09fba0 2024-09-18T14:02:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 10 monclient(hunting): my global_id is 4741 2024-09-18T14:02:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 1 -- 172.21.15.45:0/2434620158 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d6c09dce0 con 0x7f2d6c09fba0 2024-09-18T14:02:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 1 -- 172.21.15.45:0/2434620158 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1460022147 0 0) 0x7f2d580034e0 con 0x7f2d6c09ff70 2024-09-18T14:02:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 1 -- 172.21.15.45:0/2434620158 >> v1:172.21.15.84:6792/0 conn(0x7f2d6c09fba0 legacy=0x7f2d6c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 1 -- 172.21.15.45:0/2434620158 >> v1:172.21.15.45:6790/0 conn(0x7f2d6c09f7d0 legacy=0x7f2d6c132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:02:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:02:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 1 monclient: found mon.noname-e 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 1 -- 172.21.15.45:0/2434620158 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0a1c00 con 0x7f2d6c09ff70 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 1 -- 172.21.15.45:0/2434620158 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2d64001bb0 con 0x7f2d6c09ff70 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 5 monclient: authenticate success, global_id 4823 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d6a7fc640 10 monclient: dump: 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.384 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.370+0000 7f2d714af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:02:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d6a7fc640 1 -- 172.21.15.45:0/2434620158 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2d64001d90 con 0x7f2d6c09ff70 2024-09-18T14:02:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d6a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d714af640 10 monclient: get_monmap_and_config success 2024-09-18T14:02:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d714af640 4 set_mon_vals no callback set 2024-09-18T14:02:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d6a7fc640 1 -- 172.21.15.45:0/2434620158 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2d64004f20 con 0x7f2d6c09ff70 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d6a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d6a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d6a7fc640 10 monclient: dump: 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d714af640 10 monclient: shutdown 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d714af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d714af640 1 -- 172.21.15.45:0/2434620158 >> v1:172.21.15.45:6793/0 conn(0x7f2d6c09ff70 legacy=0x7f2d6c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d714af640 1 -- 172.21.15.45:0/2434620158 shutdown_connections 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d714af640 1 -- 172.21.15.45:0/2434620158 >> 172.21.15.45:0/2434620158 conn(0x7f2d6c0630b0 msgr2=0x7f2d6c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d714af640 1 -- 172.21.15.45:0/2434620158 shutdown_connections 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.371+0000 7f2d714af640 1 -- 172.21.15.45:0/2434620158 wait complete. 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.372+0000 7f2d714af640 10 monclient: build_initial_monmap 2024-09-18T14:02:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.372+0000 7f2d714af640 10 monclient: monmap: 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:04.388 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.372+0000 7f2d714af640 1 Processor -- start 2024-09-18T14:02:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.372+0000 7f2d714af640 1 -- start start 2024-09-18T14:02:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.372+0000 7f2d714af640 10 monclient: init 2024-09-18T14:02:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.372+0000 7f2d714af640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.372+0000 7f2d714af640 10 monclient: _add_conns ranks=[1,2,6,3,8,0,4,5,7] 2024-09-18T14:02:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.372+0000 7f2d714af640 10 monclient(hunting): picked mon.noname-f con 0x7f2d6c09f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:02:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.372+0000 7f2d714af640 10 monclient(hunting): picked mon.noname-b con 0x7f2d6c09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:02:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 10 monclient(hunting): picked mon.noname-d con 0x7f2d6c09ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:02:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c09d310 con 0x7f2d6c09f7d0 2024-09-18T14:02:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d63fff640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7f2d6c09fba0 legacy=0x7f2d6c1318f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:02:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c0a2560 con 0x7f2d6c09fba0 2024-09-18T14:02:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c121db0 con 0x7f2d6c09ff70 2024-09-18T14:02:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 10 monclient(hunting): authenticate will time out at 939.711115s 2024-09-18T14:02:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d6b7fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f2d6c09ff70 0x7f2d6c149a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:53906/0 (socket says 172.21.15.45:53906) 2024-09-18T14:02:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d6b7fe640 1 -- 172.21.15.45:0/3476508560 learned_addr learned my addr 172.21.15.45:0/3476508560 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219979604 0 0) 0x7f2d6c121db0 con 0x7f2d6c09ff70 2024-09-18T14:02:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): my global_id is 4677 2024-09-18T14:02:04.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d4c0034e0 con 0x7f2d6c09ff70 2024-09-18T14:02:04.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67867878 0 0) 0x7f2d6c09d310 con 0x7f2d6c09f7d0 2024-09-18T14:02:04.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): my global_id is 4690 2024-09-18T14:02:04.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d6c121db0 con 0x7f2d6c09f7d0 2024-09-18T14:02:04.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1647978455 0 0) 0x7f2d4c0034e0 con 0x7f2d6c09ff70 2024-09-18T14:02:04.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2d6c09d310 con 0x7f2d6c09ff70 2024-09-18T14:02:04.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2d64001bb0 con 0x7f2d6c09ff70 2024-09-18T14:02:04.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:02:04.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): dump: 2024-09-18T14:02:04.391 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.391 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.391 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.391 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.391 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.391 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.391 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.391 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.391 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.391 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.392 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.392 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.392 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.392 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.392 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.392 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:02:04.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1970061840 0 0) 0x7f2d6c121db0 con 0x7f2d6c09f7d0 2024-09-18T14:02:04.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2d4c0034e0 con 0x7f2d6c09f7d0 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2d5c004270 con 0x7f2d6c09f7d0 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient(hunting): dump: 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.393 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 969985189 0 0) 0x7f2d6c09d310 con 0x7f2d6c09ff70 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 >> v1:172.21.15.45:6790/0 conn(0x7f2d6c09fba0 legacy=0x7f2d6c1318f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 >> v1:172.21.15.84:6789/0 conn(0x7f2d6c09f7d0 legacy=0x7f2d6c141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 monclient: found mon.d 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:02:04.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c11ff80 con 0x7f2d6c09ff70 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 10 monclient: _finish_auth 0 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 5 monclient: authenticate success, global_id 4677 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 10 monclient: _renew_subs 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c121a40 con 0x7f2d6c09ff70 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 10 monclient: _renew_subs 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.373+0000 7f2d714af640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c141e40 con 0x7f2d6c09ff70 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.374+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2d640014d0 con 0x7f2d6c09ff70 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.374+0000 7f2d62ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.374+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2d64005250 con 0x7f2d6c09ff70 2024-09-18T14:02:04.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.374+0000 7f2d62ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.374+0000 7f2d69ffb640 4 set_mon_vals no callback set 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.374+0000 7f2d62ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.374+0000 7f2d62ffd640 10 monclient: dump: 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.374+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2d640053b0 con 0x7f2d6c09ff70 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.374+0000 7f2d714af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.374+0000 7f2d714af640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2d50006580 con 0x7f2d50002830 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.374+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(130..130 src has 1..130) v4 ==== 3939+0+0 (unknown 3552224666 0 0) 0x7f2d64046e90 con 0x7f2d6c09ff70 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d63fff640 10 monclient: get_auth_request con 0x7f2d4c03a380 auth_method 0 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255876939 0 0) 0x7f2d50006580 con 0x7f2d50002830 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 10 monclient(hunting): my global_id is 4699 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d6c09d310 con 0x7f2d50002830 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 699155939 0 0) 0x7f2d6c09d310 con 0x7f2d50002830 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2d50006580 con 0x7f2d50002830 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2d5c004a00 con 0x7f2d50002830 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 10 monclient: dump: 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3494874588 0 0) 0x7f2d50006580 con 0x7f2d50002830 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.375+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d5000a0a0 con 0x7f2d50002830 2024-09-18T14:02:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.376+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2d5c004fb0 con 0x7f2d50002830 2024-09-18T14:02:04.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.376+0000 7f2d62ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:04.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.376+0000 7f2d62ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:02:04.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.376+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 >> v1:172.21.15.84:6789/0 conn(0x7f2d50002830 legacy=0x7f2d50004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.376+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 >> v1:172.21.15.84:6789/0 conn(0x7f2d50002830 legacy=0x7f2d50004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.381+0000 7f2d714af640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:02:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.381+0000 7f2d714af640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2d50007540 con 0x7f2d500068f0 2024-09-18T14:02:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575827370 0 0) 0x7f2d50007540 con 0x7f2d500068f0 2024-09-18T14:02:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 10 monclient(hunting): my global_id is 4708 2024-09-18T14:02:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d50006580 con 0x7f2d500068f0 2024-09-18T14:02:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2937389643 0 0) 0x7f2d50006580 con 0x7f2d500068f0 2024-09-18T14:02:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2d50007540 con 0x7f2d500068f0 2024-09-18T14:02:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2d5c005710 con 0x7f2d500068f0 2024-09-18T14:02:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:02:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 10 monclient: dump: 2024-09-18T14:02:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.404 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1397796070 0 0) 0x7f2d50007540 con 0x7f2d500068f0 2024-09-18T14:02:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.382+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2d500087a0 con 0x7f2d500068f0 2024-09-18T14:02:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7523 (unknown 0 0 2721770887) 0x7f2d5c006b90 con 0x7f2d500068f0 2024-09-18T14:02:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d62ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:02:04.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d62ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:02:04.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 >> v1:172.21.15.84:6789/0 conn(0x7f2d500068f0 legacy=0x7f2d50006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d62ffd640 1 -- 172.21.15.45:0/3476508560 >> v1:172.21.15.84:6789/0 conn(0x7f2d500068f0 legacy=0x7f2d50006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:04.405 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:02:04.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:02:04.405 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:02:04.405 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:02:04.405 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 58, 2024-09-18T14:02:04.406 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:02:04.406 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:02:04.406 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:02:04.406 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:02:04.406 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:02:04.406 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:02:04.406 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:02:04.406 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.406 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:02:04.406 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:02:04.406 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:02:04.407 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:02:04.407 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.407 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.407 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.407 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.407 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.407 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.407 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.407 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.407 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.407 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.407 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:02:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:02:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.409 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:02:04.409 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:02:04.409 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:02:04.409 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:02:04.409 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:02:04.409 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:02:04.409 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:02:04.409 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:02:04.409 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:02:04.410 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:02:04.410 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:02:04.410 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:02:04.410 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:02:04.410 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:02:04.410 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:02:04.410 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:02:04.410 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:02:04.410 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.410 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.410 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.411 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.411 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.411 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.411 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.411 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.411 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.411 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:02:04.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.411 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:02:04.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.411 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:02:04.411 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:02:04.412 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.412 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.412 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:02:04.412 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.412 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.412 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.412 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:02:04.413 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:02:04.413 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.413 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.413 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.413 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:02:04.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:02:04.413 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.413 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.414 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:02:04.414 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.414 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.414 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:02:04.414 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:02:04.414 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.415 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.415 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.415 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:02:04.415 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:02:04.415 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.415 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.415 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:02:04.415 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:02:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:02:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:02:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:02:04.417 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.417 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.417 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:02:04.417 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.417 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.417 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.418 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:02:04.419 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.420 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.421 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:02:04.421 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:02:04.421 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.421 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.421 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:02:04.421 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.421 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.423 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:02:04.423 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.423 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.423 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.423 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:02:04.423 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:02:04.423 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.423 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.423 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.424 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:02:04.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:02:04.424 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.424 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.424 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:02:04.424 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.424 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.424 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.425 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:02:04.425 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:02:04.425 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.425 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.425 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.425 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.425 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:02:04.425 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.426 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:02:04.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.427 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.427 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.427 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:02:04.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.427 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.427 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:02:04.427 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:02:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d714af640 1 -- 172.21.15.45:0/3476508560 >> v1:172.21.15.84:6812/31478 conn(0x7f2d4c03a380 legacy=0x7f2d4c03c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d714af640 10 monclient: shutdown 2024-09-18T14:02:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d714af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d714af640 1 -- 172.21.15.45:0/3476508560 >> v1:172.21.15.45:6792/0 conn(0x7f2d6c09ff70 legacy=0x7f2d6c149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d6b7fe640 1 -- 172.21.15.45:0/3476508560 reap_dead start 2024-09-18T14:02:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d6b7fe640 1 -- 172.21.15.45:0/3476508560 reap_dead start 2024-09-18T14:02:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d714af640 1 -- 172.21.15.45:0/3476508560 shutdown_connections 2024-09-18T14:02:04.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d714af640 1 -- 172.21.15.45:0/3476508560 >> 172.21.15.45:0/3476508560 conn(0x7f2d6c0630b0 msgr2=0x7f2d6c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:04.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d714af640 1 -- 172.21.15.45:0/3476508560 shutdown_connections 2024-09-18T14:02:04.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.383+0000 7f2d714af640 1 -- 172.21.15.45:0/3476508560 wait complete. 2024-09-18T14:02:04.430 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-18T14:02:04.430 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-09-18T14:02:04.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:02:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.526+0000 7fed26100640 10 monclient: get_monmap_and_config 2024-09-18T14:02:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.526+0000 7fed26100640 10 monclient: build_initial_monmap 2024-09-18T14:02:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.526+0000 7fed26100640 10 monclient: monmap: 2024-09-18T14:02:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:02:04.527637+0000 2024-09-18T14:02:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:02:04.527637+0000 2024-09-18T14:02:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:04.528 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 1 Processor -- start 2024-09-18T14:02:04.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 1 -- start start 2024-09-18T14:02:04.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 10 monclient: init 2024-09-18T14:02:04.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:04.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 10 monclient: _add_conns ranks=[1,0,2,7,3,6,5,8,4] 2024-09-18T14:02:04.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 10 monclient(hunting): picked mon.noname-f con 0x7fed2009f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:02:04.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 10 monclient(hunting): picked mon.noname-a con 0x7fed2009fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:02:04.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 10 monclient(hunting): picked mon.noname-b con 0x7fed2009ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:02:04.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2005ed70 con 0x7fed2009fba0 2024-09-18T14:02:04.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2009dfc0 con 0x7fed2009f7d0 2024-09-18T14:02:04.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2009dce0 con 0x7fed2009ff70 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.527+0000 7fed26100640 10 monclient(hunting): authenticate will time out at 939.865989s 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1ffff640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7fed2009ff70 legacy=0x7fed2013b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1effd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fed2009fba0 0x7fed20137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35722/0 (socket says 172.21.15.45:35722) 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1effd640 1 -- 172.21.15.45:0/736702645 learned_addr learned my addr 172.21.15.45:0/736702645 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 1 -- 172.21.15.45:0/736702645 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026564909 0 0) 0x7fed2005ed70 con 0x7fed2009fba0 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient(hunting): my global_id is 4833 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 1 -- 172.21.15.45:0/736702645 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed080034e0 con 0x7fed2009fba0 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 1 -- 172.21.15.45:0/736702645 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048866280 0 0) 0x7fed2009dfc0 con 0x7fed2009f7d0 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient(hunting): my global_id is 4717 2024-09-18T14:02:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 1 -- 172.21.15.45:0/736702645 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed2005ed70 con 0x7fed2009f7d0 2024-09-18T14:02:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 1 -- 172.21.15.45:0/736702645 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 559697006 0 0) 0x7fed080034e0 con 0x7fed2009fba0 2024-09-18T14:02:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 1 -- 172.21.15.45:0/736702645 >> v1:172.21.15.45:6790/0 conn(0x7fed2009ff70 legacy=0x7fed2013b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:02:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 1 -- 172.21.15.45:0/736702645 >> v1:172.21.15.84:6789/0 conn(0x7fed2009f7d0 legacy=0x7fed20132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:02:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 1 monclient: found mon.noname-a 2024-09-18T14:02:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:02:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 1 -- 172.21.15.45:0/736702645 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed200a1c00 con 0x7fed2009fba0 2024-09-18T14:02:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:02:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed26100640 5 monclient: authenticate success, global_id 4833 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 1 -- 172.21.15.45:0/736702645 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fed14001830 con 0x7fed2009fba0 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient: dump: 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed26100640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 1 -- 172.21.15.45:0/736702645 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fed14001a10 con 0x7fed2009fba0 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 1 -- 172.21.15.45:0/736702645 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fed14004ba0 con 0x7fed2009fba0 2024-09-18T14:02:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed26100640 10 monclient: get_monmap_and_config success 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed26100640 4 set_mon_vals no callback set 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.528+0000 7fed1e7fc640 10 monclient: dump: 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.529+0000 7fed26100640 10 monclient: shutdown 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.529+0000 7fed26100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.529+0000 7fed26100640 1 -- 172.21.15.45:0/736702645 >> v1:172.21.15.45:6789/0 conn(0x7fed2009fba0 legacy=0x7fed20137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.529+0000 7fed26100640 1 -- 172.21.15.45:0/736702645 shutdown_connections 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.529+0000 7fed26100640 1 -- 172.21.15.45:0/736702645 >> 172.21.15.45:0/736702645 conn(0x7fed200630b0 msgr2=0x7fed200634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.529+0000 7fed26100640 1 -- 172.21.15.45:0/736702645 shutdown_connections 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.529+0000 7fed26100640 1 -- 172.21.15.45:0/736702645 wait complete. 2024-09-18T14:02:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.529+0000 7fed26100640 10 monclient: build_initial_monmap 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.529+0000 7fed26100640 10 monclient: monmap: 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 1 Processor -- start 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 1 -- start start 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 10 monclient: init 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 10 monclient: _add_conns ranks=[2,8,4,6,1,0,5,7,3] 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 10 monclient(hunting): picked mon.noname-b con 0x7fed2009f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 10 monclient(hunting): picked mon.noname-e con 0x7fed2009fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 10 monclient(hunting): picked mon.noname-c con 0x7fed2009ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed200a2560 con 0x7fed2009f7d0 2024-09-18T14:02:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed20142870 con 0x7fed2009ff70 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed1f7fe640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7fed2009f7d0 legacy=0x7fed20131780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2011e8c0 con 0x7fed2009fba0 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed1ffff640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7fed2009ff70 legacy=0x7fed2014a470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed26100640 10 monclient(hunting): authenticate will time out at 939.868825s 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed1effd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fed2009fba0 0x7fed20146d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:55648/0 (socket says 172.21.15.45:55648) 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.530+0000 7fed1effd640 1 -- 172.21.15.45:0/365807133 learned_addr learned my addr 172.21.15.45:0/365807133 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255843240 0 0) 0x7fed2011e8c0 con 0x7fed2009fba0 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient(hunting): my global_id is 4832 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed0c0034e0 con 0x7fed2009fba0 2024-09-18T14:02:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1599815708 0 0) 0x7fed0c0034e0 con 0x7fed2009fba0 2024-09-18T14:02:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed2011e8c0 con 0x7fed2009fba0 2024-09-18T14:02:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fed14001770 con 0x7fed2009fba0 2024-09-18T14:02:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:02:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient(hunting): dump: 2024-09-18T14:02:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:02:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3885873389 0 0) 0x7fed2011e8c0 con 0x7fed2009fba0 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 >> v1:172.21.15.45:6791/0 conn(0x7fed2009ff70 legacy=0x7fed2014a470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 >> v1:172.21.15.45:6790/0 conn(0x7fed2009f7d0 legacy=0x7fed20131780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient: _finish_hunting 0 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 1 monclient: found mon.e 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2011eba0 con 0x7fed2009fba0 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient: _finish_auth 0 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed26100640 5 monclient: authenticate success, global_id 4832 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed26100640 10 monclient: _renew_subs 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed26100640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed26100640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2009b9c0 con 0x7fed2009fba0 2024-09-18T14:02:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed26100640 10 monclient: _renew_subs 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed26100640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed26100640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fed20136e90 con 0x7fed2009fba0 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fed14004e20 con 0x7fed2009fba0 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fed140052d0 con 0x7fed2009fba0 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed1dffb640 4 set_mon_vals no callback set 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.531+0000 7fed077fe640 10 monclient: dump: 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.532+0000 7fed26100640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.532+0000 7fed26100640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fed00006580 con 0x7fed00002830 2024-09-18T14:02:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.532+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fed140102a0 con 0x7fed2009fba0 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.532+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(130..130 src has 1..130) v4 ==== 3939+0+0 (unknown 3552224666 0 0) 0x7fed14046e70 con 0x7fed2009fba0 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed1ffff640 10 monclient: get_auth_request con 0x7fed0c039bf0 auth_method 0 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697426304 0 0) 0x7fed00006580 con 0x7fed00002830 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 10 monclient(hunting): my global_id is 4667 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed2011e8c0 con 0x7fed00002830 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3438797660 0 0) 0x7fed2011e8c0 con 0x7fed00002830 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed00006580 con 0x7fed00002830 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fed10002d70 con 0x7fed00002830 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 10 monclient: dump: 2024-09-18T14:02:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.553 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3374004022 0 0) 0x7fed00006580 con 0x7fed00002830 2024-09-18T14:02:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.533+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed0000a0a0 con 0x7fed00002830 2024-09-18T14:02:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.534+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fed10006430 con 0x7fed00002830 2024-09-18T14:02:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.534+0000 7fed077fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.534+0000 7fed077fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:02:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.534+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 >> v1:172.21.15.84:6791/0 conn(0x7fed00002830 legacy=0x7fed00004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.534+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 >> v1:172.21.15.84:6791/0 conn(0x7fed00002830 legacy=0x7fed00004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:04.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.539+0000 7fed26100640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:02:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.539+0000 7fed26100640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fed00007540 con 0x7fed000068f0 2024-09-18T14:02:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911687962 0 0) 0x7fed00007540 con 0x7fed000068f0 2024-09-18T14:02:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 10 monclient(hunting): my global_id is 4676 2024-09-18T14:02:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed00006580 con 0x7fed000068f0 2024-09-18T14:02:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1959687820 0 0) 0x7fed00006580 con 0x7fed000068f0 2024-09-18T14:02:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed00007540 con 0x7fed000068f0 2024-09-18T14:02:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fed10004d10 con 0x7fed000068f0 2024-09-18T14:02:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 10 monclient: dump: 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.557 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1223082129 0 0) 0x7fed00007540 con 0x7fed000068f0 2024-09-18T14:02:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.540+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fed000087a0 con 0x7fed000068f0 2024-09-18T14:02:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7693 (unknown 0 0 941858406) 0x7fed100061a0 con 0x7fed000068f0 2024-09-18T14:02:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed077fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:02:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed077fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:02:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 >> v1:172.21.15.84:6791/0 conn(0x7fed000068f0 legacy=0x7fed00006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed077fe640 1 -- 172.21.15.45:0/365807133 >> v1:172.21.15.84:6791/0 conn(0x7fed000068f0 legacy=0x7fed00006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:04.558 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:02:04.558 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:02:04.558 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:02:04.558 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:02:04.558 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 58, 2024-09-18T14:02:04.558 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:02:04.558 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:02:04.558 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:02:04.558 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:02:04.558 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.559 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.560 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:02:04.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.564 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:02:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:02:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:02:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:02:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.566 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.568 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.569 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.570 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:02:04.572 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.573 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:02:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:02:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:02:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:02:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:02:04.577 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:02:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed26100640 1 -- 172.21.15.45:0/365807133 >> v1:172.21.15.84:6812/31478 conn(0x7fed0c039bf0 legacy=0x7fed0c03c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed26100640 10 monclient: shutdown 2024-09-18T14:02:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed26100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed26100640 1 -- 172.21.15.45:0/365807133 >> v1:172.21.15.45:6793/0 conn(0x7fed2009fba0 legacy=0x7fed20146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed1ffff640 1 -- 172.21.15.45:0/365807133 reap_dead start 2024-09-18T14:02:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed1ffff640 1 -- 172.21.15.45:0/365807133 reap_dead start 2024-09-18T14:02:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed26100640 1 -- 172.21.15.45:0/365807133 shutdown_connections 2024-09-18T14:02:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed26100640 1 -- 172.21.15.45:0/365807133 >> 172.21.15.45:0/365807133 conn(0x7fed200630b0 msgr2=0x7fed200a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed26100640 1 -- 172.21.15.45:0/365807133 shutdown_connections 2024-09-18T14:02:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.541+0000 7fed26100640 1 -- 172.21.15.45:0/365807133 wait complete. 2024-09-18T14:02:04.580 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-18T14:02:04.580 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon i 2024-09-18T14:02:04.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:02:04.609 INFO:tasks.rados.rados.0.smithi084.stdout:604: done (2 left) 2024-09-18T14:02:04.609 INFO:tasks.rados.rados.0.smithi084.stdout:605: read oid 6 snap -1 2024-09-18T14:02:04.609 INFO:tasks.rados.rados.0.smithi084.stdout:605: expect deleted 2024-09-18T14:02:04.609 INFO:tasks.rados.rados.0.smithi084.stdout:606: read oid 45 snap -1 2024-09-18T14:02:04.610 INFO:tasks.rados.rados.0.smithi084.stdout:606: expect deleted 2024-09-18T14:02:04.610 INFO:tasks.rados.rados.0.smithi084.stdout:607: delete oid 43 current snap is 57 2024-09-18T14:02:04.610 INFO:tasks.rados.rados.0.smithi084.stdout:601: finishing write tid 1 to smithi08434663-19 2024-09-18T14:02:04.610 INFO:tasks.rados.rados.0.smithi084.stdout:601: finishing write tid 2 to smithi08434663-19 2024-09-18T14:02:04.610 INFO:tasks.rados.rados.0.smithi084.stdout:601: finishing write tid 3 to smithi08434663-19 2024-09-18T14:02:04.610 INFO:tasks.rados.rados.0.smithi084.stdout:601: finishing write tid 4 to smithi08434663-19 2024-09-18T14:02:04.610 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 257 (ObjNum 247 snap 57 seq_num 247) dirty exists 2024-09-18T14:02:04.610 INFO:tasks.rados.rados.0.smithi084.stdout:601: left oid 19 (ObjNum 247 snap 57 seq_num 247) 2024-09-18T14:02:04.610 INFO:tasks.rados.rados.0.smithi084.stdout:603: finishing write tid 1 to smithi08434663-21 2024-09-18T14:02:04.610 INFO:tasks.rados.rados.0.smithi084.stdout:603: finishing write tid 2 to smithi08434663-21 2024-09-18T14:02:04.610 INFO:tasks.rados.rados.0.smithi084.stdout:603: finishing write tid 3 to smithi08434663-21 2024-09-18T14:02:04.610 INFO:tasks.rados.rados.0.smithi084.stdout:603: finishing write tid 4 to smithi08434663-21 2024-09-18T14:02:04.611 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 141 (ObjNum 249 snap 57 seq_num 249) dirty exists 2024-09-18T14:02:04.611 INFO:tasks.rados.rados.0.smithi084.stdout:603: left oid 21 (ObjNum 249 snap 57 seq_num 249) 2024-09-18T14:02:04.614 INFO:tasks.rados.rados.0.smithi084.stdout:601: done (4 left) 2024-09-18T14:02:04.614 INFO:tasks.rados.rados.0.smithi084.stdout:603: done (3 left) 2024-09-18T14:02:04.614 INFO:tasks.rados.rados.0.smithi084.stdout:605: done (2 left) 2024-09-18T14:02:04.614 INFO:tasks.rados.rados.0.smithi084.stdout:606: done (1 left) 2024-09-18T14:02:04.614 INFO:tasks.rados.rados.0.smithi084.stdout:607: done (0 left) 2024-09-18T14:02:04.614 INFO:tasks.rados.rados.0.smithi084.stdout:608: delete oid 16 current snap is 57 2024-09-18T14:02:04.619 INFO:tasks.rados.rados.0.smithi084.stdout:608: done (0 left) 2024-09-18T14:02:04.619 INFO:tasks.rados.rados.0.smithi084.stdout:609: snap_create 2024-09-18T14:02:04.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.683+0000 7fc0bd6c0640 10 monclient: get_monmap_and_config 2024-09-18T14:02:04.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.683+0000 7fc0bd6c0640 10 monclient: build_initial_monmap 2024-09-18T14:02:04.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.683+0000 7fc0bd6c0640 10 monclient: monmap: 2024-09-18T14:02:04.684 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:04.684 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:02:04.684632+0000 2024-09-18T14:02:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:02:04.684632+0000 2024-09-18T14:02:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:04.686 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 1 Processor -- start 2024-09-18T14:02:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 1 -- start start 2024-09-18T14:02:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 10 monclient: init 2024-09-18T14:02:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 10 monclient: _add_conns ranks=[5,1,4,2,7,8,0,6,3] 2024-09-18T14:02:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 10 monclient(hunting): picked mon.noname-h con 0x7fc0b809f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:02:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 10 monclient(hunting): picked mon.noname-f con 0x7fc0b809fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:02:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 10 monclient(hunting): picked mon.noname-c con 0x7fc0b809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:02:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b805ed70 con 0x7fc0b809fba0 2024-09-18T14:02:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b809dfc0 con 0x7fc0b809ff70 2024-09-18T14:02:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b809dce0 con 0x7fc0b809f7d0 2024-09-18T14:02:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0bd6c0640 10 monclient(hunting): authenticate will time out at 940.022856s 2024-09-18T14:02:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.684+0000 7fc0b77fe640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7fc0b809ff70 legacy=0x7fc0b813b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:02:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b67fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc0b809fba0 0x7fc0b8137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:47744/0 (socket says 172.21.15.45:47744) 2024-09-18T14:02:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b6ffd640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7fc0b809f7d0 0x7fc0b8132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:39618/0 (socket says 172.21.15.45:39618) 2024-09-18T14:02:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b67fc640 1 -- 172.21.15.45:0/2784747201 learned_addr learned my addr 172.21.15.45:0/2784747201 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 1 -- 172.21.15.45:0/2784747201 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053420753 0 0) 0x7fc0b809dce0 con 0x7fc0b809f7d0 2024-09-18T14:02:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient(hunting): my global_id is 4685 2024-09-18T14:02:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 1 -- 172.21.15.45:0/2784747201 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a00034e0 con 0x7fc0b809f7d0 2024-09-18T14:02:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 1 -- 172.21.15.45:0/2784747201 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306344957 0 0) 0x7fc0b805ed70 con 0x7fc0b809fba0 2024-09-18T14:02:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient(hunting): my global_id is 4726 2024-09-18T14:02:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 1 -- 172.21.15.45:0/2784747201 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b809dce0 con 0x7fc0b809fba0 2024-09-18T14:02:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 1 -- 172.21.15.45:0/2784747201 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2123795909 0 0) 0x7fc0a00034e0 con 0x7fc0b809f7d0 2024-09-18T14:02:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 1 -- 172.21.15.45:0/2784747201 >> v1:172.21.15.45:6791/0 conn(0x7fc0b809ff70 legacy=0x7fc0b813b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:02:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 1 -- 172.21.15.45:0/2784747201 >> v1:172.21.15.84:6789/0 conn(0x7fc0b809fba0 legacy=0x7fc0b8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:02:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 1 monclient: found mon.noname-h 2024-09-18T14:02:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T14:02:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 1 -- 172.21.15.45:0/2784747201 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b80a1c00 con 0x7fc0b809f7d0 2024-09-18T14:02:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient: _finish_auth 0 2024-09-18T14:02:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 1 -- 172.21.15.45:0/2784747201 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0a8001830 con 0x7fc0b809f7d0 2024-09-18T14:02:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:02:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient: dump: 2024-09-18T14:02:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.702 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0bd6c0640 5 monclient: authenticate success, global_id 4685 2024-09-18T14:02:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0bd6c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:02:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 1 -- 172.21.15.45:0/2784747201 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc0a8001a10 con 0x7fc0b809f7d0 2024-09-18T14:02:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0bd6c0640 10 monclient: get_monmap_and_config success 2024-09-18T14:02:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0bd6c0640 4 set_mon_vals no callback set 2024-09-18T14:02:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 1 -- 172.21.15.45:0/2784747201 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0a8004ae0 con 0x7fc0b809f7d0 2024-09-18T14:02:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:02:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.685+0000 7fc0b5ffb640 10 monclient: dump: 2024-09-18T14:02:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.704 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.704 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.704 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.704 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.686+0000 7fc0bd6c0640 10 monclient: shutdown 2024-09-18T14:02:04.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.686+0000 7fc0bd6c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:04.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.686+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/2784747201 >> v1:172.21.15.84:6791/0 conn(0x7fc0b809f7d0 legacy=0x7fc0b8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.686+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/2784747201 shutdown_connections 2024-09-18T14:02:04.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.686+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/2784747201 >> 172.21.15.45:0/2784747201 conn(0x7fc0b80630b0 msgr2=0x7fc0b80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:04.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.686+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/2784747201 shutdown_connections 2024-09-18T14:02:04.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.686+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/2784747201 wait complete. 2024-09-18T14:02:04.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0bd6c0640 10 monclient: build_initial_monmap 2024-09-18T14:02:04.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0bd6c0640 10 monclient: monmap: 2024-09-18T14:02:04.705 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:02:04.705 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:02:04.705 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:02:04.705 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:02:04.706 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:02:04.706 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:02:04.706 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:02:04.706 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:02:04.706 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:02:04.706 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:02:04.706 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:02:04.706 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:02:04.706 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:02:04.706 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:02:04.706 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:02:04.707 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0bd6c0640 1 Processor -- start 2024-09-18T14:02:04.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0bd6c0640 1 -- start start 2024-09-18T14:02:04.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0bd6c0640 10 monclient: init 2024-09-18T14:02:04.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0bd6c0640 10 monclient: _reopen_session rank -1 2024-09-18T14:02:04.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0bd6c0640 10 monclient: _add_conns ranks=[1,4,8,2,5,0,7,6,3] 2024-09-18T14:02:04.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0bd6c0640 10 monclient(hunting): picked mon.noname-f con 0x7fc0b809f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:02:04.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0bd6c0640 10 monclient(hunting): picked mon.noname-c con 0x7fc0b809fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:02:04.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0bd6c0640 10 monclient(hunting): picked mon.noname-e con 0x7fc0b809ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:02:04.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0b67fc640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7fc0b809fba0 legacy=0x7fc0b81319e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:02:04.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0bd6c0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b809d310 con 0x7fc0b809f7d0 2024-09-18T14:02:04.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.687+0000 7fc0bd6c0640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b80a2560 con 0x7fc0b809fba0 2024-09-18T14:02:04.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0bd6c0640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b81207f0 con 0x7fc0b809ff70 2024-09-18T14:02:04.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0bd6c0640 10 monclient(hunting): _renew_subs 2024-09-18T14:02:04.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0b77fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fc0b809ff70 0x7fc0b8149af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:55660/0 (socket says 172.21.15.45:55660) 2024-09-18T14:02:04.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0bd6c0640 10 monclient(hunting): authenticate will time out at 940.026051s 2024-09-18T14:02:04.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0b77fe640 1 -- 172.21.15.45:0/1094551001 learned_addr learned my addr 172.21.15.45:0/1094551001 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:02:04.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749035027 0 0) 0x7fc0b0003d80 con 0x7fc0b809ff70 2024-09-18T14:02:04.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): my global_id is 4841 2024-09-18T14:02:04.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a40034e0 con 0x7fc0b809ff70 2024-09-18T14:02:04.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442654997 0 0) 0x7fc0b809d310 con 0x7fc0b809f7d0 2024-09-18T14:02:04.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): my global_id is 4735 2024-09-18T14:02:04.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b0003d80 con 0x7fc0b809f7d0 2024-09-18T14:02:04.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4182782917 0 0) 0x7fc0b0003d80 con 0x7fc0b809f7d0 2024-09-18T14:02:04.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0b809d310 con 0x7fc0b809f7d0 2024-09-18T14:02:04.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2675792204 0 0) 0x7fc0a40034e0 con 0x7fc0b809ff70 2024-09-18T14:02:04.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0b0003d80 con 0x7fc0b809ff70 2024-09-18T14:02:04.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0a8001430 con 0x7fc0b809f7d0 2024-09-18T14:02:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:02:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): dump: 2024-09-18T14:02:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.711 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:02:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0b0004740 con 0x7fc0b809ff70 2024-09-18T14:02:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:02:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient(hunting): dump: 2024-09-18T14:02:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.713 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.713 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.713 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.713 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.713 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.713 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 755968792 0 0) 0x7fc0b809d310 con 0x7fc0b809f7d0 2024-09-18T14:02:04.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 >> v1:172.21.15.45:6793/0 conn(0x7fc0b809ff70 legacy=0x7fc0b8149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 >> v1:172.21.15.45:6791/0 conn(0x7fc0b809fba0 legacy=0x7fc0b81319e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:02:04.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient: _finish_hunting 0 2024-09-18T14:02:04.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 monclient: found mon.f 2024-09-18T14:02:04.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:02:04.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:02:04.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b8121e10 con 0x7fc0b809f7d0 2024-09-18T14:02:04.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient: _finish_auth 0 2024-09-18T14:02:04.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:02:04.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0bd6c0640 5 monclient: authenticate success, global_id 4735 2024-09-18T14:02:04.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0bd6c0640 10 monclient: _renew_subs 2024-09-18T14:02:04.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0bd6c0640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:02:04.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b8121b90 con 0x7fc0b809f7d0 2024-09-18T14:02:04.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0bd6c0640 10 monclient: _renew_subs 2024-09-18T14:02:04.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0bd6c0640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:02:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b8120000 con 0x7fc0b809f7d0 2024-09-18T14:02:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc0a8004d60 con 0x7fc0b809f7d0 2024-09-18T14:02:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:02:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0a8001250 con 0x7fc0b809f7d0 2024-09-18T14:02:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:02:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc0b57fa640 4 set_mon_vals no callback set 2024-09-18T14:02:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.688+0000 7fc09effd640 10 monclient: dump: 2024-09-18T14:02:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.715 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.716 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.716 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.689+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc0a8010080 con 0x7fc0b809f7d0 2024-09-18T14:02:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.689+0000 7fc0bd6c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.689+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc098006580 con 0x7fc098002830 2024-09-18T14:02:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.689+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(131..131 src has 1..131) v4 ==== 3939+0+0 (unknown 2675643152 0 0) 0x7fc0a8046c90 con 0x7fc0b809f7d0 2024-09-18T14:02:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc0b77fe640 10 monclient: get_auth_request con 0x7fc0a403a2f0 auth_method 0 2024-09-18T14:02:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945606362 0 0) 0x7fc098006580 con 0x7fc098002830 2024-09-18T14:02:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 10 monclient(hunting): my global_id is 4750 2024-09-18T14:02:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b809d310 con 0x7fc098002830 2024-09-18T14:02:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2352986585 0 0) 0x7fc0b809d310 con 0x7fc098002830 2024-09-18T14:02:04.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc098006580 con 0x7fc098002830 2024-09-18T14:02:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0ac001890 con 0x7fc098002830 2024-09-18T14:02:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:02:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 10 monclient: dump: 2024-09-18T14:02:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.718 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 375005380 0 0) 0x7fc098006580 con 0x7fc098002830 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc09800a0a0 con 0x7fc098002830 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc0ac005890 con 0x7fc098002830 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:02:04.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:02:04.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 >> v1:172.21.15.84:6792/0 conn(0x7fc098002830 legacy=0x7fc098004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.690+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 >> v1:172.21.15.84:6792/0 conn(0x7fc098002830 legacy=0x7fc098004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:04.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.696+0000 7fc0bd6c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:02:04.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.696+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc098009660 con 0x7fc098009040 2024-09-18T14:02:04.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106636970 0 0) 0x7fc098009660 con 0x7fc098009040 2024-09-18T14:02:04.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:02:04.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:02:04.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 10 monclient(hunting): my global_id is 4759 2024-09-18T14:02:04.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc098006580 con 0x7fc098009040 2024-09-18T14:02:04.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4252951928 0 0) 0x7fc098006580 con 0x7fc098009040 2024-09-18T14:02:04.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc098009660 con 0x7fc098009040 2024-09-18T14:02:04.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0ac004e50 con 0x7fc098009040 2024-09-18T14:02:04.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:02:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:02:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 10 monclient: dump: 2024-09-18T14:02:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:02:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:02:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:02:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:02:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:02:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:02:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:02:04.723 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:02:04.724 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:02:04.724 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:02:04.724 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:02:04.724 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:02:04.724 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:02:04.724 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:02:04.724 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:02:04.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4118883491 0 0) 0x7fc098009660 con 0x7fc098009040 2024-09-18T14:02:04.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc098009920 con 0x7fc098009040 2024-09-18T14:02:04.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7353 (unknown 0 0 1845028426) 0x7fc0ac005b00 con 0x7fc098009040 2024-09-18T14:02:04.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:02:04.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:02:04.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 >> v1:172.21.15.84:6792/0 conn(0x7fc098009040 legacy=0x7fc098008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.697+0000 7fc09effd640 1 -- 172.21.15.45:0/1094551001 >> v1:172.21.15.84:6792/0 conn(0x7fc098009040 legacy=0x7fc098008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:02:04.725 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:02:04.725 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:02:04.725 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:02:04.725 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:02:04.725 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 58, 2024-09-18T14:02:04.726 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:02:04.726 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:02:04.726 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:02:04.726 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:02:04.726 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:02:04.726 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:02:04.726 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:02:04.726 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.726 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:02:04.726 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:02:04.727 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:02:04.727 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:02:04.727 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.727 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.727 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.727 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.727 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.727 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.727 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.727 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.728 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.728 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.728 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:02:04.728 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:02:04.728 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.728 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.728 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.728 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.728 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.728 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.728 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.729 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.729 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.729 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.729 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:02:04.729 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:02:04.729 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:02:04.729 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:02:04.729 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:02:04.729 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:02:04.729 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:02:04.730 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:02:04.730 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:02:04.730 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:02:04.730 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:02:04.730 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:02:04.730 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:02:04.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:02:04.730 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:02:04.730 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:02:04.730 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:02:04.730 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:02:04.731 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:02:04.731 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:02:04.731 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:02:04.731 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:02:04.731 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:02:04.731 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:02:04.731 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:02:04.731 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:02:04.731 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.731 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:02:04.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.732 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:02:04.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.732 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:02:04.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:02:04.732 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.732 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.732 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.732 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:02:04.732 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.733 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.733 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:02:04.733 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:02:04.733 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.733 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.733 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.733 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:02:04.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:02:04.734 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.734 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.734 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:02:04.734 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.734 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.734 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.734 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:02:04.734 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:02:04.735 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.735 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.735 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.735 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:02:04.735 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:02:04.735 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.735 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.735 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.736 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:02:04.736 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.736 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.736 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.736 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:02:04.736 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:02:04.736 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.736 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.736 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.737 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:02:04.737 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:02:04.737 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.737 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.737 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.737 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:02:04.737 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.737 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.737 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.738 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:02:04.738 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:02:04.738 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.738 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.738 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.738 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:02:04.738 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:02:04.738 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.738 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.739 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:02:04.739 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.739 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.739 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.739 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:02:04.739 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:02:04.739 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.739 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.740 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.740 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:02:04.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:02:04.740 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.740 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.740 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.740 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:02:04.740 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.741 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.741 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.741 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:02:04.741 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:02:04.741 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.741 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.741 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.742 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:02:04.742 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:02:04.742 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.742 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.742 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.742 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:02:04.742 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.742 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.742 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.743 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:02:04.743 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:02:04.743 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.743 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.743 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.743 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:02:04.743 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:02:04.743 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.744 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.744 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:02:04.744 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.744 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.744 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.744 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:02:04.744 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:02:04.744 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.745 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.745 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.745 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:02:04.745 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:02:04.745 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:02:04.745 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:02:04.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:02:04.746 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:02:04.746 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:02:04.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.746 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.746 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:02:04.746 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:02:04.746 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:02:04.746 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:02:04.746 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:02:04.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.747 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.747 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:02:04.747 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:02:04.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.747 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.747 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.747 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:02:04.747 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.747 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:02:04.748 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:02:04.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:02:04.748 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:02:04.748 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:02:04.748 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:02:04.748 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:02:04.748 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:02:04.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:02:04.748 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:02:04.748 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:02:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.698+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/1094551001 >> v1:172.21.15.84:6812/31478 conn(0x7fc0a403a2f0 legacy=0x7fc0a403c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.698+0000 7fc0bd6c0640 10 monclient: shutdown 2024-09-18T14:02:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.698+0000 7fc0bd6c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:02:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.698+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/1094551001 >> v1:172.21.15.84:6789/0 conn(0x7fc0b809f7d0 legacy=0x7fc0b81417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:02:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.698+0000 7fc0b77fe640 1 -- 172.21.15.45:0/1094551001 reap_dead start 2024-09-18T14:02:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.698+0000 7fc0b77fe640 1 -- 172.21.15.45:0/1094551001 reap_dead start 2024-09-18T14:02:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.698+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/1094551001 shutdown_connections 2024-09-18T14:02:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.698+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/1094551001 >> 172.21.15.45:0/1094551001 conn(0x7fc0b80630b0 msgr2=0x7fc0b80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:02:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.698+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/1094551001 shutdown_connections 2024-09-18T14:02:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:02:04.698+0000 7fc0bd6c0640 1 -- 172.21.15.45:0/1094551001 wait complete. 2024-09-18T14:02:04.751 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-18T14:02:04.752 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-18T14:02:05.608 INFO:tasks.rados.rados.0.smithi084.stdout:609: done (0 left) 2024-09-18T14:02:05.608 INFO:tasks.rados.rados.0.smithi084.stdout:610: read oid 11 snap -1 2024-09-18T14:02:05.608 INFO:tasks.rados.rados.0.smithi084.stdout:610: expect (ObjNum 178 snap 41 seq_num 178) 2024-09-18T14:02:05.609 INFO:tasks.rados.rados.0.smithi084.stdout:611: read oid 21 snap -1 2024-09-18T14:02:05.609 INFO:tasks.rados.rados.0.smithi084.stdout:611: expect (ObjNum 249 snap 57 seq_num 249) 2024-09-18T14:02:05.609 INFO:tasks.rados.rados.0.smithi084.stdout:612: rollback oid 24 current snap is 58 2024-09-18T14:02:05.609 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 24 to 51 2024-09-18T14:02:05.609 INFO:tasks.rados.rados.0.smithi084.stdout:613: snap_remove snap 56 2024-09-18T14:02:06.623 INFO:tasks.rados.rados.0.smithi084.stdout:613: done (3 left) 2024-09-18T14:02:06.624 INFO:tasks.rados.rados.0.smithi084.stdout:614: snap_remove snap 55 2024-09-18T14:02:07.162 INFO:tasks.rados.rados.0.smithi084.stdout:611: expect (ObjNum 249 snap 57 seq_num 249) 2024-09-18T14:02:07.274 INFO:tasks.rados.rados.0.smithi084.stdout:610: expect (ObjNum 178 snap 41 seq_num 178) 2024-09-18T14:02:07.443 INFO:tasks.rados.rados.0.smithi084.stdout:612: finishing rollback tid 0 to smithi08434663-24 2024-09-18T14:02:07.443 INFO:tasks.rados.rados.0.smithi084.stdout:612: finishing rollback tid 1 to smithi08434663-24 2024-09-18T14:02:07.443 INFO:tasks.rados.rados.0.smithi084.stdout:612: finishing rollback tid 2 to smithi08434663-24 2024-09-18T14:02:07.443 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 174 (ObjNum 205 snap 46 seq_num 205) dirty exists 2024-09-18T14:02:07.443 INFO:tasks.rados.rados.0.smithi084.stdout:610: done (3 left) 2024-09-18T14:02:07.443 INFO:tasks.rados.rados.0.smithi084.stdout:611: done (2 left) 2024-09-18T14:02:07.444 INFO:tasks.rados.rados.0.smithi084.stdout:612: done (1 left) 2024-09-18T14:02:07.444 INFO:tasks.rados.rados.0.smithi084.stdout:614: done (0 left) 2024-09-18T14:02:07.444 INFO:tasks.rados.rados.0.smithi084.stdout:615: copy_from oid 45 from oid 16 current snap is 58 2024-09-18T14:02:07.444 INFO:tasks.rados.rados.0.smithi084.stdout:616: snap_create 2024-09-18T14:02:07.444 INFO:tasks.rados.rados.0.smithi084.stdout:615: finishing copy_from to smithi08434663-45 2024-09-18T14:02:07.444 INFO:tasks.rados.rados.0.smithi084.stdout:615: got expected ENOENT (src dne) 2024-09-18T14:02:07.445 INFO:tasks.rados.rados.0.smithi084.stdout:615: finishing copy_from racing read to smithi08434663-45 2024-09-18T14:02:08.167 INFO:tasks.rados.rados.0.smithi084.stdout:615: done (1 left) 2024-09-18T14:02:08.167 INFO:tasks.rados.rados.0.smithi084.stdout:616: done (0 left) 2024-09-18T14:02:08.167 INFO:tasks.rados.rados.0.smithi084.stdout:617: read oid 50 snap -1 2024-09-18T14:02:08.168 INFO:tasks.rados.rados.0.smithi084.stdout:617: expect deleted 2024-09-18T14:02:08.168 INFO:tasks.rados.rados.0.smithi084.stdout:618: write (excl) oid 27 current snap is 59 2024-09-18T14:02:08.168 INFO:tasks.rados.rados.0.smithi084.stdout:618: seq_num 252 ranges {635364=402397,1763227=472137} 2024-09-18T14:02:08.173 INFO:tasks.rados.rados.0.smithi084.stdout:618: writing smithi08434663-27 from 635364 to 1037761 tid 1 2024-09-18T14:02:08.180 INFO:tasks.rados.rados.0.smithi084.stdout:618: writing smithi08434663-27 from 1763227 to 2235364 tid 2 2024-09-18T14:02:08.180 INFO:tasks.rados.rados.0.smithi084.stdout:619: read oid 8 snap -1 2024-09-18T14:02:08.180 INFO:tasks.rados.rados.0.smithi084.stdout:619: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:02:08.180 INFO:tasks.rados.rados.0.smithi084.stdout:620: snap_create 2024-09-18T14:02:08.180 INFO:tasks.rados.rados.0.smithi084.stdout:618: finishing write tid 1 to smithi08434663-27 2024-09-18T14:02:08.184 INFO:tasks.rados.rados.0.smithi084.stdout:618: finishing write tid 2 to smithi08434663-27 2024-09-18T14:02:08.185 INFO:tasks.rados.rados.0.smithi084.stdout:618: finishing write tid 3 to smithi08434663-27 2024-09-18T14:02:08.185 INFO:tasks.rados.rados.0.smithi084.stdout:618: finishing write tid 4 to smithi08434663-27 2024-09-18T14:02:08.185 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 193 (ObjNum 252 snap 59 seq_num 252) dirty exists 2024-09-18T14:02:08.185 INFO:tasks.rados.rados.0.smithi084.stdout:618: left oid 27 (ObjNum 252 snap 59 seq_num 252) 2024-09-18T14:02:08.191 INFO:tasks.rados.rados.0.smithi084.stdout:619: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:02:09.171 INFO:tasks.rados.rados.0.smithi084.stdout:617: done (3 left) 2024-09-18T14:02:09.172 INFO:tasks.rados.rados.0.smithi084.stdout:618: done (2 left) 2024-09-18T14:02:09.172 INFO:tasks.rados.rados.0.smithi084.stdout:619: done (1 left) 2024-09-18T14:02:09.172 INFO:tasks.rados.rados.0.smithi084.stdout:620: done (0 left) 2024-09-18T14:02:09.172 INFO:tasks.rados.rados.0.smithi084.stdout:621: write oid 13 current snap is 60 2024-09-18T14:02:09.172 INFO:tasks.rados.rados.0.smithi084.stdout:621: seq_num 253 ranges {652171=477646,1821709=430462} 2024-09-18T14:02:09.177 INFO:tasks.rados.rados.0.smithi084.stdout:621: writing smithi08434663-13 from 652171 to 1129817 tid 1 2024-09-18T14:02:09.184 INFO:tasks.rados.rados.0.smithi084.stdout:621: writing smithi08434663-13 from 1821709 to 2252171 tid 2 2024-09-18T14:02:09.184 INFO:tasks.rados.rados.0.smithi084.stdout:622: delete oid 49 current snap is 60 2024-09-18T14:02:09.184 INFO:tasks.rados.rados.0.smithi084.stdout:621: finishing write tid 1 to smithi08434663-13 2024-09-18T14:02:09.186 INFO:tasks.rados.rados.0.smithi084.stdout:622: done (1 left) 2024-09-18T14:02:09.186 INFO:tasks.rados.rados.0.smithi084.stdout:623: copy_from oid 28 from oid 22 current snap is 60 2024-09-18T14:02:09.186 INFO:tasks.rados.rados.0.smithi084.stdout:624: read oid 35 snap -1 2024-09-18T14:02:09.186 INFO:tasks.rados.rados.0.smithi084.stdout:624: expect deleted 2024-09-18T14:02:09.187 INFO:tasks.rados.rados.0.smithi084.stdout:625: snap_create 2024-09-18T14:02:09.187 INFO:tasks.rados.rados.0.smithi084.stdout:621: finishing write tid 2 to smithi08434663-13 2024-09-18T14:02:09.188 INFO:tasks.rados.rados.0.smithi084.stdout:621: finishing write tid 3 to smithi08434663-13 2024-09-18T14:02:09.188 INFO:tasks.rados.rados.0.smithi084.stdout:623: finishing copy_from to smithi08434663-28 2024-09-18T14:02:09.188 INFO:tasks.rados.rados.0.smithi084.stdout:623: got expected ENOENT (src dne) 2024-09-18T14:02:09.188 INFO:tasks.rados.rados.0.smithi084.stdout:621: finishing write tid 4 to smithi08434663-13 2024-09-18T14:02:09.188 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 212 (ObjNum 253 snap 60 seq_num 253) dirty exists 2024-09-18T14:02:09.188 INFO:tasks.rados.rados.0.smithi084.stdout:621: left oid 13 (ObjNum 253 snap 60 seq_num 253) 2024-09-18T14:02:09.188 INFO:tasks.rados.rados.0.smithi084.stdout:623: finishing copy_from racing read to smithi08434663-28 2024-09-18T14:02:10.182 INFO:tasks.rados.rados.0.smithi084.stdout:621: done (3 left) 2024-09-18T14:02:10.182 INFO:tasks.rados.rados.0.smithi084.stdout:623: done (2 left) 2024-09-18T14:02:10.182 INFO:tasks.rados.rados.0.smithi084.stdout:624: done (1 left) 2024-09-18T14:02:10.182 INFO:tasks.rados.rados.0.smithi084.stdout:625: done (0 left) 2024-09-18T14:02:10.182 INFO:tasks.rados.rados.0.smithi084.stdout:626: rollback oid 46 current snap is 61 2024-09-18T14:02:10.182 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 46 to 58 2024-09-18T14:02:10.182 INFO:tasks.rados.rados.0.smithi084.stdout:627: write oid 29 current snap is 61 2024-09-18T14:02:10.183 INFO:tasks.rados.rados.0.smithi084.stdout:627: seq_num 255 ranges {685785=628144,1822320=463465} 2024-09-18T14:02:10.189 INFO:tasks.rados.rados.0.smithi084.stdout:627: writing smithi08434663-29 from 685785 to 1313929 tid 1 2024-09-18T14:02:10.194 INFO:tasks.rados.rados.0.smithi084.stdout:627: writing smithi08434663-29 from 1822320 to 2285785 tid 2 2024-09-18T14:02:10.194 INFO:tasks.rados.rados.0.smithi084.stdout:628: delete oid 36 current snap is 61 2024-09-18T14:02:10.195 INFO:tasks.rados.rados.0.smithi084.stdout:626: finishing rollback tid 0 to smithi08434663-46 2024-09-18T14:02:10.195 INFO:tasks.rados.rados.0.smithi084.stdout:626: finishing rollback tid 1 to smithi08434663-46 2024-09-18T14:02:10.195 INFO:tasks.rados.rados.0.smithi084.stdout:626: finishing rollback tid 2 to smithi08434663-46 2024-09-18T14:02:10.195 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 234 (ObjNum 218 snap 50 seq_num 218) dirty exists 2024-09-18T14:02:10.197 INFO:tasks.rados.rados.0.smithi084.stdout:627: finishing write tid 1 to smithi08434663-29 2024-09-18T14:02:10.197 INFO:tasks.rados.rados.0.smithi084.stdout:627: finishing write tid 2 to smithi08434663-29 2024-09-18T14:02:10.198 INFO:tasks.rados.rados.0.smithi084.stdout:626: done (2 left) 2024-09-18T14:02:10.199 INFO:tasks.rados.rados.0.smithi084.stdout:628: done (1 left) 2024-09-18T14:02:10.199 INFO:tasks.rados.rados.0.smithi084.stdout:629: snap_create 2024-09-18T14:02:10.199 INFO:tasks.rados.rados.0.smithi084.stdout:627: finishing write tid 3 to smithi08434663-29 2024-09-18T14:02:10.199 INFO:tasks.rados.rados.0.smithi084.stdout:627: finishing write tid 4 to smithi08434663-29 2024-09-18T14:02:10.199 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 216 (ObjNum 255 snap 61 seq_num 255) dirty exists 2024-09-18T14:02:10.199 INFO:tasks.rados.rados.0.smithi084.stdout:627: left oid 29 (ObjNum 255 snap 61 seq_num 255) 2024-09-18T14:02:11.185 INFO:tasks.rados.rados.0.smithi084.stdout:627: done (1 left) 2024-09-18T14:02:11.185 INFO:tasks.rados.rados.0.smithi084.stdout:629: done (0 left) 2024-09-18T14:02:11.186 INFO:tasks.rados.rados.0.smithi084.stdout:630: snap_remove snap 58 2024-09-18T14:02:12.163 INFO:tasks.rados.rados.0.smithi084.stdout:630: done (0 left) 2024-09-18T14:02:12.163 INFO:tasks.rados.rados.0.smithi084.stdout:631: snap_create 2024-09-18T14:02:13.168 INFO:tasks.rados.rados.0.smithi084.stdout:631: done (0 left) 2024-09-18T14:02:13.168 INFO:tasks.rados.rados.0.smithi084.stdout:632: delete oid 32 current snap is 63 2024-09-18T14:02:13.173 INFO:tasks.rados.rados.0.smithi084.stdout:632: done (0 left) 2024-09-18T14:02:13.173 INFO:tasks.rados.rados.0.smithi084.stdout:633: read oid 29 snap 51 2024-09-18T14:02:13.173 INFO:tasks.rados.rados.0.smithi084.stdout:633: expect (ObjNum 136 snap 29 seq_num 136) 2024-09-18T14:02:13.173 INFO:tasks.rados.rados.0.smithi084.stdout:634: snap_create 2024-09-18T14:02:13.179 INFO:tasks.rados.rados.0.smithi084.stdout:633: expect (ObjNum 136 snap 29 seq_num 136) 2024-09-18T14:02:14.169 INFO:tasks.rados.rados.0.smithi084.stdout:633: done (1 left) 2024-09-18T14:02:14.169 INFO:tasks.rados.rados.0.smithi084.stdout:634: done (0 left) 2024-09-18T14:02:14.169 INFO:tasks.rados.rados.0.smithi084.stdout:635: snap_remove snap 60 2024-09-18T14:02:15.176 INFO:tasks.rados.rados.0.smithi084.stdout:635: done (0 left) 2024-09-18T14:02:15.176 INFO:tasks.rados.rados.0.smithi084.stdout:636: read oid 46 snap 62 2024-09-18T14:02:15.176 INFO:tasks.rados.rados.0.smithi084.stdout:636: expect (ObjNum 218 snap 50 seq_num 218) 2024-09-18T14:02:15.176 INFO:tasks.rados.rados.0.smithi084.stdout:637: read oid 42 snap -1 2024-09-18T14:02:15.177 INFO:tasks.rados.rados.0.smithi084.stdout:637: expect (ObjNum 227 snap 53 seq_num 227) 2024-09-18T14:02:15.177 INFO:tasks.rados.rados.0.smithi084.stdout:638: delete oid 17 current snap is 64 2024-09-18T14:02:15.188 INFO:tasks.rados.rados.0.smithi084.stdout:638: done (2 left) 2024-09-18T14:02:15.188 INFO:tasks.rados.rados.0.smithi084.stdout:639: write oid 50 current snap is 64 2024-09-18T14:02:15.188 INFO:tasks.rados.rados.0.smithi084.stdout:639: seq_num 259 ranges {753013=645493,2056248=296765} 2024-09-18T14:02:15.195 INFO:tasks.rados.rados.0.smithi084.stdout:639: writing smithi08434663-50 from 753013 to 1398506 tid 1 2024-09-18T14:02:15.201 INFO:tasks.rados.rados.0.smithi084.stdout:639: writing smithi08434663-50 from 2056248 to 2353013 tid 2 2024-09-18T14:02:15.201 INFO:tasks.rados.rados.0.smithi084.stdout:640: snap_remove snap 59 2024-09-18T14:02:16.186 INFO:tasks.rados.rados.0.smithi084.stdout:640: done (3 left) 2024-09-18T14:02:16.186 INFO:tasks.rados.rados.0.smithi084.stdout:641: copy_from oid 41 from oid 35 current snap is 64 2024-09-18T14:02:16.186 INFO:tasks.rados.rados.0.smithi084.stdout:636: expect (ObjNum 218 snap 50 seq_num 218) 2024-09-18T14:02:16.441 INFO:tasks.rados.rados.0.smithi084.stdout:637: expect (ObjNum 227 snap 53 seq_num 227) 2024-09-18T14:02:16.720 INFO:tasks.rados.rados.0.smithi084.stdout:639: finishing write tid 1 to smithi08434663-50 2024-09-18T14:02:16.721 INFO:tasks.rados.rados.0.smithi084.stdout:639: finishing write tid 2 to smithi08434663-50 2024-09-18T14:02:16.721 INFO:tasks.rados.rados.0.smithi084.stdout:639: finishing write tid 3 to smithi08434663-50 2024-09-18T14:02:16.721 INFO:tasks.rados.rados.0.smithi084.stdout:639: finishing write tid 4 to smithi08434663-50 2024-09-18T14:02:16.721 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 270 (ObjNum 259 snap 64 seq_num 259) dirty exists 2024-09-18T14:02:16.721 INFO:tasks.rados.rados.0.smithi084.stdout:639: left oid 50 (ObjNum 259 snap 64 seq_num 259) 2024-09-18T14:02:16.721 INFO:tasks.rados.rados.0.smithi084.stdout:641: finishing copy_from to smithi08434663-41 2024-09-18T14:02:16.721 INFO:tasks.rados.rados.0.smithi084.stdout:641: got expected ENOENT (src dne) 2024-09-18T14:02:16.721 INFO:tasks.rados.rados.0.smithi084.stdout:641: finishing copy_from racing read to smithi08434663-41 2024-09-18T14:02:16.721 INFO:tasks.rados.rados.0.smithi084.stdout:636: done (3 left) 2024-09-18T14:02:16.721 INFO:tasks.rados.rados.0.smithi084.stdout:637: done (2 left) 2024-09-18T14:02:16.721 INFO:tasks.rados.rados.0.smithi084.stdout:639: done (1 left) 2024-09-18T14:02:16.722 INFO:tasks.rados.rados.0.smithi084.stdout:641: done (0 left) 2024-09-18T14:02:16.722 INFO:tasks.rados.rados.0.smithi084.stdout:642: snap_create 2024-09-18T14:02:17.164 INFO:tasks.rados.rados.0.smithi084.stdout:642: done (0 left) 2024-09-18T14:02:17.164 INFO:tasks.rados.rados.0.smithi084.stdout:643: snap_remove snap 63 2024-09-18T14:02:18.174 INFO:tasks.rados.rados.0.smithi084.stdout:643: done (0 left) 2024-09-18T14:02:18.175 INFO:tasks.rados.rados.0.smithi084.stdout:644: write oid 21 current snap is 65 2024-09-18T14:02:18.175 INFO:tasks.rados.rados.0.smithi084.stdout:644: seq_num 260 ranges {769820=720742,2114730=255090} 2024-09-18T14:02:18.182 INFO:tasks.rados.rados.0.smithi084.stdout:644: writing smithi08434663-21 from 769820 to 1490562 tid 1 2024-09-18T14:02:18.188 INFO:tasks.rados.rados.0.smithi084.stdout:644: writing smithi08434663-21 from 2114730 to 2369820 tid 2 2024-09-18T14:02:18.188 INFO:tasks.rados.rados.0.smithi084.stdout:645: rollback oid 12 current snap is 65 2024-09-18T14:02:18.188 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 12 to 61 2024-09-18T14:02:18.188 INFO:tasks.rados.rados.0.smithi084.stdout:646: read oid 50 snap -1 2024-09-18T14:02:18.188 INFO:tasks.rados.rados.0.smithi084.stdout:646: expect (ObjNum 259 snap 64 seq_num 259) 2024-09-18T14:02:18.188 INFO:tasks.rados.rados.0.smithi084.stdout:647: write oid 26 current snap is 65 2024-09-18T14:02:18.188 INFO:tasks.rados.rados.0.smithi084.stdout:647: seq_num 261 ranges {786627=795991,2173212=213415} 2024-09-18T14:02:18.196 INFO:tasks.rados.rados.0.smithi084.stdout:647: writing smithi08434663-26 from 786627 to 1582618 tid 1 2024-09-18T14:02:18.202 INFO:tasks.rados.rados.0.smithi084.stdout:647: writing smithi08434663-26 from 2173212 to 2386627 tid 2 2024-09-18T14:02:18.202 INFO:tasks.rados.rados.0.smithi084.stdout:648: read oid 40 snap 51 2024-09-18T14:02:18.202 INFO:tasks.rados.rados.0.smithi084.stdout:648: expect (ObjNum 207 snap 46 seq_num 207) 2024-09-18T14:02:18.202 INFO:tasks.rados.rados.0.smithi084.stdout:649: rollback oid 5 current snap is 65 2024-09-18T14:02:18.202 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 5 to 64 2024-09-18T14:02:18.202 INFO:tasks.rados.rados.0.smithi084.stdout:644: finishing write tid 1 to smithi08434663-21 2024-09-18T14:02:18.202 INFO:tasks.rados.rados.0.smithi084.stdout:644: finishing write tid 2 to smithi08434663-21 2024-09-18T14:02:18.202 INFO:tasks.rados.rados.0.smithi084.stdout:644: finishing write tid 3 to smithi08434663-21 2024-09-18T14:02:18.202 INFO:tasks.rados.rados.0.smithi084.stdout:644: finishing write tid 4 to smithi08434663-21 2024-09-18T14:02:18.202 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 151 (ObjNum 260 snap 65 seq_num 260) dirty exists 2024-09-18T14:02:18.202 INFO:tasks.rados.rados.0.smithi084.stdout:644: left oid 21 (ObjNum 260 snap 65 seq_num 260) 2024-09-18T14:02:18.203 INFO:tasks.rados.rados.0.smithi084.stdout:645: finishing rollback tid 0 to smithi08434663-12 2024-09-18T14:02:18.203 INFO:tasks.rados.rados.0.smithi084.stdout:645: finishing rollback tid 1 to smithi08434663-12 2024-09-18T14:02:18.203 INFO:tasks.rados.rados.0.smithi084.stdout:645: finishing rollback tid 2 to smithi08434663-12 2024-09-18T14:02:18.203 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 276 (ObjNum 230 snap 54 seq_num 230) dirty exists 2024-09-18T14:02:18.203 INFO:tasks.rados.rados.0.smithi084.stdout:646: expect (ObjNum 259 snap 64 seq_num 259) 2024-09-18T14:02:18.281 INFO:tasks.rados.rados.0.smithi084.stdout:647: finishing write tid 1 to smithi08434663-26 2024-09-18T14:02:18.281 INFO:tasks.rados.rados.0.smithi084.stdout:649: finishing rollback tid 0 to smithi08434663-5 2024-09-18T14:02:18.281 INFO:tasks.rados.rados.0.smithi084.stdout:649: finishing rollback tid 1 to smithi08434663-5 2024-09-18T14:02:18.281 INFO:tasks.rados.rados.0.smithi084.stdout:649: finishing rollback tid 2 to smithi08434663-5 2024-09-18T14:02:18.281 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 130 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T14:02:18.281 INFO:tasks.rados.rados.0.smithi084.stdout:647: finishing write tid 2 to smithi08434663-26 2024-09-18T14:02:18.281 INFO:tasks.rados.rados.0.smithi084.stdout:644: done (5 left) 2024-09-18T14:02:18.281 INFO:tasks.rados.rados.0.smithi084.stdout:645: done (4 left) 2024-09-18T14:02:18.281 INFO:tasks.rados.rados.0.smithi084.stdout:646: done (3 left) 2024-09-18T14:02:18.281 INFO:tasks.rados.rados.0.smithi084.stdout:649: done (2 left) 2024-09-18T14:02:18.281 INFO:tasks.rados.rados.0.smithi084.stdout:650: read oid 39 snap -1 2024-09-18T14:02:18.282 INFO:tasks.rados.rados.0.smithi084.stdout:650: expect (ObjNum 245 snap 57 seq_num 245) 2024-09-18T14:02:18.282 INFO:tasks.rados.rados.0.smithi084.stdout:651: snap_remove snap 61 2024-09-18T14:02:19.194 INFO:tasks.rados.rados.0.smithi084.stdout:651: done (3 left) 2024-09-18T14:02:19.194 INFO:tasks.rados.rados.0.smithi084.stdout:652: read oid 33 snap -1 2024-09-18T14:02:19.194 INFO:tasks.rados.rados.0.smithi084.stdout:652: expect (ObjNum 216 snap 50 seq_num 216) 2024-09-18T14:02:19.194 INFO:tasks.rados.rados.0.smithi084.stdout:653: delete oid 29 current snap is 65 2024-09-18T14:02:19.194 INFO:tasks.rados.rados.0.smithi084.stdout:647: finishing write tid 3 to smithi08434663-26 2024-09-18T14:02:19.194 INFO:tasks.rados.rados.0.smithi084.stdout:647: finishing write tid 4 to smithi08434663-26 2024-09-18T14:02:19.194 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 198 (ObjNum 261 snap 65 seq_num 261) dirty exists 2024-09-18T14:02:19.194 INFO:tasks.rados.rados.0.smithi084.stdout:647: left oid 26 (ObjNum 261 snap 65 seq_num 261) 2024-09-18T14:02:19.194 INFO:tasks.rados.rados.0.smithi084.stdout:648: expect (ObjNum 207 snap 46 seq_num 207) 2024-09-18T14:02:19.394 INFO:tasks.rados.rados.0.smithi084.stdout:650: expect (ObjNum 245 snap 57 seq_num 245) 2024-09-18T14:02:19.517 INFO:tasks.rados.rados.0.smithi084.stdout:652: expect (ObjNum 216 snap 50 seq_num 216) 2024-09-18T14:02:19.680 INFO:tasks.rados.rados.0.smithi084.stdout:647: done (4 left) 2024-09-18T14:02:19.680 INFO:tasks.rados.rados.0.smithi084.stdout:648: done (3 left) 2024-09-18T14:02:19.680 INFO:tasks.rados.rados.0.smithi084.stdout:650: done (2 left) 2024-09-18T14:02:19.680 INFO:tasks.rados.rados.0.smithi084.stdout:652: done (1 left) 2024-09-18T14:02:19.680 INFO:tasks.rados.rados.0.smithi084.stdout:653: done (0 left) 2024-09-18T14:02:19.680 INFO:tasks.rados.rados.0.smithi084.stdout:654: snap_remove snap 51 2024-09-18T14:02:20.199 INFO:tasks.rados.rados.0.smithi084.stdout:654: done (0 left) 2024-09-18T14:02:20.199 INFO:tasks.rados.rados.0.smithi084.stdout:655: read oid 43 snap -1 2024-09-18T14:02:20.199 INFO:tasks.rados.rados.0.smithi084.stdout:655: expect deleted 2024-09-18T14:02:20.200 INFO:tasks.rados.rados.0.smithi084.stdout:656: rollback oid 23 current snap is 65 2024-09-18T14:02:20.200 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 23 to 57 2024-09-18T14:02:20.200 INFO:tasks.rados.rados.0.smithi084.stdout:657: snap_create 2024-09-18T14:02:20.206 INFO:tasks.rados.rados.0.smithi084.stdout:656: finishing rollback tid 0 to smithi08434663-23 2024-09-18T14:02:20.207 INFO:tasks.rados.rados.0.smithi084.stdout:656: finishing rollback tid 1 to smithi08434663-23 2024-09-18T14:02:20.208 INFO:tasks.rados.rados.0.smithi084.stdout:656: finishing rollback tid 2 to smithi08434663-23 2024-09-18T14:02:20.208 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 179 (ObjNum 236 snap 56 seq_num 236) dirty exists 2024-09-18T14:02:21.197 INFO:tasks.rados.rados.0.smithi084.stdout:655: done (2 left) 2024-09-18T14:02:21.197 INFO:tasks.rados.rados.0.smithi084.stdout:656: done (1 left) 2024-09-18T14:02:21.197 INFO:tasks.rados.rados.0.smithi084.stdout:657: done (0 left) 2024-09-18T14:02:21.197 INFO:tasks.rados.rados.0.smithi084.stdout:658: snap_create 2024-09-18T14:02:22.165 INFO:tasks.rados.rados.0.smithi084.stdout:658: done (0 left) 2024-09-18T14:02:22.165 INFO:tasks.rados.rados.0.smithi084.stdout:659: delete oid 22 current snap is 67 2024-09-18T14:02:22.168 INFO:tasks.rados.rados.0.smithi084.stdout:659: done (0 left) 2024-09-18T14:02:22.168 INFO:tasks.rados.rados.0.smithi084.stdout:660: rollback oid 36 current snap is 67 2024-09-18T14:02:22.168 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 36 to 65 2024-09-18T14:02:22.168 INFO:tasks.rados.rados.0.smithi084.stdout:661: read oid 48 snap -1 2024-09-18T14:02:22.168 INFO:tasks.rados.rados.0.smithi084.stdout:661: expect (ObjNum 221 snap 52 seq_num 221) 2024-09-18T14:02:22.169 INFO:tasks.rados.rados.0.smithi084.stdout:662: copy_from oid 47 from oid 49 current snap is 67 2024-09-18T14:02:22.169 INFO:tasks.rados.rados.0.smithi084.stdout:663: delete oid 46 current snap is 67 2024-09-18T14:02:22.169 INFO:tasks.rados.rados.0.smithi084.stdout:662: finishing copy_from to smithi08434663-47 2024-09-18T14:02:22.169 INFO:tasks.rados.rados.0.smithi084.stdout:662: got expected ENOENT (src dne) 2024-09-18T14:02:22.169 INFO:tasks.rados.rados.0.smithi084.stdout:662: finishing copy_from racing read to smithi08434663-47 2024-09-18T14:02:22.170 INFO:tasks.rados.rados.0.smithi084.stdout:660: finishing rollback tid 1 to smithi08434663-36 2024-09-18T14:02:22.170 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 181 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-18T14:02:22.172 INFO:tasks.rados.rados.0.smithi084.stdout:660: done (3 left) 2024-09-18T14:02:22.172 INFO:tasks.rados.rados.0.smithi084.stdout:662: done (2 left) 2024-09-18T14:02:22.172 INFO:tasks.rados.rados.0.smithi084.stdout:663: done (1 left) 2024-09-18T14:02:22.172 INFO:tasks.rados.rados.0.smithi084.stdout:664: write oid 43 current snap is 67 2024-09-18T14:02:22.172 INFO:tasks.rados.rados.0.smithi084.stdout:664: seq_num 265 ranges {453855=696987,1890787=548083} 2024-09-18T14:02:22.178 INFO:tasks.rados.rados.0.smithi084.stdout:664: writing smithi08434663-43 from 453855 to 1150842 tid 1 2024-09-18T14:02:22.186 INFO:tasks.rados.rados.0.smithi084.stdout:664: writing smithi08434663-43 from 1890787 to 2438870 tid 2 2024-09-18T14:02:22.186 INFO:tasks.rados.rados.0.smithi084.stdout:665: rollback oid 18 current snap is 67 2024-09-18T14:02:22.186 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 18 to 62 2024-09-18T14:02:22.186 INFO:tasks.rados.rados.0.smithi084.stdout:661: expect (ObjNum 221 snap 52 seq_num 221) 2024-09-18T14:02:22.304 INFO:tasks.rados.rados.0.smithi084.stdout:664: finishing write tid 1 to smithi08434663-43 2024-09-18T14:02:22.305 INFO:tasks.rados.rados.0.smithi084.stdout:664: finishing write tid 2 to smithi08434663-43 2024-09-18T14:02:22.305 INFO:tasks.rados.rados.0.smithi084.stdout:664: finishing write tid 3 to smithi08434663-43 2024-09-18T14:02:22.305 INFO:tasks.rados.rados.0.smithi084.stdout:664: finishing write tid 4 to smithi08434663-43 2024-09-18T14:02:22.305 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 86 (ObjNum 265 snap 67 seq_num 265) dirty exists 2024-09-18T14:02:22.305 INFO:tasks.rados.rados.0.smithi084.stdout:664: left oid 43 (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:02:22.305 INFO:tasks.rados.rados.0.smithi084.stdout:665: finishing rollback tid 0 to smithi08434663-18 2024-09-18T14:02:22.305 INFO:tasks.rados.rados.0.smithi084.stdout:665: finishing rollback tid 1 to smithi08434663-18 2024-09-18T14:02:22.305 INFO:tasks.rados.rados.0.smithi084.stdout:665: finishing rollback tid 2 to smithi08434663-18 2024-09-18T14:02:22.305 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 138 (ObjNum 235 snap 55 seq_num 235) dirty exists 2024-09-18T14:02:22.305 INFO:tasks.rados.rados.0.smithi084.stdout:661: done (2 left) 2024-09-18T14:02:22.305 INFO:tasks.rados.rados.0.smithi084.stdout:664: done (1 left) 2024-09-18T14:02:22.305 INFO:tasks.rados.rados.0.smithi084.stdout:665: done (0 left) 2024-09-18T14:02:22.306 INFO:tasks.rados.rados.0.smithi084.stdout:666: snap_create 2024-09-18T14:02:23.207 INFO:tasks.rados.rados.0.smithi084.stdout:666: done (0 left) 2024-09-18T14:02:23.207 INFO:tasks.rados.rados.0.smithi084.stdout:667: copy_from oid 43 from oid 28 current snap is 68 2024-09-18T14:02:23.207 INFO:tasks.rados.rados.0.smithi084.stdout:668: read oid 47 snap -1 2024-09-18T14:02:23.207 INFO:tasks.rados.rados.0.smithi084.stdout:668: expect deleted 2024-09-18T14:02:23.207 INFO:tasks.rados.rados.0.smithi084.stdout:669: write (excl) oid 33 current snap is 68 2024-09-18T14:02:23.207 INFO:tasks.rados.rados.0.smithi084.stdout:669: seq_num 266 ranges {470662=772236,1832916=408094} 2024-09-18T14:02:23.213 INFO:tasks.rados.rados.0.smithi084.stdout:669: writing smithi08434663-33 from 470662 to 1242898 tid 1 2024-09-18T14:02:23.219 INFO:tasks.rados.rados.0.smithi084.stdout:669: writing smithi08434663-33 from 1832916 to 2241010 tid 2 2024-09-18T14:02:23.219 INFO:tasks.rados.rados.0.smithi084.stdout:670: write oid 42 current snap is 68 2024-09-18T14:02:23.219 INFO:tasks.rados.rados.0.smithi084.stdout:670: seq_num 267 ranges {487469=563838,1607751=551752} 2024-09-18T14:02:23.225 INFO:tasks.rados.rados.0.smithi084.stdout:670: writing smithi08434663-42 from 487469 to 1051307 tid 1 2024-09-18T14:02:23.231 INFO:tasks.rados.rados.0.smithi084.stdout:670: writing smithi08434663-42 from 1607751 to 2159503 tid 2 2024-09-18T14:02:23.231 INFO:tasks.rados.rados.0.smithi084.stdout:671: snap_remove snap 65 2024-09-18T14:02:24.223 INFO:tasks.rados.rados.0.smithi084.stdout:671: done (4 left) 2024-09-18T14:02:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:672: delete oid 13 current snap is 68 2024-09-18T14:02:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:667: finishing copy_from to smithi08434663-43 2024-09-18T14:02:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:667: got expected ENOENT (src dne) 2024-09-18T14:02:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:667: finishing copy_from racing read to smithi08434663-43 2024-09-18T14:02:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:669: finishing write tid 1 to smithi08434663-33 2024-09-18T14:02:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:669: finishing write tid 2 to smithi08434663-33 2024-09-18T14:02:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:669: finishing write tid 3 to smithi08434663-33 2024-09-18T14:02:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:669: finishing write tid 4 to smithi08434663-33 2024-09-18T14:02:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 196 (ObjNum 266 snap 68 seq_num 266) dirty exists 2024-09-18T14:02:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:669: left oid 33 (ObjNum 266 snap 68 seq_num 266) 2024-09-18T14:02:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:670: finishing write tid 1 to smithi08434663-42 2024-09-18T14:02:24.225 INFO:tasks.rados.rados.0.smithi084.stdout:670: finishing write tid 2 to smithi08434663-42 2024-09-18T14:02:24.225 INFO:tasks.rados.rados.0.smithi084.stdout:670: finishing write tid 3 to smithi08434663-42 2024-09-18T14:02:24.225 INFO:tasks.rados.rados.0.smithi084.stdout:670: finishing write tid 4 to smithi08434663-42 2024-09-18T14:02:24.225 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 183 (ObjNum 267 snap 68 seq_num 267) dirty exists 2024-09-18T14:02:24.225 INFO:tasks.rados.rados.0.smithi084.stdout:670: left oid 42 (ObjNum 267 snap 68 seq_num 267) 2024-09-18T14:02:24.227 INFO:tasks.rados.rados.0.smithi084.stdout:667: done (4 left) 2024-09-18T14:02:24.227 INFO:tasks.rados.rados.0.smithi084.stdout:668: done (3 left) 2024-09-18T14:02:24.227 INFO:tasks.rados.rados.0.smithi084.stdout:669: done (2 left) 2024-09-18T14:02:24.227 INFO:tasks.rados.rados.0.smithi084.stdout:670: done (1 left) 2024-09-18T14:02:24.227 INFO:tasks.rados.rados.0.smithi084.stdout:672: done (0 left) 2024-09-18T14:02:24.227 INFO:tasks.rados.rados.0.smithi084.stdout:673: rollback oid 45 current snap is 68 2024-09-18T14:02:24.227 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 45 to 66 2024-09-18T14:02:24.227 INFO:tasks.rados.rados.0.smithi084.stdout:674: snap_remove snap 64 2024-09-18T14:02:25.222 INFO:tasks.rados.rados.0.smithi084.stdout:674: done (1 left) 2024-09-18T14:02:25.222 INFO:tasks.rados.rados.0.smithi084.stdout:675: delete oid 23 current snap is 68 2024-09-18T14:02:25.222 INFO:tasks.rados.rados.0.smithi084.stdout:673: finishing rollback tid 1 to smithi08434663-45 2024-09-18T14:02:25.222 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 0 (ObjNum 129 snap 0 seq_num 2495445544) dirty dne 2024-09-18T14:02:25.228 INFO:tasks.rados.rados.0.smithi084.stdout:673: done (1 left) 2024-09-18T14:02:25.228 INFO:tasks.rados.rados.0.smithi084.stdout:675: done (0 left) 2024-09-18T14:02:25.228 INFO:tasks.rados.rados.0.smithi084.stdout:676: copy_from oid 35 from oid 31 current snap is 68 2024-09-18T14:02:25.229 INFO:tasks.rados.rados.0.smithi084.stdout:677: write oid 16 current snap is 68 2024-09-18T14:02:25.229 INFO:tasks.rados.rados.0.smithi084.stdout:677: seq_num 270 ranges {537890=789585,1783197=699079} 2024-09-18T14:02:25.236 INFO:tasks.rados.rados.0.smithi084.stdout:677: writing smithi08434663-16 from 537890 to 1327475 tid 1 2024-09-18T14:02:25.242 INFO:tasks.rados.rados.0.smithi084.stdout:677: writing smithi08434663-16 from 1783197 to 2482276 tid 2 2024-09-18T14:02:25.242 INFO:tasks.rados.rados.0.smithi084.stdout:678: snap_create 2024-09-18T14:02:25.242 INFO:tasks.rados.rados.0.smithi084.stdout:676: finishing copy_from to smithi08434663-35 2024-09-18T14:02:25.243 INFO:tasks.rados.rados.0.smithi084.stdout:676: got expected ENOENT (src dne) 2024-09-18T14:02:25.243 INFO:tasks.rados.rados.0.smithi084.stdout:676: finishing copy_from racing read to smithi08434663-35 2024-09-18T14:02:25.243 INFO:tasks.rados.rados.0.smithi084.stdout:677: finishing write tid 1 to smithi08434663-16 2024-09-18T14:02:25.246 INFO:tasks.rados.rados.0.smithi084.stdout:677: finishing write tid 2 to smithi08434663-16 2024-09-18T14:02:25.246 INFO:tasks.rados.rados.0.smithi084.stdout:677: finishing write tid 3 to smithi08434663-16 2024-09-18T14:02:25.246 INFO:tasks.rados.rados.0.smithi084.stdout:677: finishing write tid 4 to smithi08434663-16 2024-09-18T14:02:25.246 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 257 (ObjNum 270 snap 68 seq_num 270) dirty exists 2024-09-18T14:02:25.246 INFO:tasks.rados.rados.0.smithi084.stdout:677: left oid 16 (ObjNum 270 snap 68 seq_num 270) 2024-09-18T14:02:26.232 INFO:tasks.rados.rados.0.smithi084.stdout:676: done (2 left) 2024-09-18T14:02:26.232 INFO:tasks.rados.rados.0.smithi084.stdout:677: done (1 left) 2024-09-18T14:02:26.232 INFO:tasks.rados.rados.0.smithi084.stdout:678: done (0 left) 2024-09-18T14:02:26.232 INFO:tasks.rados.rados.0.smithi084.stdout:679: write oid 6 current snap is 69 2024-09-18T14:02:26.232 INFO:tasks.rados.rados.0.smithi084.stdout:679: seq_num 271 ranges {554697=464834,1725326=559090} 2024-09-18T14:02:26.237 INFO:tasks.rados.rados.0.smithi084.stdout:679: writing smithi08434663-6 from 554697 to 1019531 tid 1 2024-09-18T14:02:26.244 INFO:tasks.rados.rados.0.smithi084.stdout:679: writing smithi08434663-6 from 1725326 to 2284416 tid 2 2024-09-18T14:02:26.244 INFO:tasks.rados.rados.0.smithi084.stdout:680: copy_from oid 26 from oid 7 current snap is 69 2024-09-18T14:02:26.244 INFO:tasks.rados.rados.0.smithi084.stdout:679: finishing write tid 1 to smithi08434663-6 2024-09-18T14:02:26.244 INFO:tasks.rados.rados.0.smithi084.stdout:681: snap_create 2024-09-18T14:02:26.250 INFO:tasks.rados.rados.0.smithi084.stdout:679: finishing write tid 2 to smithi08434663-6 2024-09-18T14:02:26.250 INFO:tasks.rados.rados.0.smithi084.stdout:679: finishing write tid 3 to smithi08434663-6 2024-09-18T14:02:26.250 INFO:tasks.rados.rados.0.smithi084.stdout:679: finishing write tid 4 to smithi08434663-6 2024-09-18T14:02:26.250 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 183 (ObjNum 271 snap 69 seq_num 271) dirty exists 2024-09-18T14:02:26.250 INFO:tasks.rados.rados.0.smithi084.stdout:679: left oid 6 (ObjNum 271 snap 69 seq_num 271) 2024-09-18T14:02:26.257 INFO:tasks.rados.rados.0.smithi084.stdout:680: finishing copy_from racing read to smithi08434663-26 2024-09-18T14:02:26.260 INFO:tasks.rados.rados.0.smithi084.stdout:680: finishing copy_from to smithi08434663-26 2024-09-18T14:02:26.260 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 172 (ObjNum 178 snap 41 seq_num 178) dirty exists 2024-09-18T14:02:27.169 INFO:tasks.rados.rados.0.smithi084.stdout:679: done (2 left) 2024-09-18T14:02:27.169 INFO:tasks.rados.rados.0.smithi084.stdout:680: done (1 left) 2024-09-18T14:02:27.169 INFO:tasks.rados.rados.0.smithi084.stdout:681: done (0 left) 2024-09-18T14:02:27.170 INFO:tasks.rados.rados.0.smithi084.stdout:682: write (excl) oid 26 current snap is 70 2024-09-18T14:02:27.170 INFO:tasks.rados.rados.0.smithi084.stdout:682: seq_num 272 ranges {571504=540083,1783808=702748} 2024-09-18T14:02:27.175 INFO:tasks.rados.rados.0.smithi084.stdout:682: writing smithi08434663-26 from 571504 to 1111587 tid 1 2024-09-18T14:02:27.183 INFO:tasks.rados.rados.0.smithi084.stdout:682: writing smithi08434663-26 from 1783808 to 2486556 tid 2 2024-09-18T14:02:27.183 INFO:tasks.rados.rados.0.smithi084.stdout:683: snap_remove snap 69 2024-09-18T14:02:28.176 INFO:tasks.rados.rados.0.smithi084.stdout:682: finishing write tid 1 to smithi08434663-26 2024-09-18T14:02:28.176 INFO:tasks.rados.rados.0.smithi084.stdout:682: finishing write tid 2 to smithi08434663-26 2024-09-18T14:02:28.177 INFO:tasks.rados.rados.0.smithi084.stdout:682: finishing write tid 3 to smithi08434663-26 2024-09-18T14:02:28.177 INFO:tasks.rados.rados.0.smithi084.stdout:682: finishing write tid 4 to smithi08434663-26 2024-09-18T14:02:28.177 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 208 (ObjNum 272 snap 70 seq_num 272) dirty exists 2024-09-18T14:02:28.177 INFO:tasks.rados.rados.0.smithi084.stdout:682: left oid 26 (ObjNum 272 snap 70 seq_num 272) 2024-09-18T14:02:28.177 INFO:tasks.rados.rados.0.smithi084.stdout:682: done (1 left) 2024-09-18T14:02:28.177 INFO:tasks.rados.rados.0.smithi084.stdout:683: done (0 left) 2024-09-18T14:02:28.177 INFO:tasks.rados.rados.0.smithi084.stdout:684: snap_create 2024-09-18T14:02:29.181 INFO:tasks.rados.rados.0.smithi084.stdout:684: done (0 left) 2024-09-18T14:02:29.181 INFO:tasks.rados.rados.0.smithi084.stdout:685: read oid 47 snap 66 2024-09-18T14:02:29.181 INFO:tasks.rados.rados.0.smithi084.stdout:685: expect deleted 2024-09-18T14:02:29.181 INFO:tasks.rados.rados.0.smithi084.stdout:686: read oid 13 snap -1 2024-09-18T14:02:29.181 INFO:tasks.rados.rados.0.smithi084.stdout:686: expect deleted 2024-09-18T14:02:29.181 INFO:tasks.rados.rados.0.smithi084.stdout:687: copy_from oid 9 from oid 29 current snap is 71 2024-09-18T14:02:29.181 INFO:tasks.rados.rados.0.smithi084.stdout:688: write oid 31 current snap is 71 2024-09-18T14:02:29.181 INFO:tasks.rados.rados.0.smithi084.stdout:688: seq_num 273 ranges {588311=615332,1842290=562759} 2024-09-18T14:02:29.187 INFO:tasks.rados.rados.0.smithi084.stdout:688: writing smithi08434663-31 from 588311 to 1203643 tid 1 2024-09-18T14:02:29.194 INFO:tasks.rados.rados.0.smithi084.stdout:688: writing smithi08434663-31 from 1842290 to 2405049 tid 2 2024-09-18T14:02:29.194 INFO:tasks.rados.rados.0.smithi084.stdout:689: snap_remove snap 70 2024-09-18T14:02:30.190 INFO:tasks.rados.rados.0.smithi084.stdout:689: done (4 left) 2024-09-18T14:02:30.190 INFO:tasks.rados.rados.0.smithi084.stdout:690: read oid 39 snap 67 2024-09-18T14:02:30.190 INFO:tasks.rados.rados.0.smithi084.stdout:690: expect (ObjNum 245 snap 57 seq_num 245) 2024-09-18T14:02:30.190 INFO:tasks.rados.rados.0.smithi084.stdout:691: read oid 48 snap -1 2024-09-18T14:02:30.190 INFO:tasks.rados.rados.0.smithi084.stdout:691: expect (ObjNum 221 snap 52 seq_num 221) 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:687: finishing copy_from to smithi08434663-9 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:687: got expected ENOENT (src dne) 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:687: finishing copy_from racing read to smithi08434663-9 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:688: finishing write tid 1 to smithi08434663-31 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:688: finishing write tid 2 to smithi08434663-31 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:688: finishing write tid 3 to smithi08434663-31 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:688: finishing write tid 4 to smithi08434663-31 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 177 (ObjNum 273 snap 71 seq_num 273) dirty exists 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:688: left oid 31 (ObjNum 273 snap 71 seq_num 273) 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:685: done (5 left) 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:686: done (4 left) 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:687: done (3 left) 2024-09-18T14:02:30.191 INFO:tasks.rados.rados.0.smithi084.stdout:688: done (2 left) 2024-09-18T14:02:30.192 INFO:tasks.rados.rados.0.smithi084.stdout:692: write oid 28 current snap is 71 2024-09-18T14:02:30.192 INFO:tasks.rados.rados.0.smithi084.stdout:692: seq_num 274 ranges {605118=406934,1617125=706417} 2024-09-18T14:02:30.195 INFO:tasks.rados.rados.0.smithi084.stdout:692: writing smithi08434663-28 from 605118 to 1012052 tid 1 2024-09-18T14:02:30.203 INFO:tasks.rados.rados.0.smithi084.stdout:692: writing smithi08434663-28 from 1617125 to 2323542 tid 2 2024-09-18T14:02:30.203 INFO:tasks.rados.rados.0.smithi084.stdout:693: read oid 45 snap -1 2024-09-18T14:02:30.203 INFO:tasks.rados.rados.0.smithi084.stdout:693: expect deleted 2024-09-18T14:02:30.204 INFO:tasks.rados.rados.0.smithi084.stdout:694: copy_from oid 16 from oid 2 current snap is 71 2024-09-18T14:02:30.204 INFO:tasks.rados.rados.0.smithi084.stdout:690: expect (ObjNum 245 snap 57 seq_num 245) 2024-09-18T14:02:30.329 INFO:tasks.rados.rados.0.smithi084.stdout:691: expect (ObjNum 221 snap 52 seq_num 221) 2024-09-18T14:02:30.446 INFO:tasks.rados.rados.0.smithi084.stdout:692: finishing write tid 1 to smithi08434663-28 2024-09-18T14:02:30.446 INFO:tasks.rados.rados.0.smithi084.stdout:694: finishing copy_from to smithi08434663-16 2024-09-18T14:02:30.446 INFO:tasks.rados.rados.0.smithi084.stdout:694: got expected ENOENT (src dne) 2024-09-18T14:02:30.446 INFO:tasks.rados.rados.0.smithi084.stdout:694: finishing copy_from racing read to smithi08434663-16 2024-09-18T14:02:30.446 INFO:tasks.rados.rados.0.smithi084.stdout:692: finishing write tid 2 to smithi08434663-28 2024-09-18T14:02:30.446 INFO:tasks.rados.rados.0.smithi084.stdout:692: finishing write tid 3 to smithi08434663-28 2024-09-18T14:02:30.447 INFO:tasks.rados.rados.0.smithi084.stdout:690: done (4 left) 2024-09-18T14:02:30.447 INFO:tasks.rados.rados.0.smithi084.stdout:691: done (3 left) 2024-09-18T14:02:30.447 INFO:tasks.rados.rados.0.smithi084.stdout:693: done (2 left) 2024-09-18T14:02:30.447 INFO:tasks.rados.rados.0.smithi084.stdout:694: done (1 left) 2024-09-18T14:02:30.447 INFO:tasks.rados.rados.0.smithi084.stdout:695: rollback oid 5 current snap is 71 2024-09-18T14:02:30.447 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 5 to 67 2024-09-18T14:02:30.447 INFO:tasks.rados.rados.0.smithi084.stdout:692: finishing write tid 4 to smithi08434663-28 2024-09-18T14:02:30.447 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 291 (ObjNum 274 snap 71 seq_num 274) dirty exists 2024-09-18T14:02:30.447 INFO:tasks.rados.rados.0.smithi084.stdout:692: left oid 28 (ObjNum 274 snap 71 seq_num 274) 2024-09-18T14:02:30.447 INFO:tasks.rados.rados.0.smithi084.stdout:692: done (1 left) 2024-09-18T14:02:30.447 INFO:tasks.rados.rados.0.smithi084.stdout:696: snap_remove snap 57 2024-09-18T14:02:31.191 INFO:tasks.rados.rados.0.smithi084.stdout:695: finishing rollback tid 0 to smithi08434663-5 2024-09-18T14:02:31.191 INFO:tasks.rados.rados.0.smithi084.stdout:695: finishing rollback tid 1 to smithi08434663-5 2024-09-18T14:02:31.191 INFO:tasks.rados.rados.0.smithi084.stdout:695: finishing rollback tid 2 to smithi08434663-5 2024-09-18T14:02:31.191 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 133 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T14:02:31.191 INFO:tasks.rados.rados.0.smithi084.stdout:695: done (1 left) 2024-09-18T14:02:31.191 INFO:tasks.rados.rados.0.smithi084.stdout:696: done (0 left) 2024-09-18T14:02:31.191 INFO:tasks.rados.rados.0.smithi084.stdout:697: read oid 21 snap -1 2024-09-18T14:02:31.191 INFO:tasks.rados.rados.0.smithi084.stdout:697: expect (ObjNum 260 snap 65 seq_num 260) 2024-09-18T14:02:31.192 INFO:tasks.rados.rados.0.smithi084.stdout:698: snap_remove snap 67 2024-09-18T14:02:32.168 INFO:tasks.rados.rados.0.smithi084.stdout:698: done (1 left) 2024-09-18T14:02:32.168 INFO:tasks.rados.rados.0.smithi084.stdout:699: read oid 15 snap 66 2024-09-18T14:02:32.168 INFO:tasks.rados.rados.0.smithi084.stdout:699: expect (ObjNum 237 snap 56 seq_num 237) 2024-09-18T14:02:32.168 INFO:tasks.rados.rados.0.smithi084.stdout:700: delete oid 32 current snap is 71 2024-09-18T14:02:32.168 INFO:tasks.rados.rados.0.smithi084.stdout:697: expect (ObjNum 260 snap 65 seq_num 260) 2024-09-18T14:02:32.294 INFO:tasks.rados.rados.0.smithi084.stdout:699: expect (ObjNum 237 snap 56 seq_num 237) 2024-09-18T14:02:32.464 INFO:tasks.rados.rados.0.smithi084.stdout:697: done (2 left) 2024-09-18T14:02:32.464 INFO:tasks.rados.rados.0.smithi084.stdout:699: done (1 left) 2024-09-18T14:02:32.464 INFO:tasks.rados.rados.0.smithi084.stdout:700: done (0 left) 2024-09-18T14:02:32.464 INFO:tasks.rados.rados.0.smithi084.stdout:701: delete oid 20 current snap is 71 2024-09-18T14:02:32.465 INFO:tasks.rados.rados.0.smithi084.stdout:701: done (0 left) 2024-09-18T14:02:32.465 INFO:tasks.rados.rados.0.smithi084.stdout:702: rollback oid 24 current snap is 71 2024-09-18T14:02:32.465 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 24 to 68 2024-09-18T14:02:32.465 INFO:tasks.rados.rados.0.smithi084.stdout:703: delete oid 12 current snap is 71 2024-09-18T14:02:32.469 INFO:tasks.rados.rados.0.smithi084.stdout:703: done (1 left) 2024-09-18T14:02:32.469 INFO:tasks.rados.rados.0.smithi084.stdout:704: read oid 20 snap 66 2024-09-18T14:02:32.469 INFO:tasks.rados.rados.0.smithi084.stdout:704: expect deleted 2024-09-18T14:02:32.469 INFO:tasks.rados.rados.0.smithi084.stdout:705: write oid 46 current snap is 71 2024-09-18T14:02:32.469 INFO:tasks.rados.rados.0.smithi084.stdout:705: seq_num 278 ranges {672346=707930,2134700=537646} 2024-09-18T14:02:32.476 INFO:tasks.rados.rados.0.smithi084.stdout:705: writing smithi08434663-46 from 672346 to 1380276 tid 1 2024-09-18T14:02:32.483 INFO:tasks.rados.rados.0.smithi084.stdout:705: writing smithi08434663-46 from 2134700 to 2672346 tid 2 2024-09-18T14:02:32.483 INFO:tasks.rados.rados.0.smithi084.stdout:706: snap_remove snap 62 2024-09-18T14:02:33.204 INFO:tasks.rados.rados.0.smithi084.stdout:706: done (3 left) 2024-09-18T14:02:33.204 INFO:tasks.rados.rados.0.smithi084.stdout:707: copy_from oid 27 from oid 41 current snap is 71 2024-09-18T14:02:33.204 INFO:tasks.rados.rados.0.smithi084.stdout:702: finishing rollback tid 0 to smithi08434663-24 2024-09-18T14:02:33.205 INFO:tasks.rados.rados.0.smithi084.stdout:702: finishing rollback tid 1 to smithi08434663-24 2024-09-18T14:02:33.205 INFO:tasks.rados.rados.0.smithi084.stdout:702: finishing rollback tid 2 to smithi08434663-24 2024-09-18T14:02:33.205 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 180 (ObjNum 205 snap 46 seq_num 205) dirty exists 2024-09-18T14:02:33.205 INFO:tasks.rados.rados.0.smithi084.stdout:705: finishing write tid 1 to smithi08434663-46 2024-09-18T14:02:33.205 INFO:tasks.rados.rados.0.smithi084.stdout:705: finishing write tid 2 to smithi08434663-46 2024-09-18T14:02:33.205 INFO:tasks.rados.rados.0.smithi084.stdout:705: finishing write tid 3 to smithi08434663-46 2024-09-18T14:02:33.205 INFO:tasks.rados.rados.0.smithi084.stdout:705: finishing write tid 4 to smithi08434663-46 2024-09-18T14:02:33.205 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 262 (ObjNum 278 snap 71 seq_num 278) dirty exists 2024-09-18T14:02:33.205 INFO:tasks.rados.rados.0.smithi084.stdout:705: left oid 46 (ObjNum 278 snap 71 seq_num 278) 2024-09-18T14:02:33.205 INFO:tasks.rados.rados.0.smithi084.stdout:702: done (3 left) 2024-09-18T14:02:33.205 INFO:tasks.rados.rados.0.smithi084.stdout:704: done (2 left) 2024-09-18T14:02:33.205 INFO:tasks.rados.rados.0.smithi084.stdout:705: done (1 left) 2024-09-18T14:02:33.206 INFO:tasks.rados.rados.0.smithi084.stdout:708: rollback oid 9 current snap is 71 2024-09-18T14:02:33.206 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 9 to 68 2024-09-18T14:02:33.206 INFO:tasks.rados.rados.0.smithi084.stdout:709: copy_from oid 34 from oid 21 current snap is 71 2024-09-18T14:02:33.206 INFO:tasks.rados.rados.0.smithi084.stdout:710: snap_remove snap 66 2024-09-18T14:02:34.210 INFO:tasks.rados.rados.0.smithi084.stdout:710: done (3 left) 2024-09-18T14:02:34.210 INFO:tasks.rados.rados.0.smithi084.stdout:711: read oid 5 snap -1 2024-09-18T14:02:34.211 INFO:tasks.rados.rados.0.smithi084.stdout:711: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:02:34.211 INFO:tasks.rados.rados.0.smithi084.stdout:712: delete oid 13 current snap is 71 2024-09-18T14:02:34.211 INFO:tasks.rados.rados.0.smithi084.stdout:708: finishing rollback tid 0 to smithi08434663-9 2024-09-18T14:02:34.211 INFO:tasks.rados.rados.0.smithi084.stdout:708: finishing rollback tid 1 to smithi08434663-9 2024-09-18T14:02:34.211 INFO:tasks.rados.rados.0.smithi084.stdout:708: finishing rollback tid 2 to smithi08434663-9 2024-09-18T14:02:34.211 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 268 (ObjNum 246 snap 57 seq_num 246) dirty exists 2024-09-18T14:02:34.211 INFO:tasks.rados.rados.0.smithi084.stdout:707: finishing copy_from racing read to smithi08434663-27 2024-09-18T14:02:34.211 INFO:tasks.rados.rados.0.smithi084.stdout:707: finishing copy_from to smithi08434663-27 2024-09-18T14:02:34.211 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 142 (ObjNum 152 snap 31 seq_num 152) dirty exists 2024-09-18T14:02:34.211 INFO:tasks.rados.rados.0.smithi084.stdout:709: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:02:34.211 INFO:tasks.rados.rados.0.smithi084.stdout:709: finishing copy_from to smithi08434663-34 2024-09-18T14:02:34.211 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 151 (ObjNum 260 snap 65 seq_num 260) dirty exists 2024-09-18T14:02:34.216 INFO:tasks.rados.rados.0.smithi084.stdout:707: done (4 left) 2024-09-18T14:02:34.216 INFO:tasks.rados.rados.0.smithi084.stdout:708: done (3 left) 2024-09-18T14:02:34.216 INFO:tasks.rados.rados.0.smithi084.stdout:709: done (2 left) 2024-09-18T14:02:34.216 INFO:tasks.rados.rados.0.smithi084.stdout:712: done (1 left) 2024-09-18T14:02:34.216 INFO:tasks.rados.rados.0.smithi084.stdout:713: copy_from oid 11 from oid 20 current snap is 71 2024-09-18T14:02:34.216 INFO:tasks.rados.rados.0.smithi084.stdout:714: snap_remove snap 68 2024-09-18T14:02:35.218 INFO:tasks.rados.rados.0.smithi084.stdout:714: done (2 left) 2024-09-18T14:02:35.218 INFO:tasks.rados.rados.0.smithi084.stdout:715: read oid 44 snap -1 2024-09-18T14:02:35.218 INFO:tasks.rados.rados.0.smithi084.stdout:715: expect (ObjNum 88 snap 9 seq_num 88) 2024-09-18T14:02:35.218 INFO:tasks.rados.rados.0.smithi084.stdout:716: read oid 28 snap -1 2024-09-18T14:02:35.218 INFO:tasks.rados.rados.0.smithi084.stdout:716: expect (ObjNum 274 snap 71 seq_num 274) 2024-09-18T14:02:35.218 INFO:tasks.rados.rados.0.smithi084.stdout:717: copy_from oid 40 from oid 19 current snap is 71 2024-09-18T14:02:35.218 INFO:tasks.rados.rados.0.smithi084.stdout:718: write (excl) oid 25 current snap is 71 2024-09-18T14:02:35.218 INFO:tasks.rados.rados.0.smithi084.stdout:718: seq_num 280 ranges {705960=458428,1735311=717424} 2024-09-18T14:02:35.224 INFO:tasks.rados.rados.0.smithi084.stdout:718: writing smithi08434663-25 from 705960 to 1164388 tid 1 2024-09-18T14:02:35.232 INFO:tasks.rados.rados.0.smithi084.stdout:718: writing smithi08434663-25 from 1735311 to 2452735 tid 2 2024-09-18T14:02:35.232 INFO:tasks.rados.rados.0.smithi084.stdout:719: read oid 34 snap -1 2024-09-18T14:02:35.232 INFO:tasks.rados.rados.0.smithi084.stdout:719: expect (ObjNum 260 snap 65 seq_num 260) 2024-09-18T14:02:35.232 INFO:tasks.rados.rados.0.smithi084.stdout:720: read oid 41 snap -1 2024-09-18T14:02:35.233 INFO:tasks.rados.rados.0.smithi084.stdout:720: expect (ObjNum 152 snap 31 seq_num 152) 2024-09-18T14:02:35.233 INFO:tasks.rados.rados.0.smithi084.stdout:721: copy_from oid 7 from oid 48 current snap is 71 2024-09-18T14:02:35.233 INFO:tasks.rados.rados.0.smithi084.stdout:713: finishing copy_from to smithi08434663-11 2024-09-18T14:02:35.233 INFO:tasks.rados.rados.0.smithi084.stdout:713: got expected ENOENT (src dne) 2024-09-18T14:02:35.233 INFO:tasks.rados.rados.0.smithi084.stdout:713: finishing copy_from racing read to smithi08434663-11 2024-09-18T14:02:35.233 INFO:tasks.rados.rados.0.smithi084.stdout:711: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:02:35.305 INFO:tasks.rados.rados.0.smithi084.stdout:715: expect (ObjNum 88 snap 9 seq_num 88) 2024-09-18T14:02:35.463 INFO:tasks.rados.rados.0.smithi084.stdout:716: expect (ObjNum 274 snap 71 seq_num 274) 2024-09-18T14:02:35.547 INFO:tasks.rados.rados.0.smithi084.stdout:718: finishing write tid 1 to smithi08434663-25 2024-09-18T14:02:35.548 INFO:tasks.rados.rados.0.smithi084.stdout:717: finishing copy_from racing read to smithi08434663-40 2024-09-18T14:02:35.548 INFO:tasks.rados.rados.0.smithi084.stdout:718: finishing write tid 2 to smithi08434663-25 2024-09-18T14:02:35.548 INFO:tasks.rados.rados.0.smithi084.stdout:711: done (8 left) 2024-09-18T14:02:35.548 INFO:tasks.rados.rados.0.smithi084.stdout:713: done (7 left) 2024-09-18T14:02:35.548 INFO:tasks.rados.rados.0.smithi084.stdout:715: done (6 left) 2024-09-18T14:02:35.548 INFO:tasks.rados.rados.0.smithi084.stdout:716: done (5 left) 2024-09-18T14:02:35.548 INFO:tasks.rados.rados.0.smithi084.stdout:722: copy_from oid 39 from oid 21 current snap is 71 2024-09-18T14:02:35.548 INFO:tasks.rados.rados.0.smithi084.stdout:718: finishing write tid 3 to smithi08434663-25 2024-09-18T14:02:35.548 INFO:tasks.rados.rados.0.smithi084.stdout:718: finishing write tid 4 to smithi08434663-25 2024-09-18T14:02:35.548 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 164 (ObjNum 280 snap 71 seq_num 280) dirty exists 2024-09-18T14:02:35.549 INFO:tasks.rados.rados.0.smithi084.stdout:718: left oid 25 (ObjNum 280 snap 71 seq_num 280) 2024-09-18T14:02:35.549 INFO:tasks.rados.rados.0.smithi084.stdout:717: finishing copy_from to smithi08434663-40 2024-09-18T14:02:35.549 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 257 (ObjNum 247 snap 57 seq_num 247) dirty exists 2024-09-18T14:02:35.549 INFO:tasks.rados.rados.0.smithi084.stdout:719: expect (ObjNum 260 snap 65 seq_num 260) 2024-09-18T14:02:35.671 INFO:tasks.rados.rados.0.smithi084.stdout:721: finishing copy_from racing read to smithi08434663-7 2024-09-18T14:02:35.671 INFO:tasks.rados.rados.0.smithi084.stdout:721: finishing copy_from to smithi08434663-7 2024-09-18T14:02:35.671 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 130 (ObjNum 221 snap 52 seq_num 221) dirty exists 2024-09-18T14:02:35.671 INFO:tasks.rados.rados.0.smithi084.stdout:720: expect (ObjNum 152 snap 31 seq_num 152) 2024-09-18T14:02:35.789 INFO:tasks.rados.rados.0.smithi084.stdout:722: finishing copy_from racing read to smithi08434663-39 2024-09-18T14:02:35.789 INFO:tasks.rados.rados.0.smithi084.stdout:722: finishing copy_from to smithi08434663-39 2024-09-18T14:02:35.789 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 151 (ObjNum 260 snap 65 seq_num 260) dirty exists 2024-09-18T14:02:35.789 INFO:tasks.rados.rados.0.smithi084.stdout:717: done (5 left) 2024-09-18T14:02:35.789 INFO:tasks.rados.rados.0.smithi084.stdout:718: done (4 left) 2024-09-18T14:02:35.789 INFO:tasks.rados.rados.0.smithi084.stdout:719: done (3 left) 2024-09-18T14:02:35.789 INFO:tasks.rados.rados.0.smithi084.stdout:720: done (2 left) 2024-09-18T14:02:35.789 INFO:tasks.rados.rados.0.smithi084.stdout:721: done (1 left) 2024-09-18T14:02:35.790 INFO:tasks.rados.rados.0.smithi084.stdout:722: done (0 left) 2024-09-18T14:02:35.790 INFO:tasks.rados.rados.0.smithi084.stdout:723: write oid 25 current snap is 71 2024-09-18T14:02:35.790 INFO:tasks.rados.rados.0.smithi084.stdout:723: seq_num 281 ranges {722767=533677,1793793=461082} 2024-09-18T14:02:35.795 INFO:tasks.rados.rados.0.smithi084.stdout:723: writing smithi08434663-25 from 722767 to 1256444 tid 1 2024-09-18T14:02:35.801 INFO:tasks.rados.rados.0.smithi084.stdout:723: writing smithi08434663-25 from 1793793 to 2254875 tid 2 2024-09-18T14:02:35.801 INFO:tasks.rados.rados.0.smithi084.stdout:724: read oid 19 snap -1 2024-09-18T14:02:35.801 INFO:tasks.rados.rados.0.smithi084.stdout:724: expect (ObjNum 247 snap 57 seq_num 247) 2024-09-18T14:02:35.801 INFO:tasks.rados.rados.0.smithi084.stdout:725: read oid 36 snap -1 2024-09-18T14:02:35.801 INFO:tasks.rados.rados.0.smithi084.stdout:725: expect deleted 2024-09-18T14:02:35.801 INFO:tasks.rados.rados.0.smithi084.stdout:726: read oid 10 snap -1 2024-09-18T14:02:35.801 INFO:tasks.rados.rados.0.smithi084.stdout:726: expect (ObjNum 231 snap 54 seq_num 231) 2024-09-18T14:02:35.801 INFO:tasks.rados.rados.0.smithi084.stdout:727: delete oid 44 current snap is 71 2024-09-18T14:02:35.801 INFO:tasks.rados.rados.0.smithi084.stdout:723: finishing write tid 1 to smithi08434663-25 2024-09-18T14:02:35.803 INFO:tasks.rados.rados.0.smithi084.stdout:724: expect (ObjNum 247 snap 57 seq_num 247) 2024-09-18T14:02:35.954 INFO:tasks.rados.rados.0.smithi084.stdout:723: finishing write tid 2 to smithi08434663-25 2024-09-18T14:02:35.954 INFO:tasks.rados.rados.0.smithi084.stdout:723: finishing write tid 3 to smithi08434663-25 2024-09-18T14:02:35.954 INFO:tasks.rados.rados.0.smithi084.stdout:723: finishing write tid 4 to smithi08434663-25 2024-09-18T14:02:35.954 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 167 (ObjNum 281 snap 71 seq_num 281) dirty exists 2024-09-18T14:02:35.954 INFO:tasks.rados.rados.0.smithi084.stdout:723: left oid 25 (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:02:35.954 INFO:tasks.rados.rados.0.smithi084.stdout:723: done (4 left) 2024-09-18T14:02:35.955 INFO:tasks.rados.rados.0.smithi084.stdout:724: done (3 left) 2024-09-18T14:02:35.955 INFO:tasks.rados.rados.0.smithi084.stdout:725: done (2 left) 2024-09-18T14:02:35.955 INFO:tasks.rados.rados.0.smithi084.stdout:727: done (1 left) 2024-09-18T14:02:35.955 INFO:tasks.rados.rados.0.smithi084.stdout:728: read oid 11 snap -1 2024-09-18T14:02:35.955 INFO:tasks.rados.rados.0.smithi084.stdout:728: expect (ObjNum 178 snap 41 seq_num 178) 2024-09-18T14:02:35.955 INFO:tasks.rados.rados.0.smithi084.stdout:729: write (excl) oid 15 current snap is 71 2024-09-18T14:02:35.955 INFO:tasks.rados.rados.0.smithi084.stdout:726: expect (ObjNum 231 snap 54 seq_num 231) 2024-09-18T14:02:36.131 INFO:tasks.rados.rados.0.smithi084.stdout:728: expect (ObjNum 178 snap 41 seq_num 178) 2024-09-18T14:02:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:729: seq_num 283 ranges {756381=400528,1910757=464751} 2024-09-18T14:02:36.309 INFO:tasks.rados.rados.0.smithi084.stdout:729: writing smithi08434663-15 from 756381 to 1156909 tid 1 2024-09-18T14:02:36.316 INFO:tasks.rados.rados.0.smithi084.stdout:729: writing smithi08434663-15 from 1910757 to 2375508 tid 2 2024-09-18T14:02:36.317 INFO:tasks.rados.rados.0.smithi084.stdout:726: done (2 left) 2024-09-18T14:02:36.317 INFO:tasks.rados.rados.0.smithi084.stdout:728: done (1 left) 2024-09-18T14:02:36.317 INFO:tasks.rados.rados.0.smithi084.stdout:730: read oid 28 snap -1 2024-09-18T14:02:36.317 INFO:tasks.rados.rados.0.smithi084.stdout:730: expect (ObjNum 274 snap 71 seq_num 274) 2024-09-18T14:02:36.317 INFO:tasks.rados.rados.0.smithi084.stdout:731: read oid 43 snap -1 2024-09-18T14:02:36.317 INFO:tasks.rados.rados.0.smithi084.stdout:731: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:02:36.317 INFO:tasks.rados.rados.0.smithi084.stdout:732: rollback oid 46 current snap is 71 2024-09-18T14:02:36.317 INFO:tasks.rados.rados.0.smithi084.stdout:732: done (3 left) 2024-09-18T14:02:36.317 INFO:tasks.rados.rados.0.smithi084.stdout:733: rollback oid 18 current snap is 71 2024-09-18T14:02:36.317 INFO:tasks.rados.rados.0.smithi084.stdout:733: done (3 left) 2024-09-18T14:02:36.317 INFO:tasks.rados.rados.0.smithi084.stdout:734: snap_create 2024-09-18T14:02:36.317 INFO:tasks.rados.rados.0.smithi084.stdout:729: finishing write tid 1 to smithi08434663-15 2024-09-18T14:02:36.319 INFO:tasks.rados.rados.0.smithi084.stdout:729: finishing write tid 2 to smithi08434663-15 2024-09-18T14:02:36.319 INFO:tasks.rados.rados.0.smithi084.stdout:729: finishing write tid 3 to smithi08434663-15 2024-09-18T14:02:36.320 INFO:tasks.rados.rados.0.smithi084.stdout:729: finishing write tid 4 to smithi08434663-15 2024-09-18T14:02:36.320 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 277 (ObjNum 283 snap 71 seq_num 283) dirty exists 2024-09-18T14:02:36.320 INFO:tasks.rados.rados.0.smithi084.stdout:729: left oid 15 (ObjNum 283 snap 71 seq_num 283) 2024-09-18T14:02:36.320 INFO:tasks.rados.rados.0.smithi084.stdout:730: expect (ObjNum 274 snap 71 seq_num 274) 2024-09-18T14:02:36.405 INFO:tasks.rados.rados.0.smithi084.stdout:731: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:02:37.169 INFO:tasks.rados.rados.0.smithi084.stdout:729: done (3 left) 2024-09-18T14:02:37.170 INFO:tasks.rados.rados.0.smithi084.stdout:730: done (2 left) 2024-09-18T14:02:37.170 INFO:tasks.rados.rados.0.smithi084.stdout:731: done (1 left) 2024-09-18T14:02:37.170 INFO:tasks.rados.rados.0.smithi084.stdout:734: done (0 left) 2024-09-18T14:02:37.170 INFO:tasks.rados.rados.0.smithi084.stdout:735: snap_remove snap 71 2024-09-18T14:02:38.175 INFO:tasks.rados.rados.0.smithi084.stdout:735: done (0 left) 2024-09-18T14:02:38.176 INFO:tasks.rados.rados.0.smithi084.stdout:736: copy_from oid 27 from oid 8 current snap is 72 2024-09-18T14:02:38.176 INFO:tasks.rados.rados.0.smithi084.stdout:737: snap_create 2024-09-18T14:02:38.188 INFO:tasks.rados.rados.0.smithi084.stdout:736: finishing copy_from racing read to smithi08434663-27 2024-09-18T14:02:38.192 INFO:tasks.rados.rados.0.smithi084.stdout:736: finishing copy_from to smithi08434663-27 2024-09-18T14:02:38.192 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 79 (ObjNum 197 snap 46 seq_num 197) dirty exists 2024-09-18T14:02:39.177 INFO:tasks.rados.rados.0.smithi084.stdout:736: done (1 left) 2024-09-18T14:02:39.177 INFO:tasks.rados.rados.0.smithi084.stdout:737: done (0 left) 2024-09-18T14:02:39.177 INFO:tasks.rados.rados.0.smithi084.stdout:738: delete oid 9 current snap is 73 2024-09-18T14:02:39.180 INFO:tasks.rados.rados.0.smithi084.stdout:738: done (0 left) 2024-09-18T14:02:39.180 INFO:tasks.rados.rados.0.smithi084.stdout:739: copy_from oid 41 from oid 2 current snap is 73 2024-09-18T14:02:39.180 INFO:tasks.rados.rados.0.smithi084.stdout:740: snap_remove snap 72 2024-09-18T14:02:40.191 INFO:tasks.rados.rados.0.smithi084.stdout:740: done (1 left) 2024-09-18T14:02:40.191 INFO:tasks.rados.rados.0.smithi084.stdout:741: snap_create 2024-09-18T14:02:40.191 INFO:tasks.rados.rados.0.smithi084.stdout:739: finishing copy_from to smithi08434663-41 2024-09-18T14:02:40.191 INFO:tasks.rados.rados.0.smithi084.stdout:739: got expected ENOENT (src dne) 2024-09-18T14:02:40.192 INFO:tasks.rados.rados.0.smithi084.stdout:739: finishing copy_from racing read to smithi08434663-41 2024-09-18T14:02:41.189 INFO:tasks.rados.rados.0.smithi084.stdout:739: done (1 left) 2024-09-18T14:02:41.190 INFO:tasks.rados.rados.0.smithi084.stdout:741: done (0 left) 2024-09-18T14:02:41.190 INFO:tasks.rados.rados.0.smithi084.stdout:742: delete oid 18 current snap is 74 2024-09-18T14:02:41.202 INFO:tasks.rados.rados.0.smithi084.stdout:742: done (0 left) 2024-09-18T14:02:41.202 INFO:tasks.rados.rados.0.smithi084.stdout:743: snap_remove snap 73 2024-09-18T14:02:42.200 INFO:tasks.rados.rados.0.smithi084.stdout:743: done (0 left) 2024-09-18T14:02:42.200 INFO:tasks.rados.rados.0.smithi084.stdout:744: write (excl) oid 34 current snap is 74 2024-09-18T14:02:42.200 INFO:tasks.rados.rados.0.smithi084.stdout:744: seq_num 286 ranges {406802=626275,1686203=728431} 2024-09-18T14:02:42.205 INFO:tasks.rados.rados.0.smithi084.stdout:744: writing smithi08434663-34 from 406802 to 1033077 tid 1 2024-09-18T14:02:42.214 INFO:tasks.rados.rados.0.smithi084.stdout:744: writing smithi08434663-34 from 1686203 to 2414634 tid 2 2024-09-18T14:02:42.214 INFO:tasks.rados.rados.0.smithi084.stdout:745: read oid 32 snap -1 2024-09-18T14:02:42.214 INFO:tasks.rados.rados.0.smithi084.stdout:745: expect deleted 2024-09-18T14:02:42.214 INFO:tasks.rados.rados.0.smithi084.stdout:746: write (excl) oid 7 current snap is 74 2024-09-18T14:02:42.214 INFO:tasks.rados.rados.0.smithi084.stdout:744: finishing write tid 1 to smithi08434663-34 2024-09-18T14:02:42.214 INFO:tasks.rados.rados.0.smithi084.stdout:746: seq_num 287 ranges {423609=701524,1628332=472089,2648987=174622} 2024-09-18T14:02:42.220 INFO:tasks.rados.rados.0.smithi084.stdout:746: writing smithi08434663-7 from 423609 to 1125133 tid 1 2024-09-18T14:02:42.225 INFO:tasks.rados.rados.0.smithi084.stdout:746: writing smithi08434663-7 from 1628332 to 2100421 tid 2 2024-09-18T14:02:42.229 INFO:tasks.rados.rados.0.smithi084.stdout:746: writing smithi08434663-7 from 2648987 to 2823609 tid 3 2024-09-18T14:02:42.229 INFO:tasks.rados.rados.0.smithi084.stdout:747: rollback oid 20 current snap is 74 2024-09-18T14:02:42.229 INFO:tasks.rados.rados.0.smithi084.stdout:747: done (3 left) 2024-09-18T14:02:42.230 INFO:tasks.rados.rados.0.smithi084.stdout:748: write (excl) oid 22 current snap is 74 2024-09-18T14:02:42.230 INFO:tasks.rados.rados.0.smithi084.stdout:748: seq_num 288 ranges {440416=776773,1686814=732100} 2024-09-18T14:02:42.235 INFO:tasks.rados.rados.0.smithi084.stdout:748: writing smithi08434663-22 from 440416 to 1217189 tid 1 2024-09-18T14:02:42.243 INFO:tasks.rados.rados.0.smithi084.stdout:748: writing smithi08434663-22 from 1686814 to 2418914 tid 2 2024-09-18T14:02:42.243 INFO:tasks.rados.rados.0.smithi084.stdout:749: snap_create 2024-09-18T14:02:42.243 INFO:tasks.rados.rados.0.smithi084.stdout:744: finishing write tid 2 to smithi08434663-34 2024-09-18T14:02:42.243 INFO:tasks.rados.rados.0.smithi084.stdout:744: finishing write tid 3 to smithi08434663-34 2024-09-18T14:02:42.243 INFO:tasks.rados.rados.0.smithi084.stdout:744: finishing write tid 4 to smithi08434663-34 2024-09-18T14:02:42.243 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 204 (ObjNum 286 snap 74 seq_num 286) dirty exists 2024-09-18T14:02:42.243 INFO:tasks.rados.rados.0.smithi084.stdout:744: left oid 34 (ObjNum 286 snap 74 seq_num 286) 2024-09-18T14:02:42.243 INFO:tasks.rados.rados.0.smithi084.stdout:746: finishing write tid 1 to smithi08434663-7 2024-09-18T14:02:42.243 INFO:tasks.rados.rados.0.smithi084.stdout:746: finishing write tid 2 to smithi08434663-7 2024-09-18T14:02:42.243 INFO:tasks.rados.rados.0.smithi084.stdout:746: finishing write tid 3 to smithi08434663-7 2024-09-18T14:02:42.244 INFO:tasks.rados.rados.0.smithi084.stdout:746: finishing write tid 4 to smithi08434663-7 2024-09-18T14:02:42.244 INFO:tasks.rados.rados.0.smithi084.stdout:746: finishing write tid 5 to smithi08434663-7 2024-09-18T14:02:42.244 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 187 (ObjNum 287 snap 74 seq_num 287) dirty exists 2024-09-18T14:02:42.244 INFO:tasks.rados.rados.0.smithi084.stdout:746: left oid 7 (ObjNum 287 snap 74 seq_num 287) 2024-09-18T14:02:42.244 INFO:tasks.rados.rados.0.smithi084.stdout:748: finishing write tid 1 to smithi08434663-22 2024-09-18T14:02:42.248 INFO:tasks.rados.rados.0.smithi084.stdout:748: finishing write tid 2 to smithi08434663-22 2024-09-18T14:02:42.248 INFO:tasks.rados.rados.0.smithi084.stdout:748: finishing write tid 3 to smithi08434663-22 2024-09-18T14:02:42.249 INFO:tasks.rados.rados.0.smithi084.stdout:748: finishing write tid 4 to smithi08434663-22 2024-09-18T14:02:42.249 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 155 (ObjNum 288 snap 74 seq_num 288) dirty exists 2024-09-18T14:02:42.249 INFO:tasks.rados.rados.0.smithi084.stdout:748: left oid 22 (ObjNum 288 snap 74 seq_num 288) 2024-09-18T14:02:43.201 INFO:tasks.rados.rados.0.smithi084.stdout:744: done (4 left) 2024-09-18T14:02:43.202 INFO:tasks.rados.rados.0.smithi084.stdout:745: done (3 left) 2024-09-18T14:02:43.202 INFO:tasks.rados.rados.0.smithi084.stdout:746: done (2 left) 2024-09-18T14:02:43.202 INFO:tasks.rados.rados.0.smithi084.stdout:748: done (1 left) 2024-09-18T14:02:43.202 INFO:tasks.rados.rados.0.smithi084.stdout:749: done (0 left) 2024-09-18T14:02:43.202 INFO:tasks.rados.rados.0.smithi084.stdout:750: snap_remove snap 74 2024-09-18T14:02:44.216 INFO:tasks.rados.rados.0.smithi084.stdout:750: done (0 left) 2024-09-18T14:02:44.216 INFO:tasks.rados.rados.0.smithi084.stdout:751: read oid 45 snap -1 2024-09-18T14:02:44.216 INFO:tasks.rados.rados.0.smithi084.stdout:751: expect deleted 2024-09-18T14:02:44.217 INFO:tasks.rados.rados.0.smithi084.stdout:752: snap_create 2024-09-18T14:02:45.226 INFO:tasks.rados.rados.0.smithi084.stdout:751: done (1 left) 2024-09-18T14:02:45.226 INFO:tasks.rados.rados.0.smithi084.stdout:752: done (0 left) 2024-09-18T14:02:45.227 INFO:tasks.rados.rados.0.smithi084.stdout:753: read oid 40 snap -1 2024-09-18T14:02:45.227 INFO:tasks.rados.rados.0.smithi084.stdout:753: expect (ObjNum 247 snap 57 seq_num 247) 2024-09-18T14:02:45.227 INFO:tasks.rados.rados.0.smithi084.stdout:754: read oid 14 snap 75 2024-09-18T14:02:45.227 INFO:tasks.rados.rados.0.smithi084.stdout:754: expect (ObjNum 147 snap 30 seq_num 147) 2024-09-18T14:02:45.227 INFO:tasks.rados.rados.0.smithi084.stdout:755: delete oid 35 current snap is 76 2024-09-18T14:02:45.228 INFO:tasks.rados.rados.0.smithi084.stdout:755: done (2 left) 2024-09-18T14:02:45.228 INFO:tasks.rados.rados.0.smithi084.stdout:756: rollback oid 10 current snap is 76 2024-09-18T14:02:45.228 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 10 to 75 2024-09-18T14:02:45.228 INFO:tasks.rados.rados.0.smithi084.stdout:757: write (excl) oid 47 current snap is 76 2024-09-18T14:02:45.228 INFO:tasks.rados.rados.0.smithi084.stdout:757: seq_num 290 ranges {474030=643624,1520131=735769} 2024-09-18T14:02:45.237 INFO:tasks.rados.rados.0.smithi084.stdout:757: writing smithi08434663-47 from 474030 to 1117654 tid 1 2024-09-18T14:02:45.241 INFO:tasks.rados.rados.0.smithi084.stdout:757: writing smithi08434663-47 from 1520131 to 2255900 tid 2 2024-09-18T14:02:45.241 INFO:tasks.rados.rados.0.smithi084.stdout:758: read oid 15 snap -1 2024-09-18T14:02:45.241 INFO:tasks.rados.rados.0.smithi084.stdout:758: expect (ObjNum 283 snap 71 seq_num 283) 2024-09-18T14:02:45.242 INFO:tasks.rados.rados.0.smithi084.stdout:759: read oid 39 snap -1 2024-09-18T14:02:45.242 INFO:tasks.rados.rados.0.smithi084.stdout:759: expect (ObjNum 260 snap 65 seq_num 260) 2024-09-18T14:02:45.242 INFO:tasks.rados.rados.0.smithi084.stdout:760: snap_create 2024-09-18T14:02:45.242 INFO:tasks.rados.rados.0.smithi084.stdout:754: expect (ObjNum 147 snap 30 seq_num 147) 2024-09-18T14:02:45.350 INFO:tasks.rados.rados.0.smithi084.stdout:756: finishing rollback tid 0 to smithi08434663-10 2024-09-18T14:02:45.350 INFO:tasks.rados.rados.0.smithi084.stdout:756: finishing rollback tid 1 to smithi08434663-10 2024-09-18T14:02:45.350 INFO:tasks.rados.rados.0.smithi084.stdout:756: finishing rollback tid 2 to smithi08434663-10 2024-09-18T14:02:45.350 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 229 (ObjNum 231 snap 54 seq_num 231) dirty exists 2024-09-18T14:02:45.350 INFO:tasks.rados.rados.0.smithi084.stdout:753: expect (ObjNum 247 snap 57 seq_num 247) 2024-09-18T14:02:45.498 INFO:tasks.rados.rados.0.smithi084.stdout:757: finishing write tid 1 to smithi08434663-47 2024-09-18T14:02:45.498 INFO:tasks.rados.rados.0.smithi084.stdout:758: expect (ObjNum 283 snap 71 seq_num 283) 2024-09-18T14:02:45.653 INFO:tasks.rados.rados.0.smithi084.stdout:757: finishing write tid 2 to smithi08434663-47 2024-09-18T14:02:45.654 INFO:tasks.rados.rados.0.smithi084.stdout:757: finishing write tid 3 to smithi08434663-47 2024-09-18T14:02:45.654 INFO:tasks.rados.rados.0.smithi084.stdout:757: finishing write tid 4 to smithi08434663-47 2024-09-18T14:02:45.654 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 188 (ObjNum 290 snap 76 seq_num 290) dirty exists 2024-09-18T14:02:45.654 INFO:tasks.rados.rados.0.smithi084.stdout:757: left oid 47 (ObjNum 290 snap 76 seq_num 290) 2024-09-18T14:02:45.654 INFO:tasks.rados.rados.0.smithi084.stdout:759: expect (ObjNum 260 snap 65 seq_num 260) 2024-09-18T14:02:46.229 INFO:tasks.rados.rados.0.smithi084.stdout:753: done (6 left) 2024-09-18T14:02:46.229 INFO:tasks.rados.rados.0.smithi084.stdout:754: done (5 left) 2024-09-18T14:02:46.229 INFO:tasks.rados.rados.0.smithi084.stdout:756: done (4 left) 2024-09-18T14:02:46.229 INFO:tasks.rados.rados.0.smithi084.stdout:757: done (3 left) 2024-09-18T14:02:46.230 INFO:tasks.rados.rados.0.smithi084.stdout:758: done (2 left) 2024-09-18T14:02:46.230 INFO:tasks.rados.rados.0.smithi084.stdout:759: done (1 left) 2024-09-18T14:02:46.230 INFO:tasks.rados.rados.0.smithi084.stdout:760: done (0 left) 2024-09-18T14:02:46.230 INFO:tasks.rados.rados.0.smithi084.stdout:761: rollback oid 2 current snap is 77 2024-09-18T14:02:46.230 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 2 to 75 2024-09-18T14:02:46.230 INFO:tasks.rados.rados.0.smithi084.stdout:762: read oid 32 snap -1 2024-09-18T14:02:46.230 INFO:tasks.rados.rados.0.smithi084.stdout:762: expect deleted 2024-09-18T14:02:46.231 INFO:tasks.rados.rados.0.smithi084.stdout:763: read oid 1 snap -1 2024-09-18T14:02:46.231 INFO:tasks.rados.rados.0.smithi084.stdout:763: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:02:46.231 INFO:tasks.rados.rados.0.smithi084.stdout:764: read oid 16 snap -1 2024-09-18T14:02:46.231 INFO:tasks.rados.rados.0.smithi084.stdout:764: expect (ObjNum 270 snap 68 seq_num 270) 2024-09-18T14:02:46.231 INFO:tasks.rados.rados.0.smithi084.stdout:765: snap_remove snap 76 2024-09-18T14:02:47.180 INFO:tasks.rados.rados.0.smithi084.stdout:765: done (4 left) 2024-09-18T14:02:47.180 INFO:tasks.rados.rados.0.smithi084.stdout:766: write (excl) oid 50 current snap is 77 2024-09-18T14:02:47.180 INFO:tasks.rados.rados.0.smithi084.stdout:766: seq_num 291 ranges {490837=718873,1862260=479427,2875032=15805} 2024-09-18T14:02:47.186 INFO:tasks.rados.rados.0.smithi084.stdout:766: writing smithi08434663-50 from 490837 to 1209710 tid 1 2024-09-18T14:02:47.193 INFO:tasks.rados.rados.0.smithi084.stdout:766: writing smithi08434663-50 from 1862260 to 2341687 tid 2 2024-09-18T14:02:47.196 INFO:tasks.rados.rados.0.smithi084.stdout:766: writing smithi08434663-50 from 2875032 to 2890837 tid 3 2024-09-18T14:02:47.196 INFO:tasks.rados.rados.0.smithi084.stdout:767: snap_create 2024-09-18T14:02:47.197 INFO:tasks.rados.rados.0.smithi084.stdout:761: finishing rollback tid 1 to smithi08434663-2 2024-09-18T14:02:47.197 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 0 (ObjNum 129 snap 0 seq_num 2495412088) dirty dne 2024-09-18T14:02:47.197 INFO:tasks.rados.rados.0.smithi084.stdout:763: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:02:47.273 INFO:tasks.rados.rados.0.smithi084.stdout:764: expect (ObjNum 270 snap 68 seq_num 270) 2024-09-18T14:02:47.350 INFO:tasks.rados.rados.0.smithi084.stdout:766: finishing write tid 1 to smithi08434663-50 2024-09-18T14:02:47.350 INFO:tasks.rados.rados.0.smithi084.stdout:766: finishing write tid 2 to smithi08434663-50 2024-09-18T14:02:47.350 INFO:tasks.rados.rados.0.smithi084.stdout:766: finishing write tid 3 to smithi08434663-50 2024-09-18T14:02:47.350 INFO:tasks.rados.rados.0.smithi084.stdout:766: finishing write tid 4 to smithi08434663-50 2024-09-18T14:02:47.350 INFO:tasks.rados.rados.0.smithi084.stdout:766: finishing write tid 5 to smithi08434663-50 2024-09-18T14:02:47.350 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 312 (ObjNum 291 snap 77 seq_num 291) dirty exists 2024-09-18T14:02:47.350 INFO:tasks.rados.rados.0.smithi084.stdout:766: left oid 50 (ObjNum 291 snap 77 seq_num 291) 2024-09-18T14:02:48.188 INFO:tasks.rados.rados.0.smithi084.stdout:761: done (5 left) 2024-09-18T14:02:48.188 INFO:tasks.rados.rados.0.smithi084.stdout:762: done (4 left) 2024-09-18T14:02:48.188 INFO:tasks.rados.rados.0.smithi084.stdout:763: done (3 left) 2024-09-18T14:02:48.188 INFO:tasks.rados.rados.0.smithi084.stdout:764: done (2 left) 2024-09-18T14:02:48.188 INFO:tasks.rados.rados.0.smithi084.stdout:766: done (1 left) 2024-09-18T14:02:48.189 INFO:tasks.rados.rados.0.smithi084.stdout:767: done (0 left) 2024-09-18T14:02:48.189 INFO:tasks.rados.rados.0.smithi084.stdout:768: rollback oid 16 current snap is 78 2024-09-18T14:02:48.189 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 16 to 75 2024-09-18T14:02:48.189 INFO:tasks.rados.rados.0.smithi084.stdout:769: snap_create 2024-09-18T14:02:48.192 INFO:tasks.rados.rados.0.smithi084.stdout:768: finishing rollback tid 0 to smithi08434663-16 2024-09-18T14:02:48.192 INFO:tasks.rados.rados.0.smithi084.stdout:768: finishing rollback tid 1 to smithi08434663-16 2024-09-18T14:02:48.195 INFO:tasks.rados.rados.0.smithi084.stdout:768: finishing rollback tid 2 to smithi08434663-16 2024-09-18T14:02:48.196 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 286 (ObjNum 270 snap 68 seq_num 270) dirty exists 2024-09-18T14:02:49.194 INFO:tasks.rados.rados.0.smithi084.stdout:768: done (1 left) 2024-09-18T14:02:49.194 INFO:tasks.rados.rados.0.smithi084.stdout:769: done (0 left) 2024-09-18T14:02:49.194 INFO:tasks.rados.rados.0.smithi084.stdout:770: rollback oid 6 current snap is 79 2024-09-18T14:02:49.195 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 77 2024-09-18T14:02:49.195 INFO:tasks.rados.rados.0.smithi084.stdout:771: write (excl) oid 14 current snap is 79 2024-09-18T14:02:49.195 INFO:tasks.rados.rados.0.smithi084.stdout:771: seq_num 292 ranges {507644=794122,1920742=623085} 2024-09-18T14:02:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:771: writing smithi08434663-14 from 507644 to 1301766 tid 1 2024-09-18T14:02:49.209 INFO:tasks.rados.rados.0.smithi084.stdout:771: writing smithi08434663-14 from 1920742 to 2543827 tid 2 2024-09-18T14:02:49.209 INFO:tasks.rados.rados.0.smithi084.stdout:772: snap_create 2024-09-18T14:02:49.209 INFO:tasks.rados.rados.0.smithi084.stdout:770: finishing rollback tid 0 to smithi08434663-6 2024-09-18T14:02:49.209 INFO:tasks.rados.rados.0.smithi084.stdout:770: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:02:49.209 INFO:tasks.rados.rados.0.smithi084.stdout:770: finishing rollback tid 2 to smithi08434663-6 2024-09-18T14:02:49.209 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 190 (ObjNum 271 snap 69 seq_num 271) dirty exists 2024-09-18T14:02:49.211 INFO:tasks.rados.rados.0.smithi084.stdout:771: finishing write tid 1 to smithi08434663-14 2024-09-18T14:02:49.212 INFO:tasks.rados.rados.0.smithi084.stdout:771: finishing write tid 2 to smithi08434663-14 2024-09-18T14:02:49.212 INFO:tasks.rados.rados.0.smithi084.stdout:771: finishing write tid 3 to smithi08434663-14 2024-09-18T14:02:49.212 INFO:tasks.rados.rados.0.smithi084.stdout:771: finishing write tid 4 to smithi08434663-14 2024-09-18T14:02:49.212 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 316 (ObjNum 292 snap 79 seq_num 292) dirty exists 2024-09-18T14:02:49.213 INFO:tasks.rados.rados.0.smithi084.stdout:771: left oid 14 (ObjNum 292 snap 79 seq_num 292) 2024-09-18T14:02:50.205 INFO:tasks.rados.rados.0.smithi084.stdout:770: done (2 left) 2024-09-18T14:02:50.205 INFO:tasks.rados.rados.0.smithi084.stdout:771: done (1 left) 2024-09-18T14:02:50.205 INFO:tasks.rados.rados.0.smithi084.stdout:772: done (0 left) 2024-09-18T14:02:50.205 INFO:tasks.rados.rados.0.smithi084.stdout:773: snap_create 2024-09-18T14:02:51.206 INFO:tasks.rados.rados.0.smithi084.stdout:773: done (0 left) 2024-09-18T14:02:51.206 INFO:tasks.rados.rados.0.smithi084.stdout:774: read oid 14 snap -1 2024-09-18T14:02:51.206 INFO:tasks.rados.rados.0.smithi084.stdout:774: expect (ObjNum 292 snap 79 seq_num 292) 2024-09-18T14:02:51.206 INFO:tasks.rados.rados.0.smithi084.stdout:775: rollback oid 12 current snap is 81 2024-09-18T14:02:51.206 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 12 to 78 2024-09-18T14:02:51.206 INFO:tasks.rados.rados.0.smithi084.stdout:776: snap_create 2024-09-18T14:02:51.213 INFO:tasks.rados.rados.0.smithi084.stdout:774: expect (ObjNum 292 snap 79 seq_num 292) 2024-09-18T14:02:51.337 INFO:tasks.rados.rados.0.smithi084.stdout:775: finishing rollback tid 1 to smithi08434663-12 2024-09-18T14:02:51.337 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 0 (ObjNum 129 snap 0 seq_num 2494481568) dirty dne 2024-09-18T14:02:52.215 INFO:tasks.rados.rados.0.smithi084.stdout:774: done (2 left) 2024-09-18T14:02:52.216 INFO:tasks.rados.rados.0.smithi084.stdout:775: done (1 left) 2024-09-18T14:02:52.216 INFO:tasks.rados.rados.0.smithi084.stdout:776: done (0 left) 2024-09-18T14:02:52.216 INFO:tasks.rados.rados.0.smithi084.stdout:777: rollback oid 11 current snap is 82 2024-09-18T14:02:52.216 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 11 to 78 2024-09-18T14:02:52.216 INFO:tasks.rados.rados.0.smithi084.stdout:778: copy_from oid 24 from oid 42 current snap is 82 2024-09-18T14:02:52.216 INFO:tasks.rados.rados.0.smithi084.stdout:779: snap_remove snap 80 2024-09-18T14:02:53.217 INFO:tasks.rados.rados.0.smithi084.stdout:779: done (2 left) 2024-09-18T14:02:53.217 INFO:tasks.rados.rados.0.smithi084.stdout:780: read oid 5 snap -1 2024-09-18T14:02:53.218 INFO:tasks.rados.rados.0.smithi084.stdout:780: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:02:53.218 INFO:tasks.rados.rados.0.smithi084.stdout:781: delete oid 49 current snap is 82 2024-09-18T14:02:53.218 INFO:tasks.rados.rados.0.smithi084.stdout:777: finishing rollback tid 0 to smithi08434663-11 2024-09-18T14:02:53.218 INFO:tasks.rados.rados.0.smithi084.stdout:777: finishing rollback tid 1 to smithi08434663-11 2024-09-18T14:02:53.218 INFO:tasks.rados.rados.0.smithi084.stdout:777: finishing rollback tid 2 to smithi08434663-11 2024-09-18T14:02:53.218 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 191 (ObjNum 178 snap 41 seq_num 178) dirty exists 2024-09-18T14:02:53.218 INFO:tasks.rados.rados.0.smithi084.stdout:778: finishing copy_from racing read to smithi08434663-24 2024-09-18T14:02:53.218 INFO:tasks.rados.rados.0.smithi084.stdout:778: finishing copy_from to smithi08434663-24 2024-09-18T14:02:53.218 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 183 (ObjNum 267 snap 68 seq_num 267) dirty exists 2024-09-18T14:02:53.221 INFO:tasks.rados.rados.0.smithi084.stdout:780: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:02:53.291 INFO:tasks.rados.rados.0.smithi084.stdout:777: done (3 left) 2024-09-18T14:02:53.291 INFO:tasks.rados.rados.0.smithi084.stdout:778: done (2 left) 2024-09-18T14:02:53.292 INFO:tasks.rados.rados.0.smithi084.stdout:780: done (1 left) 2024-09-18T14:02:53.292 INFO:tasks.rados.rados.0.smithi084.stdout:781: done (0 left) 2024-09-18T14:02:53.292 INFO:tasks.rados.rados.0.smithi084.stdout:782: snap_create 2024-09-18T14:02:54.231 INFO:tasks.rados.rados.0.smithi084.stdout:782: done (0 left) 2024-09-18T14:02:54.232 INFO:tasks.rados.rados.0.smithi084.stdout:783: write oid 42 current snap is 83 2024-09-18T14:02:54.232 INFO:tasks.rados.rados.0.smithi084.stdout:783: seq_num 294 ranges {541258=544620,1637706=626754,2840948=100310} 2024-09-18T14:02:54.237 INFO:tasks.rados.rados.0.smithi084.stdout:783: writing smithi08434663-42 from 541258 to 1085878 tid 1 2024-09-18T14:02:54.243 INFO:tasks.rados.rados.0.smithi084.stdout:783: writing smithi08434663-42 from 1637706 to 2264460 tid 2 2024-09-18T14:02:54.248 INFO:tasks.rados.rados.0.smithi084.stdout:783: writing smithi08434663-42 from 2840948 to 2941258 tid 3 2024-09-18T14:02:54.248 INFO:tasks.rados.rados.0.smithi084.stdout:784: read oid 46 snap 77 2024-09-18T14:02:54.248 INFO:tasks.rados.rados.0.smithi084.stdout:784: expect (ObjNum 278 snap 71 seq_num 278) 2024-09-18T14:02:54.248 INFO:tasks.rados.rados.0.smithi084.stdout:785: write oid 12 current snap is 83 2024-09-18T14:02:54.248 INFO:tasks.rados.rados.0.smithi084.stdout:783: finishing write tid 1 to smithi08434663-42 2024-09-18T14:02:54.248 INFO:tasks.rados.rados.0.smithi084.stdout:783: finishing write tid 2 to smithi08434663-42 2024-09-18T14:02:54.248 INFO:tasks.rados.rados.0.smithi084.stdout:785: seq_num 295 ranges {558065=619869,1579835=486765,2468371=489694} 2024-09-18T14:02:54.254 INFO:tasks.rados.rados.0.smithi084.stdout:785: writing smithi08434663-12 from 558065 to 1177934 tid 1 2024-09-18T14:02:54.260 INFO:tasks.rados.rados.0.smithi084.stdout:785: writing smithi08434663-12 from 1579835 to 2066600 tid 2 2024-09-18T14:02:54.265 INFO:tasks.rados.rados.0.smithi084.stdout:785: writing smithi08434663-12 from 2468371 to 2958065 tid 3 2024-09-18T14:02:54.265 INFO:tasks.rados.rados.0.smithi084.stdout:783: finishing write tid 3 to smithi08434663-42 2024-09-18T14:02:54.265 INFO:tasks.rados.rados.0.smithi084.stdout:783: finishing write tid 4 to smithi08434663-42 2024-09-18T14:02:54.265 INFO:tasks.rados.rados.0.smithi084.stdout:783: finishing write tid 5 to smithi08434663-42 2024-09-18T14:02:54.266 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 209 (ObjNum 294 snap 83 seq_num 294) dirty exists 2024-09-18T14:02:54.266 INFO:tasks.rados.rados.0.smithi084.stdout:783: left oid 42 (ObjNum 294 snap 83 seq_num 294) 2024-09-18T14:02:54.266 INFO:tasks.rados.rados.0.smithi084.stdout:786: read oid 13 snap -1 2024-09-18T14:02:54.266 INFO:tasks.rados.rados.0.smithi084.stdout:786: expect deleted 2024-09-18T14:02:54.266 INFO:tasks.rados.rados.0.smithi084.stdout:783: done (3 left) 2024-09-18T14:02:54.266 INFO:tasks.rados.rados.0.smithi084.stdout:787: snap_create 2024-09-18T14:02:54.266 INFO:tasks.rados.rados.0.smithi084.stdout:784: expect (ObjNum 278 snap 71 seq_num 278) 2024-09-18T14:02:54.350 INFO:tasks.rados.rados.0.smithi084.stdout:785: finishing write tid 1 to smithi08434663-12 2024-09-18T14:02:54.350 INFO:tasks.rados.rados.0.smithi084.stdout:785: finishing write tid 2 to smithi08434663-12 2024-09-18T14:02:54.350 INFO:tasks.rados.rados.0.smithi084.stdout:785: finishing write tid 3 to smithi08434663-12 2024-09-18T14:02:54.350 INFO:tasks.rados.rados.0.smithi084.stdout:785: finishing write tid 4 to smithi08434663-12 2024-09-18T14:02:54.350 INFO:tasks.rados.rados.0.smithi084.stdout:785: finishing write tid 5 to smithi08434663-12 2024-09-18T14:02:54.350 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 320 (ObjNum 295 snap 83 seq_num 295) dirty exists 2024-09-18T14:02:54.351 INFO:tasks.rados.rados.0.smithi084.stdout:785: left oid 12 (ObjNum 295 snap 83 seq_num 295) 2024-09-18T14:02:55.231 INFO:tasks.rados.rados.0.smithi084.stdout:784: done (3 left) 2024-09-18T14:02:55.231 INFO:tasks.rados.rados.0.smithi084.stdout:785: done (2 left) 2024-09-18T14:02:55.231 INFO:tasks.rados.rados.0.smithi084.stdout:786: done (1 left) 2024-09-18T14:02:55.231 INFO:tasks.rados.rados.0.smithi084.stdout:787: done (0 left) 2024-09-18T14:02:55.231 INFO:tasks.rados.rados.0.smithi084.stdout:788: read oid 50 snap -1 2024-09-18T14:02:55.231 INFO:tasks.rados.rados.0.smithi084.stdout:788: expect (ObjNum 291 snap 77 seq_num 291) 2024-09-18T14:02:55.231 INFO:tasks.rados.rados.0.smithi084.stdout:789: snap_create 2024-09-18T14:02:55.236 INFO:tasks.rados.rados.0.smithi084.stdout:788: expect (ObjNum 291 snap 77 seq_num 291) 2024-09-18T14:02:56.233 INFO:tasks.rados.rados.0.smithi084.stdout:788: done (1 left) 2024-09-18T14:02:56.233 INFO:tasks.rados.rados.0.smithi084.stdout:789: done (0 left) 2024-09-18T14:02:56.233 INFO:tasks.rados.rados.0.smithi084.stdout:790: write (excl) oid 15 current snap is 85 2024-09-18T14:02:56.233 INFO:tasks.rados.rados.0.smithi084.stdout:790: seq_num 296 ranges {574872=695118,2038317=630423} 2024-09-18T14:02:56.240 INFO:tasks.rados.rados.0.smithi084.stdout:790: writing smithi08434663-15 from 574872 to 1269990 tid 1 2024-09-18T14:02:56.248 INFO:tasks.rados.rados.0.smithi084.stdout:790: writing smithi08434663-15 from 2038317 to 2668740 tid 2 2024-09-18T14:02:56.248 INFO:tasks.rados.rados.0.smithi084.stdout:791: snap_create 2024-09-18T14:02:56.248 INFO:tasks.rados.rados.0.smithi084.stdout:790: finishing write tid 1 to smithi08434663-15 2024-09-18T14:02:56.254 INFO:tasks.rados.rados.0.smithi084.stdout:790: finishing write tid 2 to smithi08434663-15 2024-09-18T14:02:56.254 INFO:tasks.rados.rados.0.smithi084.stdout:790: finishing write tid 3 to smithi08434663-15 2024-09-18T14:02:56.254 INFO:tasks.rados.rados.0.smithi084.stdout:790: finishing write tid 4 to smithi08434663-15 2024-09-18T14:02:56.254 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 290 (ObjNum 296 snap 85 seq_num 296) dirty exists 2024-09-18T14:02:56.254 INFO:tasks.rados.rados.0.smithi084.stdout:790: left oid 15 (ObjNum 296 snap 85 seq_num 296) 2024-09-18T14:02:57.173 INFO:tasks.rados.rados.0.smithi084.stdout:790: done (1 left) 2024-09-18T14:02:57.174 INFO:tasks.rados.rados.0.smithi084.stdout:791: done (0 left) 2024-09-18T14:02:57.174 INFO:tasks.rados.rados.0.smithi084.stdout:792: read oid 3 snap -1 2024-09-18T14:02:57.174 INFO:tasks.rados.rados.0.smithi084.stdout:792: expect (ObjNum 226 snap 53 seq_num 226) 2024-09-18T14:02:57.174 INFO:tasks.rados.rados.0.smithi084.stdout:793: snap_create 2024-09-18T14:02:57.193 INFO:tasks.rados.rados.0.smithi084.stdout:792: expect (ObjNum 226 snap 53 seq_num 226) 2024-09-18T14:02:58.180 INFO:tasks.rados.rados.0.smithi084.stdout:792: done (1 left) 2024-09-18T14:02:58.181 INFO:tasks.rados.rados.0.smithi084.stdout:793: done (0 left) 2024-09-18T14:02:58.181 INFO:tasks.rados.rados.0.smithi084.stdout:794: snap_remove snap 83 2024-09-18T14:02:59.185 INFO:tasks.rados.rados.0.smithi084.stdout:794: done (0 left) 2024-09-18T14:02:59.185 INFO:tasks.rados.rados.0.smithi084.stdout:795: write oid 6 current snap is 87 2024-09-18T14:02:59.185 INFO:tasks.rados.rados.0.smithi084.stdout:795: seq_num 297 ranges {591679=486720,1813152=490434} 2024-09-18T14:02:59.191 INFO:tasks.rados.rados.0.smithi084.stdout:795: writing smithi08434663-6 from 591679 to 1078399 tid 1 2024-09-18T14:02:59.198 INFO:tasks.rados.rados.0.smithi084.stdout:795: writing smithi08434663-6 from 1813152 to 2303586 tid 2 2024-09-18T14:02:59.198 INFO:tasks.rados.rados.0.smithi084.stdout:796: delete oid 16 current snap is 87 2024-09-18T14:02:59.198 INFO:tasks.rados.rados.0.smithi084.stdout:795: finishing write tid 1 to smithi08434663-6 2024-09-18T14:02:59.201 INFO:tasks.rados.rados.0.smithi084.stdout:796: done (1 left) 2024-09-18T14:02:59.201 INFO:tasks.rados.rados.0.smithi084.stdout:797: snap_create 2024-09-18T14:02:59.201 INFO:tasks.rados.rados.0.smithi084.stdout:795: finishing write tid 2 to smithi08434663-6 2024-09-18T14:02:59.202 INFO:tasks.rados.rados.0.smithi084.stdout:795: finishing write tid 3 to smithi08434663-6 2024-09-18T14:02:59.202 INFO:tasks.rados.rados.0.smithi084.stdout:795: finishing write tid 4 to smithi08434663-6 2024-09-18T14:02:59.202 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 193 (ObjNum 297 snap 87 seq_num 297) dirty exists 2024-09-18T14:02:59.202 INFO:tasks.rados.rados.0.smithi084.stdout:795: left oid 6 (ObjNum 297 snap 87 seq_num 297) 2024-09-18T14:03:00.198 INFO:tasks.rados.rados.0.smithi084.stdout:795: done (1 left) 2024-09-18T14:03:00.198 INFO:tasks.rados.rados.0.smithi084.stdout:797: done (0 left) 2024-09-18T14:03:00.199 INFO:tasks.rados.rados.0.smithi084.stdout:798: read oid 50 snap 79 2024-09-18T14:03:00.199 INFO:tasks.rados.rados.0.smithi084.stdout:798: expect (ObjNum 291 snap 77 seq_num 291) 2024-09-18T14:03:00.199 INFO:tasks.rados.rados.0.smithi084.stdout:799: read oid 42 snap -1 2024-09-18T14:03:00.199 INFO:tasks.rados.rados.0.smithi084.stdout:799: expect (ObjNum 294 snap 83 seq_num 294) 2024-09-18T14:03:00.199 INFO:tasks.rados.rados.0.smithi084.stdout:800: read oid 23 snap -1 2024-09-18T14:03:00.199 INFO:tasks.rados.rados.0.smithi084.stdout:800: expect deleted 2024-09-18T14:03:00.199 INFO:tasks.rados.rados.0.smithi084.stdout:801: snap_create 2024-09-18T14:03:00.205 INFO:tasks.rados.rados.0.smithi084.stdout:798: expect (ObjNum 291 snap 77 seq_num 291) 2024-09-18T14:03:00.334 INFO:tasks.rados.rados.0.smithi084.stdout:799: expect (ObjNum 294 snap 83 seq_num 294) 2024-09-18T14:03:01.204 INFO:tasks.rados.rados.0.smithi084.stdout:798: done (3 left) 2024-09-18T14:03:01.205 INFO:tasks.rados.rados.0.smithi084.stdout:799: done (2 left) 2024-09-18T14:03:01.205 INFO:tasks.rados.rados.0.smithi084.stdout:800: done (1 left) 2024-09-18T14:03:01.205 INFO:tasks.rados.rados.0.smithi084.stdout:801: done (0 left) 2024-09-18T14:03:01.205 INFO:tasks.rados.rados.0.smithi084.stdout:802: write oid 36 current snap is 89 2024-09-18T14:03:01.205 INFO:tasks.rados.rados.0.smithi084.stdout:802: seq_num 299 ranges {625293=637218,1813763=494103,2978063=47230} 2024-09-18T14:03:01.211 INFO:tasks.rados.rados.0.smithi084.stdout:802: writing smithi08434663-36 from 625293 to 1262511 tid 1 2024-09-18T14:03:01.217 INFO:tasks.rados.rados.0.smithi084.stdout:802: writing smithi08434663-36 from 1813763 to 2307866 tid 2 2024-09-18T14:03:01.221 INFO:tasks.rados.rados.0.smithi084.stdout:802: writing smithi08434663-36 from 2978063 to 3025293 tid 3 2024-09-18T14:03:01.221 INFO:tasks.rados.rados.0.smithi084.stdout:803: snap_create 2024-09-18T14:03:01.221 INFO:tasks.rados.rados.0.smithi084.stdout:802: finishing write tid 1 to smithi08434663-36 2024-09-18T14:03:01.221 INFO:tasks.rados.rados.0.smithi084.stdout:802: finishing write tid 2 to smithi08434663-36 2024-09-18T14:03:01.223 INFO:tasks.rados.rados.0.smithi084.stdout:802: finishing write tid 3 to smithi08434663-36 2024-09-18T14:03:01.223 INFO:tasks.rados.rados.0.smithi084.stdout:802: finishing write tid 4 to smithi08434663-36 2024-09-18T14:03:01.224 INFO:tasks.rados.rados.0.smithi084.stdout:802: finishing write tid 5 to smithi08434663-36 2024-09-18T14:03:01.224 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 215 (ObjNum 299 snap 89 seq_num 299) dirty exists 2024-09-18T14:03:01.224 INFO:tasks.rados.rados.0.smithi084.stdout:802: left oid 36 (ObjNum 299 snap 89 seq_num 299) 2024-09-18T14:03:02.217 INFO:tasks.rados.rados.0.smithi084.stdout:802: done (1 left) 2024-09-18T14:03:02.217 INFO:tasks.rados.rados.0.smithi084.stdout:803: done (0 left) 2024-09-18T14:03:02.217 INFO:tasks.rados.rados.0.smithi084.stdout:804: write oid 44 current snap is 90 2024-09-18T14:03:02.217 INFO:tasks.rados.rados.0.smithi084.stdout:804: seq_num 300 ranges {642100=712467,1872245=637761} 2024-09-18T14:03:02.224 INFO:tasks.rados.rados.0.smithi084.stdout:804: writing smithi08434663-44 from 642100 to 1354567 tid 1 2024-09-18T14:03:02.231 INFO:tasks.rados.rados.0.smithi084.stdout:804: writing smithi08434663-44 from 1872245 to 2510006 tid 2 2024-09-18T14:03:02.231 INFO:tasks.rados.rados.0.smithi084.stdout:805: copy_from oid 28 from oid 17 current snap is 90 2024-09-18T14:03:02.231 INFO:tasks.rados.rados.0.smithi084.stdout:804: finishing write tid 1 to smithi08434663-44 2024-09-18T14:03:02.231 INFO:tasks.rados.rados.0.smithi084.stdout:806: snap_create 2024-09-18T14:03:02.233 INFO:tasks.rados.rados.0.smithi084.stdout:805: finishing copy_from to smithi08434663-28 2024-09-18T14:03:02.234 INFO:tasks.rados.rados.0.smithi084.stdout:805: got expected ENOENT (src dne) 2024-09-18T14:03:02.234 INFO:tasks.rados.rados.0.smithi084.stdout:805: finishing copy_from racing read to smithi08434663-28 2024-09-18T14:03:02.234 INFO:tasks.rados.rados.0.smithi084.stdout:804: finishing write tid 2 to smithi08434663-44 2024-09-18T14:03:02.236 INFO:tasks.rados.rados.0.smithi084.stdout:804: finishing write tid 3 to smithi08434663-44 2024-09-18T14:03:02.236 INFO:tasks.rados.rados.0.smithi084.stdout:804: finishing write tid 4 to smithi08434663-44 2024-09-18T14:03:02.236 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 137 (ObjNum 300 snap 90 seq_num 300) dirty exists 2024-09-18T14:03:02.236 INFO:tasks.rados.rados.0.smithi084.stdout:804: left oid 44 (ObjNum 300 snap 90 seq_num 300) 2024-09-18T14:03:03.215 INFO:tasks.rados.rados.0.smithi084.stdout:804: done (2 left) 2024-09-18T14:03:03.216 INFO:tasks.rados.rados.0.smithi084.stdout:805: done (1 left) 2024-09-18T14:03:03.216 INFO:tasks.rados.rados.0.smithi084.stdout:806: done (0 left) 2024-09-18T14:03:03.216 INFO:tasks.rados.rados.0.smithi084.stdout:807: copy_from oid 42 from oid 14 current snap is 91 2024-09-18T14:03:03.216 INFO:tasks.rados.rados.0.smithi084.stdout:808: read oid 30 snap -1 2024-09-18T14:03:03.216 INFO:tasks.rados.rados.0.smithi084.stdout:808: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:03:03.216 INFO:tasks.rados.rados.0.smithi084.stdout:809: rollback oid 35 current snap is 91 2024-09-18T14:03:03.216 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 35 to 75 2024-09-18T14:03:03.216 INFO:tasks.rados.rados.0.smithi084.stdout:810: read oid 41 snap 82 2024-09-18T14:03:03.216 INFO:tasks.rados.rados.0.smithi084.stdout:810: expect (ObjNum 152 snap 31 seq_num 152) 2024-09-18T14:03:03.216 INFO:tasks.rados.rados.0.smithi084.stdout:811: write oid 21 current snap is 91 2024-09-18T14:03:03.216 INFO:tasks.rados.rados.0.smithi084.stdout:811: seq_num 301 ranges {658907=787716,1930727=497772,3032909=25998} 2024-09-18T14:03:03.224 INFO:tasks.rados.rados.0.smithi084.stdout:811: writing smithi08434663-21 from 658907 to 1446623 tid 1 2024-09-18T14:03:03.230 INFO:tasks.rados.rados.0.smithi084.stdout:811: writing smithi08434663-21 from 1930727 to 2428499 tid 2 2024-09-18T14:03:03.234 INFO:tasks.rados.rados.0.smithi084.stdout:811: writing smithi08434663-21 from 3032909 to 3058907 tid 3 2024-09-18T14:03:03.234 INFO:tasks.rados.rados.0.smithi084.stdout:812: rollback oid 48 current snap is 91 2024-09-18T14:03:03.234 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 48 to 90 2024-09-18T14:03:03.234 INFO:tasks.rados.rados.0.smithi084.stdout:809: finishing rollback tid 1 to smithi08434663-35 2024-09-18T14:03:03.234 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 0 (ObjNum 49 snap 0 seq_num 4) dirty dne 2024-09-18T14:03:03.234 INFO:tasks.rados.rados.0.smithi084.stdout:808: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:03:03.303 INFO:tasks.rados.rados.0.smithi084.stdout:810: expect (ObjNum 152 snap 31 seq_num 152) 2024-09-18T14:03:03.421 INFO:tasks.rados.rados.0.smithi084.stdout:811: finishing write tid 1 to smithi08434663-21 2024-09-18T14:03:03.421 INFO:tasks.rados.rados.0.smithi084.stdout:807: finishing copy_from racing read to smithi08434663-42 2024-09-18T14:03:03.421 INFO:tasks.rados.rados.0.smithi084.stdout:808: done (5 left) 2024-09-18T14:03:03.421 INFO:tasks.rados.rados.0.smithi084.stdout:809: done (4 left) 2024-09-18T14:03:03.421 INFO:tasks.rados.rados.0.smithi084.stdout:810: done (3 left) 2024-09-18T14:03:03.421 INFO:tasks.rados.rados.0.smithi084.stdout:813: copy_from oid 40 from oid 23 current snap is 91 2024-09-18T14:03:03.421 INFO:tasks.rados.rados.0.smithi084.stdout:807: finishing copy_from to smithi08434663-42 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 316 (ObjNum 292 snap 79 seq_num 292) dirty exists 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:811: finishing write tid 2 to smithi08434663-21 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:811: finishing write tid 3 to smithi08434663-21 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:811: finishing write tid 4 to smithi08434663-21 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:811: finishing write tid 5 to smithi08434663-21 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 172 (ObjNum 301 snap 91 seq_num 301) dirty exists 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:811: left oid 21 (ObjNum 301 snap 91 seq_num 301) 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:812: finishing rollback tid 0 to smithi08434663-48 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:812: finishing rollback tid 1 to smithi08434663-48 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:812: finishing rollback tid 2 to smithi08434663-48 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 176 (ObjNum 221 snap 52 seq_num 221) dirty exists 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:807: done (3 left) 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:811: done (2 left) 2024-09-18T14:03:03.422 INFO:tasks.rados.rados.0.smithi084.stdout:812: done (1 left) 2024-09-18T14:03:03.423 INFO:tasks.rados.rados.0.smithi084.stdout:814: delete oid 22 current snap is 91 2024-09-18T14:03:03.423 INFO:tasks.rados.rados.0.smithi084.stdout:813: finishing copy_from to smithi08434663-40 2024-09-18T14:03:03.423 INFO:tasks.rados.rados.0.smithi084.stdout:813: got expected ENOENT (src dne) 2024-09-18T14:03:03.423 INFO:tasks.rados.rados.0.smithi084.stdout:813: finishing copy_from racing read to smithi08434663-40 2024-09-18T14:03:03.423 INFO:tasks.rados.rados.0.smithi084.stdout:813: done (1 left) 2024-09-18T14:03:03.424 INFO:tasks.rados.rados.0.smithi084.stdout:814: done (0 left) 2024-09-18T14:03:03.424 INFO:tasks.rados.rados.0.smithi084.stdout:815: copy_from oid 3 from oid 42 current snap is 91 2024-09-18T14:03:03.424 INFO:tasks.rados.rados.0.smithi084.stdout:816: snap_create 2024-09-18T14:03:03.434 INFO:tasks.rados.rados.0.smithi084.stdout:815: finishing copy_from racing read to smithi08434663-3 2024-09-18T14:03:03.435 INFO:tasks.rados.rados.0.smithi084.stdout:815: finishing copy_from to smithi08434663-3 2024-09-18T14:03:03.435 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 316 (ObjNum 292 snap 79 seq_num 292) dirty exists 2024-09-18T14:03:04.224 INFO:tasks.rados.rados.0.smithi084.stdout:815: done (1 left) 2024-09-18T14:03:04.224 INFO:tasks.rados.rados.0.smithi084.stdout:816: done (0 left) 2024-09-18T14:03:04.224 INFO:tasks.rados.rados.0.smithi084.stdout:817: read oid 2 snap -1 2024-09-18T14:03:04.224 INFO:tasks.rados.rados.0.smithi084.stdout:817: expect deleted 2024-09-18T14:03:04.224 INFO:tasks.rados.rados.0.smithi084.stdout:818: snap_create 2024-09-18T14:03:05.230 INFO:tasks.rados.rados.0.smithi084.stdout:817: done (1 left) 2024-09-18T14:03:05.230 INFO:tasks.rados.rados.0.smithi084.stdout:818: done (0 left) 2024-09-18T14:03:05.230 INFO:tasks.rados.rados.0.smithi084.stdout:819: snap_remove snap 92 2024-09-18T14:03:06.245 INFO:tasks.rados.rados.0.smithi084.stdout:819: done (0 left) 2024-09-18T14:03:06.245 INFO:tasks.rados.rados.0.smithi084.stdout:820: rollback oid 17 current snap is 93 2024-09-18T14:03:06.245 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 17 to 85 2024-09-18T14:03:06.245 INFO:tasks.rados.rados.0.smithi084.stdout:821: snap_remove snap 91 2024-09-18T14:03:07.177 INFO:tasks.rados.rados.0.smithi084.stdout:821: done (1 left) 2024-09-18T14:03:07.177 INFO:tasks.rados.rados.0.smithi084.stdout:822: rollback oid 2 current snap is 93 2024-09-18T14:03:07.177 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 2 to 84 2024-09-18T14:03:07.177 INFO:tasks.rados.rados.0.smithi084.stdout:820: finishing rollback tid 1 to smithi08434663-17 2024-09-18T14:03:07.177 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 0 (ObjNum 257 snap 0 seq_num 2495494960) dirty dne 2024-09-18T14:03:07.177 INFO:tasks.rados.rados.0.smithi084.stdout:820: done (1 left) 2024-09-18T14:03:07.177 INFO:tasks.rados.rados.0.smithi084.stdout:823: write (excl) oid 13 current snap is 93 2024-09-18T14:03:07.177 INFO:tasks.rados.rados.0.smithi084.stdout:823: seq_num 303 ranges {692521=538214,1931338=785088} 2024-09-18T14:03:07.184 INFO:tasks.rados.rados.0.smithi084.stdout:823: writing smithi08434663-13 from 692521 to 1230735 tid 1 2024-09-18T14:03:07.193 INFO:tasks.rados.rados.0.smithi084.stdout:823: writing smithi08434663-13 from 1931338 to 2716426 tid 2 2024-09-18T14:03:07.193 INFO:tasks.rados.rados.0.smithi084.stdout:824: rollback oid 39 current snap is 93 2024-09-18T14:03:07.193 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 39 to 75 2024-09-18T14:03:07.193 INFO:tasks.rados.rados.0.smithi084.stdout:822: finishing rollback tid 1 to smithi08434663-2 2024-09-18T14:03:07.193 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 0 (ObjNum 129 snap 0 seq_num 2495499600) dirty dne 2024-09-18T14:03:07.193 INFO:tasks.rados.rados.0.smithi084.stdout:823: finishing write tid 1 to smithi08434663-13 2024-09-18T14:03:07.193 INFO:tasks.rados.rados.0.smithi084.stdout:822: done (2 left) 2024-09-18T14:03:07.193 INFO:tasks.rados.rados.0.smithi084.stdout:825: write oid 35 current snap is 93 2024-09-18T14:03:07.193 INFO:tasks.rados.rados.0.smithi084.stdout:825: seq_num 304 ranges {709328=613463,1989820=645099} 2024-09-18T14:03:07.200 INFO:tasks.rados.rados.0.smithi084.stdout:825: writing smithi08434663-35 from 709328 to 1322791 tid 1 2024-09-18T14:03:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:825: writing smithi08434663-35 from 1989820 to 2634919 tid 2 2024-09-18T14:03:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:826: write oid 32 current snap is 93 2024-09-18T14:03:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:824: finishing rollback tid 0 to smithi08434663-39 2024-09-18T14:03:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:823: finishing write tid 2 to smithi08434663-13 2024-09-18T14:03:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:823: finishing write tid 3 to smithi08434663-13 2024-09-18T14:03:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:823: finishing write tid 4 to smithi08434663-13 2024-09-18T14:03:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 269 (ObjNum 303 snap 93 seq_num 303) dirty exists 2024-09-18T14:03:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:823: left oid 13 (ObjNum 303 snap 93 seq_num 303) 2024-09-18T14:03:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:824: finishing rollback tid 1 to smithi08434663-39 2024-09-18T14:03:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:824: finishing rollback tid 2 to smithi08434663-39 2024-09-18T14:03:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 194 (ObjNum 260 snap 65 seq_num 260) dirty exists 2024-09-18T14:03:07.209 INFO:tasks.rados.rados.0.smithi084.stdout:825: finishing write tid 1 to smithi08434663-35 2024-09-18T14:03:07.209 INFO:tasks.rados.rados.0.smithi084.stdout:826: seq_num 305 ranges {726135=405065,1764655=788757,3026248=99887} 2024-09-18T14:03:07.213 INFO:tasks.rados.rados.0.smithi084.stdout:826: writing smithi08434663-32 from 726135 to 1131200 tid 1 2024-09-18T14:03:07.221 INFO:tasks.rados.rados.0.smithi084.stdout:826: writing smithi08434663-32 from 1764655 to 2553412 tid 2 2024-09-18T14:03:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:826: writing smithi08434663-32 from 3026248 to 3126135 tid 3 2024-09-18T14:03:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:823: done (3 left) 2024-09-18T14:03:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:824: done (2 left) 2024-09-18T14:03:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:827: read oid 42 snap -1 2024-09-18T14:03:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:827: expect (ObjNum 292 snap 79 seq_num 292) 2024-09-18T14:03:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:828: write (excl) oid 39 current snap is 93 2024-09-18T14:03:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:825: finishing write tid 2 to smithi08434663-35 2024-09-18T14:03:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:825: finishing write tid 3 to smithi08434663-35 2024-09-18T14:03:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:825: finishing write tid 4 to smithi08434663-35 2024-09-18T14:03:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 297 (ObjNum 304 snap 93 seq_num 304) dirty exists 2024-09-18T14:03:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:825: left oid 35 (ObjNum 304 snap 93 seq_num 304) 2024-09-18T14:03:07.225 INFO:tasks.rados.rados.0.smithi084.stdout:826: finishing write tid 1 to smithi08434663-32 2024-09-18T14:03:07.226 INFO:tasks.rados.rados.0.smithi084.stdout:828: seq_num 306 ranges {742942=480314,1823137=648768} 2024-09-18T14:03:07.231 INFO:tasks.rados.rados.0.smithi084.stdout:828: writing smithi08434663-39 from 742942 to 1223256 tid 1 2024-09-18T14:03:07.238 INFO:tasks.rados.rados.0.smithi084.stdout:828: writing smithi08434663-39 from 1823137 to 2471905 tid 2 2024-09-18T14:03:07.238 INFO:tasks.rados.rados.0.smithi084.stdout:825: done (3 left) 2024-09-18T14:03:07.238 INFO:tasks.rados.rados.0.smithi084.stdout:829: snap_create 2024-09-18T14:03:07.238 INFO:tasks.rados.rados.0.smithi084.stdout:826: finishing write tid 2 to smithi08434663-32 2024-09-18T14:03:07.239 INFO:tasks.rados.rados.0.smithi084.stdout:826: finishing write tid 3 to smithi08434663-32 2024-09-18T14:03:07.239 INFO:tasks.rados.rados.0.smithi084.stdout:826: finishing write tid 4 to smithi08434663-32 2024-09-18T14:03:07.239 INFO:tasks.rados.rados.0.smithi084.stdout:826: finishing write tid 5 to smithi08434663-32 2024-09-18T14:03:07.239 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 325 (ObjNum 305 snap 93 seq_num 305) dirty exists 2024-09-18T14:03:07.239 INFO:tasks.rados.rados.0.smithi084.stdout:826: left oid 32 (ObjNum 305 snap 93 seq_num 305) 2024-09-18T14:03:07.239 INFO:tasks.rados.rados.0.smithi084.stdout:827: expect (ObjNum 292 snap 79 seq_num 292) 2024-09-18T14:03:07.363 INFO:tasks.rados.rados.0.smithi084.stdout:828: finishing write tid 1 to smithi08434663-39 2024-09-18T14:03:07.363 INFO:tasks.rados.rados.0.smithi084.stdout:828: finishing write tid 2 to smithi08434663-39 2024-09-18T14:03:07.364 INFO:tasks.rados.rados.0.smithi084.stdout:828: finishing write tid 3 to smithi08434663-39 2024-09-18T14:03:07.364 INFO:tasks.rados.rados.0.smithi084.stdout:828: finishing write tid 4 to smithi08434663-39 2024-09-18T14:03:07.364 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 197 (ObjNum 306 snap 93 seq_num 306) dirty exists 2024-09-18T14:03:07.364 INFO:tasks.rados.rados.0.smithi084.stdout:828: left oid 39 (ObjNum 306 snap 93 seq_num 306) 2024-09-18T14:03:08.239 INFO:tasks.rados.rados.0.smithi084.stdout:826: done (3 left) 2024-09-18T14:03:08.239 INFO:tasks.rados.rados.0.smithi084.stdout:827: done (2 left) 2024-09-18T14:03:08.240 INFO:tasks.rados.rados.0.smithi084.stdout:828: done (1 left) 2024-09-18T14:03:08.240 INFO:tasks.rados.rados.0.smithi084.stdout:829: done (0 left) 2024-09-18T14:03:08.240 INFO:tasks.rados.rados.0.smithi084.stdout:830: snap_create 2024-09-18T14:03:09.242 INFO:tasks.rados.rados.0.smithi084.stdout:830: done (0 left) 2024-09-18T14:03:09.242 INFO:tasks.rados.rados.0.smithi084.stdout:831: snap_create 2024-09-18T14:03:10.251 INFO:tasks.rados.rados.0.smithi084.stdout:831: done (0 left) 2024-09-18T14:03:10.251 INFO:tasks.rados.rados.0.smithi084.stdout:832: snap_create 2024-09-18T14:03:11.255 INFO:tasks.rados.rados.0.smithi084.stdout:832: done (0 left) 2024-09-18T14:03:11.255 INFO:tasks.rados.rados.0.smithi084.stdout:833: read oid 3 snap -1 2024-09-18T14:03:11.255 INFO:tasks.rados.rados.0.smithi084.stdout:833: expect (ObjNum 292 snap 79 seq_num 292) 2024-09-18T14:03:11.255 INFO:tasks.rados.rados.0.smithi084.stdout:834: snap_remove snap 79 2024-09-18T14:03:12.268 INFO:tasks.rados.rados.0.smithi084.stdout:834: done (1 left) 2024-09-18T14:03:12.268 INFO:tasks.rados.rados.0.smithi084.stdout:835: snap_remove snap 93 2024-09-18T14:03:13.274 INFO:tasks.rados.rados.0.smithi084.stdout:835: done (1 left) 2024-09-18T14:03:13.274 INFO:tasks.rados.rados.0.smithi084.stdout:836: rollback oid 10 current snap is 97 2024-09-18T14:03:13.274 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 10 to 87 2024-09-18T14:03:13.274 INFO:tasks.rados.rados.0.smithi084.stdout:833: expect (ObjNum 292 snap 79 seq_num 292) 2024-09-18T14:03:13.400 INFO:tasks.rados.rados.0.smithi084.stdout:836: finishing rollback tid 0 to smithi08434663-10 2024-09-18T14:03:13.400 INFO:tasks.rados.rados.0.smithi084.stdout:836: finishing rollback tid 1 to smithi08434663-10 2024-09-18T14:03:13.400 INFO:tasks.rados.rados.0.smithi084.stdout:836: finishing rollback tid 2 to smithi08434663-10 2024-09-18T14:03:13.400 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 233 (ObjNum 231 snap 54 seq_num 231) dirty exists 2024-09-18T14:03:13.400 INFO:tasks.rados.rados.0.smithi084.stdout:833: done (1 left) 2024-09-18T14:03:13.400 INFO:tasks.rados.rados.0.smithi084.stdout:836: done (0 left) 2024-09-18T14:03:13.401 INFO:tasks.rados.rados.0.smithi084.stdout:837: rollback oid 4 current snap is 97 2024-09-18T14:03:13.401 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 4 to 82 2024-09-18T14:03:13.401 INFO:tasks.rados.rados.0.smithi084.stdout:838: write oid 22 current snap is 97 2024-09-18T14:03:13.401 INFO:tasks.rados.rados.0.smithi084.stdout:838: seq_num 307 ranges {759749=555563,1765266=792426,2964741=195008} 2024-09-18T14:03:13.406 INFO:tasks.rados.rados.0.smithi084.stdout:838: writing smithi08434663-22 from 759749 to 1315312 tid 1 2024-09-18T14:03:13.413 INFO:tasks.rados.rados.0.smithi084.stdout:838: writing smithi08434663-22 from 1765266 to 2557692 tid 2 2024-09-18T14:03:13.417 INFO:tasks.rados.rados.0.smithi084.stdout:838: writing smithi08434663-22 from 2964741 to 3159749 tid 3 2024-09-18T14:03:13.417 INFO:tasks.rados.rados.0.smithi084.stdout:839: copy_from oid 1 from oid 33 current snap is 97 2024-09-18T14:03:13.417 INFO:tasks.rados.rados.0.smithi084.stdout:837: finishing rollback tid 0 to smithi08434663-4 2024-09-18T14:03:13.417 INFO:tasks.rados.rados.0.smithi084.stdout:837: finishing rollback tid 1 to smithi08434663-4 2024-09-18T14:03:13.417 INFO:tasks.rados.rados.0.smithi084.stdout:837: finishing rollback tid 2 to smithi08434663-4 2024-09-18T14:03:13.418 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 319 (ObjNum 219 snap 50 seq_num 219) dirty exists 2024-09-18T14:03:13.418 INFO:tasks.rados.rados.0.smithi084.stdout:837: done (2 left) 2024-09-18T14:03:13.418 INFO:tasks.rados.rados.0.smithi084.stdout:840: delete oid 36 current snap is 97 2024-09-18T14:03:13.418 INFO:tasks.rados.rados.0.smithi084.stdout:838: finishing write tid 1 to smithi08434663-22 2024-09-18T14:03:13.418 INFO:tasks.rados.rados.0.smithi084.stdout:838: finishing write tid 2 to smithi08434663-22 2024-09-18T14:03:13.419 INFO:tasks.rados.rados.0.smithi084.stdout:838: finishing write tid 3 to smithi08434663-22 2024-09-18T14:03:13.419 INFO:tasks.rados.rados.0.smithi084.stdout:838: finishing write tid 4 to smithi08434663-22 2024-09-18T14:03:13.420 INFO:tasks.rados.rados.0.smithi084.stdout:838: finishing write tid 5 to smithi08434663-22 2024-09-18T14:03:13.420 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 320 (ObjNum 307 snap 97 seq_num 307) dirty exists 2024-09-18T14:03:13.420 INFO:tasks.rados.rados.0.smithi084.stdout:838: left oid 22 (ObjNum 307 snap 97 seq_num 307) 2024-09-18T14:03:13.423 INFO:tasks.rados.rados.0.smithi084.stdout:838: done (2 left) 2024-09-18T14:03:13.423 INFO:tasks.rados.rados.0.smithi084.stdout:840: done (1 left) 2024-09-18T14:03:13.423 INFO:tasks.rados.rados.0.smithi084.stdout:841: snap_remove snap 86 2024-09-18T14:03:14.285 INFO:tasks.rados.rados.0.smithi084.stdout:841: done (1 left) 2024-09-18T14:03:14.285 INFO:tasks.rados.rados.0.smithi084.stdout:842: snap_remove snap 75 2024-09-18T14:03:15.280 INFO:tasks.rados.rados.0.smithi084.stdout:842: done (1 left) 2024-09-18T14:03:15.281 INFO:tasks.rados.rados.0.smithi084.stdout:843: rollback oid 43 current snap is 97 2024-09-18T14:03:15.281 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 78 2024-09-18T14:03:15.281 INFO:tasks.rados.rados.0.smithi084.stdout:844: snap_remove snap 94 2024-09-18T14:03:16.291 INFO:tasks.rados.rados.0.smithi084.stdout:844: done (2 left) 2024-09-18T14:03:16.291 INFO:tasks.rados.rados.0.smithi084.stdout:845: snap_remove snap 88 2024-09-18T14:03:17.179 INFO:tasks.rados.rados.0.smithi084.stdout:845: done (2 left) 2024-09-18T14:03:17.179 INFO:tasks.rados.rados.0.smithi084.stdout:846: delete oid 13 current snap is 97 2024-09-18T14:03:17.179 INFO:tasks.rados.rados.0.smithi084.stdout:839: finishing copy_from racing read to smithi08434663-1 2024-09-18T14:03:17.179 INFO:tasks.rados.rados.0.smithi084.stdout:839: finishing copy_from to smithi08434663-1 2024-09-18T14:03:17.179 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 196 (ObjNum 266 snap 68 seq_num 266) dirty exists 2024-09-18T14:03:17.179 INFO:tasks.rados.rados.0.smithi084.stdout:843: finishing rollback tid 0 to smithi08434663-43 2024-09-18T14:03:17.179 INFO:tasks.rados.rados.0.smithi084.stdout:843: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:03:17.179 INFO:tasks.rados.rados.0.smithi084.stdout:843: finishing rollback tid 2 to smithi08434663-43 2024-09-18T14:03:17.179 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 89 (ObjNum 265 snap 67 seq_num 265) dirty exists 2024-09-18T14:03:17.190 INFO:tasks.rados.rados.0.smithi084.stdout:839: done (2 left) 2024-09-18T14:03:17.190 INFO:tasks.rados.rados.0.smithi084.stdout:843: done (1 left) 2024-09-18T14:03:17.190 INFO:tasks.rados.rados.0.smithi084.stdout:846: done (0 left) 2024-09-18T14:03:17.190 INFO:tasks.rados.rados.0.smithi084.stdout:847: copy_from oid 17 from oid 27 current snap is 97 2024-09-18T14:03:17.190 INFO:tasks.rados.rados.0.smithi084.stdout:848: copy_from oid 34 from oid 37 current snap is 97 2024-09-18T14:03:17.190 INFO:tasks.rados.rados.0.smithi084.stdout:849: rollback oid 47 current snap is 97 2024-09-18T14:03:17.190 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 47 to 84 2024-09-18T14:03:17.190 INFO:tasks.rados.rados.0.smithi084.stdout:850: read oid 40 snap -1 2024-09-18T14:03:17.190 INFO:tasks.rados.rados.0.smithi084.stdout:850: expect (ObjNum 247 snap 57 seq_num 247) 2024-09-18T14:03:17.190 INFO:tasks.rados.rados.0.smithi084.stdout:851: delete oid 38 current snap is 97 2024-09-18T14:03:17.195 INFO:tasks.rados.rados.0.smithi084.stdout:849: finishing rollback tid 0 to smithi08434663-47 2024-09-18T14:03:17.196 INFO:tasks.rados.rados.0.smithi084.stdout:850: expect (ObjNum 247 snap 57 seq_num 247) 2024-09-18T14:03:17.347 INFO:tasks.rados.rados.0.smithi084.stdout:849: finishing rollback tid 1 to smithi08434663-47 2024-09-18T14:03:17.347 INFO:tasks.rados.rados.0.smithi084.stdout:849: finishing rollback tid 2 to smithi08434663-47 2024-09-18T14:03:17.347 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 200 (ObjNum 290 snap 76 seq_num 290) dirty exists 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:848: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:848: finishing copy_from to smithi08434663-34 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 107 (ObjNum 138 snap 29 seq_num 138) dirty exists 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:847: finishing copy_from racing read to smithi08434663-17 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:848: done (4 left) 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:849: done (3 left) 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:850: done (2 left) 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:851: done (1 left) 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:852: read oid 15 snap 81 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:852: expect (ObjNum 283 snap 71 seq_num 283) 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:853: delete oid 14 current snap is 97 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:847: finishing copy_from to smithi08434663-17 2024-09-18T14:03:17.348 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 79 (ObjNum 197 snap 46 seq_num 197) dirty exists 2024-09-18T14:03:17.350 INFO:tasks.rados.rados.0.smithi084.stdout:847: done (2 left) 2024-09-18T14:03:17.350 INFO:tasks.rados.rados.0.smithi084.stdout:853: done (1 left) 2024-09-18T14:03:17.350 INFO:tasks.rados.rados.0.smithi084.stdout:854: read oid 19 snap -1 2024-09-18T14:03:17.350 INFO:tasks.rados.rados.0.smithi084.stdout:854: expect (ObjNum 247 snap 57 seq_num 247) 2024-09-18T14:03:17.350 INFO:tasks.rados.rados.0.smithi084.stdout:855: read oid 14 snap -1 2024-09-18T14:03:17.350 INFO:tasks.rados.rados.0.smithi084.stdout:855: expect deleted 2024-09-18T14:03:17.350 INFO:tasks.rados.rados.0.smithi084.stdout:856: copy_from oid 8 from oid 24 current snap is 97 2024-09-18T14:03:17.351 INFO:tasks.rados.rados.0.smithi084.stdout:857: delete oid 42 current snap is 97 2024-09-18T14:03:17.351 INFO:tasks.rados.rados.0.smithi084.stdout:852: expect (ObjNum 283 snap 71 seq_num 283) 2024-09-18T14:03:17.508 INFO:tasks.rados.rados.0.smithi084.stdout:854: expect (ObjNum 247 snap 57 seq_num 247) 2024-09-18T14:03:17.658 INFO:tasks.rados.rados.0.smithi084.stdout:856: finishing copy_from racing read to smithi08434663-8 2024-09-18T14:03:17.658 INFO:tasks.rados.rados.0.smithi084.stdout:856: finishing copy_from to smithi08434663-8 2024-09-18T14:03:17.658 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 183 (ObjNum 267 snap 68 seq_num 267) dirty exists 2024-09-18T14:03:17.658 INFO:tasks.rados.rados.0.smithi084.stdout:852: done (4 left) 2024-09-18T14:03:17.658 INFO:tasks.rados.rados.0.smithi084.stdout:854: done (3 left) 2024-09-18T14:03:17.659 INFO:tasks.rados.rados.0.smithi084.stdout:855: done (2 left) 2024-09-18T14:03:17.659 INFO:tasks.rados.rados.0.smithi084.stdout:856: done (1 left) 2024-09-18T14:03:17.659 INFO:tasks.rados.rados.0.smithi084.stdout:857: done (0 left) 2024-09-18T14:03:17.659 INFO:tasks.rados.rados.0.smithi084.stdout:858: read oid 22 snap -1 2024-09-18T14:03:17.659 INFO:tasks.rados.rados.0.smithi084.stdout:858: expect (ObjNum 307 snap 97 seq_num 307) 2024-09-18T14:03:17.659 INFO:tasks.rados.rados.0.smithi084.stdout:859: read oid 12 snap 90 2024-09-18T14:03:17.659 INFO:tasks.rados.rados.0.smithi084.stdout:859: expect (ObjNum 295 snap 83 seq_num 295) 2024-09-18T14:03:17.659 INFO:tasks.rados.rados.0.smithi084.stdout:860: delete oid 8 current snap is 97 2024-09-18T14:03:17.659 INFO:tasks.rados.rados.0.smithi084.stdout:860: done (2 left) 2024-09-18T14:03:17.660 INFO:tasks.rados.rados.0.smithi084.stdout:861: snap_remove snap 96 2024-09-18T14:03:18.300 INFO:tasks.rados.rados.0.smithi084.stdout:861: done (2 left) 2024-09-18T14:03:18.300 INFO:tasks.rados.rados.0.smithi084.stdout:862: delete oid 15 current snap is 97 2024-09-18T14:03:18.301 INFO:tasks.rados.rados.0.smithi084.stdout:859: expect (ObjNum 295 snap 83 seq_num 295) 2024-09-18T14:03:18.395 INFO:tasks.rados.rados.0.smithi084.stdout:858: expect (ObjNum 307 snap 97 seq_num 307) 2024-09-18T14:03:18.495 INFO:tasks.rados.rados.0.smithi084.stdout:858: done (2 left) 2024-09-18T14:03:18.495 INFO:tasks.rados.rados.0.smithi084.stdout:859: done (1 left) 2024-09-18T14:03:18.495 INFO:tasks.rados.rados.0.smithi084.stdout:862: done (0 left) 2024-09-18T14:03:18.495 INFO:tasks.rados.rados.0.smithi084.stdout:863: delete oid 23 current snap is 97 2024-09-18T14:03:18.497 INFO:tasks.rados.rados.0.smithi084.stdout:863: done (0 left) 2024-09-18T14:03:18.497 INFO:tasks.rados.rados.0.smithi084.stdout:864: rollback oid 37 current snap is 97 2024-09-18T14:03:18.497 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 37 to 81 2024-09-18T14:03:18.497 INFO:tasks.rados.rados.0.smithi084.stdout:865: delete oid 45 current snap is 97 2024-09-18T14:03:18.498 INFO:tasks.rados.rados.0.smithi084.stdout:865: done (1 left) 2024-09-18T14:03:18.498 INFO:tasks.rados.rados.0.smithi084.stdout:866: write oid 44 current snap is 97 2024-09-18T14:03:18.498 INFO:tasks.rados.rados.0.smithi084.stdout:866: seq_num 317 ranges {527819=624406,1833733=410771,2722618=601581} 2024-09-18T14:03:18.504 INFO:tasks.rados.rados.0.smithi084.stdout:866: writing smithi08434663-44 from 527819 to 1152225 tid 1 2024-09-18T14:03:18.510 INFO:tasks.rados.rados.0.smithi084.stdout:866: writing smithi08434663-44 from 1833733 to 2244504 tid 2 2024-09-18T14:03:18.516 INFO:tasks.rados.rados.0.smithi084.stdout:866: writing smithi08434663-44 from 2722618 to 3324199 tid 3 2024-09-18T14:03:18.516 INFO:tasks.rados.rados.0.smithi084.stdout:867: snap_remove snap 82 2024-09-18T14:03:19.302 INFO:tasks.rados.rados.0.smithi084.stdout:867: done (2 left) 2024-09-18T14:03:19.302 INFO:tasks.rados.rados.0.smithi084.stdout:868: delete oid 3 current snap is 97 2024-09-18T14:03:19.302 INFO:tasks.rados.rados.0.smithi084.stdout:864: finishing rollback tid 0 to smithi08434663-37 2024-09-18T14:03:19.302 INFO:tasks.rados.rados.0.smithi084.stdout:864: finishing rollback tid 1 to smithi08434663-37 2024-09-18T14:03:19.302 INFO:tasks.rados.rados.0.smithi084.stdout:864: finishing rollback tid 2 to smithi08434663-37 2024-09-18T14:03:19.302 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 225 (ObjNum 138 snap 29 seq_num 138) dirty exists 2024-09-18T14:03:19.302 INFO:tasks.rados.rados.0.smithi084.stdout:866: finishing write tid 1 to smithi08434663-44 2024-09-18T14:03:19.302 INFO:tasks.rados.rados.0.smithi084.stdout:866: finishing write tid 2 to smithi08434663-44 2024-09-18T14:03:19.302 INFO:tasks.rados.rados.0.smithi084.stdout:866: finishing write tid 3 to smithi08434663-44 2024-09-18T14:03:19.302 INFO:tasks.rados.rados.0.smithi084.stdout:866: finishing write tid 4 to smithi08434663-44 2024-09-18T14:03:19.302 INFO:tasks.rados.rados.0.smithi084.stdout:866: finishing write tid 5 to smithi08434663-44 2024-09-18T14:03:19.303 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 141 (ObjNum 317 snap 97 seq_num 317) dirty exists 2024-09-18T14:03:19.303 INFO:tasks.rados.rados.0.smithi084.stdout:866: left oid 44 (ObjNum 317 snap 97 seq_num 317) 2024-09-18T14:03:19.310 INFO:tasks.rados.rados.0.smithi084.stdout:864: done (2 left) 2024-09-18T14:03:19.310 INFO:tasks.rados.rados.0.smithi084.stdout:866: done (1 left) 2024-09-18T14:03:19.310 INFO:tasks.rados.rados.0.smithi084.stdout:868: done (0 left) 2024-09-18T14:03:19.311 INFO:tasks.rados.rados.0.smithi084.stdout:869: write oid 6 current snap is 97 2024-09-18T14:03:19.311 INFO:tasks.rados.rados.0.smithi084.stdout:869: seq_num 319 ranges {561433=774904,1950697=414440,2777464=583969} 2024-09-18T14:03:19.317 INFO:tasks.rados.rados.0.smithi084.stdout:869: writing smithi08434663-6 from 561433 to 1336337 tid 1 2024-09-18T14:03:19.323 INFO:tasks.rados.rados.0.smithi084.stdout:869: writing smithi08434663-6 from 1950697 to 2365137 tid 2 2024-09-18T14:03:19.329 INFO:tasks.rados.rados.0.smithi084.stdout:869: writing smithi08434663-6 from 2777464 to 3361433 tid 3 2024-09-18T14:03:19.329 INFO:tasks.rados.rados.0.smithi084.stdout:870: write (excl) oid 35 current snap is 97 2024-09-18T14:03:19.329 INFO:tasks.rados.rados.0.smithi084.stdout:870: seq_num 320 ranges {578240=566506,1609179=558098,2804887=551750} 2024-09-18T14:03:19.335 INFO:tasks.rados.rados.0.smithi084.stdout:870: writing smithi08434663-35 from 578240 to 1144746 tid 1 2024-09-18T14:03:19.341 INFO:tasks.rados.rados.0.smithi084.stdout:870: writing smithi08434663-35 from 1609179 to 2167277 tid 2 2024-09-18T14:03:19.348 INFO:tasks.rados.rados.0.smithi084.stdout:870: writing smithi08434663-35 from 2804887 to 3356637 tid 3 2024-09-18T14:03:19.348 INFO:tasks.rados.rados.0.smithi084.stdout:871: read oid 43 snap -1 2024-09-18T14:03:19.348 INFO:tasks.rados.rados.0.smithi084.stdout:871: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:03:19.348 INFO:tasks.rados.rados.0.smithi084.stdout:872: delete oid 5 current snap is 97 2024-09-18T14:03:19.348 INFO:tasks.rados.rados.0.smithi084.stdout:869: finishing write tid 1 to smithi08434663-6 2024-09-18T14:03:19.348 INFO:tasks.rados.rados.0.smithi084.stdout:869: finishing write tid 2 to smithi08434663-6 2024-09-18T14:03:19.348 INFO:tasks.rados.rados.0.smithi084.stdout:869: finishing write tid 3 to smithi08434663-6 2024-09-18T14:03:19.348 INFO:tasks.rados.rados.0.smithi084.stdout:869: finishing write tid 4 to smithi08434663-6 2024-09-18T14:03:19.348 INFO:tasks.rados.rados.0.smithi084.stdout:869: finishing write tid 5 to smithi08434663-6 2024-09-18T14:03:19.348 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 197 (ObjNum 319 snap 97 seq_num 319) dirty exists 2024-09-18T14:03:19.349 INFO:tasks.rados.rados.0.smithi084.stdout:869: left oid 6 (ObjNum 319 snap 97 seq_num 319) 2024-09-18T14:03:19.349 INFO:tasks.rados.rados.0.smithi084.stdout:870: finishing write tid 1 to smithi08434663-35 2024-09-18T14:03:19.349 INFO:tasks.rados.rados.0.smithi084.stdout:870: finishing write tid 2 to smithi08434663-35 2024-09-18T14:03:19.350 INFO:tasks.rados.rados.0.smithi084.stdout:869: done (3 left) 2024-09-18T14:03:19.350 INFO:tasks.rados.rados.0.smithi084.stdout:872: done (2 left) 2024-09-18T14:03:19.350 INFO:tasks.rados.rados.0.smithi084.stdout:873: snap_create 2024-09-18T14:03:19.351 INFO:tasks.rados.rados.0.smithi084.stdout:870: finishing write tid 3 to smithi08434663-35 2024-09-18T14:03:19.351 INFO:tasks.rados.rados.0.smithi084.stdout:870: finishing write tid 4 to smithi08434663-35 2024-09-18T14:03:19.351 INFO:tasks.rados.rados.0.smithi084.stdout:870: finishing write tid 5 to smithi08434663-35 2024-09-18T14:03:19.351 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 318 (ObjNum 320 snap 97 seq_num 320) dirty exists 2024-09-18T14:03:19.351 INFO:tasks.rados.rados.0.smithi084.stdout:870: left oid 35 (ObjNum 320 snap 97 seq_num 320) 2024-09-18T14:03:19.355 INFO:tasks.rados.rados.0.smithi084.stdout:871: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:03:20.310 INFO:tasks.rados.rados.0.smithi084.stdout:870: done (2 left) 2024-09-18T14:03:20.310 INFO:tasks.rados.rados.0.smithi084.stdout:871: done (1 left) 2024-09-18T14:03:20.310 INFO:tasks.rados.rados.0.smithi084.stdout:873: done (0 left) 2024-09-18T14:03:20.310 INFO:tasks.rados.rados.0.smithi084.stdout:874: delete oid 38 current snap is 98 2024-09-18T14:03:20.313 INFO:tasks.rados.rados.0.smithi084.stdout:874: done (0 left) 2024-09-18T14:03:20.313 INFO:tasks.rados.rados.0.smithi084.stdout:875: write (excl) oid 50 current snap is 98 2024-09-18T14:03:20.313 INFO:tasks.rados.rados.0.smithi084.stdout:875: seq_num 323 ranges {628661=792253,2184625=421778,3403509=25152} 2024-09-18T14:03:20.320 INFO:tasks.rados.rados.0.smithi084.stdout:875: writing smithi08434663-50 from 628661 to 1420914 tid 1 2024-09-18T14:03:20.327 INFO:tasks.rados.rados.0.smithi084.stdout:875: writing smithi08434663-50 from 2184625 to 2606403 tid 2 2024-09-18T14:03:20.332 INFO:tasks.rados.rados.0.smithi084.stdout:875: writing smithi08434663-50 from 3403509 to 3428661 tid 3 2024-09-18T14:03:20.332 INFO:tasks.rados.rados.0.smithi084.stdout:876: read oid 21 snap -1 2024-09-18T14:03:20.332 INFO:tasks.rados.rados.0.smithi084.stdout:876: expect (ObjNum 301 snap 91 seq_num 301) 2024-09-18T14:03:20.332 INFO:tasks.rados.rados.0.smithi084.stdout:877: snap_create 2024-09-18T14:03:20.332 INFO:tasks.rados.rados.0.smithi084.stdout:875: finishing write tid 1 to smithi08434663-50 2024-09-18T14:03:20.332 INFO:tasks.rados.rados.0.smithi084.stdout:875: finishing write tid 2 to smithi08434663-50 2024-09-18T14:03:20.333 INFO:tasks.rados.rados.0.smithi084.stdout:875: finishing write tid 3 to smithi08434663-50 2024-09-18T14:03:20.334 INFO:tasks.rados.rados.0.smithi084.stdout:875: finishing write tid 4 to smithi08434663-50 2024-09-18T14:03:20.334 INFO:tasks.rados.rados.0.smithi084.stdout:875: finishing write tid 5 to smithi08434663-50 2024-09-18T14:03:20.334 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 340 (ObjNum 323 snap 98 seq_num 323) dirty exists 2024-09-18T14:03:20.334 INFO:tasks.rados.rados.0.smithi084.stdout:875: left oid 50 (ObjNum 323 snap 98 seq_num 323) 2024-09-18T14:03:20.338 INFO:tasks.rados.rados.0.smithi084.stdout:876: expect (ObjNum 301 snap 91 seq_num 301) 2024-09-18T14:03:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:875: done (2 left) 2024-09-18T14:03:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:876: done (1 left) 2024-09-18T14:03:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:877: done (0 left) 2024-09-18T14:03:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:878: snap_remove snap 95 2024-09-18T14:03:22.187 INFO:tasks.rados.rados.0.smithi084.stdout:878: done (0 left) 2024-09-18T14:03:22.187 INFO:tasks.rados.rados.0.smithi084.stdout:879: write (excl) oid 11 current snap is 99 2024-09-18T14:03:22.187 INFO:tasks.rados.rados.0.smithi084.stdout:879: seq_num 324 ranges {645468=467502,1726754=565436,2798226=596838} 2024-09-18T14:03:22.193 INFO:tasks.rados.rados.0.smithi084.stdout:879: writing smithi08434663-11 from 645468 to 1112970 tid 1 2024-09-18T14:03:22.200 INFO:tasks.rados.rados.0.smithi084.stdout:879: writing smithi08434663-11 from 1726754 to 2292190 tid 2 2024-09-18T14:03:22.209 INFO:tasks.rados.rados.0.smithi084.stdout:879: writing smithi08434663-11 from 2798226 to 3395064 tid 3 2024-09-18T14:03:22.209 INFO:tasks.rados.rados.0.smithi084.stdout:880: copy_from oid 38 from oid 46 current snap is 99 2024-09-18T14:03:22.209 INFO:tasks.rados.rados.0.smithi084.stdout:879: finishing write tid 1 to smithi08434663-11 2024-09-18T14:03:22.209 INFO:tasks.rados.rados.0.smithi084.stdout:879: finishing write tid 2 to smithi08434663-11 2024-09-18T14:03:22.210 INFO:tasks.rados.rados.0.smithi084.stdout:881: read oid 44 snap -1 2024-09-18T14:03:22.210 INFO:tasks.rados.rados.0.smithi084.stdout:881: expect (ObjNum 317 snap 97 seq_num 317) 2024-09-18T14:03:22.210 INFO:tasks.rados.rados.0.smithi084.stdout:882: snap_remove snap 84 2024-09-18T14:03:23.182 INFO:tasks.rados.rados.0.smithi084.stdout:882: done (3 left) 2024-09-18T14:03:23.182 INFO:tasks.rados.rados.0.smithi084.stdout:883: write oid 35 current snap is 99 2024-09-18T14:03:23.183 INFO:tasks.rados.rados.0.smithi084.stdout:879: finishing write tid 3 to smithi08434663-11 2024-09-18T14:03:23.183 INFO:tasks.rados.rados.0.smithi084.stdout:881: expect (ObjNum 317 snap 97 seq_num 317) 2024-09-18T14:03:23.335 INFO:tasks.rados.rados.0.smithi084.stdout:879: finishing write tid 4 to smithi08434663-11 2024-09-18T14:03:23.336 INFO:tasks.rados.rados.0.smithi084.stdout:879: finishing write tid 5 to smithi08434663-11 2024-09-18T14:03:23.336 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 204 (ObjNum 324 snap 99 seq_num 324) dirty exists 2024-09-18T14:03:23.336 INFO:tasks.rados.rados.0.smithi084.stdout:879: left oid 11 (ObjNum 324 snap 99 seq_num 324) 2024-09-18T14:03:23.336 INFO:tasks.rados.rados.0.smithi084.stdout:880: finishing copy_from racing read to smithi08434663-38 2024-09-18T14:03:23.336 INFO:tasks.rados.rados.0.smithi084.stdout:880: finishing copy_from to smithi08434663-38 2024-09-18T14:03:23.336 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 262 (ObjNum 278 snap 71 seq_num 278) dirty exists 2024-09-18T14:03:23.336 INFO:tasks.rados.rados.0.smithi084.stdout:883: seq_num 325 ranges {662275=542751,1785236=425447,2942002=520273} 2024-09-18T14:03:23.342 INFO:tasks.rados.rados.0.smithi084.stdout:883: writing smithi08434663-35 from 662275 to 1205026 tid 1 2024-09-18T14:03:23.347 INFO:tasks.rados.rados.0.smithi084.stdout:883: writing smithi08434663-35 from 1785236 to 2210683 tid 2 2024-09-18T14:03:23.355 INFO:tasks.rados.rados.0.smithi084.stdout:883: writing smithi08434663-35 from 2942002 to 3462275 tid 3 2024-09-18T14:03:23.355 INFO:tasks.rados.rados.0.smithi084.stdout:879: done (3 left) 2024-09-18T14:03:23.355 INFO:tasks.rados.rados.0.smithi084.stdout:880: done (2 left) 2024-09-18T14:03:23.355 INFO:tasks.rados.rados.0.smithi084.stdout:881: done (1 left) 2024-09-18T14:03:23.355 INFO:tasks.rados.rados.0.smithi084.stdout:884: copy_from oid 44 from oid 17 current snap is 99 2024-09-18T14:03:23.355 INFO:tasks.rados.rados.0.smithi084.stdout:883: finishing write tid 1 to smithi08434663-35 2024-09-18T14:03:23.355 INFO:tasks.rados.rados.0.smithi084.stdout:883: finishing write tid 2 to smithi08434663-35 2024-09-18T14:03:23.355 INFO:tasks.rados.rados.0.smithi084.stdout:885: copy_from oid 28 from oid 9 current snap is 99 2024-09-18T14:03:23.355 INFO:tasks.rados.rados.0.smithi084.stdout:886: read oid 30 snap 90 2024-09-18T14:03:23.355 INFO:tasks.rados.rados.0.smithi084.stdout:886: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:03:23.356 INFO:tasks.rados.rados.0.smithi084.stdout:887: delete oid 22 current snap is 99 2024-09-18T14:03:23.357 INFO:tasks.rados.rados.0.smithi084.stdout:885: finishing copy_from to smithi08434663-28 2024-09-18T14:03:23.357 INFO:tasks.rados.rados.0.smithi084.stdout:885: got expected ENOENT (src dne) 2024-09-18T14:03:23.357 INFO:tasks.rados.rados.0.smithi084.stdout:885: finishing copy_from racing read to smithi08434663-28 2024-09-18T14:03:23.358 INFO:tasks.rados.rados.0.smithi084.stdout:883: finishing write tid 3 to smithi08434663-35 2024-09-18T14:03:23.358 INFO:tasks.rados.rados.0.smithi084.stdout:886: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:03:23.429 INFO:tasks.rados.rados.0.smithi084.stdout:883: finishing write tid 4 to smithi08434663-35 2024-09-18T14:03:23.429 INFO:tasks.rados.rados.0.smithi084.stdout:883: finishing write tid 5 to smithi08434663-35 2024-09-18T14:03:23.430 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 331 (ObjNum 325 snap 99 seq_num 325) dirty exists 2024-09-18T14:03:23.430 INFO:tasks.rados.rados.0.smithi084.stdout:883: left oid 35 (ObjNum 325 snap 99 seq_num 325) 2024-09-18T14:03:23.430 INFO:tasks.rados.rados.0.smithi084.stdout:884: finishing copy_from racing read to smithi08434663-44 2024-09-18T14:03:23.430 INFO:tasks.rados.rados.0.smithi084.stdout:884: finishing copy_from to smithi08434663-44 2024-09-18T14:03:23.430 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 79 (ObjNum 197 snap 46 seq_num 197) dirty exists 2024-09-18T14:03:23.430 INFO:tasks.rados.rados.0.smithi084.stdout:883: done (4 left) 2024-09-18T14:03:23.430 INFO:tasks.rados.rados.0.smithi084.stdout:884: done (3 left) 2024-09-18T14:03:23.430 INFO:tasks.rados.rados.0.smithi084.stdout:885: done (2 left) 2024-09-18T14:03:23.430 INFO:tasks.rados.rados.0.smithi084.stdout:886: done (1 left) 2024-09-18T14:03:23.430 INFO:tasks.rados.rados.0.smithi084.stdout:887: done (0 left) 2024-09-18T14:03:23.431 INFO:tasks.rados.rados.0.smithi084.stdout:888: read oid 37 snap 85 2024-09-18T14:03:23.431 INFO:tasks.rados.rados.0.smithi084.stdout:888: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:03:23.431 INFO:tasks.rados.rados.0.smithi084.stdout:889: snap_create 2024-09-18T14:03:23.438 INFO:tasks.rados.rados.0.smithi084.stdout:888: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:03:24.186 INFO:tasks.rados.rados.0.smithi084.stdout:888: done (1 left) 2024-09-18T14:03:24.186 INFO:tasks.rados.rados.0.smithi084.stdout:889: done (0 left) 2024-09-18T14:03:24.186 INFO:tasks.rados.rados.0.smithi084.stdout:890: rollback oid 48 current snap is 100 2024-09-18T14:03:24.186 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 48 to 87 2024-09-18T14:03:24.187 INFO:tasks.rados.rados.0.smithi084.stdout:891: copy_from oid 31 from oid 27 current snap is 100 2024-09-18T14:03:24.187 INFO:tasks.rados.rados.0.smithi084.stdout:892: snap_create 2024-09-18T14:03:24.189 INFO:tasks.rados.rados.0.smithi084.stdout:890: finishing rollback tid 0 to smithi08434663-48 2024-09-18T14:03:24.189 INFO:tasks.rados.rados.0.smithi084.stdout:890: finishing rollback tid 1 to smithi08434663-48 2024-09-18T14:03:24.192 INFO:tasks.rados.rados.0.smithi084.stdout:890: finishing rollback tid 2 to smithi08434663-48 2024-09-18T14:03:24.192 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 204 (ObjNum 221 snap 52 seq_num 221) dirty exists 2024-09-18T14:03:24.204 INFO:tasks.rados.rados.0.smithi084.stdout:891: finishing copy_from racing read to smithi08434663-31 2024-09-18T14:03:24.205 INFO:tasks.rados.rados.0.smithi084.stdout:891: finishing copy_from to smithi08434663-31 2024-09-18T14:03:24.205 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 79 (ObjNum 197 snap 46 seq_num 197) dirty exists 2024-09-18T14:03:25.187 INFO:tasks.rados.rados.0.smithi084.stdout:890: done (2 left) 2024-09-18T14:03:25.187 INFO:tasks.rados.rados.0.smithi084.stdout:891: done (1 left) 2024-09-18T14:03:25.187 INFO:tasks.rados.rados.0.smithi084.stdout:892: done (0 left) 2024-09-18T14:03:25.187 INFO:tasks.rados.rados.0.smithi084.stdout:893: write oid 10 current snap is 101 2024-09-18T14:03:25.187 INFO:tasks.rados.rados.0.smithi084.stdout:893: seq_num 327 ranges {695889=409602,1618553=712763,2996848=499041} 2024-09-18T14:03:25.193 INFO:tasks.rados.rados.0.smithi084.stdout:893: writing smithi08434663-10 from 695889 to 1105491 tid 1 2024-09-18T14:03:25.199 INFO:tasks.rados.rados.0.smithi084.stdout:893: writing smithi08434663-10 from 1618553 to 2331316 tid 2 2024-09-18T14:03:25.206 INFO:tasks.rados.rados.0.smithi084.stdout:893: writing smithi08434663-10 from 2996848 to 3495889 tid 3 2024-09-18T14:03:25.206 INFO:tasks.rados.rados.0.smithi084.stdout:894: copy_from oid 3 from oid 7 current snap is 101 2024-09-18T14:03:25.206 INFO:tasks.rados.rados.0.smithi084.stdout:893: finishing write tid 1 to smithi08434663-10 2024-09-18T14:03:25.207 INFO:tasks.rados.rados.0.smithi084.stdout:893: finishing write tid 2 to smithi08434663-10 2024-09-18T14:03:25.207 INFO:tasks.rados.rados.0.smithi084.stdout:895: write (excl) oid 50 current snap is 101 2024-09-18T14:03:25.207 INFO:tasks.rados.rados.0.smithi084.stdout:895: seq_num 328 ranges {712696=484851,1960682=572774,3307918=204778} 2024-09-18T14:03:25.213 INFO:tasks.rados.rados.0.smithi084.stdout:895: writing smithi08434663-50 from 712696 to 1197547 tid 1 2024-09-18T14:03:25.220 INFO:tasks.rados.rados.0.smithi084.stdout:895: writing smithi08434663-50 from 1960682 to 2533456 tid 2 2024-09-18T14:03:25.226 INFO:tasks.rados.rados.0.smithi084.stdout:895: writing smithi08434663-50 from 3307918 to 3512696 tid 3 2024-09-18T14:03:25.226 INFO:tasks.rados.rados.0.smithi084.stdout:896: delete oid 4 current snap is 101 2024-09-18T14:03:25.226 INFO:tasks.rados.rados.0.smithi084.stdout:893: finishing write tid 3 to smithi08434663-10 2024-09-18T14:03:25.226 INFO:tasks.rados.rados.0.smithi084.stdout:893: finishing write tid 4 to smithi08434663-10 2024-09-18T14:03:25.226 INFO:tasks.rados.rados.0.smithi084.stdout:893: finishing write tid 5 to smithi08434663-10 2024-09-18T14:03:25.227 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 272 (ObjNum 327 snap 101 seq_num 327) dirty exists 2024-09-18T14:03:25.227 INFO:tasks.rados.rados.0.smithi084.stdout:893: left oid 10 (ObjNum 327 snap 101 seq_num 327) 2024-09-18T14:03:25.227 INFO:tasks.rados.rados.0.smithi084.stdout:895: finishing write tid 1 to smithi08434663-50 2024-09-18T14:03:25.227 INFO:tasks.rados.rados.0.smithi084.stdout:894: finishing copy_from racing read to smithi08434663-3 2024-09-18T14:03:25.227 INFO:tasks.rados.rados.0.smithi084.stdout:894: finishing copy_from to smithi08434663-3 2024-09-18T14:03:25.227 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 187 (ObjNum 287 snap 74 seq_num 287) dirty exists 2024-09-18T14:03:25.227 INFO:tasks.rados.rados.0.smithi084.stdout:895: finishing write tid 2 to smithi08434663-50 2024-09-18T14:03:25.229 INFO:tasks.rados.rados.0.smithi084.stdout:893: done (3 left) 2024-09-18T14:03:25.229 INFO:tasks.rados.rados.0.smithi084.stdout:894: done (2 left) 2024-09-18T14:03:25.229 INFO:tasks.rados.rados.0.smithi084.stdout:896: done (1 left) 2024-09-18T14:03:25.229 INFO:tasks.rados.rados.0.smithi084.stdout:897: snap_remove snap 78 2024-09-18T14:03:26.202 INFO:tasks.rados.rados.0.smithi084.stdout:897: done (1 left) 2024-09-18T14:03:26.202 INFO:tasks.rados.rados.0.smithi084.stdout:898: write (excl) oid 29 current snap is 101 2024-09-18T14:03:26.202 INFO:tasks.rados.rados.0.smithi084.stdout:898: seq_num 330 ranges {746310=635349,2077646=576443,3362764=183546} 2024-09-18T14:03:26.209 INFO:tasks.rados.rados.0.smithi084.stdout:898: writing smithi08434663-29 from 746310 to 1381659 tid 1 2024-09-18T14:03:26.217 INFO:tasks.rados.rados.0.smithi084.stdout:898: writing smithi08434663-29 from 2077646 to 2654089 tid 2 2024-09-18T14:03:26.222 INFO:tasks.rados.rados.0.smithi084.stdout:898: writing smithi08434663-29 from 3362764 to 3546310 tid 3 2024-09-18T14:03:26.222 INFO:tasks.rados.rados.0.smithi084.stdout:899: rollback oid 9 current snap is 101 2024-09-18T14:03:26.223 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 9 to 85 2024-09-18T14:03:26.223 INFO:tasks.rados.rados.0.smithi084.stdout:895: finishing write tid 3 to smithi08434663-50 2024-09-18T14:03:26.223 INFO:tasks.rados.rados.0.smithi084.stdout:895: finishing write tid 4 to smithi08434663-50 2024-09-18T14:03:26.223 INFO:tasks.rados.rados.0.smithi084.stdout:895: finishing write tid 5 to smithi08434663-50 2024-09-18T14:03:26.223 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 353 (ObjNum 328 snap 101 seq_num 328) dirty exists 2024-09-18T14:03:26.223 INFO:tasks.rados.rados.0.smithi084.stdout:895: left oid 50 (ObjNum 328 snap 101 seq_num 328) 2024-09-18T14:03:26.223 INFO:tasks.rados.rados.0.smithi084.stdout:898: finishing write tid 1 to smithi08434663-29 2024-09-18T14:03:26.223 INFO:tasks.rados.rados.0.smithi084.stdout:898: finishing write tid 2 to smithi08434663-29 2024-09-18T14:03:26.223 INFO:tasks.rados.rados.0.smithi084.stdout:895: done (2 left) 2024-09-18T14:03:26.223 INFO:tasks.rados.rados.0.smithi084.stdout:900: snap_create 2024-09-18T14:03:26.224 INFO:tasks.rados.rados.0.smithi084.stdout:899: finishing rollback tid 1 to smithi08434663-9 2024-09-18T14:03:26.224 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:03:26.224 INFO:tasks.rados.rados.0.smithi084.stdout:898: finishing write tid 3 to smithi08434663-29 2024-09-18T14:03:26.225 INFO:tasks.rados.rados.0.smithi084.stdout:898: finishing write tid 4 to smithi08434663-29 2024-09-18T14:03:26.225 INFO:tasks.rados.rados.0.smithi084.stdout:898: finishing write tid 5 to smithi08434663-29 2024-09-18T14:03:26.225 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 280 (ObjNum 330 snap 101 seq_num 330) dirty exists 2024-09-18T14:03:26.225 INFO:tasks.rados.rados.0.smithi084.stdout:898: left oid 29 (ObjNum 330 snap 101 seq_num 330) 2024-09-18T14:03:27.182 INFO:tasks.rados.rados.0.smithi084.stdout:898: done (2 left) 2024-09-18T14:03:27.182 INFO:tasks.rados.rados.0.smithi084.stdout:899: done (1 left) 2024-09-18T14:03:27.182 INFO:tasks.rados.rados.0.smithi084.stdout:900: done (0 left) 2024-09-18T14:03:27.182 INFO:tasks.rados.rados.0.smithi084.stdout:901: rollback oid 42 current snap is 102 2024-09-18T14:03:27.182 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 42 to 97 2024-09-18T14:03:27.182 INFO:tasks.rados.rados.0.smithi084.stdout:902: read oid 21 snap 85 2024-09-18T14:03:27.182 INFO:tasks.rados.rados.0.smithi084.stdout:902: expect (ObjNum 260 snap 65 seq_num 260) 2024-09-18T14:03:27.182 INFO:tasks.rados.rados.0.smithi084.stdout:903: snap_create 2024-09-18T14:03:27.183 INFO:tasks.rados.rados.0.smithi084.stdout:901: finishing rollback tid 1 to smithi08434663-42 2024-09-18T14:03:27.183 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 320 (ObjNum 97 snap 0 seq_num 2494480224) dirty dne 2024-09-18T14:03:27.185 INFO:tasks.rados.rados.0.smithi084.stdout:902: expect (ObjNum 260 snap 65 seq_num 260) 2024-09-18T14:03:28.190 INFO:tasks.rados.rados.0.smithi084.stdout:901: done (2 left) 2024-09-18T14:03:28.190 INFO:tasks.rados.rados.0.smithi084.stdout:902: done (1 left) 2024-09-18T14:03:28.190 INFO:tasks.rados.rados.0.smithi084.stdout:903: done (0 left) 2024-09-18T14:03:28.190 INFO:tasks.rados.rados.0.smithi084.stdout:904: copy_from oid 12 from oid 7 current snap is 103 2024-09-18T14:03:28.190 INFO:tasks.rados.rados.0.smithi084.stdout:905: read oid 9 snap -1 2024-09-18T14:03:28.191 INFO:tasks.rados.rados.0.smithi084.stdout:905: expect deleted 2024-09-18T14:03:28.191 INFO:tasks.rados.rados.0.smithi084.stdout:906: delete oid 14 current snap is 103 2024-09-18T14:03:28.193 INFO:tasks.rados.rados.0.smithi084.stdout:905: done (2 left) 2024-09-18T14:03:28.194 INFO:tasks.rados.rados.0.smithi084.stdout:906: done (1 left) 2024-09-18T14:03:28.194 INFO:tasks.rados.rados.0.smithi084.stdout:907: rollback oid 28 current snap is 103 2024-09-18T14:03:28.194 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 28 to 81 2024-09-18T14:03:28.194 INFO:tasks.rados.rados.0.smithi084.stdout:908: delete oid 18 current snap is 103 2024-09-18T14:03:28.195 INFO:tasks.rados.rados.0.smithi084.stdout:908: done (2 left) 2024-09-18T14:03:28.196 INFO:tasks.rados.rados.0.smithi084.stdout:909: write oid 17 current snap is 103 2024-09-18T14:03:28.196 INFO:tasks.rados.rados.0.smithi084.stdout:909: seq_num 333 ranges {796731=461096,1736739=723770,2928680=668051} 2024-09-18T14:03:28.203 INFO:tasks.rados.rados.0.smithi084.stdout:909: writing smithi08434663-17 from 796731 to 1257827 tid 1 2024-09-18T14:03:28.210 INFO:tasks.rados.rados.0.smithi084.stdout:909: writing smithi08434663-17 from 1736739 to 2460509 tid 2 2024-09-18T14:03:28.217 INFO:tasks.rados.rados.0.smithi084.stdout:909: writing smithi08434663-17 from 2928680 to 3596731 tid 3 2024-09-18T14:03:28.217 INFO:tasks.rados.rados.0.smithi084.stdout:910: write (excl) oid 36 current snap is 103 2024-09-18T14:03:28.217 INFO:tasks.rados.rados.0.smithi084.stdout:907: finishing rollback tid 0 to smithi08434663-28 2024-09-18T14:03:28.217 INFO:tasks.rados.rados.0.smithi084.stdout:907: finishing rollback tid 1 to smithi08434663-28 2024-09-18T14:03:28.217 INFO:tasks.rados.rados.0.smithi084.stdout:907: finishing rollback tid 2 to smithi08434663-28 2024-09-18T14:03:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 362 (ObjNum 274 snap 71 seq_num 274) dirty exists 2024-09-18T14:03:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:904: finishing copy_from racing read to smithi08434663-12 2024-09-18T14:03:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:909: finishing write tid 1 to smithi08434663-17 2024-09-18T14:03:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:904: finishing copy_from to smithi08434663-12 2024-09-18T14:03:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 187 (ObjNum 287 snap 74 seq_num 287) dirty exists 2024-09-18T14:03:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:909: finishing write tid 2 to smithi08434663-17 2024-09-18T14:03:28.218 INFO:tasks.rados.rados.0.smithi084.stdout:910: seq_num 334 ranges {413538=536345,1395221=583781,2556103=658617} 2024-09-18T14:03:28.221 INFO:tasks.rados.rados.0.smithi084.stdout:910: writing smithi08434663-36 from 413538 to 949883 tid 1 2024-09-18T14:03:28.227 INFO:tasks.rados.rados.0.smithi084.stdout:910: writing smithi08434663-36 from 1395221 to 1979002 tid 2 2024-09-18T14:03:28.234 INFO:tasks.rados.rados.0.smithi084.stdout:910: writing smithi08434663-36 from 2556103 to 3214720 tid 3 2024-09-18T14:03:28.234 INFO:tasks.rados.rados.0.smithi084.stdout:904: done (3 left) 2024-09-18T14:03:28.234 INFO:tasks.rados.rados.0.smithi084.stdout:907: done (2 left) 2024-09-18T14:03:28.234 INFO:tasks.rados.rados.0.smithi084.stdout:911: snap_remove snap 81 2024-09-18T14:03:29.204 INFO:tasks.rados.rados.0.smithi084.stdout:911: done (2 left) 2024-09-18T14:03:29.204 INFO:tasks.rados.rados.0.smithi084.stdout:912: rollback oid 19 current snap is 103 2024-09-18T14:03:29.204 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 19 to 97 2024-09-18T14:03:29.204 INFO:tasks.rados.rados.0.smithi084.stdout:909: finishing write tid 3 to smithi08434663-17 2024-09-18T14:03:29.204 INFO:tasks.rados.rados.0.smithi084.stdout:909: finishing write tid 4 to smithi08434663-17 2024-09-18T14:03:29.204 INFO:tasks.rados.rados.0.smithi084.stdout:909: finishing write tid 5 to smithi08434663-17 2024-09-18T14:03:29.204 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 285 (ObjNum 333 snap 103 seq_num 333) dirty exists 2024-09-18T14:03:29.204 INFO:tasks.rados.rados.0.smithi084.stdout:909: left oid 17 (ObjNum 333 snap 103 seq_num 333) 2024-09-18T14:03:29.204 INFO:tasks.rados.rados.0.smithi084.stdout:910: finishing write tid 1 to smithi08434663-36 2024-09-18T14:03:29.205 INFO:tasks.rados.rados.0.smithi084.stdout:910: finishing write tid 2 to smithi08434663-36 2024-09-18T14:03:29.205 INFO:tasks.rados.rados.0.smithi084.stdout:910: finishing write tid 3 to smithi08434663-36 2024-09-18T14:03:29.205 INFO:tasks.rados.rados.0.smithi084.stdout:910: finishing write tid 4 to smithi08434663-36 2024-09-18T14:03:29.205 INFO:tasks.rados.rados.0.smithi084.stdout:910: finishing write tid 5 to smithi08434663-36 2024-09-18T14:03:29.205 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 239 (ObjNum 334 snap 103 seq_num 334) dirty exists 2024-09-18T14:03:29.205 INFO:tasks.rados.rados.0.smithi084.stdout:910: left oid 36 (ObjNum 334 snap 103 seq_num 334) 2024-09-18T14:03:29.205 INFO:tasks.rados.rados.0.smithi084.stdout:909: done (2 left) 2024-09-18T14:03:29.206 INFO:tasks.rados.rados.0.smithi084.stdout:910: done (1 left) 2024-09-18T14:03:29.206 INFO:tasks.rados.rados.0.smithi084.stdout:913: delete oid 40 current snap is 103 2024-09-18T14:03:29.210 INFO:tasks.rados.rados.0.smithi084.stdout:913: done (1 left) 2024-09-18T14:03:29.210 INFO:tasks.rados.rados.0.smithi084.stdout:914: write (excl) oid 36 current snap is 103 2024-09-18T14:03:29.211 INFO:tasks.rados.rados.0.smithi084.stdout:914: seq_num 336 ranges {447152=403196,1512185=587450,2727302=681161} 2024-09-18T14:03:29.215 INFO:tasks.rados.rados.0.smithi084.stdout:914: writing smithi08434663-36 from 447152 to 850348 tid 1 2024-09-18T14:03:29.222 INFO:tasks.rados.rados.0.smithi084.stdout:914: writing smithi08434663-36 from 1512185 to 2099635 tid 2 2024-09-18T14:03:29.230 INFO:tasks.rados.rados.0.smithi084.stdout:914: writing smithi08434663-36 from 2727302 to 3408463 tid 3 2024-09-18T14:03:29.230 INFO:tasks.rados.rados.0.smithi084.stdout:915: write oid 28 current snap is 103 2024-09-18T14:03:29.230 INFO:tasks.rados.rados.0.smithi084.stdout:912: finishing rollback tid 0 to smithi08434663-19 2024-09-18T14:03:29.230 INFO:tasks.rados.rados.0.smithi084.stdout:915: seq_num 337 ranges {463959=478445,1570667=731108,3038372=492433} 2024-09-18T14:03:29.235 INFO:tasks.rados.rados.0.smithi084.stdout:915: writing smithi08434663-28 from 463959 to 942404 tid 1 2024-09-18T14:03:29.242 INFO:tasks.rados.rados.0.smithi084.stdout:915: writing smithi08434663-28 from 1570667 to 2301775 tid 2 2024-09-18T14:03:29.253 INFO:tasks.rados.rados.0.smithi084.stdout:915: writing smithi08434663-28 from 3038372 to 3530805 tid 3 2024-09-18T14:03:29.253 INFO:tasks.rados.rados.0.smithi084.stdout:916: read oid 42 snap -1 2024-09-18T14:03:29.253 INFO:tasks.rados.rados.0.smithi084.stdout:916: expect deleted 2024-09-18T14:03:29.253 INFO:tasks.rados.rados.0.smithi084.stdout:917: read oid 26 snap -1 2024-09-18T14:03:29.253 INFO:tasks.rados.rados.0.smithi084.stdout:917: expect (ObjNum 272 snap 70 seq_num 272) 2024-09-18T14:03:29.253 INFO:tasks.rados.rados.0.smithi084.stdout:918: read oid 11 snap -1 2024-09-18T14:03:29.253 INFO:tasks.rados.rados.0.smithi084.stdout:918: expect (ObjNum 324 snap 99 seq_num 324) 2024-09-18T14:03:29.253 INFO:tasks.rados.rados.0.smithi084.stdout:919: snap_remove snap 102 2024-09-18T14:03:30.215 INFO:tasks.rados.rados.0.smithi084.stdout:919: done (6 left) 2024-09-18T14:03:30.215 INFO:tasks.rados.rados.0.smithi084.stdout:920: snap_remove snap 99 2024-09-18T14:03:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:920: done (6 left) 2024-09-18T14:03:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:921: delete oid 33 current snap is 103 2024-09-18T14:03:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:912: finishing rollback tid 1 to smithi08434663-19 2024-09-18T14:03:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:912: finishing rollback tid 2 to smithi08434663-19 2024-09-18T14:03:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 368 (ObjNum 247 snap 57 seq_num 247) dirty exists 2024-09-18T14:03:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:914: finishing write tid 1 to smithi08434663-36 2024-09-18T14:03:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:914: finishing write tid 2 to smithi08434663-36 2024-09-18T14:03:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:914: finishing write tid 3 to smithi08434663-36 2024-09-18T14:03:31.213 INFO:tasks.rados.rados.0.smithi084.stdout:914: finishing write tid 4 to smithi08434663-36 2024-09-18T14:03:31.214 INFO:tasks.rados.rados.0.smithi084.stdout:914: finishing write tid 5 to smithi08434663-36 2024-09-18T14:03:31.214 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 245 (ObjNum 336 snap 103 seq_num 336) dirty exists 2024-09-18T14:03:31.214 INFO:tasks.rados.rados.0.smithi084.stdout:914: left oid 36 (ObjNum 336 snap 103 seq_num 336) 2024-09-18T14:03:31.214 INFO:tasks.rados.rados.0.smithi084.stdout:915: finishing write tid 1 to smithi08434663-28 2024-09-18T14:03:31.214 INFO:tasks.rados.rados.0.smithi084.stdout:915: finishing write tid 2 to smithi08434663-28 2024-09-18T14:03:31.214 INFO:tasks.rados.rados.0.smithi084.stdout:915: finishing write tid 3 to smithi08434663-28 2024-09-18T14:03:31.214 INFO:tasks.rados.rados.0.smithi084.stdout:915: finishing write tid 4 to smithi08434663-28 2024-09-18T14:03:31.214 INFO:tasks.rados.rados.0.smithi084.stdout:915: finishing write tid 5 to smithi08434663-28 2024-09-18T14:03:31.214 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 378 (ObjNum 337 snap 103 seq_num 337) dirty exists 2024-09-18T14:03:31.215 INFO:tasks.rados.rados.0.smithi084.stdout:915: left oid 28 (ObjNum 337 snap 103 seq_num 337) 2024-09-18T14:03:31.215 INFO:tasks.rados.rados.0.smithi084.stdout:918: expect (ObjNum 324 snap 99 seq_num 324) 2024-09-18T14:03:31.445 INFO:tasks.rados.rados.0.smithi084.stdout:917: expect (ObjNum 272 snap 70 seq_num 272) 2024-09-18T14:03:31.595 INFO:tasks.rados.rados.0.smithi084.stdout:912: done (6 left) 2024-09-18T14:03:31.595 INFO:tasks.rados.rados.0.smithi084.stdout:914: done (5 left) 2024-09-18T14:03:31.595 INFO:tasks.rados.rados.0.smithi084.stdout:915: done (4 left) 2024-09-18T14:03:31.595 INFO:tasks.rados.rados.0.smithi084.stdout:916: done (3 left) 2024-09-18T14:03:31.595 INFO:tasks.rados.rados.0.smithi084.stdout:917: done (2 left) 2024-09-18T14:03:31.595 INFO:tasks.rados.rados.0.smithi084.stdout:918: done (1 left) 2024-09-18T14:03:31.595 INFO:tasks.rados.rados.0.smithi084.stdout:921: done (0 left) 2024-09-18T14:03:31.595 INFO:tasks.rados.rados.0.smithi084.stdout:922: snap_create 2024-09-18T14:03:32.220 INFO:tasks.rados.rados.0.smithi084.stdout:922: done (0 left) 2024-09-18T14:03:32.221 INFO:tasks.rados.rados.0.smithi084.stdout:923: rollback oid 34 current snap is 104 2024-09-18T14:03:32.221 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 34 to 103 2024-09-18T14:03:32.221 INFO:tasks.rados.rados.0.smithi084.stdout:924: delete oid 9 current snap is 104 2024-09-18T14:03:32.222 INFO:tasks.rados.rados.0.smithi084.stdout:924: done (1 left) 2024-09-18T14:03:32.222 INFO:tasks.rados.rados.0.smithi084.stdout:925: copy_from oid 1 from oid 25 current snap is 104 2024-09-18T14:03:32.222 INFO:tasks.rados.rados.0.smithi084.stdout:926: delete oid 6 current snap is 104 2024-09-18T14:03:32.224 INFO:tasks.rados.rados.0.smithi084.stdout:923: finishing rollback tid 0 to smithi08434663-34 2024-09-18T14:03:32.225 INFO:tasks.rados.rados.0.smithi084.stdout:923: finishing rollback tid 1 to smithi08434663-34 2024-09-18T14:03:32.225 INFO:tasks.rados.rados.0.smithi084.stdout:923: finishing rollback tid 2 to smithi08434663-34 2024-09-18T14:03:32.225 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 324 (ObjNum 138 snap 29 seq_num 138) dirty exists 2024-09-18T14:03:32.226 INFO:tasks.rados.rados.0.smithi084.stdout:923: done (2 left) 2024-09-18T14:03:32.226 INFO:tasks.rados.rados.0.smithi084.stdout:926: done (1 left) 2024-09-18T14:03:32.226 INFO:tasks.rados.rados.0.smithi084.stdout:927: rollback oid 46 current snap is 104 2024-09-18T14:03:32.226 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 46 to 103 2024-09-18T14:03:32.226 INFO:tasks.rados.rados.0.smithi084.stdout:928: read oid 27 snap 89 2024-09-18T14:03:32.227 INFO:tasks.rados.rados.0.smithi084.stdout:928: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:03:32.227 INFO:tasks.rados.rados.0.smithi084.stdout:929: snap_remove snap 100 2024-09-18T14:03:33.223 INFO:tasks.rados.rados.0.smithi084.stdout:929: done (3 left) 2024-09-18T14:03:33.223 INFO:tasks.rados.rados.0.smithi084.stdout:930: copy_from oid 2 from oid 34 current snap is 104 2024-09-18T14:03:33.223 INFO:tasks.rados.rados.0.smithi084.stdout:927: finishing rollback tid 0 to smithi08434663-46 2024-09-18T14:03:33.223 INFO:tasks.rados.rados.0.smithi084.stdout:927: finishing rollback tid 1 to smithi08434663-46 2024-09-18T14:03:33.223 INFO:tasks.rados.rados.0.smithi084.stdout:928: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:03:33.330 INFO:tasks.rados.rados.0.smithi084.stdout:927: finishing rollback tid 2 to smithi08434663-46 2024-09-18T14:03:33.330 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 344 (ObjNum 278 snap 71 seq_num 278) dirty exists 2024-09-18T14:03:33.330 INFO:tasks.rados.rados.0.smithi084.stdout:925: finishing copy_from racing read to smithi08434663-1 2024-09-18T14:03:33.330 INFO:tasks.rados.rados.0.smithi084.stdout:925: finishing copy_from to smithi08434663-1 2024-09-18T14:03:33.330 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 167 (ObjNum 281 snap 71 seq_num 281) dirty exists 2024-09-18T14:03:33.330 INFO:tasks.rados.rados.0.smithi084.stdout:925: done (3 left) 2024-09-18T14:03:33.330 INFO:tasks.rados.rados.0.smithi084.stdout:927: done (2 left) 2024-09-18T14:03:33.331 INFO:tasks.rados.rados.0.smithi084.stdout:928: done (1 left) 2024-09-18T14:03:33.331 INFO:tasks.rados.rados.0.smithi084.stdout:931: write oid 22 current snap is 104 2024-09-18T14:03:33.331 INFO:tasks.rados.rados.0.smithi084.stdout:930: finishing copy_from racing read to smithi08434663-2 2024-09-18T14:03:33.331 INFO:tasks.rados.rados.0.smithi084.stdout:930: finishing copy_from to smithi08434663-2 2024-09-18T14:03:33.331 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 324 (ObjNum 138 snap 29 seq_num 138) dirty exists 2024-09-18T14:03:33.331 INFO:tasks.rados.rados.0.smithi084.stdout:931: seq_num 341 ranges {531187=779441,2088242=738446,3431711=299476} 2024-09-18T14:03:33.336 INFO:tasks.rados.rados.0.smithi084.stdout:931: writing smithi08434663-22 from 531187 to 1310628 tid 1 2024-09-18T14:03:33.345 INFO:tasks.rados.rados.0.smithi084.stdout:931: writing smithi08434663-22 from 2088242 to 2826688 tid 2 2024-09-18T14:03:33.350 INFO:tasks.rados.rados.0.smithi084.stdout:931: writing smithi08434663-22 from 3431711 to 3731187 tid 3 2024-09-18T14:03:33.351 INFO:tasks.rados.rados.0.smithi084.stdout:930: done (1 left) 2024-09-18T14:03:33.351 INFO:tasks.rados.rados.0.smithi084.stdout:932: read oid 44 snap -1 2024-09-18T14:03:33.351 INFO:tasks.rados.rados.0.smithi084.stdout:932: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:03:33.351 INFO:tasks.rados.rados.0.smithi084.stdout:933: write (excl) oid 42 current snap is 104 2024-09-18T14:03:33.351 INFO:tasks.rados.rados.0.smithi084.stdout:931: finishing write tid 1 to smithi08434663-22 2024-09-18T14:03:33.351 INFO:tasks.rados.rados.0.smithi084.stdout:931: finishing write tid 2 to smithi08434663-22 2024-09-18T14:03:33.351 INFO:tasks.rados.rados.0.smithi084.stdout:933: seq_num 342 ranges {547994=454690,1746724=482104,2659134=465146,3605976=142018} 2024-09-18T14:03:33.356 INFO:tasks.rados.rados.0.smithi084.stdout:933: writing smithi08434663-42 from 547994 to 1002684 tid 1 2024-09-18T14:03:33.362 INFO:tasks.rados.rados.0.smithi084.stdout:933: writing smithi08434663-42 from 1746724 to 2228828 tid 2 2024-09-18T14:03:33.367 INFO:tasks.rados.rados.0.smithi084.stdout:933: writing smithi08434663-42 from 2659134 to 3124280 tid 3 2024-09-18T14:03:33.370 INFO:tasks.rados.rados.0.smithi084.stdout:933: writing smithi08434663-42 from 3605976 to 3747994 tid 4 2024-09-18T14:03:33.370 INFO:tasks.rados.rados.0.smithi084.stdout:934: write (excl) oid 36 current snap is 104 2024-09-18T14:03:33.371 INFO:tasks.rados.rados.0.smithi084.stdout:934: seq_num 343 ranges {564801=646292,1921559=742115,3202910=561891} 2024-09-18T14:03:33.378 INFO:tasks.rados.rados.0.smithi084.stdout:934: writing smithi08434663-36 from 564801 to 1211093 tid 1 2024-09-18T14:03:33.385 INFO:tasks.rados.rados.0.smithi084.stdout:934: writing smithi08434663-36 from 1921559 to 2663674 tid 2 2024-09-18T14:03:33.391 INFO:tasks.rados.rados.0.smithi084.stdout:934: writing smithi08434663-36 from 3202910 to 3764801 tid 3 2024-09-18T14:03:33.391 INFO:tasks.rados.rados.0.smithi084.stdout:935: delete oid 14 current snap is 104 2024-09-18T14:03:33.391 INFO:tasks.rados.rados.0.smithi084.stdout:931: finishing write tid 3 to smithi08434663-22 2024-09-18T14:03:33.391 INFO:tasks.rados.rados.0.smithi084.stdout:931: finishing write tid 4 to smithi08434663-22 2024-09-18T14:03:33.391 INFO:tasks.rados.rados.0.smithi084.stdout:931: finishing write tid 5 to smithi08434663-22 2024-09-18T14:03:33.392 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 326 (ObjNum 341 snap 104 seq_num 341) dirty exists 2024-09-18T14:03:33.392 INFO:tasks.rados.rados.0.smithi084.stdout:931: left oid 22 (ObjNum 341 snap 104 seq_num 341) 2024-09-18T14:03:33.392 INFO:tasks.rados.rados.0.smithi084.stdout:932: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:03:33.497 INFO:tasks.rados.rados.0.smithi084.stdout:933: finishing write tid 1 to smithi08434663-42 2024-09-18T14:03:33.497 INFO:tasks.rados.rados.0.smithi084.stdout:933: finishing write tid 2 to smithi08434663-42 2024-09-18T14:03:33.497 INFO:tasks.rados.rados.0.smithi084.stdout:933: finishing write tid 3 to smithi08434663-42 2024-09-18T14:03:33.497 INFO:tasks.rados.rados.0.smithi084.stdout:933: finishing write tid 4 to smithi08434663-42 2024-09-18T14:03:33.497 INFO:tasks.rados.rados.0.smithi084.stdout:933: finishing write tid 5 to smithi08434663-42 2024-09-18T14:03:33.497 INFO:tasks.rados.rados.0.smithi084.stdout:933: finishing write tid 6 to smithi08434663-42 2024-09-18T14:03:33.497 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 329 (ObjNum 342 snap 104 seq_num 342) dirty exists 2024-09-18T14:03:33.497 INFO:tasks.rados.rados.0.smithi084.stdout:933: left oid 42 (ObjNum 342 snap 104 seq_num 342) 2024-09-18T14:03:33.497 INFO:tasks.rados.rados.0.smithi084.stdout:934: finishing write tid 1 to smithi08434663-36 2024-09-18T14:03:33.497 INFO:tasks.rados.rados.0.smithi084.stdout:931: done (4 left) 2024-09-18T14:03:33.498 INFO:tasks.rados.rados.0.smithi084.stdout:932: done (3 left) 2024-09-18T14:03:33.498 INFO:tasks.rados.rados.0.smithi084.stdout:933: done (2 left) 2024-09-18T14:03:33.498 INFO:tasks.rados.rados.0.smithi084.stdout:935: done (1 left) 2024-09-18T14:03:33.498 INFO:tasks.rados.rados.0.smithi084.stdout:936: write (excl) oid 47 current snap is 104 2024-09-18T14:03:33.498 INFO:tasks.rados.rados.0.smithi084.stdout:934: finishing write tid 2 to smithi08434663-36 2024-09-18T14:03:33.498 INFO:tasks.rados.rados.0.smithi084.stdout:934: finishing write tid 3 to smithi08434663-36 2024-09-18T14:03:33.498 INFO:tasks.rados.rados.0.smithi084.stdout:934: finishing write tid 4 to smithi08434663-36 2024-09-18T14:03:33.498 INFO:tasks.rados.rados.0.smithi084.stdout:934: finishing write tid 5 to smithi08434663-36 2024-09-18T14:03:33.498 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 254 (ObjNum 343 snap 104 seq_num 343) dirty exists 2024-09-18T14:03:33.498 INFO:tasks.rados.rados.0.smithi084.stdout:934: left oid 36 (ObjNum 343 snap 104 seq_num 343) 2024-09-18T14:03:33.498 INFO:tasks.rados.rados.0.smithi084.stdout:936: seq_num 345 ranges {598415=796790,1922170=745784,3141403=415315} 2024-09-18T14:03:33.503 INFO:tasks.rados.rados.0.smithi084.stdout:936: writing smithi08434663-47 from 598415 to 1395205 tid 1 2024-09-18T14:03:33.510 INFO:tasks.rados.rados.0.smithi084.stdout:936: writing smithi08434663-47 from 1922170 to 2667954 tid 2 2024-09-18T14:03:33.517 INFO:tasks.rados.rados.0.smithi084.stdout:936: writing smithi08434663-47 from 3141403 to 3556718 tid 3 2024-09-18T14:03:33.517 INFO:tasks.rados.rados.0.smithi084.stdout:934: done (1 left) 2024-09-18T14:03:33.517 INFO:tasks.rados.rados.0.smithi084.stdout:937: read oid 24 snap -1 2024-09-18T14:03:33.517 INFO:tasks.rados.rados.0.smithi084.stdout:937: expect (ObjNum 267 snap 68 seq_num 267) 2024-09-18T14:03:33.517 INFO:tasks.rados.rados.0.smithi084.stdout:938: copy_from oid 23 from oid 10 current snap is 104 2024-09-18T14:03:33.517 INFO:tasks.rados.rados.0.smithi084.stdout:936: finishing write tid 1 to smithi08434663-47 2024-09-18T14:03:33.517 INFO:tasks.rados.rados.0.smithi084.stdout:936: finishing write tid 2 to smithi08434663-47 2024-09-18T14:03:33.517 INFO:tasks.rados.rados.0.smithi084.stdout:939: read oid 37 snap -1 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:939: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:940: rollback oid 41 current snap is 104 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 41 to 101 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:941: read oid 26 snap -1 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:941: expect (ObjNum 272 snap 70 seq_num 272) 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:942: copy_from oid 9 from oid 40 current snap is 104 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:943: rollback oid 45 current snap is 104 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 45 to 85 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:944: rollback oid 44 current snap is 104 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 44 to 103 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:945: rollback oid 8 current snap is 104 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 8 to 98 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:946: rollback oid 38 current snap is 104 2024-09-18T14:03:33.518 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 38 to 98 2024-09-18T14:03:33.519 INFO:tasks.rados.rados.0.smithi084.stdout:947: write (excl) oid 29 current snap is 104 2024-09-18T14:03:33.519 INFO:tasks.rados.rados.0.smithi084.stdout:947: seq_num 346 ranges {615222=472039,1580652=489442,2768826=626587} 2024-09-18T14:03:33.522 INFO:tasks.rados.rados.0.smithi084.stdout:947: writing smithi08434663-29 from 615222 to 1087261 tid 1 2024-09-18T14:03:33.528 INFO:tasks.rados.rados.0.smithi084.stdout:947: writing smithi08434663-29 from 1580652 to 2070094 tid 2 2024-09-18T14:03:33.535 INFO:tasks.rados.rados.0.smithi084.stdout:947: writing smithi08434663-29 from 2768826 to 3395413 tid 3 2024-09-18T14:03:33.535 INFO:tasks.rados.rados.0.smithi084.stdout:948: read oid 27 snap -1 2024-09-18T14:03:33.535 INFO:tasks.rados.rados.0.smithi084.stdout:948: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:03:33.536 INFO:tasks.rados.rados.0.smithi084.stdout:949: delete oid 3 current snap is 104 2024-09-18T14:03:33.536 INFO:tasks.rados.rados.0.smithi084.stdout:942: finishing copy_from to smithi08434663-9 2024-09-18T14:03:33.536 INFO:tasks.rados.rados.0.smithi084.stdout:942: got expected ENOENT (src dne) 2024-09-18T14:03:33.536 INFO:tasks.rados.rados.0.smithi084.stdout:942: finishing copy_from racing read to smithi08434663-9 2024-09-18T14:03:33.536 INFO:tasks.rados.rados.0.smithi084.stdout:941: expect (ObjNum 272 snap 70 seq_num 272) 2024-09-18T14:03:33.689 INFO:tasks.rados.rados.0.smithi084.stdout:945: finishing rollback tid 1 to smithi08434663-8 2024-09-18T14:03:33.689 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 184 (ObjNum 33 snap 0 seq_num 2495121456) dirty dne 2024-09-18T14:03:33.690 INFO:tasks.rados.rados.0.smithi084.stdout:936: finishing write tid 3 to smithi08434663-47 2024-09-18T14:03:33.690 INFO:tasks.rados.rados.0.smithi084.stdout:936: finishing write tid 4 to smithi08434663-47 2024-09-18T14:03:33.690 INFO:tasks.rados.rados.0.smithi084.stdout:946: finishing rollback tid 0 to smithi08434663-38 2024-09-18T14:03:33.690 INFO:tasks.rados.rados.0.smithi084.stdout:946: finishing rollback tid 1 to smithi08434663-38 2024-09-18T14:03:33.690 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 264 (ObjNum 33 snap 0 seq_num 3384845502) dirty dne 2024-09-18T14:03:33.690 INFO:tasks.rados.rados.0.smithi084.stdout:936: finishing write tid 5 to smithi08434663-47 2024-09-18T14:03:33.690 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 210 (ObjNum 345 snap 104 seq_num 345) dirty exists 2024-09-18T14:03:33.690 INFO:tasks.rados.rados.0.smithi084.stdout:936: left oid 47 (ObjNum 345 snap 104 seq_num 345) 2024-09-18T14:03:33.690 INFO:tasks.rados.rados.0.smithi084.stdout:944: finishing rollback tid 0 to smithi08434663-44 2024-09-18T14:03:33.690 INFO:tasks.rados.rados.0.smithi084.stdout:936: done (13 left) 2024-09-18T14:03:33.690 INFO:tasks.rados.rados.0.smithi084.stdout:941: done (12 left) 2024-09-18T14:03:33.690 INFO:tasks.rados.rados.0.smithi084.stdout:942: done (11 left) 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:945: done (10 left) 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:946: done (9 left) 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:949: done (8 left) 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:950: write oid 16 current snap is 104 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:944: finishing rollback tid 1 to smithi08434663-44 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:944: finishing rollback tid 2 to smithi08434663-44 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 154 (ObjNum 197 snap 46 seq_num 197) dirty exists 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:940: finishing rollback tid 0 to smithi08434663-41 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:940: finishing rollback tid 1 to smithi08434663-41 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:940: finishing rollback tid 2 to smithi08434663-41 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 332 (ObjNum 152 snap 31 seq_num 152) dirty exists 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:943: finishing rollback tid 1 to smithi08434663-45 2024-09-18T14:03:33.691 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 0 (ObjNum 33 snap 0 seq_num 2495142992) dirty dne 2024-09-18T14:03:33.692 INFO:tasks.rados.rados.0.smithi084.stdout:939: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:03:33.765 INFO:tasks.rados.rados.0.smithi084.stdout:947: finishing write tid 1 to smithi08434663-29 2024-09-18T14:03:33.765 INFO:tasks.rados.rados.0.smithi084.stdout:937: expect (ObjNum 267 snap 68 seq_num 267) 2024-09-18T14:03:33.932 INFO:tasks.rados.rados.0.smithi084.stdout:938: finishing copy_from racing read to smithi08434663-23 2024-09-18T14:03:33.932 INFO:tasks.rados.rados.0.smithi084.stdout:947: finishing write tid 2 to smithi08434663-29 2024-09-18T14:03:33.932 INFO:tasks.rados.rados.0.smithi084.stdout:938: finishing copy_from to smithi08434663-23 2024-09-18T14:03:33.932 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 272 (ObjNum 327 snap 101 seq_num 327) dirty exists 2024-09-18T14:03:33.932 INFO:tasks.rados.rados.0.smithi084.stdout:948: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:03:34.036 INFO:tasks.rados.rados.0.smithi084.stdout:947: finishing write tid 3 to smithi08434663-29 2024-09-18T14:03:34.036 INFO:tasks.rados.rados.0.smithi084.stdout:947: finishing write tid 4 to smithi08434663-29 2024-09-18T14:03:34.037 INFO:tasks.rados.rados.0.smithi084.stdout:947: finishing write tid 5 to smithi08434663-29 2024-09-18T14:03:34.037 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 304 (ObjNum 346 snap 104 seq_num 346) dirty exists 2024-09-18T14:03:34.037 INFO:tasks.rados.rados.0.smithi084.stdout:947: left oid 29 (ObjNum 346 snap 104 seq_num 346) 2024-09-18T14:03:34.037 INFO:tasks.rados.rados.0.smithi084.stdout:950: seq_num 348 ranges {648836=622537,1981263=493111,3107319=741517} 2024-09-18T14:03:34.043 INFO:tasks.rados.rados.0.smithi084.stdout:950: writing smithi08434663-16 from 648836 to 1271373 tid 1 2024-09-18T14:03:34.049 INFO:tasks.rados.rados.0.smithi084.stdout:950: writing smithi08434663-16 from 1981263 to 2474374 tid 2 2024-09-18T14:03:34.057 INFO:tasks.rados.rados.0.smithi084.stdout:950: writing smithi08434663-16 from 3107319 to 3848836 tid 3 2024-09-18T14:03:34.057 INFO:tasks.rados.rados.0.smithi084.stdout:937: done (8 left) 2024-09-18T14:03:34.057 INFO:tasks.rados.rados.0.smithi084.stdout:938: done (7 left) 2024-09-18T14:03:34.057 INFO:tasks.rados.rados.0.smithi084.stdout:939: done (6 left) 2024-09-18T14:03:34.057 INFO:tasks.rados.rados.0.smithi084.stdout:940: done (5 left) 2024-09-18T14:03:34.057 INFO:tasks.rados.rados.0.smithi084.stdout:943: done (4 left) 2024-09-18T14:03:34.058 INFO:tasks.rados.rados.0.smithi084.stdout:944: done (3 left) 2024-09-18T14:03:34.058 INFO:tasks.rados.rados.0.smithi084.stdout:947: done (2 left) 2024-09-18T14:03:34.058 INFO:tasks.rados.rados.0.smithi084.stdout:948: done (1 left) 2024-09-18T14:03:34.058 INFO:tasks.rados.rados.0.smithi084.stdout:951: read oid 45 snap -1 2024-09-18T14:03:34.058 INFO:tasks.rados.rados.0.smithi084.stdout:951: expect deleted 2024-09-18T14:03:34.058 INFO:tasks.rados.rados.0.smithi084.stdout:952: write oid 34 current snap is 104 2024-09-18T14:03:34.058 INFO:tasks.rados.rados.0.smithi084.stdout:950: finishing write tid 1 to smithi08434663-16 2024-09-18T14:03:34.058 INFO:tasks.rados.rados.0.smithi084.stdout:950: finishing write tid 2 to smithi08434663-16 2024-09-18T14:03:34.058 INFO:tasks.rados.rados.0.smithi084.stdout:952: seq_num 349 ranges {665643=697786,2039745=753122,3534742=330901} 2024-09-18T14:03:34.064 INFO:tasks.rados.rados.0.smithi084.stdout:952: writing smithi08434663-34 from 665643 to 1363429 tid 1 2024-09-18T14:03:34.072 INFO:tasks.rados.rados.0.smithi084.stdout:952: writing smithi08434663-34 from 2039745 to 2792867 tid 2 2024-09-18T14:03:34.078 INFO:tasks.rados.rados.0.smithi084.stdout:952: writing smithi08434663-34 from 3534742 to 3865643 tid 3 2024-09-18T14:03:34.079 INFO:tasks.rados.rados.0.smithi084.stdout:953: read oid 44 snap -1 2024-09-18T14:03:34.079 INFO:tasks.rados.rados.0.smithi084.stdout:953: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:03:34.079 INFO:tasks.rados.rados.0.smithi084.stdout:954: read oid 13 snap -1 2024-09-18T14:03:34.079 INFO:tasks.rados.rados.0.smithi084.stdout:954: expect deleted 2024-09-18T14:03:34.079 INFO:tasks.rados.rados.0.smithi084.stdout:955: write oid 50 current snap is 104 2024-09-18T14:03:34.079 INFO:tasks.rados.rados.0.smithi084.stdout:955: seq_num 350 ranges {682450=489388,1814580=496780,2878518=788028} 2024-09-18T14:03:34.084 INFO:tasks.rados.rados.0.smithi084.stdout:955: writing smithi08434663-50 from 682450 to 1171838 tid 1 2024-09-18T14:03:34.090 INFO:tasks.rados.rados.0.smithi084.stdout:955: writing smithi08434663-50 from 1814580 to 2311360 tid 2 2024-09-18T14:03:34.098 INFO:tasks.rados.rados.0.smithi084.stdout:955: writing smithi08434663-50 from 2878518 to 3666546 tid 3 2024-09-18T14:03:34.098 INFO:tasks.rados.rados.0.smithi084.stdout:956: read oid 43 snap -1 2024-09-18T14:03:34.098 INFO:tasks.rados.rados.0.smithi084.stdout:956: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:03:34.098 INFO:tasks.rados.rados.0.smithi084.stdout:957: snap_remove snap 101 2024-09-18T14:03:34.238 INFO:tasks.rados.rados.0.smithi084.stdout:957: done (7 left) 2024-09-18T14:03:34.238 INFO:tasks.rados.rados.0.smithi084.stdout:958: snap_create 2024-09-18T14:03:34.238 INFO:tasks.rados.rados.0.smithi084.stdout:950: finishing write tid 3 to smithi08434663-16 2024-09-18T14:03:34.238 INFO:tasks.rados.rados.0.smithi084.stdout:950: finishing write tid 4 to smithi08434663-16 2024-09-18T14:03:34.238 INFO:tasks.rados.rados.0.smithi084.stdout:950: finishing write tid 5 to smithi08434663-16 2024-09-18T14:03:34.238 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 350 (ObjNum 348 snap 104 seq_num 348) dirty exists 2024-09-18T14:03:34.238 INFO:tasks.rados.rados.0.smithi084.stdout:950: left oid 16 (ObjNum 348 snap 104 seq_num 348) 2024-09-18T14:03:34.238 INFO:tasks.rados.rados.0.smithi084.stdout:952: finishing write tid 1 to smithi08434663-34 2024-09-18T14:03:34.238 INFO:tasks.rados.rados.0.smithi084.stdout:952: finishing write tid 2 to smithi08434663-34 2024-09-18T14:03:34.238 INFO:tasks.rados.rados.0.smithi084.stdout:953: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:03:34.344 INFO:tasks.rados.rados.0.smithi084.stdout:952: finishing write tid 3 to smithi08434663-34 2024-09-18T14:03:34.344 INFO:tasks.rados.rados.0.smithi084.stdout:952: finishing write tid 4 to smithi08434663-34 2024-09-18T14:03:34.344 INFO:tasks.rados.rados.0.smithi084.stdout:952: finishing write tid 5 to smithi08434663-34 2024-09-18T14:03:34.344 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 336 (ObjNum 349 snap 104 seq_num 349) dirty exists 2024-09-18T14:03:34.344 INFO:tasks.rados.rados.0.smithi084.stdout:952: left oid 34 (ObjNum 349 snap 104 seq_num 349) 2024-09-18T14:03:34.344 INFO:tasks.rados.rados.0.smithi084.stdout:955: finishing write tid 1 to smithi08434663-50 2024-09-18T14:03:34.344 INFO:tasks.rados.rados.0.smithi084.stdout:955: finishing write tid 2 to smithi08434663-50 2024-09-18T14:03:34.345 INFO:tasks.rados.rados.0.smithi084.stdout:955: finishing write tid 3 to smithi08434663-50 2024-09-18T14:03:34.345 INFO:tasks.rados.rados.0.smithi084.stdout:955: finishing write tid 4 to smithi08434663-50 2024-09-18T14:03:34.345 INFO:tasks.rados.rados.0.smithi084.stdout:955: finishing write tid 5 to smithi08434663-50 2024-09-18T14:03:34.345 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 407 (ObjNum 350 snap 104 seq_num 350) dirty exists 2024-09-18T14:03:34.345 INFO:tasks.rados.rados.0.smithi084.stdout:955: left oid 50 (ObjNum 350 snap 104 seq_num 350) 2024-09-18T14:03:34.345 INFO:tasks.rados.rados.0.smithi084.stdout:956: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:03:34.753 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-18T14:03:34.753 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-18T14:03:34.753 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-18T14:03:34.753 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-18T14:03:34.756 INFO:tasks.ceph.mon.g:Started 2024-09-18T14:03:34.757 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-18T14:03:34.757 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-18T14:03:34.757 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-18T14:03:34.757 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-18T14:03:34.760 INFO:tasks.ceph.mon.b:Started 2024-09-18T14:03:34.760 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-18T14:03:34.760 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-18T14:03:34.760 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T14:03:34.760 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T14:03:34.762 INFO:tasks.ceph.mon.c:Started 2024-09-18T14:03:34.762 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-09-18T14:03:34.763 INFO:tasks.ceph.mon.a:Sent signal 19 2024-09-18T14:03:34.763 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-18T14:03:34.763 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-18T14:03:34.763 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-18T14:03:44.945 INFO:tasks.rados.rados.0.smithi084.stdout:950: done (7 left) 2024-09-18T14:03:44.945 INFO:tasks.rados.rados.0.smithi084.stdout:951: done (6 left) 2024-09-18T14:03:44.945 INFO:tasks.rados.rados.0.smithi084.stdout:952: done (5 left) 2024-09-18T14:03:44.945 INFO:tasks.rados.rados.0.smithi084.stdout:953: done (4 left) 2024-09-18T14:03:44.945 INFO:tasks.rados.rados.0.smithi084.stdout:954: done (3 left) 2024-09-18T14:03:44.945 INFO:tasks.rados.rados.0.smithi084.stdout:955: done (2 left) 2024-09-18T14:03:44.945 INFO:tasks.rados.rados.0.smithi084.stdout:956: done (1 left) 2024-09-18T14:03:44.945 INFO:tasks.rados.rados.0.smithi084.stdout:958: done (0 left) 2024-09-18T14:03:44.945 INFO:tasks.rados.rados.0.smithi084.stdout:959: read oid 16 snap -1 2024-09-18T14:03:44.946 INFO:tasks.rados.rados.0.smithi084.stdout:959: expect (ObjNum 348 snap 104 seq_num 348) 2024-09-18T14:03:44.946 INFO:tasks.rados.rados.0.smithi084.stdout:960: rollback oid 13 current snap is 105 2024-09-18T14:03:44.946 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 13 to 89 2024-09-18T14:03:44.946 INFO:tasks.rados.rados.0.smithi084.stdout:961: read oid 49 snap -1 2024-09-18T14:03:44.946 INFO:tasks.rados.rados.0.smithi084.stdout:961: expect deleted 2024-09-18T14:03:44.946 INFO:tasks.rados.rados.0.smithi084.stdout:962: snap_create 2024-09-18T14:03:44.953 INFO:tasks.rados.rados.0.smithi084.stdout:960: finishing rollback tid 1 to smithi08434663-13 2024-09-18T14:03:44.953 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 0 (ObjNum 129 snap 0 seq_num 3384933780) dirty dne 2024-09-18T14:03:44.954 INFO:tasks.rados.rados.0.smithi084.stdout:959: expect (ObjNum 348 snap 104 seq_num 348) 2024-09-18T14:03:45.949 INFO:tasks.rados.rados.0.smithi084.stdout:959: done (3 left) 2024-09-18T14:03:45.949 INFO:tasks.rados.rados.0.smithi084.stdout:960: done (2 left) 2024-09-18T14:03:45.949 INFO:tasks.rados.rados.0.smithi084.stdout:961: done (1 left) 2024-09-18T14:03:45.949 INFO:tasks.rados.rados.0.smithi084.stdout:962: done (0 left) 2024-09-18T14:03:45.949 INFO:tasks.rados.rados.0.smithi084.stdout:963: snap_remove snap 104 2024-09-18T14:03:46.955 INFO:tasks.rados.rados.0.smithi084.stdout:963: done (0 left) 2024-09-18T14:03:46.955 INFO:tasks.rados.rados.0.smithi084.stdout:964: write (excl) oid 20 current snap is 106 2024-09-18T14:03:46.955 INFO:tasks.rados.rados.0.smithi084.stdout:964: seq_num 351 ranges {699257=564637,1873062=640438,3305941=593316} 2024-09-18T14:03:46.962 INFO:tasks.rados.rados.0.smithi084.stdout:964: writing smithi08434663-20 from 699257 to 1263894 tid 1 2024-09-18T14:03:46.969 INFO:tasks.rados.rados.0.smithi084.stdout:964: writing smithi08434663-20 from 1873062 to 2513500 tid 2 2024-09-18T14:03:46.976 INFO:tasks.rados.rados.0.smithi084.stdout:964: writing smithi08434663-20 from 3305941 to 3899257 tid 3 2024-09-18T14:03:46.976 INFO:tasks.rados.rados.0.smithi084.stdout:965: snap_create 2024-09-18T14:03:46.977 INFO:tasks.rados.rados.0.smithi084.stdout:964: finishing write tid 1 to smithi08434663-20 2024-09-18T14:03:46.977 INFO:tasks.rados.rados.0.smithi084.stdout:964: finishing write tid 2 to smithi08434663-20 2024-09-18T14:03:46.980 INFO:tasks.rados.rados.0.smithi084.stdout:964: finishing write tid 3 to smithi08434663-20 2024-09-18T14:03:46.980 INFO:tasks.rados.rados.0.smithi084.stdout:964: finishing write tid 4 to smithi08434663-20 2024-09-18T14:03:46.980 INFO:tasks.rados.rados.0.smithi084.stdout:964: finishing write tid 5 to smithi08434663-20 2024-09-18T14:03:46.980 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 188 (ObjNum 351 snap 106 seq_num 351) dirty exists 2024-09-18T14:03:46.980 INFO:tasks.rados.rados.0.smithi084.stdout:964: left oid 20 (ObjNum 351 snap 106 seq_num 351) 2024-09-18T14:03:47.185 INFO:tasks.rados.rados.0.smithi084.stdout:964: done (1 left) 2024-09-18T14:03:47.185 INFO:tasks.rados.rados.0.smithi084.stdout:965: done (0 left) 2024-09-18T14:03:47.185 INFO:tasks.rados.rados.0.smithi084.stdout:966: read oid 46 snap -1 2024-09-18T14:03:47.185 INFO:tasks.rados.rados.0.smithi084.stdout:966: expect (ObjNum 278 snap 71 seq_num 278) 2024-09-18T14:03:47.185 INFO:tasks.rados.rados.0.smithi084.stdout:967: write (excl) oid 10 current snap is 107 2024-09-18T14:03:47.185 INFO:tasks.rados.rados.0.smithi084.stdout:967: seq_num 352 ranges {716064=639886,1815191=500449,2817011=526925} 2024-09-18T14:03:47.192 INFO:tasks.rados.rados.0.smithi084.stdout:967: writing smithi08434663-10 from 716064 to 1355950 tid 1 2024-09-18T14:03:47.198 INFO:tasks.rados.rados.0.smithi084.stdout:967: writing smithi08434663-10 from 1815191 to 2315640 tid 2 2024-09-18T14:03:47.204 INFO:tasks.rados.rados.0.smithi084.stdout:967: writing smithi08434663-10 from 2817011 to 3343936 tid 3 2024-09-18T14:03:47.204 INFO:tasks.rados.rados.0.smithi084.stdout:968: rollback oid 32 current snap is 107 2024-09-18T14:03:47.204 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 32 to 85 2024-09-18T14:03:47.204 INFO:tasks.rados.rados.0.smithi084.stdout:966: expect (ObjNum 278 snap 71 seq_num 278) 2024-09-18T14:03:47.291 INFO:tasks.rados.rados.0.smithi084.stdout:967: finishing write tid 1 to smithi08434663-10 2024-09-18T14:03:47.291 INFO:tasks.rados.rados.0.smithi084.stdout:967: finishing write tid 2 to smithi08434663-10 2024-09-18T14:03:47.292 INFO:tasks.rados.rados.0.smithi084.stdout:967: finishing write tid 3 to smithi08434663-10 2024-09-18T14:03:47.292 INFO:tasks.rados.rados.0.smithi084.stdout:967: finishing write tid 4 to smithi08434663-10 2024-09-18T14:03:47.292 INFO:tasks.rados.rados.0.smithi084.stdout:967: finishing write tid 5 to smithi08434663-10 2024-09-18T14:03:47.292 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 295 (ObjNum 352 snap 107 seq_num 352) dirty exists 2024-09-18T14:03:47.292 INFO:tasks.rados.rados.0.smithi084.stdout:967: left oid 10 (ObjNum 352 snap 107 seq_num 352) 2024-09-18T14:03:47.292 INFO:tasks.rados.rados.0.smithi084.stdout:968: finishing rollback tid 0 to smithi08434663-32 2024-09-18T14:03:47.292 INFO:tasks.rados.rados.0.smithi084.stdout:968: finishing rollback tid 1 to smithi08434663-32 2024-09-18T14:03:47.292 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 418 (ObjNum 129 snap 0 seq_num 2494554768) dirty dne 2024-09-18T14:03:47.292 INFO:tasks.rados.rados.0.smithi084.stdout:966: done (2 left) 2024-09-18T14:03:47.292 INFO:tasks.rados.rados.0.smithi084.stdout:967: done (1 left) 2024-09-18T14:03:47.292 INFO:tasks.rados.rados.0.smithi084.stdout:968: done (0 left) 2024-09-18T14:03:47.292 INFO:tasks.rados.rados.0.smithi084.stdout:969: delete oid 30 current snap is 107 2024-09-18T14:03:47.293 INFO:tasks.rados.rados.0.smithi084.stdout:969: done (0 left) 2024-09-18T14:03:47.293 INFO:tasks.rados.rados.0.smithi084.stdout:970: read oid 32 snap -1 2024-09-18T14:03:47.294 INFO:tasks.rados.rados.0.smithi084.stdout:970: expect deleted 2024-09-18T14:03:47.294 INFO:tasks.rados.rados.0.smithi084.stdout:971: snap_remove snap 103 2024-09-18T14:03:48.196 INFO:tasks.rados.rados.0.smithi084.stdout:971: done (1 left) 2024-09-18T14:03:48.196 INFO:tasks.rados.rados.0.smithi084.stdout:972: read oid 50 snap -1 2024-09-18T14:03:48.196 INFO:tasks.rados.rados.0.smithi084.stdout:972: expect (ObjNum 350 snap 104 seq_num 350) 2024-09-18T14:03:48.196 INFO:tasks.rados.rados.0.smithi084.stdout:973: copy_from oid 41 from oid 4 current snap is 107 2024-09-18T14:03:48.196 INFO:tasks.rados.rados.0.smithi084.stdout:970: done (2 left) 2024-09-18T14:03:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:974: read oid 21 snap 77 2024-09-18T14:03:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:974: expect (ObjNum 260 snap 65 seq_num 260) 2024-09-18T14:03:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:975: read oid 49 snap -1 2024-09-18T14:03:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:975: expect deleted 2024-09-18T14:03:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:976: copy_from oid 28 from oid 45 current snap is 107 2024-09-18T14:03:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:977: snap_create 2024-09-18T14:03:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:976: finishing copy_from to smithi08434663-28 2024-09-18T14:03:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:976: got expected ENOENT (src dne) 2024-09-18T14:03:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:973: finishing copy_from to smithi08434663-41 2024-09-18T14:03:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:973: got expected ENOENT (src dne) 2024-09-18T14:03:48.199 INFO:tasks.rados.rados.0.smithi084.stdout:973: finishing copy_from racing read to smithi08434663-41 2024-09-18T14:03:48.199 INFO:tasks.rados.rados.0.smithi084.stdout:974: expect (ObjNum 260 snap 65 seq_num 260) 2024-09-18T14:03:48.327 INFO:tasks.rados.rados.0.smithi084.stdout:976: finishing copy_from racing read to smithi08434663-28 2024-09-18T14:03:48.327 INFO:tasks.rados.rados.0.smithi084.stdout:972: expect (ObjNum 350 snap 104 seq_num 350) 2024-09-18T14:03:49.200 INFO:tasks.rados.rados.0.smithi084.stdout:972: done (5 left) 2024-09-18T14:03:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:973: done (4 left) 2024-09-18T14:03:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:974: done (3 left) 2024-09-18T14:03:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:975: done (2 left) 2024-09-18T14:03:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:976: done (1 left) 2024-09-18T14:03:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:977: done (0 left) 2024-09-18T14:03:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:978: read oid 30 snap -1 2024-09-18T14:03:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:978: expect deleted 2024-09-18T14:03:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:979: copy_from oid 34 from oid 31 current snap is 108 2024-09-18T14:03:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:980: rollback oid 38 current snap is 108 2024-09-18T14:03:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 38 to 90 2024-09-18T14:03:49.201 INFO:tasks.rados.rados.0.smithi084.stdout:981: snap_create 2024-09-18T14:03:49.208 INFO:tasks.rados.rados.0.smithi084.stdout:980: finishing rollback tid 1 to smithi08434663-38 2024-09-18T14:03:49.208 INFO:tasks.rados.rados.0.smithi084.stdout:980: finishing rollback tid 2 to smithi08434663-38 2024-09-18T14:03:49.208 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 266 (ObjNum 72 snap 6 seq_num 72) dirty exists 2024-09-18T14:03:49.216 INFO:tasks.rados.rados.0.smithi084.stdout:979: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:03:49.217 INFO:tasks.rados.rados.0.smithi084.stdout:979: finishing copy_from to smithi08434663-34 2024-09-18T14:03:49.217 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 79 (ObjNum 197 snap 46 seq_num 197) dirty exists 2024-09-18T14:03:49.764 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-09-18T14:03:49.764 INFO:tasks.ceph.mon.a:Sent signal 18 2024-09-18T14:03:49.764 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-18T14:03:49.765 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-18T14:03:49.765 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:03:49.765 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:03:49.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.881+0000 7fc4ea59d640 10 monclient: get_monmap_and_config 2024-09-18T14:03:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.881+0000 7fc4ea59d640 10 monclient: build_initial_monmap 2024-09-18T14:03:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.881+0000 7fc4ea59d640 10 monclient: monmap: 2024-09-18T14:03:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:49.882324+0000 2024-09-18T14:03:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:49.882324+0000 2024-09-18T14:03:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:49.883 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 1 Processor -- start 2024-09-18T14:03:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 1 -- start start 2024-09-18T14:03:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 10 monclient: init 2024-09-18T14:03:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 10 monclient: _add_conns ranks=[5,0,4,8,6,3,2,1,7] 2024-09-18T14:03:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 10 monclient(hunting): picked mon.noname-h con 0x7fc4e40a2610 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 10 monclient(hunting): picked mon.noname-a con 0x7fc4e41330b0 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 10 monclient(hunting): picked mon.noname-c con 0x7fc4e4096a20 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e405ed70 con 0x7fc4e41330b0 2024-09-18T14:03:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e40a1740 con 0x7fc4e4096a20 2024-09-18T14:03:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e40a29e0 con 0x7fc4e40a2610 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4ea59d640 10 monclient(hunting): authenticate will time out at 1045.220717s 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4e37fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc4e41330b0 0x7fc4e4137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44782/0 (socket says 172.21.15.45:44782) 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4e8b3a640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fc4e4096a20 0x7fc4e413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38334/0 (socket says 172.21.15.45:38334) 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.882+0000 7fc4e37fe640 1 -- 172.21.15.45:0/1796258320 learned_addr learned my addr 172.21.15.45:0/1796258320 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486582873 0 0) 0x7fc4e40a1740 con 0x7fc4e4096a20 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient(hunting): my global_id is 144103 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4cc0034e0 con 0x7fc4e4096a20 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573179342 0 0) 0x7fc4e405ed70 con 0x7fc4e41330b0 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient(hunting): my global_id is 4842 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e40a1740 con 0x7fc4e41330b0 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600100440 0 0) 0x7fc4e40a29e0 con 0x7fc4e40a2610 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient(hunting): my global_id is 4694 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e405ed70 con 0x7fc4e40a2610 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1917396760 0 0) 0x7fc4cc0034e0 con 0x7fc4e4096a20 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 >> v1:172.21.15.84:6791/0 conn(0x7fc4e40a2610 legacy=0x7fc4e41329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 >> v1:172.21.15.45:6789/0 conn(0x7fc4e41330b0 legacy=0x7fc4e4137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 monclient: found mon.noname-c 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:03:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e40970a0 con 0x7fc4e4096a20 2024-09-18T14:03:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient: _finish_auth 0 2024-09-18T14:03:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4dc001bb0 con 0x7fc4e4096a20 2024-09-18T14:03:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:03:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient: dump: 2024-09-18T14:03:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:49.896 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4ea59d640 5 monclient: authenticate success, global_id 144103 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4ea59d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc4dc001d90 con 0x7fc4e4096a20 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/1796258320 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4dc004e60 con 0x7fc4e4096a20 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4e2ffd640 10 monclient: dump: 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:49.898 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4ea59d640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.883+0000 7fc4ea59d640 4 set_mon_vals no callback set 2024-09-18T14:03:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.884+0000 7fc4ea59d640 10 monclient: shutdown 2024-09-18T14:03:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.884+0000 7fc4ea59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.884+0000 7fc4ea59d640 1 -- 172.21.15.45:0/1796258320 >> v1:172.21.15.45:6791/0 conn(0x7fc4e4096a20 legacy=0x7fc4e413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.884+0000 7fc4ea59d640 1 -- 172.21.15.45:0/1796258320 shutdown_connections 2024-09-18T14:03:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.884+0000 7fc4ea59d640 1 -- 172.21.15.45:0/1796258320 >> 172.21.15.45:0/1796258320 conn(0x7fc4e40630f0 msgr2=0x7fc4e40634f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.884+0000 7fc4ea59d640 1 -- 172.21.15.45:0/1796258320 shutdown_connections 2024-09-18T14:03:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.884+0000 7fc4ea59d640 1 -- 172.21.15.45:0/1796258320 wait complete. 2024-09-18T14:03:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.884+0000 7fc4ea59d640 10 monclient: build_initial_monmap 2024-09-18T14:03:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.884+0000 7fc4ea59d640 10 monclient: monmap: 2024-09-18T14:03:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 1 Processor -- start 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 1 -- start start 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 10 monclient: init 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 10 monclient: _add_conns ranks=[7,6,4,3,1,2,5,8,0] 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 10 monclient(hunting): picked mon.noname-i con 0x7fc4e4096a20 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 10 monclient(hunting): picked mon.noname-d con 0x7fc4e40a2610 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 10 monclient(hunting): picked mon.noname-c con 0x7fc4e41330b0 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e411e280 con 0x7fc4e41330b0 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e40a2350 con 0x7fc4e40a2610 2024-09-18T14:03:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e411f730 con 0x7fc4e4096a20 2024-09-18T14:03:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4ea59d640 10 monclient(hunting): authenticate will time out at 1045.223729s 2024-09-18T14:03:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4e8b3a640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fc4e41330b0 0x7fc4e4149920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38336/0 (socket says 172.21.15.45:38336) 2024-09-18T14:03:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4e8b3a640 1 -- 172.21.15.45:0/2212163914 learned_addr learned my addr 172.21.15.45:0/2212163914 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.885+0000 7fc4e37fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fc4e40a2610 0x7fc4e411c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:58882/0 (socket says 172.21.15.45:58882) 2024-09-18T14:03:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473618925 0 0) 0x7fc4e40a2350 con 0x7fc4e40a2610 2024-09-18T14:03:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): my global_id is 4686 2024-09-18T14:03:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4d00034e0 con 0x7fc4e40a2610 2024-09-18T14:03:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127850229 0 0) 0x7fc4e411e280 con 0x7fc4e41330b0 2024-09-18T14:03:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): my global_id is 144112 2024-09-18T14:03:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e40a2350 con 0x7fc4e41330b0 2024-09-18T14:03:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694272898 0 0) 0x7fc4e411f730 con 0x7fc4e4096a20 2024-09-18T14:03:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): my global_id is 4768 2024-09-18T14:03:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e411e280 con 0x7fc4e4096a20 2024-09-18T14:03:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1043956430 0 0) 0x7fc4d00034e0 con 0x7fc4e40a2610 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4e411f730 con 0x7fc4e40a2610 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4d8001960 con 0x7fc4e40a2610 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): dump: 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1987467745 0 0) 0x7fc4e40a2350 con 0x7fc4e41330b0 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4d00034e0 con 0x7fc4e41330b0 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4dc004ee0 con 0x7fc4e41330b0 2024-09-18T14:03:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): dump: 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:49.907 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2621483946 0 0) 0x7fc4e411e280 con 0x7fc4e4096a20 2024-09-18T14:03:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4e40a2350 con 0x7fc4e4096a20 2024-09-18T14:03:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4d4001a10 con 0x7fc4e4096a20 2024-09-18T14:03:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:03:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient(hunting): dump: 2024-09-18T14:03:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:49.908 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1647194639 0 0) 0x7fc4e411f730 con 0x7fc4e40a2610 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 >> v1:172.21.15.84:6792/0 conn(0x7fc4e4096a20 legacy=0x7fc4e411c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 >> v1:172.21.15.45:6791/0 conn(0x7fc4e41330b0 legacy=0x7fc4e4149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient: _finish_hunting 0 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 monclient: found mon.d 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e411f450 con 0x7fc4e40a2610 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient: _finish_auth 0 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4ea59d640 5 monclient: authenticate success, global_id 4686 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4ea59d640 10 monclient: _renew_subs 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4ea59d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4ea59d640 1 -- 172.21.15.45:0/2212163914 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e411d030 con 0x7fc4e40a2610 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4ea59d640 10 monclient: _renew_subs 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4ea59d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4ea59d640 1 -- 172.21.15.45:0/2212163914 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e411f170 con 0x7fc4e40a2610 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc4d80047a0 con 0x7fc4e40a2610 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4d8004c90 con 0x7fc4e40a2610 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4cbfff640 10 monclient: dump: 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.886+0000 7fc4e27fc640 4 set_mon_vals no callback set 2024-09-18T14:03:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.887+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc4d800fce0 con 0x7fc4e40a2610 2024-09-18T14:03:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.887+0000 7fc4ea59d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.887+0000 7fc4ea59d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.887+0000 7fc4ea59d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.887+0000 7fc4ea59d640 1 -- 172.21.15.45:0/2212163914 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e404c4a0 con 0x7fc4e40a2610 2024-09-18T14:03:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.888+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(226..226 src has 1..226) v4 ==== 3911+0+0 (unknown 679261631 0 0) 0x7fc4d8046920 con 0x7fc4e40a2610 2024-09-18T14:03:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.888+0000 7fc4e3fff640 10 monclient: get_auth_request con 0x7fc4d003aa10 auth_method 0 2024-09-18T14:03:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.892+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fc4d8011280 con 0x7fc4e40a2610 2024-09-18T14:03:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.892+0000 7fc4cbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:49.892+0000 7fc4cbfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.115+0000 7fc4ea59d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:03:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.115+0000 7fc4ea59d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:03:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.115+0000 7fc4ea59d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.115+0000 7fc4ea59d640 1 -- 172.21.15.45:0/2212163914 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc4e404b720 con 0x7fc4e40a2610 2024-09-18T14:03:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.116+0000 7fc4cbfff640 1 -- 172.21.15.45:0/2212163914 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 2462847050) 0x7fc4d8014b60 con 0x7fc4e40a2610 2024-09-18T14:03:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.116+0000 7fc4cbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:03:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.116+0000 7fc4cbfff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:50.117 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":64,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:03:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.118+0000 7fc4ea59d640 1 -- 172.21.15.45:0/2212163914 >> v1:172.21.15.84:6812/31478 conn(0x7fc4d003aa10 legacy=0x7fc4d003cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.118+0000 7fc4ea59d640 10 monclient: shutdown 2024-09-18T14:03:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.118+0000 7fc4ea59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.118+0000 7fc4ea59d640 1 -- 172.21.15.45:0/2212163914 >> v1:172.21.15.45:6792/0 conn(0x7fc4e40a2610 legacy=0x7fc4e411c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.118+0000 7fc4ea59d640 1 -- 172.21.15.45:0/2212163914 shutdown_connections 2024-09-18T14:03:50.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.118+0000 7fc4ea59d640 1 -- 172.21.15.45:0/2212163914 >> 172.21.15.45:0/2212163914 conn(0x7fc4e40630f0 msgr2=0x7fc4e4131d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:50.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.118+0000 7fc4ea59d640 1 -- 172.21.15.45:0/2212163914 shutdown_connections 2024-09-18T14:03:50.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.118+0000 7fc4ea59d640 1 -- 172.21.15.45:0/2212163914 wait complete. 2024-09-18T14:03:50.128 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:03:50.129 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:03:50.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:03:50.209 INFO:tasks.rados.rados.0.smithi084.stdout:978: done (3 left) 2024-09-18T14:03:50.209 INFO:tasks.rados.rados.0.smithi084.stdout:979: done (2 left) 2024-09-18T14:03:50.209 INFO:tasks.rados.rados.0.smithi084.stdout:980: done (1 left) 2024-09-18T14:03:50.209 INFO:tasks.rados.rados.0.smithi084.stdout:981: done (0 left) 2024-09-18T14:03:50.209 INFO:tasks.rados.rados.0.smithi084.stdout:982: write oid 40 current snap is 109 2024-09-18T14:03:50.209 INFO:tasks.rados.rados.0.smithi084.stdout:982: seq_num 354 ranges {749678=790384,2332155=504118,3271857=549469} 2024-09-18T14:03:50.217 INFO:tasks.rados.rados.0.smithi084.stdout:982: writing smithi08434663-40 from 749678 to 1540062 tid 1 2024-09-18T14:03:50.224 INFO:tasks.rados.rados.0.smithi084.stdout:982: writing smithi08434663-40 from 2332155 to 2836273 tid 2 2024-09-18T14:03:50.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.229+0000 7f6821f9d640 10 monclient: get_monmap_and_config 2024-09-18T14:03:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.229+0000 7f6821f9d640 10 monclient: build_initial_monmap 2024-09-18T14:03:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.229+0000 7f6821f9d640 10 monclient: monmap: 2024-09-18T14:03:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:50.230441+0000 2024-09-18T14:03:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:50.230441+0000 2024-09-18T14:03:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:50.231 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:50.231 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:50.231 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:50.231 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:50.231 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:50.231 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:50.231 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.232 INFO:tasks.rados.rados.0.smithi084.stdout:982: writing smithi08434663-40 from 3271857 to 3821326 tid 3 2024-09-18T14:03:50.232 INFO:tasks.rados.rados.0.smithi084.stdout:983: write oid 39 current snap is 109 2024-09-18T14:03:50.232 INFO:tasks.rados.rados.0.smithi084.stdout:983: seq_num 355 ranges {766485=465633,1990637=647776,3299280=667205} 2024-09-18T14:03:50.238 INFO:tasks.rados.rados.0.smithi084.stdout:983: writing smithi08434663-39 from 766485 to 1232118 tid 1 2024-09-18T14:03:50.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 1 Processor -- start 2024-09-18T14:03:50.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 1 -- start start 2024-09-18T14:03:50.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 10 monclient: init 2024-09-18T14:03:50.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 10 monclient: _add_conns ranks=[8,7,4,5,1,0,6,2,3] 2024-09-18T14:03:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f681c06e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:03:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 10 monclient(hunting): picked mon.noname-i con 0x7f681c0a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 10 monclient(hunting): picked mon.noname-c con 0x7f681c0a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681c05ed70 con 0x7f681c0a1080 2024-09-18T14:03:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681c097620 con 0x7f681c0a0cb0 2024-09-18T14:03:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681c097340 con 0x7f681c06e810 2024-09-18T14:03:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f6821f9d640 10 monclient(hunting): authenticate will time out at 1045.568861s 2024-09-18T14:03:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f681bfff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f681c0a1080 0x7f681c13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38348/0 (socket says 172.21.15.45:38348) 2024-09-18T14:03:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681bfff640 1 -- 172.21.15.45:0/3867700081 learned_addr learned my addr 172.21.15.45:0/3867700081 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.230+0000 7f681b7fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f681c06e810 0x7f681c0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:49970/0 (socket says 172.21.15.45:49970) 2024-09-18T14:03:50.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545585977 0 0) 0x7f681c097620 con 0x7f681c0a0cb0 2024-09-18T14:03:50.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient(hunting): my global_id is 4777 2024-09-18T14:03:50.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68040034e0 con 0x7f681c0a0cb0 2024-09-18T14:03:50.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663727782 0 0) 0x7f681c097340 con 0x7f681c06e810 2024-09-18T14:03:50.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient(hunting): my global_id is 4850 2024-09-18T14:03:50.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f681c097620 con 0x7f681c06e810 2024-09-18T14:03:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283644930 0 0) 0x7f681c05ed70 con 0x7f681c0a1080 2024-09-18T14:03:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient(hunting): my global_id is 144121 2024-09-18T14:03:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f681c097340 con 0x7f681c0a1080 2024-09-18T14:03:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1423298516 0 0) 0x7f68040034e0 con 0x7f681c0a0cb0 2024-09-18T14:03:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 >> v1:172.21.15.45:6793/0 conn(0x7f681c06e810 legacy=0x7f681c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 >> v1:172.21.15.45:6791/0 conn(0x7f681c0a1080 legacy=0x7f681c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:03:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 monclient: found mon.noname-i 2024-09-18T14:03:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c0a1660 con 0x7f681c0a0cb0 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6810001830 con 0x7f681c0a0cb0 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient: dump: 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f6821f9d640 5 monclient: authenticate success, global_id 4777 2024-09-18T14:03:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f6821f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6810001a10 con 0x7f681c0a0cb0 2024-09-18T14:03:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f681a7fc640 1 -- 172.21.15.45:0/3867700081 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6810001c00 con 0x7f681c0a0cb0 2024-09-18T14:03:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f6821f9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.232+0000 7f681a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.231+0000 7f6821f9d640 4 set_mon_vals no callback set 2024-09-18T14:03:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.232+0000 7f681a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:03:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.232+0000 7f681a7fc640 10 monclient: dump: 2024-09-18T14:03:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.232+0000 7f6821f9d640 10 monclient: shutdown 2024-09-18T14:03:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.232+0000 7f6821f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.232+0000 7f6821f9d640 1 -- 172.21.15.45:0/3867700081 >> v1:172.21.15.84:6792/0 conn(0x7f681c0a0cb0 legacy=0x7f681c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.232+0000 7f6821f9d640 1 -- 172.21.15.45:0/3867700081 shutdown_connections 2024-09-18T14:03:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.232+0000 7f6821f9d640 1 -- 172.21.15.45:0/3867700081 >> 172.21.15.45:0/3867700081 conn(0x7f681c0a5bb0 msgr2=0x7f681c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.233+0000 7f6821f9d640 1 -- 172.21.15.45:0/3867700081 shutdown_connections 2024-09-18T14:03:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.233+0000 7f6821f9d640 1 -- 172.21.15.45:0/3867700081 wait complete. 2024-09-18T14:03:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.233+0000 7f6821f9d640 10 monclient: build_initial_monmap 2024-09-18T14:03:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.233+0000 7f6821f9d640 10 monclient: monmap: 2024-09-18T14:03:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:50.250 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 1 Processor -- start 2024-09-18T14:03:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 1 -- start start 2024-09-18T14:03:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 10 monclient: init 2024-09-18T14:03:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 10 monclient: _add_conns ranks=[5,1,7,4,0,6,3,8,2] 2024-09-18T14:03:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 10 monclient(hunting): picked mon.noname-h con 0x7f681c06e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 10 monclient(hunting): picked mon.noname-f con 0x7f681c0a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 10 monclient(hunting): picked mon.noname-i con 0x7f681c0a1080 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681c122f00 con 0x7f681c0a0cb0 2024-09-18T14:03:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681c13dcf0 con 0x7f681c06e810 2024-09-18T14:03:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681c122940 con 0x7f681c0a1080 2024-09-18T14:03:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f6821f9d640 10 monclient(hunting): authenticate will time out at 1045.572618s 2024-09-18T14:03:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f681b7fe640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f681c06e810 0x7f681c13d5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:45626/0 (socket says 172.21.15.45:45626) 2024-09-18T14:03:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f681affd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f681c0a0cb0 0x7f681c146810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:42226/0 (socket says 172.21.15.45:42226) 2024-09-18T14:03:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.234+0000 7f681b7fe640 1 -- 172.21.15.45:0/3320229989 learned_addr learned my addr 172.21.15.45:0/3320229989 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874911304 0 0) 0x7f681c122940 con 0x7f681c0a1080 2024-09-18T14:03:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): my global_id is 4786 2024-09-18T14:03:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68080034e0 con 0x7f681c0a1080 2024-09-18T14:03:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405827939 0 0) 0x7f681c13dcf0 con 0x7f681c06e810 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): my global_id is 4703 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f681c122940 con 0x7f681c06e810 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267925733 0 0) 0x7f681c122f00 con 0x7f681c0a0cb0 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): my global_id is 4744 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f681c13dcf0 con 0x7f681c0a0cb0 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1776470855 0 0) 0x7f68080034e0 con 0x7f681c0a1080 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f681c122f00 con 0x7f681c0a1080 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6814001d80 con 0x7f681c0a1080 2024-09-18T14:03:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): dump: 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 66741718 0 0) 0x7f681c13dcf0 con 0x7f681c0a0cb0 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68080034e0 con 0x7f681c0a0cb0 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 234173750 0 0) 0x7f681c122940 con 0x7f681c06e810 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f681c13dcf0 con 0x7f681c06e810 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6810002200 con 0x7f681c0a0cb0 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:03:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): dump: 2024-09-18T14:03:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f680c001b20 con 0x7f681c06e810 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient(hunting): dump: 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.257 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2865058008 0 0) 0x7f681c122f00 con 0x7f681c0a1080 2024-09-18T14:03:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 >> v1:172.21.15.84:6791/0 conn(0x7f681c06e810 legacy=0x7f681c13d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 >> v1:172.21.15.84:6789/0 conn(0x7f681c0a0cb0 legacy=0x7f681c146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient: _finish_hunting 0 2024-09-18T14:03:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 monclient: found mon.i 2024-09-18T14:03:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:03:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c120490 con 0x7f681c0a1080 2024-09-18T14:03:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient: _finish_auth 0 2024-09-18T14:03:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f6821f9d640 5 monclient: authenticate success, global_id 4786 2024-09-18T14:03:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f6821f9d640 10 monclient: _renew_subs 2024-09-18T14:03:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f6821f9d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:03:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f6821f9d640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681c122660 con 0x7f681c0a1080 2024-09-18T14:03:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f6821f9d640 10 monclient: _renew_subs 2024-09-18T14:03:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f6821f9d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:03:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f6821f9d640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f681c121790 con 0x7f681c0a1080 2024-09-18T14:03:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6814004ba0 con 0x7f681c0a1080 2024-09-18T14:03:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6814005050 con 0x7f681c0a1080 2024-09-18T14:03:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f6819ffb640 4 set_mon_vals no callback set 2024-09-18T14:03:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:03:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.235+0000 7f68037fe640 10 monclient: dump: 2024-09-18T14:03:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.262 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.236+0000 7f6821f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.236+0000 7f6821f9d640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f67fc006580 con 0x7f67fc002830 2024-09-18T14:03:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.236+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f68140051b0 con 0x7f681c0a1080 2024-09-18T14:03:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.236+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(227..227 src has 1..227) v4 ==== 3911+0+0 (unknown 472785454 0 0) 0x7f6814046e20 con 0x7f681c0a1080 2024-09-18T14:03:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.236+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626876192 0 0) 0x7f67fc006580 con 0x7f67fc002830 2024-09-18T14:03:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.236+0000 7f68037fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.236+0000 7f68037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.236+0000 7f68037fe640 10 monclient(hunting): my global_id is 4851 2024-09-18T14:03:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.236+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f681c122f00 con 0x7f67fc002830 2024-09-18T14:03:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.236+0000 7f681affd640 10 monclient: get_auth_request con 0x7f680803aac0 auth_method 0 2024-09-18T14:03:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3045888939 0 0) 0x7f681c122f00 con 0x7f67fc002830 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f67fc006580 con 0x7f67fc002830 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f680c004e10 con 0x7f67fc002830 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 10 monclient: dump: 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.264 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1250156699 0 0) 0x7f67fc006580 con 0x7f67fc002830 2024-09-18T14:03:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67fc00a0a0 con 0x7f67fc002830 2024-09-18T14:03:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f680c004ff0 con 0x7f67fc002830 2024-09-18T14:03:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 >> v1:172.21.15.45:6789/0 conn(0x7f67fc002830 legacy=0x7f67fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.237+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 >> v1:172.21.15.45:6789/0 conn(0x7f67fc002830 legacy=0x7f67fc004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f6821f9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f6821f9d640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f67fc0095f0 con 0x7f67fc008ff0 2024-09-18T14:03:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793891259 0 0) 0x7f67fc0095f0 con 0x7f67fc008ff0 2024-09-18T14:03:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f68037fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f68037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f68037fe640 10 monclient(hunting): my global_id is 4860 2024-09-18T14:03:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67fc006580 con 0x7f67fc008ff0 2024-09-18T14:03:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2742894202 0 0) 0x7f67fc006580 con 0x7f67fc008ff0 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f67fc0095f0 con 0x7f67fc008ff0 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f680c0054f0 con 0x7f67fc008ff0 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f68037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f68037fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.243+0000 7f68037fe640 10 monclient: dump: 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4168480998 0 0) 0x7f67fc0095f0 con 0x7f67fc008ff0 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f67fc0098b0 con 0x7f67fc008ff0 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 2016963762) 0x7f680c005b10 con 0x7f67fc008ff0 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f68037fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f68037fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 >> v1:172.21.15.45:6789/0 conn(0x7f67fc008ff0 legacy=0x7f67fc008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f68037fe640 1 -- 172.21.15.45:0/3320229989 >> v1:172.21.15.45:6789/0 conn(0x7f67fc008ff0 legacy=0x7f67fc008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:50.270 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:50.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:50.270 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:50.270 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:50.270 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:50.270 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:50.270 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:50.271 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.272 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:03:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:50.289 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:50.290 INFO:tasks.rados.rados.0.smithi084.stdout:983: writing smithi08434663-39 from 1990637 to 2638413 tid 2 2024-09-18T14:03:50.290 INFO:tasks.rados.rados.0.smithi084.stdout:983: writing smithi08434663-39 from 3299280 to 3966485 tid 3 2024-09-18T14:03:50.290 INFO:tasks.rados.rados.0.smithi084.stdout:984: rollback oid 27 current snap is 109 2024-09-18T14:03:50.290 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 27 to 98 2024-09-18T14:03:50.290 INFO:tasks.rados.rados.0.smithi084.stdout:982: finishing write tid 1 to smithi08434663-40 2024-09-18T14:03:50.290 INFO:tasks.rados.rados.0.smithi084.stdout:982: finishing write tid 2 to smithi08434663-40 2024-09-18T14:03:50.290 INFO:tasks.rados.rados.0.smithi084.stdout:982: finishing write tid 3 to smithi08434663-40 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:982: finishing write tid 4 to smithi08434663-40 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:982: finishing write tid 5 to smithi08434663-40 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 317 (ObjNum 354 snap 109 seq_num 354) dirty exists 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:982: left oid 40 (ObjNum 354 snap 109 seq_num 354) 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:983: finishing write tid 1 to smithi08434663-39 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:983: finishing write tid 2 to smithi08434663-39 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:982: done (2 left) 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:985: rollback oid 3 current snap is 109 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 3 to 89 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:986: snap_create 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:983: finishing write tid 3 to smithi08434663-39 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:983: finishing write tid 4 to smithi08434663-39 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:985: finishing rollback tid 1 to smithi08434663-3 2024-09-18T14:03:50.291 INFO:tasks.rados.rados.0.smithi084.stdout:985: finishing rollback tid 2 to smithi08434663-3 2024-09-18T14:03:50.292 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 329 (ObjNum 226 snap 53 seq_num 226) dirty exists 2024-09-18T14:03:50.292 INFO:tasks.rados.rados.0.smithi084.stdout:983: finishing write tid 5 to smithi08434663-39 2024-09-18T14:03:50.292 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 219 (ObjNum 355 snap 109 seq_num 355) dirty exists 2024-09-18T14:03:50.292 INFO:tasks.rados.rados.0.smithi084.stdout:983: left oid 39 (ObjNum 355 snap 109 seq_num 355) 2024-09-18T14:03:50.292 INFO:tasks.rados.rados.0.smithi084.stdout:984: finishing rollback tid 0 to smithi08434663-27 2024-09-18T14:03:50.292 INFO:tasks.rados.rados.0.smithi084.stdout:984: finishing rollback tid 1 to smithi08434663-27 2024-09-18T14:03:50.292 INFO:tasks.rados.rados.0.smithi084.stdout:984: finishing rollback tid 2 to smithi08434663-27 2024-09-18T14:03:50.292 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 303 (ObjNum 197 snap 46 seq_num 197) dirty exists 2024-09-18T14:03:50.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f6821f9d640 1 -- 172.21.15.45:0/3320229989 >> v1:172.21.15.84:6812/31478 conn(0x7f680803aac0 legacy=0x7f680803cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f6821f9d640 10 monclient: shutdown 2024-09-18T14:03:50.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f6821f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:50.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f6821f9d640 1 -- 172.21.15.45:0/3320229989 >> v1:172.21.15.84:6792/0 conn(0x7f681c0a1080 legacy=0x7f681c149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f681bfff640 1 -- 172.21.15.45:0/3320229989 reap_dead start 2024-09-18T14:03:50.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f681bfff640 1 -- 172.21.15.45:0/3320229989 reap_dead start 2024-09-18T14:03:50.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f6821f9d640 1 -- 172.21.15.45:0/3320229989 shutdown_connections 2024-09-18T14:03:50.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f6821f9d640 1 -- 172.21.15.45:0/3320229989 >> 172.21.15.45:0/3320229989 conn(0x7f681c0a5bb0 msgr2=0x7f681c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:50.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f6821f9d640 1 -- 172.21.15.45:0/3320229989 shutdown_connections 2024-09-18T14:03:50.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.244+0000 7f6821f9d640 1 -- 172.21.15.45:0/3320229989 wait complete. 2024-09-18T14:03:50.294 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:50.295 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:03:50.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:03:50.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.391+0000 7f9ce6b5d640 10 monclient: get_monmap_and_config 2024-09-18T14:03:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.391+0000 7f9ce6b5d640 10 monclient: build_initial_monmap 2024-09-18T14:03:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.391+0000 7f9ce6b5d640 10 monclient: monmap: 2024-09-18T14:03:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:50.392536+0000 2024-09-18T14:03:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:50.392536+0000 2024-09-18T14:03:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:50.393 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:50.393 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:50.393 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:50.393 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:50.393 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:50.393 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:50.393 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 1 Processor -- start 2024-09-18T14:03:50.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 1 -- start start 2024-09-18T14:03:50.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 10 monclient: init 2024-09-18T14:03:50.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:50.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 10 monclient: _add_conns ranks=[0,7,3,4,1,6,8,5,2] 2024-09-18T14:03:50.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 10 monclient(hunting): picked mon.noname-a con 0x7f9ce009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:50.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 10 monclient(hunting): picked mon.noname-i con 0x7f9ce009fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:50.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 10 monclient(hunting): picked mon.noname-g con 0x7f9ce009ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce005ed70 con 0x7f9ce009f7d0 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce009dfc0 con 0x7f9ce009ff70 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce009dce0 con 0x7f9ce009fba0 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.392+0000 7f9ce6b5d640 10 monclient(hunting): authenticate will time out at 1045.730955s 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9ce48f9640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9ce009f7d0 0x7f9ce0132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44824/0 (socket says 172.21.15.45:44824) 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9ce48f9640 1 -- 172.21.15.45:0/3950609379 learned_addr learned my addr 172.21.15.45:0/3950609379 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475793299 0 0) 0x7f9ce005ed70 con 0x7f9ce009f7d0 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient(hunting): my global_id is 4869 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cd00034e0 con 0x7f9ce009f7d0 2024-09-18T14:03:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561014839 0 0) 0x7f9ce009dfc0 con 0x7f9ce009ff70 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient(hunting): my global_id is 144102 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce005ed70 con 0x7f9ce009ff70 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800658215 0 0) 0x7f9ce009dce0 con 0x7f9ce009fba0 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient(hunting): my global_id is 4795 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce009dfc0 con 0x7f9ce009fba0 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2242651790 0 0) 0x7f9cd00034e0 con 0x7f9ce009f7d0 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 >> v1:172.21.15.84:6792/0 conn(0x7f9ce009fba0 legacy=0x7f9ce0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 >> v1:172.21.15.84:6790/0 conn(0x7f9ce009ff70 legacy=0x7f9ce013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 monclient: found mon.noname-a 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce00a1c00 con 0x7f9ce009f7d0 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient: _finish_auth 0 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9cd4001830 con 0x7f9ce009f7d0 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9ce6b5d640 5 monclient: authenticate success, global_id 4869 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9ce6b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient: dump: 2024-09-18T14:03:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9ce6b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9cd4001a10 con 0x7f9ce009f7d0 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 1 -- 172.21.15.45:0/3950609379 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9cd4004ba0 con 0x7f9ce009f7d0 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9cdf7fe640 10 monclient: dump: 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9ce6b5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.393+0000 7f9ce6b5d640 4 set_mon_vals no callback set 2024-09-18T14:03:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.394+0000 7f9ce6b5d640 10 monclient: shutdown 2024-09-18T14:03:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.394+0000 7f9ce6b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.394+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/3950609379 >> v1:172.21.15.45:6789/0 conn(0x7f9ce009f7d0 legacy=0x7f9ce0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.394+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/3950609379 shutdown_connections 2024-09-18T14:03:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.394+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/3950609379 >> 172.21.15.45:0/3950609379 conn(0x7f9ce00630b0 msgr2=0x7f9ce00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.394+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/3950609379 shutdown_connections 2024-09-18T14:03:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.394+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/3950609379 wait complete. 2024-09-18T14:03:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.394+0000 7f9ce6b5d640 10 monclient: build_initial_monmap 2024-09-18T14:03:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.394+0000 7f9ce6b5d640 10 monclient: monmap: 2024-09-18T14:03:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 1 Processor -- start 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 1 -- start start 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 10 monclient: init 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 10 monclient: _add_conns ranks=[5,0,2,6,8,3,4,7,1] 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 10 monclient(hunting): picked mon.noname-h con 0x7f9ce009f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 10 monclient(hunting): picked mon.noname-a con 0x7f9ce009fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 10 monclient(hunting): picked mon.noname-b con 0x7f9ce009ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce009d310 con 0x7f9ce009fba0 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce00a2560 con 0x7f9ce009ff70 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce009b9a0 con 0x7f9ce009f7d0 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:50.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9ce6b5d640 10 monclient(hunting): authenticate will time out at 1045.733722s 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9cdffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9ce009fba0 0x7f9ce0146cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44840/0 (socket says 172.21.15.45:44840) 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.395+0000 7f9cdffff640 1 -- 172.21.15.45:0/1425056941 learned_addr learned my addr 172.21.15.45:0/1425056941 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768395041 0 0) 0x7f9ce009d310 con 0x7f9ce009fba0 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): my global_id is 4878 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cc40034e0 con 0x7f9ce009fba0 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763282476 0 0) 0x7f9ce00a2560 con 0x7f9ce009ff70 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): my global_id is 144101 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce009d310 con 0x7f9ce009ff70 2024-09-18T14:03:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400878246 0 0) 0x7f9ce009b9a0 con 0x7f9ce009f7d0 2024-09-18T14:03:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): my global_id is 4712 2024-09-18T14:03:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce00a2560 con 0x7f9ce009f7d0 2024-09-18T14:03:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1118642938 0 0) 0x7f9cc40034e0 con 0x7f9ce009fba0 2024-09-18T14:03:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9ce009b9a0 con 0x7f9ce009fba0 2024-09-18T14:03:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9ccc002bf0 con 0x7f9ce009fba0 2024-09-18T14:03:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:03:50.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): dump: 2024-09-18T14:03:50.414 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.414 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.414 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.414 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.414 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.414 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.414 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.414 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1575926737 0 0) 0x7f9ce009d310 con 0x7f9ce009ff70 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9cc40034e0 con 0x7f9ce009ff70 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1109564898 0 0) 0x7f9ce00a2560 con 0x7f9ce009f7d0 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9ce009d310 con 0x7f9ce009f7d0 2024-09-18T14:03:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9cd8001cc0 con 0x7f9ce009ff70 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): dump: 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.417 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9cd4004ae0 con 0x7f9ce009f7d0 2024-09-18T14:03:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:03:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient(hunting): dump: 2024-09-18T14:03:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3653522224 0 0) 0x7f9ce009b9a0 con 0x7f9ce009fba0 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 >> v1:172.21.15.84:6791/0 conn(0x7f9ce009f7d0 legacy=0x7f9ce0131700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 >> v1:172.21.15.45:6790/0 conn(0x7f9ce009ff70 legacy=0x7f9ce014a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient: _finish_hunting 0 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 monclient: found mon.a 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce011e820 con 0x7f9ce009fba0 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient: _finish_auth 0 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9ce6b5d640 5 monclient: authenticate success, global_id 4878 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9ce6b5d640 10 monclient: _renew_subs 2024-09-18T14:03:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9ce6b5d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce011d8c0 con 0x7f9ce009fba0 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9ce6b5d640 10 monclient: _renew_subs 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9ce6b5d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce0131fb0 con 0x7f9ce009fba0 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9ccc001660 con 0x7f9ce009fba0 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9ccc001bd0 con 0x7f9ce009fba0 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdeffd640 4 set_mon_vals no callback set 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.396+0000 7f9cdcff9640 10 monclient: dump: 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.397+0000 7f9ce6b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.397+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9cc0006580 con 0x7f9cc0002830 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.397+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f9ccc001d30 con 0x7f9ce009fba0 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.397+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(227..227 src has 1..227) v4 ==== 3911+0+0 (unknown 472785454 0 0) 0x7f9ccc046ad0 con 0x7f9ce009fba0 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.397+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360946586 0 0) 0x7f9cc0006580 con 0x7f9cc0002830 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.397+0000 7f9cdcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.397+0000 7f9cdcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.397+0000 7f9cdcff9640 10 monclient(hunting): my global_id is 144110 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.397+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce009b9a0 con 0x7f9cc0002830 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.397+0000 7f9ce50fa640 10 monclient: get_auth_request con 0x7f9cc403abb0 auth_method 0 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 581435469 0 0) 0x7f9ce009b9a0 con 0x7f9cc0002830 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9cc0006580 con 0x7f9cc0002830 2024-09-18T14:03:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9cd4001770 con 0x7f9cc0002830 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 10 monclient: dump: 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2693053201 0 0) 0x7f9cc0006580 con 0x7f9cc0002830 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9cc000a0a0 con 0x7f9cc0002830 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9cd4006fc0 con 0x7f9cc0002830 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 >> v1:172.21.15.45:6790/0 conn(0x7f9cc0002830 legacy=0x7f9cc0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.398+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 >> v1:172.21.15.45:6790/0 conn(0x7f9cc0002830 legacy=0x7f9cc0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:50.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9ce6b5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:50.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9cc0007540 con 0x7f9cc00068f0 2024-09-18T14:03:50.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454731450 0 0) 0x7f9cc0007540 con 0x7f9cc00068f0 2024-09-18T14:03:50.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 10 monclient(hunting): my global_id is 144119 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cc0006580 con 0x7f9cc00068f0 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3290803315 0 0) 0x7f9cc0006580 con 0x7f9cc00068f0 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9cc0007540 con 0x7f9cc00068f0 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9cd40055f0 con 0x7f9cc00068f0 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 10 monclient: dump: 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.429 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.429 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3830284379 0 0) 0x7f9cc0007540 con 0x7f9cc00068f0 2024-09-18T14:03:50.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.404+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9cc00087a0 con 0x7f9cc00068f0 2024-09-18T14:03:50.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3414911727) 0x7f9cd4006aa0 con 0x7f9cc00068f0 2024-09-18T14:03:50.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9cdcff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:50.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9cdcff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:50.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 >> v1:172.21.15.45:6790/0 conn(0x7f9cc00068f0 legacy=0x7f9cc0006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9cdcff9640 1 -- 172.21.15.45:0/1425056941 >> v1:172.21.15.45:6790/0 conn(0x7f9cc00068f0 legacy=0x7f9cc0006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:50.429 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:50.449 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:50.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/1425056941 >> v1:172.21.15.84:6812/31478 conn(0x7f9cc403abb0 legacy=0x7f9cc403d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9ce6b5d640 10 monclient: shutdown 2024-09-18T14:03:50.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9ce6b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:50.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/1425056941 >> v1:172.21.15.45:6789/0 conn(0x7f9ce009fba0 legacy=0x7f9ce0146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9ce50fa640 1 -- 172.21.15.45:0/1425056941 reap_dead start 2024-09-18T14:03:50.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9ce50fa640 1 -- 172.21.15.45:0/1425056941 reap_dead start 2024-09-18T14:03:50.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/1425056941 shutdown_connections 2024-09-18T14:03:50.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/1425056941 >> 172.21.15.45:0/1425056941 conn(0x7f9ce00630b0 msgr2=0x7f9ce00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:50.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/1425056941 shutdown_connections 2024-09-18T14:03:50.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.405+0000 7f9ce6b5d640 1 -- 172.21.15.45:0/1425056941 wait complete. 2024-09-18T14:03:50.452 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:50.452 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:03:50.453 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:03:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.548+0000 7f5cb919d640 10 monclient: get_monmap_and_config 2024-09-18T14:03:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.548+0000 7f5cb919d640 10 monclient: build_initial_monmap 2024-09-18T14:03:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.548+0000 7f5cb919d640 10 monclient: monmap: 2024-09-18T14:03:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:50.549846+0000 2024-09-18T14:03:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:50.549846+0000 2024-09-18T14:03:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:50.551 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.549+0000 7f5cb919d640 1 Processor -- start 2024-09-18T14:03:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.549+0000 7f5cb919d640 1 -- start start 2024-09-18T14:03:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.549+0000 7f5cb919d640 10 monclient: init 2024-09-18T14:03:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb919d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb919d640 10 monclient: _add_conns ranks=[4,8,6,7,3,5,0,1,2] 2024-09-18T14:03:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb919d640 10 monclient(hunting): picked mon.noname-c con 0x7f5cb409f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb919d640 10 monclient(hunting): picked mon.noname-e con 0x7f5cb409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:03:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb919d640 10 monclient(hunting): picked mon.noname-d con 0x7f5cb409ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb919d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb405ed70 con 0x7f5cb409f7d0 2024-09-18T14:03:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb919d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb409dfc0 con 0x7f5cb409ff70 2024-09-18T14:03:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb919d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb409dce0 con 0x7f5cb409fba0 2024-09-18T14:03:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb919d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb2d9d640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f5cb409f7d0 0x7f5cb4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38362/0 (socket says 172.21.15.45:38362) 2024-09-18T14:03:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb259c640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f5cb409fba0 0x7f5cb4137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:49986/0 (socket says 172.21.15.45:49986) 2024-09-18T14:03:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb919d640 10 monclient(hunting): authenticate will time out at 1045.888321s 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb359e640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f5cb409ff70 0x7f5cb413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:58898/0 (socket says 172.21.15.45:58898) 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb259c640 1 -- 172.21.15.45:0/1819421075 learned_addr learned my addr 172.21.15.45:0/1819421075 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529244704 0 0) 0x7f5cb409dfc0 con 0x7f5cb409ff70 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient(hunting): my global_id is 4695 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c9c0034e0 con 0x7f5cb409ff70 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473395190 0 0) 0x7f5cb405ed70 con 0x7f5cb409f7d0 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient(hunting): my global_id is 144130 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb409dfc0 con 0x7f5cb409f7d0 2024-09-18T14:03:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771175646 0 0) 0x7f5cb409dce0 con 0x7f5cb409fba0 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient(hunting): my global_id is 4859 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb405ed70 con 0x7f5cb409fba0 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2388843020 0 0) 0x7f5c9c0034e0 con 0x7f5cb409ff70 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 >> v1:172.21.15.45:6793/0 conn(0x7f5cb409fba0 legacy=0x7f5cb4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 >> v1:172.21.15.45:6791/0 conn(0x7f5cb409f7d0 legacy=0x7f5cb4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 1 monclient: found mon.noname-d 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb40a1c00 con 0x7f5cb409ff70 2024-09-18T14:03:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient: _finish_auth 0 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5cac001bb0 con 0x7f5cb409ff70 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb919d640 5 monclient: authenticate success, global_id 4695 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb919d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.550+0000 7f5cb1d9b640 10 monclient: dump: 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.564 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb919d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5cac001d10 con 0x7f5cb409ff70 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb1d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb1d9b640 1 -- 172.21.15.45:0/1819421075 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5cac004e60 con 0x7f5cb409ff70 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb919d640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb919d640 4 set_mon_vals no callback set 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb1d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb1d9b640 10 monclient: dump: 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb919d640 10 monclient: shutdown 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb919d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb919d640 1 -- 172.21.15.45:0/1819421075 >> v1:172.21.15.45:6792/0 conn(0x7f5cb409ff70 legacy=0x7f5cb413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb919d640 1 -- 172.21.15.45:0/1819421075 shutdown_connections 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb919d640 1 -- 172.21.15.45:0/1819421075 >> 172.21.15.45:0/1819421075 conn(0x7f5cb40630b0 msgr2=0x7f5cb40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb919d640 1 -- 172.21.15.45:0/1819421075 shutdown_connections 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.551+0000 7f5cb919d640 1 -- 172.21.15.45:0/1819421075 wait complete. 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.552+0000 7f5cb919d640 10 monclient: build_initial_monmap 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.552+0000 7f5cb919d640 10 monclient: monmap: 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.552+0000 7f5cb919d640 1 Processor -- start 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.552+0000 7f5cb919d640 1 -- start start 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.552+0000 7f5cb919d640 10 monclient: init 2024-09-18T14:03:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.552+0000 7f5cb919d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.552+0000 7f5cb919d640 10 monclient: _add_conns ranks=[8,7,0,1,2,6,5,4,3] 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 10 monclient(hunting): picked mon.noname-e con 0x7f5cb409f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 10 monclient(hunting): picked mon.noname-i con 0x7f5cb409fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 10 monclient(hunting): picked mon.noname-a con 0x7f5cb409ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb409d310 con 0x7f5cb409ff70 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb40a2560 con 0x7f5cb409fba0 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb411d620 con 0x7f5cb409f7d0 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 10 monclient(hunting): authenticate will time out at 1045.891060s 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb2d9d640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f5cb409f7d0 0x7f5cb409a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:49994/0 (socket says 172.21.15.45:49994) 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb2d9d640 1 -- 172.21.15.45:0/2255237478 learned_addr learned my addr 172.21.15.45:0/2255237478 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759410453 0 0) 0x7f5cb409d310 con 0x7f5cb409ff70 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): my global_id is 4887 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ca00034e0 con 0x7f5cb409ff70 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520627515 0 0) 0x7f5cb411d620 con 0x7f5cb409f7d0 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): my global_id is 4868 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb409d310 con 0x7f5cb409f7d0 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 70227515 0 0) 0x7f5cb409d310 con 0x7f5cb409f7d0 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5cb411d620 con 0x7f5cb409f7d0 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 729170928 0 0) 0x7f5ca00034e0 con 0x7f5cb409ff70 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5cb409d310 con 0x7f5cb409ff70 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5ca40014f0 con 0x7f5cb409f7d0 2024-09-18T14:03:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): dump: 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.572 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5cac001af0 con 0x7f5cb409ff70 2024-09-18T14:03:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): dump: 2024-09-18T14:03:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.573 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906626055 0 0) 0x7f5cb40a2560 con 0x7f5cb409fba0 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient(hunting): my global_id is 4804 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ca00034e0 con 0x7f5cb409fba0 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4062819522 0 0) 0x7f5cb411d620 con 0x7f5cb409f7d0 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 >> v1:172.21.15.84:6792/0 conn(0x7f5cb409fba0 legacy=0x7f5cb409ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 >> v1:172.21.15.45:6789/0 conn(0x7f5cb409ff70 legacy=0x7f5cb4149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient: _finish_hunting 0 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 monclient: found mon.e 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:03:50.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb409be30 con 0x7f5cb409f7d0 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient: _finish_auth 0 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 5 monclient: authenticate success, global_id 4868 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 10 monclient: _renew_subs 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb411d3a0 con 0x7f5cb409f7d0 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 10 monclient: _renew_subs 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb919d640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb412f510 con 0x7f5cb409f7d0 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5ca4001790 con 0x7f5cb409f7d0 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5ca4004d90 con 0x7f5cb409f7d0 2024-09-18T14:03:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5c9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.553+0000 7f5cb159a640 4 set_mon_vals no callback set 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.554+0000 7f5c9affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.554+0000 7f5c9affd640 10 monclient: dump: 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.554+0000 7f5cb919d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.554+0000 7f5cb919d640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5c94006580 con 0x7f5c94002830 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.554+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f5ca400fd60 con 0x7f5cb409f7d0 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(227..227 src has 1..227) v4 ==== 3911+0+0 (unknown 472785454 0 0) 0x7f5ca40469e0 con 0x7f5cb409f7d0 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104041822 0 0) 0x7f5c94006580 con 0x7f5c94002830 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 10 monclient(hunting): my global_id is 144139 2024-09-18T14:03:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb411d620 con 0x7f5c94002830 2024-09-18T14:03:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5cb359e640 10 monclient: get_auth_request con 0x7f5ca003a640 auth_method 0 2024-09-18T14:03:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2519601577 0 0) 0x7f5cb411d620 con 0x7f5c94002830 2024-09-18T14:03:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c94006580 con 0x7f5c94002830 2024-09-18T14:03:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5ca8001320 con 0x7f5c94002830 2024-09-18T14:03:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 10 monclient: dump: 2024-09-18T14:03:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 447857071 0 0) 0x7f5c94006580 con 0x7f5c94002830 2024-09-18T14:03:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c9400a0a0 con 0x7f5c94002830 2024-09-18T14:03:50.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5ca80047a0 con 0x7f5c94002830 2024-09-18T14:03:50.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:50.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:50.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 >> v1:172.21.15.45:6791/0 conn(0x7f5c94002830 legacy=0x7f5c94004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.555+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 >> v1:172.21.15.45:6791/0 conn(0x7f5c94002830 legacy=0x7f5c94004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:50.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.561+0000 7f5cb919d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:50.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.561+0000 7f5cb919d640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5c94009660 con 0x7f5c94009040 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.561+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102499538 0 0) 0x7f5c94009660 con 0x7f5c94009040 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.561+0000 7f5c9affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.561+0000 7f5c9affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.561+0000 7f5c9affd640 10 monclient(hunting): my global_id is 144148 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.561+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c94006580 con 0x7f5c94009040 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.561+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3797577640 0 0) 0x7f5c94006580 con 0x7f5c94009040 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c94009660 con 0x7f5c94009040 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5ca8001d40 con 0x7f5c94009040 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5c9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5c9affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5c9affd640 10 monclient: dump: 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.582 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.584 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 632175256 0 0) 0x7f5c94009660 con 0x7f5c94009040 2024-09-18T14:03:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5c94009920 con 0x7f5c94009040 2024-09-18T14:03:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 797811824) 0x7f5ca8007040 con 0x7f5c94009040 2024-09-18T14:03:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5c9affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5c9affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 >> v1:172.21.15.45:6791/0 conn(0x7f5c94009040 legacy=0x7f5c94008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5c9affd640 1 -- 172.21.15.45:0/2255237478 >> v1:172.21.15.45:6791/0 conn(0x7f5c94009040 legacy=0x7f5c94008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:50.584 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:50.585 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:50.586 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:50.586 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.586 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:03:50.586 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:50.586 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:50.586 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:50.586 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.586 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.586 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.586 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.586 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.586 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.587 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.587 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.587 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.587 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.587 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:50.587 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:50.587 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.587 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.587 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:50.588 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:50.589 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:50.589 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:50.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:50.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:50.589 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:50.589 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:50.589 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:50.589 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:50.589 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:50.589 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:50.589 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:50.589 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:50.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.591 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:50.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:50.591 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.591 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.591 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:50.591 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.591 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.591 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.591 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.592 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:50.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.594 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.595 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.596 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.597 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:50.598 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.599 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.601 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:50.601 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.601 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.601 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.601 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:50.601 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:50.601 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.603 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.604 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:50.605 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5cb919d640 1 -- 172.21.15.45:0/2255237478 >> v1:172.21.15.84:6812/31478 conn(0x7f5ca003a640 legacy=0x7f5ca003caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5cb919d640 10 monclient: shutdown 2024-09-18T14:03:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5cb919d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5cb919d640 1 -- 172.21.15.45:0/2255237478 >> v1:172.21.15.45:6793/0 conn(0x7f5cb409f7d0 legacy=0x7f5cb409a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5cb359e640 1 -- 172.21.15.45:0/2255237478 reap_dead start 2024-09-18T14:03:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5cb359e640 1 -- 172.21.15.45:0/2255237478 reap_dead start 2024-09-18T14:03:50.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5cb919d640 1 -- 172.21.15.45:0/2255237478 shutdown_connections 2024-09-18T14:03:50.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5cb919d640 1 -- 172.21.15.45:0/2255237478 >> 172.21.15.45:0/2255237478 conn(0x7f5cb40630b0 msgr2=0x7f5cb4130b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:50.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5cb919d640 1 -- 172.21.15.45:0/2255237478 shutdown_connections 2024-09-18T14:03:50.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.562+0000 7f5cb919d640 1 -- 172.21.15.45:0/2255237478 wait complete. 2024-09-18T14:03:50.607 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:50.607 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:03:50.607 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:03:50.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.704+0000 7fae85b1d640 10 monclient: get_monmap_and_config 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.704+0000 7fae85b1d640 10 monclient: build_initial_monmap 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.704+0000 7fae85b1d640 10 monclient: monmap: 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:50.705102+0000 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:50.705102+0000 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:50.705 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:50.706 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:50.706 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.704+0000 7fae85b1d640 1 Processor -- start 2024-09-18T14:03:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.704+0000 7fae85b1d640 1 -- start start 2024-09-18T14:03:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.704+0000 7fae85b1d640 10 monclient: init 2024-09-18T14:03:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae85b1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae85b1d640 10 monclient: _add_conns ranks=[1,7,6,4,3,0,5,8,2] 2024-09-18T14:03:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae85b1d640 10 monclient(hunting): picked mon.noname-f con 0x7fae8006e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae85b1d640 10 monclient(hunting): picked mon.noname-i con 0x7fae800a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae85b1d640 10 monclient(hunting): picked mon.noname-d con 0x7fae800a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae85b1d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8005ed70 con 0x7fae8006e810 2024-09-18T14:03:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae85b1d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae80097620 con 0x7fae800a1080 2024-09-18T14:03:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae85b1d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae80097340 con 0x7fae800a0cb0 2024-09-18T14:03:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae85b1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae85b1d640 10 monclient(hunting): authenticate will time out at 1046.043454s 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7ffff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fae800a1080 0x7fae8013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:58900/0 (socket says 172.21.15.45:58900) 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7ffff640 1 -- 172.21.15.45:0/2603842084 learned_addr learned my addr 172.21.15.45:0/2603842084 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276603363 0 0) 0x7fae80097620 con 0x7fae800a1080 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 10 monclient(hunting): my global_id is 4704 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae6c0034e0 con 0x7fae800a1080 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028480649 0 0) 0x7fae80097340 con 0x7fae800a0cb0 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 10 monclient(hunting): my global_id is 4813 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae80097620 con 0x7fae800a0cb0 2024-09-18T14:03:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751863167 0 0) 0x7fae8005ed70 con 0x7fae8006e810 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 10 monclient(hunting): my global_id is 4753 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.705+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae80097340 con 0x7fae8006e810 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1218098556 0 0) 0x7fae6c0034e0 con 0x7fae800a1080 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 >> v1:172.21.15.84:6792/0 conn(0x7fae800a0cb0 legacy=0x7fae80137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 >> v1:172.21.15.84:6789/0 conn(0x7fae8006e810 legacy=0x7fae800a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 1 monclient: found mon.noname-d 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae800a1660 con 0x7fae800a1080 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:03:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae85b1d640 5 monclient: authenticate success, global_id 4704 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae78001bb0 con 0x7fae800a1080 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae85b1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 10 monclient: dump: 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.718 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae85b1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fae78001d90 con 0x7fae800a1080 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae85b1d640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:50.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 1 -- 172.21.15.45:0/2603842084 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae78004f20 con 0x7fae800a1080 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae85b1d640 4 set_mon_vals no callback set 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae7e7fc640 10 monclient: dump: 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae85b1d640 10 monclient: shutdown 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae85b1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae85b1d640 1 -- 172.21.15.45:0/2603842084 >> v1:172.21.15.45:6792/0 conn(0x7fae800a1080 legacy=0x7fae8013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae85b1d640 1 -- 172.21.15.45:0/2603842084 shutdown_connections 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae85b1d640 1 -- 172.21.15.45:0/2603842084 >> 172.21.15.45:0/2603842084 conn(0x7fae800a5bb0 msgr2=0x7fae800a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae85b1d640 1 -- 172.21.15.45:0/2603842084 shutdown_connections 2024-09-18T14:03:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.706+0000 7fae85b1d640 1 -- 172.21.15.45:0/2603842084 wait complete. 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.707+0000 7fae85b1d640 10 monclient: build_initial_monmap 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.707+0000 7fae85b1d640 10 monclient: monmap: 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.707+0000 7fae85b1d640 1 Processor -- start 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.707+0000 7fae85b1d640 1 -- start start 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.707+0000 7fae85b1d640 10 monclient: init 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae85b1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae85b1d640 10 monclient: _add_conns ranks=[1,6,5,2,7,4,0,3,8] 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae85b1d640 10 monclient(hunting): picked mon.noname-f con 0x7fae8006e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae85b1d640 10 monclient(hunting): picked mon.noname-d con 0x7fae800a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae85b1d640 10 monclient(hunting): picked mon.noname-h con 0x7fae800a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae85b1d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae80122f20 con 0x7fae8006e810 2024-09-18T14:03:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae85b1d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8013dd10 con 0x7fae800a1080 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae85b1d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae80122960 con 0x7fae800a0cb0 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae85b1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae85b1d640 10 monclient(hunting): authenticate will time out at 1046.046217s 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae7effd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fae800a0cb0 0x7fae80146830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:58904/0 (socket says 172.21.15.45:58904) 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae7effd640 1 -- 172.21.15.45:0/766691657 learned_addr learned my addr 172.21.15.45:0/766691657 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571913113 0 0) 0x7fae80122960 con 0x7fae800a0cb0 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): my global_id is 4713 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae600034e0 con 0x7fae800a0cb0 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776241462 0 0) 0x7fae80122f20 con 0x7fae8006e810 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): my global_id is 4762 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae80122960 con 0x7fae8006e810 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304340955 0 0) 0x7fae8013dd10 con 0x7fae800a1080 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): my global_id is 4721 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae80122f20 con 0x7fae800a1080 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1479283515 0 0) 0x7fae600034e0 con 0x7fae800a0cb0 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fae8013dd10 con 0x7fae800a0cb0 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae70002c10 con 0x7fae800a0cb0 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:03:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): dump: 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2090516220 0 0) 0x7fae80122960 con 0x7fae8006e810 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fae600034e0 con 0x7fae8006e810 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae68001a60 con 0x7fae8006e810 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 10 monclient(hunting): dump: 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.727 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 380548985 0 0) 0x7fae80122f20 con 0x7fae800a1080 2024-09-18T14:03:50.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.708+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fae80122960 con 0x7fae800a1080 2024-09-18T14:03:50.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae78001b70 con 0x7fae800a1080 2024-09-18T14:03:50.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:03:50.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 10 monclient(hunting): dump: 2024-09-18T14:03:50.729 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.729 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.729 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.729 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.729 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 168318915 0 0) 0x7fae8013dd10 con 0x7fae800a0cb0 2024-09-18T14:03:50.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 >> v1:172.21.15.84:6791/0 conn(0x7fae800a1080 legacy=0x7fae80149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 >> v1:172.21.15.84:6789/0 conn(0x7fae8006e810 legacy=0x7fae8013d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 10 monclient: _finish_hunting 0 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 1 monclient: found mon.d 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80120470 con 0x7fae800a0cb0 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 10 monclient: _finish_auth 0 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae85b1d640 5 monclient: authenticate success, global_id 4713 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae85b1d640 10 monclient: _renew_subs 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae85b1d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae85b1d640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae80122680 con 0x7fae800a0cb0 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae85b1d640 10 monclient: _renew_subs 2024-09-18T14:03:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae85b1d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae85b1d640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fae801217b0 con 0x7fae800a0cb0 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fae70001760 con 0x7fae800a0cb0 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae70001c90 con 0x7fae800a0cb0 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae7dffb640 4 set_mon_vals no callback set 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae777fe640 10 monclient: dump: 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae85b1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:50.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.709+0000 7fae85b1d640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fae64006580 con 0x7fae64002830 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fae70001e70 con 0x7fae800a0cb0 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(227..227 src has 1..227) v4 ==== 3911+0+0 (unknown 472785454 0 0) 0x7fae70046cb0 con 0x7fae800a0cb0 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654869205 0 0) 0x7fae64006580 con 0x7fae64002830 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 10 monclient(hunting): my global_id is 4722 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae8013dd10 con 0x7fae64002830 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae7ffff640 10 monclient: get_auth_request con 0x7fae6003a9f0 auth_method 0 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1939301932 0 0) 0x7fae8013dd10 con 0x7fae64002830 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fae64006580 con 0x7fae64002830 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae68002130 con 0x7fae64002830 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 10 monclient: dump: 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.736 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3476920421 0 0) 0x7fae64006580 con 0x7fae64002830 2024-09-18T14:03:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.710+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae6400a0a0 con 0x7fae64002830 2024-09-18T14:03:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.711+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fae68004410 con 0x7fae64002830 2024-09-18T14:03:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.711+0000 7fae777fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.711+0000 7fae777fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.711+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 >> v1:172.21.15.45:6792/0 conn(0x7fae64002830 legacy=0x7fae64004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.711+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 >> v1:172.21.15.45:6792/0 conn(0x7fae64002830 legacy=0x7fae64004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.716+0000 7fae85b1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.716+0000 7fae85b1d640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fae64009e70 con 0x7fae640068f0 2024-09-18T14:03:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438066885 0 0) 0x7fae64009e70 con 0x7fae640068f0 2024-09-18T14:03:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 10 monclient(hunting): my global_id is 4731 2024-09-18T14:03:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae64006580 con 0x7fae640068f0 2024-09-18T14:03:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3574117558 0 0) 0x7fae64006580 con 0x7fae640068f0 2024-09-18T14:03:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fae64009e70 con 0x7fae640068f0 2024-09-18T14:03:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae68002b70 con 0x7fae640068f0 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 10 monclient: dump: 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.740 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2254646673 0 0) 0x7fae64009e70 con 0x7fae640068f0 2024-09-18T14:03:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fae64008750 con 0x7fae640068f0 2024-09-18T14:03:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 1284204747) 0x7fae68004020 con 0x7fae640068f0 2024-09-18T14:03:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 >> v1:172.21.15.45:6792/0 conn(0x7fae640068f0 legacy=0x7fae64006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.717+0000 7fae777fe640 1 -- 172.21.15.45:0/766691657 >> v1:172.21.15.45:6792/0 conn(0x7fae640068f0 legacy=0x7fae64006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:50.741 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:50.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:50.741 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:50.741 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:50.741 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:50.741 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:50.741 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:50.741 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:50.741 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:50.741 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:50.741 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:50.742 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:50.742 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:50.742 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:50.742 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:50.742 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.742 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:03:50.742 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:50.742 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:50.742 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:50.742 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.742 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.742 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:50.744 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.745 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.749 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:50.750 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.751 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.751 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.751 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:50.751 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:50.751 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.751 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.754 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:50.755 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.756 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.756 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:50.756 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.756 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.756 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.756 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:50.756 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:50.756 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.756 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.757 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.759 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:50.759 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.759 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.759 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.759 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:50.759 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:50.759 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.759 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.759 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.759 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.759 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:03:50.761 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.761 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.761 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:50.761 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:50.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.718+0000 7fae85b1d640 1 -- 172.21.15.45:0/766691657 >> v1:172.21.15.84:6812/31478 conn(0x7fae6003a9f0 legacy=0x7fae6003cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.718+0000 7fae85b1d640 10 monclient: shutdown 2024-09-18T14:03:50.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.718+0000 7fae85b1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:50.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.718+0000 7fae85b1d640 1 -- 172.21.15.45:0/766691657 >> v1:172.21.15.45:6792/0 conn(0x7fae800a0cb0 legacy=0x7fae80146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.718+0000 7fae7ffff640 1 -- 172.21.15.45:0/766691657 reap_dead start 2024-09-18T14:03:50.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.718+0000 7fae7ffff640 1 -- 172.21.15.45:0/766691657 reap_dead start 2024-09-18T14:03:50.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.718+0000 7fae85b1d640 1 -- 172.21.15.45:0/766691657 shutdown_connections 2024-09-18T14:03:50.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.718+0000 7fae85b1d640 1 -- 172.21.15.45:0/766691657 >> 172.21.15.45:0/766691657 conn(0x7fae800a5bb0 msgr2=0x7fae800a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:50.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.718+0000 7fae85b1d640 1 -- 172.21.15.45:0/766691657 shutdown_connections 2024-09-18T14:03:50.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.718+0000 7fae85b1d640 1 -- 172.21.15.45:0/766691657 wait complete. 2024-09-18T14:03:50.763 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:50.763 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:03:50.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:03:50.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.864+0000 7f187e634640 10 monclient: get_monmap_and_config 2024-09-18T14:03:50.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.864+0000 7f187e634640 10 monclient: build_initial_monmap 2024-09-18T14:03:50.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.864+0000 7f187e634640 10 monclient: monmap: 2024-09-18T14:03:50.866 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:50.866 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.866 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:50.865395+0000 2024-09-18T14:03:50.866 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:50.865395+0000 2024-09-18T14:03:50.867 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:50.867 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.867 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:50.867 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:50.867 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:50.868 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:50.868 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:50.868 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:50.868 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:50.868 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:50.869 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:50.869 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 1 Processor -- start 2024-09-18T14:03:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 1 -- start start 2024-09-18T14:03:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 10 monclient: init 2024-09-18T14:03:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 10 monclient: _add_conns ranks=[0,5,1,4,8,2,6,3,7] 2024-09-18T14:03:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 10 monclient(hunting): picked mon.noname-a con 0x7f187809f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 10 monclient(hunting): picked mon.noname-h con 0x7f187809fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 10 monclient(hunting): picked mon.noname-f con 0x7f187809ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187805ed70 con 0x7f187809f7d0 2024-09-18T14:03:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187809dfc0 con 0x7f187809ff70 2024-09-18T14:03:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187809dce0 con 0x7f187809fba0 2024-09-18T14:03:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f187e634640 10 monclient(hunting): authenticate will time out at 1046.203779s 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f1877fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f187809f7d0 0x7f1878132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44858/0 (socket says 172.21.15.45:44858) 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.865+0000 7f1877fff640 1 -- 172.21.15.45:0/3952404831 learned_addr learned my addr 172.21.15.45:0/3952404831 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368224765 0 0) 0x7f187805ed70 con 0x7f187809f7d0 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient(hunting): my global_id is 4896 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18600034e0 con 0x7f187809f7d0 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175532076 0 0) 0x7f187809dce0 con 0x7f187809fba0 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient(hunting): my global_id is 4730 2024-09-18T14:03:50.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f187805ed70 con 0x7f187809fba0 2024-09-18T14:03:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362543395 0 0) 0x7f187809dfc0 con 0x7f187809ff70 2024-09-18T14:03:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient(hunting): my global_id is 4771 2024-09-18T14:03:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f187809dce0 con 0x7f187809ff70 2024-09-18T14:03:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2426987063 0 0) 0x7f18600034e0 con 0x7f187809f7d0 2024-09-18T14:03:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 >> v1:172.21.15.84:6791/0 conn(0x7f187809fba0 legacy=0x7f1878137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 >> v1:172.21.15.84:6789/0 conn(0x7f187809ff70 legacy=0x7f187813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:03:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 monclient: found mon.noname-a 2024-09-18T14:03:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:03:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18780a1c00 con 0x7f187809f7d0 2024-09-18T14:03:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient: _finish_auth 0 2024-09-18T14:03:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1868001830 con 0x7f187809f7d0 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient: dump: 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f187e634640 5 monclient: authenticate success, global_id 4896 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f187e634640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1868001a10 con 0x7f187809f7d0 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f187e634640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 1 -- 172.21.15.45:0/3952404831 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1868004ba0 con 0x7f187809f7d0 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f187e634640 4 set_mon_vals no callback set 2024-09-18T14:03:50.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f1876ffd640 10 monclient: dump: 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.888 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f187e634640 10 monclient: shutdown 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f187e634640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f187e634640 1 -- 172.21.15.45:0/3952404831 >> v1:172.21.15.45:6789/0 conn(0x7f187809f7d0 legacy=0x7f1878132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f187e634640 1 -- 172.21.15.45:0/3952404831 shutdown_connections 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.866+0000 7f187e634640 1 -- 172.21.15.45:0/3952404831 >> 172.21.15.45:0/3952404831 conn(0x7f18780630b0 msgr2=0x7f18780634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.867+0000 7f187e634640 1 -- 172.21.15.45:0/3952404831 shutdown_connections 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.867+0000 7f187e634640 1 -- 172.21.15.45:0/3952404831 wait complete. 2024-09-18T14:03:50.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.867+0000 7f187e634640 10 monclient: build_initial_monmap 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.867+0000 7f187e634640 10 monclient: monmap: 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:50.890 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.867+0000 7f187e634640 1 Processor -- start 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f187e634640 1 -- start start 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f187e634640 10 monclient: init 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f187e634640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f187e634640 10 monclient: _add_conns ranks=[1,4,3,6,7,8,5,0,2] 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f187e634640 10 monclient(hunting): picked mon.noname-f con 0x7f187809f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f187e634640 10 monclient(hunting): picked mon.noname-c con 0x7f187809fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f187e634640 10 monclient(hunting): picked mon.noname-g con 0x7f187809ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f187e634640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18780a2560 con 0x7f187809f7d0 2024-09-18T14:03:50.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f187e634640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187809d310 con 0x7f187809ff70 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f187e634640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1878142050 con 0x7f187809fba0 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f187e634640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f187e634640 10 monclient(hunting): authenticate will time out at 1046.206590s 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f18777fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f187809fba0 0x7f1878131b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38382/0 (socket says 172.21.15.45:38382) 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f18777fe640 1 -- 172.21.15.45:0/1611771443 learned_addr learned my addr 172.21.15.45:0/1611771443 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998053477 0 0) 0x7f1878142050 con 0x7f187809fba0 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f185ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f185ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f185ffff640 10 monclient(hunting): my global_id is 144157 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.868+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18640034e0 con 0x7f187809fba0 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3624685061 0 0) 0x7f18640034e0 con 0x7f187809fba0 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1878142050 con 0x7f187809fba0 2024-09-18T14:03:50.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f186c002c10 con 0x7f187809fba0 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient(hunting): dump: 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606919435 0 0) 0x7f18780a2560 con 0x7f187809f7d0 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient(hunting): my global_id is 4780 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18640034e0 con 0x7f187809f7d0 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965054511 0 0) 0x7f187809d310 con 0x7f187809ff70 2024-09-18T14:03:50.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient(hunting): my global_id is 144111 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18780a2560 con 0x7f187809ff70 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2708164295 0 0) 0x7f1878142050 con 0x7f187809fba0 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 >> v1:172.21.15.84:6790/0 conn(0x7f187809ff70 legacy=0x7f1878149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 >> v1:172.21.15.84:6789/0 conn(0x7f187809f7d0 legacy=0x7f1878141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient: _finish_hunting 0 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 monclient: found mon.c 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1878122540 con 0x7f187809fba0 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient: _finish_auth 0 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:50.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f187e634640 5 monclient: authenticate success, global_id 144157 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f187e634640 10 monclient: _renew_subs 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f187e634640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f187e634640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18781208c0 con 0x7f187809fba0 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f187e634640 10 monclient: _renew_subs 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f187e634640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f187e634640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1878120170 con 0x7f187809fba0 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f186c0017a0 con 0x7f187809fba0 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f186c001cd0 con 0x7f187809fba0 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f185ffff640 10 monclient: dump: 2024-09-18T14:03:50.896 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.897 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.897 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.897 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.897 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.897 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.897 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.897 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.897 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.897 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.897 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.897 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.897 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.869+0000 7f18767fc640 4 set_mon_vals no callback set 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f186c001eb0 con 0x7f187809fba0 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f187e634640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f187e634640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f187804c4a0 con 0x7f1878132260 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(227..227 src has 1..227) v4 ==== 3911+0+0 (unknown 472785454 0 0) 0x7f186c046c30 con 0x7f187809fba0 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f187cbd1640 10 monclient: get_auth_request con 0x7f1864039f70 auth_method 0 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880802217 0 0) 0x7f187804c4a0 con 0x7f1878132260 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 10 monclient(hunting): my global_id is 4877 2024-09-18T14:03:50.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1878142050 con 0x7f1878132260 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3814540485 0 0) 0x7f1878142050 con 0x7f1878132260 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f187804c4a0 con 0x7f1878132260 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1868004ae0 con 0x7f1878132260 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 10 monclient: dump: 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.899 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.870+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2902834895 0 0) 0x7f187804c4a0 con 0x7f1878132260 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.871+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f187804b720 con 0x7f1878132260 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.871+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f18680014a0 con 0x7f1878132260 2024-09-18T14:03:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.871+0000 7f185ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.871+0000 7f185ffff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.871+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 >> v1:172.21.15.45:6793/0 conn(0x7f1878132260 legacy=0x7f1878099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.871+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 >> v1:172.21.15.45:6793/0 conn(0x7f1878132260 legacy=0x7f1878099a40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f187e634640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f187e634640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1878071550 con 0x7f187802a8d0 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073282348 0 0) 0x7f1878071550 con 0x7f187802a8d0 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 10 monclient(hunting): my global_id is 4886 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f187804c4a0 con 0x7f187802a8d0 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2990656655 0 0) 0x7f187804c4a0 con 0x7f187802a8d0 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1878071550 con 0x7f187802a8d0 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1868005340 con 0x7f187802a8d0 2024-09-18T14:03:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 10 monclient: dump: 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:50.902 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:50.903 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:50.903 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:50.903 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:50.903 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:50.903 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:50.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2420461372 0 0) 0x7f1878071550 con 0x7f187802a8d0 2024-09-18T14:03:50.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.877+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f187809d670 con 0x7f187802a8d0 2024-09-18T14:03:50.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7558 (unknown 0 0 342005672) 0x7f1868007560 con 0x7f187802a8d0 2024-09-18T14:03:50.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f185ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:50.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f185ffff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:50.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 >> v1:172.21.15.45:6793/0 conn(0x7f187802a8d0 legacy=0x7f187809a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f185ffff640 1 -- 172.21.15.45:0/1611771443 >> v1:172.21.15.45:6793/0 conn(0x7f187802a8d0 legacy=0x7f187809a150 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:50.904 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:03:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:03:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:03:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:50.925 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f187e634640 1 -- 172.21.15.45:0/1611771443 >> v1:172.21.15.84:6812/31478 conn(0x7f1864039f70 legacy=0x7f186403c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f187e634640 10 monclient: shutdown 2024-09-18T14:03:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f187e634640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f187e634640 1 -- 172.21.15.45:0/1611771443 >> v1:172.21.15.45:6791/0 conn(0x7f187809fba0 legacy=0x7f1878131b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f187cbd1640 1 -- 172.21.15.45:0/1611771443 reap_dead start 2024-09-18T14:03:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f187cbd1640 1 -- 172.21.15.45:0/1611771443 reap_dead start 2024-09-18T14:03:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f187e634640 1 -- 172.21.15.45:0/1611771443 shutdown_connections 2024-09-18T14:03:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f187e634640 1 -- 172.21.15.45:0/1611771443 >> 172.21.15.45:0/1611771443 conn(0x7f18780630b0 msgr2=0x7f18780a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f187e634640 1 -- 172.21.15.45:0/1611771443 shutdown_connections 2024-09-18T14:03:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:50.878+0000 7f187e634640 1 -- 172.21.15.45:0/1611771443 wait complete. 2024-09-18T14:03:50.928 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:50.928 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:03:50.928 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:03:51.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.029+0000 7fecba35d640 10 monclient: get_monmap_and_config 2024-09-18T14:03:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.029+0000 7fecba35d640 10 monclient: build_initial_monmap 2024-09-18T14:03:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.029+0000 7fecba35d640 10 monclient: monmap: 2024-09-18T14:03:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:51.030439+0000 2024-09-18T14:03:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:51.030439+0000 2024-09-18T14:03:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:51.031 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 1 Processor -- start 2024-09-18T14:03:51.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 1 -- start start 2024-09-18T14:03:51.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 10 monclient: init 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 10 monclient: _add_conns ranks=[6,3,8,5,1,4,2,0,7] 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 10 monclient(hunting): picked mon.noname-d con 0x7fecb409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 10 monclient(hunting): picked mon.noname-g con 0x7fecb409fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 10 monclient(hunting): picked mon.noname-e con 0x7fecb409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb405ed70 con 0x7fecb409fba0 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb409dfc0 con 0x7fecb409f7d0 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb409dce0 con 0x7fecb409ff70 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecba35d640 10 monclient(hunting): authenticate will time out at 1046.368853s 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecb88fa640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fecb409ff70 0x7fecb413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:50028/0 (socket says 172.21.15.45:50028) 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecb88fa640 1 -- 172.21.15.45:0/2012120068 learned_addr learned my addr 172.21.15.45:0/2012120068 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.030+0000 7fecb3fff640 1 --1- 172.21.15.45:0/2012120068 >> v1:172.21.15.45:6792/0 conn(0x7fecb409f7d0 0x7fecb4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:58938/0 (socket says 172.21.15.45:58938) 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392310345 0 0) 0x7fecb409dce0 con 0x7fecb409ff70 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient(hunting): my global_id is 4895 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec9c0034e0 con 0x7fecb409ff70 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472061871 0 0) 0x7fecb409dfc0 con 0x7fecb409f7d0 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient(hunting): my global_id is 4740 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb409dce0 con 0x7fecb409f7d0 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620640089 0 0) 0x7fecb405ed70 con 0x7fecb409fba0 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient(hunting): my global_id is 144120 2024-09-18T14:03:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb409dfc0 con 0x7fecb409fba0 2024-09-18T14:03:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 539551224 0 0) 0x7fecb409dce0 con 0x7fecb409f7d0 2024-09-18T14:03:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 >> v1:172.21.15.45:6793/0 conn(0x7fecb409ff70 legacy=0x7fecb413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 >> v1:172.21.15.84:6790/0 conn(0x7fecb409fba0 legacy=0x7fecb4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:03:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 monclient: found mon.noname-d 2024-09-18T14:03:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:03:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb40a1c00 con 0x7fecb409f7d0 2024-09-18T14:03:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient: _finish_auth 0 2024-09-18T14:03:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecba35d640 5 monclient: authenticate success, global_id 4740 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feca4001810 con 0x7fecb409f7d0 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecba35d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient: dump: 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecba35d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feca4001970 con 0x7fecb409f7d0 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecba35d640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 1 -- 172.21.15.45:0/2012120068 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feca4004ae0 con 0x7fecb409f7d0 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecba35d640 4 set_mon_vals no callback set 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.031+0000 7fecb2ffd640 10 monclient: dump: 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.032+0000 7fecba35d640 10 monclient: shutdown 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.032+0000 7fecba35d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.032+0000 7fecba35d640 1 -- 172.21.15.45:0/2012120068 >> v1:172.21.15.45:6792/0 conn(0x7fecb409f7d0 legacy=0x7fecb4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.032+0000 7fecba35d640 1 -- 172.21.15.45:0/2012120068 shutdown_connections 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.032+0000 7fecba35d640 1 -- 172.21.15.45:0/2012120068 >> 172.21.15.45:0/2012120068 conn(0x7fecb40630b0 msgr2=0x7fecb40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.032+0000 7fecba35d640 1 -- 172.21.15.45:0/2012120068 shutdown_connections 2024-09-18T14:03:51.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.032+0000 7fecba35d640 1 -- 172.21.15.45:0/2012120068 wait complete. 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.032+0000 7fecba35d640 10 monclient: build_initial_monmap 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.032+0000 7fecba35d640 10 monclient: monmap: 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 1 Processor -- start 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 1 -- start start 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 10 monclient: init 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 10 monclient: _add_conns ranks=[2,4,6,5,0,3,1,8,7] 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 10 monclient(hunting): picked mon.noname-b con 0x7fecb409f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 10 monclient(hunting): picked mon.noname-c con 0x7fecb409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 10 monclient(hunting): picked mon.noname-d con 0x7fecb409ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb40a2560 con 0x7fecb409f7d0 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb4121e00 con 0x7fecb409fba0 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb4120060 con 0x7fecb409ff70 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecba35d640 10 monclient(hunting): authenticate will time out at 1046.371810s 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecb3fff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fecb409f7d0 0x7fecb41319d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49064/0 (socket says 172.21.15.45:49064) 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.033+0000 7fecb3fff640 1 -- 172.21.15.45:0/1594721872 learned_addr learned my addr 172.21.15.45:0/1594721872 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234470530 0 0) 0x7fecb40a2560 con 0x7fecb409f7d0 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): my global_id is 144128 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feca00034e0 con 0x7fecb409f7d0 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194159971 0 0) 0x7fecb4121e00 con 0x7fecb409fba0 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): my global_id is 144166 2024-09-18T14:03:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb40a2560 con 0x7fecb409fba0 2024-09-18T14:03:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970034082 0 0) 0x7fecb4120060 con 0x7fecb409ff70 2024-09-18T14:03:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): my global_id is 4749 2024-09-18T14:03:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb4121e00 con 0x7fecb409ff70 2024-09-18T14:03:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3806441146 0 0) 0x7feca00034e0 con 0x7fecb409f7d0 2024-09-18T14:03:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fecb4120060 con 0x7fecb409f7d0 2024-09-18T14:03:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feca4001410 con 0x7fecb409f7d0 2024-09-18T14:03:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:03:51.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): dump: 2024-09-18T14:03:51.052 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.052 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.052 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.052 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.052 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.052 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.052 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.052 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.053 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3101766203 0 0) 0x7fecb4121e00 con 0x7fecb409ff70 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feca00034e0 con 0x7fecb409ff70 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 307094374 0 0) 0x7fecb40a2560 con 0x7fecb409fba0 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fecb4121e00 con 0x7fecb409fba0 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fecac001d60 con 0x7fecb409ff70 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): dump: 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feca8001a60 con 0x7fecb409fba0 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient(hunting): dump: 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.056 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 956648463 0 0) 0x7fecb4120060 con 0x7fecb409f7d0 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 >> v1:172.21.15.45:6792/0 conn(0x7fecb409ff70 legacy=0x7fecb414a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 >> v1:172.21.15.45:6791/0 conn(0x7fecb409fba0 legacy=0x7fecb4146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 monclient: found mon.b 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb4140070 con 0x7fecb409f7d0 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient: _finish_auth 0 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fecba35d640 5 monclient: authenticate success, global_id 144128 2024-09-18T14:03:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fecba35d640 10 monclient: _renew_subs 2024-09-18T14:03:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fecba35d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:03:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fecba35d640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb4121b20 con 0x7fecb409f7d0 2024-09-18T14:03:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fecba35d640 10 monclient: _renew_subs 2024-09-18T14:03:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fecba35d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:03:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fecba35d640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb413fe30 con 0x7fecb409f7d0 2024-09-18T14:03:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feca4004d60 con 0x7fecb409f7d0 2024-09-18T14:03:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feca4005210 con 0x7fecb409f7d0 2024-09-18T14:03:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fecb27fc640 4 set_mon_vals no callback set 2024-09-18T14:03:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.034+0000 7fec9bfff640 10 monclient: dump: 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.060 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.060 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.035+0000 7fecba35d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:51.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.035+0000 7fecba35d640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fec94006580 con 0x7fec94002830 2024-09-18T14:03:51.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.035+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7feca40101e0 con 0x7fecb409f7d0 2024-09-18T14:03:51.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.035+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(227..227 src has 1..227) v4 ==== 3911+0+0 (unknown 472785454 0 0) 0x7feca4046bd0 con 0x7fecb409f7d0 2024-09-18T14:03:51.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fecb88fa640 10 monclient: get_auth_request con 0x7feca003aa10 auth_method 0 2024-09-18T14:03:51.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692695810 0 0) 0x7fec94006580 con 0x7fec94002830 2024-09-18T14:03:51.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 10 monclient(hunting): my global_id is 4789 2024-09-18T14:03:51.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb4120060 con 0x7fec94002830 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3746165738 0 0) 0x7fecb4120060 con 0x7fec94002830 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fec94006580 con 0x7fec94002830 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feca80021b0 con 0x7fec94002830 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 10 monclient: dump: 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2770568324 0 0) 0x7fec94006580 con 0x7fec94002830 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec9400a0a0 con 0x7fec94002830 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7feca80044b0 con 0x7fec94002830 2024-09-18T14:03:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:51.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:51.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 >> v1:172.21.15.84:6789/0 conn(0x7fec94002830 legacy=0x7fec94004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.036+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 >> v1:172.21.15.84:6789/0 conn(0x7fec94002830 legacy=0x7fec94004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:51.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.042+0000 7fecba35d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:51.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.042+0000 7fecba35d640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fec940074f0 con 0x7fec940068a0 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386691184 0 0) 0x7fec940074f0 con 0x7fec940068a0 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 10 monclient(hunting): my global_id is 4798 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec94006580 con 0x7fec940068a0 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1208178412 0 0) 0x7fec94006580 con 0x7fec940068a0 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fec940074f0 con 0x7fec940068a0 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feca8002c70 con 0x7fec940068a0 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 10 monclient: dump: 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.067 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 720403472 0 0) 0x7fec940074f0 con 0x7fec940068a0 2024-09-18T14:03:51.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fec94008750 con 0x7fec940068a0 2024-09-18T14:03:51.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 763933110) 0x7feca8004120 con 0x7fec940068a0 2024-09-18T14:03:51.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:51.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:51.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 >> v1:172.21.15.84:6789/0 conn(0x7fec940068a0 legacy=0x7fec94006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.043+0000 7fec9bfff640 1 -- 172.21.15.45:0/1594721872 >> v1:172.21.15.84:6789/0 conn(0x7fec940068a0 legacy=0x7fec94006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:51.067 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:51.068 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:51.068 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:51.068 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:51.068 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:51.068 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:51.068 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:51.068 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:51.068 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:51.068 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:51.069 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:51.069 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:51.069 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:51.069 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:51.069 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:51.069 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.069 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:03:51.069 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:51.069 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:51.069 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:51.069 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:51.070 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:51.071 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:51.072 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:51.072 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:51.072 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:51.072 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:51.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:51.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:51.072 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:51.072 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:51.072 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:51.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:51.072 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:51.072 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.073 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:51.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.074 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:51.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:51.074 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.074 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.074 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:51.074 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.074 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.074 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:51.076 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.076 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.076 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:51.076 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:51.076 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.076 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.076 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.076 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:51.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:51.077 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:51.077 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.077 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.077 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:51.077 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.078 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.079 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.081 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.082 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:51.084 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.084 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.084 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.084 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:51.084 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.084 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.084 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.084 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:51.084 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:51.084 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.085 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:51.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.087 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:51.087 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:51.087 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:51.087 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.087 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.087 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:03:51.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.087 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:51.087 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:51.088 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:51.088 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.088 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.088 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:51.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.088 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:51.088 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:51.088 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:51.088 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.088 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.088 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:51.089 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:51.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.044+0000 7fecba35d640 1 -- 172.21.15.45:0/1594721872 >> v1:172.21.15.84:6812/31478 conn(0x7feca003aa10 legacy=0x7feca003cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.044+0000 7fecba35d640 10 monclient: shutdown 2024-09-18T14:03:51.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.044+0000 7fecba35d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:51.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.044+0000 7fecba35d640 1 -- 172.21.15.45:0/1594721872 >> v1:172.21.15.45:6790/0 conn(0x7fecb409f7d0 legacy=0x7fecb41319d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.044+0000 7fecb88fa640 1 -- 172.21.15.45:0/1594721872 reap_dead start 2024-09-18T14:03:51.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.044+0000 7fecb88fa640 1 -- 172.21.15.45:0/1594721872 reap_dead start 2024-09-18T14:03:51.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.044+0000 7fecba35d640 1 -- 172.21.15.45:0/1594721872 shutdown_connections 2024-09-18T14:03:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.044+0000 7fecba35d640 1 -- 172.21.15.45:0/1594721872 >> 172.21.15.45:0/1594721872 conn(0x7fecb40630b0 msgr2=0x7fecb40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.044+0000 7fecba35d640 1 -- 172.21.15.45:0/1594721872 shutdown_connections 2024-09-18T14:03:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.044+0000 7fecba35d640 1 -- 172.21.15.45:0/1594721872 wait complete. 2024-09-18T14:03:51.091 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:51.091 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:03:51.091 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:03:51.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.187+0000 7f3bfb79d640 10 monclient: get_monmap_and_config 2024-09-18T14:03:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.187+0000 7f3bfb79d640 10 monclient: build_initial_monmap 2024-09-18T14:03:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.187+0000 7f3bfb79d640 10 monclient: monmap: 2024-09-18T14:03:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:51.188828+0000 2024-09-18T14:03:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:51.188828+0000 2024-09-18T14:03:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:51.190 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:51.190 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:51.190 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:51.190 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:51.190 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:51.190 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:51.190 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:51.190 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.188+0000 7f3bfb79d640 1 Processor -- start 2024-09-18T14:03:51.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.188+0000 7f3bfb79d640 1 -- start start 2024-09-18T14:03:51.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.188+0000 7f3bfb79d640 10 monclient: init 2024-09-18T14:03:51.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bfb79d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:51.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bfb79d640 10 monclient: _add_conns ranks=[4,5,8,1,6,0,2,3,7] 2024-09-18T14:03:51.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bfb79d640 10 monclient(hunting): picked mon.noname-c con 0x7f3bf409f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:51.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bfb79d640 10 monclient(hunting): picked mon.noname-h con 0x7f3bf409fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bfb79d640 10 monclient(hunting): picked mon.noname-e con 0x7f3bf409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bfb79d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf405ed70 con 0x7f3bf409f7d0 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bfb79d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf409dfc0 con 0x7f3bf409fba0 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bfb79d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf409dce0 con 0x7f3bf409ff70 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bfb79d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bfb79d640 10 monclient(hunting): authenticate will time out at 1046.527242s 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf9d3a640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f3bf409ff70 0x7f3bf413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:50034/0 (socket says 172.21.15.45:50034) 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf9539640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f3bf409f7d0 0x7f3bf4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38410/0 (socket says 172.21.15.45:38410) 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf9d3a640 1 -- 172.21.15.45:0/3127520378 learned_addr learned my addr 172.21.15.45:0/3127520378 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696494187 0 0) 0x7f3bf409dce0 con 0x7f3bf409ff70 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient(hunting): my global_id is 4904 2024-09-18T14:03:51.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bdc0034e0 con 0x7f3bf409ff70 2024-09-18T14:03:51.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879600705 0 0) 0x7f3bf405ed70 con 0x7f3bf409f7d0 2024-09-18T14:03:51.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient(hunting): my global_id is 144175 2024-09-18T14:03:51.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bf409dce0 con 0x7f3bf409f7d0 2024-09-18T14:03:51.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047619800 0 0) 0x7f3bf409dfc0 con 0x7f3bf409fba0 2024-09-18T14:03:51.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient(hunting): my global_id is 4739 2024-09-18T14:03:51.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bf405ed70 con 0x7f3bf409fba0 2024-09-18T14:03:51.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 872074363 0 0) 0x7f3bf409dce0 con 0x7f3bf409f7d0 2024-09-18T14:03:51.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 >> v1:172.21.15.45:6793/0 conn(0x7f3bf409ff70 legacy=0x7f3bf413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 >> v1:172.21.15.84:6791/0 conn(0x7f3bf409fba0 legacy=0x7f3bf4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient: _finish_hunting 0 2024-09-18T14:03:51.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 1 monclient: found mon.noname-c 2024-09-18T14:03:51.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:51.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:03:51.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf40a1c00 con 0x7f3bf409f7d0 2024-09-18T14:03:51.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient: _finish_auth 0 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bfb79d640 5 monclient: authenticate success, global_id 144175 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3bec001790 con 0x7f3bf409f7d0 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.189+0000 7f3bf3fff640 10 monclient: dump: 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bfb79d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3bec0018f0 con 0x7f3bf409f7d0 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bf3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bf3fff640 1 -- 172.21.15.45:0/3127520378 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3bec0047a0 con 0x7f3bf409f7d0 2024-09-18T14:03:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bf3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bf3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bf3fff640 10 monclient: dump: 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bfb79d640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bfb79d640 4 set_mon_vals no callback set 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bfb79d640 10 monclient: shutdown 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bfb79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bfb79d640 1 -- 172.21.15.45:0/3127520378 >> v1:172.21.15.45:6791/0 conn(0x7f3bf409f7d0 legacy=0x7f3bf4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bfb79d640 1 -- 172.21.15.45:0/3127520378 shutdown_connections 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bfb79d640 1 -- 172.21.15.45:0/3127520378 >> 172.21.15.45:0/3127520378 conn(0x7f3bf40630b0 msgr2=0x7f3bf40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bfb79d640 1 -- 172.21.15.45:0/3127520378 shutdown_connections 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.190+0000 7f3bfb79d640 1 -- 172.21.15.45:0/3127520378 wait complete. 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.191+0000 7f3bfb79d640 10 monclient: build_initial_monmap 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.191+0000 7f3bfb79d640 10 monclient: monmap: 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.191+0000 7f3bfb79d640 1 Processor -- start 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.191+0000 7f3bfb79d640 1 -- start start 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.191+0000 7f3bfb79d640 10 monclient: init 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 10 monclient: _add_conns ranks=[1,5,4,8,6,0,2,3,7] 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 10 monclient(hunting): picked mon.noname-f con 0x7f3bf409f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 10 monclient(hunting): picked mon.noname-h con 0x7f3bf409fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 10 monclient(hunting): picked mon.noname-c con 0x7f3bf409ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf409d310 con 0x7f3bf409f7d0 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf40a2560 con 0x7f3bf409ff70 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf409b7a0 con 0x7f3bf409fba0 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 10 monclient(hunting): authenticate will time out at 1046.530251s 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf9d3a640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f3bf409ff70 0x7f3bf414a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38422/0 (socket says 172.21.15.45:38422) 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf9d3a640 1 -- 172.21.15.45:0/1767289377 learned_addr learned my addr 172.21.15.45:0/1767289377 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408478500 0 0) 0x7f3bf40a2560 con 0x7f3bf409ff70 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): my global_id is 144184 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be00034e0 con 0x7f3bf409ff70 2024-09-18T14:03:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183296090 0 0) 0x7f3bf409b7a0 con 0x7f3bf409fba0 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): my global_id is 4748 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bf40a2560 con 0x7f3bf409fba0 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009553228 0 0) 0x7f3bf409d310 con 0x7f3bf409f7d0 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): my global_id is 4807 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bf409b7a0 con 0x7f3bf409f7d0 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2880404594 0 0) 0x7f3be00034e0 con 0x7f3bf409ff70 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3bf409d310 con 0x7f3bf409ff70 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3be4001de0 con 0x7f3bf409ff70 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): dump: 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3084499251 0 0) 0x7f3bf40a2560 con 0x7f3bf409fba0 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3be00034e0 con 0x7f3bf409fba0 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3be8001aa0 con 0x7f3bf409fba0 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): dump: 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.216 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3658587454 0 0) 0x7f3bf409b7a0 con 0x7f3bf409f7d0 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3bf40a2560 con 0x7f3bf409f7d0 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3bec0016d0 con 0x7f3bf409f7d0 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient(hunting): dump: 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 33784181 0 0) 0x7f3bf409d310 con 0x7f3bf409ff70 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 >> v1:172.21.15.84:6791/0 conn(0x7f3bf409fba0 legacy=0x7f3bf4146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 >> v1:172.21.15.84:6789/0 conn(0x7f3bf409f7d0 legacy=0x7f3bf4131640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient: _finish_hunting 0 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 monclient: found mon.c 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf411e760 con 0x7f3bf409ff70 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 10 monclient: _finish_auth 0 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bf17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 5 monclient: authenticate success, global_id 144184 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 10 monclient: _renew_subs 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.192+0000 7f3bfb79d640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf411ea40 con 0x7f3bf409ff70 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bfb79d640 10 monclient: _renew_subs 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bfb79d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bfb79d640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf4136ee0 con 0x7f3bf409ff70 2024-09-18T14:03:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3be4002040 con 0x7f3bf409ff70 2024-09-18T14:03:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bf17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3be4005110 con 0x7f3bf409ff70 2024-09-18T14:03:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bf17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bf17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bf17fa640 10 monclient: dump: 2024-09-18T14:03:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bf37fe640 4 set_mon_vals no callback set 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bfb79d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bfb79d640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3bf404c4a0 con 0x7f3bf409ac50 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.193+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f3be40100e0 con 0x7f3bf409ff70 2024-09-18T14:03:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(227..227 src has 1..227) v4 ==== 3911+0+0 (unknown 472785454 0 0) 0x7f3be4046d50 con 0x7f3bf409ff70 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559542507 0 0) 0x7f3bf404c4a0 con 0x7f3bf409ac50 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 10 monclient(hunting): my global_id is 144129 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bf409d310 con 0x7f3bf409ac50 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf8d38640 10 monclient: get_auth_request con 0x7f3be003a9d0 auth_method 0 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2902978927 0 0) 0x7f3bf409d310 con 0x7f3bf409ac50 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3bf404c4a0 con 0x7f3bf409ac50 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3bec001190 con 0x7f3bf409ac50 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 10 monclient: dump: 2024-09-18T14:03:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.224 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3704767725 0 0) 0x7f3bf404c4a0 con 0x7f3bf409ac50 2024-09-18T14:03:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.194+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3bf411f090 con 0x7f3bf409ac50 2024-09-18T14:03:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.195+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3bec005190 con 0x7f3bf409ac50 2024-09-18T14:03:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.195+0000 7f3bf17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.195+0000 7f3bf17fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.195+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 >> v1:172.21.15.84:6790/0 conn(0x7f3bf409ac50 legacy=0x7f3bf4096a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.195+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 >> v1:172.21.15.84:6790/0 conn(0x7f3bf409ac50 legacy=0x7f3bf4096a20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.200+0000 7f3bfb79d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.200+0000 7f3bfb79d640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3bf404b720 con 0x7f3bf402a8d0 2024-09-18T14:03:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745857522 0 0) 0x7f3bf404b720 con 0x7f3bf402a8d0 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 10 monclient(hunting): my global_id is 144138 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bf404c4a0 con 0x7f3bf402a8d0 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2190738211 0 0) 0x7f3bf404c4a0 con 0x7f3bf402a8d0 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3bf404b720 con 0x7f3bf402a8d0 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3bec005950 con 0x7f3bf402a8d0 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 10 monclient: dump: 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 610875541 0 0) 0x7f3bf404b720 con 0x7f3bf402a8d0 2024-09-18T14:03:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.201+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3bf4071550 con 0x7f3bf402a8d0 2024-09-18T14:03:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 1756172181) 0x7f3bec005cb0 con 0x7f3bf402a8d0 2024-09-18T14:03:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bf17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bf17fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 >> v1:172.21.15.84:6790/0 conn(0x7f3bf402a8d0 legacy=0x7f3bf4097130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bf17fa640 1 -- 172.21.15.45:0/1767289377 >> v1:172.21.15.84:6790/0 conn(0x7f3bf402a8d0 legacy=0x7f3bf4097130 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:51.227 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:51.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:51.227 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:51.228 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.229 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:03:51.229 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:51.229 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:51.229 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:51.229 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:51.229 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:51.229 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:51.229 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:51.229 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:51.229 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:51.229 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:51.229 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:51.230 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:51.231 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:51.232 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:51.232 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:51.232 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:51.232 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:51.232 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:51.232 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:51.232 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:51.232 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:51.232 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:51.232 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:51.232 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:51.232 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.234 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:51.235 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.237 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:51.239 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.241 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:51.244 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:51.245 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.246 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.247 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:51.247 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:51.248 INFO:tasks.rados.rados.0.smithi084.stdout:983: done (3 left) 2024-09-18T14:03:51.249 INFO:tasks.rados.rados.0.smithi084.stdout:984: done (2 left) 2024-09-18T14:03:51.249 INFO:tasks.rados.rados.0.smithi084.stdout:985: done (1 left) 2024-09-18T14:03:51.249 INFO:tasks.rados.rados.0.smithi084.stdout:986: done (0 left) 2024-09-18T14:03:51.249 INFO:tasks.rados.rados.0.smithi084.stdout:987: write oid 26 current snap is 110 2024-09-18T14:03:51.249 INFO:tasks.rados.rados.0.smithi084.stdout:987: seq_num 356 ranges {783292=540882,1932766=507787,3210350=572013} 2024-09-18T14:03:51.249 INFO:tasks.rados.rados.0.smithi084.stdout:987: writing smithi08434663-26 from 783292 to 1324174 tid 1 2024-09-18T14:03:51.249 INFO:tasks.rados.rados.0.smithi084.stdout:987: writing smithi08434663-26 from 1932766 to 2440553 tid 2 2024-09-18T14:03:51.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bfb79d640 1 -- 172.21.15.45:0/1767289377 >> v1:172.21.15.84:6812/31478 conn(0x7f3be003a9d0 legacy=0x7f3be003ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bfb79d640 10 monclient: shutdown 2024-09-18T14:03:51.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bfb79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:51.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bfb79d640 1 -- 172.21.15.45:0/1767289377 >> v1:172.21.15.45:6791/0 conn(0x7f3bf409ff70 legacy=0x7f3bf414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bf9d3a640 1 -- 172.21.15.45:0/1767289377 reap_dead start 2024-09-18T14:03:51.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bf9d3a640 1 -- 172.21.15.45:0/1767289377 reap_dead start 2024-09-18T14:03:51.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bfb79d640 1 -- 172.21.15.45:0/1767289377 shutdown_connections 2024-09-18T14:03:51.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bfb79d640 1 -- 172.21.15.45:0/1767289377 >> 172.21.15.45:0/1767289377 conn(0x7f3bf40630b0 msgr2=0x7f3bf40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:51.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bfb79d640 1 -- 172.21.15.45:0/1767289377 shutdown_connections 2024-09-18T14:03:51.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.202+0000 7f3bfb79d640 1 -- 172.21.15.45:0/1767289377 wait complete. 2024-09-18T14:03:51.250 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:51.250 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:03:51.251 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:03:51.252 INFO:tasks.rados.rados.0.smithi084.stdout:987: writing smithi08434663-26 from 3210350 to 3782363 tid 3 2024-09-18T14:03:51.253 INFO:tasks.rados.rados.0.smithi084.stdout:988: rollback oid 49 current snap is 110 2024-09-18T14:03:51.253 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 49 to 85 2024-09-18T14:03:51.253 INFO:tasks.rados.rados.0.smithi084.stdout:987: finishing write tid 1 to smithi08434663-26 2024-09-18T14:03:51.253 INFO:tasks.rados.rados.0.smithi084.stdout:987: finishing write tid 2 to smithi08434663-26 2024-09-18T14:03:51.253 INFO:tasks.rados.rados.0.smithi084.stdout:989: snap_create 2024-09-18T14:03:51.254 INFO:tasks.rados.rados.0.smithi084.stdout:988: finishing rollback tid 1 to smithi08434663-49 2024-09-18T14:03:51.254 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 0 (ObjNum 33 snap 0 seq_num 3385422139) dirty dne 2024-09-18T14:03:51.255 INFO:tasks.rados.rados.0.smithi084.stdout:987: finishing write tid 3 to smithi08434663-26 2024-09-18T14:03:51.257 INFO:tasks.rados.rados.0.smithi084.stdout:987: finishing write tid 4 to smithi08434663-26 2024-09-18T14:03:51.257 INFO:tasks.rados.rados.0.smithi084.stdout:987: finishing write tid 5 to smithi08434663-26 2024-09-18T14:03:51.257 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 308 (ObjNum 356 snap 110 seq_num 356) dirty exists 2024-09-18T14:03:51.257 INFO:tasks.rados.rados.0.smithi084.stdout:987: left oid 26 (ObjNum 356 snap 110 seq_num 356) 2024-09-18T14:03:51.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.347+0000 7f33e7460640 10 monclient: get_monmap_and_config 2024-09-18T14:03:51.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.347+0000 7f33e7460640 10 monclient: build_initial_monmap 2024-09-18T14:03:51.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.347+0000 7f33e7460640 10 monclient: monmap: 2024-09-18T14:03:51.348 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:51.348 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.348 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:51.348605+0000 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:51.348605+0000 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:51.349 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.348+0000 7f33e7460640 1 Processor -- start 2024-09-18T14:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.348+0000 7f33e7460640 1 -- start start 2024-09-18T14:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.348+0000 7f33e7460640 10 monclient: init 2024-09-18T14:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.348+0000 7f33e7460640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.348+0000 7f33e7460640 10 monclient: _add_conns ranks=[6,5,8,1,2,4,7,3,0] 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.348+0000 7f33e7460640 10 monclient(hunting): picked mon.noname-d con 0x7f33e009f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.348+0000 7f33e7460640 10 monclient(hunting): picked mon.noname-h con 0x7f33e009fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.348+0000 7f33e7460640 10 monclient(hunting): picked mon.noname-e con 0x7f33e009ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.348+0000 7f33e7460640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e005ed70 con 0x7f33e009fba0 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.348+0000 7f33e7460640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e009dfc0 con 0x7f33e009f7d0 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33e7460640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e009dce0 con 0x7f33e009ff70 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33e7460640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33e7460640 10 monclient(hunting): authenticate will time out at 1046.687031s 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33e51fc640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f33e009f7d0 0x7f33e0132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:58944/0 (socket says 172.21.15.45:58944) 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33e51fc640 1 -- 172.21.15.45:0/2885239152 learned_addr learned my addr 172.21.15.45:0/2885239152 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257828865 0 0) 0x7f33e009dfc0 con 0x7f33e009f7d0 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient(hunting): my global_id is 4758 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33cc0034e0 con 0x7f33e009f7d0 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275835809 0 0) 0x7f33e009dce0 con 0x7f33e009ff70 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient(hunting): my global_id is 4913 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e009dfc0 con 0x7f33e009ff70 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057585211 0 0) 0x7f33e005ed70 con 0x7f33e009fba0 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient(hunting): my global_id is 4757 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e009dce0 con 0x7f33e009fba0 2024-09-18T14:03:51.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2021051699 0 0) 0x7f33cc0034e0 con 0x7f33e009f7d0 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 >> v1:172.21.15.45:6793/0 conn(0x7f33e009ff70 legacy=0x7f33e013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 >> v1:172.21.15.84:6791/0 conn(0x7f33e009fba0 legacy=0x7f33e0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient: _finish_hunting 0 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 monclient: found mon.noname-d 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e00a1c00 con 0x7f33e009f7d0 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient: _finish_auth 0 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f33dc001830 con 0x7f33e009f7d0 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33e7460640 5 monclient: authenticate success, global_id 4758 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33e7460640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:51.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:03:51.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient: dump: 2024-09-18T14:03:51.361 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.361 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.361 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.361 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.361 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.361 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.361 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.361 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.361 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.361 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33e7460640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f33dc001a10 con 0x7f33e009f7d0 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 1 -- 172.21.15.45:0/2885239152 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f33dc004ba0 con 0x7f33e009f7d0 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33e7460640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33e7460640 4 set_mon_vals no callback set 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:51.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.349+0000 7f33d7fff640 10 monclient: dump: 2024-09-18T14:03:51.363 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.363 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.363 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.363 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.363 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.363 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.363 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.363 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.363 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.363 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.363 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.364 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.364 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.364 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.364 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.364 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.350+0000 7f33e7460640 10 monclient: shutdown 2024-09-18T14:03:51.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.350+0000 7f33e7460640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:51.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.350+0000 7f33e7460640 1 -- 172.21.15.45:0/2885239152 >> v1:172.21.15.45:6792/0 conn(0x7f33e009f7d0 legacy=0x7f33e0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.350+0000 7f33e7460640 1 -- 172.21.15.45:0/2885239152 shutdown_connections 2024-09-18T14:03:51.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.350+0000 7f33e7460640 1 -- 172.21.15.45:0/2885239152 >> 172.21.15.45:0/2885239152 conn(0x7f33e00630b0 msgr2=0x7f33e00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.350+0000 7f33e7460640 1 -- 172.21.15.45:0/2885239152 shutdown_connections 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.350+0000 7f33e7460640 1 -- 172.21.15.45:0/2885239152 wait complete. 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.350+0000 7f33e7460640 10 monclient: build_initial_monmap 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.350+0000 7f33e7460640 10 monclient: monmap: 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:51.365 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.351+0000 7f33e7460640 1 Processor -- start 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.351+0000 7f33e7460640 1 -- start start 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.351+0000 7f33e7460640 10 monclient: init 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.351+0000 7f33e7460640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.351+0000 7f33e7460640 10 monclient: _add_conns ranks=[1,4,7,8,3,6,5,0,2] 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.351+0000 7f33e7460640 10 monclient(hunting): picked mon.noname-f con 0x7f33e009f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.351+0000 7f33e7460640 10 monclient(hunting): picked mon.noname-c con 0x7f33e009fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e7460640 10 monclient(hunting): picked mon.noname-i con 0x7f33e009ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e7460640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e009d310 con 0x7f33e009f7d0 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e7460640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e00a2560 con 0x7f33e009fba0 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e7460640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e006b220 con 0x7f33e009ff70 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e7460640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e7460640 10 monclient(hunting): authenticate will time out at 1046.690084s 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e49fb640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f33e009fba0 0x7f33e006ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38432/0 (socket says 172.21.15.45:38432) 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e49fb640 1 -- 172.21.15.45:0/2350526602 learned_addr learned my addr 172.21.15.45:0/2350526602 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673686319 0 0) 0x7f33e00a2560 con 0x7f33e009fba0 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): my global_id is 144193 2024-09-18T14:03:51.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33c40034e0 con 0x7f33e009fba0 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147342906 0 0) 0x7f33e009d310 con 0x7f33e009f7d0 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): my global_id is 4816 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e00a2560 con 0x7f33e009f7d0 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389602261 0 0) 0x7f33e006b220 con 0x7f33e009ff70 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): my global_id is 4822 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e009d310 con 0x7f33e009ff70 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3156654947 0 0) 0x7f33c40034e0 con 0x7f33e009fba0 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f33e006b220 con 0x7f33e009fba0 2024-09-18T14:03:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f33d0002b70 con 0x7f33e009fba0 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): dump: 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.370 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.370 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.370 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.370 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.370 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.370 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:51.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1774188115 0 0) 0x7f33e00a2560 con 0x7f33e009f7d0 2024-09-18T14:03:51.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f33c40034e0 con 0x7f33e009f7d0 2024-09-18T14:03:51.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f33dc004b20 con 0x7f33e009f7d0 2024-09-18T14:03:51.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient(hunting): dump: 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.371 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.372 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.372 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.372 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.372 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3119158273 0 0) 0x7f33e006b220 con 0x7f33e009fba0 2024-09-18T14:03:51.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 >> v1:172.21.15.84:6792/0 conn(0x7f33e009ff70 legacy=0x7f33e014eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 >> v1:172.21.15.84:6789/0 conn(0x7f33e009f7d0 legacy=0x7f33e006a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient: _finish_hunting 0 2024-09-18T14:03:51.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 monclient: found mon.c 2024-09-18T14:03:51.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:51.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:51.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e009b890 con 0x7f33e009fba0 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 10 monclient: _finish_auth 0 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e7460640 5 monclient: authenticate success, global_id 144193 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e7460640 10 monclient: _renew_subs 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e7460640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e7460640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e006c720 con 0x7f33e009fba0 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.352+0000 7f33e7460640 10 monclient: _renew_subs 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.353+0000 7f33e7460640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.353+0000 7f33e7460640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e006b7c0 con 0x7f33e009fba0 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.353+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f33d0001660 con 0x7f33e009fba0 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.353+0000 7f33d57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.353+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f33d0002db0 con 0x7f33e009fba0 2024-09-18T14:03:51.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.353+0000 7f33d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.353+0000 7f33d77fe640 4 set_mon_vals no callback set 2024-09-18T14:03:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.353+0000 7f33d57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.353+0000 7f33d57fa640 10 monclient: dump: 2024-09-18T14:03:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.353+0000 7f33e7460640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.353+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f33d0001c10 con 0x7f33e009fba0 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.353+0000 7f33e7460640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f33c0006580 con 0x7f33c0002830 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(228..228 src has 1..228) v4 ==== 3911+0+0 (unknown 2442380505 0 0) 0x7f33d00469c0 con 0x7f33e009fba0 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33e59fd640 10 monclient: get_auth_request con 0x7f33c403a520 auth_method 0 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004825922 0 0) 0x7f33c0006580 con 0x7f33c0002830 2024-09-18T14:03:51.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 10 monclient(hunting): my global_id is 4766 2024-09-18T14:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e006b220 con 0x7f33c0002830 2024-09-18T14:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 22952888 0 0) 0x7f33e006b220 con 0x7f33c0002830 2024-09-18T14:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f33c0006580 con 0x7f33c0002830 2024-09-18T14:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f33dc0017a0 con 0x7f33c0002830 2024-09-18T14:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 10 monclient: dump: 2024-09-18T14:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.377 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2159726641 0 0) 0x7f33c0006580 con 0x7f33c0002830 2024-09-18T14:03:51.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33c000a0a0 con 0x7f33c0002830 2024-09-18T14:03:51.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f33dc005250 con 0x7f33c0002830 2024-09-18T14:03:51.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:51.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:51.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 >> v1:172.21.15.84:6791/0 conn(0x7f33c0002830 legacy=0x7f33c0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.354+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 >> v1:172.21.15.84:6791/0 conn(0x7f33c0002830 legacy=0x7f33c0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:51.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.360+0000 7f33e7460640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:51.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.360+0000 7f33e7460640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f33c0009660 con 0x7f33c0009040 2024-09-18T14:03:51.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752417234 0 0) 0x7f33c0009660 con 0x7f33c0009040 2024-09-18T14:03:51.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 10 monclient(hunting): my global_id is 4775 2024-09-18T14:03:51.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33c0006580 con 0x7f33c0009040 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3084996296 0 0) 0x7f33c0006580 con 0x7f33c0009040 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f33c0009660 con 0x7f33c0009040 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f33dc005b30 con 0x7f33c0009040 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 10 monclient: dump: 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.382 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3783501742 0 0) 0x7f33c0009660 con 0x7f33c0009040 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f33c0009920 con 0x7f33c0009040 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7726 (unknown 0 0 767532114) 0x7f33dc008240 con 0x7f33c0009040 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 >> v1:172.21.15.84:6791/0 conn(0x7f33c0009040 legacy=0x7f33c0008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.361+0000 7f33d57fa640 1 -- 172.21.15.45:0/2350526602 >> v1:172.21.15.84:6791/0 conn(0x7f33c0009040 legacy=0x7f33c0008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:51.384 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:51.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:51.384 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:51.384 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:51.384 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:51.384 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:51.384 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:51.384 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:51.384 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:51.385 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:51.385 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:51.385 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:51.385 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:51.385 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:51.385 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:51.385 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.385 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:03:51.385 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:51.385 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:51.385 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:51.385 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:51.386 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:51.387 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:51.388 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:51.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:51.390 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:51.391 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:51.392 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.393 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.394 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:51.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:51.394 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.394 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.394 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:51.394 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.394 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:51.397 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.398 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:51.401 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.402 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.403 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:03:51.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.403 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:51.403 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:51.403 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:03:51.403 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.403 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.403 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:51.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.403 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:51.403 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:51.403 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:51.404 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.404 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.404 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:03:51.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.404 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:51.404 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:51.404 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:51.404 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.404 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.404 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:51.404 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.362+0000 7f33e7460640 1 -- 172.21.15.45:0/2350526602 >> v1:172.21.15.84:6812/31478 conn(0x7f33c403a520 legacy=0x7f33c403c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.362+0000 7f33e7460640 10 monclient: shutdown 2024-09-18T14:03:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.362+0000 7f33e7460640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.362+0000 7f33e7460640 1 -- 172.21.15.45:0/2350526602 >> v1:172.21.15.45:6791/0 conn(0x7f33e009fba0 legacy=0x7f33e006ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.362+0000 7f33e59fd640 1 -- 172.21.15.45:0/2350526602 reap_dead start 2024-09-18T14:03:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.362+0000 7f33e59fd640 1 -- 172.21.15.45:0/2350526602 reap_dead start 2024-09-18T14:03:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.362+0000 7f33e7460640 1 -- 172.21.15.45:0/2350526602 shutdown_connections 2024-09-18T14:03:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.362+0000 7f33e7460640 1 -- 172.21.15.45:0/2350526602 >> 172.21.15.45:0/2350526602 conn(0x7f33e00630b0 msgr2=0x7f33e00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.362+0000 7f33e7460640 1 -- 172.21.15.45:0/2350526602 shutdown_connections 2024-09-18T14:03:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.362+0000 7f33e7460640 1 -- 172.21.15.45:0/2350526602 wait complete. 2024-09-18T14:03:51.407 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:51.407 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:03:51.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:03:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.504+0000 7f6d7bd1d640 10 monclient: get_monmap_and_config 2024-09-18T14:03:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.504+0000 7f6d7bd1d640 10 monclient: build_initial_monmap 2024-09-18T14:03:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.504+0000 7f6d7bd1d640 10 monclient: monmap: 2024-09-18T14:03:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:51.505148+0000 2024-09-18T14:03:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:51.505148+0000 2024-09-18T14:03:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:51.506 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.504+0000 7f6d7bd1d640 1 Processor -- start 2024-09-18T14:03:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7bd1d640 1 -- start start 2024-09-18T14:03:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7bd1d640 10 monclient: init 2024-09-18T14:03:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7bd1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7bd1d640 10 monclient: _add_conns ranks=[3,2,4,8,1,6,0,5,7] 2024-09-18T14:03:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7bd1d640 10 monclient(hunting): picked mon.noname-g con 0x7f6d7406e810 addr v1:172.21.15.84:6790/0 2024-09-18T14:03:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7bd1d640 10 monclient(hunting): picked mon.noname-b con 0x7f6d740a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:03:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7bd1d640 10 monclient(hunting): picked mon.noname-c con 0x7f6d740a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7bd1d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7405ed70 con 0x7f6d740a0cb0 2024-09-18T14:03:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7bd1d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d74097620 con 0x7f6d7406e810 2024-09-18T14:03:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7bd1d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d74097340 con 0x7f6d740a1080 2024-09-18T14:03:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7bd1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7bd1d640 10 monclient(hunting): authenticate will time out at 1046.843641s 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d792b8640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f6d740a0cb0 0x7f6d74137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49070/0 (socket says 172.21.15.45:49070) 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d792b8640 1 -- 172.21.15.45:0/1152387240 learned_addr learned my addr 172.21.15.45:0/1152387240 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.505+0000 7f6d7a2ba640 1 --1- 172.21.15.45:0/1152387240 >> v1:172.21.15.45:6791/0 conn(0x7f6d740a1080 0x7f6d7413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38438/0 (socket says 172.21.15.45:38438) 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448414260 0 0) 0x7f6d7405ed70 con 0x7f6d740a0cb0 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient(hunting): my global_id is 144137 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d600034e0 con 0x7f6d740a0cb0 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916753922 0 0) 0x7f6d74097340 con 0x7f6d740a1080 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient(hunting): my global_id is 144202 2024-09-18T14:03:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d7405ed70 con 0x7f6d740a1080 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191265136 0 0) 0x7f6d74097620 con 0x7f6d7406e810 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient(hunting): my global_id is 144147 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d74097340 con 0x7f6d7406e810 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1105456206 0 0) 0x7f6d600034e0 con 0x7f6d740a0cb0 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 >> v1:172.21.15.45:6791/0 conn(0x7f6d740a1080 legacy=0x7f6d7413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 >> v1:172.21.15.84:6790/0 conn(0x7f6d7406e810 legacy=0x7f6d740a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient: _finish_hunting 0 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 monclient: found mon.noname-b 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:03:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d740a1660 con 0x7f6d740a0cb0 2024-09-18T14:03:51.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient: _finish_auth 0 2024-09-18T14:03:51.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:51.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d7bd1d640 5 monclient: authenticate success, global_id 144137 2024-09-18T14:03:51.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d64001830 con 0x7f6d740a0cb0 2024-09-18T14:03:51.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d7bd1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:51.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:03:51.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient: dump: 2024-09-18T14:03:51.518 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.518 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.518 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.518 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d7bd1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6d64001a10 con 0x7f6d740a0cb0 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 1 -- 172.21.15.45:0/1152387240 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d64004ba0 con 0x7f6d740a0cb0 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d7bd1d640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d7bd1d640 4 set_mon_vals no callback set 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d78ab7640 10 monclient: dump: 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.520 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d7bd1d640 10 monclient: shutdown 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d7bd1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/1152387240 >> v1:172.21.15.45:6790/0 conn(0x7f6d740a0cb0 legacy=0x7f6d74137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/1152387240 shutdown_connections 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.506+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/1152387240 >> 172.21.15.45:0/1152387240 conn(0x7f6d740a5bb0 msgr2=0x7f6d740a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.507+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/1152387240 shutdown_connections 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.507+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/1152387240 wait complete. 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.507+0000 7f6d7bd1d640 10 monclient: build_initial_monmap 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.507+0000 7f6d7bd1d640 10 monclient: monmap: 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:51.522 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.507+0000 7f6d7bd1d640 1 Processor -- start 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.507+0000 7f6d7bd1d640 1 -- start start 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.507+0000 7f6d7bd1d640 10 monclient: init 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d7bd1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d7bd1d640 10 monclient: _add_conns ranks=[7,5,2,4,1,6,8,3,0] 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d7bd1d640 10 monclient(hunting): picked mon.noname-i con 0x7f6d7406e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d7bd1d640 10 monclient(hunting): picked mon.noname-h con 0x7f6d740a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:51.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d7bd1d640 10 monclient(hunting): picked mon.noname-b con 0x7f6d740a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:03:51.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d7bd1d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d74122df0 con 0x7f6d740a1080 2024-09-18T14:03:51.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d7bd1d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7413dbe0 con 0x7f6d740a0cb0 2024-09-18T14:03:51.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d7bd1d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d74122830 con 0x7f6d7406e810 2024-09-18T14:03:51.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d7bd1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:51.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d7bd1d640 10 monclient(hunting): authenticate will time out at 1046.846460s 2024-09-18T14:03:51.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d7a2ba640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f6d740a1080 0x7f6d74149e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49076/0 (socket says 172.21.15.45:49076) 2024-09-18T14:03:51.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d7a2ba640 1 -- 172.21.15.45:0/2053342394 learned_addr learned my addr 172.21.15.45:0/2053342394 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:51.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339197388 0 0) 0x7f6d74122df0 con 0x7f6d740a1080 2024-09-18T14:03:51.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d69ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d69ffb640 10 monclient(hunting): my global_id is 144146 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d580034e0 con 0x7f6d740a1080 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2822982535 0 0) 0x7f6d580034e0 con 0x7f6d740a1080 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6d74122df0 con 0x7f6d740a1080 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d6c001dc0 con 0x7f6d740a1080 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d69ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d69ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.508+0000 7f6d69ffb640 10 monclient(hunting): dump: 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.525 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265992203 0 0) 0x7f6d7413dbe0 con 0x7f6d740a0cb0 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient(hunting): my global_id is 4784 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d580034e0 con 0x7f6d740a0cb0 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354041560 0 0) 0x7f6d74122830 con 0x7f6d7406e810 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient(hunting): my global_id is 4831 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d7413dbe0 con 0x7f6d7406e810 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 665600789 0 0) 0x7f6d74122df0 con 0x7f6d740a1080 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 >> v1:172.21.15.84:6792/0 conn(0x7f6d7406e810 legacy=0x7f6d7413d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 >> v1:172.21.15.84:6791/0 conn(0x7f6d740a0cb0 legacy=0x7f6d74146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 1 monclient: found mon.b 2024-09-18T14:03:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d74122550 con 0x7f6d740a1080 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient: _finish_auth 0 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d7bd1d640 5 monclient: authenticate success, global_id 144146 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d7bd1d640 10 monclient: _renew_subs 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d7bd1d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d74122b10 con 0x7f6d740a1080 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6d6c0020e0 con 0x7f6d740a1080 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d6c0051d0 con 0x7f6d740a1080 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d69ffb640 10 monclient: dump: 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d6bfff640 4 set_mon_vals no callback set 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d7bd1d640 10 monclient: _renew_subs 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d7bd1d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.509+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d74121680 con 0x7f6d740a1080 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f6d6c0101a0 con 0x7f6d740a1080 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d7bd1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6d7404c4a0 con 0x7f6d7413e780 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(228..228 src has 1..228) v4 ==== 3911+0+0 (unknown 2442380505 0 0) 0x7f6d6c046e70 con 0x7f6d740a1080 2024-09-18T14:03:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d792b8640 10 monclient: get_auth_request con 0x7f6d5803a020 auth_method 0 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910519935 0 0) 0x7f6d7404c4a0 con 0x7f6d7413e780 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 10 monclient(hunting): my global_id is 4840 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d74122df0 con 0x7f6d7413e780 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 643376067 0 0) 0x7f6d74122df0 con 0x7f6d7413e780 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6d7404c4a0 con 0x7f6d7413e780 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d70001920 con 0x7f6d7413e780 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.510+0000 7f6d69ffb640 10 monclient: dump: 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.531 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.533 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.511+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2651279217 0 0) 0x7f6d7404c4a0 con 0x7f6d7413e780 2024-09-18T14:03:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.511+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d74121920 con 0x7f6d7413e780 2024-09-18T14:03:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.511+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6d70001e70 con 0x7f6d7413e780 2024-09-18T14:03:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.511+0000 7f6d69ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.511+0000 7f6d69ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.511+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 >> v1:172.21.15.84:6792/0 conn(0x7f6d7413e780 legacy=0x7f6d7409bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.511+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 >> v1:172.21.15.84:6792/0 conn(0x7f6d7413e780 legacy=0x7f6d7409bb50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:51.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d7bd1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:51.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6d7413e1d0 con 0x7f6d7413de00 2024-09-18T14:03:51.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503631078 0 0) 0x7f6d7413e1d0 con 0x7f6d7413de00 2024-09-18T14:03:51.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 10 monclient(hunting): my global_id is 4849 2024-09-18T14:03:51.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d7404c4a0 con 0x7f6d7413de00 2024-09-18T14:03:51.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1472424276 0 0) 0x7f6d7404c4a0 con 0x7f6d7413de00 2024-09-18T14:03:51.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6d7413e1d0 con 0x7f6d7413de00 2024-09-18T14:03:51.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d70002630 con 0x7f6d7413de00 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 10 monclient: dump: 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.536 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3003745321 0 0) 0x7f6d7413e1d0 con 0x7f6d7413de00 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.517+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6d7404b720 con 0x7f6d7413de00 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 4062625478) 0x7f6d70002790 con 0x7f6d7413de00 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d69ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d69ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 >> v1:172.21.15.84:6792/0 conn(0x7f6d7413de00 legacy=0x7f6d7409c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d69ffb640 1 -- 172.21.15.45:0/2053342394 >> v1:172.21.15.84:6792/0 conn(0x7f6d7413de00 legacy=0x7f6d7409c260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:51.538 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:51.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:51.538 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:51.538 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:51.538 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:51.538 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:51.538 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:51.538 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:51.538 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:51.538 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:51.539 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:51.540 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:51.541 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:51.542 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:51.543 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:51.544 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.544 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.544 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:51.544 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.544 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.544 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.544 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:51.544 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:51.544 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.545 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:51.548 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.549 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.550 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.552 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.553 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:51.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:51.555 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:51.556 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:51.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:51.557 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:51.557 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:51.557 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:03:51.557 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:51.557 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:51.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:51.557 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:51.557 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:51.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/2053342394 >> v1:172.21.15.84:6812/31478 conn(0x7f6d5803a020 legacy=0x7f6d5803c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d7bd1d640 10 monclient: shutdown 2024-09-18T14:03:51.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d7bd1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:51.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/2053342394 >> v1:172.21.15.45:6790/0 conn(0x7f6d740a1080 legacy=0x7f6d74149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d7a2ba640 1 -- 172.21.15.45:0/2053342394 reap_dead start 2024-09-18T14:03:51.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d7a2ba640 1 -- 172.21.15.45:0/2053342394 reap_dead start 2024-09-18T14:03:51.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/2053342394 shutdown_connections 2024-09-18T14:03:51.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.518+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/2053342394 >> 172.21.15.45:0/2053342394 conn(0x7f6d740a5bb0 msgr2=0x7f6d740a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:51.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.519+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/2053342394 shutdown_connections 2024-09-18T14:03:51.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.519+0000 7f6d7bd1d640 1 -- 172.21.15.45:0/2053342394 wait complete. 2024-09-18T14:03:51.560 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:51.560 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-18T14:03:51.560 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-18T14:03:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.654+0000 7fd46e6c9640 10 monclient: get_monmap_and_config 2024-09-18T14:03:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.654+0000 7fd46e6c9640 10 monclient: build_initial_monmap 2024-09-18T14:03:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.654+0000 7fd46e6c9640 10 monclient: monmap: 2024-09-18T14:03:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:51.655564+0000 2024-09-18T14:03:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:51.655564+0000 2024-09-18T14:03:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:51.656 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 1 Processor -- start 2024-09-18T14:03:51.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 1 -- start start 2024-09-18T14:03:51.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 10 monclient: init 2024-09-18T14:03:51.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:51.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 10 monclient: _add_conns ranks=[2,1,0,3,6,4,5,8,7] 2024-09-18T14:03:51.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 10 monclient(hunting): picked mon.noname-b con 0x7fd46809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:03:51.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 10 monclient(hunting): picked mon.noname-f con 0x7fd46809fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:51.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 10 monclient(hunting): picked mon.noname-a con 0x7fd46809ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:51.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46805ed70 con 0x7fd46809ff70 2024-09-18T14:03:51.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46809dfc0 con 0x7fd46809fba0 2024-09-18T14:03:51.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46809dce0 con 0x7fd46809f7d0 2024-09-18T14:03:51.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:51.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.655+0000 7fd46e6c9640 10 monclient(hunting): authenticate will time out at 1046.993928s 2024-09-18T14:03:51.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd467fff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fd46809f7d0 0x7fd468132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49082/0 (socket says 172.21.15.45:49082) 2024-09-18T14:03:51.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd46cc66640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd46809ff70 0x7fd46813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44862/0 (socket says 172.21.15.45:44862) 2024-09-18T14:03:51.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd467fff640 1 -- 172.21.15.45:0/2164587589 learned_addr learned my addr 172.21.15.45:0/2164587589 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:51.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678538110 0 0) 0x7fd46809dce0 con 0x7fd46809f7d0 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient(hunting): my global_id is 144155 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4500034e0 con 0x7fd46809f7d0 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541599980 0 0) 0x7fd46805ed70 con 0x7fd46809ff70 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient(hunting): my global_id is 4905 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd46809dce0 con 0x7fd46809ff70 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525789773 0 0) 0x7fd46809dfc0 con 0x7fd46809fba0 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient(hunting): my global_id is 4825 2024-09-18T14:03:51.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd46805ed70 con 0x7fd46809fba0 2024-09-18T14:03:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3404699922 0 0) 0x7fd46809dce0 con 0x7fd46809ff70 2024-09-18T14:03:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 >> v1:172.21.15.45:6790/0 conn(0x7fd46809f7d0 legacy=0x7fd468132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 >> v1:172.21.15.84:6789/0 conn(0x7fd46809fba0 legacy=0x7fd468137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:03:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 monclient: found mon.noname-a 2024-09-18T14:03:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:03:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4680a1c00 con 0x7fd46809ff70 2024-09-18T14:03:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient: _finish_auth 0 2024-09-18T14:03:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd460001b10 con 0x7fd46809ff70 2024-09-18T14:03:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd46e6c9640 5 monclient: authenticate success, global_id 4905 2024-09-18T14:03:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd46e6c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:03:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient: dump: 2024-09-18T14:03:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd46e6c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd460001c70 con 0x7fd46809ff70 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd46e6c9640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 1 -- 172.21.15.45:0/2164587589 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd460004b20 con 0x7fd46809ff70 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd46e6c9640 4 set_mon_vals no callback set 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.656+0000 7fd466ffd640 10 monclient: dump: 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.671 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.672 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.657+0000 7fd46e6c9640 10 monclient: shutdown 2024-09-18T14:03:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.657+0000 7fd46e6c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.657+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2164587589 >> v1:172.21.15.45:6789/0 conn(0x7fd46809ff70 legacy=0x7fd46813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.657+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2164587589 shutdown_connections 2024-09-18T14:03:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.657+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2164587589 >> 172.21.15.45:0/2164587589 conn(0x7fd4680630b0 msgr2=0x7fd4680634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.657+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2164587589 shutdown_connections 2024-09-18T14:03:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.657+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2164587589 wait complete. 2024-09-18T14:03:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.657+0000 7fd46e6c9640 10 monclient: build_initial_monmap 2024-09-18T14:03:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.657+0000 7fd46e6c9640 10 monclient: monmap: 2024-09-18T14:03:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 1 Processor -- start 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 1 -- start start 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 10 monclient: init 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 10 monclient: _add_conns ranks=[6,8,1,2,5,7,3,0,4] 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 10 monclient(hunting): picked mon.noname-d con 0x7fd46809f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 10 monclient(hunting): picked mon.noname-e con 0x7fd46809fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 10 monclient(hunting): picked mon.noname-f con 0x7fd46809ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4680a2560 con 0x7fd46809ff70 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46811e720 con 0x7fd46809f7d0 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd468136ea0 con 0x7fd46809fba0 2024-09-18T14:03:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd46e6c9640 10 monclient(hunting): authenticate will time out at 1046.996837s 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd467fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fd46809f7d0 0x7fd468131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:58946/0 (socket says 172.21.15.45:58946) 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd467fff640 1 -- 172.21.15.45:0/2254131856 learned_addr learned my addr 172.21.15.45:0/2254131856 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.658+0000 7fd4677fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fd46809fba0 0x7fd468146cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:50062/0 (socket says 172.21.15.45:50062) 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560139158 0 0) 0x7fd46811e720 con 0x7fd46809f7d0 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): my global_id is 4767 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4540034e0 con 0x7fd46809f7d0 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390648505 0 0) 0x7fd468136ea0 con 0x7fd46809fba0 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): my global_id is 4922 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd46811e720 con 0x7fd46809fba0 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204578735 0 0) 0x7fd4680a2560 con 0x7fd46809ff70 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): my global_id is 4834 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd468136ea0 con 0x7fd46809ff70 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 552153139 0 0) 0x7fd4540034e0 con 0x7fd46809f7d0 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd4680a2560 con 0x7fd46809f7d0 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd458001520 con 0x7fd46809f7d0 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): dump: 2024-09-18T14:03:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.678 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3240243022 0 0) 0x7fd46811e720 con 0x7fd46809fba0 2024-09-18T14:03:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd4540034e0 con 0x7fd46809fba0 2024-09-18T14:03:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd45c001aa0 con 0x7fd46809fba0 2024-09-18T14:03:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:03:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): dump: 2024-09-18T14:03:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1115724976 0 0) 0x7fd468136ea0 con 0x7fd46809ff70 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd46811e720 con 0x7fd46809ff70 2024-09-18T14:03:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd460001a50 con 0x7fd46809ff70 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient(hunting): dump: 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.681 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.682 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.682 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.682 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.682 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.682 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.682 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 886729064 0 0) 0x7fd4680a2560 con 0x7fd46809f7d0 2024-09-18T14:03:51.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 >> v1:172.21.15.45:6793/0 conn(0x7fd46809fba0 legacy=0x7fd468146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 >> v1:172.21.15.84:6789/0 conn(0x7fd46809ff70 legacy=0x7fd46814a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient: _finish_hunting 0 2024-09-18T14:03:51.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 monclient: found mon.d 2024-09-18T14:03:51.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46811ece0 con 0x7fd46809f7d0 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient: _finish_auth 0 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd46e6c9640 5 monclient: authenticate success, global_id 4767 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd46e6c9640 10 monclient: _renew_subs 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd46e6c9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2254131856 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46811efc0 con 0x7fd46809f7d0 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd46e6c9640 10 monclient: _renew_subs 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd46e6c9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2254131856 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46809b800 con 0x7fd46809f7d0 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd458004a50 con 0x7fd46809f7d0 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:51.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd458004f00 con 0x7fd46809f7d0 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd4667fc640 4 set_mon_vals no callback set 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.659+0000 7fd44ffff640 10 monclient: dump: 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:51.684 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.660+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd45800fed0 con 0x7fd46809f7d0 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.660+0000 7fd46e6c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.660+0000 7fd46e6c9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.660+0000 7fd46e6c9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.660+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2254131856 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd448005010 con 0x7fd46809f7d0 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.660+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(228..228 src has 1..228) v4 ==== 3911+0+0 (unknown 2442380505 0 0) 0x7fd458046ae0 con 0x7fd46809f7d0 2024-09-18T14:03:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.660+0000 7fd4677fe640 10 monclient: get_auth_request con 0x7fd45403aa00 auth_method 0 2024-09-18T14:03:51.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.665+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fd458014ea0 con 0x7fd46809f7d0 2024-09-18T14:03:51.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.665+0000 7fd44ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:51.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.665+0000 7fd44ffff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:51.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.891+0000 7fd46e6c9640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-18T14:03:51.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.891+0000 7fd46e6c9640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-18T14:03:51.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.891+0000 7fd46e6c9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:51.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.891+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2254131856 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fd448005ad0 con 0x7fd46809f7d0 2024-09-18T14:03:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.893+0000 7fd44ffff640 1 -- 172.21.15.45:0/2254131856 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7fd458014cc0 con 0x7fd46809f7d0 2024-09-18T14:03:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.893+0000 7fd44ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-18T14:03:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.893+0000 7fd44ffff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:51.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=2427205994,config=3140514009,health=3887269300,kv=594742435,logm=914431919}) 2024-09-18T14:03:51.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=21,config=1,health=12,kv=7,logm=59} crc {auth=266059306,config=1228700967,health=3887269300,kv=1989676319,logm=3594436687}) 2024-09-18T14:03:51.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=2427205994,config=3140514009,health=3887269300,kv=594742435,logm=914431919}) 2024-09-18T14:03:51.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=1,health=12,kv=7,logm=61} crc {auth=2348731068,config=1228700967,health=3806841325,kv=210776046,logm=1072932470}) 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=2427205994,config=3140514009,health=3887269300,kv=594742435,logm=914431919}) 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=21,config=2,health=12,kv=8,logm=57} crc {auth=570004115,config=3140514009,health=3887269300,kv=594742435,logm=3481129164}) 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=2427205994,config=3140514009,health=3887269300,kv=594742435,logm=914431919}) 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=20,config=2,health=11,kv=8,logm=59} crc {auth=2699488284,config=3140514009,health=1430206638,kv=594742435,logm=1969856839}) 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=2427205994,config=3140514009,health=3887269300,kv=594742435,logm=914431919}) 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=50106953,config=3140514009,health=3887269300,kv=594742435,logm=1414161688}) 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=2427205994,config=3140514009,health=3887269300,kv=594742435,logm=914431919}) 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=7,logm=60} crc {auth=3396010190,config=3140514009,health=1864161850,kv=11119927,logm=1905246386}) 2024-09-18T14:03:51.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=2427205994,config=3140514009,health=3887269300,kv=594742435,logm=914431919}) 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=20,config=2,health=12,kv=7,logm=59} crc {auth=2727779937,config=3140514009,health=3887269300,kv=4140378858,logm=1868536196}) 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=2427205994,config=3140514009,health=3887269300,kv=594742435,logm=914431919}) 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.893+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=19,config=1,health=12,kv=7,logm=61} crc {auth=204864371,config=1228700967,health=3887269300,kv=594742435,logm=1113894286}) 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.894+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.894+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280584422}) 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.894+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1237521887}) 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.894+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.894+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280584422}) 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.894+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1152121778}) 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.894+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280584422}) 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4228096303}) 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280584422}) 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1922794286}) 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280584422}) 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2162840114}) 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280584422}) 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=507851998}) 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280584422}) 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2454929147}) 2024-09-18T14:03:51.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280584422}) 2024-09-18T14:03:51.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.895+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2729080790}) 2024-09-18T14:03:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.895+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2254131856 >> v1:172.21.15.84:6812/31478 conn(0x7fd45403aa00 legacy=0x7fd45403ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.896+0000 7fd46e6c9640 10 monclient: shutdown 2024-09-18T14:03:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.896+0000 7fd46e6c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.896+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2254131856 >> v1:172.21.15.45:6792/0 conn(0x7fd46809f7d0 legacy=0x7fd468131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.896+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2254131856 shutdown_connections 2024-09-18T14:03:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.896+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2254131856 >> 172.21.15.45:0/2254131856 conn(0x7fd4680630b0 msgr2=0x7fd4680a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:51.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1027967503}) 2024-09-18T14:03:51.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=33159951}) 2024-09-18T14:03:51.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.896+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2254131856 shutdown_connections 2024-09-18T14:03:51.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1027967503}) 2024-09-18T14:03:51.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1745669076}) 2024-09-18T14:03:51.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:51.896+0000 7fd46e6c9640 1 -- 172.21.15.45:0/2254131856 wait complete. 2024-09-18T14:03:51.921 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1027967503}) 2024-09-18T14:03:51.921 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2003341985}) 2024-09-18T14:03:51.921 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.921 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1027967503}) 2024-09-18T14:03:51.921 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1279107865}) 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1027967503}) 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3581907684}) 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1027967503}) 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1512364092}) 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1027967503}) 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2741463721}) 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1027967503}) 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.896+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1535654475}) 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.922 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31969406}) 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914498399}) 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31969406}) 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1675182083}) 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31969406}) 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3866097571}) 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31969406}) 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1161070983}) 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31969406}) 2024-09-18T14:03:51.923 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3127313027}) 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31969406}) 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3763692004}) 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31969406}) 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1240693217}) 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31969406}) 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.897+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1250163848}) 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.898+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.898+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=540660454}) 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.898+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888159393}) 2024-09-18T14:03:51.924 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.898+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.925 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.898+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=540660454}) 2024-09-18T14:03:51.925 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.898+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2903148245}) 2024-09-18T14:03:51.925 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.898+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.925 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.898+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=540660454}) 2024-09-18T14:03:51.925 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.898+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3124712763}) 2024-09-18T14:03:51.925 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.898+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.925 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.898+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=540660454}) 2024-09-18T14:03:51.925 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.898+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3349357884}) 2024-09-18T14:03:51.925 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.899+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.925 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.899+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=540660454}) 2024-09-18T14:03:51.925 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.899+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3562389568}) 2024-09-18T14:03:51.925 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.899+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.926 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.899+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=540660454}) 2024-09-18T14:03:51.926 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.899+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=181702222}) 2024-09-18T14:03:51.926 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.899+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.926 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.899+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=540660454}) 2024-09-18T14:03:51.926 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.899+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2605524118}) 2024-09-18T14:03:51.926 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.899+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.927 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.899+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=540660454}) 2024-09-18T14:03:51.927 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.899+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=697428965}) 2024-09-18T14:03:51.927 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.927 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3933720689}) 2024-09-18T14:03:51.927 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2514878947}) 2024-09-18T14:03:51.927 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.927 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3933720689}) 2024-09-18T14:03:51.927 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=906277137}) 2024-09-18T14:03:51.927 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.927 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3933720689}) 2024-09-18T14:03:51.927 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4213182752}) 2024-09-18T14:03:51.927 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3933720689}) 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1515020434}) 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3933720689}) 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2150816129}) 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3933720689}) 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2756986557}) 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3933720689}) 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3283897347}) 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.928 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3933720689}) 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.900+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1882736279}) 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4095626659}) 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3470813136}) 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4095626659}) 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2521039975}) 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4095626659}) 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1031023345}) 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4095626659}) 2024-09-18T14:03:51.929 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1565759478}) 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4095626659}) 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=623849739}) 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4095626659}) 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1621444365}) 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4095626659}) 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2880053827}) 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4095626659}) 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.901+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3363739358}) 2024-09-18T14:03:51.930 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2724493537}) 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3462849181}) 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2724493537}) 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2096416437}) 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2724493537}) 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3140314460}) 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2724493537}) 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2005936901}) 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2724493537}) 2024-09-18T14:03:51.931 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1711052579}) 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2724493537}) 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=479905168}) 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2724493537}) 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1847603932}) 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2724493537}) 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.903+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4145931153}) 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.904+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.904+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3352197117}) 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1217490945}) 2024-09-18T14:03:51.932 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3352197117}) 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2011774218}) 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3352197117}) 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=856297962}) 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3352197117}) 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3375939180}) 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3352197117}) 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3896996334}) 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3352197117}) 2024-09-18T14:03:51.933 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2503904316}) 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3352197117}) 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1979766364}) 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3352197117}) 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.905+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1883096187}) 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1040895669}) 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1883503871}) 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1040895669}) 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2850992419}) 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.934 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1040895669}) 2024-09-18T14:03:51.935 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2451840785}) 2024-09-18T14:03:51.935 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.935 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1040895669}) 2024-09-18T14:03:51.935 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1239485358}) 2024-09-18T14:03:51.935 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.935 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1040895669}) 2024-09-18T14:03:51.935 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2067496162}) 2024-09-18T14:03:51.935 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.935 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1040895669}) 2024-09-18T14:03:51.935 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=556674056}) 2024-09-18T14:03:51.935 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1040895669}) 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4173566579}) 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1040895669}) 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.906+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3877884791}) 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447304919}) 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2619763793}) 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447304919}) 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3802137988}) 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447304919}) 2024-09-18T14:03:51.936 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2958430817}) 2024-09-18T14:03:51.937 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.937 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447304919}) 2024-09-18T14:03:51.937 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=563399070}) 2024-09-18T14:03:51.937 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.937 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447304919}) 2024-09-18T14:03:51.937 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=635151212}) 2024-09-18T14:03:51.937 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.937 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447304919}) 2024-09-18T14:03:51.937 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2176344389}) 2024-09-18T14:03:51.938 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.938 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447304919}) 2024-09-18T14:03:51.938 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3894094867}) 2024-09-18T14:03:51.938 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.938 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447304919}) 2024-09-18T14:03:51.938 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.908+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=971091786}) 2024-09-18T14:03:51.938 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.939 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4090284359}) 2024-09-18T14:03:51.939 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=561709863}) 2024-09-18T14:03:51.939 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.939 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4090284359}) 2024-09-18T14:03:51.939 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1899265395}) 2024-09-18T14:03:51.939 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.939 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4090284359}) 2024-09-18T14:03:51.940 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3364523344}) 2024-09-18T14:03:51.940 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.940 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4090284359}) 2024-09-18T14:03:51.940 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=94850228}) 2024-09-18T14:03:51.940 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.940 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4090284359}) 2024-09-18T14:03:51.940 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=495675092}) 2024-09-18T14:03:51.940 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.940 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4090284359}) 2024-09-18T14:03:51.941 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4193127896}) 2024-09-18T14:03:51.941 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.941 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4090284359}) 2024-09-18T14:03:51.941 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3730475994}) 2024-09-18T14:03:51.941 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.941 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4090284359}) 2024-09-18T14:03:51.941 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.909+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1573778051}) 2024-09-18T14:03:51.942 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-18T14:03:51.951 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.910+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.951 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.910+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=90,mdsmap=3,mgr=7} crc {logm=1337700600,mdsmap=1531151339,mgr=2394081750}) 2024-09-18T14:03:51.951 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.910+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=89,mdsmap=3,mgr=7,mgr_command_descs=1} crc {logm=3735845615,mdsmap=1531151339,mgr=1821329879,mgr_command_descs=3545737492}) 2024-09-18T14:03:51.951 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.951 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=90,mdsmap=3,mgr=7} crc {logm=1337700600,mdsmap=1531151339,mgr=2394081750}) 2024-09-18T14:03:51.952 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=90,mdsmap=3,mgr=7} crc {logm=1373919193,mdsmap=1531151339,mgr=1301466042}) 2024-09-18T14:03:51.952 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.952 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=90,mdsmap=3,mgr=7} crc {logm=1337700600,mdsmap=1531151339,mgr=2394081750}) 2024-09-18T14:03:51.952 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=88,mdsmap=3,mgr=8,mgr_command_descs=1} crc {logm=136620635,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492}) 2024-09-18T14:03:51.952 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.952 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=90,mdsmap=3,mgr=7} crc {logm=1337700600,mdsmap=1531151339,mgr=2394081750}) 2024-09-18T14:03:51.953 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=91,mdsmap=3,mgr=6} crc {logm=544823911,mdsmap=1531151339,mgr=215977734}) 2024-09-18T14:03:51.953 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.953 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=90,mdsmap=3,mgr=7} crc {logm=1337700600,mdsmap=1531151339,mgr=2394081750}) 2024-09-18T14:03:51.953 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=88,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1} crc {logm=1121923875,mdsmap=1531151339,mgr=1301466042,mgr_command_descs=3545737492,mgr_metadata=270970526}) 2024-09-18T14:03:51.953 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.954 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=90,mdsmap=3,mgr=7} crc {logm=1337700600,mdsmap=1531151339,mgr=2394081750}) 2024-09-18T14:03:51.954 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=89,mdsmap=3,mgr=7,mgr_command_descs=1} crc {logm=3178690248,mdsmap=1531151339,mgr=3670438315,mgr_command_descs=3545737492}) 2024-09-18T14:03:51.954 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.954 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=90,mdsmap=3,mgr=7} crc {logm=1337700600,mdsmap=1531151339,mgr=2394081750}) 2024-09-18T14:03:51.954 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=85,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=2} crc {logm=3408290834,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3585531655}) 2024-09-18T14:03:51.955 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.955 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=90,mdsmap=3,mgr=7} crc {logm=1337700600,mdsmap=1531151339,mgr=2394081750}) 2024-09-18T14:03:51.955 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.911+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=87,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1} crc {logm=2806727615,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526}) 2024-09-18T14:03:51.955 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.955 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=37,osdmap=37} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1974992031,mon_config_key=1810658514,monmap=455718034,osd_metadata=3328057993,osd_pg_creating=2005445240,osd_snap=1869096063,osdmap=716007401}) 2024-09-18T14:03:51.955 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=38,osdmap=36} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3873397444,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2005445240,osd_snap=357174982,osdmap=91081770}) 2024-09-18T14:03:51.955 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.956 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=37,osdmap=37} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1974992031,mon_config_key=1810658514,monmap=455718034,osd_metadata=3328057993,osd_pg_creating=2005445240,osd_snap=1869096063,osdmap=716007401}) 2024-09-18T14:03:51.956 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37,osdmap=36} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1974992031,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2005445240,osd_snap=3614621692,osdmap=742011391}) 2024-09-18T14:03:51.956 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.956 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=37,osdmap=37} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1974992031,mon_config_key=1810658514,monmap=455718034,osd_metadata=3328057993,osd_pg_creating=2005445240,osd_snap=1869096063,osdmap=716007401}) 2024-09-18T14:03:51.956 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=38,osdmap=36} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=52438326,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2005445240,osd_snap=3941699571,osdmap=548737443}) 2024-09-18T14:03:51.956 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.956 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=37,osdmap=37} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1974992031,mon_config_key=1810658514,monmap=455718034,osd_metadata=3328057993,osd_pg_creating=2005445240,osd_snap=1869096063,osdmap=716007401}) 2024-09-18T14:03:51.956 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_snap=36,osdmap=40} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2600958375,mon_config_key=1045651642,monmap=455718034,osd_metadata=2800182873,osd_snap=3802913740,osdmap=2004491193}) 2024-09-18T14:03:51.957 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.957 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=37,osdmap=37} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1974992031,mon_config_key=1810658514,monmap=455718034,osd_metadata=3328057993,osd_pg_creating=2005445240,osd_snap=1869096063,osdmap=716007401}) 2024-09-18T14:03:51.957 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=39,osdmap=35} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1100944807,mon_config_key=1810658514,monmap=455718034,osd_metadata=456487060,osd_pg_creating=2005445240,osd_snap=956719594,osdmap=4056499967}) 2024-09-18T14:03:51.957 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.957 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=37,osdmap=37} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1974992031,mon_config_key=1810658514,monmap=455718034,osd_metadata=3328057993,osd_pg_creating=2005445240,osd_snap=1869096063,osdmap=716007401}) 2024-09-18T14:03:51.957 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgr_command_descs=1,mgrstat=10,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39,osdmap=36} crc {mgr_command_descs=3545737492,mgrstat=1974992031,mon_config_key=2098055267,monmap=455718034,osd_metadata=322097640,osd_pg_creating=2005445240,osd_snap=956719594,osdmap=313274912}) 2024-09-18T14:03:51.957 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.957 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=37,osdmap=37} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1974992031,mon_config_key=1810658514,monmap=455718034,osd_metadata=3328057993,osd_pg_creating=2005445240,osd_snap=1869096063,osdmap=716007401}) 2024-09-18T14:03:51.957 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=37} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1974992031,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2005445240,osd_snap=1438644354,osdmap=3212416889}) 2024-09-18T14:03:51.958 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.958 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=37,osdmap=37} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1974992031,mon_config_key=1810658514,monmap=455718034,osd_metadata=3328057993,osd_pg_creating=2005445240,osd_snap=1869096063,osdmap=716007401}) 2024-09-18T14:03:51.958 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.912+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37,osdmap=37} crc {mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2470431063,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2005445240,osd_snap=3590948473,osdmap=2207386026}) 2024-09-18T14:03:51.958 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.913+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.958 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.913+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=165162821}) 2024-09-18T14:03:51.958 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.913+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3038138733}) 2024-09-18T14:03:51.958 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.913+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.958 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.913+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=165162821}) 2024-09-18T14:03:51.958 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.913+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4192064868}) 2024-09-18T14:03:51.958 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.913+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.959 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=165162821}) 2024-09-18T14:03:51.959 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4172564659}) 2024-09-18T14:03:51.959 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.959 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=165162821}) 2024-09-18T14:03:51.959 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3465604631}) 2024-09-18T14:03:51.959 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.959 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=165162821}) 2024-09-18T14:03:51.960 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1651296890}) 2024-09-18T14:03:51.960 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.960 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=165162821}) 2024-09-18T14:03:51.960 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=880883533}) 2024-09-18T14:03:51.960 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.960 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=165162821}) 2024-09-18T14:03:51.961 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1495735263}) 2024-09-18T14:03:51.961 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.961 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=165162821}) 2024-09-18T14:03:51.961 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.914+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1669657868}) 2024-09-18T14:03:51.961 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.961 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=609834862}) 2024-09-18T14:03:51.962 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2283546944}) 2024-09-18T14:03:51.962 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.962 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=609834862}) 2024-09-18T14:03:51.962 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1103598863}) 2024-09-18T14:03:51.962 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.962 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=609834862}) 2024-09-18T14:03:51.963 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1578905074}) 2024-09-18T14:03:51.963 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.963 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=609834862}) 2024-09-18T14:03:51.963 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3616233939}) 2024-09-18T14:03:51.963 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.963 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=609834862}) 2024-09-18T14:03:51.964 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3783399399}) 2024-09-18T14:03:51.964 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.964 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=609834862}) 2024-09-18T14:03:51.964 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1968421339}) 2024-09-18T14:03:51.964 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.964 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=609834862}) 2024-09-18T14:03:51.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1417330190}) 2024-09-18T14:03:51.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=609834862}) 2024-09-18T14:03:51.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.915+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3628804067}) 2024-09-18T14:03:51.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1549336048}) 2024-09-18T14:03:51.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1243957251}) 2024-09-18T14:03:51.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1549336048}) 2024-09-18T14:03:51.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2211759838}) 2024-09-18T14:03:51.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1549336048}) 2024-09-18T14:03:51.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2749815744}) 2024-09-18T14:03:51.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1549336048}) 2024-09-18T14:03:51.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=98} crc {osdmap=1383483157}) 2024-09-18T14:03:51.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1549336048}) 2024-09-18T14:03:51.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3117578136}) 2024-09-18T14:03:51.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1549336048}) 2024-09-18T14:03:51.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=805092979}) 2024-09-18T14:03:51.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.969 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1549336048}) 2024-09-18T14:03:51.969 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3438417181}) 2024-09-18T14:03:51.969 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:03:51.969 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1549336048}) 2024-09-18T14:03:51.969 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:03:51.917+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3264881962}) 2024-09-18T14:03:52.187 INFO:tasks.rados.rados.0.smithi084.stdout:987: done (2 left) 2024-09-18T14:03:52.187 INFO:tasks.rados.rados.0.smithi084.stdout:988: done (1 left) 2024-09-18T14:03:52.188 INFO:tasks.rados.rados.0.smithi084.stdout:989: done (0 left) 2024-09-18T14:03:52.188 INFO:tasks.rados.rados.0.smithi084.stdout:990: rollback oid 41 current snap is 111 2024-09-18T14:03:52.188 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 41 to 87 2024-09-18T14:03:52.188 INFO:tasks.rados.rados.0.smithi084.stdout:991: read oid 45 snap 105 2024-09-18T14:03:52.188 INFO:tasks.rados.rados.0.smithi084.stdout:991: expect deleted 2024-09-18T14:03:52.188 INFO:tasks.rados.rados.0.smithi084.stdout:992: delete oid 5 current snap is 111 2024-09-18T14:03:52.189 INFO:tasks.rados.rados.0.smithi084.stdout:992: done (2 left) 2024-09-18T14:03:52.189 INFO:tasks.rados.rados.0.smithi084.stdout:993: read oid 30 snap 87 2024-09-18T14:03:52.189 INFO:tasks.rados.rados.0.smithi084.stdout:993: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:03:52.189 INFO:tasks.rados.rados.0.smithi084.stdout:994: read oid 24 snap -1 2024-09-18T14:03:52.190 INFO:tasks.rados.rados.0.smithi084.stdout:994: expect (ObjNum 267 snap 68 seq_num 267) 2024-09-18T14:03:52.190 INFO:tasks.rados.rados.0.smithi084.stdout:995: write oid 4 current snap is 111 2024-09-18T14:03:52.190 INFO:tasks.rados.rados.0.smithi084.stdout:995: seq_num 358 ranges {416906=407733,1366083=511456} 2024-09-18T14:03:52.194 INFO:tasks.rados.rados.0.smithi084.stdout:995: writing smithi08434663-4 from 416906 to 824639 tid 1 2024-09-18T14:03:52.200 INFO:tasks.rados.rados.0.smithi084.stdout:995: writing smithi08434663-4 from 1366083 to 1877539 tid 2 2024-09-18T14:03:52.200 INFO:tasks.rados.rados.0.smithi084.stdout:996: write oid 13 current snap is 111 2024-09-18T14:03:52.200 INFO:tasks.rados.rados.0.smithi084.stdout:996: seq_num 359 ranges {433713=482982,1424565=609148} 2024-09-18T14:03:52.205 INFO:tasks.rados.rados.0.smithi084.stdout:996: writing smithi08434663-13 from 433713 to 916695 tid 1 2024-09-18T14:03:52.211 INFO:tasks.rados.rados.0.smithi084.stdout:996: writing smithi08434663-13 from 1424565 to 2033713 tid 2 2024-09-18T14:03:52.211 INFO:tasks.rados.rados.0.smithi084.stdout:997: snap_create 2024-09-18T14:03:52.212 INFO:tasks.rados.rados.0.smithi084.stdout:990: finishing rollback tid 0 to smithi08434663-41 2024-09-18T14:03:52.212 INFO:tasks.rados.rados.0.smithi084.stdout:990: finishing rollback tid 1 to smithi08434663-41 2024-09-18T14:03:52.212 INFO:tasks.rados.rados.0.smithi084.stdout:990: finishing rollback tid 2 to smithi08434663-41 2024-09-18T14:03:52.212 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 339 (ObjNum 152 snap 31 seq_num 152) dirty exists 2024-09-18T14:03:52.212 INFO:tasks.rados.rados.0.smithi084.stdout:993: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T14:03:52.289 INFO:tasks.rados.rados.0.smithi084.stdout:995: finishing write tid 1 to smithi08434663-4 2024-09-18T14:03:52.289 INFO:tasks.rados.rados.0.smithi084.stdout:994: expect (ObjNum 267 snap 68 seq_num 267) 2024-09-18T14:03:52.457 INFO:tasks.rados.rados.0.smithi084.stdout:995: finishing write tid 2 to smithi08434663-4 2024-09-18T14:03:52.457 INFO:tasks.rados.rados.0.smithi084.stdout:995: finishing write tid 3 to smithi08434663-4 2024-09-18T14:03:52.457 INFO:tasks.rados.rados.0.smithi084.stdout:995: finishing write tid 4 to smithi08434663-4 2024-09-18T14:03:52.457 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 342 (ObjNum 358 snap 111 seq_num 358) dirty exists 2024-09-18T14:03:52.457 INFO:tasks.rados.rados.0.smithi084.stdout:995: left oid 4 (ObjNum 358 snap 111 seq_num 358) 2024-09-18T14:03:52.457 INFO:tasks.rados.rados.0.smithi084.stdout:996: finishing write tid 1 to smithi08434663-13 2024-09-18T14:03:52.458 INFO:tasks.rados.rados.0.smithi084.stdout:996: finishing write tid 2 to smithi08434663-13 2024-09-18T14:03:52.458 INFO:tasks.rados.rados.0.smithi084.stdout:996: finishing write tid 3 to smithi08434663-13 2024-09-18T14:03:52.458 INFO:tasks.rados.rados.0.smithi084.stdout:996: finishing write tid 4 to smithi08434663-13 2024-09-18T14:03:52.458 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 320 (ObjNum 359 snap 111 seq_num 359) dirty exists 2024-09-18T14:03:52.458 INFO:tasks.rados.rados.0.smithi084.stdout:996: left oid 13 (ObjNum 359 snap 111 seq_num 359) 2024-09-18T14:03:52.944 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:03:52.944 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:03:53.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.052+0000 7f4f82260640 10 monclient: get_monmap_and_config 2024-09-18T14:03:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.052+0000 7f4f82260640 10 monclient: build_initial_monmap 2024-09-18T14:03:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.052+0000 7f4f82260640 10 monclient: monmap: 2024-09-18T14:03:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:53.053213+0000 2024-09-18T14:03:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:53.053213+0000 2024-09-18T14:03:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:53.054 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:53.054 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:53.054 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:53.054 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:53.054 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:53.054 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 1 Processor -- start 2024-09-18T14:03:53.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 1 -- start start 2024-09-18T14:03:53.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 10 monclient: init 2024-09-18T14:03:53.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:53.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 10 monclient: _add_conns ranks=[7,0,4,5,6,8,1,3,2] 2024-09-18T14:03:53.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 10 monclient(hunting): picked mon.noname-i con 0x7f4f7c136da0 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:53.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 10 monclient(hunting): picked mon.noname-a con 0x7f4f7c0a15b0 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 10 monclient(hunting): picked mon.noname-c con 0x7f4f7c13cd90 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7c05ed70 con 0x7f4f7c0a15b0 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7c0a1980 con 0x7f4f7c13cd90 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7c06bb70 con 0x7f4f7c136da0 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f82260640 10 monclient(hunting): authenticate will time out at 1048.391664s 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f7affd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4f7c0a15b0 0x7f4f7c137170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44870/0 (socket says 172.21.15.45:44870) 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.053+0000 7f4f7affd640 1 -- 172.21.15.45:0/1902758291 learned_addr learned my addr 172.21.15.45:0/1902758291 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091533990 0 0) 0x7f4f7c0a1980 con 0x7f4f7c13cd90 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient(hunting): my global_id is 144211 2024-09-18T14:03:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f640034e0 con 0x7f4f7c13cd90 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053658708 0 0) 0x7f4f7c05ed70 con 0x7f4f7c0a15b0 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient(hunting): my global_id is 4914 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f7c0a1980 con 0x7f4f7c0a15b0 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156872335 0 0) 0x7f4f7c06bb70 con 0x7f4f7c136da0 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient(hunting): my global_id is 4858 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f7c05ed70 con 0x7f4f7c136da0 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3002401881 0 0) 0x7f4f7c05ed70 con 0x7f4f7c136da0 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 >> v1:172.21.15.45:6791/0 conn(0x7f4f7c13cd90 legacy=0x7f4f7c13f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 >> v1:172.21.15.45:6789/0 conn(0x7f4f7c0a15b0 legacy=0x7f4f7c137170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 monclient: found mon.noname-i 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c06be50 con 0x7f4f7c136da0 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f6c0016d0 con 0x7f4f7c136da0 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient: dump: 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f82260640 5 monclient: authenticate success, global_id 4858 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f82260640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4f6c001830 con 0x7f4f7c136da0 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f82260640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 1 -- 172.21.15.45:0/1902758291 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f6c001de0 con 0x7f4f7c136da0 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f82260640 4 set_mon_vals no callback set 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.054+0000 7f4f7a7fc640 10 monclient: dump: 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.055+0000 7f4f82260640 10 monclient: shutdown 2024-09-18T14:03:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.055+0000 7f4f82260640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.055+0000 7f4f82260640 1 -- 172.21.15.45:0/1902758291 >> v1:172.21.15.84:6792/0 conn(0x7f4f7c136da0 legacy=0x7f4f7c06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.055+0000 7f4f82260640 1 -- 172.21.15.45:0/1902758291 shutdown_connections 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.055+0000 7f4f82260640 1 -- 172.21.15.45:0/1902758291 >> 172.21.15.45:0/1902758291 conn(0x7f4f7c0a5bb0 msgr2=0x7f4f7c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.055+0000 7f4f82260640 1 -- 172.21.15.45:0/1902758291 shutdown_connections 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.055+0000 7f4f82260640 1 -- 172.21.15.45:0/1902758291 wait complete. 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.055+0000 7f4f82260640 10 monclient: build_initial_monmap 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.055+0000 7f4f82260640 10 monclient: monmap: 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 1 Processor -- start 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 1 -- start start 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 10 monclient: init 2024-09-18T14:03:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 10 monclient: _add_conns ranks=[0,4,5,2,6,8,3,1,7] 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 10 monclient(hunting): picked mon.noname-a con 0x7f4f7c0a15b0 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 10 monclient(hunting): picked mon.noname-c con 0x7f4f7c136da0 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 10 monclient(hunting): picked mon.noname-h con 0x7f4f7c13cd90 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7c13ac20 con 0x7f4f7c0a15b0 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7c144500 con 0x7f4f7c136da0 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7c13b1e0 con 0x7f4f7c13cd90 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f82260640 10 monclient(hunting): authenticate will time out at 1048.394730s 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f7b7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4f7c0a15b0 0x7f4f7c144910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44876/0 (socket says 172.21.15.45:44876) 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f7affd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f4f7c136da0 0x7f4f7c14a9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38466/0 (socket says 172.21.15.45:38466) 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.056+0000 7f4f7b7fe640 1 -- 172.21.15.45:0/517733699 learned_addr learned my addr 172.21.15.45:0/517733699 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653242980 0 0) 0x7f4f7c144500 con 0x7f4f7c136da0 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): my global_id is 144220 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f680034e0 con 0x7f4f7c136da0 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83480639 0 0) 0x7f4f7c13ac20 con 0x7f4f7c0a15b0 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): my global_id is 4923 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f7c144500 con 0x7f4f7c0a15b0 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1887051082 0 0) 0x7f4f680034e0 con 0x7f4f7c136da0 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f7c13ac20 con 0x7f4f7c136da0 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f70001530 con 0x7f4f7c136da0 2024-09-18T14:03:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): dump: 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 714586513 0 0) 0x7f4f7c144500 con 0x7f4f7c0a15b0 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f680034e0 con 0x7f4f7c0a15b0 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254415400 0 0) 0x7f4f7c13b1e0 con 0x7f4f7c13cd90 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): my global_id is 4793 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f7c144500 con 0x7f4f7c13cd90 2024-09-18T14:03:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f6c001e20 con 0x7f4f7c0a15b0 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient(hunting): dump: 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3498648593 0 0) 0x7f4f7c13ac20 con 0x7f4f7c136da0 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 >> v1:172.21.15.84:6791/0 conn(0x7f4f7c13cd90 legacy=0x7f4f7c14e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 >> v1:172.21.15.45:6789/0 conn(0x7f4f7c0a15b0 legacy=0x7f4f7c144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient: _finish_hunting 0 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 monclient: found mon.c 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c13b4c0 con 0x7f4f7c136da0 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient: _finish_auth 0 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f82260640 5 monclient: authenticate success, global_id 144220 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f82260640 10 monclient: _renew_subs 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f82260640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f82260640 1 -- 172.21.15.45:0/517733699 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7c13af00 con 0x7f4f7c136da0 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f82260640 10 monclient: _renew_subs 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f82260640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f82260640 1 -- 172.21.15.45:0/517733699 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7c133710 con 0x7f4f7c136da0 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4f70001730 con 0x7f4f7c136da0 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f70004e10 con 0x7f4f7c136da0 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f79ffb640 4 set_mon_vals no callback set 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.057+0000 7f4f637fe640 10 monclient: dump: 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.077 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.077 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.077 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.058+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4f7000fde0 con 0x7f4f7c136da0 2024-09-18T14:03:53.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.058+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(229..229 src has 1..229) v4 ==== 3911+0+0 (unknown 2780910408 0 0) 0x7f4f70046b40 con 0x7f4f7c136da0 2024-09-18T14:03:53.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.058+0000 7f4f82260640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:53.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.058+0000 7f4f82260640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:53.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.058+0000 7f4f82260640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:53.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.058+0000 7f4f82260640 1 -- 172.21.15.45:0/517733699 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f5c005010 con 0x7f4f7c136da0 2024-09-18T14:03:53.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.058+0000 7f4f7b7fe640 10 monclient: get_auth_request con 0x7f4f6803a5b0 auth_method 0 2024-09-18T14:03:53.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.063+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.4 v1:172.21.15.45:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f4f700152c0 con 0x7f4f7c136da0 2024-09-18T14:03:53.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.063+0000 7f4f637fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:53.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.063+0000 7f4f637fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:53.185 INFO:tasks.rados.rados.0.smithi084.stdout:990: done (6 left) 2024-09-18T14:03:53.185 INFO:tasks.rados.rados.0.smithi084.stdout:991: done (5 left) 2024-09-18T14:03:53.185 INFO:tasks.rados.rados.0.smithi084.stdout:993: done (4 left) 2024-09-18T14:03:53.185 INFO:tasks.rados.rados.0.smithi084.stdout:994: done (3 left) 2024-09-18T14:03:53.185 INFO:tasks.rados.rados.0.smithi084.stdout:995: done (2 left) 2024-09-18T14:03:53.185 INFO:tasks.rados.rados.0.smithi084.stdout:996: done (1 left) 2024-09-18T14:03:53.185 INFO:tasks.rados.rados.0.smithi084.stdout:997: done (0 left) 2024-09-18T14:03:53.185 INFO:tasks.rados.rados.0.smithi084.stdout:998: write oid 39 current snap is 112 2024-09-18T14:03:53.186 INFO:tasks.rados.rados.0.smithi084.stdout:998: seq_num 360 ranges {450520=558231,1483047=515125} 2024-09-18T14:03:53.190 INFO:tasks.rados.rados.0.smithi084.stdout:998: writing smithi08434663-39 from 450520 to 1008751 tid 1 2024-09-18T14:03:53.196 INFO:tasks.rados.rados.0.smithi084.stdout:998: writing smithi08434663-39 from 1483047 to 1998172 tid 2 2024-09-18T14:03:53.196 INFO:tasks.rados.rados.0.smithi084.stdout:999: snap_remove snap 90 2024-09-18T14:03:53.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.291+0000 7f4f82260640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:03:53.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.291+0000 7f4f82260640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:03:53.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.291+0000 7f4f82260640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:53.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.291+0000 7f4f82260640 1 -- 172.21.15.45:0/517733699 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4f5c005840 con 0x7f4f7c136da0 2024-09-18T14:03:53.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.292+0000 7f4f637fe640 1 -- 172.21.15.45:0/517733699 <== mon.4 v1:172.21.15.45:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 1901058204) 0x7f4f70014cc0 con 0x7f4f7c136da0 2024-09-18T14:03:53.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.292+0000 7f4f637fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:03:53.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.292+0000 7f4f637fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:53.293 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":64,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:03:53.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.294+0000 7f4f82260640 1 -- 172.21.15.45:0/517733699 >> v1:172.21.15.84:6812/31478 conn(0x7f4f6803a5b0 legacy=0x7f4f6803ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.294+0000 7f4f82260640 10 monclient: shutdown 2024-09-18T14:03:53.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.294+0000 7f4f82260640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.294+0000 7f4f82260640 1 -- 172.21.15.45:0/517733699 >> v1:172.21.15.45:6791/0 conn(0x7f4f7c136da0 legacy=0x7f4f7c14a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.294+0000 7f4f82260640 1 -- 172.21.15.45:0/517733699 shutdown_connections 2024-09-18T14:03:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.294+0000 7f4f82260640 1 -- 172.21.15.45:0/517733699 >> 172.21.15.45:0/517733699 conn(0x7f4f7c0a5bb0 msgr2=0x7f4f7c134080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.294+0000 7f4f82260640 1 -- 172.21.15.45:0/517733699 shutdown_connections 2024-09-18T14:03:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.294+0000 7f4f82260640 1 -- 172.21.15.45:0/517733699 wait complete. 2024-09-18T14:03:53.304 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:03:53.304 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-18T14:03:53.304 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:03:53.305 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:03:53.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.401+0000 7fe396b9d640 10 monclient: get_monmap_and_config 2024-09-18T14:03:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.401+0000 7fe396b9d640 10 monclient: build_initial_monmap 2024-09-18T14:03:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.401+0000 7fe396b9d640 10 monclient: monmap: 2024-09-18T14:03:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:53.402212+0000 2024-09-18T14:03:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:53.402212+0000 2024-09-18T14:03:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:53.403 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.401+0000 7fe396b9d640 1 Processor -- start 2024-09-18T14:03:53.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe396b9d640 1 -- start start 2024-09-18T14:03:53.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe396b9d640 10 monclient: init 2024-09-18T14:03:53.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe396b9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:53.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe396b9d640 10 monclient: _add_conns ranks=[3,0,7,8,5,1,2,4,6] 2024-09-18T14:03:53.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe396b9d640 10 monclient(hunting): picked mon.noname-g con 0x7fe39009f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:03:53.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe396b9d640 10 monclient(hunting): picked mon.noname-a con 0x7fe39009fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:53.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe396b9d640 10 monclient(hunting): picked mon.noname-i con 0x7fe39009ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:53.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe396b9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39005ed70 con 0x7fe39009fba0 2024-09-18T14:03:53.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe396b9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39009dfc0 con 0x7fe39009f7d0 2024-09-18T14:03:53.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe396b9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39009dce0 con 0x7fe39009ff70 2024-09-18T14:03:53.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe396b9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:53.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe396b9d640 10 monclient(hunting): authenticate will time out at 1048.740551s 2024-09-18T14:03:53.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe38ffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe39009fba0 0x7fe390137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44880/0 (socket says 172.21.15.45:44880) 2024-09-18T14:03:53.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe38ffff640 1 -- 172.21.15.45:0/2461488689 learned_addr learned my addr 172.21.15.45:0/2461488689 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:53.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955725244 0 0) 0x7fe39005ed70 con 0x7fe39009fba0 2024-09-18T14:03:53.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe38f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe38f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe38f7fe640 10 monclient(hunting): my global_id is 4932 2024-09-18T14:03:53.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe37c0034e0 con 0x7fe39009fba0 2024-09-18T14:03:53.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029431274 0 0) 0x7fe39009dfc0 con 0x7fe39009f7d0 2024-09-18T14:03:53.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe38f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe38f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.402+0000 7fe38f7fe640 10 monclient(hunting): my global_id is 144156 2024-09-18T14:03:53.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39005ed70 con 0x7fe39009f7d0 2024-09-18T14:03:53.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639943822 0 0) 0x7fe39009dce0 con 0x7fe39009ff70 2024-09-18T14:03:53.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient(hunting): my global_id is 4867 2024-09-18T14:03:53.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39009dfc0 con 0x7fe39009ff70 2024-09-18T14:03:53.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3252915429 0 0) 0x7fe37c0034e0 con 0x7fe39009fba0 2024-09-18T14:03:53.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 >> v1:172.21.15.84:6792/0 conn(0x7fe39009ff70 legacy=0x7fe39013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 >> v1:172.21.15.84:6790/0 conn(0x7fe39009f7d0 legacy=0x7fe390132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:03:53.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 1 monclient: found mon.noname-a 2024-09-18T14:03:53.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:53.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:03:53.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3900a1c00 con 0x7fe39009fba0 2024-09-18T14:03:53.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient: _finish_auth 0 2024-09-18T14:03:53.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:53.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe380001830 con 0x7fe39009fba0 2024-09-18T14:03:53.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe396b9d640 5 monclient: authenticate success, global_id 4932 2024-09-18T14:03:53.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe396b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:53.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:03:53.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient: dump: 2024-09-18T14:03:53.418 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.418 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.418 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.419 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.419 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.419 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.419 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.419 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.419 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.419 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.419 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.419 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.420 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.420 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.420 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.420 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe396b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:53.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe380001a10 con 0x7fe39009fba0 2024-09-18T14:03:53.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:53.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 1 -- 172.21.15.45:0/2461488689 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe380004ba0 con 0x7fe39009fba0 2024-09-18T14:03:53.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe396b9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:53.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe396b9d640 4 set_mon_vals no callback set 2024-09-18T14:03:53.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:53.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe38f7fe640 10 monclient: dump: 2024-09-18T14:03:53.421 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.421 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.421 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.421 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.421 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.421 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.422 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.422 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.422 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.422 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.422 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.423 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.424 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.424 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.424 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.424 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe396b9d640 10 monclient: shutdown 2024-09-18T14:03:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe396b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe396b9d640 1 -- 172.21.15.45:0/2461488689 >> v1:172.21.15.45:6789/0 conn(0x7fe39009fba0 legacy=0x7fe390137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe396b9d640 1 -- 172.21.15.45:0/2461488689 shutdown_connections 2024-09-18T14:03:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe396b9d640 1 -- 172.21.15.45:0/2461488689 >> 172.21.15.45:0/2461488689 conn(0x7fe3900630b0 msgr2=0x7fe3900634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe396b9d640 1 -- 172.21.15.45:0/2461488689 shutdown_connections 2024-09-18T14:03:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.403+0000 7fe396b9d640 1 -- 172.21.15.45:0/2461488689 wait complete. 2024-09-18T14:03:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.404+0000 7fe396b9d640 10 monclient: build_initial_monmap 2024-09-18T14:03:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.404+0000 7fe396b9d640 10 monclient: monmap: 2024-09-18T14:03:53.426 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:53.426 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:53.426 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:53.426 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:53.426 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:53.427 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:53.427 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:53.427 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:53.427 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:53.427 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:53.428 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:53.428 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:53.428 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:53.428 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:53.428 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:53.429 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.404+0000 7fe396b9d640 1 Processor -- start 2024-09-18T14:03:53.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.404+0000 7fe396b9d640 1 -- start start 2024-09-18T14:03:53.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.404+0000 7fe396b9d640 10 monclient: init 2024-09-18T14:03:53.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe396b9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:53.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe396b9d640 10 monclient: _add_conns ranks=[2,6,8,3,1,7,0,4,5] 2024-09-18T14:03:53.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe396b9d640 10 monclient(hunting): picked mon.noname-b con 0x7fe39009f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:03:53.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe396b9d640 10 monclient(hunting): picked mon.noname-d con 0x7fe39009fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:53.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe396b9d640 10 monclient(hunting): picked mon.noname-e con 0x7fe39009ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:03:53.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe396b9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39009d310 con 0x7fe39009f7d0 2024-09-18T14:03:53.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe396b9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3900a2560 con 0x7fe39009fba0 2024-09-18T14:03:53.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe396b9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe390121cb0 con 0x7fe39009ff70 2024-09-18T14:03:53.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe396b9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:53.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe396b9d640 10 monclient(hunting): authenticate will time out at 1048.743390s 2024-09-18T14:03:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe394939640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fe39009f7d0 0x7fe390141810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49096/0 (socket says 172.21.15.45:49096) 2024-09-18T14:03:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe394939640 1 -- 172.21.15.45:0/2905116812 learned_addr learned my addr 172.21.15.45:0/2905116812 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003926534 0 0) 0x7fe3900a2560 con 0x7fe39009fba0 2024-09-18T14:03:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 10 monclient(hunting): my global_id is 4776 2024-09-18T14:03:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3700034e0 con 0x7fe39009fba0 2024-09-18T14:03:53.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146279248 0 0) 0x7fe390121cb0 con 0x7fe39009ff70 2024-09-18T14:03:53.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 10 monclient(hunting): my global_id is 4931 2024-09-18T14:03:53.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3900a2560 con 0x7fe39009ff70 2024-09-18T14:03:53.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011558297 0 0) 0x7fe39009d310 con 0x7fe39009f7d0 2024-09-18T14:03:53.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 10 monclient(hunting): my global_id is 144164 2024-09-18T14:03:53.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.405+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe390121cb0 con 0x7fe39009f7d0 2024-09-18T14:03:53.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4269538001 0 0) 0x7fe3700034e0 con 0x7fe39009fba0 2024-09-18T14:03:53.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe39009d310 con 0x7fe39009fba0 2024-09-18T14:03:53.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe380004b20 con 0x7fe39009fba0 2024-09-18T14:03:53.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:03:53.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient(hunting): dump: 2024-09-18T14:03:53.433 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.433 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.434 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.434 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.434 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.434 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.434 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.434 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.434 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.435 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.435 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.435 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.435 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.435 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.435 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.435 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:53.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2349343169 0 0) 0x7fe3900a2560 con 0x7fe39009ff70 2024-09-18T14:03:53.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe3700034e0 con 0x7fe39009ff70 2024-09-18T14:03:53.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1268651643 0 0) 0x7fe390121cb0 con 0x7fe39009f7d0 2024-09-18T14:03:53.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe3900a2560 con 0x7fe39009f7d0 2024-09-18T14:03:53.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe378001a50 con 0x7fe39009f7d0 2024-09-18T14:03:53.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:53.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient(hunting): dump: 2024-09-18T14:03:53.437 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.437 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.437 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.437 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.437 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.438 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.438 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.438 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.438 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.438 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.438 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.438 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.439 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.439 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.439 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.439 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe388001d00 con 0x7fe39009ff70 2024-09-18T14:03:53.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:03:53.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient(hunting): dump: 2024-09-18T14:03:53.440 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.440 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.440 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.440 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.440 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.441 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.441 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.441 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.441 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.441 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.441 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.441 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.442 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.442 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.442 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.442 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2298553764 0 0) 0x7fe39009d310 con 0x7fe39009fba0 2024-09-18T14:03:53.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6793/0 conn(0x7fe39009ff70 legacy=0x7fe390149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6790/0 conn(0x7fe39009f7d0 legacy=0x7fe390141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient: _finish_hunting 0 2024-09-18T14:03:53.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 monclient: found mon.d 2024-09-18T14:03:53.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:53.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:53.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39011ff10 con 0x7fe39009fba0 2024-09-18T14:03:53.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient: _finish_auth 0 2024-09-18T14:03:53.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:53.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe396b9d640 5 monclient: authenticate success, global_id 4776 2024-09-18T14:03:53.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe396b9d640 10 monclient: _renew_subs 2024-09-18T14:03:53.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe396b9d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:53.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe396b9d640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3901219d0 con 0x7fe39009fba0 2024-09-18T14:03:53.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe396b9d640 10 monclient: _renew_subs 2024-09-18T14:03:53.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe396b9d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe396b9d640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe390132130 con 0x7fe39009fba0 2024-09-18T14:03:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe380001780 con 0x7fe39009fba0 2024-09-18T14:03:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe380005160 con 0x7fe39009fba0 2024-09-18T14:03:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38effd640 4 set_mon_vals no callback set 2024-09-18T14:03:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.406+0000 7fe38cff9640 10 monclient: dump: 2024-09-18T14:03:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.447 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.407+0000 7fe396b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.407+0000 7fe396b9d640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe374006580 con 0x7fe374002830 2024-09-18T14:03:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.407+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fe380010130 con 0x7fe39009fba0 2024-09-18T14:03:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.407+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(230..230 src has 1..230) v4 ==== 3911+0+0 (unknown 4181126139 0 0) 0x7fe380046c40 con 0x7fe39009fba0 2024-09-18T14:03:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.407+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995706914 0 0) 0x7fe374006580 con 0x7fe374002830 2024-09-18T14:03:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.407+0000 7fe38cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.407+0000 7fe38cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.407+0000 7fe38cff9640 10 monclient(hunting): my global_id is 4941 2024-09-18T14:03:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.407+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39009d310 con 0x7fe374002830 2024-09-18T14:03:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.407+0000 7fe39513a640 10 monclient: get_auth_request con 0x7fe37003abb0 auth_method 0 2024-09-18T14:03:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.407+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 811339534 0 0) 0x7fe39009d310 con 0x7fe374002830 2024-09-18T14:03:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.408+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe374006580 con 0x7fe374002830 2024-09-18T14:03:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.408+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe378002160 con 0x7fe374002830 2024-09-18T14:03:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.408+0000 7fe38cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.408+0000 7fe38cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.408+0000 7fe38cff9640 10 monclient: dump: 2024-09-18T14:03:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.451 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.408+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3094330259 0 0) 0x7fe374006580 con 0x7fe374002830 2024-09-18T14:03:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.408+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe37400a0a0 con 0x7fe374002830 2024-09-18T14:03:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.408+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe3780024c0 con 0x7fe374002830 2024-09-18T14:03:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.408+0000 7fe38cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.408+0000 7fe38cff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.408+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6789/0 conn(0x7fe374002830 legacy=0x7fe374004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.408+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6789/0 conn(0x7fe374002830 legacy=0x7fe374004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe396b9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe396b9d640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe3740074f0 con 0x7fe3740068a0 2024-09-18T14:03:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535143503 0 0) 0x7fe3740074f0 con 0x7fe3740068a0 2024-09-18T14:03:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 10 monclient(hunting): my global_id is 4950 2024-09-18T14:03:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe374006580 con 0x7fe3740068a0 2024-09-18T14:03:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1328363424 0 0) 0x7fe374006580 con 0x7fe3740068a0 2024-09-18T14:03:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe3740074f0 con 0x7fe3740068a0 2024-09-18T14:03:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe378005630 con 0x7fe3740068a0 2024-09-18T14:03:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 10 monclient: dump: 2024-09-18T14:03:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.458 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe394939640 1 -- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6789/0 conn(0x7fe3740068a0 legacy=0x7fe374006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-18T14:03:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe394939640 1 -- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6789/0 conn(0x7fe3740068a0 legacy=0x7fe374006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:03:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe394939640 1 --1- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6789/0 conn(0x7fe3740068a0 0x7fe374006d20 :-1 s=OPENED pgs=222 cs=1 l=1).handle_message read tag failed 2024-09-18T14:03:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe394939640 1 --1- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6789/0 conn(0x7fe3740068a0 0x7fe374006d20 :-1 s=OPENED pgs=222 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:03:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6789/0 conn(0x7fe3740068a0 legacy=0x7fe374006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6789/0 conn(0x7fe3740068a0 legacy=0x7fe374006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.414+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe374006580 con 0x7fe370042dc0 2024-09-18T14:03:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683868239 0 0) 0x7fe374006580 con 0x7fe370042dc0 2024-09-18T14:03:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 10 monclient(hunting): my global_id is 4959 2024-09-18T14:03:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39009d310 con 0x7fe370042dc0 2024-09-18T14:03:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2834086353 0 0) 0x7fe39009d310 con 0x7fe370042dc0 2024-09-18T14:03:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe374006580 con 0x7fe370042dc0 2024-09-18T14:03:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe378005790 con 0x7fe370042dc0 2024-09-18T14:03:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 10 monclient: dump: 2024-09-18T14:03:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.461 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.461 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.461 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.461 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.461 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.461 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3829242286 0 0) 0x7fe374006580 con 0x7fe370042dc0 2024-09-18T14:03:53.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe3780024c0 con 0x7fe370042dc0 2024-09-18T14:03:53.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 459535115) 0x7fe3780082a0 con 0x7fe370042dc0 2024-09-18T14:03:53.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:53.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:53.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6789/0 conn(0x7fe370042dc0 legacy=0x7fe370043190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe38cff9640 1 -- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6789/0 conn(0x7fe370042dc0 legacy=0x7fe370043190 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:53.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.415+0000 7fe39513a640 1 -- 172.21.15.45:0/2905116812 reap_dead start 2024-09-18T14:03:53.463 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:53.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:53.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:53.464 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:53.464 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:53.464 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:53.464 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:53.464 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:53.465 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:53.465 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:53.465 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:53.465 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:53.465 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:53.466 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:53.466 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:53.466 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.466 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:03:53.466 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:53.467 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:53.467 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:53.467 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:53.467 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:53.467 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:53.468 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:53.468 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:53.468 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:53.468 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:53.468 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:53.469 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:53.469 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.469 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:53.469 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:53.469 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:53.470 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:53.470 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:53.470 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:53.470 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:53.470 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:53.471 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:53.471 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:53.471 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:53.471 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.471 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:53.472 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:53.472 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:53.472 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:53.472 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:53.472 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:53.473 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:53.473 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:53.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:53.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:53.473 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:53.473 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:53.473 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:53.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:53.474 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:53.474 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:53.474 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:53.474 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:53.474 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:53.474 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:53.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:53.474 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:53.474 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:53.474 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:53.475 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:53.475 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:53.475 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.475 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:53.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.475 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:53.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:53.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:53.475 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.475 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.476 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:53.476 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.476 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.476 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.476 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:53.476 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:53.476 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.476 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.477 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.477 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:53.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:53.477 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.477 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.478 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:53.478 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.478 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:53.478 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:53.479 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.479 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.479 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.479 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:53.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:53.479 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.479 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.480 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:53.480 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.480 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.480 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.480 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:53.480 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:53.480 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.480 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.480 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.481 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:53.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:53.481 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.481 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.481 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:53.482 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.482 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.482 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:53.482 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:53.482 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.482 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.482 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.483 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:53.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:53.483 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.483 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.483 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:53.484 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.484 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.484 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.484 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:53.484 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:53.484 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.484 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.484 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.485 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:53.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:53.485 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.485 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.485 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:53.485 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:53.486 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.487 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.487 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:53.487 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.487 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.487 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.487 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:53.487 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:53.487 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.488 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.488 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.488 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:53.488 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:53.488 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.488 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.488 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.489 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:53.489 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.489 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.489 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.489 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:53.489 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:53.489 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.489 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.489 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.490 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:53.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:53.490 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.490 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.490 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:53.491 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.491 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.491 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:53.491 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:53.491 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.491 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.491 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.492 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.492 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:53.492 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:53.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.492 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:53.492 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:53.492 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:53.492 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.492 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.493 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:53.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.493 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:53.493 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:53.493 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:03:53.493 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.493 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.493 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:53.494 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:53.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.416+0000 7fe396b9d640 1 -- 172.21.15.45:0/2905116812 >> v1:172.21.15.84:6812/31478 conn(0x7fe37003abb0 legacy=0x7fe37003d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.416+0000 7fe396b9d640 10 monclient: shutdown 2024-09-18T14:03:53.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.416+0000 7fe396b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:53.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.416+0000 7fe396b9d640 1 -- 172.21.15.45:0/2905116812 >> v1:172.21.15.45:6792/0 conn(0x7fe39009fba0 legacy=0x7fe390131880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.416+0000 7fe396b9d640 1 -- 172.21.15.45:0/2905116812 shutdown_connections 2024-09-18T14:03:53.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.416+0000 7fe396b9d640 1 -- 172.21.15.45:0/2905116812 >> 172.21.15.45:0/2905116812 conn(0x7fe3900630b0 msgr2=0x7fe3900a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:53.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.416+0000 7fe396b9d640 1 -- 172.21.15.45:0/2905116812 shutdown_connections 2024-09-18T14:03:53.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.416+0000 7fe396b9d640 1 -- 172.21.15.45:0/2905116812 wait complete. 2024-09-18T14:03:53.497 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:53.497 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:03:53.498 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:03:53.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.594+0000 7f68c2954640 10 monclient: get_monmap_and_config 2024-09-18T14:03:53.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.594+0000 7f68c2954640 10 monclient: build_initial_monmap 2024-09-18T14:03:53.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.595+0000 7f68c2954640 10 monclient: monmap: 2024-09-18T14:03:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:53.595954+0000 2024-09-18T14:03:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:53.595954+0000 2024-09-18T14:03:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:53.597 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:53.597 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:53.597 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:53.597 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:53.597 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.595+0000 7f68c2954640 1 Processor -- start 2024-09-18T14:03:53.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.595+0000 7f68c2954640 1 -- start start 2024-09-18T14:03:53.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.595+0000 7f68c2954640 10 monclient: init 2024-09-18T14:03:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c2954640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c2954640 10 monclient: _add_conns ranks=[5,6,8,1,4,7,2,0,3] 2024-09-18T14:03:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c2954640 10 monclient(hunting): picked mon.noname-h con 0x7f68bc06e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c2954640 10 monclient(hunting): picked mon.noname-d con 0x7f68bc0a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c2954640 10 monclient(hunting): picked mon.noname-e con 0x7f68bc0a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:03:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c2954640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc05ed70 con 0x7f68bc06e810 2024-09-18T14:03:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c2954640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc097620 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c2954640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc097340 con 0x7f68bc0a1080 2024-09-18T14:03:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c2954640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c2954640 10 monclient(hunting): authenticate will time out at 1048.934445s 2024-09-18T14:03:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c0ef1640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f68bc0a1080 0x7f68bc13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:50080/0 (socket says 172.21.15.45:50080) 2024-09-18T14:03:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68bb7fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f68bc0a0cb0 0x7f68bc137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:58974/0 (socket says 172.21.15.45:58974) 2024-09-18T14:03:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c0ef1640 1 -- 172.21.15.45:0/1690165391 learned_addr learned my addr 172.21.15.45:0/1690165391 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c0ef1640 1 -- 172.21.15.45:0/1690165391 >> v1:172.21.15.45:6793/0 conn(0x7f68bc0a1080 legacy=0x7f68bc13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-18T14:03:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c0ef1640 1 -- 172.21.15.45:0/1690165391 >> v1:172.21.15.45:6793/0 conn(0x7f68bc0a1080 legacy=0x7f68bc13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-18T14:03:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68c0ef1640 1 --1- 172.21.15.45:0/1690165391 >> v1:172.21.15.45:6793/0 conn(0x7f68bc0a1080 0x7f68bc13af40 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=0).handle_connect_reply_1 read connect reply failed 2024-09-18T14:03:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68baffd640 1 -- 172.21.15.45:0/1690165391 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707479563 0 0) 0x7f68bc097620 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68baffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68baffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68baffd640 10 monclient(hunting): my global_id is 4785 2024-09-18T14:03:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68baffd640 1 -- 172.21.15.45:0/1690165391 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68a40034e0 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68baffd640 1 -- 172.21.15.45:0/1690165391 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652316343 0 0) 0x7f68bc05ed70 con 0x7f68bc06e810 2024-09-18T14:03:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68baffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68baffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68baffd640 10 monclient(hunting): my global_id is 4802 2024-09-18T14:03:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.596+0000 7f68baffd640 1 -- 172.21.15.45:0/1690165391 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68bc097620 con 0x7f68bc06e810 2024-09-18T14:03:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 1 -- 172.21.15.45:0/1690165391 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2363323708 0 0) 0x7f68a40034e0 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 1 -- 172.21.15.45:0/1690165391 >> v1:172.21.15.45:6793/0 conn(0x7f68bc0a1080 legacy=0x7f68bc13af40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:03:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 1 -- 172.21.15.45:0/1690165391 >> v1:172.21.15.84:6791/0 conn(0x7f68bc06e810 legacy=0x7f68bc0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 10 monclient: _finish_hunting 0 2024-09-18T14:03:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 1 monclient: found mon.noname-d 2024-09-18T14:03:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:03:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 1 -- 172.21.15.45:0/1690165391 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc0a1660 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 10 monclient: _finish_auth 0 2024-09-18T14:03:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68c2954640 5 monclient: authenticate success, global_id 4785 2024-09-18T14:03:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 1 -- 172.21.15.45:0/1690165391 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68b0001830 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:03:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 10 monclient: dump: 2024-09-18T14:03:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.612 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68c2954640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 1 -- 172.21.15.45:0/1690165391 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f68b0001a10 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 1 -- 172.21.15.45:0/1690165391 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68b0004ae0 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68c2954640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68c2954640 4 set_mon_vals no callback set 2024-09-18T14:03:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68baffd640 10 monclient: dump: 2024-09-18T14:03:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.614 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.614 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.614 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.614 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.614 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68c2954640 10 monclient: shutdown 2024-09-18T14:03:53.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68c2954640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:53.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68c2954640 1 -- 172.21.15.45:0/1690165391 >> v1:172.21.15.45:6792/0 conn(0x7f68bc0a0cb0 legacy=0x7f68bc137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68c2954640 1 -- 172.21.15.45:0/1690165391 shutdown_connections 2024-09-18T14:03:53.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68c2954640 1 -- 172.21.15.45:0/1690165391 >> 172.21.15.45:0/1690165391 conn(0x7f68bc0a5bb0 msgr2=0x7f68bc0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:53.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68c2954640 1 -- 172.21.15.45:0/1690165391 shutdown_connections 2024-09-18T14:03:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.597+0000 7f68c2954640 1 -- 172.21.15.45:0/1690165391 wait complete. 2024-09-18T14:03:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.598+0000 7f68c2954640 10 monclient: build_initial_monmap 2024-09-18T14:03:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.598+0000 7f68c2954640 10 monclient: monmap: 2024-09-18T14:03:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:53.616 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.598+0000 7f68c2954640 1 Processor -- start 2024-09-18T14:03:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.598+0000 7f68c2954640 1 -- start start 2024-09-18T14:03:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.598+0000 7f68c2954640 10 monclient: init 2024-09-18T14:03:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68c2954640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68c2954640 10 monclient: _add_conns ranks=[1,0,7,5,3,4,2,8,6] 2024-09-18T14:03:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68c2954640 10 monclient(hunting): picked mon.noname-f con 0x7f68bc06e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68c2954640 10 monclient(hunting): picked mon.noname-a con 0x7f68bc0a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68c2954640 10 monclient(hunting): picked mon.noname-i con 0x7f68bc0a1080 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68c2954640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc122cd0 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68c2954640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc13daa0 con 0x7f68bc06e810 2024-09-18T14:03:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68c2954640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc122710 con 0x7f68bc0a1080 2024-09-18T14:03:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68c2954640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68c2954640 10 monclient(hunting): authenticate will time out at 1048.937230s 2024-09-18T14:03:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68bb7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f68bc0a0cb0 0x7f68bc1465e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44904/0 (socket says 172.21.15.45:44904) 2024-09-18T14:03:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68bb7fe640 1 -- 172.21.15.45:0/1063797864 learned_addr learned my addr 172.21.15.45:0/1063797864 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661539590 0 0) 0x7f68bc122cd0 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): my global_id is 4968 2024-09-18T14:03:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68a80034e0 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439751492 0 0) 0x7f68bc122710 con 0x7f68bc0a1080 2024-09-18T14:03:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): my global_id is 4876 2024-09-18T14:03:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68bc122cd0 con 0x7f68bc0a1080 2024-09-18T14:03:53.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3701282853 0 0) 0x7f68a80034e0 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68bc122710 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285114833 0 0) 0x7f68bc13daa0 con 0x7f68bc06e810 2024-09-18T14:03:53.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): my global_id is 4843 2024-09-18T14:03:53.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68a80034e0 con 0x7f68bc06e810 2024-09-18T14:03:53.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68b00047a0 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:03:53.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): dump: 2024-09-18T14:03:53.620 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.620 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.620 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.620 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.620 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.620 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.620 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.620 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.620 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.620 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.620 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.621 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.621 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.621 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.621 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.621 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:53.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3232815260 0 0) 0x7f68bc122cd0 con 0x7f68bc0a1080 2024-09-18T14:03:53.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68bc13daa0 con 0x7f68bc0a1080 2024-09-18T14:03:53.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68b40054e0 con 0x7f68bc0a1080 2024-09-18T14:03:53.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:03:53.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.599+0000 7f68a3fff640 10 monclient(hunting): dump: 2024-09-18T14:03:53.622 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.622 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.622 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.622 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.622 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.622 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.622 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.622 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.622 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.622 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.623 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.623 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.623 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.623 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.623 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.623 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1883452122 0 0) 0x7f68bc122710 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 >> v1:172.21.15.84:6792/0 conn(0x7f68bc0a1080 legacy=0x7f68bc149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 >> v1:172.21.15.84:6789/0 conn(0x7f68bc06e810 legacy=0x7f68bc13d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 10 monclient: _finish_hunting 0 2024-09-18T14:03:53.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 1 monclient: found mon.a 2024-09-18T14:03:53.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:53.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:03:53.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc122430 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 10 monclient: _finish_auth 0 2024-09-18T14:03:53.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:53.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68c2954640 5 monclient: authenticate success, global_id 4968 2024-09-18T14:03:53.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68c2954640 10 monclient: _renew_subs 2024-09-18T14:03:53.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68c2954640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:03:53.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68c2954640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68bc1229f0 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68c2954640 10 monclient: _renew_subs 2024-09-18T14:03:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68c2954640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:03:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68c2954640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f68bc121560 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f68b0004da0 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68b0005250 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68ba7fc640 4 set_mon_vals no callback set 2024-09-18T14:03:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 10 monclient: dump: 2024-09-18T14:03:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68c2954640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68c2954640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f689c006580 con 0x7f689c002830 2024-09-18T14:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.600+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f68b0010220 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(230..230 src has 1..230) v4 ==== 3911+0+0 (unknown 4181126139 0 0) 0x7f68b0046cc0 con 0x7f68bc0a0cb0 2024-09-18T14:03:53.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078630748 0 0) 0x7f689c006580 con 0x7f689c002830 2024-09-18T14:03:53.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 10 monclient(hunting): my global_id is 144173 2024-09-18T14:03:53.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68bc122710 con 0x7f689c002830 2024-09-18T14:03:53.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68c0ef1640 10 monclient: get_auth_request con 0x7f68a803a640 auth_method 0 2024-09-18T14:03:53.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4128834993 0 0) 0x7f68bc122710 con 0x7f689c002830 2024-09-18T14:03:53.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f689c006580 con 0x7f689c002830 2024-09-18T14:03:53.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68ac001820 con 0x7f689c002830 2024-09-18T14:03:53.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 10 monclient: dump: 2024-09-18T14:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2656301193 0 0) 0x7f689c006580 con 0x7f689c002830 2024-09-18T14:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.601+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f689c00a0a0 con 0x7f689c002830 2024-09-18T14:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.602+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f68ac0043a0 con 0x7f689c002830 2024-09-18T14:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.602+0000 7f68a3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.602+0000 7f68a3fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.602+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 >> v1:172.21.15.45:6790/0 conn(0x7f689c002830 legacy=0x7f689c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.602+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 >> v1:172.21.15.45:6790/0 conn(0x7f689c002830 legacy=0x7f689c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.607+0000 7f68c2954640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.607+0000 7f68c2954640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f689c009640 con 0x7f689c009020 2024-09-18T14:03:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893735441 0 0) 0x7f689c009640 con 0x7f689c009020 2024-09-18T14:03:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 10 monclient(hunting): my global_id is 144182 2024-09-18T14:03:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f689c006580 con 0x7f689c009020 2024-09-18T14:03:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2410228357 0 0) 0x7f689c006580 con 0x7f689c009020 2024-09-18T14:03:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f689c009640 con 0x7f689c009020 2024-09-18T14:03:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68ac002c30 con 0x7f689c009020 2024-09-18T14:03:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 10 monclient: dump: 2024-09-18T14:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3434779087 0 0) 0x7f689c009640 con 0x7f689c009020 2024-09-18T14:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f689c009900 con 0x7f689c009020 2024-09-18T14:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 2832266582) 0x7f68ac007390 con 0x7f689c009020 2024-09-18T14:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:53.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 >> v1:172.21.15.45:6790/0 conn(0x7f689c009020 legacy=0x7f689c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.608+0000 7f68a3fff640 1 -- 172.21.15.45:0/1063797864 >> v1:172.21.15.45:6790/0 conn(0x7f689c009020 legacy=0x7f689c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:53.637 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:03:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.647 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.647 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.647 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:53.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:53.647 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.647 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.647 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:53.647 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.648 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.648 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.648 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:53.648 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:53.648 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.648 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.648 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.648 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:53.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:53.649 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.649 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.649 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:53.649 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.649 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.649 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.651 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.651 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:53.651 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.651 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.651 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:53.651 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:53.651 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.651 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.652 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.652 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:53.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:53.652 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.652 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.652 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:53.652 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.653 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.653 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:53.653 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:53.653 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.653 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.653 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.653 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:53.655 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.655 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.655 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.655 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:53.655 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:53.655 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.655 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.655 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:53.656 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.656 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.656 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.656 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:53.656 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:53.656 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.656 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.656 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.657 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:53.657 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:53.657 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.657 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.657 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:53.657 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.657 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.657 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.657 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.658 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:53.659 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:53.659 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:53.659 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.659 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.659 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:53.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.659 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:53.659 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:53.659 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:53.659 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.659 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.660 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:53.660 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:53.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.609+0000 7f68c2954640 1 -- 172.21.15.45:0/1063797864 >> v1:172.21.15.84:6812/31478 conn(0x7f68a803a640 legacy=0x7f68a803caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.609+0000 7f68c2954640 10 monclient: shutdown 2024-09-18T14:03:53.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.609+0000 7f68c2954640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:53.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.609+0000 7f68c2954640 1 -- 172.21.15.45:0/1063797864 >> v1:172.21.15.45:6789/0 conn(0x7f68bc0a0cb0 legacy=0x7f68bc1465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.609+0000 7f68c0ef1640 1 -- 172.21.15.45:0/1063797864 reap_dead start 2024-09-18T14:03:53.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.609+0000 7f68c0ef1640 1 -- 172.21.15.45:0/1063797864 reap_dead start 2024-09-18T14:03:53.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.609+0000 7f68c2954640 1 -- 172.21.15.45:0/1063797864 shutdown_connections 2024-09-18T14:03:53.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.609+0000 7f68c2954640 1 -- 172.21.15.45:0/1063797864 >> 172.21.15.45:0/1063797864 conn(0x7f68bc0a5bb0 msgr2=0x7f68bc0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:53.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.609+0000 7f68c2954640 1 -- 172.21.15.45:0/1063797864 shutdown_connections 2024-09-18T14:03:53.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.609+0000 7f68c2954640 1 -- 172.21.15.45:0/1063797864 wait complete. 2024-09-18T14:03:53.662 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:53.663 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:03:53.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:03:53.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.760+0000 7f67f1cf8640 10 monclient: get_monmap_and_config 2024-09-18T14:03:53.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.760+0000 7f67f1cf8640 10 monclient: build_initial_monmap 2024-09-18T14:03:53.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.760+0000 7f67f1cf8640 10 monclient: monmap: 2024-09-18T14:03:53.761 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:53.761 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.761 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:53.761463+0000 2024-09-18T14:03:53.761 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:53.761463+0000 2024-09-18T14:03:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:53.763 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:53.763 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 1 Processor -- start 2024-09-18T14:03:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 1 -- start start 2024-09-18T14:03:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 10 monclient: init 2024-09-18T14:03:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 10 monclient: _add_conns ranks=[5,0,2,1,3,7,8,6,4] 2024-09-18T14:03:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 10 monclient(hunting): picked mon.noname-h con 0x7f67ec09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 10 monclient(hunting): picked mon.noname-a con 0x7f67ec09fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 10 monclient(hunting): picked mon.noname-b con 0x7f67ec09ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:03:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec05ed70 con 0x7f67ec09fba0 2024-09-18T14:03:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec09dfc0 con 0x7f67ec09ff70 2024-09-18T14:03:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec09dce0 con 0x7f67ec09f7d0 2024-09-18T14:03:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.761+0000 7f67f1cf8640 10 monclient(hunting): authenticate will time out at 1049.099993s 2024-09-18T14:03:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67eaffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f67ec09fba0 0x7f67ec137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44914/0 (socket says 172.21.15.45:44914) 2024-09-18T14:03:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ebfff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f67ec09ff70 0x7f67ec13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49120/0 (socket says 172.21.15.45:49120) 2024-09-18T14:03:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67eaffd640 1 -- 172.21.15.45:0/100179901 learned_addr learned my addr 172.21.15.45:0/100179901 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246938800 0 0) 0x7f67ec09dfc0 con 0x7f67ec09ff70 2024-09-18T14:03:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient(hunting): my global_id is 144191 2024-09-18T14:03:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d40034e0 con 0x7f67ec09ff70 2024-09-18T14:03:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928533407 0 0) 0x7f67ec05ed70 con 0x7f67ec09fba0 2024-09-18T14:03:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient(hunting): my global_id is 4977 2024-09-18T14:03:53.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ec09dfc0 con 0x7f67ec09fba0 2024-09-18T14:03:53.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561308363 0 0) 0x7f67ec09dce0 con 0x7f67ec09f7d0 2024-09-18T14:03:53.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient(hunting): my global_id is 4811 2024-09-18T14:03:53.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ec05ed70 con 0x7f67ec09f7d0 2024-09-18T14:03:53.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 459082496 0 0) 0x7f67d40034e0 con 0x7f67ec09ff70 2024-09-18T14:03:53.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 >> v1:172.21.15.84:6791/0 conn(0x7f67ec09f7d0 legacy=0x7f67ec132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 >> v1:172.21.15.45:6789/0 conn(0x7f67ec09fba0 legacy=0x7f67ec137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:03:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 monclient: found mon.noname-b 2024-09-18T14:03:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:03:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec0a1c00 con 0x7f67ec09ff70 2024-09-18T14:03:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient: _finish_auth 0 2024-09-18T14:03:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67f1cf8640 5 monclient: authenticate success, global_id 144191 2024-09-18T14:03:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f67e4001bb0 con 0x7f67ec09ff70 2024-09-18T14:03:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67f1cf8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:03:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient: dump: 2024-09-18T14:03:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.778 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67f1cf8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f67e4001d90 con 0x7f67ec09ff70 2024-09-18T14:03:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67f1cf8640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 1 -- 172.21.15.45:0/100179901 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f67e4004e60 con 0x7f67ec09ff70 2024-09-18T14:03:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67f1cf8640 4 set_mon_vals no callback set 2024-09-18T14:03:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.762+0000 7f67ea7fc640 10 monclient: dump: 2024-09-18T14:03:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.780 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.763+0000 7f67f1cf8640 10 monclient: shutdown 2024-09-18T14:03:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.763+0000 7f67f1cf8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.763+0000 7f67f1cf8640 1 -- 172.21.15.45:0/100179901 >> v1:172.21.15.45:6790/0 conn(0x7f67ec09ff70 legacy=0x7f67ec13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.763+0000 7f67f1cf8640 1 -- 172.21.15.45:0/100179901 shutdown_connections 2024-09-18T14:03:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.763+0000 7f67f1cf8640 1 -- 172.21.15.45:0/100179901 >> 172.21.15.45:0/100179901 conn(0x7f67ec0630b0 msgr2=0x7f67ec0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.763+0000 7f67f1cf8640 1 -- 172.21.15.45:0/100179901 shutdown_connections 2024-09-18T14:03:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.763+0000 7f67f1cf8640 1 -- 172.21.15.45:0/100179901 wait complete. 2024-09-18T14:03:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.763+0000 7f67f1cf8640 10 monclient: build_initial_monmap 2024-09-18T14:03:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.763+0000 7f67f1cf8640 10 monclient: monmap: 2024-09-18T14:03:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:53.783 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.764+0000 7f67f1cf8640 1 Processor -- start 2024-09-18T14:03:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.764+0000 7f67f1cf8640 1 -- start start 2024-09-18T14:03:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.764+0000 7f67f1cf8640 10 monclient: init 2024-09-18T14:03:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.764+0000 7f67f1cf8640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.764+0000 7f67f1cf8640 10 monclient: _add_conns ranks=[7,6,3,8,1,2,4,5,0] 2024-09-18T14:03:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.764+0000 7f67f1cf8640 10 monclient(hunting): picked mon.noname-i con 0x7f67ec09f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.764+0000 7f67f1cf8640 10 monclient(hunting): picked mon.noname-d con 0x7f67ec09fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.764+0000 7f67f1cf8640 10 monclient(hunting): picked mon.noname-g con 0x7f67ec09ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:03:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.764+0000 7f67f1cf8640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec09d310 con 0x7f67ec09ff70 2024-09-18T14:03:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67f1cf8640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec0a2560 con 0x7f67ec09fba0 2024-09-18T14:03:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67f1cf8640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec11d620 con 0x7f67ec09f7d0 2024-09-18T14:03:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67f1cf8640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67f1cf8640 10 monclient(hunting): authenticate will time out at 1049.103034s 2024-09-18T14:03:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67eaffd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f67ec09fba0 0x7f67ec09ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:58976/0 (socket says 172.21.15.45:58976) 2024-09-18T14:03:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67eaffd640 1 -- 172.21.15.45:0/1324924850 learned_addr learned my addr 172.21.15.45:0/1324924850 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600621171 0 0) 0x7f67ec0a2560 con 0x7f67ec09fba0 2024-09-18T14:03:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): my global_id is 4794 2024-09-18T14:03:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d80034e0 con 0x7f67ec09fba0 2024-09-18T14:03:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325892264 0 0) 0x7f67ec11d620 con 0x7f67ec09f7d0 2024-09-18T14:03:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): my global_id is 4885 2024-09-18T14:03:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ec0a2560 con 0x7f67ec09f7d0 2024-09-18T14:03:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579228539 0 0) 0x7f67ec09d310 con 0x7f67ec09ff70 2024-09-18T14:03:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): my global_id is 144165 2024-09-18T14:03:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ec11d620 con 0x7f67ec09ff70 2024-09-18T14:03:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3469300309 0 0) 0x7f67d80034e0 con 0x7f67ec09fba0 2024-09-18T14:03:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f67ec09d310 con 0x7f67ec09fba0 2024-09-18T14:03:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f67e0001480 con 0x7f67ec09fba0 2024-09-18T14:03:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:03:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): dump: 2024-09-18T14:03:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.787 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2644922957 0 0) 0x7f67ec09d310 con 0x7f67ec09fba0 2024-09-18T14:03:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 >> v1:172.21.15.84:6792/0 conn(0x7f67ec09f7d0 legacy=0x7f67ec09a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 >> v1:172.21.15.84:6790/0 conn(0x7f67ec09ff70 legacy=0x7f67ec149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient: _finish_hunting 0 2024-09-18T14:03:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 monclient: found mon.d 2024-09-18T14:03:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec09be30 con 0x7f67ec09fba0 2024-09-18T14:03:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient: _finish_auth 0 2024-09-18T14:03:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67f1cf8640 5 monclient: authenticate success, global_id 4794 2024-09-18T14:03:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67f1cf8640 10 monclient: _renew_subs 2024-09-18T14:03:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67f1cf8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67f1cf8640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ec11d3a0 con 0x7f67ec09fba0 2024-09-18T14:03:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67f1cf8640 10 monclient: _renew_subs 2024-09-18T14:03:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67f1cf8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67f1cf8640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ec11c5d0 con 0x7f67ec09fba0 2024-09-18T14:03:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f67e0001700 con 0x7f67ec09fba0 2024-09-18T14:03:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.765+0000 7f67d37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.766+0000 7f67e9ffb640 4 set_mon_vals no callback set 2024-09-18T14:03:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.766+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f67e0004d90 con 0x7f67ec09fba0 2024-09-18T14:03:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.766+0000 7f67d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.766+0000 7f67d37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.766+0000 7f67d37fe640 10 monclient: dump: 2024-09-18T14:03:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.792 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.766+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f67e000fe20 con 0x7f67ec09fba0 2024-09-18T14:03:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.766+0000 7f67f1cf8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.766+0000 7f67f1cf8640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f67ec04c4a0 con 0x7f67ec09b320 2024-09-18T14:03:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.766+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(230..230 src has 1..230) v4 ==== 3911+0+0 (unknown 4181126139 0 0) 0x7f67e0046ab0 con 0x7f67ec09fba0 2024-09-18T14:03:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67eb7fe640 10 monclient: get_auth_request con 0x7f67d8039f80 auth_method 0 2024-09-18T14:03:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631392605 0 0) 0x7f67ec04c4a0 con 0x7f67ec09b320 2024-09-18T14:03:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 10 monclient(hunting): my global_id is 144229 2024-09-18T14:03:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ec09d310 con 0x7f67ec09b320 2024-09-18T14:03:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2371822031 0 0) 0x7f67ec09d310 con 0x7f67ec09b320 2024-09-18T14:03:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f67ec04c4a0 con 0x7f67ec09b320 2024-09-18T14:03:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f67e4005050 con 0x7f67ec09b320 2024-09-18T14:03:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 10 monclient: dump: 2024-09-18T14:03:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.795 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2194963751 0 0) 0x7f67ec04c4a0 con 0x7f67ec09b320 2024-09-18T14:03:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.767+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67ec11d7b0 con 0x7f67ec09b320 2024-09-18T14:03:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.768+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f67e4005360 con 0x7f67ec09b320 2024-09-18T14:03:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.768+0000 7f67d37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.768+0000 7f67d37fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.768+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 >> v1:172.21.15.45:6791/0 conn(0x7f67ec09b320 legacy=0x7f67ec0a44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.768+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 >> v1:172.21.15.45:6791/0 conn(0x7f67ec09b320 legacy=0x7f67ec0a44a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.774+0000 7f67f1cf8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.774+0000 7f67f1cf8640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f67ec04b720 con 0x7f67ec02a8d0 2024-09-18T14:03:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290892076 0 0) 0x7f67ec04b720 con 0x7f67ec02a8d0 2024-09-18T14:03:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 10 monclient(hunting): my global_id is 144238 2024-09-18T14:03:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ec04c4a0 con 0x7f67ec02a8d0 2024-09-18T14:03:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 452185357 0 0) 0x7f67ec04c4a0 con 0x7f67ec02a8d0 2024-09-18T14:03:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f67ec04b720 con 0x7f67ec02a8d0 2024-09-18T14:03:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f67e4005c60 con 0x7f67ec02a8d0 2024-09-18T14:03:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 10 monclient: dump: 2024-09-18T14:03:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.799 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3169749557 0 0) 0x7f67ec04b720 con 0x7f67ec02a8d0 2024-09-18T14:03:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f67ec071550 con 0x7f67ec02a8d0 2024-09-18T14:03:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 1289108425) 0x7f67e4008350 con 0x7f67ec02a8d0 2024-09-18T14:03:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:53.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:53.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 >> v1:172.21.15.45:6791/0 conn(0x7f67ec02a8d0 legacy=0x7f67ec0a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.775+0000 7f67d37fe640 1 -- 172.21.15.45:0/1324924850 >> v1:172.21.15.45:6791/0 conn(0x7f67ec02a8d0 legacy=0x7f67ec0a4bb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:53.800 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:53.800 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:53.800 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:53.800 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:53.800 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:53.802 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.802 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:03:53.802 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:53.802 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:53.802 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:53.802 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:53.802 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:53.802 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:53.802 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:53.802 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:53.802 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:53.803 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:53.803 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:53.803 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:53.803 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.803 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:53.803 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:53.803 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:53.803 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:53.803 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:53.803 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:53.803 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:53.803 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:53.804 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:53.804 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:53.804 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:53.804 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.804 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:53.804 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:53.804 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:53.804 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:53.804 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:53.805 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:53.805 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:53.805 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:53.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:53.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:53.805 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:53.805 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:53.805 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:53.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:53.805 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:53.805 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:53.806 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:53.806 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:53.806 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:53.806 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:53.806 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:53.806 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:53.806 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:53.806 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:53.806 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:53.806 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:53.806 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.807 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:53.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.807 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:53.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.807 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:53.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:53.807 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.807 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.807 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:53.807 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.808 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.808 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.808 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:53.808 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:53.808 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.808 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.808 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.808 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:53.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:53.809 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.809 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.809 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:53.809 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.810 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.810 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.810 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:53.810 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:53.810 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.810 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.810 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.810 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.824 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.824 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:53.824 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:53.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.776+0000 7f67f1cf8640 1 -- 172.21.15.45:0/1324924850 >> v1:172.21.15.84:6812/31478 conn(0x7f67d8039f80 legacy=0x7f67d803c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.776+0000 7f67f1cf8640 10 monclient: shutdown 2024-09-18T14:03:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.776+0000 7f67f1cf8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.776+0000 7f67f1cf8640 1 -- 172.21.15.45:0/1324924850 >> v1:172.21.15.45:6792/0 conn(0x7f67ec09fba0 legacy=0x7f67ec09ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.776+0000 7f67ebfff640 1 -- 172.21.15.45:0/1324924850 reap_dead start 2024-09-18T14:03:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.776+0000 7f67ebfff640 1 -- 172.21.15.45:0/1324924850 reap_dead start 2024-09-18T14:03:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.776+0000 7f67f1cf8640 1 -- 172.21.15.45:0/1324924850 shutdown_connections 2024-09-18T14:03:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.776+0000 7f67f1cf8640 1 -- 172.21.15.45:0/1324924850 >> 172.21.15.45:0/1324924850 conn(0x7f67ec0630b0 msgr2=0x7f67ec130b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.776+0000 7f67f1cf8640 1 -- 172.21.15.45:0/1324924850 shutdown_connections 2024-09-18T14:03:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.776+0000 7f67f1cf8640 1 -- 172.21.15.45:0/1324924850 wait complete. 2024-09-18T14:03:53.827 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:53.827 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:03:53.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:03:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.924+0000 7f574d114640 10 monclient: get_monmap_and_config 2024-09-18T14:03:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.924+0000 7f574d114640 10 monclient: build_initial_monmap 2024-09-18T14:03:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.924+0000 7f574d114640 10 monclient: monmap: 2024-09-18T14:03:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:53.925600+0000 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:53.925600+0000 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:53.927 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 1 Processor -- start 2024-09-18T14:03:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 1 -- start start 2024-09-18T14:03:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 10 monclient: init 2024-09-18T14:03:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 10 monclient: _add_conns ranks=[4,7,1,8,0,5,6,2,3] 2024-09-18T14:03:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 10 monclient(hunting): picked mon.noname-c con 0x7f574809f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 10 monclient(hunting): picked mon.noname-i con 0x7f574809fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 10 monclient(hunting): picked mon.noname-f con 0x7f574809ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574805ed70 con 0x7f574809ff70 2024-09-18T14:03:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574809dfc0 con 0x7f574809f7d0 2024-09-18T14:03:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574809dce0 con 0x7f574809fba0 2024-09-18T14:03:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f574d114640 10 monclient(hunting): authenticate will time out at 1049.263843s 2024-09-18T14:03:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f5746d9d640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f574809f7d0 0x7f5748132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38496/0 (socket says 172.21.15.45:38496) 2024-09-18T14:03:53.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.925+0000 7f5746d9d640 1 -- 172.21.15.45:0/3256189116 learned_addr learned my addr 172.21.15.45:0/3256189116 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 1 -- 172.21.15.45:0/3256189116 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279871885 0 0) 0x7f574809dfc0 con 0x7f574809f7d0 2024-09-18T14:03:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient(hunting): my global_id is 144247 2024-09-18T14:03:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 1 -- 172.21.15.45:0/3256189116 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57340034e0 con 0x7f574809f7d0 2024-09-18T14:03:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 1 -- 172.21.15.45:0/3256189116 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 559372274 0 0) 0x7f57340034e0 con 0x7f574809f7d0 2024-09-18T14:03:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 1 -- 172.21.15.45:0/3256189116 >> v1:172.21.15.84:6792/0 conn(0x7f574809fba0 legacy=0x7f5748137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 1 -- 172.21.15.45:0/3256189116 >> v1:172.21.15.84:6789/0 conn(0x7f574809ff70 legacy=0x7f574813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:03:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 1 monclient: found mon.noname-c 2024-09-18T14:03:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:03:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 1 -- 172.21.15.45:0/3256189116 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57480a1c00 con 0x7f574809f7d0 2024-09-18T14:03:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient: _finish_auth 0 2024-09-18T14:03:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 1 -- 172.21.15.45:0/3256189116 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5738001830 con 0x7f574809f7d0 2024-09-18T14:03:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:03:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient: dump: 2024-09-18T14:03:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f574d114640 5 monclient: authenticate success, global_id 144247 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f574d114640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 1 -- 172.21.15.45:0/3256189116 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5738001a10 con 0x7f574809f7d0 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f574d114640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 1 -- 172.21.15.45:0/3256189116 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5738004ba0 con 0x7f574809f7d0 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f574d114640 4 set_mon_vals no callback set 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f5745d9b640 10 monclient: dump: 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.941 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f574d114640 10 monclient: shutdown 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f574d114640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.926+0000 7f574d114640 1 -- 172.21.15.45:0/3256189116 >> v1:172.21.15.45:6791/0 conn(0x7f574809f7d0 legacy=0x7f5748132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.927+0000 7f574d114640 1 -- 172.21.15.45:0/3256189116 shutdown_connections 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.927+0000 7f574d114640 1 -- 172.21.15.45:0/3256189116 >> 172.21.15.45:0/3256189116 conn(0x7f57480630b0 msgr2=0x7f57480634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.927+0000 7f574d114640 1 -- 172.21.15.45:0/3256189116 shutdown_connections 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.927+0000 7f574d114640 1 -- 172.21.15.45:0/3256189116 wait complete. 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.927+0000 7f574d114640 10 monclient: build_initial_monmap 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.927+0000 7f574d114640 10 monclient: monmap: 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 1 Processor -- start 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 1 -- start start 2024-09-18T14:03:53.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 10 monclient: init 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 10 monclient: _add_conns ranks=[7,6,4,5,3,1,8,0,2] 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 10 monclient(hunting): picked mon.noname-i con 0x7f574809f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 10 monclient(hunting): picked mon.noname-d con 0x7f574809fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 10 monclient(hunting): picked mon.noname-c con 0x7f574809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57480a2560 con 0x7f574809ff70 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574811eba0 con 0x7f574809fba0 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574809bb90 con 0x7f574809f7d0 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574d114640 10 monclient(hunting): authenticate will time out at 1049.266618s 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574759e640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f574809ff70 0x7f574814a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38506/0 (socket says 172.21.15.45:38506) 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f574759e640 1 -- 172.21.15.45:0/1790793419 learned_addr learned my addr 172.21.15.45:0/1790793419 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:53.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192215968 0 0) 0x7f57480a2560 con 0x7f574809ff70 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f573effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f573effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f573effd640 10 monclient(hunting): my global_id is 144256 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57280034e0 con 0x7f574809ff70 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618811970 0 0) 0x7f574811eba0 con 0x7f574809fba0 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f573effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f573effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f573effd640 10 monclient(hunting): my global_id is 4803 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57480a2560 con 0x7f574809fba0 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.928+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174835946 0 0) 0x7f574809bb90 con 0x7f574809f7d0 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): my global_id is 4903 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f574811eba0 con 0x7f574809f7d0 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 925013040 0 0) 0x7f57280034e0 con 0x7f574809ff70 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f574809bb90 con 0x7f574809ff70 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5740001e40 con 0x7f574809ff70 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): dump: 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1203163016 0 0) 0x7f574811eba0 con 0x7f574809f7d0 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f57280034e0 con 0x7f574809f7d0 2024-09-18T14:03:53.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2455879902 0 0) 0x7f57480a2560 con 0x7f574809fba0 2024-09-18T14:03:53.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f574811eba0 con 0x7f574809fba0 2024-09-18T14:03:53.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5738004ae0 con 0x7f574809f7d0 2024-09-18T14:03:53.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:03:53.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): dump: 2024-09-18T14:03:53.948 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.948 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.948 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.948 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f57300019e0 con 0x7f574809fba0 2024-09-18T14:03:53.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient(hunting): dump: 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.950 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.951 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 619702608 0 0) 0x7f574809bb90 con 0x7f574809ff70 2024-09-18T14:03:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 >> v1:172.21.15.84:6792/0 conn(0x7f574809f7d0 legacy=0x7f5748131780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 >> v1:172.21.15.45:6792/0 conn(0x7f574809fba0 legacy=0x7f5748146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient: _finish_hunting 0 2024-09-18T14:03:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 monclient: found mon.c 2024-09-18T14:03:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57481428c0 con 0x7f574809ff70 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient: _finish_auth 0 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f574d114640 5 monclient: authenticate success, global_id 144256 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f574d114640 10 monclient: _renew_subs 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f574d114640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f574d114640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574811e8c0 con 0x7f574809ff70 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f574d114640 10 monclient: _renew_subs 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f574d114640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f574d114640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5748131e90 con 0x7f574809ff70 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5740002100 con 0x7f574809ff70 2024-09-18T14:03:53.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5740002870 con 0x7f574809ff70 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f574559a640 4 set_mon_vals no callback set 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.929+0000 7f573effd640 10 monclient: dump: 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f574d114640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f57400029d0 con 0x7f574809ff70 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f574d114640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f572c006580 con 0x7f572c002830 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(230..230 src has 1..230) v4 ==== 3911+0+0 (unknown 4181126139 0 0) 0x7f5740046e20 con 0x7f574809ff70 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475946274 0 0) 0x7f572c006580 con 0x7f572c002830 2024-09-18T14:03:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 10 monclient(hunting): my global_id is 4812 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f574809bb90 con 0x7f572c002830 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f574659c640 10 monclient: get_auth_request con 0x7f572803ab60 auth_method 0 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 872105270 0 0) 0x7f574809bb90 con 0x7f572c002830 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f572c006580 con 0x7f572c002830 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5738001770 con 0x7f572c002830 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.930+0000 7f573effd640 10 monclient: dump: 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.931+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1577291172 0 0) 0x7f572c006580 con 0x7f572c002830 2024-09-18T14:03:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.931+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f572c00a0a0 con 0x7f572c002830 2024-09-18T14:03:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.931+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5738006fc0 con 0x7f572c002830 2024-09-18T14:03:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.931+0000 7f573effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.931+0000 7f573effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.931+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 >> v1:172.21.15.45:6792/0 conn(0x7f572c002830 legacy=0x7f572c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.931+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 >> v1:172.21.15.45:6792/0 conn(0x7f572c002830 legacy=0x7f572c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f574d114640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f574d114640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f572c009660 con 0x7f572c009040 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256267401 0 0) 0x7f572c009660 con 0x7f572c009040 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 10 monclient(hunting): my global_id is 4821 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f572c006580 con 0x7f572c009040 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1393224015 0 0) 0x7f572c006580 con 0x7f572c009040 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f572c009660 con 0x7f572c009040 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f57380055b0 con 0x7f572c009040 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 10 monclient: dump: 2024-09-18T14:03:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:53.961 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3150843059 0 0) 0x7f572c009660 con 0x7f572c009040 2024-09-18T14:03:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f572c009920 con 0x7f572c009040 2024-09-18T14:03:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.937+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 1752385309) 0x7f5738006a60 con 0x7f572c009040 2024-09-18T14:03:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f573effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f573effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 >> v1:172.21.15.45:6792/0 conn(0x7f572c009040 legacy=0x7f572c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f573effd640 1 -- 172.21.15.45:0/1790793419 >> v1:172.21.15.45:6792/0 conn(0x7f572c009040 legacy=0x7f572c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:53.961 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:03:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:53.981 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:53.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f574d114640 1 -- 172.21.15.45:0/1790793419 >> v1:172.21.15.84:6812/31478 conn(0x7f572803ab60 legacy=0x7f572803d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f574d114640 10 monclient: shutdown 2024-09-18T14:03:53.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f574d114640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:53.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f574d114640 1 -- 172.21.15.45:0/1790793419 >> v1:172.21.15.45:6791/0 conn(0x7f574809ff70 legacy=0x7f574814a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:53.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f574759e640 1 -- 172.21.15.45:0/1790793419 reap_dead start 2024-09-18T14:03:53.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f574759e640 1 -- 172.21.15.45:0/1790793419 reap_dead start 2024-09-18T14:03:53.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f574d114640 1 -- 172.21.15.45:0/1790793419 shutdown_connections 2024-09-18T14:03:53.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f574d114640 1 -- 172.21.15.45:0/1790793419 >> 172.21.15.45:0/1790793419 conn(0x7f57480630b0 msgr2=0x7f57480a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:53.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f574d114640 1 -- 172.21.15.45:0/1790793419 shutdown_connections 2024-09-18T14:03:53.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:53.938+0000 7f574d114640 1 -- 172.21.15.45:0/1790793419 wait complete. 2024-09-18T14:03:53.983 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:53.984 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:03:53.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:03:54.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.080+0000 7f110499d640 10 monclient: get_monmap_and_config 2024-09-18T14:03:54.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.080+0000 7f110499d640 10 monclient: build_initial_monmap 2024-09-18T14:03:54.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.080+0000 7f110499d640 10 monclient: monmap: 2024-09-18T14:03:54.081 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:54.081 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.081 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:54.081591+0000 2024-09-18T14:03:54.081 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:54.081591+0000 2024-09-18T14:03:54.081 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:54.082 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 1 Processor -- start 2024-09-18T14:03:54.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 1 -- start start 2024-09-18T14:03:54.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 10 monclient: init 2024-09-18T14:03:54.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:54.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 10 monclient: _add_conns ranks=[4,3,6,1,8,0,5,2,7] 2024-09-18T14:03:54.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 10 monclient(hunting): picked mon.noname-c con 0x7f110009f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:54.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 10 monclient(hunting): picked mon.noname-g con 0x7f110009fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:03:54.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 10 monclient(hunting): picked mon.noname-d con 0x7f110009ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:54.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110005ed70 con 0x7f110009fba0 2024-09-18T14:03:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110009dfc0 con 0x7f110009f7d0 2024-09-18T14:03:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110009dce0 con 0x7f110009ff70 2024-09-18T14:03:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f110499d640 10 monclient(hunting): authenticate will time out at 1049.419915s 2024-09-18T14:03:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.081+0000 7f10fe59c640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f110009f7d0 0x7f1100132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38518/0 (socket says 172.21.15.45:38518) 2024-09-18T14:03:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fe59c640 1 -- 172.21.15.45:0/589128927 learned_addr learned my addr 172.21.15.45:0/589128927 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 1 -- 172.21.15.45:0/589128927 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530401298 0 0) 0x7f110009dfc0 con 0x7f110009f7d0 2024-09-18T14:03:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient(hunting): my global_id is 144265 2024-09-18T14:03:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 1 -- 172.21.15.45:0/589128927 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e80034e0 con 0x7f110009f7d0 2024-09-18T14:03:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 1 -- 172.21.15.45:0/589128927 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398715047 0 0) 0x7f110009dce0 con 0x7f110009ff70 2024-09-18T14:03:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient(hunting): my global_id is 4830 2024-09-18T14:03:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 1 -- 172.21.15.45:0/589128927 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f110009dfc0 con 0x7f110009ff70 2024-09-18T14:03:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 1 -- 172.21.15.45:0/589128927 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2587465750 0 0) 0x7f10e80034e0 con 0x7f110009f7d0 2024-09-18T14:03:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 1 -- 172.21.15.45:0/589128927 >> v1:172.21.15.45:6792/0 conn(0x7f110009ff70 legacy=0x7f110013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 1 -- 172.21.15.45:0/589128927 >> v1:172.21.15.84:6790/0 conn(0x7f110009fba0 legacy=0x7f1100137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient: _finish_hunting 0 2024-09-18T14:03:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 1 monclient: found mon.noname-c 2024-09-18T14:03:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 1 -- 172.21.15.45:0/589128927 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11000a1c00 con 0x7f110009f7d0 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient: _finish_auth 0 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 1 -- 172.21.15.45:0/589128927 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10f0001830 con 0x7f110009f7d0 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient: dump: 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f110499d640 5 monclient: authenticate success, global_id 144265 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f110499d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 1 -- 172.21.15.45:0/589128927 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f10f0001a10 con 0x7f110009f7d0 2024-09-18T14:03:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 1 -- 172.21.15.45:0/589128927 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10f0004ba0 con 0x7f110009f7d0 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f110499d640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f110499d640 4 set_mon_vals no callback set 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.082+0000 7f10fd59a640 10 monclient: dump: 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.097 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.083+0000 7f110499d640 10 monclient: shutdown 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.083+0000 7f110499d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.083+0000 7f110499d640 1 -- 172.21.15.45:0/589128927 >> v1:172.21.15.45:6791/0 conn(0x7f110009f7d0 legacy=0x7f1100132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.083+0000 7f110499d640 1 -- 172.21.15.45:0/589128927 shutdown_connections 2024-09-18T14:03:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.083+0000 7f110499d640 1 -- 172.21.15.45:0/589128927 >> 172.21.15.45:0/589128927 conn(0x7f11000630b0 msgr2=0x7f11000634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.083+0000 7f110499d640 1 -- 172.21.15.45:0/589128927 shutdown_connections 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.083+0000 7f110499d640 1 -- 172.21.15.45:0/589128927 wait complete. 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.083+0000 7f110499d640 10 monclient: build_initial_monmap 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.083+0000 7f110499d640 10 monclient: monmap: 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:54.099 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:54.100 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 1 Processor -- start 2024-09-18T14:03:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 1 -- start start 2024-09-18T14:03:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 10 monclient: init 2024-09-18T14:03:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 10 monclient: _add_conns ranks=[4,2,3,7,1,0,8,5,6] 2024-09-18T14:03:54.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 10 monclient(hunting): picked mon.noname-c con 0x7f110009f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:54.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 10 monclient(hunting): picked mon.noname-b con 0x7f110009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:03:54.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 10 monclient(hunting): picked mon.noname-g con 0x7f110009ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:03:54.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11000a2560 con 0x7f110009fba0 2024-09-18T14:03:54.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1100142050 con 0x7f110009ff70 2024-09-18T14:03:54.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1100132270 con 0x7f110009f7d0 2024-09-18T14:03:54.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:54.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f110499d640 10 monclient(hunting): authenticate will time out at 1049.422570s 2024-09-18T14:03:54.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10fe59c640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f110009f7d0 0x7f1100141940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38526/0 (socket says 172.21.15.45:38526) 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10fe59c640 1 -- 172.21.15.45:0/261509189 learned_addr learned my addr 172.21.15.45:0/261509189 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265289549 0 0) 0x7f1100132270 con 0x7f110009f7d0 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 10 monclient(hunting): my global_id is 144274 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e00034e0 con 0x7f110009f7d0 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091235302 0 0) 0x7f11000a2560 con 0x7f110009fba0 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 10 monclient(hunting): my global_id is 144200 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1100132270 con 0x7f110009fba0 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004416046 0 0) 0x7f1100142050 con 0x7f110009ff70 2024-09-18T14:03:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 10 monclient(hunting): my global_id is 144183 2024-09-18T14:03:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.084+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11000a2560 con 0x7f110009ff70 2024-09-18T14:03:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3557920826 0 0) 0x7f10e00034e0 con 0x7f110009f7d0 2024-09-18T14:03:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1100142050 con 0x7f110009f7d0 2024-09-18T14:03:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10f0004c20 con 0x7f110009f7d0 2024-09-18T14:03:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:03:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient(hunting): dump: 2024-09-18T14:03:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 773980871 0 0) 0x7f1100132270 con 0x7f110009fba0 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10e00034e0 con 0x7f110009fba0 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10f4001a60 con 0x7f110009fba0 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient(hunting): dump: 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 434299913 0 0) 0x7f11000a2560 con 0x7f110009ff70 2024-09-18T14:03:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1100132270 con 0x7f110009ff70 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10f8001d40 con 0x7f110009ff70 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient(hunting): dump: 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1582561756 0 0) 0x7f1100142050 con 0x7f110009f7d0 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 >> v1:172.21.15.84:6790/0 conn(0x7f110009ff70 legacy=0x7f1100149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 >> v1:172.21.15.45:6790/0 conn(0x7f110009fba0 legacy=0x7f1100131b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 monclient: found mon.c 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:54.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1100121f90 con 0x7f110009f7d0 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient: _finish_auth 0 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f110499d640 5 monclient: authenticate success, global_id 144274 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f110499d640 10 monclient: _renew_subs 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f110499d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f110499d640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1100120d50 con 0x7f110009f7d0 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f110499d640 10 monclient: _renew_subs 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f110499d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f110499d640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1100120180 con 0x7f110009f7d0 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f10f0001210 con 0x7f110009f7d0 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10f0001860 con 0x7f110009f7d0 2024-09-18T14:03:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10fcd99640 4 set_mon_vals no callback set 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.085+0000 7f10ee7fc640 10 monclient: dump: 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.111 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f10f0010050 con 0x7f110009f7d0 2024-09-18T14:03:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f110499d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f110499d640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f10e4006580 con 0x7f10e4002830 2024-09-18T14:03:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(230..230 src has 1..230) v4 ==== 3911+0+0 (unknown 4181126139 0 0) 0x7f10f0046c60 con 0x7f110009f7d0 2024-09-18T14:03:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070990129 0 0) 0x7f10e4006580 con 0x7f10e4002830 2024-09-18T14:03:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 10 monclient(hunting): my global_id is 4940 2024-09-18T14:03:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1100142050 con 0x7f10e4002830 2024-09-18T14:03:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10fed9d640 10 monclient: get_auth_request con 0x7f10e003a980 auth_method 0 2024-09-18T14:03:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 336958427 0 0) 0x7f1100142050 con 0x7f10e4002830 2024-09-18T14:03:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10e4006580 con 0x7f10e4002830 2024-09-18T14:03:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10f40022b0 con 0x7f10e4002830 2024-09-18T14:03:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:03:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.086+0000 7f10ee7fc640 10 monclient: dump: 2024-09-18T14:03:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.114 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.087+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3884711299 0 0) 0x7f10e4006580 con 0x7f10e4002830 2024-09-18T14:03:54.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.087+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f10e400a0a0 con 0x7f10e4002830 2024-09-18T14:03:54.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.087+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f10f40026f0 con 0x7f10e4002830 2024-09-18T14:03:54.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.087+0000 7f10ee7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:54.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.087+0000 7f10ee7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:54.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.087+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 >> v1:172.21.15.45:6793/0 conn(0x7f10e4002830 legacy=0x7f10e4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.087+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 >> v1:172.21.15.45:6793/0 conn(0x7f10e4002830 legacy=0x7f10e4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f110499d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f110499d640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f10e4009660 con 0x7f10e4009040 2024-09-18T14:03:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821214773 0 0) 0x7f10e4009660 con 0x7f10e4009040 2024-09-18T14:03:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 10 monclient(hunting): my global_id is 4949 2024-09-18T14:03:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e4006580 con 0x7f10e4009040 2024-09-18T14:03:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 745809870 0 0) 0x7f10e4006580 con 0x7f10e4009040 2024-09-18T14:03:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10e4009660 con 0x7f10e4009040 2024-09-18T14:03:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10f4003340 con 0x7f10e4009040 2024-09-18T14:03:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 10 monclient: dump: 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.118 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3560597472 0 0) 0x7f10e4009660 con 0x7f10e4009040 2024-09-18T14:03:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.093+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f10e4009920 con 0x7f10e4009040 2024-09-18T14:03:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7558 (unknown 0 0 3725136180) 0x7f10f40035e0 con 0x7f10e4009040 2024-09-18T14:03:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f10ee7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f10ee7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 >> v1:172.21.15.45:6793/0 conn(0x7f10e4009040 legacy=0x7f10e4008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f10ee7fc640 1 -- 172.21.15.45:0/261509189 >> v1:172.21.15.45:6793/0 conn(0x7f10e4009040 legacy=0x7f10e4008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:54.119 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:54.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:54.119 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:54.119 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:03:54.119 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:54.119 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:54.119 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:54.119 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:54.119 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:54.119 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:54.120 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:54.120 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:54.120 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:54.120 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:54.120 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:54.120 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.120 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:03:54.120 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:54.120 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:54.120 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:54.120 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.120 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.121 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.121 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.121 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.121 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.121 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.121 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.121 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.121 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.121 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:54.121 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:54.121 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.121 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:54.122 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:54.123 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.124 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.125 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:54.126 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:54.126 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.126 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.126 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.126 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:54.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:54.126 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.126 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.126 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:54.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.128 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.129 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:54.130 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:54.131 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.132 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.132 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.132 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:54.132 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.132 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.132 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:54.132 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:54.132 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.132 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.133 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.137 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:54.138 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.138 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.138 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:03:54.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.138 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.138 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.138 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:54.138 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.138 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.138 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:54.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.139 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.139 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.139 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:03:54.139 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.139 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.139 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:54.139 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:54.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f110499d640 1 -- 172.21.15.45:0/261509189 >> v1:172.21.15.84:6812/31478 conn(0x7f10e003a980 legacy=0x7f10e003ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f110499d640 10 monclient: shutdown 2024-09-18T14:03:54.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f110499d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:54.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f110499d640 1 -- 172.21.15.45:0/261509189 >> v1:172.21.15.45:6791/0 conn(0x7f110009f7d0 legacy=0x7f1100141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f10fed9d640 1 -- 172.21.15.45:0/261509189 reap_dead start 2024-09-18T14:03:54.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f10fed9d640 1 -- 172.21.15.45:0/261509189 reap_dead start 2024-09-18T14:03:54.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f110499d640 1 -- 172.21.15.45:0/261509189 shutdown_connections 2024-09-18T14:03:54.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f110499d640 1 -- 172.21.15.45:0/261509189 >> 172.21.15.45:0/261509189 conn(0x7f11000630b0 msgr2=0x7f11000a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:54.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f110499d640 1 -- 172.21.15.45:0/261509189 shutdown_connections 2024-09-18T14:03:54.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.094+0000 7f110499d640 1 -- 172.21.15.45:0/261509189 wait complete. 2024-09-18T14:03:54.142 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:54.142 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:03:54.142 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:03:54.191 INFO:tasks.rados.rados.0.smithi084.stdout:999: done (1 left) 2024-09-18T14:03:54.192 INFO:tasks.rados.rados.0.smithi084.stdout:1000: write (excl) oid 29 current snap is 112 2024-09-18T14:03:54.192 INFO:tasks.rados.rados.0.smithi084.stdout:998: finishing write tid 1 to smithi08434663-39 2024-09-18T14:03:54.192 INFO:tasks.rados.rados.0.smithi084.stdout:1000: seq_num 361 ranges {467327=633480,1825176=242151} 2024-09-18T14:03:54.198 INFO:tasks.rados.rados.0.smithi084.stdout:1000: writing smithi08434663-29 from 467327 to 1100807 tid 1 2024-09-18T14:03:54.204 INFO:tasks.rados.rados.0.smithi084.stdout:1000: writing smithi08434663-29 from 1825176 to 2067327 tid 2 2024-09-18T14:03:54.204 INFO:tasks.rados.rados.0.smithi084.stdout:1001: copy_from oid 31 from oid 25 current snap is 112 2024-09-18T14:03:54.205 INFO:tasks.rados.rados.0.smithi084.stdout:998: finishing write tid 2 to smithi08434663-39 2024-09-18T14:03:54.205 INFO:tasks.rados.rados.0.smithi084.stdout:998: finishing write tid 3 to smithi08434663-39 2024-09-18T14:03:54.205 INFO:tasks.rados.rados.0.smithi084.stdout:998: finishing write tid 4 to smithi08434663-39 2024-09-18T14:03:54.205 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 223 (ObjNum 360 snap 112 seq_num 360) dirty exists 2024-09-18T14:03:54.205 INFO:tasks.rados.rados.0.smithi084.stdout:998: left oid 39 (ObjNum 360 snap 112 seq_num 360) 2024-09-18T14:03:54.205 INFO:tasks.rados.rados.0.smithi084.stdout:1000: finishing write tid 1 to smithi08434663-29 2024-09-18T14:03:54.205 INFO:tasks.rados.rados.0.smithi084.stdout:998: done (2 left) 2024-09-18T14:03:54.205 INFO:tasks.rados.rados.0.smithi084.stdout:1002: read oid 50 snap -1 2024-09-18T14:03:54.205 INFO:tasks.rados.rados.0.smithi084.stdout:1002: expect (ObjNum 350 snap 104 seq_num 350) 2024-09-18T14:03:54.205 INFO:tasks.rados.rados.0.smithi084.stdout:1003: copy_from oid 15 from oid 22 current snap is 112 2024-09-18T14:03:54.205 INFO:tasks.rados.rados.0.smithi084.stdout:1004: read oid 36 snap -1 2024-09-18T14:03:54.206 INFO:tasks.rados.rados.0.smithi084.stdout:1004: expect (ObjNum 343 snap 104 seq_num 343) 2024-09-18T14:03:54.206 INFO:tasks.rados.rados.0.smithi084.stdout:1005: snap_remove snap 107 2024-09-18T14:03:54.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.251+0000 7f233506f640 10 monclient: get_monmap_and_config 2024-09-18T14:03:54.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.251+0000 7f233506f640 10 monclient: build_initial_monmap 2024-09-18T14:03:54.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.251+0000 7f233506f640 10 monclient: monmap: 2024-09-18T14:03:54.252 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:54.252 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.252 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:54.252585+0000 2024-09-18T14:03:54.252 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:54.252585+0000 2024-09-18T14:03:54.252 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:54.253 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.253 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:54.253 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:54.253 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:54.253 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:54.253 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:54.253 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:54.253 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:54.253 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:54.253 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:54.253 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 1 Processor -- start 2024-09-18T14:03:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 1 -- start start 2024-09-18T14:03:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 10 monclient: init 2024-09-18T14:03:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 10 monclient: _add_conns ranks=[8,1,0,7,3,2,4,6,5] 2024-09-18T14:03:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 10 monclient(hunting): picked mon.noname-e con 0x7f233006e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:03:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 10 monclient(hunting): picked mon.noname-f con 0x7f23300a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 10 monclient(hunting): picked mon.noname-a con 0x7f23300a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233005ed70 con 0x7f23300a1080 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2330097620 con 0x7f23300a0cb0 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2330097340 con 0x7f233006e810 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.252+0000 7f233506f640 10 monclient(hunting): authenticate will time out at 1049.590910s 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232f59e640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f23300a1080 0x7f233013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44924/0 (socket says 172.21.15.45:44924) 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232f59e640 1 -- 172.21.15.45:0/1407714109 learned_addr learned my addr 172.21.15.45:0/1407714109 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 1 -- 172.21.15.45:0/1407714109 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068962207 0 0) 0x7f2330097340 con 0x7f233006e810 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 10 monclient(hunting): my global_id is 4958 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 1 -- 172.21.15.45:0/1407714109 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23180034e0 con 0x7f233006e810 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 1 -- 172.21.15.45:0/1407714109 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1758390475 0 0) 0x7f23180034e0 con 0x7f233006e810 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 1 -- 172.21.15.45:0/1407714109 >> v1:172.21.15.84:6789/0 conn(0x7f23300a0cb0 legacy=0x7f2330137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 1 -- 172.21.15.45:0/1407714109 >> v1:172.21.15.45:6789/0 conn(0x7f23300a1080 legacy=0x7f233013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 10 monclient: _finish_hunting 0 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 1 monclient: found mon.noname-e 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 1 -- 172.21.15.45:0/1407714109 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23300a1660 con 0x7f233006e810 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 10 monclient: _finish_auth 0 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 1 -- 172.21.15.45:0/1407714109 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2320001830 con 0x7f233006e810 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f232dd9b640 10 monclient: dump: 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f233506f640 5 monclient: authenticate success, global_id 4958 2024-09-18T14:03:54.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.253+0000 7f233506f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:54.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.256+0000 7f232dd9b640 1 -- 172.21.15.45:0/1407714109 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2320001a10 con 0x7f233006e810 2024-09-18T14:03:54.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.256+0000 7f232dd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:54.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.256+0000 7f232dd9b640 1 -- 172.21.15.45:0/1407714109 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2320002240 con 0x7f233006e810 2024-09-18T14:03:54.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.256+0000 7f232dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.256+0000 7f232dd9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:03:54.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.256+0000 7f232dd9b640 10 monclient: dump: 2024-09-18T14:03:54.267 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.267 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.267 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.267 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.267 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.268 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.256+0000 7f233506f640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.256+0000 7f233506f640 4 set_mon_vals no callback set 2024-09-18T14:03:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.257+0000 7f233506f640 10 monclient: shutdown 2024-09-18T14:03:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.257+0000 7f233506f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.257+0000 7f233506f640 1 -- 172.21.15.45:0/1407714109 >> v1:172.21.15.45:6793/0 conn(0x7f233006e810 legacy=0x7f23300a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.257+0000 7f233506f640 1 -- 172.21.15.45:0/1407714109 shutdown_connections 2024-09-18T14:03:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.257+0000 7f233506f640 1 -- 172.21.15.45:0/1407714109 >> 172.21.15.45:0/1407714109 conn(0x7f23300a5bb0 msgr2=0x7f23300a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.257+0000 7f233506f640 1 -- 172.21.15.45:0/1407714109 shutdown_connections 2024-09-18T14:03:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.257+0000 7f233506f640 1 -- 172.21.15.45:0/1407714109 wait complete. 2024-09-18T14:03:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.258+0000 7f233506f640 10 monclient: build_initial_monmap 2024-09-18T14:03:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.258+0000 7f233506f640 10 monclient: monmap: 2024-09-18T14:03:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:54.271 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:54.271 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:54.271 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:54.271 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:54.271 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:54.271 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:54.271 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:54.271 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:54.271 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:54.271 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.259+0000 7f233506f640 1 Processor -- start 2024-09-18T14:03:54.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.259+0000 7f233506f640 1 -- start start 2024-09-18T14:03:54.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.259+0000 7f233506f640 10 monclient: init 2024-09-18T14:03:54.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f233506f640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:54.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f233506f640 10 monclient: _add_conns ranks=[4,5,6,1,3,7,8,2,0] 2024-09-18T14:03:54.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f233506f640 10 monclient(hunting): picked mon.noname-c con 0x7f233006e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:54.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f233506f640 10 monclient(hunting): picked mon.noname-h con 0x7f23300a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:54.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f233506f640 10 monclient(hunting): picked mon.noname-d con 0x7f23300a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:54.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f233506f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233009fa10 con 0x7f233006e810 2024-09-18T14:03:54.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f233506f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23300a2270 con 0x7f23300a0cb0 2024-09-18T14:03:54.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f233506f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23301227a0 con 0x7f23300a1080 2024-09-18T14:03:54.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f233506f640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:54.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f233506f640 10 monclient(hunting): authenticate will time out at 1049.598502s 2024-09-18T14:03:54.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f232ed9d640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f233006e810 0x7f23301353a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38534/0 (socket says 172.21.15.45:38534) 2024-09-18T14:03:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f232ed9d640 1 -- 172.21.15.45:0/2911573778 learned_addr learned my addr 172.21.15.45:0/2911573778 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28883146 0 0) 0x7f233009fa10 con 0x7f233006e810 2024-09-18T14:03:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f2316ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f2316ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f2316ffd640 10 monclient(hunting): my global_id is 144283 2024-09-18T14:03:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231c0034e0 con 0x7f233006e810 2024-09-18T14:03:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58931168 0 0) 0x7f23301227a0 con 0x7f23300a1080 2024-09-18T14:03:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f2316ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f2316ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f2316ffd640 10 monclient(hunting): my global_id is 4839 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.260+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f233009fa10 con 0x7f23300a1080 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111013979 0 0) 0x7f23300a2270 con 0x7f23300a0cb0 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient(hunting): my global_id is 4820 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23301227a0 con 0x7f23300a0cb0 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2310754679 0 0) 0x7f231c0034e0 con 0x7f233006e810 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f23300a2270 con 0x7f233006e810 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2320001ec0 con 0x7f233006e810 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:03:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient(hunting): dump: 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.276 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 860785551 0 0) 0x7f233009fa10 con 0x7f23300a1080 2024-09-18T14:03:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f231c0034e0 con 0x7f23300a1080 2024-09-18T14:03:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2328001c60 con 0x7f23300a1080 2024-09-18T14:03:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient(hunting): dump: 2024-09-18T14:03:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.277 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.277 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.277 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.277 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.277 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.277 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.277 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.277 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.277 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2450581591 0 0) 0x7f23301227a0 con 0x7f23300a0cb0 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f233009fa10 con 0x7f23300a0cb0 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1485879859 0 0) 0x7f23300a2270 con 0x7f233006e810 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 >> v1:172.21.15.45:6792/0 conn(0x7f23300a1080 legacy=0x7f2330149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 >> v1:172.21.15.84:6791/0 conn(0x7f23300a0cb0 legacy=0x7f23301468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 monclient: found mon.c 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:54.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23301224c0 con 0x7f233006e810 2024-09-18T14:03:54.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient: _finish_auth 0 2024-09-18T14:03:54.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:54.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f233506f640 5 monclient: authenticate success, global_id 144283 2024-09-18T14:03:54.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f233506f640 10 monclient: _renew_subs 2024-09-18T14:03:54.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f233506f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:54.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f233506f640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2330120a00 con 0x7f233006e810 2024-09-18T14:03:54.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f233506f640 10 monclient: _renew_subs 2024-09-18T14:03:54.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f233506f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:03:54.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f233506f640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2330135ab0 con 0x7f233006e810 2024-09-18T14:03:54.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2320002500 con 0x7f233006e810 2024-09-18T14:03:54.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:54.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2320002ab0 con 0x7f233006e810 2024-09-18T14:03:54.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f232d59a640 4 set_mon_vals no callback set 2024-09-18T14:03:54.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:54.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.261+0000 7f2316ffd640 10 monclient: dump: 2024-09-18T14:03:54.280 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.280 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.280 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.280 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.280 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.280 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.280 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.262+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2320002c10 con 0x7f233006e810 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.262+0000 7f233506f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.262+0000 7f233506f640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2310006580 con 0x7f2310002830 2024-09-18T14:03:54.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.262+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(231..231 src has 1..231) v4 ==== 3955+0+0 (unknown 3788058898 0 0) 0x7f2320046cc0 con 0x7f233006e810 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.262+0000 7f232f59e640 10 monclient: get_auth_request con 0x7f231c03a8f0 auth_method 0 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.262+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812805410 0 0) 0x7f2310006580 con 0x7f2310002830 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.262+0000 7f2316ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.262+0000 7f2316ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.262+0000 7f2316ffd640 10 monclient(hunting): my global_id is 4861 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.262+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23300a2270 con 0x7f2310002830 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 564804655 0 0) 0x7f23300a2270 con 0x7f2310002830 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2310006580 con 0x7f2310002830 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f23240021f0 con 0x7f2310002830 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 10 monclient: dump: 2024-09-18T14:03:54.282 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.284 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.284 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.284 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2225618233 0 0) 0x7f2310006580 con 0x7f2310002830 2024-09-18T14:03:54.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f231000a0a0 con 0x7f2310002830 2024-09-18T14:03:54.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2324004410 con 0x7f2310002830 2024-09-18T14:03:54.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:54.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:54.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 >> v1:172.21.15.84:6789/0 conn(0x7f2310002830 legacy=0x7f2310004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.263+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 >> v1:172.21.15.84:6789/0 conn(0x7f2310002830 legacy=0x7f2310004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.269+0000 7f233506f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.269+0000 7f233506f640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f23100095f0 con 0x7f2310008ff0 2024-09-18T14:03:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.269+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328485408 0 0) 0x7f23100095f0 con 0x7f2310008ff0 2024-09-18T14:03:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.269+0000 7f2316ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.269+0000 7f2316ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.269+0000 7f2316ffd640 10 monclient(hunting): my global_id is 4870 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.269+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2310006580 con 0x7f2310008ff0 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2640476150 0 0) 0x7f2310006580 con 0x7f2310008ff0 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f23100095f0 con 0x7f2310008ff0 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2324002cc0 con 0x7f2310008ff0 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 10 monclient: dump: 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4193459611 0 0) 0x7f23100095f0 con 0x7f2310008ff0 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f23100098b0 con 0x7f2310008ff0 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 3890427690) 0x7f23240046a0 con 0x7f2310008ff0 2024-09-18T14:03:54.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:54.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:54.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 >> v1:172.21.15.84:6789/0 conn(0x7f2310008ff0 legacy=0x7f2310008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f2316ffd640 1 -- 172.21.15.45:0/2911573778 >> v1:172.21.15.84:6789/0 conn(0x7f2310008ff0 legacy=0x7f2310008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:54.289 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:54.289 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:54.289 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:54.289 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:54.289 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:54.289 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:54.290 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:54.290 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:54.290 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:54.290 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:54.290 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:54.290 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:54.290 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:54.290 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:54.290 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:54.290 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.290 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:03:54.290 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.291 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:54.292 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:54.292 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.292 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.292 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.292 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.292 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.292 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.292 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.292 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.292 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.292 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:54.293 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:54.294 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:54.294 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:54.294 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:54.294 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:54.294 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:54.294 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.294 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.294 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.294 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.294 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.294 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.294 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.296 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:54.297 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:54.297 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:54.297 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.297 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:54.297 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:54.298 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.298 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.298 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.298 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:54.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:54.298 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.298 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.298 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:54.299 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.299 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.299 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:54.299 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:54.299 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.299 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.299 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.299 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:54.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:54.300 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.300 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.300 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:54.300 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.300 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.300 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.300 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:54.300 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:54.301 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.301 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.301 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.301 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:54.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:54.301 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.301 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.302 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:54.302 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.302 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.302 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.302 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:54.302 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:54.302 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.302 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.303 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.305 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.305 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:54.305 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.305 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.305 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.305 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:54.305 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:54.305 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.305 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.306 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.306 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:54.306 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:54.306 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.306 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.306 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.306 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:54.306 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.307 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.307 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.307 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:54.307 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:54.307 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.307 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.307 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.308 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:54.308 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:54.308 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.308 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.308 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:54.308 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.308 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.308 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.309 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:54.309 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:54.309 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.309 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.309 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.309 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.309 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.309 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:54.310 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.311 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.311 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:54.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.311 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.311 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.311 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:54.311 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.311 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.311 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:54.312 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:54.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f233506f640 1 -- 172.21.15.45:0/2911573778 >> v1:172.21.15.84:6812/31478 conn(0x7f231c03a8f0 legacy=0x7f231c03cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f233506f640 10 monclient: shutdown 2024-09-18T14:03:54.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f233506f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:54.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.270+0000 7f233506f640 1 -- 172.21.15.45:0/2911573778 >> v1:172.21.15.45:6791/0 conn(0x7f233006e810 legacy=0x7f23301353a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.271+0000 7f232f59e640 1 -- 172.21.15.45:0/2911573778 reap_dead start 2024-09-18T14:03:54.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.271+0000 7f232f59e640 1 -- 172.21.15.45:0/2911573778 reap_dead start 2024-09-18T14:03:54.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.271+0000 7f233506f640 1 -- 172.21.15.45:0/2911573778 shutdown_connections 2024-09-18T14:03:54.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.271+0000 7f233506f640 1 -- 172.21.15.45:0/2911573778 >> 172.21.15.45:0/2911573778 conn(0x7f23300a5bb0 msgr2=0x7f23300a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:54.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.271+0000 7f233506f640 1 -- 172.21.15.45:0/2911573778 shutdown_connections 2024-09-18T14:03:54.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.271+0000 7f233506f640 1 -- 172.21.15.45:0/2911573778 wait complete. 2024-09-18T14:03:54.315 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:54.315 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:03:54.315 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:03:54.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.412+0000 7f7730620640 10 monclient: get_monmap_and_config 2024-09-18T14:03:54.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.412+0000 7f7730620640 10 monclient: build_initial_monmap 2024-09-18T14:03:54.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.412+0000 7f7730620640 10 monclient: monmap: 2024-09-18T14:03:54.412 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:54.413010+0000 2024-09-18T14:03:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:54.413010+0000 2024-09-18T14:03:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:54.414 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:54.414 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:54.414 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:54.414 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:54.414 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.412+0000 7f7730620640 1 Processor -- start 2024-09-18T14:03:54.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.412+0000 7f7730620640 1 -- start start 2024-09-18T14:03:54.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.412+0000 7f7730620640 10 monclient: init 2024-09-18T14:03:54.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f7730620640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:54.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f7730620640 10 monclient: _add_conns ranks=[3,5,6,0,8,4,7,1,2] 2024-09-18T14:03:54.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f7730620640 10 monclient(hunting): picked mon.noname-g con 0x7f772806e9d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:03:54.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f7730620640 10 monclient(hunting): picked mon.noname-h con 0x7f772806a730 addr v1:172.21.15.84:6791/0 2024-09-18T14:03:54.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f7730620640 10 monclient(hunting): picked mon.noname-d con 0x7f77281348b0 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:54.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f7730620640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772805ed70 con 0x7f772806e9d0 2024-09-18T14:03:54.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f7730620640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772809bf60 con 0x7f772806a730 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f7730620640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772806eda0 con 0x7f77281348b0 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f7730620640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f7730620640 10 monclient(hunting): authenticate will time out at 1049.751494s 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772ebbd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f77281348b0 0x7f7728136c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:59030/0 (socket says 172.21.15.45:59030) 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772ebbd640 1 -- 172.21.15.45:0/1403859795 learned_addr learned my addr 172.21.15.45:0/1403859795 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772e3bc640 1 --1- 172.21.15.45:0/1403859795 >> v1:172.21.15.84:6790/0 conn(0x7f772806e9d0 0x7f772806c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:53064/0 (socket says 172.21.15.45:53064) 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669658334 0 0) 0x7f772806eda0 con 0x7f77281348b0 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 10 monclient(hunting): my global_id is 4848 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77140034e0 con 0x7f77281348b0 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232577320 0 0) 0x7f772805ed70 con 0x7f772806e9d0 2024-09-18T14:03:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 10 monclient(hunting): my global_id is 144192 2024-09-18T14:03:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772806eda0 con 0x7f772806e9d0 2024-09-18T14:03:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211863121 0 0) 0x7f772809bf60 con 0x7f772806a730 2024-09-18T14:03:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 10 monclient(hunting): my global_id is 4829 2024-09-18T14:03:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.413+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772805ed70 con 0x7f772806a730 2024-09-18T14:03:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 819649996 0 0) 0x7f77140034e0 con 0x7f77281348b0 2024-09-18T14:03:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 >> v1:172.21.15.84:6791/0 conn(0x7f772806a730 legacy=0x7f7728133190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 >> v1:172.21.15.84:6790/0 conn(0x7f772806e9d0 legacy=0x7f772806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 10 monclient: _finish_hunting 0 2024-09-18T14:03:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 1 monclient: found mon.noname-d 2024-09-18T14:03:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:03:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772806c0d0 con 0x7f77281348b0 2024-09-18T14:03:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 10 monclient: _finish_auth 0 2024-09-18T14:03:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7720001830 con 0x7f77281348b0 2024-09-18T14:03:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:03:54.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 10 monclient: dump: 2024-09-18T14:03:54.426 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.426 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.427 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f7730620640 5 monclient: authenticate success, global_id 4848 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f7730620640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7720001a10 con 0x7f77281348b0 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 1 -- 172.21.15.45:0/1403859795 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7720004ba0 con 0x7f77281348b0 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f772d3ba640 10 monclient: dump: 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.428 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f7730620640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f7730620640 4 set_mon_vals no callback set 2024-09-18T14:03:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f7730620640 10 monclient: shutdown 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f7730620640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f7730620640 1 -- 172.21.15.45:0/1403859795 >> v1:172.21.15.45:6792/0 conn(0x7f77281348b0 legacy=0x7f7728136c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f7730620640 1 -- 172.21.15.45:0/1403859795 shutdown_connections 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f7730620640 1 -- 172.21.15.45:0/1403859795 >> 172.21.15.45:0/1403859795 conn(0x7f77280a19a0 msgr2=0x7f77280a1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f7730620640 1 -- 172.21.15.45:0/1403859795 shutdown_connections 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.414+0000 7f7730620640 1 -- 172.21.15.45:0/1403859795 wait complete. 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.415+0000 7f7730620640 10 monclient: build_initial_monmap 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.415+0000 7f7730620640 10 monclient: monmap: 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:54.430 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.415+0000 7f7730620640 1 Processor -- start 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.415+0000 7f7730620640 1 -- start start 2024-09-18T14:03:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.415+0000 7f7730620640 10 monclient: init 2024-09-18T14:03:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f7730620640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f7730620640 10 monclient: _add_conns ranks=[6,2,0,7,4,5,8,3,1] 2024-09-18T14:03:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f7730620640 10 monclient(hunting): picked mon.noname-d con 0x7f772806a730 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f7730620640 10 monclient(hunting): picked mon.noname-b con 0x7f772806e9d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:03:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f7730620640 10 monclient(hunting): picked mon.noname-a con 0x7f77281348b0 addr v1:172.21.15.45:6789/0 2024-09-18T14:03:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f7730620640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772806af80 con 0x7f77281348b0 2024-09-18T14:03:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f7730620640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772812dc40 con 0x7f772806e9d0 2024-09-18T14:03:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f7730620640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772809b540 con 0x7f772806a730 2024-09-18T14:03:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f7730620640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f7730620640 10 monclient(hunting): authenticate will time out at 1049.754203s 2024-09-18T14:03:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f772e3bc640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f772806a730 0x7f772813cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:59036/0 (socket says 172.21.15.45:59036) 2024-09-18T14:03:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f772e3bc640 1 -- 172.21.15.45:0/1921480566 learned_addr learned my addr 172.21.15.45:0/1921480566 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247501741 0 0) 0x7f772809b540 con 0x7f772806a730 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): my global_id is 4857 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f770c0034e0 con 0x7f772806a730 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229181798 0 0) 0x7f772812dc40 con 0x7f772806e9d0 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): my global_id is 144209 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772809b540 con 0x7f772806e9d0 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846803878 0 0) 0x7f772806af80 con 0x7f77281348b0 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): my global_id is 4986 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772812dc40 con 0x7f77281348b0 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3084997732 0 0) 0x7f772809b540 con 0x7f772806e9d0 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f772806af80 con 0x7f772806e9d0 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7718001660 con 0x7f772806e9d0 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): dump: 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.435 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 73042976 0 0) 0x7f772812dc40 con 0x7f77281348b0 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f772809b540 con 0x7f77281348b0 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 192975168 0 0) 0x7f770c0034e0 con 0x7f772806a730 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f772812dc40 con 0x7f772806a730 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7720001770 con 0x7f77281348b0 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): dump: 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7724001910 con 0x7f772806a730 2024-09-18T14:03:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 10 monclient(hunting): dump: 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1448569134 0 0) 0x7f772806af80 con 0x7f772806e9d0 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.416+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 >> v1:172.21.15.45:6792/0 conn(0x7f772806a730 legacy=0x7f772813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 >> v1:172.21.15.45:6789/0 conn(0x7f77281348b0 legacy=0x7f7728145230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 1 monclient: found mon.b 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:03:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772809b260 con 0x7f772806e9d0 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f7730620640 5 monclient: authenticate success, global_id 144209 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f7730620640 10 monclient: _renew_subs 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f7730620640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f7730620640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772813dc20 con 0x7f772806e9d0 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f7730620640 10 monclient: _renew_subs 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f7730620640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f7730620640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7728098f20 con 0x7f772806e9d0 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7718001f00 con 0x7f772806e9d0 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f77180023f0 con 0x7f772806e9d0 2024-09-18T14:03:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f772cbb9640 4 set_mon_vals no callback set 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.417+0000 7f771e7fc640 10 monclient: dump: 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f7730620640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f7730620640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7710006580 con 0x7f7710002830 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7718002550 con 0x7f772806e9d0 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(231..231 src has 1..231) v4 ==== 3955+0+0 (unknown 3788058898 0 0) 0x7f7718044160 con 0x7f772806e9d0 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f772ebbd640 10 monclient: get_auth_request con 0x7f770c043260 auth_method 0 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159940440 0 0) 0x7f7710006580 con 0x7f7710002830 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 10 monclient(hunting): my global_id is 144201 2024-09-18T14:03:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772806af80 con 0x7f7710002830 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2321436992 0 0) 0x7f772806af80 con 0x7f7710002830 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7710006580 con 0x7f7710002830 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7724001a70 con 0x7f7710002830 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 10 monclient: dump: 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 42000664 0 0) 0x7f7710006580 con 0x7f7710002830 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.418+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f771000a0a0 con 0x7f7710002830 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.419+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f77240070d0 con 0x7f7710002830 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.419+0000 7f771e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.419+0000 7f771e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.419+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 >> v1:172.21.15.84:6790/0 conn(0x7f7710002830 legacy=0x7f7710004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.419+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 >> v1:172.21.15.84:6790/0 conn(0x7f7710002830 legacy=0x7f7710004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:54.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f7730620640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f7730620640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7710009660 con 0x7f7710009040 2024-09-18T14:03:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164764480 0 0) 0x7f7710009660 con 0x7f7710009040 2024-09-18T14:03:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 10 monclient(hunting): my global_id is 144210 2024-09-18T14:03:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7710006580 con 0x7f7710009040 2024-09-18T14:03:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3355525751 0 0) 0x7f7710006580 con 0x7f7710009040 2024-09-18T14:03:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7710009660 con 0x7f7710009040 2024-09-18T14:03:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7724005a30 con 0x7f7710009040 2024-09-18T14:03:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 10 monclient: dump: 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.450 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3941851926 0 0) 0x7f7710009660 con 0x7f7710009040 2024-09-18T14:03:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.425+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7710009920 con 0x7f7710009040 2024-09-18T14:03:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3443634014) 0x7f772400a0c0 con 0x7f7710009040 2024-09-18T14:03:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f771e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f771e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 >> v1:172.21.15.84:6790/0 conn(0x7f7710009040 legacy=0x7f7710008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f771e7fc640 1 -- 172.21.15.45:0/1921480566 >> v1:172.21.15.84:6790/0 conn(0x7f7710009040 legacy=0x7f7710008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:54.450 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:03:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:54.470 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:54.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f7730620640 1 -- 172.21.15.45:0/1921480566 >> v1:172.21.15.84:6812/31478 conn(0x7f770c043260 legacy=0x7f770c045710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f7730620640 10 monclient: shutdown 2024-09-18T14:03:54.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f7730620640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:54.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f7730620640 1 -- 172.21.15.45:0/1921480566 >> v1:172.21.15.45:6790/0 conn(0x7f772806e9d0 legacy=0x7f772812d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f772ebbd640 1 -- 172.21.15.45:0/1921480566 reap_dead start 2024-09-18T14:03:54.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f772ebbd640 1 -- 172.21.15.45:0/1921480566 reap_dead start 2024-09-18T14:03:54.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f7730620640 1 -- 172.21.15.45:0/1921480566 shutdown_connections 2024-09-18T14:03:54.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f7730620640 1 -- 172.21.15.45:0/1921480566 >> 172.21.15.45:0/1921480566 conn(0x7f77280a19a0 msgr2=0x7f772809dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:54.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f7730620640 1 -- 172.21.15.45:0/1921480566 shutdown_connections 2024-09-18T14:03:54.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.426+0000 7f7730620640 1 -- 172.21.15.45:0/1921480566 wait complete. 2024-09-18T14:03:54.473 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:54.473 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:03:54.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:03:54.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.569+0000 7fc8a379d640 10 monclient: get_monmap_and_config 2024-09-18T14:03:54.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.569+0000 7fc8a379d640 10 monclient: build_initial_monmap 2024-09-18T14:03:54.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.569+0000 7fc8a379d640 10 monclient: monmap: 2024-09-18T14:03:54.570 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:54.570 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.570 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:54.570296+0000 2024-09-18T14:03:54.570 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:54.570296+0000 2024-09-18T14:03:54.570 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:54.570 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.570 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:54.570 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:54.571 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:54.571 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:54.571 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:54.571 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:54.571 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:54.571 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:54.571 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:54.571 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.569+0000 7fc8a379d640 1 Processor -- start 2024-09-18T14:03:54.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a379d640 1 -- start start 2024-09-18T14:03:54.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a379d640 10 monclient: init 2024-09-18T14:03:54.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a379d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a379d640 10 monclient: _add_conns ranks=[4,6,3,7,5,1,2,0,8] 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a379d640 10 monclient(hunting): picked mon.noname-c con 0x7fc89c09f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a379d640 10 monclient(hunting): picked mon.noname-d con 0x7fc89c09fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a379d640 10 monclient(hunting): picked mon.noname-g con 0x7fc89c09ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a379d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c05ed70 con 0x7fc89c09ff70 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a379d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c09dfc0 con 0x7fc89c09f7d0 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a379d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c09dce0 con 0x7fc89c09fba0 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a379d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a379d640 10 monclient(hunting): authenticate will time out at 1049.908639s 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a1539640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fc89c09f7d0 0x7fc89c132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:38536/0 (socket says 172.21.15.45:38536) 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a0d38640 1 --1- 172.21.15.45:0/2183459310 >> v1:172.21.15.45:6792/0 conn(0x7fc89c09fba0 0x7fc89c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:59044/0 (socket says 172.21.15.45:59044) 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.570+0000 7fc8a1539640 1 -- 172.21.15.45:0/2183459310 learned_addr learned my addr 172.21.15.45:0/2183459310 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:54.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285710487 0 0) 0x7fc89c09dce0 con 0x7fc89c09fba0 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient(hunting): my global_id is 4866 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8880034e0 con 0x7fc89c09fba0 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249684365 0 0) 0x7fc89c09dfc0 con 0x7fc89c09f7d0 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient(hunting): my global_id is 144292 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc89c09dce0 con 0x7fc89c09f7d0 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 109556830 0 0) 0x7fc89c05ed70 con 0x7fc89c09ff70 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient(hunting): my global_id is 144219 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc89c09dfc0 con 0x7fc89c09ff70 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1679951842 0 0) 0x7fc89c09dce0 con 0x7fc89c09f7d0 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 >> v1:172.21.15.45:6792/0 conn(0x7fc89c09fba0 legacy=0x7fc89c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 >> v1:172.21.15.84:6790/0 conn(0x7fc89c09ff70 legacy=0x7fc89c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient: _finish_hunting 0 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 monclient: found mon.noname-c 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0a1c00 con 0x7fc89c09f7d0 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient: _finish_auth 0 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc8a379d640 5 monclient: authenticate success, global_id 144292 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc898001b90 con 0x7fc89c09f7d0 2024-09-18T14:03:54.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc8a379d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient: dump: 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.583 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc8a379d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc898001cf0 con 0x7fc89c09f7d0 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 1 -- 172.21.15.45:0/2183459310 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc898004f20 con 0x7fc89c09f7d0 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc8a379d640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc8a379d640 4 set_mon_vals no callback set 2024-09-18T14:03:54.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc893fff640 10 monclient: dump: 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.585 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc8a379d640 10 monclient: shutdown 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc8a379d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.571+0000 7fc8a379d640 1 -- 172.21.15.45:0/2183459310 >> v1:172.21.15.45:6791/0 conn(0x7fc89c09f7d0 legacy=0x7fc89c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.572+0000 7fc8a379d640 1 -- 172.21.15.45:0/2183459310 shutdown_connections 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.572+0000 7fc8a379d640 1 -- 172.21.15.45:0/2183459310 >> 172.21.15.45:0/2183459310 conn(0x7fc89c0630b0 msgr2=0x7fc89c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.572+0000 7fc8a379d640 1 -- 172.21.15.45:0/2183459310 shutdown_connections 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.572+0000 7fc8a379d640 1 -- 172.21.15.45:0/2183459310 wait complete. 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.572+0000 7fc8a379d640 10 monclient: build_initial_monmap 2024-09-18T14:03:54.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.572+0000 7fc8a379d640 10 monclient: monmap: 2024-09-18T14:03:54.587 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:54.587 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:54.587 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:54.587 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:54.587 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:54.587 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:54.587 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:54.587 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:54.587 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:54.587 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.572+0000 7fc8a379d640 1 Processor -- start 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.572+0000 7fc8a379d640 1 -- start start 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.572+0000 7fc8a379d640 10 monclient: init 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8a379d640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8a379d640 10 monclient: _add_conns ranks=[6,2,4,8,7,3,0,5,1] 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8a379d640 10 monclient(hunting): picked mon.noname-d con 0x7fc89c09f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:03:54.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8a379d640 10 monclient(hunting): picked mon.noname-b con 0x7fc89c09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8a379d640 10 monclient(hunting): picked mon.noname-c con 0x7fc89c09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8a379d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c0a2560 con 0x7fc89c09fba0 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8a379d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c142610 con 0x7fc89c09ff70 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8a379d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c11ffe0 con 0x7fc89c09f7d0 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8a379d640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8a379d640 10 monclient(hunting): authenticate will time out at 1049.911619s 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8a1539640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fc89c09f7d0 0x7fc89c1417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:59048/0 (socket says 172.21.15.45:59048) 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8a1539640 1 -- 172.21.15.45:0/1849435540 learned_addr learned my addr 172.21.15.45:0/1849435540 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515030349 0 0) 0x7fc89c11ffe0 con 0x7fc89c09f7d0 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 10 monclient(hunting): my global_id is 4875 2024-09-18T14:03:54.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc880003510 con 0x7fc89c09f7d0 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242592137 0 0) 0x7fc89c142610 con 0x7fc89c09ff70 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 10 monclient(hunting): my global_id is 144301 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc89c11ffe0 con 0x7fc89c09ff70 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254782008 0 0) 0x7fc89c0a2560 con 0x7fc89c09fba0 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 10 monclient(hunting): my global_id is 144218 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.573+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc89c142610 con 0x7fc89c09fba0 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2731612854 0 0) 0x7fc880003510 con 0x7fc89c09f7d0 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc89c0a2560 con 0x7fc89c09f7d0 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc898004b20 con 0x7fc89c09f7d0 2024-09-18T14:03:54.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient(hunting): dump: 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1792383218 0 0) 0x7fc89c11ffe0 con 0x7fc89c09ff70 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc880003510 con 0x7fc89c09ff70 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1605474653 0 0) 0x7fc89c142610 con 0x7fc89c09fba0 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc89c11ffe0 con 0x7fc89c09fba0 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc894001a00 con 0x7fc89c09ff70 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient(hunting): dump: 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc88c0015b0 con 0x7fc89c09fba0 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient(hunting): dump: 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 122001687 0 0) 0x7fc89c0a2560 con 0x7fc89c09f7d0 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 >> v1:172.21.15.45:6791/0 conn(0x7fc89c09ff70 legacy=0x7fc89c149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 >> v1:172.21.15.45:6790/0 conn(0x7fc89c09fba0 legacy=0x7fc89c1319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient: _finish_hunting 0 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 monclient: found mon.d 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c121b10 con 0x7fc89c09f7d0 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient: _finish_auth 0 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8a379d640 5 monclient: authenticate success, global_id 4875 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8a379d640 10 monclient: _renew_subs 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8a379d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8a379d640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c121df0 con 0x7fc89c09f7d0 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8a379d640 10 monclient: _renew_subs 2024-09-18T14:03:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8a379d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8a379d640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c141eb0 con 0x7fc89c09f7d0 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc8980051a0 con 0x7fc89c09f7d0 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc898005650 con 0x7fc89c09f7d0 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8937fe640 4 set_mon_vals no callback set 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.574+0000 7fc8917fa640 10 monclient: dump: 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.575+0000 7fc8a379d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.575+0000 7fc8a379d640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc87c006580 con 0x7fc87c002830 2024-09-18T14:03:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.575+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc898010620 con 0x7fc89c09f7d0 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.575+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(231..231 src has 1..231) v4 ==== 3955+0+0 (unknown 3788058898 0 0) 0x7fc898047110 con 0x7fc89c09f7d0 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.575+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989360867 0 0) 0x7fc87c006580 con 0x7fc87c002830 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.575+0000 7fc8917fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.575+0000 7fc8917fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.575+0000 7fc8917fa640 10 monclient(hunting): my global_id is 4838 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.575+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc89c0a2560 con 0x7fc87c002830 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.575+0000 7fc8a1d3a640 10 monclient: get_auth_request con 0x7fc88003aa10 auth_method 0 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 315956562 0 0) 0x7fc89c0a2560 con 0x7fc87c002830 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc87c006580 con 0x7fc87c002830 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc88c0010a0 con 0x7fc87c002830 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 10 monclient: dump: 2024-09-18T14:03:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.601 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1338161337 0 0) 0x7fc87c006580 con 0x7fc87c002830 2024-09-18T14:03:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc87c00a0a0 con 0x7fc87c002830 2024-09-18T14:03:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc88c004f30 con 0x7fc87c002830 2024-09-18T14:03:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 >> v1:172.21.15.84:6791/0 conn(0x7fc87c002830 legacy=0x7fc87c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.576+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 >> v1:172.21.15.84:6791/0 conn(0x7fc87c002830 legacy=0x7fc87c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.582+0000 7fc8a379d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.582+0000 7fc8a379d640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc87c0074f0 con 0x7fc87c0068a0 2024-09-18T14:03:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.582+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737472232 0 0) 0x7fc87c0074f0 con 0x7fc87c0068a0 2024-09-18T14:03:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.582+0000 7fc8917fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.582+0000 7fc8917fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.582+0000 7fc8917fa640 10 monclient(hunting): my global_id is 4847 2024-09-18T14:03:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.582+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc87c006580 con 0x7fc87c0068a0 2024-09-18T14:03:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2780893286 0 0) 0x7fc87c006580 con 0x7fc87c0068a0 2024-09-18T14:03:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc87c0074f0 con 0x7fc87c0068a0 2024-09-18T14:03:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc88c005690 con 0x7fc87c0068a0 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 10 monclient: dump: 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.605 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3686364244 0 0) 0x7fc87c0074f0 con 0x7fc87c0068a0 2024-09-18T14:03:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc87c008750 con 0x7fc87c0068a0 2024-09-18T14:03:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7726 (unknown 0 0 136294043) 0x7fc88c006a80 con 0x7fc87c0068a0 2024-09-18T14:03:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 >> v1:172.21.15.84:6791/0 conn(0x7fc87c0068a0 legacy=0x7fc87c006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.583+0000 7fc8917fa640 1 -- 172.21.15.45:0/1849435540 >> v1:172.21.15.84:6791/0 conn(0x7fc87c0068a0 legacy=0x7fc87c006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:54.606 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.607 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.608 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.608 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.608 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.608 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.608 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.608 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.608 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.608 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.608 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:54.608 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:54.608 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.608 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:54.609 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:54.610 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:54.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:54.612 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:54.613 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.614 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.616 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:54.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:54.616 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.616 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.616 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:54.617 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:54.618 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.618 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.618 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:54.618 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:54.619 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.621 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:54.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:54.621 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.621 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.621 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:54.622 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.623 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.625 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:54.627 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.584+0000 7fc8a379d640 1 -- 172.21.15.45:0/1849435540 >> v1:172.21.15.84:6812/31478 conn(0x7fc88003aa10 legacy=0x7fc88003cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.584+0000 7fc8a379d640 10 monclient: shutdown 2024-09-18T14:03:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.584+0000 7fc8a379d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.584+0000 7fc8a379d640 1 -- 172.21.15.45:0/1849435540 >> v1:172.21.15.45:6792/0 conn(0x7fc89c09f7d0 legacy=0x7fc89c1417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.584+0000 7fc8a1d3a640 1 -- 172.21.15.45:0/1849435540 reap_dead start 2024-09-18T14:03:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.584+0000 7fc8a1d3a640 1 -- 172.21.15.45:0/1849435540 reap_dead start 2024-09-18T14:03:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.584+0000 7fc8a379d640 1 -- 172.21.15.45:0/1849435540 shutdown_connections 2024-09-18T14:03:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.584+0000 7fc8a379d640 1 -- 172.21.15.45:0/1849435540 >> 172.21.15.45:0/1849435540 conn(0x7fc89c0630b0 msgr2=0x7fc89c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:54.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.584+0000 7fc8a379d640 1 -- 172.21.15.45:0/1849435540 shutdown_connections 2024-09-18T14:03:54.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.584+0000 7fc8a379d640 1 -- 172.21.15.45:0/1849435540 wait complete. 2024-09-18T14:03:54.630 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:54.630 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:03:54.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:03:54.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.727+0000 7f2f2ac4f640 10 monclient: get_monmap_and_config 2024-09-18T14:03:54.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.727+0000 7f2f2ac4f640 10 monclient: build_initial_monmap 2024-09-18T14:03:54.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.727+0000 7f2f2ac4f640 10 monclient: monmap: 2024-09-18T14:03:54.728 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:54.728 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.728 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:03:54.728592+0000 2024-09-18T14:03:54.728 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:03:54.728592+0000 2024-09-18T14:03:54.728 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:54.728 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.728 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:54.729 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:54.729 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:54.729 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:54.729 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:54.729 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:54.729 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:54.729 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:54.729 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:54.729 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.728+0000 7f2f2ac4f640 1 Processor -- start 2024-09-18T14:03:54.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.728+0000 7f2f2ac4f640 1 -- start start 2024-09-18T14:03:54.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.728+0000 7f2f2ac4f640 10 monclient: init 2024-09-18T14:03:54.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.728+0000 7f2f2ac4f640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:54.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.728+0000 7f2f2ac4f640 10 monclient: _add_conns ranks=[2,8,7,1,0,5,4,3,6] 2024-09-18T14:03:54.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.728+0000 7f2f2ac4f640 10 monclient(hunting): picked mon.noname-b con 0x7f2f2409f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:03:54.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f2ac4f640 10 monclient(hunting): picked mon.noname-e con 0x7f2f2409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:03:54.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f2ac4f640 10 monclient(hunting): picked mon.noname-i con 0x7f2f2409ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:03:54.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f2ac4f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2405ed70 con 0x7f2f2409f7d0 2024-09-18T14:03:54.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f2ac4f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2409dfc0 con 0x7f2f2409ff70 2024-09-18T14:03:54.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f2ac4f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2409dce0 con 0x7f2f2409fba0 2024-09-18T14:03:54.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f2ac4f640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:54.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f2ac4f640 10 monclient(hunting): authenticate will time out at 1050.067131s 2024-09-18T14:03:54.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f23fff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f2f2409fba0 0x7f2f24137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:50112/0 (socket says 172.21.15.45:50112) 2024-09-18T14:03:54.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f289eb640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f2f2409f7d0 0x7f2f24132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49138/0 (socket says 172.21.15.45:49138) 2024-09-18T14:03:54.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f23fff640 1 -- 172.21.15.45:0/780570013 learned_addr learned my addr 172.21.15.45:0/780570013 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 1 -- 172.21.15.45:0/780570013 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158889001 0 0) 0x7f2f2405ed70 con 0x7f2f2409f7d0 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient(hunting): my global_id is 144227 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 1 -- 172.21.15.45:0/780570013 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f140034e0 con 0x7f2f2409f7d0 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 1 -- 172.21.15.45:0/780570013 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899563487 0 0) 0x7f2f2409dce0 con 0x7f2f2409fba0 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient(hunting): my global_id is 4967 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 1 -- 172.21.15.45:0/780570013 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f2405ed70 con 0x7f2f2409fba0 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 1 -- 172.21.15.45:0/780570013 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 934052169 0 0) 0x7f2f140034e0 con 0x7f2f2409f7d0 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 1 -- 172.21.15.45:0/780570013 >> v1:172.21.15.45:6793/0 conn(0x7f2f2409fba0 legacy=0x7f2f24137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 1 -- 172.21.15.45:0/780570013 >> v1:172.21.15.84:6792/0 conn(0x7f2f2409ff70 legacy=0x7f2f2413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient: _finish_hunting 0 2024-09-18T14:03:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 1 monclient: found mon.noname-b 2024-09-18T14:03:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:03:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 1 -- 172.21.15.45:0/780570013 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f240a1c00 con 0x7f2f2409f7d0 2024-09-18T14:03:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient: _finish_auth 0 2024-09-18T14:03:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 1 -- 172.21.15.45:0/780570013 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f18001830 con 0x7f2f2409f7d0 2024-09-18T14:03:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f2ac4f640 5 monclient: authenticate success, global_id 144227 2024-09-18T14:03:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f2ac4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:03:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient: dump: 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f2ac4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 1 -- 172.21.15.45:0/780570013 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f18001990 con 0x7f2f2409f7d0 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 1 -- 172.21.15.45:0/780570013 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f18004ae0 con 0x7f2f2409f7d0 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f2ac4f640 10 monclient: get_monmap_and_config success 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f2ac4f640 4 set_mon_vals no callback set 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.729+0000 7f2f237fe640 10 monclient: dump: 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.730+0000 7f2f2ac4f640 10 monclient: shutdown 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.730+0000 7f2f2ac4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.730+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/780570013 >> v1:172.21.15.45:6790/0 conn(0x7f2f2409f7d0 legacy=0x7f2f24132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.730+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/780570013 shutdown_connections 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.730+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/780570013 >> 172.21.15.45:0/780570013 conn(0x7f2f240630b0 msgr2=0x7f2f240634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.730+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/780570013 shutdown_connections 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.730+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/780570013 wait complete. 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.730+0000 7f2f2ac4f640 10 monclient: build_initial_monmap 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.731+0000 7f2f2ac4f640 10 monclient: monmap: 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:03:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.731+0000 7f2f2ac4f640 1 Processor -- start 2024-09-18T14:03:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.731+0000 7f2f2ac4f640 1 -- start start 2024-09-18T14:03:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.731+0000 7f2f2ac4f640 10 monclient: init 2024-09-18T14:03:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.731+0000 7f2f2ac4f640 10 monclient: _reopen_session rank -1 2024-09-18T14:03:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.731+0000 7f2f2ac4f640 10 monclient: _add_conns ranks=[3,1,2,8,5,7,0,6,4] 2024-09-18T14:03:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 10 monclient(hunting): picked mon.noname-g con 0x7f2f2409f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:03:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 10 monclient(hunting): picked mon.noname-f con 0x7f2f2409fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:03:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 10 monclient(hunting): picked mon.noname-b con 0x7f2f2409ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:03:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2409d310 con 0x7f2f2409fba0 2024-09-18T14:03:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f240a2560 con 0x7f2f2409ff70 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f24141d40 con 0x7f2f2409f7d0 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 10 monclient(hunting): _renew_subs 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 10 monclient(hunting): authenticate will time out at 1050.070133s 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f291ec640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f2f2409ff70 0x7f2f24149980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49152/0 (socket says 172.21.15.45:49152) 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f291ec640 1 -- 172.21.15.45:0/1932786358 learned_addr learned my addr 172.21.15.45:0/1932786358 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438584283 0 0) 0x7f2f240a2560 con 0x7f2f2409ff70 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): my global_id is 144236 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f080034e0 con 0x7f2f2409ff70 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748870777 0 0) 0x7f2f24141d40 con 0x7f2f2409f7d0 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): my global_id is 144228 2024-09-18T14:03:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f240a2560 con 0x7f2f2409f7d0 2024-09-18T14:03:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287677659 0 0) 0x7f2f2409d310 con 0x7f2f2409fba0 2024-09-18T14:03:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): my global_id is 4879 2024-09-18T14:03:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f24141d40 con 0x7f2f2409fba0 2024-09-18T14:03:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1427104688 0 0) 0x7f2f080034e0 con 0x7f2f2409ff70 2024-09-18T14:03:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f2409d310 con 0x7f2f2409ff70 2024-09-18T14:03:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f1c001dc0 con 0x7f2f2409ff70 2024-09-18T14:03:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): dump: 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.749 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.749 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.749 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:03:54.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3503961777 0 0) 0x7f2f2409d310 con 0x7f2f2409ff70 2024-09-18T14:03:54.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 >> v1:172.21.15.84:6790/0 conn(0x7f2f2409f7d0 legacy=0x7f2f24141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 >> v1:172.21.15.84:6789/0 conn(0x7f2f2409fba0 legacy=0x7f2f241307f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 monclient: found mon.b 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24120940 con 0x7f2f2409ff70 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 10 monclient: _finish_auth 0 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f20ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 5 monclient: authenticate success, global_id 144236 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 10 monclient: _renew_subs 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f24130f00 con 0x7f2f2409ff70 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 10 monclient: _renew_subs 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.732+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2411fb80 con 0x7f2f2409ff70 2024-09-18T14:03:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.733+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f1c002080 con 0x7f2f2409ff70 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.733+0000 7f2f20ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.733+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f1c002970 con 0x7f2f2409ff70 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.733+0000 7f2f20ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.733+0000 7f2f20ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.733+0000 7f2f22ffd640 4 set_mon_vals no callback set 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.733+0000 7f2f20ff9640 10 monclient: dump: 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.733+0000 7f2f2ac4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.733+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2f04006580 con 0x7f2f04002830 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.733+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2f1c002b50 con 0x7f2f2409ff70 2024-09-18T14:03:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(231..231 src has 1..231) v4 ==== 3955+0+0 (unknown 3788058898 0 0) 0x7f2f1c046ed0 con 0x7f2f2409ff70 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f23fff640 10 monclient: get_auth_request con 0x7f2f08039ea0 auth_method 0 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537796109 0 0) 0x7f2f04006580 con 0x7f2f04002830 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 10 monclient(hunting): my global_id is 4921 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f2409d310 con 0x7f2f04002830 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2739116565 0 0) 0x7f2f2409d310 con 0x7f2f04002830 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f04006580 con 0x7f2f04002830 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f18001240 con 0x7f2f04002830 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 10 monclient: dump: 2024-09-18T14:03:54.753 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.754 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.755 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.755 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 301560656 0 0) 0x7f2f04006580 con 0x7f2f04002830 2024-09-18T14:03:54.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.734+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f0400a0a0 con 0x7f2f04002830 2024-09-18T14:03:54.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.735+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2f180051e0 con 0x7f2f04002830 2024-09-18T14:03:54.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.735+0000 7f2f20ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:03:54.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.735+0000 7f2f20ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:03:54.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.735+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 >> v1:172.21.15.84:6792/0 conn(0x7f2f04002830 legacy=0x7f2f04004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.735+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 >> v1:172.21.15.84:6792/0 conn(0x7f2f04002830 legacy=0x7f2f04004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:54.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.740+0000 7f2f2ac4f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:03:54.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.740+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2f04007540 con 0x7f2f040068f0 2024-09-18T14:03:54.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898550922 0 0) 0x7f2f04007540 con 0x7f2f040068f0 2024-09-18T14:03:54.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:03:54.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:03:54.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 10 monclient(hunting): my global_id is 4930 2024-09-18T14:03:54.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f04006580 con 0x7f2f040068f0 2024-09-18T14:03:54.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3768301310 0 0) 0x7f2f04006580 con 0x7f2f040068f0 2024-09-18T14:03:54.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f04007540 con 0x7f2f040068f0 2024-09-18T14:03:54.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f18005b00 con 0x7f2f040068f0 2024-09-18T14:03:54.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 10 monclient: dump: 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:03:54.758 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:03:54.759 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:03:54.759 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:03:54.759 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:03:54.759 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:03:54.759 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:03:54.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3139404750 0 0) 0x7f2f04007540 con 0x7f2f040068f0 2024-09-18T14:03:54.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.741+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2f040087a0 con 0x7f2f040068f0 2024-09-18T14:03:54.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 1472863757) 0x7f2f180081d0 con 0x7f2f040068f0 2024-09-18T14:03:54.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f20ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:03:54.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f20ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:03:54.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 >> v1:172.21.15.84:6792/0 conn(0x7f2f040068f0 legacy=0x7f2f04006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f20ff9640 1 -- 172.21.15.45:0/1932786358 >> v1:172.21.15.84:6792/0 conn(0x7f2f040068f0 legacy=0x7f2f04006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:03:54.760 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:03:54.760 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:54.760 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:54.760 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:03:54.760 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 64, 2024-09-18T14:03:54.760 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:03:54.761 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:03:54.761 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:03:54.761 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:03:54.761 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:03:54.761 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:03:54.761 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:03:54.761 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:03:54.761 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:03:54.761 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:03:54.761 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.761 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:03:54.761 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.762 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:03:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:03:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.764 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:03:54.764 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:03:54.764 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:03:54.764 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:03:54.764 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:03:54.764 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:03:54.764 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:54.764 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:03:54.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:03:54.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:03:54.765 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:03:54.765 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:03:54.765 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:03:54.765 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:03:54.765 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:03:54.765 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:03:54.765 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:03:54.765 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:03:54.765 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:03:54.765 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:03:54.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.767 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.768 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.770 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:03:54.771 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.772 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.773 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.775 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.776 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:03:54.777 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:03:54.778 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:03:54.779 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:03:54.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:03:54.780 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:03:54.780 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:03:54.780 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:03:54.780 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:03:54.780 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:03:54.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:03:54.780 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:03:54.780 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:03:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/1932786358 >> v1:172.21.15.84:6812/31478 conn(0x7f2f08039ea0 legacy=0x7f2f0803c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f2ac4f640 10 monclient: shutdown 2024-09-18T14:03:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f2ac4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:03:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/1932786358 >> v1:172.21.15.45:6790/0 conn(0x7f2f2409ff70 legacy=0x7f2f24149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:03:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f291ec640 1 -- 172.21.15.45:0/1932786358 reap_dead start 2024-09-18T14:03:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f291ec640 1 -- 172.21.15.45:0/1932786358 reap_dead start 2024-09-18T14:03:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/1932786358 shutdown_connections 2024-09-18T14:03:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/1932786358 >> 172.21.15.45:0/1932786358 conn(0x7f2f240630b0 msgr2=0x7f2f240a47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:03:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/1932786358 shutdown_connections 2024-09-18T14:03:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:03:54.742+0000 7f2f2ac4f640 1 -- 172.21.15.45:0/1932786358 wait complete. 2024-09-18T14:03:54.783 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:03:54.783 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['g', 'c'] 2024-09-18T14:03:54.783 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['i'] 2024-09-18T14:03:54.783 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-18T14:03:54.783 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-18T14:03:54.784 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-18T14:03:54.784 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:03:54.796 INFO:tasks.ceph.mon.g:Stopped 2024-09-18T14:03:54.796 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-18T14:03:54.796 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-18T14:03:54.796 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T14:03:54.796 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:03:54.806 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T14:03:54.806 INFO:tasks.mon_thrash:Sending STOP to mon i 2024-09-18T14:03:54.806 INFO:tasks.ceph.mon.i:Sent signal 19 2024-09-18T14:03:54.806 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-18T14:04:09.807 INFO:tasks.mon_thrash:Sending CONT to mon i 2024-09-18T14:04:09.808 INFO:tasks.ceph.mon.i:Sent signal 18 2024-09-18T14:04:09.809 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-18T14:04:09.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:04:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.917+0000 7fd949d9d640 10 monclient: get_monmap_and_config 2024-09-18T14:04:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.917+0000 7fd949d9d640 10 monclient: build_initial_monmap 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.917+0000 7fd949d9d640 10 monclient: monmap: 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:04:09.918160+0000 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:04:09.918160+0000 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:09.919 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 1 Processor -- start 2024-09-18T14:04:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 1 -- start start 2024-09-18T14:04:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 10 monclient: init 2024-09-18T14:04:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 10 monclient: _add_conns ranks=[6,3,8,7,5,2,0,1,4] 2024-09-18T14:04:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 10 monclient(hunting): picked mon.noname-d con 0x7fd94406e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:04:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 10 monclient(hunting): picked mon.noname-g con 0x7fd9440a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:04:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 10 monclient(hunting): picked mon.noname-e con 0x7fd9440a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:04:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94405ed70 con 0x7fd9440a0cb0 2024-09-18T14:04:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd944097620 con 0x7fd94406e810 2024-09-18T14:04:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd944097340 con 0x7fd9440a1080 2024-09-18T14:04:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd949d9d640 10 monclient(hunting): authenticate will time out at 1065.256760s 2024-09-18T14:04:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd9437fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fd94406e810 0x7fd9440a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:38140/0 (socket says 172.21.15.45:38140) 2024-09-18T14:04:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd9437fe640 1 -- 172.21.15.45:0/2936799835 learned_addr learned my addr 172.21.15.45:0/2936799835 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:09.918+0000 7fd942ffd640 1 -- 172.21.15.45:0/2936799835 >> v1:172.21.15.84:6790/0 conn(0x7fd9440a0cb0 legacy=0x7fd944137810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:04:10.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:10.119+0000 7fd942ffd640 1 -- 172.21.15.45:0/2936799835 >> v1:172.21.15.84:6790/0 conn(0x7fd9440a0cb0 legacy=0x7fd944137810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:04:10.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:10.520+0000 7fd942ffd640 1 -- 172.21.15.45:0/2936799835 >> v1:172.21.15.84:6790/0 conn(0x7fd9440a0cb0 legacy=0x7fd944137810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:04:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:11.321+0000 7fd942ffd640 1 -- 172.21.15.45:0/2936799835 >> v1:172.21.15.84:6790/0 conn(0x7fd9440a0cb0 legacy=0x7fd944137810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:04:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 10 monclient(hunting): tick 2024-09-18T14:04:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 1 monclient(hunting): continuing hunt 2024-09-18T14:04:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:04:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 1 -- 172.21.15.45:0/2936799835 >> v1:172.21.15.45:6793/0 conn(0x7fd9440a1080 legacy=0x7fd94413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 1 -- 172.21.15.45:0/2936799835 >> v1:172.21.15.45:6792/0 conn(0x7fd94406e810 legacy=0x7fd9440a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 1 -- 172.21.15.45:0/2936799835 >> v1:172.21.15.84:6790/0 conn(0x7fd9440a0cb0 legacy=0x7fd944137810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:04:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 10 monclient: _add_conns ranks=[0,1,2,7,4,5] 2024-09-18T14:04:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 10 monclient(hunting): picked mon.noname-a con 0x7fd930003780 addr v1:172.21.15.45:6789/0 2024-09-18T14:04:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 10 monclient(hunting): picked mon.noname-f con 0x7fd930007400 addr v1:172.21.15.84:6789/0 2024-09-18T14:04:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 10 monclient(hunting): picked mon.noname-b con 0x7fd93000b140 addr v1:172.21.15.45:6790/0 2024-09-18T14:04:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 1 -- 172.21.15.45:0/2936799835 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9440a1660 con 0x7fd930003780 2024-09-18T14:04:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 1 -- 172.21.15.45:0/2936799835 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94405ed70 con 0x7fd930007400 2024-09-18T14:04:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 1 -- 172.21.15.45:0/2936799835 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd930010fa0 con 0x7fd93000b140 2024-09-18T14:04:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:12.919+0000 7fd9417fa640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:14.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559319233 0 0) 0x7fd94405ed70 con 0x7fd930007400 2024-09-18T14:04:14.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:14.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:14.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient(hunting): my global_id is 4888 2024-09-18T14:04:14.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd92c0034e0 con 0x7fd930007400 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978653354 0 0) 0x7fd930010fa0 con 0x7fd93000b140 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient(hunting): my global_id is 144245 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd92c004860 con 0x7fd93000b140 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421925271 0 0) 0x7fd9440a1660 con 0x7fd930003780 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient(hunting): my global_id is 4995 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd930010fa0 con 0x7fd930003780 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2879983064 0 0) 0x7fd92c0034e0 con 0x7fd930007400 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 >> v1:172.21.15.45:6790/0 conn(0x7fd93000b140 legacy=0x7fd93000d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 >> v1:172.21.15.45:6789/0 conn(0x7fd930003780 legacy=0x7fd930005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient: _finish_hunting 0 2024-09-18T14:04:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 1 monclient: found mon.noname-f 2024-09-18T14:04:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T14:04:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd930012260 con 0x7fd930007400 2024-09-18T14:04:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient: _finish_auth 0 2024-09-18T14:04:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd943fff640 1 -- 172.21.15.45:0/2936799835 reap_dead start 2024-09-18T14:04:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd9380042b0 con 0x7fd930007400 2024-09-18T14:04:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:04:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.816+0000 7fd9427fc640 10 monclient: dump: 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:14.822 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.817+0000 7fd949d9d640 5 monclient: authenticate success, global_id 4888 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.817+0000 7fd949d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.817+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd938004480 con 0x7fd930007400 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.817+0000 7fd9427fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.817+0000 7fd9427fc640 1 -- 172.21.15.45:0/2936799835 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd938004a30 con 0x7fd930007400 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.817+0000 7fd949d9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.817+0000 7fd949d9d640 4 set_mon_vals no callback set 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.817+0000 7fd9427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.817+0000 7fd9427fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.817+0000 7fd9427fc640 10 monclient: dump: 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.818+0000 7fd949d9d640 10 monclient: shutdown 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.818+0000 7fd949d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.818+0000 7fd949d9d640 1 -- 172.21.15.45:0/2936799835 >> v1:172.21.15.84:6789/0 conn(0x7fd930007400 legacy=0x7fd930009870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.818+0000 7fd949d9d640 1 -- 172.21.15.45:0/2936799835 shutdown_connections 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.818+0000 7fd949d9d640 1 -- 172.21.15.45:0/2936799835 >> 172.21.15.45:0/2936799835 conn(0x7fd9440a5bb0 msgr2=0x7fd9440a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.818+0000 7fd949d9d640 1 -- 172.21.15.45:0/2936799835 shutdown_connections 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.818+0000 7fd949d9d640 1 -- 172.21.15.45:0/2936799835 wait complete. 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.818+0000 7fd949d9d640 10 monclient: build_initial_monmap 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.818+0000 7fd949d9d640 10 monclient: monmap: 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:04:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:04:14.826 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:14.826 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:04:14.826 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:14.826 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:14.826 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:14.826 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:14.826 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:14.826 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:14.826 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:14.826 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:14.826 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:14.826 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:14.827 INFO:tasks.rados.rados.0.smithi084.stdout:1005: done (5 left) 2024-09-18T14:04:14.827 INFO:tasks.rados.rados.0.smithi084.stdout:1006: snap_remove snap 77 2024-09-18T14:04:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.819+0000 7fd949d9d640 1 Processor -- start 2024-09-18T14:04:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.819+0000 7fd949d9d640 1 -- start start 2024-09-18T14:04:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.819+0000 7fd949d9d640 10 monclient: init 2024-09-18T14:04:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd949d9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd949d9d640 10 monclient: _add_conns ranks=[6,4,1,3,5,7,2,8,0] 2024-09-18T14:04:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd949d9d640 10 monclient(hunting): picked mon.noname-d con 0x7fd930007400 addr v1:172.21.15.45:6792/0 2024-09-18T14:04:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd949d9d640 10 monclient(hunting): picked mon.noname-c con 0x7fd94411fee0 addr v1:172.21.15.45:6791/0 2024-09-18T14:04:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd949d9d640 10 monclient(hunting): picked mon.noname-f con 0x7fd944147fd0 addr v1:172.21.15.84:6789/0 2024-09-18T14:04:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd949d9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94409fa10 con 0x7fd944147fd0 2024-09-18T14:04:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd949d9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9440a2270 con 0x7fd94411fee0 2024-09-18T14:04:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd949d9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9441202b0 con 0x7fd930007400 2024-09-18T14:04:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd942ffd640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7fd94411fee0 legacy=0x7fd9441468b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:04:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd949d9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd949d9d640 10 monclient(hunting): authenticate will time out at 1070.158524s 2024-09-18T14:04:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd9437fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fd930007400 0x7fd944135370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:38156/0 (socket says 172.21.15.45:38156) 2024-09-18T14:04:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.820+0000 7fd9437fe640 1 -- 172.21.15.45:0/3853434888 learned_addr learned my addr 172.21.15.45:0/3853434888 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138954077 0 0) 0x7fd9441202b0 con 0x7fd930007400 2024-09-18T14:04:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 10 monclient(hunting): my global_id is 4884 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd930010650 con 0x7fd930007400 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3057037415 0 0) 0x7fd930010650 con 0x7fd930007400 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd9441202b0 con 0x7fd930007400 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd934001a40 con 0x7fd930007400 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 10 monclient(hunting): dump: 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:04:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723702883 0 0) 0x7fd94409fa10 con 0x7fd944147fd0 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 10 monclient(hunting): my global_id is 4897 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.823+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd930010650 con 0x7fd944147fd0 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2303541776 0 0) 0x7fd9441202b0 con 0x7fd930007400 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 >> v1:172.21.15.45:6791/0 conn(0x7fd94411fee0 legacy=0x7fd9441468b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 >> v1:172.21.15.84:6789/0 conn(0x7fd944147fd0 legacy=0x7fd94414a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 1 monclient: found mon.d 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd944142490 con 0x7fd930007400 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 10 monclient: _finish_auth 0 2024-09-18T14:04:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd949d9d640 5 monclient: authenticate success, global_id 4884 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd949d9d640 10 monclient: _renew_subs 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd949d9d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd949d9d640 1 -- 172.21.15.45:0/3853434888 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd944120ee0 con 0x7fd930007400 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd949d9d640 10 monclient: _renew_subs 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd949d9d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd949d9d640 1 -- 172.21.15.45:0/3853434888 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9441215a0 con 0x7fd930007400 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd934001d60 con 0x7fd930007400 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd934004e10 con 0x7fd930007400 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:04:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd92b7fe640 10 monclient: dump: 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:14.838 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.824+0000 7fd941ffb640 4 set_mon_vals no callback set 2024-09-18T14:04:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.825+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd93400fde0 con 0x7fd930007400 2024-09-18T14:04:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.825+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(232..232 src has 1..232) v4 ==== 3971+0+0 (unknown 4008190398 0 0) 0x7fd934046aa0 con 0x7fd930007400 2024-09-18T14:04:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.825+0000 7fd949d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.825+0000 7fd949d9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.825+0000 7fd949d9d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:04:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.825+0000 7fd949d9d640 1 -- 172.21.15.45:0/3853434888 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd924005010 con 0x7fd930007400 2024-09-18T14:04:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.825+0000 7fd942ffd640 10 monclient: get_auth_request con 0x7fd93003b250 auth_method 0 2024-09-18T14:04:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.830+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fd934014e20 con 0x7fd930007400 2024-09-18T14:04:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.830+0000 7fd92b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:14.830+0000 7fd92b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:04:15.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.066+0000 7fd949d9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:04:15.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.066+0000 7fd949d9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:04:15.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.066+0000 7fd949d9d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:04:15.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.066+0000 7fd949d9d640 1 -- 172.21.15.45:0/3853434888 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd924005840 con 0x7fd930007400 2024-09-18T14:04:15.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.066+0000 7fd92b7fe640 1 -- 172.21.15.45:0/3853434888 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2676 (unknown 254746657 0 3560181144) 0x7fd934014c40 con 0x7fd930007400 2024-09-18T14:04:15.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.066+0000 7fd92b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:04:15.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.066+0000 7fd92b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:04:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":70,"quorum":[0,1,2,5,6,7,8],"quorum_names":["a","f","b","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:04:15.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.068+0000 7fd949d9d640 1 -- 172.21.15.45:0/3853434888 >> v1:172.21.15.84:6812/31478 conn(0x7fd93003b250 legacy=0x7fd93003d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.068+0000 7fd949d9d640 10 monclient: shutdown 2024-09-18T14:04:15.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.068+0000 7fd949d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:15.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.068+0000 7fd949d9d640 1 -- 172.21.15.45:0/3853434888 >> v1:172.21.15.45:6792/0 conn(0x7fd930007400 legacy=0x7fd944135370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.068+0000 7fd949d9d640 1 -- 172.21.15.45:0/3853434888 shutdown_connections 2024-09-18T14:04:15.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.068+0000 7fd949d9d640 1 -- 172.21.15.45:0/3853434888 >> 172.21.15.45:0/3853434888 conn(0x7fd9440a5bb0 msgr2=0x7fd9440a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:15.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.068+0000 7fd949d9d640 1 -- 172.21.15.45:0/3853434888 shutdown_connections 2024-09-18T14:04:15.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.068+0000 7fd949d9d640 1 -- 172.21.15.45:0/3853434888 wait complete. 2024-09-18T14:04:15.079 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-18T14:04:15.079 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-09-18T14:04:15.079 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:04:15.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.175+0000 7f239262b640 10 monclient: get_monmap_and_config 2024-09-18T14:04:15.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.175+0000 7f239262b640 10 monclient: build_initial_monmap 2024-09-18T14:04:15.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.175+0000 7f239262b640 10 monclient: monmap: 2024-09-18T14:04:15.177 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:15.177 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.177 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:04:15.176667+0000 2024-09-18T14:04:15.177 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:04:15.176667+0000 2024-09-18T14:04:15.177 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:15.177 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.177 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:15.177 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:15.177 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:15.178 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:15.178 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:15.178 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:15.178 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:15.178 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:15.178 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:15.178 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 1 Processor -- start 2024-09-18T14:04:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 1 -- start start 2024-09-18T14:04:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 10 monclient: init 2024-09-18T14:04:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 10 monclient: _add_conns ranks=[1,4,2,6,0,7,8,5,3] 2024-09-18T14:04:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 10 monclient(hunting): picked mon.noname-f con 0x7f238c09f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:04:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 10 monclient(hunting): picked mon.noname-c con 0x7f238c09fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 10 monclient(hunting): picked mon.noname-b con 0x7f238c09ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c05ed70 con 0x7f238c09f7d0 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c09dfc0 con 0x7f238c09ff70 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c09dce0 con 0x7f238c09fba0 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.176+0000 7f239262b640 10 monclient(hunting): authenticate will time out at 1070.514988s 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238b7fe640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f238c09fba0 legacy=0x7f238c137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f2390bc8640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f238c09ff70 0x7f238c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49054/0 (socket says 172.21.15.45:49054) 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f2390bc8640 1 -- 172.21.15.45:0/1259727408 learned_addr learned my addr 172.21.15.45:0/1259727408 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 1 -- 172.21.15.45:0/1259727408 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784239572 0 0) 0x7f238c09dfc0 con 0x7f238c09ff70 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient(hunting): my global_id is 144254 2024-09-18T14:04:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 1 -- 172.21.15.45:0/1259727408 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23740034e0 con 0x7f238c09ff70 2024-09-18T14:04:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 1 -- 172.21.15.45:0/1259727408 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 670270220 0 0) 0x7f23740034e0 con 0x7f238c09ff70 2024-09-18T14:04:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 1 -- 172.21.15.45:0/1259727408 >> v1:172.21.15.45:6791/0 conn(0x7f238c09fba0 legacy=0x7f238c137a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:04:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 1 -- 172.21.15.45:0/1259727408 >> v1:172.21.15.84:6789/0 conn(0x7f238c09f7d0 legacy=0x7f238c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient: _finish_hunting 0 2024-09-18T14:04:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 1 monclient: found mon.noname-b 2024-09-18T14:04:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:04:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 1 -- 172.21.15.45:0/1259727408 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c0a1c00 con 0x7f238c09ff70 2024-09-18T14:04:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient: _finish_auth 0 2024-09-18T14:04:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 1 -- 172.21.15.45:0/1259727408 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2384001bb0 con 0x7f238c09ff70 2024-09-18T14:04:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:04:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient: dump: 2024-09-18T14:04:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f239262b640 5 monclient: authenticate success, global_id 144254 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f239262b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 1 -- 172.21.15.45:0/1259727408 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2384001d90 con 0x7f238c09ff70 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 1 -- 172.21.15.45:0/1259727408 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2384002040 con 0x7f238c09ff70 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f239262b640 10 monclient: get_monmap_and_config success 2024-09-18T14:04:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f239262b640 4 set_mon_vals no callback set 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.177+0000 7f238affd640 10 monclient: dump: 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.178+0000 7f239262b640 10 monclient: shutdown 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.178+0000 7f239262b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.178+0000 7f239262b640 1 -- 172.21.15.45:0/1259727408 >> v1:172.21.15.45:6790/0 conn(0x7f238c09ff70 legacy=0x7f238c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.178+0000 7f239262b640 1 -- 172.21.15.45:0/1259727408 shutdown_connections 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.178+0000 7f239262b640 1 -- 172.21.15.45:0/1259727408 >> 172.21.15.45:0/1259727408 conn(0x7f238c0630b0 msgr2=0x7f238c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.178+0000 7f239262b640 1 -- 172.21.15.45:0/1259727408 shutdown_connections 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.178+0000 7f239262b640 1 -- 172.21.15.45:0/1259727408 wait complete. 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.178+0000 7f239262b640 10 monclient: build_initial_monmap 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.178+0000 7f239262b640 10 monclient: monmap: 2024-09-18T14:04:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:15.192 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.179+0000 7f239262b640 1 Processor -- start 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.179+0000 7f239262b640 1 -- start start 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.179+0000 7f239262b640 10 monclient: init 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.179+0000 7f239262b640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.179+0000 7f239262b640 10 monclient: _add_conns ranks=[1,3,4,6,7,2,5,0,8] 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.179+0000 7f239262b640 10 monclient(hunting): picked mon.noname-f con 0x7f238c09f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.179+0000 7f239262b640 10 monclient(hunting): picked mon.noname-g con 0x7f238c09fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.179+0000 7f239262b640 10 monclient(hunting): picked mon.noname-c con 0x7f238c09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.179+0000 7f239262b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c09d310 con 0x7f238c09f7d0 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.179+0000 7f2390bc8640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f238c09ff70 legacy=0x7f238c149b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.179+0000 7f238b7fe640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7f238c09fba0 legacy=0x7f238c1319b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f239262b640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c0a2560 con 0x7f238c09fba0 2024-09-18T14:04:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.179+0000 7f238bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f238c09f7d0 0x7f238c1417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:34072/0 (socket says 172.21.15.45:34072) 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f238bfff640 1 -- 172.21.15.45:0/2700777277 learned_addr learned my addr 172.21.15.45:0/2700777277 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f239262b640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c141eb0 con 0x7f238c09ff70 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f239262b640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f239262b640 10 monclient(hunting): authenticate will time out at 1070.518050s 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072708868 0 0) 0x7f238c09d310 con 0x7f238c09f7d0 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 10 monclient(hunting): my global_id is 4915 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23780034e0 con 0x7f238c09f7d0 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 563989185 0 0) 0x7f23780034e0 con 0x7f238c09f7d0 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f238c09d310 con 0x7f238c09f7d0 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f237c001bc0 con 0x7f238c09f7d0 2024-09-18T14:04:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 10 monclient(hunting): dump: 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1579126388 0 0) 0x7f238c09d310 con 0x7f238c09f7d0 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 >> v1:172.21.15.45:6791/0 conn(0x7f238c09ff70 legacy=0x7f238c149b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 >> v1:172.21.15.84:6790/0 conn(0x7f238c09fba0 legacy=0x7f238c1319b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 10 monclient: _finish_hunting 0 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 1 monclient: found mon.f 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:04:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c142100 con 0x7f238c09f7d0 2024-09-18T14:04:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 10 monclient: _finish_auth 0 2024-09-18T14:04:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f2373fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f239262b640 5 monclient: authenticate success, global_id 4915 2024-09-18T14:04:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f239262b640 10 monclient: _renew_subs 2024-09-18T14:04:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f239262b640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:04:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.180+0000 7f239262b640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238c121b00 con 0x7f238c09f7d0 2024-09-18T14:04:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.181+0000 7f239262b640 10 monclient: _renew_subs 2024-09-18T14:04:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.181+0000 7f239262b640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:04:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.181+0000 7f239262b640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f238c1320c0 con 0x7f238c09f7d0 2024-09-18T14:04:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.181+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f237c0022b0 con 0x7f238c09f7d0 2024-09-18T14:04:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.181+0000 7f2373fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.181+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f237c002820 con 0x7f238c09f7d0 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.181+0000 7f2373fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.181+0000 7f238a7fc640 4 set_mon_vals no callback set 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.181+0000 7f2373fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.181+0000 7f2373fff640 10 monclient: dump: 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.181+0000 7f239262b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.181+0000 7f239262b640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f236c006580 con 0x7f236c002830 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f237c002980 con 0x7f238c09f7d0 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(232..232 src has 1..232) v4 ==== 3971+0+0 (unknown 4008190398 0 0) 0x7f237c046cf0 con 0x7f238c09f7d0 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874368571 0 0) 0x7f236c006580 con 0x7f236c002830 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 10 monclient(hunting): my global_id is 5004 2024-09-18T14:04:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f238c09d310 con 0x7f236c002830 2024-09-18T14:04:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2390bc8640 10 monclient: get_auth_request con 0x7f2378039b70 auth_method 0 2024-09-18T14:04:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3921421029 0 0) 0x7f238c09d310 con 0x7f236c002830 2024-09-18T14:04:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f236c006580 con 0x7f236c002830 2024-09-18T14:04:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2380004080 con 0x7f236c002830 2024-09-18T14:04:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:04:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 10 monclient: dump: 2024-09-18T14:04:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.201 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1720836133 0 0) 0x7f236c006580 con 0x7f236c002830 2024-09-18T14:04:15.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.182+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f236c00a0a0 con 0x7f236c002830 2024-09-18T14:04:15.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.183+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2380006360 con 0x7f236c002830 2024-09-18T14:04:15.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.183+0000 7f2373fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:15.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.183+0000 7f2373fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:04:15.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.183+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 >> v1:172.21.15.45:6789/0 conn(0x7f236c002830 legacy=0x7f236c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.183+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 >> v1:172.21.15.45:6789/0 conn(0x7f236c002830 legacy=0x7f236c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:15.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.188+0000 7f239262b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:04:15.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.188+0000 7f239262b640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f236c009660 con 0x7f236c009040 2024-09-18T14:04:15.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671306025 0 0) 0x7f236c009660 con 0x7f236c009040 2024-09-18T14:04:15.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 10 monclient(hunting): my global_id is 5013 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f236c006580 con 0x7f236c009040 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3026535139 0 0) 0x7f236c006580 con 0x7f236c009040 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f236c009660 con 0x7f236c009040 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2380004bc0 con 0x7f236c009040 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 10 monclient: dump: 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3584924260 0 0) 0x7f236c009660 con 0x7f236c009040 2024-09-18T14:04:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f236c009920 con 0x7f236c009040 2024-09-18T14:04:15.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7364 (unknown 0 0 2686767163) 0x7f23800060d0 con 0x7f236c009040 2024-09-18T14:04:15.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:04:15.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:04:15.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 >> v1:172.21.15.45:6789/0 conn(0x7f236c009040 legacy=0x7f236c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.189+0000 7f2373fff640 1 -- 172.21.15.45:0/2700777277 >> v1:172.21.15.45:6789/0 conn(0x7f236c009040 legacy=0x7f236c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:15.206 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:04:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:04:15.206 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:04:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:04:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 70, 2024-09-18T14:04:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:04:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:04:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:04:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:04:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:04:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:04:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:04:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:04:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.207 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.208 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:04:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:04:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.209 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:04:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.211 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:04:15.212 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:04:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:04:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:04:15.213 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:04:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:04:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:04:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:04:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:04:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:04:15.216 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.217 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.218 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:04:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:04:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:04:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:04:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:04:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:04:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:04:15.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:04:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:04:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:04:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:04:15.224 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.225 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:04:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:04:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.226 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:04:15.226 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:04:15.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.190+0000 7f239262b640 1 -- 172.21.15.45:0/2700777277 >> v1:172.21.15.84:6812/31478 conn(0x7f2378039b70 legacy=0x7f237803c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.190+0000 7f239262b640 10 monclient: shutdown 2024-09-18T14:04:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.190+0000 7f239262b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.190+0000 7f239262b640 1 -- 172.21.15.45:0/2700777277 >> v1:172.21.15.84:6789/0 conn(0x7f238c09f7d0 legacy=0x7f238c1417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.190+0000 7f2390bc8640 1 -- 172.21.15.45:0/2700777277 reap_dead start 2024-09-18T14:04:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.190+0000 7f2390bc8640 1 -- 172.21.15.45:0/2700777277 reap_dead start 2024-09-18T14:04:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.190+0000 7f239262b640 1 -- 172.21.15.45:0/2700777277 shutdown_connections 2024-09-18T14:04:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.190+0000 7f239262b640 1 -- 172.21.15.45:0/2700777277 >> 172.21.15.45:0/2700777277 conn(0x7f238c0630b0 msgr2=0x7f238c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.190+0000 7f239262b640 1 -- 172.21.15.45:0/2700777277 shutdown_connections 2024-09-18T14:04:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.190+0000 7f239262b640 1 -- 172.21.15.45:0/2700777277 wait complete. 2024-09-18T14:04:15.229 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:04:15.229 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-09-18T14:04:15.229 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:04:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.326+0000 7f9a3979d640 10 monclient: get_monmap_and_config 2024-09-18T14:04:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.326+0000 7f9a3979d640 10 monclient: build_initial_monmap 2024-09-18T14:04:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.326+0000 7f9a3979d640 10 monclient: monmap: 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:04:15.327183+0000 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:04:15.327183+0000 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:15.328 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:15.328 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:15.328 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.326+0000 7f9a3979d640 1 Processor -- start 2024-09-18T14:04:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.326+0000 7f9a3979d640 1 -- start start 2024-09-18T14:04:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.326+0000 7f9a3979d640 10 monclient: init 2024-09-18T14:04:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a3979d640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a3979d640 10 monclient: _add_conns ranks=[6,1,0,5,7,8,3,2,4] 2024-09-18T14:04:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a3979d640 10 monclient(hunting): picked mon.noname-d con 0x7f9a3406e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:04:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a3979d640 10 monclient(hunting): picked mon.noname-f con 0x7f9a340a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:04:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a3979d640 10 monclient(hunting): picked mon.noname-a con 0x7f9a340a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:04:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a3979d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3405ed70 con 0x7f9a340a1080 2024-09-18T14:04:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a3979d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a34097620 con 0x7f9a340a0cb0 2024-09-18T14:04:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a3979d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a34097340 con 0x7f9a3406e810 2024-09-18T14:04:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a3979d640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a3979d640 10 monclient(hunting): authenticate will time out at 1070.665374s 2024-09-18T14:04:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a32ffd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f9a3406e810 0x7f9a340a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:38172/0 (socket says 172.21.15.45:38172) 2024-09-18T14:04:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a32ffd640 1 -- 172.21.15.45:0/1709324135 learned_addr learned my addr 172.21.15.45:0/1709324135 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167882326 0 0) 0x7f9a3405ed70 con 0x7f9a340a1080 2024-09-18T14:04:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 10 monclient(hunting): my global_id is 5022 2024-09-18T14:04:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a200034e0 con 0x7f9a340a1080 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698962762 0 0) 0x7f9a34097340 con 0x7f9a3406e810 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 10 monclient(hunting): my global_id is 4893 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a3405ed70 con 0x7f9a3406e810 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056457582 0 0) 0x7f9a34097620 con 0x7f9a340a0cb0 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 10 monclient(hunting): my global_id is 4924 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a34097340 con 0x7f9a340a0cb0 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3248647769 0 0) 0x7f9a200034e0 con 0x7f9a340a1080 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.327+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 >> v1:172.21.15.45:6792/0 conn(0x7f9a3406e810 legacy=0x7f9a340a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 >> v1:172.21.15.84:6789/0 conn(0x7f9a340a0cb0 legacy=0x7f9a34137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 1 monclient: found mon.noname-a 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a340a1660 con 0x7f9a340a1080 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 10 monclient: _finish_auth 0 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9a2c001760 con 0x7f9a340a1080 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a3979d640 5 monclient: authenticate success, global_id 5022 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a3979d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:04:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 10 monclient: dump: 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a3979d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9a2c0018c0 con 0x7f9a340a1080 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 1 -- 172.21.15.45:0/1709324135 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9a2c001df0 con 0x7f9a340a1080 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a31ffb640 10 monclient: dump: 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.342 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a3979d640 10 monclient: get_monmap_and_config success 2024-09-18T14:04:15.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a3979d640 4 set_mon_vals no callback set 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a3979d640 10 monclient: shutdown 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a3979d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a3979d640 1 -- 172.21.15.45:0/1709324135 >> v1:172.21.15.45:6789/0 conn(0x7f9a340a1080 legacy=0x7f9a3413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a3979d640 1 -- 172.21.15.45:0/1709324135 shutdown_connections 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a3979d640 1 -- 172.21.15.45:0/1709324135 >> 172.21.15.45:0/1709324135 conn(0x7f9a340a5bb0 msgr2=0x7f9a340a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a3979d640 1 -- 172.21.15.45:0/1709324135 shutdown_connections 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.328+0000 7f9a3979d640 1 -- 172.21.15.45:0/1709324135 wait complete. 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.329+0000 7f9a3979d640 10 monclient: build_initial_monmap 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.329+0000 7f9a3979d640 10 monclient: monmap: 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:04:15.344 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.329+0000 7f9a3979d640 1 Processor -- start 2024-09-18T14:04:15.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.329+0000 7f9a3979d640 1 -- start start 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.329+0000 7f9a3979d640 10 monclient: init 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 10 monclient: _add_conns ranks=[6,4,1,7,2,0,3,8,5] 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 10 monclient(hunting): picked mon.noname-d con 0x7f9a3406e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 10 monclient(hunting): picked mon.noname-c con 0x7f9a340a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 10 monclient(hunting): picked mon.noname-f con 0x7f9a340a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a34122e60 con 0x7f9a340a1080 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3413dc50 con 0x7f9a340a0cb0 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a327fc640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f9a340a0cb0 legacy=0x7f9a34146770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a341228a0 con 0x7f9a3406e810 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 10 monclient(hunting): authenticate will time out at 1070.668221s 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a32ffd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f9a3406e810 0x7f9a3413d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:38176/0 (socket says 172.21.15.45:38176) 2024-09-18T14:04:15.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a32ffd640 1 -- 172.21.15.45:0/2590656093 learned_addr learned my addr 172.21.15.45:0/2590656093 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:15.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682479052 0 0) 0x7f9a341228a0 con 0x7f9a3406e810 2024-09-18T14:04:15.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient(hunting): my global_id is 4902 2024-09-18T14:04:15.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a140034e0 con 0x7f9a3406e810 2024-09-18T14:04:15.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615378774 0 0) 0x7f9a34122e60 con 0x7f9a340a1080 2024-09-18T14:04:15.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient(hunting): my global_id is 4933 2024-09-18T14:04:15.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a341228a0 con 0x7f9a340a1080 2024-09-18T14:04:15.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1162092375 0 0) 0x7f9a140034e0 con 0x7f9a3406e810 2024-09-18T14:04:15.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9a34122e60 con 0x7f9a3406e810 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9a240018f0 con 0x7f9a3406e810 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient(hunting): dump: 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.348 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2866220197 0 0) 0x7f9a34122e60 con 0x7f9a3406e810 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 >> v1:172.21.15.45:6791/0 conn(0x7f9a340a0cb0 legacy=0x7f9a34146770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 >> v1:172.21.15.84:6789/0 conn(0x7f9a340a1080 legacy=0x7f9a34149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient: _finish_hunting 0 2024-09-18T14:04:15.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 1 monclient: found mon.d 2024-09-18T14:04:15.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:15.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:04:15.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a341225c0 con 0x7f9a3406e810 2024-09-18T14:04:15.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 10 monclient: _finish_auth 0 2024-09-18T14:04:15.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a2affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:15.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 5 monclient: authenticate success, global_id 4902 2024-09-18T14:04:15.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 10 monclient: _renew_subs 2024-09-18T14:04:15.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:04:15.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a34122b80 con 0x7f9a3406e810 2024-09-18T14:04:15.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 10 monclient: _renew_subs 2024-09-18T14:04:15.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:04:15.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.330+0000 7f9a3979d640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a341216f0 con 0x7f9a3406e810 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.331+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9a24004dd0 con 0x7f9a3406e810 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.331+0000 7f9a2affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.331+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9a24005280 con 0x7f9a3406e810 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.331+0000 7f9a2affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.331+0000 7f9a317fa640 4 set_mon_vals no callback set 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.331+0000 7f9a2affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.331+0000 7f9a2affd640 10 monclient: dump: 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.351 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.352 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.352 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.352 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.352 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.352 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.352 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.352 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.352 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.352 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.331+0000 7f9a3979d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:15.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.331+0000 7f9a3979d640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9a18006580 con 0x7f9a18002830 2024-09-18T14:04:15.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.331+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f9a24010250 con 0x7f9a3406e810 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(232..232 src has 1..232) v4 ==== 3971+0+0 (unknown 4008190398 0 0) 0x7f9a24046f20 con 0x7f9a3406e810 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144090672 0 0) 0x7f9a18006580 con 0x7f9a18002830 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 10 monclient(hunting): my global_id is 144263 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a34122e60 con 0x7f9a18002830 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a337fe640 10 monclient: get_auth_request con 0x7f9a14039c90 auth_method 0 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2174636919 0 0) 0x7f9a34122e60 con 0x7f9a18002830 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9a18006580 con 0x7f9a18002830 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9a1c002d00 con 0x7f9a18002830 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:04:15.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 10 monclient: dump: 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.354 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.355 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.355 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3528328104 0 0) 0x7f9a18006580 con 0x7f9a18002830 2024-09-18T14:04:15.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a1800a0a0 con 0x7f9a18002830 2024-09-18T14:04:15.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9a1c006430 con 0x7f9a18002830 2024-09-18T14:04:15.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:15.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:04:15.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 >> v1:172.21.15.45:6790/0 conn(0x7f9a18002830 legacy=0x7f9a18004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.332+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 >> v1:172.21.15.45:6790/0 conn(0x7f9a18002830 legacy=0x7f9a18004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.338+0000 7f9a3979d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:04:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.338+0000 7f9a3979d640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9a18009660 con 0x7f9a18009040 2024-09-18T14:04:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.338+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220239099 0 0) 0x7f9a18009660 con 0x7f9a18009040 2024-09-18T14:04:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.338+0000 7f9a2affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.338+0000 7f9a2affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.338+0000 7f9a2affd640 10 monclient(hunting): my global_id is 144272 2024-09-18T14:04:15.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.338+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a18006580 con 0x7f9a18009040 2024-09-18T14:04:15.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3753639386 0 0) 0x7f9a18006580 con 0x7f9a18009040 2024-09-18T14:04:15.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9a18009660 con 0x7f9a18009040 2024-09-18T14:04:15.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9a1c004cd0 con 0x7f9a18009040 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 10 monclient: dump: 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.362 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 225917447 0 0) 0x7f9a18009660 con 0x7f9a18009040 2024-09-18T14:04:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9a18009920 con 0x7f9a18009040 2024-09-18T14:04:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7364 (unknown 0 0 3150096974) 0x7f9a1c006680 con 0x7f9a18009040 2024-09-18T14:04:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:04:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:04:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 >> v1:172.21.15.45:6790/0 conn(0x7f9a18009040 legacy=0x7f9a18008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a2affd640 1 -- 172.21.15.45:0/2590656093 >> v1:172.21.15.45:6790/0 conn(0x7f9a18009040 legacy=0x7f9a18008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:15.363 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:04:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:04:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:04:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:04:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 70, 2024-09-18T14:04:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:04:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:04:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:04:15.363 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:04:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:04:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:04:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:04:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:04:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:04:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:04:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:04:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.364 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.365 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:04:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.367 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.368 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:04:15.369 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:04:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:04:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:04:15.372 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.373 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.374 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:04:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:04:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:04:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.378 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:04:15.379 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:04:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:04:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:04:15.380 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:04:15.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:04:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:04:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:15.382 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.383 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:04:15.383 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:04:15.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a3979d640 1 -- 172.21.15.45:0/2590656093 >> v1:172.21.15.84:6812/31478 conn(0x7f9a14039c90 legacy=0x7f9a1403c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a3979d640 10 monclient: shutdown 2024-09-18T14:04:15.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a3979d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:15.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a3979d640 1 -- 172.21.15.45:0/2590656093 >> v1:172.21.15.45:6792/0 conn(0x7f9a3406e810 legacy=0x7f9a3413d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a337fe640 1 -- 172.21.15.45:0/2590656093 reap_dead start 2024-09-18T14:04:15.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a337fe640 1 -- 172.21.15.45:0/2590656093 reap_dead start 2024-09-18T14:04:15.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a3979d640 1 -- 172.21.15.45:0/2590656093 shutdown_connections 2024-09-18T14:04:15.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.339+0000 7f9a3979d640 1 -- 172.21.15.45:0/2590656093 >> 172.21.15.45:0/2590656093 conn(0x7f9a340a5bb0 msgr2=0x7f9a340a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:15.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.340+0000 7f9a3979d640 1 -- 172.21.15.45:0/2590656093 shutdown_connections 2024-09-18T14:04:15.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.340+0000 7f9a3979d640 1 -- 172.21.15.45:0/2590656093 wait complete. 2024-09-18T14:04:15.385 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:04:15.385 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-18T14:04:15.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:04:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.482+0000 7f194b554640 10 monclient: get_monmap_and_config 2024-09-18T14:04:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.482+0000 7f194b554640 10 monclient: build_initial_monmap 2024-09-18T14:04:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.482+0000 7f194b554640 10 monclient: monmap: 2024-09-18T14:04:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:04:15.483564+0000 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:04:15.483564+0000 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 1 Processor -- start 2024-09-18T14:04:15.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 1 -- start start 2024-09-18T14:04:15.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 10 monclient: init 2024-09-18T14:04:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 10 monclient: _add_conns ranks=[1,7,4,0,3,8,6,2,5] 2024-09-18T14:04:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 10 monclient(hunting): picked mon.noname-f con 0x7f194409f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:04:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 10 monclient(hunting): picked mon.noname-i con 0x7f194409fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:04:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 10 monclient(hunting): picked mon.noname-c con 0x7f194409ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:04:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194405ed70 con 0x7f194409f7d0 2024-09-18T14:04:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194409dfc0 con 0x7f194409ff70 2024-09-18T14:04:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194409dce0 con 0x7f194409fba0 2024-09-18T14:04:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f194b554640 10 monclient(hunting): authenticate will time out at 1070.821906s 2024-09-18T14:04:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.483+0000 7f1949af1640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f194409ff70 legacy=0x7f194413b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:04:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f19492f0640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f194409f7d0 0x7f1944132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:34102/0 (socket says 172.21.15.45:34102) 2024-09-18T14:04:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f19492f0640 1 -- 172.21.15.45:0/233511802 learned_addr learned my addr 172.21.15.45:0/233511802 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 1 -- 172.21.15.45:0/233511802 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095158566 0 0) 0x7f194405ed70 con 0x7f194409f7d0 2024-09-18T14:04:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient(hunting): my global_id is 4942 2024-09-18T14:04:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 1 -- 172.21.15.45:0/233511802 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19300034e0 con 0x7f194409f7d0 2024-09-18T14:04:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 1 -- 172.21.15.45:0/233511802 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879674781 0 0) 0x7f194409dce0 con 0x7f194409fba0 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient(hunting): my global_id is 4939 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 1 -- 172.21.15.45:0/233511802 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f194405ed70 con 0x7f194409fba0 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 1 -- 172.21.15.45:0/233511802 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3433684513 0 0) 0x7f19300034e0 con 0x7f194409f7d0 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 1 -- 172.21.15.45:0/233511802 >> v1:172.21.15.84:6792/0 conn(0x7f194409fba0 legacy=0x7f1944137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 1 -- 172.21.15.45:0/233511802 >> v1:172.21.15.45:6791/0 conn(0x7f194409ff70 legacy=0x7f194413b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 1 monclient: found mon.noname-f 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 1 -- 172.21.15.45:0/233511802 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19440a1c00 con 0x7f194409f7d0 2024-09-18T14:04:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient: _finish_auth 0 2024-09-18T14:04:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 1 -- 172.21.15.45:0/233511802 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1940001830 con 0x7f194409f7d0 2024-09-18T14:04:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:04:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient: dump: 2024-09-18T14:04:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f194b554640 5 monclient: authenticate success, global_id 4942 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f194b554640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 1 -- 172.21.15.45:0/233511802 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1940001a10 con 0x7f194409f7d0 2024-09-18T14:04:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f194b554640 10 monclient: get_monmap_and_config success 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 1 -- 172.21.15.45:0/233511802 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1940004ba0 con 0x7f194409f7d0 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f194b554640 4 set_mon_vals no callback set 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.484+0000 7f193bfff640 10 monclient: dump: 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.485+0000 7f194b554640 10 monclient: shutdown 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.485+0000 7f194b554640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.485+0000 7f194b554640 1 -- 172.21.15.45:0/233511802 >> v1:172.21.15.84:6789/0 conn(0x7f194409f7d0 legacy=0x7f1944132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.485+0000 7f194b554640 1 -- 172.21.15.45:0/233511802 shutdown_connections 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.485+0000 7f194b554640 1 -- 172.21.15.45:0/233511802 >> 172.21.15.45:0/233511802 conn(0x7f19440630b0 msgr2=0x7f19440634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.485+0000 7f194b554640 1 -- 172.21.15.45:0/233511802 shutdown_connections 2024-09-18T14:04:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.485+0000 7f194b554640 1 -- 172.21.15.45:0/233511802 wait complete. 2024-09-18T14:04:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.485+0000 7f194b554640 10 monclient: build_initial_monmap 2024-09-18T14:04:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.485+0000 7f194b554640 10 monclient: monmap: 2024-09-18T14:04:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:04:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:04:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:04:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:04:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 1 Processor -- start 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 1 -- start start 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 10 monclient: init 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 10 monclient: _add_conns ranks=[7,2,5,0,8,6,3,1,4] 2024-09-18T14:04:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 10 monclient(hunting): picked mon.noname-i con 0x7f194409f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 10 monclient(hunting): picked mon.noname-b con 0x7f194409fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 10 monclient(hunting): picked mon.noname-h con 0x7f194409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1944142640 con 0x7f194409fba0 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194409d310 con 0x7f194409ff70 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19440a2560 con 0x7f194409f7d0 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f194b554640 10 monclient(hunting): authenticate will time out at 1070.824725s 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f1948aef640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f194409fba0 0x7f1944131950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49074/0 (socket says 172.21.15.45:49074) 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.486+0000 7f1948aef640 1 -- 172.21.15.45:0/1306800058 learned_addr learned my addr 172.21.15.45:0/1306800058 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400216364 0 0) 0x7f1944142640 con 0x7f194409fba0 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): my global_id is 144281 2024-09-18T14:04:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19280034e0 con 0x7f194409fba0 2024-09-18T14:04:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030847470 0 0) 0x7f19440a2560 con 0x7f194409f7d0 2024-09-18T14:04:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): my global_id is 4948 2024-09-18T14:04:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1944142640 con 0x7f194409f7d0 2024-09-18T14:04:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918119863 0 0) 0x7f194409d310 con 0x7f194409ff70 2024-09-18T14:04:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): my global_id is 4856 2024-09-18T14:04:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19440a2560 con 0x7f194409ff70 2024-09-18T14:04:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3159438133 0 0) 0x7f19280034e0 con 0x7f194409fba0 2024-09-18T14:04:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f194409d310 con 0x7f194409fba0 2024-09-18T14:04:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1934002bf0 con 0x7f194409fba0 2024-09-18T14:04:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:04:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): dump: 2024-09-18T14:04:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2325254633 0 0) 0x7f1944142640 con 0x7f194409f7d0 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19280034e0 con 0x7f194409f7d0 2024-09-18T14:04:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1940004b20 con 0x7f194409f7d0 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): dump: 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.506 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2169860128 0 0) 0x7f19440a2560 con 0x7f194409ff70 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1944142640 con 0x7f194409ff70 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f193c0044e0 con 0x7f194409ff70 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient(hunting): dump: 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.507 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.508 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3937304403 0 0) 0x7f194409d310 con 0x7f194409fba0 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 >> v1:172.21.15.84:6792/0 conn(0x7f194409f7d0 legacy=0x7f19441417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 >> v1:172.21.15.84:6791/0 conn(0x7f194409ff70 legacy=0x7f1944149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient: _finish_hunting 0 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 monclient: found mon.b 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1944120750 con 0x7f194409fba0 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient: _finish_auth 0 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f194b554640 5 monclient: authenticate success, global_id 144281 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f194b554640 10 monclient: _renew_subs 2024-09-18T14:04:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f194b554640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f194b554640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194411ffe0 con 0x7f194409fba0 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f194b554640 10 monclient: _renew_subs 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f194b554640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f194b554640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1944132060 con 0x7f194409fba0 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1934001760 con 0x7f194409fba0 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1934001c90 con 0x7f194409fba0 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f193b7fe640 4 set_mon_vals no callback set 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.487+0000 7f19397fa640 10 monclient: dump: 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.510 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.511 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.488+0000 7f194b554640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.488+0000 7f194b554640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1924006580 con 0x7f1924002830 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.488+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1934001df0 con 0x7f194409fba0 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(232..232 src has 1..232) v4 ==== 3971+0+0 (unknown 4008190398 0 0) 0x7f1934046c50 con 0x7f194409fba0 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f1949af1640 10 monclient: get_auth_request con 0x7f192803aa20 auth_method 0 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912969205 0 0) 0x7f1924006580 con 0x7f1924002830 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 10 monclient(hunting): my global_id is 4911 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f194409d310 con 0x7f1924002830 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1210878788 0 0) 0x7f194409d310 con 0x7f1924002830 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1924006580 con 0x7f1924002830 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1940001210 con 0x7f1924002830 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 10 monclient: dump: 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.514 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2181772009 0 0) 0x7f1924006580 con 0x7f1924002830 2024-09-18T14:04:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f192400a0a0 con 0x7f1924002830 2024-09-18T14:04:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1940005160 con 0x7f1924002830 2024-09-18T14:04:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:04:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 >> v1:172.21.15.45:6792/0 conn(0x7f1924002830 legacy=0x7f1924004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.489+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 >> v1:172.21.15.45:6792/0 conn(0x7f1924002830 legacy=0x7f1924004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.495+0000 7f194b554640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:04:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.495+0000 7f194b554640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1924009660 con 0x7f1924009040 2024-09-18T14:04:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.495+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910520936 0 0) 0x7f1924009660 con 0x7f1924009040 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.495+0000 7f19397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.495+0000 7f19397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.495+0000 7f19397fa640 10 monclient(hunting): my global_id is 4920 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.495+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1924006580 con 0x7f1924009040 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1488618963 0 0) 0x7f1924006580 con 0x7f1924009040 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1924009660 con 0x7f1924009040 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f19400059c0 con 0x7f1924009040 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 10 monclient: dump: 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.517 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1235406662 0 0) 0x7f1924009660 con 0x7f1924009040 2024-09-18T14:04:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1924009920 con 0x7f1924009040 2024-09-18T14:04:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7364 (unknown 0 0 889327342) 0x7f1940008150 con 0x7f1924009040 2024-09-18T14:04:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:04:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:04:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 >> v1:172.21.15.45:6792/0 conn(0x7f1924009040 legacy=0x7f1924008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f19397fa640 1 -- 172.21.15.45:0/1306800058 >> v1:172.21.15.45:6792/0 conn(0x7f1924009040 legacy=0x7f1924008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:15.518 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:04:15.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:04:15.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:04:15.519 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:04:15.519 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 70, 2024-09-18T14:04:15.519 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:04:15.519 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:04:15.519 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:04:15.519 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:04:15.519 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:04:15.519 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:04:15.519 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:04:15.519 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:04:15.519 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.519 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:04:15.520 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:15.520 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:04:15.520 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:04:15.520 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.520 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.520 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.520 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.520 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.520 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.520 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.520 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.520 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.521 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.522 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:04:15.522 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:04:15.522 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:04:15.522 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:04:15.522 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:04:15.522 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:04:15.522 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:15.522 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:15.522 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:04:15.522 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.523 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.525 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:04:15.525 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.525 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.525 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.525 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:04:15.525 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:04:15.525 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.525 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.525 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.526 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:04:15.527 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.528 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:04:15.530 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:04:15.531 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:04:15.532 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:04:15.535 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:04:15.535 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.535 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.535 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:04:15.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:04:15.535 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.535 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:15.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.538 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.538 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:04:15.538 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:04:15.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f194b554640 1 -- 172.21.15.45:0/1306800058 >> v1:172.21.15.84:6812/31478 conn(0x7f192803aa20 legacy=0x7f192803ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f194b554640 10 monclient: shutdown 2024-09-18T14:04:15.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f194b554640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:15.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f194b554640 1 -- 172.21.15.45:0/1306800058 >> v1:172.21.15.45:6790/0 conn(0x7f194409fba0 legacy=0x7f1944131950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f1949af1640 1 -- 172.21.15.45:0/1306800058 reap_dead start 2024-09-18T14:04:15.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f1949af1640 1 -- 172.21.15.45:0/1306800058 reap_dead start 2024-09-18T14:04:15.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.496+0000 7f194b554640 1 -- 172.21.15.45:0/1306800058 shutdown_connections 2024-09-18T14:04:15.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.497+0000 7f194b554640 1 -- 172.21.15.45:0/1306800058 >> 172.21.15.45:0/1306800058 conn(0x7f19440630b0 msgr2=0x7f19440a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:15.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.497+0000 7f194b554640 1 -- 172.21.15.45:0/1306800058 shutdown_connections 2024-09-18T14:04:15.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.497+0000 7f194b554640 1 -- 172.21.15.45:0/1306800058 wait complete. 2024-09-18T14:04:15.540 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:04:15.540 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-09-18T14:04:15.540 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:04:15.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.637+0000 7fbf26154640 10 monclient: get_monmap_and_config 2024-09-18T14:04:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.637+0000 7fbf26154640 10 monclient: build_initial_monmap 2024-09-18T14:04:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.637+0000 7fbf26154640 10 monclient: monmap: 2024-09-18T14:04:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:04:15.638207+0000 2024-09-18T14:04:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:04:15.638207+0000 2024-09-18T14:04:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:15.639 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:15.639 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:15.639 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:15.639 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:15.639 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:15.639 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:15.639 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.637+0000 7fbf26154640 1 Processor -- start 2024-09-18T14:04:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf26154640 1 -- start start 2024-09-18T14:04:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf26154640 10 monclient: init 2024-09-18T14:04:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf26154640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf26154640 10 monclient: _add_conns ranks=[0,4,2,8,7,3,6,5,1] 2024-09-18T14:04:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf26154640 10 monclient(hunting): picked mon.noname-a con 0x7fbf2009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:04:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf26154640 10 monclient(hunting): picked mon.noname-c con 0x7fbf2009fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:04:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf26154640 10 monclient(hunting): picked mon.noname-b con 0x7fbf2009ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:04:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf26154640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2005ed70 con 0x7fbf2009f7d0 2024-09-18T14:04:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf26154640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2009dfc0 con 0x7fbf2009ff70 2024-09-18T14:04:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf26154640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2009dce0 con 0x7fbf2009fba0 2024-09-18T14:04:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf26154640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf26154640 10 monclient(hunting): authenticate will time out at 1070.976546s 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1effd640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7fbf2009fba0 legacy=0x7fbf20137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1f7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbf2009f7d0 0x7fbf20132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54810/0 (socket says 172.21.15.45:54810) 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1ffff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fbf2009ff70 0x7fbf2013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49084/0 (socket says 172.21.15.45:49084) 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1f7fe640 1 -- 172.21.15.45:0/2072024705 learned_addr learned my addr 172.21.15.45:0/2072024705 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1e7fc640 1 -- 172.21.15.45:0/2072024705 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973127885 0 0) 0x7fbf2009dfc0 con 0x7fbf2009ff70 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1e7fc640 10 monclient(hunting): my global_id is 144290 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1e7fc640 1 -- 172.21.15.45:0/2072024705 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf080034e0 con 0x7fbf2009ff70 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1e7fc640 1 -- 172.21.15.45:0/2072024705 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650027072 0 0) 0x7fbf2005ed70 con 0x7fbf2009f7d0 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1e7fc640 10 monclient(hunting): my global_id is 5031 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.638+0000 7fbf1e7fc640 1 -- 172.21.15.45:0/2072024705 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf2009dfc0 con 0x7fbf2009f7d0 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 1 -- 172.21.15.45:0/2072024705 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3589308465 0 0) 0x7fbf080034e0 con 0x7fbf2009ff70 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 1 -- 172.21.15.45:0/2072024705 >> v1:172.21.15.45:6791/0 conn(0x7fbf2009fba0 legacy=0x7fbf20137a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 1 -- 172.21.15.45:0/2072024705 >> v1:172.21.15.45:6789/0 conn(0x7fbf2009f7d0 legacy=0x7fbf20132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 1 monclient: found mon.noname-b 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 1 -- 172.21.15.45:0/2072024705 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf200a1c00 con 0x7fbf2009ff70 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 1 -- 172.21.15.45:0/2072024705 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbf18001ae0 con 0x7fbf2009ff70 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 10 monclient: dump: 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf26154640 5 monclient: authenticate success, global_id 144290 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf26154640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 1 -- 172.21.15.45:0/2072024705 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbf18001c40 con 0x7fbf2009ff70 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 1 -- 172.21.15.45:0/2072024705 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbf18004b20 con 0x7fbf2009ff70 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf26154640 10 monclient: get_monmap_and_config success 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf26154640 4 set_mon_vals no callback set 2024-09-18T14:04:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:04:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf1e7fc640 10 monclient: dump: 2024-09-18T14:04:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf26154640 10 monclient: shutdown 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf26154640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf26154640 1 -- 172.21.15.45:0/2072024705 >> v1:172.21.15.45:6790/0 conn(0x7fbf2009ff70 legacy=0x7fbf2013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf26154640 1 -- 172.21.15.45:0/2072024705 shutdown_connections 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf26154640 1 -- 172.21.15.45:0/2072024705 >> 172.21.15.45:0/2072024705 conn(0x7fbf200630b0 msgr2=0x7fbf200634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf26154640 1 -- 172.21.15.45:0/2072024705 shutdown_connections 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.639+0000 7fbf26154640 1 -- 172.21.15.45:0/2072024705 wait complete. 2024-09-18T14:04:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.640+0000 7fbf26154640 10 monclient: build_initial_monmap 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.640+0000 7fbf26154640 10 monclient: monmap: 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:15.657 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:15.657 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:15.657 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.640+0000 7fbf26154640 1 Processor -- start 2024-09-18T14:04:15.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.640+0000 7fbf26154640 1 -- start start 2024-09-18T14:04:15.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.640+0000 7fbf26154640 10 monclient: init 2024-09-18T14:04:15.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf26154640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:15.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf26154640 10 monclient: _add_conns ranks=[2,7,1,6,5,0,8,4,3] 2024-09-18T14:04:15.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf26154640 10 monclient(hunting): picked mon.noname-b con 0x7fbf2009f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:04:15.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf26154640 10 monclient(hunting): picked mon.noname-i con 0x7fbf2009fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:04:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf26154640 10 monclient(hunting): picked mon.noname-f con 0x7fbf2009ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:04:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf26154640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2009d310 con 0x7fbf2009ff70 2024-09-18T14:04:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf26154640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf200a2580 con 0x7fbf2009f7d0 2024-09-18T14:04:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf26154640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf20120580 con 0x7fbf2009fba0 2024-09-18T14:04:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf1f7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fbf2009f7d0 0x7fbf2011d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49088/0 (socket says 172.21.15.45:49088) 2024-09-18T14:04:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf26154640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf1f7fe640 1 -- 172.21.15.45:0/2660988154 learned_addr learned my addr 172.21.15.45:0/2660988154 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf26154640 10 monclient(hunting): authenticate will time out at 1070.979509s 2024-09-18T14:04:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942417901 0 0) 0x7fbf200a2580 con 0x7fbf2009f7d0 2024-09-18T14:04:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf077fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf077fe640 10 monclient(hunting): my global_id is 144299 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf0c0034e0 con 0x7fbf2009f7d0 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 554849761 0 0) 0x7fbf0c0034e0 con 0x7fbf2009f7d0 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf200a2580 con 0x7fbf2009f7d0 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbf10001bb0 con 0x7fbf2009f7d0 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf077fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf077fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf077fe640 10 monclient(hunting): dump: 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.641+0000 7fbf077fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480128665 0 0) 0x7fbf2009d310 con 0x7fbf2009ff70 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient(hunting): my global_id is 4951 2024-09-18T14:04:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf0c0034e0 con 0x7fbf2009ff70 2024-09-18T14:04:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949736367 0 0) 0x7fbf20120580 con 0x7fbf2009fba0 2024-09-18T14:04:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient(hunting): my global_id is 4957 2024-09-18T14:04:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf2009d310 con 0x7fbf2009fba0 2024-09-18T14:04:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1919125081 0 0) 0x7fbf200a2580 con 0x7fbf2009f7d0 2024-09-18T14:04:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 >> v1:172.21.15.84:6792/0 conn(0x7fbf2009fba0 legacy=0x7fbf2011d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 >> v1:172.21.15.84:6789/0 conn(0x7fbf2009ff70 legacy=0x7fbf20149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient: _finish_hunting 0 2024-09-18T14:04:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 1 monclient: found mon.b 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf20121780 con 0x7fbf2009f7d0 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient: _finish_auth 0 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf26154640 5 monclient: authenticate success, global_id 144299 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf26154640 10 monclient: _renew_subs 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf26154640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf26154640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf20120af0 con 0x7fbf2009f7d0 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf26154640 10 monclient: _renew_subs 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf26154640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf26154640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2011e9a0 con 0x7fbf2009f7d0 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbf10004a50 con 0x7fbf2009f7d0 2024-09-18T14:04:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbf10004f00 con 0x7fbf2009f7d0 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf1dffb640 4 set_mon_vals no callback set 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.642+0000 7fbf077fe640 10 monclient: dump: 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf26154640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fbf1000ff50 con 0x7fbf2009f7d0 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf26154640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbf00006580 con 0x7fbf00002830 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(232..232 src has 1..232) v4 ==== 3971+0+0 (unknown 4008190398 0 0) 0x7fbf10046be0 con 0x7fbf2009f7d0 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052999058 0 0) 0x7fbf00006580 con 0x7fbf00002830 2024-09-18T14:04:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 10 monclient(hunting): my global_id is 4976 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf200a2580 con 0x7fbf00002830 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf1ffff640 10 monclient: get_auth_request con 0x7fbf0c039eb0 auth_method 0 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 688491165 0 0) 0x7fbf200a2580 con 0x7fbf00002830 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf00006580 con 0x7fbf00002830 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbf14004200 con 0x7fbf00002830 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 10 monclient: dump: 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3848032364 0 0) 0x7fbf00006580 con 0x7fbf00002830 2024-09-18T14:04:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.643+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf0000a0a0 con 0x7fbf00002830 2024-09-18T14:04:15.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.644+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fbf140064a0 con 0x7fbf00002830 2024-09-18T14:04:15.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.644+0000 7fbf077fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:15.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.644+0000 7fbf077fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:04:15.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.644+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 >> v1:172.21.15.45:6793/0 conn(0x7fbf00002830 legacy=0x7fbf00004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.644+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 >> v1:172.21.15.45:6793/0 conn(0x7fbf00002830 legacy=0x7fbf00004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:15.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.649+0000 7fbf26154640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:04:15.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.649+0000 7fbf26154640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbf00007540 con 0x7fbf000068f0 2024-09-18T14:04:15.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488010736 0 0) 0x7fbf00007540 con 0x7fbf000068f0 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 10 monclient(hunting): my global_id is 4985 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf00006580 con 0x7fbf000068f0 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1518824923 0 0) 0x7fbf00006580 con 0x7fbf000068f0 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf00007540 con 0x7fbf000068f0 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbf14004b90 con 0x7fbf000068f0 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 10 monclient: dump: 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 576187563 0 0) 0x7fbf00007540 con 0x7fbf000068f0 2024-09-18T14:04:15.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbf000087a0 con 0x7fbf000068f0 2024-09-18T14:04:15.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7536 (unknown 0 0 1938178228) 0x7fbf14006000 con 0x7fbf000068f0 2024-09-18T14:04:15.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:04:15.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:04:15.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 >> v1:172.21.15.45:6793/0 conn(0x7fbf000068f0 legacy=0x7fbf00006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.650+0000 7fbf077fe640 1 -- 172.21.15.45:0/2660988154 >> v1:172.21.15.45:6793/0 conn(0x7fbf000068f0 legacy=0x7fbf00006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:15.671 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:04:15.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:04:15.671 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 70, 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:15.672 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:04:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:04:15.674 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:15.675 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:04:15.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:04:15.677 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:04:15.678 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.678 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.678 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.678 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:04:15.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:04:15.678 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:04:15.680 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:04:15.681 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:04:15.683 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.683 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:04:15.683 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:04:15.683 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.683 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:04:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:04:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:04:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.687 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:04:15.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:04:15.687 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.687 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.687 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:04:15.687 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.687 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.687 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.688 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:04:15.688 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:04:15.688 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.688 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.688 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.688 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:04:15.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:04:15.689 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.689 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.689 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:04:15.689 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.689 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.689 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:04:15.689 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:04:15.689 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.690 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.690 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.690 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.690 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.690 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:04:15.690 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:04:15.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.690 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:15.690 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.691 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:15.692 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:15.692 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:04:15.692 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.692 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.692 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:04:15.692 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:04:15.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.651+0000 7fbf26154640 1 -- 172.21.15.45:0/2660988154 >> v1:172.21.15.84:6812/31478 conn(0x7fbf0c039eb0 legacy=0x7fbf0c03c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.651+0000 7fbf26154640 10 monclient: shutdown 2024-09-18T14:04:15.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.651+0000 7fbf26154640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:15.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.651+0000 7fbf26154640 1 -- 172.21.15.45:0/2660988154 >> v1:172.21.15.45:6790/0 conn(0x7fbf2009f7d0 legacy=0x7fbf2011d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.651+0000 7fbf1ffff640 1 -- 172.21.15.45:0/2660988154 reap_dead start 2024-09-18T14:04:15.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.651+0000 7fbf1ffff640 1 -- 172.21.15.45:0/2660988154 reap_dead start 2024-09-18T14:04:15.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.651+0000 7fbf26154640 1 -- 172.21.15.45:0/2660988154 shutdown_connections 2024-09-18T14:04:15.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.651+0000 7fbf26154640 1 -- 172.21.15.45:0/2660988154 >> 172.21.15.45:0/2660988154 conn(0x7fbf200630b0 msgr2=0x7fbf200a4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:15.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.651+0000 7fbf26154640 1 -- 172.21.15.45:0/2660988154 shutdown_connections 2024-09-18T14:04:15.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.651+0000 7fbf26154640 1 -- 172.21.15.45:0/2660988154 wait complete. 2024-09-18T14:04:15.694 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:04:15.694 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-09-18T14:04:15.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:04:15.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.792+0000 7fd65c75d640 10 monclient: get_monmap_and_config 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.792+0000 7fd65c75d640 10 monclient: build_initial_monmap 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.792+0000 7fd65c75d640 10 monclient: monmap: 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:04:15.793308+0000 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:04:15.793308+0000 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:15.794 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:15.794 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:15.794 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:15.794 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.792+0000 7fd65c75d640 1 Processor -- start 2024-09-18T14:04:15.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.792+0000 7fd65c75d640 1 -- start start 2024-09-18T14:04:15.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65c75d640 10 monclient: init 2024-09-18T14:04:15.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65c75d640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:15.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65c75d640 10 monclient: _add_conns ranks=[0,7,1,6,5,8,3,2,4] 2024-09-18T14:04:15.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65c75d640 10 monclient(hunting): picked mon.noname-a con 0x7fd65409f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65c75d640 10 monclient(hunting): picked mon.noname-i con 0x7fd65409fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65c75d640 10 monclient(hunting): picked mon.noname-f con 0x7fd65409ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65c75d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65405ed70 con 0x7fd65409f7d0 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65c75d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65409dfc0 con 0x7fd65409ff70 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65c75d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65409dce0 con 0x7fd65409fba0 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65c75d640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65c75d640 10 monclient(hunting): authenticate will time out at 1071.131646s 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65a4f9640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd65409f7d0 0x7fd654132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54822/0 (socket says 172.21.15.45:54822) 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd65a4f9640 1 -- 172.21.15.45:0/2431499360 learned_addr learned my addr 172.21.15.45:0/2431499360 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd6594f7640 1 -- 172.21.15.45:0/2431499360 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138332454 0 0) 0x7fd65405ed70 con 0x7fd65409f7d0 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd6594f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd6594f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.793+0000 7fd6594f7640 10 monclient(hunting): my global_id is 5040 2024-09-18T14:04:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 1 -- 172.21.15.45:0/2431499360 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6400034e0 con 0x7fd65409f7d0 2024-09-18T14:04:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 1 -- 172.21.15.45:0/2431499360 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866474910 0 0) 0x7fd65409dce0 con 0x7fd65409fba0 2024-09-18T14:04:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient(hunting): my global_id is 4966 2024-09-18T14:04:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 1 -- 172.21.15.45:0/2431499360 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd65405ed70 con 0x7fd65409fba0 2024-09-18T14:04:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 1 -- 172.21.15.45:0/2431499360 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2728870198 0 0) 0x7fd6400034e0 con 0x7fd65409f7d0 2024-09-18T14:04:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 1 -- 172.21.15.45:0/2431499360 >> v1:172.21.15.84:6792/0 conn(0x7fd65409fba0 legacy=0x7fd654137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 1 -- 172.21.15.45:0/2431499360 >> v1:172.21.15.84:6789/0 conn(0x7fd65409ff70 legacy=0x7fd65413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient: _finish_hunting 0 2024-09-18T14:04:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 1 monclient: found mon.noname-a 2024-09-18T14:04:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 1 -- 172.21.15.45:0/2431499360 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6540a1c00 con 0x7fd65409f7d0 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient: _finish_auth 0 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 1 -- 172.21.15.45:0/2431499360 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd650001830 con 0x7fd65409f7d0 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd65c75d640 5 monclient: authenticate success, global_id 5040 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd65c75d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient: dump: 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd65c75d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:04:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 1 -- 172.21.15.45:0/2431499360 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd650001990 con 0x7fd65409f7d0 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 1 -- 172.21.15.45:0/2431499360 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd650004ba0 con 0x7fd65409f7d0 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd65c75d640 10 monclient: get_monmap_and_config success 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd65c75d640 4 set_mon_vals no callback set 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd6594f7640 10 monclient: dump: 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.808 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.808 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.808 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.808 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.808 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.808 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.808 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.808 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.808 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd65c75d640 10 monclient: shutdown 2024-09-18T14:04:15.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd65c75d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:15.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd65c75d640 1 -- 172.21.15.45:0/2431499360 >> v1:172.21.15.45:6789/0 conn(0x7fd65409f7d0 legacy=0x7fd654132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd65c75d640 1 -- 172.21.15.45:0/2431499360 shutdown_connections 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.794+0000 7fd65c75d640 1 -- 172.21.15.45:0/2431499360 >> 172.21.15.45:0/2431499360 conn(0x7fd6540630b0 msgr2=0x7fd6540634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.795+0000 7fd65c75d640 1 -- 172.21.15.45:0/2431499360 shutdown_connections 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.795+0000 7fd65c75d640 1 -- 172.21.15.45:0/2431499360 wait complete. 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.795+0000 7fd65c75d640 10 monclient: build_initial_monmap 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.795+0000 7fd65c75d640 10 monclient: monmap: 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:04:15.809 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.795+0000 7fd65c75d640 1 Processor -- start 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.795+0000 7fd65c75d640 1 -- start start 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.795+0000 7fd65c75d640 10 monclient: init 2024-09-18T14:04:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd65c75d640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd65c75d640 10 monclient: _add_conns ranks=[0,6,5,4,8,1,2,3,7] 2024-09-18T14:04:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd65c75d640 10 monclient(hunting): picked mon.noname-a con 0x7fd65409f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:04:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd65c75d640 10 monclient(hunting): picked mon.noname-d con 0x7fd65409fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:04:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd65c75d640 10 monclient(hunting): picked mon.noname-h con 0x7fd65409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:04:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd65c75d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd654142750 con 0x7fd65409f7d0 2024-09-18T14:04:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd65c75d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65409d380 con 0x7fd65409ff70 2024-09-18T14:04:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd65c75d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd654121900 con 0x7fd65409fba0 2024-09-18T14:04:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd65c75d640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd65c75d640 10 monclient(hunting): authenticate will time out at 1071.134281s 2024-09-18T14:04:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd65a4f9640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd65409f7d0 0x7fd6541317b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54832/0 (socket says 172.21.15.45:54832) 2024-09-18T14:04:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd65a4f9640 1 -- 172.21.15.45:0/2138329484 learned_addr learned my addr 172.21.15.45:0/2138329484 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789599778 0 0) 0x7fd6540a1c00 con 0x7fd65409f7d0 2024-09-18T14:04:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): my global_id is 5049 2024-09-18T14:04:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6380034e0 con 0x7fd65409f7d0 2024-09-18T14:04:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034837605 0 0) 0x7fd654121900 con 0x7fd65409fba0 2024-09-18T14:04:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): my global_id is 4929 2024-09-18T14:04:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6540a1c00 con 0x7fd65409fba0 2024-09-18T14:04:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196306949 0 0) 0x7fd64c0013f0 con 0x7fd65409ff70 2024-09-18T14:04:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): my global_id is 4865 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd654121900 con 0x7fd65409ff70 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1237898052 0 0) 0x7fd6380034e0 con 0x7fd65409f7d0 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd64c0013f0 con 0x7fd65409f7d0 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 59293847 0 0) 0x7fd6540a1c00 con 0x7fd65409fba0 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6380034e0 con 0x7fd65409fba0 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd650001830 con 0x7fd65409f7d0 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): dump: 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.814 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd644001c70 con 0x7fd65409fba0 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 10 monclient(hunting): dump: 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1325649942 0 0) 0x7fd654121900 con 0x7fd65409ff70 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.796+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6540a1c00 con 0x7fd65409ff70 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd64c001db0 con 0x7fd65409ff70 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 10 monclient(hunting): dump: 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4045386833 0 0) 0x7fd64c0013f0 con 0x7fd65409f7d0 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 >> v1:172.21.15.45:6792/0 conn(0x7fd65409fba0 legacy=0x7fd654146ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 >> v1:172.21.15.84:6791/0 conn(0x7fd65409ff70 legacy=0x7fd65414a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 1 monclient: found mon.a 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6540a2560 con 0x7fd65409f7d0 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:04:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd65c75d640 5 monclient: authenticate success, global_id 5049 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd65c75d640 10 monclient: _renew_subs 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd65c75d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd65c75d640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd654121be0 con 0x7fd65409f7d0 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd65c75d640 10 monclient: _renew_subs 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd65c75d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd65c75d640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd654132060 con 0x7fd65409f7d0 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd650001210 con 0x7fd65409f7d0 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd650004bd0 con 0x7fd65409f7d0 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64bfff640 4 set_mon_vals no callback set 2024-09-18T14:04:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd64a7fc640 10 monclient: dump: 2024-09-18T14:04:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd65c75d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.797+0000 7fd65c75d640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd63c006580 con 0x7fd63c002830 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd650010010 con 0x7fd65409f7d0 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(232..232 src has 1..232) v4 ==== 3971+0+0 (unknown 4008190398 0 0) 0x7fd650046a10 con 0x7fd65409f7d0 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252523501 0 0) 0x7fd63c006580 con 0x7fd63c002830 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 10 monclient(hunting): my global_id is 4969 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd64c0013f0 con 0x7fd63c002830 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd65acfa640 10 monclient: get_auth_request con 0x7fd63803a9b0 auth_method 0 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1517906551 0 0) 0x7fd64c0013f0 con 0x7fd63c002830 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd63c006580 con 0x7fd63c002830 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd644002770 con 0x7fd63c002830 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 10 monclient: dump: 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.823 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2908872285 0 0) 0x7fd63c006580 con 0x7fd63c002830 2024-09-18T14:04:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.798+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd63c00a0a0 con 0x7fd63c002830 2024-09-18T14:04:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.799+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd644002930 con 0x7fd63c002830 2024-09-18T14:04:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.799+0000 7fd64a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.799+0000 7fd64a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:04:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.799+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 >> v1:172.21.15.84:6789/0 conn(0x7fd63c002830 legacy=0x7fd63c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.799+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 >> v1:172.21.15.84:6789/0 conn(0x7fd63c002830 legacy=0x7fd63c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.804+0000 7fd65c75d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.804+0000 7fd65c75d640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd63c009640 con 0x7fd63c009020 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3770896958 0 0) 0x7fd63c009640 con 0x7fd63c009020 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 10 monclient(hunting): my global_id is 4978 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd63c006580 con 0x7fd63c009020 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1632471043 0 0) 0x7fd63c006580 con 0x7fd63c009020 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd63c009640 con 0x7fd63c009020 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd644003340 con 0x7fd63c009020 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 10 monclient: dump: 2024-09-18T14:04:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.828 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 58711819 0 0) 0x7fd63c009640 con 0x7fd63c009020 2024-09-18T14:04:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.805+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd63c009900 con 0x7fd63c009020 2024-09-18T14:04:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7534 (unknown 0 0 3022824541) 0x7fd6440036d0 con 0x7fd63c009020 2024-09-18T14:04:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd64a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:04:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd64a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:04:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 >> v1:172.21.15.84:6789/0 conn(0x7fd63c009020 legacy=0x7fd63c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd64a7fc640 1 -- 172.21.15.45:0/2138329484 >> v1:172.21.15.84:6789/0 conn(0x7fd63c009020 legacy=0x7fd63c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:15.828 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:04:15.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:04:15.828 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 70, 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:04:15.829 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.830 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:04:15.831 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:04:15.832 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:04:15.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:04:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:04:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:04:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:04:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.835 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:04:15.835 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:04:15.835 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.835 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.836 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.836 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:04:15.836 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:04:15.836 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.836 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.836 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:04:15.836 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.839 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:04:15.840 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:04:15.841 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.844 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:04:15.845 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.846 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.847 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.848 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:04:15.849 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:04:15.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd65c75d640 1 -- 172.21.15.45:0/2138329484 >> v1:172.21.15.84:6812/31478 conn(0x7fd63803a9b0 legacy=0x7fd63803ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd65c75d640 10 monclient: shutdown 2024-09-18T14:04:15.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd65c75d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:15.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd65c75d640 1 -- 172.21.15.45:0/2138329484 >> v1:172.21.15.45:6789/0 conn(0x7fd65409f7d0 legacy=0x7fd6541317b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd65acfa640 1 -- 172.21.15.45:0/2138329484 reap_dead start 2024-09-18T14:04:15.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd65acfa640 1 -- 172.21.15.45:0/2138329484 reap_dead start 2024-09-18T14:04:15.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd65c75d640 1 -- 172.21.15.45:0/2138329484 shutdown_connections 2024-09-18T14:04:15.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd65c75d640 1 -- 172.21.15.45:0/2138329484 >> 172.21.15.45:0/2138329484 conn(0x7fd6540630b0 msgr2=0x7fd6540a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:15.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd65c75d640 1 -- 172.21.15.45:0/2138329484 shutdown_connections 2024-09-18T14:04:15.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.806+0000 7fd65c75d640 1 -- 172.21.15.45:0/2138329484 wait complete. 2024-09-18T14:04:15.851 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:04:15.851 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-18T14:04:15.851 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:04:15.852 INFO:tasks.rados.rados.0.smithi084.stdout:1006: done (5 left) 2024-09-18T14:04:15.852 INFO:tasks.rados.rados.0.smithi084.stdout:1007: rollback oid 3 current snap is 112 2024-09-18T14:04:15.853 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 3 to 109 2024-09-18T14:04:15.853 INFO:tasks.rados.rados.0.smithi084.stdout:1000: finishing write tid 2 to smithi08434663-29 2024-09-18T14:04:15.853 INFO:tasks.rados.rados.0.smithi084.stdout:1000: finishing write tid 3 to smithi08434663-29 2024-09-18T14:04:15.853 INFO:tasks.rados.rados.0.smithi084.stdout:1000: finishing write tid 4 to smithi08434663-29 2024-09-18T14:04:15.853 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 326 (ObjNum 361 snap 112 seq_num 361) dirty exists 2024-09-18T14:04:15.853 INFO:tasks.rados.rados.0.smithi084.stdout:1000: left oid 29 (ObjNum 361 snap 112 seq_num 361) 2024-09-18T14:04:15.853 INFO:tasks.rados.rados.0.smithi084.stdout:1002: expect (ObjNum 350 snap 104 seq_num 350) 2024-09-18T14:04:15.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.950+0000 7f311cedd640 10 monclient: get_monmap_and_config 2024-09-18T14:04:15.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.950+0000 7f311cedd640 10 monclient: build_initial_monmap 2024-09-18T14:04:15.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.950+0000 7f311cedd640 10 monclient: monmap: 2024-09-18T14:04:15.951 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:15.951 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.951 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:04:15.951632+0000 2024-09-18T14:04:15.951 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:04:15.951632+0000 2024-09-18T14:04:15.952 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:15.952 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.952 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:15.952 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:15.952 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:15.952 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:15.952 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:15.952 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:15.952 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:15.952 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:15.952 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:15.952 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 1 Processor -- start 2024-09-18T14:04:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 1 -- start start 2024-09-18T14:04:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 10 monclient: init 2024-09-18T14:04:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 10 monclient: _add_conns ranks=[2,7,3,6,1,8,0,4,5] 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 10 monclient(hunting): picked mon.noname-b con 0x7f311809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 10 monclient(hunting): picked mon.noname-i con 0x7f311809fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 10 monclient(hunting): picked mon.noname-g con 0x7f311809ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311805ed70 con 0x7f311809f7d0 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311809dfc0 con 0x7f311809ff70 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311809dce0 con 0x7f311809fba0 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.951+0000 7f311cedd640 10 monclient(hunting): authenticate will time out at 1071.289943s 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3116d9d640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7f311809ff70 legacy=0x7f311813b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f311659c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f311809f7d0 0x7f3118132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49096/0 (socket says 172.21.15.45:49096) 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f311659c640 1 -- 172.21.15.45:0/2028758986 learned_addr learned my addr 172.21.15.45:0/2028758986 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 1 -- 172.21.15.45:0/2028758986 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595392593 0 0) 0x7f311805ed70 con 0x7f311809f7d0 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient(hunting): my global_id is 144308 2024-09-18T14:04:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 1 -- 172.21.15.45:0/2028758986 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31040034e0 con 0x7f311809f7d0 2024-09-18T14:04:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 1 -- 172.21.15.45:0/2028758986 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905705773 0 0) 0x7f311809dce0 con 0x7f311809fba0 2024-09-18T14:04:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient(hunting): my global_id is 4975 2024-09-18T14:04:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 1 -- 172.21.15.45:0/2028758986 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f311805ed70 con 0x7f311809fba0 2024-09-18T14:04:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 1 -- 172.21.15.45:0/2028758986 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2333062785 0 0) 0x7f31040034e0 con 0x7f311809f7d0 2024-09-18T14:04:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 1 -- 172.21.15.45:0/2028758986 >> v1:172.21.15.84:6792/0 conn(0x7f311809fba0 legacy=0x7f3118137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 1 -- 172.21.15.45:0/2028758986 >> v1:172.21.15.84:6790/0 conn(0x7f311809ff70 legacy=0x7f311813b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:04:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:04:15.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 1 monclient: found mon.noname-b 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 1 -- 172.21.15.45:0/2028758986 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31180a1c00 con 0x7f311809f7d0 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient: _finish_auth 0 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 1 -- 172.21.15.45:0/2028758986 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3108001830 con 0x7f311809f7d0 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f311cedd640 5 monclient: authenticate success, global_id 144308 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f311cedd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient: dump: 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.964 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f311cedd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 1 -- 172.21.15.45:0/2028758986 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3108001a10 con 0x7f311809f7d0 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 1 -- 172.21.15.45:0/2028758986 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3108004ba0 con 0x7f311809f7d0 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f311cedd640 10 monclient: get_monmap_and_config success 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f311cedd640 4 set_mon_vals no callback set 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.952+0000 7f3115d9b640 10 monclient: dump: 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.966 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.967 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.953+0000 7f311cedd640 10 monclient: shutdown 2024-09-18T14:04:15.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.953+0000 7f311cedd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:15.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.953+0000 7f311cedd640 1 -- 172.21.15.45:0/2028758986 >> v1:172.21.15.45:6790/0 conn(0x7f311809f7d0 legacy=0x7f3118132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.953+0000 7f311cedd640 1 -- 172.21.15.45:0/2028758986 shutdown_connections 2024-09-18T14:04:15.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.953+0000 7f311cedd640 1 -- 172.21.15.45:0/2028758986 >> 172.21.15.45:0/2028758986 conn(0x7f31180630b0 msgr2=0x7f31180634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:15.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.953+0000 7f311cedd640 1 -- 172.21.15.45:0/2028758986 shutdown_connections 2024-09-18T14:04:15.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.953+0000 7f311cedd640 1 -- 172.21.15.45:0/2028758986 wait complete. 2024-09-18T14:04:15.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.953+0000 7f311cedd640 10 monclient: build_initial_monmap 2024-09-18T14:04:15.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.953+0000 7f311cedd640 10 monclient: monmap: 2024-09-18T14:04:15.968 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:15.968 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:04:15.968 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:04:15.969 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:04:15.969 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:15.969 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:04:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:15.973 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 1 Processor -- start 2024-09-18T14:04:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 1 -- start start 2024-09-18T14:04:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 10 monclient: init 2024-09-18T14:04:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 10 monclient: _add_conns ranks=[2,4,5,3,7,6,8,0,1] 2024-09-18T14:04:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 10 monclient(hunting): picked mon.noname-b con 0x7f311809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:04:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 10 monclient(hunting): picked mon.noname-c con 0x7f311809fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:04:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 10 monclient(hunting): picked mon.noname-h con 0x7f311809ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:04:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3118142640 con 0x7f311809f7d0 2024-09-18T14:04:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f310ffff640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f311809fba0 legacy=0x7f3118131950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:04:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311809d310 con 0x7f311809fba0 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31180a2560 con 0x7f311809ff70 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311cedd640 10 monclient(hunting): authenticate will time out at 1071.292683s 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311659c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f311809f7d0 0x7f31181417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:49100/0 (socket says 172.21.15.45:49100) 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f311659c640 1 -- 172.21.15.45:0/2159902426 learned_addr learned my addr 172.21.15.45:0/2159902426 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537993786 0 0) 0x7f3118142640 con 0x7f311809f7d0 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f310e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f310e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f310e7fc640 10 monclient(hunting): my global_id is 144317 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30f80034e0 con 0x7f311809f7d0 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.954+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355486907 0 0) 0x7f31180a2560 con 0x7f311809ff70 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient(hunting): my global_id is 4874 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3118142640 con 0x7f311809ff70 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1037646560 0 0) 0x7f30f80034e0 con 0x7f311809f7d0 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f31180a2560 con 0x7f311809f7d0 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3108004ae0 con 0x7f311809f7d0 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient(hunting): dump: 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:04:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 319348542 0 0) 0x7f3118142640 con 0x7f311809ff70 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f30f80034e0 con 0x7f311809ff70 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f31100045a0 con 0x7f311809ff70 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient(hunting): dump: 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1072853396 0 0) 0x7f31180a2560 con 0x7f311809f7d0 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 >> v1:172.21.15.84:6791/0 conn(0x7f311809ff70 legacy=0x7f3118149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 >> v1:172.21.15.45:6791/0 conn(0x7f311809fba0 legacy=0x7f3118131950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 monclient: found mon.b 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:15.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:04:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3118120750 con 0x7f311809f7d0 2024-09-18T14:04:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:04:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f311cedd640 5 monclient: authenticate success, global_id 144317 2024-09-18T14:04:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f311cedd640 10 monclient: _renew_subs 2024-09-18T14:04:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f311cedd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:04:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f311cedd640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311811ffe0 con 0x7f311809f7d0 2024-09-18T14:04:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f311cedd640 10 monclient: _renew_subs 2024-09-18T14:04:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f311cedd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:04:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f311cedd640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3118132060 con 0x7f311809f7d0 2024-09-18T14:04:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3108004c40 con 0x7f311809f7d0 2024-09-18T14:04:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f31080051a0 con 0x7f311809f7d0 2024-09-18T14:04:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f311559a640 4 set_mon_vals no callback set 2024-09-18T14:04:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:04:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.955+0000 7f310e7fc640 10 monclient: dump: 2024-09-18T14:04:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f311cedd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f3108010170 con 0x7f311809f7d0 2024-09-18T14:04:15.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f311cedd640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f30fc006580 con 0x7f30fc002830 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(233..233 src has 1..233) v4 ==== 3987+0+0 (unknown 1578135236 0 0) 0x7f3108046d40 con 0x7f311809f7d0 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f3116d9d640 10 monclient: get_auth_request con 0x7f30f803a3b0 auth_method 0 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145320656 0 0) 0x7f30fc006580 con 0x7f30fc002830 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 10 monclient(hunting): my global_id is 4883 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31180a2560 con 0x7f30fc002830 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1533560474 0 0) 0x7f31180a2560 con 0x7f30fc002830 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f30fc006580 con 0x7f30fc002830 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3100002cf0 con 0x7f30fc002830 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:04:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 10 monclient: dump: 2024-09-18T14:04:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.985 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3207394436 0 0) 0x7f30fc006580 con 0x7f30fc002830 2024-09-18T14:04:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.956+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30fc00a0a0 con 0x7f30fc002830 2024-09-18T14:04:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.957+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3100006450 con 0x7f30fc002830 2024-09-18T14:04:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.957+0000 7f310e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.957+0000 7f310e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:04:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.957+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 >> v1:172.21.15.84:6791/0 conn(0x7f30fc002830 legacy=0x7f30fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.957+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 >> v1:172.21.15.84:6791/0 conn(0x7f30fc002830 legacy=0x7f30fc004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f311cedd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:04:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f311cedd640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f30fc009660 con 0x7f30fc009040 2024-09-18T14:04:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685412040 0 0) 0x7f30fc009660 con 0x7f30fc009040 2024-09-18T14:04:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 10 monclient(hunting): my global_id is 4892 2024-09-18T14:04:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30fc006580 con 0x7f30fc009040 2024-09-18T14:04:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1952176629 0 0) 0x7f30fc006580 con 0x7f30fc009040 2024-09-18T14:04:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f30fc009660 con 0x7f30fc009040 2024-09-18T14:04:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3100004c90 con 0x7f30fc009040 2024-09-18T14:04:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:04:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 10 monclient: dump: 2024-09-18T14:04:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 489264278 0 0) 0x7f30fc009660 con 0x7f30fc009040 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.963+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f30fc009920 con 0x7f30fc009040 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7704 (unknown 0 0 3290375092) 0x7f3100006150 con 0x7f30fc009040 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f310e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f310e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:04:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 >> v1:172.21.15.84:6791/0 conn(0x7f30fc009040 legacy=0x7f30fc008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f310e7fc640 1 -- 172.21.15.45:0/2159902426 >> v1:172.21.15.84:6791/0 conn(0x7f30fc009040 legacy=0x7f30fc008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:15.990 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:04:15.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:04:15.990 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:04:15.990 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:04:15.990 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 70, 2024-09-18T14:04:15.990 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:04:15.990 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.991 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.992 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:04:15.993 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:04:15.994 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:04:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.995 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.996 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:04:15.998 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.998 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.998 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.998 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:04:15.998 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:04:15.998 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:15.998 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:15.998 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:15.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.998 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:04:15.999 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.000 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.000 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.000 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:04:16.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:04:16.000 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.001 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:04:16.001 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.001 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.001 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.001 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:04:16.001 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:04:16.001 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.001 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.001 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.002 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:04:16.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:04:16.002 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.002 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.002 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:04:16.002 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.002 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.004 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.005 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.006 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.007 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:04:16.007 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:04:16.007 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.007 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.007 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.007 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:04:16.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:04:16.007 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.007 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:04:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:04:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:04:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:04:16.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:04:16.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.011 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:16.011 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:16.011 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:16.011 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.011 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.011 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:04:16.011 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:04:16.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f311cedd640 1 -- 172.21.15.45:0/2159902426 >> v1:172.21.15.84:6812/31478 conn(0x7f30f803a3b0 legacy=0x7f30f803c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:16.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f311cedd640 10 monclient: shutdown 2024-09-18T14:04:16.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f311cedd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:16.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f311cedd640 1 -- 172.21.15.45:0/2159902426 >> v1:172.21.15.45:6790/0 conn(0x7f311809f7d0 legacy=0x7f31181417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:16.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f3116d9d640 1 -- 172.21.15.45:0/2159902426 reap_dead start 2024-09-18T14:04:16.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f3116d9d640 1 -- 172.21.15.45:0/2159902426 reap_dead start 2024-09-18T14:04:16.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f311cedd640 1 -- 172.21.15.45:0/2159902426 shutdown_connections 2024-09-18T14:04:16.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f311cedd640 1 -- 172.21.15.45:0/2159902426 >> 172.21.15.45:0/2159902426 conn(0x7f31180630b0 msgr2=0x7f31180a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:16.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f311cedd640 1 -- 172.21.15.45:0/2159902426 shutdown_connections 2024-09-18T14:04:16.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:15.964+0000 7f311cedd640 1 -- 172.21.15.45:0/2159902426 wait complete. 2024-09-18T14:04:16.014 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:04:16.014 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-09-18T14:04:16.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:04:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.111+0000 7f0a6c509640 10 monclient: get_monmap_and_config 2024-09-18T14:04:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.111+0000 7f0a6c509640 10 monclient: build_initial_monmap 2024-09-18T14:04:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.111+0000 7f0a6c509640 10 monclient: monmap: 2024-09-18T14:04:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:04:16.112366+0000 2024-09-18T14:04:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:04:16.112366+0000 2024-09-18T14:04:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:16.113 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:16.113 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:16.113 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:16.113 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:16.113 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:16.113 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.111+0000 7f0a6c509640 1 Processor -- start 2024-09-18T14:04:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6c509640 1 -- start start 2024-09-18T14:04:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6c509640 10 monclient: init 2024-09-18T14:04:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6c509640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6c509640 10 monclient: _add_conns ranks=[4,6,5,7,2,1,8,0,3] 2024-09-18T14:04:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6c509640 10 monclient(hunting): picked mon.noname-c con 0x7f0a6409f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:04:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6c509640 10 monclient(hunting): picked mon.noname-d con 0x7f0a6409fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:04:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6c509640 10 monclient(hunting): picked mon.noname-h con 0x7f0a6409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:04:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6c509640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6405ed70 con 0x7f0a6409f7d0 2024-09-18T14:04:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6c509640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6409dfc0 con 0x7f0a6409ff70 2024-09-18T14:04:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6c509640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6409dce0 con 0x7f0a6409fba0 2024-09-18T14:04:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6a2a5640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f0a6409f7d0 legacy=0x7f0a64132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:04:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6c509640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a6c509640 10 monclient(hunting): authenticate will time out at 1071.450744s 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a69aa4640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f0a6409fba0 0x7f0a64137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:38216/0 (socket says 172.21.15.45:38216) 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.112+0000 7f0a69aa4640 1 -- 172.21.15.45:0/770832120 learned_addr learned my addr 172.21.15.45:0/770832120 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 1 -- 172.21.15.45:0/770832120 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278694233 0 0) 0x7f0a6409dce0 con 0x7f0a6409fba0 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient(hunting): my global_id is 4938 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 1 -- 172.21.15.45:0/770832120 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a500034e0 con 0x7f0a6409fba0 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 1 -- 172.21.15.45:0/770832120 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124554839 0 0) 0x7f0a6409dfc0 con 0x7f0a6409ff70 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient(hunting): my global_id is 4901 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 1 -- 172.21.15.45:0/770832120 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6409dce0 con 0x7f0a6409ff70 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 1 -- 172.21.15.45:0/770832120 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1345429540 0 0) 0x7f0a500034e0 con 0x7f0a6409fba0 2024-09-18T14:04:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 1 -- 172.21.15.45:0/770832120 >> v1:172.21.15.84:6791/0 conn(0x7f0a6409ff70 legacy=0x7f0a6413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 1 -- 172.21.15.45:0/770832120 >> v1:172.21.15.45:6791/0 conn(0x7f0a6409f7d0 legacy=0x7f0a64132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient: _finish_hunting 0 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 1 monclient: found mon.noname-d 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 1 -- 172.21.15.45:0/770832120 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a640a1c00 con 0x7f0a6409fba0 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient: _finish_auth 0 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 1 -- 172.21.15.45:0/770832120 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0a54001830 con 0x7f0a6409fba0 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a6c509640 5 monclient: authenticate success, global_id 4938 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a6c509640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient: dump: 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a6c509640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 1 -- 172.21.15.45:0/770832120 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0a54001a10 con 0x7f0a6409fba0 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 1 -- 172.21.15.45:0/770832120 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0a54004ba0 con 0x7f0a6409fba0 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a6c509640 10 monclient: get_monmap_and_config success 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a6c509640 4 set_mon_vals no callback set 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:04:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a692a3640 10 monclient: dump: 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:16.127 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a6c509640 10 monclient: shutdown 2024-09-18T14:04:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a6c509640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.113+0000 7f0a6c509640 1 -- 172.21.15.45:0/770832120 >> v1:172.21.15.45:6792/0 conn(0x7f0a6409fba0 legacy=0x7f0a64137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.114+0000 7f0a6c509640 1 -- 172.21.15.45:0/770832120 shutdown_connections 2024-09-18T14:04:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.114+0000 7f0a6c509640 1 -- 172.21.15.45:0/770832120 >> 172.21.15.45:0/770832120 conn(0x7f0a640630b0 msgr2=0x7f0a640634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.114+0000 7f0a6c509640 1 -- 172.21.15.45:0/770832120 shutdown_connections 2024-09-18T14:04:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.114+0000 7f0a6c509640 1 -- 172.21.15.45:0/770832120 wait complete. 2024-09-18T14:04:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.114+0000 7f0a6c509640 10 monclient: build_initial_monmap 2024-09-18T14:04:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.114+0000 7f0a6c509640 10 monclient: monmap: 2024-09-18T14:04:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:04:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.114+0000 7f0a6c509640 1 Processor -- start 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6c509640 1 -- start start 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6c509640 10 monclient: init 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6c509640 10 monclient: _reopen_session rank -1 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6c509640 10 monclient: _add_conns ranks=[6,1,4,8,5,3,7,0,2] 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6c509640 10 monclient(hunting): picked mon.noname-d con 0x7f0a6409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6c509640 10 monclient(hunting): picked mon.noname-f con 0x7f0a6409fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6c509640 10 monclient(hunting): picked mon.noname-c con 0x7f0a6409ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6c509640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a640a2560 con 0x7f0a6409fba0 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6c509640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a64121f20 con 0x7f0a6409ff70 2024-09-18T14:04:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6aaa6640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f0a6409ff70 legacy=0x7f0a6414a040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6a2a5640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f0a6409f7d0 0x7f0a64131810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:38226/0 (socket says 172.21.15.45:38226) 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6c509640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a64121c40 con 0x7f0a6409f7d0 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6c509640 10 monclient(hunting): _renew_subs 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6a2a5640 1 -- 172.21.15.45:0/3367278687 learned_addr learned my addr 172.21.15.45:0/3367278687 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a6c509640 10 monclient(hunting): authenticate will time out at 1071.453564s 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930755966 0 0) 0x7f0a64121c40 con 0x7f0a6409f7d0 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 10 monclient(hunting): my global_id is 4947 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a480034e0 con 0x7f0a6409f7d0 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225189889 0 0) 0x7f0a640a2560 con 0x7f0a6409fba0 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 10 monclient(hunting): my global_id is 4987 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a64121c40 con 0x7f0a6409fba0 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1156336323 0 0) 0x7f0a480034e0 con 0x7f0a6409f7d0 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a640a2560 con 0x7f0a6409f7d0 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0a60004340 con 0x7f0a6409f7d0 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.115+0000 7f0a5a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 10 monclient(hunting): dump: 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2040485388 0 0) 0x7f0a64121c40 con 0x7f0a6409fba0 2024-09-18T14:04:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a480034e0 con 0x7f0a6409fba0 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3295023122 0 0) 0x7f0a640a2560 con 0x7f0a6409f7d0 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 >> v1:172.21.15.45:6791/0 conn(0x7f0a6409ff70 legacy=0x7f0a6414a040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 >> v1:172.21.15.84:6789/0 conn(0x7f0a6409fba0 legacy=0x7f0a64146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 1 monclient: found mon.d 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a64121960 con 0x7f0a6409f7d0 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a6c509640 5 monclient: authenticate success, global_id 4947 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a6c509640 10 monclient: _renew_subs 2024-09-18T14:04:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a6c509640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a6c509640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a64131f20 con 0x7f0a6409f7d0 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a6c509640 10 monclient: _renew_subs 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a6c509640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a6c509640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a64120ba0 con 0x7f0a6409f7d0 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0a60004520 con 0x7f0a6409f7d0 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0a60004a50 con 0x7f0a6409f7d0 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a68aa2640 4 set_mon_vals no callback set 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.116+0000 7f0a5a7fc640 10 monclient: dump: 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:16.135 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a6c509640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a6c509640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0a4c006580 con 0x7f0a4c002830 2024-09-18T14:04:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f0a60004c30 con 0x7f0a6409f7d0 2024-09-18T14:04:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(233..233 src has 1..233) v4 ==== 3987+0+0 (unknown 1578135236 0 0) 0x7f0a60046c30 con 0x7f0a6409f7d0 2024-09-18T14:04:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038552385 0 0) 0x7f0a4c006580 con 0x7f0a4c002830 2024-09-18T14:04:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 10 monclient(hunting): my global_id is 4984 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a640a2560 con 0x7f0a4c002830 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a6aaa6640 10 monclient: get_auth_request con 0x7f0a48039f00 auth_method 0 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3210890146 0 0) 0x7f0a640a2560 con 0x7f0a4c002830 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a4c006580 con 0x7f0a4c002830 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0a540016e0 con 0x7f0a4c002830 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 10 monclient: dump: 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:16.139 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3858741573 0 0) 0x7f0a4c006580 con 0x7f0a4c002830 2024-09-18T14:04:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.117+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a4c00a0a0 con 0x7f0a4c002830 2024-09-18T14:04:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.118+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0a54005160 con 0x7f0a4c002830 2024-09-18T14:04:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.118+0000 7f0a5a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:04:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.118+0000 7f0a5a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:04:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.118+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 >> v1:172.21.15.84:6792/0 conn(0x7f0a4c002830 legacy=0x7f0a4c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.118+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 >> v1:172.21.15.84:6792/0 conn(0x7f0a4c002830 legacy=0x7f0a4c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:16.140 INFO:tasks.rados.rados.0.smithi084.stdout:1003: finishing copy_from racing read to smithi08434663-15 2024-09-18T14:04:16.140 INFO:tasks.rados.rados.0.smithi084.stdout:1003: finishing copy_from to smithi08434663-15 2024-09-18T14:04:16.140 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 326 (ObjNum 341 snap 104 seq_num 341) dirty exists 2024-09-18T14:04:16.140 INFO:tasks.rados.rados.0.smithi084.stdout:1004: expect (ObjNum 343 snap 104 seq_num 343) 2024-09-18T14:04:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.123+0000 7f0a6c509640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:04:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.123+0000 7f0a6c509640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0a4c007520 con 0x7f0a4c0068d0 2024-09-18T14:04:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949551028 0 0) 0x7f0a4c007520 con 0x7f0a4c0068d0 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 10 monclient(hunting): my global_id is 4993 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a4c006580 con 0x7f0a4c0068d0 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3240515719 0 0) 0x7f0a4c006580 con 0x7f0a4c0068d0 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a4c007520 con 0x7f0a4c0068d0 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0a54005a60 con 0x7f0a4c0068d0 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 10 monclient: dump: 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:04:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:04:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3136098683 0 0) 0x7f0a4c007520 con 0x7f0a4c0068d0 2024-09-18T14:04:16.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0a4c0087a0 con 0x7f0a4c0068d0 2024-09-18T14:04:16.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7364 (unknown 0 0 1954847596) 0x7f0a54008150 con 0x7f0a4c0068d0 2024-09-18T14:04:16.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:04:16.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:04:16.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 >> v1:172.21.15.84:6792/0 conn(0x7f0a4c0068d0 legacy=0x7f0a4c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:16.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.124+0000 7f0a5a7fc640 1 -- 172.21.15.45:0/3367278687 >> v1:172.21.15.84:6792/0 conn(0x7f0a4c0068d0 legacy=0x7f0a4c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:04:16.144 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:04:16.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:04:16.144 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:04:16.144 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 70, 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:16.145 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:04:16.147 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:04:16.148 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.149 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:04:16.150 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.151 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.152 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:04:16.154 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:04:16.155 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.156 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:04:16.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:04:16.161 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:04:16.162 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:04:16.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:04:16.163 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:04:16.163 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:04:16.163 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:04:16.163 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:04:16.163 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:04:16.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:04:16.163 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:04:16.163 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:04:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.125+0000 7f0a6c509640 1 -- 172.21.15.45:0/3367278687 >> v1:172.21.15.84:6812/31478 conn(0x7f0a48039f00 legacy=0x7f0a4803c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.125+0000 7f0a6c509640 10 monclient: shutdown 2024-09-18T14:04:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.125+0000 7f0a6c509640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:04:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.125+0000 7f0a6c509640 1 -- 172.21.15.45:0/3367278687 >> v1:172.21.15.45:6792/0 conn(0x7f0a6409f7d0 legacy=0x7f0a64131810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:04:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.125+0000 7f0a6aaa6640 1 -- 172.21.15.45:0/3367278687 reap_dead start 2024-09-18T14:04:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.125+0000 7f0a6aaa6640 1 -- 172.21.15.45:0/3367278687 reap_dead start 2024-09-18T14:04:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.125+0000 7f0a6c509640 1 -- 172.21.15.45:0/3367278687 shutdown_connections 2024-09-18T14:04:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.125+0000 7f0a6c509640 1 -- 172.21.15.45:0/3367278687 >> 172.21.15.45:0/3367278687 conn(0x7f0a640630b0 msgr2=0x7f0a640a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:04:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.125+0000 7f0a6c509640 1 -- 172.21.15.45:0/3367278687 shutdown_connections 2024-09-18T14:04:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:04:16.125+0000 7f0a6c509640 1 -- 172.21.15.45:0/3367278687 wait complete. 2024-09-18T14:04:16.166 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:04:16.166 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-18T14:04:16.326 INFO:tasks.rados.rados.0.smithi084.stdout:1001: finishing copy_from racing read to smithi08434663-31 2024-09-18T14:04:16.326 INFO:tasks.rados.rados.0.smithi084.stdout:1001: finishing copy_from to smithi08434663-31 2024-09-18T14:04:16.326 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 167 (ObjNum 281 snap 71 seq_num 281) dirty exists 2024-09-18T14:04:16.326 INFO:tasks.rados.rados.0.smithi084.stdout:1007: finishing rollback tid 0 to smithi08434663-3 2024-09-18T14:04:16.326 INFO:tasks.rados.rados.0.smithi084.stdout:1007: finishing rollback tid 1 to smithi08434663-3 2024-09-18T14:04:16.326 INFO:tasks.rados.rados.0.smithi084.stdout:1007: finishing rollback tid 2 to smithi08434663-3 2024-09-18T14:04:16.326 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 332 (ObjNum 226 snap 53 seq_num 226) dirty exists 2024-09-18T14:04:16.326 INFO:tasks.rados.rados.0.smithi084.stdout:1000: done (5 left) 2024-09-18T14:04:16.326 INFO:tasks.rados.rados.0.smithi084.stdout:1001: done (4 left) 2024-09-18T14:04:16.326 INFO:tasks.rados.rados.0.smithi084.stdout:1002: done (3 left) 2024-09-18T14:04:16.327 INFO:tasks.rados.rados.0.smithi084.stdout:1003: done (2 left) 2024-09-18T14:04:16.327 INFO:tasks.rados.rados.0.smithi084.stdout:1004: done (1 left) 2024-09-18T14:04:16.327 INFO:tasks.rados.rados.0.smithi084.stdout:1007: done (0 left) 2024-09-18T14:04:16.327 INFO:tasks.rados.rados.0.smithi084.stdout:1008: delete oid 11 current snap is 112 2024-09-18T14:04:16.328 INFO:tasks.rados.rados.0.smithi084.stdout:1008: done (0 left) 2024-09-18T14:04:16.329 INFO:tasks.rados.rados.0.smithi084.stdout:1009: read oid 25 snap -1 2024-09-18T14:04:16.329 INFO:tasks.rados.rados.0.smithi084.stdout:1009: expect (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:04:16.329 INFO:tasks.rados.rados.0.smithi084.stdout:1010: read oid 10 snap -1 2024-09-18T14:04:16.329 INFO:tasks.rados.rados.0.smithi084.stdout:1010: expect (ObjNum 352 snap 107 seq_num 352) 2024-09-18T14:04:16.329 INFO:tasks.rados.rados.0.smithi084.stdout:1011: write (excl) oid 38 current snap is 112 2024-09-18T14:04:16.329 INFO:tasks.rados.rados.0.smithi084.stdout:1011: seq_num 363 ranges {500941=783978,1942140=158801} 2024-09-18T14:04:16.335 INFO:tasks.rados.rados.0.smithi084.stdout:1011: writing smithi08434663-38 from 500941 to 1284919 tid 1 2024-09-18T14:04:16.340 INFO:tasks.rados.rados.0.smithi084.stdout:1011: writing smithi08434663-38 from 1942140 to 2100941 tid 2 2024-09-18T14:04:16.340 INFO:tasks.rados.rados.0.smithi084.stdout:1012: read oid 1 snap -1 2024-09-18T14:04:16.341 INFO:tasks.rados.rados.0.smithi084.stdout:1012: expect (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:04:16.341 INFO:tasks.rados.rados.0.smithi084.stdout:1013: delete oid 21 current snap is 112 2024-09-18T14:04:16.341 INFO:tasks.rados.rados.0.smithi084.stdout:1009: expect (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:04:16.608 INFO:tasks.rados.rados.0.smithi084.stdout:1010: expect (ObjNum 352 snap 107 seq_num 352) 2024-09-18T14:04:16.816 INFO:tasks.rados.rados.0.smithi084.stdout:1011: finishing write tid 1 to smithi08434663-38 2024-09-18T14:04:16.816 INFO:tasks.rados.rados.0.smithi084.stdout:1011: finishing write tid 2 to smithi08434663-38 2024-09-18T14:04:16.816 INFO:tasks.rados.rados.0.smithi084.stdout:1009: done (4 left) 2024-09-18T14:04:16.817 INFO:tasks.rados.rados.0.smithi084.stdout:1010: done (3 left) 2024-09-18T14:04:16.817 INFO:tasks.rados.rados.0.smithi084.stdout:1013: done (2 left) 2024-09-18T14:04:16.817 INFO:tasks.rados.rados.0.smithi084.stdout:1014: snap_create 2024-09-18T14:04:16.817 INFO:tasks.rados.rados.0.smithi084.stdout:1011: finishing write tid 3 to smithi08434663-38 2024-09-18T14:04:16.817 INFO:tasks.rados.rados.0.smithi084.stdout:1012: expect (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:04:17.087 INFO:tasks.rados.rados.0.smithi084.stdout:1011: finishing write tid 4 to smithi08434663-38 2024-09-18T14:04:17.088 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 269 (ObjNum 363 snap 112 seq_num 363) dirty exists 2024-09-18T14:04:17.088 INFO:tasks.rados.rados.0.smithi084.stdout:1011: left oid 38 (ObjNum 363 snap 112 seq_num 363) 2024-09-18T14:04:17.088 INFO:tasks.rados.rados.0.smithi084.stdout:1011: done (2 left) 2024-09-18T14:04:17.088 INFO:tasks.rados.rados.0.smithi084.stdout:1012: done (1 left) 2024-09-18T14:04:17.088 INFO:tasks.rados.rados.0.smithi084.stdout:1014: done (0 left) 2024-09-18T14:04:17.088 INFO:tasks.rados.rados.0.smithi084.stdout:1015: read oid 23 snap -1 2024-09-18T14:04:17.088 INFO:tasks.rados.rados.0.smithi084.stdout:1015: expect (ObjNum 327 snap 101 seq_num 327) 2024-09-18T14:04:17.088 INFO:tasks.rados.rados.0.smithi084.stdout:1016: write oid 45 current snap is 113 2024-09-18T14:04:17.088 INFO:tasks.rados.rados.0.smithi084.stdout:1016: seq_num 365 ranges {534555=650829,1659104=475451} 2024-09-18T14:04:17.093 INFO:tasks.rados.rados.0.smithi084.stdout:1016: writing smithi08434663-45 from 534555 to 1185384 tid 1 2024-09-18T14:04:17.099 INFO:tasks.rados.rados.0.smithi084.stdout:1016: writing smithi08434663-45 from 1659104 to 2134555 tid 2 2024-09-18T14:04:17.099 INFO:tasks.rados.rados.0.smithi084.stdout:1017: snap_create 2024-09-18T14:04:17.099 INFO:tasks.rados.rados.0.smithi084.stdout:1016: finishing write tid 1 to smithi08434663-45 2024-09-18T14:04:17.100 INFO:tasks.rados.rados.0.smithi084.stdout:1015: expect (ObjNum 327 snap 101 seq_num 327) 2024-09-18T14:04:17.266 INFO:tasks.rados.rados.0.smithi084.stdout:1016: finishing write tid 2 to smithi08434663-45 2024-09-18T14:04:17.266 INFO:tasks.rados.rados.0.smithi084.stdout:1016: finishing write tid 3 to smithi08434663-45 2024-09-18T14:04:17.266 INFO:tasks.rados.rados.0.smithi084.stdout:1016: finishing write tid 4 to smithi08434663-45 2024-09-18T14:04:17.266 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 345 (ObjNum 365 snap 113 seq_num 365) dirty exists 2024-09-18T14:04:17.267 INFO:tasks.rados.rados.0.smithi084.stdout:1016: left oid 45 (ObjNum 365 snap 113 seq_num 365) 2024-09-18T14:04:17.267 INFO:tasks.rados.rados.0.smithi084.stdout:1015: done (2 left) 2024-09-18T14:04:17.267 INFO:tasks.rados.rados.0.smithi084.stdout:1016: done (1 left) 2024-09-18T14:04:17.267 INFO:tasks.rados.rados.0.smithi084.stdout:1017: done (0 left) 2024-09-18T14:04:17.267 INFO:tasks.rados.rados.0.smithi084.stdout:1018: write (excl) oid 25 current snap is 114 2024-09-18T14:04:17.267 INFO:tasks.rados.rados.0.smithi084.stdout:1018: seq_num 366 ranges {551362=726078,1717586=409779} 2024-09-18T14:04:17.272 INFO:tasks.rados.rados.0.smithi084.stdout:1018: writing smithi08434663-25 from 551362 to 1277440 tid 1 2024-09-18T14:04:17.277 INFO:tasks.rados.rados.0.smithi084.stdout:1018: writing smithi08434663-25 from 1717586 to 2127365 tid 2 2024-09-18T14:04:17.277 INFO:tasks.rados.rados.0.smithi084.stdout:1019: delete oid 23 current snap is 114 2024-09-18T14:04:17.278 INFO:tasks.rados.rados.0.smithi084.stdout:1018: finishing write tid 1 to smithi08434663-25 2024-09-18T14:04:17.281 INFO:tasks.rados.rados.0.smithi084.stdout:1018: finishing write tid 2 to smithi08434663-25 2024-09-18T14:04:17.282 INFO:tasks.rados.rados.0.smithi084.stdout:1018: finishing write tid 3 to smithi08434663-25 2024-09-18T14:04:17.282 INFO:tasks.rados.rados.0.smithi084.stdout:1018: finishing write tid 4 to smithi08434663-25 2024-09-18T14:04:17.282 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 228 (ObjNum 366 snap 114 seq_num 366) dirty exists 2024-09-18T14:04:17.282 INFO:tasks.rados.rados.0.smithi084.stdout:1018: left oid 25 (ObjNum 366 snap 114 seq_num 366) 2024-09-18T14:04:17.283 INFO:tasks.rados.rados.0.smithi084.stdout:1018: done (1 left) 2024-09-18T14:04:17.283 INFO:tasks.rados.rados.0.smithi084.stdout:1019: done (0 left) 2024-09-18T14:04:17.283 INFO:tasks.rados.rados.0.smithi084.stdout:1020: rollback oid 29 current snap is 114 2024-09-18T14:04:17.283 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 29 to 110 2024-09-18T14:04:17.283 INFO:tasks.rados.rados.0.smithi084.stdout:1021: read oid 41 snap 98 2024-09-18T14:04:17.283 INFO:tasks.rados.rados.0.smithi084.stdout:1021: expect (ObjNum 152 snap 31 seq_num 152) 2024-09-18T14:04:17.284 INFO:tasks.rados.rados.0.smithi084.stdout:1022: write oid 42 current snap is 114 2024-09-18T14:04:17.284 INFO:tasks.rados.rados.0.smithi084.stdout:1022: seq_num 368 ranges {584976=476576,1834550=350426} 2024-09-18T14:04:17.289 INFO:tasks.rados.rados.0.smithi084.stdout:1022: writing smithi08434663-42 from 584976 to 1061552 tid 1 2024-09-18T14:04:17.296 INFO:tasks.rados.rados.0.smithi084.stdout:1022: writing smithi08434663-42 from 1834550 to 2184976 tid 2 2024-09-18T14:04:17.296 INFO:tasks.rados.rados.0.smithi084.stdout:1023: read oid 13 snap -1 2024-09-18T14:04:17.296 INFO:tasks.rados.rados.0.smithi084.stdout:1023: expect (ObjNum 359 snap 111 seq_num 359) 2024-09-18T14:04:17.296 INFO:tasks.rados.rados.0.smithi084.stdout:1024: delete oid 35 current snap is 114 2024-09-18T14:04:17.296 INFO:tasks.rados.rados.0.smithi084.stdout:1020: finishing rollback tid 0 to smithi08434663-29 2024-09-18T14:04:17.296 INFO:tasks.rados.rados.0.smithi084.stdout:1020: finishing rollback tid 1 to smithi08434663-29 2024-09-18T14:04:17.296 INFO:tasks.rados.rados.0.smithi084.stdout:1020: finishing rollback tid 2 to smithi08434663-29 2024-09-18T14:04:17.296 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 334 (ObjNum 346 snap 104 seq_num 346) dirty exists 2024-09-18T14:04:17.296 INFO:tasks.rados.rados.0.smithi084.stdout:1021: expect (ObjNum 152 snap 31 seq_num 152) 2024-09-18T14:04:17.414 INFO:tasks.rados.rados.0.smithi084.stdout:1022: finishing write tid 1 to smithi08434663-42 2024-09-18T14:04:17.414 INFO:tasks.rados.rados.0.smithi084.stdout:1022: finishing write tid 2 to smithi08434663-42 2024-09-18T14:04:17.414 INFO:tasks.rados.rados.0.smithi084.stdout:1022: finishing write tid 3 to smithi08434663-42 2024-09-18T14:04:17.414 INFO:tasks.rados.rados.0.smithi084.stdout:1022: finishing write tid 4 to smithi08434663-42 2024-09-18T14:04:17.414 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 348 (ObjNum 368 snap 114 seq_num 368) dirty exists 2024-09-18T14:04:17.414 INFO:tasks.rados.rados.0.smithi084.stdout:1022: left oid 42 (ObjNum 368 snap 114 seq_num 368) 2024-09-18T14:04:17.414 INFO:tasks.rados.rados.0.smithi084.stdout:1023: expect (ObjNum 359 snap 111 seq_num 359) 2024-09-18T14:04:17.476 INFO:tasks.rados.rados.0.smithi084.stdout:1020: done (4 left) 2024-09-18T14:04:17.476 INFO:tasks.rados.rados.0.smithi084.stdout:1021: done (3 left) 2024-09-18T14:04:17.476 INFO:tasks.rados.rados.0.smithi084.stdout:1022: done (2 left) 2024-09-18T14:04:17.476 INFO:tasks.rados.rados.0.smithi084.stdout:1023: done (1 left) 2024-09-18T14:04:17.476 INFO:tasks.rados.rados.0.smithi084.stdout:1024: done (0 left) 2024-09-18T14:04:17.476 INFO:tasks.rados.rados.0.smithi084.stdout:1025: copy_from oid 3 from oid 24 current snap is 114 2024-09-18T14:04:17.476 INFO:tasks.rados.rados.0.smithi084.stdout:1026: read oid 32 snap -1 2024-09-18T14:04:17.477 INFO:tasks.rados.rados.0.smithi084.stdout:1026: expect deleted 2024-09-18T14:04:17.477 INFO:tasks.rados.rados.0.smithi084.stdout:1027: copy_from oid 10 from oid 22 current snap is 114 2024-09-18T14:04:17.477 INFO:tasks.rados.rados.0.smithi084.stdout:1028: rollback oid 45 current snap is 114 2024-09-18T14:04:17.477 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 45 to 111 2024-09-18T14:04:17.477 INFO:tasks.rados.rados.0.smithi084.stdout:1029: read oid 27 snap -1 2024-09-18T14:04:17.477 INFO:tasks.rados.rados.0.smithi084.stdout:1029: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:04:17.477 INFO:tasks.rados.rados.0.smithi084.stdout:1030: delete oid 15 current snap is 114 2024-09-18T14:04:17.479 INFO:tasks.rados.rados.0.smithi084.stdout:1026: done (5 left) 2024-09-18T14:04:17.479 INFO:tasks.rados.rados.0.smithi084.stdout:1030: done (4 left) 2024-09-18T14:04:17.479 INFO:tasks.rados.rados.0.smithi084.stdout:1031: write (excl) oid 26 current snap is 114 2024-09-18T14:04:17.479 INFO:tasks.rados.rados.0.smithi084.stdout:1031: seq_num 371 ranges {635397=702323,1893643=341754} 2024-09-18T14:04:17.487 INFO:tasks.rados.rados.0.smithi084.stdout:1031: writing smithi08434663-26 from 635397 to 1337720 tid 1 2024-09-18T14:04:17.492 INFO:tasks.rados.rados.0.smithi084.stdout:1031: writing smithi08434663-26 from 1893643 to 2235397 tid 2 2024-09-18T14:04:17.492 INFO:tasks.rados.rados.0.smithi084.stdout:1032: read oid 44 snap -1 2024-09-18T14:04:17.493 INFO:tasks.rados.rados.0.smithi084.stdout:1032: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:04:17.493 INFO:tasks.rados.rados.0.smithi084.stdout:1033: delete oid 32 current snap is 114 2024-09-18T14:04:17.493 INFO:tasks.rados.rados.0.smithi084.stdout:1028: finishing rollback tid 0 to smithi08434663-45 2024-09-18T14:04:17.493 INFO:tasks.rados.rados.0.smithi084.stdout:1028: finishing rollback tid 1 to smithi08434663-45 2024-09-18T14:04:17.493 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 350 (ObjNum 33 snap 0 seq_num 2475021688) dirty dne 2024-09-18T14:04:17.493 INFO:tasks.rados.rados.0.smithi084.stdout:1029: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:04:17.601 INFO:tasks.rados.rados.0.smithi084.stdout:1025: finishing copy_from racing read to smithi08434663-3 2024-09-18T14:04:17.601 INFO:tasks.rados.rados.0.smithi084.stdout:1025: finishing copy_from to smithi08434663-3 2024-09-18T14:04:17.601 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 183 (ObjNum 267 snap 68 seq_num 267) dirty exists 2024-09-18T14:04:17.601 INFO:tasks.rados.rados.0.smithi084.stdout:1032: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:04:17.705 INFO:tasks.rados.rados.0.smithi084.stdout:1027: finishing copy_from racing read to smithi08434663-10 2024-09-18T14:04:17.705 INFO:tasks.rados.rados.0.smithi084.stdout:1031: finishing write tid 1 to smithi08434663-26 2024-09-18T14:04:17.705 INFO:tasks.rados.rados.0.smithi084.stdout:1027: finishing copy_from to smithi08434663-10 2024-09-18T14:04:17.705 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 326 (ObjNum 341 snap 104 seq_num 341) dirty exists 2024-09-18T14:04:17.705 INFO:tasks.rados.rados.0.smithi084.stdout:1031: finishing write tid 2 to smithi08434663-26 2024-09-18T14:04:17.705 INFO:tasks.rados.rados.0.smithi084.stdout:1031: finishing write tid 3 to smithi08434663-26 2024-09-18T14:04:17.705 INFO:tasks.rados.rados.0.smithi084.stdout:1031: finishing write tid 4 to smithi08434663-26 2024-09-18T14:04:17.705 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 334 (ObjNum 371 snap 114 seq_num 371) dirty exists 2024-09-18T14:04:17.706 INFO:tasks.rados.rados.0.smithi084.stdout:1031: left oid 26 (ObjNum 371 snap 114 seq_num 371) 2024-09-18T14:04:17.707 INFO:tasks.rados.rados.0.smithi084.stdout:1025: done (6 left) 2024-09-18T14:04:17.707 INFO:tasks.rados.rados.0.smithi084.stdout:1027: done (5 left) 2024-09-18T14:04:17.707 INFO:tasks.rados.rados.0.smithi084.stdout:1028: done (4 left) 2024-09-18T14:04:17.707 INFO:tasks.rados.rados.0.smithi084.stdout:1029: done (3 left) 2024-09-18T14:04:17.707 INFO:tasks.rados.rados.0.smithi084.stdout:1031: done (2 left) 2024-09-18T14:04:17.707 INFO:tasks.rados.rados.0.smithi084.stdout:1032: done (1 left) 2024-09-18T14:04:17.707 INFO:tasks.rados.rados.0.smithi084.stdout:1033: done (0 left) 2024-09-18T14:04:17.707 INFO:tasks.rados.rados.0.smithi084.stdout:1034: snap_remove snap 106 2024-09-18T14:04:18.196 INFO:tasks.rados.rados.0.smithi084.stdout:1034: done (0 left) 2024-09-18T14:04:18.196 INFO:tasks.rados.rados.0.smithi084.stdout:1035: snap_remove snap 113 2024-09-18T14:04:19.195 INFO:tasks.rados.rados.0.smithi084.stdout:1035: done (0 left) 2024-09-18T14:04:19.196 INFO:tasks.rados.rados.0.smithi084.stdout:1036: read oid 40 snap -1 2024-09-18T14:04:19.196 INFO:tasks.rados.rados.0.smithi084.stdout:1036: expect (ObjNum 354 snap 109 seq_num 354) 2024-09-18T14:04:19.196 INFO:tasks.rados.rados.0.smithi084.stdout:1037: snap_remove snap 98 2024-09-18T14:04:20.206 INFO:tasks.rados.rados.0.smithi084.stdout:1037: done (1 left) 2024-09-18T14:04:20.206 INFO:tasks.rados.rados.0.smithi084.stdout:1038: snap_create 2024-09-18T14:04:20.206 INFO:tasks.rados.rados.0.smithi084.stdout:1036: expect (ObjNum 354 snap 109 seq_num 354) 2024-09-18T14:04:21.210 INFO:tasks.rados.rados.0.smithi084.stdout:1036: done (1 left) 2024-09-18T14:04:21.211 INFO:tasks.rados.rados.0.smithi084.stdout:1038: done (0 left) 2024-09-18T14:04:21.211 INFO:tasks.rados.rados.0.smithi084.stdout:1039: copy_from oid 3 from oid 22 current snap is 115 2024-09-18T14:04:21.211 INFO:tasks.rados.rados.0.smithi084.stdout:1040: rollback oid 39 current snap is 115 2024-09-18T14:04:21.211 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 39 to 97 2024-09-18T14:04:21.211 INFO:tasks.rados.rados.0.smithi084.stdout:1041: write oid 47 current snap is 115 2024-09-18T14:04:21.211 INFO:tasks.rados.rados.0.smithi084.stdout:1041: seq_num 373 ranges {669011=569174,2010607=258404} 2024-09-18T14:04:21.217 INFO:tasks.rados.rados.0.smithi084.stdout:1041: writing smithi08434663-47 from 669011 to 1238185 tid 1 2024-09-18T14:04:21.224 INFO:tasks.rados.rados.0.smithi084.stdout:1041: writing smithi08434663-47 from 2010607 to 2269011 tid 2 2024-09-18T14:04:21.224 INFO:tasks.rados.rados.0.smithi084.stdout:1042: delete oid 29 current snap is 115 2024-09-18T14:04:21.224 INFO:tasks.rados.rados.0.smithi084.stdout:1040: finishing rollback tid 0 to smithi08434663-39 2024-09-18T14:04:21.224 INFO:tasks.rados.rados.0.smithi084.stdout:1040: finishing rollback tid 1 to smithi08434663-39 2024-09-18T14:04:21.224 INFO:tasks.rados.rados.0.smithi084.stdout:1040: finishing rollback tid 2 to smithi08434663-39 2024-09-18T14:04:21.224 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 253 (ObjNum 306 snap 93 seq_num 306) dirty exists 2024-09-18T14:04:21.224 INFO:tasks.rados.rados.0.smithi084.stdout:1039: finishing copy_from racing read to smithi08434663-3 2024-09-18T14:04:21.224 INFO:tasks.rados.rados.0.smithi084.stdout:1041: finishing write tid 1 to smithi08434663-47 2024-09-18T14:04:21.226 INFO:tasks.rados.rados.0.smithi084.stdout:1039: finishing copy_from to smithi08434663-3 2024-09-18T14:04:21.226 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 326 (ObjNum 341 snap 104 seq_num 341) dirty exists 2024-09-18T14:04:21.227 INFO:tasks.rados.rados.0.smithi084.stdout:1041: finishing write tid 2 to smithi08434663-47 2024-09-18T14:04:21.227 INFO:tasks.rados.rados.0.smithi084.stdout:1039: done (3 left) 2024-09-18T14:04:21.227 INFO:tasks.rados.rados.0.smithi084.stdout:1040: done (2 left) 2024-09-18T14:04:21.227 INFO:tasks.rados.rados.0.smithi084.stdout:1042: done (1 left) 2024-09-18T14:04:21.227 INFO:tasks.rados.rados.0.smithi084.stdout:1043: snap_remove snap 108 2024-09-18T14:04:22.189 INFO:tasks.rados.rados.0.smithi084.stdout:1043: done (1 left) 2024-09-18T14:04:22.189 INFO:tasks.rados.rados.0.smithi084.stdout:1044: snap_create 2024-09-18T14:04:22.189 INFO:tasks.rados.rados.0.smithi084.stdout:1041: finishing write tid 3 to smithi08434663-47 2024-09-18T14:04:22.189 INFO:tasks.rados.rados.0.smithi084.stdout:1041: finishing write tid 4 to smithi08434663-47 2024-09-18T14:04:22.189 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 257 (ObjNum 373 snap 115 seq_num 373) dirty exists 2024-09-18T14:04:22.189 INFO:tasks.rados.rados.0.smithi084.stdout:1041: left oid 47 (ObjNum 373 snap 115 seq_num 373) 2024-09-18T14:04:23.189 INFO:tasks.rados.rados.0.smithi084.stdout:1041: done (1 left) 2024-09-18T14:04:23.190 INFO:tasks.rados.rados.0.smithi084.stdout:1044: done (0 left) 2024-09-18T14:04:23.190 INFO:tasks.rados.rados.0.smithi084.stdout:1045: copy_from oid 11 from oid 19 current snap is 116 2024-09-18T14:04:23.190 INFO:tasks.rados.rados.0.smithi084.stdout:1046: read oid 25 snap -1 2024-09-18T14:04:23.190 INFO:tasks.rados.rados.0.smithi084.stdout:1046: expect (ObjNum 366 snap 114 seq_num 366) 2024-09-18T14:04:23.190 INFO:tasks.rados.rados.0.smithi084.stdout:1047: read oid 17 snap -1 2024-09-18T14:04:23.190 INFO:tasks.rados.rados.0.smithi084.stdout:1047: expect (ObjNum 333 snap 103 seq_num 333) 2024-09-18T14:04:23.190 INFO:tasks.rados.rados.0.smithi084.stdout:1048: snap_remove snap 110 2024-09-18T14:04:24.191 INFO:tasks.rados.rados.0.smithi084.stdout:1048: done (3 left) 2024-09-18T14:04:24.191 INFO:tasks.rados.rados.0.smithi084.stdout:1049: read oid 44 snap -1 2024-09-18T14:04:24.191 INFO:tasks.rados.rados.0.smithi084.stdout:1049: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:04:24.191 INFO:tasks.rados.rados.0.smithi084.stdout:1050: delete oid 20 current snap is 116 2024-09-18T14:04:24.191 INFO:tasks.rados.rados.0.smithi084.stdout:1046: expect (ObjNum 366 snap 114 seq_num 366) 2024-09-18T14:04:24.369 INFO:tasks.rados.rados.0.smithi084.stdout:1045: finishing copy_from racing read to smithi08434663-11 2024-09-18T14:04:24.369 INFO:tasks.rados.rados.0.smithi084.stdout:1047: expect (ObjNum 333 snap 103 seq_num 333) 2024-09-18T14:04:24.501 INFO:tasks.rados.rados.0.smithi084.stdout:1045: finishing copy_from to smithi08434663-11 2024-09-18T14:04:24.501 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 368 (ObjNum 247 snap 57 seq_num 247) dirty exists 2024-09-18T14:04:24.501 INFO:tasks.rados.rados.0.smithi084.stdout:1045: done (4 left) 2024-09-18T14:04:24.502 INFO:tasks.rados.rados.0.smithi084.stdout:1046: done (3 left) 2024-09-18T14:04:24.502 INFO:tasks.rados.rados.0.smithi084.stdout:1047: done (2 left) 2024-09-18T14:04:24.502 INFO:tasks.rados.rados.0.smithi084.stdout:1050: done (1 left) 2024-09-18T14:04:24.502 INFO:tasks.rados.rados.0.smithi084.stdout:1051: rollback oid 9 current snap is 116 2024-09-18T14:04:24.502 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 9 to 105 2024-09-18T14:04:24.502 INFO:tasks.rados.rados.0.smithi084.stdout:1049: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:04:24.606 INFO:tasks.rados.rados.0.smithi084.stdout:1051: finishing rollback tid 1 to smithi08434663-9 2024-09-18T14:04:24.606 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 0 (ObjNum 33 snap 0 seq_num 2495086800) dirty dne 2024-09-18T14:04:24.606 INFO:tasks.rados.rados.0.smithi084.stdout:1049: done (1 left) 2024-09-18T14:04:24.606 INFO:tasks.rados.rados.0.smithi084.stdout:1051: done (0 left) 2024-09-18T14:04:24.606 INFO:tasks.rados.rados.0.smithi084.stdout:1052: rollback oid 14 current snap is 116 2024-09-18T14:04:24.606 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 14 to 97 2024-09-18T14:04:24.606 INFO:tasks.rados.rados.0.smithi084.stdout:1053: write oid 24 current snap is 116 2024-09-18T14:04:24.606 INFO:tasks.rados.rados.0.smithi084.stdout:1053: seq_num 376 ranges {719432=794921,2186053=133379} 2024-09-18T14:04:24.614 INFO:tasks.rados.rados.0.smithi084.stdout:1053: writing smithi08434663-24 from 719432 to 1514353 tid 1 2024-09-18T14:04:24.618 INFO:tasks.rados.rados.0.smithi084.stdout:1053: writing smithi08434663-24 from 2186053 to 2319432 tid 2 2024-09-18T14:04:24.618 INFO:tasks.rados.rados.0.smithi084.stdout:1054: snap_create 2024-09-18T14:04:24.618 INFO:tasks.rados.rados.0.smithi084.stdout:1052: finishing rollback tid 1 to smithi08434663-14 2024-09-18T14:04:24.618 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 331 (ObjNum 33 snap 0 seq_num 3977800368) dirty dne 2024-09-18T14:04:24.620 INFO:tasks.rados.rados.0.smithi084.stdout:1053: finishing write tid 1 to smithi08434663-24 2024-09-18T14:04:24.621 INFO:tasks.rados.rados.0.smithi084.stdout:1053: finishing write tid 2 to smithi08434663-24 2024-09-18T14:04:24.621 INFO:tasks.rados.rados.0.smithi084.stdout:1053: finishing write tid 3 to smithi08434663-24 2024-09-18T14:04:24.621 INFO:tasks.rados.rados.0.smithi084.stdout:1053: finishing write tid 4 to smithi08434663-24 2024-09-18T14:04:24.621 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 186 (ObjNum 376 snap 116 seq_num 376) dirty exists 2024-09-18T14:04:24.621 INFO:tasks.rados.rados.0.smithi084.stdout:1053: left oid 24 (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:04:25.217 INFO:tasks.rados.rados.0.smithi084.stdout:1052: done (2 left) 2024-09-18T14:04:25.218 INFO:tasks.rados.rados.0.smithi084.stdout:1053: done (1 left) 2024-09-18T14:04:25.218 INFO:tasks.rados.rados.0.smithi084.stdout:1054: done (0 left) 2024-09-18T14:04:25.218 INFO:tasks.rados.rados.0.smithi084.stdout:1055: snap_remove snap 89 2024-09-18T14:04:26.225 INFO:tasks.rados.rados.0.smithi084.stdout:1055: done (0 left) 2024-09-18T14:04:26.225 INFO:tasks.rados.rados.0.smithi084.stdout:1056: delete oid 4 current snap is 117 2024-09-18T14:04:26.232 INFO:tasks.rados.rados.0.smithi084.stdout:1056: done (0 left) 2024-09-18T14:04:26.232 INFO:tasks.rados.rados.0.smithi084.stdout:1057: copy_from oid 14 from oid 40 current snap is 117 2024-09-18T14:04:26.232 INFO:tasks.rados.rados.0.smithi084.stdout:1058: read oid 2 snap -1 2024-09-18T14:04:26.232 INFO:tasks.rados.rados.0.smithi084.stdout:1058: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:04:26.233 INFO:tasks.rados.rados.0.smithi084.stdout:1059: write oid 16 current snap is 117 2024-09-18T14:04:26.233 INFO:tasks.rados.rados.0.smithi084.stdout:1059: seq_num 378 ranges {753046=545419,1786664=431793} 2024-09-18T14:04:26.240 INFO:tasks.rados.rados.0.smithi084.stdout:1059: writing smithi08434663-16 from 753046 to 1298465 tid 1 2024-09-18T14:04:26.245 INFO:tasks.rados.rados.0.smithi084.stdout:1059: writing smithi08434663-16 from 1786664 to 2218457 tid 2 2024-09-18T14:04:26.246 INFO:tasks.rados.rados.0.smithi084.stdout:1060: snap_remove snap 111 2024-09-18T14:04:27.190 INFO:tasks.rados.rados.0.smithi084.stdout:1060: done (3 left) 2024-09-18T14:04:27.190 INFO:tasks.rados.rados.0.smithi084.stdout:1061: snap_create 2024-09-18T14:04:27.190 INFO:tasks.rados.rados.0.smithi084.stdout:1058: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:04:27.271 INFO:tasks.rados.rados.0.smithi084.stdout:1057: finishing copy_from racing read to smithi08434663-14 2024-09-18T14:04:27.271 INFO:tasks.rados.rados.0.smithi084.stdout:1059: finishing write tid 1 to smithi08434663-16 2024-09-18T14:04:27.271 INFO:tasks.rados.rados.0.smithi084.stdout:1057: finishing copy_from to smithi08434663-14 2024-09-18T14:04:27.271 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 317 (ObjNum 354 snap 109 seq_num 354) dirty exists 2024-09-18T14:04:27.271 INFO:tasks.rados.rados.0.smithi084.stdout:1059: finishing write tid 2 to smithi08434663-16 2024-09-18T14:04:27.271 INFO:tasks.rados.rados.0.smithi084.stdout:1059: finishing write tid 3 to smithi08434663-16 2024-09-18T14:04:27.271 INFO:tasks.rados.rados.0.smithi084.stdout:1059: finishing write tid 4 to smithi08434663-16 2024-09-18T14:04:27.271 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 392 (ObjNum 378 snap 117 seq_num 378) dirty exists 2024-09-18T14:04:27.271 INFO:tasks.rados.rados.0.smithi084.stdout:1059: left oid 16 (ObjNum 378 snap 117 seq_num 378) 2024-09-18T14:04:28.195 INFO:tasks.rados.rados.0.smithi084.stdout:1057: done (3 left) 2024-09-18T14:04:28.195 INFO:tasks.rados.rados.0.smithi084.stdout:1058: done (2 left) 2024-09-18T14:04:28.195 INFO:tasks.rados.rados.0.smithi084.stdout:1059: done (1 left) 2024-09-18T14:04:28.195 INFO:tasks.rados.rados.0.smithi084.stdout:1061: done (0 left) 2024-09-18T14:04:28.195 INFO:tasks.rados.rados.0.smithi084.stdout:1062: snap_create 2024-09-18T14:04:29.198 INFO:tasks.rados.rados.0.smithi084.stdout:1062: done (0 left) 2024-09-18T14:04:29.198 INFO:tasks.rados.rados.0.smithi084.stdout:1063: write (excl) oid 4 current snap is 119 2024-09-18T14:04:29.198 INFO:tasks.rados.rados.0.smithi084.stdout:1063: seq_num 379 ranges {769853=620668,1845146=524707} 2024-09-18T14:04:29.206 INFO:tasks.rados.rados.0.smithi084.stdout:1063: writing smithi08434663-4 from 769853 to 1390521 tid 1 2024-09-18T14:04:29.211 INFO:tasks.rados.rados.0.smithi084.stdout:1063: writing smithi08434663-4 from 1845146 to 2369853 tid 2 2024-09-18T14:04:29.212 INFO:tasks.rados.rados.0.smithi084.stdout:1064: read oid 44 snap -1 2024-09-18T14:04:29.212 INFO:tasks.rados.rados.0.smithi084.stdout:1064: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:04:29.212 INFO:tasks.rados.rados.0.smithi084.stdout:1065: delete oid 14 current snap is 119 2024-09-18T14:04:29.212 INFO:tasks.rados.rados.0.smithi084.stdout:1063: finishing write tid 1 to smithi08434663-4 2024-09-18T14:04:29.214 INFO:tasks.rados.rados.0.smithi084.stdout:1063: finishing write tid 2 to smithi08434663-4 2024-09-18T14:04:29.215 INFO:tasks.rados.rados.0.smithi084.stdout:1063: finishing write tid 3 to smithi08434663-4 2024-09-18T14:04:29.215 INFO:tasks.rados.rados.0.smithi084.stdout:1063: finishing write tid 4 to smithi08434663-4 2024-09-18T14:04:29.215 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 387 (ObjNum 379 snap 119 seq_num 379) dirty exists 2024-09-18T14:04:29.215 INFO:tasks.rados.rados.0.smithi084.stdout:1063: left oid 4 (ObjNum 379 snap 119 seq_num 379) 2024-09-18T14:04:29.215 INFO:tasks.rados.rados.0.smithi084.stdout:1063: done (2 left) 2024-09-18T14:04:29.216 INFO:tasks.rados.rados.0.smithi084.stdout:1065: done (1 left) 2024-09-18T14:04:29.216 INFO:tasks.rados.rados.0.smithi084.stdout:1066: rollback oid 32 current snap is 119 2024-09-18T14:04:29.216 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 32 to 109 2024-09-18T14:04:29.216 INFO:tasks.rados.rados.0.smithi084.stdout:1067: delete oid 22 current snap is 119 2024-09-18T14:04:29.217 INFO:tasks.rados.rados.0.smithi084.stdout:1066: finishing rollback tid 1 to smithi08434663-32 2024-09-18T14:04:29.218 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 418 (ObjNum 129 snap 0 seq_num 3385323984) dirty dne 2024-09-18T14:04:29.219 INFO:tasks.rados.rados.0.smithi084.stdout:1064: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:04:29.325 INFO:tasks.rados.rados.0.smithi084.stdout:1064: done (2 left) 2024-09-18T14:04:29.325 INFO:tasks.rados.rados.0.smithi084.stdout:1066: done (1 left) 2024-09-18T14:04:29.325 INFO:tasks.rados.rados.0.smithi084.stdout:1067: done (0 left) 2024-09-18T14:04:29.325 INFO:tasks.rados.rados.0.smithi084.stdout:1068: read oid 18 snap -1 2024-09-18T14:04:29.325 INFO:tasks.rados.rados.0.smithi084.stdout:1068: expect deleted 2024-09-18T14:04:29.325 INFO:tasks.rados.rados.0.smithi084.stdout:1069: read oid 17 snap 97 2024-09-18T14:04:29.325 INFO:tasks.rados.rados.0.smithi084.stdout:1069: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:04:29.325 INFO:tasks.rados.rados.0.smithi084.stdout:1070: snap_remove snap 85 2024-09-18T14:04:30.217 INFO:tasks.rados.rados.0.smithi084.stdout:1070: done (2 left) 2024-09-18T14:04:30.218 INFO:tasks.rados.rados.0.smithi084.stdout:1071: snap_remove snap 116 2024-09-18T14:04:31.223 INFO:tasks.rados.rados.0.smithi084.stdout:1071: done (2 left) 2024-09-18T14:04:31.223 INFO:tasks.rados.rados.0.smithi084.stdout:1072: delete oid 27 current snap is 119 2024-09-18T14:04:31.223 INFO:tasks.rados.rados.0.smithi084.stdout:1069: expect (ObjNum 197 snap 46 seq_num 197) 2024-09-18T14:04:31.329 INFO:tasks.rados.rados.0.smithi084.stdout:1068: done (2 left) 2024-09-18T14:04:31.329 INFO:tasks.rados.rados.0.smithi084.stdout:1069: done (1 left) 2024-09-18T14:04:31.329 INFO:tasks.rados.rados.0.smithi084.stdout:1072: done (0 left) 2024-09-18T14:04:31.329 INFO:tasks.rados.rados.0.smithi084.stdout:1073: write (excl) oid 26 current snap is 119 2024-09-18T14:04:31.329 INFO:tasks.rados.rados.0.smithi084.stdout:1073: seq_num 383 ranges {437081=638017,1679074=582789} 2024-09-18T14:04:31.334 INFO:tasks.rados.rados.0.smithi084.stdout:1073: writing smithi08434663-26 from 437081 to 1075098 tid 1 2024-09-18T14:04:31.341 INFO:tasks.rados.rados.0.smithi084.stdout:1073: writing smithi08434663-26 from 1679074 to 2261863 tid 2 2024-09-18T14:04:31.341 INFO:tasks.rados.rados.0.smithi084.stdout:1074: delete oid 10 current snap is 119 2024-09-18T14:04:31.341 INFO:tasks.rados.rados.0.smithi084.stdout:1073: finishing write tid 1 to smithi08434663-26 2024-09-18T14:04:31.346 INFO:tasks.rados.rados.0.smithi084.stdout:1073: finishing write tid 2 to smithi08434663-26 2024-09-18T14:04:31.350 INFO:tasks.rados.rados.0.smithi084.stdout:1073: finishing write tid 3 to smithi08434663-26 2024-09-18T14:04:31.350 INFO:tasks.rados.rados.0.smithi084.stdout:1073: finishing write tid 4 to smithi08434663-26 2024-09-18T14:04:31.350 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 388 (ObjNum 383 snap 119 seq_num 383) dirty exists 2024-09-18T14:04:31.350 INFO:tasks.rados.rados.0.smithi084.stdout:1073: left oid 26 (ObjNum 383 snap 119 seq_num 383) 2024-09-18T14:04:31.350 INFO:tasks.rados.rados.0.smithi084.stdout:1073: done (1 left) 2024-09-18T14:04:31.350 INFO:tasks.rados.rados.0.smithi084.stdout:1074: done (0 left) 2024-09-18T14:04:31.350 INFO:tasks.rados.rados.0.smithi084.stdout:1075: read oid 2 snap -1 2024-09-18T14:04:31.350 INFO:tasks.rados.rados.0.smithi084.stdout:1075: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:04:31.350 INFO:tasks.rados.rados.0.smithi084.stdout:1076: rollback oid 13 current snap is 119 2024-09-18T14:04:31.350 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 13 to 112 2024-09-18T14:04:31.350 INFO:tasks.rados.rados.0.smithi084.stdout:1077: write (excl) oid 39 current snap is 119 2024-09-18T14:04:31.350 INFO:tasks.rados.rados.0.smithi084.stdout:1077: seq_num 385 ranges {470695=788515,1679685=586458} 2024-09-18T14:04:31.356 INFO:tasks.rados.rados.0.smithi084.stdout:1077: writing smithi08434663-39 from 470695 to 1259210 tid 1 2024-09-18T14:04:31.362 INFO:tasks.rados.rados.0.smithi084.stdout:1077: writing smithi08434663-39 from 1679685 to 2266143 tid 2 2024-09-18T14:04:31.362 INFO:tasks.rados.rados.0.smithi084.stdout:1078: write oid 3 current snap is 119 2024-09-18T14:04:31.362 INFO:tasks.rados.rados.0.smithi084.stdout:1078: seq_num 386 ranges {487502=463764,1738167=446469} 2024-09-18T14:04:31.368 INFO:tasks.rados.rados.0.smithi084.stdout:1078: writing smithi08434663-3 from 487502 to 951266 tid 1 2024-09-18T14:04:31.374 INFO:tasks.rados.rados.0.smithi084.stdout:1078: writing smithi08434663-3 from 1738167 to 2184636 tid 2 2024-09-18T14:04:31.374 INFO:tasks.rados.rados.0.smithi084.stdout:1079: write oid 14 current snap is 119 2024-09-18T14:04:31.375 INFO:tasks.rados.rados.0.smithi084.stdout:1079: seq_num 387 ranges {504309=539013,1796649=590127} 2024-09-18T14:04:31.380 INFO:tasks.rados.rados.0.smithi084.stdout:1079: writing smithi08434663-14 from 504309 to 1043322 tid 1 2024-09-18T14:04:31.387 INFO:tasks.rados.rados.0.smithi084.stdout:1079: writing smithi08434663-14 from 1796649 to 2386776 tid 2 2024-09-18T14:04:31.388 INFO:tasks.rados.rados.0.smithi084.stdout:1080: read oid 8 snap -1 2024-09-18T14:04:31.388 INFO:tasks.rados.rados.0.smithi084.stdout:1080: expect deleted 2024-09-18T14:04:31.388 INFO:tasks.rados.rados.0.smithi084.stdout:1081: delete oid 50 current snap is 119 2024-09-18T14:04:31.388 INFO:tasks.rados.rados.0.smithi084.stdout:1075: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:04:31.464 INFO:tasks.rados.rados.0.smithi084.stdout:1076: finishing rollback tid 0 to smithi08434663-13 2024-09-18T14:04:31.464 INFO:tasks.rados.rados.0.smithi084.stdout:1076: finishing rollback tid 1 to smithi08434663-13 2024-09-18T14:04:31.464 INFO:tasks.rados.rados.0.smithi084.stdout:1076: finishing rollback tid 2 to smithi08434663-13 2024-09-18T14:04:31.465 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 358 (ObjNum 359 snap 111 seq_num 359) dirty exists 2024-09-18T14:04:31.465 INFO:tasks.rados.rados.0.smithi084.stdout:1077: finishing write tid 1 to smithi08434663-39 2024-09-18T14:04:31.465 INFO:tasks.rados.rados.0.smithi084.stdout:1077: finishing write tid 2 to smithi08434663-39 2024-09-18T14:04:31.465 INFO:tasks.rados.rados.0.smithi084.stdout:1077: finishing write tid 3 to smithi08434663-39 2024-09-18T14:04:31.465 INFO:tasks.rados.rados.0.smithi084.stdout:1077: finishing write tid 4 to smithi08434663-39 2024-09-18T14:04:31.465 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 371 (ObjNum 385 snap 119 seq_num 385) dirty exists 2024-09-18T14:04:31.465 INFO:tasks.rados.rados.0.smithi084.stdout:1077: left oid 39 (ObjNum 385 snap 119 seq_num 385) 2024-09-18T14:04:31.465 INFO:tasks.rados.rados.0.smithi084.stdout:1078: finishing write tid 1 to smithi08434663-3 2024-09-18T14:04:31.465 INFO:tasks.rados.rados.0.smithi084.stdout:1075: done (6 left) 2024-09-18T14:04:31.466 INFO:tasks.rados.rados.0.smithi084.stdout:1076: done (5 left) 2024-09-18T14:04:31.466 INFO:tasks.rados.rados.0.smithi084.stdout:1077: done (4 left) 2024-09-18T14:04:31.466 INFO:tasks.rados.rados.0.smithi084.stdout:1081: done (3 left) 2024-09-18T14:04:31.466 INFO:tasks.rados.rados.0.smithi084.stdout:1082: copy_from oid 32 from oid 29 current snap is 119 2024-09-18T14:04:31.466 INFO:tasks.rados.rados.0.smithi084.stdout:1078: finishing write tid 2 to smithi08434663-3 2024-09-18T14:04:31.466 INFO:tasks.rados.rados.0.smithi084.stdout:1078: finishing write tid 3 to smithi08434663-3 2024-09-18T14:04:31.466 INFO:tasks.rados.rados.0.smithi084.stdout:1078: finishing write tid 4 to smithi08434663-3 2024-09-18T14:04:31.467 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 336 (ObjNum 386 snap 119 seq_num 386) dirty exists 2024-09-18T14:04:31.467 INFO:tasks.rados.rados.0.smithi084.stdout:1078: left oid 3 (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:04:31.467 INFO:tasks.rados.rados.0.smithi084.stdout:1079: finishing write tid 1 to smithi08434663-14 2024-09-18T14:04:31.467 INFO:tasks.rados.rados.0.smithi084.stdout:1079: finishing write tid 2 to smithi08434663-14 2024-09-18T14:04:31.467 INFO:tasks.rados.rados.0.smithi084.stdout:1079: finishing write tid 3 to smithi08434663-14 2024-09-18T14:04:31.467 INFO:tasks.rados.rados.0.smithi084.stdout:1079: finishing write tid 4 to smithi08434663-14 2024-09-18T14:04:31.467 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 481 (ObjNum 387 snap 119 seq_num 387) dirty exists 2024-09-18T14:04:31.467 INFO:tasks.rados.rados.0.smithi084.stdout:1079: left oid 14 (ObjNum 387 snap 119 seq_num 387) 2024-09-18T14:04:31.467 INFO:tasks.rados.rados.0.smithi084.stdout:1078: done (3 left) 2024-09-18T14:04:31.468 INFO:tasks.rados.rados.0.smithi084.stdout:1079: done (2 left) 2024-09-18T14:04:31.468 INFO:tasks.rados.rados.0.smithi084.stdout:1080: done (1 left) 2024-09-18T14:04:31.468 INFO:tasks.rados.rados.0.smithi084.stdout:1083: write oid 26 current snap is 119 2024-09-18T14:04:31.468 INFO:tasks.rados.rados.0.smithi084.stdout:1083: seq_num 389 ranges {537923=405864,1513613=593796} 2024-09-18T14:04:31.468 INFO:tasks.rados.rados.0.smithi084.stdout:1083: writing smithi08434663-26 from 537923 to 943787 tid 1 2024-09-18T14:04:31.475 INFO:tasks.rados.rados.0.smithi084.stdout:1083: writing smithi08434663-26 from 1513613 to 2107409 tid 2 2024-09-18T14:04:31.475 INFO:tasks.rados.rados.0.smithi084.stdout:1084: copy_from oid 34 from oid 49 current snap is 119 2024-09-18T14:04:31.475 INFO:tasks.rados.rados.0.smithi084.stdout:1082: finishing copy_from to smithi08434663-32 2024-09-18T14:04:31.475 INFO:tasks.rados.rados.0.smithi084.stdout:1082: got expected ENOENT (src dne) 2024-09-18T14:04:31.475 INFO:tasks.rados.rados.0.smithi084.stdout:1082: finishing copy_from racing read to smithi08434663-32 2024-09-18T14:04:31.475 INFO:tasks.rados.rados.0.smithi084.stdout:1083: finishing write tid 1 to smithi08434663-26 2024-09-18T14:04:31.476 INFO:tasks.rados.rados.0.smithi084.stdout:1082: done (2 left) 2024-09-18T14:04:31.476 INFO:tasks.rados.rados.0.smithi084.stdout:1085: snap_create 2024-09-18T14:04:31.476 INFO:tasks.rados.rados.0.smithi084.stdout:1084: finishing copy_from to smithi08434663-34 2024-09-18T14:04:31.476 INFO:tasks.rados.rados.0.smithi084.stdout:1084: got expected ENOENT (src dne) 2024-09-18T14:04:31.476 INFO:tasks.rados.rados.0.smithi084.stdout:1084: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:04:31.481 INFO:tasks.rados.rados.0.smithi084.stdout:1083: finishing write tid 2 to smithi08434663-26 2024-09-18T14:04:31.481 INFO:tasks.rados.rados.0.smithi084.stdout:1083: finishing write tid 3 to smithi08434663-26 2024-09-18T14:04:31.481 INFO:tasks.rados.rados.0.smithi084.stdout:1083: finishing write tid 4 to smithi08434663-26 2024-09-18T14:04:31.481 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 393 (ObjNum 389 snap 119 seq_num 389) dirty exists 2024-09-18T14:04:31.481 INFO:tasks.rados.rados.0.smithi084.stdout:1083: left oid 26 (ObjNum 389 snap 119 seq_num 389) 2024-09-18T14:04:32.222 INFO:tasks.rados.rados.0.smithi084.stdout:1083: done (2 left) 2024-09-18T14:04:32.223 INFO:tasks.rados.rados.0.smithi084.stdout:1084: done (1 left) 2024-09-18T14:04:32.223 INFO:tasks.rados.rados.0.smithi084.stdout:1085: done (0 left) 2024-09-18T14:04:32.223 INFO:tasks.rados.rados.0.smithi084.stdout:1086: snap_remove snap 105 2024-09-18T14:04:33.225 INFO:tasks.rados.rados.0.smithi084.stdout:1086: done (0 left) 2024-09-18T14:04:33.226 INFO:tasks.rados.rados.0.smithi084.stdout:1087: read oid 26 snap -1 2024-09-18T14:04:33.226 INFO:tasks.rados.rados.0.smithi084.stdout:1087: expect (ObjNum 389 snap 119 seq_num 389) 2024-09-18T14:04:33.226 INFO:tasks.rados.rados.0.smithi084.stdout:1088: read oid 3 snap 118 2024-09-18T14:04:33.226 INFO:tasks.rados.rados.0.smithi084.stdout:1088: expect (ObjNum 341 snap 104 seq_num 341) 2024-09-18T14:04:33.226 INFO:tasks.rados.rados.0.smithi084.stdout:1089: read oid 24 snap -1 2024-09-18T14:04:33.226 INFO:tasks.rados.rados.0.smithi084.stdout:1089: expect (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:04:33.226 INFO:tasks.rados.rados.0.smithi084.stdout:1090: rollback oid 32 current snap is 120 2024-09-18T14:04:33.226 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 32 to 118 2024-09-18T14:04:33.227 INFO:tasks.rados.rados.0.smithi084.stdout:1091: write oid 45 current snap is 120 2024-09-18T14:04:33.227 INFO:tasks.rados.rados.0.smithi084.stdout:1091: seq_num 390 ranges {554730=481113,1572095=737454} 2024-09-18T14:04:33.232 INFO:tasks.rados.rados.0.smithi084.stdout:1091: writing smithi08434663-45 from 554730 to 1035843 tid 1 2024-09-18T14:04:33.240 INFO:tasks.rados.rados.0.smithi084.stdout:1091: writing smithi08434663-45 from 1572095 to 2309549 tid 2 2024-09-18T14:04:33.240 INFO:tasks.rados.rados.0.smithi084.stdout:1092: delete oid 46 current snap is 120 2024-09-18T14:04:33.241 INFO:tasks.rados.rados.0.smithi084.stdout:1087: expect (ObjNum 389 snap 119 seq_num 389) 2024-09-18T14:04:33.399 INFO:tasks.rados.rados.0.smithi084.stdout:1091: finishing write tid 1 to smithi08434663-45 2024-09-18T14:04:33.399 INFO:tasks.rados.rados.0.smithi084.stdout:1090: finishing rollback tid 1 to smithi08434663-32 2024-09-18T14:04:33.399 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 418 (ObjNum 257 snap 0 seq_num 2494572128) dirty dne 2024-09-18T14:04:33.399 INFO:tasks.rados.rados.0.smithi084.stdout:1091: finishing write tid 2 to smithi08434663-45 2024-09-18T14:04:33.399 INFO:tasks.rados.rados.0.smithi084.stdout:1091: finishing write tid 3 to smithi08434663-45 2024-09-18T14:04:33.399 INFO:tasks.rados.rados.0.smithi084.stdout:1091: finishing write tid 4 to smithi08434663-45 2024-09-18T14:04:33.399 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 406 (ObjNum 390 snap 120 seq_num 390) dirty exists 2024-09-18T14:04:33.399 INFO:tasks.rados.rados.0.smithi084.stdout:1091: left oid 45 (ObjNum 390 snap 120 seq_num 390) 2024-09-18T14:04:33.400 INFO:tasks.rados.rados.0.smithi084.stdout:1089: expect (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:04:33.532 INFO:tasks.rados.rados.0.smithi084.stdout:1088: expect (ObjNum 341 snap 104 seq_num 341) 2024-09-18T14:04:33.648 INFO:tasks.rados.rados.0.smithi084.stdout:1087: done (5 left) 2024-09-18T14:04:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:1088: done (4 left) 2024-09-18T14:04:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:1089: done (3 left) 2024-09-18T14:04:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:1090: done (2 left) 2024-09-18T14:04:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:1091: done (1 left) 2024-09-18T14:04:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:1092: done (0 left) 2024-09-18T14:04:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:1093: read oid 33 snap -1 2024-09-18T14:04:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:1093: expect deleted 2024-09-18T14:04:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:1094: rollback oid 43 current snap is 120 2024-09-18T14:04:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 119 2024-09-18T14:04:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:1095: write (excl) oid 25 current snap is 120 2024-09-18T14:04:33.649 INFO:tasks.rados.rados.0.smithi084.stdout:1095: seq_num 392 ranges {588344=631611,1689059=741123} 2024-09-18T14:04:33.655 INFO:tasks.rados.rados.0.smithi084.stdout:1095: writing smithi08434663-25 from 588344 to 1219955 tid 1 2024-09-18T14:04:33.662 INFO:tasks.rados.rados.0.smithi084.stdout:1095: writing smithi08434663-25 from 1689059 to 2430182 tid 2 2024-09-18T14:04:33.662 INFO:tasks.rados.rados.0.smithi084.stdout:1096: rollback oid 16 current snap is 120 2024-09-18T14:04:33.662 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 16 to 97 2024-09-18T14:04:33.662 INFO:tasks.rados.rados.0.smithi084.stdout:1094: finishing rollback tid 0 to smithi08434663-43 2024-09-18T14:04:33.662 INFO:tasks.rados.rados.0.smithi084.stdout:1094: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:04:33.662 INFO:tasks.rados.rados.0.smithi084.stdout:1094: finishing rollback tid 2 to smithi08434663-43 2024-09-18T14:04:33.662 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 272 (ObjNum 265 snap 67 seq_num 265) dirty exists 2024-09-18T14:04:33.662 INFO:tasks.rados.rados.0.smithi084.stdout:1095: finishing write tid 1 to smithi08434663-25 2024-09-18T14:04:33.662 INFO:tasks.rados.rados.0.smithi084.stdout:1093: done (3 left) 2024-09-18T14:04:33.662 INFO:tasks.rados.rados.0.smithi084.stdout:1094: done (2 left) 2024-09-18T14:04:33.662 INFO:tasks.rados.rados.0.smithi084.stdout:1097: snap_remove snap 112 2024-09-18T14:04:34.241 INFO:tasks.rados.rados.0.smithi084.stdout:1097: done (2 left) 2024-09-18T14:04:34.241 INFO:tasks.rados.rados.0.smithi084.stdout:1098: write (excl) oid 42 current snap is 120 2024-09-18T14:04:34.241 INFO:tasks.rados.rados.0.smithi084.stdout:1096: finishing rollback tid 0 to smithi08434663-16 2024-09-18T14:04:34.241 INFO:tasks.rados.rados.0.smithi084.stdout:1098: seq_num 393 ranges {605151=706860,2031188=573963} 2024-09-18T14:04:34.248 INFO:tasks.rados.rados.0.smithi084.stdout:1098: writing smithi08434663-42 from 605151 to 1312011 tid 1 2024-09-18T14:04:34.256 INFO:tasks.rados.rados.0.smithi084.stdout:1098: writing smithi08434663-42 from 2031188 to 2605151 tid 2 2024-09-18T14:04:34.256 INFO:tasks.rados.rados.0.smithi084.stdout:1099: delete oid 34 current snap is 120 2024-09-18T14:04:34.256 INFO:tasks.rados.rados.0.smithi084.stdout:1096: finishing rollback tid 1 to smithi08434663-16 2024-09-18T14:04:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 413 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:04:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:1095: finishing write tid 2 to smithi08434663-25 2024-09-18T14:04:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:1095: finishing write tid 3 to smithi08434663-25 2024-09-18T14:04:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:1095: finishing write tid 4 to smithi08434663-25 2024-09-18T14:04:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 272 (ObjNum 392 snap 120 seq_num 392) dirty exists 2024-09-18T14:04:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:1095: left oid 25 (ObjNum 392 snap 120 seq_num 392) 2024-09-18T14:04:34.257 INFO:tasks.rados.rados.0.smithi084.stdout:1098: finishing write tid 1 to smithi08434663-42 2024-09-18T14:04:34.260 INFO:tasks.rados.rados.0.smithi084.stdout:1098: finishing write tid 2 to smithi08434663-42 2024-09-18T14:04:34.260 INFO:tasks.rados.rados.0.smithi084.stdout:1098: finishing write tid 3 to smithi08434663-42 2024-09-18T14:04:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:1098: finishing write tid 4 to smithi08434663-42 2024-09-18T14:04:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 414 (ObjNum 393 snap 120 seq_num 393) dirty exists 2024-09-18T14:04:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:1098: left oid 42 (ObjNum 393 snap 120 seq_num 393) 2024-09-18T14:04:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:1095: done (3 left) 2024-09-18T14:04:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:1096: done (2 left) 2024-09-18T14:04:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:1098: done (1 left) 2024-09-18T14:04:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:1099: done (0 left) 2024-09-18T14:04:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:1100: copy_from oid 29 from oid 28 current snap is 120 2024-09-18T14:04:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:1101: delete oid 17 current snap is 120 2024-09-18T14:04:34.263 INFO:tasks.rados.rados.0.smithi084.stdout:1101: done (1 left) 2024-09-18T14:04:34.263 INFO:tasks.rados.rados.0.smithi084.stdout:1102: delete oid 32 current snap is 120 2024-09-18T14:04:34.265 INFO:tasks.rados.rados.0.smithi084.stdout:1102: done (1 left) 2024-09-18T14:04:34.265 INFO:tasks.rados.rados.0.smithi084.stdout:1103: snap_create 2024-09-18T14:04:34.274 INFO:tasks.rados.rados.0.smithi084.stdout:1100: finishing copy_from racing read to smithi08434663-29 2024-09-18T14:04:34.274 INFO:tasks.rados.rados.0.smithi084.stdout:1100: finishing copy_from to smithi08434663-29 2024-09-18T14:04:34.275 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 378 (ObjNum 337 snap 103 seq_num 337) dirty exists 2024-09-18T14:04:35.238 INFO:tasks.rados.rados.0.smithi084.stdout:1100: done (1 left) 2024-09-18T14:04:35.238 INFO:tasks.rados.rados.0.smithi084.stdout:1103: done (0 left) 2024-09-18T14:04:35.238 INFO:tasks.rados.rados.0.smithi084.stdout:1104: copy_from oid 33 from oid 28 current snap is 121 2024-09-18T14:04:35.239 INFO:tasks.rados.rados.0.smithi084.stdout:1105: read oid 43 snap -1 2024-09-18T14:04:35.239 INFO:tasks.rados.rados.0.smithi084.stdout:1105: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:04:35.239 INFO:tasks.rados.rados.0.smithi084.stdout:1106: write (excl) oid 49 current snap is 121 2024-09-18T14:04:35.239 INFO:tasks.rados.rados.0.smithi084.stdout:1106: seq_num 397 ranges {672379=724209,1865116=608472} 2024-09-18T14:04:35.246 INFO:tasks.rados.rados.0.smithi084.stdout:1106: writing smithi08434663-49 from 672379 to 1396588 tid 1 2024-09-18T14:04:35.253 INFO:tasks.rados.rados.0.smithi084.stdout:1106: writing smithi08434663-49 from 1865116 to 2473588 tid 2 2024-09-18T14:04:35.253 INFO:tasks.rados.rados.0.smithi084.stdout:1107: write oid 25 current snap is 121 2024-09-18T14:04:35.253 INFO:tasks.rados.rados.0.smithi084.stdout:1107: seq_num 398 ranges {689186=799458,1923598=752130} 2024-09-18T14:04:35.262 INFO:tasks.rados.rados.0.smithi084.stdout:1107: writing smithi08434663-25 from 689186 to 1488644 tid 1 2024-09-18T14:04:35.269 INFO:tasks.rados.rados.0.smithi084.stdout:1107: writing smithi08434663-25 from 1923598 to 2675728 tid 2 2024-09-18T14:04:35.269 INFO:tasks.rados.rados.0.smithi084.stdout:1108: write oid 32 current snap is 121 2024-09-18T14:04:35.269 INFO:tasks.rados.rados.0.smithi084.stdout:1108: seq_num 399 ranges {705993=474707,1582080=612141} 2024-09-18T14:04:35.275 INFO:tasks.rados.rados.0.smithi084.stdout:1108: writing smithi08434663-32 from 705993 to 1180700 tid 1 2024-09-18T14:04:35.281 INFO:tasks.rados.rados.0.smithi084.stdout:1108: writing smithi08434663-32 from 1582080 to 2194221 tid 2 2024-09-18T14:04:35.281 INFO:tasks.rados.rados.0.smithi084.stdout:1109: snap_create 2024-09-18T14:04:35.281 INFO:tasks.rados.rados.0.smithi084.stdout:1105: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:04:35.361 INFO:tasks.rados.rados.0.smithi084.stdout:1104: finishing copy_from racing read to smithi08434663-33 2024-09-18T14:04:35.361 INFO:tasks.rados.rados.0.smithi084.stdout:1106: finishing write tid 1 to smithi08434663-49 2024-09-18T14:04:35.361 INFO:tasks.rados.rados.0.smithi084.stdout:1104: finishing copy_from to smithi08434663-33 2024-09-18T14:04:35.361 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 378 (ObjNum 337 snap 103 seq_num 337) dirty exists 2024-09-18T14:04:35.362 INFO:tasks.rados.rados.0.smithi084.stdout:1106: finishing write tid 2 to smithi08434663-49 2024-09-18T14:04:35.362 INFO:tasks.rados.rados.0.smithi084.stdout:1106: finishing write tid 3 to smithi08434663-49 2024-09-18T14:04:35.362 INFO:tasks.rados.rados.0.smithi084.stdout:1106: finishing write tid 4 to smithi08434663-49 2024-09-18T14:04:35.362 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 380 (ObjNum 397 snap 121 seq_num 397) dirty exists 2024-09-18T14:04:35.362 INFO:tasks.rados.rados.0.smithi084.stdout:1106: left oid 49 (ObjNum 397 snap 121 seq_num 397) 2024-09-18T14:04:35.362 INFO:tasks.rados.rados.0.smithi084.stdout:1107: finishing write tid 1 to smithi08434663-25 2024-09-18T14:04:35.363 INFO:tasks.rados.rados.0.smithi084.stdout:1107: finishing write tid 2 to smithi08434663-25 2024-09-18T14:04:35.363 INFO:tasks.rados.rados.0.smithi084.stdout:1107: finishing write tid 3 to smithi08434663-25 2024-09-18T14:04:35.363 INFO:tasks.rados.rados.0.smithi084.stdout:1107: finishing write tid 4 to smithi08434663-25 2024-09-18T14:04:35.363 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 278 (ObjNum 398 snap 121 seq_num 398) dirty exists 2024-09-18T14:04:35.363 INFO:tasks.rados.rados.0.smithi084.stdout:1107: left oid 25 (ObjNum 398 snap 121 seq_num 398) 2024-09-18T14:04:35.364 INFO:tasks.rados.rados.0.smithi084.stdout:1108: finishing write tid 1 to smithi08434663-32 2024-09-18T14:04:35.364 INFO:tasks.rados.rados.0.smithi084.stdout:1108: finishing write tid 2 to smithi08434663-32 2024-09-18T14:04:35.364 INFO:tasks.rados.rados.0.smithi084.stdout:1108: finishing write tid 3 to smithi08434663-32 2024-09-18T14:04:35.364 INFO:tasks.rados.rados.0.smithi084.stdout:1108: finishing write tid 4 to smithi08434663-32 2024-09-18T14:04:35.364 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 494 (ObjNum 399 snap 121 seq_num 399) dirty exists 2024-09-18T14:04:35.364 INFO:tasks.rados.rados.0.smithi084.stdout:1108: left oid 32 (ObjNum 399 snap 121 seq_num 399) 2024-09-18T14:04:36.255 INFO:tasks.rados.rados.0.smithi084.stdout:1104: done (5 left) 2024-09-18T14:04:36.255 INFO:tasks.rados.rados.0.smithi084.stdout:1105: done (4 left) 2024-09-18T14:04:36.255 INFO:tasks.rados.rados.0.smithi084.stdout:1106: done (3 left) 2024-09-18T14:04:36.255 INFO:tasks.rados.rados.0.smithi084.stdout:1107: done (2 left) 2024-09-18T14:04:36.255 INFO:tasks.rados.rados.0.smithi084.stdout:1108: done (1 left) 2024-09-18T14:04:36.255 INFO:tasks.rados.rados.0.smithi084.stdout:1109: done (0 left) 2024-09-18T14:04:36.255 INFO:tasks.rados.rados.0.smithi084.stdout:1110: read oid 43 snap -1 2024-09-18T14:04:36.255 INFO:tasks.rados.rados.0.smithi084.stdout:1110: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:04:36.255 INFO:tasks.rados.rados.0.smithi084.stdout:1111: write oid 10 current snap is 122 2024-09-18T14:04:36.255 INFO:tasks.rados.rados.0.smithi084.stdout:1111: seq_num 400 ranges {722800=549956,2040562=682238} 2024-09-18T14:04:36.262 INFO:tasks.rados.rados.0.smithi084.stdout:1111: writing smithi08434663-10 from 722800 to 1272756 tid 1 2024-09-18T14:04:36.270 INFO:tasks.rados.rados.0.smithi084.stdout:1111: writing smithi08434663-10 from 2040562 to 2722800 tid 2 2024-09-18T14:04:36.270 INFO:tasks.rados.rados.0.smithi084.stdout:1112: read oid 23 snap 119 2024-09-18T14:04:36.271 INFO:tasks.rados.rados.0.smithi084.stdout:1112: expect deleted 2024-09-18T14:04:36.271 INFO:tasks.rados.rados.0.smithi084.stdout:1113: snap_create 2024-09-18T14:04:36.271 INFO:tasks.rados.rados.0.smithi084.stdout:1110: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:04:36.348 INFO:tasks.rados.rados.0.smithi084.stdout:1111: finishing write tid 1 to smithi08434663-10 2024-09-18T14:04:36.348 INFO:tasks.rados.rados.0.smithi084.stdout:1111: finishing write tid 2 to smithi08434663-10 2024-09-18T14:04:36.348 INFO:tasks.rados.rados.0.smithi084.stdout:1111: finishing write tid 3 to smithi08434663-10 2024-09-18T14:04:36.348 INFO:tasks.rados.rados.0.smithi084.stdout:1111: finishing write tid 4 to smithi08434663-10 2024-09-18T14:04:36.349 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 408 (ObjNum 400 snap 122 seq_num 400) dirty exists 2024-09-18T14:04:36.349 INFO:tasks.rados.rados.0.smithi084.stdout:1111: left oid 10 (ObjNum 400 snap 122 seq_num 400) 2024-09-18T14:04:37.194 INFO:tasks.rados.rados.0.smithi084.stdout:1110: done (3 left) 2024-09-18T14:04:37.194 INFO:tasks.rados.rados.0.smithi084.stdout:1111: done (2 left) 2024-09-18T14:04:37.194 INFO:tasks.rados.rados.0.smithi084.stdout:1112: done (1 left) 2024-09-18T14:04:37.194 INFO:tasks.rados.rados.0.smithi084.stdout:1113: done (0 left) 2024-09-18T14:04:37.194 INFO:tasks.rados.rados.0.smithi084.stdout:1114: rollback oid 12 current snap is 123 2024-09-18T14:04:37.194 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 12 to 117 2024-09-18T14:04:37.195 INFO:tasks.rados.rados.0.smithi084.stdout:1115: read oid 50 snap -1 2024-09-18T14:04:37.195 INFO:tasks.rados.rados.0.smithi084.stdout:1115: expect deleted 2024-09-18T14:04:37.195 INFO:tasks.rados.rados.0.smithi084.stdout:1116: snap_remove snap 121 2024-09-18T14:04:38.200 INFO:tasks.rados.rados.0.smithi084.stdout:1116: done (2 left) 2024-09-18T14:04:38.200 INFO:tasks.rados.rados.0.smithi084.stdout:1117: write (excl) oid 13 current snap is 123 2024-09-18T14:04:38.200 INFO:tasks.rados.rados.0.smithi084.stdout:1117: seq_num 401 ranges {739607=625205,1982691=499457} 2024-09-18T14:04:38.207 INFO:tasks.rados.rados.0.smithi084.stdout:1117: writing smithi08434663-13 from 739607 to 1364812 tid 1 2024-09-18T14:04:38.213 INFO:tasks.rados.rados.0.smithi084.stdout:1117: writing smithi08434663-13 from 1982691 to 2482148 tid 2 2024-09-18T14:04:38.214 INFO:tasks.rados.rados.0.smithi084.stdout:1118: write oid 15 current snap is 123 2024-09-18T14:04:38.214 INFO:tasks.rados.rados.0.smithi084.stdout:1118: seq_num 402 ranges {756414=700454,2041173=715241} 2024-09-18T14:04:38.221 INFO:tasks.rados.rados.0.smithi084.stdout:1118: writing smithi08434663-15 from 756414 to 1456868 tid 1 2024-09-18T14:04:38.228 INFO:tasks.rados.rados.0.smithi084.stdout:1118: writing smithi08434663-15 from 2041173 to 2756414 tid 2 2024-09-18T14:04:38.229 INFO:tasks.rados.rados.0.smithi084.stdout:1119: write (excl) oid 48 current snap is 123 2024-09-18T14:04:38.229 INFO:tasks.rados.rados.0.smithi084.stdout:1119: seq_num 403 ranges {773221=492056,1816008=503126} 2024-09-18T14:04:38.235 INFO:tasks.rados.rados.0.smithi084.stdout:1119: writing smithi08434663-48 from 773221 to 1265277 tid 1 2024-09-18T14:04:38.242 INFO:tasks.rados.rados.0.smithi084.stdout:1119: writing smithi08434663-48 from 1816008 to 2319134 tid 2 2024-09-18T14:04:38.242 INFO:tasks.rados.rados.0.smithi084.stdout:1120: read oid 18 snap -1 2024-09-18T14:04:38.242 INFO:tasks.rados.rados.0.smithi084.stdout:1120: expect deleted 2024-09-18T14:04:38.242 INFO:tasks.rados.rados.0.smithi084.stdout:1121: snap_create 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:1114: finishing rollback tid 0 to smithi08434663-12 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:1114: finishing rollback tid 1 to smithi08434663-12 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:1114: finishing rollback tid 2 to smithi08434663-12 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 498 (ObjNum 287 snap 74 seq_num 287) dirty exists 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:1117: finishing write tid 1 to smithi08434663-13 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:1117: finishing write tid 2 to smithi08434663-13 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:1117: finishing write tid 3 to smithi08434663-13 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:1117: finishing write tid 4 to smithi08434663-13 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 381 (ObjNum 401 snap 123 seq_num 401) dirty exists 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:1117: left oid 13 (ObjNum 401 snap 123 seq_num 401) 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:1118: finishing write tid 1 to smithi08434663-15 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:1118: finishing write tid 2 to smithi08434663-15 2024-09-18T14:04:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:1118: finishing write tid 3 to smithi08434663-15 2024-09-18T14:04:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1118: finishing write tid 4 to smithi08434663-15 2024-09-18T14:04:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 424 (ObjNum 402 snap 123 seq_num 402) dirty exists 2024-09-18T14:04:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1118: left oid 15 (ObjNum 402 snap 123 seq_num 402) 2024-09-18T14:04:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1119: finishing write tid 1 to smithi08434663-48 2024-09-18T14:04:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:1119: finishing write tid 2 to smithi08434663-48 2024-09-18T14:04:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:1119: finishing write tid 3 to smithi08434663-48 2024-09-18T14:04:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:1119: finishing write tid 4 to smithi08434663-48 2024-09-18T14:04:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 282 (ObjNum 403 snap 123 seq_num 403) dirty exists 2024-09-18T14:04:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:1119: left oid 48 (ObjNum 403 snap 123 seq_num 403) 2024-09-18T14:04:39.204 INFO:tasks.rados.rados.0.smithi084.stdout:1114: done (6 left) 2024-09-18T14:04:39.205 INFO:tasks.rados.rados.0.smithi084.stdout:1115: done (5 left) 2024-09-18T14:04:39.205 INFO:tasks.rados.rados.0.smithi084.stdout:1117: done (4 left) 2024-09-18T14:04:39.205 INFO:tasks.rados.rados.0.smithi084.stdout:1118: done (3 left) 2024-09-18T14:04:39.205 INFO:tasks.rados.rados.0.smithi084.stdout:1119: done (2 left) 2024-09-18T14:04:39.205 INFO:tasks.rados.rados.0.smithi084.stdout:1120: done (1 left) 2024-09-18T14:04:39.205 INFO:tasks.rados.rados.0.smithi084.stdout:1121: done (0 left) 2024-09-18T14:04:39.205 INFO:tasks.rados.rados.0.smithi084.stdout:1122: write (excl) oid 8 current snap is 124 2024-09-18T14:04:39.205 INFO:tasks.rados.rados.0.smithi084.stdout:1122: seq_num 404 ranges {790028=567305,1874490=763137} 2024-09-18T14:04:39.212 INFO:tasks.rados.rados.0.smithi084.stdout:1122: writing smithi08434663-8 from 790028 to 1357333 tid 1 2024-09-18T14:04:39.219 INFO:tasks.rados.rados.0.smithi084.stdout:1122: writing smithi08434663-8 from 1874490 to 2637627 tid 2 2024-09-18T14:04:39.219 INFO:tasks.rados.rados.0.smithi084.stdout:1123: read oid 34 snap -1 2024-09-18T14:04:39.219 INFO:tasks.rados.rados.0.smithi084.stdout:1123: expect deleted 2024-09-18T14:04:39.219 INFO:tasks.rados.rados.0.smithi084.stdout:1122: finishing write tid 1 to smithi08434663-8 2024-09-18T14:04:39.219 INFO:tasks.rados.rados.0.smithi084.stdout:1124: read oid 36 snap 118 2024-09-18T14:04:39.219 INFO:tasks.rados.rados.0.smithi084.stdout:1124: expect (ObjNum 343 snap 104 seq_num 343) 2024-09-18T14:04:39.219 INFO:tasks.rados.rados.0.smithi084.stdout:1125: rollback oid 45 current snap is 124 2024-09-18T14:04:39.219 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 45 to 87 2024-09-18T14:04:39.220 INFO:tasks.rados.rados.0.smithi084.stdout:1126: write oid 18 current snap is 124 2024-09-18T14:04:39.220 INFO:tasks.rados.rados.0.smithi084.stdout:1126: seq_num 405 ranges {406835=642554,1532972=506795,2772312=34523} 2024-09-18T14:04:39.224 INFO:tasks.rados.rados.0.smithi084.stdout:1126: writing smithi08434663-18 from 406835 to 1049389 tid 1 2024-09-18T14:04:39.231 INFO:tasks.rados.rados.0.smithi084.stdout:1126: writing smithi08434663-18 from 1532972 to 2039767 tid 2 2024-09-18T14:04:39.236 INFO:tasks.rados.rados.0.smithi084.stdout:1126: writing smithi08434663-18 from 2772312 to 2806835 tid 3 2024-09-18T14:04:39.236 INFO:tasks.rados.rados.0.smithi084.stdout:1127: rollback oid 4 current snap is 124 2024-09-18T14:04:39.236 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 4 to 97 2024-09-18T14:04:39.236 INFO:tasks.rados.rados.0.smithi084.stdout:1122: finishing write tid 2 to smithi08434663-8 2024-09-18T14:04:39.236 INFO:tasks.rados.rados.0.smithi084.stdout:1122: finishing write tid 3 to smithi08434663-8 2024-09-18T14:04:39.236 INFO:tasks.rados.rados.0.smithi084.stdout:1122: finishing write tid 4 to smithi08434663-8 2024-09-18T14:04:39.236 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 192 (ObjNum 404 snap 124 seq_num 404) dirty exists 2024-09-18T14:04:39.236 INFO:tasks.rados.rados.0.smithi084.stdout:1122: left oid 8 (ObjNum 404 snap 124 seq_num 404) 2024-09-18T14:04:39.236 INFO:tasks.rados.rados.0.smithi084.stdout:1125: finishing rollback tid 0 to smithi08434663-45 2024-09-18T14:04:39.237 INFO:tasks.rados.rados.0.smithi084.stdout:1125: finishing rollback tid 1 to smithi08434663-45 2024-09-18T14:04:39.237 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 419 (ObjNum 33 snap 0 seq_num 3385025970) dirty dne 2024-09-18T14:04:39.237 INFO:tasks.rados.rados.0.smithi084.stdout:1126: finishing write tid 1 to smithi08434663-18 2024-09-18T14:04:39.237 INFO:tasks.rados.rados.0.smithi084.stdout:1124: expect (ObjNum 343 snap 104 seq_num 343) 2024-09-18T14:04:39.429 INFO:tasks.rados.rados.0.smithi084.stdout:1126: finishing write tid 2 to smithi08434663-18 2024-09-18T14:04:39.429 INFO:tasks.rados.rados.0.smithi084.stdout:1126: finishing write tid 3 to smithi08434663-18 2024-09-18T14:04:39.431 INFO:tasks.rados.rados.0.smithi084.stdout:1126: finishing write tid 4 to smithi08434663-18 2024-09-18T14:04:39.431 INFO:tasks.rados.rados.0.smithi084.stdout:1126: finishing write tid 5 to smithi08434663-18 2024-09-18T14:04:39.431 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 340 (ObjNum 405 snap 124 seq_num 405) dirty exists 2024-09-18T14:04:39.431 INFO:tasks.rados.rados.0.smithi084.stdout:1126: left oid 18 (ObjNum 405 snap 124 seq_num 405) 2024-09-18T14:04:39.431 INFO:tasks.rados.rados.0.smithi084.stdout:1127: finishing rollback tid 0 to smithi08434663-4 2024-09-18T14:04:39.431 INFO:tasks.rados.rados.0.smithi084.stdout:1127: finishing rollback tid 1 to smithi08434663-4 2024-09-18T14:04:39.431 INFO:tasks.rados.rados.0.smithi084.stdout:1127: finishing rollback tid 2 to smithi08434663-4 2024-09-18T14:04:39.431 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 423 (ObjNum 219 snap 50 seq_num 219) dirty exists 2024-09-18T14:04:39.431 INFO:tasks.rados.rados.0.smithi084.stdout:1122: done (5 left) 2024-09-18T14:04:39.432 INFO:tasks.rados.rados.0.smithi084.stdout:1123: done (4 left) 2024-09-18T14:04:39.432 INFO:tasks.rados.rados.0.smithi084.stdout:1124: done (3 left) 2024-09-18T14:04:39.432 INFO:tasks.rados.rados.0.smithi084.stdout:1125: done (2 left) 2024-09-18T14:04:39.432 INFO:tasks.rados.rados.0.smithi084.stdout:1126: done (1 left) 2024-09-18T14:04:39.432 INFO:tasks.rados.rados.0.smithi084.stdout:1127: done (0 left) 2024-09-18T14:04:39.432 INFO:tasks.rados.rados.0.smithi084.stdout:1128: snap_create 2024-09-18T14:04:40.210 INFO:tasks.rados.rados.0.smithi084.stdout:1128: done (0 left) 2024-09-18T14:04:40.210 INFO:tasks.rados.rados.0.smithi084.stdout:1129: write (excl) oid 18 current snap is 125 2024-09-18T14:04:40.210 INFO:tasks.rados.rados.0.smithi084.stdout:1129: seq_num 406 ranges {423642=717803,1875101=766806} 2024-09-18T14:04:40.216 INFO:tasks.rados.rados.0.smithi084.stdout:1129: writing smithi08434663-18 from 423642 to 1141445 tid 1 2024-09-18T14:04:40.225 INFO:tasks.rados.rados.0.smithi084.stdout:1129: writing smithi08434663-18 from 1875101 to 2641907 tid 2 2024-09-18T14:04:40.225 INFO:tasks.rados.rados.0.smithi084.stdout:1130: read oid 13 snap -1 2024-09-18T14:04:40.225 INFO:tasks.rados.rados.0.smithi084.stdout:1130: expect (ObjNum 401 snap 123 seq_num 401) 2024-09-18T14:04:40.225 INFO:tasks.rados.rados.0.smithi084.stdout:1131: read oid 20 snap -1 2024-09-18T14:04:40.225 INFO:tasks.rados.rados.0.smithi084.stdout:1131: expect deleted 2024-09-18T14:04:40.225 INFO:tasks.rados.rados.0.smithi084.stdout:1132: snap_remove snap 117 2024-09-18T14:04:41.217 INFO:tasks.rados.rados.0.smithi084.stdout:1132: done (3 left) 2024-09-18T14:04:41.217 INFO:tasks.rados.rados.0.smithi084.stdout:1133: read oid 28 snap -1 2024-09-18T14:04:41.218 INFO:tasks.rados.rados.0.smithi084.stdout:1133: expect (ObjNum 337 snap 103 seq_num 337) 2024-09-18T14:04:41.218 INFO:tasks.rados.rados.0.smithi084.stdout:1134: read oid 4 snap -1 2024-09-18T14:04:41.218 INFO:tasks.rados.rados.0.smithi084.stdout:1134: expect (ObjNum 219 snap 50 seq_num 219) 2024-09-18T14:04:41.218 INFO:tasks.rados.rados.0.smithi084.stdout:1135: delete oid 37 current snap is 125 2024-09-18T14:04:41.218 INFO:tasks.rados.rados.0.smithi084.stdout:1129: finishing write tid 1 to smithi08434663-18 2024-09-18T14:04:41.218 INFO:tasks.rados.rados.0.smithi084.stdout:1129: finishing write tid 2 to smithi08434663-18 2024-09-18T14:04:41.218 INFO:tasks.rados.rados.0.smithi084.stdout:1130: expect (ObjNum 401 snap 123 seq_num 401) 2024-09-18T14:04:41.337 INFO:tasks.rados.rados.0.smithi084.stdout:1129: finishing write tid 3 to smithi08434663-18 2024-09-18T14:04:41.337 INFO:tasks.rados.rados.0.smithi084.stdout:1129: finishing write tid 4 to smithi08434663-18 2024-09-18T14:04:41.337 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 343 (ObjNum 406 snap 125 seq_num 406) dirty exists 2024-09-18T14:04:41.337 INFO:tasks.rados.rados.0.smithi084.stdout:1129: left oid 18 (ObjNum 406 snap 125 seq_num 406) 2024-09-18T14:04:41.338 INFO:tasks.rados.rados.0.smithi084.stdout:1129: done (5 left) 2024-09-18T14:04:41.338 INFO:tasks.rados.rados.0.smithi084.stdout:1130: done (4 left) 2024-09-18T14:04:41.338 INFO:tasks.rados.rados.0.smithi084.stdout:1131: done (3 left) 2024-09-18T14:04:41.338 INFO:tasks.rados.rados.0.smithi084.stdout:1135: done (2 left) 2024-09-18T14:04:41.338 INFO:tasks.rados.rados.0.smithi084.stdout:1136: snap_remove snap 109 2024-09-18T14:04:42.228 INFO:tasks.rados.rados.0.smithi084.stdout:1136: done (2 left) 2024-09-18T14:04:42.228 INFO:tasks.rados.rados.0.smithi084.stdout:1137: read oid 6 snap -1 2024-09-18T14:04:42.228 INFO:tasks.rados.rados.0.smithi084.stdout:1137: expect deleted 2024-09-18T14:04:42.228 INFO:tasks.rados.rados.0.smithi084.stdout:1138: snap_remove snap 87 2024-09-18T14:04:43.230 INFO:tasks.rados.rados.0.smithi084.stdout:1138: done (3 left) 2024-09-18T14:04:43.230 INFO:tasks.rados.rados.0.smithi084.stdout:1139: write oid 14 current snap is 125 2024-09-18T14:04:43.230 INFO:tasks.rados.rados.0.smithi084.stdout:1139: seq_num 408 ranges {457256=468301,1592065=770475,2854581=2675} 2024-09-18T14:04:43.235 INFO:tasks.rados.rados.0.smithi084.stdout:1139: writing smithi08434663-14 from 457256 to 925557 tid 1 2024-09-18T14:04:43.244 INFO:tasks.rados.rados.0.smithi084.stdout:1139: writing smithi08434663-14 from 1592065 to 2362540 tid 2 2024-09-18T14:04:43.247 INFO:tasks.rados.rados.0.smithi084.stdout:1139: writing smithi08434663-14 from 2854581 to 2857256 tid 3 2024-09-18T14:04:43.247 INFO:tasks.rados.rados.0.smithi084.stdout:1140: delete oid 41 current snap is 125 2024-09-18T14:04:43.247 INFO:tasks.rados.rados.0.smithi084.stdout:1133: expect (ObjNum 337 snap 103 seq_num 337) 2024-09-18T14:04:43.419 INFO:tasks.rados.rados.0.smithi084.stdout:1134: expect (ObjNum 219 snap 50 seq_num 219) 2024-09-18T14:04:43.583 INFO:tasks.rados.rados.0.smithi084.stdout:1139: finishing write tid 1 to smithi08434663-14 2024-09-18T14:04:43.583 INFO:tasks.rados.rados.0.smithi084.stdout:1139: finishing write tid 2 to smithi08434663-14 2024-09-18T14:04:43.583 INFO:tasks.rados.rados.0.smithi084.stdout:1139: finishing write tid 3 to smithi08434663-14 2024-09-18T14:04:43.583 INFO:tasks.rados.rados.0.smithi084.stdout:1139: finishing write tid 4 to smithi08434663-14 2024-09-18T14:04:43.584 INFO:tasks.rados.rados.0.smithi084.stdout:1139: finishing write tid 5 to smithi08434663-14 2024-09-18T14:04:43.584 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 525 (ObjNum 408 snap 125 seq_num 408) dirty exists 2024-09-18T14:04:43.584 INFO:tasks.rados.rados.0.smithi084.stdout:1139: left oid 14 (ObjNum 408 snap 125 seq_num 408) 2024-09-18T14:04:43.584 INFO:tasks.rados.rados.0.smithi084.stdout:1133: done (4 left) 2024-09-18T14:04:43.584 INFO:tasks.rados.rados.0.smithi084.stdout:1134: done (3 left) 2024-09-18T14:04:43.584 INFO:tasks.rados.rados.0.smithi084.stdout:1137: done (2 left) 2024-09-18T14:04:43.585 INFO:tasks.rados.rados.0.smithi084.stdout:1139: done (1 left) 2024-09-18T14:04:43.585 INFO:tasks.rados.rados.0.smithi084.stdout:1140: done (0 left) 2024-09-18T14:04:43.585 INFO:tasks.rados.rados.0.smithi084.stdout:1141: delete oid 6 current snap is 125 2024-09-18T14:04:43.586 INFO:tasks.rados.rados.0.smithi084.stdout:1141: done (0 left) 2024-09-18T14:04:43.586 INFO:tasks.rados.rados.0.smithi084.stdout:1142: read oid 40 snap 119 2024-09-18T14:04:43.586 INFO:tasks.rados.rados.0.smithi084.stdout:1142: expect (ObjNum 354 snap 109 seq_num 354) 2024-09-18T14:04:43.586 INFO:tasks.rados.rados.0.smithi084.stdout:1143: delete oid 5 current snap is 125 2024-09-18T14:04:43.587 INFO:tasks.rados.rados.0.smithi084.stdout:1143: done (1 left) 2024-09-18T14:04:43.587 INFO:tasks.rados.rados.0.smithi084.stdout:1144: snap_create 2024-09-18T14:04:43.599 INFO:tasks.rados.rados.0.smithi084.stdout:1142: expect (ObjNum 354 snap 109 seq_num 354) 2024-09-18T14:04:44.237 INFO:tasks.rados.rados.0.smithi084.stdout:1142: done (1 left) 2024-09-18T14:04:44.237 INFO:tasks.rados.rados.0.smithi084.stdout:1144: done (0 left) 2024-09-18T14:04:44.237 INFO:tasks.rados.rados.0.smithi084.stdout:1145: copy_from oid 48 from oid 7 current snap is 126 2024-09-18T14:04:44.237 INFO:tasks.rados.rados.0.smithi084.stdout:1146: delete oid 8 current snap is 126 2024-09-18T14:04:44.240 INFO:tasks.rados.rados.0.smithi084.stdout:1146: done (1 left) 2024-09-18T14:04:44.240 INFO:tasks.rados.rados.0.smithi084.stdout:1147: snap_remove snap 122 2024-09-18T14:04:45.238 INFO:tasks.rados.rados.0.smithi084.stdout:1147: done (1 left) 2024-09-18T14:04:45.239 INFO:tasks.rados.rados.0.smithi084.stdout:1148: write oid 34 current snap is 126 2024-09-18T14:04:45.239 INFO:tasks.rados.rados.0.smithi084.stdout:1148: seq_num 413 ranges {541291=560899,1884475=521471} 2024-09-18T14:04:45.245 INFO:tasks.rados.rados.0.smithi084.stdout:1148: writing smithi08434663-34 from 541291 to 1102190 tid 1 2024-09-18T14:04:45.252 INFO:tasks.rados.rados.0.smithi084.stdout:1148: writing smithi08434663-34 from 1884475 to 2405946 tid 2 2024-09-18T14:04:45.252 INFO:tasks.rados.rados.0.smithi084.stdout:1149: snap_remove snap 119 2024-09-18T14:04:45.252 INFO:tasks.rados.rados.0.smithi084.stdout:1145: finishing copy_from racing read to smithi08434663-48 2024-09-18T14:04:46.250 INFO:tasks.rados.rados.0.smithi084.stdout:1149: done (2 left) 2024-09-18T14:04:46.250 INFO:tasks.rados.rados.0.smithi084.stdout:1150: read oid 12 snap -1 2024-09-18T14:04:46.250 INFO:tasks.rados.rados.0.smithi084.stdout:1150: expect (ObjNum 287 snap 74 seq_num 287) 2024-09-18T14:04:46.250 INFO:tasks.rados.rados.0.smithi084.stdout:1151: snap_create 2024-09-18T14:04:46.251 INFO:tasks.rados.rados.0.smithi084.stdout:1145: finishing copy_from to smithi08434663-48 2024-09-18T14:04:46.251 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 187 (ObjNum 287 snap 74 seq_num 287) dirty exists 2024-09-18T14:04:46.251 INFO:tasks.rados.rados.0.smithi084.stdout:1148: finishing write tid 1 to smithi08434663-34 2024-09-18T14:04:46.251 INFO:tasks.rados.rados.0.smithi084.stdout:1148: finishing write tid 2 to smithi08434663-34 2024-09-18T14:04:46.251 INFO:tasks.rados.rados.0.smithi084.stdout:1148: finishing write tid 3 to smithi08434663-34 2024-09-18T14:04:46.251 INFO:tasks.rados.rados.0.smithi084.stdout:1148: finishing write tid 4 to smithi08434663-34 2024-09-18T14:04:46.251 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 452 (ObjNum 413 snap 126 seq_num 413) dirty exists 2024-09-18T14:04:46.251 INFO:tasks.rados.rados.0.smithi084.stdout:1148: left oid 34 (ObjNum 413 snap 126 seq_num 413) 2024-09-18T14:04:46.261 INFO:tasks.rados.rados.0.smithi084.stdout:1150: expect (ObjNum 287 snap 74 seq_num 287) 2024-09-18T14:04:47.197 INFO:tasks.rados.rados.0.smithi084.stdout:1145: done (3 left) 2024-09-18T14:04:47.197 INFO:tasks.rados.rados.0.smithi084.stdout:1148: done (2 left) 2024-09-18T14:04:47.197 INFO:tasks.rados.rados.0.smithi084.stdout:1150: done (1 left) 2024-09-18T14:04:47.197 INFO:tasks.rados.rados.0.smithi084.stdout:1151: done (0 left) 2024-09-18T14:04:47.197 INFO:tasks.rados.rados.0.smithi084.stdout:1152: delete oid 29 current snap is 127 2024-09-18T14:04:47.207 INFO:tasks.rados.rados.0.smithi084.stdout:1152: done (0 left) 2024-09-18T14:04:47.207 INFO:tasks.rados.rados.0.smithi084.stdout:1153: copy_from oid 15 from oid 42 current snap is 127 2024-09-18T14:04:47.207 INFO:tasks.rados.rados.0.smithi084.stdout:1154: snap_create 2024-09-18T14:04:47.218 INFO:tasks.rados.rados.0.smithi084.stdout:1153: finishing copy_from racing read to smithi08434663-15 2024-09-18T14:04:47.220 INFO:tasks.rados.rados.0.smithi084.stdout:1153: finishing copy_from to smithi08434663-15 2024-09-18T14:04:47.220 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 414 (ObjNum 393 snap 120 seq_num 393) dirty exists 2024-09-18T14:04:48.203 INFO:tasks.rados.rados.0.smithi084.stdout:1153: done (1 left) 2024-09-18T14:04:48.203 INFO:tasks.rados.rados.0.smithi084.stdout:1154: done (0 left) 2024-09-18T14:04:48.204 INFO:tasks.rados.rados.0.smithi084.stdout:1155: delete oid 44 current snap is 128 2024-09-18T14:04:48.207 INFO:tasks.rados.rados.0.smithi084.stdout:1155: done (0 left) 2024-09-18T14:04:48.207 INFO:tasks.rados.rados.0.smithi084.stdout:1156: snap_remove snap 125 2024-09-18T14:04:49.209 INFO:tasks.rados.rados.0.smithi084.stdout:1156: done (0 left) 2024-09-18T14:04:49.210 INFO:tasks.rados.rados.0.smithi084.stdout:1157: read oid 42 snap 97 2024-09-18T14:04:49.210 INFO:tasks.rados.rados.0.smithi084.stdout:1157: expect deleted 2024-09-18T14:04:49.210 INFO:tasks.rados.rados.0.smithi084.stdout:1158: rollback oid 43 current snap is 128 2024-09-18T14:04:49.210 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 114 2024-09-18T14:04:49.210 INFO:tasks.rados.rados.0.smithi084.stdout:1159: snap_remove snap 118 2024-09-18T14:04:50.215 INFO:tasks.rados.rados.0.smithi084.stdout:1159: done (2 left) 2024-09-18T14:04:50.216 INFO:tasks.rados.rados.0.smithi084.stdout:1160: read oid 31 snap 97 2024-09-18T14:04:50.216 INFO:tasks.rados.rados.0.smithi084.stdout:1160: expect (ObjNum 273 snap 71 seq_num 273) 2024-09-18T14:04:50.216 INFO:tasks.rados.rados.0.smithi084.stdout:1161: rollback oid 37 current snap is 128 2024-09-18T14:04:50.216 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 37 to 123 2024-09-18T14:04:50.216 INFO:tasks.rados.rados.0.smithi084.stdout:1158: finishing rollback tid 0 to smithi08434663-43 2024-09-18T14:04:50.216 INFO:tasks.rados.rados.0.smithi084.stdout:1158: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:04:50.216 INFO:tasks.rados.rados.0.smithi084.stdout:1158: finishing rollback tid 2 to smithi08434663-43 2024-09-18T14:04:50.216 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 275 (ObjNum 265 snap 67 seq_num 265) dirty exists 2024-09-18T14:04:50.216 INFO:tasks.rados.rados.0.smithi084.stdout:1157: done (3 left) 2024-09-18T14:04:50.216 INFO:tasks.rados.rados.0.smithi084.stdout:1158: done (2 left) 2024-09-18T14:04:50.216 INFO:tasks.rados.rados.0.smithi084.stdout:1162: rollback oid 39 current snap is 128 2024-09-18T14:04:50.216 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 39 to 114 2024-09-18T14:04:50.217 INFO:tasks.rados.rados.0.smithi084.stdout:1163: snap_remove snap 127 2024-09-18T14:04:51.216 INFO:tasks.rados.rados.0.smithi084.stdout:1163: done (3 left) 2024-09-18T14:04:51.216 INFO:tasks.rados.rados.0.smithi084.stdout:1164: write oid 9 current snap is 128 2024-09-18T14:04:51.216 INFO:tasks.rados.rados.0.smithi084.stdout:1164: seq_num 416 ranges {591712=786646,1943568=668798} 2024-09-18T14:04:51.223 INFO:tasks.rados.rados.0.smithi084.stdout:1164: writing smithi08434663-9 from 591712 to 1378358 tid 1 2024-09-18T14:04:51.230 INFO:tasks.rados.rados.0.smithi084.stdout:1164: writing smithi08434663-9 from 1943568 to 2612366 tid 2 2024-09-18T14:04:51.231 INFO:tasks.rados.rados.0.smithi084.stdout:1165: write oid 21 current snap is 128 2024-09-18T14:04:51.231 INFO:tasks.rados.rados.0.smithi084.stdout:1165: seq_num 417 ranges {608519=461895,1602050=528809,2585035=423484} 2024-09-18T14:04:51.236 INFO:tasks.rados.rados.0.smithi084.stdout:1165: writing smithi08434663-21 from 608519 to 1070414 tid 1 2024-09-18T14:04:51.242 INFO:tasks.rados.rados.0.smithi084.stdout:1165: writing smithi08434663-21 from 1602050 to 2130859 tid 2 2024-09-18T14:04:51.247 INFO:tasks.rados.rados.0.smithi084.stdout:1165: writing smithi08434663-21 from 2585035 to 3008519 tid 3 2024-09-18T14:04:51.247 INFO:tasks.rados.rados.0.smithi084.stdout:1166: copy_from oid 15 from oid 24 current snap is 128 2024-09-18T14:04:51.247 INFO:tasks.rados.rados.0.smithi084.stdout:1161: finishing rollback tid 1 to smithi08434663-37 2024-09-18T14:04:51.247 INFO:tasks.rados.rados.0.smithi084.stdout:1161: finishing rollback tid 2 to smithi08434663-37 2024-09-18T14:04:51.247 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 383 (ObjNum 138 snap 29 seq_num 138) dirty exists 2024-09-18T14:04:51.247 INFO:tasks.rados.rados.0.smithi084.stdout:1160: expect (ObjNum 273 snap 71 seq_num 273) 2024-09-18T14:04:51.331 INFO:tasks.rados.rados.0.smithi084.stdout:1162: finishing rollback tid 0 to smithi08434663-39 2024-09-18T14:04:51.331 INFO:tasks.rados.rados.0.smithi084.stdout:1162: finishing rollback tid 1 to smithi08434663-39 2024-09-18T14:04:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:1162: finishing rollback tid 2 to smithi08434663-39 2024-09-18T14:04:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 374 (ObjNum 360 snap 112 seq_num 360) dirty exists 2024-09-18T14:04:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:1164: finishing write tid 1 to smithi08434663-9 2024-09-18T14:04:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:1164: finishing write tid 2 to smithi08434663-9 2024-09-18T14:04:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:1164: finishing write tid 3 to smithi08434663-9 2024-09-18T14:04:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:1160: done (5 left) 2024-09-18T14:04:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:1161: done (4 left) 2024-09-18T14:04:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:1162: done (3 left) 2024-09-18T14:04:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:1167: write (excl) oid 32 current snap is 128 2024-09-18T14:04:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:1167: seq_num 418 ranges {625326=537144,1944179=672467} 2024-09-18T14:04:51.337 INFO:tasks.rados.rados.0.smithi084.stdout:1167: writing smithi08434663-32 from 625326 to 1162470 tid 1 2024-09-18T14:04:51.345 INFO:tasks.rados.rados.0.smithi084.stdout:1167: writing smithi08434663-32 from 1944179 to 2616646 tid 2 2024-09-18T14:04:51.345 INFO:tasks.rados.rados.0.smithi084.stdout:1168: snap_create 2024-09-18T14:04:51.345 INFO:tasks.rados.rados.0.smithi084.stdout:1164: finishing write tid 4 to smithi08434663-9 2024-09-18T14:04:51.346 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 453 (ObjNum 416 snap 128 seq_num 416) dirty exists 2024-09-18T14:04:51.346 INFO:tasks.rados.rados.0.smithi084.stdout:1164: left oid 9 (ObjNum 416 snap 128 seq_num 416) 2024-09-18T14:04:51.346 INFO:tasks.rados.rados.0.smithi084.stdout:1165: finishing write tid 1 to smithi08434663-21 2024-09-18T14:04:51.346 INFO:tasks.rados.rados.0.smithi084.stdout:1165: finishing write tid 2 to smithi08434663-21 2024-09-18T14:04:51.346 INFO:tasks.rados.rados.0.smithi084.stdout:1165: finishing write tid 3 to smithi08434663-21 2024-09-18T14:04:51.346 INFO:tasks.rados.rados.0.smithi084.stdout:1165: finishing write tid 4 to smithi08434663-21 2024-09-18T14:04:51.346 INFO:tasks.rados.rados.0.smithi084.stdout:1165: finishing write tid 5 to smithi08434663-21 2024-09-18T14:04:51.346 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 316 (ObjNum 417 snap 128 seq_num 417) dirty exists 2024-09-18T14:04:51.346 INFO:tasks.rados.rados.0.smithi084.stdout:1165: left oid 21 (ObjNum 417 snap 128 seq_num 417) 2024-09-18T14:04:51.346 INFO:tasks.rados.rados.0.smithi084.stdout:1166: finishing copy_from racing read to smithi08434663-15 2024-09-18T14:04:51.346 INFO:tasks.rados.rados.0.smithi084.stdout:1166: finishing copy_from to smithi08434663-15 2024-09-18T14:04:51.347 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 186 (ObjNum 376 snap 116 seq_num 376) dirty exists 2024-09-18T14:04:51.347 INFO:tasks.rados.rados.0.smithi084.stdout:1167: finishing write tid 1 to smithi08434663-32 2024-09-18T14:04:51.350 INFO:tasks.rados.rados.0.smithi084.stdout:1167: finishing write tid 2 to smithi08434663-32 2024-09-18T14:04:51.350 INFO:tasks.rados.rados.0.smithi084.stdout:1167: finishing write tid 3 to smithi08434663-32 2024-09-18T14:04:51.350 INFO:tasks.rados.rados.0.smithi084.stdout:1167: finishing write tid 4 to smithi08434663-32 2024-09-18T14:04:51.350 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 543 (ObjNum 418 snap 128 seq_num 418) dirty exists 2024-09-18T14:04:51.350 INFO:tasks.rados.rados.0.smithi084.stdout:1167: left oid 32 (ObjNum 418 snap 128 seq_num 418) 2024-09-18T14:04:52.196 INFO:tasks.rados.rados.0.smithi084.stdout:1164: done (4 left) 2024-09-18T14:04:52.197 INFO:tasks.rados.rados.0.smithi084.stdout:1165: done (3 left) 2024-09-18T14:04:52.197 INFO:tasks.rados.rados.0.smithi084.stdout:1166: done (2 left) 2024-09-18T14:04:52.197 INFO:tasks.rados.rados.0.smithi084.stdout:1167: done (1 left) 2024-09-18T14:04:52.197 INFO:tasks.rados.rados.0.smithi084.stdout:1168: done (0 left) 2024-09-18T14:04:52.197 INFO:tasks.rados.rados.0.smithi084.stdout:1169: snap_remove snap 123 2024-09-18T14:04:53.201 INFO:tasks.rados.rados.0.smithi084.stdout:1169: done (0 left) 2024-09-18T14:04:53.201 INFO:tasks.rados.rados.0.smithi084.stdout:1170: delete oid 25 current snap is 129 2024-09-18T14:04:53.217 INFO:tasks.rados.rados.0.smithi084.stdout:1170: done (0 left) 2024-09-18T14:04:53.217 INFO:tasks.rados.rados.0.smithi084.stdout:1171: delete oid 39 current snap is 129 2024-09-18T14:04:53.224 INFO:tasks.rados.rados.0.smithi084.stdout:1171: done (0 left) 2024-09-18T14:04:53.225 INFO:tasks.rados.rados.0.smithi084.stdout:1172: read oid 13 snap -1 2024-09-18T14:04:53.225 INFO:tasks.rados.rados.0.smithi084.stdout:1172: expect (ObjNum 401 snap 123 seq_num 401) 2024-09-18T14:04:53.225 INFO:tasks.rados.rados.0.smithi084.stdout:1173: snap_create 2024-09-18T14:04:53.231 INFO:tasks.rados.rados.0.smithi084.stdout:1172: expect (ObjNum 401 snap 123 seq_num 401) 2024-09-18T14:04:54.204 INFO:tasks.rados.rados.0.smithi084.stdout:1172: done (1 left) 2024-09-18T14:04:54.204 INFO:tasks.rados.rados.0.smithi084.stdout:1173: done (0 left) 2024-09-18T14:04:54.204 INFO:tasks.rados.rados.0.smithi084.stdout:1174: copy_from oid 12 from oid 26 current snap is 130 2024-09-18T14:04:54.204 INFO:tasks.rados.rados.0.smithi084.stdout:1175: read oid 13 snap 115 2024-09-18T14:04:54.204 INFO:tasks.rados.rados.0.smithi084.stdout:1175: expect (ObjNum 359 snap 111 seq_num 359) 2024-09-18T14:04:54.204 INFO:tasks.rados.rados.0.smithi084.stdout:1176: snap_remove snap 114 2024-09-18T14:04:55.202 INFO:tasks.rados.rados.0.smithi084.stdout:1176: done (2 left) 2024-09-18T14:04:55.202 INFO:tasks.rados.rados.0.smithi084.stdout:1177: snap_create 2024-09-18T14:04:55.202 INFO:tasks.rados.rados.0.smithi084.stdout:1175: expect (ObjNum 359 snap 111 seq_num 359) 2024-09-18T14:04:55.269 INFO:tasks.rados.rados.0.smithi084.stdout:1174: finishing copy_from racing read to smithi08434663-12 2024-09-18T14:04:55.269 INFO:tasks.rados.rados.0.smithi084.stdout:1174: finishing copy_from to smithi08434663-12 2024-09-18T14:04:55.269 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 393 (ObjNum 389 snap 119 seq_num 389) dirty exists 2024-09-18T14:04:56.214 INFO:tasks.rados.rados.0.smithi084.stdout:1174: done (2 left) 2024-09-18T14:04:56.215 INFO:tasks.rados.rados.0.smithi084.stdout:1175: done (1 left) 2024-09-18T14:04:56.215 INFO:tasks.rados.rados.0.smithi084.stdout:1177: done (0 left) 2024-09-18T14:04:56.215 INFO:tasks.rados.rados.0.smithi084.stdout:1178: delete oid 45 current snap is 131 2024-09-18T14:04:56.216 INFO:tasks.rados.rados.0.smithi084.stdout:1178: done (0 left) 2024-09-18T14:04:56.216 INFO:tasks.rados.rados.0.smithi084.stdout:1179: delete oid 21 current snap is 131 2024-09-18T14:04:56.219 INFO:tasks.rados.rados.0.smithi084.stdout:1179: done (0 left) 2024-09-18T14:04:56.219 INFO:tasks.rados.rados.0.smithi084.stdout:1180: snap_create 2024-09-18T14:04:57.198 INFO:tasks.rados.rados.0.smithi084.stdout:1180: done (0 left) 2024-09-18T14:04:57.198 INFO:tasks.rados.rados.0.smithi084.stdout:1181: copy_from oid 37 from oid 22 current snap is 132 2024-09-18T14:04:57.198 INFO:tasks.rados.rados.0.smithi084.stdout:1182: read oid 27 snap -1 2024-09-18T14:04:57.199 INFO:tasks.rados.rados.0.smithi084.stdout:1182: expect deleted 2024-09-18T14:04:57.199 INFO:tasks.rados.rados.0.smithi084.stdout:1183: snap_create 2024-09-18T14:04:57.211 INFO:tasks.rados.rados.0.smithi084.stdout:1181: finishing copy_from to smithi08434663-37 2024-09-18T14:04:57.211 INFO:tasks.rados.rados.0.smithi084.stdout:1181: got expected ENOENT (src dne) 2024-09-18T14:04:57.211 INFO:tasks.rados.rados.0.smithi084.stdout:1181: finishing copy_from racing read to smithi08434663-37 2024-09-18T14:04:58.206 INFO:tasks.rados.rados.0.smithi084.stdout:1181: done (2 left) 2024-09-18T14:04:58.206 INFO:tasks.rados.rados.0.smithi084.stdout:1182: done (1 left) 2024-09-18T14:04:58.206 INFO:tasks.rados.rados.0.smithi084.stdout:1183: done (0 left) 2024-09-18T14:04:58.207 INFO:tasks.rados.rados.0.smithi084.stdout:1184: copy_from oid 28 from oid 3 current snap is 133 2024-09-18T14:04:58.207 INFO:tasks.rados.rados.0.smithi084.stdout:1185: snap_remove snap 130 2024-09-18T14:04:59.217 INFO:tasks.rados.rados.0.smithi084.stdout:1185: done (1 left) 2024-09-18T14:04:59.217 INFO:tasks.rados.rados.0.smithi084.stdout:1186: read oid 16 snap -1 2024-09-18T14:04:59.217 INFO:tasks.rados.rados.0.smithi084.stdout:1186: expect deleted 2024-09-18T14:04:59.217 INFO:tasks.rados.rados.0.smithi084.stdout:1187: write oid 42 current snap is 133 2024-09-18T14:04:59.217 INFO:tasks.rados.rados.0.smithi084.stdout:1184: finishing copy_from racing read to smithi08434663-28 2024-09-18T14:04:59.217 INFO:tasks.rados.rados.0.smithi084.stdout:1184: finishing copy_from to smithi08434663-28 2024-09-18T14:04:59.217 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 336 (ObjNum 386 snap 119 seq_num 386) dirty exists 2024-09-18T14:04:59.218 INFO:tasks.rados.rados.0.smithi084.stdout:1187: seq_num 423 ranges {709361=629742,1836589=539816,3033220=76141} 2024-09-18T14:04:59.224 INFO:tasks.rados.rados.0.smithi084.stdout:1187: writing smithi08434663-42 from 709361 to 1339103 tid 1 2024-09-18T14:04:59.229 INFO:tasks.rados.rados.0.smithi084.stdout:1187: writing smithi08434663-42 from 1836589 to 2376405 tid 2 2024-09-18T14:04:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1187: writing smithi08434663-42 from 3033220 to 3109361 tid 3 2024-09-18T14:04:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1184: done (2 left) 2024-09-18T14:04:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1188: delete oid 23 current snap is 133 2024-09-18T14:04:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1187: finishing write tid 1 to smithi08434663-42 2024-09-18T14:04:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1187: finishing write tid 2 to smithi08434663-42 2024-09-18T14:04:59.236 INFO:tasks.rados.rados.0.smithi084.stdout:1186: done (2 left) 2024-09-18T14:04:59.236 INFO:tasks.rados.rados.0.smithi084.stdout:1188: done (1 left) 2024-09-18T14:04:59.236 INFO:tasks.rados.rados.0.smithi084.stdout:1189: read oid 31 snap -1 2024-09-18T14:04:59.236 INFO:tasks.rados.rados.0.smithi084.stdout:1189: expect (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:04:59.236 INFO:tasks.rados.rados.0.smithi084.stdout:1190: read oid 30 snap -1 2024-09-18T14:04:59.236 INFO:tasks.rados.rados.0.smithi084.stdout:1190: expect deleted 2024-09-18T14:04:59.236 INFO:tasks.rados.rados.0.smithi084.stdout:1191: rollback oid 7 current snap is 133 2024-09-18T14:04:59.236 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 7 to 124 2024-09-18T14:04:59.236 INFO:tasks.rados.rados.0.smithi084.stdout:1192: write oid 19 current snap is 133 2024-09-18T14:04:59.236 INFO:tasks.rados.rados.0.smithi084.stdout:1192: seq_num 425 ranges {742975=780240,1953553=427132,2971713=171262} 2024-09-18T14:04:59.244 INFO:tasks.rados.rados.0.smithi084.stdout:1192: writing smithi08434663-19 from 742975 to 1523215 tid 1 2024-09-18T14:04:59.249 INFO:tasks.rados.rados.0.smithi084.stdout:1192: writing smithi08434663-19 from 1953553 to 2380685 tid 2 2024-09-18T14:04:59.253 INFO:tasks.rados.rados.0.smithi084.stdout:1192: writing smithi08434663-19 from 2971713 to 3142975 tid 3 2024-09-18T14:04:59.253 INFO:tasks.rados.rados.0.smithi084.stdout:1193: copy_from oid 26 from oid 3 current snap is 133 2024-09-18T14:04:59.254 INFO:tasks.rados.rados.0.smithi084.stdout:1187: finishing write tid 3 to smithi08434663-42 2024-09-18T14:04:59.254 INFO:tasks.rados.rados.0.smithi084.stdout:1187: finishing write tid 4 to smithi08434663-42 2024-09-18T14:04:59.254 INFO:tasks.rados.rados.0.smithi084.stdout:1187: finishing write tid 5 to smithi08434663-42 2024-09-18T14:04:59.254 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 486 (ObjNum 423 snap 133 seq_num 423) dirty exists 2024-09-18T14:04:59.254 INFO:tasks.rados.rados.0.smithi084.stdout:1187: left oid 42 (ObjNum 423 snap 133 seq_num 423) 2024-09-18T14:04:59.254 INFO:tasks.rados.rados.0.smithi084.stdout:1191: finishing rollback tid 0 to smithi08434663-7 2024-09-18T14:04:59.254 INFO:tasks.rados.rados.0.smithi084.stdout:1191: finishing rollback tid 1 to smithi08434663-7 2024-09-18T14:04:59.254 INFO:tasks.rados.rados.0.smithi084.stdout:1191: finishing rollback tid 2 to smithi08434663-7 2024-09-18T14:04:59.254 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 276 (ObjNum 287 snap 74 seq_num 287) dirty exists 2024-09-18T14:04:59.254 INFO:tasks.rados.rados.0.smithi084.stdout:1189: expect (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:04:59.522 INFO:tasks.rados.rados.0.smithi084.stdout:1192: finishing write tid 1 to smithi08434663-19 2024-09-18T14:04:59.523 INFO:tasks.rados.rados.0.smithi084.stdout:1192: finishing write tid 2 to smithi08434663-19 2024-09-18T14:04:59.523 INFO:tasks.rados.rados.0.smithi084.stdout:1187: done (5 left) 2024-09-18T14:04:59.523 INFO:tasks.rados.rados.0.smithi084.stdout:1189: done (4 left) 2024-09-18T14:04:59.523 INFO:tasks.rados.rados.0.smithi084.stdout:1190: done (3 left) 2024-09-18T14:04:59.523 INFO:tasks.rados.rados.0.smithi084.stdout:1191: done (2 left) 2024-09-18T14:04:59.523 INFO:tasks.rados.rados.0.smithi084.stdout:1194: read oid 15 snap -1 2024-09-18T14:04:59.523 INFO:tasks.rados.rados.0.smithi084.stdout:1194: expect (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:04:59.523 INFO:tasks.rados.rados.0.smithi084.stdout:1195: snap_create 2024-09-18T14:04:59.523 INFO:tasks.rados.rados.0.smithi084.stdout:1192: finishing write tid 3 to smithi08434663-19 2024-09-18T14:04:59.523 INFO:tasks.rados.rados.0.smithi084.stdout:1192: finishing write tid 4 to smithi08434663-19 2024-09-18T14:04:59.523 INFO:tasks.rados.rados.0.smithi084.stdout:1192: finishing write tid 5 to smithi08434663-19 2024-09-18T14:04:59.524 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 562 (ObjNum 425 snap 133 seq_num 425) dirty exists 2024-09-18T14:04:59.524 INFO:tasks.rados.rados.0.smithi084.stdout:1192: left oid 19 (ObjNum 425 snap 133 seq_num 425) 2024-09-18T14:04:59.524 INFO:tasks.rados.rados.0.smithi084.stdout:1193: finishing copy_from racing read to smithi08434663-26 2024-09-18T14:04:59.524 INFO:tasks.rados.rados.0.smithi084.stdout:1193: finishing copy_from to smithi08434663-26 2024-09-18T14:04:59.524 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 336 (ObjNum 386 snap 119 seq_num 386) dirty exists 2024-09-18T14:04:59.527 INFO:tasks.rados.rados.0.smithi084.stdout:1194: expect (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:05:00.228 INFO:tasks.rados.rados.0.smithi084.stdout:1192: done (3 left) 2024-09-18T14:05:00.228 INFO:tasks.rados.rados.0.smithi084.stdout:1193: done (2 left) 2024-09-18T14:05:00.228 INFO:tasks.rados.rados.0.smithi084.stdout:1194: done (1 left) 2024-09-18T14:05:00.228 INFO:tasks.rados.rados.0.smithi084.stdout:1195: done (0 left) 2024-09-18T14:05:00.228 INFO:tasks.rados.rados.0.smithi084.stdout:1196: write oid 42 current snap is 134 2024-09-18T14:05:00.229 INFO:tasks.rados.rados.0.smithi084.stdout:1196: seq_num 426 ranges {759782=571842,2012035=687143} 2024-09-18T14:05:00.235 INFO:tasks.rados.rados.0.smithi084.stdout:1196: writing smithi08434663-42 from 759782 to 1331624 tid 1 2024-09-18T14:05:00.243 INFO:tasks.rados.rados.0.smithi084.stdout:1196: writing smithi08434663-42 from 2012035 to 2699178 tid 2 2024-09-18T14:05:00.243 INFO:tasks.rados.rados.0.smithi084.stdout:1197: rollback oid 39 current snap is 134 2024-09-18T14:05:00.243 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 39 to 97 2024-09-18T14:05:00.243 INFO:tasks.rados.rados.0.smithi084.stdout:1196: finishing write tid 1 to smithi08434663-42 2024-09-18T14:05:00.243 INFO:tasks.rados.rados.0.smithi084.stdout:1198: write (excl) oid 23 current snap is 134 2024-09-18T14:05:00.243 INFO:tasks.rados.rados.0.smithi084.stdout:1198: seq_num 427 ranges {776589=647091,2070517=430801,3026559=150030} 2024-09-18T14:05:00.250 INFO:tasks.rados.rados.0.smithi084.stdout:1198: writing smithi08434663-23 from 776589 to 1423680 tid 1 2024-09-18T14:05:00.256 INFO:tasks.rados.rados.0.smithi084.stdout:1198: writing smithi08434663-23 from 2070517 to 2501318 tid 2 2024-09-18T14:05:00.261 INFO:tasks.rados.rados.0.smithi084.stdout:1198: writing smithi08434663-23 from 3026559 to 3176589 tid 3 2024-09-18T14:05:00.261 INFO:tasks.rados.rados.0.smithi084.stdout:1199: rollback oid 43 current snap is 134 2024-09-18T14:05:00.261 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 132 2024-09-18T14:05:00.261 INFO:tasks.rados.rados.0.smithi084.stdout:1196: finishing write tid 2 to smithi08434663-42 2024-09-18T14:05:00.261 INFO:tasks.rados.rados.0.smithi084.stdout:1196: finishing write tid 3 to smithi08434663-42 2024-09-18T14:05:00.261 INFO:tasks.rados.rados.0.smithi084.stdout:1196: finishing write tid 4 to smithi08434663-42 2024-09-18T14:05:00.261 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 490 (ObjNum 426 snap 134 seq_num 426) dirty exists 2024-09-18T14:05:00.261 INFO:tasks.rados.rados.0.smithi084.stdout:1196: left oid 42 (ObjNum 426 snap 134 seq_num 426) 2024-09-18T14:05:00.262 INFO:tasks.rados.rados.0.smithi084.stdout:1197: finishing rollback tid 1 to smithi08434663-39 2024-09-18T14:05:00.262 INFO:tasks.rados.rados.0.smithi084.stdout:1197: finishing rollback tid 2 to smithi08434663-39 2024-09-18T14:05:00.262 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 377 (ObjNum 306 snap 93 seq_num 306) dirty exists 2024-09-18T14:05:00.262 INFO:tasks.rados.rados.0.smithi084.stdout:1198: finishing write tid 1 to smithi08434663-23 2024-09-18T14:05:00.262 INFO:tasks.rados.rados.0.smithi084.stdout:1198: finishing write tid 2 to smithi08434663-23 2024-09-18T14:05:00.262 INFO:tasks.rados.rados.0.smithi084.stdout:1196: done (3 left) 2024-09-18T14:05:00.262 INFO:tasks.rados.rados.0.smithi084.stdout:1197: done (2 left) 2024-09-18T14:05:00.262 INFO:tasks.rados.rados.0.smithi084.stdout:1200: write (excl) oid 39 current snap is 134 2024-09-18T14:05:00.262 INFO:tasks.rados.rados.0.smithi084.stdout:1200: seq_num 428 ranges {793396=722340,2128999=690812} 2024-09-18T14:05:00.268 INFO:tasks.rados.rados.0.smithi084.stdout:1200: writing smithi08434663-39 from 793396 to 1515736 tid 1 2024-09-18T14:05:00.277 INFO:tasks.rados.rados.0.smithi084.stdout:1200: writing smithi08434663-39 from 2128999 to 2819811 tid 2 2024-09-18T14:05:00.277 INFO:tasks.rados.rados.0.smithi084.stdout:1201: read oid 5 snap -1 2024-09-18T14:05:00.277 INFO:tasks.rados.rados.0.smithi084.stdout:1201: expect deleted 2024-09-18T14:05:00.277 INFO:tasks.rados.rados.0.smithi084.stdout:1202: write oid 9 current snap is 134 2024-09-18T14:05:00.277 INFO:tasks.rados.rados.0.smithi084.stdout:1198: finishing write tid 3 to smithi08434663-23 2024-09-18T14:05:00.277 INFO:tasks.rados.rados.0.smithi084.stdout:1202: seq_num 429 ranges {410203=797589,1787481=434470,2681405=528798} 2024-09-18T14:05:00.282 INFO:tasks.rados.rados.0.smithi084.stdout:1202: writing smithi08434663-9 from 410203 to 1207792 tid 1 2024-09-18T14:05:00.288 INFO:tasks.rados.rados.0.smithi084.stdout:1202: writing smithi08434663-9 from 1787481 to 2221951 tid 2 2024-09-18T14:05:00.294 INFO:tasks.rados.rados.0.smithi084.stdout:1202: writing smithi08434663-9 from 2681405 to 3210203 tid 3 2024-09-18T14:05:00.294 INFO:tasks.rados.rados.0.smithi084.stdout:1203: write (excl) oid 18 current snap is 134 2024-09-18T14:05:00.294 INFO:tasks.rados.rados.0.smithi084.stdout:1198: finishing write tid 4 to smithi08434663-23 2024-09-18T14:05:00.294 INFO:tasks.rados.rados.0.smithi084.stdout:1198: finishing write tid 5 to smithi08434663-23 2024-09-18T14:05:00.294 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 280 (ObjNum 427 snap 134 seq_num 427) dirty exists 2024-09-18T14:05:00.294 INFO:tasks.rados.rados.0.smithi084.stdout:1198: left oid 23 (ObjNum 427 snap 134 seq_num 427) 2024-09-18T14:05:00.294 INFO:tasks.rados.rados.0.smithi084.stdout:1199: finishing rollback tid 0 to smithi08434663-43 2024-09-18T14:05:00.294 INFO:tasks.rados.rados.0.smithi084.stdout:1199: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:05:00.295 INFO:tasks.rados.rados.0.smithi084.stdout:1199: finishing rollback tid 2 to smithi08434663-43 2024-09-18T14:05:00.295 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 278 (ObjNum 265 snap 67 seq_num 265) dirty exists 2024-09-18T14:05:00.295 INFO:tasks.rados.rados.0.smithi084.stdout:1200: finishing write tid 1 to smithi08434663-39 2024-09-18T14:05:00.295 INFO:tasks.rados.rados.0.smithi084.stdout:1203: seq_num 430 ranges {427010=472838,1329610=694481,2708828=518182} 2024-09-18T14:05:00.298 INFO:tasks.rados.rados.0.smithi084.stdout:1203: writing smithi08434663-18 from 427010 to 899848 tid 1 2024-09-18T14:05:00.305 INFO:tasks.rados.rados.0.smithi084.stdout:1203: writing smithi08434663-18 from 1329610 to 2024091 tid 2 2024-09-18T14:05:00.313 INFO:tasks.rados.rados.0.smithi084.stdout:1203: writing smithi08434663-18 from 2708828 to 3227010 tid 3 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:1198: done (5 left) 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:1199: done (4 left) 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:1204: rollback oid 22 current snap is 134 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 22 to 124 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:1200: finishing write tid 2 to smithi08434663-39 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:1200: finishing write tid 3 to smithi08434663-39 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:1200: finishing write tid 4 to smithi08434663-39 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 380 (ObjNum 428 snap 134 seq_num 428) dirty exists 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:1200: left oid 39 (ObjNum 428 snap 134 seq_num 428) 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:1202: finishing write tid 1 to smithi08434663-9 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:1200: done (4 left) 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:1201: done (3 left) 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:1205: read oid 21 snap -1 2024-09-18T14:05:00.314 INFO:tasks.rados.rados.0.smithi084.stdout:1205: expect deleted 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1206: read oid 1 snap -1 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1206: expect (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1207: read oid 2 snap -1 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1207: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1202: finishing write tid 2 to smithi08434663-9 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1202: finishing write tid 3 to smithi08434663-9 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1202: finishing write tid 4 to smithi08434663-9 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1202: finishing write tid 5 to smithi08434663-9 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 465 (ObjNum 429 snap 134 seq_num 429) dirty exists 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1202: left oid 9 (ObjNum 429 snap 134 seq_num 429) 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1203: finishing write tid 1 to smithi08434663-18 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1203: finishing write tid 2 to smithi08434663-18 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1208: read oid 15 snap -1 2024-09-18T14:05:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:1208: expect (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:05:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:1202: done (6 left) 2024-09-18T14:05:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:1209: copy_from oid 12 from oid 30 current snap is 134 2024-09-18T14:05:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:1210: read oid 46 snap 131 2024-09-18T14:05:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:1210: expect deleted 2024-09-18T14:05:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:1211: read oid 35 snap -1 2024-09-18T14:05:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:1211: expect deleted 2024-09-18T14:05:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:1212: read oid 40 snap -1 2024-09-18T14:05:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:1212: expect (ObjNum 354 snap 109 seq_num 354) 2024-09-18T14:05:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:1213: write oid 14 current snap is 134 2024-09-18T14:05:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:1213: seq_num 431 ranges {443817=548087,1788092=438139,3019898=223919} 2024-09-18T14:05:00.319 INFO:tasks.rados.rados.0.smithi084.stdout:1213: writing smithi08434663-14 from 443817 to 991904 tid 1 2024-09-18T14:05:00.326 INFO:tasks.rados.rados.0.smithi084.stdout:1213: writing smithi08434663-14 from 1788092 to 2226231 tid 2 2024-09-18T14:05:00.331 INFO:tasks.rados.rados.0.smithi084.stdout:1213: writing smithi08434663-14 from 3019898 to 3243817 tid 3 2024-09-18T14:05:00.331 INFO:tasks.rados.rados.0.smithi084.stdout:1214: copy_from oid 3 from oid 26 current snap is 134 2024-09-18T14:05:00.332 INFO:tasks.rados.rados.0.smithi084.stdout:1206: expect (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:05:00.601 INFO:tasks.rados.rados.0.smithi084.stdout:1208: expect (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:05:00.733 INFO:tasks.rados.rados.0.smithi084.stdout:1205: done (11 left) 2024-09-18T14:05:00.733 INFO:tasks.rados.rados.0.smithi084.stdout:1206: done (10 left) 2024-09-18T14:05:00.733 INFO:tasks.rados.rados.0.smithi084.stdout:1208: done (9 left) 2024-09-18T14:05:00.733 INFO:tasks.rados.rados.0.smithi084.stdout:1210: done (8 left) 2024-09-18T14:05:00.733 INFO:tasks.rados.rados.0.smithi084.stdout:1215: snap_remove snap 132 2024-09-18T14:05:01.230 INFO:tasks.rados.rados.0.smithi084.stdout:1215: done (8 left) 2024-09-18T14:05:01.230 INFO:tasks.rados.rados.0.smithi084.stdout:1216: write (excl) oid 25 current snap is 134 2024-09-18T14:05:01.230 INFO:tasks.rados.rados.0.smithi084.stdout:1216: seq_num 432 ranges {460624=623336,1846574=698150,3163674=96950} 2024-09-18T14:05:01.235 INFO:tasks.rados.rados.0.smithi084.stdout:1216: writing smithi08434663-25 from 460624 to 1083960 tid 1 2024-09-18T14:05:01.244 INFO:tasks.rados.rados.0.smithi084.stdout:1216: writing smithi08434663-25 from 1846574 to 2544724 tid 2 2024-09-18T14:05:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:1216: writing smithi08434663-25 from 3163674 to 3260624 tid 3 2024-09-18T14:05:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:1211: done (8 left) 2024-09-18T14:05:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:1217: copy_from oid 16 from oid 47 current snap is 134 2024-09-18T14:05:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:1207: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:05:01.325 INFO:tasks.rados.rados.0.smithi084.stdout:1209: finishing copy_from to smithi08434663-12 2024-09-18T14:05:01.325 INFO:tasks.rados.rados.0.smithi084.stdout:1209: got expected ENOENT (src dne) 2024-09-18T14:05:01.325 INFO:tasks.rados.rados.0.smithi084.stdout:1209: finishing copy_from racing read to smithi08434663-12 2024-09-18T14:05:01.325 INFO:tasks.rados.rados.0.smithi084.stdout:1207: done (8 left) 2024-09-18T14:05:01.325 INFO:tasks.rados.rados.0.smithi084.stdout:1209: done (7 left) 2024-09-18T14:05:01.325 INFO:tasks.rados.rados.0.smithi084.stdout:1218: rollback oid 13 current snap is 134 2024-09-18T14:05:01.325 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 13 to 133 2024-09-18T14:05:01.325 INFO:tasks.rados.rados.0.smithi084.stdout:1203: finishing write tid 3 to smithi08434663-18 2024-09-18T14:05:01.325 INFO:tasks.rados.rados.0.smithi084.stdout:1213: finishing write tid 1 to smithi08434663-14 2024-09-18T14:05:01.325 INFO:tasks.rados.rados.0.smithi084.stdout:1203: finishing write tid 4 to smithi08434663-18 2024-09-18T14:05:01.326 INFO:tasks.rados.rados.0.smithi084.stdout:1203: finishing write tid 5 to smithi08434663-18 2024-09-18T14:05:01.326 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 347 (ObjNum 430 snap 134 seq_num 430) dirty exists 2024-09-18T14:05:01.326 INFO:tasks.rados.rados.0.smithi084.stdout:1203: left oid 18 (ObjNum 430 snap 134 seq_num 430) 2024-09-18T14:05:01.326 INFO:tasks.rados.rados.0.smithi084.stdout:1204: finishing rollback tid 1 to smithi08434663-22 2024-09-18T14:05:01.326 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 333 (ObjNum 129 snap 0 seq_num 1568230310) dirty dne 2024-09-18T14:05:01.326 INFO:tasks.rados.rados.0.smithi084.stdout:1212: expect (ObjNum 354 snap 109 seq_num 354) 2024-09-18T14:05:01.448 INFO:tasks.rados.rados.0.smithi084.stdout:1213: finishing write tid 2 to smithi08434663-14 2024-09-18T14:05:01.449 INFO:tasks.rados.rados.0.smithi084.stdout:1203: done (7 left) 2024-09-18T14:05:01.449 INFO:tasks.rados.rados.0.smithi084.stdout:1204: done (6 left) 2024-09-18T14:05:01.449 INFO:tasks.rados.rados.0.smithi084.stdout:1212: done (5 left) 2024-09-18T14:05:01.449 INFO:tasks.rados.rados.0.smithi084.stdout:1219: write oid 10 current snap is 134 2024-09-18T14:05:01.449 INFO:tasks.rados.rados.0.smithi084.stdout:1213: finishing write tid 3 to smithi08434663-14 2024-09-18T14:05:01.449 INFO:tasks.rados.rados.0.smithi084.stdout:1219: seq_num 433 ranges {477431=698585,1905056=441808,3074744=202687} 2024-09-18T14:05:01.454 INFO:tasks.rados.rados.0.smithi084.stdout:1219: writing smithi08434663-10 from 477431 to 1176016 tid 1 2024-09-18T14:05:01.461 INFO:tasks.rados.rados.0.smithi084.stdout:1219: writing smithi08434663-10 from 1905056 to 2346864 tid 2 2024-09-18T14:05:01.467 INFO:tasks.rados.rados.0.smithi084.stdout:1219: writing smithi08434663-10 from 3074744 to 3277431 tid 3 2024-09-18T14:05:01.467 INFO:tasks.rados.rados.0.smithi084.stdout:1220: copy_from oid 46 from oid 31 current snap is 134 2024-09-18T14:05:01.467 INFO:tasks.rados.rados.0.smithi084.stdout:1213: finishing write tid 4 to smithi08434663-14 2024-09-18T14:05:01.467 INFO:tasks.rados.rados.0.smithi084.stdout:1213: finishing write tid 5 to smithi08434663-14 2024-09-18T14:05:01.467 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 567 (ObjNum 431 snap 134 seq_num 431) dirty exists 2024-09-18T14:05:01.467 INFO:tasks.rados.rados.0.smithi084.stdout:1213: left oid 14 (ObjNum 431 snap 134 seq_num 431) 2024-09-18T14:05:01.467 INFO:tasks.rados.rados.0.smithi084.stdout:1214: finishing copy_from racing read to smithi08434663-3 2024-09-18T14:05:01.467 INFO:tasks.rados.rados.0.smithi084.stdout:1214: finishing copy_from to smithi08434663-3 2024-09-18T14:05:01.467 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 336 (ObjNum 386 snap 119 seq_num 386) dirty exists 2024-09-18T14:05:01.467 INFO:tasks.rados.rados.0.smithi084.stdout:1216: finishing write tid 1 to smithi08434663-25 2024-09-18T14:05:01.468 INFO:tasks.rados.rados.0.smithi084.stdout:1216: finishing write tid 2 to smithi08434663-25 2024-09-18T14:05:01.468 INFO:tasks.rados.rados.0.smithi084.stdout:1216: finishing write tid 3 to smithi08434663-25 2024-09-18T14:05:01.468 INFO:tasks.rados.rados.0.smithi084.stdout:1216: finishing write tid 4 to smithi08434663-25 2024-09-18T14:05:01.468 INFO:tasks.rados.rados.0.smithi084.stdout:1213: done (6 left) 2024-09-18T14:05:01.468 INFO:tasks.rados.rados.0.smithi084.stdout:1214: done (5 left) 2024-09-18T14:05:01.468 INFO:tasks.rados.rados.0.smithi084.stdout:1221: snap_create 2024-09-18T14:05:01.468 INFO:tasks.rados.rados.0.smithi084.stdout:1216: finishing write tid 5 to smithi08434663-25 2024-09-18T14:05:01.468 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 332 (ObjNum 432 snap 134 seq_num 432) dirty exists 2024-09-18T14:05:01.468 INFO:tasks.rados.rados.0.smithi084.stdout:1216: left oid 25 (ObjNum 432 snap 134 seq_num 432) 2024-09-18T14:05:01.468 INFO:tasks.rados.rados.0.smithi084.stdout:1217: finishing copy_from racing read to smithi08434663-16 2024-09-18T14:05:01.469 INFO:tasks.rados.rados.0.smithi084.stdout:1217: finishing copy_from to smithi08434663-16 2024-09-18T14:05:01.469 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 257 (ObjNum 373 snap 115 seq_num 373) dirty exists 2024-09-18T14:05:01.469 INFO:tasks.rados.rados.0.smithi084.stdout:1218: finishing rollback tid 0 to smithi08434663-13 2024-09-18T14:05:01.469 INFO:tasks.rados.rados.0.smithi084.stdout:1218: finishing rollback tid 1 to smithi08434663-13 2024-09-18T14:05:01.469 INFO:tasks.rados.rados.0.smithi084.stdout:1218: finishing rollback tid 2 to smithi08434663-13 2024-09-18T14:05:01.469 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 407 (ObjNum 401 snap 123 seq_num 401) dirty exists 2024-09-18T14:05:01.469 INFO:tasks.rados.rados.0.smithi084.stdout:1219: finishing write tid 1 to smithi08434663-10 2024-09-18T14:05:01.469 INFO:tasks.rados.rados.0.smithi084.stdout:1219: finishing write tid 2 to smithi08434663-10 2024-09-18T14:05:01.470 INFO:tasks.rados.rados.0.smithi084.stdout:1219: finishing write tid 3 to smithi08434663-10 2024-09-18T14:05:01.473 INFO:tasks.rados.rados.0.smithi084.stdout:1219: finishing write tid 4 to smithi08434663-10 2024-09-18T14:05:01.473 INFO:tasks.rados.rados.0.smithi084.stdout:1219: finishing write tid 5 to smithi08434663-10 2024-09-18T14:05:01.473 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 449 (ObjNum 433 snap 134 seq_num 433) dirty exists 2024-09-18T14:05:01.473 INFO:tasks.rados.rados.0.smithi084.stdout:1219: left oid 10 (ObjNum 433 snap 134 seq_num 433) 2024-09-18T14:05:01.477 INFO:tasks.rados.rados.0.smithi084.stdout:1220: finishing copy_from racing read to smithi08434663-46 2024-09-18T14:05:01.480 INFO:tasks.rados.rados.0.smithi084.stdout:1220: finishing copy_from to smithi08434663-46 2024-09-18T14:05:01.480 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 167 (ObjNum 281 snap 71 seq_num 281) dirty exists 2024-09-18T14:05:02.234 INFO:tasks.rados.rados.0.smithi084.stdout:1216: done (5 left) 2024-09-18T14:05:02.235 INFO:tasks.rados.rados.0.smithi084.stdout:1217: done (4 left) 2024-09-18T14:05:02.235 INFO:tasks.rados.rados.0.smithi084.stdout:1218: done (3 left) 2024-09-18T14:05:02.235 INFO:tasks.rados.rados.0.smithi084.stdout:1219: done (2 left) 2024-09-18T14:05:02.235 INFO:tasks.rados.rados.0.smithi084.stdout:1220: done (1 left) 2024-09-18T14:05:02.235 INFO:tasks.rados.rados.0.smithi084.stdout:1221: done (0 left) 2024-09-18T14:05:02.235 INFO:tasks.rados.rados.0.smithi084.stdout:1222: read oid 30 snap -1 2024-09-18T14:05:02.235 INFO:tasks.rados.rados.0.smithi084.stdout:1222: expect deleted 2024-09-18T14:05:02.235 INFO:tasks.rados.rados.0.smithi084.stdout:1223: delete oid 48 current snap is 135 2024-09-18T14:05:02.245 INFO:tasks.rados.rados.0.smithi084.stdout:1222: done (1 left) 2024-09-18T14:05:02.245 INFO:tasks.rados.rados.0.smithi084.stdout:1223: done (0 left) 2024-09-18T14:05:02.245 INFO:tasks.rados.rados.0.smithi084.stdout:1224: read oid 23 snap -1 2024-09-18T14:05:02.245 INFO:tasks.rados.rados.0.smithi084.stdout:1224: expect (ObjNum 427 snap 134 seq_num 427) 2024-09-18T14:05:02.245 INFO:tasks.rados.rados.0.smithi084.stdout:1225: write oid 33 current snap is 135 2024-09-18T14:05:02.245 INFO:tasks.rados.rados.0.smithi084.stdout:1225: seq_num 435 ranges {511045=565436,1622020=445477,2729590=556855} 2024-09-18T14:05:02.250 INFO:tasks.rados.rados.0.smithi084.stdout:1225: writing smithi08434663-33 from 511045 to 1076481 tid 1 2024-09-18T14:05:02.255 INFO:tasks.rados.rados.0.smithi084.stdout:1225: writing smithi08434663-33 from 1622020 to 2067497 tid 2 2024-09-18T14:05:02.262 INFO:tasks.rados.rados.0.smithi084.stdout:1225: writing smithi08434663-33 from 2729590 to 3286445 tid 3 2024-09-18T14:05:02.262 INFO:tasks.rados.rados.0.smithi084.stdout:1226: delete oid 21 current snap is 135 2024-09-18T14:05:02.262 INFO:tasks.rados.rados.0.smithi084.stdout:1224: expect (ObjNum 427 snap 134 seq_num 427) 2024-09-18T14:05:02.366 INFO:tasks.rados.rados.0.smithi084.stdout:1225: finishing write tid 1 to smithi08434663-33 2024-09-18T14:05:02.366 INFO:tasks.rados.rados.0.smithi084.stdout:1225: finishing write tid 2 to smithi08434663-33 2024-09-18T14:05:02.366 INFO:tasks.rados.rados.0.smithi084.stdout:1224: done (2 left) 2024-09-18T14:05:02.366 INFO:tasks.rados.rados.0.smithi084.stdout:1226: done (1 left) 2024-09-18T14:05:02.367 INFO:tasks.rados.rados.0.smithi084.stdout:1227: copy_from oid 46 from oid 29 current snap is 135 2024-09-18T14:05:02.367 INFO:tasks.rados.rados.0.smithi084.stdout:1225: finishing write tid 3 to smithi08434663-33 2024-09-18T14:05:02.367 INFO:tasks.rados.rados.0.smithi084.stdout:1225: finishing write tid 4 to smithi08434663-33 2024-09-18T14:05:02.367 INFO:tasks.rados.rados.0.smithi084.stdout:1225: finishing write tid 5 to smithi08434663-33 2024-09-18T14:05:02.367 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 387 (ObjNum 435 snap 135 seq_num 435) dirty exists 2024-09-18T14:05:02.367 INFO:tasks.rados.rados.0.smithi084.stdout:1225: left oid 33 (ObjNum 435 snap 135 seq_num 435) 2024-09-18T14:05:02.367 INFO:tasks.rados.rados.0.smithi084.stdout:1225: done (1 left) 2024-09-18T14:05:02.367 INFO:tasks.rados.rados.0.smithi084.stdout:1228: write oid 34 current snap is 135 2024-09-18T14:05:02.367 INFO:tasks.rados.rados.0.smithi084.stdout:1228: seq_num 437 ranges {544659=715934,1738984=449146,2784436=560223} 2024-09-18T14:05:02.372 INFO:tasks.rados.rados.0.smithi084.stdout:1228: writing smithi08434663-34 from 544659 to 1260593 tid 1 2024-09-18T14:05:02.378 INFO:tasks.rados.rados.0.smithi084.stdout:1228: writing smithi08434663-34 from 1738984 to 2188130 tid 2 2024-09-18T14:05:02.384 INFO:tasks.rados.rados.0.smithi084.stdout:1228: writing smithi08434663-34 from 2784436 to 3344659 tid 3 2024-09-18T14:05:02.384 INFO:tasks.rados.rados.0.smithi084.stdout:1229: delete oid 18 current snap is 135 2024-09-18T14:05:02.384 INFO:tasks.rados.rados.0.smithi084.stdout:1227: finishing copy_from to smithi08434663-46 2024-09-18T14:05:02.384 INFO:tasks.rados.rados.0.smithi084.stdout:1227: got expected ENOENT (src dne) 2024-09-18T14:05:02.384 INFO:tasks.rados.rados.0.smithi084.stdout:1227: finishing copy_from racing read to smithi08434663-46 2024-09-18T14:05:02.384 INFO:tasks.rados.rados.0.smithi084.stdout:1228: finishing write tid 1 to smithi08434663-34 2024-09-18T14:05:02.384 INFO:tasks.rados.rados.0.smithi084.stdout:1228: finishing write tid 2 to smithi08434663-34 2024-09-18T14:05:02.387 INFO:tasks.rados.rados.0.smithi084.stdout:1228: finishing write tid 3 to smithi08434663-34 2024-09-18T14:05:02.387 INFO:tasks.rados.rados.0.smithi084.stdout:1228: finishing write tid 4 to smithi08434663-34 2024-09-18T14:05:02.388 INFO:tasks.rados.rados.0.smithi084.stdout:1228: finishing write tid 5 to smithi08434663-34 2024-09-18T14:05:02.388 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 497 (ObjNum 437 snap 135 seq_num 437) dirty exists 2024-09-18T14:05:02.388 INFO:tasks.rados.rados.0.smithi084.stdout:1228: left oid 34 (ObjNum 437 snap 135 seq_num 437) 2024-09-18T14:05:02.388 INFO:tasks.rados.rados.0.smithi084.stdout:1227: done (2 left) 2024-09-18T14:05:02.388 INFO:tasks.rados.rados.0.smithi084.stdout:1228: done (1 left) 2024-09-18T14:05:02.388 INFO:tasks.rados.rados.0.smithi084.stdout:1229: done (0 left) 2024-09-18T14:05:02.388 INFO:tasks.rados.rados.0.smithi084.stdout:1230: snap_remove snap 129 2024-09-18T14:05:03.239 INFO:tasks.rados.rados.0.smithi084.stdout:1230: done (0 left) 2024-09-18T14:05:03.240 INFO:tasks.rados.rados.0.smithi084.stdout:1231: snap_create 2024-09-18T14:05:04.254 INFO:tasks.rados.rados.0.smithi084.stdout:1231: done (0 left) 2024-09-18T14:05:04.254 INFO:tasks.rados.rados.0.smithi084.stdout:1232: read oid 5 snap -1 2024-09-18T14:05:04.254 INFO:tasks.rados.rados.0.smithi084.stdout:1232: expect deleted 2024-09-18T14:05:04.254 INFO:tasks.rados.rados.0.smithi084.stdout:1233: snap_create 2024-09-18T14:05:05.257 INFO:tasks.rados.rados.0.smithi084.stdout:1232: done (1 left) 2024-09-18T14:05:05.257 INFO:tasks.rados.rados.0.smithi084.stdout:1233: done (0 left) 2024-09-18T14:05:05.257 INFO:tasks.rados.rados.0.smithi084.stdout:1234: read oid 50 snap -1 2024-09-18T14:05:05.257 INFO:tasks.rados.rados.0.smithi084.stdout:1234: expect deleted 2024-09-18T14:05:05.257 INFO:tasks.rados.rados.0.smithi084.stdout:1235: write oid 23 current snap is 137 2024-09-18T14:05:05.258 INFO:tasks.rados.rados.0.smithi084.stdout:1235: seq_num 439 ranges {578273=466432,1739595=452815,2722929=655344} 2024-09-18T14:05:05.262 INFO:tasks.rados.rados.0.smithi084.stdout:1235: writing smithi08434663-23 from 578273 to 1044705 tid 1 2024-09-18T14:05:05.269 INFO:tasks.rados.rados.0.smithi084.stdout:1235: writing smithi08434663-23 from 1739595 to 2192410 tid 2 2024-09-18T14:05:05.275 INFO:tasks.rados.rados.0.smithi084.stdout:1235: writing smithi08434663-23 from 2722929 to 3378273 tid 3 2024-09-18T14:05:05.276 INFO:tasks.rados.rados.0.smithi084.stdout:1236: snap_create 2024-09-18T14:05:05.276 INFO:tasks.rados.rados.0.smithi084.stdout:1235: finishing write tid 1 to smithi08434663-23 2024-09-18T14:05:05.276 INFO:tasks.rados.rados.0.smithi084.stdout:1235: finishing write tid 2 to smithi08434663-23 2024-09-18T14:05:05.279 INFO:tasks.rados.rados.0.smithi084.stdout:1235: finishing write tid 3 to smithi08434663-23 2024-09-18T14:05:05.279 INFO:tasks.rados.rados.0.smithi084.stdout:1235: finishing write tid 4 to smithi08434663-23 2024-09-18T14:05:05.279 INFO:tasks.rados.rados.0.smithi084.stdout:1235: finishing write tid 5 to smithi08434663-23 2024-09-18T14:05:05.279 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 284 (ObjNum 439 snap 137 seq_num 439) dirty exists 2024-09-18T14:05:05.279 INFO:tasks.rados.rados.0.smithi084.stdout:1235: left oid 23 (ObjNum 439 snap 137 seq_num 439) 2024-09-18T14:05:06.273 INFO:tasks.rados.rados.0.smithi084.stdout:1234: done (2 left) 2024-09-18T14:05:06.274 INFO:tasks.rados.rados.0.smithi084.stdout:1235: done (1 left) 2024-09-18T14:05:06.274 INFO:tasks.rados.rados.0.smithi084.stdout:1236: done (0 left) 2024-09-18T14:05:06.274 INFO:tasks.rados.rados.0.smithi084.stdout:1237: write oid 41 current snap is 138 2024-09-18T14:05:06.274 INFO:tasks.rados.rados.0.smithi084.stdout:1237: seq_num 440 ranges {595080=541681,1798077=596473,3150352=244728} 2024-09-18T14:05:06.279 INFO:tasks.rados.rados.0.smithi084.stdout:1237: writing smithi08434663-41 from 595080 to 1136761 tid 1 2024-09-18T14:05:06.285 INFO:tasks.rados.rados.0.smithi084.stdout:1237: writing smithi08434663-41 from 1798077 to 2394550 tid 2 2024-09-18T14:05:06.291 INFO:tasks.rados.rados.0.smithi084.stdout:1237: writing smithi08434663-41 from 3150352 to 3395080 tid 3 2024-09-18T14:05:06.292 INFO:tasks.rados.rados.0.smithi084.stdout:1238: read oid 24 snap -1 2024-09-18T14:05:06.292 INFO:tasks.rados.rados.0.smithi084.stdout:1238: expect (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:05:06.292 INFO:tasks.rados.rados.0.smithi084.stdout:1239: read oid 29 snap -1 2024-09-18T14:05:06.292 INFO:tasks.rados.rados.0.smithi084.stdout:1239: expect deleted 2024-09-18T14:05:06.292 INFO:tasks.rados.rados.0.smithi084.stdout:1240: write (excl) oid 44 current snap is 138 2024-09-18T14:05:06.292 INFO:tasks.rados.rados.0.smithi084.stdout:1237: finishing write tid 1 to smithi08434663-41 2024-09-18T14:05:06.292 INFO:tasks.rados.rados.0.smithi084.stdout:1237: finishing write tid 2 to smithi08434663-41 2024-09-18T14:05:06.292 INFO:tasks.rados.rados.0.smithi084.stdout:1240: seq_num 441 ranges {611887=733283,1972912=456484,2894128=517759} 2024-09-18T14:05:06.299 INFO:tasks.rados.rados.0.smithi084.stdout:1240: writing smithi08434663-44 from 611887 to 1345170 tid 1 2024-09-18T14:05:06.305 INFO:tasks.rados.rados.0.smithi084.stdout:1240: writing smithi08434663-44 from 1972912 to 2429396 tid 2 2024-09-18T14:05:06.312 INFO:tasks.rados.rados.0.smithi084.stdout:1240: writing smithi08434663-44 from 2894128 to 3411887 tid 3 2024-09-18T14:05:06.312 INFO:tasks.rados.rados.0.smithi084.stdout:1241: copy_from oid 8 from oid 47 current snap is 138 2024-09-18T14:05:06.312 INFO:tasks.rados.rados.0.smithi084.stdout:1237: finishing write tid 3 to smithi08434663-41 2024-09-18T14:05:06.312 INFO:tasks.rados.rados.0.smithi084.stdout:1237: finishing write tid 4 to smithi08434663-41 2024-09-18T14:05:06.313 INFO:tasks.rados.rados.0.smithi084.stdout:1237: finishing write tid 5 to smithi08434663-41 2024-09-18T14:05:06.313 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 505 (ObjNum 440 snap 138 seq_num 440) dirty exists 2024-09-18T14:05:06.313 INFO:tasks.rados.rados.0.smithi084.stdout:1237: left oid 41 (ObjNum 440 snap 138 seq_num 440) 2024-09-18T14:05:06.313 INFO:tasks.rados.rados.0.smithi084.stdout:1238: expect (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:05:06.445 INFO:tasks.rados.rados.0.smithi084.stdout:1240: finishing write tid 1 to smithi08434663-44 2024-09-18T14:05:06.445 INFO:tasks.rados.rados.0.smithi084.stdout:1240: finishing write tid 2 to smithi08434663-44 2024-09-18T14:05:06.445 INFO:tasks.rados.rados.0.smithi084.stdout:1240: finishing write tid 3 to smithi08434663-44 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:1240: finishing write tid 4 to smithi08434663-44 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:1240: finishing write tid 5 to smithi08434663-44 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 190 (ObjNum 441 snap 138 seq_num 441) dirty exists 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:1240: left oid 44 (ObjNum 441 snap 138 seq_num 441) 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:1241: finishing copy_from racing read to smithi08434663-8 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:1241: finishing copy_from to smithi08434663-8 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 257 (ObjNum 373 snap 115 seq_num 373) dirty exists 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:1237: done (4 left) 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:1238: done (3 left) 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:1239: done (2 left) 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:1240: done (1 left) 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:1241: done (0 left) 2024-09-18T14:05:06.446 INFO:tasks.rados.rados.0.smithi084.stdout:1242: snap_remove snap 135 2024-09-18T14:05:07.201 INFO:tasks.rados.rados.0.smithi084.stdout:1242: done (0 left) 2024-09-18T14:05:07.201 INFO:tasks.rados.rados.0.smithi084.stdout:1243: read oid 19 snap -1 2024-09-18T14:05:07.201 INFO:tasks.rados.rados.0.smithi084.stdout:1243: expect (ObjNum 425 snap 133 seq_num 425) 2024-09-18T14:05:07.201 INFO:tasks.rados.rados.0.smithi084.stdout:1244: delete oid 34 current snap is 138 2024-09-18T14:05:07.208 INFO:tasks.rados.rados.0.smithi084.stdout:1243: expect (ObjNum 425 snap 133 seq_num 425) 2024-09-18T14:05:07.408 INFO:tasks.rados.rados.0.smithi084.stdout:1243: done (1 left) 2024-09-18T14:05:07.408 INFO:tasks.rados.rados.0.smithi084.stdout:1244: done (0 left) 2024-09-18T14:05:07.408 INFO:tasks.rados.rados.0.smithi084.stdout:1245: read oid 24 snap -1 2024-09-18T14:05:07.408 INFO:tasks.rados.rados.0.smithi084.stdout:1245: expect (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:05:07.408 INFO:tasks.rados.rados.0.smithi084.stdout:1246: read oid 42 snap -1 2024-09-18T14:05:07.408 INFO:tasks.rados.rados.0.smithi084.stdout:1246: expect (ObjNum 426 snap 134 seq_num 426) 2024-09-18T14:05:07.408 INFO:tasks.rados.rados.0.smithi084.stdout:1247: write oid 10 current snap is 138 2024-09-18T14:05:07.408 INFO:tasks.rados.rados.0.smithi084.stdout:1247: seq_num 443 ranges {645501=483781,1573523=460153,2548974=763384} 2024-09-18T14:05:07.413 INFO:tasks.rados.rados.0.smithi084.stdout:1247: writing smithi08434663-10 from 645501 to 1129282 tid 1 2024-09-18T14:05:07.420 INFO:tasks.rados.rados.0.smithi084.stdout:1247: writing smithi08434663-10 from 1573523 to 2033676 tid 2 2024-09-18T14:05:07.427 INFO:tasks.rados.rados.0.smithi084.stdout:1247: writing smithi08434663-10 from 2548974 to 3312358 tid 3 2024-09-18T14:05:07.427 INFO:tasks.rados.rados.0.smithi084.stdout:1248: write oid 13 current snap is 138 2024-09-18T14:05:07.427 INFO:tasks.rados.rados.0.smithi084.stdout:1248: seq_num 444 ranges {662308=559030,1632005=603811,2860044=602264} 2024-09-18T14:05:07.434 INFO:tasks.rados.rados.0.smithi084.stdout:1248: writing smithi08434663-13 from 662308 to 1221338 tid 1 2024-09-18T14:05:07.439 INFO:tasks.rados.rados.0.smithi084.stdout:1248: writing smithi08434663-13 from 1632005 to 2235816 tid 2 2024-09-18T14:05:07.446 INFO:tasks.rados.rados.0.smithi084.stdout:1248: writing smithi08434663-13 from 2860044 to 3462308 tid 3 2024-09-18T14:05:07.446 INFO:tasks.rados.rados.0.smithi084.stdout:1249: rollback oid 14 current snap is 138 2024-09-18T14:05:07.446 INFO:tasks.rados.rados.0.smithi084.stdout:1245: expect (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:05:07.577 INFO:tasks.rados.rados.0.smithi084.stdout:1246: expect (ObjNum 426 snap 134 seq_num 426) 2024-09-18T14:05:07.821 INFO:tasks.rados.rados.0.smithi084.stdout:1247: finishing write tid 1 to smithi08434663-10 2024-09-18T14:05:07.821 INFO:tasks.rados.rados.0.smithi084.stdout:1247: finishing write tid 2 to smithi08434663-10 2024-09-18T14:05:07.821 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 14 to 120 2024-09-18T14:05:07.821 INFO:tasks.rados.rados.0.smithi084.stdout:1247: finishing write tid 3 to smithi08434663-10 2024-09-18T14:05:07.821 INFO:tasks.rados.rados.0.smithi084.stdout:1247: finishing write tid 4 to smithi08434663-10 2024-09-18T14:05:07.821 INFO:tasks.rados.rados.0.smithi084.stdout:1247: finishing write tid 5 to smithi08434663-10 2024-09-18T14:05:07.821 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 458 (ObjNum 443 snap 138 seq_num 443) dirty exists 2024-09-18T14:05:07.821 INFO:tasks.rados.rados.0.smithi084.stdout:1247: left oid 10 (ObjNum 443 snap 138 seq_num 443) 2024-09-18T14:05:07.821 INFO:tasks.rados.rados.0.smithi084.stdout:1248: finishing write tid 1 to smithi08434663-13 2024-09-18T14:05:07.821 INFO:tasks.rados.rados.0.smithi084.stdout:1248: finishing write tid 2 to smithi08434663-13 2024-09-18T14:05:07.822 INFO:tasks.rados.rados.0.smithi084.stdout:1248: finishing write tid 3 to smithi08434663-13 2024-09-18T14:05:07.822 INFO:tasks.rados.rados.0.smithi084.stdout:1248: finishing write tid 4 to smithi08434663-13 2024-09-18T14:05:07.822 INFO:tasks.rados.rados.0.smithi084.stdout:1248: finishing write tid 5 to smithi08434663-13 2024-09-18T14:05:07.822 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 414 (ObjNum 444 snap 138 seq_num 444) dirty exists 2024-09-18T14:05:07.822 INFO:tasks.rados.rados.0.smithi084.stdout:1248: left oid 13 (ObjNum 444 snap 138 seq_num 444) 2024-09-18T14:05:07.822 INFO:tasks.rados.rados.0.smithi084.stdout:1245: done (4 left) 2024-09-18T14:05:07.822 INFO:tasks.rados.rados.0.smithi084.stdout:1246: done (3 left) 2024-09-18T14:05:07.822 INFO:tasks.rados.rados.0.smithi084.stdout:1247: done (2 left) 2024-09-18T14:05:07.822 INFO:tasks.rados.rados.0.smithi084.stdout:1248: done (1 left) 2024-09-18T14:05:07.822 INFO:tasks.rados.rados.0.smithi084.stdout:1250: write oid 22 current snap is 138 2024-09-18T14:05:07.822 INFO:tasks.rados.rados.0.smithi084.stdout:1250: seq_num 445 ranges {679115=634279,2090487=463822,3003820=475295} 2024-09-18T14:05:07.827 INFO:tasks.rados.rados.0.smithi084.stdout:1250: writing smithi08434663-22 from 679115 to 1313394 tid 1 2024-09-18T14:05:07.833 INFO:tasks.rados.rados.0.smithi084.stdout:1250: writing smithi08434663-22 from 2090487 to 2554309 tid 2 2024-09-18T14:05:07.839 INFO:tasks.rados.rados.0.smithi084.stdout:1250: writing smithi08434663-22 from 3003820 to 3479115 tid 3 2024-09-18T14:05:07.839 INFO:tasks.rados.rados.0.smithi084.stdout:1251: read oid 32 snap -1 2024-09-18T14:05:07.839 INFO:tasks.rados.rados.0.smithi084.stdout:1251: expect (ObjNum 418 snap 128 seq_num 418) 2024-09-18T14:05:07.839 INFO:tasks.rados.rados.0.smithi084.stdout:1252: copy_from oid 1 from oid 43 current snap is 138 2024-09-18T14:05:07.839 INFO:tasks.rados.rados.0.smithi084.stdout:1249: finishing rollback tid 0 to smithi08434663-14 2024-09-18T14:05:07.839 INFO:tasks.rados.rados.0.smithi084.stdout:1249: finishing rollback tid 1 to smithi08434663-14 2024-09-18T14:05:07.839 INFO:tasks.rados.rados.0.smithi084.stdout:1249: finishing rollback tid 2 to smithi08434663-14 2024-09-18T14:05:07.840 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 585 (ObjNum 387 snap 119 seq_num 387) dirty exists 2024-09-18T14:05:07.840 INFO:tasks.rados.rados.0.smithi084.stdout:1250: finishing write tid 1 to smithi08434663-22 2024-09-18T14:05:07.840 INFO:tasks.rados.rados.0.smithi084.stdout:1250: finishing write tid 2 to smithi08434663-22 2024-09-18T14:05:07.840 INFO:tasks.rados.rados.0.smithi084.stdout:1249: done (3 left) 2024-09-18T14:05:07.840 INFO:tasks.rados.rados.0.smithi084.stdout:1253: delete oid 25 current snap is 138 2024-09-18T14:05:07.842 INFO:tasks.rados.rados.0.smithi084.stdout:1250: finishing write tid 3 to smithi08434663-22 2024-09-18T14:05:07.842 INFO:tasks.rados.rados.0.smithi084.stdout:1250: finishing write tid 4 to smithi08434663-22 2024-09-18T14:05:07.843 INFO:tasks.rados.rados.0.smithi084.stdout:1253: done (3 left) 2024-09-18T14:05:07.843 INFO:tasks.rados.rados.0.smithi084.stdout:1254: write oid 15 current snap is 138 2024-09-18T14:05:07.843 INFO:tasks.rados.rados.0.smithi084.stdout:1254: seq_num 447 ranges {712729=784777,2091098=467491,3342313=170416} 2024-09-18T14:05:07.851 INFO:tasks.rados.rados.0.smithi084.stdout:1254: writing smithi08434663-15 from 712729 to 1497506 tid 1 2024-09-18T14:05:07.856 INFO:tasks.rados.rados.0.smithi084.stdout:1254: writing smithi08434663-15 from 2091098 to 2558589 tid 2 2024-09-18T14:05:07.862 INFO:tasks.rados.rados.0.smithi084.stdout:1254: writing smithi08434663-15 from 3342313 to 3512729 tid 3 2024-09-18T14:05:07.862 INFO:tasks.rados.rados.0.smithi084.stdout:1255: write oid 39 current snap is 138 2024-09-18T14:05:07.862 INFO:tasks.rados.rados.0.smithi084.stdout:1255: seq_num 448 ranges {729536=460026,1749580=611149,2853383=452450} 2024-09-18T14:05:07.868 INFO:tasks.rados.rados.0.smithi084.stdout:1255: writing smithi08434663-39 from 729536 to 1189562 tid 1 2024-09-18T14:05:07.875 INFO:tasks.rados.rados.0.smithi084.stdout:1255: writing smithi08434663-39 from 1749580 to 2360729 tid 2 2024-09-18T14:05:07.880 INFO:tasks.rados.rados.0.smithi084.stdout:1255: writing smithi08434663-39 from 2853383 to 3305833 tid 3 2024-09-18T14:05:07.881 INFO:tasks.rados.rados.0.smithi084.stdout:1256: write (excl) oid 13 current snap is 138 2024-09-18T14:05:07.881 INFO:tasks.rados.rados.0.smithi084.stdout:1250: finishing write tid 5 to smithi08434663-22 2024-09-18T14:05:07.881 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 352 (ObjNum 445 snap 138 seq_num 445) dirty exists 2024-09-18T14:05:07.881 INFO:tasks.rados.rados.0.smithi084.stdout:1250: left oid 22 (ObjNum 445 snap 138 seq_num 445) 2024-09-18T14:05:07.881 INFO:tasks.rados.rados.0.smithi084.stdout:1251: expect (ObjNum 418 snap 128 seq_num 418) 2024-09-18T14:05:08.017 INFO:tasks.rados.rados.0.smithi084.stdout:1252: finishing copy_from racing read to smithi08434663-1 2024-09-18T14:05:08.017 INFO:tasks.rados.rados.0.smithi084.stdout:1252: finishing copy_from to smithi08434663-1 2024-09-18T14:05:08.017 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 278 (ObjNum 265 snap 67 seq_num 265) dirty exists 2024-09-18T14:05:08.017 INFO:tasks.rados.rados.0.smithi084.stdout:1254: finishing write tid 1 to smithi08434663-15 2024-09-18T14:05:08.018 INFO:tasks.rados.rados.0.smithi084.stdout:1254: finishing write tid 2 to smithi08434663-15 2024-09-18T14:05:08.018 INFO:tasks.rados.rados.0.smithi084.stdout:1256: seq_num 449 ranges {746343=651628,1924415=754807,3397159=149184} 2024-09-18T14:05:08.024 INFO:tasks.rados.rados.0.smithi084.stdout:1256: writing smithi08434663-13 from 746343 to 1397971 tid 1 2024-09-18T14:05:08.031 INFO:tasks.rados.rados.0.smithi084.stdout:1256: writing smithi08434663-13 from 1924415 to 2679222 tid 2 2024-09-18T14:05:08.037 INFO:tasks.rados.rados.0.smithi084.stdout:1256: writing smithi08434663-13 from 3397159 to 3546343 tid 3 2024-09-18T14:05:08.037 INFO:tasks.rados.rados.0.smithi084.stdout:1250: done (5 left) 2024-09-18T14:05:08.038 INFO:tasks.rados.rados.0.smithi084.stdout:1251: done (4 left) 2024-09-18T14:05:08.038 INFO:tasks.rados.rados.0.smithi084.stdout:1252: done (3 left) 2024-09-18T14:05:08.038 INFO:tasks.rados.rados.0.smithi084.stdout:1257: read oid 36 snap -1 2024-09-18T14:05:08.038 INFO:tasks.rados.rados.0.smithi084.stdout:1257: expect (ObjNum 343 snap 104 seq_num 343) 2024-09-18T14:05:08.038 INFO:tasks.rados.rados.0.smithi084.stdout:1258: rollback oid 40 current snap is 138 2024-09-18T14:05:08.038 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 40 to 97 2024-09-18T14:05:08.038 INFO:tasks.rados.rados.0.smithi084.stdout:1254: finishing write tid 3 to smithi08434663-15 2024-09-18T14:05:08.038 INFO:tasks.rados.rados.0.smithi084.stdout:1254: finishing write tid 4 to smithi08434663-15 2024-09-18T14:05:08.038 INFO:tasks.rados.rados.0.smithi084.stdout:1254: finishing write tid 5 to smithi08434663-15 2024-09-18T14:05:08.038 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 476 (ObjNum 447 snap 138 seq_num 447) dirty exists 2024-09-18T14:05:08.038 INFO:tasks.rados.rados.0.smithi084.stdout:1254: left oid 15 (ObjNum 447 snap 138 seq_num 447) 2024-09-18T14:05:08.038 INFO:tasks.rados.rados.0.smithi084.stdout:1255: finishing write tid 1 to smithi08434663-39 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:1255: finishing write tid 2 to smithi08434663-39 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:1255: finishing write tid 3 to smithi08434663-39 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:1255: finishing write tid 4 to smithi08434663-39 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:1255: finishing write tid 5 to smithi08434663-39 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 384 (ObjNum 448 snap 138 seq_num 448) dirty exists 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:1255: left oid 39 (ObjNum 448 snap 138 seq_num 448) 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:1256: finishing write tid 1 to smithi08434663-13 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:1256: finishing write tid 2 to smithi08434663-13 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:1254: done (4 left) 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:1255: done (3 left) 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:1259: rollback oid 22 current snap is 138 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 22 to 131 2024-09-18T14:05:08.039 INFO:tasks.rados.rados.0.smithi084.stdout:1260: snap_create 2024-09-18T14:05:08.040 INFO:tasks.rados.rados.0.smithi084.stdout:1259: finishing rollback tid 0 to smithi08434663-22 2024-09-18T14:05:08.040 INFO:tasks.rados.rados.0.smithi084.stdout:1259: finishing rollback tid 1 to smithi08434663-22 2024-09-18T14:05:08.040 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 354 (ObjNum 0 snap 0 seq_num 304) dirty dne 2024-09-18T14:05:08.043 INFO:tasks.rados.rados.0.smithi084.stdout:1256: finishing write tid 3 to smithi08434663-13 2024-09-18T14:05:08.043 INFO:tasks.rados.rados.0.smithi084.stdout:1256: finishing write tid 4 to smithi08434663-13 2024-09-18T14:05:08.043 INFO:tasks.rados.rados.0.smithi084.stdout:1256: finishing write tid 5 to smithi08434663-13 2024-09-18T14:05:08.043 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 418 (ObjNum 449 snap 138 seq_num 449) dirty exists 2024-09-18T14:05:08.043 INFO:tasks.rados.rados.0.smithi084.stdout:1256: left oid 13 (ObjNum 449 snap 138 seq_num 449) 2024-09-18T14:05:08.043 INFO:tasks.rados.rados.0.smithi084.stdout:1258: finishing rollback tid 0 to smithi08434663-40 2024-09-18T14:05:08.044 INFO:tasks.rados.rados.0.smithi084.stdout:1258: finishing rollback tid 1 to smithi08434663-40 2024-09-18T14:05:08.044 INFO:tasks.rados.rados.0.smithi084.stdout:1258: finishing rollback tid 2 to smithi08434663-40 2024-09-18T14:05:08.045 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 422 (ObjNum 247 snap 57 seq_num 247) dirty exists 2024-09-18T14:05:08.048 INFO:tasks.rados.rados.0.smithi084.stdout:1257: expect (ObjNum 343 snap 104 seq_num 343) 2024-09-18T14:05:08.276 INFO:tasks.rados.rados.0.smithi084.stdout:1256: done (4 left) 2024-09-18T14:05:08.276 INFO:tasks.rados.rados.0.smithi084.stdout:1257: done (3 left) 2024-09-18T14:05:08.277 INFO:tasks.rados.rados.0.smithi084.stdout:1258: done (2 left) 2024-09-18T14:05:08.277 INFO:tasks.rados.rados.0.smithi084.stdout:1259: done (1 left) 2024-09-18T14:05:08.277 INFO:tasks.rados.rados.0.smithi084.stdout:1260: done (0 left) 2024-09-18T14:05:08.277 INFO:tasks.rados.rados.0.smithi084.stdout:1261: write oid 46 current snap is 139 2024-09-18T14:05:08.277 INFO:tasks.rados.rados.0.smithi084.stdout:1261: seq_num 450 ranges {763150=726877,1982897=614818,3024582=474994} 2024-09-18T14:05:08.284 INFO:tasks.rados.rados.0.smithi084.stdout:1261: writing smithi08434663-46 from 763150 to 1490027 tid 1 2024-09-18T14:05:08.290 INFO:tasks.rados.rados.0.smithi084.stdout:1261: writing smithi08434663-46 from 1982897 to 2597715 tid 2 2024-09-18T14:05:08.295 INFO:tasks.rados.rados.0.smithi084.stdout:1261: writing smithi08434663-46 from 3024582 to 3499576 tid 3 2024-09-18T14:05:08.295 INFO:tasks.rados.rados.0.smithi084.stdout:1262: delete oid 42 current snap is 139 2024-09-18T14:05:08.295 INFO:tasks.rados.rados.0.smithi084.stdout:1261: finishing write tid 1 to smithi08434663-46 2024-09-18T14:05:08.295 INFO:tasks.rados.rados.0.smithi084.stdout:1261: finishing write tid 2 to smithi08434663-46 2024-09-18T14:05:08.299 INFO:tasks.rados.rados.0.smithi084.stdout:1261: finishing write tid 3 to smithi08434663-46 2024-09-18T14:05:08.299 INFO:tasks.rados.rados.0.smithi084.stdout:1261: finishing write tid 4 to smithi08434663-46 2024-09-18T14:05:08.299 INFO:tasks.rados.rados.0.smithi084.stdout:1261: finishing write tid 5 to smithi08434663-46 2024-09-18T14:05:08.299 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 481 (ObjNum 450 snap 139 seq_num 450) dirty exists 2024-09-18T14:05:08.299 INFO:tasks.rados.rados.0.smithi084.stdout:1261: left oid 46 (ObjNum 450 snap 139 seq_num 450) 2024-09-18T14:05:08.299 INFO:tasks.rados.rados.0.smithi084.stdout:1261: done (1 left) 2024-09-18T14:05:08.299 INFO:tasks.rados.rados.0.smithi084.stdout:1262: done (0 left) 2024-09-18T14:05:08.299 INFO:tasks.rados.rados.0.smithi084.stdout:1263: snap_remove snap 120 2024-09-18T14:05:09.278 INFO:tasks.rados.rados.0.smithi084.stdout:1263: done (0 left) 2024-09-18T14:05:09.278 INFO:tasks.rados.rados.0.smithi084.stdout:1264: rollback oid 36 current snap is 139 2024-09-18T14:05:09.278 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 36 to 138 2024-09-18T14:05:09.278 INFO:tasks.rados.rados.0.smithi084.stdout:1265: read oid 50 snap -1 2024-09-18T14:05:09.278 INFO:tasks.rados.rados.0.smithi084.stdout:1265: expect deleted 2024-09-18T14:05:09.279 INFO:tasks.rados.rados.0.smithi084.stdout:1266: copy_from oid 39 from oid 46 current snap is 139 2024-09-18T14:05:09.279 INFO:tasks.rados.rados.0.smithi084.stdout:1267: snap_remove snap 115 2024-09-18T14:05:10.293 INFO:tasks.rados.rados.0.smithi084.stdout:1264: finishing rollback tid 0 to smithi08434663-36 2024-09-18T14:05:10.293 INFO:tasks.rados.rados.0.smithi084.stdout:1264: finishing rollback tid 1 to smithi08434663-36 2024-09-18T14:05:10.293 INFO:tasks.rados.rados.0.smithi084.stdout:1264: finishing rollback tid 2 to smithi08434663-36 2024-09-18T14:05:10.293 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 390 (ObjNum 343 snap 104 seq_num 343) dirty exists 2024-09-18T14:05:10.293 INFO:tasks.rados.rados.0.smithi084.stdout:1266: finishing copy_from racing read to smithi08434663-39 2024-09-18T14:05:10.293 INFO:tasks.rados.rados.0.smithi084.stdout:1266: finishing copy_from to smithi08434663-39 2024-09-18T14:05:10.294 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 481 (ObjNum 450 snap 139 seq_num 450) dirty exists 2024-09-18T14:05:10.294 INFO:tasks.rados.rados.0.smithi084.stdout:1264: done (3 left) 2024-09-18T14:05:10.294 INFO:tasks.rados.rados.0.smithi084.stdout:1265: done (2 left) 2024-09-18T14:05:10.294 INFO:tasks.rados.rados.0.smithi084.stdout:1266: done (1 left) 2024-09-18T14:05:10.294 INFO:tasks.rados.rados.0.smithi084.stdout:1267: done (0 left) 2024-09-18T14:05:10.294 INFO:tasks.rados.rados.0.smithi084.stdout:1268: write oid 5 current snap is 139 2024-09-18T14:05:10.294 INFO:tasks.rados.rados.0.smithi084.stdout:1268: seq_num 452 ranges {796764=477375,1983508=618487,3363075=233689} 2024-09-18T14:05:10.300 INFO:tasks.rados.rados.0.smithi084.stdout:1268: writing smithi08434663-5 from 796764 to 1274139 tid 1 2024-09-18T14:05:10.307 INFO:tasks.rados.rados.0.smithi084.stdout:1268: writing smithi08434663-5 from 1983508 to 2601995 tid 2 2024-09-18T14:05:10.313 INFO:tasks.rados.rados.0.smithi084.stdout:1268: writing smithi08434663-5 from 3363075 to 3596764 tid 3 2024-09-18T14:05:10.313 INFO:tasks.rados.rados.0.smithi084.stdout:1269: snap_remove snap 126 2024-09-18T14:05:11.299 INFO:tasks.rados.rados.0.smithi084.stdout:1269: done (1 left) 2024-09-18T14:05:11.299 INFO:tasks.rados.rados.0.smithi084.stdout:1270: read oid 25 snap -1 2024-09-18T14:05:11.299 INFO:tasks.rados.rados.0.smithi084.stdout:1270: expect deleted 2024-09-18T14:05:11.299 INFO:tasks.rados.rados.0.smithi084.stdout:1271: write (excl) oid 19 current snap is 139 2024-09-18T14:05:11.299 INFO:tasks.rados.rados.0.smithi084.stdout:1268: finishing write tid 1 to smithi08434663-5 2024-09-18T14:05:11.300 INFO:tasks.rados.rados.0.smithi084.stdout:1268: finishing write tid 2 to smithi08434663-5 2024-09-18T14:05:11.300 INFO:tasks.rados.rados.0.smithi084.stdout:1268: finishing write tid 3 to smithi08434663-5 2024-09-18T14:05:11.300 INFO:tasks.rados.rados.0.smithi084.stdout:1268: finishing write tid 4 to smithi08434663-5 2024-09-18T14:05:11.300 INFO:tasks.rados.rados.0.smithi084.stdout:1268: finishing write tid 5 to smithi08434663-5 2024-09-18T14:05:11.300 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 198 (ObjNum 452 snap 139 seq_num 452) dirty exists 2024-09-18T14:05:11.300 INFO:tasks.rados.rados.0.smithi084.stdout:1268: left oid 5 (ObjNum 452 snap 139 seq_num 452) 2024-09-18T14:05:11.300 INFO:tasks.rados.rados.0.smithi084.stdout:1271: seq_num 453 ranges {413571=552624,1641990=762145,2990498=425163} 2024-09-18T14:05:11.305 INFO:tasks.rados.rados.0.smithi084.stdout:1271: writing smithi08434663-19 from 413571 to 966195 tid 1 2024-09-18T14:05:11.313 INFO:tasks.rados.rados.0.smithi084.stdout:1271: writing smithi08434663-19 from 1641990 to 2404135 tid 2 2024-09-18T14:05:11.319 INFO:tasks.rados.rados.0.smithi084.stdout:1271: writing smithi08434663-19 from 2990498 to 3415661 tid 3 2024-09-18T14:05:11.319 INFO:tasks.rados.rados.0.smithi084.stdout:1268: done (2 left) 2024-09-18T14:05:11.319 INFO:tasks.rados.rados.0.smithi084.stdout:1272: copy_from oid 7 from oid 28 current snap is 139 2024-09-18T14:05:11.319 INFO:tasks.rados.rados.0.smithi084.stdout:1270: done (2 left) 2024-09-18T14:05:11.319 INFO:tasks.rados.rados.0.smithi084.stdout:1273: snap_remove snap 137 2024-09-18T14:05:12.313 INFO:tasks.rados.rados.0.smithi084.stdout:1273: done (2 left) 2024-09-18T14:05:12.313 INFO:tasks.rados.rados.0.smithi084.stdout:1274: read oid 47 snap 131 2024-09-18T14:05:12.313 INFO:tasks.rados.rados.0.smithi084.stdout:1274: expect (ObjNum 373 snap 115 seq_num 373) 2024-09-18T14:05:12.313 INFO:tasks.rados.rados.0.smithi084.stdout:1275: copy_from oid 23 from oid 12 current snap is 139 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:1271: finishing write tid 1 to smithi08434663-19 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:1271: finishing write tid 2 to smithi08434663-19 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:1271: finishing write tid 3 to smithi08434663-19 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:1271: finishing write tid 4 to smithi08434663-19 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:1271: finishing write tid 5 to smithi08434663-19 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 614 (ObjNum 453 snap 139 seq_num 453) dirty exists 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:1271: left oid 19 (ObjNum 453 snap 139 seq_num 453) 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:1272: finishing copy_from racing read to smithi08434663-7 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:1272: finishing copy_from to smithi08434663-7 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 336 (ObjNum 386 snap 119 seq_num 386) dirty exists 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:1271: done (3 left) 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:1272: done (2 left) 2024-09-18T14:05:12.314 INFO:tasks.rados.rados.0.smithi084.stdout:1276: delete oid 44 current snap is 139 2024-09-18T14:05:12.318 INFO:tasks.rados.rados.0.smithi084.stdout:1276: done (2 left) 2024-09-18T14:05:12.318 INFO:tasks.rados.rados.0.smithi084.stdout:1277: read oid 34 snap 128 2024-09-18T14:05:12.318 INFO:tasks.rados.rados.0.smithi084.stdout:1277: expect (ObjNum 413 snap 126 seq_num 413) 2024-09-18T14:05:12.318 INFO:tasks.rados.rados.0.smithi084.stdout:1278: read oid 7 snap -1 2024-09-18T14:05:12.318 INFO:tasks.rados.rados.0.smithi084.stdout:1278: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:12.318 INFO:tasks.rados.rados.0.smithi084.stdout:1279: snap_create 2024-09-18T14:05:12.319 INFO:tasks.rados.rados.0.smithi084.stdout:1274: expect (ObjNum 373 snap 115 seq_num 373) 2024-09-18T14:05:12.447 INFO:tasks.rados.rados.0.smithi084.stdout:1277: expect (ObjNum 413 snap 126 seq_num 413) 2024-09-18T14:05:12.543 INFO:tasks.rados.rados.0.smithi084.stdout:1278: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:12.654 INFO:tasks.rados.rados.0.smithi084.stdout:1275: finishing copy_from racing read to smithi08434663-23 2024-09-18T14:05:12.654 INFO:tasks.rados.rados.0.smithi084.stdout:1275: finishing copy_from to smithi08434663-23 2024-09-18T14:05:12.654 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 393 (ObjNum 389 snap 119 seq_num 389) dirty exists 2024-09-18T14:05:13.313 INFO:tasks.rados.rados.0.smithi084.stdout:1274: done (4 left) 2024-09-18T14:05:13.314 INFO:tasks.rados.rados.0.smithi084.stdout:1275: done (3 left) 2024-09-18T14:05:13.314 INFO:tasks.rados.rados.0.smithi084.stdout:1277: done (2 left) 2024-09-18T14:05:13.314 INFO:tasks.rados.rados.0.smithi084.stdout:1278: done (1 left) 2024-09-18T14:05:13.314 INFO:tasks.rados.rados.0.smithi084.stdout:1279: done (0 left) 2024-09-18T14:05:13.314 INFO:tasks.rados.rados.0.smithi084.stdout:1280: copy_from oid 11 from oid 24 current snap is 140 2024-09-18T14:05:13.314 INFO:tasks.rados.rados.0.smithi084.stdout:1281: delete oid 12 current snap is 140 2024-09-18T14:05:13.316 INFO:tasks.rados.rados.0.smithi084.stdout:1281: done (1 left) 2024-09-18T14:05:13.316 INFO:tasks.rados.rados.0.smithi084.stdout:1282: copy_from oid 30 from oid 32 current snap is 140 2024-09-18T14:05:13.316 INFO:tasks.rados.rados.0.smithi084.stdout:1283: read oid 16 snap 128 2024-09-18T14:05:13.316 INFO:tasks.rados.rados.0.smithi084.stdout:1283: expect deleted 2024-09-18T14:05:13.317 INFO:tasks.rados.rados.0.smithi084.stdout:1284: copy_from oid 38 from oid 33 current snap is 140 2024-09-18T14:05:13.317 INFO:tasks.rados.rados.0.smithi084.stdout:1285: snap_create 2024-09-18T14:05:13.324 INFO:tasks.rados.rados.0.smithi084.stdout:1282: finishing copy_from racing read to smithi08434663-30 2024-09-18T14:05:13.328 INFO:tasks.rados.rados.0.smithi084.stdout:1280: finishing copy_from racing read to smithi08434663-11 2024-09-18T14:05:13.328 INFO:tasks.rados.rados.0.smithi084.stdout:1282: finishing copy_from to smithi08434663-30 2024-09-18T14:05:13.328 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 543 (ObjNum 418 snap 128 seq_num 418) dirty exists 2024-09-18T14:05:13.332 INFO:tasks.rados.rados.0.smithi084.stdout:1284: finishing copy_from racing read to smithi08434663-38 2024-09-18T14:05:13.332 INFO:tasks.rados.rados.0.smithi084.stdout:1280: finishing copy_from to smithi08434663-11 2024-09-18T14:05:13.332 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 186 (ObjNum 376 snap 116 seq_num 376) dirty exists 2024-09-18T14:05:13.333 INFO:tasks.rados.rados.0.smithi084.stdout:1284: finishing copy_from to smithi08434663-38 2024-09-18T14:05:13.333 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 387 (ObjNum 435 snap 135 seq_num 435) dirty exists 2024-09-18T14:05:14.324 INFO:tasks.rados.rados.0.smithi084.stdout:1280: done (4 left) 2024-09-18T14:05:14.324 INFO:tasks.rados.rados.0.smithi084.stdout:1282: done (3 left) 2024-09-18T14:05:14.324 INFO:tasks.rados.rados.0.smithi084.stdout:1283: done (2 left) 2024-09-18T14:05:14.325 INFO:tasks.rados.rados.0.smithi084.stdout:1284: done (1 left) 2024-09-18T14:05:14.325 INFO:tasks.rados.rados.0.smithi084.stdout:1285: done (0 left) 2024-09-18T14:05:14.325 INFO:tasks.rados.rados.0.smithi084.stdout:1286: delete oid 32 current snap is 141 2024-09-18T14:05:14.328 INFO:tasks.rados.rados.0.smithi084.stdout:1286: done (0 left) 2024-09-18T14:05:14.328 INFO:tasks.rados.rados.0.smithi084.stdout:1287: read oid 5 snap -1 2024-09-18T14:05:14.328 INFO:tasks.rados.rados.0.smithi084.stdout:1287: expect (ObjNum 452 snap 139 seq_num 452) 2024-09-18T14:05:14.328 INFO:tasks.rados.rados.0.smithi084.stdout:1288: rollback oid 1 current snap is 141 2024-09-18T14:05:14.329 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 1 to 136 2024-09-18T14:05:14.329 INFO:tasks.rados.rados.0.smithi084.stdout:1289: delete oid 2 current snap is 141 2024-09-18T14:05:14.332 INFO:tasks.rados.rados.0.smithi084.stdout:1288: finishing rollback tid 0 to smithi08434663-1 2024-09-18T14:05:14.334 INFO:tasks.rados.rados.0.smithi084.stdout:1289: done (2 left) 2024-09-18T14:05:14.334 INFO:tasks.rados.rados.0.smithi084.stdout:1290: write oid 33 current snap is 141 2024-09-18T14:05:14.334 INFO:tasks.rados.rados.0.smithi084.stdout:1290: seq_num 458 ranges {497606=645222,1934400=629494,3243966=453640} 2024-09-18T14:05:14.341 INFO:tasks.rados.rados.0.smithi084.stdout:1290: writing smithi08434663-33 from 497606 to 1142828 tid 1 2024-09-18T14:05:14.348 INFO:tasks.rados.rados.0.smithi084.stdout:1290: writing smithi08434663-33 from 1934400 to 2563894 tid 2 2024-09-18T14:05:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:1290: writing smithi08434663-33 from 3243966 to 3697606 tid 3 2024-09-18T14:05:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:1291: copy_from oid 50 from oid 28 current snap is 141 2024-09-18T14:05:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:1288: finishing rollback tid 1 to smithi08434663-1 2024-09-18T14:05:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:1288: finishing rollback tid 2 to smithi08434663-1 2024-09-18T14:05:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 486 (ObjNum 281 snap 71 seq_num 281) dirty exists 2024-09-18T14:05:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:1287: expect (ObjNum 452 snap 139 seq_num 452) 2024-09-18T14:05:14.476 INFO:tasks.rados.rados.0.smithi084.stdout:1290: finishing write tid 1 to smithi08434663-33 2024-09-18T14:05:14.476 INFO:tasks.rados.rados.0.smithi084.stdout:1290: finishing write tid 2 to smithi08434663-33 2024-09-18T14:05:14.476 INFO:tasks.rados.rados.0.smithi084.stdout:1287: done (3 left) 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:1288: done (2 left) 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:1292: read oid 14 snap -1 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:1292: expect (ObjNum 387 snap 119 seq_num 387) 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:1293: read oid 19 snap -1 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:1293: expect (ObjNum 453 snap 139 seq_num 453) 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:1294: rollback oid 18 current snap is 141 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 18 to 97 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:1290: finishing write tid 3 to smithi08434663-33 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:1290: finishing write tid 4 to smithi08434663-33 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:1290: finishing write tid 5 to smithi08434663-33 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 394 (ObjNum 458 snap 141 seq_num 458) dirty exists 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:1290: left oid 33 (ObjNum 458 snap 141 seq_num 458) 2024-09-18T14:05:14.477 INFO:tasks.rados.rados.0.smithi084.stdout:1291: finishing copy_from racing read to smithi08434663-50 2024-09-18T14:05:14.478 INFO:tasks.rados.rados.0.smithi084.stdout:1291: finishing copy_from to smithi08434663-50 2024-09-18T14:05:14.478 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 336 (ObjNum 386 snap 119 seq_num 386) dirty exists 2024-09-18T14:05:14.478 INFO:tasks.rados.rados.0.smithi084.stdout:1290: done (4 left) 2024-09-18T14:05:14.478 INFO:tasks.rados.rados.0.smithi084.stdout:1291: done (3 left) 2024-09-18T14:05:14.478 INFO:tasks.rados.rados.0.smithi084.stdout:1295: read oid 25 snap -1 2024-09-18T14:05:14.478 INFO:tasks.rados.rados.0.smithi084.stdout:1295: expect deleted 2024-09-18T14:05:14.478 INFO:tasks.rados.rados.0.smithi084.stdout:1296: rollback oid 38 current snap is 141 2024-09-18T14:05:14.478 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 38 to 136 2024-09-18T14:05:14.478 INFO:tasks.rados.rados.0.smithi084.stdout:1297: delete oid 15 current snap is 141 2024-09-18T14:05:14.478 INFO:tasks.rados.rados.0.smithi084.stdout:1294: finishing rollback tid 1 to smithi08434663-18 2024-09-18T14:05:14.478 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 348 (ObjNum 81 snap 0 seq_num 3386661650) dirty dne 2024-09-18T14:05:14.479 INFO:tasks.rados.rados.0.smithi084.stdout:1296: finishing rollback tid 0 to smithi08434663-38 2024-09-18T14:05:14.479 INFO:tasks.rados.rados.0.smithi084.stdout:1292: expect (ObjNum 387 snap 119 seq_num 387) 2024-09-18T14:05:14.559 INFO:tasks.rados.rados.0.smithi084.stdout:1296: finishing rollback tid 1 to smithi08434663-38 2024-09-18T14:05:14.559 INFO:tasks.rados.rados.0.smithi084.stdout:1296: finishing rollback tid 2 to smithi08434663-38 2024-09-18T14:05:14.559 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 390 (ObjNum 363 snap 112 seq_num 363) dirty exists 2024-09-18T14:05:14.559 INFO:tasks.rados.rados.0.smithi084.stdout:1293: expect (ObjNum 453 snap 139 seq_num 453) 2024-09-18T14:05:14.785 INFO:tasks.rados.rados.0.smithi084.stdout:1292: done (5 left) 2024-09-18T14:05:14.786 INFO:tasks.rados.rados.0.smithi084.stdout:1293: done (4 left) 2024-09-18T14:05:14.786 INFO:tasks.rados.rados.0.smithi084.stdout:1294: done (3 left) 2024-09-18T14:05:14.786 INFO:tasks.rados.rados.0.smithi084.stdout:1295: done (2 left) 2024-09-18T14:05:14.786 INFO:tasks.rados.rados.0.smithi084.stdout:1296: done (1 left) 2024-09-18T14:05:14.786 INFO:tasks.rados.rados.0.smithi084.stdout:1297: done (0 left) 2024-09-18T14:05:14.786 INFO:tasks.rados.rados.0.smithi084.stdout:1298: write (excl) oid 32 current snap is 141 2024-09-18T14:05:14.786 INFO:tasks.rados.rados.0.smithi084.stdout:1298: seq_num 460 ranges {531220=795720,1935011=516810,3066106=420420} 2024-09-18T14:05:14.792 INFO:tasks.rados.rados.0.smithi084.stdout:1298: writing smithi08434663-32 from 531220 to 1326940 tid 1 2024-09-18T14:05:14.798 INFO:tasks.rados.rados.0.smithi084.stdout:1298: writing smithi08434663-32 from 1935011 to 2451821 tid 2 2024-09-18T14:05:14.804 INFO:tasks.rados.rados.0.smithi084.stdout:1298: writing smithi08434663-32 from 3066106 to 3486526 tid 3 2024-09-18T14:05:14.804 INFO:tasks.rados.rados.0.smithi084.stdout:1299: snap_remove snap 140 2024-09-18T14:05:15.327 INFO:tasks.rados.rados.0.smithi084.stdout:1299: done (1 left) 2024-09-18T14:05:15.328 INFO:tasks.rados.rados.0.smithi084.stdout:1300: read oid 7 snap -1 2024-09-18T14:05:15.328 INFO:tasks.rados.rados.0.smithi084.stdout:1300: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:15.328 INFO:tasks.rados.rados.0.smithi084.stdout:1301: snap_remove snap 136 2024-09-18T14:05:16.342 INFO:tasks.rados.rados.0.smithi084.stdout:1301: done (2 left) 2024-09-18T14:05:16.342 INFO:tasks.rados.rados.0.smithi084.stdout:1302: snap_remove snap 128 2024-09-18T14:05:17.206 INFO:tasks.rados.rados.0.smithi084.stdout:1302: done (2 left) 2024-09-18T14:05:17.206 INFO:tasks.rados.rados.0.smithi084.stdout:1303: write oid 17 current snap is 141 2024-09-18T14:05:17.206 INFO:tasks.rados.rados.0.smithi084.stdout:1298: finishing write tid 1 to smithi08434663-32 2024-09-18T14:05:17.206 INFO:tasks.rados.rados.0.smithi084.stdout:1303: seq_num 461 ranges {548027=470969,1593493=776821,3093529=631692} 2024-09-18T14:05:17.215 INFO:tasks.rados.rados.0.smithi084.stdout:1303: writing smithi08434663-17 from 548027 to 1018996 tid 1 2024-09-18T14:05:17.224 INFO:tasks.rados.rados.0.smithi084.stdout:1303: writing smithi08434663-17 from 1593493 to 2370314 tid 2 2024-09-18T14:05:17.232 INFO:tasks.rados.rados.0.smithi084.stdout:1303: writing smithi08434663-17 from 3093529 to 3725221 tid 3 2024-09-18T14:05:17.232 INFO:tasks.rados.rados.0.smithi084.stdout:1304: read oid 26 snap -1 2024-09-18T14:05:17.232 INFO:tasks.rados.rados.0.smithi084.stdout:1304: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:17.232 INFO:tasks.rados.rados.0.smithi084.stdout:1305: snap_create 2024-09-18T14:05:17.232 INFO:tasks.rados.rados.0.smithi084.stdout:1298: finishing write tid 2 to smithi08434663-32 2024-09-18T14:05:17.232 INFO:tasks.rados.rados.0.smithi084.stdout:1298: finishing write tid 3 to smithi08434663-32 2024-09-18T14:05:17.232 INFO:tasks.rados.rados.0.smithi084.stdout:1298: finishing write tid 4 to smithi08434663-32 2024-09-18T14:05:17.232 INFO:tasks.rados.rados.0.smithi084.stdout:1298: finishing write tid 5 to smithi08434663-32 2024-09-18T14:05:17.232 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 631 (ObjNum 460 snap 141 seq_num 460) dirty exists 2024-09-18T14:05:17.232 INFO:tasks.rados.rados.0.smithi084.stdout:1298: left oid 32 (ObjNum 460 snap 141 seq_num 460) 2024-09-18T14:05:17.233 INFO:tasks.rados.rados.0.smithi084.stdout:1300: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:17.347 INFO:tasks.rados.rados.0.smithi084.stdout:1303: finishing write tid 1 to smithi08434663-17 2024-09-18T14:05:17.347 INFO:tasks.rados.rados.0.smithi084.stdout:1303: finishing write tid 2 to smithi08434663-17 2024-09-18T14:05:17.347 INFO:tasks.rados.rados.0.smithi084.stdout:1304: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:17.458 INFO:tasks.rados.rados.0.smithi084.stdout:1303: finishing write tid 3 to smithi08434663-17 2024-09-18T14:05:17.459 INFO:tasks.rados.rados.0.smithi084.stdout:1303: finishing write tid 4 to smithi08434663-17 2024-09-18T14:05:17.459 INFO:tasks.rados.rados.0.smithi084.stdout:1303: finishing write tid 5 to smithi08434663-17 2024-09-18T14:05:17.459 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 456 (ObjNum 461 snap 141 seq_num 461) dirty exists 2024-09-18T14:05:17.459 INFO:tasks.rados.rados.0.smithi084.stdout:1303: left oid 17 (ObjNum 461 snap 141 seq_num 461) 2024-09-18T14:05:18.222 INFO:tasks.rados.rados.0.smithi084.stdout:1298: done (4 left) 2024-09-18T14:05:18.222 INFO:tasks.rados.rados.0.smithi084.stdout:1300: done (3 left) 2024-09-18T14:05:18.222 INFO:tasks.rados.rados.0.smithi084.stdout:1303: done (2 left) 2024-09-18T14:05:18.222 INFO:tasks.rados.rados.0.smithi084.stdout:1304: done (1 left) 2024-09-18T14:05:18.222 INFO:tasks.rados.rados.0.smithi084.stdout:1305: done (0 left) 2024-09-18T14:05:18.222 INFO:tasks.rados.rados.0.smithi084.stdout:1306: rollback oid 40 current snap is 142 2024-09-18T14:05:18.222 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 40 to 124 2024-09-18T14:05:18.222 INFO:tasks.rados.rados.0.smithi084.stdout:1307: read oid 30 snap -1 2024-09-18T14:05:18.222 INFO:tasks.rados.rados.0.smithi084.stdout:1307: expect (ObjNum 418 snap 128 seq_num 418) 2024-09-18T14:05:18.222 INFO:tasks.rados.rados.0.smithi084.stdout:1308: delete oid 4 current snap is 142 2024-09-18T14:05:18.227 INFO:tasks.rados.rados.0.smithi084.stdout:1308: done (2 left) 2024-09-18T14:05:18.227 INFO:tasks.rados.rados.0.smithi084.stdout:1309: snap_remove snap 139 2024-09-18T14:05:19.224 INFO:tasks.rados.rados.0.smithi084.stdout:1309: done (2 left) 2024-09-18T14:05:19.224 INFO:tasks.rados.rados.0.smithi084.stdout:1310: read oid 25 snap -1 2024-09-18T14:05:19.224 INFO:tasks.rados.rados.0.smithi084.stdout:1310: expect deleted 2024-09-18T14:05:19.224 INFO:tasks.rados.rados.0.smithi084.stdout:1311: write oid 18 current snap is 142 2024-09-18T14:05:19.224 INFO:tasks.rados.rados.0.smithi084.stdout:1311: seq_num 463 ranges {581641=621467,1994104=780490,3432022=349619} 2024-09-18T14:05:19.230 INFO:tasks.rados.rados.0.smithi084.stdout:1311: writing smithi08434663-18 from 581641 to 1203108 tid 1 2024-09-18T14:05:19.239 INFO:tasks.rados.rados.0.smithi084.stdout:1311: writing smithi08434663-18 from 1994104 to 2774594 tid 2 2024-09-18T14:05:19.245 INFO:tasks.rados.rados.0.smithi084.stdout:1311: writing smithi08434663-18 from 3432022 to 3781641 tid 3 2024-09-18T14:05:19.245 INFO:tasks.rados.rados.0.smithi084.stdout:1312: read oid 50 snap -1 2024-09-18T14:05:19.245 INFO:tasks.rados.rados.0.smithi084.stdout:1312: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:19.245 INFO:tasks.rados.rados.0.smithi084.stdout:1313: snap_create 2024-09-18T14:05:19.245 INFO:tasks.rados.rados.0.smithi084.stdout:1307: expect (ObjNum 418 snap 128 seq_num 418) 2024-09-18T14:05:19.382 INFO:tasks.rados.rados.0.smithi084.stdout:1306: finishing rollback tid 0 to smithi08434663-40 2024-09-18T14:05:19.382 INFO:tasks.rados.rados.0.smithi084.stdout:1306: finishing rollback tid 1 to smithi08434663-40 2024-09-18T14:05:19.382 INFO:tasks.rados.rados.0.smithi084.stdout:1306: finishing rollback tid 2 to smithi08434663-40 2024-09-18T14:05:19.382 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 460 (ObjNum 354 snap 109 seq_num 354) dirty exists 2024-09-18T14:05:19.382 INFO:tasks.rados.rados.0.smithi084.stdout:1311: finishing write tid 1 to smithi08434663-18 2024-09-18T14:05:19.382 INFO:tasks.rados.rados.0.smithi084.stdout:1311: finishing write tid 2 to smithi08434663-18 2024-09-18T14:05:19.382 INFO:tasks.rados.rados.0.smithi084.stdout:1311: finishing write tid 3 to smithi08434663-18 2024-09-18T14:05:19.382 INFO:tasks.rados.rados.0.smithi084.stdout:1311: finishing write tid 4 to smithi08434663-18 2024-09-18T14:05:19.382 INFO:tasks.rados.rados.0.smithi084.stdout:1311: finishing write tid 5 to smithi08434663-18 2024-09-18T14:05:19.383 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 358 (ObjNum 463 snap 142 seq_num 463) dirty exists 2024-09-18T14:05:19.383 INFO:tasks.rados.rados.0.smithi084.stdout:1311: left oid 18 (ObjNum 463 snap 142 seq_num 463) 2024-09-18T14:05:19.383 INFO:tasks.rados.rados.0.smithi084.stdout:1312: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:20.238 INFO:tasks.rados.rados.0.smithi084.stdout:1306: done (5 left) 2024-09-18T14:05:20.238 INFO:tasks.rados.rados.0.smithi084.stdout:1307: done (4 left) 2024-09-18T14:05:20.238 INFO:tasks.rados.rados.0.smithi084.stdout:1310: done (3 left) 2024-09-18T14:05:20.238 INFO:tasks.rados.rados.0.smithi084.stdout:1311: done (2 left) 2024-09-18T14:05:20.238 INFO:tasks.rados.rados.0.smithi084.stdout:1312: done (1 left) 2024-09-18T14:05:20.238 INFO:tasks.rados.rados.0.smithi084.stdout:1313: done (0 left) 2024-09-18T14:05:20.238 INFO:tasks.rados.rados.0.smithi084.stdout:1314: snap_create 2024-09-18T14:05:21.240 INFO:tasks.rados.rados.0.smithi084.stdout:1314: done (0 left) 2024-09-18T14:05:21.241 INFO:tasks.rados.rados.0.smithi084.stdout:1315: rollback oid 42 current snap is 144 2024-09-18T14:05:21.241 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 42 to 143 2024-09-18T14:05:21.241 INFO:tasks.rados.rados.0.smithi084.stdout:1316: write (excl) oid 34 current snap is 144 2024-09-18T14:05:21.241 INFO:tasks.rados.rados.0.smithi084.stdout:1316: seq_num 464 ranges {598448=413069,1768939=524148,2775798=581861,3781488=16960} 2024-09-18T14:05:21.246 INFO:tasks.rados.rados.0.smithi084.stdout:1316: writing smithi08434663-34 from 598448 to 1011517 tid 1 2024-09-18T14:05:21.253 INFO:tasks.rados.rados.0.smithi084.stdout:1316: writing smithi08434663-34 from 1768939 to 2293087 tid 2 2024-09-18T14:05:21.259 INFO:tasks.rados.rados.0.smithi084.stdout:1316: writing smithi08434663-34 from 2775798 to 3357659 tid 3 2024-09-18T14:05:21.261 INFO:tasks.rados.rados.0.smithi084.stdout:1316: writing smithi08434663-34 from 3781488 to 3798448 tid 4 2024-09-18T14:05:21.262 INFO:tasks.rados.rados.0.smithi084.stdout:1317: write (excl) oid 3 current snap is 144 2024-09-18T14:05:21.262 INFO:tasks.rados.rados.0.smithi084.stdout:1315: finishing rollback tid 1 to smithi08434663-42 2024-09-18T14:05:21.262 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 509 (ObjNum 129 snap 0 seq_num 2494534336) dirty dne 2024-09-18T14:05:21.262 INFO:tasks.rados.rados.0.smithi084.stdout:1316: finishing write tid 1 to smithi08434663-34 2024-09-18T14:05:21.262 INFO:tasks.rados.rados.0.smithi084.stdout:1316: finishing write tid 2 to smithi08434663-34 2024-09-18T14:05:21.262 INFO:tasks.rados.rados.0.smithi084.stdout:1317: seq_num 465 ranges {615255=488318,1827421=784159,3203221=612034} 2024-09-18T14:05:21.267 INFO:tasks.rados.rados.0.smithi084.stdout:1317: writing smithi08434663-3 from 615255 to 1103573 tid 1 2024-09-18T14:05:21.275 INFO:tasks.rados.rados.0.smithi084.stdout:1317: writing smithi08434663-3 from 1827421 to 2611580 tid 2 2024-09-18T14:05:21.282 INFO:tasks.rados.rados.0.smithi084.stdout:1317: writing smithi08434663-3 from 3203221 to 3815255 tid 3 2024-09-18T14:05:21.282 INFO:tasks.rados.rados.0.smithi084.stdout:1315: done (2 left) 2024-09-18T14:05:21.282 INFO:tasks.rados.rados.0.smithi084.stdout:1318: write (excl) oid 1 current snap is 144 2024-09-18T14:05:21.282 INFO:tasks.rados.rados.0.smithi084.stdout:1316: finishing write tid 3 to smithi08434663-34 2024-09-18T14:05:21.282 INFO:tasks.rados.rados.0.smithi084.stdout:1316: finishing write tid 4 to smithi08434663-34 2024-09-18T14:05:21.283 INFO:tasks.rados.rados.0.smithi084.stdout:1316: finishing write tid 5 to smithi08434663-34 2024-09-18T14:05:21.283 INFO:tasks.rados.rados.0.smithi084.stdout:1316: finishing write tid 6 to smithi08434663-34 2024-09-18T14:05:21.283 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 551 (ObjNum 464 snap 144 seq_num 464) dirty exists 2024-09-18T14:05:21.283 INFO:tasks.rados.rados.0.smithi084.stdout:1316: left oid 34 (ObjNum 464 snap 144 seq_num 464) 2024-09-18T14:05:21.283 INFO:tasks.rados.rados.0.smithi084.stdout:1317: finishing write tid 1 to smithi08434663-3 2024-09-18T14:05:21.283 INFO:tasks.rados.rados.0.smithi084.stdout:1318: seq_num 466 ranges {632062=563567,1885903=527817,2830644=720758} 2024-09-18T14:05:21.288 INFO:tasks.rados.rados.0.smithi084.stdout:1318: writing smithi08434663-1 from 632062 to 1195629 tid 1 2024-09-18T14:05:21.295 INFO:tasks.rados.rados.0.smithi084.stdout:1318: writing smithi08434663-1 from 1885903 to 2413720 tid 2 2024-09-18T14:05:21.302 INFO:tasks.rados.rados.0.smithi084.stdout:1318: writing smithi08434663-1 from 2830644 to 3551402 tid 3 2024-09-18T14:05:21.302 INFO:tasks.rados.rados.0.smithi084.stdout:1316: done (2 left) 2024-09-18T14:05:21.302 INFO:tasks.rados.rados.0.smithi084.stdout:1319: snap_create 2024-09-18T14:05:21.302 INFO:tasks.rados.rados.0.smithi084.stdout:1317: finishing write tid 2 to smithi08434663-3 2024-09-18T14:05:21.302 INFO:tasks.rados.rados.0.smithi084.stdout:1317: finishing write tid 3 to smithi08434663-3 2024-09-18T14:05:21.302 INFO:tasks.rados.rados.0.smithi084.stdout:1317: finishing write tid 4 to smithi08434663-3 2024-09-18T14:05:21.302 INFO:tasks.rados.rados.0.smithi084.stdout:1317: finishing write tid 5 to smithi08434663-3 2024-09-18T14:05:21.302 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 362 (ObjNum 465 snap 144 seq_num 465) dirty exists 2024-09-18T14:05:21.303 INFO:tasks.rados.rados.0.smithi084.stdout:1317: left oid 3 (ObjNum 465 snap 144 seq_num 465) 2024-09-18T14:05:21.303 INFO:tasks.rados.rados.0.smithi084.stdout:1318: finishing write tid 1 to smithi08434663-1 2024-09-18T14:05:21.303 INFO:tasks.rados.rados.0.smithi084.stdout:1318: finishing write tid 2 to smithi08434663-1 2024-09-18T14:05:21.305 INFO:tasks.rados.rados.0.smithi084.stdout:1318: finishing write tid 3 to smithi08434663-1 2024-09-18T14:05:21.309 INFO:tasks.rados.rados.0.smithi084.stdout:1318: finishing write tid 4 to smithi08434663-1 2024-09-18T14:05:21.309 INFO:tasks.rados.rados.0.smithi084.stdout:1318: finishing write tid 5 to smithi08434663-1 2024-09-18T14:05:21.309 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 505 (ObjNum 466 snap 144 seq_num 466) dirty exists 2024-09-18T14:05:21.309 INFO:tasks.rados.rados.0.smithi084.stdout:1318: left oid 1 (ObjNum 466 snap 144 seq_num 466) 2024-09-18T14:05:22.254 INFO:tasks.rados.rados.0.smithi084.stdout:1317: done (2 left) 2024-09-18T14:05:22.254 INFO:tasks.rados.rados.0.smithi084.stdout:1318: done (1 left) 2024-09-18T14:05:22.254 INFO:tasks.rados.rados.0.smithi084.stdout:1319: done (0 left) 2024-09-18T14:05:22.254 INFO:tasks.rados.rados.0.smithi084.stdout:1320: write (excl) oid 45 current snap is 145 2024-09-18T14:05:22.254 INFO:tasks.rados.rados.0.smithi084.stdout:1320: seq_num 467 ranges {648869=638816,1828032=787828,3141714=532030} 2024-09-18T14:05:22.260 INFO:tasks.rados.rados.0.smithi084.stdout:1320: writing smithi08434663-45 from 648869 to 1287685 tid 1 2024-09-18T14:05:22.267 INFO:tasks.rados.rados.0.smithi084.stdout:1320: writing smithi08434663-45 from 1828032 to 2615860 tid 2 2024-09-18T14:05:22.274 INFO:tasks.rados.rados.0.smithi084.stdout:1320: writing smithi08434663-45 from 3141714 to 3673744 tid 3 2024-09-18T14:05:22.274 INFO:tasks.rados.rados.0.smithi084.stdout:1321: read oid 5 snap -1 2024-09-18T14:05:22.274 INFO:tasks.rados.rados.0.smithi084.stdout:1321: expect (ObjNum 452 snap 139 seq_num 452) 2024-09-18T14:05:22.274 INFO:tasks.rados.rados.0.smithi084.stdout:1322: read oid 31 snap -1 2024-09-18T14:05:22.274 INFO:tasks.rados.rados.0.smithi084.stdout:1322: expect (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:05:22.274 INFO:tasks.rados.rados.0.smithi084.stdout:1323: copy_from oid 41 from oid 48 current snap is 145 2024-09-18T14:05:22.274 INFO:tasks.rados.rados.0.smithi084.stdout:1320: finishing write tid 1 to smithi08434663-45 2024-09-18T14:05:22.275 INFO:tasks.rados.rados.0.smithi084.stdout:1320: finishing write tid 2 to smithi08434663-45 2024-09-18T14:05:22.275 INFO:tasks.rados.rados.0.smithi084.stdout:1324: read oid 17 snap 144 2024-09-18T14:05:22.275 INFO:tasks.rados.rados.0.smithi084.stdout:1324: expect (ObjNum 461 snap 141 seq_num 461) 2024-09-18T14:05:22.275 INFO:tasks.rados.rados.0.smithi084.stdout:1325: read oid 43 snap -1 2024-09-18T14:05:22.275 INFO:tasks.rados.rados.0.smithi084.stdout:1325: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:05:22.275 INFO:tasks.rados.rados.0.smithi084.stdout:1326: write (excl) oid 23 current snap is 145 2024-09-18T14:05:22.275 INFO:tasks.rados.rados.0.smithi084.stdout:1326: seq_num 468 ranges {665676=714065,1886514=531486,3169137=696539} 2024-09-18T14:05:22.281 INFO:tasks.rados.rados.0.smithi084.stdout:1326: writing smithi08434663-23 from 665676 to 1379741 tid 1 2024-09-18T14:05:22.287 INFO:tasks.rados.rados.0.smithi084.stdout:1326: writing smithi08434663-23 from 1886514 to 2418000 tid 2 2024-09-18T14:05:22.295 INFO:tasks.rados.rados.0.smithi084.stdout:1326: writing smithi08434663-23 from 3169137 to 3865676 tid 3 2024-09-18T14:05:22.295 INFO:tasks.rados.rados.0.smithi084.stdout:1327: snap_create 2024-09-18T14:05:22.295 INFO:tasks.rados.rados.0.smithi084.stdout:1321: expect (ObjNum 452 snap 139 seq_num 452) 2024-09-18T14:05:22.414 INFO:tasks.rados.rados.0.smithi084.stdout:1323: finishing copy_from to smithi08434663-41 2024-09-18T14:05:22.414 INFO:tasks.rados.rados.0.smithi084.stdout:1323: got expected ENOENT (src dne) 2024-09-18T14:05:22.414 INFO:tasks.rados.rados.0.smithi084.stdout:1323: finishing copy_from racing read to smithi08434663-41 2024-09-18T14:05:22.414 INFO:tasks.rados.rados.0.smithi084.stdout:1325: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:05:22.490 INFO:tasks.rados.rados.0.smithi084.stdout:1320: finishing write tid 3 to smithi08434663-45 2024-09-18T14:05:22.490 INFO:tasks.rados.rados.0.smithi084.stdout:1320: finishing write tid 4 to smithi08434663-45 2024-09-18T14:05:22.490 INFO:tasks.rados.rados.0.smithi084.stdout:1320: finishing write tid 5 to smithi08434663-45 2024-09-18T14:05:22.490 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 555 (ObjNum 467 snap 145 seq_num 467) dirty exists 2024-09-18T14:05:22.490 INFO:tasks.rados.rados.0.smithi084.stdout:1320: left oid 45 (ObjNum 467 snap 145 seq_num 467) 2024-09-18T14:05:22.490 INFO:tasks.rados.rados.0.smithi084.stdout:1322: expect (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:05:22.757 INFO:tasks.rados.rados.0.smithi084.stdout:1324: expect (ObjNum 461 snap 141 seq_num 461) 2024-09-18T14:05:22.874 INFO:tasks.rados.rados.0.smithi084.stdout:1326: finishing write tid 1 to smithi08434663-23 2024-09-18T14:05:22.874 INFO:tasks.rados.rados.0.smithi084.stdout:1326: finishing write tid 2 to smithi08434663-23 2024-09-18T14:05:22.874 INFO:tasks.rados.rados.0.smithi084.stdout:1326: finishing write tid 3 to smithi08434663-23 2024-09-18T14:05:22.874 INFO:tasks.rados.rados.0.smithi084.stdout:1326: finishing write tid 4 to smithi08434663-23 2024-09-18T14:05:22.874 INFO:tasks.rados.rados.0.smithi084.stdout:1326: finishing write tid 5 to smithi08434663-23 2024-09-18T14:05:22.874 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 397 (ObjNum 468 snap 145 seq_num 468) dirty exists 2024-09-18T14:05:22.874 INFO:tasks.rados.rados.0.smithi084.stdout:1326: left oid 23 (ObjNum 468 snap 145 seq_num 468) 2024-09-18T14:05:23.255 INFO:tasks.rados.rados.0.smithi084.stdout:1320: done (7 left) 2024-09-18T14:05:23.255 INFO:tasks.rados.rados.0.smithi084.stdout:1321: done (6 left) 2024-09-18T14:05:23.255 INFO:tasks.rados.rados.0.smithi084.stdout:1322: done (5 left) 2024-09-18T14:05:23.255 INFO:tasks.rados.rados.0.smithi084.stdout:1323: done (4 left) 2024-09-18T14:05:23.255 INFO:tasks.rados.rados.0.smithi084.stdout:1324: done (3 left) 2024-09-18T14:05:23.255 INFO:tasks.rados.rados.0.smithi084.stdout:1325: done (2 left) 2024-09-18T14:05:23.255 INFO:tasks.rados.rados.0.smithi084.stdout:1326: done (1 left) 2024-09-18T14:05:23.255 INFO:tasks.rados.rados.0.smithi084.stdout:1327: done (0 left) 2024-09-18T14:05:23.255 INFO:tasks.rados.rados.0.smithi084.stdout:1328: snap_remove snap 131 2024-09-18T14:05:24.266 INFO:tasks.rados.rados.0.smithi084.stdout:1328: done (0 left) 2024-09-18T14:05:24.266 INFO:tasks.rados.rados.0.smithi084.stdout:1329: delete oid 12 current snap is 146 2024-09-18T14:05:24.269 INFO:tasks.rados.rados.0.smithi084.stdout:1329: done (0 left) 2024-09-18T14:05:24.269 INFO:tasks.rados.rados.0.smithi084.stdout:1330: snap_create 2024-09-18T14:05:25.270 INFO:tasks.rados.rados.0.smithi084.stdout:1330: done (0 left) 2024-09-18T14:05:25.270 INFO:tasks.rados.rados.0.smithi084.stdout:1331: rollback oid 48 current snap is 147 2024-09-18T14:05:25.270 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 48 to 138 2024-09-18T14:05:25.270 INFO:tasks.rados.rados.0.smithi084.stdout:1332: write (excl) oid 44 current snap is 147 2024-09-18T14:05:25.270 INFO:tasks.rados.rados.0.smithi084.stdout:1332: seq_num 470 ranges {699290=464563,1603478=535155,2823983=765846} 2024-09-18T14:05:25.276 INFO:tasks.rados.rados.0.smithi084.stdout:1332: writing smithi08434663-44 from 699290 to 1163853 tid 1 2024-09-18T14:05:25.281 INFO:tasks.rados.rados.0.smithi084.stdout:1332: writing smithi08434663-44 from 1603478 to 2138633 tid 2 2024-09-18T14:05:25.289 INFO:tasks.rados.rados.0.smithi084.stdout:1332: writing smithi08434663-44 from 2823983 to 3589829 tid 3 2024-09-18T14:05:25.290 INFO:tasks.rados.rados.0.smithi084.stdout:1333: snap_create 2024-09-18T14:05:25.290 INFO:tasks.rados.rados.0.smithi084.stdout:1331: finishing rollback tid 1 to smithi08434663-48 2024-09-18T14:05:25.290 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 334 (ObjNum 129 snap 0 seq_num 2495122384) dirty dne 2024-09-18T14:05:25.290 INFO:tasks.rados.rados.0.smithi084.stdout:1332: finishing write tid 1 to smithi08434663-44 2024-09-18T14:05:25.290 INFO:tasks.rados.rados.0.smithi084.stdout:1332: finishing write tid 2 to smithi08434663-44 2024-09-18T14:05:25.293 INFO:tasks.rados.rados.0.smithi084.stdout:1332: finishing write tid 3 to smithi08434663-44 2024-09-18T14:05:25.293 INFO:tasks.rados.rados.0.smithi084.stdout:1332: finishing write tid 4 to smithi08434663-44 2024-09-18T14:05:25.293 INFO:tasks.rados.rados.0.smithi084.stdout:1332: finishing write tid 5 to smithi08434663-44 2024-09-18T14:05:25.293 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 206 (ObjNum 470 snap 147 seq_num 470) dirty exists 2024-09-18T14:05:25.293 INFO:tasks.rados.rados.0.smithi084.stdout:1332: left oid 44 (ObjNum 470 snap 147 seq_num 470) 2024-09-18T14:05:26.280 INFO:tasks.rados.rados.0.smithi084.stdout:1331: done (2 left) 2024-09-18T14:05:26.280 INFO:tasks.rados.rados.0.smithi084.stdout:1332: done (1 left) 2024-09-18T14:05:26.280 INFO:tasks.rados.rados.0.smithi084.stdout:1333: done (0 left) 2024-09-18T14:05:26.280 INFO:tasks.rados.rados.0.smithi084.stdout:1334: copy_from oid 16 from oid 12 current snap is 148 2024-09-18T14:05:26.280 INFO:tasks.rados.rados.0.smithi084.stdout:1335: snap_remove snap 145 2024-09-18T14:05:27.208 INFO:tasks.rados.rados.0.smithi084.stdout:1335: done (1 left) 2024-09-18T14:05:27.209 INFO:tasks.rados.rados.0.smithi084.stdout:1336: read oid 31 snap -1 2024-09-18T14:05:27.209 INFO:tasks.rados.rados.0.smithi084.stdout:1336: expect (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:05:27.209 INFO:tasks.rados.rados.0.smithi084.stdout:1337: write (excl) oid 50 current snap is 148 2024-09-18T14:05:27.209 INFO:tasks.rados.rados.0.smithi084.stdout:1334: finishing copy_from to smithi08434663-16 2024-09-18T14:05:27.209 INFO:tasks.rados.rados.0.smithi084.stdout:1334: got expected ENOENT (src dne) 2024-09-18T14:05:27.209 INFO:tasks.rados.rados.0.smithi084.stdout:1334: finishing copy_from racing read to smithi08434663-16 2024-09-18T14:05:27.209 INFO:tasks.rados.rados.0.smithi084.stdout:1337: seq_num 471 ranges {716097=539812,1945607=795166,3535053=381044} 2024-09-18T14:05:27.215 INFO:tasks.rados.rados.0.smithi084.stdout:1337: writing smithi08434663-50 from 716097 to 1255909 tid 1 2024-09-18T14:05:27.225 INFO:tasks.rados.rados.0.smithi084.stdout:1337: writing smithi08434663-50 from 1945607 to 2740773 tid 2 2024-09-18T14:05:27.232 INFO:tasks.rados.rados.0.smithi084.stdout:1337: writing smithi08434663-50 from 3535053 to 3916097 tid 3 2024-09-18T14:05:27.232 INFO:tasks.rados.rados.0.smithi084.stdout:1334: done (2 left) 2024-09-18T14:05:27.232 INFO:tasks.rados.rados.0.smithi084.stdout:1338: write (excl) oid 23 current snap is 148 2024-09-18T14:05:27.232 INFO:tasks.rados.rados.0.smithi084.stdout:1338: seq_num 472 ranges {732904=731414,2120442=538824,3278829=504743} 2024-09-18T14:05:27.239 INFO:tasks.rados.rados.0.smithi084.stdout:1338: writing smithi08434663-23 from 732904 to 1464318 tid 1 2024-09-18T14:05:27.246 INFO:tasks.rados.rados.0.smithi084.stdout:1338: writing smithi08434663-23 from 2120442 to 2659266 tid 2 2024-09-18T14:05:27.252 INFO:tasks.rados.rados.0.smithi084.stdout:1338: writing smithi08434663-23 from 3278829 to 3783572 tid 3 2024-09-18T14:05:27.253 INFO:tasks.rados.rados.0.smithi084.stdout:1339: write oid 38 current snap is 148 2024-09-18T14:05:27.253 INFO:tasks.rados.rados.0.smithi084.stdout:1339: seq_num 473 ranges {749711=406663,1778924=682482,3189899=716015} 2024-09-18T14:05:27.258 INFO:tasks.rados.rados.0.smithi084.stdout:1339: writing smithi08434663-38 from 749711 to 1156374 tid 1 2024-09-18T14:05:27.265 INFO:tasks.rados.rados.0.smithi084.stdout:1339: writing smithi08434663-38 from 1778924 to 2461406 tid 2 2024-09-18T14:05:27.273 INFO:tasks.rados.rados.0.smithi084.stdout:1339: writing smithi08434663-38 from 3189899 to 3905914 tid 3 2024-09-18T14:05:27.273 INFO:tasks.rados.rados.0.smithi084.stdout:1340: snap_remove snap 134 2024-09-18T14:05:28.286 INFO:tasks.rados.rados.0.smithi084.stdout:1340: done (4 left) 2024-09-18T14:05:28.286 INFO:tasks.rados.rados.0.smithi084.stdout:1341: read oid 22 snap -1 2024-09-18T14:05:28.286 INFO:tasks.rados.rados.0.smithi084.stdout:1341: expect deleted 2024-09-18T14:05:28.286 INFO:tasks.rados.rados.0.smithi084.stdout:1342: write (excl) oid 10 current snap is 148 2024-09-18T14:05:28.287 INFO:tasks.rados.rados.0.smithi084.stdout:1342: seq_num 474 ranges {766518=481912,1837406=542493,2933675=527287,3876584=89934} 2024-09-18T14:05:28.293 INFO:tasks.rados.rados.0.smithi084.stdout:1342: writing smithi08434663-10 from 766518 to 1248430 tid 1 2024-09-18T14:05:28.300 INFO:tasks.rados.rados.0.smithi084.stdout:1342: writing smithi08434663-10 from 1837406 to 2379899 tid 2 2024-09-18T14:05:28.306 INFO:tasks.rados.rados.0.smithi084.stdout:1342: writing smithi08434663-10 from 2933675 to 3460962 tid 3 2024-09-18T14:05:28.309 INFO:tasks.rados.rados.0.smithi084.stdout:1342: writing smithi08434663-10 from 3876584 to 3966518 tid 4 2024-09-18T14:05:28.309 INFO:tasks.rados.rados.0.smithi084.stdout:1343: read oid 37 snap -1 2024-09-18T14:05:28.309 INFO:tasks.rados.rados.0.smithi084.stdout:1343: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:05:28.309 INFO:tasks.rados.rados.0.smithi084.stdout:1344: snap_remove snap 143 2024-09-18T14:05:29.286 INFO:tasks.rados.rados.0.smithi084.stdout:1344: done (7 left) 2024-09-18T14:05:29.286 INFO:tasks.rados.rados.0.smithi084.stdout:1345: write oid 9 current snap is 148 2024-09-18T14:05:29.286 INFO:tasks.rados.rados.0.smithi084.stdout:1336: expect (ObjNum 281 snap 71 seq_num 281) 2024-09-18T14:05:29.557 INFO:tasks.rados.rados.0.smithi084.stdout:1337: finishing write tid 1 to smithi08434663-50 2024-09-18T14:05:29.557 INFO:tasks.rados.rados.0.smithi084.stdout:1337: finishing write tid 2 to smithi08434663-50 2024-09-18T14:05:29.557 INFO:tasks.rados.rados.0.smithi084.stdout:1345: seq_num 475 ranges {783325=557161,1779535=686151,3244745=454912} 2024-09-18T14:05:29.564 INFO:tasks.rados.rados.0.smithi084.stdout:1345: writing smithi08434663-9 from 783325 to 1340486 tid 1 2024-09-18T14:05:29.570 INFO:tasks.rados.rados.0.smithi084.stdout:1345: writing smithi08434663-9 from 1779535 to 2465686 tid 2 2024-09-18T14:05:29.577 INFO:tasks.rados.rados.0.smithi084.stdout:1345: writing smithi08434663-9 from 3244745 to 3699657 tid 3 2024-09-18T14:05:29.577 INFO:tasks.rados.rados.0.smithi084.stdout:1336: done (7 left) 2024-09-18T14:05:29.577 INFO:tasks.rados.rados.0.smithi084.stdout:1346: write (excl) oid 4 current snap is 148 2024-09-18T14:05:29.578 INFO:tasks.rados.rados.0.smithi084.stdout:1346: seq_num 476 ranges {400132=632410,1438017=546162} 2024-09-18T14:05:29.582 INFO:tasks.rados.rados.0.smithi084.stdout:1346: writing smithi08434663-4 from 400132 to 1032542 tid 1 2024-09-18T14:05:29.588 INFO:tasks.rados.rados.0.smithi084.stdout:1346: writing smithi08434663-4 from 1438017 to 1984179 tid 2 2024-09-18T14:05:29.588 INFO:tasks.rados.rados.0.smithi084.stdout:1347: read oid 43 snap 146 2024-09-18T14:05:29.589 INFO:tasks.rados.rados.0.smithi084.stdout:1347: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:05:29.589 INFO:tasks.rados.rados.0.smithi084.stdout:1348: copy_from oid 16 from oid 21 current snap is 148 2024-09-18T14:05:29.589 INFO:tasks.rados.rados.0.smithi084.stdout:1337: finishing write tid 3 to smithi08434663-50 2024-09-18T14:05:29.589 INFO:tasks.rados.rados.0.smithi084.stdout:1337: finishing write tid 4 to smithi08434663-50 2024-09-18T14:05:29.589 INFO:tasks.rados.rados.0.smithi084.stdout:1337: finishing write tid 5 to smithi08434663-50 2024-09-18T14:05:29.589 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 681 (ObjNum 471 snap 148 seq_num 471) dirty exists 2024-09-18T14:05:29.589 INFO:tasks.rados.rados.0.smithi084.stdout:1337: left oid 50 (ObjNum 471 snap 148 seq_num 471) 2024-09-18T14:05:29.589 INFO:tasks.rados.rados.0.smithi084.stdout:1338: finishing write tid 1 to smithi08434663-23 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1338: finishing write tid 2 to smithi08434663-23 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1338: finishing write tid 3 to smithi08434663-23 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1338: finishing write tid 4 to smithi08434663-23 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1338: finishing write tid 5 to smithi08434663-23 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 401 (ObjNum 472 snap 148 seq_num 472) dirty exists 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1338: left oid 23 (ObjNum 472 snap 148 seq_num 472) 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1339: finishing write tid 1 to smithi08434663-38 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1339: finishing write tid 2 to smithi08434663-38 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1339: finishing write tid 3 to smithi08434663-38 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1337: done (9 left) 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1338: done (8 left) 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1349: read oid 24 snap -1 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1349: expect (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:05:29.590 INFO:tasks.rados.rados.0.smithi084.stdout:1350: snap_create 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:1339: finishing write tid 4 to smithi08434663-38 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:1339: finishing write tid 5 to smithi08434663-38 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 394 (ObjNum 473 snap 148 seq_num 473) dirty exists 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:1339: left oid 38 (ObjNum 473 snap 148 seq_num 473) 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:1342: finishing write tid 1 to smithi08434663-10 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:1342: finishing write tid 2 to smithi08434663-10 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:1342: finishing write tid 3 to smithi08434663-10 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:1342: finishing write tid 4 to smithi08434663-10 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:1342: finishing write tid 5 to smithi08434663-10 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:1342: finishing write tid 6 to smithi08434663-10 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 521 (ObjNum 474 snap 148 seq_num 474) dirty exists 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:1342: left oid 10 (ObjNum 474 snap 148 seq_num 474) 2024-09-18T14:05:29.591 INFO:tasks.rados.rados.0.smithi084.stdout:1343: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:05:29.664 INFO:tasks.rados.rados.0.smithi084.stdout:1345: finishing write tid 1 to smithi08434663-9 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:1345: finishing write tid 2 to smithi08434663-9 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:1345: finishing write tid 3 to smithi08434663-9 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:1345: finishing write tid 4 to smithi08434663-9 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:1345: finishing write tid 5 to smithi08434663-9 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 518 (ObjNum 475 snap 148 seq_num 475) dirty exists 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:1345: left oid 9 (ObjNum 475 snap 148 seq_num 475) 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:1346: finishing write tid 1 to smithi08434663-4 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:1348: finishing copy_from to smithi08434663-16 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:1348: got expected ENOENT (src dne) 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:1348: finishing copy_from racing read to smithi08434663-16 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:1346: finishing write tid 2 to smithi08434663-4 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:1346: finishing write tid 3 to smithi08434663-4 2024-09-18T14:05:29.665 INFO:tasks.rados.rados.0.smithi084.stdout:1346: finishing write tid 4 to smithi08434663-4 2024-09-18T14:05:29.666 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 570 (ObjNum 476 snap 148 seq_num 476) dirty exists 2024-09-18T14:05:29.666 INFO:tasks.rados.rados.0.smithi084.stdout:1346: left oid 4 (ObjNum 476 snap 148 seq_num 476) 2024-09-18T14:05:29.666 INFO:tasks.rados.rados.0.smithi084.stdout:1349: expect (ObjNum 376 snap 116 seq_num 376) 2024-09-18T14:05:29.797 INFO:tasks.rados.rados.0.smithi084.stdout:1347: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:05:30.297 INFO:tasks.rados.rados.0.smithi084.stdout:1339: done (9 left) 2024-09-18T14:05:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1341: done (8 left) 2024-09-18T14:05:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1342: done (7 left) 2024-09-18T14:05:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1343: done (6 left) 2024-09-18T14:05:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1345: done (5 left) 2024-09-18T14:05:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1346: done (4 left) 2024-09-18T14:05:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1347: done (3 left) 2024-09-18T14:05:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1348: done (2 left) 2024-09-18T14:05:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1349: done (1 left) 2024-09-18T14:05:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1350: done (0 left) 2024-09-18T14:05:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1351: read oid 33 snap -1 2024-09-18T14:05:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1351: expect (ObjNum 458 snap 141 seq_num 458) 2024-09-18T14:05:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1352: snap_remove snap 97 2024-09-18T14:05:31.298 INFO:tasks.rados.rados.0.smithi084.stdout:1352: done (1 left) 2024-09-18T14:05:31.298 INFO:tasks.rados.rados.0.smithi084.stdout:1353: read oid 28 snap -1 2024-09-18T14:05:31.298 INFO:tasks.rados.rados.0.smithi084.stdout:1353: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:31.298 INFO:tasks.rados.rados.0.smithi084.stdout:1354: rollback oid 1 current snap is 149 2024-09-18T14:05:31.299 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 1 to 147 2024-09-18T14:05:31.299 INFO:tasks.rados.rados.0.smithi084.stdout:1355: read oid 35 snap -1 2024-09-18T14:05:31.299 INFO:tasks.rados.rados.0.smithi084.stdout:1355: expect deleted 2024-09-18T14:05:31.299 INFO:tasks.rados.rados.0.smithi084.stdout:1356: read oid 44 snap -1 2024-09-18T14:05:31.299 INFO:tasks.rados.rados.0.smithi084.stdout:1356: expect (ObjNum 470 snap 147 seq_num 470) 2024-09-18T14:05:31.299 INFO:tasks.rados.rados.0.smithi084.stdout:1357: copy_from oid 31 from oid 3 current snap is 149 2024-09-18T14:05:31.299 INFO:tasks.rados.rados.0.smithi084.stdout:1358: write oid 43 current snap is 149 2024-09-18T14:05:31.300 INFO:tasks.rados.rados.0.smithi084.stdout:1358: seq_num 477 ranges {416939=707659,1896499=120440} 2024-09-18T14:05:31.306 INFO:tasks.rados.rados.0.smithi084.stdout:1358: writing smithi08434663-43 from 416939 to 1124598 tid 1 2024-09-18T14:05:31.311 INFO:tasks.rados.rados.0.smithi084.stdout:1358: writing smithi08434663-43 from 1896499 to 2016939 tid 2 2024-09-18T14:05:31.312 INFO:tasks.rados.rados.0.smithi084.stdout:1359: write oid 30 current snap is 149 2024-09-18T14:05:31.312 INFO:tasks.rados.rados.0.smithi084.stdout:1359: seq_num 478 ranges {433746=782908,1954981=78765} 2024-09-18T14:05:31.320 INFO:tasks.rados.rados.0.smithi084.stdout:1359: writing smithi08434663-30 from 433746 to 1216654 tid 1 2024-09-18T14:05:31.324 INFO:tasks.rados.rados.0.smithi084.stdout:1359: writing smithi08434663-30 from 1954981 to 2033746 tid 2 2024-09-18T14:05:31.324 INFO:tasks.rados.rados.0.smithi084.stdout:1360: copy_from oid 22 from oid 50 current snap is 149 2024-09-18T14:05:31.325 INFO:tasks.rados.rados.0.smithi084.stdout:1351: expect (ObjNum 458 snap 141 seq_num 458) 2024-09-18T14:05:31.547 INFO:tasks.rados.rados.0.smithi084.stdout:1353: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:31.658 INFO:tasks.rados.rados.0.smithi084.stdout:1356: expect (ObjNum 470 snap 147 seq_num 470) 2024-09-18T14:05:31.782 INFO:tasks.rados.rados.0.smithi084.stdout:1354: finishing rollback tid 0 to smithi08434663-1 2024-09-18T14:05:31.782 INFO:tasks.rados.rados.0.smithi084.stdout:1354: finishing rollback tid 1 to smithi08434663-1 2024-09-18T14:05:31.782 INFO:tasks.rados.rados.0.smithi084.stdout:1354: finishing rollback tid 2 to smithi08434663-1 2024-09-18T14:05:31.782 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 529 (ObjNum 466 snap 144 seq_num 466) dirty exists 2024-09-18T14:05:31.782 INFO:tasks.rados.rados.0.smithi084.stdout:1358: finishing write tid 1 to smithi08434663-43 2024-09-18T14:05:31.782 INFO:tasks.rados.rados.0.smithi084.stdout:1358: finishing write tid 2 to smithi08434663-43 2024-09-18T14:05:31.782 INFO:tasks.rados.rados.0.smithi084.stdout:1358: finishing write tid 3 to smithi08434663-43 2024-09-18T14:05:31.782 INFO:tasks.rados.rados.0.smithi084.stdout:1358: finishing write tid 4 to smithi08434663-43 2024-09-18T14:05:31.782 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 397 (ObjNum 477 snap 149 seq_num 477) dirty exists 2024-09-18T14:05:31.782 INFO:tasks.rados.rados.0.smithi084.stdout:1358: left oid 43 (ObjNum 477 snap 149 seq_num 477) 2024-09-18T14:05:31.782 INFO:tasks.rados.rados.0.smithi084.stdout:1357: finishing copy_from racing read to smithi08434663-31 2024-09-18T14:05:31.782 INFO:tasks.rados.rados.0.smithi084.stdout:1357: finishing copy_from to smithi08434663-31 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 362 (ObjNum 465 snap 144 seq_num 465) dirty exists 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:1359: finishing write tid 1 to smithi08434663-30 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:1359: finishing write tid 2 to smithi08434663-30 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:1359: finishing write tid 3 to smithi08434663-30 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:1359: finishing write tid 4 to smithi08434663-30 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 546 (ObjNum 478 snap 149 seq_num 478) dirty exists 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:1359: left oid 30 (ObjNum 478 snap 149 seq_num 478) 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:1360: finishing copy_from racing read to smithi08434663-22 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:1360: finishing copy_from to smithi08434663-22 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 681 (ObjNum 471 snap 148 seq_num 471) dirty exists 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:1351: done (8 left) 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:1353: done (7 left) 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:1354: done (6 left) 2024-09-18T14:05:31.783 INFO:tasks.rados.rados.0.smithi084.stdout:1355: done (5 left) 2024-09-18T14:05:31.784 INFO:tasks.rados.rados.0.smithi084.stdout:1356: done (4 left) 2024-09-18T14:05:31.784 INFO:tasks.rados.rados.0.smithi084.stdout:1357: done (3 left) 2024-09-18T14:05:31.784 INFO:tasks.rados.rados.0.smithi084.stdout:1358: done (2 left) 2024-09-18T14:05:31.784 INFO:tasks.rados.rados.0.smithi084.stdout:1359: done (1 left) 2024-09-18T14:05:31.784 INFO:tasks.rados.rados.0.smithi084.stdout:1360: done (0 left) 2024-09-18T14:05:31.784 INFO:tasks.rados.rados.0.smithi084.stdout:1361: snap_create 2024-09-18T14:05:32.310 INFO:tasks.rados.rados.0.smithi084.stdout:1361: done (0 left) 2024-09-18T14:05:32.310 INFO:tasks.rados.rados.0.smithi084.stdout:1362: snap_create 2024-09-18T14:05:33.321 INFO:tasks.rados.rados.0.smithi084.stdout:1362: done (0 left) 2024-09-18T14:05:33.321 INFO:tasks.rados.rados.0.smithi084.stdout:1363: read oid 37 snap -1 2024-09-18T14:05:33.321 INFO:tasks.rados.rados.0.smithi084.stdout:1363: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:05:33.321 INFO:tasks.rados.rados.0.smithi084.stdout:1364: read oid 5 snap 144 2024-09-18T14:05:33.321 INFO:tasks.rados.rados.0.smithi084.stdout:1364: expect (ObjNum 452 snap 139 seq_num 452) 2024-09-18T14:05:33.321 INFO:tasks.rados.rados.0.smithi084.stdout:1365: rollback oid 42 current snap is 151 2024-09-18T14:05:33.321 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 42 to 138 2024-09-18T14:05:33.322 INFO:tasks.rados.rados.0.smithi084.stdout:1366: read oid 7 snap -1 2024-09-18T14:05:33.322 INFO:tasks.rados.rados.0.smithi084.stdout:1366: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:33.322 INFO:tasks.rados.rados.0.smithi084.stdout:1367: read oid 8 snap -1 2024-09-18T14:05:33.322 INFO:tasks.rados.rados.0.smithi084.stdout:1367: expect (ObjNum 373 snap 115 seq_num 373) 2024-09-18T14:05:33.322 INFO:tasks.rados.rados.0.smithi084.stdout:1368: copy_from oid 34 from oid 9 current snap is 151 2024-09-18T14:05:33.322 INFO:tasks.rados.rados.0.smithi084.stdout:1369: read oid 10 snap -1 2024-09-18T14:05:33.322 INFO:tasks.rados.rados.0.smithi084.stdout:1369: expect (ObjNum 474 snap 148 seq_num 474) 2024-09-18T14:05:33.322 INFO:tasks.rados.rados.0.smithi084.stdout:1370: write (excl) oid 13 current snap is 151 2024-09-18T14:05:33.322 INFO:tasks.rados.rados.0.smithi084.stdout:1370: seq_num 479 ranges {450553=574510,1613463=437090} 2024-09-18T14:05:33.327 INFO:tasks.rados.rados.0.smithi084.stdout:1370: writing smithi08434663-13 from 450553 to 1025063 tid 1 2024-09-18T14:05:33.333 INFO:tasks.rados.rados.0.smithi084.stdout:1370: writing smithi08434663-13 from 1613463 to 2050553 tid 2 2024-09-18T14:05:33.333 INFO:tasks.rados.rados.0.smithi084.stdout:1371: write oid 31 current snap is 151 2024-09-18T14:05:33.333 INFO:tasks.rados.rados.0.smithi084.stdout:1371: seq_num 480 ranges {467360=649759,1671945=395415} 2024-09-18T14:05:33.339 INFO:tasks.rados.rados.0.smithi084.stdout:1371: writing smithi08434663-31 from 467360 to 1117119 tid 1 2024-09-18T14:05:33.345 INFO:tasks.rados.rados.0.smithi084.stdout:1371: writing smithi08434663-31 from 1671945 to 2067360 tid 2 2024-09-18T14:05:33.345 INFO:tasks.rados.rados.0.smithi084.stdout:1372: snap_remove snap 124 2024-09-18T14:05:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:1372: done (9 left) 2024-09-18T14:05:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:1373: read oid 28 snap -1 2024-09-18T14:05:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:1373: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:1364: expect (ObjNum 452 snap 139 seq_num 452) 2024-09-18T14:05:34.455 INFO:tasks.rados.rados.0.smithi084.stdout:1367: expect (ObjNum 373 snap 115 seq_num 373) 2024-09-18T14:05:34.580 INFO:tasks.rados.rados.0.smithi084.stdout:1369: expect (ObjNum 474 snap 148 seq_num 474) 2024-09-18T14:05:34.796 INFO:tasks.rados.rados.0.smithi084.stdout:1365: finishing rollback tid 1 to smithi08434663-42 2024-09-18T14:05:34.796 INFO:tasks.rados.rados.0.smithi084.stdout:1365: finishing rollback tid 2 to smithi08434663-42 2024-09-18T14:05:34.796 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 578 (ObjNum 426 snap 134 seq_num 426) dirty exists 2024-09-18T14:05:34.796 INFO:tasks.rados.rados.0.smithi084.stdout:1363: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:05:34.871 INFO:tasks.rados.rados.0.smithi084.stdout:1366: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:34.984 INFO:tasks.rados.rados.0.smithi084.stdout:1370: finishing write tid 1 to smithi08434663-13 2024-09-18T14:05:34.984 INFO:tasks.rados.rados.0.smithi084.stdout:1370: finishing write tid 2 to smithi08434663-13 2024-09-18T14:05:34.984 INFO:tasks.rados.rados.0.smithi084.stdout:1370: finishing write tid 3 to smithi08434663-13 2024-09-18T14:05:34.984 INFO:tasks.rados.rados.0.smithi084.stdout:1370: finishing write tid 4 to smithi08434663-13 2024-09-18T14:05:34.984 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 478 (ObjNum 479 snap 151 seq_num 479) dirty exists 2024-09-18T14:05:34.984 INFO:tasks.rados.rados.0.smithi084.stdout:1370: left oid 13 (ObjNum 479 snap 151 seq_num 479) 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1371: finishing write tid 1 to smithi08434663-31 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1363: done (9 left) 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1364: done (8 left) 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1365: done (7 left) 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1366: done (6 left) 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1367: done (5 left) 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1369: done (4 left) 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1370: done (3 left) 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1374: read oid 37 snap 147 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1374: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1375: delete oid 12 current snap is 151 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1368: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:1368: finishing copy_from to smithi08434663-34 2024-09-18T14:05:34.985 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 518 (ObjNum 475 snap 148 seq_num 475) dirty exists 2024-09-18T14:05:34.986 INFO:tasks.rados.rados.0.smithi084.stdout:1371: finishing write tid 2 to smithi08434663-31 2024-09-18T14:05:34.986 INFO:tasks.rados.rados.0.smithi084.stdout:1371: finishing write tid 3 to smithi08434663-31 2024-09-18T14:05:34.986 INFO:tasks.rados.rados.0.smithi084.stdout:1371: finishing write tid 4 to smithi08434663-31 2024-09-18T14:05:34.986 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 365 (ObjNum 480 snap 151 seq_num 480) dirty exists 2024-09-18T14:05:34.986 INFO:tasks.rados.rados.0.smithi084.stdout:1371: left oid 31 (ObjNum 480 snap 151 seq_num 480) 2024-09-18T14:05:34.986 INFO:tasks.rados.rados.0.smithi084.stdout:1373: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:05:35.096 INFO:tasks.rados.rados.0.smithi084.stdout:1374: expect (ObjNum 138 snap 29 seq_num 138) 2024-09-18T14:05:35.171 INFO:tasks.rados.rados.0.smithi084.stdout:1368: done (4 left) 2024-09-18T14:05:35.171 INFO:tasks.rados.rados.0.smithi084.stdout:1371: done (3 left) 2024-09-18T14:05:35.171 INFO:tasks.rados.rados.0.smithi084.stdout:1373: done (2 left) 2024-09-18T14:05:35.171 INFO:tasks.rados.rados.0.smithi084.stdout:1374: done (1 left) 2024-09-18T14:05:35.171 INFO:tasks.rados.rados.0.smithi084.stdout:1375: done (0 left) 2024-09-18T14:05:35.171 INFO:tasks.rados.rados.0.smithi084.stdout:1376: copy_from oid 13 from oid 45 current snap is 151 2024-09-18T14:05:35.171 INFO:tasks.rados.rados.0.smithi084.stdout:1377: rollback oid 21 current snap is 151 2024-09-18T14:05:35.171 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 21 to 149 2024-09-18T14:05:35.171 INFO:tasks.rados.rados.0.smithi084.stdout:1378: read oid 4 snap 144 2024-09-18T14:05:35.172 INFO:tasks.rados.rados.0.smithi084.stdout:1378: expect deleted 2024-09-18T14:05:35.172 INFO:tasks.rados.rados.0.smithi084.stdout:1379: write (excl) oid 41 current snap is 151 2024-09-18T14:05:35.172 INFO:tasks.rados.rados.0.smithi084.stdout:1379: seq_num 482 ranges {500974=400257,1388909=557169} 2024-09-18T14:05:35.175 INFO:tasks.rados.rados.0.smithi084.stdout:1379: writing smithi08434663-41 from 500974 to 901231 tid 1 2024-09-18T14:05:35.181 INFO:tasks.rados.rados.0.smithi084.stdout:1379: writing smithi08434663-41 from 1388909 to 1946078 tid 2 2024-09-18T14:05:35.181 INFO:tasks.rados.rados.0.smithi084.stdout:1380: rollback oid 17 current snap is 151 2024-09-18T14:05:35.182 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 17 to 148 2024-09-18T14:05:35.182 INFO:tasks.rados.rados.0.smithi084.stdout:1377: finishing rollback tid 1 to smithi08434663-21 2024-09-18T14:05:35.182 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-18T14:05:35.182 INFO:tasks.rados.rados.0.smithi084.stdout:1377: done (4 left) 2024-09-18T14:05:35.182 INFO:tasks.rados.rados.0.smithi084.stdout:1378: done (3 left) 2024-09-18T14:05:35.182 INFO:tasks.rados.rados.0.smithi084.stdout:1381: read oid 31 snap -1 2024-09-18T14:05:35.182 INFO:tasks.rados.rados.0.smithi084.stdout:1381: expect (ObjNum 480 snap 151 seq_num 480) 2024-09-18T14:05:35.182 INFO:tasks.rados.rados.0.smithi084.stdout:1382: rollback oid 27 current snap is 151 2024-09-18T14:05:35.182 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 27 to 144 2024-09-18T14:05:35.182 INFO:tasks.rados.rados.0.smithi084.stdout:1383: rollback oid 9 current snap is 151 2024-09-18T14:05:35.182 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 9 to 141 2024-09-18T14:05:35.182 INFO:tasks.rados.rados.0.smithi084.stdout:1384: snap_remove snap 150 2024-09-18T14:05:35.333 INFO:tasks.rados.rados.0.smithi084.stdout:1384: done (6 left) 2024-09-18T14:05:35.333 INFO:tasks.rados.rados.0.smithi084.stdout:1385: rollback oid 1 current snap is 151 2024-09-18T14:05:35.333 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 1 to 138 2024-09-18T14:05:35.333 INFO:tasks.rados.rados.0.smithi084.stdout:1382: finishing rollback tid 1 to smithi08434663-27 2024-09-18T14:05:35.333 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 0 (ObjNum 33 snap 0 seq_num 3154195104) dirty dne 2024-09-18T14:05:35.333 INFO:tasks.rados.rados.0.smithi084.stdout:1379: finishing write tid 1 to smithi08434663-41 2024-09-18T14:05:35.333 INFO:tasks.rados.rados.0.smithi084.stdout:1376: finishing copy_from racing read to smithi08434663-13 2024-09-18T14:05:35.333 INFO:tasks.rados.rados.0.smithi084.stdout:1383: finishing rollback tid 0 to smithi08434663-9 2024-09-18T14:05:35.333 INFO:tasks.rados.rados.0.smithi084.stdout:1383: finishing rollback tid 1 to smithi08434663-9 2024-09-18T14:05:35.334 INFO:tasks.rados.rados.0.smithi084.stdout:1383: finishing rollback tid 2 to smithi08434663-9 2024-09-18T14:05:35.334 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 528 (ObjNum 429 snap 134 seq_num 429) dirty exists 2024-09-18T14:05:35.334 INFO:tasks.rados.rados.0.smithi084.stdout:1381: expect (ObjNum 480 snap 151 seq_num 480) 2024-09-18T14:05:35.418 INFO:tasks.rados.rados.0.smithi084.stdout:1376: finishing copy_from to smithi08434663-13 2024-09-18T14:05:35.418 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 555 (ObjNum 467 snap 145 seq_num 467) dirty exists 2024-09-18T14:05:35.418 INFO:tasks.rados.rados.0.smithi084.stdout:1379: finishing write tid 2 to smithi08434663-41 2024-09-18T14:05:35.418 INFO:tasks.rados.rados.0.smithi084.stdout:1380: finishing rollback tid 0 to smithi08434663-17 2024-09-18T14:05:35.418 INFO:tasks.rados.rados.0.smithi084.stdout:1380: finishing rollback tid 1 to smithi08434663-17 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:1379: finishing write tid 3 to smithi08434663-41 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:1379: finishing write tid 4 to smithi08434663-41 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 590 (ObjNum 482 snap 151 seq_num 482) dirty exists 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:1379: left oid 41 (ObjNum 482 snap 151 seq_num 482) 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:1380: finishing rollback tid 2 to smithi08434663-17 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 558 (ObjNum 461 snap 141 seq_num 461) dirty exists 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:1376: done (6 left) 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:1379: done (5 left) 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:1380: done (4 left) 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:1381: done (3 left) 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:1382: done (2 left) 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:1383: done (1 left) 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:1386: copy_from oid 6 from oid 17 current snap is 151 2024-09-18T14:05:35.419 INFO:tasks.rados.rados.0.smithi084.stdout:1385: finishing rollback tid 0 to smithi08434663-1 2024-09-18T14:05:35.420 INFO:tasks.rados.rados.0.smithi084.stdout:1385: finishing rollback tid 1 to smithi08434663-1 2024-09-18T14:05:35.420 INFO:tasks.rados.rados.0.smithi084.stdout:1385: finishing rollback tid 2 to smithi08434663-1 2024-09-18T14:05:35.420 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 547 (ObjNum 265 snap 67 seq_num 265) dirty exists 2024-09-18T14:05:35.420 INFO:tasks.rados.rados.0.smithi084.stdout:1385: done (1 left) 2024-09-18T14:05:35.420 INFO:tasks.rados.rados.0.smithi084.stdout:1387: read oid 19 snap -1 2024-09-18T14:05:35.420 INFO:tasks.rados.rados.0.smithi084.stdout:1387: expect (ObjNum 453 snap 139 seq_num 453) 2024-09-18T14:05:35.420 INFO:tasks.rados.rados.0.smithi084.stdout:1388: delete oid 21 current snap is 151 2024-09-18T14:05:35.421 INFO:tasks.rados.rados.0.smithi084.stdout:1388: done (2 left) 2024-09-18T14:05:35.421 INFO:tasks.rados.rados.0.smithi084.stdout:1389: write (excl) oid 27 current snap is 151 2024-09-18T14:05:35.421 INFO:tasks.rados.rados.0.smithi084.stdout:1389: seq_num 484 ranges {534588=550755,1789520=345068} 2024-09-18T14:05:35.427 INFO:tasks.rados.rados.0.smithi084.stdout:1389: writing smithi08434663-27 from 534588 to 1085343 tid 1 2024-09-18T14:05:35.433 INFO:tasks.rados.rados.0.smithi084.stdout:1389: writing smithi08434663-27 from 1789520 to 2134588 tid 2 2024-09-18T14:05:35.433 INFO:tasks.rados.rados.0.smithi084.stdout:1390: write oid 37 current snap is 151 2024-09-18T14:05:35.433 INFO:tasks.rados.rados.0.smithi084.stdout:1390: seq_num 485 ranges {551395=626004,1848002=303393} 2024-09-18T14:05:35.439 INFO:tasks.rados.rados.0.smithi084.stdout:1390: writing smithi08434663-37 from 551395 to 1177399 tid 1 2024-09-18T14:05:35.444 INFO:tasks.rados.rados.0.smithi084.stdout:1390: writing smithi08434663-37 from 1848002 to 2151395 tid 2 2024-09-18T14:05:35.445 INFO:tasks.rados.rados.0.smithi084.stdout:1391: write oid 43 current snap is 151 2024-09-18T14:05:35.445 INFO:tasks.rados.rados.0.smithi084.stdout:1386: finishing copy_from racing read to smithi08434663-6 2024-09-18T14:05:35.445 INFO:tasks.rados.rados.0.smithi084.stdout:1387: expect (ObjNum 453 snap 139 seq_num 453) 2024-09-18T14:05:35.672 INFO:tasks.rados.rados.0.smithi084.stdout:1386: finishing copy_from to smithi08434663-6 2024-09-18T14:05:35.672 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 558 (ObjNum 461 snap 141 seq_num 461) dirty exists 2024-09-18T14:05:35.673 INFO:tasks.rados.rados.0.smithi084.stdout:1389: finishing write tid 1 to smithi08434663-27 2024-09-18T14:05:35.673 INFO:tasks.rados.rados.0.smithi084.stdout:1389: finishing write tid 2 to smithi08434663-27 2024-09-18T14:05:35.673 INFO:tasks.rados.rados.0.smithi084.stdout:1389: finishing write tid 3 to smithi08434663-27 2024-09-18T14:05:35.673 INFO:tasks.rados.rados.0.smithi084.stdout:1389: finishing write tid 4 to smithi08434663-27 2024-09-18T14:05:35.673 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 550 (ObjNum 484 snap 151 seq_num 484) dirty exists 2024-09-18T14:05:35.674 INFO:tasks.rados.rados.0.smithi084.stdout:1389: left oid 27 (ObjNum 484 snap 151 seq_num 484) 2024-09-18T14:05:35.674 INFO:tasks.rados.rados.0.smithi084.stdout:1390: finishing write tid 1 to smithi08434663-37 2024-09-18T14:05:35.674 INFO:tasks.rados.rados.0.smithi084.stdout:1390: finishing write tid 2 to smithi08434663-37 2024-09-18T14:05:35.674 INFO:tasks.rados.rados.0.smithi084.stdout:1390: finishing write tid 3 to smithi08434663-37 2024-09-18T14:05:35.674 INFO:tasks.rados.rados.0.smithi084.stdout:1390: finishing write tid 4 to smithi08434663-37 2024-09-18T14:05:35.674 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 397 (ObjNum 485 snap 151 seq_num 485) dirty exists 2024-09-18T14:05:35.675 INFO:tasks.rados.rados.0.smithi084.stdout:1390: left oid 37 (ObjNum 485 snap 151 seq_num 485) 2024-09-18T14:05:35.675 INFO:tasks.rados.rados.0.smithi084.stdout:1391: seq_num 486 ranges {568202=701253,1906484=261718} 2024-09-18T14:05:35.678 INFO:tasks.rados.rados.0.smithi084.stdout:1391: writing smithi08434663-43 from 568202 to 1269455 tid 1 2024-09-18T14:05:35.683 INFO:tasks.rados.rados.0.smithi084.stdout:1391: writing smithi08434663-43 from 1906484 to 2168202 tid 2 2024-09-18T14:05:35.684 INFO:tasks.rados.rados.0.smithi084.stdout:1386: done (4 left) 2024-09-18T14:05:35.684 INFO:tasks.rados.rados.0.smithi084.stdout:1387: done (3 left) 2024-09-18T14:05:35.684 INFO:tasks.rados.rados.0.smithi084.stdout:1389: done (2 left) 2024-09-18T14:05:35.684 INFO:tasks.rados.rados.0.smithi084.stdout:1390: done (1 left) 2024-09-18T14:05:35.684 INFO:tasks.rados.rados.0.smithi084.stdout:1392: copy_from oid 2 from oid 6 current snap is 151 2024-09-18T14:05:35.684 INFO:tasks.rados.rados.0.smithi084.stdout:1393: write oid 32 current snap is 151 2024-09-18T14:05:35.684 INFO:tasks.rados.rados.0.smithi084.stdout:1393: seq_num 487 ranges {585009=492855,1564966=620043} 2024-09-18T14:05:35.689 INFO:tasks.rados.rados.0.smithi084.stdout:1393: writing smithi08434663-32 from 585009 to 1077864 tid 1 2024-09-18T14:05:35.696 INFO:tasks.rados.rados.0.smithi084.stdout:1393: writing smithi08434663-32 from 1564966 to 2185009 tid 2 2024-09-18T14:05:35.696 INFO:tasks.rados.rados.0.smithi084.stdout:1394: delete oid 16 current snap is 151 2024-09-18T14:05:35.696 INFO:tasks.rados.rados.0.smithi084.stdout:1391: finishing write tid 1 to smithi08434663-43 2024-09-18T14:05:35.696 INFO:tasks.rados.rados.0.smithi084.stdout:1391: finishing write tid 2 to smithi08434663-43 2024-09-18T14:05:35.696 INFO:tasks.rados.rados.0.smithi084.stdout:1391: finishing write tid 3 to smithi08434663-43 2024-09-18T14:05:35.696 INFO:tasks.rados.rados.0.smithi084.stdout:1391: finishing write tid 4 to smithi08434663-43 2024-09-18T14:05:35.696 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 400 (ObjNum 486 snap 151 seq_num 486) dirty exists 2024-09-18T14:05:35.696 INFO:tasks.rados.rados.0.smithi084.stdout:1391: left oid 43 (ObjNum 486 snap 151 seq_num 486) 2024-09-18T14:05:35.696 INFO:tasks.rados.rados.0.smithi084.stdout:1392: finishing copy_from racing read to smithi08434663-2 2024-09-18T14:05:35.697 INFO:tasks.rados.rados.0.smithi084.stdout:1393: finishing write tid 1 to smithi08434663-32 2024-09-18T14:05:35.699 INFO:tasks.rados.rados.0.smithi084.stdout:1391: done (3 left) 2024-09-18T14:05:35.699 INFO:tasks.rados.rados.0.smithi084.stdout:1394: done (2 left) 2024-09-18T14:05:35.699 INFO:tasks.rados.rados.0.smithi084.stdout:1395: snap_create 2024-09-18T14:05:35.700 INFO:tasks.rados.rados.0.smithi084.stdout:1392: finishing copy_from to smithi08434663-2 2024-09-18T14:05:35.700 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 558 (ObjNum 461 snap 141 seq_num 461) dirty exists 2024-09-18T14:05:35.701 INFO:tasks.rados.rados.0.smithi084.stdout:1393: finishing write tid 2 to smithi08434663-32 2024-09-18T14:05:35.703 INFO:tasks.rados.rados.0.smithi084.stdout:1393: finishing write tid 3 to smithi08434663-32 2024-09-18T14:05:35.703 INFO:tasks.rados.rados.0.smithi084.stdout:1393: finishing write tid 4 to smithi08434663-32 2024-09-18T14:05:35.703 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 721 (ObjNum 487 snap 151 seq_num 487) dirty exists 2024-09-18T14:05:35.703 INFO:tasks.rados.rados.0.smithi084.stdout:1393: left oid 32 (ObjNum 487 snap 151 seq_num 487) 2024-09-18T14:05:36.343 INFO:tasks.rados.rados.0.smithi084.stdout:1392: done (2 left) 2024-09-18T14:05:36.343 INFO:tasks.rados.rados.0.smithi084.stdout:1393: done (1 left) 2024-09-18T14:05:36.343 INFO:tasks.rados.rados.0.smithi084.stdout:1395: done (0 left) 2024-09-18T14:05:36.343 INFO:tasks.rados.rados.0.smithi084.stdout:1396: rollback oid 42 current snap is 152 2024-09-18T14:05:36.343 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 42 to 133 2024-09-18T14:05:36.343 INFO:tasks.rados.rados.0.smithi084.stdout:1397: write oid 37 current snap is 152 2024-09-18T14:05:36.343 INFO:tasks.rados.rados.0.smithi084.stdout:1397: seq_num 489 ranges {618623=643353,1681930=536693} 2024-09-18T14:05:36.349 INFO:tasks.rados.rados.0.smithi084.stdout:1397: writing smithi08434663-37 from 618623 to 1261976 tid 1 2024-09-18T14:05:36.355 INFO:tasks.rados.rados.0.smithi084.stdout:1397: writing smithi08434663-37 from 1681930 to 2218623 tid 2 2024-09-18T14:05:36.355 INFO:tasks.rados.rados.0.smithi084.stdout:1398: write oid 29 current snap is 152 2024-09-18T14:05:36.355 INFO:tasks.rados.rados.0.smithi084.stdout:1396: finishing rollback tid 0 to smithi08434663-42 2024-09-18T14:05:36.355 INFO:tasks.rados.rados.0.smithi084.stdout:1396: finishing rollback tid 1 to smithi08434663-42 2024-09-18T14:05:36.356 INFO:tasks.rados.rados.0.smithi084.stdout:1398: seq_num 490 ranges {635430=718602,2140412=95018} 2024-09-18T14:05:36.362 INFO:tasks.rados.rados.0.smithi084.stdout:1398: writing smithi08434663-29 from 635430 to 1354032 tid 1 2024-09-18T14:05:36.368 INFO:tasks.rados.rados.0.smithi084.stdout:1398: writing smithi08434663-29 from 2140412 to 2235430 tid 2 2024-09-18T14:05:36.369 INFO:tasks.rados.rados.0.smithi084.stdout:1399: copy_from oid 4 from oid 32 current snap is 152 2024-09-18T14:05:36.369 INFO:tasks.rados.rados.0.smithi084.stdout:1396: finishing rollback tid 2 to smithi08434663-42 2024-09-18T14:05:36.369 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 598 (ObjNum 423 snap 133 seq_num 423) dirty exists 2024-09-18T14:05:36.369 INFO:tasks.rados.rados.0.smithi084.stdout:1397: finishing write tid 1 to smithi08434663-37 2024-09-18T14:05:36.369 INFO:tasks.rados.rados.0.smithi084.stdout:1397: finishing write tid 2 to smithi08434663-37 2024-09-18T14:05:36.369 INFO:tasks.rados.rados.0.smithi084.stdout:1397: finishing write tid 3 to smithi08434663-37 2024-09-18T14:05:36.369 INFO:tasks.rados.rados.0.smithi084.stdout:1397: finishing write tid 4 to smithi08434663-37 2024-09-18T14:05:36.369 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 400 (ObjNum 489 snap 152 seq_num 489) dirty exists 2024-09-18T14:05:36.370 INFO:tasks.rados.rados.0.smithi084.stdout:1397: left oid 37 (ObjNum 489 snap 152 seq_num 489) 2024-09-18T14:05:36.370 INFO:tasks.rados.rados.0.smithi084.stdout:1396: done (3 left) 2024-09-18T14:05:36.370 INFO:tasks.rados.rados.0.smithi084.stdout:1397: done (2 left) 2024-09-18T14:05:36.370 INFO:tasks.rados.rados.0.smithi084.stdout:1400: read oid 18 snap -1 2024-09-18T14:05:36.370 INFO:tasks.rados.rados.0.smithi084.stdout:1400: expect (ObjNum 463 snap 142 seq_num 463) 2024-09-18T14:05:36.370 INFO:tasks.rados.rados.0.smithi084.stdout:1401: write (excl) oid 30 current snap is 152 2024-09-18T14:05:36.371 INFO:tasks.rados.rados.0.smithi084.stdout:1398: finishing write tid 1 to smithi08434663-29 2024-09-18T14:05:36.371 INFO:tasks.rados.rados.0.smithi084.stdout:1401: seq_num 491 ranges {652237=793851,2198894=53343} 2024-09-18T14:05:36.375 INFO:tasks.rados.rados.0.smithi084.stdout:1401: writing smithi08434663-30 from 652237 to 1446088 tid 1 2024-09-18T14:05:36.380 INFO:tasks.rados.rados.0.smithi084.stdout:1401: writing smithi08434663-30 from 2198894 to 2252237 tid 2 2024-09-18T14:05:36.380 INFO:tasks.rados.rados.0.smithi084.stdout:1402: write oid 3 current snap is 152 2024-09-18T14:05:36.380 INFO:tasks.rados.rados.0.smithi084.stdout:1398: finishing write tid 2 to smithi08434663-29 2024-09-18T14:05:36.380 INFO:tasks.rados.rados.0.smithi084.stdout:1402: seq_num 492 ranges {669044=469100,1741023=459161} 2024-09-18T14:05:36.386 INFO:tasks.rados.rados.0.smithi084.stdout:1402: writing smithi08434663-3 from 669044 to 1138144 tid 1 2024-09-18T14:05:36.392 INFO:tasks.rados.rados.0.smithi084.stdout:1402: writing smithi08434663-3 from 1741023 to 2200184 tid 2 2024-09-18T14:05:36.392 INFO:tasks.rados.rados.0.smithi084.stdout:1403: snap_remove snap 144 2024-09-18T14:05:37.207 INFO:tasks.rados.rados.0.smithi084.stdout:1403: done (5 left) 2024-09-18T14:05:37.208 INFO:tasks.rados.rados.0.smithi084.stdout:1404: delete oid 23 current snap is 152 2024-09-18T14:05:37.208 INFO:tasks.rados.rados.0.smithi084.stdout:1398: finishing write tid 3 to smithi08434663-29 2024-09-18T14:05:37.208 INFO:tasks.rados.rados.0.smithi084.stdout:1398: finishing write tid 4 to smithi08434663-29 2024-09-18T14:05:37.208 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 561 (ObjNum 490 snap 152 seq_num 490) dirty exists 2024-09-18T14:05:37.208 INFO:tasks.rados.rados.0.smithi084.stdout:1398: left oid 29 (ObjNum 490 snap 152 seq_num 490) 2024-09-18T14:05:37.209 INFO:tasks.rados.rados.0.smithi084.stdout:1400: expect (ObjNum 463 snap 142 seq_num 463) 2024-09-18T14:05:37.335 INFO:tasks.rados.rados.0.smithi084.stdout:1399: finishing copy_from racing read to smithi08434663-4 2024-09-18T14:05:37.335 INFO:tasks.rados.rados.0.smithi084.stdout:1399: finishing copy_from to smithi08434663-4 2024-09-18T14:05:37.335 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 721 (ObjNum 487 snap 151 seq_num 487) dirty exists 2024-09-18T14:05:37.335 INFO:tasks.rados.rados.0.smithi084.stdout:1401: finishing write tid 1 to smithi08434663-30 2024-09-18T14:05:37.335 INFO:tasks.rados.rados.0.smithi084.stdout:1401: finishing write tid 2 to smithi08434663-30 2024-09-18T14:05:37.335 INFO:tasks.rados.rados.0.smithi084.stdout:1401: finishing write tid 3 to smithi08434663-30 2024-09-18T14:05:37.335 INFO:tasks.rados.rados.0.smithi084.stdout:1401: finishing write tid 4 to smithi08434663-30 2024-09-18T14:05:37.335 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 549 (ObjNum 491 snap 152 seq_num 491) dirty exists 2024-09-18T14:05:37.336 INFO:tasks.rados.rados.0.smithi084.stdout:1401: left oid 30 (ObjNum 491 snap 152 seq_num 491) 2024-09-18T14:05:37.336 INFO:tasks.rados.rados.0.smithi084.stdout:1402: finishing write tid 1 to smithi08434663-3 2024-09-18T14:05:37.336 INFO:tasks.rados.rados.0.smithi084.stdout:1398: done (5 left) 2024-09-18T14:05:37.336 INFO:tasks.rados.rados.0.smithi084.stdout:1399: done (4 left) 2024-09-18T14:05:37.336 INFO:tasks.rados.rados.0.smithi084.stdout:1400: done (3 left) 2024-09-18T14:05:37.336 INFO:tasks.rados.rados.0.smithi084.stdout:1401: done (2 left) 2024-09-18T14:05:37.336 INFO:tasks.rados.rados.0.smithi084.stdout:1404: done (1 left) 2024-09-18T14:05:37.336 INFO:tasks.rados.rados.0.smithi084.stdout:1405: write oid 5 current snap is 152 2024-09-18T14:05:37.336 INFO:tasks.rados.rados.0.smithi084.stdout:1402: finishing write tid 2 to smithi08434663-3 2024-09-18T14:05:37.337 INFO:tasks.rados.rados.0.smithi084.stdout:1405: seq_num 494 ranges {702658=619598,1857987=444671} 2024-09-18T14:05:37.341 INFO:tasks.rados.rados.0.smithi084.stdout:1405: writing smithi08434663-5 from 702658 to 1322256 tid 1 2024-09-18T14:05:37.348 INFO:tasks.rados.rados.0.smithi084.stdout:1405: writing smithi08434663-5 from 1857987 to 2302658 tid 2 2024-09-18T14:05:37.348 INFO:tasks.rados.rados.0.smithi084.stdout:1406: read oid 12 snap 151 2024-09-18T14:05:37.348 INFO:tasks.rados.rados.0.smithi084.stdout:1406: expect deleted 2024-09-18T14:05:37.348 INFO:tasks.rados.rados.0.smithi084.stdout:1407: read oid 46 snap -1 2024-09-18T14:05:37.349 INFO:tasks.rados.rados.0.smithi084.stdout:1407: expect (ObjNum 450 snap 139 seq_num 450) 2024-09-18T14:05:37.349 INFO:tasks.rados.rados.0.smithi084.stdout:1408: rollback oid 14 current snap is 152 2024-09-18T14:05:37.349 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 14 to 148 2024-09-18T14:05:37.349 INFO:tasks.rados.rados.0.smithi084.stdout:1402: finishing write tid 3 to smithi08434663-3 2024-09-18T14:05:37.349 INFO:tasks.rados.rados.0.smithi084.stdout:1402: finishing write tid 4 to smithi08434663-3 2024-09-18T14:05:37.349 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 684 (ObjNum 492 snap 152 seq_num 492) dirty exists 2024-09-18T14:05:37.349 INFO:tasks.rados.rados.0.smithi084.stdout:1402: left oid 3 (ObjNum 492 snap 152 seq_num 492) 2024-09-18T14:05:37.349 INFO:tasks.rados.rados.0.smithi084.stdout:1405: finishing write tid 1 to smithi08434663-5 2024-09-18T14:05:37.349 INFO:tasks.rados.rados.0.smithi084.stdout:1402: done (4 left) 2024-09-18T14:05:37.349 INFO:tasks.rados.rados.0.smithi084.stdout:1409: read oid 25 snap -1 2024-09-18T14:05:37.349 INFO:tasks.rados.rados.0.smithi084.stdout:1409: expect deleted 2024-09-18T14:05:37.350 INFO:tasks.rados.rados.0.smithi084.stdout:1410: rollback oid 50 current snap is 152 2024-09-18T14:05:37.350 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 50 to 141 2024-09-18T14:05:37.350 INFO:tasks.rados.rados.0.smithi084.stdout:1411: read oid 29 snap -1 2024-09-18T14:05:37.350 INFO:tasks.rados.rados.0.smithi084.stdout:1411: expect (ObjNum 490 snap 152 seq_num 490) 2024-09-18T14:05:37.350 INFO:tasks.rados.rados.0.smithi084.stdout:1412: read oid 44 snap -1 2024-09-18T14:05:37.350 INFO:tasks.rados.rados.0.smithi084.stdout:1412: expect (ObjNum 470 snap 147 seq_num 470) 2024-09-18T14:05:37.350 INFO:tasks.rados.rados.0.smithi084.stdout:1413: write oid 10 current snap is 152 2024-09-18T14:05:37.350 INFO:tasks.rados.rados.0.smithi084.stdout:1413: seq_num 495 ranges {719465=411200,1632822=686643} 2024-09-18T14:05:37.354 INFO:tasks.rados.rados.0.smithi084.stdout:1413: writing smithi08434663-10 from 719465 to 1130665 tid 1 2024-09-18T14:05:37.362 INFO:tasks.rados.rados.0.smithi084.stdout:1413: writing smithi08434663-10 from 1632822 to 2319465 tid 2 2024-09-18T14:05:37.362 INFO:tasks.rados.rados.0.smithi084.stdout:1414: snap_create 2024-09-18T14:05:37.362 INFO:tasks.rados.rados.0.smithi084.stdout:1407: expect (ObjNum 450 snap 139 seq_num 450) 2024-09-18T14:05:37.641 INFO:tasks.rados.rados.0.smithi084.stdout:1411: expect (ObjNum 490 snap 152 seq_num 490) 2024-09-18T14:05:37.713 INFO:tasks.rados.rados.0.smithi084.stdout:1412: expect (ObjNum 470 snap 147 seq_num 470) 2024-09-18T14:05:37.837 INFO:tasks.rados.rados.0.smithi084.stdout:1405: finishing write tid 2 to smithi08434663-5 2024-09-18T14:05:37.837 INFO:tasks.rados.rados.0.smithi084.stdout:1405: finishing write tid 3 to smithi08434663-5 2024-09-18T14:05:37.837 INFO:tasks.rados.rados.0.smithi084.stdout:1405: finishing write tid 4 to smithi08434663-5 2024-09-18T14:05:37.837 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 214 (ObjNum 494 snap 152 seq_num 494) dirty exists 2024-09-18T14:05:37.837 INFO:tasks.rados.rados.0.smithi084.stdout:1405: left oid 5 (ObjNum 494 snap 152 seq_num 494) 2024-09-18T14:05:37.838 INFO:tasks.rados.rados.0.smithi084.stdout:1408: finishing rollback tid 0 to smithi08434663-14 2024-09-18T14:05:37.838 INFO:tasks.rados.rados.0.smithi084.stdout:1408: finishing rollback tid 1 to smithi08434663-14 2024-09-18T14:05:37.838 INFO:tasks.rados.rados.0.smithi084.stdout:1408: finishing rollback tid 2 to smithi08434663-14 2024-09-18T14:05:37.838 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 729 (ObjNum 387 snap 119 seq_num 387) dirty exists 2024-09-18T14:05:37.838 INFO:tasks.rados.rados.0.smithi084.stdout:1410: finishing rollback tid 0 to smithi08434663-50 2024-09-18T14:05:37.838 INFO:tasks.rados.rados.0.smithi084.stdout:1410: finishing rollback tid 1 to smithi08434663-50 2024-09-18T14:05:37.838 INFO:tasks.rados.rados.0.smithi084.stdout:1410: finishing rollback tid 2 to smithi08434663-50 2024-09-18T14:05:37.838 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 733 (ObjNum 386 snap 119 seq_num 386) dirty exists 2024-09-18T14:05:37.838 INFO:tasks.rados.rados.0.smithi084.stdout:1413: finishing write tid 1 to smithi08434663-10 2024-09-18T14:05:37.839 INFO:tasks.rados.rados.0.smithi084.stdout:1413: finishing write tid 2 to smithi08434663-10 2024-09-18T14:05:37.839 INFO:tasks.rados.rados.0.smithi084.stdout:1413: finishing write tid 3 to smithi08434663-10 2024-09-18T14:05:37.839 INFO:tasks.rados.rados.0.smithi084.stdout:1413: finishing write tid 4 to smithi08434663-10 2024-09-18T14:05:37.839 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 558 (ObjNum 495 snap 152 seq_num 495) dirty exists 2024-09-18T14:05:37.839 INFO:tasks.rados.rados.0.smithi084.stdout:1413: left oid 10 (ObjNum 495 snap 152 seq_num 495) 2024-09-18T14:05:38.349 INFO:tasks.rados.rados.0.smithi084.stdout:1405: done (9 left) 2024-09-18T14:05:38.350 INFO:tasks.rados.rados.0.smithi084.stdout:1406: done (8 left) 2024-09-18T14:05:38.350 INFO:tasks.rados.rados.0.smithi084.stdout:1407: done (7 left) 2024-09-18T14:05:38.350 INFO:tasks.rados.rados.0.smithi084.stdout:1408: done (6 left) 2024-09-18T14:05:38.350 INFO:tasks.rados.rados.0.smithi084.stdout:1409: done (5 left) 2024-09-18T14:05:38.350 INFO:tasks.rados.rados.0.smithi084.stdout:1410: done (4 left) 2024-09-18T14:05:38.350 INFO:tasks.rados.rados.0.smithi084.stdout:1411: done (3 left) 2024-09-18T14:05:38.350 INFO:tasks.rados.rados.0.smithi084.stdout:1412: done (2 left) 2024-09-18T14:05:38.350 INFO:tasks.rados.rados.0.smithi084.stdout:1413: done (1 left) 2024-09-18T14:05:38.350 INFO:tasks.rados.rados.0.smithi084.stdout:1414: done (0 left) 2024-09-18T14:05:38.350 INFO:tasks.rados.rados.0.smithi084.stdout:1415: copy_from oid 35 from oid 2 current snap is 153 2024-09-18T14:05:38.350 INFO:tasks.rados.rados.0.smithi084.stdout:1416: read oid 15 snap -1 2024-09-18T14:05:38.350 INFO:tasks.rados.rados.0.smithi084.stdout:1416: expect deleted 2024-09-18T14:05:38.351 INFO:tasks.rados.rados.0.smithi084.stdout:1417: read oid 42 snap 152 2024-09-18T14:05:38.351 INFO:tasks.rados.rados.0.smithi084.stdout:1417: expect (ObjNum 423 snap 133 seq_num 423) 2024-09-18T14:05:38.351 INFO:tasks.rados.rados.0.smithi084.stdout:1418: read oid 10 snap 149 2024-09-18T14:05:38.351 INFO:tasks.rados.rados.0.smithi084.stdout:1418: expect (ObjNum 474 snap 148 seq_num 474) 2024-09-18T14:05:38.351 INFO:tasks.rados.rados.0.smithi084.stdout:1419: write oid 38 current snap is 153 2024-09-18T14:05:38.351 INFO:tasks.rados.rados.0.smithi084.stdout:1419: seq_num 496 ranges {736272=486449,1974951=361321} 2024-09-18T14:05:38.356 INFO:tasks.rados.rados.0.smithi084.stdout:1419: writing smithi08434663-38 from 736272 to 1222721 tid 1 2024-09-18T14:05:38.363 INFO:tasks.rados.rados.0.smithi084.stdout:1419: writing smithi08434663-38 from 1974951 to 2336272 tid 2 2024-09-18T14:05:38.363 INFO:tasks.rados.rados.0.smithi084.stdout:1420: rollback oid 9 current snap is 153 2024-09-18T14:05:38.363 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 9 to 146 2024-09-18T14:05:38.364 INFO:tasks.rados.rados.0.smithi084.stdout:1418: expect (ObjNum 474 snap 148 seq_num 474) 2024-09-18T14:05:38.583 INFO:tasks.rados.rados.0.smithi084.stdout:1417: expect (ObjNum 423 snap 133 seq_num 423) 2024-09-18T14:05:38.774 INFO:tasks.rados.rados.0.smithi084.stdout:1415: finishing copy_from racing read to smithi08434663-35 2024-09-18T14:05:38.775 INFO:tasks.rados.rados.0.smithi084.stdout:1419: finishing write tid 1 to smithi08434663-38 2024-09-18T14:05:38.775 INFO:tasks.rados.rados.0.smithi084.stdout:1419: finishing write tid 2 to smithi08434663-38 2024-09-18T14:05:38.775 INFO:tasks.rados.rados.0.smithi084.stdout:1419: finishing write tid 3 to smithi08434663-38 2024-09-18T14:05:38.775 INFO:tasks.rados.rados.0.smithi084.stdout:1419: finishing write tid 4 to smithi08434663-38 2024-09-18T14:05:38.775 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 403 (ObjNum 496 snap 153 seq_num 496) dirty exists 2024-09-18T14:05:38.775 INFO:tasks.rados.rados.0.smithi084.stdout:1419: left oid 38 (ObjNum 496 snap 153 seq_num 496) 2024-09-18T14:05:38.775 INFO:tasks.rados.rados.0.smithi084.stdout:1415: finishing copy_from to smithi08434663-35 2024-09-18T14:05:38.775 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 558 (ObjNum 461 snap 141 seq_num 461) dirty exists 2024-09-18T14:05:38.775 INFO:tasks.rados.rados.0.smithi084.stdout:1420: finishing rollback tid 0 to smithi08434663-9 2024-09-18T14:05:38.775 INFO:tasks.rados.rados.0.smithi084.stdout:1420: finishing rollback tid 1 to smithi08434663-9 2024-09-18T14:05:38.775 INFO:tasks.rados.rados.0.smithi084.stdout:1415: done (5 left) 2024-09-18T14:05:38.775 INFO:tasks.rados.rados.0.smithi084.stdout:1416: done (4 left) 2024-09-18T14:05:38.776 INFO:tasks.rados.rados.0.smithi084.stdout:1417: done (3 left) 2024-09-18T14:05:38.776 INFO:tasks.rados.rados.0.smithi084.stdout:1418: done (2 left) 2024-09-18T14:05:38.776 INFO:tasks.rados.rados.0.smithi084.stdout:1419: done (1 left) 2024-09-18T14:05:38.776 INFO:tasks.rados.rados.0.smithi084.stdout:1421: read oid 3 snap -1 2024-09-18T14:05:38.776 INFO:tasks.rados.rados.0.smithi084.stdout:1421: expect (ObjNum 492 snap 152 seq_num 492) 2024-09-18T14:05:38.776 INFO:tasks.rados.rados.0.smithi084.stdout:1422: write oid 1 current snap is 153 2024-09-18T14:05:38.776 INFO:tasks.rados.rados.0.smithi084.stdout:1420: finishing rollback tid 2 to smithi08434663-9 2024-09-18T14:05:38.776 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 561 (ObjNum 429 snap 134 seq_num 429) dirty exists 2024-09-18T14:05:38.776 INFO:tasks.rados.rados.0.smithi084.stdout:1422: seq_num 497 ranges {753079=561698,2033433=319646} 2024-09-18T14:05:38.781 INFO:tasks.rados.rados.0.smithi084.stdout:1422: writing smithi08434663-1 from 753079 to 1314777 tid 1 2024-09-18T14:05:38.787 INFO:tasks.rados.rados.0.smithi084.stdout:1422: writing smithi08434663-1 from 2033433 to 2353079 tid 2 2024-09-18T14:05:38.787 INFO:tasks.rados.rados.0.smithi084.stdout:1420: done (2 left) 2024-09-18T14:05:38.787 INFO:tasks.rados.rados.0.smithi084.stdout:1423: write oid 11 current snap is 153 2024-09-18T14:05:38.787 INFO:tasks.rados.rados.0.smithi084.stdout:1423: seq_num 498 ranges {769886=636947,2091915=277971} 2024-09-18T14:05:38.795 INFO:tasks.rados.rados.0.smithi084.stdout:1423: writing smithi08434663-11 from 769886 to 1406833 tid 1 2024-09-18T14:05:38.800 INFO:tasks.rados.rados.0.smithi084.stdout:1423: writing smithi08434663-11 from 2091915 to 2369886 tid 2 2024-09-18T14:05:38.800 INFO:tasks.rados.rados.0.smithi084.stdout:1424: snap_remove snap 148 2024-09-18T14:05:39.354 INFO:tasks.rados.rados.0.smithi084.stdout:1424: done (3 left) 2024-09-18T14:05:39.354 INFO:tasks.rados.rados.0.smithi084.stdout:1425: copy_from oid 23 from oid 49 current snap is 153 2024-09-18T14:05:39.354 INFO:tasks.rados.rados.0.smithi084.stdout:1421: expect (ObjNum 492 snap 152 seq_num 492) 2024-09-18T14:05:39.508 INFO:tasks.rados.rados.0.smithi084.stdout:1422: finishing write tid 1 to smithi08434663-1 2024-09-18T14:05:39.508 INFO:tasks.rados.rados.0.smithi084.stdout:1422: finishing write tid 2 to smithi08434663-1 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:1422: finishing write tid 3 to smithi08434663-1 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:1422: finishing write tid 4 to smithi08434663-1 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 562 (ObjNum 497 snap 153 seq_num 497) dirty exists 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:1422: left oid 1 (ObjNum 497 snap 153 seq_num 497) 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:1423: finishing write tid 1 to smithi08434663-11 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:1421: done (3 left) 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:1422: done (2 left) 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:1426: read oid 39 snap -1 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:1426: expect (ObjNum 450 snap 139 seq_num 450) 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:1427: delete oid 48 current snap is 153 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:1423: finishing write tid 2 to smithi08434663-11 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:1423: finishing write tid 3 to smithi08434663-11 2024-09-18T14:05:39.509 INFO:tasks.rados.rados.0.smithi084.stdout:1423: finishing write tid 4 to smithi08434663-11 2024-09-18T14:05:39.510 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 484 (ObjNum 498 snap 153 seq_num 498) dirty exists 2024-09-18T14:05:39.510 INFO:tasks.rados.rados.0.smithi084.stdout:1423: left oid 11 (ObjNum 498 snap 153 seq_num 498) 2024-09-18T14:05:39.510 INFO:tasks.rados.rados.0.smithi084.stdout:1425: finishing copy_from racing read to smithi08434663-23 2024-09-18T14:05:39.510 INFO:tasks.rados.rados.0.smithi084.stdout:1425: finishing copy_from to smithi08434663-23 2024-09-18T14:05:39.510 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 380 (ObjNum 397 snap 121 seq_num 397) dirty exists 2024-09-18T14:05:39.510 INFO:tasks.rados.rados.0.smithi084.stdout:1423: done (3 left) 2024-09-18T14:05:39.510 INFO:tasks.rados.rados.0.smithi084.stdout:1425: done (2 left) 2024-09-18T14:05:39.510 INFO:tasks.rados.rados.0.smithi084.stdout:1427: done (1 left) 2024-09-18T14:05:39.510 INFO:tasks.rados.rados.0.smithi084.stdout:1428: write oid 10 current snap is 153 2024-09-18T14:05:39.511 INFO:tasks.rados.rados.0.smithi084.stdout:1428: seq_num 500 ranges {403500=787445,1692526=473837} 2024-09-18T14:05:39.516 INFO:tasks.rados.rados.0.smithi084.stdout:1428: writing smithi08434663-10 from 403500 to 1190945 tid 1 2024-09-18T14:05:39.522 INFO:tasks.rados.rados.0.smithi084.stdout:1428: writing smithi08434663-10 from 1692526 to 2166363 tid 2 2024-09-18T14:05:39.522 INFO:tasks.rados.rados.0.smithi084.stdout:1429: snap_remove snap 146 2024-09-18T14:05:40.362 INFO:tasks.rados.rados.0.smithi084.stdout:1429: done (2 left) 2024-09-18T14:05:40.363 INFO:tasks.rados.rados.0.smithi084.stdout:1430: copy_from oid 25 from oid 31 current snap is 153 2024-09-18T14:05:40.363 INFO:tasks.rados.rados.0.smithi084.stdout:1426: expect (ObjNum 450 snap 139 seq_num 450) 2024-09-18T14:05:40.642 INFO:tasks.rados.rados.0.smithi084.stdout:1428: finishing write tid 1 to smithi08434663-10 2024-09-18T14:05:40.643 INFO:tasks.rados.rados.0.smithi084.stdout:1428: finishing write tid 2 to smithi08434663-10 2024-09-18T14:05:40.643 INFO:tasks.rados.rados.0.smithi084.stdout:1428: finishing write tid 3 to smithi08434663-10 2024-09-18T14:05:40.643 INFO:tasks.rados.rados.0.smithi084.stdout:1428: finishing write tid 4 to smithi08434663-10 2024-09-18T14:05:40.643 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 570 (ObjNum 500 snap 153 seq_num 500) dirty exists 2024-09-18T14:05:40.643 INFO:tasks.rados.rados.0.smithi084.stdout:1428: left oid 10 (ObjNum 500 snap 153 seq_num 500) 2024-09-18T14:05:40.643 INFO:tasks.rados.rados.0.smithi084.stdout:1426: done (2 left) 2024-09-18T14:05:40.643 INFO:tasks.rados.rados.0.smithi084.stdout:1428: done (1 left) 2024-09-18T14:05:40.643 INFO:tasks.rados.rados.0.smithi084.stdout:1431: read oid 15 snap -1 2024-09-18T14:05:40.643 INFO:tasks.rados.rados.0.smithi084.stdout:1431: expect deleted 2024-09-18T14:05:40.643 INFO:tasks.rados.rados.0.smithi084.stdout:1432: snap_remove snap 147 2024-09-18T14:05:41.366 INFO:tasks.rados.rados.0.smithi084.stdout:1432: done (2 left) 2024-09-18T14:05:41.366 INFO:tasks.rados.rados.0.smithi084.stdout:1433: write oid 46 current snap is 153 2024-09-18T14:05:41.366 INFO:tasks.rados.rados.0.smithi084.stdout:1433: seq_num 501 ranges {420307=462694,1351008=617495} 2024-09-18T14:05:41.371 INFO:tasks.rados.rados.0.smithi084.stdout:1433: writing smithi08434663-46 from 420307 to 883001 tid 1 2024-09-18T14:05:41.378 INFO:tasks.rados.rados.0.smithi084.stdout:1433: writing smithi08434663-46 from 1351008 to 1968503 tid 2 2024-09-18T14:05:41.378 INFO:tasks.rados.rados.0.smithi084.stdout:1434: snap_create 2024-09-18T14:05:41.378 INFO:tasks.rados.rados.0.smithi084.stdout:1430: finishing copy_from racing read to smithi08434663-25 2024-09-18T14:05:41.378 INFO:tasks.rados.rados.0.smithi084.stdout:1430: finishing copy_from to smithi08434663-25 2024-09-18T14:05:41.378 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 365 (ObjNum 480 snap 151 seq_num 480) dirty exists 2024-09-18T14:05:41.388 INFO:tasks.rados.rados.0.smithi084.stdout:1433: finishing write tid 1 to smithi08434663-46 2024-09-18T14:05:41.390 INFO:tasks.rados.rados.0.smithi084.stdout:1433: finishing write tid 2 to smithi08434663-46 2024-09-18T14:05:41.391 INFO:tasks.rados.rados.0.smithi084.stdout:1433: finishing write tid 3 to smithi08434663-46 2024-09-18T14:05:41.391 INFO:tasks.rados.rados.0.smithi084.stdout:1433: finishing write tid 4 to smithi08434663-46 2024-09-18T14:05:41.391 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 564 (ObjNum 501 snap 153 seq_num 501) dirty exists 2024-09-18T14:05:41.391 INFO:tasks.rados.rados.0.smithi084.stdout:1433: left oid 46 (ObjNum 501 snap 153 seq_num 501) 2024-09-18T14:05:42.384 INFO:tasks.rados.rados.0.smithi084.stdout:1430: done (3 left) 2024-09-18T14:05:42.384 INFO:tasks.rados.rados.0.smithi084.stdout:1431: done (2 left) 2024-09-18T14:05:42.384 INFO:tasks.rados.rados.0.smithi084.stdout:1433: done (1 left) 2024-09-18T14:05:42.384 INFO:tasks.rados.rados.0.smithi084.stdout:1434: done (0 left) 2024-09-18T14:05:42.384 INFO:tasks.rados.rados.0.smithi084.stdout:1435: read oid 35 snap -1 2024-09-18T14:05:42.384 INFO:tasks.rados.rados.0.smithi084.stdout:1435: expect (ObjNum 461 snap 141 seq_num 461) 2024-09-18T14:05:42.384 INFO:tasks.rados.rados.0.smithi084.stdout:1436: rollback oid 33 current snap is 154 2024-09-18T14:05:42.385 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 33 to 142 2024-09-18T14:05:42.385 INFO:tasks.rados.rados.0.smithi084.stdout:1437: copy_from oid 20 from oid 19 current snap is 154 2024-09-18T14:05:42.385 INFO:tasks.rados.rados.0.smithi084.stdout:1438: write (excl) oid 25 current snap is 154 2024-09-18T14:05:42.385 INFO:tasks.rados.rados.0.smithi084.stdout:1438: seq_num 502 ranges {437114=654296,1525843=477506} 2024-09-18T14:05:42.388 INFO:tasks.rados.rados.0.smithi084.stdout:1438: writing smithi08434663-25 from 437114 to 1091410 tid 1 2024-09-18T14:05:42.393 INFO:tasks.rados.rados.0.smithi084.stdout:1438: writing smithi08434663-25 from 1525843 to 2003349 tid 2 2024-09-18T14:05:42.394 INFO:tasks.rados.rados.0.smithi084.stdout:1439: write oid 37 current snap is 154 2024-09-18T14:05:42.394 INFO:tasks.rados.rados.0.smithi084.stdout:1439: seq_num 503 ranges {453921=729545,1584325=621164} 2024-09-18T14:05:42.400 INFO:tasks.rados.rados.0.smithi084.stdout:1439: writing smithi08434663-37 from 453921 to 1183466 tid 1 2024-09-18T14:05:42.406 INFO:tasks.rados.rados.0.smithi084.stdout:1439: writing smithi08434663-37 from 1584325 to 2205489 tid 2 2024-09-18T14:05:42.406 INFO:tasks.rados.rados.0.smithi084.stdout:1440: snap_create 2024-09-18T14:05:42.406 INFO:tasks.rados.rados.0.smithi084.stdout:1435: expect (ObjNum 461 snap 141 seq_num 461) 2024-09-18T14:05:42.523 INFO:tasks.rados.rados.0.smithi084.stdout:1436: finishing rollback tid 0 to smithi08434663-33 2024-09-18T14:05:42.523 INFO:tasks.rados.rados.0.smithi084.stdout:1436: finishing rollback tid 1 to smithi08434663-33 2024-09-18T14:05:42.523 INFO:tasks.rados.rados.0.smithi084.stdout:1436: finishing rollback tid 2 to smithi08434663-33 2024-09-18T14:05:42.523 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 403 (ObjNum 458 snap 141 seq_num 458) dirty exists 2024-09-18T14:05:42.523 INFO:tasks.rados.rados.0.smithi084.stdout:1437: finishing copy_from racing read to smithi08434663-20 2024-09-18T14:05:42.523 INFO:tasks.rados.rados.0.smithi084.stdout:1437: finishing copy_from to smithi08434663-20 2024-09-18T14:05:42.523 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 614 (ObjNum 453 snap 139 seq_num 453) dirty exists 2024-09-18T14:05:42.523 INFO:tasks.rados.rados.0.smithi084.stdout:1438: finishing write tid 1 to smithi08434663-25 2024-09-18T14:05:42.523 INFO:tasks.rados.rados.0.smithi084.stdout:1438: finishing write tid 2 to smithi08434663-25 2024-09-18T14:05:42.524 INFO:tasks.rados.rados.0.smithi084.stdout:1438: finishing write tid 3 to smithi08434663-25 2024-09-18T14:05:42.524 INFO:tasks.rados.rados.0.smithi084.stdout:1438: finishing write tid 4 to smithi08434663-25 2024-09-18T14:05:42.524 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 387 (ObjNum 502 snap 154 seq_num 502) dirty exists 2024-09-18T14:05:42.524 INFO:tasks.rados.rados.0.smithi084.stdout:1438: left oid 25 (ObjNum 502 snap 154 seq_num 502) 2024-09-18T14:05:42.524 INFO:tasks.rados.rados.0.smithi084.stdout:1439: finishing write tid 1 to smithi08434663-37 2024-09-18T14:05:42.524 INFO:tasks.rados.rados.0.smithi084.stdout:1439: finishing write tid 2 to smithi08434663-37 2024-09-18T14:05:42.524 INFO:tasks.rados.rados.0.smithi084.stdout:1439: finishing write tid 3 to smithi08434663-37 2024-09-18T14:05:42.524 INFO:tasks.rados.rados.0.smithi084.stdout:1439: finishing write tid 4 to smithi08434663-37 2024-09-18T14:05:42.524 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 406 (ObjNum 503 snap 154 seq_num 503) dirty exists 2024-09-18T14:05:42.524 INFO:tasks.rados.rados.0.smithi084.stdout:1439: left oid 37 (ObjNum 503 snap 154 seq_num 503) 2024-09-18T14:05:43.392 INFO:tasks.rados.rados.0.smithi084.stdout:1435: done (5 left) 2024-09-18T14:05:43.392 INFO:tasks.rados.rados.0.smithi084.stdout:1436: done (4 left) 2024-09-18T14:05:43.393 INFO:tasks.rados.rados.0.smithi084.stdout:1437: done (3 left) 2024-09-18T14:05:43.393 INFO:tasks.rados.rados.0.smithi084.stdout:1438: done (2 left) 2024-09-18T14:05:43.393 INFO:tasks.rados.rados.0.smithi084.stdout:1439: done (1 left) 2024-09-18T14:05:43.393 INFO:tasks.rados.rados.0.smithi084.stdout:1440: done (0 left) 2024-09-18T14:05:43.393 INFO:tasks.rados.rados.0.smithi084.stdout:1441: snap_remove snap 154 2024-09-18T14:05:44.393 INFO:tasks.rados.rados.0.smithi084.stdout:1441: done (0 left) 2024-09-18T14:05:44.393 INFO:tasks.rados.rados.0.smithi084.stdout:1442: write (excl) oid 13 current snap is 155 2024-09-18T14:05:44.393 INFO:tasks.rados.rados.0.smithi084.stdout:1442: seq_num 504 ranges {470728=404794,1526454=481175} 2024-09-18T14:05:44.398 INFO:tasks.rados.rados.0.smithi084.stdout:1442: writing smithi08434663-13 from 470728 to 875522 tid 1 2024-09-18T14:05:44.404 INFO:tasks.rados.rados.0.smithi084.stdout:1442: writing smithi08434663-13 from 1526454 to 2007629 tid 2 2024-09-18T14:05:44.404 INFO:tasks.rados.rados.0.smithi084.stdout:1443: rollback oid 18 current snap is 155 2024-09-18T14:05:44.404 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 18 to 142 2024-09-18T14:05:44.404 INFO:tasks.rados.rados.0.smithi084.stdout:1442: finishing write tid 1 to smithi08434663-13 2024-09-18T14:05:44.404 INFO:tasks.rados.rados.0.smithi084.stdout:1444: write (excl) oid 16 current snap is 155 2024-09-18T14:05:44.404 INFO:tasks.rados.rados.0.smithi084.stdout:1444: seq_num 505 ranges {487535=480043,1584936=624833} 2024-09-18T14:05:44.409 INFO:tasks.rados.rados.0.smithi084.stdout:1444: writing smithi08434663-16 from 487535 to 967578 tid 1 2024-09-18T14:05:44.416 INFO:tasks.rados.rados.0.smithi084.stdout:1444: writing smithi08434663-16 from 1584936 to 2209769 tid 2 2024-09-18T14:05:44.417 INFO:tasks.rados.rados.0.smithi084.stdout:1445: delete oid 21 current snap is 155 2024-09-18T14:05:44.417 INFO:tasks.rados.rados.0.smithi084.stdout:1442: finishing write tid 2 to smithi08434663-13 2024-09-18T14:05:44.417 INFO:tasks.rados.rados.0.smithi084.stdout:1442: finishing write tid 3 to smithi08434663-13 2024-09-18T14:05:44.417 INFO:tasks.rados.rados.0.smithi084.stdout:1442: finishing write tid 4 to smithi08434663-13 2024-09-18T14:05:44.417 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 564 (ObjNum 504 snap 155 seq_num 504) dirty exists 2024-09-18T14:05:44.417 INFO:tasks.rados.rados.0.smithi084.stdout:1442: left oid 13 (ObjNum 504 snap 155 seq_num 504) 2024-09-18T14:05:44.417 INFO:tasks.rados.rados.0.smithi084.stdout:1443: finishing rollback tid 0 to smithi08434663-18 2024-09-18T14:05:44.417 INFO:tasks.rados.rados.0.smithi084.stdout:1443: finishing rollback tid 1 to smithi08434663-18 2024-09-18T14:05:44.417 INFO:tasks.rados.rados.0.smithi084.stdout:1443: finishing rollback tid 2 to smithi08434663-18 2024-09-18T14:05:44.417 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 687 (ObjNum 463 snap 142 seq_num 463) dirty exists 2024-09-18T14:05:44.417 INFO:tasks.rados.rados.0.smithi084.stdout:1444: finishing write tid 1 to smithi08434663-16 2024-09-18T14:05:44.420 INFO:tasks.rados.rados.0.smithi084.stdout:1444: finishing write tid 2 to smithi08434663-16 2024-09-18T14:05:44.420 INFO:tasks.rados.rados.0.smithi084.stdout:1444: finishing write tid 3 to smithi08434663-16 2024-09-18T14:05:44.420 INFO:tasks.rados.rados.0.smithi084.stdout:1444: finishing write tid 4 to smithi08434663-16 2024-09-18T14:05:44.420 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 567 (ObjNum 505 snap 155 seq_num 505) dirty exists 2024-09-18T14:05:44.420 INFO:tasks.rados.rados.0.smithi084.stdout:1444: left oid 16 (ObjNum 505 snap 155 seq_num 505) 2024-09-18T14:05:44.421 INFO:tasks.rados.rados.0.smithi084.stdout:1442: done (3 left) 2024-09-18T14:05:44.421 INFO:tasks.rados.rados.0.smithi084.stdout:1443: done (2 left) 2024-09-18T14:05:44.421 INFO:tasks.rados.rados.0.smithi084.stdout:1444: done (1 left) 2024-09-18T14:05:44.421 INFO:tasks.rados.rados.0.smithi084.stdout:1445: done (0 left) 2024-09-18T14:05:44.421 INFO:tasks.rados.rados.0.smithi084.stdout:1446: snap_create 2024-09-18T14:05:45.403 INFO:tasks.rados.rados.0.smithi084.stdout:1446: done (0 left) 2024-09-18T14:05:45.403 INFO:tasks.rados.rados.0.smithi084.stdout:1447: write (excl) oid 7 current snap is 156 2024-09-18T14:05:45.403 INFO:tasks.rados.rados.0.smithi084.stdout:1447: seq_num 507 ranges {521149=630541,1701900=628502} 2024-09-18T14:05:45.409 INFO:tasks.rados.rados.0.smithi084.stdout:1447: writing smithi08434663-7 from 521149 to 1151690 tid 1 2024-09-18T14:05:45.415 INFO:tasks.rados.rados.0.smithi084.stdout:1447: writing smithi08434663-7 from 1701900 to 2330402 tid 2 2024-09-18T14:05:45.415 INFO:tasks.rados.rados.0.smithi084.stdout:1448: snap_remove snap 133 2024-09-18T14:05:46.167 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-18T14:05:46.167 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-18T14:05:46.167 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-18T14:05:46.167 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-18T14:05:46.171 INFO:tasks.ceph.mon.g:Started 2024-09-18T14:05:46.171 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-18T14:05:46.171 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-18T14:05:46.171 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T14:05:46.171 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T14:05:46.174 INFO:tasks.ceph.mon.c:Started 2024-09-18T14:05:46.174 INFO:tasks.mon_thrash:Sending STOP to mon i 2024-09-18T14:05:46.175 INFO:tasks.ceph.mon.i:Sent signal 19 2024-09-18T14:05:46.175 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-18T14:06:01.176 INFO:tasks.mon_thrash:Sending CONT to mon i 2024-09-18T14:06:01.177 INFO:tasks.ceph.mon.i:Sent signal 18 2024-09-18T14:06:01.177 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:06:01.177 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:06:01.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.295+0000 7f033f4e8640 10 monclient: get_monmap_and_config 2024-09-18T14:06:01.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.295+0000 7f033f4e8640 10 monclient: build_initial_monmap 2024-09-18T14:06:01.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.295+0000 7f033f4e8640 10 monclient: monmap: 2024-09-18T14:06:01.296 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:01.296 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.296 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:01.296163+0000 2024-09-18T14:06:01.296 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:01.296163+0000 2024-09-18T14:06:01.296 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:01.296 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.296 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:01.296 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:01.296 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:01.296 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:01.296 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:01.297 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:01.297 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:01.297 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:01.297 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:01.297 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.295+0000 7f033f4e8640 1 Processor -- start 2024-09-18T14:06:01.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033f4e8640 1 -- start start 2024-09-18T14:06:01.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033f4e8640 10 monclient: init 2024-09-18T14:06:01.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033f4e8640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033f4e8640 10 monclient: _add_conns ranks=[0,5,6,4,1,8,7,2,3] 2024-09-18T14:06:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033f4e8640 10 monclient(hunting): picked mon.noname-a con 0x7f033806e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033f4e8640 10 monclient(hunting): picked mon.noname-h con 0x7f03380a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033f4e8640 10 monclient(hunting): picked mon.noname-d con 0x7f03380a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033f4e8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033805ed70 con 0x7f033806e810 2024-09-18T14:06:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033f4e8640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0338097620 con 0x7f03380a0cb0 2024-09-18T14:06:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033f4e8640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0338097340 con 0x7f03380a1080 2024-09-18T14:06:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033f4e8640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033d284640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f033806e810 0x7f03380a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33524/0 (socket says 172.21.15.45:33524) 2024-09-18T14:06:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033f4e8640 10 monclient(hunting): authenticate will time out at 1176.634808s 2024-09-18T14:06:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.296+0000 7f033d284640 1 -- 172.21.15.45:0/595631628 learned_addr learned my addr 172.21.15.45:0/595631628 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513940427 0 0) 0x7f0338097340 con 0x7f03380a1080 2024-09-18T14:06:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient(hunting): my global_id is 4956 2024-09-18T14:06:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03240034e0 con 0x7f03380a1080 2024-09-18T14:06:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798372031 0 0) 0x7f033805ed70 con 0x7f033806e810 2024-09-18T14:06:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient(hunting): my global_id is 5058 2024-09-18T14:06:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0338097340 con 0x7f033806e810 2024-09-18T14:06:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114757096 0 0) 0x7f0338097620 con 0x7f03380a0cb0 2024-09-18T14:06:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient(hunting): my global_id is 4910 2024-09-18T14:06:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f033805ed70 con 0x7f03380a0cb0 2024-09-18T14:06:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4257795499 0 0) 0x7f03240034e0 con 0x7f03380a1080 2024-09-18T14:06:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 >> v1:172.21.15.84:6791/0 conn(0x7f03380a0cb0 legacy=0x7f0338137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 >> v1:172.21.15.45:6789/0 conn(0x7f033806e810 legacy=0x7f03380a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:06:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 monclient: found mon.noname-d 2024-09-18T14:06:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:06:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03380a1660 con 0x7f03380a1080 2024-09-18T14:06:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient: _finish_auth 0 2024-09-18T14:06:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f033f4e8640 5 monclient: authenticate success, global_id 4956 2024-09-18T14:06:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0330001ae0 con 0x7f03380a1080 2024-09-18T14:06:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f033f4e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:06:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient: dump: 2024-09-18T14:06:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f033f4e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0330001cc0 con 0x7f03380a1080 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f033f4e8640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 1 -- 172.21.15.45:0/595631628 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f03300021f0 con 0x7f03380a1080 2024-09-18T14:06:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f033f4e8640 4 set_mon_vals no callback set 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.297+0000 7f032bfff640 10 monclient: dump: 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.298+0000 7f033f4e8640 10 monclient: shutdown 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.298+0000 7f033f4e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.298+0000 7f033f4e8640 1 -- 172.21.15.45:0/595631628 >> v1:172.21.15.45:6792/0 conn(0x7f03380a1080 legacy=0x7f033813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.298+0000 7f033f4e8640 1 -- 172.21.15.45:0/595631628 shutdown_connections 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.298+0000 7f033f4e8640 1 -- 172.21.15.45:0/595631628 >> 172.21.15.45:0/595631628 conn(0x7f03380a5bb0 msgr2=0x7f03380a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.298+0000 7f033f4e8640 1 -- 172.21.15.45:0/595631628 shutdown_connections 2024-09-18T14:06:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.298+0000 7f033f4e8640 1 -- 172.21.15.45:0/595631628 wait complete. 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.298+0000 7f033f4e8640 10 monclient: build_initial_monmap 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.298+0000 7f033f4e8640 10 monclient: monmap: 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:01.314 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 1 Processor -- start 2024-09-18T14:06:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 1 -- start start 2024-09-18T14:06:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 10 monclient: init 2024-09-18T14:06:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 10 monclient: _add_conns ranks=[2,1,7,0,4,6,8,5,3] 2024-09-18T14:06:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 10 monclient(hunting): picked mon.noname-b con 0x7f033806e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 10 monclient(hunting): picked mon.noname-f con 0x7f03380a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 10 monclient(hunting): picked mon.noname-i con 0x7f03380a1080 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0338122d20 con 0x7f03380a0cb0 2024-09-18T14:06:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033813db10 con 0x7f033806e810 2024-09-18T14:06:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0338122760 con 0x7f03380a1080 2024-09-18T14:06:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033f4e8640 10 monclient(hunting): authenticate will time out at 1176.637971s 2024-09-18T14:06:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033d284640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f033806e810 0x7f033813d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39338/0 (socket says 172.21.15.45:39338) 2024-09-18T14:06:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.299+0000 7f033d284640 1 -- 172.21.15.45:0/921027508 learned_addr learned my addr 172.21.15.45:0/921027508 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121702824 0 0) 0x7f033813db10 con 0x7f033806e810 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): my global_id is 144326 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f031c0034e0 con 0x7f033806e810 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4094033317 0 0) 0x7f031c0034e0 con 0x7f033806e810 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f033813db10 con 0x7f033806e810 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0334001610 con 0x7f033806e810 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): dump: 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.317 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346041334 0 0) 0x7f0338122760 con 0x7f03380a1080 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): my global_id is 5002 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f031c0034e0 con 0x7f03380a1080 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818166139 0 0) 0x7f0338122d20 con 0x7f03380a0cb0 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient(hunting): my global_id is 4996 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0338122760 con 0x7f03380a0cb0 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4201462310 0 0) 0x7f033813db10 con 0x7f033806e810 2024-09-18T14:06:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 >> v1:172.21.15.84:6792/0 conn(0x7f03380a1080 legacy=0x7f0338149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 >> v1:172.21.15.84:6789/0 conn(0x7f03380a0cb0 legacy=0x7f0338146630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient: _finish_hunting 0 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 monclient: found mon.b 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03381202b0 con 0x7f033806e810 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient: _finish_auth 0 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f033f4e8640 5 monclient: authenticate success, global_id 144326 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f033f4e8640 10 monclient: _renew_subs 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f033f4e8640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f033f4e8640 1 -- 172.21.15.45:0/921027508 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0338122480 con 0x7f033806e810 2024-09-18T14:06:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f033f4e8640 10 monclient: _renew_subs 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f033f4e8640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f033f4e8640 1 -- 172.21.15.45:0/921027508 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0338121750 con 0x7f033806e810 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0334004a50 con 0x7f033806e810 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0334004f40 con 0x7f033806e810 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f032b7fe640 4 set_mon_vals no callback set 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.300+0000 7f03297fa640 10 monclient: dump: 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.301+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f033400ff10 con 0x7f033806e810 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.301+0000 7f033f4e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.301+0000 7f033f4e8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:01.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.301+0000 7f033f4e8640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:01.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.301+0000 7f033f4e8640 1 -- 172.21.15.45:0/921027508 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0318005010 con 0x7f033806e810 2024-09-18T14:06:01.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.301+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(324..324 src has 126..324) v4 ==== 3975+0+0 (unknown 727861532 0 0) 0x7f0334046b90 con 0x7f033806e810 2024-09-18T14:06:01.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.302+0000 7f033ca83640 10 monclient: get_auth_request con 0x7f031c039ee0 auth_method 0 2024-09-18T14:06:01.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.306+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f0334015390 con 0x7f033806e810 2024-09-18T14:06:01.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.306+0000 7f03297fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:01.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.306+0000 7f03297fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:01.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.530+0000 7f033f4e8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:06:01.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.530+0000 7f033f4e8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:06:01.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.530+0000 7f033f4e8640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:01.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.530+0000 7f033f4e8640 1 -- 172.21.15.45:0/921027508 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0318005840 con 0x7f033806e810 2024-09-18T14:06:01.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.530+0000 7f03297fa640 1 -- 172.21.15.45:0/921027508 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 939897023) 0x7f0334014d90 con 0x7f033806e810 2024-09-18T14:06:01.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.530+0000 7f03297fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:06:01.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.530+0000 7f03297fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:01.531 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":76,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:06:01.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.532+0000 7f033f4e8640 1 -- 172.21.15.45:0/921027508 >> v1:172.21.15.84:6812/31478 conn(0x7f031c039ee0 legacy=0x7f031c03c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.532+0000 7f033f4e8640 10 monclient: shutdown 2024-09-18T14:06:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.532+0000 7f033f4e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.532+0000 7f033f4e8640 1 -- 172.21.15.45:0/921027508 >> v1:172.21.15.45:6790/0 conn(0x7f033806e810 legacy=0x7f033813d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.532+0000 7f033f4e8640 1 -- 172.21.15.45:0/921027508 shutdown_connections 2024-09-18T14:06:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.532+0000 7f033f4e8640 1 -- 172.21.15.45:0/921027508 >> 172.21.15.45:0/921027508 conn(0x7f03380a5bb0 msgr2=0x7f03380a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.532+0000 7f033f4e8640 1 -- 172.21.15.45:0/921027508 shutdown_connections 2024-09-18T14:06:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.532+0000 7f033f4e8640 1 -- 172.21.15.45:0/921027508 wait complete. 2024-09-18T14:06:01.543 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:06:01.543 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:06:01.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:06:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.639+0000 7f7500f5d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.639+0000 7f7500f5d640 10 monclient: build_initial_monmap 2024-09-18T14:06:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.640+0000 7f7500f5d640 10 monclient: monmap: 2024-09-18T14:06:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:01.640961+0000 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:01.640961+0000 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:01.641 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.640+0000 7f7500f5d640 1 Processor -- start 2024-09-18T14:06:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.640+0000 7f7500f5d640 1 -- start start 2024-09-18T14:06:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.640+0000 7f7500f5d640 10 monclient: init 2024-09-18T14:06:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f7500f5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f7500f5d640 10 monclient: _add_conns ranks=[6,3,4,7,0,2,5,1,8] 2024-09-18T14:06:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f7500f5d640 10 monclient(hunting): picked mon.noname-d con 0x7f74fc09f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f7500f5d640 10 monclient(hunting): picked mon.noname-g con 0x7f74fc09fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f7500f5d640 10 monclient(hunting): picked mon.noname-c con 0x7f74fc09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f7500f5d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74fc05ed70 con 0x7f74fc09fba0 2024-09-18T14:06:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f7500f5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74fc09dfc0 con 0x7f74fc09ff70 2024-09-18T14:06:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f7500f5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74fc09dce0 con 0x7f74fc09f7d0 2024-09-18T14:06:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f7500f5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f7500f5d640 10 monclient(hunting): authenticate will time out at 1176.979317s 2024-09-18T14:06:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74fad9d640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f74fc09ff70 0x7f74fc13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58452/0 (socket says 172.21.15.45:58452) 2024-09-18T14:06:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74fa59c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f74fc09f7d0 0x7f74fc132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:54788/0 (socket says 172.21.15.45:54788) 2024-09-18T14:06:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74fad9d640 1 -- 172.21.15.45:0/1508835318 learned_addr learned my addr 172.21.15.45:0/1508835318 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873683740 0 0) 0x7f74fc09dfc0 con 0x7f74fc09ff70 2024-09-18T14:06:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 10 monclient(hunting): my global_id is 154102 2024-09-18T14:06:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74e40034e0 con 0x7f74fc09ff70 2024-09-18T14:06:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313447650 0 0) 0x7f74fc09dce0 con 0x7f74fc09f7d0 2024-09-18T14:06:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 10 monclient(hunting): my global_id is 4965 2024-09-18T14:06:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74fc09dfc0 con 0x7f74fc09f7d0 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236342872 0 0) 0x7f74fc05ed70 con 0x7f74fc09fba0 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 10 monclient(hunting): my global_id is 154101 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74fc09dce0 con 0x7f74fc09fba0 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1862169258 0 0) 0x7f74fc09dfc0 con 0x7f74fc09f7d0 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 >> v1:172.21.15.45:6791/0 conn(0x7f74fc09ff70 legacy=0x7f74fc13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 >> v1:172.21.15.84:6790/0 conn(0x7f74fc09fba0 legacy=0x7f74fc137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 10 monclient: _finish_hunting 0 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 1 monclient: found mon.noname-d 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.641+0000 7f74f959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:06:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74fc0a1c00 con 0x7f74fc09f7d0 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 10 monclient: _finish_auth 0 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f74ec001b90 con 0x7f74fc09f7d0 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f7500f5d640 5 monclient: authenticate success, global_id 4965 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f7500f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 10 monclient: dump: 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f7500f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f74ec001cf0 con 0x7f74fc09f7d0 2024-09-18T14:06:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 1 -- 172.21.15.45:0/1508835318 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f74ec004e60 con 0x7f74fc09f7d0 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f7500f5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f7500f5d640 4 set_mon_vals no callback set 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f74f959a640 10 monclient: dump: 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f7500f5d640 10 monclient: shutdown 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f7500f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f7500f5d640 1 -- 172.21.15.45:0/1508835318 >> v1:172.21.15.45:6792/0 conn(0x7f74fc09f7d0 legacy=0x7f74fc132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f7500f5d640 1 -- 172.21.15.45:0/1508835318 shutdown_connections 2024-09-18T14:06:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f7500f5d640 1 -- 172.21.15.45:0/1508835318 >> 172.21.15.45:0/1508835318 conn(0x7f74fc0630b0 msgr2=0x7f74fc0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f7500f5d640 1 -- 172.21.15.45:0/1508835318 shutdown_connections 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.642+0000 7f7500f5d640 1 -- 172.21.15.45:0/1508835318 wait complete. 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.643+0000 7f7500f5d640 10 monclient: build_initial_monmap 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.643+0000 7f7500f5d640 10 monclient: monmap: 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.643+0000 7f7500f5d640 1 Processor -- start 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.643+0000 7f7500f5d640 1 -- start start 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.643+0000 7f7500f5d640 10 monclient: init 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.643+0000 7f7500f5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.643+0000 7f7500f5d640 10 monclient: _add_conns ranks=[5,7,3,8,4,1,6,0,2] 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.643+0000 7f7500f5d640 10 monclient(hunting): picked mon.noname-h con 0x7f74fc09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.643+0000 7f7500f5d640 10 monclient(hunting): picked mon.noname-i con 0x7f74fc09fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f7500f5d640 10 monclient(hunting): picked mon.noname-g con 0x7f74fc09ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f7500f5d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74fc09d310 con 0x7f74fc09ff70 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f7500f5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74fc0a2560 con 0x7f74fc09f7d0 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f7500f5d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74fc11e960 con 0x7f74fc09fba0 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f7500f5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f7500f5d640 10 monclient(hunting): authenticate will time out at 1176.982074s 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74fad9d640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7f74fc09ff70 0x7f74fc14a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:60890/0 (socket says 172.21.15.45:60890) 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74fa59c640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f74fc09f7d0 0x7f74fc131560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:52926/0 (socket says 172.21.15.45:52926) 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74fad9d640 1 -- 172.21.15.45:0/135298965 learned_addr learned my addr 172.21.15.45:0/135298965 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282567563 0 0) 0x7f74fc09d310 con 0x7f74fc09ff70 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): my global_id is 154110 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74dc003510 con 0x7f74fc09ff70 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329907845 0 0) 0x7f74fc11e960 con 0x7f74fc09fba0 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): my global_id is 5011 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74fc09d310 con 0x7f74fc09fba0 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082224228 0 0) 0x7f74fc0a2560 con 0x7f74fc09f7d0 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): my global_id is 4919 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74fc11e960 con 0x7f74fc09f7d0 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2546475614 0 0) 0x7f74dc003510 con 0x7f74fc09ff70 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74fc0a2560 con 0x7f74fc09ff70 2024-09-18T14:06:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 783538517 0 0) 0x7f74fc11e960 con 0x7f74fc09f7d0 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74dc003510 con 0x7f74fc09f7d0 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2359378174 0 0) 0x7f74fc09d310 con 0x7f74fc09fba0 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74fc11e960 con 0x7f74fc09fba0 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f74f4001960 con 0x7f74fc09ff70 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): dump: 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.662 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f74ec001ad0 con 0x7f74fc09f7d0 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): dump: 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f74f0001a20 con 0x7f74fc09fba0 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient(hunting): dump: 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 150868126 0 0) 0x7f74fc0a2560 con 0x7f74fc09ff70 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 >> v1:172.21.15.84:6792/0 conn(0x7f74fc09fba0 legacy=0x7f74fc146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 >> v1:172.21.15.84:6791/0 conn(0x7f74fc09f7d0 legacy=0x7f74fc131560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 monclient: found mon.g 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74fc11e680 con 0x7f74fc09ff70 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 10 monclient: _finish_auth 0 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f74ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f7500f5d640 5 monclient: authenticate success, global_id 154110 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f7500f5d640 10 monclient: _renew_subs 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.644+0000 7f7500f5d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f7500f5d640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74fc09b7e0 con 0x7f74fc09ff70 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f7500f5d640 10 monclient: _renew_subs 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f7500f5d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:06:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f7500f5d640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f74fc11d720 con 0x7f74fc09ff70 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f74f4001c80 con 0x7f74fc09ff70 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f74ea7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f74f4004d90 con 0x7f74fc09ff70 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f74ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f74ea7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f74f8d99640 4 set_mon_vals no callback set 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f74ea7fc640 10 monclient: dump: 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.668 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f7500f5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f7500f5d640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f74e0006580 con 0x7f74e0002830 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.645+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f74f400fd60 con 0x7f74fc09ff70 2024-09-18T14:06:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(324..324 src has 126..324) v4 ==== 3975+0+0 (unknown 727861532 0 0) 0x7f74f4046a70 con 0x7f74fc09ff70 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638969776 0 0) 0x7f74e0006580 con 0x7f74e0002830 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 10 monclient(hunting): my global_id is 5067 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74fc0a2560 con 0x7f74e0002830 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74f9d9b640 10 monclient: get_auth_request con 0x7f74dc03aa10 auth_method 0 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1709209164 0 0) 0x7f74fc0a2560 con 0x7f74e0002830 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74e0006580 con 0x7f74e0002830 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f74ec005460 con 0x7f74e0002830 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 10 monclient: dump: 2024-09-18T14:06:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.672 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 440373529 0 0) 0x7f74e0006580 con 0x7f74e0002830 2024-09-18T14:06:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.646+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74e000a0a0 con 0x7f74e0002830 2024-09-18T14:06:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.647+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f74ec005700 con 0x7f74e0002830 2024-09-18T14:06:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.647+0000 7f74ea7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.647+0000 7f74ea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.647+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 >> v1:172.21.15.45:6789/0 conn(0x7f74e0002830 legacy=0x7f74e0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.647+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 >> v1:172.21.15.45:6789/0 conn(0x7f74e0002830 legacy=0x7f74e0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.652+0000 7f7500f5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.652+0000 7f7500f5d640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f74e0007520 con 0x7f74e00068d0 2024-09-18T14:06:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215795608 0 0) 0x7f74e0007520 con 0x7f74e00068d0 2024-09-18T14:06:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 10 monclient(hunting): my global_id is 5076 2024-09-18T14:06:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74e0006580 con 0x7f74e00068d0 2024-09-18T14:06:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3816772480 0 0) 0x7f74e0006580 con 0x7f74e00068d0 2024-09-18T14:06:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74e0007520 con 0x7f74e00068d0 2024-09-18T14:06:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f74ec005f40 con 0x7f74e00068d0 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 10 monclient: dump: 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.676 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3253064781 0 0) 0x7f74e0007520 con 0x7f74e00068d0 2024-09-18T14:06:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f74e00087a0 con 0x7f74e00068d0 2024-09-18T14:06:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3222064444) 0x7f74ec008160 con 0x7f74e00068d0 2024-09-18T14:06:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 >> v1:172.21.15.45:6789/0 conn(0x7f74e00068d0 legacy=0x7f74e0006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.653+0000 7f74ea7fc640 1 -- 172.21.15.45:0/135298965 >> v1:172.21.15.45:6789/0 conn(0x7f74e00068d0 legacy=0x7f74e0006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:01.677 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:01.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:01.677 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:01.677 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:01.677 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:01.677 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:01.677 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:01.677 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:01.677 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:01.677 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:01.678 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:01.679 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:01.680 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:01.680 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:01.680 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:01.680 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:01.680 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:01.680 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.680 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:01.680 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:01.680 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:01.680 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:01.680 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:01.681 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:01.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:01.683 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:01.684 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:01.685 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.686 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.688 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:01.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.690 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.693 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.694 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:01.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.696 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:01.696 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:01.696 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:06:01.696 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.696 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.696 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:01.696 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.654+0000 7f7500f5d640 1 -- 172.21.15.45:0/135298965 >> v1:172.21.15.84:6812/31478 conn(0x7f74dc03aa10 legacy=0x7f74dc03cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.654+0000 7f7500f5d640 10 monclient: shutdown 2024-09-18T14:06:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.654+0000 7f7500f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:01.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.654+0000 7f7500f5d640 1 -- 172.21.15.45:0/135298965 >> v1:172.21.15.84:6790/0 conn(0x7f74fc09ff70 legacy=0x7f74fc14a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.654+0000 7f74fad9d640 1 -- 172.21.15.45:0/135298965 reap_dead start 2024-09-18T14:06:01.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.654+0000 7f74fad9d640 1 -- 172.21.15.45:0/135298965 reap_dead start 2024-09-18T14:06:01.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.654+0000 7f7500f5d640 1 -- 172.21.15.45:0/135298965 shutdown_connections 2024-09-18T14:06:01.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.654+0000 7f7500f5d640 1 -- 172.21.15.45:0/135298965 >> 172.21.15.45:0/135298965 conn(0x7f74fc0630b0 msgr2=0x7f74fc0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:01.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.654+0000 7f7500f5d640 1 -- 172.21.15.45:0/135298965 shutdown_connections 2024-09-18T14:06:01.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.654+0000 7f7500f5d640 1 -- 172.21.15.45:0/135298965 wait complete. 2024-09-18T14:06:01.698 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:01.698 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:06:01.699 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:06:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.795+0000 7f22d6eaf640 10 monclient: get_monmap_and_config 2024-09-18T14:06:01.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.795+0000 7f22d6eaf640 10 monclient: build_initial_monmap 2024-09-18T14:06:01.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.795+0000 7f22d6eaf640 10 monclient: monmap: 2024-09-18T14:06:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:01.796522+0000 2024-09-18T14:06:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:01.796522+0000 2024-09-18T14:06:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:01.798 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:01.798 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:01.798 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:01.798 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:01.798 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 1 Processor -- start 2024-09-18T14:06:01.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 1 -- start start 2024-09-18T14:06:01.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 10 monclient: init 2024-09-18T14:06:01.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:01.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 10 monclient: _add_conns ranks=[7,8,2,3,4,0,1,6,5] 2024-09-18T14:06:01.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 10 monclient(hunting): picked mon.noname-i con 0x7f22d009f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:01.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 10 monclient(hunting): picked mon.noname-e con 0x7f22d009fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:01.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 10 monclient(hunting): picked mon.noname-b con 0x7f22d009ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d005ed70 con 0x7f22d009ff70 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d009dfc0 con 0x7f22d009f7d0 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d009dce0 con 0x7f22d009fba0 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.796+0000 7f22d6eaf640 10 monclient(hunting): authenticate will time out at 1177.134996s 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c7fff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f22d009fba0 0x7f22d0137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:39754/0 (socket says 172.21.15.45:39754) 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22d544c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f22d009ff70 0x7f22d013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39344/0 (socket says 172.21.15.45:39344) 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c7fff640 1 -- 172.21.15.45:0/1877597390 learned_addr learned my addr 172.21.15.45:0/1877597390 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472596335 0 0) 0x7f22d005ed70 con 0x7f22d009ff70 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient(hunting): my global_id is 144335 2024-09-18T14:06:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22b80034e0 con 0x7f22d009ff70 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651191791 0 0) 0x7f22d009dce0 con 0x7f22d009fba0 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient(hunting): my global_id is 4994 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d005ed70 con 0x7f22d009fba0 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290247024 0 0) 0x7f22d009dfc0 con 0x7f22d009f7d0 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient(hunting): my global_id is 5020 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d009dce0 con 0x7f22d009f7d0 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 74891504 0 0) 0x7f22b80034e0 con 0x7f22d009ff70 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 >> v1:172.21.15.45:6793/0 conn(0x7f22d009fba0 legacy=0x7f22d0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 >> v1:172.21.15.84:6792/0 conn(0x7f22d009f7d0 legacy=0x7f22d0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient: _finish_hunting 0 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 monclient: found mon.noname-b 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d00a1c00 con 0x7f22d009ff70 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient: _finish_auth 0 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f22c8001bb0 con 0x7f22d009ff70 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22d6eaf640 5 monclient: authenticate success, global_id 144335 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient: dump: 2024-09-18T14:06:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22d6eaf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f22c8001d90 con 0x7f22d009ff70 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22d6eaf640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 1 -- 172.21.15.45:0/1877597390 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f22c8002000 con 0x7f22d009ff70 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22d6eaf640 4 set_mon_vals no callback set 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.797+0000 7f22c77fe640 10 monclient: dump: 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.798+0000 7f22d6eaf640 10 monclient: shutdown 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.798+0000 7f22d6eaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.798+0000 7f22d6eaf640 1 -- 172.21.15.45:0/1877597390 >> v1:172.21.15.45:6790/0 conn(0x7f22d009ff70 legacy=0x7f22d013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.798+0000 7f22d6eaf640 1 -- 172.21.15.45:0/1877597390 shutdown_connections 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.798+0000 7f22d6eaf640 1 -- 172.21.15.45:0/1877597390 >> 172.21.15.45:0/1877597390 conn(0x7f22d00630b0 msgr2=0x7f22d00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.798+0000 7f22d6eaf640 1 -- 172.21.15.45:0/1877597390 shutdown_connections 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.798+0000 7f22d6eaf640 1 -- 172.21.15.45:0/1877597390 wait complete. 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.798+0000 7f22d6eaf640 10 monclient: build_initial_monmap 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.798+0000 7f22d6eaf640 10 monclient: monmap: 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:01.814 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 1 Processor -- start 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 1 -- start start 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 10 monclient: init 2024-09-18T14:06:01.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 10 monclient: _add_conns ranks=[2,1,7,4,8,0,3,5,6] 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 10 monclient(hunting): picked mon.noname-b con 0x7f22d009f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 10 monclient(hunting): picked mon.noname-f con 0x7f22d009fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 10 monclient(hunting): picked mon.noname-i con 0x7f22d009ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d01428c0 con 0x7f22d009fba0 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d009bdd0 con 0x7f22d009f7d0 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d00a2560 con 0x7f22d009ff70 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d6eaf640 10 monclient(hunting): authenticate will time out at 1177.137729s 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d4c4b640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f22d009f7d0 0x7f22d01316b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39354/0 (socket says 172.21.15.45:39354) 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.799+0000 7f22d4c4b640 1 -- 172.21.15.45:0/49511445 learned_addr learned my addr 172.21.15.45:0/49511445 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286789532 0 0) 0x7f22d009bdd0 con 0x7f22d009f7d0 2024-09-18T14:06:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): my global_id is 144344 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22bc0034e0 con 0x7f22d009f7d0 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831085609 0 0) 0x7f22d00a2560 con 0x7f22d009ff70 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): my global_id is 5029 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d009bdd0 con 0x7f22d009ff70 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736638398 0 0) 0x7f22d01428c0 con 0x7f22d009fba0 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): my global_id is 5005 2024-09-18T14:06:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d00a2560 con 0x7f22d009fba0 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3499825338 0 0) 0x7f22bc0034e0 con 0x7f22d009f7d0 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22d01428c0 con 0x7f22d009f7d0 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f22cc002bf0 con 0x7f22d009f7d0 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 768894103 0 0) 0x7f22d009bdd0 con 0x7f22d009ff70 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22bc0034e0 con 0x7f22d009ff70 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f22c8001af0 con 0x7f22d009ff70 2024-09-18T14:06:01.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1793714039 0 0) 0x7f22d00a2560 con 0x7f22d009fba0 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22d009bdd0 con 0x7f22d009fba0 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f22c00019d0 con 0x7f22d009fba0 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.821 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.822 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 268418759 0 0) 0x7f22d01428c0 con 0x7f22d009f7d0 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 >> v1:172.21.15.84:6792/0 conn(0x7f22d009ff70 legacy=0x7f22d014a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 >> v1:172.21.15.84:6789/0 conn(0x7f22d009fba0 legacy=0x7f22d0146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 monclient: found mon.b 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d009d310 con 0x7f22d009f7d0 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient: _finish_auth 0 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22d6eaf640 5 monclient: authenticate success, global_id 144344 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22d6eaf640 10 monclient: _renew_subs 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22d6eaf640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22d6eaf640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d011d870 con 0x7f22d009f7d0 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22d6eaf640 10 monclient: _renew_subs 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22d6eaf640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22d6eaf640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d0136ea0 con 0x7f22d009f7d0 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f22cc001660 con 0x7f22d009f7d0 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f22cc001bd0 con 0x7f22d009f7d0 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c6ffd640 4 set_mon_vals no callback set 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.800+0000 7f22c4ff9640 10 monclient: dump: 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.801+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f22cc001d30 con 0x7f22d009f7d0 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.801+0000 7f22d6eaf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.801+0000 7f22d6eaf640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f22b0006580 con 0x7f22b0002830 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.801+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(324..324 src has 126..324) v4 ==== 3975+0+0 (unknown 727861532 0 0) 0x7f22cc046b30 con 0x7f22d009f7d0 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.801+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148173370 0 0) 0x7f22b0006580 con 0x7f22b0002830 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.801+0000 7f22c4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.801+0000 7f22c4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.801+0000 7f22c4ff9640 10 monclient(hunting): my global_id is 144353 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.801+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d01428c0 con 0x7f22b0002830 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.801+0000 7f22d544c640 10 monclient: get_auth_request con 0x7f22bc03a9a0 auth_method 0 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.801+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 339962378 0 0) 0x7f22d01428c0 con 0x7f22b0002830 2024-09-18T14:06:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.801+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22b0006580 con 0x7f22b0002830 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.802+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f22c0004ae0 con 0x7f22b0002830 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.802+0000 7f22c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.802+0000 7f22c4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.802+0000 7f22c4ff9640 10 monclient: dump: 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.827 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.802+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 319998255 0 0) 0x7f22b0006580 con 0x7f22b0002830 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.802+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22b000a0a0 con 0x7f22b0002830 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.802+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f22c0005890 con 0x7f22b0002830 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.802+0000 7f22c4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.802+0000 7f22c4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:01.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.802+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 >> v1:172.21.15.45:6790/0 conn(0x7f22b0002830 legacy=0x7f22b0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.802+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 >> v1:172.21.15.45:6790/0 conn(0x7f22b0002830 legacy=0x7f22b0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:01.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22d6eaf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:01.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22d6eaf640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f22b0009640 con 0x7f22b0009020 2024-09-18T14:06:01.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119417865 0 0) 0x7f22b0009640 con 0x7f22b0009020 2024-09-18T14:06:01.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 10 monclient(hunting): my global_id is 144362 2024-09-18T14:06:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22b0006580 con 0x7f22b0009020 2024-09-18T14:06:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2561546021 0 0) 0x7f22b0006580 con 0x7f22b0009020 2024-09-18T14:06:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22b0009640 con 0x7f22b0009020 2024-09-18T14:06:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f22c0005420 con 0x7f22b0009020 2024-09-18T14:06:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 10 monclient: dump: 2024-09-18T14:06:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1963379016 0 0) 0x7f22b0009640 con 0x7f22b0009020 2024-09-18T14:06:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.808+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f22b0009900 con 0x7f22b0009020 2024-09-18T14:06:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 792322884) 0x7f22c0005b20 con 0x7f22b0009020 2024-09-18T14:06:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22c4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22c4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 >> v1:172.21.15.45:6790/0 conn(0x7f22b0009020 legacy=0x7f22b0008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22c4ff9640 1 -- 172.21.15.45:0/49511445 >> v1:172.21.15.45:6790/0 conn(0x7f22b0009020 legacy=0x7f22b0008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:01.833 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:01.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:01.833 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:01.833 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.848 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.850 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:01.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:01.850 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.850 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.850 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:01.850 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.850 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.850 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.851 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:01.851 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:01.851 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.851 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.851 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.851 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.851 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.851 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:01.851 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:01.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.853 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:01.853 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:01.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22d6eaf640 1 -- 172.21.15.45:0/49511445 >> v1:172.21.15.84:6812/31478 conn(0x7f22bc03a9a0 legacy=0x7f22bc03ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22d6eaf640 10 monclient: shutdown 2024-09-18T14:06:01.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22d6eaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:01.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22d6eaf640 1 -- 172.21.15.45:0/49511445 >> v1:172.21.15.45:6790/0 conn(0x7f22d009f7d0 legacy=0x7f22d01316b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22d544c640 1 -- 172.21.15.45:0/49511445 reap_dead start 2024-09-18T14:06:01.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22d544c640 1 -- 172.21.15.45:0/49511445 reap_dead start 2024-09-18T14:06:01.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22d6eaf640 1 -- 172.21.15.45:0/49511445 shutdown_connections 2024-09-18T14:06:01.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22d6eaf640 1 -- 172.21.15.45:0/49511445 >> 172.21.15.45:0/49511445 conn(0x7f22d00630b0 msgr2=0x7f22d00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:01.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22d6eaf640 1 -- 172.21.15.45:0/49511445 shutdown_connections 2024-09-18T14:06:01.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.809+0000 7f22d6eaf640 1 -- 172.21.15.45:0/49511445 wait complete. 2024-09-18T14:06:01.855 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:01.855 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:06:01.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:06:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.952+0000 7f35228c0640 10 monclient: get_monmap_and_config 2024-09-18T14:06:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.952+0000 7f35228c0640 10 monclient: build_initial_monmap 2024-09-18T14:06:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.952+0000 7f35228c0640 10 monclient: monmap: 2024-09-18T14:06:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:01.953030+0000 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:01.953030+0000 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:01.954 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.952+0000 7f35228c0640 1 Processor -- start 2024-09-18T14:06:01.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f35228c0640 1 -- start start 2024-09-18T14:06:01.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f35228c0640 10 monclient: init 2024-09-18T14:06:01.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f35228c0640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:01.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f35228c0640 10 monclient: _add_conns ranks=[2,3,1,5,7,6,8,4,0] 2024-09-18T14:06:01.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f35228c0640 10 monclient(hunting): picked mon.noname-b con 0x7f351c06e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:01.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f35228c0640 10 monclient(hunting): picked mon.noname-g con 0x7f351c0a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:01.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f35228c0640 10 monclient(hunting): picked mon.noname-f con 0x7f351c0a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:01.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f35228c0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c05ed70 con 0x7f351c0a1080 2024-09-18T14:06:01.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f35228c0640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c097620 con 0x7f351c06e810 2024-09-18T14:06:01.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f35228c0640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c097340 con 0x7f351c0a0cb0 2024-09-18T14:06:01.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f35228c0640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:01.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f35228c0640 10 monclient(hunting): authenticate will time out at 1177.291653s 2024-09-18T14:06:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f351bfff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f351c06e810 0x7f351c0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39376/0 (socket says 172.21.15.45:39376) 2024-09-18T14:06:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f351bfff640 1 -- 172.21.15.45:0/4176566761 learned_addr learned my addr 172.21.15.45:0/4176566761 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853177756 0 0) 0x7f351c097620 con 0x7f351c06e810 2024-09-18T14:06:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f351b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f351b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.953+0000 7f351b7fe640 10 monclient(hunting): my global_id is 144371 2024-09-18T14:06:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35080034e0 con 0x7f351c06e810 2024-09-18T14:06:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078334167 0 0) 0x7f351c05ed70 con 0x7f351c0a1080 2024-09-18T14:06:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient(hunting): my global_id is 5014 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f351c097620 con 0x7f351c0a1080 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587822332 0 0) 0x7f351c097340 con 0x7f351c0a0cb0 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient(hunting): my global_id is 154119 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f351c05ed70 con 0x7f351c0a0cb0 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 285185704 0 0) 0x7f35080034e0 con 0x7f351c06e810 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 >> v1:172.21.15.84:6790/0 conn(0x7f351c0a0cb0 legacy=0x7f351c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 >> v1:172.21.15.84:6789/0 conn(0x7f351c0a1080 legacy=0x7f351c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 monclient: found mon.noname-b 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c0a1660 con 0x7f351c06e810 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient: _finish_auth 0 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f35228c0640 5 monclient: authenticate success, global_id 144371 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f350c001830 con 0x7f351c06e810 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f35228c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient: dump: 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f35228c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f350c001a10 con 0x7f351c06e810 2024-09-18T14:06:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 1 -- 172.21.15.45:0/4176566761 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f350c004ba0 con 0x7f351c06e810 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f35228c0640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f35228c0640 4 set_mon_vals no callback set 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f351b7fe640 10 monclient: dump: 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f35228c0640 10 monclient: shutdown 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f35228c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f35228c0640 1 -- 172.21.15.45:0/4176566761 >> v1:172.21.15.45:6790/0 conn(0x7f351c06e810 legacy=0x7f351c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f35228c0640 1 -- 172.21.15.45:0/4176566761 shutdown_connections 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.954+0000 7f35228c0640 1 -- 172.21.15.45:0/4176566761 >> 172.21.15.45:0/4176566761 conn(0x7f351c0a5bb0 msgr2=0x7f351c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.955+0000 7f35228c0640 1 -- 172.21.15.45:0/4176566761 shutdown_connections 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.955+0000 7f35228c0640 1 -- 172.21.15.45:0/4176566761 wait complete. 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.955+0000 7f35228c0640 10 monclient: build_initial_monmap 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.955+0000 7f35228c0640 10 monclient: monmap: 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.955+0000 7f35228c0640 1 Processor -- start 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.955+0000 7f35228c0640 1 -- start start 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.955+0000 7f35228c0640 10 monclient: init 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f35228c0640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f35228c0640 10 monclient: _add_conns ranks=[8,5,6,1,2,3,7,0,4] 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f35228c0640 10 monclient(hunting): picked mon.noname-e con 0x7f351c06e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f35228c0640 10 monclient(hunting): picked mon.noname-h con 0x7f351c0a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f35228c0640 10 monclient(hunting): picked mon.noname-d con 0x7f351c0a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f35228c0640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c09fa10 con 0x7f351c0a0cb0 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f35228c0640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c13dcc0 con 0x7f351c0a1080 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f35228c0640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c0a2270 con 0x7f351c06e810 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f35228c0640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f35228c0640 10 monclient(hunting): authenticate will time out at 1177.294320s 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f351bfff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f351c06e810 0x7f351c13d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:39758/0 (socket says 172.21.15.45:39758) 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f351bfff640 1 -- 172.21.15.45:0/2248860926 learned_addr learned my addr 172.21.15.45:0/2248860926 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3191392285 0 0) 0x7f351c0a2270 con 0x7f351c06e810 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): my global_id is 5003 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34fc0034e0 con 0x7f351c06e810 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325961440 0 0) 0x7f351c13dcc0 con 0x7f351c0a1080 2024-09-18T14:06:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): my global_id is 4974 2024-09-18T14:06:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f351c0a2270 con 0x7f351c0a1080 2024-09-18T14:06:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852720808 0 0) 0x7f351c09fa10 con 0x7f351c0a0cb0 2024-09-18T14:06:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): my global_id is 4928 2024-09-18T14:06:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f351c13dcc0 con 0x7f351c0a0cb0 2024-09-18T14:06:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2676549854 0 0) 0x7f351c0a2270 con 0x7f351c0a1080 2024-09-18T14:06:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f351c09fa10 con 0x7f351c0a1080 2024-09-18T14:06:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3514001d40 con 0x7f351c0a1080 2024-09-18T14:06:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2194738785 0 0) 0x7f34fc0034e0 con 0x7f351c06e810 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f351c0a2270 con 0x7f351c06e810 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f350c0018b0 con 0x7f351c06e810 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.956+0000 7f3518ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4283989606 0 0) 0x7f351c09fa10 con 0x7f351c0a1080 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 >> v1:172.21.15.45:6793/0 conn(0x7f351c06e810 legacy=0x7f351c13d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 >> v1:172.21.15.84:6791/0 conn(0x7f351c0a0cb0 legacy=0x7f351c1467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 1 monclient: found mon.d 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c13ead0 con 0x7f351c0a1080 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 10 monclient: _finish_auth 0 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f35228c0640 5 monclient: authenticate success, global_id 4974 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f35228c0640 10 monclient: _renew_subs 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f35228c0640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:06:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f35228c0640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351c122650 con 0x7f351c0a1080 2024-09-18T14:06:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f35228c0640 10 monclient: _renew_subs 2024-09-18T14:06:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f35228c0640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:06:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f35228c0640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c120260 con 0x7f351c0a1080 2024-09-18T14:06:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3514003030 con 0x7f351c0a1080 2024-09-18T14:06:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3514002610 con 0x7f351c0a1080 2024-09-18T14:06:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f351affd640 4 set_mon_vals no callback set 2024-09-18T14:06:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f3518ff9640 10 monclient: dump: 2024-09-18T14:06:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f35228c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.957+0000 7f35228c0640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3500006580 con 0x7f3500002830 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f35140027f0 con 0x7f351c0a1080 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(324..324 src has 126..324) v4 ==== 3975+0+0 (unknown 727861532 0 0) 0x7f35140445e0 con 0x7f351c0a1080 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245922679 0 0) 0x7f3500006580 con 0x7f3500002830 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 10 monclient(hunting): my global_id is 154111 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f351c09fa10 con 0x7f3500002830 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3513fff640 10 monclient: get_auth_request con 0x7f34fc03a5a0 auth_method 0 2024-09-18T14:06:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1217966000 0 0) 0x7f351c09fa10 con 0x7f3500002830 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3500006580 con 0x7f3500002830 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f350c004bf0 con 0x7f3500002830 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 10 monclient: dump: 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.981 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2775826498 0 0) 0x7f3500006580 con 0x7f3500002830 2024-09-18T14:06:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.958+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f350000a0a0 con 0x7f3500002830 2024-09-18T14:06:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.959+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f350c0051e0 con 0x7f3500002830 2024-09-18T14:06:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.959+0000 7f3518ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.959+0000 7f3518ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.959+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 >> v1:172.21.15.45:6791/0 conn(0x7f3500002830 legacy=0x7f3500004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.959+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 >> v1:172.21.15.45:6791/0 conn(0x7f3500002830 legacy=0x7f3500004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.964+0000 7f35228c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.964+0000 7f35228c0640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3500009640 con 0x7f3500009020 2024-09-18T14:06:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236353233 0 0) 0x7f3500009640 con 0x7f3500009020 2024-09-18T14:06:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 10 monclient(hunting): my global_id is 154120 2024-09-18T14:06:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3500006580 con 0x7f3500009020 2024-09-18T14:06:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 830139112 0 0) 0x7f3500006580 con 0x7f3500009020 2024-09-18T14:06:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3500009640 con 0x7f3500009020 2024-09-18T14:06:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f350c005a60 con 0x7f3500009020 2024-09-18T14:06:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 10 monclient: dump: 2024-09-18T14:06:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3860625177 0 0) 0x7f3500009640 con 0x7f3500009020 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3500009900 con 0x7f3500009020 2024-09-18T14:06:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 2545485310) 0x7f350c0081d0 con 0x7f3500009020 2024-09-18T14:06:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 >> v1:172.21.15.45:6791/0 conn(0x7f3500009020 legacy=0x7f3500008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.965+0000 7f3518ff9640 1 -- 172.21.15.45:0/2248860926 >> v1:172.21.15.45:6791/0 conn(0x7f3500009020 legacy=0x7f3500008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:01.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:01.986 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:01.986 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:01.986 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:01.987 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:01.987 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:01.987 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:01.987 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:01.987 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:01.987 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:01.987 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:01.987 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:01.987 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:01.987 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:01.987 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:01.987 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:01.988 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:01.989 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:01.989 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:01.989 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:01.989 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:01.989 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:01.989 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:01.989 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:01.989 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:01.989 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:01.989 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:01.989 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:01.989 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.990 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:01.990 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:01.990 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:01.990 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:01.990 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:01.990 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:01.990 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:01.990 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:01.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:01.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:01.990 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:01.991 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.993 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:01.993 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.993 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.993 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.993 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:01.993 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:01.993 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.993 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.993 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:01.994 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:01.995 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.995 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.995 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.995 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:01.995 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:01.995 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.995 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.995 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:01.996 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.996 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.996 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.996 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:01.996 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:01.996 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.996 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.996 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.996 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:01.997 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:01.998 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:01.999 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:01.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.999 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:01.999 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:01.999 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:01.999 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:01.999 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:01.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:01.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:01.999 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:01.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:01.999 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:02.000 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.000 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.000 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.000 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.000 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.001 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.001 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.001 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:02.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:02.001 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.001 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.001 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:02.001 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.002 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.002 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.002 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.002 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.002 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.002 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.002 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:02.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:02.003 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:02.003 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.003 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.004 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.004 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.004 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.004 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.004 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:02.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:02.004 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.004 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:02.005 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.005 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.005 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.005 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.005 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.005 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.005 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.006 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.006 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.006 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:02.006 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:02.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.006 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.006 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.006 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.006 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.006 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.006 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:02.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.007 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.007 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.007 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.007 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.007 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:02.007 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.966+0000 7f35228c0640 1 -- 172.21.15.45:0/2248860926 >> v1:172.21.15.84:6812/31478 conn(0x7f34fc03a5a0 legacy=0x7f34fc03ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.966+0000 7f35228c0640 10 monclient: shutdown 2024-09-18T14:06:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.966+0000 7f35228c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.966+0000 7f35228c0640 1 -- 172.21.15.45:0/2248860926 >> v1:172.21.15.45:6792/0 conn(0x7f351c0a1080 legacy=0x7f351c149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.966+0000 7f3520e5d640 1 -- 172.21.15.45:0/2248860926 reap_dead start 2024-09-18T14:06:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.966+0000 7f3520e5d640 1 -- 172.21.15.45:0/2248860926 reap_dead start 2024-09-18T14:06:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.966+0000 7f35228c0640 1 -- 172.21.15.45:0/2248860926 shutdown_connections 2024-09-18T14:06:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.966+0000 7f35228c0640 1 -- 172.21.15.45:0/2248860926 >> 172.21.15.45:0/2248860926 conn(0x7f351c0a5bb0 msgr2=0x7f351c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.966+0000 7f35228c0640 1 -- 172.21.15.45:0/2248860926 shutdown_connections 2024-09-18T14:06:02.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:01.966+0000 7f35228c0640 1 -- 172.21.15.45:0/2248860926 wait complete. 2024-09-18T14:06:02.010 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:02.010 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:06:02.010 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:06:02.038 INFO:tasks.rados.rados.0.smithi084.stdout:1448: done (1 left) 2024-09-18T14:06:02.038 INFO:tasks.rados.rados.0.smithi084.stdout:1449: copy_from oid 30 from oid 39 current snap is 156 2024-09-18T14:06:02.038 INFO:tasks.rados.rados.0.smithi084.stdout:1447: finishing write tid 1 to smithi08434663-7 2024-09-18T14:06:02.038 INFO:tasks.rados.rados.0.smithi084.stdout:1447: finishing write tid 2 to smithi08434663-7 2024-09-18T14:06:02.038 INFO:tasks.rados.rados.0.smithi084.stdout:1447: finishing write tid 3 to smithi08434663-7 2024-09-18T14:06:02.038 INFO:tasks.rados.rados.0.smithi084.stdout:1447: finishing write tid 4 to smithi08434663-7 2024-09-18T14:06:02.038 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 562 (ObjNum 507 snap 156 seq_num 507) dirty exists 2024-09-18T14:06:02.038 INFO:tasks.rados.rados.0.smithi084.stdout:1447: left oid 7 (ObjNum 507 snap 156 seq_num 507) 2024-09-18T14:06:02.038 INFO:tasks.rados.rados.0.smithi084.stdout:1447: done (1 left) 2024-09-18T14:06:02.038 INFO:tasks.rados.rados.0.smithi084.stdout:1450: rollback oid 40 current snap is 156 2024-09-18T14:06:02.039 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 40 to 155 2024-09-18T14:06:02.039 INFO:tasks.rados.rados.0.smithi084.stdout:1451: write (excl) oid 3 current snap is 156 2024-09-18T14:06:02.039 INFO:tasks.rados.rados.0.smithi084.stdout:1451: seq_num 508 ranges {537956=705790,1644029=772160} 2024-09-18T14:06:02.045 INFO:tasks.rados.rados.0.smithi084.stdout:1451: writing smithi08434663-3 from 537956 to 1243746 tid 1 2024-09-18T14:06:02.053 INFO:tasks.rados.rados.0.smithi084.stdout:1451: writing smithi08434663-3 from 1644029 to 2416189 tid 2 2024-09-18T14:06:02.053 INFO:tasks.rados.rados.0.smithi084.stdout:1452: read oid 36 snap 138 2024-09-18T14:06:02.053 INFO:tasks.rados.rados.0.smithi084.stdout:1452: expect (ObjNum 343 snap 104 seq_num 343) 2024-09-18T14:06:02.053 INFO:tasks.rados.rados.0.smithi084.stdout:1453: rollback oid 22 current snap is 156 2024-09-18T14:06:02.053 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 22 to 142 2024-09-18T14:06:02.053 INFO:tasks.rados.rados.0.smithi084.stdout:1450: finishing rollback tid 0 to smithi08434663-40 2024-09-18T14:06:02.053 INFO:tasks.rados.rados.0.smithi084.stdout:1450: finishing rollback tid 1 to smithi08434663-40 2024-09-18T14:06:02.053 INFO:tasks.rados.rados.0.smithi084.stdout:1450: finishing rollback tid 2 to smithi08434663-40 2024-09-18T14:06:02.053 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 567 (ObjNum 354 snap 109 seq_num 354) dirty exists 2024-09-18T14:06:02.054 INFO:tasks.rados.rados.0.smithi084.stdout:1450: done (4 left) 2024-09-18T14:06:02.054 INFO:tasks.rados.rados.0.smithi084.stdout:1454: write (excl) oid 19 current snap is 156 2024-09-18T14:06:02.054 INFO:tasks.rados.rados.0.smithi084.stdout:1454: seq_num 509 ranges {554763=781039,2102511=452252} 2024-09-18T14:06:02.060 INFO:tasks.rados.rados.0.smithi084.stdout:1454: writing smithi08434663-19 from 554763 to 1335802 tid 1 2024-09-18T14:06:02.067 INFO:tasks.rados.rados.0.smithi084.stdout:1454: writing smithi08434663-19 from 2102511 to 2554763 tid 2 2024-09-18T14:06:02.068 INFO:tasks.rados.rados.0.smithi084.stdout:1455: copy_from oid 25 from oid 21 current snap is 156 2024-09-18T14:06:02.068 INFO:tasks.rados.rados.0.smithi084.stdout:1449: finishing copy_from racing read to smithi08434663-30 2024-09-18T14:06:02.068 INFO:tasks.rados.rados.0.smithi084.stdout:1449: finishing copy_from to smithi08434663-30 2024-09-18T14:06:02.068 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 481 (ObjNum 450 snap 139 seq_num 450) dirty exists 2024-09-18T14:06:02.068 INFO:tasks.rados.rados.0.smithi084.stdout:1451: finishing write tid 1 to smithi08434663-3 2024-09-18T14:06:02.068 INFO:tasks.rados.rados.0.smithi084.stdout:1451: finishing write tid 2 to smithi08434663-3 2024-09-18T14:06:02.068 INFO:tasks.rados.rados.0.smithi084.stdout:1451: finishing write tid 3 to smithi08434663-3 2024-09-18T14:06:02.068 INFO:tasks.rados.rados.0.smithi084.stdout:1451: finishing write tid 4 to smithi08434663-3 2024-09-18T14:06:02.068 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 690 (ObjNum 508 snap 156 seq_num 508) dirty exists 2024-09-18T14:06:02.068 INFO:tasks.rados.rados.0.smithi084.stdout:1451: left oid 3 (ObjNum 508 snap 156 seq_num 508) 2024-09-18T14:06:02.068 INFO:tasks.rados.rados.0.smithi084.stdout:1453: finishing rollback tid 0 to smithi08434663-22 2024-09-18T14:06:02.068 INFO:tasks.rados.rados.0.smithi084.stdout:1453: finishing rollback tid 1 to smithi08434663-22 2024-09-18T14:06:02.069 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 692 (ObjNum 289 snap 0 seq_num 3384903676) dirty dne 2024-09-18T14:06:02.069 INFO:tasks.rados.rados.0.smithi084.stdout:1454: finishing write tid 1 to smithi08434663-19 2024-09-18T14:06:02.069 INFO:tasks.rados.rados.0.smithi084.stdout:1449: done (5 left) 2024-09-18T14:06:02.069 INFO:tasks.rados.rados.0.smithi084.stdout:1451: done (4 left) 2024-09-18T14:06:02.069 INFO:tasks.rados.rados.0.smithi084.stdout:1453: done (3 left) 2024-09-18T14:06:02.069 INFO:tasks.rados.rados.0.smithi084.stdout:1456: snap_create 2024-09-18T14:06:02.069 INFO:tasks.rados.rados.0.smithi084.stdout:1455: finishing copy_from to smithi08434663-25 2024-09-18T14:06:02.069 INFO:tasks.rados.rados.0.smithi084.stdout:1455: got expected ENOENT (src dne) 2024-09-18T14:06:02.070 INFO:tasks.rados.rados.0.smithi084.stdout:1452: expect (ObjNum 343 snap 104 seq_num 343) 2024-09-18T14:06:02.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.120+0000 7fbb2e06f640 10 monclient: get_monmap_and_config 2024-09-18T14:06:02.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.120+0000 7fbb2e06f640 10 monclient: build_initial_monmap 2024-09-18T14:06:02.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.120+0000 7fbb2e06f640 10 monclient: monmap: 2024-09-18T14:06:02.121 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:02.121 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.121 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:02.121440+0000 2024-09-18T14:06:02.121 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:02.121440+0000 2024-09-18T14:06:02.121 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:02.121 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.121 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:02.121 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:02.121 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:02.122 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 1 Processor -- start 2024-09-18T14:06:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 1 -- start start 2024-09-18T14:06:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 10 monclient: init 2024-09-18T14:06:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 10 monclient: _add_conns ranks=[3,2,4,7,1,8,0,6,5] 2024-09-18T14:06:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 10 monclient(hunting): picked mon.noname-g con 0x7fbb2809f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 10 monclient(hunting): picked mon.noname-b con 0x7fbb2809fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 10 monclient(hunting): picked mon.noname-c con 0x7fbb2809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2805ed70 con 0x7fbb2809fba0 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2809dfc0 con 0x7fbb2809f7d0 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2809dce0 con 0x7fbb2809ff70 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb2e06f640 10 monclient(hunting): authenticate will time out at 1177.459785s 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb26ffd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fbb2809fba0 0x7fbb28137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39390/0 (socket says 172.21.15.45:39390) 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.121+0000 7fbb26ffd640 1 -- 172.21.15.45:0/4081838436 learned_addr learned my addr 172.21.15.45:0/4081838436 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250806679 0 0) 0x7fbb2809dce0 con 0x7fbb2809ff70 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient(hunting): my global_id is 154129 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb100034e0 con 0x7fbb2809ff70 2024-09-18T14:06:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217802893 0 0) 0x7fbb2805ed70 con 0x7fbb2809fba0 2024-09-18T14:06:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient(hunting): my global_id is 144380 2024-09-18T14:06:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb2809dce0 con 0x7fbb2809fba0 2024-09-18T14:06:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686030190 0 0) 0x7fbb2809dfc0 con 0x7fbb2809f7d0 2024-09-18T14:06:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient(hunting): my global_id is 154128 2024-09-18T14:06:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb2805ed70 con 0x7fbb2809f7d0 2024-09-18T14:06:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 304316549 0 0) 0x7fbb2805ed70 con 0x7fbb2809f7d0 2024-09-18T14:06:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 >> v1:172.21.15.45:6791/0 conn(0x7fbb2809ff70 legacy=0x7fbb2813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 >> v1:172.21.15.45:6790/0 conn(0x7fbb2809fba0 legacy=0x7fbb28137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient: _finish_hunting 0 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 monclient: found mon.noname-g 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb280a1c00 con 0x7fbb2809f7d0 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient: _finish_auth 0 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbb180016d0 con 0x7fbb2809f7d0 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb2e06f640 5 monclient: authenticate success, global_id 154128 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb2e06f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:06:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient: dump: 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb2e06f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbb18001830 con 0x7fbb2809f7d0 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 1 -- 172.21.15.45:0/4081838436 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbb18001de0 con 0x7fbb2809f7d0 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb2e06f640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb2e06f640 4 set_mon_vals no callback set 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb267fc640 10 monclient: dump: 2024-09-18T14:06:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.122+0000 7fbb2e06f640 10 monclient: shutdown 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.123+0000 7fbb2e06f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.123+0000 7fbb2e06f640 1 -- 172.21.15.45:0/4081838436 >> v1:172.21.15.84:6790/0 conn(0x7fbb2809f7d0 legacy=0x7fbb28132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.123+0000 7fbb2e06f640 1 -- 172.21.15.45:0/4081838436 shutdown_connections 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.123+0000 7fbb2e06f640 1 -- 172.21.15.45:0/4081838436 >> 172.21.15.45:0/4081838436 conn(0x7fbb280630b0 msgr2=0x7fbb280634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.123+0000 7fbb2e06f640 1 -- 172.21.15.45:0/4081838436 shutdown_connections 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.123+0000 7fbb2e06f640 1 -- 172.21.15.45:0/4081838436 wait complete. 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.123+0000 7fbb2e06f640 10 monclient: build_initial_monmap 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.123+0000 7fbb2e06f640 10 monclient: monmap: 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:02.140 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 1 Processor -- start 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 1 -- start start 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 10 monclient: init 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 10 monclient: _add_conns ranks=[1,0,4,5,3,2,7,6,8] 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 10 monclient(hunting): picked mon.noname-f con 0x7fbb2809f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 10 monclient(hunting): picked mon.noname-a con 0x7fbb2809fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 10 monclient(hunting): picked mon.noname-c con 0x7fbb2809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2809d310 con 0x7fbb2809fba0 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb280a2560 con 0x7fbb2809f7d0 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb28121cf0 con 0x7fbb2809ff70 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb2e06f640 10 monclient(hunting): authenticate will time out at 1177.462559s 2024-09-18T14:06:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb26ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbb2809fba0 0x7fbb281318c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33556/0 (socket says 172.21.15.45:33556) 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb27fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fbb2809ff70 0x7fbb28149c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58486/0 (socket says 172.21.15.45:58486) 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb26ffd640 1 -- 172.21.15.45:0/1105066416 learned_addr learned my addr 172.21.15.45:0/1105066416 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506034894 0 0) 0x7fbb28121cf0 con 0x7fbb2809ff70 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 10 monclient(hunting): my global_id is 154138 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb140034e0 con 0x7fbb2809ff70 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594703811 0 0) 0x7fbb2809d310 con 0x7fbb2809fba0 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 10 monclient(hunting): my global_id is 5085 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb28121cf0 con 0x7fbb2809fba0 2024-09-18T14:06:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772957523 0 0) 0x7fbb280a2560 con 0x7fbb2809f7d0 2024-09-18T14:06:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 10 monclient(hunting): my global_id is 5023 2024-09-18T14:06:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.124+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb2809d310 con 0x7fbb2809f7d0 2024-09-18T14:06:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2122180927 0 0) 0x7fbb140034e0 con 0x7fbb2809ff70 2024-09-18T14:06:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb280a2560 con 0x7fbb2809ff70 2024-09-18T14:06:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbb20001de0 con 0x7fbb2809ff70 2024-09-18T14:06:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:06:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient(hunting): dump: 2024-09-18T14:06:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.145 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1911040204 0 0) 0x7fbb28121cf0 con 0x7fbb2809fba0 2024-09-18T14:06:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb140034e0 con 0x7fbb2809fba0 2024-09-18T14:06:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1168541480 0 0) 0x7fbb2809d310 con 0x7fbb2809f7d0 2024-09-18T14:06:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb28121cf0 con 0x7fbb2809f7d0 2024-09-18T14:06:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbb1c0013e0 con 0x7fbb2809fba0 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient(hunting): dump: 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbb18001ce0 con 0x7fbb2809f7d0 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient(hunting): dump: 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 114535979 0 0) 0x7fbb280a2560 con 0x7fbb2809ff70 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 >> v1:172.21.15.84:6789/0 conn(0x7fbb2809f7d0 legacy=0x7fbb281418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 >> v1:172.21.15.45:6789/0 conn(0x7fbb2809fba0 legacy=0x7fbb281318c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 monclient: found mon.c 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb28142760 con 0x7fbb2809ff70 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient: _finish_auth 0 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb2e06f640 5 monclient: authenticate success, global_id 154138 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb2e06f640 10 monclient: _renew_subs 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb2e06f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb2e06f640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb28121a30 con 0x7fbb2809ff70 2024-09-18T14:06:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb2e06f640 10 monclient: _renew_subs 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb2e06f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb2e06f640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb28131fd0 con 0x7fbb2809ff70 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbb200022e0 con 0x7fbb2809ff70 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbb20005190 con 0x7fbb2809ff70 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb25ffb640 4 set_mon_vals no callback set 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.125+0000 7fbb0f7fe640 10 monclient: dump: 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.126+0000 7fbb2e06f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.126+0000 7fbb2e06f640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbb08006580 con 0x7fbb08002830 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.126+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fbb20010160 con 0x7fbb2809ff70 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.126+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(325..325 src has 126..325) v4 ==== 3923+0+0 (unknown 847000907 0 0) 0x7fbb20046df0 con 0x7fbb2809ff70 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.126+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448086604 0 0) 0x7fbb08006580 con 0x7fbb08002830 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.126+0000 7fbb0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.126+0000 7fbb0f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.126+0000 7fbb0f7fe640 10 monclient(hunting): my global_id is 4983 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.126+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb280a2560 con 0x7fbb08002830 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.126+0000 7fbb26ffd640 10 monclient: get_auth_request con 0x7fbb1403aaf0 auth_method 0 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.126+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2580017639 0 0) 0x7fbb280a2560 con 0x7fbb08002830 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.126+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb08006580 con 0x7fbb08002830 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.127+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbb18001e40 con 0x7fbb08002830 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.127+0000 7fbb0f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.127+0000 7fbb0f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.127+0000 7fbb0f7fe640 10 monclient: dump: 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.154 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.127+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2313467166 0 0) 0x7fbb08006580 con 0x7fbb08002830 2024-09-18T14:06:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.127+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb0800a0a0 con 0x7fbb08002830 2024-09-18T14:06:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.127+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fbb18002640 con 0x7fbb08002830 2024-09-18T14:06:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.127+0000 7fbb0f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.127+0000 7fbb0f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.127+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 >> v1:172.21.15.45:6792/0 conn(0x7fbb08002830 legacy=0x7fbb08004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.127+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 >> v1:172.21.15.45:6792/0 conn(0x7fbb08002830 legacy=0x7fbb08004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb2e06f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb2e06f640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbb08009640 con 0x7fbb08009020 2024-09-18T14:06:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77835622 0 0) 0x7fbb08009640 con 0x7fbb08009020 2024-09-18T14:06:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 10 monclient(hunting): my global_id is 4992 2024-09-18T14:06:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb08006580 con 0x7fbb08009020 2024-09-18T14:06:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1192364152 0 0) 0x7fbb08006580 con 0x7fbb08009020 2024-09-18T14:06:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb08009640 con 0x7fbb08009020 2024-09-18T14:06:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbb18002f40 con 0x7fbb08009020 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 10 monclient: dump: 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2329547751 0 0) 0x7fbb08009640 con 0x7fbb08009020 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.133+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbb08009900 con 0x7fbb08009020 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 2822365536) 0x7fbb18005200 con 0x7fbb08009020 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb0f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb0f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 >> v1:172.21.15.45:6792/0 conn(0x7fbb08009020 legacy=0x7fbb08008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb0f7fe640 1 -- 172.21.15.45:0/1105066416 >> v1:172.21.15.45:6792/0 conn(0x7fbb08009020 legacy=0x7fbb08008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.159 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:02.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:02.159 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:02.159 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:02.159 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:02.159 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:02.159 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:02.159 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:02.159 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:02.159 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.160 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.161 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.163 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:02.165 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.166 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.168 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:02.168 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.168 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.168 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:02.168 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.170 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.172 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:02.173 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.174 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.175 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:02.175 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.175 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.175 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.175 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.175 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.175 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.176 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.177 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.178 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:02.179 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:02.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb2e06f640 1 -- 172.21.15.45:0/1105066416 >> v1:172.21.15.84:6812/31478 conn(0x7fbb1403aaf0 legacy=0x7fbb1403cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb2e06f640 10 monclient: shutdown 2024-09-18T14:06:02.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb2e06f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb2e06f640 1 -- 172.21.15.45:0/1105066416 >> v1:172.21.15.45:6791/0 conn(0x7fbb2809ff70 legacy=0x7fbb28149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb27fff640 1 -- 172.21.15.45:0/1105066416 reap_dead start 2024-09-18T14:06:02.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb27fff640 1 -- 172.21.15.45:0/1105066416 reap_dead start 2024-09-18T14:06:02.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb2e06f640 1 -- 172.21.15.45:0/1105066416 shutdown_connections 2024-09-18T14:06:02.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb2e06f640 1 -- 172.21.15.45:0/1105066416 >> 172.21.15.45:0/1105066416 conn(0x7fbb280630b0 msgr2=0x7fbb280a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb2e06f640 1 -- 172.21.15.45:0/1105066416 shutdown_connections 2024-09-18T14:06:02.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.134+0000 7fbb2e06f640 1 -- 172.21.15.45:0/1105066416 wait complete. 2024-09-18T14:06:02.181 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:02.181 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:06:02.181 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:06:02.266 INFO:tasks.rados.rados.0.smithi084.stdout:1455: finishing copy_from racing read to smithi08434663-25 2024-09-18T14:06:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1454: finishing write tid 2 to smithi08434663-19 2024-09-18T14:06:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1454: finishing write tid 3 to smithi08434663-19 2024-09-18T14:06:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1454: finishing write tid 4 to smithi08434663-19 2024-09-18T14:06:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 765 (ObjNum 509 snap 156 seq_num 509) dirty exists 2024-09-18T14:06:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1454: left oid 19 (ObjNum 509 snap 156 seq_num 509) 2024-09-18T14:06:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.280+0000 7f5fbad5d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.280+0000 7f5fbad5d640 10 monclient: build_initial_monmap 2024-09-18T14:06:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.280+0000 7f5fbad5d640 10 monclient: monmap: 2024-09-18T14:06:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:02.281409+0000 2024-09-18T14:06:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:02.281409+0000 2024-09-18T14:06:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:02.282 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:02.282 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:02.282 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:02.282 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:02.282 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:02.282 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 1 Processor -- start 2024-09-18T14:06:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 1 -- start start 2024-09-18T14:06:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 10 monclient: init 2024-09-18T14:06:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 10 monclient: _add_conns ranks=[0,4,7,5,3,8,1,6,2] 2024-09-18T14:06:02.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 10 monclient(hunting): picked mon.noname-a con 0x7f5fb409f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:02.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 10 monclient(hunting): picked mon.noname-c con 0x7f5fb409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:02.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 10 monclient(hunting): picked mon.noname-i con 0x7f5fb409ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:02.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb405ed70 con 0x7f5fb409f7d0 2024-09-18T14:06:02.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb409dfc0 con 0x7f5fb409fba0 2024-09-18T14:06:02.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb409dce0 con 0x7f5fb409ff70 2024-09-18T14:06:02.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:02.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fbad5d640 10 monclient(hunting): authenticate will time out at 1177.619746s 2024-09-18T14:06:02.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fb3fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f5fb409fba0 0x7f5fb4137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58488/0 (socket says 172.21.15.45:58488) 2024-09-18T14:06:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fb8af9640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5fb409f7d0 0x7f5fb4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33572/0 (socket says 172.21.15.45:33572) 2024-09-18T14:06:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.281+0000 7f5fb3fff640 1 -- 172.21.15.45:0/3040992569 learned_addr learned my addr 172.21.15.45:0/3040992569 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166175551 0 0) 0x7f5fb409dfc0 con 0x7f5fb409fba0 2024-09-18T14:06:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient(hunting): my global_id is 154147 2024-09-18T14:06:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fa40034e0 con 0x7f5fb409fba0 2024-09-18T14:06:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924553801 0 0) 0x7f5fb405ed70 con 0x7f5fb409f7d0 2024-09-18T14:06:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient(hunting): my global_id is 5094 2024-09-18T14:06:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb409dfc0 con 0x7f5fb409f7d0 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388905154 0 0) 0x7f5fb409dce0 con 0x7f5fb409ff70 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient(hunting): my global_id is 5038 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb405ed70 con 0x7f5fb409ff70 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642591408 0 0) 0x7f5fa40034e0 con 0x7f5fb409fba0 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 >> v1:172.21.15.84:6792/0 conn(0x7f5fb409ff70 legacy=0x7f5fb413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 >> v1:172.21.15.45:6789/0 conn(0x7f5fb409f7d0 legacy=0x7f5fb4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient: _finish_hunting 0 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 monclient: found mon.noname-c 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb40a1c00 con 0x7f5fb409fba0 2024-09-18T14:06:02.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient: _finish_auth 0 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5fa0001830 con 0x7f5fb409fba0 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fbad5d640 5 monclient: authenticate success, global_id 154147 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fbad5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient: dump: 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.294 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fbad5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5fa0001990 con 0x7f5fb409fba0 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 1 -- 172.21.15.45:0/3040992569 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5fa0004ba0 con 0x7f5fb409fba0 2024-09-18T14:06:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fbad5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fbad5d640 4 set_mon_vals no callback set 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fb37fe640 10 monclient: dump: 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.282+0000 7f5fbad5d640 10 monclient: shutdown 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.283+0000 7f5fbad5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.283+0000 7f5fbad5d640 1 -- 172.21.15.45:0/3040992569 >> v1:172.21.15.45:6791/0 conn(0x7f5fb409fba0 legacy=0x7f5fb4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.283+0000 7f5fbad5d640 1 -- 172.21.15.45:0/3040992569 shutdown_connections 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.283+0000 7f5fbad5d640 1 -- 172.21.15.45:0/3040992569 >> 172.21.15.45:0/3040992569 conn(0x7f5fb40630b0 msgr2=0x7f5fb40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.283+0000 7f5fbad5d640 1 -- 172.21.15.45:0/3040992569 shutdown_connections 2024-09-18T14:06:02.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.283+0000 7f5fbad5d640 1 -- 172.21.15.45:0/3040992569 wait complete. 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.283+0000 7f5fbad5d640 10 monclient: build_initial_monmap 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.283+0000 7f5fbad5d640 10 monclient: monmap: 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.283+0000 7f5fbad5d640 1 Processor -- start 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fbad5d640 1 -- start start 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fbad5d640 10 monclient: init 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fbad5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fbad5d640 10 monclient: _add_conns ranks=[5,4,7,3,1,6,2,0,8] 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fbad5d640 10 monclient(hunting): picked mon.noname-h con 0x7f5fb409f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fbad5d640 10 monclient(hunting): picked mon.noname-c con 0x7f5fb409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fbad5d640 10 monclient(hunting): picked mon.noname-i con 0x7f5fb409ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fbad5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb409d310 con 0x7f5fb409fba0 2024-09-18T14:06:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fbad5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb40a2560 con 0x7f5fb409f7d0 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fbad5d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb411e9f0 con 0x7f5fb409ff70 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fbad5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fbad5d640 10 monclient(hunting): authenticate will time out at 1177.622572s 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb3fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f5fb409fba0 0x7f5fb4146cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58498/0 (socket says 172.21.15.45:58498) 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb3fff640 1 -- 172.21.15.45:0/2868809655 learned_addr learned my addr 172.21.15.45:0/2868809655 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837753638 0 0) 0x7f5fb409d310 con 0x7f5fb409fba0 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 10 monclient(hunting): my global_id is 154156 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f980034e0 con 0x7f5fb409fba0 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793714078 0 0) 0x7f5fb40a2560 con 0x7f5fb409f7d0 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 10 monclient(hunting): my global_id is 4937 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb409d310 con 0x7f5fb409f7d0 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868026700 0 0) 0x7f5fb411e9f0 con 0x7f5fb409ff70 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.284+0000 7f5fb0ff9640 10 monclient(hunting): my global_id is 5047 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb40a2560 con 0x7f5fb409ff70 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1231096404 0 0) 0x7f5f980034e0 con 0x7f5fb409fba0 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5fb411e9f0 con 0x7f5fb409fba0 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5fa00047a0 con 0x7f5fb409fba0 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:06:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2428269134 0 0) 0x7f5fb409d310 con 0x7f5fb409f7d0 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f980034e0 con 0x7f5fb409f7d0 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5fa80012f0 con 0x7f5fb409f7d0 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.303 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3880208412 0 0) 0x7f5fb40a2560 con 0x7f5fb409ff70 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5fb409d310 con 0x7f5fb409ff70 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5fac002bf0 con 0x7f5fb409ff70 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1287113736 0 0) 0x7f5fb411e9f0 con 0x7f5fb409fba0 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 >> v1:172.21.15.84:6792/0 conn(0x7f5fb409ff70 legacy=0x7f5fb414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 >> v1:172.21.15.84:6791/0 conn(0x7f5fb409f7d0 legacy=0x7f5fb4131560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 monclient: found mon.c 2024-09-18T14:06:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb4136ea0 con 0x7f5fb409fba0 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient: _finish_auth 0 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fbad5d640 5 monclient: authenticate success, global_id 154156 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fbad5d640 10 monclient: _renew_subs 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fbad5d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fbad5d640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb411d790 con 0x7f5fb409fba0 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fbad5d640 10 monclient: _renew_subs 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fbad5d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fbad5d640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb409b7c0 con 0x7f5fb409fba0 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5fa0004bb0 con 0x7f5fb409fba0 2024-09-18T14:06:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5fa00051a0 con 0x7f5fb409fba0 2024-09-18T14:06:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb2ffd640 4 set_mon_vals no callback set 2024-09-18T14:06:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.285+0000 7f5fb0ff9640 10 monclient: dump: 2024-09-18T14:06:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fbad5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f5fa0010170 con 0x7f5fb409fba0 2024-09-18T14:06:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fbad5d640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5f94006580 con 0x7f5f94002830 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(325..325 src has 126..325) v4 ==== 3923+0+0 (unknown 847000907 0 0) 0x7f5fa0046b00 con 0x7f5fb409fba0 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb92fa640 10 monclient: get_auth_request con 0x7f5f9803aaa0 auth_method 0 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373290135 0 0) 0x7f5f94006580 con 0x7f5f94002830 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 10 monclient(hunting): my global_id is 5012 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb411e9f0 con 0x7f5f94002830 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3788741874 0 0) 0x7f5fb411e9f0 con 0x7f5f94002830 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f94006580 con 0x7f5f94002830 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5fa8004ae0 con 0x7f5f94002830 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.286+0000 7f5fb0ff9640 10 monclient: dump: 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.312 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.287+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3794665852 0 0) 0x7f5f94006580 con 0x7f5f94002830 2024-09-18T14:06:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.287+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f9400a0a0 con 0x7f5f94002830 2024-09-18T14:06:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.287+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5fa8004f40 con 0x7f5f94002830 2024-09-18T14:06:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.287+0000 7f5fb0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.287+0000 7f5fb0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.287+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 >> v1:172.21.15.45:6793/0 conn(0x7f5f94002830 legacy=0x7f5f94004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.287+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 >> v1:172.21.15.45:6793/0 conn(0x7f5f94002830 legacy=0x7f5f94004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fbad5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fbad5d640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5f94009660 con 0x7f5f94009040 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980813886 0 0) 0x7f5f94009660 con 0x7f5f94009040 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 10 monclient(hunting): my global_id is 5021 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f94006580 con 0x7f5f94009040 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3702337933 0 0) 0x7f5f94006580 con 0x7f5f94009040 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f94009660 con 0x7f5f94009040 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5fa8005740 con 0x7f5f94009040 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 10 monclient: dump: 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.316 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2100778479 0 0) 0x7f5f94009660 con 0x7f5f94009040 2024-09-18T14:06:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.293+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5f94009920 con 0x7f5f94009040 2024-09-18T14:06:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7558 (unknown 0 0 2271237078) 0x7f5fa8006b10 con 0x7f5f94009040 2024-09-18T14:06:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fb0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fb0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 >> v1:172.21.15.45:6793/0 conn(0x7f5f94009040 legacy=0x7f5f94008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fb0ff9640 1 -- 172.21.15.45:0/2868809655 >> v1:172.21.15.45:6793/0 conn(0x7f5f94009040 legacy=0x7f5f94008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.316 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.332 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:02.333 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.334 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.334 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.334 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.334 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.334 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.334 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.334 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.334 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.334 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.334 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.335 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.336 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.336 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:02.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.336 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.336 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.336 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:06:02.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.336 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.336 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:02.336 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fbad5d640 1 -- 172.21.15.45:0/2868809655 >> v1:172.21.15.84:6812/31478 conn(0x7f5f9803aaa0 legacy=0x7f5f9803cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fbad5d640 10 monclient: shutdown 2024-09-18T14:06:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fbad5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fbad5d640 1 -- 172.21.15.45:0/2868809655 >> v1:172.21.15.45:6791/0 conn(0x7f5fb409fba0 legacy=0x7f5fb4146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fb92fa640 1 -- 172.21.15.45:0/2868809655 reap_dead start 2024-09-18T14:06:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fb92fa640 1 -- 172.21.15.45:0/2868809655 reap_dead start 2024-09-18T14:06:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fbad5d640 1 -- 172.21.15.45:0/2868809655 shutdown_connections 2024-09-18T14:06:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fbad5d640 1 -- 172.21.15.45:0/2868809655 >> 172.21.15.45:0/2868809655 conn(0x7f5fb40630b0 msgr2=0x7f5fb40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fbad5d640 1 -- 172.21.15.45:0/2868809655 shutdown_connections 2024-09-18T14:06:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.294+0000 7f5fbad5d640 1 -- 172.21.15.45:0/2868809655 wait complete. 2024-09-18T14:06:02.339 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:02.339 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:06:02.339 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:06:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.436+0000 7feb2e82b640 10 monclient: get_monmap_and_config 2024-09-18T14:06:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.436+0000 7feb2e82b640 10 monclient: build_initial_monmap 2024-09-18T14:06:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.436+0000 7feb2e82b640 10 monclient: monmap: 2024-09-18T14:06:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:02.437031+0000 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:02.437031+0000 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:02.438 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.436+0000 7feb2e82b640 1 Processor -- start 2024-09-18T14:06:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2e82b640 1 -- start start 2024-09-18T14:06:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2e82b640 10 monclient: init 2024-09-18T14:06:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2e82b640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2e82b640 10 monclient: _add_conns ranks=[7,5,1,6,0,8,4,3,2] 2024-09-18T14:06:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2e82b640 10 monclient(hunting): picked mon.noname-i con 0x7feb2806e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2e82b640 10 monclient(hunting): picked mon.noname-h con 0x7feb280a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2e82b640 10 monclient(hunting): picked mon.noname-f con 0x7feb280a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2e82b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2805ed70 con 0x7feb280a1080 2024-09-18T14:06:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2e82b640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb28097620 con 0x7feb280a0cb0 2024-09-18T14:06:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2e82b640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb28097340 con 0x7feb2806e810 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2e82b640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2e82b640 10 monclient(hunting): authenticate will time out at 1177.775627s 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb27fff640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7feb2806e810 0x7feb280a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:43396/0 (socket says 172.21.15.45:43396) 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb1ffff640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7feb280a0cb0 0x7feb28137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:52952/0 (socket says 172.21.15.45:52952) 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb27fff640 1 -- 172.21.15.45:0/1474205951 learned_addr learned my addr 172.21.15.45:0/1474205951 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.437+0000 7feb2cdc8640 1 --1- 172.21.15.45:0/1474205951 >> v1:172.21.15.84:6789/0 conn(0x7feb280a1080 0x7feb2813af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:43834/0 (socket says 172.21.15.45:43834) 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679495242 0 0) 0x7feb28097340 con 0x7feb2806e810 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient(hunting): my global_id is 5056 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb140034e0 con 0x7feb2806e810 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625807269 0 0) 0x7feb28097620 con 0x7feb280a0cb0 2024-09-18T14:06:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient(hunting): my global_id is 4946 2024-09-18T14:06:02.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb28097340 con 0x7feb280a0cb0 2024-09-18T14:06:02.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739962704 0 0) 0x7feb2805ed70 con 0x7feb280a1080 2024-09-18T14:06:02.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient(hunting): my global_id is 5032 2024-09-18T14:06:02.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb28097620 con 0x7feb280a1080 2024-09-18T14:06:02.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 676800830 0 0) 0x7feb140034e0 con 0x7feb2806e810 2024-09-18T14:06:02.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 >> v1:172.21.15.84:6791/0 conn(0x7feb280a0cb0 legacy=0x7feb28137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 >> v1:172.21.15.84:6789/0 conn(0x7feb280a1080 legacy=0x7feb2813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient: _finish_hunting 0 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 monclient: found mon.noname-i 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb280a1660 con 0x7feb2806e810 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient: _finish_auth 0 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb2e82b640 5 monclient: authenticate success, global_id 5056 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb18001760 con 0x7feb2806e810 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient: dump: 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb2e82b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feb180018c0 con 0x7feb2806e810 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 1 -- 172.21.15.45:0/1474205951 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb18004860 con 0x7feb2806e810 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb2e82b640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb2e82b640 4 set_mon_vals no callback set 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb277fe640 10 monclient: dump: 2024-09-18T14:06:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.462 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.462 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.462 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.462 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.462 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.462 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.462 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.462 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.462 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.463 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb2e82b640 10 monclient: shutdown 2024-09-18T14:06:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb2e82b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.438+0000 7feb2e82b640 1 -- 172.21.15.45:0/1474205951 >> v1:172.21.15.84:6792/0 conn(0x7feb2806e810 legacy=0x7feb280a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.439+0000 7feb2e82b640 1 -- 172.21.15.45:0/1474205951 shutdown_connections 2024-09-18T14:06:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.439+0000 7feb2e82b640 1 -- 172.21.15.45:0/1474205951 >> 172.21.15.45:0/1474205951 conn(0x7feb280a5bb0 msgr2=0x7feb280a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.439+0000 7feb2e82b640 1 -- 172.21.15.45:0/1474205951 shutdown_connections 2024-09-18T14:06:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.439+0000 7feb2e82b640 1 -- 172.21.15.45:0/1474205951 wait complete. 2024-09-18T14:06:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.439+0000 7feb2e82b640 10 monclient: build_initial_monmap 2024-09-18T14:06:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.439+0000 7feb2e82b640 10 monclient: monmap: 2024-09-18T14:06:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.439+0000 7feb2e82b640 1 Processor -- start 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.439+0000 7feb2e82b640 1 -- start start 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.439+0000 7feb2e82b640 10 monclient: init 2024-09-18T14:06:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb2e82b640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb2e82b640 10 monclient: _add_conns ranks=[5,2,7,6,3,8,4,0,1] 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb2e82b640 10 monclient(hunting): picked mon.noname-h con 0x7feb2806e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb2e82b640 10 monclient(hunting): picked mon.noname-b con 0x7feb280a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb2e82b640 10 monclient(hunting): picked mon.noname-i con 0x7feb280a1080 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb2e82b640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2809fa10 con 0x7feb280a0cb0 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb2e82b640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb280a2270 con 0x7feb2806e810 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb2e82b640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2811d120 con 0x7feb280a1080 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb2e82b640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb2e82b640 10 monclient(hunting): authenticate will time out at 1177.778470s 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb1ffff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7feb280a0cb0 0x7feb2809dc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39404/0 (socket says 172.21.15.45:39404) 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb1ffff640 1 -- 172.21.15.45:0/687998797 learned_addr learned my addr 172.21.15.45:0/687998797 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb27fff640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7feb2806e810 0x7feb2809d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:52954/0 (socket says 172.21.15.45:52954) 2024-09-18T14:06:02.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590891341 0 0) 0x7feb2809fa10 con 0x7feb280a0cb0 2024-09-18T14:06:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 10 monclient(hunting): my global_id is 144389 2024-09-18T14:06:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb080034e0 con 0x7feb280a0cb0 2024-09-18T14:06:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507588516 0 0) 0x7feb280a2270 con 0x7feb2806e810 2024-09-18T14:06:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 10 monclient(hunting): my global_id is 4955 2024-09-18T14:06:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb2809fa10 con 0x7feb2806e810 2024-09-18T14:06:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045277653 0 0) 0x7feb2811d120 con 0x7feb280a1080 2024-09-18T14:06:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 10 monclient(hunting): my global_id is 5065 2024-09-18T14:06:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb280a2270 con 0x7feb280a1080 2024-09-18T14:06:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1255857121 0 0) 0x7feb080034e0 con 0x7feb280a0cb0 2024-09-18T14:06:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.440+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb2811d120 con 0x7feb280a0cb0 2024-09-18T14:06:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb10001bd0 con 0x7feb280a0cb0 2024-09-18T14:06:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.471 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1889280089 0 0) 0x7feb2809fa10 con 0x7feb2806e810 2024-09-18T14:06:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb080034e0 con 0x7feb2806e810 2024-09-18T14:06:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2335306698 0 0) 0x7feb280a2270 con 0x7feb280a1080 2024-09-18T14:06:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb2809fa10 con 0x7feb280a1080 2024-09-18T14:06:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb18001720 con 0x7feb2806e810 2024-09-18T14:06:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.476 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb20001be0 con 0x7feb280a1080 2024-09-18T14:06:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.479 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2899052962 0 0) 0x7feb2811d120 con 0x7feb280a0cb0 2024-09-18T14:06:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 >> v1:172.21.15.84:6792/0 conn(0x7feb280a1080 legacy=0x7feb2809e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 >> v1:172.21.15.84:6791/0 conn(0x7feb2806e810 legacy=0x7feb2809d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:06:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 monclient: found mon.b 2024-09-18T14:06:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2811c4b0 con 0x7feb280a0cb0 2024-09-18T14:06:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient: _finish_auth 0 2024-09-18T14:06:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb2e82b640 5 monclient: authenticate success, global_id 144389 2024-09-18T14:06:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb2e82b640 10 monclient: _renew_subs 2024-09-18T14:06:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb2e82b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb2e82b640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2811df70 con 0x7feb280a0cb0 2024-09-18T14:06:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb2e82b640 10 monclient: _renew_subs 2024-09-18T14:06:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb2e82b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb2e82b640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7feb281301c0 con 0x7feb280a0cb0 2024-09-18T14:06:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feb10001d70 con 0x7feb280a0cb0 2024-09-18T14:06:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb10004e10 con 0x7feb280a0cb0 2024-09-18T14:06:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb26ffd640 4 set_mon_vals no callback set 2024-09-18T14:06:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.441+0000 7feb24ff9640 10 monclient: dump: 2024-09-18T14:06:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.485 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb2e82b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb2e82b640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7feb0c006580 con 0x7feb0c002830 2024-09-18T14:06:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7feb1000fe60 con 0x7feb280a0cb0 2024-09-18T14:06:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(325..325 src has 126..325) v4 ==== 3923+0+0 (unknown 847000907 0 0) 0x7feb10046b50 con 0x7feb280a0cb0 2024-09-18T14:06:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb2cdc8640 10 monclient: get_auth_request con 0x7feb0803a9e0 auth_method 0 2024-09-18T14:06:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4093575444 0 0) 0x7feb0c006580 con 0x7feb0c002830 2024-09-18T14:06:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 10 monclient(hunting): my global_id is 5041 2024-09-18T14:06:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb2811d120 con 0x7feb0c002830 2024-09-18T14:06:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2138779738 0 0) 0x7feb2811d120 con 0x7feb0c002830 2024-09-18T14:06:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb0c006580 con 0x7feb0c002830 2024-09-18T14:06:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb18005110 con 0x7feb0c002830 2024-09-18T14:06:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:06:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 10 monclient: dump: 2024-09-18T14:06:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.488 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2192232787 0 0) 0x7feb0c006580 con 0x7feb0c002830 2024-09-18T14:06:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.442+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb0c00a0a0 con 0x7feb0c002830 2024-09-18T14:06:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.443+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7feb180054b0 con 0x7feb0c002830 2024-09-18T14:06:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.443+0000 7feb24ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.443+0000 7feb24ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.443+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 >> v1:172.21.15.84:6789/0 conn(0x7feb0c002830 legacy=0x7feb0c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.443+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 >> v1:172.21.15.84:6789/0 conn(0x7feb0c002830 legacy=0x7feb0c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb2e82b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb2e82b640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7feb0c007540 con 0x7feb0c0068f0 2024-09-18T14:06:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904991240 0 0) 0x7feb0c007540 con 0x7feb0c0068f0 2024-09-18T14:06:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 10 monclient(hunting): my global_id is 5050 2024-09-18T14:06:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb0c006580 con 0x7feb0c0068f0 2024-09-18T14:06:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 17890482 0 0) 0x7feb0c006580 con 0x7feb0c0068f0 2024-09-18T14:06:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb0c007540 con 0x7feb0c0068f0 2024-09-18T14:06:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb18005cb0 con 0x7feb0c0068f0 2024-09-18T14:06:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:06:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 10 monclient: dump: 2024-09-18T14:06:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.494 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 940559421 0 0) 0x7feb0c007540 con 0x7feb0c0068f0 2024-09-18T14:06:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.449+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7feb0c0087a0 con 0x7feb0c0068f0 2024-09-18T14:06:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 3196745160) 0x7feb18007d90 con 0x7feb0c0068f0 2024-09-18T14:06:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb24ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:02.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb24ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:02.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 >> v1:172.21.15.84:6789/0 conn(0x7feb0c0068f0 legacy=0x7feb0c006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb24ff9640 1 -- 172.21.15.45:0/687998797 >> v1:172.21.15.84:6789/0 conn(0x7feb0c0068f0 legacy=0x7feb0c006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.495 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:06:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.516 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.518 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:02.518 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb2e82b640 1 -- 172.21.15.45:0/687998797 >> v1:172.21.15.84:6812/31478 conn(0x7feb0803a9e0 legacy=0x7feb0803ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb2e82b640 10 monclient: shutdown 2024-09-18T14:06:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb2e82b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb2e82b640 1 -- 172.21.15.45:0/687998797 >> v1:172.21.15.45:6790/0 conn(0x7feb280a0cb0 legacy=0x7feb2809dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb2cdc8640 1 -- 172.21.15.45:0/687998797 reap_dead start 2024-09-18T14:06:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb2cdc8640 1 -- 172.21.15.45:0/687998797 reap_dead start 2024-09-18T14:06:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb2e82b640 1 -- 172.21.15.45:0/687998797 shutdown_connections 2024-09-18T14:06:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb2e82b640 1 -- 172.21.15.45:0/687998797 >> 172.21.15.45:0/687998797 conn(0x7feb280a5bb0 msgr2=0x7feb281333f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb2e82b640 1 -- 172.21.15.45:0/687998797 shutdown_connections 2024-09-18T14:06:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.450+0000 7feb2e82b640 1 -- 172.21.15.45:0/687998797 wait complete. 2024-09-18T14:06:02.520 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:02.520 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:06:02.521 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:06:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.617+0000 7fbfd1f1d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.617+0000 7fbfd1f1d640 10 monclient: build_initial_monmap 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.617+0000 7fbfd1f1d640 10 monclient: monmap: 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:02.618346+0000 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:02.618346+0000 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:02.619 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:02.619 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:02.619 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 1 Processor -- start 2024-09-18T14:06:02.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 1 -- start start 2024-09-18T14:06:02.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 10 monclient: init 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 10 monclient: _add_conns ranks=[7,3,0,1,4,5,6,2,8] 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 10 monclient(hunting): picked mon.noname-i con 0x7fbfcc09f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 10 monclient(hunting): picked mon.noname-g con 0x7fbfcc09fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 10 monclient(hunting): picked mon.noname-a con 0x7fbfcc09ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfcc05ed70 con 0x7fbfcc09ff70 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfcc09dfc0 con 0x7fbfcc09fba0 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfcc09dce0 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfd1f1d640 10 monclient(hunting): authenticate will time out at 1177.956825s 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfcbfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbfcc09ff70 0x7fbfcc13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33580/0 (socket says 172.21.15.45:33580) 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.618+0000 7fbfcbfff640 1 -- 172.21.15.45:0/3422894028 learned_addr learned my addr 172.21.15.45:0/3422894028 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606445417 0 0) 0x7fbfcc05ed70 con 0x7fbfcc09ff70 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient(hunting): my global_id is 5103 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb80034e0 con 0x7fbfcc09ff70 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2706758076 0 0) 0x7fbfcc09dce0 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient(hunting): my global_id is 5074 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfcc05ed70 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 787914078 0 0) 0x7fbfcc09dfc0 con 0x7fbfcc09fba0 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient(hunting): my global_id is 154137 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfcc09dce0 con 0x7fbfcc09fba0 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2484439629 0 0) 0x7fbfb80034e0 con 0x7fbfcc09ff70 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 >> v1:172.21.15.84:6792/0 conn(0x7fbfcc09f7d0 legacy=0x7fbfcc132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 >> v1:172.21.15.84:6790/0 conn(0x7fbfcc09fba0 legacy=0x7fbfcc137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient: _finish_hunting 0 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 monclient: found mon.noname-a 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc0a1c00 con 0x7fbfcc09ff70 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient: _finish_auth 0 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfd1f1d640 5 monclient: authenticate success, global_id 5103 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbfc4001bb0 con 0x7fbfcc09ff70 2024-09-18T14:06:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfd1f1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient: dump: 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfd1f1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbfc4001d10 con 0x7fbfcc09ff70 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfd1f1d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 1 -- 172.21.15.45:0/3422894028 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbfc4004f20 con 0x7fbfcc09ff70 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfd1f1d640 4 set_mon_vals no callback set 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.619+0000 7fbfcaffd640 10 monclient: dump: 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.620+0000 7fbfd1f1d640 10 monclient: shutdown 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.620+0000 7fbfd1f1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.620+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3422894028 >> v1:172.21.15.45:6789/0 conn(0x7fbfcc09ff70 legacy=0x7fbfcc13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.620+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3422894028 shutdown_connections 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.620+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3422894028 >> 172.21.15.45:0/3422894028 conn(0x7fbfcc0630b0 msgr2=0x7fbfcc0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.620+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3422894028 shutdown_connections 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.620+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3422894028 wait complete. 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.620+0000 7fbfd1f1d640 10 monclient: build_initial_monmap 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.620+0000 7fbfd1f1d640 10 monclient: monmap: 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:02.634 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.620+0000 7fbfd1f1d640 1 Processor -- start 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfd1f1d640 1 -- start start 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfd1f1d640 10 monclient: init 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfd1f1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfd1f1d640 10 monclient: _add_conns ranks=[0,1,7,2,4,8,3,6,5] 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfd1f1d640 10 monclient(hunting): picked mon.noname-a con 0x7fbfcc09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfd1f1d640 10 monclient(hunting): picked mon.noname-f con 0x7fbfcc09fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfd1f1d640 10 monclient(hunting): picked mon.noname-i con 0x7fbfcc09ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfd1f1d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfcc11d710 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfd1f1d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfcc0a2560 con 0x7fbfcc09fba0 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfd1f1d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfcc09d310 con 0x7fbfcc09ff70 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfd1f1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfd1f1d640 10 monclient(hunting): authenticate will time out at 1177.959472s 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfcb7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbfcc09f7d0 0x7fbfcc131720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33592/0 (socket says 172.21.15.45:33592) 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfcb7fe640 1 -- 172.21.15.45:0/3338771299 learned_addr learned my addr 172.21.15.45:0/3338771299 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991816453 0 0) 0x7fbfcc11d710 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 10 monclient(hunting): my global_id is 5112 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfac0034e0 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452027920 0 0) 0x7fbfcc0a2560 con 0x7fbfcc09fba0 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 10 monclient(hunting): my global_id is 5059 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfcc11d710 con 0x7fbfcc09fba0 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075761362 0 0) 0x7fbfcc09d310 con 0x7fbfcc09ff70 2024-09-18T14:06:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 10 monclient(hunting): my global_id is 5083 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfcc0a2560 con 0x7fbfcc09ff70 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2376801436 0 0) 0x7fbfac0034e0 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbfcc09d310 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.621+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbfbc001be0 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient(hunting): dump: 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1309863051 0 0) 0x7fbfcc11d710 con 0x7fbfcc09fba0 2024-09-18T14:06:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbfac0034e0 con 0x7fbfcc09fba0 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbfb4001aa0 con 0x7fbfcc09fba0 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient(hunting): dump: 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 813156773 0 0) 0x7fbfcc0a2560 con 0x7fbfcc09ff70 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbfcc11d710 con 0x7fbfcc09ff70 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3158487002 0 0) 0x7fbfcc09d310 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 >> v1:172.21.15.84:6792/0 conn(0x7fbfcc09ff70 legacy=0x7fbfcc14a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 >> v1:172.21.15.84:6789/0 conn(0x7fbfcc09fba0 legacy=0x7fbfcc146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient: _finish_hunting 0 2024-09-18T14:06:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 1 monclient: found mon.a 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc09ba00 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient: _finish_auth 0 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfd1f1d640 5 monclient: authenticate success, global_id 5112 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfd1f1d640 10 monclient: _renew_subs 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfd1f1d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfcc11ee00 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfd1f1d640 10 monclient: _renew_subs 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfd1f1d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfcc131e30 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbfbc001ff0 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbfbc0024a0 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfca7fc640 4 set_mon_vals no callback set 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.622+0000 7fbfc37fe640 10 monclient: dump: 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfd1f1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fbfbc002600 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbfb0006580 con 0x7fbfb0002830 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(325..325 src has 126..325) v4 ==== 3923+0+0 (unknown 847000907 0 0) 0x7fbfbc046a60 con 0x7fbfcc09f7d0 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748470062 0 0) 0x7fbfb0006580 con 0x7fbfb0002830 2024-09-18T14:06:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 10 monclient(hunting): my global_id is 154146 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfcc09d310 con 0x7fbfb0002830 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfcbfff640 10 monclient: get_auth_request con 0x7fbfac03a8c0 auth_method 0 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3351430646 0 0) 0x7fbfcc09d310 con 0x7fbfb0002830 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbfb0006580 con 0x7fbfb0002830 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbfb4002270 con 0x7fbfb0002830 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 10 monclient: dump: 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4077076088 0 0) 0x7fbfb0006580 con 0x7fbfb0002830 2024-09-18T14:06:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.623+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfb000a0a0 con 0x7fbfb0002830 2024-09-18T14:06:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.624+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fbfb4002750 con 0x7fbfb0002830 2024-09-18T14:06:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.624+0000 7fbfc37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.624+0000 7fbfc37fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.624+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 >> v1:172.21.15.84:6790/0 conn(0x7fbfb0002830 legacy=0x7fbfb0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.624+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 >> v1:172.21.15.84:6790/0 conn(0x7fbfb0002830 legacy=0x7fbfb0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfd1f1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbfb00074d0 con 0x7fbfb00068a0 2024-09-18T14:06:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521536844 0 0) 0x7fbfb00074d0 con 0x7fbfb00068a0 2024-09-18T14:06:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 10 monclient(hunting): my global_id is 154155 2024-09-18T14:06:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb0006580 con 0x7fbfb00068a0 2024-09-18T14:06:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1018317386 0 0) 0x7fbfb0006580 con 0x7fbfb00068a0 2024-09-18T14:06:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbfb00074d0 con 0x7fbfb00068a0 2024-09-18T14:06:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbfb4002cd0 con 0x7fbfb00068a0 2024-09-18T14:06:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:06:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 10 monclient: dump: 2024-09-18T14:06:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.650 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4217159037 0 0) 0x7fbfb00074d0 con 0x7fbfb00068a0 2024-09-18T14:06:02.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.630+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbfb0008750 con 0x7fbfb00068a0 2024-09-18T14:06:02.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 1564175015) 0x7fbfb4003480 con 0x7fbfb00068a0 2024-09-18T14:06:02.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfc37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:02.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfc37fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:02.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 >> v1:172.21.15.84:6790/0 conn(0x7fbfb00068a0 legacy=0x7fbfb0006d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfc37fe640 1 -- 172.21.15.45:0/3338771299 >> v1:172.21.15.84:6790/0 conn(0x7fbfb00068a0 legacy=0x7fbfb0006d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.650 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:02.670 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:02.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3338771299 >> v1:172.21.15.84:6812/31478 conn(0x7fbfac03a8c0 legacy=0x7fbfac03cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfd1f1d640 10 monclient: shutdown 2024-09-18T14:06:02.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfd1f1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3338771299 >> v1:172.21.15.45:6789/0 conn(0x7fbfcc09f7d0 legacy=0x7fbfcc131720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfcbfff640 1 -- 172.21.15.45:0/3338771299 reap_dead start 2024-09-18T14:06:02.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfcbfff640 1 -- 172.21.15.45:0/3338771299 reap_dead start 2024-09-18T14:06:02.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3338771299 shutdown_connections 2024-09-18T14:06:02.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3338771299 >> 172.21.15.45:0/3338771299 conn(0x7fbfcc0630b0 msgr2=0x7fbfcc0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3338771299 shutdown_connections 2024-09-18T14:06:02.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.631+0000 7fbfd1f1d640 1 -- 172.21.15.45:0/3338771299 wait complete. 2024-09-18T14:06:02.672 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:02.672 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:06:02.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:06:02.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.769+0000 7f6fd769f640 10 monclient: get_monmap_and_config 2024-09-18T14:06:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.769+0000 7f6fd769f640 10 monclient: build_initial_monmap 2024-09-18T14:06:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.769+0000 7f6fd769f640 10 monclient: monmap: 2024-09-18T14:06:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:02.770538+0000 2024-09-18T14:06:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:02.770538+0000 2024-09-18T14:06:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:02.771 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:02.771 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:02.771 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:02.771 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:02.771 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:02.771 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 1 Processor -- start 2024-09-18T14:06:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 1 -- start start 2024-09-18T14:06:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 10 monclient: init 2024-09-18T14:06:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 10 monclient: _add_conns ranks=[0,2,3,7,4,8,6,5,1] 2024-09-18T14:06:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 10 monclient(hunting): picked mon.noname-a con 0x7f6fd009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 10 monclient(hunting): picked mon.noname-b con 0x7f6fd009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 10 monclient(hunting): picked mon.noname-g con 0x7f6fd009ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd005ed70 con 0x7f6fd009f7d0 2024-09-18T14:06:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd009dfc0 con 0x7f6fd009fba0 2024-09-18T14:06:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd009dce0 con 0x7f6fd009ff70 2024-09-18T14:06:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd769f640 10 monclient(hunting): authenticate will time out at 1178.108889s 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.770+0000 7f6fd4c3a640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f6fd009fba0 0x7f6fd0137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39408/0 (socket says 172.21.15.45:39408) 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fd543b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6fd009f7d0 0x7f6fd0132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33600/0 (socket says 172.21.15.45:33600) 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fd4c3a640 1 -- 172.21.15.45:0/2102885786 learned_addr learned my addr 172.21.15.45:0/2102885786 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642662229 0 0) 0x7f6fd009dfc0 con 0x7f6fd009fba0 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient(hunting): my global_id is 144398 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fbc0034e0 con 0x7f6fd009fba0 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049118564 0 0) 0x7f6fd005ed70 con 0x7f6fd009f7d0 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient(hunting): my global_id is 5121 2024-09-18T14:06:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd009dfc0 con 0x7f6fd009f7d0 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506073837 0 0) 0x7f6fd009dce0 con 0x7f6fd009ff70 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient(hunting): my global_id is 154164 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd005ed70 con 0x7f6fd009ff70 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1362327912 0 0) 0x7f6fbc0034e0 con 0x7f6fd009fba0 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 >> v1:172.21.15.84:6790/0 conn(0x7f6fd009ff70 legacy=0x7f6fd013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 >> v1:172.21.15.45:6789/0 conn(0x7f6fd009f7d0 legacy=0x7f6fd0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient: _finish_hunting 0 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 monclient: found mon.noname-b 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:06:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00a1c00 con 0x7f6fd009fba0 2024-09-18T14:06:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient: _finish_auth 0 2024-09-18T14:06:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fc0001830 con 0x7f6fd009fba0 2024-09-18T14:06:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fd769f640 5 monclient: authenticate success, global_id 144398 2024-09-18T14:06:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fd769f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient: dump: 2024-09-18T14:06:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fd769f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6fc0001a10 con 0x7f6fd009fba0 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 1 -- 172.21.15.45:0/2102885786 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fc0004ba0 con 0x7f6fd009fba0 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fd769f640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fd769f640 4 set_mon_vals no callback set 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.771+0000 7f6fc7fff640 10 monclient: dump: 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.772+0000 7f6fd769f640 10 monclient: shutdown 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.772+0000 7f6fd769f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.772+0000 7f6fd769f640 1 -- 172.21.15.45:0/2102885786 >> v1:172.21.15.45:6790/0 conn(0x7f6fd009fba0 legacy=0x7f6fd0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.772+0000 7f6fd769f640 1 -- 172.21.15.45:0/2102885786 shutdown_connections 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.772+0000 7f6fd769f640 1 -- 172.21.15.45:0/2102885786 >> 172.21.15.45:0/2102885786 conn(0x7f6fd00630b0 msgr2=0x7f6fd00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.772+0000 7f6fd769f640 1 -- 172.21.15.45:0/2102885786 shutdown_connections 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.772+0000 7f6fd769f640 1 -- 172.21.15.45:0/2102885786 wait complete. 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.772+0000 7f6fd769f640 10 monclient: build_initial_monmap 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.772+0000 7f6fd769f640 10 monclient: monmap: 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 1 Processor -- start 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 1 -- start start 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 10 monclient: init 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 10 monclient: _add_conns ranks=[0,1,3,8,4,5,7,2,6] 2024-09-18T14:06:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 10 monclient(hunting): picked mon.noname-a con 0x7f6fd009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 10 monclient(hunting): picked mon.noname-f con 0x7f6fd009fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 10 monclient(hunting): picked mon.noname-g con 0x7f6fd009ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd009d310 con 0x7f6fd009f7d0 2024-09-18T14:06:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd00a2560 con 0x7f6fd009fba0 2024-09-18T14:06:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd0142600 con 0x7f6fd009ff70 2024-09-18T14:06:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd769f640 10 monclient(hunting): authenticate will time out at 1178.111786s 2024-09-18T14:06:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd543b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6fd009f7d0 0x7f6fd0141730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33610/0 (socket says 172.21.15.45:33610) 2024-09-18T14:06:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.773+0000 7f6fd543b640 1 -- 172.21.15.45:0/1295082386 learned_addr learned my addr 172.21.15.45:0/1295082386 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395961359 0 0) 0x7f6fd009d310 con 0x7f6fd009f7d0 2024-09-18T14:06:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): my global_id is 5130 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb40034e0 con 0x7f6fd009f7d0 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179707674 0 0) 0x7f6fd0142600 con 0x7f6fd009ff70 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): my global_id is 154173 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd009d310 con 0x7f6fd009ff70 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048846735 0 0) 0x7f6fd00a2560 con 0x7f6fd009fba0 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): my global_id is 5068 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd0142600 con 0x7f6fd009fba0 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3109902033 0 0) 0x7f6fb40034e0 con 0x7f6fd009f7d0 2024-09-18T14:06:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fd00a2560 con 0x7f6fd009f7d0 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fcc001540 con 0x7f6fd009f7d0 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): dump: 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1907875887 0 0) 0x7f6fd009d310 con 0x7f6fd009ff70 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fb40034e0 con 0x7f6fd009ff70 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fc8001d00 con 0x7f6fd009ff70 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:06:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient(hunting): dump: 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1835549158 0 0) 0x7f6fd00a2560 con 0x7f6fd009f7d0 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 >> v1:172.21.15.84:6790/0 conn(0x7f6fd009ff70 legacy=0x7f6fd0149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 >> v1:172.21.15.84:6789/0 conn(0x7f6fd009fba0 legacy=0x7f6fd0131930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient: _finish_hunting 0 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 monclient: found mon.a 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd0120b20 con 0x7f6fd009f7d0 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 10 monclient: _finish_auth 0 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fc57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fd769f640 5 monclient: authenticate success, global_id 5130 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fd769f640 10 monclient: _renew_subs 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fd769f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fd769f640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd0121a80 con 0x7f6fd009f7d0 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fd769f640 10 monclient: _renew_subs 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fd769f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.774+0000 7f6fd769f640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd0120650 con 0x7f6fd009f7d0 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.775+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6fcc004a50 con 0x7f6fd009f7d0 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.775+0000 7f6fc57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.775+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fcc004f00 con 0x7f6fd009f7d0 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.775+0000 7f6fc57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.775+0000 7f6fc77fe640 4 set_mon_vals no callback set 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.775+0000 7f6fc57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.775+0000 7f6fc57fa640 10 monclient: dump: 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.775+0000 7f6fd769f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.775+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f6fcc00fed0 con 0x7f6fd009f7d0 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.775+0000 7f6fd769f640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6fb0006580 con 0x7f6fb0002830 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.775+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(325..325 src has 126..325) v4 ==== 3923+0+0 (unknown 847000907 0 0) 0x7f6fcc046aa0 con 0x7f6fd009f7d0 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.775+0000 7f6fd4c3a640 10 monclient: get_auth_request con 0x7f6fb403a600 auth_method 0 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782197369 0 0) 0x7f6fb0006580 con 0x7f6fb0002830 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 10 monclient(hunting): my global_id is 4964 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd00a2560 con 0x7f6fb0002830 2024-09-18T14:06:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2071458132 0 0) 0x7f6fd00a2560 con 0x7f6fb0002830 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fb0006580 con 0x7f6fb0002830 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fc80021a0 con 0x7f6fb0002830 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 10 monclient: dump: 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3836706482 0 0) 0x7f6fb0006580 con 0x7f6fb0002830 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6fb000a0a0 con 0x7f6fb0002830 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6fc8002520 con 0x7f6fb0002830 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 >> v1:172.21.15.84:6791/0 conn(0x7f6fb0002830 legacy=0x7f6fb0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.776+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 >> v1:172.21.15.84:6791/0 conn(0x7f6fb0002830 legacy=0x7f6fb0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.782+0000 7f6fd769f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.782+0000 7f6fd769f640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6fb0009610 con 0x7f6fb0008ff0 2024-09-18T14:06:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.782+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120019249 0 0) 0x7f6fb0009610 con 0x7f6fb0008ff0 2024-09-18T14:06:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.782+0000 7f6fc57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.782+0000 7f6fc57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.782+0000 7f6fc57fa640 10 monclient(hunting): my global_id is 4973 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.782+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb0006580 con 0x7f6fb0008ff0 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1735104293 0 0) 0x7f6fb0006580 con 0x7f6fb0008ff0 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fb0009610 con 0x7f6fb0008ff0 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fc8002d20 con 0x7f6fb0008ff0 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 10 monclient: dump: 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 490395860 0 0) 0x7f6fb0009610 con 0x7f6fb0008ff0 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6fb00098d0 con 0x7f6fb0008ff0 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 2488005319) 0x7f6fc80062a0 con 0x7f6fb0008ff0 2024-09-18T14:06:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:02.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:02.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 >> v1:172.21.15.84:6791/0 conn(0x7f6fb0008ff0 legacy=0x7f6fb0008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fc57fa640 1 -- 172.21.15.45:0/1295082386 >> v1:172.21.15.84:6791/0 conn(0x7f6fb0008ff0 legacy=0x7f6fb0008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.803 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:02.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:02.803 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:02.803 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:02.803 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:02.803 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:02.803 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:02.805 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:02.823 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:02.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fd769f640 1 -- 172.21.15.45:0/1295082386 >> v1:172.21.15.84:6812/31478 conn(0x7f6fb403a600 legacy=0x7f6fb403ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fd769f640 10 monclient: shutdown 2024-09-18T14:06:02.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fd769f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fd769f640 1 -- 172.21.15.45:0/1295082386 >> v1:172.21.15.45:6789/0 conn(0x7f6fd009f7d0 legacy=0x7f6fd0141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fd5c3c640 1 -- 172.21.15.45:0/1295082386 reap_dead start 2024-09-18T14:06:02.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.783+0000 7f6fd5c3c640 1 -- 172.21.15.45:0/1295082386 reap_dead start 2024-09-18T14:06:02.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.784+0000 7f6fd769f640 1 -- 172.21.15.45:0/1295082386 shutdown_connections 2024-09-18T14:06:02.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.784+0000 7f6fd769f640 1 -- 172.21.15.45:0/1295082386 >> 172.21.15.45:0/1295082386 conn(0x7f6fd00630b0 msgr2=0x7f6fd00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.784+0000 7f6fd769f640 1 -- 172.21.15.45:0/1295082386 shutdown_connections 2024-09-18T14:06:02.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.784+0000 7f6fd769f640 1 -- 172.21.15.45:0/1295082386 wait complete. 2024-09-18T14:06:02.825 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:02.825 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:06:02.825 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:06:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.922+0000 7f384b8ef640 10 monclient: get_monmap_and_config 2024-09-18T14:06:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.922+0000 7f384b8ef640 10 monclient: build_initial_monmap 2024-09-18T14:06:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.922+0000 7f384b8ef640 10 monclient: monmap: 2024-09-18T14:06:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:02.923614+0000 2024-09-18T14:06:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:02.923614+0000 2024-09-18T14:06:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:02.924 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:02.924 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:02.924 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:02.924 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:02.924 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:02.924 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:02.924 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:02.924 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 1 Processor -- start 2024-09-18T14:06:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 1 -- start start 2024-09-18T14:06:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 10 monclient: init 2024-09-18T14:06:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 10 monclient: _add_conns ranks=[0,8,1,6,4,3,7,5,2] 2024-09-18T14:06:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 10 monclient(hunting): picked mon.noname-a con 0x7f384409f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 10 monclient(hunting): picked mon.noname-e con 0x7f384409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 10 monclient(hunting): picked mon.noname-f con 0x7f384409ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384405ed70 con 0x7f384409f7d0 2024-09-18T14:06:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384409dfc0 con 0x7f384409ff70 2024-09-18T14:06:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384409dce0 con 0x7f384409fba0 2024-09-18T14:06:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.923+0000 7f384b8ef640 10 monclient(hunting): authenticate will time out at 1178.261944s 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f384968b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f384409f7d0 0x7f3844132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33622/0 (socket says 172.21.15.45:33622) 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f384968b640 1 -- 172.21.15.45:0/3762757393 learned_addr learned my addr 172.21.15.45:0/3762757393 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 514423812 0 0) 0x7f384405ed70 con 0x7f384409f7d0 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient(hunting): my global_id is 5139 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38300034e0 con 0x7f384409f7d0 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455969573 0 0) 0x7f384409dce0 con 0x7f384409fba0 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient(hunting): my global_id is 5030 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f384405ed70 con 0x7f384409fba0 2024-09-18T14:06:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378931565 0 0) 0x7f384409dfc0 con 0x7f384409ff70 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient(hunting): my global_id is 5077 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f384409dce0 con 0x7f384409ff70 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 441746483 0 0) 0x7f384405ed70 con 0x7f384409fba0 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 >> v1:172.21.15.84:6789/0 conn(0x7f384409ff70 legacy=0x7f384413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 >> v1:172.21.15.45:6789/0 conn(0x7f384409f7d0 legacy=0x7f3844132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 monclient: found mon.noname-e 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38440a1c00 con 0x7f384409fba0 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient: _finish_auth 0 2024-09-18T14:06:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3834001790 con 0x7f384409fba0 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f384b8ef640 5 monclient: authenticate success, global_id 5030 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f384b8ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient: dump: 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f384b8ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f38340018f0 con 0x7f384409fba0 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 1 -- 172.21.15.45:0/3762757393 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3834001ea0 con 0x7f384409fba0 2024-09-18T14:06:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f384b8ef640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f384b8ef640 4 set_mon_vals no callback set 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.924+0000 7f383bfff640 10 monclient: dump: 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.925+0000 7f384b8ef640 10 monclient: shutdown 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.925+0000 7f384b8ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.925+0000 7f384b8ef640 1 -- 172.21.15.45:0/3762757393 >> v1:172.21.15.45:6793/0 conn(0x7f384409fba0 legacy=0x7f3844137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.925+0000 7f384b8ef640 1 -- 172.21.15.45:0/3762757393 shutdown_connections 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.925+0000 7f384b8ef640 1 -- 172.21.15.45:0/3762757393 >> 172.21.15.45:0/3762757393 conn(0x7f38440630b0 msgr2=0x7f38440634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.925+0000 7f384b8ef640 1 -- 172.21.15.45:0/3762757393 shutdown_connections 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.925+0000 7f384b8ef640 1 -- 172.21.15.45:0/3762757393 wait complete. 2024-09-18T14:06:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.925+0000 7f384b8ef640 10 monclient: build_initial_monmap 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.925+0000 7f384b8ef640 10 monclient: monmap: 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 1 Processor -- start 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 1 -- start start 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 10 monclient: init 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 10 monclient: _add_conns ranks=[5,6,2,8,4,3,0,7,1] 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 10 monclient(hunting): picked mon.noname-h con 0x7f384409f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 10 monclient(hunting): picked mon.noname-d con 0x7f384409fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 10 monclient(hunting): picked mon.noname-b con 0x7f384409ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384409d310 con 0x7f384409ff70 2024-09-18T14:06:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38440a2560 con 0x7f384409f7d0 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3844142050 con 0x7f384409fba0 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f384b8ef640 10 monclient(hunting): authenticate will time out at 1178.264812s 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f3849e8c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f384409ff70 0x7f3844149cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39420/0 (socket says 172.21.15.45:39420) 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f3849e8c640 1 -- 172.21.15.45:0/2030292105 learned_addr learned my addr 172.21.15.45:0/2030292105 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.926+0000 7f3848e8a640 1 --1- 172.21.15.45:0/2030292105 >> v1:172.21.15.45:6792/0 conn(0x7f384409fba0 0x7f3844131bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:54816/0 (socket says 172.21.15.45:54816) 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535989066 0 0) 0x7f384409d310 con 0x7f384409ff70 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): my global_id is 144407 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38280034e0 con 0x7f384409ff70 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396356130 0 0) 0x7f3844142050 con 0x7f384409fba0 2024-09-18T14:06:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): my global_id is 5001 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f384409d310 con 0x7f384409fba0 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674672382 0 0) 0x7f38440a2560 con 0x7f384409f7d0 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): my global_id is 4982 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3844142050 con 0x7f384409f7d0 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f384968b640 1 -- 172.21.15.45:0/2030292105 >> v1:172.21.15.84:6791/0 conn(0x7f384409f7d0 legacy=0x7f3844141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f384968b640 1 -- 172.21.15.45:0/2030292105 >> v1:172.21.15.84:6791/0 conn(0x7f384409f7d0 legacy=0x7f3844141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f384968b640 1 --1- 172.21.15.45:0/2030292105 >> v1:172.21.15.84:6791/0 conn(0x7f384409f7d0 0x7f3844141940 :-1 s=OPENED pgs=246 cs=1 l=1).handle_message read tag failed 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f384968b640 1 --1- 172.21.15.45:0/2030292105 >> v1:172.21.15.84:6791/0 conn(0x7f384409f7d0 0x7f3844141940 :-1 s=OPENED pgs=246 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.84:6791/0 2024-09-18T14:06:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2360588313 0 0) 0x7f38280034e0 con 0x7f384409ff70 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f38440a2560 con 0x7f384409ff70 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f383c001d60 con 0x7f384409ff70 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): dump: 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1724529840 0 0) 0x7f384409d310 con 0x7f384409fba0 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f38280034e0 con 0x7f384409fba0 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3834001da0 con 0x7f384409fba0 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient(hunting): dump: 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2313228854 0 0) 0x7f38440a2560 con 0x7f384409ff70 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 >> v1:172.21.15.45:6792/0 conn(0x7f384409fba0 legacy=0x7f3844131bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 >> v1:172.21.15.84:6791/0 conn(0x7f384409f7d0 legacy=0x7f3844141940 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient: _finish_hunting 0 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 monclient: found mon.b 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3844142800 con 0x7f384409ff70 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient: _finish_auth 0 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f384b8ef640 5 monclient: authenticate success, global_id 144407 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f384b8ef640 10 monclient: _renew_subs 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f384b8ef640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f384b8ef640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3844120db0 con 0x7f384409ff70 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f384b8ef640 10 monclient: _renew_subs 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f384b8ef640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f384b8ef640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f38441201e0 con 0x7f384409ff70 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f383c004c80 con 0x7f384409ff70 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f383c005130 con 0x7f384409ff70 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f383affd640 4 set_mon_vals no callback set 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.927+0000 7f38397fa640 10 monclient: dump: 2024-09-18T14:06:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.928+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f383c010100 con 0x7f384409ff70 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.928+0000 7f384b8ef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.928+0000 7f384b8ef640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3824006580 con 0x7f3824002830 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.928+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(325..325 src has 126..325) v4 ==== 3923+0+0 (unknown 847000907 0 0) 0x7f383c046e80 con 0x7f384409ff70 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.928+0000 7f3848e8a640 10 monclient: get_auth_request con 0x7f382803a530 auth_method 0 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.928+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322738853 0 0) 0x7f3824006580 con 0x7f3824002830 2024-09-18T14:06:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.928+0000 7f38397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.928+0000 7f38397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.928+0000 7f38397fa640 10 monclient(hunting): my global_id is 5092 2024-09-18T14:06:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.928+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38440a2560 con 0x7f3824002830 2024-09-18T14:06:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4043307914 0 0) 0x7f38440a2560 con 0x7f3824002830 2024-09-18T14:06:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3824006580 con 0x7f3824002830 2024-09-18T14:06:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3840001650 con 0x7f3824002830 2024-09-18T14:06:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 10 monclient: dump: 2024-09-18T14:06:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 908163590 0 0) 0x7f3824006580 con 0x7f3824002830 2024-09-18T14:06:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f382400a0a0 con 0x7f3824002830 2024-09-18T14:06:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3840004910 con 0x7f3824002830 2024-09-18T14:06:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 >> v1:172.21.15.84:6792/0 conn(0x7f3824002830 legacy=0x7f3824004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.929+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 >> v1:172.21.15.84:6792/0 conn(0x7f3824002830 legacy=0x7f3824004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.935+0000 7f384b8ef640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.935+0000 7f384b8ef640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3824009660 con 0x7f3824009040 2024-09-18T14:06:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.935+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375476624 0 0) 0x7f3824009660 con 0x7f3824009040 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.935+0000 7f38397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.935+0000 7f38397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.935+0000 7f38397fa640 10 monclient(hunting): my global_id is 5101 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.935+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3824006580 con 0x7f3824009040 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2612461865 0 0) 0x7f3824006580 con 0x7f3824009040 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3824009660 con 0x7f3824009040 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f38400050d0 con 0x7f3824009040 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 10 monclient: dump: 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:02.955 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 60544058 0 0) 0x7f3824009660 con 0x7f3824009040 2024-09-18T14:06:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3824009920 con 0x7f3824009040 2024-09-18T14:06:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 1242701640) 0x7f3840005460 con 0x7f3824009040 2024-09-18T14:06:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 >> v1:172.21.15.84:6792/0 conn(0x7f3824009040 legacy=0x7f3824008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f38397fa640 1 -- 172.21.15.45:0/2030292105 >> v1:172.21.15.84:6792/0 conn(0x7f3824009040 legacy=0x7f3824008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:02.956 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:02.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:02.956 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:02.956 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.958 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.959 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:02.975 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:02.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f384b8ef640 1 -- 172.21.15.45:0/2030292105 >> v1:172.21.15.84:6812/31478 conn(0x7f382803a530 legacy=0x7f382803c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f384b8ef640 10 monclient: shutdown 2024-09-18T14:06:02.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f384b8ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:02.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f384b8ef640 1 -- 172.21.15.45:0/2030292105 >> v1:172.21.15.45:6790/0 conn(0x7f384409ff70 legacy=0x7f3844149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:02.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f3849e8c640 1 -- 172.21.15.45:0/2030292105 reap_dead start 2024-09-18T14:06:02.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f3849e8c640 1 -- 172.21.15.45:0/2030292105 reap_dead start 2024-09-18T14:06:02.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f384b8ef640 1 -- 172.21.15.45:0/2030292105 shutdown_connections 2024-09-18T14:06:02.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.936+0000 7f384b8ef640 1 -- 172.21.15.45:0/2030292105 >> 172.21.15.45:0/2030292105 conn(0x7f38440630b0 msgr2=0x7f38440a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:02.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.937+0000 7f384b8ef640 1 -- 172.21.15.45:0/2030292105 shutdown_connections 2024-09-18T14:06:02.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:02.937+0000 7f384b8ef640 1 -- 172.21.15.45:0/2030292105 wait complete. 2024-09-18T14:06:02.978 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:02.978 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-18T14:06:02.978 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-18T14:06:03.041 INFO:tasks.rados.rados.0.smithi084.stdout:1452: done (3 left) 2024-09-18T14:06:03.041 INFO:tasks.rados.rados.0.smithi084.stdout:1454: done (2 left) 2024-09-18T14:06:03.041 INFO:tasks.rados.rados.0.smithi084.stdout:1455: done (1 left) 2024-09-18T14:06:03.041 INFO:tasks.rados.rados.0.smithi084.stdout:1456: done (0 left) 2024-09-18T14:06:03.041 INFO:tasks.rados.rados.0.smithi084.stdout:1457: delete oid 49 current snap is 157 2024-09-18T14:06:03.050 INFO:tasks.rados.rados.0.smithi084.stdout:1457: done (0 left) 2024-09-18T14:06:03.050 INFO:tasks.rados.rados.0.smithi084.stdout:1458: read oid 32 snap 156 2024-09-18T14:06:03.050 INFO:tasks.rados.rados.0.smithi084.stdout:1458: expect (ObjNum 487 snap 151 seq_num 487) 2024-09-18T14:06:03.050 INFO:tasks.rados.rados.0.smithi084.stdout:1459: write (excl) oid 7 current snap is 157 2024-09-18T14:06:03.050 INFO:tasks.rados.rados.0.smithi084.stdout:1459: seq_num 511 ranges {588377=647890,1935828=635840} 2024-09-18T14:06:03.057 INFO:tasks.rados.rados.0.smithi084.stdout:1459: writing smithi08434663-7 from 588377 to 1236267 tid 1 2024-09-18T14:06:03.064 INFO:tasks.rados.rados.0.smithi084.stdout:1459: writing smithi08434663-7 from 1935828 to 2571668 tid 2 2024-09-18T14:06:03.064 INFO:tasks.rados.rados.0.smithi084.stdout:1460: copy_from oid 36 from oid 38 current snap is 157 2024-09-18T14:06:03.064 INFO:tasks.rados.rados.0.smithi084.stdout:1458: expect (ObjNum 487 snap 151 seq_num 487) 2024-09-18T14:06:03.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.083+0000 7f018e59d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:03.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.083+0000 7f018e59d640 10 monclient: build_initial_monmap 2024-09-18T14:06:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.083+0000 7f018e59d640 10 monclient: monmap: 2024-09-18T14:06:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:03.084527+0000 2024-09-18T14:06:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:03.084527+0000 2024-09-18T14:06:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:03.085 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:03.085 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:03.085 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:03.085 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:03.085 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 1 Processor -- start 2024-09-18T14:06:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 1 -- start start 2024-09-18T14:06:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 10 monclient: init 2024-09-18T14:06:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 10 monclient: _add_conns ranks=[1,8,0,2,7,3,4,6,5] 2024-09-18T14:06:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 10 monclient(hunting): picked mon.noname-f con 0x7f01880a2520 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 10 monclient(hunting): picked mon.noname-e con 0x7f0188096a20 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 10 monclient(hunting): picked mon.noname-a con 0x7f0188132fc0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f018805ed70 con 0x7f0188132fc0 2024-09-18T14:06:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0188097040 con 0x7f01880a2520 2024-09-18T14:06:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01880a28f0 con 0x7f0188096a20 2024-09-18T14:06:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.084+0000 7f018e59d640 10 monclient(hunting): authenticate will time out at 1178.422916s 2024-09-18T14:06:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f018cb3a640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0188132fc0 0x7f018813ae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33630/0 (socket says 172.21.15.45:33630) 2024-09-18T14:06:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f018cb3a640 1 -- 172.21.15.45:0/3320526635 learned_addr learned my addr 172.21.15.45:0/3320526635 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913274871 0 0) 0x7f018805ed70 con 0x7f0188132fc0 2024-09-18T14:06:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient(hunting): my global_id is 5148 2024-09-18T14:06:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01700034e0 con 0x7f0188132fc0 2024-09-18T14:06:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811531515 0 0) 0x7f01880a28f0 con 0x7f0188096a20 2024-09-18T14:06:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient(hunting): my global_id is 5039 2024-09-18T14:06:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f018805ed70 con 0x7f0188096a20 2024-09-18T14:06:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539058228 0 0) 0x7f0188097040 con 0x7f01880a2520 2024-09-18T14:06:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient(hunting): my global_id is 5086 2024-09-18T14:06:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01880a28f0 con 0x7f01880a2520 2024-09-18T14:06:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3615042793 0 0) 0x7f01700034e0 con 0x7f0188132fc0 2024-09-18T14:06:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 >> v1:172.21.15.45:6793/0 conn(0x7f0188096a20 legacy=0x7f0188137750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 >> v1:172.21.15.84:6789/0 conn(0x7f01880a2520 legacy=0x7f01881328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:06:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 monclient: found mon.noname-a 2024-09-18T14:06:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:06:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018809f4b0 con 0x7f0188132fc0 2024-09-18T14:06:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient: _finish_auth 0 2024-09-18T14:06:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0180001bb0 con 0x7f0188132fc0 2024-09-18T14:06:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f018e59d640 5 monclient: authenticate success, global_id 5148 2024-09-18T14:06:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f018e59d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:06:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient: dump: 2024-09-18T14:06:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:03.101 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f018e59d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0180001d10 con 0x7f0188132fc0 2024-09-18T14:06:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 1 -- 172.21.15.45:0/3320526635 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0180004f20 con 0x7f0188132fc0 2024-09-18T14:06:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f018e59d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f018e59d640 4 set_mon_vals no callback set 2024-09-18T14:06:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.085+0000 7f0186ffd640 10 monclient: dump: 2024-09-18T14:06:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:03.104 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.086+0000 7f018e59d640 10 monclient: shutdown 2024-09-18T14:06:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.086+0000 7f018e59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.086+0000 7f018e59d640 1 -- 172.21.15.45:0/3320526635 >> v1:172.21.15.45:6789/0 conn(0x7f0188132fc0 legacy=0x7f018813ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.086+0000 7f018e59d640 1 -- 172.21.15.45:0/3320526635 shutdown_connections 2024-09-18T14:06:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.086+0000 7f018e59d640 1 -- 172.21.15.45:0/3320526635 >> 172.21.15.45:0/3320526635 conn(0x7f0188063130 msgr2=0x7f0188063500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.086+0000 7f018e59d640 1 -- 172.21.15.45:0/3320526635 shutdown_connections 2024-09-18T14:06:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.086+0000 7f018e59d640 1 -- 172.21.15.45:0/3320526635 wait complete. 2024-09-18T14:06:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.086+0000 7f018e59d640 10 monclient: build_initial_monmap 2024-09-18T14:06:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.086+0000 7f018e59d640 10 monclient: monmap: 2024-09-18T14:06:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:03.106 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 1 Processor -- start 2024-09-18T14:06:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 1 -- start start 2024-09-18T14:06:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 10 monclient: init 2024-09-18T14:06:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 10 monclient: _add_conns ranks=[6,4,1,2,5,3,8,0,7] 2024-09-18T14:06:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 10 monclient(hunting): picked mon.noname-d con 0x7f0188096a20 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 10 monclient(hunting): picked mon.noname-c con 0x7f01880a2520 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 10 monclient(hunting): picked mon.noname-f con 0x7f0188132fc0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f018809f8e0 con 0x7f0188132fc0 2024-09-18T14:06:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01880a2270 con 0x7f01880a2520 2024-09-18T14:06:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f018809c120 con 0x7f0188096a20 2024-09-18T14:06:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f018e59d640 10 monclient(hunting): authenticate will time out at 1178.425735s 2024-09-18T14:06:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f0187fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f0188096a20 0x7f018809a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:54818/0 (socket says 172.21.15.45:54818) 2024-09-18T14:06:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f0187fff640 1 -- 172.21.15.45:0/2974595256 learned_addr learned my addr 172.21.15.45:0/2974595256 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.087+0000 7f01877fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f01880a2520 0x7f018809af10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58500/0 (socket says 172.21.15.45:58500) 2024-09-18T14:06:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657592842 0 0) 0x7f018809c120 con 0x7f0188096a20 2024-09-18T14:06:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): my global_id is 5010 2024-09-18T14:06:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01740034e0 con 0x7f0188096a20 2024-09-18T14:06:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590338550 0 0) 0x7f01880a2270 con 0x7f01880a2520 2024-09-18T14:06:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): my global_id is 154165 2024-09-18T14:06:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f018809c120 con 0x7f01880a2520 2024-09-18T14:06:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413912276 0 0) 0x7f018809f8e0 con 0x7f0188132fc0 2024-09-18T14:06:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): my global_id is 5095 2024-09-18T14:06:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01880a2270 con 0x7f0188132fc0 2024-09-18T14:06:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3751425035 0 0) 0x7f018809c120 con 0x7f01880a2520 2024-09-18T14:06:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f018809f8e0 con 0x7f01880a2520 2024-09-18T14:06:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f017c001cc0 con 0x7f01880a2520 2024-09-18T14:06:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:06:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): dump: 2024-09-18T14:06:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:03.111 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1946028763 0 0) 0x7f01740034e0 con 0x7f0188096a20 2024-09-18T14:06:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f018809c120 con 0x7f0188096a20 2024-09-18T14:06:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1181999666 0 0) 0x7f01880a2270 con 0x7f0188132fc0 2024-09-18T14:06:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f01740034e0 con 0x7f0188132fc0 2024-09-18T14:06:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0178001a10 con 0x7f0188096a20 2024-09-18T14:06:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): dump: 2024-09-18T14:06:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0180001af0 con 0x7f0188132fc0 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient(hunting): dump: 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:03.114 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1049075369 0 0) 0x7f018809f8e0 con 0x7f01880a2520 2024-09-18T14:06:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 >> v1:172.21.15.45:6792/0 conn(0x7f0188096a20 legacy=0x7f018809a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 >> v1:172.21.15.84:6789/0 conn(0x7f0188132fc0 legacy=0x7f0188149880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient: _finish_hunting 0 2024-09-18T14:06:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 monclient: found mon.c 2024-09-18T14:06:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018809dbe0 con 0x7f01880a2520 2024-09-18T14:06:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient: _finish_auth 0 2024-09-18T14:06:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f018e59d640 5 monclient: authenticate success, global_id 154165 2024-09-18T14:06:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f018e59d640 10 monclient: _renew_subs 2024-09-18T14:06:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f018e59d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f018e59d640 1 -- 172.21.15.45:0/2974595256 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f018812cfe0 con 0x7f01880a2520 2024-09-18T14:06:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f018e59d640 10 monclient: _renew_subs 2024-09-18T14:06:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f018e59d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f018e59d640 1 -- 172.21.15.45:0/2974595256 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f018809b620 con 0x7f01880a2520 2024-09-18T14:06:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f017c001770 con 0x7f01880a2520 2024-09-18T14:06:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f017c002510 con 0x7f01880a2520 2024-09-18T14:06:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f01867fc640 4 set_mon_vals no callback set 2024-09-18T14:06:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.088+0000 7f016ffff640 10 monclient: dump: 2024-09-18T14:06:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:03.118 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.089+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f017c010050 con 0x7f01880a2520 2024-09-18T14:06:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.089+0000 7f018e59d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.089+0000 7f018e59d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.089+0000 7f018e59d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.089+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(326..326 src has 126..326) v4 ==== 3895+0+0 (unknown 3313254731 0 0) 0x7f017c046a40 con 0x7f01880a2520 2024-09-18T14:06:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.089+0000 7f018e59d640 1 -- 172.21.15.45:0/2974595256 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0168005010 con 0x7f01880a2520 2024-09-18T14:06:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.089+0000 7f0187fff640 10 monclient: get_auth_request con 0x7f017403aa60 auth_method 0 2024-09-18T14:06:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.094+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.4 v1:172.21.15.45:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f017c015230 con 0x7f01880a2520 2024-09-18T14:06:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.094+0000 7f016ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.094+0000 7f016ffff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:03.152 INFO:tasks.rados.rados.0.smithi084.stdout:1459: finishing write tid 1 to smithi08434663-7 2024-09-18T14:06:03.152 INFO:tasks.rados.rados.0.smithi084.stdout:1459: finishing write tid 2 to smithi08434663-7 2024-09-18T14:06:03.152 INFO:tasks.rados.rados.0.smithi084.stdout:1459: finishing write tid 3 to smithi08434663-7 2024-09-18T14:06:03.152 INFO:tasks.rados.rados.0.smithi084.stdout:1459: finishing write tid 4 to smithi08434663-7 2024-09-18T14:06:03.152 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 565 (ObjNum 511 snap 157 seq_num 511) dirty exists 2024-09-18T14:06:03.152 INFO:tasks.rados.rados.0.smithi084.stdout:1459: left oid 7 (ObjNum 511 snap 157 seq_num 511) 2024-09-18T14:06:03.152 INFO:tasks.rados.rados.0.smithi084.stdout:1460: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:06:03.152 INFO:tasks.rados.rados.0.smithi084.stdout:1460: finishing copy_from to smithi08434663-36 2024-09-18T14:06:03.153 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 403 (ObjNum 496 snap 153 seq_num 496) dirty exists 2024-09-18T14:06:03.153 INFO:tasks.rados.rados.0.smithi084.stdout:1458: done (2 left) 2024-09-18T14:06:03.153 INFO:tasks.rados.rados.0.smithi084.stdout:1459: done (1 left) 2024-09-18T14:06:03.153 INFO:tasks.rados.rados.0.smithi084.stdout:1460: done (0 left) 2024-09-18T14:06:03.153 INFO:tasks.rados.rados.0.smithi084.stdout:1461: read oid 5 snap 141 2024-09-18T14:06:03.153 INFO:tasks.rados.rados.0.smithi084.stdout:1461: expect (ObjNum 452 snap 139 seq_num 452) 2024-09-18T14:06:03.153 INFO:tasks.rados.rados.0.smithi084.stdout:1462: write oid 31 current snap is 157 2024-09-18T14:06:03.153 INFO:tasks.rados.rados.0.smithi084.stdout:1462: seq_num 512 ranges {605184=723139,1877957=727227} 2024-09-18T14:06:03.158 INFO:tasks.rados.rados.0.smithi084.stdout:1462: writing smithi08434663-31 from 605184 to 1328323 tid 1 2024-09-18T14:06:03.165 INFO:tasks.rados.rados.0.smithi084.stdout:1462: writing smithi08434663-31 from 1877957 to 2605184 tid 2 2024-09-18T14:06:03.165 INFO:tasks.rados.rados.0.smithi084.stdout:1463: snap_create 2024-09-18T14:06:03.165 INFO:tasks.rados.rados.0.smithi084.stdout:1461: expect (ObjNum 452 snap 139 seq_num 452) 2024-09-18T14:06:03.285 INFO:tasks.rados.rados.0.smithi084.stdout:1462: finishing write tid 1 to smithi08434663-31 2024-09-18T14:06:03.285 INFO:tasks.rados.rados.0.smithi084.stdout:1462: finishing write tid 2 to smithi08434663-31 2024-09-18T14:06:03.285 INFO:tasks.rados.rados.0.smithi084.stdout:1462: finishing write tid 3 to smithi08434663-31 2024-09-18T14:06:03.286 INFO:tasks.rados.rados.0.smithi084.stdout:1462: finishing write tid 4 to smithi08434663-31 2024-09-18T14:06:03.286 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 368 (ObjNum 512 snap 157 seq_num 512) dirty exists 2024-09-18T14:06:03.286 INFO:tasks.rados.rados.0.smithi084.stdout:1462: left oid 31 (ObjNum 512 snap 157 seq_num 512) 2024-09-18T14:06:03.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.326+0000 7f018e59d640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-18T14:06:03.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.326+0000 7f018e59d640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-18T14:06:03.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.326+0000 7f018e59d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:03.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.326+0000 7f018e59d640 1 -- 172.21.15.45:0/2974595256 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f0168005ad0 con 0x7f01880a2520 2024-09-18T14:06:03.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f016ffff640 1 -- 172.21.15.45:0/2974595256 <== mon.4 v1:172.21.15.45:6791/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f017c014c30 con 0x7f01880a2520 2024-09-18T14:06:03.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f016ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-18T14:06:03.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f016ffff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:03.329 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.329 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=2916162509,config=3140514009,health=2677522725,kv=594742435,logm=965370430}) 2024-09-18T14:06:03.329 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=21,config=2,health=12,kv=7,logm=58} crc {auth=2727432087,config=3140514009,health=2677522725,kv=2464314623,logm=523665261}) 2024-09-18T14:06:03.329 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.329 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=2916162509,config=3140514009,health=2677522725,kv=594742435,logm=965370430}) 2024-09-18T14:06:03.329 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=2118973216,config=3140514009,health=2677522725,kv=594742435,logm=1518631826}) 2024-09-18T14:06:03.329 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.329 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=2916162509,config=3140514009,health=2677522725,kv=594742435,logm=965370430}) 2024-09-18T14:06:03.329 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=22,config=2,health=11,kv=8,logm=57} crc {auth=2916162509,config=3140514009,health=3428715404,kv=594742435,logm=748751266}) 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=2916162509,config=3140514009,health=2677522725,kv=594742435,logm=965370430}) 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=21,config=2,health=12,kv=8,logm=57} crc {auth=480487253,config=3140514009,health=2677522725,kv=594742435,logm=1826093754}) 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=2916162509,config=3140514009,health=2677522725,kv=594742435,logm=965370430}) 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=2751327795,config=3140514009,health=2677522725,kv=594742435,logm=3000645578}) 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=2916162509,config=3140514009,health=2677522725,kv=594742435,logm=965370430}) 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=8,logm=59} crc {auth=3270191821,config=3140514009,health=1850441919,kv=594742435,logm=3167941512}) 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=2916162509,config=3140514009,health=2677522725,kv=594742435,logm=965370430}) 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=21,config=2,health=11,kv=8,logm=58} crc {auth=3576908621,config=3140514009,health=1214664604,kv=2284840295,logm=1955051592}) 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.330 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=2916162509,config=3140514009,health=2677522725,kv=594742435,logm=965370430}) 2024-09-18T14:06:03.331 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.328+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=22,config=2,health=11,kv=8,logm=57} crc {auth=3841637308,config=3140514009,health=4224009463,kv=594742435,logm=4277921222}) 2024-09-18T14:06:03.333 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.333 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1133377906}) 2024-09-18T14:06:03.333 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1681024414}) 2024-09-18T14:06:03.333 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.333 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1133377906}) 2024-09-18T14:06:03.333 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1142669268}) 2024-09-18T14:06:03.333 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.333 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1133377906}) 2024-09-18T14:06:03.333 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2328908980}) 2024-09-18T14:06:03.333 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.333 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1133377906}) 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3293429101}) 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1133377906}) 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1517184719}) 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1133377906}) 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1611597549}) 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1133377906}) 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3410947969}) 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1133377906}) 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.329+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3294063454}) 2024-09-18T14:06:03.334 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.335 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=297439572}) 2024-09-18T14:06:03.335 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2262640779}) 2024-09-18T14:06:03.335 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.335 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=297439572}) 2024-09-18T14:06:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f018e59d640 1 -- 172.21.15.45:0/2974595256 >> v1:172.21.15.84:6812/31478 conn(0x7f017403aa60 legacy=0x7f017403cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:03.335 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=727218298}) 2024-09-18T14:06:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f018e59d640 10 monclient: shutdown 2024-09-18T14:06:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f018e59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f018e59d640 1 -- 172.21.15.45:0/2974595256 >> v1:172.21.15.45:6791/0 conn(0x7f01880a2520 legacy=0x7f018809af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:03.336 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.336 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=297439572}) 2024-09-18T14:06:03.336 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1387067697}) 2024-09-18T14:06:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f018e59d640 1 -- 172.21.15.45:0/2974595256 shutdown_connections 2024-09-18T14:06:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f018e59d640 1 -- 172.21.15.45:0/2974595256 >> 172.21.15.45:0/2974595256 conn(0x7f0188063130 msgr2=0x7f01880a42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:03.338 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.338 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=297439572}) 2024-09-18T14:06:03.338 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1043199657}) 2024-09-18T14:06:03.338 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.338 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=297439572}) 2024-09-18T14:06:03.338 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2779356739}) 2024-09-18T14:06:03.338 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.338 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=297439572}) 2024-09-18T14:06:03.338 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3283347670}) 2024-09-18T14:06:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f018e59d640 1 -- 172.21.15.45:0/2974595256 shutdown_connections 2024-09-18T14:06:03.339 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f018e59d640 1 -- 172.21.15.45:0/2974595256 wait complete. 2024-09-18T14:06:03.350 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=297439572}) 2024-09-18T14:06:03.350 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2877550848}) 2024-09-18T14:06:03.350 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.350 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=297439572}) 2024-09-18T14:06:03.350 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.331+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3345429585}) 2024-09-18T14:06:03.350 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.350 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3177107457}) 2024-09-18T14:06:03.350 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2797466783}) 2024-09-18T14:06:03.350 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3177107457}) 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3634266630}) 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3177107457}) 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1318807534}) 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3177107457}) 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1604305503}) 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3177107457}) 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2512340347}) 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3177107457}) 2024-09-18T14:06:03.351 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2278898984}) 2024-09-18T14:06:03.352 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.352 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3177107457}) 2024-09-18T14:06:03.352 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1079783982}) 2024-09-18T14:06:03.352 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.352 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3177107457}) 2024-09-18T14:06:03.352 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.332+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=404123428}) 2024-09-18T14:06:03.352 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.352 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1840140046}) 2024-09-18T14:06:03.352 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3106967731}) 2024-09-18T14:06:03.352 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.352 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1840140046}) 2024-09-18T14:06:03.352 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1200300133}) 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1840140046}) 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4028615189}) 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1840140046}) 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1265837433}) 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1840140046}) 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=906364098}) 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1840140046}) 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3053326425}) 2024-09-18T14:06:03.353 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1840140046}) 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3177436086}) 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1840140046}) 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.333+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4168993960}) 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3088594668}) 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1290095967}) 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3088594668}) 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3491094223}) 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3088594668}) 2024-09-18T14:06:03.354 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2191794861}) 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3088594668}) 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=22067640}) 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3088594668}) 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2778219703}) 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3088594668}) 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3296516341}) 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3088594668}) 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3260421939}) 2024-09-18T14:06:03.355 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3088594668}) 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.334+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1751946334}) 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2400125684}) 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3114971578}) 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2400125684}) 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2736440896}) 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2400125684}) 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1816689219}) 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.356 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2400125684}) 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3992865481}) 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2400125684}) 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3442922912}) 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2400125684}) 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1562361709}) 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2400125684}) 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2433445889}) 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2400125684}) 2024-09-18T14:06:03.357 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.336+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4132809222}) 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=648506205}) 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=844579635}) 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=648506205}) 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2207210689}) 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=648506205}) 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=306908348}) 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=648506205}) 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3293416026}) 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.358 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=648506205}) 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=818439159}) 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=648506205}) 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2234887521}) 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=648506205}) 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1954239235}) 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=648506205}) 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.337+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3890326523}) 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1859250164}) 2024-09-18T14:06:03.359 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2220066889}) 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1859250164}) 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1778534936}) 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1859250164}) 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4003777415}) 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1859250164}) 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1972669647}) 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1859250164}) 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=977682374}) 2024-09-18T14:06:03.360 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1859250164}) 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=645967489}) 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1859250164}) 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4157356884}) 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1859250164}) 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.338+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=532470858}) 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2201516873}) 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2754262151}) 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.361 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2201516873}) 2024-09-18T14:06:03.362 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=417019885}) 2024-09-18T14:06:03.362 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.362 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2201516873}) 2024-09-18T14:06:03.362 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1002626237}) 2024-09-18T14:06:03.362 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.362 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2201516873}) 2024-09-18T14:06:03.362 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=57544799}) 2024-09-18T14:06:03.362 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.362 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2201516873}) 2024-09-18T14:06:03.362 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3333276335}) 2024-09-18T14:06:03.362 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.362 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2201516873}) 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2643919242}) 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2201516873}) 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2239708395}) 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2201516873}) 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.339+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=777565372}) 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.340+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.340+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3008460097}) 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.340+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2539477408}) 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.340+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3008460097}) 2024-09-18T14:06:03.363 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2809085933}) 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3008460097}) 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2826181301}) 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3008460097}) 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3963094363}) 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3008460097}) 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4194448226}) 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3008460097}) 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1859211531}) 2024-09-18T14:06:03.364 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3008460097}) 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=408386198}) 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3008460097}) 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.341+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2497740666}) 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3408039515}) 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=609691983}) 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3408039515}) 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2344389726}) 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.365 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3408039515}) 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2141742614}) 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3408039515}) 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1229082342}) 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3408039515}) 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1115496406}) 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3408039515}) 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=75399643}) 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3408039515}) 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3878371486}) 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.366 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3408039515}) 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.342+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1159064585}) 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3790722933}) 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4245078589}) 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3790722933}) 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1998380265}) 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3790722933}) 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3473343082}) 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3790722933}) 2024-09-18T14:06:03.367 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2001222814}) 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3790722933}) 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2472306421}) 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3790722933}) 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2192083849}) 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3790722933}) 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1231587417}) 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3790722933}) 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.343+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1114170806}) 2024-09-18T14:06:03.368 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.369 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140511630}) 2024-09-18T14:06:03.369 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3697474308}) 2024-09-18T14:06:03.369 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.369 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140511630}) 2024-09-18T14:06:03.369 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2262965158}) 2024-09-18T14:06:03.369 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.369 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140511630}) 2024-09-18T14:06:03.369 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2089446577}) 2024-09-18T14:06:03.369 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.369 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140511630}) 2024-09-18T14:06:03.369 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2183289102}) 2024-09-18T14:06:03.369 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.370 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140511630}) 2024-09-18T14:06:03.370 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2511617974}) 2024-09-18T14:06:03.370 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.370 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140511630}) 2024-09-18T14:06:03.370 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3892443506}) 2024-09-18T14:06:03.370 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.370 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140511630}) 2024-09-18T14:06:03.370 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3976396750}) 2024-09-18T14:06:03.370 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.370 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140511630}) 2024-09-18T14:06:03.370 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.344+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3612793333}) 2024-09-18T14:06:03.371 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-18T14:06:03.388 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.345+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.388 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.345+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=26596505}) 2024-09-18T14:06:03.388 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.345+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3156167296}) 2024-09-18T14:06:03.388 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.345+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.388 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.345+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=26596505}) 2024-09-18T14:06:03.388 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.345+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=648650858}) 2024-09-18T14:06:03.388 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.345+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.388 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.345+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=26596505}) 2024-09-18T14:06:03.388 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.345+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=485739723}) 2024-09-18T14:06:03.388 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.345+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.388 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.345+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=26596505}) 2024-09-18T14:06:03.388 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1951311949}) 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=26596505}) 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3158472742}) 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=26596505}) 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2654995439}) 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=26596505}) 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2466148125}) 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=26596505}) 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1111518126}) 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.346+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.389 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3795980086}) 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=109769060}) 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3795980086}) 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3507312197}) 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3795980086}) 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=376375041}) 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3795980086}) 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3696068052}) 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3795980086}) 2024-09-18T14:06:03.390 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=879249791}) 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3795980086}) 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3878321264}) 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3795980086}) 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4240371962}) 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3795980086}) 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.347+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=561707481}) 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=901491003}) 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=290624290}) 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.391 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=901491003}) 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2052089779}) 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=901491003}) 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=731171858}) 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=901491003}) 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=324616172}) 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=901491003}) 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=254507642}) 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=901491003}) 2024-09-18T14:06:03.392 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2987906718}) 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=901491003}) 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2378544556}) 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=901491003}) 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.348+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1288788863}) 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=694425528}) 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=754090827}) 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=694425528}) 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3122144809}) 2024-09-18T14:06:03.393 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=694425528}) 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1788597649}) 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=694425528}) 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4097843473}) 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=694425528}) 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=599598970}) 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=694425528}) 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4033854409}) 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=694425528}) 2024-09-18T14:06:03.394 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3587007947}) 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=694425528}) 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.349+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2576479628}) 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3443655856}) 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2557283596}) 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3443655856}) 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=396398340}) 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3443655856}) 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2184826029}) 2024-09-18T14:06:03.395 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3443655856}) 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1697843857}) 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.350+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3443655856}) 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.351+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=971445526}) 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.351+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.351+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3443655856}) 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.351+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3429726108}) 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.351+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.351+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3443655856}) 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.351+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2850825987}) 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.351+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.351+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3443655856}) 2024-09-18T14:06:03.396 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.351+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1877212124}) 2024-09-18T14:06:03.397 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.397 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3734952905}) 2024-09-18T14:06:03.397 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4264458014}) 2024-09-18T14:06:03.397 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.397 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3734952905}) 2024-09-18T14:06:03.397 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1200926314}) 2024-09-18T14:06:03.397 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.397 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3734952905}) 2024-09-18T14:06:03.397 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=390885101}) 2024-09-18T14:06:03.397 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.397 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3734952905}) 2024-09-18T14:06:03.397 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3921039971}) 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3734952905}) 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1916007975}) 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3734952905}) 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1244921264}) 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3734952905}) 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3217466945}) 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3734952905}) 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.352+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3014131556}) 2024-09-18T14:06:03.398 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=28,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=33} crc {logm=3518026301,mdsmap=1531151339,mgr=1359932699,mgr_metadata=270970526,mgrstat=2593073466,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=1814009033}) 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=31,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=28} crc {logm=2855596534,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=4010496354,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=4234286643}) 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=28,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=33} crc {logm=3518026301,mdsmap=1531151339,mgr=1359932699,mgr_metadata=270970526,mgrstat=2593073466,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=1814009033}) 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=30,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=4,osd_pg_creating=1,osd_snap=33} crc {logm=2242987592,mdsmap=3140514009,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3229294801,mon_config_key=1810658514,monmap=455718034,osd_metadata=3522187836,osd_pg_creating=1204018762,osd_snap=433722990}) 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=28,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=33} crc {logm=3518026301,mdsmap=1531151339,mgr=1359932699,mgr_metadata=270970526,mgrstat=2593073466,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=1814009033}) 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=31,mdsmap=3,mgr=5,mgr_command_descs=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=33} crc {logm=2855596534,mdsmap=1531151339,mgr=1947666192,mgr_command_descs=3545737492,mgrstat=1157641345,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=1754490584}) 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=28,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=33} crc {logm=3518026301,mdsmap=1531151339,mgr=1359932699,mgr_metadata=270970526,mgrstat=2593073466,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=1814009033}) 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=31,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=31} crc {logm=2855596534,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3425097834,mon_config_key=1810658514,monmap=455718034,osd_metadata=1798237175,osd_pg_creating=1204018762,osd_snap=718004025}) 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.353+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.399 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.354+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=28,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=33} crc {logm=3518026301,mdsmap=1531151339,mgr=1359932699,mgr_metadata=270970526,mgrstat=2593073466,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=1814009033}) 2024-09-18T14:06:03.400 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.354+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=30,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=30} crc {logm=2242987592,mdsmap=1531151339,mgr=3670438315,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=4010496354,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=2015273769}) 2024-09-18T14:06:03.400 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.354+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.400 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.354+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=28,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=33} crc {logm=3518026301,mdsmap=1531151339,mgr=1359932699,mgr_metadata=270970526,mgrstat=2593073466,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=1814009033}) 2024-09-18T14:06:03.400 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.354+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=29,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=4,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=33} crc {logm=832142905,mdsmap=1531151339,mgr=2831785783,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=4010496354,mon_config_key=1045651642,monmap=455718034,osd_metadata=1798237175,osd_pg_creating=1204018762,osd_snap=514129326}) 2024-09-18T14:06:03.400 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.354+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.400 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.354+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=28,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=33} crc {logm=3518026301,mdsmap=1531151339,mgr=1359932699,mgr_metadata=270970526,mgrstat=2593073466,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=1814009033}) 2024-09-18T14:06:03.400 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.354+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=30,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=31} crc {logm=471633719,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgrstat=1521041526,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=3216490238}) 2024-09-18T14:06:03.400 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.354+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.400 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.354+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=28,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=33} crc {logm=3518026301,mdsmap=1531151339,mgr=1359932699,mgr_metadata=270970526,mgrstat=2593073466,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=1814009033}) 2024-09-18T14:06:03.400 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.354+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=29,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=32} crc {logm=1931307589,mdsmap=3140514009,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2456846116,mon_config_key=1810658514,monmap=2358078507,osd_metadata=2800182873,osd_pg_creating=1204018762,osd_snap=320779023}) 2024-09-18T14:06:03.400 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.400 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=645013441,osdmap=1415212057}) 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=4232039180,osdmap=3178796753}) 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=645013441,osdmap=1415212057}) 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=2223237198,osdmap=2872037777}) 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=645013441,osdmap=1415212057}) 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=20,osdmap=80} crc {osd_snap=2367917638,osdmap=2008140242}) 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=645013441,osdmap=1415212057}) 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=22,osdmap=78} crc {osd_snap=3497038823,osdmap=1316537423}) 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=645013441,osdmap=1415212057}) 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=3632723860,osdmap=1271736677}) 2024-09-18T14:06:03.401 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=645013441,osdmap=1415212057}) 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=2623478292,osdmap=2954381403}) 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=645013441,osdmap=1415212057}) 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=22,osdmap=78} crc {osd_snap=3497038823,osdmap=4282113900}) 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=645013441,osdmap=1415212057}) 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.355+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=2075371857,osdmap=2615407055}) 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2617090064}) 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1207328084}) 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.402 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2617090064}) 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1828543478}) 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2617090064}) 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4142109622}) 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2617090064}) 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4182583012}) 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2617090064}) 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2901172036}) 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2617090064}) 2024-09-18T14:06:03.403 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3081642446}) 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2617090064}) 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1591891688}) 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2617090064}) 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.356+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3783276472}) 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3194480365}) 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2372633191}) 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3194480365}) 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3363054617}) 2024-09-18T14:06:03.404 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3194480365}) 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=519475073}) 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3194480365}) 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4057772546}) 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3194480365}) 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=576082557}) 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3194480365}) 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3831442641}) 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.405 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3194480365}) 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2653829374}) 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3194480365}) 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.358+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=532818757}) 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=20} crc {osdmap=271422581}) 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=20} crc {osdmap=513633946}) 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=20} crc {osdmap=271422581}) 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=20} crc {osdmap=2412585943}) 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=20} crc {osdmap=271422581}) 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=18} crc {osdmap=3547627126}) 2024-09-18T14:06:03.406 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=20} crc {osdmap=271422581}) 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=21} crc {osdmap=3779272312}) 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=20} crc {osdmap=271422581}) 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=21} crc {osdmap=1507253533}) 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=20} crc {osdmap=271422581}) 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=19} crc {osdmap=2351606877}) 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=20} crc {osdmap=271422581}) 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=20} crc {osdmap=4005147063}) 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:06:03.407 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=20} crc {osdmap=271422581}) 2024-09-18T14:06:03.408 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:06:03.359+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=21} crc {osdmap=1507253533}) 2024-09-18T14:06:04.073 INFO:tasks.rados.rados.0.smithi084.stdout:1461: done (2 left) 2024-09-18T14:06:04.073 INFO:tasks.rados.rados.0.smithi084.stdout:1462: done (1 left) 2024-09-18T14:06:04.073 INFO:tasks.rados.rados.0.smithi084.stdout:1463: done (0 left) 2024-09-18T14:06:04.073 INFO:tasks.rados.rados.0.smithi084.stdout:1464: rollback oid 21 current snap is 158 2024-09-18T14:06:04.073 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 21 to 157 2024-09-18T14:06:04.073 INFO:tasks.rados.rados.0.smithi084.stdout:1465: copy_from oid 23 from oid 49 current snap is 158 2024-09-18T14:06:04.073 INFO:tasks.rados.rados.0.smithi084.stdout:1466: write (excl) oid 14 current snap is 158 2024-09-18T14:06:04.073 INFO:tasks.rados.rados.0.smithi084.stdout:1466: seq_num 513 ranges {621991=798388,1936439=639509} 2024-09-18T14:06:04.080 INFO:tasks.rados.rados.0.smithi084.stdout:1466: writing smithi08434663-14 from 621991 to 1420379 tid 1 2024-09-18T14:06:04.086 INFO:tasks.rados.rados.0.smithi084.stdout:1466: writing smithi08434663-14 from 1936439 to 2575948 tid 2 2024-09-18T14:06:04.086 INFO:tasks.rados.rados.0.smithi084.stdout:1467: snap_remove snap 152 2024-09-18T14:06:04.373 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:06:04.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:06:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.473+0000 7f09ced9d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.473+0000 7f09ced9d640 10 monclient: build_initial_monmap 2024-09-18T14:06:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.473+0000 7f09ced9d640 10 monclient: monmap: 2024-09-18T14:06:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:04.474151+0000 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:04.474151+0000 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:04.475 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.473+0000 7f09ced9d640 1 Processor -- start 2024-09-18T14:06:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.473+0000 7f09ced9d640 1 -- start start 2024-09-18T14:06:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09ced9d640 10 monclient: init 2024-09-18T14:06:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09ced9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09ced9d640 10 monclient: _add_conns ranks=[1,8,6,0,2,5,3,7,4] 2024-09-18T14:06:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09ced9d640 10 monclient(hunting): picked mon.noname-f con 0x7f09c806e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09ced9d640 10 monclient(hunting): picked mon.noname-e con 0x7f09c80a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09ced9d640 10 monclient(hunting): picked mon.noname-d con 0x7f09c80a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09ced9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c805ed70 con 0x7f09c806e810 2024-09-18T14:06:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09ced9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c8097620 con 0x7f09c80a1080 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09ced9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c8097340 con 0x7f09c80a0cb0 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09ced9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09ced9d640 10 monclient(hunting): authenticate will time out at 1179.812544s 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09cd33a640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f09c80a1080 0x7f09c813af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:54828/0 (socket says 172.21.15.45:54828) 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09cd33a640 1 -- 172.21.15.45:0/1270636296 learned_addr learned my addr 172.21.15.45:0/1270636296 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c7fff640 1 --1- 172.21.15.45:0/1270636296 >> v1:172.21.15.45:6793/0 conn(0x7f09c80a0cb0 0x7f09c8137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:39798/0 (socket says 172.21.15.45:39798) 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844840276 0 0) 0x7f09c8097620 con 0x7f09c80a1080 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 10 monclient(hunting): my global_id is 5019 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09b00034e0 con 0x7f09c80a1080 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092634239 0 0) 0x7f09c8097340 con 0x7f09c80a0cb0 2024-09-18T14:06:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 10 monclient(hunting): my global_id is 5048 2024-09-18T14:06:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c8097620 con 0x7f09c80a0cb0 2024-09-18T14:06:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264773527 0 0) 0x7f09c805ed70 con 0x7f09c806e810 2024-09-18T14:06:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 10 monclient(hunting): my global_id is 5104 2024-09-18T14:06:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.474+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c8097340 con 0x7f09c806e810 2024-09-18T14:06:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3913089988 0 0) 0x7f09c8097620 con 0x7f09c80a0cb0 2024-09-18T14:06:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 >> v1:172.21.15.45:6792/0 conn(0x7f09c80a1080 legacy=0x7f09c813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 >> v1:172.21.15.84:6789/0 conn(0x7f09c806e810 legacy=0x7f09c80a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 10 monclient: _finish_hunting 0 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 1 monclient: found mon.noname-e 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c80a1660 con 0x7f09c80a0cb0 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 10 monclient: _finish_auth 0 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f09bc001740 con 0x7f09c80a0cb0 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09ced9d640 5 monclient: authenticate success, global_id 5048 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09ced9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 10 monclient: dump: 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.486 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09ced9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f09bc0018a0 con 0x7f09c80a0cb0 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09ced9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 1 -- 172.21.15.45:0/1270636296 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f09bc0047a0 con 0x7f09c80a0cb0 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09ced9d640 4 set_mon_vals no callback set 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09c77fe640 10 monclient: dump: 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09ced9d640 10 monclient: shutdown 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09ced9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09ced9d640 1 -- 172.21.15.45:0/1270636296 >> v1:172.21.15.45:6793/0 conn(0x7f09c80a0cb0 legacy=0x7f09c8137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09ced9d640 1 -- 172.21.15.45:0/1270636296 shutdown_connections 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09ced9d640 1 -- 172.21.15.45:0/1270636296 >> 172.21.15.45:0/1270636296 conn(0x7f09c80a5bb0 msgr2=0x7f09c80a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09ced9d640 1 -- 172.21.15.45:0/1270636296 shutdown_connections 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.475+0000 7f09ced9d640 1 -- 172.21.15.45:0/1270636296 wait complete. 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.476+0000 7f09ced9d640 10 monclient: build_initial_monmap 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.476+0000 7f09ced9d640 10 monclient: monmap: 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.476+0000 7f09ced9d640 1 Processor -- start 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.476+0000 7f09ced9d640 1 -- start start 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.476+0000 7f09ced9d640 10 monclient: init 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09ced9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09ced9d640 10 monclient: _add_conns ranks=[2,6,8,4,5,0,3,7,1] 2024-09-18T14:06:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09ced9d640 10 monclient(hunting): picked mon.noname-b con 0x7f09c806e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09ced9d640 10 monclient(hunting): picked mon.noname-d con 0x7f09c80a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09ced9d640 10 monclient(hunting): picked mon.noname-e con 0x7f09c80a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09ced9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c809fa10 con 0x7f09c806e810 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09ced9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c813da50 con 0x7f09c80a0cb0 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09ced9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c80a2270 con 0x7f09c80a1080 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09ced9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09ced9d640 10 monclient(hunting): authenticate will time out at 1179.815418s 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c7fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f09c80a0cb0 0x7f09c8146550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:54838/0 (socket says 172.21.15.45:54838) 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09ccb39640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f09c806e810 0x7f09c813d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39434/0 (socket says 172.21.15.45:39434) 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c7fff640 1 -- 172.21.15.45:0/640275863 learned_addr learned my addr 172.21.15.45:0/640275863 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09cd33a640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f09c80a1080 0x7f09c8149c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:39806/0 (socket says 172.21.15.45:39806) 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986381204 0 0) 0x7f09c80a2270 con 0x7f09c80a1080 2024-09-18T14:06:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 10 monclient(hunting): my global_id is 5057 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09b40034e0 con 0x7f09c80a1080 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964363791 0 0) 0x7f09c813da50 con 0x7f09c80a0cb0 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 10 monclient(hunting): my global_id is 5028 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c80a2270 con 0x7f09c80a0cb0 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941525896 0 0) 0x7f09c809fa10 con 0x7f09c806e810 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 10 monclient(hunting): my global_id is 144416 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.477+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c813da50 con 0x7f09c806e810 2024-09-18T14:06:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 792002581 0 0) 0x7f09b40034e0 con 0x7f09c80a1080 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09c809fa10 con 0x7f09c80a1080 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f09c0001dc0 con 0x7f09c80a1080 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2470273415 0 0) 0x7f09c80a2270 con 0x7f09c80a0cb0 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09b40034e0 con 0x7f09c80a0cb0 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 431262835 0 0) 0x7f09c813da50 con 0x7f09c806e810 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09c80a2270 con 0x7f09c806e810 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f09bc001600 con 0x7f09c80a0cb0 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f09b8001910 con 0x7f09c806e810 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2233428734 0 0) 0x7f09c809fa10 con 0x7f09c80a1080 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 >> v1:172.21.15.45:6792/0 conn(0x7f09c80a0cb0 legacy=0x7f09c8146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 >> v1:172.21.15.45:6790/0 conn(0x7f09c806e810 legacy=0x7f09c813d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 monclient: found mon.e 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c813e860 con 0x7f09c80a1080 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient: _finish_auth 0 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09ced9d640 5 monclient: authenticate success, global_id 5057 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09ced9d640 10 monclient: _renew_subs 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09ced9d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09ced9d640 1 -- 172.21.15.45:0/640275863 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c81223e0 con 0x7f09c80a1080 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09ced9d640 10 monclient: _renew_subs 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09ced9d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09ced9d640 1 -- 172.21.15.45:0/640275863 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c8120080 con 0x7f09c80a1080 2024-09-18T14:06:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f09c00020d0 con 0x7f09c80a1080 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f09c0005110 con 0x7f09c80a1080 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c4ff9640 10 monclient: dump: 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.478+0000 7f09c6ffd640 4 set_mon_vals no callback set 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.479+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f09c0010160 con 0x7f09c80a1080 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.479+0000 7f09ced9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.479+0000 7f09ced9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.479+0000 7f09ced9d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.479+0000 7f09ced9d640 1 -- 172.21.15.45:0/640275863 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c804c4a0 con 0x7f09c80a1080 2024-09-18T14:06:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.479+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(327..327 src has 126..327) v4 ==== 3895+0+0 (unknown 2911941051 0 0) 0x7f09c0046dc0 con 0x7f09c80a1080 2024-09-18T14:06:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.479+0000 7f09ccb39640 10 monclient: get_auth_request con 0x7f09b403aba0 auth_method 0 2024-09-18T14:06:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.483+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f09c00155b0 con 0x7f09c80a1080 2024-09-18T14:06:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.483+0000 7f09c4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.483+0000 7f09c4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:04.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.709+0000 7f09ced9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:06:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.709+0000 7f09ced9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:06:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.709+0000 7f09ced9d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.709+0000 7f09ced9d640 1 -- 172.21.15.45:0/640275863 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f09c804b720 con 0x7f09c80a1080 2024-09-18T14:06:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.709+0000 7f09c4ff9640 1 -- 172.21.15.45:0/640275863 <== mon.8 v1:172.21.15.45:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 3683022953) 0x7f09c0017a90 con 0x7f09c80a1080 2024-09-18T14:06:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.709+0000 7f09c4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:06:04.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.709+0000 7f09c4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:04.710 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":76,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:06:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.711+0000 7f09ced9d640 1 -- 172.21.15.45:0/640275863 >> v1:172.21.15.84:6812/31478 conn(0x7f09b403aba0 legacy=0x7f09b403d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.711+0000 7f09ced9d640 10 monclient: shutdown 2024-09-18T14:06:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.711+0000 7f09ced9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.711+0000 7f09ced9d640 1 -- 172.21.15.45:0/640275863 >> v1:172.21.15.45:6793/0 conn(0x7f09c80a1080 legacy=0x7f09c8149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.711+0000 7f09ced9d640 1 -- 172.21.15.45:0/640275863 shutdown_connections 2024-09-18T14:06:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.711+0000 7f09ced9d640 1 -- 172.21.15.45:0/640275863 >> 172.21.15.45:0/640275863 conn(0x7f09c80a5bb0 msgr2=0x7f09c80a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.711+0000 7f09ced9d640 1 -- 172.21.15.45:0/640275863 shutdown_connections 2024-09-18T14:06:04.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.711+0000 7f09ced9d640 1 -- 172.21.15.45:0/640275863 wait complete. 2024-09-18T14:06:04.722 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:06:04.722 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-18T14:06:04.722 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:06:04.722 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:06:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.819+0000 7fd2ad59d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.819+0000 7fd2ad59d640 10 monclient: build_initial_monmap 2024-09-18T14:06:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.819+0000 7fd2ad59d640 10 monclient: monmap: 2024-09-18T14:06:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:04.820758+0000 2024-09-18T14:06:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:04.820758+0000 2024-09-18T14:06:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:04.821 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 1 Processor -- start 2024-09-18T14:06:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 1 -- start start 2024-09-18T14:06:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 10 monclient: init 2024-09-18T14:06:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 10 monclient: _add_conns ranks=[1,2,5,0,7,3,6,4,8] 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 10 monclient(hunting): picked mon.noname-f con 0x7fd2a809f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 10 monclient(hunting): picked mon.noname-b con 0x7fd2a809fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 10 monclient(hunting): picked mon.noname-h con 0x7fd2a809ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a805ed70 con 0x7fd2a809f7d0 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a809dfc0 con 0x7fd2a809fba0 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a809dce0 con 0x7fd2a809ff70 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.820+0000 7fd2ad59d640 10 monclient(hunting): authenticate will time out at 1180.158959s 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a67fc640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fd2a809fba0 0x7fd2a8137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39450/0 (socket says 172.21.15.45:39450) 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a67fc640 1 -- 172.21.15.45:0/1172613503 learned_addr learned my addr 172.21.15.45:0/1172613503 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608626539 0 0) 0x7fd2a809dfc0 con 0x7fd2a809fba0 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient(hunting): my global_id is 144425 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2940034e0 con 0x7fd2a809fba0 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26356244 0 0) 0x7fd2a809dce0 con 0x7fd2a809ff70 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient(hunting): my global_id is 4991 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2a809dfc0 con 0x7fd2a809ff70 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994509435 0 0) 0x7fd2a805ed70 con 0x7fd2a809f7d0 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient(hunting): my global_id is 5113 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2a809dce0 con 0x7fd2a809f7d0 2024-09-18T14:06:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1384279088 0 0) 0x7fd2940034e0 con 0x7fd2a809fba0 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 >> v1:172.21.15.84:6791/0 conn(0x7fd2a809ff70 legacy=0x7fd2a813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 >> v1:172.21.15.84:6789/0 conn(0x7fd2a809f7d0 legacy=0x7fd2a8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 monclient: found mon.noname-b 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a80a1c00 con 0x7fd2a809fba0 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient: _finish_auth 0 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd290001830 con 0x7fd2a809fba0 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient: dump: 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2ad59d640 5 monclient: authenticate success, global_id 144425 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2ad59d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd290001a10 con 0x7fd2a809fba0 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 1 -- 172.21.15.45:0/1172613503 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd290004ba0 con 0x7fd2a809fba0 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2ad59d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2ad59d640 4 set_mon_vals no callback set 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.821+0000 7fd2a5ffb640 10 monclient: dump: 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.836 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.822+0000 7fd2ad59d640 10 monclient: shutdown 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.822+0000 7fd2ad59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.822+0000 7fd2ad59d640 1 -- 172.21.15.45:0/1172613503 >> v1:172.21.15.45:6790/0 conn(0x7fd2a809fba0 legacy=0x7fd2a8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.822+0000 7fd2ad59d640 1 -- 172.21.15.45:0/1172613503 shutdown_connections 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.822+0000 7fd2ad59d640 1 -- 172.21.15.45:0/1172613503 >> 172.21.15.45:0/1172613503 conn(0x7fd2a80630b0 msgr2=0x7fd2a80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.822+0000 7fd2ad59d640 1 -- 172.21.15.45:0/1172613503 shutdown_connections 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.822+0000 7fd2ad59d640 1 -- 172.21.15.45:0/1172613503 wait complete. 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.822+0000 7fd2ad59d640 10 monclient: build_initial_monmap 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 10 monclient: monmap: 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 1 Processor -- start 2024-09-18T14:06:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 1 -- start start 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 10 monclient: init 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 10 monclient: _add_conns ranks=[5,7,2,0,8,4,3,6,1] 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 10 monclient(hunting): picked mon.noname-h con 0x7fd2a809f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 10 monclient(hunting): picked mon.noname-i con 0x7fd2a809fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 10 monclient(hunting): picked mon.noname-b con 0x7fd2a809ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a809d310 con 0x7fd2a809ff70 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a80a2560 con 0x7fd2a809f7d0 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a811eba0 con 0x7fd2a809fba0 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.823+0000 7fd2ad59d640 10 monclient(hunting): authenticate will time out at 1180.161975s 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd2a77fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fd2a809ff70 0x7fd2a814a470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39466/0 (socket says 172.21.15.45:39466) 2024-09-18T14:06:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd2a77fe640 1 -- 172.21.15.45:0/501117293 learned_addr learned my addr 172.21.15.45:0/501117293 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219945696 0 0) 0x7fd2a809d310 con 0x7fd2a809ff70 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): my global_id is 144434 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2880034e0 con 0x7fd2a809ff70 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956120280 0 0) 0x7fd2a80a2560 con 0x7fd2a809f7d0 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): my global_id is 5000 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2a809d310 con 0x7fd2a809f7d0 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426685349 0 0) 0x7fd2a811eba0 con 0x7fd2a809fba0 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): my global_id is 5110 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2a80a2560 con 0x7fd2a809fba0 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1359310850 0 0) 0x7fd2880034e0 con 0x7fd2a809ff70 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2a811eba0 con 0x7fd2a809ff70 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd298002af0 con 0x7fd2a809ff70 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): dump: 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 5237883 0 0) 0x7fd2a809d310 con 0x7fd2a809f7d0 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2880034e0 con 0x7fd2a809f7d0 2024-09-18T14:06:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd2a0001de0 con 0x7fd2a809f7d0 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient(hunting): dump: 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2849723999 0 0) 0x7fd2a80a2560 con 0x7fd2a809fba0 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2a809d310 con 0x7fd2a809fba0 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 37042650 0 0) 0x7fd2a811eba0 con 0x7fd2a809ff70 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 >> v1:172.21.15.84:6792/0 conn(0x7fd2a809fba0 legacy=0x7fd2a8146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 >> v1:172.21.15.84:6791/0 conn(0x7fd2a809f7d0 legacy=0x7fd2a8131780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient: _finish_hunting 0 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 monclient: found mon.b 2024-09-18T14:06:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a811d960 con 0x7fd2a809ff70 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient: _finish_auth 0 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd2ad59d640 5 monclient: authenticate success, global_id 144434 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd2ad59d640 10 monclient: _renew_subs 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd2ad59d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd2ad59d640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a811f160 con 0x7fd2a809ff70 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd2ad59d640 10 monclient: _renew_subs 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd2ad59d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd2ad59d640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a809bbc0 con 0x7fd2a809ff70 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd298001580 con 0x7fd2a809ff70 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd298001af0 con 0x7fd2a809ff70 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd2a4ff9640 4 set_mon_vals no callback set 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.824+0000 7fd29effd640 10 monclient: dump: 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.825+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd298001cd0 con 0x7fd2a809ff70 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.825+0000 7fd2ad59d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.825+0000 7fd2ad59d640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd2a804c4a0 con 0x7fd2a809b100 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.825+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(327..327 src has 126..327) v4 ==== 3895+0+0 (unknown 2911941051 0 0) 0x7fd298046a00 con 0x7fd2a809ff70 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682692834 0 0) 0x7fd2a804c4a0 con 0x7fd2a809b100 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 10 monclient(hunting): my global_id is 5157 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2a811eba0 con 0x7fd2a809b100 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd2a67fc640 10 monclient: get_auth_request con 0x7fd28803a850 auth_method 0 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4171621351 0 0) 0x7fd2a811eba0 con 0x7fd2a809b100 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2a804c4a0 con 0x7fd2a809b100 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd2a0002530 con 0x7fd2a809b100 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 10 monclient: dump: 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.848 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3010792119 0 0) 0x7fd2a804c4a0 con 0x7fd2a809b100 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd2a804b720 con 0x7fd2a809b100 2024-09-18T14:06:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd2a0004810 con 0x7fd2a809b100 2024-09-18T14:06:04.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:04.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:04.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 >> v1:172.21.15.45:6789/0 conn(0x7fd2a809b100 legacy=0x7fd2a8096a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.826+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 >> v1:172.21.15.45:6789/0 conn(0x7fd2a809b100 legacy=0x7fd2a8096a20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:04.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd2ad59d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:04.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd2ad59d640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd2a811ede0 con 0x7fd2a802a8d0 2024-09-18T14:06:04.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997937302 0 0) 0x7fd2880034e0 con 0x7fd2a802a8d0 2024-09-18T14:06:04.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd29effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd29effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd29effd640 10 monclient(hunting): my global_id is 5166 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2a804c4a0 con 0x7fd2a802a8d0 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3107553209 0 0) 0x7fd2a804c4a0 con 0x7fd2a802a8d0 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2880034e0 con 0x7fd2a802a8d0 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd2a0003030 con 0x7fd2a802a8d0 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd29effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd29effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.832+0000 7fd29effd640 10 monclient: dump: 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3565307105 0 0) 0x7fd2880034e0 con 0x7fd2a802a8d0 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd2a8071550 con 0x7fd2a802a8d0 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 2740408965) 0x7fd2a0004a40 con 0x7fd2a802a8d0 2024-09-18T14:06:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd29effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:04.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd29effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:04.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 >> v1:172.21.15.45:6789/0 conn(0x7fd2a802a8d0 legacy=0x7fd2a8097130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd29effd640 1 -- 172.21.15.45:0/501117293 >> v1:172.21.15.45:6789/0 conn(0x7fd2a802a8d0 legacy=0x7fd2a8097130 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:04.854 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:04.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:04.863 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:04.864 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:04.867 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:04.868 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:04.871 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:04.872 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:04.873 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:04.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:04.873 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:04.873 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:04.873 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:06:04.873 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:04.873 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:04.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:04.873 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:04.873 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:04.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd2ad59d640 1 -- 172.21.15.45:0/501117293 >> v1:172.21.15.84:6812/31478 conn(0x7fd28803a850 legacy=0x7fd28803cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd2ad59d640 10 monclient: shutdown 2024-09-18T14:06:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd2ad59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd2ad59d640 1 -- 172.21.15.45:0/501117293 >> v1:172.21.15.45:6790/0 conn(0x7fd2a809ff70 legacy=0x7fd2a814a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd2a77fe640 1 -- 172.21.15.45:0/501117293 reap_dead start 2024-09-18T14:06:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd2a77fe640 1 -- 172.21.15.45:0/501117293 reap_dead start 2024-09-18T14:06:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd2ad59d640 1 -- 172.21.15.45:0/501117293 shutdown_connections 2024-09-18T14:06:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd2ad59d640 1 -- 172.21.15.45:0/501117293 >> 172.21.15.45:0/501117293 conn(0x7fd2a80630b0 msgr2=0x7fd2a80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd2ad59d640 1 -- 172.21.15.45:0/501117293 shutdown_connections 2024-09-18T14:06:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.833+0000 7fd2ad59d640 1 -- 172.21.15.45:0/501117293 wait complete. 2024-09-18T14:06:04.875 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:04.876 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:06:04.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:06:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.972+0000 7fc69d6dd640 10 monclient: get_monmap_and_config 2024-09-18T14:06:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.972+0000 7fc69d6dd640 10 monclient: build_initial_monmap 2024-09-18T14:06:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.972+0000 7fc69d6dd640 10 monclient: monmap: 2024-09-18T14:06:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:04.973654+0000 2024-09-18T14:06:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:04.973654+0000 2024-09-18T14:06:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:04.974 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.973+0000 7fc69d6dd640 1 Processor -- start 2024-09-18T14:06:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.973+0000 7fc69d6dd640 1 -- start start 2024-09-18T14:06:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.973+0000 7fc69d6dd640 10 monclient: init 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.973+0000 7fc69d6dd640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.973+0000 7fc69d6dd640 10 monclient: _add_conns ranks=[8,0,1,7,4,5,6,2,3] 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.973+0000 7fc69d6dd640 10 monclient(hunting): picked mon.noname-e con 0x7fc69809f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.973+0000 7fc69d6dd640 10 monclient(hunting): picked mon.noname-a con 0x7fc69809fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc69d6dd640 10 monclient(hunting): picked mon.noname-f con 0x7fc69809ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc69d6dd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69805ed70 con 0x7fc69809fba0 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc69d6dd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69809dfc0 con 0x7fc69809ff70 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc69d6dd640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69809dce0 con 0x7fc69809f7d0 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc69d6dd640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc69d6dd640 10 monclient(hunting): authenticate will time out at 1180.312117s 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc68ffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc69809fba0 0x7fc698137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33654/0 (socket says 172.21.15.45:33654) 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc68ffff640 1 -- 172.21.15.45:0/958075817 learned_addr learned my addr 172.21.15.45:0/958075817 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc696ffd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fc69809f7d0 0x7fc698132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:39814/0 (socket says 172.21.15.45:39814) 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793347952 0 0) 0x7fc69809dce0 con 0x7fc69809f7d0 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient(hunting): my global_id is 5066 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6840034e0 con 0x7fc69809f7d0 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311167978 0 0) 0x7fc69805ed70 con 0x7fc69809fba0 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient(hunting): my global_id is 5175 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc69809dce0 con 0x7fc69809fba0 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723262548 0 0) 0x7fc69809dfc0 con 0x7fc69809ff70 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient(hunting): my global_id is 5122 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc69805ed70 con 0x7fc69809ff70 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3238569354 0 0) 0x7fc69809dce0 con 0x7fc69809fba0 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 >> v1:172.21.15.45:6793/0 conn(0x7fc69809f7d0 legacy=0x7fc698132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 >> v1:172.21.15.84:6789/0 conn(0x7fc69809ff70 legacy=0x7fc69813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient: _finish_hunting 0 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 monclient: found mon.noname-a 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6980a1c00 con 0x7fc69809fba0 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient: _finish_auth 0 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc680001740 con 0x7fc69809fba0 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc69d6dd640 5 monclient: authenticate success, global_id 5175 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc69d6dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient: dump: 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc69d6dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc680001920 con 0x7fc69809fba0 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 1 -- 172.21.15.45:0/958075817 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc680004860 con 0x7fc69809fba0 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc69d6dd640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc69d6dd640 4 set_mon_vals no callback set 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.974+0000 7fc6967fc640 10 monclient: dump: 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.975+0000 7fc69d6dd640 10 monclient: shutdown 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.975+0000 7fc69d6dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.975+0000 7fc69d6dd640 1 -- 172.21.15.45:0/958075817 >> v1:172.21.15.45:6789/0 conn(0x7fc69809fba0 legacy=0x7fc698137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.975+0000 7fc69d6dd640 1 -- 172.21.15.45:0/958075817 shutdown_connections 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.975+0000 7fc69d6dd640 1 -- 172.21.15.45:0/958075817 >> 172.21.15.45:0/958075817 conn(0x7fc6980630b0 msgr2=0x7fc6980634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.975+0000 7fc69d6dd640 1 -- 172.21.15.45:0/958075817 shutdown_connections 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.975+0000 7fc69d6dd640 1 -- 172.21.15.45:0/958075817 wait complete. 2024-09-18T14:06:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.975+0000 7fc69d6dd640 10 monclient: build_initial_monmap 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.975+0000 7fc69d6dd640 10 monclient: monmap: 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 1 Processor -- start 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 1 -- start start 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 10 monclient: init 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 10 monclient: _add_conns ranks=[8,6,7,3,2,5,1,4,0] 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 10 monclient(hunting): picked mon.noname-e con 0x7fc69809f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 10 monclient(hunting): picked mon.noname-d con 0x7fc69809fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 10 monclient(hunting): picked mon.noname-i con 0x7fc69809ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69809d310 con 0x7fc69809fba0 2024-09-18T14:06:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6980a2560 con 0x7fc69809ff70 2024-09-18T14:06:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc698121b70 con 0x7fc69809f7d0 2024-09-18T14:06:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc69d6dd640 10 monclient(hunting): authenticate will time out at 1180.314735s 2024-09-18T14:06:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc696ffd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fc69809f7d0 0x7fc698131740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:39818/0 (socket says 172.21.15.45:39818) 2024-09-18T14:06:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc68ffff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fc69809fba0 0x7fc698146b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:54854/0 (socket says 172.21.15.45:54854) 2024-09-18T14:06:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc696ffd640 1 -- 172.21.15.45:0/1890524458 learned_addr learned my addr 172.21.15.45:0/1890524458 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.976+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438426536 0 0) 0x7fc698121b70 con 0x7fc69809f7d0 2024-09-18T14:06:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): my global_id is 5075 2024-09-18T14:06:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6780034e0 con 0x7fc69809f7d0 2024-09-18T14:06:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175097602 0 0) 0x7fc69809d310 con 0x7fc69809fba0 2024-09-18T14:06:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): my global_id is 5037 2024-09-18T14:06:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc698121b70 con 0x7fc69809fba0 2024-09-18T14:06:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1680925858 0 0) 0x7fc6780034e0 con 0x7fc69809f7d0 2024-09-18T14:06:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc69809d310 con 0x7fc69809f7d0 2024-09-18T14:06:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198169000 0 0) 0x7fc6980a2560 con 0x7fc69809ff70 2024-09-18T14:06:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): my global_id is 5119 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6780034e0 con 0x7fc69809ff70 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc6880015b0 con 0x7fc69809f7d0 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): dump: 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3013441691 0 0) 0x7fc698121b70 con 0x7fc69809fba0 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc6980a2560 con 0x7fc69809fba0 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc680001680 con 0x7fc69809fba0 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient(hunting): dump: 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3505311989 0 0) 0x7fc69809d310 con 0x7fc69809f7d0 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 >> v1:172.21.15.84:6792/0 conn(0x7fc69809ff70 legacy=0x7fc69814a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 >> v1:172.21.15.45:6792/0 conn(0x7fc69809fba0 legacy=0x7fc698146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient: _finish_hunting 0 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 monclient: found mon.e 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69811fd60 con 0x7fc69809f7d0 2024-09-18T14:06:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient: _finish_auth 0 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc69d6dd640 5 monclient: authenticate success, global_id 5075 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc69d6dd640 10 monclient: _renew_subs 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc69d6dd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc69d6dd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc698121890 con 0x7fc69809f7d0 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc69d6dd640 10 monclient: _renew_subs 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc69d6dd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc69d6dd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc698131e50 con 0x7fc69809f7d0 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc688001830 con 0x7fc69809f7d0 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc688004d90 con 0x7fc69809f7d0 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc695ffb640 4 set_mon_vals no callback set 2024-09-18T14:06:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:05.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.977+0000 7fc68effd640 10 monclient: dump: 2024-09-18T14:06:05.000 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.000 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.000 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.000 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.000 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.000 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.000 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.000 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.000 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.001 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.001 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.001 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.001 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.001 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.001 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.001 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.978+0000 7fc69d6dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.978+0000 7fc69d6dd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc67c006580 con 0x7fc67c002830 2024-09-18T14:06:05.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.978+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc68800fd60 con 0x7fc69809f7d0 2024-09-18T14:06:05.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.978+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(327..327 src has 126..327) v4 ==== 3895+0+0 (unknown 2911941051 0 0) 0x7fc688046a20 con 0x7fc69809f7d0 2024-09-18T14:06:05.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.978+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369894133 0 0) 0x7fc67c006580 con 0x7fc67c002830 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.978+0000 7fc68effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.978+0000 7fc68effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.978+0000 7fc68effd640 10 monclient(hunting): my global_id is 144443 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.978+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc69809d310 con 0x7fc67c002830 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.978+0000 7fc6977fe640 10 monclient: get_auth_request con 0x7fc67803a650 auth_method 0 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2495294218 0 0) 0x7fc69809d310 con 0x7fc67c002830 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc67c006580 con 0x7fc67c002830 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc680004e20 con 0x7fc67c002830 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 10 monclient: dump: 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2960418938 0 0) 0x7fc67c006580 con 0x7fc67c002830 2024-09-18T14:06:05.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc67c00a0a0 con 0x7fc67c002830 2024-09-18T14:06:05.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc6800051b0 con 0x7fc67c002830 2024-09-18T14:06:05.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:05.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 >> v1:172.21.15.45:6790/0 conn(0x7fc67c002830 legacy=0x7fc67c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.979+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 >> v1:172.21.15.45:6790/0 conn(0x7fc67c002830 legacy=0x7fc67c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:05.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc69d6dd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:05.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc69d6dd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc67c009660 con 0x7fc67c009040 2024-09-18T14:06:05.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3905745757 0 0) 0x7fc67c009660 con 0x7fc67c009040 2024-09-18T14:06:05.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 10 monclient(hunting): my global_id is 144452 2024-09-18T14:06:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc67c006580 con 0x7fc67c009040 2024-09-18T14:06:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2556713910 0 0) 0x7fc67c006580 con 0x7fc67c009040 2024-09-18T14:06:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc67c009660 con 0x7fc67c009040 2024-09-18T14:06:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc680005a90 con 0x7fc67c009040 2024-09-18T14:06:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 10 monclient: dump: 2024-09-18T14:06:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.009 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3190586141 0 0) 0x7fc67c009660 con 0x7fc67c009040 2024-09-18T14:06:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.985+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc67c009920 con 0x7fc67c009040 2024-09-18T14:06:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 283399384) 0x7fc6800081a0 con 0x7fc67c009040 2024-09-18T14:06:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc68effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc68effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 >> v1:172.21.15.45:6790/0 conn(0x7fc67c009040 legacy=0x7fc67c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc68effd640 1 -- 172.21.15.45:0/1890524458 >> v1:172.21.15.45:6790/0 conn(0x7fc67c009040 legacy=0x7fc67c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:05.010 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:06:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:05.015 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:05.016 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:05.016 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.016 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.016 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.016 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.016 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.016 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.016 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.016 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.016 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.016 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.016 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:05.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.017 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:05.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.017 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:05.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:05.017 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.017 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.018 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:05.018 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.018 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.018 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.018 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.019 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:05.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:05.021 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.021 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.021 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:05.021 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.021 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.021 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.022 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:05.022 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:05.022 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.022 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.022 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.022 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:05.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:05.022 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.022 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.023 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:05.023 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.023 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.023 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:05.024 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:05.024 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.024 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.024 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.024 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:05.024 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:05.024 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.024 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.027 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:05.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:05.029 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.029 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.029 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:05.029 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.029 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.029 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.029 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:05.029 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:05.030 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.030 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.030 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.030 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:05.030 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:05.030 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.030 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.031 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:05.031 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.031 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.031 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.031 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:05.032 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:05.032 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.032 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.032 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.032 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.032 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.032 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:05.032 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:05.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.033 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.033 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.033 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.033 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.033 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.034 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:05.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.034 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.034 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.034 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.034 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.035 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.035 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:05.035 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:05.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc69d6dd640 1 -- 172.21.15.45:0/1890524458 >> v1:172.21.15.84:6812/31478 conn(0x7fc67803a650 legacy=0x7fc67803cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc69d6dd640 10 monclient: shutdown 2024-09-18T14:06:05.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc69d6dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:05.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc69d6dd640 1 -- 172.21.15.45:0/1890524458 >> v1:172.21.15.45:6793/0 conn(0x7fc69809f7d0 legacy=0x7fc698131740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc6977fe640 1 -- 172.21.15.45:0/1890524458 reap_dead start 2024-09-18T14:06:05.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc6977fe640 1 -- 172.21.15.45:0/1890524458 reap_dead start 2024-09-18T14:06:05.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc69d6dd640 1 -- 172.21.15.45:0/1890524458 shutdown_connections 2024-09-18T14:06:05.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc69d6dd640 1 -- 172.21.15.45:0/1890524458 >> 172.21.15.45:0/1890524458 conn(0x7fc6980630b0 msgr2=0x7fc6980a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:05.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc69d6dd640 1 -- 172.21.15.45:0/1890524458 shutdown_connections 2024-09-18T14:06:05.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:04.986+0000 7fc69d6dd640 1 -- 172.21.15.45:0/1890524458 wait complete. 2024-09-18T14:06:05.038 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:05.038 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:06:05.038 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:06:05.063 INFO:tasks.rados.rados.0.smithi084.stdout:1467: done (3 left) 2024-09-18T14:06:05.063 INFO:tasks.rados.rados.0.smithi084.stdout:1468: copy_from oid 11 from oid 40 current snap is 158 2024-09-18T14:06:05.063 INFO:tasks.rados.rados.0.smithi084.stdout:1465: finishing copy_from to smithi08434663-23 2024-09-18T14:06:05.063 INFO:tasks.rados.rados.0.smithi084.stdout:1465: got expected ENOENT (src dne) 2024-09-18T14:06:05.063 INFO:tasks.rados.rados.0.smithi084.stdout:1465: finishing copy_from racing read to smithi08434663-23 2024-09-18T14:06:05.063 INFO:tasks.rados.rados.0.smithi084.stdout:1464: finishing rollback tid 1 to smithi08434663-21 2024-09-18T14:06:05.063 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-18T14:06:05.063 INFO:tasks.rados.rados.0.smithi084.stdout:1466: finishing write tid 1 to smithi08434663-14 2024-09-18T14:06:05.063 INFO:tasks.rados.rados.0.smithi084.stdout:1466: finishing write tid 2 to smithi08434663-14 2024-09-18T14:06:05.064 INFO:tasks.rados.rados.0.smithi084.stdout:1466: finishing write tid 3 to smithi08434663-14 2024-09-18T14:06:05.064 INFO:tasks.rados.rados.0.smithi084.stdout:1466: finishing write tid 4 to smithi08434663-14 2024-09-18T14:06:05.064 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 769 (ObjNum 513 snap 158 seq_num 513) dirty exists 2024-09-18T14:06:05.064 INFO:tasks.rados.rados.0.smithi084.stdout:1466: left oid 14 (ObjNum 513 snap 158 seq_num 513) 2024-09-18T14:06:05.064 INFO:tasks.rados.rados.0.smithi084.stdout:1464: done (3 left) 2024-09-18T14:06:05.064 INFO:tasks.rados.rados.0.smithi084.stdout:1465: done (2 left) 2024-09-18T14:06:05.064 INFO:tasks.rados.rados.0.smithi084.stdout:1466: done (1 left) 2024-09-18T14:06:05.064 INFO:tasks.rados.rados.0.smithi084.stdout:1469: read oid 15 snap -1 2024-09-18T14:06:05.064 INFO:tasks.rados.rados.0.smithi084.stdout:1469: expect deleted 2024-09-18T14:06:05.064 INFO:tasks.rados.rados.0.smithi084.stdout:1470: delete oid 8 current snap is 158 2024-09-18T14:06:05.074 INFO:tasks.rados.rados.0.smithi084.stdout:1469: done (2 left) 2024-09-18T14:06:05.074 INFO:tasks.rados.rados.0.smithi084.stdout:1470: done (1 left) 2024-09-18T14:06:05.074 INFO:tasks.rados.rados.0.smithi084.stdout:1471: write (excl) oid 24 current snap is 158 2024-09-18T14:06:05.074 INFO:tasks.rados.rados.0.smithi084.stdout:1471: seq_num 515 ranges {655605=548886,1653403=643178} 2024-09-18T14:06:05.081 INFO:tasks.rados.rados.0.smithi084.stdout:1471: writing smithi08434663-24 from 655605 to 1204491 tid 1 2024-09-18T14:06:05.088 INFO:tasks.rados.rados.0.smithi084.stdout:1471: writing smithi08434663-24 from 1653403 to 2296581 tid 2 2024-09-18T14:06:05.088 INFO:tasks.rados.rados.0.smithi084.stdout:1472: delete oid 39 current snap is 158 2024-09-18T14:06:05.088 INFO:tasks.rados.rados.0.smithi084.stdout:1471: finishing write tid 1 to smithi08434663-24 2024-09-18T14:06:05.088 INFO:tasks.rados.rados.0.smithi084.stdout:1468: finishing copy_from racing read to smithi08434663-11 2024-09-18T14:06:05.091 INFO:tasks.rados.rados.0.smithi084.stdout:1471: finishing write tid 2 to smithi08434663-24 2024-09-18T14:06:05.091 INFO:tasks.rados.rados.0.smithi084.stdout:1471: finishing write tid 3 to smithi08434663-24 2024-09-18T14:06:05.092 INFO:tasks.rados.rados.0.smithi084.stdout:1471: finishing write tid 4 to smithi08434663-24 2024-09-18T14:06:05.092 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 371 (ObjNum 515 snap 158 seq_num 515) dirty exists 2024-09-18T14:06:05.092 INFO:tasks.rados.rados.0.smithi084.stdout:1471: left oid 24 (ObjNum 515 snap 158 seq_num 515) 2024-09-18T14:06:05.092 INFO:tasks.rados.rados.0.smithi084.stdout:1468: finishing copy_from to smithi08434663-11 2024-09-18T14:06:05.092 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 567 (ObjNum 354 snap 109 seq_num 354) dirty exists 2024-09-18T14:06:05.093 INFO:tasks.rados.rados.0.smithi084.stdout:1468: done (2 left) 2024-09-18T14:06:05.093 INFO:tasks.rados.rados.0.smithi084.stdout:1471: done (1 left) 2024-09-18T14:06:05.093 INFO:tasks.rados.rados.0.smithi084.stdout:1472: done (0 left) 2024-09-18T14:06:05.093 INFO:tasks.rados.rados.0.smithi084.stdout:1473: copy_from oid 21 from oid 45 current snap is 158 2024-09-18T14:06:05.093 INFO:tasks.rados.rados.0.smithi084.stdout:1474: delete oid 19 current snap is 158 2024-09-18T14:06:05.095 INFO:tasks.rados.rados.0.smithi084.stdout:1474: done (1 left) 2024-09-18T14:06:05.095 INFO:tasks.rados.rados.0.smithi084.stdout:1475: rollback oid 43 current snap is 158 2024-09-18T14:06:05.095 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 142 2024-09-18T14:06:05.095 INFO:tasks.rados.rados.0.smithi084.stdout:1476: snap_create 2024-09-18T14:06:05.099 INFO:tasks.rados.rados.0.smithi084.stdout:1475: finishing rollback tid 0 to smithi08434663-43 2024-09-18T14:06:05.099 INFO:tasks.rados.rados.0.smithi084.stdout:1475: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:06:05.099 INFO:tasks.rados.rados.0.smithi084.stdout:1475: finishing rollback tid 2 to smithi08434663-43 2024-09-18T14:06:05.099 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 406 (ObjNum 265 snap 67 seq_num 265) dirty exists 2024-09-18T14:06:05.102 INFO:tasks.rados.rados.0.smithi084.stdout:1473: finishing copy_from racing read to smithi08434663-21 2024-09-18T14:06:05.105 INFO:tasks.rados.rados.0.smithi084.stdout:1473: finishing copy_from to smithi08434663-21 2024-09-18T14:06:05.105 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 555 (ObjNum 467 snap 145 seq_num 467) dirty exists 2024-09-18T14:06:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.153+0000 7f06bee9f640 10 monclient: get_monmap_and_config 2024-09-18T14:06:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.153+0000 7f06bee9f640 10 monclient: build_initial_monmap 2024-09-18T14:06:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.153+0000 7f06bee9f640 10 monclient: monmap: 2024-09-18T14:06:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:05.154263+0000 2024-09-18T14:06:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:05.154263+0000 2024-09-18T14:06:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:05.155 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:05.155 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:05.155 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.153+0000 7f06bee9f640 1 Processor -- start 2024-09-18T14:06:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bee9f640 1 -- start start 2024-09-18T14:06:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bee9f640 10 monclient: init 2024-09-18T14:06:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bee9f640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bee9f640 10 monclient: _add_conns ranks=[0,8,7,3,5,6,2,1,4] 2024-09-18T14:06:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bee9f640 10 monclient(hunting): picked mon.noname-a con 0x7f06b809f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bee9f640 10 monclient(hunting): picked mon.noname-e con 0x7f06b809fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bee9f640 10 monclient(hunting): picked mon.noname-i con 0x7f06b809ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bee9f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b805ed70 con 0x7f06b809f7d0 2024-09-18T14:06:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bee9f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b809dfc0 con 0x7f06b809ff70 2024-09-18T14:06:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bee9f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b809dce0 con 0x7f06b809fba0 2024-09-18T14:06:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bee9f640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bee9f640 10 monclient(hunting): authenticate will time out at 1180.492551s 2024-09-18T14:06:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06b7fff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f06b809fba0 0x7f06b8137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:39822/0 (socket says 172.21.15.45:39822) 2024-09-18T14:06:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06bcc3b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f06b809f7d0 0x7f06b8132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33658/0 (socket says 172.21.15.45:33658) 2024-09-18T14:06:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06b7fff640 1 -- 172.21.15.45:0/3950589239 learned_addr learned my addr 172.21.15.45:0/3950589239 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919480523 0 0) 0x7f06b809dce0 con 0x7f06b809fba0 2024-09-18T14:06:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.154+0000 7f06b77fe640 10 monclient(hunting): my global_id is 5084 2024-09-18T14:06:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06a80034e0 con 0x7f06b809fba0 2024-09-18T14:06:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921178244 0 0) 0x7f06b805ed70 con 0x7f06b809f7d0 2024-09-18T14:06:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient(hunting): my global_id is 5184 2024-09-18T14:06:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06b809dce0 con 0x7f06b809f7d0 2024-09-18T14:06:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329782242 0 0) 0x7f06b809dfc0 con 0x7f06b809ff70 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient(hunting): my global_id is 5128 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06b805ed70 con 0x7f06b809ff70 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 692970809 0 0) 0x7f06a80034e0 con 0x7f06b809fba0 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 >> v1:172.21.15.84:6792/0 conn(0x7f06b809ff70 legacy=0x7f06b813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 >> v1:172.21.15.45:6789/0 conn(0x7f06b809f7d0 legacy=0x7f06b8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient: _finish_hunting 0 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 monclient: found mon.noname-e 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b80a1c00 con 0x7f06b809fba0 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient: _finish_auth 0 2024-09-18T14:06:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06bee9f640 5 monclient: authenticate success, global_id 5084 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f06a4001830 con 0x7f06b809fba0 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06bee9f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient: dump: 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06bee9f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f06a4001990 con 0x7f06b809fba0 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06bee9f640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 1 -- 172.21.15.45:0/3950589239 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f06a4004ba0 con 0x7f06b809fba0 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06bee9f640 4 set_mon_vals no callback set 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06b77fe640 10 monclient: dump: 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06bee9f640 10 monclient: shutdown 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06bee9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06bee9f640 1 -- 172.21.15.45:0/3950589239 >> v1:172.21.15.45:6793/0 conn(0x7f06b809fba0 legacy=0x7f06b8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06bee9f640 1 -- 172.21.15.45:0/3950589239 shutdown_connections 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.155+0000 7f06bee9f640 1 -- 172.21.15.45:0/3950589239 >> 172.21.15.45:0/3950589239 conn(0x7f06b80630b0 msgr2=0x7f06b80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.156+0000 7f06bee9f640 1 -- 172.21.15.45:0/3950589239 shutdown_connections 2024-09-18T14:06:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.156+0000 7f06bee9f640 1 -- 172.21.15.45:0/3950589239 wait complete. 2024-09-18T14:06:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.156+0000 7f06bee9f640 10 monclient: build_initial_monmap 2024-09-18T14:06:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.156+0000 7f06bee9f640 10 monclient: monmap: 2024-09-18T14:06:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.156+0000 7f06bee9f640 1 Processor -- start 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.156+0000 7f06bee9f640 1 -- start start 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.156+0000 7f06bee9f640 10 monclient: init 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06bee9f640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06bee9f640 10 monclient: _add_conns ranks=[4,0,8,1,6,2,3,7,5] 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06bee9f640 10 monclient(hunting): picked mon.noname-c con 0x7f06b809f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06bee9f640 10 monclient(hunting): picked mon.noname-a con 0x7f06b809fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06bee9f640 10 monclient(hunting): picked mon.noname-e con 0x7f06b809ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06bee9f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b809d310 con 0x7f06b809fba0 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06bee9f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b80a2560 con 0x7f06b809f7d0 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06bee9f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b811eb80 con 0x7f06b809ff70 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b7fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f06b809fba0 0x7f06b8146db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33668/0 (socket says 172.21.15.45:33668) 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b7fff640 1 -- 172.21.15.45:0/1658726759 learned_addr learned my addr 172.21.15.45:0/1658726759 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06bee9f640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06bee9f640 10 monclient(hunting): authenticate will time out at 1180.495572s 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114078706 0 0) 0x7f06b811eb80 con 0x7f06b809ff70 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 10 monclient(hunting): my global_id is 5093 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f069c0034e0 con 0x7f06b809ff70 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962100559 0 0) 0x7f06b809d310 con 0x7f06b809fba0 2024-09-18T14:06:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 10 monclient(hunting): my global_id is 5193 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06b811eb80 con 0x7f06b809fba0 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258919987 0 0) 0x7f06b80a2560 con 0x7f06b809f7d0 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 10 monclient(hunting): my global_id is 154174 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.157+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06b809d310 con 0x7f06b809f7d0 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3741321412 0 0) 0x7f06b809d310 con 0x7f06b809f7d0 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f06b80a2560 con 0x7f06b809f7d0 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f06ac001480 con 0x7f06b809f7d0 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:06:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.177 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3221869972 0 0) 0x7f06b811eb80 con 0x7f06b809fba0 2024-09-18T14:06:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f06b809d310 con 0x7f06b809fba0 2024-09-18T14:06:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3077876718 0 0) 0x7f069c0034e0 con 0x7f06b809ff70 2024-09-18T14:06:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f06b811eb80 con 0x7f06b809ff70 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f06a4001770 con 0x7f06b809fba0 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f06b0002cb0 con 0x7f06b809ff70 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3500270956 0 0) 0x7f06b80a2560 con 0x7f06b809f7d0 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 >> v1:172.21.15.45:6793/0 conn(0x7f06b809ff70 legacy=0x7f06b814a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 >> v1:172.21.15.45:6789/0 conn(0x7f06b809fba0 legacy=0x7f06b8146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 monclient: found mon.c 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b809b8e0 con 0x7f06b809f7d0 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient: _finish_auth 0 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06bee9f640 5 monclient: authenticate success, global_id 154174 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06bee9f640 10 monclient: _renew_subs 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06bee9f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06bee9f640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b811e8a0 con 0x7f06b809f7d0 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06bee9f640 10 monclient: _renew_subs 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06bee9f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06bee9f640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b8131e90 con 0x7f06b809f7d0 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f06ac001660 con 0x7f06b809f7d0 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f06ac004d90 con 0x7f06b809f7d0 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b6ffd640 4 set_mon_vals no callback set 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.158+0000 7f06b4ff9640 10 monclient: dump: 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f06ac00fd60 con 0x7f06b809f7d0 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06bee9f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06bee9f640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0698006580 con 0x7f0698002830 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(328..328 src has 126..328) v4 ==== 3939+0+0 (unknown 1446098396 0 0) 0x7f06ac046960 con 0x7f06b809f7d0 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297857419 0 0) 0x7f0698006580 con 0x7f0698002830 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 10 monclient(hunting): my global_id is 154183 2024-09-18T14:06:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06b80a2560 con 0x7f0698002830 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06bd43c640 10 monclient: get_auth_request con 0x7f069c03abf0 auth_method 0 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2385715477 0 0) 0x7f06b80a2560 con 0x7f0698002830 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0698006580 con 0x7f0698002830 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f06a4004b20 con 0x7f0698002830 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 10 monclient: dump: 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.186 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 529083864 0 0) 0x7f0698006580 con 0x7f0698002830 2024-09-18T14:06:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.159+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f069800a0a0 con 0x7f0698002830 2024-09-18T14:06:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.160+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f06a4005120 con 0x7f0698002830 2024-09-18T14:06:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.160+0000 7f06b4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.160+0000 7f06b4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.160+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 >> v1:172.21.15.45:6791/0 conn(0x7f0698002830 legacy=0x7f0698004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.160+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 >> v1:172.21.15.45:6791/0 conn(0x7f0698002830 legacy=0x7f0698004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.165+0000 7f06bee9f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.165+0000 7f06bee9f640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0698009610 con 0x7f0698008ff0 2024-09-18T14:06:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223011631 0 0) 0x7f0698009610 con 0x7f0698008ff0 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 10 monclient(hunting): my global_id is 154192 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0698006580 con 0x7f0698008ff0 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 908949559 0 0) 0x7f0698006580 con 0x7f0698008ff0 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0698009610 con 0x7f0698008ff0 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f06a40058a0 con 0x7f0698008ff0 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 10 monclient: dump: 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.190 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2008450064 0 0) 0x7f0698009610 con 0x7f0698008ff0 2024-09-18T14:06:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f06980098d0 con 0x7f0698008ff0 2024-09-18T14:06:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 2825171554) 0x7f06a4006cf0 con 0x7f0698008ff0 2024-09-18T14:06:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 >> v1:172.21.15.45:6791/0 conn(0x7f0698008ff0 legacy=0x7f0698008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.166+0000 7f06b4ff9640 1 -- 172.21.15.45:0/1658726759 >> v1:172.21.15.45:6791/0 conn(0x7f0698008ff0 legacy=0x7f0698008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:05.191 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:05.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:05.191 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:05.192 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:05.192 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:05.192 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:05.192 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:05.192 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:05.192 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:05.192 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:05.192 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:05.192 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:05.192 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:05.192 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.193 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.194 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:05.196 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:05.196 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:05.196 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:05.196 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:05.196 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:05.196 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:05.196 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:05.196 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.196 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.196 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.196 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.196 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.197 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.197 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.197 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.197 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.197 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.197 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:05.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.197 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:05.200 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.200 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.200 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.200 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:05.200 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:05.200 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.200 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.200 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.200 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.203 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:05.203 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:05.203 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.203 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.203 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.203 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:05.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:05.203 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.203 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.205 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:05.206 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:05.206 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.206 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.206 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.206 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:05.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:05.206 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.206 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.207 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:05.207 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.207 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.207 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.207 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:05.207 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:05.207 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.207 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.207 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.210 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:06:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.212 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.212 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:05.212 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.167+0000 7f06bee9f640 1 -- 172.21.15.45:0/1658726759 >> v1:172.21.15.84:6812/31478 conn(0x7f069c03abf0 legacy=0x7f069c03d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.167+0000 7f06bee9f640 10 monclient: shutdown 2024-09-18T14:06:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.167+0000 7f06bee9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.167+0000 7f06bee9f640 1 -- 172.21.15.45:0/1658726759 >> v1:172.21.15.45:6791/0 conn(0x7f06b809f7d0 legacy=0x7f06b8131780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.167+0000 7f06bd43c640 1 -- 172.21.15.45:0/1658726759 reap_dead start 2024-09-18T14:06:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.167+0000 7f06bd43c640 1 -- 172.21.15.45:0/1658726759 reap_dead start 2024-09-18T14:06:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.167+0000 7f06bee9f640 1 -- 172.21.15.45:0/1658726759 shutdown_connections 2024-09-18T14:06:05.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.167+0000 7f06bee9f640 1 -- 172.21.15.45:0/1658726759 >> 172.21.15.45:0/1658726759 conn(0x7f06b80630b0 msgr2=0x7f06b80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:05.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.167+0000 7f06bee9f640 1 -- 172.21.15.45:0/1658726759 shutdown_connections 2024-09-18T14:06:05.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.167+0000 7f06bee9f640 1 -- 172.21.15.45:0/1658726759 wait complete. 2024-09-18T14:06:05.214 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:05.214 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:06:05.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:06:05.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.312+0000 7f3fa915d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:05.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.312+0000 7f3fa915d640 10 monclient: build_initial_monmap 2024-09-18T14:06:05.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.312+0000 7f3fa915d640 10 monclient: monmap: 2024-09-18T14:06:05.313 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:05.313 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.313 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:05.313125+0000 2024-09-18T14:06:05.313 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:05.313125+0000 2024-09-18T14:06:05.313 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:05.313 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.313 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:05.313 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:05.313 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:05.313 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:05.313 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:05.314 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:05.314 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:05.314 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:05.314 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:05.314 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.312+0000 7f3fa915d640 1 Processor -- start 2024-09-18T14:06:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.312+0000 7f3fa915d640 1 -- start start 2024-09-18T14:06:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.312+0000 7f3fa915d640 10 monclient: init 2024-09-18T14:06:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa915d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa915d640 10 monclient: _add_conns ranks=[8,6,1,3,0,7,2,5,4] 2024-09-18T14:06:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa915d640 10 monclient(hunting): picked mon.noname-e con 0x7f3fa406e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa915d640 10 monclient(hunting): picked mon.noname-d con 0x7f3fa40a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa915d640 10 monclient(hunting): picked mon.noname-f con 0x7f3fa40a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa915d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa405ed70 con 0x7f3fa40a1080 2024-09-18T14:06:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa915d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa4097620 con 0x7f3fa40a0cb0 2024-09-18T14:06:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa915d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa4097340 con 0x7f3fa406e810 2024-09-18T14:06:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa915d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa915d640 10 monclient(hunting): authenticate will time out at 1180.651344s 2024-09-18T14:06:05.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa259c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f3fa40a0cb0 0x7f3fa4137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:54858/0 (socket says 172.21.15.45:54858) 2024-09-18T14:06:05.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa2d9d640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f3fa406e810 0x7f3fa40a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:39834/0 (socket says 172.21.15.45:39834) 2024-09-18T14:06:05.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa259c640 1 -- 172.21.15.45:0/2197304116 learned_addr learned my addr 172.21.15.45:0/2197304116 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:05.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903899538 0 0) 0x7f3fa4097620 con 0x7f3fa40a0cb0 2024-09-18T14:06:05.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 10 monclient(hunting): my global_id is 5046 2024-09-18T14:06:05.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f8c0034e0 con 0x7f3fa40a0cb0 2024-09-18T14:06:05.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984203634 0 0) 0x7f3fa4097340 con 0x7f3fa406e810 2024-09-18T14:06:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 10 monclient(hunting): my global_id is 5102 2024-09-18T14:06:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa4097620 con 0x7f3fa406e810 2024-09-18T14:06:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162533345 0 0) 0x7f3fa405ed70 con 0x7f3fa40a1080 2024-09-18T14:06:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 10 monclient(hunting): my global_id is 5131 2024-09-18T14:06:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa4097340 con 0x7f3fa40a1080 2024-09-18T14:06:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.313+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3726756778 0 0) 0x7f3fa4097620 con 0x7f3fa406e810 2024-09-18T14:06:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 >> v1:172.21.15.45:6792/0 conn(0x7f3fa40a0cb0 legacy=0x7f3fa4137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 >> v1:172.21.15.84:6789/0 conn(0x7f3fa40a1080 legacy=0x7f3fa413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:06:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 1 monclient: found mon.noname-e 2024-09-18T14:06:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:06:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa40a1660 con 0x7f3fa406e810 2024-09-18T14:06:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 10 monclient: _finish_auth 0 2024-09-18T14:06:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa915d640 5 monclient: authenticate success, global_id 5102 2024-09-18T14:06:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa915d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3f94001790 con 0x7f3fa406e810 2024-09-18T14:06:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:06:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 10 monclient: dump: 2024-09-18T14:06:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.330 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa915d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3f94001970 con 0x7f3fa406e810 2024-09-18T14:06:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 1 -- 172.21.15.45:0/2197304116 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3f940047a0 con 0x7f3fa406e810 2024-09-18T14:06:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa915d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa915d640 4 set_mon_vals no callback set 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa1d9b640 10 monclient: dump: 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa915d640 10 monclient: shutdown 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa915d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa915d640 1 -- 172.21.15.45:0/2197304116 >> v1:172.21.15.45:6793/0 conn(0x7f3fa406e810 legacy=0x7f3fa40a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa915d640 1 -- 172.21.15.45:0/2197304116 shutdown_connections 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa915d640 1 -- 172.21.15.45:0/2197304116 >> 172.21.15.45:0/2197304116 conn(0x7f3fa40a5bb0 msgr2=0x7f3fa40a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa915d640 1 -- 172.21.15.45:0/2197304116 shutdown_connections 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.314+0000 7f3fa915d640 1 -- 172.21.15.45:0/2197304116 wait complete. 2024-09-18T14:06:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.315+0000 7f3fa915d640 10 monclient: build_initial_monmap 2024-09-18T14:06:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.315+0000 7f3fa915d640 10 monclient: monmap: 2024-09-18T14:06:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:05.334 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.315+0000 7f3fa915d640 1 Processor -- start 2024-09-18T14:06:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.315+0000 7f3fa915d640 1 -- start start 2024-09-18T14:06:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.315+0000 7f3fa915d640 10 monclient: init 2024-09-18T14:06:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3fa915d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3fa915d640 10 monclient: _add_conns ranks=[0,7,5,6,1,4,8,2,3] 2024-09-18T14:06:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3fa915d640 10 monclient(hunting): picked mon.noname-a con 0x7f3fa406e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3fa915d640 10 monclient(hunting): picked mon.noname-i con 0x7f3fa40a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3fa915d640 10 monclient(hunting): picked mon.noname-h con 0x7f3fa40a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3fa915d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa4122d50 con 0x7f3fa406e810 2024-09-18T14:06:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3fa915d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa413db40 con 0x7f3fa40a1080 2024-09-18T14:06:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3fa915d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa4122790 con 0x7f3fa40a0cb0 2024-09-18T14:06:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3fa915d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3fa915d640 10 monclient(hunting): authenticate will time out at 1180.654292s 2024-09-18T14:06:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3fa2d9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3fa406e810 0x7f3fa413d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33672/0 (socket says 172.21.15.45:33672) 2024-09-18T14:06:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3fa2d9d640 1 -- 172.21.15.45:0/518435651 learned_addr learned my addr 172.21.15.45:0/518435651 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3772388064 0 0) 0x7f3fa4122d50 con 0x7f3fa406e810 2024-09-18T14:06:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): my global_id is 5202 2024-09-18T14:06:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f900034e0 con 0x7f3fa406e810 2024-09-18T14:06:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3322352409 0 0) 0x7f3f900034e0 con 0x7f3fa406e810 2024-09-18T14:06:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3fa4122d50 con 0x7f3fa406e810 2024-09-18T14:06:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3f94001750 con 0x7f3fa406e810 2024-09-18T14:06:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:06:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): dump: 2024-09-18T14:06:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074534087 0 0) 0x7f3fa4122790 con 0x7f3fa40a0cb0 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): my global_id is 5137 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f900034e0 con 0x7f3fa40a0cb0 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193444683 0 0) 0x7f3fa413db40 con 0x7f3fa40a1080 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 10 monclient(hunting): my global_id is 5009 2024-09-18T14:06:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa4122790 con 0x7f3fa40a1080 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.316+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2056549594 0 0) 0x7f3fa4122d50 con 0x7f3fa406e810 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 >> v1:172.21.15.84:6792/0 conn(0x7f3fa40a0cb0 legacy=0x7f3fa4146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 >> v1:172.21.15.84:6791/0 conn(0x7f3fa40a1080 legacy=0x7f3fa4149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 10 monclient: _finish_hunting 0 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 1 monclient: found mon.a 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa4120250 con 0x7f3fa406e810 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 10 monclient: _finish_auth 0 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3fa915d640 5 monclient: authenticate success, global_id 5202 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3fa915d640 10 monclient: _renew_subs 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3fa915d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3fa915d640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa41224b0 con 0x7f3fa406e810 2024-09-18T14:06:05.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3fa915d640 10 monclient: _renew_subs 2024-09-18T14:06:05.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3fa915d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:05.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3fa915d640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa41215e0 con 0x7f3fa406e810 2024-09-18T14:06:05.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3f94004dd0 con 0x7f3fa406e810 2024-09-18T14:06:05.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:05.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3f940052c0 con 0x7f3fa406e810 2024-09-18T14:06:05.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3fa159a640 4 set_mon_vals no callback set 2024-09-18T14:06:05.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 10 monclient: dump: 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3fa915d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3fa915d640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3f84006580 con 0x7f3f84002830 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.317+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f3f940054a0 con 0x7f3fa406e810 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(328..328 src has 126..328) v4 ==== 3939+0+0 (unknown 1446098396 0 0) 0x7f3f94046de0 con 0x7f3fa406e810 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761131005 0 0) 0x7f3f84006580 con 0x7f3f84002830 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 10 monclient(hunting): my global_id is 5055 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa4122d50 con 0x7f3f84002830 2024-09-18T14:06:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3fa359e640 10 monclient: get_auth_request con 0x7f3f90039fa0 auth_method 0 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3308797691 0 0) 0x7f3fa4122d50 con 0x7f3f84002830 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3f84006580 con 0x7f3f84002830 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3f980014c0 con 0x7f3f84002830 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 10 monclient: dump: 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2903738486 0 0) 0x7f3f84006580 con 0x7f3f84002830 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f8400a0a0 con 0x7f3f84002830 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3f98001c10 con 0x7f3f84002830 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 >> v1:172.21.15.45:6792/0 conn(0x7f3f84002830 legacy=0x7f3f84004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.318+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 >> v1:172.21.15.45:6792/0 conn(0x7f3f84002830 legacy=0x7f3f84004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:05.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.324+0000 7f3fa915d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:05.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.324+0000 7f3fa915d640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3f84009660 con 0x7f3f84009040 2024-09-18T14:06:05.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.324+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072907688 0 0) 0x7f3f84009660 con 0x7f3f84009040 2024-09-18T14:06:05.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.324+0000 7f3f8affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.324+0000 7f3f8affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.324+0000 7f3f8affd640 10 monclient(hunting): my global_id is 5064 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.324+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f84006580 con 0x7f3f84009040 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 503996998 0 0) 0x7f3f84006580 con 0x7f3f84009040 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3f84009660 con 0x7f3f84009040 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3f98004df0 con 0x7f3f84009040 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 10 monclient: dump: 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3829431247 0 0) 0x7f3f84009660 con 0x7f3f84009040 2024-09-18T14:06:05.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3f84009920 con 0x7f3f84009040 2024-09-18T14:06:05.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3494373011) 0x7f3f98007090 con 0x7f3f84009040 2024-09-18T14:06:05.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:05.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:05.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 >> v1:172.21.15.45:6792/0 conn(0x7f3f84009040 legacy=0x7f3f84008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3f8affd640 1 -- 172.21.15.45:0/518435651 >> v1:172.21.15.45:6792/0 conn(0x7f3f84009040 legacy=0x7f3f84008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:05.350 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:05.350 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:05.350 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:05.350 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:05.350 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:05.350 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:05.350 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:05.350 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:05.350 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:05.351 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:05.351 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:05.351 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:05.351 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:05.351 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:05.351 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:05.351 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.351 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:06:05.351 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:05.351 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:05.351 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:05.351 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.352 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.353 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.353 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.353 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.353 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.353 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.353 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.353 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.353 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.353 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:05.353 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:05.353 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:05.354 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.356 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:05.357 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:05.357 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.357 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.357 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.357 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:05.357 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:05.357 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.357 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:05.358 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.358 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.358 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:05.358 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:05.359 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.359 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.359 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.359 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:05.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:05.359 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:05.360 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.360 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.360 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.360 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:05.360 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:05.360 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.360 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.360 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.361 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:05.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:05.361 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.361 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.361 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:05.361 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:05.362 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.364 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:05.365 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:05.365 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.365 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.365 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.365 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:05.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:05.365 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.365 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:05.371 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:05.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3fa915d640 1 -- 172.21.15.45:0/518435651 >> v1:172.21.15.84:6812/31478 conn(0x7f3f90039fa0 legacy=0x7f3f9003c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3fa915d640 10 monclient: shutdown 2024-09-18T14:06:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3fa915d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3fa915d640 1 -- 172.21.15.45:0/518435651 >> v1:172.21.15.45:6789/0 conn(0x7f3fa406e810 legacy=0x7f3fa413d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3fa359e640 1 -- 172.21.15.45:0/518435651 reap_dead start 2024-09-18T14:06:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.325+0000 7f3fa359e640 1 -- 172.21.15.45:0/518435651 reap_dead start 2024-09-18T14:06:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.326+0000 7f3fa915d640 1 -- 172.21.15.45:0/518435651 shutdown_connections 2024-09-18T14:06:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.326+0000 7f3fa915d640 1 -- 172.21.15.45:0/518435651 >> 172.21.15.45:0/518435651 conn(0x7f3fa40a5bb0 msgr2=0x7f3fa40a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.326+0000 7f3fa915d640 1 -- 172.21.15.45:0/518435651 shutdown_connections 2024-09-18T14:06:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.326+0000 7f3fa915d640 1 -- 172.21.15.45:0/518435651 wait complete. 2024-09-18T14:06:05.373 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:05.374 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:06:05.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:06:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.471+0000 7fc4c5edd640 10 monclient: get_monmap_and_config 2024-09-18T14:06:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.471+0000 7fc4c5edd640 10 monclient: build_initial_monmap 2024-09-18T14:06:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.471+0000 7fc4c5edd640 10 monclient: monmap: 2024-09-18T14:06:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:05.472032+0000 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:05.472032+0000 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:05.472 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.471+0000 7fc4c5edd640 1 Processor -- start 2024-09-18T14:06:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.471+0000 7fc4c5edd640 1 -- start start 2024-09-18T14:06:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.471+0000 7fc4c5edd640 10 monclient: init 2024-09-18T14:06:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4c5edd640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4c5edd640 10 monclient: _add_conns ranks=[8,6,5,1,3,0,4,2,7] 2024-09-18T14:06:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4c5edd640 10 monclient(hunting): picked mon.noname-e con 0x7fc4c006e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4c5edd640 10 monclient(hunting): picked mon.noname-d con 0x7fc4c00a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4c5edd640 10 monclient(hunting): picked mon.noname-h con 0x7fc4c00a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4c5edd640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c005ed70 con 0x7fc4c00a1080 2024-09-18T14:06:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4c5edd640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c0097620 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4c5edd640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c0097340 con 0x7fc4c006e810 2024-09-18T14:06:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4c5edd640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4c5edd640 10 monclient(hunting): authenticate will time out at 1180.810419s 2024-09-18T14:06:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4bf7fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fc4c006e810 0x7fc4c00a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:39848/0 (socket says 172.21.15.45:39848) 2024-09-18T14:06:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4beffd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fc4c00a0cb0 0x7fc4c0137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:54892/0 (socket says 172.21.15.45:54892) 2024-09-18T14:06:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4bf7fe640 1 -- 172.21.15.45:0/906827392 learned_addr learned my addr 172.21.15.45:0/906827392 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053267354 0 0) 0x7fc4c0097620 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 10 monclient(hunting): my global_id is 5073 2024-09-18T14:06:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a80034e0 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135112453 0 0) 0x7fc4c0097340 con 0x7fc4c006e810 2024-09-18T14:06:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 10 monclient(hunting): my global_id is 5111 2024-09-18T14:06:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c0097620 con 0x7fc4c006e810 2024-09-18T14:06:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623185334 0 0) 0x7fc4c005ed70 con 0x7fc4c00a1080 2024-09-18T14:06:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 10 monclient(hunting): my global_id is 5018 2024-09-18T14:06:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.472+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c0097340 con 0x7fc4c00a1080 2024-09-18T14:06:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3848200553 0 0) 0x7fc4a80034e0 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 >> v1:172.21.15.45:6793/0 conn(0x7fc4c006e810 legacy=0x7fc4c00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 >> v1:172.21.15.84:6791/0 conn(0x7fc4c00a1080 legacy=0x7fc4c013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:06:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 1 monclient: found mon.noname-d 2024-09-18T14:06:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:06:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c00a1660 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 10 monclient: _finish_auth 0 2024-09-18T14:06:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4c5edd640 5 monclient: authenticate success, global_id 5073 2024-09-18T14:06:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4b4001830 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 10 monclient: dump: 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4c5edd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc4b4001990 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4c5edd640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4c5edd640 4 set_mon_vals no callback set 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 1 -- 172.21.15.45:0/906827392 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4b40047a0 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4be7fc640 10 monclient: dump: 2024-09-18T14:06:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.493 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4c5edd640 10 monclient: shutdown 2024-09-18T14:06:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4c5edd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4c5edd640 1 -- 172.21.15.45:0/906827392 >> v1:172.21.15.45:6792/0 conn(0x7fc4c00a0cb0 legacy=0x7fc4c0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4c5edd640 1 -- 172.21.15.45:0/906827392 shutdown_connections 2024-09-18T14:06:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4c5edd640 1 -- 172.21.15.45:0/906827392 >> 172.21.15.45:0/906827392 conn(0x7fc4c00a5bb0 msgr2=0x7fc4c00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4c5edd640 1 -- 172.21.15.45:0/906827392 shutdown_connections 2024-09-18T14:06:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.473+0000 7fc4c5edd640 1 -- 172.21.15.45:0/906827392 wait complete. 2024-09-18T14:06:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.474+0000 7fc4c5edd640 10 monclient: build_initial_monmap 2024-09-18T14:06:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.474+0000 7fc4c5edd640 10 monclient: monmap: 2024-09-18T14:06:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.474+0000 7fc4c5edd640 1 Processor -- start 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.474+0000 7fc4c5edd640 1 -- start start 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.474+0000 7fc4c5edd640 10 monclient: init 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4c5edd640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4c5edd640 10 monclient: _add_conns ranks=[2,7,8,6,1,3,0,4,5] 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4c5edd640 10 monclient(hunting): picked mon.noname-b con 0x7fc4c006e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4c5edd640 10 monclient(hunting): picked mon.noname-i con 0x7fc4c00a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4c5edd640 10 monclient(hunting): picked mon.noname-e con 0x7fc4c00a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4c5edd640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c0122c30 con 0x7fc4c006e810 2024-09-18T14:06:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4c5edd640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c013da20 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4c5edd640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c0122670 con 0x7fc4c00a1080 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4c5edd640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4c5edd640 10 monclient(hunting): authenticate will time out at 1180.813350s 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4bf7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fc4c006e810 0x7fc4c013d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39486/0 (socket says 172.21.15.45:39486) 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4bf7fe640 1 -- 172.21.15.45:0/849937859 learned_addr learned my addr 172.21.15.45:0/849937859 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944603323 0 0) 0x7fc4c0122c30 con 0x7fc4c006e810 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): my global_id is 144461 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4ac0034e0 con 0x7fc4c006e810 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849010734 0 0) 0x7fc4c0122670 con 0x7fc4c00a1080 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): my global_id is 5120 2024-09-18T14:06:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c0122c30 con 0x7fc4c00a1080 2024-09-18T14:06:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473095878 0 0) 0x7fc4c013da20 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): my global_id is 5146 2024-09-18T14:06:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c0122670 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 309807453 0 0) 0x7fc4ac0034e0 con 0x7fc4c006e810 2024-09-18T14:06:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4c013da20 con 0x7fc4c006e810 2024-09-18T14:06:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4b0001650 con 0x7fc4c006e810 2024-09-18T14:06:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): dump: 2024-09-18T14:06:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3651205521 0 0) 0x7fc4c0122c30 con 0x7fc4c00a1080 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4ac0034e0 con 0x7fc4c00a1080 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4b8001c50 con 0x7fc4c00a1080 2024-09-18T14:06:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 10 monclient(hunting): dump: 2024-09-18T14:06:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.505 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1288319597 0 0) 0x7fc4c0122670 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.475+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4c0122c30 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4b40016f0 con 0x7fc4c00a0cb0 2024-09-18T14:06:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 10 monclient(hunting): dump: 2024-09-18T14:06:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.509 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4200029890 0 0) 0x7fc4c013da20 con 0x7fc4c006e810 2024-09-18T14:06:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 >> v1:172.21.15.45:6793/0 conn(0x7fc4c00a1080 legacy=0x7fc4c0149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 >> v1:172.21.15.84:6792/0 conn(0x7fc4c00a0cb0 legacy=0x7fc4c0146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 10 monclient: _finish_hunting 0 2024-09-18T14:06:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 1 monclient: found mon.b 2024-09-18T14:06:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c0120180 con 0x7fc4c006e810 2024-09-18T14:06:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 10 monclient: _finish_auth 0 2024-09-18T14:06:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4c5edd640 5 monclient: authenticate success, global_id 144461 2024-09-18T14:06:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4c5edd640 10 monclient: _renew_subs 2024-09-18T14:06:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4c5edd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4c5edd640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c0122390 con 0x7fc4c006e810 2024-09-18T14:06:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4c5edd640 10 monclient: _renew_subs 2024-09-18T14:06:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4c5edd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4c5edd640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c01214c0 con 0x7fc4c006e810 2024-09-18T14:06:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc4b0004a50 con 0x7fc4c006e810 2024-09-18T14:06:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4b0004f00 con 0x7fc4c006e810 2024-09-18T14:06:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4bdffb640 4 set_mon_vals no callback set 2024-09-18T14:06:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4a77fe640 10 monclient: dump: 2024-09-18T14:06:05.514 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.514 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.514 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.514 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.514 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.516 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4c5edd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.476+0000 7fc4c5edd640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc4a0006580 con 0x7fc4a0002830 2024-09-18T14:06:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc4b000ff50 con 0x7fc4c006e810 2024-09-18T14:06:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(328..328 src has 126..328) v4 ==== 3939+0+0 (unknown 1446098396 0 0) 0x7fc4b0046c00 con 0x7fc4c006e810 2024-09-18T14:06:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133769017 0 0) 0x7fc4a0006580 con 0x7fc4a0002830 2024-09-18T14:06:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 10 monclient(hunting): my global_id is 5129 2024-09-18T14:06:05.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c013da20 con 0x7fc4a0002830 2024-09-18T14:06:05.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4bffff640 10 monclient: get_auth_request con 0x7fc4ac03a9a0 auth_method 0 2024-09-18T14:06:05.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2194094449 0 0) 0x7fc4c013da20 con 0x7fc4a0002830 2024-09-18T14:06:05.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4a0006580 con 0x7fc4a0002830 2024-09-18T14:06:05.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4b40047a0 con 0x7fc4a0002830 2024-09-18T14:06:05.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:05.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 10 monclient: dump: 2024-09-18T14:06:05.519 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.519 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.519 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.520 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.520 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.520 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.520 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.520 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.521 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.521 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.521 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.521 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.521 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.521 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.522 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.522 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4012613723 0 0) 0x7fc4a0006580 con 0x7fc4a0002830 2024-09-18T14:06:05.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.477+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4a000a0a0 con 0x7fc4a0002830 2024-09-18T14:06:05.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.478+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc4b4006fc0 con 0x7fc4a0002830 2024-09-18T14:06:05.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.478+0000 7fc4a77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.478+0000 7fc4a77fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:05.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.478+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 >> v1:172.21.15.45:6793/0 conn(0x7fc4a0002830 legacy=0x7fc4a0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.478+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 >> v1:172.21.15.45:6793/0 conn(0x7fc4a0002830 legacy=0x7fc4a0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:05.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.483+0000 7fc4c5edd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:05.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.483+0000 7fc4c5edd640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc4a0009660 con 0x7fc4a0009040 2024-09-18T14:06:05.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824283660 0 0) 0x7fc4a0009660 con 0x7fc4a0009040 2024-09-18T14:06:05.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 10 monclient(hunting): my global_id is 5138 2024-09-18T14:06:05.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a0006580 con 0x7fc4a0009040 2024-09-18T14:06:05.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2743315914 0 0) 0x7fc4a0006580 con 0x7fc4a0009040 2024-09-18T14:06:05.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4a0009660 con 0x7fc4a0009040 2024-09-18T14:06:05.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4b4005570 con 0x7fc4a0009040 2024-09-18T14:06:05.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 10 monclient: dump: 2024-09-18T14:06:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.529 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.530 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.531 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.532 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2473253436 0 0) 0x7fc4a0009660 con 0x7fc4a0009040 2024-09-18T14:06:05.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc4a0009920 con 0x7fc4a0009040 2024-09-18T14:06:05.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7558 (unknown 0 0 1292621130) 0x7fc4b4007250 con 0x7fc4a0009040 2024-09-18T14:06:05.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:05.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:05.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 >> v1:172.21.15.45:6793/0 conn(0x7fc4a0009040 legacy=0x7fc4a0008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.484+0000 7fc4a77fe640 1 -- 172.21.15.45:0/849937859 >> v1:172.21.15.45:6793/0 conn(0x7fc4a0009040 legacy=0x7fc4a0008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:05.533 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:06:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:06:05.537 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:05.537 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:05.537 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:05.537 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.537 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.537 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.538 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.538 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.538 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.538 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.538 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.538 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.539 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.539 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:05.539 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:05.539 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.539 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.540 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.540 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.540 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.540 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.540 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.541 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.541 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.541 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.541 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:05.541 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:05.542 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:05.542 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:05.542 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:05.542 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:05.542 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:05.543 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:05.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:05.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:05.543 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:05.543 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:05.543 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:05.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:05.544 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:05.544 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:05.544 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:05.544 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.545 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.545 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.545 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.545 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.545 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.545 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.546 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.546 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.546 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.546 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:05.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.546 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:05.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.546 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:05.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:05.546 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.547 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.549 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:05.549 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:05.549 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.549 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.549 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.549 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:05.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:05.549 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.549 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.553 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.556 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.558 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:06:05.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:05.561 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.485+0000 7fc4c5edd640 1 -- 172.21.15.45:0/849937859 >> v1:172.21.15.84:6812/31478 conn(0x7fc4ac03a9a0 legacy=0x7fc4ac03ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.485+0000 7fc4c5edd640 10 monclient: shutdown 2024-09-18T14:06:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.485+0000 7fc4c5edd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.485+0000 7fc4c5edd640 1 -- 172.21.15.45:0/849937859 >> v1:172.21.15.45:6790/0 conn(0x7fc4c006e810 legacy=0x7fc4c013d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.485+0000 7fc4bffff640 1 -- 172.21.15.45:0/849937859 reap_dead start 2024-09-18T14:06:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.485+0000 7fc4bffff640 1 -- 172.21.15.45:0/849937859 reap_dead start 2024-09-18T14:06:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.485+0000 7fc4c5edd640 1 -- 172.21.15.45:0/849937859 shutdown_connections 2024-09-18T14:06:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.485+0000 7fc4c5edd640 1 -- 172.21.15.45:0/849937859 >> 172.21.15.45:0/849937859 conn(0x7fc4c00a5bb0 msgr2=0x7fc4c00a2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:05.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.485+0000 7fc4c5edd640 1 -- 172.21.15.45:0/849937859 shutdown_connections 2024-09-18T14:06:05.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.485+0000 7fc4c5edd640 1 -- 172.21.15.45:0/849937859 wait complete. 2024-09-18T14:06:05.563 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:05.563 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:06:05.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:06:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.660+0000 7f607a62b640 10 monclient: get_monmap_and_config 2024-09-18T14:06:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.660+0000 7f607a62b640 10 monclient: build_initial_monmap 2024-09-18T14:06:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.660+0000 7f607a62b640 10 monclient: monmap: 2024-09-18T14:06:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:05.661724+0000 2024-09-18T14:06:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:05.661724+0000 2024-09-18T14:06:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:05.662 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.661+0000 7f607a62b640 1 Processor -- start 2024-09-18T14:06:05.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.661+0000 7f607a62b640 1 -- start start 2024-09-18T14:06:05.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.661+0000 7f607a62b640 10 monclient: init 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.661+0000 7f607a62b640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.661+0000 7f607a62b640 10 monclient: _add_conns ranks=[1,6,0,4,2,3,5,7,8] 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.661+0000 7f607a62b640 10 monclient(hunting): picked mon.noname-f con 0x7f607409f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f607a62b640 10 monclient(hunting): picked mon.noname-d con 0x7f607409fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f607a62b640 10 monclient(hunting): picked mon.noname-a con 0x7f607409ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f607a62b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607405ed70 con 0x7f607409ff70 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f607a62b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607409dfc0 con 0x7f607409f7d0 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f607a62b640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607409dce0 con 0x7f607409fba0 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f607a62b640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f607a62b640 10 monclient(hunting): authenticate will time out at 1181.000064s 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6078bc8640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f607409ff70 0x7f607413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33674/0 (socket says 172.21.15.45:33674) 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6078bc8640 1 -- 172.21.15.45:0/1673966917 learned_addr learned my addr 172.21.15.45:0/1673966917 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f60737fe640 1 --1- 172.21.15.45:0/1673966917 >> v1:172.21.15.45:6792/0 conn(0x7f607409fba0 0x7f6074137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:54902/0 (socket says 172.21.15.45:54902) 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007056789 0 0) 0x7f607409dce0 con 0x7f607409fba0 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient(hunting): my global_id is 5082 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f605c0034e0 con 0x7f607409fba0 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101908914 0 0) 0x7f607405ed70 con 0x7f607409ff70 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient(hunting): my global_id is 5211 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607409dce0 con 0x7f607409ff70 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593899019 0 0) 0x7f607409dfc0 con 0x7f607409f7d0 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient(hunting): my global_id is 5140 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607405ed70 con 0x7f607409f7d0 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1444775814 0 0) 0x7f605c0034e0 con 0x7f607409fba0 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 >> v1:172.21.15.84:6789/0 conn(0x7f607409f7d0 legacy=0x7f6074132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 >> v1:172.21.15.45:6789/0 conn(0x7f607409ff70 legacy=0x7f607413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 monclient: found mon.noname-d 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60740a1c00 con 0x7f607409fba0 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient: _finish_auth 0 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6068001760 con 0x7f607409fba0 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f607a62b640 5 monclient: authenticate success, global_id 5082 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f607a62b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient: dump: 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.674 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f607a62b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6068001940 con 0x7f607409fba0 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 1 -- 172.21.15.45:0/1673966917 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f60680047a0 con 0x7f607409fba0 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f607a62b640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f607a62b640 4 set_mon_vals no callback set 2024-09-18T14:06:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.662+0000 7f6072ffd640 10 monclient: dump: 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.663+0000 7f607a62b640 10 monclient: shutdown 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.663+0000 7f607a62b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.663+0000 7f607a62b640 1 -- 172.21.15.45:0/1673966917 >> v1:172.21.15.45:6792/0 conn(0x7f607409fba0 legacy=0x7f6074137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.663+0000 7f607a62b640 1 -- 172.21.15.45:0/1673966917 shutdown_connections 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.663+0000 7f607a62b640 1 -- 172.21.15.45:0/1673966917 >> 172.21.15.45:0/1673966917 conn(0x7f60740630b0 msgr2=0x7f60740634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.663+0000 7f607a62b640 1 -- 172.21.15.45:0/1673966917 shutdown_connections 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.663+0000 7f607a62b640 1 -- 172.21.15.45:0/1673966917 wait complete. 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.663+0000 7f607a62b640 10 monclient: build_initial_monmap 2024-09-18T14:06:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.663+0000 7f607a62b640 10 monclient: monmap: 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:05.679 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.664+0000 7f607a62b640 1 Processor -- start 2024-09-18T14:06:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.664+0000 7f607a62b640 1 -- start start 2024-09-18T14:06:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.664+0000 7f607a62b640 10 monclient: init 2024-09-18T14:06:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.664+0000 7f607a62b640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.664+0000 7f607a62b640 10 monclient: _add_conns ranks=[0,3,7,2,8,4,6,5,1] 2024-09-18T14:06:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.664+0000 7f607a62b640 10 monclient(hunting): picked mon.noname-a con 0x7f607409f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.664+0000 7f607a62b640 10 monclient(hunting): picked mon.noname-g con 0x7f607409fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.664+0000 7f607a62b640 10 monclient(hunting): picked mon.noname-i con 0x7f607409ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.664+0000 7f607a62b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60740a2560 con 0x7f607409f7d0 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.664+0000 7f607a62b640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607409d310 con 0x7f607409fba0 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.664+0000 7f607a62b640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6074120b10 con 0x7f607409ff70 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f607a62b640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f607a62b640 10 monclient(hunting): authenticate will time out at 1181.003019s 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f6073fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f607409f7d0 0x7f60741418a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33686/0 (socket says 172.21.15.45:33686) 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f6073fff640 1 -- 172.21.15.45:0/2474761887 learned_addr learned my addr 172.21.15.45:0/2474761887 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799458923 0 0) 0x7f60740a2560 con 0x7f607409f7d0 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): my global_id is 5220 2024-09-18T14:06:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60600034e0 con 0x7f607409f7d0 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687660079 0 0) 0x7f607409d310 con 0x7f607409fba0 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): my global_id is 154182 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60740a2560 con 0x7f607409fba0 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973708758 0 0) 0x7f6074120b10 con 0x7f607409ff70 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): my global_id is 5155 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607409d310 con 0x7f607409ff70 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2508274015 0 0) 0x7f60600034e0 con 0x7f607409f7d0 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6074120b10 con 0x7f607409f7d0 2024-09-18T14:06:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6064002bf0 con 0x7f607409f7d0 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): dump: 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1369163885 0 0) 0x7f60740a2560 con 0x7f607409fba0 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f60600034e0 con 0x7f607409fba0 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f60680016a0 con 0x7f607409fba0 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): dump: 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.684 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 102998249 0 0) 0x7f607409d310 con 0x7f607409ff70 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f60740a2560 con 0x7f607409ff70 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f606c001ca0 con 0x7f607409ff70 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient(hunting): dump: 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.685 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2912899418 0 0) 0x7f6074120b10 con 0x7f607409f7d0 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 >> v1:172.21.15.84:6792/0 conn(0x7f607409ff70 legacy=0x7f6074149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 >> v1:172.21.15.84:6790/0 conn(0x7f607409fba0 legacy=0x7f6074131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:06:05.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 monclient: found mon.a 2024-09-18T14:06:05.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:05.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:05.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60741426e0 con 0x7f607409f7d0 2024-09-18T14:06:05.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient: _finish_auth 0 2024-09-18T14:06:05.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:05.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f607a62b640 5 monclient: authenticate success, global_id 5220 2024-09-18T14:06:05.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f607a62b640 10 monclient: _renew_subs 2024-09-18T14:06:05.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f607a62b640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:05.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f607a62b640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607411ff40 con 0x7f607409f7d0 2024-09-18T14:06:05.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f607a62b640 10 monclient: _renew_subs 2024-09-18T14:06:05.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f607a62b640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f607a62b640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6074132030 con 0x7f607409f7d0 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6064001660 con 0x7f607409f7d0 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6064001bd0 con 0x7f607409f7d0 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f605bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.665+0000 7f60727fc640 4 set_mon_vals no callback set 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.666+0000 7f605bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.666+0000 7f605bfff640 10 monclient: dump: 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.666+0000 7f607a62b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.666+0000 7f607a62b640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6054006580 con 0x7f6054002830 2024-09-18T14:06:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.666+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f6064001d30 con 0x7f607409f7d0 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(328..328 src has 126..328) v4 ==== 3939+0+0 (unknown 1446098396 0 0) 0x7f6064046b80 con 0x7f607409f7d0 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673110802 0 0) 0x7f6054006580 con 0x7f6054002830 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 10 monclient(hunting): my global_id is 5149 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6074120b10 con 0x7f6054002830 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f6078bc8640 10 monclient: get_auth_request con 0x7f606003a950 auth_method 0 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1304546089 0 0) 0x7f6074120b10 con 0x7f6054002830 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6054006580 con 0x7f6054002830 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6068001140 con 0x7f6054002830 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:06:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 10 monclient: dump: 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.692 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1806332236 0 0) 0x7f6054006580 con 0x7f6054002830 2024-09-18T14:06:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.667+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f605400a0a0 con 0x7f6054002830 2024-09-18T14:06:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.668+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f60680072a0 con 0x7f6054002830 2024-09-18T14:06:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.668+0000 7f605bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.668+0000 7f605bfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.668+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 >> v1:172.21.15.84:6789/0 conn(0x7f6054002830 legacy=0x7f6054004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.668+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 >> v1:172.21.15.84:6789/0 conn(0x7f6054002830 legacy=0x7f6054004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.673+0000 7f607a62b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.673+0000 7f607a62b640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6054007540 con 0x7f60540068f0 2024-09-18T14:06:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481747521 0 0) 0x7f6054007540 con 0x7f60540068f0 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 10 monclient(hunting): my global_id is 5158 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6054006580 con 0x7f60540068f0 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3181904655 0 0) 0x7f6054006580 con 0x7f60540068f0 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6054007540 con 0x7f60540068f0 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6068005ba0 con 0x7f60540068f0 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 10 monclient: dump: 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1121806709 0 0) 0x7f6054007540 con 0x7f60540068f0 2024-09-18T14:06:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.674+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f60540087a0 con 0x7f60540068f0 2024-09-18T14:06:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 1960964948) 0x7f606800a290 con 0x7f60540068f0 2024-09-18T14:06:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f605bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f605bfff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 >> v1:172.21.15.84:6789/0 conn(0x7f60540068f0 legacy=0x7f6054006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f605bfff640 1 -- 172.21.15.45:0/2474761887 >> v1:172.21.15.84:6789/0 conn(0x7f60540068f0 legacy=0x7f6054006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:05.696 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:05.696 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:05.696 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:05.696 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.697 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.698 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.699 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.701 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.702 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:05.705 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:05.706 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:05.707 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.708 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.708 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.708 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:05.708 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.708 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.708 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.708 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:05.708 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:05.708 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.708 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.709 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.715 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.716 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.716 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.716 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.716 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.716 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:05.716 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:05.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f607a62b640 1 -- 172.21.15.45:0/2474761887 >> v1:172.21.15.84:6812/31478 conn(0x7f606003a950 legacy=0x7f606003ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f607a62b640 10 monclient: shutdown 2024-09-18T14:06:05.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f607a62b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:05.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f607a62b640 1 -- 172.21.15.45:0/2474761887 >> v1:172.21.15.45:6789/0 conn(0x7f607409f7d0 legacy=0x7f60741418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f6078bc8640 1 -- 172.21.15.45:0/2474761887 reap_dead start 2024-09-18T14:06:05.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f6078bc8640 1 -- 172.21.15.45:0/2474761887 reap_dead start 2024-09-18T14:06:05.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f607a62b640 1 -- 172.21.15.45:0/2474761887 shutdown_connections 2024-09-18T14:06:05.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f607a62b640 1 -- 172.21.15.45:0/2474761887 >> 172.21.15.45:0/2474761887 conn(0x7f60740630b0 msgr2=0x7f60740a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:05.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f607a62b640 1 -- 172.21.15.45:0/2474761887 shutdown_connections 2024-09-18T14:06:05.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.675+0000 7f607a62b640 1 -- 172.21.15.45:0/2474761887 wait complete. 2024-09-18T14:06:05.718 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:05.718 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:06:05.719 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:06:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.816+0000 7f49f6314640 10 monclient: get_monmap_and_config 2024-09-18T14:06:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.816+0000 7f49f6314640 10 monclient: build_initial_monmap 2024-09-18T14:06:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.816+0000 7f49f6314640 10 monclient: monmap: 2024-09-18T14:06:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:05.817043+0000 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:05.817043+0000 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:05.818 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:05.818 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.816+0000 7f49f6314640 1 Processor -- start 2024-09-18T14:06:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f6314640 1 -- start start 2024-09-18T14:06:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f6314640 10 monclient: init 2024-09-18T14:06:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f6314640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f6314640 10 monclient: _add_conns ranks=[8,5,0,3,1,4,6,2,7] 2024-09-18T14:06:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f6314640 10 monclient(hunting): picked mon.noname-e con 0x7f49f006e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f6314640 10 monclient(hunting): picked mon.noname-h con 0x7f49f00a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f6314640 10 monclient(hunting): picked mon.noname-a con 0x7f49f00a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f6314640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f005ed70 con 0x7f49f00a1080 2024-09-18T14:06:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f6314640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f0097620 con 0x7f49f00a0cb0 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f6314640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f0097340 con 0x7f49f006e810 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f6314640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f6314640 10 monclient(hunting): authenticate will time out at 1181.155515s 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f48b1640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f49f00a1080 0x7f49f013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33696/0 (socket says 172.21.15.45:33696) 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49f48b1640 1 -- 172.21.15.45:0/821869412 learned_addr learned my addr 172.21.15.45:0/821869412 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49effff640 1 --1- 172.21.15.45:0/821869412 >> v1:172.21.15.45:6793/0 conn(0x7f49f006e810 0x7f49f00a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:39890/0 (socket says 172.21.15.45:39890) 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415026204 0 0) 0x7f49f005ed70 con 0x7f49f00a1080 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49ef7fe640 10 monclient(hunting): my global_id is 5229 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49dc0034e0 con 0x7f49f00a1080 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099675680 0 0) 0x7f49f0097340 con 0x7f49f006e810 2024-09-18T14:06:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49ef7fe640 10 monclient(hunting): my global_id is 5147 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.817+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f005ed70 con 0x7f49f006e810 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019205379 0 0) 0x7f49f0097620 con 0x7f49f00a0cb0 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient(hunting): my global_id is 5027 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f0097340 con 0x7f49f00a0cb0 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4217600768 0 0) 0x7f49f005ed70 con 0x7f49f006e810 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 >> v1:172.21.15.84:6791/0 conn(0x7f49f00a0cb0 legacy=0x7f49f0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 >> v1:172.21.15.45:6789/0 conn(0x7f49f00a1080 legacy=0x7f49f013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 1 monclient: found mon.noname-e 2024-09-18T14:06:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f00a1660 con 0x7f49f006e810 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient: _finish_auth 0 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49f6314640 5 monclient: authenticate success, global_id 5147 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49f6314640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49e0001810 con 0x7f49f006e810 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient: dump: 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49f6314640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f49e0001970 con 0x7f49f006e810 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 1 -- 172.21.15.45:0/821869412 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49e00047a0 con 0x7f49f006e810 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49ef7fe640 10 monclient: dump: 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49f6314640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49f6314640 4 set_mon_vals no callback set 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49f6314640 10 monclient: shutdown 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49f6314640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49f6314640 1 -- 172.21.15.45:0/821869412 >> v1:172.21.15.45:6793/0 conn(0x7f49f006e810 legacy=0x7f49f00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49f6314640 1 -- 172.21.15.45:0/821869412 shutdown_connections 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.818+0000 7f49f6314640 1 -- 172.21.15.45:0/821869412 >> 172.21.15.45:0/821869412 conn(0x7f49f00a5bb0 msgr2=0x7f49f00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.819+0000 7f49f6314640 1 -- 172.21.15.45:0/821869412 shutdown_connections 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.819+0000 7f49f6314640 1 -- 172.21.15.45:0/821869412 wait complete. 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.819+0000 7f49f6314640 10 monclient: build_initial_monmap 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.819+0000 7f49f6314640 10 monclient: monmap: 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.819+0000 7f49f6314640 1 Processor -- start 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.819+0000 7f49f6314640 1 -- start start 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.819+0000 7f49f6314640 10 monclient: init 2024-09-18T14:06:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49f6314640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49f6314640 10 monclient: _add_conns ranks=[7,4,5,2,6,8,0,1,3] 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49f6314640 10 monclient(hunting): picked mon.noname-i con 0x7f49f006e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49f6314640 10 monclient(hunting): picked mon.noname-c con 0x7f49f00a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49f6314640 10 monclient(hunting): picked mon.noname-h con 0x7f49f00a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49f6314640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f0122d40 con 0x7f49f00a0cb0 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49f6314640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f013db30 con 0x7f49f00a1080 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49f6314640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f0122780 con 0x7f49f006e810 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49f6314640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49f6314640 10 monclient(hunting): authenticate will time out at 1181.158422s 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49e7fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f49f00a0cb0 0x7f49f0146650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58520/0 (socket says 172.21.15.45:58520) 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49e7fff640 1 -- 172.21.15.45:0/3830461609 learned_addr learned my addr 172.21.15.45:0/3830461609 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407949775 0 0) 0x7f49f0122d40 con 0x7f49f00a0cb0 2024-09-18T14:06:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 10 monclient(hunting): my global_id is 154201 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d00034e0 con 0x7f49f00a0cb0 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974473264 0 0) 0x7f49f0122780 con 0x7f49f006e810 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 10 monclient(hunting): my global_id is 5164 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f0122d40 con 0x7f49f006e810 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299058673 0 0) 0x7f49f013db30 con 0x7f49f00a1080 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 10 monclient(hunting): my global_id is 5036 2024-09-18T14:06:05.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f0122780 con 0x7f49f00a1080 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 834381206 0 0) 0x7f49d00034e0 con 0x7f49f00a0cb0 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49f013db30 con 0x7f49f00a0cb0 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49d8002bf0 con 0x7f49f00a0cb0 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.820+0000 7f49ecff9640 10 monclient(hunting): dump: 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.837 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3519058534 0 0) 0x7f49f0122d40 con 0x7f49f006e810 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49d00034e0 con 0x7f49f006e810 2024-09-18T14:06:05.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49e00017d0 con 0x7f49f006e810 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient(hunting): dump: 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.840 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3330211660 0 0) 0x7f49f0122780 con 0x7f49f00a1080 2024-09-18T14:06:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49f0122d40 con 0x7f49f00a1080 2024-09-18T14:06:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49e8001de0 con 0x7f49f00a1080 2024-09-18T14:06:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient(hunting): dump: 2024-09-18T14:06:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.841 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 610133933 0 0) 0x7f49f013db30 con 0x7f49f00a0cb0 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 >> v1:172.21.15.84:6792/0 conn(0x7f49f006e810 legacy=0x7f49f013d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 >> v1:172.21.15.84:6791/0 conn(0x7f49f00a1080 legacy=0x7f49f0149d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient: _finish_hunting 0 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 monclient: found mon.c 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f0120290 con 0x7f49f00a0cb0 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient: _finish_auth 0 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49f6314640 5 monclient: authenticate success, global_id 154201 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49f6314640 10 monclient: _renew_subs 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49f6314640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49f6314640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f01224a0 con 0x7f49f00a0cb0 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49f6314640 10 monclient: _renew_subs 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49f6314640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49f6314640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f01215d0 con 0x7f49f00a0cb0 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f49d8001660 con 0x7f49f00a0cb0 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49d8001bd0 con 0x7f49f00a0cb0 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49eeffd640 4 set_mon_vals no callback set 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49ecff9640 10 monclient: dump: 2024-09-18T14:06:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.821+0000 7f49f6314640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49f6314640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f49d4006580 con 0x7f49d4002830 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f49d8001db0 con 0x7f49f00a0cb0 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(328..328 src has 126..328) v4 ==== 3939+0+0 (unknown 1446098396 0 0) 0x7f49d8046bd0 con 0x7f49f00a0cb0 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563614245 0 0) 0x7f49d4006580 con 0x7f49d4002830 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 10 monclient(hunting): my global_id is 154191 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f013db30 con 0x7f49d4002830 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49f48b1640 10 monclient: get_auth_request con 0x7f49d003a8a0 auth_method 0 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2190940303 0 0) 0x7f49f013db30 con 0x7f49d4002830 2024-09-18T14:06:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49d4006580 con 0x7f49d4002830 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49e0004f50 con 0x7f49d4002830 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 10 monclient: dump: 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.846 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3250600422 0 0) 0x7f49d4006580 con 0x7f49d4002830 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.822+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49d400a0a0 con 0x7f49d4002830 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.823+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f49e0005260 con 0x7f49d4002830 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.823+0000 7f49ecff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.823+0000 7f49ecff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.823+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 >> v1:172.21.15.84:6790/0 conn(0x7f49d4002830 legacy=0x7f49d4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.823+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 >> v1:172.21.15.84:6790/0 conn(0x7f49d4002830 legacy=0x7f49d4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49f6314640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49f6314640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f49d4009660 con 0x7f49d4009040 2024-09-18T14:06:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005954124 0 0) 0x7f49d4009660 con 0x7f49d4009040 2024-09-18T14:06:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 10 monclient(hunting): my global_id is 154200 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d4006580 con 0x7f49d4009040 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3096995260 0 0) 0x7f49d4006580 con 0x7f49d4009040 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49d4009660 con 0x7f49d4009040 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f49e0005ac0 con 0x7f49d4009040 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 10 monclient: dump: 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2642562781 0 0) 0x7f49d4009660 con 0x7f49d4009040 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.829+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f49d4009920 con 0x7f49d4009040 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 2538662971) 0x7f49e0008250 con 0x7f49d4009040 2024-09-18T14:06:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49ecff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49ecff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 >> v1:172.21.15.84:6790/0 conn(0x7f49d4009040 legacy=0x7f49d4008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49ecff9640 1 -- 172.21.15.45:0/3830461609 >> v1:172.21.15.84:6790/0 conn(0x7f49d4009040 legacy=0x7f49d4008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:05.852 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:05.852 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:05.852 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:05.852 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:05.852 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:05.852 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:05.853 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:05.854 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:05.856 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.857 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.858 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:05.861 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.862 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:05.865 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.867 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.869 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.870 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:05.871 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:05.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:05.872 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:05.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:05.872 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:05.872 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:05.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49f6314640 1 -- 172.21.15.45:0/3830461609 >> v1:172.21.15.84:6812/31478 conn(0x7f49d003a8a0 legacy=0x7f49d003cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49f6314640 10 monclient: shutdown 2024-09-18T14:06:05.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49f6314640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:05.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49f6314640 1 -- 172.21.15.45:0/3830461609 >> v1:172.21.15.45:6791/0 conn(0x7f49f00a0cb0 legacy=0x7f49f0146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49f48b1640 1 -- 172.21.15.45:0/3830461609 reap_dead start 2024-09-18T14:06:05.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49f48b1640 1 -- 172.21.15.45:0/3830461609 reap_dead start 2024-09-18T14:06:05.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49f6314640 1 -- 172.21.15.45:0/3830461609 shutdown_connections 2024-09-18T14:06:05.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49f6314640 1 -- 172.21.15.45:0/3830461609 >> 172.21.15.45:0/3830461609 conn(0x7f49f00a5bb0 msgr2=0x7f49f00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:05.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49f6314640 1 -- 172.21.15.45:0/3830461609 shutdown_connections 2024-09-18T14:06:05.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.830+0000 7f49f6314640 1 -- 172.21.15.45:0/3830461609 wait complete. 2024-09-18T14:06:05.874 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:05.874 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:06:05.874 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:06:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.971+0000 7fa5df79d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.971+0000 7fa5df79d640 10 monclient: build_initial_monmap 2024-09-18T14:06:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.971+0000 7fa5df79d640 10 monclient: monmap: 2024-09-18T14:06:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:05.972420+0000 2024-09-18T14:06:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:05.972420+0000 2024-09-18T14:06:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:05.973 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 1 Processor -- start 2024-09-18T14:06:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 1 -- start start 2024-09-18T14:06:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 10 monclient: init 2024-09-18T14:06:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 10 monclient: _add_conns ranks=[4,1,7,3,0,2,8,5,6] 2024-09-18T14:06:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 10 monclient(hunting): picked mon.noname-c con 0x7fa5d809f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 10 monclient(hunting): picked mon.noname-f con 0x7fa5d809fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 10 monclient(hunting): picked mon.noname-i con 0x7fa5d809ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d805ed70 con 0x7fa5d809fba0 2024-09-18T14:06:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d809dfc0 con 0x7fa5d809f7d0 2024-09-18T14:06:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d809dce0 con 0x7fa5d809ff70 2024-09-18T14:06:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5df79d640 10 monclient(hunting): authenticate will time out at 1181.310803s 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5dd539640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fa5d809f7d0 0x7fa5d8132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58534/0 (socket says 172.21.15.45:58534) 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.972+0000 7fa5dd539640 1 -- 172.21.15.45:0/1600440117 learned_addr learned my addr 172.21.15.45:0/1600440117 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 1 -- 172.21.15.45:0/1600440117 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860010232 0 0) 0x7fa5d809dfc0 con 0x7fa5d809f7d0 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient(hunting): my global_id is 154210 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 1 -- 172.21.15.45:0/1600440117 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5c40034e0 con 0x7fa5d809f7d0 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 1 -- 172.21.15.45:0/1600440117 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029048813 0 0) 0x7fa5c40034e0 con 0x7fa5d809f7d0 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 1 -- 172.21.15.45:0/1600440117 >> v1:172.21.15.84:6792/0 conn(0x7fa5d809ff70 legacy=0x7fa5d813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 1 -- 172.21.15.45:0/1600440117 >> v1:172.21.15.84:6789/0 conn(0x7fa5d809fba0 legacy=0x7fa5d8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient: _finish_hunting 0 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 1 monclient: found mon.noname-c 2024-09-18T14:06:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 1 -- 172.21.15.45:0/1600440117 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d80a1c00 con 0x7fa5d809f7d0 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient: _finish_auth 0 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 1 -- 172.21.15.45:0/1600440117 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa5d4001830 con 0x7fa5d809f7d0 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5df79d640 5 monclient: authenticate success, global_id 154210 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5df79d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient: dump: 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5df79d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 1 -- 172.21.15.45:0/1600440117 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa5d4001a10 con 0x7fa5d809f7d0 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 1 -- 172.21.15.45:0/1600440117 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa5d4004ba0 con 0x7fa5d809f7d0 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5cbfff640 10 monclient: dump: 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5df79d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.973+0000 7fa5df79d640 4 set_mon_vals no callback set 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.974+0000 7fa5df79d640 10 monclient: shutdown 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.974+0000 7fa5df79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.974+0000 7fa5df79d640 1 -- 172.21.15.45:0/1600440117 >> v1:172.21.15.45:6791/0 conn(0x7fa5d809f7d0 legacy=0x7fa5d8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.974+0000 7fa5df79d640 1 -- 172.21.15.45:0/1600440117 shutdown_connections 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.974+0000 7fa5df79d640 1 -- 172.21.15.45:0/1600440117 >> 172.21.15.45:0/1600440117 conn(0x7fa5d80630b0 msgr2=0x7fa5d80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.974+0000 7fa5df79d640 1 -- 172.21.15.45:0/1600440117 shutdown_connections 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.974+0000 7fa5df79d640 1 -- 172.21.15.45:0/1600440117 wait complete. 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.974+0000 7fa5df79d640 10 monclient: build_initial_monmap 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.974+0000 7fa5df79d640 10 monclient: monmap: 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.974+0000 7fa5df79d640 1 Processor -- start 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5df79d640 1 -- start start 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5df79d640 10 monclient: init 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5df79d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5df79d640 10 monclient: _add_conns ranks=[2,4,6,0,3,7,8,5,1] 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5df79d640 10 monclient(hunting): picked mon.noname-b con 0x7fa5d809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5df79d640 10 monclient(hunting): picked mon.noname-c con 0x7fa5d809fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5df79d640 10 monclient(hunting): picked mon.noname-d con 0x7fa5d809ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5df79d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d809d310 con 0x7fa5d809f7d0 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5df79d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d80a2560 con 0x7fa5d809fba0 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5df79d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d811ff20 con 0x7fa5d809ff70 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5df79d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5df79d640 10 monclient(hunting): authenticate will time out at 1181.313664s 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5dd539640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fa5d809f7d0 0x7fa5d8141930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39488/0 (socket says 172.21.15.45:39488) 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5dd539640 1 -- 172.21.15.45:0/3181744132 learned_addr learned my addr 172.21.15.45:0/3181744132 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.975+0000 7fa5dcd38640 1 --1- 172.21.15.45:0/3181744132 >> v1:172.21.15.45:6791/0 conn(0x7fa5d809fba0 0x7fa5d8131900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58536/0 (socket says 172.21.15.45:58536) 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467281337 0 0) 0x7fa5d80a2560 con 0x7fa5d809fba0 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): my global_id is 154219 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5bc0034e0 con 0x7fa5d809fba0 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404337424 0 0) 0x7fa5d809d310 con 0x7fa5d809f7d0 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): my global_id is 144470 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d80a2560 con 0x7fa5d809f7d0 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949350460 0 0) 0x7fa5d811ff20 con 0x7fa5d809ff70 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): my global_id is 5091 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d809d310 con 0x7fa5d809ff70 2024-09-18T14:06:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 839804709 0 0) 0x7fa5d809d310 con 0x7fa5d809ff70 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5d811ff20 con 0x7fa5d809ff70 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3357230856 0 0) 0x7fa5d80a2560 con 0x7fa5d809f7d0 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5d809d310 con 0x7fa5d809f7d0 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 875127438 0 0) 0x7fa5bc0034e0 con 0x7fa5d809fba0 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5d80a2560 con 0x7fa5d809fba0 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa5d0001be0 con 0x7fa5d809ff70 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): dump: 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa5d4004ae0 con 0x7fa5d809f7d0 2024-09-18T14:06:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): dump: 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa5cc001aa0 con 0x7fa5d809fba0 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient(hunting): dump: 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3045911386 0 0) 0x7fa5d811ff20 con 0x7fa5d809ff70 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 >> v1:172.21.15.45:6791/0 conn(0x7fa5d809fba0 legacy=0x7fa5d8131900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 >> v1:172.21.15.45:6790/0 conn(0x7fa5d809f7d0 legacy=0x7fa5d8141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient: _finish_hunting 0 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 monclient: found mon.d 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d8121a50 con 0x7fa5d809ff70 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient: _finish_auth 0 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5df79d640 5 monclient: authenticate success, global_id 5091 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5df79d640 10 monclient: _renew_subs 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5df79d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5df79d640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d8120620 con 0x7fa5d809ff70 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5df79d640 10 monclient: _renew_subs 2024-09-18T14:06:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5df79d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5df79d640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d8120af0 con 0x7fa5d809ff70 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa5d0001d40 con 0x7fa5d809ff70 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa5d00022f0 con 0x7fa5d809ff70 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5caffd640 4 set_mon_vals no callback set 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.976+0000 7fa5c97fa640 10 monclient: dump: 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.977+0000 7fa5df79d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.977+0000 7fa5df79d640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa5b8006580 con 0x7fa5b8002830 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.977+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fa5d0002450 con 0x7fa5d809ff70 2024-09-18T14:06:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.977+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(328..328 src has 126..328) v4 ==== 3939+0+0 (unknown 1446098396 0 0) 0x7fa5d0044340 con 0x7fa5d809ff70 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179817935 0 0) 0x7fa5b8006580 con 0x7fa5b8002830 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 10 monclient(hunting): my global_id is 5045 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d811ff20 con 0x7fa5b8002830 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5dcd38640 10 monclient: get_auth_request con 0x7fa5bc03aaf0 auth_method 0 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1297162503 0 0) 0x7fa5d811ff20 con 0x7fa5b8002830 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5b8006580 con 0x7fa5b8002830 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa5d4005160 con 0x7fa5b8002830 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 10 monclient: dump: 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.001 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2596283903 0 0) 0x7fa5b8006580 con 0x7fa5b8002830 2024-09-18T14:06:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5b800a0a0 con 0x7fa5b8002830 2024-09-18T14:06:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa5d4005470 con 0x7fa5b8002830 2024-09-18T14:06:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 >> v1:172.21.15.84:6791/0 conn(0x7fa5b8002830 legacy=0x7fa5b8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.978+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 >> v1:172.21.15.84:6791/0 conn(0x7fa5b8002830 legacy=0x7fa5b8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.984+0000 7fa5df79d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.984+0000 7fa5df79d640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa5b8007540 con 0x7fa5b80068f0 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980402480 0 0) 0x7fa5b8007540 con 0x7fa5b80068f0 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 10 monclient(hunting): my global_id is 5054 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b8006580 con 0x7fa5b80068f0 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2564852665 0 0) 0x7fa5b8006580 con 0x7fa5b80068f0 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5b8007540 con 0x7fa5b80068f0 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa5d4005c50 con 0x7fa5b80068f0 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 10 monclient: dump: 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.005 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3212603917 0 0) 0x7fa5b8007540 con 0x7fa5b80068f0 2024-09-18T14:06:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.985+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa5b80087a0 con 0x7fa5b80068f0 2024-09-18T14:06:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 1579185243) 0x7fa5d4005ff0 con 0x7fa5b80068f0 2024-09-18T14:06:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5c97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5c97fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 >> v1:172.21.15.84:6791/0 conn(0x7fa5b80068f0 legacy=0x7fa5b8006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5c97fa640 1 -- 172.21.15.45:0/3181744132 >> v1:172.21.15.84:6791/0 conn(0x7fa5b80068f0 legacy=0x7fa5b8006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:06.005 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:06.023 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.024 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:06.025 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:06.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.025 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:06.025 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:06.025 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:06.025 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.025 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.025 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:06.025 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5df79d640 1 -- 172.21.15.45:0/3181744132 >> v1:172.21.15.84:6812/31478 conn(0x7fa5bc03aaf0 legacy=0x7fa5bc03cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5df79d640 10 monclient: shutdown 2024-09-18T14:06:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5df79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5df79d640 1 -- 172.21.15.45:0/3181744132 >> v1:172.21.15.45:6792/0 conn(0x7fa5d809ff70 legacy=0x7fa5d8149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5ddd3a640 1 -- 172.21.15.45:0/3181744132 reap_dead start 2024-09-18T14:06:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5ddd3a640 1 -- 172.21.15.45:0/3181744132 reap_dead start 2024-09-18T14:06:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5df79d640 1 -- 172.21.15.45:0/3181744132 shutdown_connections 2024-09-18T14:06:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5df79d640 1 -- 172.21.15.45:0/3181744132 >> 172.21.15.45:0/3181744132 conn(0x7fa5d80630b0 msgr2=0x7fa5d80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5df79d640 1 -- 172.21.15.45:0/3181744132 shutdown_connections 2024-09-18T14:06:06.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:05.986+0000 7fa5df79d640 1 -- 172.21.15.45:0/3181744132 wait complete. 2024-09-18T14:06:06.028 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:06.028 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:06:06.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:06:06.072 INFO:tasks.rados.rados.0.smithi084.stdout:1473: done (2 left) 2024-09-18T14:06:06.072 INFO:tasks.rados.rados.0.smithi084.stdout:1475: done (1 left) 2024-09-18T14:06:06.072 INFO:tasks.rados.rados.0.smithi084.stdout:1476: done (0 left) 2024-09-18T14:06:06.072 INFO:tasks.rados.rados.0.smithi084.stdout:1477: write (excl) oid 4 current snap is 159 2024-09-18T14:06:06.072 INFO:tasks.rados.rados.0.smithi084.stdout:1477: seq_num 518 ranges {706026=490986,1828849=790505} 2024-09-18T14:06:06.078 INFO:tasks.rados.rados.0.smithi084.stdout:1477: writing smithi08434663-4 from 706026 to 1197012 tid 1 2024-09-18T14:06:06.086 INFO:tasks.rados.rados.0.smithi084.stdout:1477: writing smithi08434663-4 from 1828849 to 2619354 tid 2 2024-09-18T14:06:06.086 INFO:tasks.rados.rados.0.smithi084.stdout:1478: snap_create 2024-09-18T14:06:06.086 INFO:tasks.rados.rados.0.smithi084.stdout:1477: finishing write tid 1 to smithi08434663-4 2024-09-18T14:06:06.090 INFO:tasks.rados.rados.0.smithi084.stdout:1477: finishing write tid 2 to smithi08434663-4 2024-09-18T14:06:06.090 INFO:tasks.rados.rados.0.smithi084.stdout:1477: finishing write tid 3 to smithi08434663-4 2024-09-18T14:06:06.091 INFO:tasks.rados.rados.0.smithi084.stdout:1477: finishing write tid 4 to smithi08434663-4 2024-09-18T14:06:06.091 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 724 (ObjNum 518 snap 159 seq_num 518) dirty exists 2024-09-18T14:06:06.091 INFO:tasks.rados.rados.0.smithi084.stdout:1477: left oid 4 (ObjNum 518 snap 159 seq_num 518) 2024-09-18T14:06:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.132+0000 7f6bad4c0640 10 monclient: get_monmap_and_config 2024-09-18T14:06:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.132+0000 7f6bad4c0640 10 monclient: build_initial_monmap 2024-09-18T14:06:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.132+0000 7f6bad4c0640 10 monclient: monmap: 2024-09-18T14:06:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:06.133349+0000 2024-09-18T14:06:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:06.133349+0000 2024-09-18T14:06:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:06.134 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 1 Processor -- start 2024-09-18T14:06:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 1 -- start start 2024-09-18T14:06:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 10 monclient: init 2024-09-18T14:06:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 10 monclient: _add_conns ranks=[8,3,0,2,6,7,5,4,1] 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 10 monclient(hunting): picked mon.noname-e con 0x7f6ba809f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 10 monclient(hunting): picked mon.noname-g con 0x7f6ba809fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 10 monclient(hunting): picked mon.noname-a con 0x7f6ba809ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba805ed70 con 0x7f6ba809ff70 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba809dfc0 con 0x7f6ba809fba0 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba809dce0 con 0x7f6ba809f7d0 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6bad4c0640 10 monclient(hunting): authenticate will time out at 1181.471670s 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6ba77fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6ba809ff70 0x7f6ba813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33712/0 (socket says 172.21.15.45:33712) 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6ba77fe640 1 -- 172.21.15.45:0/2573532791 learned_addr learned my addr 172.21.15.45:0/2573532791 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.133+0000 7f6ba6ffd640 1 --1- 172.21.15.45:0/2573532791 >> v1:172.21.15.45:6793/0 conn(0x7f6ba809f7d0 0x7f6ba8132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:39894/0 (socket says 172.21.15.45:39894) 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625150265 0 0) 0x7f6ba805ed70 con 0x7f6ba809ff70 2024-09-18T14:06:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient(hunting): my global_id is 5238 2024-09-18T14:06:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b940034e0 con 0x7f6ba809ff70 2024-09-18T14:06:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704111303 0 0) 0x7f6ba809dce0 con 0x7f6ba809f7d0 2024-09-18T14:06:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient(hunting): my global_id is 5156 2024-09-18T14:06:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba805ed70 con 0x7f6ba809f7d0 2024-09-18T14:06:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964030023 0 0) 0x7f6ba809dfc0 con 0x7f6ba809fba0 2024-09-18T14:06:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient(hunting): my global_id is 154209 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba809dce0 con 0x7f6ba809fba0 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3422800707 0 0) 0x7f6b940034e0 con 0x7f6ba809ff70 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 >> v1:172.21.15.45:6793/0 conn(0x7f6ba809f7d0 legacy=0x7f6ba8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 >> v1:172.21.15.84:6790/0 conn(0x7f6ba809fba0 legacy=0x7f6ba8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient: _finish_hunting 0 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 monclient: found mon.noname-a 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba80a1c00 con 0x7f6ba809ff70 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient: _finish_auth 0 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6bad4c0640 5 monclient: authenticate success, global_id 5238 2024-09-18T14:06:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6ba0001bb0 con 0x7f6ba809ff70 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient: dump: 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6bad4c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6ba0001d90 con 0x7f6ba809ff70 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 1 -- 172.21.15.45:0/2573532791 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6ba0004f20 con 0x7f6ba809ff70 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6bad4c0640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6bad4c0640 4 set_mon_vals no callback set 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6ba67fc640 10 monclient: dump: 2024-09-18T14:06:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6bad4c0640 10 monclient: shutdown 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6bad4c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.134+0000 7f6bad4c0640 1 -- 172.21.15.45:0/2573532791 >> v1:172.21.15.45:6789/0 conn(0x7f6ba809ff70 legacy=0x7f6ba813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.135+0000 7f6bad4c0640 1 -- 172.21.15.45:0/2573532791 shutdown_connections 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.135+0000 7f6bad4c0640 1 -- 172.21.15.45:0/2573532791 >> 172.21.15.45:0/2573532791 conn(0x7f6ba80630b0 msgr2=0x7f6ba80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.135+0000 7f6bad4c0640 1 -- 172.21.15.45:0/2573532791 shutdown_connections 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.135+0000 7f6bad4c0640 1 -- 172.21.15.45:0/2573532791 wait complete. 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.135+0000 7f6bad4c0640 10 monclient: build_initial_monmap 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.135+0000 7f6bad4c0640 10 monclient: monmap: 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.135+0000 7f6bad4c0640 1 Processor -- start 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.135+0000 7f6bad4c0640 1 -- start start 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.135+0000 7f6bad4c0640 10 monclient: init 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6bad4c0640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6bad4c0640 10 monclient: _add_conns ranks=[3,4,0,5,8,6,1,7,2] 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6bad4c0640 10 monclient(hunting): picked mon.noname-g con 0x7f6ba809f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6bad4c0640 10 monclient(hunting): picked mon.noname-c con 0x7f6ba809fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6bad4c0640 10 monclient(hunting): picked mon.noname-a con 0x7f6ba809ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6bad4c0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba81428e0 con 0x7f6ba809ff70 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6bad4c0640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba80a2560 con 0x7f6ba809f7d0 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6bad4c0640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba809b800 con 0x7f6ba809fba0 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6bad4c0640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6bad4c0640 10 monclient(hunting): authenticate will time out at 1181.474519s 2024-09-18T14:06:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9ffff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f6ba809fba0 0x7f6ba8146cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58542/0 (socket says 172.21.15.45:58542) 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6ba77fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6ba809ff70 0x7f6ba814a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33718/0 (socket says 172.21.15.45:33718) 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9ffff640 1 -- 172.21.15.45:0/454896019 learned_addr learned my addr 172.21.15.45:0/454896019 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024497986 0 0) 0x7f6ba81428e0 con 0x7f6ba809ff70 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 10 monclient(hunting): my global_id is 5247 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b880034e0 con 0x7f6ba809ff70 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430438916 0 0) 0x7f6ba809b800 con 0x7f6ba809fba0 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 10 monclient(hunting): my global_id is 154228 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba81428e0 con 0x7f6ba809fba0 2024-09-18T14:06:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2841369173 0 0) 0x7f6ba80a2560 con 0x7f6ba809f7d0 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 10 monclient(hunting): my global_id is 154218 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.136+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba809b800 con 0x7f6ba809f7d0 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2215571129 0 0) 0x7f6b880034e0 con 0x7f6ba809ff70 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ba80a2560 con 0x7f6ba809ff70 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6ba0001af0 con 0x7f6ba809ff70 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient(hunting): dump: 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3566535605 0 0) 0x7f6ba809b800 con 0x7f6ba809f7d0 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6b880034e0 con 0x7f6ba809f7d0 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3596588779 0 0) 0x7f6ba81428e0 con 0x7f6ba809fba0 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ba809b800 con 0x7f6ba809fba0 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6b98001550 con 0x7f6ba809f7d0 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient(hunting): dump: 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6b90001b20 con 0x7f6ba809fba0 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:06:06.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient(hunting): dump: 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 723805852 0 0) 0x7f6ba80a2560 con 0x7f6ba809ff70 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 >> v1:172.21.15.45:6791/0 conn(0x7f6ba809fba0 legacy=0x7f6ba8146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 >> v1:172.21.15.84:6790/0 conn(0x7f6ba809f7d0 legacy=0x7f6ba8131670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient: _finish_hunting 0 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 monclient: found mon.a 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba811e790 con 0x7f6ba809ff70 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient: _finish_auth 0 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6bad4c0640 5 monclient: authenticate success, global_id 5247 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6bad4c0640 10 monclient: _renew_subs 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6bad4c0640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6bad4c0640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba811ea70 con 0x7f6ba809ff70 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6bad4c0640 10 monclient: _renew_subs 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6bad4c0640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6bad4c0640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba811d4a0 con 0x7f6ba809ff70 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6ba0004ed0 con 0x7f6ba809ff70 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6ba0005440 con 0x7f6ba809ff70 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6ba5ffb640 4 set_mon_vals no callback set 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.137+0000 7f6b9effd640 10 monclient: dump: 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.138+0000 7f6bad4c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.138+0000 7f6bad4c0640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6b8c006580 con 0x7f6b8c002830 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.138+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f6ba00055a0 con 0x7f6ba809ff70 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.138+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(329..329 src has 126..329) v4 ==== 3911+0+0 (unknown 3883595493 0 0) 0x7f6ba0046e40 con 0x7f6ba809ff70 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.138+0000 7f6b9ffff640 10 monclient: get_auth_request con 0x7f6b8803a920 auth_method 0 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.138+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843439357 0 0) 0x7f6b8c006580 con 0x7f6b8c002830 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.138+0000 7f6b9effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.138+0000 7f6b9effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.138+0000 7f6b9effd640 10 monclient(hunting): my global_id is 5182 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.138+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba80a2560 con 0x7f6b8c002830 2024-09-18T14:06:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.138+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2369830494 0 0) 0x7f6ba80a2560 con 0x7f6b8c002830 2024-09-18T14:06:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.139+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6b8c006580 con 0x7f6b8c002830 2024-09-18T14:06:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.139+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6b98004d90 con 0x7f6b8c002830 2024-09-18T14:06:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.139+0000 7f6b9effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.139+0000 7f6b9effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.139+0000 7f6b9effd640 10 monclient: dump: 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.166 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.139+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3798773436 0 0) 0x7f6b8c006580 con 0x7f6b8c002830 2024-09-18T14:06:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.139+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b8c00a0a0 con 0x7f6b8c002830 2024-09-18T14:06:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.139+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6b98005340 con 0x7f6b8c002830 2024-09-18T14:06:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.139+0000 7f6b9effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.139+0000 7f6b9effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.139+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 >> v1:172.21.15.84:6792/0 conn(0x7f6b8c002830 legacy=0x7f6b8c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.139+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 >> v1:172.21.15.84:6792/0 conn(0x7f6b8c002830 legacy=0x7f6b8c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6bad4c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6bad4c0640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6b8c009660 con 0x7f6b8c009040 2024-09-18T14:06:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422897986 0 0) 0x7f6b8c009660 con 0x7f6b8c009040 2024-09-18T14:06:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6b9effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6b9effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6b9effd640 10 monclient(hunting): my global_id is 5191 2024-09-18T14:06:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b8c006580 con 0x7f6b8c009040 2024-09-18T14:06:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1046902035 0 0) 0x7f6b8c006580 con 0x7f6b8c009040 2024-09-18T14:06:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6b8c009660 con 0x7f6b8c009040 2024-09-18T14:06:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6b98005aa0 con 0x7f6b8c009040 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6b9effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6b9effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.145+0000 7f6b9effd640 10 monclient: dump: 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.170 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 259003049 0 0) 0x7f6b8c009660 con 0x7f6b8c009040 2024-09-18T14:06:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6b8c009920 con 0x7f6b8c009040 2024-09-18T14:06:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 4026449795) 0x7f6b98007cf0 con 0x7f6b8c009040 2024-09-18T14:06:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6b9effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6b9effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 >> v1:172.21.15.84:6792/0 conn(0x7f6b8c009040 legacy=0x7f6b8c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6b9effd640 1 -- 172.21.15.45:0/454896019 >> v1:172.21.15.84:6792/0 conn(0x7f6b8c009040 legacy=0x7f6b8c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 76, 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.180 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:06.180 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:06.180 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.180 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.180 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.180 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:06.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:06.180 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.180 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:06.182 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:06.183 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:06:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:06.190 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6bad4c0640 1 -- 172.21.15.45:0/454896019 >> v1:172.21.15.84:6812/31478 conn(0x7f6b8803a920 legacy=0x7f6b8803cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6bad4c0640 10 monclient: shutdown 2024-09-18T14:06:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6bad4c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6bad4c0640 1 -- 172.21.15.45:0/454896019 >> v1:172.21.15.45:6789/0 conn(0x7f6ba809ff70 legacy=0x7f6ba814a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6ba77fe640 1 -- 172.21.15.45:0/454896019 reap_dead start 2024-09-18T14:06:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6ba77fe640 1 -- 172.21.15.45:0/454896019 reap_dead start 2024-09-18T14:06:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6bad4c0640 1 -- 172.21.15.45:0/454896019 shutdown_connections 2024-09-18T14:06:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6bad4c0640 1 -- 172.21.15.45:0/454896019 >> 172.21.15.45:0/454896019 conn(0x7f6ba80630b0 msgr2=0x7f6ba80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6bad4c0640 1 -- 172.21.15.45:0/454896019 shutdown_connections 2024-09-18T14:06:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.146+0000 7f6bad4c0640 1 -- 172.21.15.45:0/454896019 wait complete. 2024-09-18T14:06:06.192 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:06:06.193 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c', 'f'] 2024-09-18T14:06:06.193 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-18T14:06:06.193 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-18T14:06:06.193 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-18T14:06:06.193 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T14:06:06.193 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:06:06.203 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T14:06:06.203 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-18T14:06:06.203 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-18T14:06:06.203 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-18T14:06:06.203 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:06:06.216 INFO:tasks.ceph.mon.f:Stopped 2024-09-18T14:06:06.216 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-18T14:06:06.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:06:06.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.338+0000 7f0a1c26f640 10 monclient: get_monmap_and_config 2024-09-18T14:06:06.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.338+0000 7f0a1c26f640 10 monclient: build_initial_monmap 2024-09-18T14:06:06.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.338+0000 7f0a1c26f640 10 monclient: monmap: 2024-09-18T14:06:06.339 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:06.339 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.339 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:06.339499+0000 2024-09-18T14:06:06.339 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:06.339499+0000 2024-09-18T14:06:06.339 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:06.339 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.339 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:06.339 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:06.339 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:06.340 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.339+0000 7f0a1c26f640 1 Processor -- start 2024-09-18T14:06:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.339+0000 7f0a1c26f640 1 -- start start 2024-09-18T14:06:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.339+0000 7f0a1c26f640 10 monclient: init 2024-09-18T14:06:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.339+0000 7f0a1c26f640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.339+0000 7f0a1c26f640 10 monclient: _add_conns ranks=[4,6,2,0,8,7,5,1,3] 2024-09-18T14:06:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.339+0000 7f0a1c26f640 10 monclient(hunting): picked mon.noname-c con 0x7f0a1409f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.339+0000 7f0a1c26f640 10 monclient(hunting): picked mon.noname-d con 0x7f0a1409fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.339+0000 7f0a1c26f640 10 monclient(hunting): picked mon.noname-b con 0x7f0a1409ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.339+0000 7f0a1c26f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1405ed70 con 0x7f0a1409ff70 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.339+0000 7f0a1a00b640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f0a1409f7d0 legacy=0x7f0a14132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.339+0000 7f0a1c26f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1409dfc0 con 0x7f0a1409f7d0 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.339+0000 7f0a1c26f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1409dce0 con 0x7f0a1409fba0 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a1c26f640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a1c26f640 10 monclient(hunting): authenticate will time out at 1181.678020s 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a1980a640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f0a1409fba0 0x7f0a14137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:54918/0 (socket says 172.21.15.45:54918) 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a1980a640 1 -- 172.21.15.45:0/3523958667 learned_addr learned my addr 172.21.15.45:0/3523958667 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a1a80c640 1 --1- 172.21.15.45:0/3523958667 >> v1:172.21.15.45:6790/0 conn(0x7f0a1409ff70 0x7f0a1413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39502/0 (socket says 172.21.15.45:39502) 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 1 -- 172.21.15.45:0/3523958667 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435239634 0 0) 0x7f0a1409dce0 con 0x7f0a1409fba0 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient(hunting): my global_id is 5100 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 1 -- 172.21.15.45:0/3523958667 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a000034e0 con 0x7f0a1409fba0 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 1 -- 172.21.15.45:0/3523958667 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548206197 0 0) 0x7f0a1405ed70 con 0x7f0a1409ff70 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient(hunting): my global_id is 144479 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 1 -- 172.21.15.45:0/3523958667 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a1409dce0 con 0x7f0a1409ff70 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 1 -- 172.21.15.45:0/3523958667 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3926966636 0 0) 0x7f0a000034e0 con 0x7f0a1409fba0 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 1 -- 172.21.15.45:0/3523958667 >> v1:172.21.15.45:6791/0 conn(0x7f0a1409f7d0 legacy=0x7f0a14132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 1 -- 172.21.15.45:0/3523958667 >> v1:172.21.15.45:6790/0 conn(0x7f0a1409ff70 legacy=0x7f0a1413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient: _finish_hunting 0 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 1 monclient: found mon.noname-d 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:06:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 1 -- 172.21.15.45:0/3523958667 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a140a1c00 con 0x7f0a1409fba0 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient: _finish_auth 0 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a1c26f640 5 monclient: authenticate success, global_id 5100 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 1 -- 172.21.15.45:0/3523958667 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0a04001830 con 0x7f0a1409fba0 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a1c26f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient: dump: 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a1c26f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 1 -- 172.21.15.45:0/3523958667 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0a04001a10 con 0x7f0a1409fba0 2024-09-18T14:06:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a1c26f640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 1 -- 172.21.15.45:0/3523958667 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0a04004ba0 con 0x7f0a1409fba0 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a1c26f640 4 set_mon_vals no callback set 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.340+0000 7f0a19009640 10 monclient: dump: 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.353 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.353 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.353 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.353 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.353 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.353 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.353 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.353 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.341+0000 7f0a1c26f640 10 monclient: shutdown 2024-09-18T14:06:06.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.341+0000 7f0a1c26f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.341+0000 7f0a1c26f640 1 -- 172.21.15.45:0/3523958667 >> v1:172.21.15.45:6792/0 conn(0x7f0a1409fba0 legacy=0x7f0a14137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.341+0000 7f0a1c26f640 1 -- 172.21.15.45:0/3523958667 shutdown_connections 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.341+0000 7f0a1c26f640 1 -- 172.21.15.45:0/3523958667 >> 172.21.15.45:0/3523958667 conn(0x7f0a140630b0 msgr2=0x7f0a140634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.341+0000 7f0a1c26f640 1 -- 172.21.15.45:0/3523958667 shutdown_connections 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.341+0000 7f0a1c26f640 1 -- 172.21.15.45:0/3523958667 wait complete. 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.341+0000 7f0a1c26f640 10 monclient: build_initial_monmap 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.341+0000 7f0a1c26f640 10 monclient: monmap: 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 1 Processor -- start 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 1 -- start start 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 10 monclient: init 2024-09-18T14:06:06.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 10 monclient: _add_conns ranks=[1,0,7,8,5,4,2,3,6] 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 10 monclient(hunting): picked mon.noname-f con 0x7f0a1409f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 10 monclient(hunting): picked mon.noname-a con 0x7f0a1409fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 10 monclient(hunting): picked mon.noname-i con 0x7f0a1409ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1409d310 con 0x7f0a1409fba0 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a140a2560 con 0x7f0a1409f7d0 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a14120600 con 0x7f0a1409ff70 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1c26f640 10 monclient(hunting): authenticate will time out at 1181.680570s 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1980a640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0a1409fba0 0x7f0a14146d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33730/0 (socket says 172.21.15.45:33730) 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1980a640 1 -- 172.21.15.45:0/959673671 learned_addr learned my addr 172.21.15.45:0/959673671 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a1a00b640 1 -- 172.21.15.45:0/959673671 >> v1:172.21.15.84:6789/0 conn(0x7f0a1409f7d0 legacy=0x7f0a141314c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916461500 0 0) 0x7f0a1409d310 con 0x7f0a1409fba0 2024-09-18T14:06:06.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a0a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a0a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a0a7fc640 10 monclient(hunting): my global_id is 5256 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.342+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f80034e0 con 0x7f0a1409fba0 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4000195409 0 0) 0x7f09f80034e0 con 0x7f0a1409fba0 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a1409d310 con 0x7f0a1409fba0 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0a040018b0 con 0x7f0a1409fba0 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient(hunting): dump: 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734881323 0 0) 0x7f0a14120600 con 0x7f0a1409ff70 2024-09-18T14:06:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:06.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:06.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient(hunting): my global_id is 5200 2024-09-18T14:06:06.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f80034e0 con 0x7f0a1409ff70 2024-09-18T14:06:06.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 456195930 0 0) 0x7f0a1409d310 con 0x7f0a1409fba0 2024-09-18T14:06:06.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 >> v1:172.21.15.84:6792/0 conn(0x7f0a1409ff70 legacy=0x7f0a1414a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 >> v1:172.21.15.84:6789/0 conn(0x7f0a1409f7d0 legacy=0x7f0a141314c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:06:06.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:06:06.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 1 monclient: found mon.a 2024-09-18T14:06:06.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a141208e0 con 0x7f0a1409fba0 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a1c26f640 5 monclient: authenticate success, global_id 5256 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a1c26f640 10 monclient: _renew_subs 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a1c26f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a1c26f640 1 -- 172.21.15.45:0/959673671 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1411ead0 con 0x7f0a1409fba0 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a1c26f640 10 monclient: _renew_subs 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a1c26f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a1c26f640 1 -- 172.21.15.45:0/959673671 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1411f6a0 con 0x7f0a1409fba0 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0a04004c10 con 0x7f0a1409fba0 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:06.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0a040051a0 con 0x7f0a1409fba0 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0a7fc640 10 monclient: dump: 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:06.361 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.343+0000 7f0a0bfff640 4 set_mon_vals no callback set 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.344+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f0a040101f0 con 0x7f0a1409fba0 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.344+0000 7f0a1c26f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.344+0000 7f0a1c26f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.344+0000 7f0a1c26f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.344+0000 7f0a1c26f640 1 -- 172.21.15.45:0/959673671 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1404c4a0 con 0x7f0a1409fba0 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.344+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(329..329 src has 126..329) v4 ==== 3911+0+0 (unknown 3883595493 0 0) 0x7f0a04046d40 con 0x7f0a1409fba0 2024-09-18T14:06:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.344+0000 7f0a1a00b640 10 monclient: get_auth_request con 0x7f09f8039cd0 auth_method 0 2024-09-18T14:06:06.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.348+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f0a04015580 con 0x7f0a1409fba0 2024-09-18T14:06:06.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.348+0000 7f0a0a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:06.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.348+0000 7f0a0a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.579+0000 7f0a1c26f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:06:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.579+0000 7f0a1c26f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:06:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.579+0000 7f0a1c26f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.579+0000 7f0a1c26f640 1 -- 172.21.15.45:0/959673671 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0a1404b720 con 0x7f0a1409fba0 2024-09-18T14:06:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.579+0000 7f0a0a7fc640 1 -- 172.21.15.45:0/959673671 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 426614580) 0x7f0a04014f80 con 0x7f0a1409fba0 2024-09-18T14:06:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.579+0000 7f0a0a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:06:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.579+0000 7f0a0a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:06.580 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":76,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":5,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:06:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.581+0000 7f0a1c26f640 1 -- 172.21.15.45:0/959673671 >> v1:172.21.15.84:6812/31478 conn(0x7f09f8039cd0 legacy=0x7f09f803c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.581+0000 7f0a1c26f640 10 monclient: shutdown 2024-09-18T14:06:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.581+0000 7f0a1c26f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.581+0000 7f0a1c26f640 1 -- 172.21.15.45:0/959673671 >> v1:172.21.15.45:6789/0 conn(0x7f0a1409fba0 legacy=0x7f0a14146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.581+0000 7f0a1c26f640 1 -- 172.21.15.45:0/959673671 shutdown_connections 2024-09-18T14:06:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.581+0000 7f0a1c26f640 1 -- 172.21.15.45:0/959673671 >> 172.21.15.45:0/959673671 conn(0x7f0a140630b0 msgr2=0x7f0a140a4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.582+0000 7f0a1c26f640 1 -- 172.21.15.45:0/959673671 shutdown_connections 2024-09-18T14:06:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:06.582+0000 7f0a1c26f640 1 -- 172.21.15.45:0/959673671 wait complete. 2024-09-18T14:06:09.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:06:09.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.701+0000 7f709d99d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:09.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.701+0000 7f709d99d640 10 monclient: build_initial_monmap 2024-09-18T14:06:09.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.701+0000 7f709d99d640 10 monclient: monmap: 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:09.702291+0000 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:09.702291+0000 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:09.702 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:09.703 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:09.703 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:09.703 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:09.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.701+0000 7f709d99d640 1 Processor -- start 2024-09-18T14:06:09.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f709d99d640 1 -- start start 2024-09-18T14:06:09.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f709d99d640 10 monclient: init 2024-09-18T14:06:09.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f709d99d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:09.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f709d99d640 10 monclient: _add_conns ranks=[3,1,6,5,2,7,8,4,0] 2024-09-18T14:06:09.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f709d99d640 10 monclient(hunting): picked mon.noname-g con 0x7f709809f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:09.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f709d99d640 10 monclient(hunting): picked mon.noname-f con 0x7f709809fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:09.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f709d99d640 10 monclient(hunting): picked mon.noname-d con 0x7f709809ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:09.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f709d99d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709805ed70 con 0x7f709809fba0 2024-09-18T14:06:09.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f709d99d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709809dfc0 con 0x7f709809f7d0 2024-09-18T14:06:09.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f709d99d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709809dce0 con 0x7f709809ff70 2024-09-18T14:06:09.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f709d99d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:09.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f70967fc640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7f709809fba0 legacy=0x7f7098137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:09.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f709d99d640 10 monclient(hunting): authenticate will time out at 1185.040768s 2024-09-18T14:06:09.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f70977fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f709809ff70 0x7f709813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:44798/0 (socket says 172.21.15.45:44798) 2024-09-18T14:06:09.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.702+0000 7f70977fe640 1 -- 172.21.15.45:0/452714304 learned_addr learned my addr 172.21.15.45:0/452714304 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:09.903+0000 7f70967fc640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6789/0 conn(0x7f709809fba0 legacy=0x7f7098137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:10.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:10.304+0000 7f70967fc640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6789/0 conn(0x7f709809fba0 legacy=0x7f7098137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:11.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:11.105+0000 7f70967fc640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6789/0 conn(0x7f709809fba0 legacy=0x7f7098137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 10 monclient(hunting): tick 2024-09-18T14:06:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 1 monclient(hunting): continuing hunt 2024-09-18T14:06:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:06:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.45:6792/0 conn(0x7f709809ff70 legacy=0x7f709813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6790/0 conn(0x7f709809f7d0 legacy=0x7f7098132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6789/0 conn(0x7f709809fba0 legacy=0x7f7098137a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:06:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 10 monclient: _add_conns ranks=[5,2,8,4,7,0] 2024-09-18T14:06:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f7084003780 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f7084007360 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f708400b0a0 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70980a1c00 con 0x7f7084007360 2024-09-18T14:06:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709805ed70 con 0x7f7084003780 2024-09-18T14:06:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7084010f90 con 0x7f708400b0a0 2024-09-18T14:06:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:12.702+0000 7f7094ff9640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 10 monclient(hunting): tick 2024-09-18T14:06:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 1 monclient(hunting): continuing hunt 2024-09-18T14:06:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:06:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.45:6793/0 conn(0x7f708400b0a0 legacy=0x7f708400d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6791/0 conn(0x7f7084003780 legacy=0x7f7084005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.45:6790/0 conn(0x7f7084007360 legacy=0x7f70840097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 10 monclient: _add_conns ranks=[7,4,0] 2024-09-18T14:06:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f708400f010 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f70977fe640 1 -- 172.21.15.45:0/452714304 reap_dead start 2024-09-18T14:06:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f7084011120 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f7084011c30 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f70967fc640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.45:6791/0 conn(0x7f7084011120 legacy=0x7f7084011520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:06:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70840121f0 con 0x7f7084011c30 2024-09-18T14:06:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708400dd70 con 0x7f7084011120 2024-09-18T14:06:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708400dfa0 con 0x7f708400f010 2024-09-18T14:06:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.703+0000 7f7094ff9640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:15.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:15.904+0000 7f70967fc640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.45:6791/0 conn(0x7f7084011120 legacy=0x7f7084011520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:06:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:16.305+0000 7f70967fc640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.45:6791/0 conn(0x7f7084011120 legacy=0x7f7084011520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:06:17.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:17.106+0000 7f70967fc640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.45:6791/0 conn(0x7f7084011120 legacy=0x7f7084011520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:06:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 10 monclient(hunting): tick 2024-09-18T14:06:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 1 monclient(hunting): continuing hunt 2024-09-18T14:06:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:06:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6792/0 conn(0x7f708400f010 legacy=0x7f708400f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.45:6791/0 conn(0x7f7084011120 legacy=0x7f7084011520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:06:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.45:6789/0 conn(0x7f7084011c30 legacy=0x7f70840097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:18.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 10 monclient: _add_conns ranks=[5,2,1,0,7,3,6,8,4] 2024-09-18T14:06:18.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f7084005ec0 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:18.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f708400e8e0 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:18.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f7084018550 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:18.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7084005c70 con 0x7f7084018550 2024-09-18T14:06:18.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708400dd70 con 0x7f708400e8e0 2024-09-18T14:06:18.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 1 -- 172.21.15.45:0/452714304 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708401e300 con 0x7f7084005ec0 2024-09-18T14:06:18.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f7094ff9640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:18.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.703+0000 7f70977fe640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6789/0 conn(0x7f7084018550 legacy=0x7f708401a9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:18.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:18.904+0000 7f70977fe640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6789/0 conn(0x7f7084018550 legacy=0x7f708401a9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:19.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:19.305+0000 7f70977fe640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6789/0 conn(0x7f7084018550 legacy=0x7f708401a9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:20.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:20.106+0000 7f70977fe640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6789/0 conn(0x7f7084018550 legacy=0x7f708401a9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 1 -- 172.21.15.45:0/452714304 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446813548 0 0) 0x7f708400dd70 con 0x7f708400e8e0 2024-09-18T14:06:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 10 monclient(hunting): my global_id is 144488 2024-09-18T14:06:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 1 -- 172.21.15.45:0/452714304 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70800034e0 con 0x7f708400e8e0 2024-09-18T14:06:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 1 -- 172.21.15.45:0/452714304 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 691271157 0 0) 0x7f70800034e0 con 0x7f708400e8e0 2024-09-18T14:06:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6791/0 conn(0x7f7084005ec0 legacy=0x7f708400e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.84:6789/0 conn(0x7f7084018550 legacy=0x7f708401a9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:06:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:06:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 1 monclient: found mon.noname-b 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 1 -- 172.21.15.45:0/452714304 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708401f540 con 0x7f708400e8e0 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 10 monclient: _finish_auth 0 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f70977fe640 1 -- 172.21.15.45:0/452714304 reap_dead start 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 1 -- 172.21.15.45:0/452714304 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f708c0042f0 con 0x7f708400e8e0 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f7095ffb640 10 monclient: dump: 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f709d99d640 5 monclient: authenticate success, global_id 144488 2024-09-18T14:06:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.082+0000 7f709d99d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.083+0000 7f7095ffb640 1 -- 172.21.15.45:0/452714304 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f708c004450 con 0x7f708400e8e0 2024-09-18T14:06:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.083+0000 7f7095ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.083+0000 7f7095ffb640 1 -- 172.21.15.45:0/452714304 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f708c004a40 con 0x7f708400e8e0 2024-09-18T14:06:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.083+0000 7f709d99d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.083+0000 7f709d99d640 4 set_mon_vals no callback set 2024-09-18T14:06:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.083+0000 7f7095ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.083+0000 7f7095ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.083+0000 7f7095ffb640 10 monclient: dump: 2024-09-18T14:06:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.088 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.084+0000 7f709d99d640 10 monclient: shutdown 2024-09-18T14:06:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.084+0000 7f709d99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.084+0000 7f709d99d640 1 -- 172.21.15.45:0/452714304 >> v1:172.21.15.45:6790/0 conn(0x7f708400e8e0 legacy=0x7f708400d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.084+0000 7f709d99d640 1 -- 172.21.15.45:0/452714304 shutdown_connections 2024-09-18T14:06:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.084+0000 7f709d99d640 1 -- 172.21.15.45:0/452714304 >> 172.21.15.45:0/452714304 conn(0x7f70980630b0 msgr2=0x7f70980634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.084+0000 7f709d99d640 1 -- 172.21.15.45:0/452714304 shutdown_connections 2024-09-18T14:06:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.084+0000 7f709d99d640 1 -- 172.21.15.45:0/452714304 wait complete. 2024-09-18T14:06:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.085+0000 7f709d99d640 10 monclient: build_initial_monmap 2024-09-18T14:06:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.085+0000 7f709d99d640 10 monclient: monmap: 2024-09-18T14:06:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.086+0000 7f709d99d640 1 Processor -- start 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.086+0000 7f709d99d640 1 -- start start 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.086+0000 7f709d99d640 10 monclient: init 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.086+0000 7f709d99d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.086+0000 7f709d99d640 10 monclient: _add_conns ranks=[6,5,7,8,2,1,4,0,3] 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.086+0000 7f709d99d640 10 monclient(hunting): picked mon.noname-d con 0x7f708400e8e0 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.086+0000 7f709d99d640 10 monclient(hunting): picked mon.noname-h con 0x7f7098131930 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.086+0000 7f709d99d640 10 monclient(hunting): picked mon.noname-i con 0x7f7098131d00 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.086+0000 7f709d99d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70981428c0 con 0x7f7098131930 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.087+0000 7f709d99d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7098120710 con 0x7f708400e8e0 2024-09-18T14:06:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.087+0000 7f709d99d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709809d310 con 0x7f7098131d00 2024-09-18T14:06:21.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.087+0000 7f709d99d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:21.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.087+0000 7f709d99d640 10 monclient(hunting): authenticate will time out at 1196.425134s 2024-09-18T14:06:21.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.087+0000 7f7096ffd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f708400e8e0 0x7f70981419e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34004/0 (socket says 172.21.15.45:34004) 2024-09-18T14:06:21.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.087+0000 7f7096ffd640 1 -- 172.21.15.45:0/1580059034 learned_addr learned my addr 172.21.15.45:0/1580059034 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:21.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.092+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071023506 0 0) 0x7f708401f540 con 0x7f7098131930 2024-09-18T14:06:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.092+0000 7f707effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.092+0000 7f707effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.092+0000 7f707effd640 10 monclient(hunting): my global_id is 5072 2024-09-18T14:06:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.092+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f708400ee00 con 0x7f7098131930 2024-09-18T14:06:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.092+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730241338 0 0) 0x7f7098120710 con 0x7f708400e8e0 2024-09-18T14:06:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.092+0000 7f707effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.092+0000 7f707effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.092+0000 7f707effd640 10 monclient(hunting): my global_id is 5109 2024-09-18T14:06:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.092+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f708401f540 con 0x7f708400e8e0 2024-09-18T14:06:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1548163876 0 0) 0x7f708401f540 con 0x7f708400e8e0 2024-09-18T14:06:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7098120710 con 0x7f708400e8e0 2024-09-18T14:06:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3526526388 0 0) 0x7f708400ee00 con 0x7f7098131930 2024-09-18T14:06:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f708401f540 con 0x7f7098131930 2024-09-18T14:06:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7088001ae0 con 0x7f708400e8e0 2024-09-18T14:06:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:06:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient(hunting): dump: 2024-09-18T14:06:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f708c004900 con 0x7f7098131930 2024-09-18T14:06:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient(hunting): dump: 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.101 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2054237094 0 0) 0x7f7098120710 con 0x7f708400e8e0 2024-09-18T14:06:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 >> v1:172.21.15.84:6792/0 conn(0x7f7098131d00 legacy=0x7f709814a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 >> v1:172.21.15.84:6791/0 conn(0x7f7098131930 legacy=0x7f7098146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient: _finish_hunting 0 2024-09-18T14:06:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 monclient: found mon.d 2024-09-18T14:06:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:06:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70980a2560 con 0x7f708400e8e0 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient: _finish_auth 0 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f709d99d640 5 monclient: authenticate success, global_id 5109 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f709d99d640 10 monclient: _renew_subs 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f709d99d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f709d99d640 1 -- 172.21.15.45:0/1580059034 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7098120b20 con 0x7f708400e8e0 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f709d99d640 10 monclient: _renew_subs 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f709d99d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f709d99d640 1 -- 172.21.15.45:0/1580059034 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f709811ff50 con 0x7f708400e8e0 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7088002d90 con 0x7f708400e8e0 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f70880022f0 con 0x7f708400e8e0 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f70957fa640 4 set_mon_vals no callback set 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.093+0000 7f707effd640 10 monclient: dump: 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.104 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.094+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7088002450 con 0x7f708400e8e0 2024-09-18T14:06:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.094+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(329..329 src has 126..329) v4 ==== 3911+0+0 (unknown 3883595493 0 0) 0x7f7088046a50 con 0x7f708400e8e0 2024-09-18T14:06:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.094+0000 7f709d99d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.094+0000 7f709d99d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.094+0000 7f709d99d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:06:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.094+0000 7f709d99d640 1 -- 172.21.15.45:0/1580059034 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7078005010 con 0x7f708400e8e0 2024-09-18T14:06:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.094+0000 7f70967fc640 10 monclient: get_auth_request con 0x7f708403c700 auth_method 0 2024-09-18T14:06:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.099+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f7088015230 con 0x7f708400e8e0 2024-09-18T14:06:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.099+0000 7f707effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.099+0000 7f707effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:21.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.328+0000 7f709d99d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:06:21.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.328+0000 7f709d99d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:06:21.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.328+0000 7f709d99d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:06:21.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.328+0000 7f709d99d640 1 -- 172.21.15.45:0/1580059034 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7078005840 con 0x7f708400e8e0 2024-09-18T14:06:21.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.328+0000 7f707effd640 1 -- 172.21.15.45:0/1580059034 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2676 (unknown 254746657 0 762696702) 0x7f7088014c30 con 0x7f708400e8e0 2024-09-18T14:06:21.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.328+0000 7f707effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:06:21.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.328+0000 7f707effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:21.330 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":80,"quorum":[0,2,3,5,6,7,8],"quorum_names":["a","b","g","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:06:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.330+0000 7f709d99d640 1 -- 172.21.15.45:0/1580059034 >> v1:172.21.15.84:6812/31478 conn(0x7f708403c700 legacy=0x7f708403ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.330+0000 7f709d99d640 10 monclient: shutdown 2024-09-18T14:06:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.330+0000 7f709d99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.330+0000 7f709d99d640 1 -- 172.21.15.45:0/1580059034 >> v1:172.21.15.45:6792/0 conn(0x7f708400e8e0 legacy=0x7f70981419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.330+0000 7f709d99d640 1 -- 172.21.15.45:0/1580059034 shutdown_connections 2024-09-18T14:06:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.330+0000 7f709d99d640 1 -- 172.21.15.45:0/1580059034 >> 172.21.15.45:0/1580059034 conn(0x7f70980630b0 msgr2=0x7f70980a0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.330+0000 7f709d99d640 1 -- 172.21.15.45:0/1580059034 shutdown_connections 2024-09-18T14:06:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.330+0000 7f709d99d640 1 -- 172.21.15.45:0/1580059034 wait complete. 2024-09-18T14:06:21.342 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-18T14:06:21.342 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-09-18T14:06:21.342 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:06:21.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.438+0000 7f917051d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:21.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.438+0000 7f917051d640 10 monclient: build_initial_monmap 2024-09-18T14:06:21.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.438+0000 7f917051d640 10 monclient: monmap: 2024-09-18T14:06:21.456 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:21.456 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.456 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:21.439872+0000 2024-09-18T14:06:21.456 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:21.439872+0000 2024-09-18T14:06:21.456 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:21.456 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.456 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:21.456 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:21.456 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.439+0000 7f917051d640 1 Processor -- start 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.439+0000 7f917051d640 1 -- start start 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.439+0000 7f917051d640 10 monclient: init 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f917051d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f917051d640 10 monclient: _add_conns ranks=[3,1,7,2,8,0,5,6,4] 2024-09-18T14:06:21.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f917051d640 10 monclient(hunting): picked mon.noname-g con 0x7f916809f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f917051d640 10 monclient(hunting): picked mon.noname-f con 0x7f916809fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f917051d640 10 monclient(hunting): picked mon.noname-i con 0x7f916809ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f917051d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916805ed70 con 0x7f916809fba0 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f917051d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916809dfc0 con 0x7f916809f7d0 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f917051d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916809dce0 con 0x7f916809ff70 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f917051d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f917051d640 10 monclient(hunting): authenticate will time out at 1196.778304s 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916dab8640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7f916809fba0 legacy=0x7f9168137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916e2b9640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7f916809f7d0 0x7f9168132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:33424/0 (socket says 172.21.15.45:33424) 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916e2b9640 1 -- 172.21.15.45:0/1042135077 learned_addr learned my addr 172.21.15.45:0/1042135077 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916d2b7640 1 -- 172.21.15.45:0/1042135077 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454132983 0 0) 0x7f916809dce0 con 0x7f916809ff70 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916d2b7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916d2b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916d2b7640 10 monclient(hunting): my global_id is 5209 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916d2b7640 1 -- 172.21.15.45:0/1042135077 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91500034e0 con 0x7f916809ff70 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916d2b7640 1 -- 172.21.15.45:0/1042135077 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688517269 0 0) 0x7f916809dfc0 con 0x7f916809f7d0 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916d2b7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916d2b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916d2b7640 10 monclient(hunting): my global_id is 154227 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.440+0000 7f916d2b7640 1 -- 172.21.15.45:0/1042135077 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f916809dce0 con 0x7f916809f7d0 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 1 -- 172.21.15.45:0/1042135077 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1716965727 0 0) 0x7f91500034e0 con 0x7f916809ff70 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 1 -- 172.21.15.45:0/1042135077 >> v1:172.21.15.84:6790/0 conn(0x7f916809f7d0 legacy=0x7f9168132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 1 -- 172.21.15.45:0/1042135077 >> v1:172.21.15.84:6789/0 conn(0x7f916809fba0 legacy=0x7f9168137a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 10 monclient: _finish_hunting 0 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 1 monclient: found mon.noname-i 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 1 -- 172.21.15.45:0/1042135077 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91680a1c00 con 0x7f916809ff70 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 10 monclient: _finish_auth 0 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f917051d640 5 monclient: authenticate success, global_id 5209 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f917051d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 1 -- 172.21.15.45:0/1042135077 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9160001830 con 0x7f916809ff70 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 10 monclient: dump: 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.461 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.461 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.461 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.461 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.462 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f917051d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 1 -- 172.21.15.45:0/1042135077 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9160001990 con 0x7f916809ff70 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f917051d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f917051d640 4 set_mon_vals no callback set 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 1 -- 172.21.15.45:0/1042135077 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9160001f40 con 0x7f916809ff70 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f916d2b7640 10 monclient: dump: 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f917051d640 10 monclient: shutdown 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f917051d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f917051d640 1 -- 172.21.15.45:0/1042135077 >> v1:172.21.15.84:6792/0 conn(0x7f916809ff70 legacy=0x7f916813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f917051d640 1 -- 172.21.15.45:0/1042135077 shutdown_connections 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f917051d640 1 -- 172.21.15.45:0/1042135077 >> 172.21.15.45:0/1042135077 conn(0x7f91680630b0 msgr2=0x7f91680634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f917051d640 1 -- 172.21.15.45:0/1042135077 shutdown_connections 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.441+0000 7f917051d640 1 -- 172.21.15.45:0/1042135077 wait complete. 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.442+0000 7f917051d640 10 monclient: build_initial_monmap 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.442+0000 7f917051d640 10 monclient: monmap: 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:21.466 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:21.466 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:21.466 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:21.466 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:21.466 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:21.466 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:21.466 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:21.466 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.442+0000 7f917051d640 1 Processor -- start 2024-09-18T14:06:21.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.442+0000 7f917051d640 1 -- start start 2024-09-18T14:06:21.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.442+0000 7f917051d640 10 monclient: init 2024-09-18T14:06:21.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f917051d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f917051d640 10 monclient: _add_conns ranks=[8,1,7,3,4,0,5,6,2] 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f917051d640 10 monclient(hunting): picked mon.noname-e con 0x7f916809f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f917051d640 10 monclient(hunting): picked mon.noname-f con 0x7f916809fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f917051d640 10 monclient(hunting): picked mon.noname-i con 0x7f916809ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f917051d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9168120990 con 0x7f916809fba0 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f917051d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916811ede0 con 0x7f916809ff70 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f917051d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916809d310 con 0x7f916809f7d0 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f917051d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f916dab8640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7f916809fba0 legacy=0x7f916811d7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f917051d640 10 monclient(hunting): authenticate will time out at 1196.781297s 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f916e2b9640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f916809f7d0 0x7f916811d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:41182/0 (socket says 172.21.15.45:41182) 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f916e2b9640 1 -- 172.21.15.45:0/1775717429 learned_addr learned my addr 172.21.15.45:0/1775717429 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708489727 0 0) 0x7f916809d310 con 0x7f916809f7d0 2024-09-18T14:06:21.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 10 monclient(hunting): my global_id is 5165 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91540034e0 con 0x7f916809f7d0 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231666393 0 0) 0x7f916811ede0 con 0x7f916809ff70 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 10 monclient(hunting): my global_id is 5218 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f916809d310 con 0x7f916809ff70 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 493988387 0 0) 0x7f91540034e0 con 0x7f916809f7d0 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f916811ede0 con 0x7f916809f7d0 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91580018f0 con 0x7f916809f7d0 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:06:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 10 monclient(hunting): dump: 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.443+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3575733058 0 0) 0x7f916811ede0 con 0x7f916809f7d0 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 >> v1:172.21.15.84:6792/0 conn(0x7f916809ff70 legacy=0x7f9168149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 >> v1:172.21.15.84:6789/0 conn(0x7f916809fba0 legacy=0x7f916811d7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 10 monclient: _finish_hunting 0 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 1 monclient: found mon.e 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91680a2560 con 0x7f916809f7d0 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 10 monclient: _finish_auth 0 2024-09-18T14:06:21.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f917051d640 5 monclient: authenticate success, global_id 5165 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f917051d640 10 monclient: _renew_subs 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f917051d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f917051d640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916811f470 con 0x7f916809f7d0 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f917051d640 10 monclient: _renew_subs 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f917051d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f917051d640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f916811dee0 con 0x7f916809f7d0 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9158001b30 con 0x7f916809f7d0 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9158005110 con 0x7f916809f7d0 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f916cab6640 4 set_mon_vals no callback set 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 10 monclient: dump: 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.473 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f917051d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f917051d640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9148006580 con 0x7f9148002830 2024-09-18T14:06:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.444+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f91580100e0 con 0x7f916809f7d0 2024-09-18T14:06:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(329..329 src has 126..329) v4 ==== 3911+0+0 (unknown 3883595493 0 0) 0x7f9158046d60 con 0x7f916809f7d0 2024-09-18T14:06:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300466130 0 0) 0x7f9148006580 con 0x7f9148002830 2024-09-18T14:06:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 10 monclient(hunting): my global_id is 5265 2024-09-18T14:06:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f916811ede0 con 0x7f9148002830 2024-09-18T14:06:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f916eaba640 10 monclient: get_auth_request con 0x7f9154039e10 auth_method 0 2024-09-18T14:06:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1606600120 0 0) 0x7f916811ede0 con 0x7f9148002830 2024-09-18T14:06:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9148006580 con 0x7f9148002830 2024-09-18T14:06:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f915c004040 con 0x7f9148002830 2024-09-18T14:06:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 10 monclient: dump: 2024-09-18T14:06:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4017480113 0 0) 0x7f9148006580 con 0x7f9148002830 2024-09-18T14:06:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.445+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f914800a0a0 con 0x7f9148002830 2024-09-18T14:06:21.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.446+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f915c0062c0 con 0x7f9148002830 2024-09-18T14:06:21.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.446+0000 7f91667fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:21.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.446+0000 7f91667fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:21.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.446+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 >> v1:172.21.15.45:6789/0 conn(0x7f9148002830 legacy=0x7f9148004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.446+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 >> v1:172.21.15.45:6789/0 conn(0x7f9148002830 legacy=0x7f9148004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:21.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.451+0000 7f917051d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:21.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.451+0000 7f917051d640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9148007540 con 0x7f91480068f0 2024-09-18T14:06:21.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039020157 0 0) 0x7f9148007540 con 0x7f91480068f0 2024-09-18T14:06:21.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 10 monclient(hunting): my global_id is 5274 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9148006580 con 0x7f91480068f0 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2043305233 0 0) 0x7f9148006580 con 0x7f91480068f0 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9148007540 con 0x7f91480068f0 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f915c004b00 con 0x7f91480068f0 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 10 monclient: dump: 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.478 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1892423901 0 0) 0x7f9148007540 con 0x7f91480068f0 2024-09-18T14:06:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f91480087a0 con 0x7f91480068f0 2024-09-18T14:06:21.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7534 (unknown 0 0 1443610584) 0x7f915c006070 con 0x7f91480068f0 2024-09-18T14:06:21.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:21.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:21.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 >> v1:172.21.15.45:6789/0 conn(0x7f91480068f0 legacy=0x7f9148006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.452+0000 7f91667fc640 1 -- 172.21.15.45:0/1775717429 >> v1:172.21.15.45:6789/0 conn(0x7f91480068f0 legacy=0x7f9148006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:21.480 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 80, 2024-09-18T14:06:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:06:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:06:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:06:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:21.504 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.453+0000 7f917051d640 1 -- 172.21.15.45:0/1775717429 >> v1:172.21.15.84:6812/31478 conn(0x7f9154039e10 legacy=0x7f915403c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.453+0000 7f917051d640 10 monclient: shutdown 2024-09-18T14:06:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.453+0000 7f917051d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.453+0000 7f917051d640 1 -- 172.21.15.45:0/1775717429 >> v1:172.21.15.45:6793/0 conn(0x7f916809f7d0 legacy=0x7f916811d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.453+0000 7f916eaba640 1 -- 172.21.15.45:0/1775717429 reap_dead start 2024-09-18T14:06:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.453+0000 7f916eaba640 1 -- 172.21.15.45:0/1775717429 reap_dead start 2024-09-18T14:06:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.453+0000 7f917051d640 1 -- 172.21.15.45:0/1775717429 shutdown_connections 2024-09-18T14:06:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.453+0000 7f917051d640 1 -- 172.21.15.45:0/1775717429 >> 172.21.15.45:0/1775717429 conn(0x7f91680630b0 msgr2=0x7f91680a45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.453+0000 7f917051d640 1 -- 172.21.15.45:0/1775717429 shutdown_connections 2024-09-18T14:06:21.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.453+0000 7f917051d640 1 -- 172.21.15.45:0/1775717429 wait complete. 2024-09-18T14:06:21.510 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:06:21.510 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-09-18T14:06:21.510 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:06:21.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.607+0000 7f3c2d634640 10 monclient: get_monmap_and_config 2024-09-18T14:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.607+0000 7f3c2d634640 10 monclient: build_initial_monmap 2024-09-18T14:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.607+0000 7f3c2d634640 10 monclient: monmap: 2024-09-18T14:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:21.608138+0000 2024-09-18T14:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:21.608138+0000 2024-09-18T14:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.607+0000 7f3c2d634640 1 Processor -- start 2024-09-18T14:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.607+0000 7f3c2d634640 1 -- start start 2024-09-18T14:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.607+0000 7f3c2d634640 10 monclient: init 2024-09-18T14:06:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c2d634640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c2d634640 10 monclient: _add_conns ranks=[6,7,1,8,2,0,5,3,4] 2024-09-18T14:06:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c2d634640 10 monclient(hunting): picked mon.noname-d con 0x7f3c2809f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c2d634640 10 monclient(hunting): picked mon.noname-i con 0x7f3c2809fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c2d634640 10 monclient(hunting): picked mon.noname-f con 0x7f3c2809ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c2d634640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2805ed70 con 0x7f3c2809ff70 2024-09-18T14:06:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c2d634640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2809dfc0 con 0x7f3c2809f7d0 2024-09-18T14:06:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c2d634640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2809dce0 con 0x7f3c2809fba0 2024-09-18T14:06:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c2d634640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c2d634640 10 monclient(hunting): authenticate will time out at 1196.946365s 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c26ffd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f3c2809f7d0 0x7f3c28132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34018/0 (socket says 172.21.15.45:34018) 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c26ffd640 1 -- 172.21.15.45:0/342040213 learned_addr learned my addr 172.21.15.45:0/342040213 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c277fe640 1 -- 172.21.15.45:0/342040213 >> v1:172.21.15.84:6789/0 conn(0x7f3c2809ff70 legacy=0x7f3c2813b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c25ffb640 1 -- 172.21.15.45:0/342040213 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151653364 0 0) 0x7f3c2809dfc0 con 0x7f3c2809f7d0 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c25ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c25ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c25ffb640 10 monclient(hunting): my global_id is 5118 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c25ffb640 1 -- 172.21.15.45:0/342040213 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c140034e0 con 0x7f3c2809f7d0 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c25ffb640 1 -- 172.21.15.45:0/342040213 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158593343 0 0) 0x7f3c2809dce0 con 0x7f3c2809fba0 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c25ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c25ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c25ffb640 10 monclient(hunting): my global_id is 5227 2024-09-18T14:06:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.608+0000 7f3c25ffb640 1 -- 172.21.15.45:0/342040213 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c2809dfc0 con 0x7f3c2809fba0 2024-09-18T14:06:21.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 1 -- 172.21.15.45:0/342040213 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1715880706 0 0) 0x7f3c140034e0 con 0x7f3c2809f7d0 2024-09-18T14:06:21.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 1 -- 172.21.15.45:0/342040213 >> v1:172.21.15.84:6792/0 conn(0x7f3c2809fba0 legacy=0x7f3c28137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 1 -- 172.21.15.45:0/342040213 >> v1:172.21.15.84:6789/0 conn(0x7f3c2809ff70 legacy=0x7f3c2813b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:06:21.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:06:21.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 1 monclient: found mon.noname-d 2024-09-18T14:06:21.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:21.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:06:21.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 1 -- 172.21.15.45:0/342040213 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c280a1c00 con 0x7f3c2809f7d0 2024-09-18T14:06:21.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 10 monclient: _finish_auth 0 2024-09-18T14:06:21.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:21.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 1 -- 172.21.15.45:0/342040213 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3c18001bb0 con 0x7f3c2809f7d0 2024-09-18T14:06:21.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c2d634640 5 monclient: authenticate success, global_id 5118 2024-09-18T14:06:21.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c2d634640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:21.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:06:21.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 10 monclient: dump: 2024-09-18T14:06:21.620 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.623 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c2d634640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:21.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 1 -- 172.21.15.45:0/342040213 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3c18001d90 con 0x7f3c2809f7d0 2024-09-18T14:06:21.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:21.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 1 -- 172.21.15.45:0/342040213 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3c18004f20 con 0x7f3c2809f7d0 2024-09-18T14:06:21.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c2d634640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:21.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c2d634640 4 set_mon_vals no callback set 2024-09-18T14:06:21.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:21.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c25ffb640 10 monclient: dump: 2024-09-18T14:06:21.624 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.624 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.624 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.626 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.626 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.626 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c2d634640 10 monclient: shutdown 2024-09-18T14:06:21.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c2d634640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:21.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c2d634640 1 -- 172.21.15.45:0/342040213 >> v1:172.21.15.45:6792/0 conn(0x7f3c2809f7d0 legacy=0x7f3c28132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c2d634640 1 -- 172.21.15.45:0/342040213 shutdown_connections 2024-09-18T14:06:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c2d634640 1 -- 172.21.15.45:0/342040213 >> 172.21.15.45:0/342040213 conn(0x7f3c280630b0 msgr2=0x7f3c280634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c2d634640 1 -- 172.21.15.45:0/342040213 shutdown_connections 2024-09-18T14:06:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.609+0000 7f3c2d634640 1 -- 172.21.15.45:0/342040213 wait complete. 2024-09-18T14:06:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.610+0000 7f3c2d634640 10 monclient: build_initial_monmap 2024-09-18T14:06:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.610+0000 7f3c2d634640 10 monclient: monmap: 2024-09-18T14:06:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:21.628 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.610+0000 7f3c2d634640 1 Processor -- start 2024-09-18T14:06:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.610+0000 7f3c2d634640 1 -- start start 2024-09-18T14:06:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.610+0000 7f3c2d634640 10 monclient: init 2024-09-18T14:06:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c2d634640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c2d634640 10 monclient: _add_conns ranks=[7,2,8,3,1,5,0,4,6] 2024-09-18T14:06:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c2d634640 10 monclient(hunting): picked mon.noname-i con 0x7f3c2809f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c2d634640 10 monclient(hunting): picked mon.noname-b con 0x7f3c2809fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c2d634640 10 monclient(hunting): picked mon.noname-e con 0x7f3c2809ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c2d634640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2809d310 con 0x7f3c2809fba0 2024-09-18T14:06:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c2d634640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c280a2560 con 0x7f3c2809f7d0 2024-09-18T14:06:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c2d634640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c28120090 con 0x7f3c2809ff70 2024-09-18T14:06:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c2d634640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c2d634640 10 monclient(hunting): authenticate will time out at 1196.949230s 2024-09-18T14:06:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c267fc640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f3c2809fba0 0x7f3c28131a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:53950/0 (socket says 172.21.15.45:53950) 2024-09-18T14:06:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c267fc640 1 -- 172.21.15.45:0/173316591 learned_addr learned my addr 172.21.15.45:0/173316591 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999683579 0 0) 0x7f3c28120090 con 0x7f3c2809ff70 2024-09-18T14:06:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): my global_id is 5174 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c080034e0 con 0x7f3c2809ff70 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513046287 0 0) 0x7f3c2809d310 con 0x7f3c2809fba0 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): my global_id is 144497 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c28120090 con 0x7f3c2809fba0 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326843652 0 0) 0x7f3c280a2560 con 0x7f3c2809f7d0 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): my global_id is 5236 2024-09-18T14:06:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c2809d310 con 0x7f3c2809f7d0 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 270849059 0 0) 0x7f3c080034e0 con 0x7f3c2809ff70 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3c280a2560 con 0x7f3c2809ff70 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3c20004220 con 0x7f3c2809ff70 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): dump: 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.633 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2014213454 0 0) 0x7f3c28120090 con 0x7f3c2809fba0 2024-09-18T14:06:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3c080034e0 con 0x7f3c2809fba0 2024-09-18T14:06:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3c10001b20 con 0x7f3c2809fba0 2024-09-18T14:06:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): dump: 2024-09-18T14:06:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2249901742 0 0) 0x7f3c2809d310 con 0x7f3c2809f7d0 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3c28120090 con 0x7f3c2809f7d0 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3c18001bb0 con 0x7f3c2809f7d0 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.611+0000 7f3c1effd640 10 monclient(hunting): dump: 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.640 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2636554048 0 0) 0x7f3c280a2560 con 0x7f3c2809ff70 2024-09-18T14:06:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 >> v1:172.21.15.84:6792/0 conn(0x7f3c2809f7d0 legacy=0x7f3c28141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 >> v1:172.21.15.45:6790/0 conn(0x7f3c2809fba0 legacy=0x7f3c28131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 10 monclient: _finish_hunting 0 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 1 monclient: found mon.e 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28121b50 con 0x7f3c2809ff70 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 10 monclient: _finish_auth 0 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c2d634640 5 monclient: authenticate success, global_id 5174 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c2d634640 10 monclient: _renew_subs 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c2d634640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c2d634640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c28120c60 con 0x7f3c2809ff70 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c2d634640 10 monclient: _renew_subs 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c2d634640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c2d634640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c28132110 con 0x7f3c2809ff70 2024-09-18T14:06:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3c20004380 con 0x7f3c2809ff70 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3c200048b0 con 0x7f3c2809ff70 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c257fa640 4 set_mon_vals no callback set 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c1effd640 10 monclient: dump: 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.643 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.612+0000 7f3c2d634640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c2d634640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3c0c006580 con 0x7f3c0c002830 2024-09-18T14:06:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f3c20004a10 con 0x7f3c2809ff70 2024-09-18T14:06:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(329..329 src has 126..329) v4 ==== 3911+0+0 (unknown 3883595493 0 0) 0x7f3c200468c0 con 0x7f3c2809ff70 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018145083 0 0) 0x7f3c0c006580 con 0x7f3c0c002830 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 10 monclient(hunting): my global_id is 144506 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c280a2560 con 0x7f3c0c002830 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c267fc640 10 monclient: get_auth_request con 0x7f3c0803a920 auth_method 0 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2359579195 0 0) 0x7f3c280a2560 con 0x7f3c0c002830 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3c0c006580 con 0x7f3c0c002830 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3c18001590 con 0x7f3c0c002830 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 10 monclient: dump: 2024-09-18T14:06:21.644 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.646 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3291725639 0 0) 0x7f3c0c006580 con 0x7f3c0c002830 2024-09-18T14:06:21.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.613+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c0c00a0a0 con 0x7f3c0c002830 2024-09-18T14:06:21.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.614+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3c18007260 con 0x7f3c0c002830 2024-09-18T14:06:21.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.614+0000 7f3c1effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:21.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.614+0000 7f3c1effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:21.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.614+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 >> v1:172.21.15.45:6790/0 conn(0x7f3c0c002830 legacy=0x7f3c0c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.614+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 >> v1:172.21.15.45:6790/0 conn(0x7f3c0c002830 legacy=0x7f3c0c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:21.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.619+0000 7f3c2d634640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:21.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c2d634640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3c0c009660 con 0x7f3c0c009040 2024-09-18T14:06:21.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407767883 0 0) 0x7f3c0c009660 con 0x7f3c0c009040 2024-09-18T14:06:21.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 10 monclient(hunting): my global_id is 144515 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c0c006580 con 0x7f3c0c009040 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2646061316 0 0) 0x7f3c0c006580 con 0x7f3c0c009040 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3c0c009660 con 0x7f3c0c009040 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3c18005b00 con 0x7f3c0c009040 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 10 monclient: dump: 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 705357726 0 0) 0x7f3c0c009660 con 0x7f3c0c009040 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3c0c009920 con 0x7f3c0c009040 2024-09-18T14:06:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7364 (unknown 0 0 419221818) 0x7f3c18006ed0 con 0x7f3c0c009040 2024-09-18T14:06:21.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:21.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:21.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 >> v1:172.21.15.45:6790/0 conn(0x7f3c0c009040 legacy=0x7f3c0c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.620+0000 7f3c1effd640 1 -- 172.21.15.45:0/173316591 >> v1:172.21.15.45:6790/0 conn(0x7f3c0c009040 legacy=0x7f3c0c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:21.651 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 80, 2024-09-18T14:06:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:06:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:21.672 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:21.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.621+0000 7f3c2d634640 1 -- 172.21.15.45:0/173316591 >> v1:172.21.15.84:6812/31478 conn(0x7f3c0803a920 legacy=0x7f3c0803cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.621+0000 7f3c2d634640 10 monclient: shutdown 2024-09-18T14:06:21.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.621+0000 7f3c2d634640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:21.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.621+0000 7f3c2d634640 1 -- 172.21.15.45:0/173316591 >> v1:172.21.15.45:6793/0 conn(0x7f3c2809ff70 legacy=0x7f3c28149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.621+0000 7f3c277fe640 1 -- 172.21.15.45:0/173316591 reap_dead start 2024-09-18T14:06:21.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.621+0000 7f3c277fe640 1 -- 172.21.15.45:0/173316591 reap_dead start 2024-09-18T14:06:21.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.621+0000 7f3c2d634640 1 -- 172.21.15.45:0/173316591 shutdown_connections 2024-09-18T14:06:21.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.621+0000 7f3c2d634640 1 -- 172.21.15.45:0/173316591 >> 172.21.15.45:0/173316591 conn(0x7f3c280630b0 msgr2=0x7f3c280a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:21.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.621+0000 7f3c2d634640 1 -- 172.21.15.45:0/173316591 shutdown_connections 2024-09-18T14:06:21.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.621+0000 7f3c2d634640 1 -- 172.21.15.45:0/173316591 wait complete. 2024-09-18T14:06:21.675 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:06:21.675 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-18T14:06:21.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:06:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.772+0000 7f164ed9d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.772+0000 7f164ed9d640 10 monclient: build_initial_monmap 2024-09-18T14:06:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.772+0000 7f164ed9d640 10 monclient: monmap: 2024-09-18T14:06:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:21.773549+0000 2024-09-18T14:06:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:21.773549+0000 2024-09-18T14:06:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:21.774 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 1 Processor -- start 2024-09-18T14:06:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 1 -- start start 2024-09-18T14:06:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 10 monclient: init 2024-09-18T14:06:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 10 monclient: _add_conns ranks=[2,4,3,5,1,7,0,6,8] 2024-09-18T14:06:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 10 monclient(hunting): picked mon.noname-b con 0x7f164809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 10 monclient(hunting): picked mon.noname-c con 0x7f164809fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 10 monclient(hunting): picked mon.noname-g con 0x7f164809ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164805ed70 con 0x7f164809f7d0 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164809dfc0 con 0x7f164809ff70 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164809dce0 con 0x7f164809fba0 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164ed9d640 10 monclient(hunting): authenticate will time out at 1197.111891s 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f1647fff640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f164809fba0 legacy=0x7f1648137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.773+0000 7f164cb39640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f164809f7d0 0x7f1648132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:53974/0 (socket says 172.21.15.45:53974) 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f164cb39640 1 -- 172.21.15.45:0/4294004347 learned_addr learned my addr 172.21.15.45:0/4294004347 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 1 -- 172.21.15.45:0/4294004347 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454304129 0 0) 0x7f164805ed70 con 0x7f164809f7d0 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient(hunting): my global_id is 144524 2024-09-18T14:06:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 1 -- 172.21.15.45:0/4294004347 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16380034e0 con 0x7f164809f7d0 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 1 -- 172.21.15.45:0/4294004347 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1858036059 0 0) 0x7f16380034e0 con 0x7f164809f7d0 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 1 -- 172.21.15.45:0/4294004347 >> v1:172.21.15.45:6791/0 conn(0x7f164809fba0 legacy=0x7f1648137a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 1 -- 172.21.15.45:0/4294004347 >> v1:172.21.15.84:6790/0 conn(0x7f164809ff70 legacy=0x7f164813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient: _finish_hunting 0 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 1 monclient: found mon.noname-b 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 1 -- 172.21.15.45:0/4294004347 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16480a1c00 con 0x7f164809f7d0 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient: _finish_auth 0 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f164ed9d640 5 monclient: authenticate success, global_id 144524 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 1 -- 172.21.15.45:0/4294004347 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f163c001830 con 0x7f164809f7d0 2024-09-18T14:06:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f164ed9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient: dump: 2024-09-18T14:06:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f164ed9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 1 -- 172.21.15.45:0/4294004347 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f163c001a10 con 0x7f164809f7d0 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 1 -- 172.21.15.45:0/4294004347 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f163c004ba0 con 0x7f164809f7d0 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f164ed9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f164ed9d640 4 set_mon_vals no callback set 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.774+0000 7f16477fe640 10 monclient: dump: 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.775+0000 7f164ed9d640 10 monclient: shutdown 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.775+0000 7f164ed9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.775+0000 7f164ed9d640 1 -- 172.21.15.45:0/4294004347 >> v1:172.21.15.45:6790/0 conn(0x7f164809f7d0 legacy=0x7f1648132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.775+0000 7f164ed9d640 1 -- 172.21.15.45:0/4294004347 shutdown_connections 2024-09-18T14:06:21.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.775+0000 7f164ed9d640 1 -- 172.21.15.45:0/4294004347 >> 172.21.15.45:0/4294004347 conn(0x7f16480630b0 msgr2=0x7f16480634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.775+0000 7f164ed9d640 1 -- 172.21.15.45:0/4294004347 shutdown_connections 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.775+0000 7f164ed9d640 1 -- 172.21.15.45:0/4294004347 wait complete. 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.775+0000 7f164ed9d640 10 monclient: build_initial_monmap 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.775+0000 7f164ed9d640 10 monclient: monmap: 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.775+0000 7f164ed9d640 1 Processor -- start 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164ed9d640 1 -- start start 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164ed9d640 10 monclient: init 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164ed9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164ed9d640 10 monclient: _add_conns ranks=[2,5,8,0,6,1,4,3,7] 2024-09-18T14:06:21.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164ed9d640 10 monclient(hunting): picked mon.noname-b con 0x7f164809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164ed9d640 10 monclient(hunting): picked mon.noname-h con 0x7f164809fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164ed9d640 10 monclient(hunting): picked mon.noname-e con 0x7f164809ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164ed9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16480a2560 con 0x7f164809f7d0 2024-09-18T14:06:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164ed9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1648120d80 con 0x7f164809fba0 2024-09-18T14:06:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164ed9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1648121ce0 con 0x7f164809ff70 2024-09-18T14:06:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164ed9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164ed9d640 10 monclient(hunting): authenticate will time out at 1197.114702s 2024-09-18T14:06:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164cb39640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f164809f7d0 0x7f1648141940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:53986/0 (socket says 172.21.15.45:53986) 2024-09-18T14:06:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f164cb39640 1 -- 172.21.15.45:0/2312766192 learned_addr learned my addr 172.21.15.45:0/2312766192 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996975439 0 0) 0x7f16480a2560 con 0x7f164809f7d0 2024-09-18T14:06:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f1644ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f1644ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f1644ff9640 10 monclient(hunting): my global_id is 144533 2024-09-18T14:06:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.776+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f162c0034e0 con 0x7f164809f7d0 2024-09-18T14:06:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112619961 0 0) 0x7f1648121ce0 con 0x7f164809ff70 2024-09-18T14:06:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): my global_id is 5183 2024-09-18T14:06:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16480a2560 con 0x7f164809ff70 2024-09-18T14:06:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392089536 0 0) 0x7f1648120d80 con 0x7f164809fba0 2024-09-18T14:06:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): my global_id is 5081 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1648121ce0 con 0x7f164809fba0 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 358841997 0 0) 0x7f162c0034e0 con 0x7f164809f7d0 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1648120d80 con 0x7f164809f7d0 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f163c001770 con 0x7f164809f7d0 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1322792869 0 0) 0x7f16480a2560 con 0x7f164809ff70 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f162c0034e0 con 0x7f164809ff70 2024-09-18T14:06:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1640001c20 con 0x7f164809ff70 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3236892156 0 0) 0x7f1648121ce0 con 0x7f164809fba0 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f16480a2560 con 0x7f164809fba0 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f16340041b0 con 0x7f164809fba0 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient(hunting): dump: 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1521257148 0 0) 0x7f1648120d80 con 0x7f164809f7d0 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 >> v1:172.21.15.45:6793/0 conn(0x7f164809ff70 legacy=0x7f1648149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 >> v1:172.21.15.84:6791/0 conn(0x7f164809fba0 legacy=0x7f1648131b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 monclient: found mon.b 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1648121fc0 con 0x7f164809f7d0 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient: _finish_auth 0 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f164ed9d640 5 monclient: authenticate success, global_id 144533 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f164ed9d640 10 monclient: _renew_subs 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f164ed9d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f164ed9d640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16481201b0 con 0x7f164809f7d0 2024-09-18T14:06:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f164ed9d640 10 monclient: _renew_subs 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f164ed9d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f164ed9d640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1648132440 con 0x7f164809f7d0 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f163c004e20 con 0x7f164809f7d0 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f163c0052d0 con 0x7f164809f7d0 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1646ffd640 4 set_mon_vals no callback set 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.777+0000 7f1644ff9640 10 monclient: dump: 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.778+0000 7f164ed9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.778+0000 7f164ed9d640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1628006580 con 0x7f1628002830 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.778+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f163c0102a0 con 0x7f164809f7d0 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.778+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(329..329 src has 126..329) v4 ==== 3911+0+0 (unknown 3883595493 0 0) 0x7f163c046d60 con 0x7f164809f7d0 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.778+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782345233 0 0) 0x7f1628006580 con 0x7f1628002830 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.778+0000 7f1644ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.778+0000 7f1644ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.778+0000 7f1644ff9640 10 monclient(hunting): my global_id is 5127 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.778+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1648120d80 con 0x7f1628002830 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.778+0000 7f164d33a640 10 monclient: get_auth_request con 0x7f162c03aa40 auth_method 0 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 296388333 0 0) 0x7f1648120d80 con 0x7f1628002830 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1628006580 con 0x7f1628002830 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1634004880 con 0x7f1628002830 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 10 monclient: dump: 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.804 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2433566665 0 0) 0x7f1628006580 con 0x7f1628002830 2024-09-18T14:06:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f162800a0a0 con 0x7f1628002830 2024-09-18T14:06:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1634006b00 con 0x7f1628002830 2024-09-18T14:06:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 >> v1:172.21.15.45:6792/0 conn(0x7f1628002830 legacy=0x7f1628004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.779+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 >> v1:172.21.15.45:6792/0 conn(0x7f1628002830 legacy=0x7f1628004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f164ed9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f164ed9d640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1628009e70 con 0x7f16280068f0 2024-09-18T14:06:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486424452 0 0) 0x7f1628009e70 con 0x7f16280068f0 2024-09-18T14:06:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 10 monclient(hunting): my global_id is 5136 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1628006580 con 0x7f16280068f0 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3299141145 0 0) 0x7f1628006580 con 0x7f16280068f0 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1628009e70 con 0x7f16280068f0 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1634005340 con 0x7f16280068f0 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 10 monclient: dump: 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1626539787 0 0) 0x7f1628009e70 con 0x7f16280068f0 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.785+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1628008750 con 0x7f16280068f0 2024-09-18T14:06:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7364 (unknown 0 0 2520544666) 0x7f16340068b0 con 0x7f16280068f0 2024-09-18T14:06:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f1644ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f1644ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 >> v1:172.21.15.45:6792/0 conn(0x7f16280068f0 legacy=0x7f1628006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f1644ff9640 1 -- 172.21.15.45:0/2312766192 >> v1:172.21.15.45:6792/0 conn(0x7f16280068f0 legacy=0x7f1628006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:21.809 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 80, 2024-09-18T14:06:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:06:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.846 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:21.846 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.846 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:21.852 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f164ed9d640 1 -- 172.21.15.45:0/2312766192 >> v1:172.21.15.84:6812/31478 conn(0x7f162c03aa40 legacy=0x7f162c03cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f164ed9d640 10 monclient: shutdown 2024-09-18T14:06:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f164ed9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f164ed9d640 1 -- 172.21.15.45:0/2312766192 >> v1:172.21.15.45:6790/0 conn(0x7f164809f7d0 legacy=0x7f1648141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f164d33a640 1 -- 172.21.15.45:0/2312766192 reap_dead start 2024-09-18T14:06:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f164d33a640 1 -- 172.21.15.45:0/2312766192 reap_dead start 2024-09-18T14:06:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f164ed9d640 1 -- 172.21.15.45:0/2312766192 shutdown_connections 2024-09-18T14:06:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f164ed9d640 1 -- 172.21.15.45:0/2312766192 >> 172.21.15.45:0/2312766192 conn(0x7f16480630b0 msgr2=0x7f16480a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f164ed9d640 1 -- 172.21.15.45:0/2312766192 shutdown_connections 2024-09-18T14:06:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.786+0000 7f164ed9d640 1 -- 172.21.15.45:0/2312766192 wait complete. 2024-09-18T14:06:21.857 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:06:21.857 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-09-18T14:06:21.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:06:21.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.955+0000 7f029c39d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.955+0000 7f029c39d640 10 monclient: build_initial_monmap 2024-09-18T14:06:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.955+0000 7f029c39d640 10 monclient: monmap: 2024-09-18T14:06:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:21.956114+0000 2024-09-18T14:06:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:21.956114+0000 2024-09-18T14:06:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:21.957 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.955+0000 7f029c39d640 1 Processor -- start 2024-09-18T14:06:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.955+0000 7f029c39d640 1 -- start start 2024-09-18T14:06:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.955+0000 7f029c39d640 10 monclient: init 2024-09-18T14:06:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029c39d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029c39d640 10 monclient: _add_conns ranks=[8,2,6,4,7,5,0,3,1] 2024-09-18T14:06:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029c39d640 10 monclient(hunting): picked mon.noname-e con 0x7f029409f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029c39d640 10 monclient(hunting): picked mon.noname-b con 0x7f029409fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029c39d640 10 monclient(hunting): picked mon.noname-d con 0x7f029409ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029c39d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029405ed70 con 0x7f029409fba0 2024-09-18T14:06:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029c39d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029409dfc0 con 0x7f029409ff70 2024-09-18T14:06:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029c39d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029409dce0 con 0x7f029409f7d0 2024-09-18T14:06:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029c39d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029c39d640 10 monclient(hunting): authenticate will time out at 1197.294474s 2024-09-18T14:06:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029a93a640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f029409ff70 0x7f029413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34046/0 (socket says 172.21.15.45:34046) 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029a139640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f029409f7d0 0x7f0294132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:41222/0 (socket says 172.21.15.45:41222) 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f029a93a640 1 -- 172.21.15.45:0/3693466895 learned_addr learned my addr 172.21.15.45:0/3693466895 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258919943 0 0) 0x7f029409dce0 con 0x7f029409f7d0 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 10 monclient(hunting): my global_id is 5192 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02800034e0 con 0x7f029409f7d0 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602432822 0 0) 0x7f029409dfc0 con 0x7f029409ff70 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 10 monclient(hunting): my global_id is 5145 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f029409dce0 con 0x7f029409ff70 2024-09-18T14:06:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969409988 0 0) 0x7f029405ed70 con 0x7f029409fba0 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 10 monclient(hunting): my global_id is 144542 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.956+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f029409dfc0 con 0x7f029409fba0 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3001863608 0 0) 0x7f029409dce0 con 0x7f029409ff70 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 >> v1:172.21.15.45:6793/0 conn(0x7f029409f7d0 legacy=0x7f0294132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 >> v1:172.21.15.45:6790/0 conn(0x7f029409fba0 legacy=0x7f0294137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 10 monclient: _finish_hunting 0 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 1 monclient: found mon.noname-d 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02940a1c00 con 0x7f029409ff70 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 10 monclient: _finish_auth 0 2024-09-18T14:06:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f028c001b10 con 0x7f029409ff70 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 10 monclient: dump: 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f029c39d640 5 monclient: authenticate success, global_id 5145 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f029c39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f028c001c70 con 0x7f029409ff70 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 1 -- 172.21.15.45:0/3693466895 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f028c004be0 con 0x7f029409ff70 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f029c39d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f029c39d640 4 set_mon_vals no callback set 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f0299137640 10 monclient: dump: 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f029c39d640 10 monclient: shutdown 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f029c39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f029c39d640 1 -- 172.21.15.45:0/3693466895 >> v1:172.21.15.45:6792/0 conn(0x7f029409ff70 legacy=0x7f029413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f029c39d640 1 -- 172.21.15.45:0/3693466895 shutdown_connections 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f029c39d640 1 -- 172.21.15.45:0/3693466895 >> 172.21.15.45:0/3693466895 conn(0x7f02940630b0 msgr2=0x7f02940634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f029c39d640 1 -- 172.21.15.45:0/3693466895 shutdown_connections 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.957+0000 7f029c39d640 1 -- 172.21.15.45:0/3693466895 wait complete. 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.958+0000 7f029c39d640 10 monclient: build_initial_monmap 2024-09-18T14:06:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.958+0000 7f029c39d640 10 monclient: monmap: 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.958+0000 7f029c39d640 1 Processor -- start 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.958+0000 7f029c39d640 1 -- start start 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.958+0000 7f029c39d640 10 monclient: init 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f029c39d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f029c39d640 10 monclient: _add_conns ranks=[8,2,7,0,5,3,1,6,4] 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f029c39d640 10 monclient(hunting): picked mon.noname-e con 0x7f029409f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f029c39d640 10 monclient(hunting): picked mon.noname-b con 0x7f029409fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f029c39d640 10 monclient(hunting): picked mon.noname-i con 0x7f029409ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f029c39d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029409d380 con 0x7f029409fba0 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f029c39d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029411d610 con 0x7f029409ff70 2024-09-18T14:06:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f029c39d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0294130480 con 0x7f029409f7d0 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f029c39d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f029a139640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f029409f7d0 0x7f0294141c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:41236/0 (socket says 172.21.15.45:41236) 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f029a139640 1 -- 172.21.15.45:0/4179750499 learned_addr learned my addr 172.21.15.45:0/4179750499 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f029c39d640 10 monclient(hunting): authenticate will time out at 1197.297362s 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151271370 0 0) 0x7f0284001040 con 0x7f029409fba0 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): my global_id is 144551 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02780034e0 con 0x7f029409fba0 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809849454 0 0) 0x7f0294130480 con 0x7f029409f7d0 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): my global_id is 5201 2024-09-18T14:06:21.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0284001040 con 0x7f029409f7d0 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2004092520 0 0) 0x7f02780034e0 con 0x7f029409fba0 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0294130480 con 0x7f029409fba0 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0284001c50 con 0x7f029409fba0 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): dump: 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629373214 0 0) 0x7f029411d610 con 0x7f029409ff70 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): my global_id is 5245 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02780034e0 con 0x7f029409ff70 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 932157678 0 0) 0x7f0284001040 con 0x7f029409f7d0 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f029411d610 con 0x7f029409f7d0 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f02900015a0 con 0x7f029409f7d0 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.959+0000 7f028a7fc640 10 monclient(hunting): dump: 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3046682674 0 0) 0x7f0294130480 con 0x7f029409fba0 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 >> v1:172.21.15.45:6793/0 conn(0x7f029409f7d0 legacy=0x7f0294141c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 >> v1:172.21.15.84:6792/0 conn(0x7f029409ff70 legacy=0x7f0294149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 1 monclient: found mon.b 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294142a80 con 0x7f029409fba0 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f029c39d640 5 monclient: authenticate success, global_id 144551 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f029c39d640 10 monclient: _renew_subs 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f029c39d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f029c39d640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02940a2560 con 0x7f029409fba0 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f029c39d640 10 monclient: _renew_subs 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f029c39d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f029c39d640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f029411e330 con 0x7f029409fba0 2024-09-18T14:06:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0284002160 con 0x7f029409fba0 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0284002610 con 0x7f029409fba0 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f0298936640 4 set_mon_vals no callback set 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f028a7fc640 10 monclient: dump: 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f029c39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.960+0000 7f029c39d640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f027c006580 con 0x7f027c002830 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f0284002770 con 0x7f029409fba0 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(329..329 src has 126..329) v4 ==== 3911+0+0 (unknown 3883595493 0 0) 0x7f0284044170 con 0x7f029409fba0 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388331374 0 0) 0x7f027c006580 con 0x7f027c002830 2024-09-18T14:06:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 10 monclient(hunting): my global_id is 5210 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0294130480 con 0x7f027c002830 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f029a93a640 10 monclient: get_auth_request con 0x7f027803a690 auth_method 0 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4139134100 0 0) 0x7f0294130480 con 0x7f027c002830 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f027c006580 con 0x7f027c002830 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0290004b20 con 0x7f027c002830 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 10 monclient: dump: 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2661485733 0 0) 0x7f027c006580 con 0x7f027c002830 2024-09-18T14:06:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.961+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f027c00a0a0 con 0x7f027c002830 2024-09-18T14:06:21.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.962+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0290005050 con 0x7f027c002830 2024-09-18T14:06:21.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.962+0000 7f028a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:21.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.962+0000 7f028a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:21.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.962+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 >> v1:172.21.15.45:6793/0 conn(0x7f027c002830 legacy=0x7f027c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.962+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 >> v1:172.21.15.45:6793/0 conn(0x7f027c002830 legacy=0x7f027c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.967+0000 7f029c39d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.967+0000 7f029c39d640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f027c009e70 con 0x7f027c0068f0 2024-09-18T14:06:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913317079 0 0) 0x7f027c009e70 con 0x7f027c0068f0 2024-09-18T14:06:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 10 monclient(hunting): my global_id is 5219 2024-09-18T14:06:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f027c006580 con 0x7f027c0068f0 2024-09-18T14:06:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3960285136 0 0) 0x7f027c006580 con 0x7f027c0068f0 2024-09-18T14:06:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f027c009e70 con 0x7f027c0068f0 2024-09-18T14:06:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f02900058b0 con 0x7f027c0068f0 2024-09-18T14:06:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:06:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 10 monclient: dump: 2024-09-18T14:06:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1882400829 0 0) 0x7f027c009e70 con 0x7f027c0068f0 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f027c008750 con 0x7f027c0068f0 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7536 (unknown 0 0 613002712) 0x7f0290005a10 con 0x7f027c0068f0 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 >> v1:172.21.15.45:6793/0 conn(0x7f027c0068f0 legacy=0x7f027c006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.968+0000 7f028a7fc640 1 -- 172.21.15.45:0/4179750499 >> v1:172.21.15.45:6793/0 conn(0x7f027c0068f0 legacy=0x7f027c006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:21.989 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:06:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 80, 2024-09-18T14:06:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:06:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:22.008 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.969+0000 7f029c39d640 1 -- 172.21.15.45:0/4179750499 >> v1:172.21.15.84:6812/31478 conn(0x7f027803a690 legacy=0x7f027803cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.969+0000 7f029c39d640 10 monclient: shutdown 2024-09-18T14:06:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.969+0000 7f029c39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.969+0000 7f029c39d640 1 -- 172.21.15.45:0/4179750499 >> v1:172.21.15.45:6790/0 conn(0x7f029409fba0 legacy=0x7f0294142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.969+0000 7f029a93a640 1 -- 172.21.15.45:0/4179750499 reap_dead start 2024-09-18T14:06:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.969+0000 7f029a93a640 1 -- 172.21.15.45:0/4179750499 reap_dead start 2024-09-18T14:06:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.969+0000 7f029c39d640 1 -- 172.21.15.45:0/4179750499 shutdown_connections 2024-09-18T14:06:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.969+0000 7f029c39d640 1 -- 172.21.15.45:0/4179750499 >> 172.21.15.45:0/4179750499 conn(0x7f02940630b0 msgr2=0x7f0294131e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.969+0000 7f029c39d640 1 -- 172.21.15.45:0/4179750499 shutdown_connections 2024-09-18T14:06:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:21.969+0000 7f029c39d640 1 -- 172.21.15.45:0/4179750499 wait complete. 2024-09-18T14:06:22.011 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:06:22.011 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-09-18T14:06:22.011 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:06:22.090 INFO:tasks.rados.rados.0.smithi084.stdout:1477: done (1 left) 2024-09-18T14:06:22.090 INFO:tasks.rados.rados.0.smithi084.stdout:1478: done (0 left) 2024-09-18T14:06:22.090 INFO:tasks.rados.rados.0.smithi084.stdout:1479: write oid 33 current snap is 160 2024-09-18T14:06:22.090 INFO:tasks.rados.rados.0.smithi084.stdout:1479: seq_num 519 ranges {722833=566235,1887331=650516} 2024-09-18T14:06:22.096 INFO:tasks.rados.rados.0.smithi084.stdout:1479: writing smithi08434663-33 from 722833 to 1289068 tid 1 2024-09-18T14:06:22.104 INFO:tasks.rados.rados.0.smithi084.stdout:1479: writing smithi08434663-33 from 1887331 to 2537847 tid 2 2024-09-18T14:06:22.104 INFO:tasks.rados.rados.0.smithi084.stdout:1480: write oid 31 current snap is 160 2024-09-18T14:06:22.104 INFO:tasks.rados.rados.0.smithi084.stdout:1480: seq_num 520 ranges {739640=641484,1829460=794174} 2024-09-18T14:06:22.111 INFO:tasks.rados.rados.0.smithi084.stdout:1480: writing smithi08434663-31 from 739640 to 1381124 tid 1 2024-09-18T14:06:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.113+0000 7f9d2b91d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.113+0000 7f9d2b91d640 10 monclient: build_initial_monmap 2024-09-18T14:06:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.113+0000 7f9d2b91d640 10 monclient: monmap: 2024-09-18T14:06:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:22.114278+0000 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:22.114278+0000 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:22.114 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.123 INFO:tasks.rados.rados.0.smithi084.stdout:1480: writing smithi08434663-31 from 1829460 to 2623634 tid 2 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:1481: snap_create 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:1479: finishing write tid 1 to smithi08434663-33 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:1479: finishing write tid 2 to smithi08434663-33 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:1479: finishing write tid 3 to smithi08434663-33 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:1479: finishing write tid 4 to smithi08434663-33 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 410 (ObjNum 519 snap 160 seq_num 519) dirty exists 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:1479: left oid 33 (ObjNum 519 snap 160 seq_num 519) 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:1480: finishing write tid 1 to smithi08434663-31 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:1480: finishing write tid 2 to smithi08434663-31 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:1480: finishing write tid 3 to smithi08434663-31 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:1480: finishing write tid 4 to smithi08434663-31 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 374 (ObjNum 520 snap 160 seq_num 520) dirty exists 2024-09-18T14:06:22.124 INFO:tasks.rados.rados.0.smithi084.stdout:1480: left oid 31 (ObjNum 520 snap 160 seq_num 520) 2024-09-18T14:06:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.113+0000 7f9d2b91d640 1 Processor -- start 2024-09-18T14:06:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d2b91d640 1 -- start start 2024-09-18T14:06:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d2b91d640 10 monclient: init 2024-09-18T14:06:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d2b91d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d2b91d640 10 monclient: _add_conns ranks=[2,7,4,1,5,8,3,6,0] 2024-09-18T14:06:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d2b91d640 10 monclient(hunting): picked mon.noname-b con 0x7f9d2406e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d2b91d640 10 monclient(hunting): picked mon.noname-i con 0x7f9d240a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d2b91d640 10 monclient(hunting): picked mon.noname-c con 0x7f9d240a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:06:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d2b91d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2405ed70 con 0x7f9d2406e810 2024-09-18T14:06:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d2b91d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d24097620 con 0x7f9d240a1080 2024-09-18T14:06:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d2b91d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d24097340 con 0x7f9d240a0cb0 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d2b91d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d2b91d640 10 monclient(hunting): authenticate will time out at 1197.452843s 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d29eba640 1 -- >> v1:172.21.15.45:6791/0 conn(0x7f9d240a1080 legacy=0x7f9d2413af40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6791/0 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d296b9640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f9d2406e810 0x7f9d240a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:54020/0 (socket says 172.21.15.45:54020) 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.114+0000 7f9d296b9640 1 -- 172.21.15.45:0/3680475876 learned_addr learned my addr 172.21.15.45:0/3680475876 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 1 -- 172.21.15.45:0/3680475876 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589438237 0 0) 0x7f9d2405ed70 con 0x7f9d2406e810 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient(hunting): my global_id is 144560 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 1 -- 172.21.15.45:0/3680475876 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d100034e0 con 0x7f9d2406e810 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 1 -- 172.21.15.45:0/3680475876 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002604694 0 0) 0x7f9d24097340 con 0x7f9d240a0cb0 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient(hunting): my global_id is 5254 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 1 -- 172.21.15.45:0/3680475876 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d2405ed70 con 0x7f9d240a0cb0 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 1 -- 172.21.15.45:0/3680475876 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3763944730 0 0) 0x7f9d100034e0 con 0x7f9d2406e810 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 1 -- 172.21.15.45:0/3680475876 >> v1:172.21.15.84:6792/0 conn(0x7f9d240a0cb0 legacy=0x7f9d24137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 1 -- 172.21.15.45:0/3680475876 >> v1:172.21.15.45:6791/0 conn(0x7f9d240a1080 legacy=0x7f9d2413af40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 1 monclient: found mon.noname-b 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 1 -- 172.21.15.45:0/3680475876 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d240a1660 con 0x7f9d2406e810 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient: _finish_auth 0 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 1 -- 172.21.15.45:0/3680475876 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d20001830 con 0x7f9d2406e810 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d2b91d640 5 monclient: authenticate success, global_id 144560 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d2b91d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient: dump: 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d2b91d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 1 -- 172.21.15.45:0/3680475876 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9d20001a10 con 0x7f9d2406e810 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 1 -- 172.21.15.45:0/3680475876 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d20004ba0 con 0x7f9d2406e810 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d1bfff640 10 monclient: dump: 2024-09-18T14:06:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d2b91d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.115+0000 7f9d2b91d640 4 set_mon_vals no callback set 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.116+0000 7f9d2b91d640 10 monclient: shutdown 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.116+0000 7f9d2b91d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.116+0000 7f9d2b91d640 1 -- 172.21.15.45:0/3680475876 >> v1:172.21.15.45:6790/0 conn(0x7f9d2406e810 legacy=0x7f9d240a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.116+0000 7f9d2b91d640 1 -- 172.21.15.45:0/3680475876 shutdown_connections 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.116+0000 7f9d2b91d640 1 -- 172.21.15.45:0/3680475876 >> 172.21.15.45:0/3680475876 conn(0x7f9d240a5bb0 msgr2=0x7f9d240a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.116+0000 7f9d2b91d640 1 -- 172.21.15.45:0/3680475876 shutdown_connections 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.116+0000 7f9d2b91d640 1 -- 172.21.15.45:0/3680475876 wait complete. 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.117+0000 7f9d2b91d640 10 monclient: build_initial_monmap 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.117+0000 7f9d2b91d640 10 monclient: monmap: 2024-09-18T14:06:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:22.134 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.117+0000 7f9d2b91d640 1 Processor -- start 2024-09-18T14:06:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.117+0000 7f9d2b91d640 1 -- start start 2024-09-18T14:06:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.117+0000 7f9d2b91d640 10 monclient: init 2024-09-18T14:06:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.117+0000 7f9d2b91d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.117+0000 7f9d2b91d640 10 monclient: _add_conns ranks=[2,6,3,4,0,7,5,8,1] 2024-09-18T14:06:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.117+0000 7f9d2b91d640 10 monclient(hunting): picked mon.noname-b con 0x7f9d2406e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d2b91d640 10 monclient(hunting): picked mon.noname-d con 0x7f9d240a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:06:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d2b91d640 10 monclient(hunting): picked mon.noname-g con 0x7f9d240a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d2b91d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d241227b0 con 0x7f9d2406e810 2024-09-18T14:06:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d2b91d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2413e8a0 con 0x7f9d240a1080 2024-09-18T14:06:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d2b91d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2413db40 con 0x7f9d240a0cb0 2024-09-18T14:06:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d2b91d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d2b91d640 10 monclient(hunting): authenticate will time out at 1197.456059s 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d296b9640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f9d2406e810 0x7f9d2413d430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:54034/0 (socket says 172.21.15.45:54034) 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d296b9640 1 -- 172.21.15.45:0/1476576561 learned_addr learned my addr 172.21.15.45:0/1476576561 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855558561 0 0) 0x7f9d241227b0 con 0x7f9d2406e810 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): my global_id is 144569 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d080034e0 con 0x7f9d2406e810 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778873759 0 0) 0x7f9d2413db40 con 0x7f9d240a0cb0 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): my global_id is 5154 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d241227b0 con 0x7f9d240a0cb0 2024-09-18T14:06:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939849607 0 0) 0x7f9d2413e8a0 con 0x7f9d240a1080 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): my global_id is 154245 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d2413db40 con 0x7f9d240a1080 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3575544134 0 0) 0x7f9d080034e0 con 0x7f9d2406e810 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d2413e8a0 con 0x7f9d2406e810 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d200047a0 con 0x7f9d2406e810 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): dump: 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2782619818 0 0) 0x7f9d241227b0 con 0x7f9d240a0cb0 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d080034e0 con 0x7f9d240a0cb0 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d14001cb0 con 0x7f9d240a0cb0 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient(hunting): dump: 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 957872481 0 0) 0x7f9d2413e8a0 con 0x7f9d2406e810 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 >> v1:172.21.15.45:6792/0 conn(0x7f9d240a0cb0 legacy=0x7f9d24146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 >> v1:172.21.15.84:6790/0 conn(0x7f9d240a1080 legacy=0x7f9d24149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient: _finish_hunting 0 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 monclient: found mon.b 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d241224d0 con 0x7f9d2406e810 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 10 monclient: _finish_auth 0 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d197fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d2b91d640 5 monclient: authenticate success, global_id 144569 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d2b91d640 10 monclient: _renew_subs 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d2b91d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.118+0000 7f9d2b91d640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d24122a90 con 0x7f9d2406e810 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.119+0000 7f9d2b91d640 10 monclient: _renew_subs 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.119+0000 7f9d2b91d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.119+0000 7f9d2b91d640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d241217a0 con 0x7f9d2406e810 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.119+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9d20004c10 con 0x7f9d2406e810 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.119+0000 7f9d197fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.119+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d200051a0 con 0x7f9d2406e810 2024-09-18T14:06:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.119+0000 7f9d197fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.119+0000 7f9d1b7fe640 4 set_mon_vals no callback set 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.119+0000 7f9d197fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.119+0000 7f9d197fa640 10 monclient: dump: 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.143 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.119+0000 7f9d2b91d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.119+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f9d20010170 con 0x7f9d2406e810 2024-09-18T14:06:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.120+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(330..330 src has 126..330) v4 ==== 3911+0+0 (unknown 3146600022 0 0) 0x7f9d20046d40 con 0x7f9d2406e810 2024-09-18T14:06:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.120+0000 7f9d2b91d640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9d04006580 con 0x7f9d04002830 2024-09-18T14:06:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.120+0000 7f9d28eb8640 10 monclient: get_auth_request con 0x7f9d0803a620 auth_method 0 2024-09-18T14:06:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.120+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365448160 0 0) 0x7f9d04006580 con 0x7f9d04002830 2024-09-18T14:06:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.120+0000 7f9d197fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.120+0000 7f9d197fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.120+0000 7f9d197fa640 10 monclient(hunting): my global_id is 154254 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.120+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d2413e8a0 con 0x7f9d04002830 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3270615475 0 0) 0x7f9d2413e8a0 con 0x7f9d04002830 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d04006580 con 0x7f9d04002830 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d1c0044b0 con 0x7f9d04002830 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 10 monclient: dump: 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1253186816 0 0) 0x7f9d04006580 con 0x7f9d04002830 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d0400a0a0 con 0x7f9d04002830 2024-09-18T14:06:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9d1c006790 con 0x7f9d04002830 2024-09-18T14:06:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 >> v1:172.21.15.84:6790/0 conn(0x7f9d04002830 legacy=0x7f9d04004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.121+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 >> v1:172.21.15.84:6790/0 conn(0x7f9d04002830 legacy=0x7f9d04004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.127+0000 7f9d2b91d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.127+0000 7f9d2b91d640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9d04007540 con 0x7f9d040068f0 2024-09-18T14:06:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560738446 0 0) 0x7f9d04007540 con 0x7f9d040068f0 2024-09-18T14:06:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 10 monclient(hunting): my global_id is 154263 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d04006580 con 0x7f9d040068f0 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1775720975 0 0) 0x7f9d04006580 con 0x7f9d040068f0 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d04007540 con 0x7f9d040068f0 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9d1c004f70 con 0x7f9d040068f0 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 10 monclient: dump: 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 709008465 0 0) 0x7f9d04007540 con 0x7f9d040068f0 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.128+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9d040087a0 con 0x7f9d040068f0 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7534 (unknown 0 0 4035304527) 0x7f9d1c0063a0 con 0x7f9d040068f0 2024-09-18T14:06:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d197fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d197fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 >> v1:172.21.15.84:6790/0 conn(0x7f9d040068f0 legacy=0x7f9d04006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d197fa640 1 -- 172.21.15.45:0/1476576561 >> v1:172.21.15.84:6790/0 conn(0x7f9d040068f0 legacy=0x7f9d04006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:22.150 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:22.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:22.150 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:22.150 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:22.150 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 80, 2024-09-18T14:06:22.150 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:22.150 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:22.151 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:22.152 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:22.153 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:22.154 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.155 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:22.157 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.158 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:22.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.161 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.162 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:22.166 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:22.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:22.169 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d2b91d640 1 -- 172.21.15.45:0/1476576561 >> v1:172.21.15.84:6812/31478 conn(0x7f9d0803a620 legacy=0x7f9d0803cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d2b91d640 10 monclient: shutdown 2024-09-18T14:06:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d2b91d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d2b91d640 1 -- 172.21.15.45:0/1476576561 >> v1:172.21.15.45:6790/0 conn(0x7f9d2406e810 legacy=0x7f9d2413d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d29eba640 1 -- 172.21.15.45:0/1476576561 reap_dead start 2024-09-18T14:06:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d29eba640 1 -- 172.21.15.45:0/1476576561 reap_dead start 2024-09-18T14:06:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d2b91d640 1 -- 172.21.15.45:0/1476576561 shutdown_connections 2024-09-18T14:06:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d2b91d640 1 -- 172.21.15.45:0/1476576561 >> 172.21.15.45:0/1476576561 conn(0x7f9d240a5bb0 msgr2=0x7f9d240a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d2b91d640 1 -- 172.21.15.45:0/1476576561 shutdown_connections 2024-09-18T14:06:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.129+0000 7f9d2b91d640 1 -- 172.21.15.45:0/1476576561 wait complete. 2024-09-18T14:06:22.171 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:06:22.171 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-18T14:06:22.171 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:06:22.213 INFO:tasks.rados.rados.0.smithi084.stdout:1479: done (2 left) 2024-09-18T14:06:22.213 INFO:tasks.rados.rados.0.smithi084.stdout:1480: done (1 left) 2024-09-18T14:06:22.213 INFO:tasks.rados.rados.0.smithi084.stdout:1481: done (0 left) 2024-09-18T14:06:22.213 INFO:tasks.rados.rados.0.smithi084.stdout:1482: snap_remove snap 158 2024-09-18T14:06:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.273+0000 7f25b959d640 10 monclient: get_monmap_and_config 2024-09-18T14:06:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.273+0000 7f25b959d640 10 monclient: build_initial_monmap 2024-09-18T14:06:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.273+0000 7f25b959d640 10 monclient: monmap: 2024-09-18T14:06:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:22.274060+0000 2024-09-18T14:06:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:22.274060+0000 2024-09-18T14:06:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:22.275 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:22.275 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:22.275 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.273+0000 7f25b959d640 1 Processor -- start 2024-09-18T14:06:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.273+0000 7f25b959d640 1 -- start start 2024-09-18T14:06:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.273+0000 7f25b959d640 10 monclient: init 2024-09-18T14:06:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b959d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b959d640 10 monclient: _add_conns ranks=[2,3,8,5,7,1,6,4,0] 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b959d640 10 monclient(hunting): picked mon.noname-b con 0x7f25b406e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b959d640 10 monclient(hunting): picked mon.noname-g con 0x7f25b40a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b959d640 10 monclient(hunting): picked mon.noname-e con 0x7f25b40a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b959d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b405ed70 con 0x7f25b406e810 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b959d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b4097620 con 0x7f25b40a0cb0 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b959d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b4097340 con 0x7f25b40a1080 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b959d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b959d640 10 monclient(hunting): authenticate will time out at 1197.612268s 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b2ffd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f25b406e810 0x7f25b40a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:54036/0 (socket says 172.21.15.45:54036) 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b2ffd640 1 -- 172.21.15.45:0/1550369314 learned_addr learned my addr 172.21.15.45:0/1550369314 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b37fe640 1 --1- 172.21.15.45:0/1550369314 >> v1:172.21.15.45:6793/0 conn(0x7f25b40a1080 0x7f25b413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:41274/0 (socket says 172.21.15.45:41274) 2024-09-18T14:06:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610640648 0 0) 0x7f25b405ed70 con 0x7f25b406e810 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient(hunting): my global_id is 144578 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25a00034e0 con 0x7f25b406e810 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344897855 0 0) 0x7f25b4097340 con 0x7f25b40a1080 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient(hunting): my global_id is 5228 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b405ed70 con 0x7f25b40a1080 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477548319 0 0) 0x7f25b4097620 con 0x7f25b40a0cb0 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient(hunting): my global_id is 154272 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b4097340 con 0x7f25b40a0cb0 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1885336855 0 0) 0x7f25a00034e0 con 0x7f25b406e810 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 >> v1:172.21.15.45:6793/0 conn(0x7f25b40a1080 legacy=0x7f25b413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 >> v1:172.21.15.84:6790/0 conn(0x7f25b40a0cb0 legacy=0x7f25b4137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 1 monclient: found mon.noname-b 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b40a1660 con 0x7f25b406e810 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient: _finish_auth 0 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b959d640 5 monclient: authenticate success, global_id 144578 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f25a4001bb0 con 0x7f25b406e810 2024-09-18T14:06:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b1ffb640 10 monclient: dump: 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.274+0000 7f25b959d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f25a4001d90 con 0x7f25b406e810 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b959d640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b1ffb640 1 -- 172.21.15.45:0/1550369314 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f25a4004f20 con 0x7f25b406e810 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b959d640 4 set_mon_vals no callback set 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b1ffb640 10 monclient: dump: 2024-09-18T14:06:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b959d640 10 monclient: shutdown 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b959d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b959d640 1 -- 172.21.15.45:0/1550369314 >> v1:172.21.15.45:6790/0 conn(0x7f25b406e810 legacy=0x7f25b40a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b959d640 1 -- 172.21.15.45:0/1550369314 shutdown_connections 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b959d640 1 -- 172.21.15.45:0/1550369314 >> 172.21.15.45:0/1550369314 conn(0x7f25b40a5bb0 msgr2=0x7f25b40a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b959d640 1 -- 172.21.15.45:0/1550369314 shutdown_connections 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.275+0000 7f25b959d640 1 -- 172.21.15.45:0/1550369314 wait complete. 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.276+0000 7f25b959d640 10 monclient: build_initial_monmap 2024-09-18T14:06:22.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.276+0000 7f25b959d640 10 monclient: monmap: 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.276+0000 7f25b959d640 1 Processor -- start 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.276+0000 7f25b959d640 1 -- start start 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.276+0000 7f25b959d640 10 monclient: init 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 10 monclient: _add_conns ranks=[1,7,2,0,6,5,4,3,8] 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 10 monclient(hunting): picked mon.noname-f con 0x7f25b406e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 10 monclient(hunting): picked mon.noname-i con 0x7f25b40a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 10 monclient(hunting): picked mon.noname-b con 0x7f25b40a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b4122cf0 con 0x7f25b406e810 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b2ffd640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7f25b406e810 legacy=0x7f25b413d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b413dae0 con 0x7f25b40a1080 2024-09-18T14:06:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b4122730 con 0x7f25b40a0cb0 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 10 monclient(hunting): authenticate will time out at 1197.615214s 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b37fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f25b40a1080 0x7f25b4149d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:54048/0 (socket says 172.21.15.45:54048) 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b37fe640 1 -- 172.21.15.45:0/1534600616 learned_addr learned my addr 172.21.15.45:0/1534600616 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279965183 0 0) 0x7f25b413dae0 con 0x7f25b40a1080 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): my global_id is 144587 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25940034e0 con 0x7f25b40a1080 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267083231 0 0) 0x7f25b4122730 con 0x7f25b40a0cb0 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): my global_id is 5263 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b413dae0 con 0x7f25b40a0cb0 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 481359529 0 0) 0x7f25940034e0 con 0x7f25b40a1080 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f25b4122730 con 0x7f25b40a1080 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f25ac001a30 con 0x7f25b40a1080 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): dump: 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.293 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1245756732 0 0) 0x7f25b413dae0 con 0x7f25b40a0cb0 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f25940034e0 con 0x7f25b40a0cb0 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f259c001a60 con 0x7f25b40a0cb0 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient(hunting): dump: 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.295 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 355512551 0 0) 0x7f25b4122730 con 0x7f25b40a1080 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 >> v1:172.21.15.84:6792/0 conn(0x7f25b40a0cb0 legacy=0x7f25b4146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 >> v1:172.21.15.84:6789/0 conn(0x7f25b406e810 legacy=0x7f25b413d3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient: _finish_hunting 0 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 monclient: found mon.b 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b41201f0 con 0x7f25b40a1080 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 10 monclient: _finish_auth 0 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:22.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 5 monclient: authenticate success, global_id 144587 2024-09-18T14:06:22.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 10 monclient: _renew_subs 2024-09-18T14:06:22.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:22.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.277+0000 7f25b959d640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b4122450 con 0x7f25b40a1080 2024-09-18T14:06:22.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25b959d640 10 monclient: _renew_subs 2024-09-18T14:06:22.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25b959d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:06:22.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25b959d640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b4121580 con 0x7f25b40a1080 2024-09-18T14:06:22.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f25ac001c70 con 0x7f25b40a1080 2024-09-18T14:06:22.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25aaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:22.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f25ac004d90 con 0x7f25b40a1080 2024-09-18T14:06:22.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25b17fa640 4 set_mon_vals no callback set 2024-09-18T14:06:22.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25aaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:06:22.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25aaffd640 10 monclient: dump: 2024-09-18T14:06:22.298 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.298 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.298 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.298 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.298 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.298 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.298 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.298 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.298 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.298 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.299 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.299 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.299 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.299 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.299 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.299 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f25ac00fd60 con 0x7f25b40a1080 2024-09-18T14:06:22.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25b959d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:22.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.278+0000 7f25b959d640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2598006580 con 0x7f2598002830 2024-09-18T14:06:22.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(331..331 src has 126..331) v4 ==== 3911+0+0 (unknown 2413812167 0 0) 0x7f25ac046a20 con 0x7f25b40a1080 2024-09-18T14:06:22.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25b27fc640 10 monclient: get_auth_request con 0x7f259403a370 auth_method 0 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212205975 0 0) 0x7f2598006580 con 0x7f2598002830 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 10 monclient(hunting): my global_id is 5090 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b4122730 con 0x7f2598002830 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1570448423 0 0) 0x7f25b4122730 con 0x7f2598002830 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2598006580 con 0x7f2598002830 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f25a4004f20 con 0x7f2598002830 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 10 monclient: dump: 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2754434741 0 0) 0x7f2598006580 con 0x7f2598002830 2024-09-18T14:06:22.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.279+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f259800a0a0 con 0x7f2598002830 2024-09-18T14:06:22.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.280+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f25a4001870 con 0x7f2598002830 2024-09-18T14:06:22.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.280+0000 7f25aaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:22.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.280+0000 7f25aaffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:22.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.280+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 >> v1:172.21.15.84:6791/0 conn(0x7f2598002830 legacy=0x7f2598004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.280+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 >> v1:172.21.15.84:6791/0 conn(0x7f2598002830 legacy=0x7f2598004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:22.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.285+0000 7f25b959d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:22.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.285+0000 7f25b959d640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2598007510 con 0x7f25980068c0 2024-09-18T14:06:22.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265465402 0 0) 0x7f2598007510 con 0x7f25980068c0 2024-09-18T14:06:22.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 10 monclient(hunting): my global_id is 5099 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2598006580 con 0x7f25980068c0 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2808416605 0 0) 0x7f2598006580 con 0x7f25980068c0 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2598007510 con 0x7f25980068c0 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f25a40054e0 con 0x7f25980068c0 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 10 monclient: dump: 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2131108732 0 0) 0x7f2598007510 con 0x7f25980068c0 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.286+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f25980087a0 con 0x7f25980068c0 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7534 (unknown 0 0 678065320) 0x7f25a4008190 con 0x7f25980068c0 2024-09-18T14:06:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25aaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:22.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25aaffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:22.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 >> v1:172.21.15.84:6791/0 conn(0x7f25980068c0 legacy=0x7f2598006d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25aaffd640 1 -- 172.21.15.45:0/1534600616 >> v1:172.21.15.84:6791/0 conn(0x7f25980068c0 legacy=0x7f2598006d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:22.306 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 80, 2024-09-18T14:06:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:06:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:06:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:22.325 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:22.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25b959d640 1 -- 172.21.15.45:0/1534600616 >> v1:172.21.15.84:6812/31478 conn(0x7f259403a370 legacy=0x7f259403c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25b959d640 10 monclient: shutdown 2024-09-18T14:06:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25b959d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25b959d640 1 -- 172.21.15.45:0/1534600616 >> v1:172.21.15.45:6790/0 conn(0x7f25b40a1080 legacy=0x7f25b4149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25b37fe640 1 -- 172.21.15.45:0/1534600616 reap_dead start 2024-09-18T14:06:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25b37fe640 1 -- 172.21.15.45:0/1534600616 reap_dead start 2024-09-18T14:06:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25b959d640 1 -- 172.21.15.45:0/1534600616 shutdown_connections 2024-09-18T14:06:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25b959d640 1 -- 172.21.15.45:0/1534600616 >> 172.21.15.45:0/1534600616 conn(0x7f25b40a5bb0 msgr2=0x7f25b40a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25b959d640 1 -- 172.21.15.45:0/1534600616 shutdown_connections 2024-09-18T14:06:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.287+0000 7f25b959d640 1 -- 172.21.15.45:0/1534600616 wait complete. 2024-09-18T14:06:22.327 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:06:22.328 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-09-18T14:06:22.328 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:06:22.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.424+0000 7f017c268640 10 monclient: get_monmap_and_config 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.424+0000 7f017c268640 10 monclient: build_initial_monmap 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.424+0000 7f017c268640 10 monclient: monmap: 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:06:22.425513+0000 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:06:22.425513+0000 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:22.426 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 1 Processor -- start 2024-09-18T14:06:22.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 1 -- start start 2024-09-18T14:06:22.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 10 monclient: init 2024-09-18T14:06:22.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:22.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 10 monclient: _add_conns ranks=[7,8,5,0,2,1,3,4,6] 2024-09-18T14:06:22.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 10 monclient(hunting): picked mon.noname-i con 0x7f017409f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 10 monclient(hunting): picked mon.noname-e con 0x7f017409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 10 monclient(hunting): picked mon.noname-h con 0x7f017409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017405ed70 con 0x7f017409ff70 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017409dfc0 con 0x7f017409f7d0 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017409dce0 con 0x7f017409fba0 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.425+0000 7f017c268640 10 monclient(hunting): authenticate will time out at 1197.763896s 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179803640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f017409fba0 0x7f0174137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:41276/0 (socket says 172.21.15.45:41276) 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179803640 1 -- 172.21.15.45:0/1875324452 learned_addr learned my addr 172.21.15.45:0/1875324452 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731834417 0 0) 0x7f017409dce0 con 0x7f017409fba0 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient(hunting): my global_id is 5237 2024-09-18T14:06:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01600034e0 con 0x7f017409fba0 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767310409 0 0) 0x7f017409dfc0 con 0x7f017409f7d0 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient(hunting): my global_id is 5272 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f017409dce0 con 0x7f017409f7d0 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189160218 0 0) 0x7f017405ed70 con 0x7f017409ff70 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient(hunting): my global_id is 5108 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f017409dfc0 con 0x7f017409ff70 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1272453661 0 0) 0x7f017409dce0 con 0x7f017409f7d0 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 >> v1:172.21.15.45:6793/0 conn(0x7f017409fba0 legacy=0x7f0174137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 >> v1:172.21.15.84:6791/0 conn(0x7f017409ff70 legacy=0x7f017413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient: _finish_hunting 0 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 monclient: found mon.noname-i 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01740a1c00 con 0x7f017409f7d0 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient: _finish_auth 0 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0170001810 con 0x7f017409f7d0 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f017c268640 5 monclient: authenticate success, global_id 5272 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f017c268640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient: dump: 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.437 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f017c268640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f01700019f0 con 0x7f017409f7d0 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 1 -- 172.21.15.45:0/1875324452 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0170001fa0 con 0x7f017409f7d0 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f017c268640 10 monclient: get_monmap_and_config success 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f017c268640 4 set_mon_vals no callback set 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.426+0000 7f0179002640 10 monclient: dump: 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.427+0000 7f017c268640 10 monclient: shutdown 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.427+0000 7f017c268640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.427+0000 7f017c268640 1 -- 172.21.15.45:0/1875324452 >> v1:172.21.15.84:6792/0 conn(0x7f017409f7d0 legacy=0x7f0174132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.427+0000 7f017c268640 1 -- 172.21.15.45:0/1875324452 shutdown_connections 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.427+0000 7f017c268640 1 -- 172.21.15.45:0/1875324452 >> 172.21.15.45:0/1875324452 conn(0x7f01740630b0 msgr2=0x7f01740634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.427+0000 7f017c268640 1 -- 172.21.15.45:0/1875324452 shutdown_connections 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.427+0000 7f017c268640 1 -- 172.21.15.45:0/1875324452 wait complete. 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 10 monclient: build_initial_monmap 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 10 monclient: monmap: 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:06:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 1 Processor -- start 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 1 -- start start 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 10 monclient: init 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 10 monclient: _reopen_session rank -1 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 10 monclient: _add_conns ranks=[0,3,1,8,5,6,2,7,4] 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 10 monclient(hunting): picked mon.noname-a con 0x7f017409f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:06:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 10 monclient(hunting): picked mon.noname-g con 0x7f017409fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 10 monclient(hunting): picked mon.noname-f con 0x7f017409ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01740a2560 con 0x7f017409f7d0 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0174120830 con 0x7f017409ff70 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0174121df0 con 0x7f017409fba0 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 10 monclient(hunting): _renew_subs 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017c268640 10 monclient(hunting): authenticate will time out at 1197.766916s 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017a805640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7f017409ff70 legacy=0x7f0174149b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017a004640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f017409f7d0 0x7f01741417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46868/0 (socket says 172.21.15.45:46868) 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.428+0000 7f017a004640 1 -- 172.21.15.45:0/2892503107 learned_addr learned my addr 172.21.15.45:0/2892503107 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1282123407 0 0) 0x7f01740a2560 con 0x7f017409f7d0 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient(hunting): my global_id is 5283 2024-09-18T14:06:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01580034e0 con 0x7f017409f7d0 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1916004820 0 0) 0x7f01580034e0 con 0x7f017409f7d0 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f01740a2560 con 0x7f017409f7d0 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0170001ea0 con 0x7f017409f7d0 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient(hunting): dump: 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692432170 0 0) 0x7f0174121df0 con 0x7f017409fba0 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient(hunting): my global_id is 154281 2024-09-18T14:06:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01580034e0 con 0x7f017409fba0 2024-09-18T14:06:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4169040096 0 0) 0x7f01740a2560 con 0x7f017409f7d0 2024-09-18T14:06:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 >> v1:172.21.15.84:6790/0 conn(0x7f017409fba0 legacy=0x7f01741319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 >> v1:172.21.15.84:6789/0 conn(0x7f017409ff70 legacy=0x7f0174149b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:06:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:06:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 monclient: found mon.a 2024-09-18T14:06:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:06:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0174142170 con 0x7f017409f7d0 2024-09-18T14:06:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:06:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:06:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f017c268640 5 monclient: authenticate success, global_id 5283 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f017c268640 10 monclient: _renew_subs 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f017c268640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f017c268640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0174141eb0 con 0x7f017409f7d0 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f017c268640 10 monclient: _renew_subs 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f017c268640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f017c268640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f01741320d0 con 0x7f017409f7d0 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0170002590 con 0x7f017409f7d0 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0170002a40 con 0x7f017409f7d0 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f0178801640 4 set_mon_vals no callback set 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.429+0000 7f016a7fc640 10 monclient: dump: 2024-09-18T14:06:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.430+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f0170002c20 con 0x7f017409f7d0 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.430+0000 7f017c268640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.430+0000 7f017c268640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f015c006580 con 0x7f015c002830 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.430+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(331..331 src has 126..331) v4 ==== 3911+0+0 (unknown 2413812167 0 0) 0x7f0170046df0 con 0x7f017409f7d0 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.430+0000 7f017a805640 10 monclient: get_auth_request con 0x7f0158039cd0 auth_method 0 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051296861 0 0) 0x7f015c006580 con 0x7f015c002830 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 10 monclient(hunting): my global_id is 5281 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01740a2560 con 0x7f015c002830 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4037947535 0 0) 0x7f01740a2560 con 0x7f015c002830 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f015c006580 con 0x7f015c002830 2024-09-18T14:06:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0164001360 con 0x7f015c002830 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 10 monclient: dump: 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.450 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3719317081 0 0) 0x7f015c006580 con 0x7f015c002830 2024-09-18T14:06:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f015c00a0a0 con 0x7f015c002830 2024-09-18T14:06:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f01640017c0 con 0x7f015c002830 2024-09-18T14:06:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:06:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:06:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 >> v1:172.21.15.84:6792/0 conn(0x7f015c002830 legacy=0x7f015c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.431+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 >> v1:172.21.15.84:6792/0 conn(0x7f015c002830 legacy=0x7f015c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.437+0000 7f017c268640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:06:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.437+0000 7f017c268640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f015c007540 con 0x7f015c0068f0 2024-09-18T14:06:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238870040 0 0) 0x7f015c007540 con 0x7f015c0068f0 2024-09-18T14:06:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:06:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:06:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 10 monclient(hunting): my global_id is 5290 2024-09-18T14:06:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f015c006580 con 0x7f015c0068f0 2024-09-18T14:06:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3553715503 0 0) 0x7f015c006580 con 0x7f015c0068f0 2024-09-18T14:06:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f015c007540 con 0x7f015c0068f0 2024-09-18T14:06:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0164004f30 con 0x7f015c0068f0 2024-09-18T14:06:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 10 monclient: dump: 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:06:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:06:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:06:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:06:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:06:22.454 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:06:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 982117341 0 0) 0x7f015c007540 con 0x7f015c0068f0 2024-09-18T14:06:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.438+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f015c0087a0 con 0x7f015c0068f0 2024-09-18T14:06:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7364 (unknown 0 0 3619348504) 0x7f0164007170 con 0x7f015c0068f0 2024-09-18T14:06:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f016a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:06:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f016a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:06:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 >> v1:172.21.15.84:6792/0 conn(0x7f015c0068f0 legacy=0x7f015c006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f016a7fc640 1 -- 172.21.15.45:0/2892503107 >> v1:172.21.15.84:6792/0 conn(0x7f015c0068f0 legacy=0x7f015c006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:06:22.454 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 80, 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:06:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:06:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:06:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:06:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:06:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:06:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:06:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:06:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:06:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:06:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:06:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:06:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:06:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:06:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:06:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:06:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:06:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:06:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:06:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:06:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:06:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:06:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:06:22.473 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:06:22.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f017c268640 1 -- 172.21.15.45:0/2892503107 >> v1:172.21.15.84:6812/31478 conn(0x7f0158039cd0 legacy=0x7f015803c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f017c268640 10 monclient: shutdown 2024-09-18T14:06:22.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f017c268640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:06:22.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f017c268640 1 -- 172.21.15.45:0/2892503107 >> v1:172.21.15.45:6789/0 conn(0x7f017409f7d0 legacy=0x7f01741417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:06:22.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f017a805640 1 -- 172.21.15.45:0/2892503107 reap_dead start 2024-09-18T14:06:22.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f017a805640 1 -- 172.21.15.45:0/2892503107 reap_dead start 2024-09-18T14:06:22.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f017c268640 1 -- 172.21.15.45:0/2892503107 shutdown_connections 2024-09-18T14:06:22.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f017c268640 1 -- 172.21.15.45:0/2892503107 >> 172.21.15.45:0/2892503107 conn(0x7f01740630b0 msgr2=0x7f01740a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:06:22.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f017c268640 1 -- 172.21.15.45:0/2892503107 shutdown_connections 2024-09-18T14:06:22.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:06:22.439+0000 7f017c268640 1 -- 172.21.15.45:0/2892503107 wait complete. 2024-09-18T14:06:22.475 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:06:22.475 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-18T14:06:23.219 INFO:tasks.rados.rados.0.smithi084.stdout:1482: done (0 left) 2024-09-18T14:06:23.219 INFO:tasks.rados.rados.0.smithi084.stdout:1483: read oid 26 snap -1 2024-09-18T14:06:23.219 INFO:tasks.rados.rados.0.smithi084.stdout:1483: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:06:23.219 INFO:tasks.rados.rados.0.smithi084.stdout:1484: write oid 30 current snap is 161 2024-09-18T14:06:23.220 INFO:tasks.rados.rados.0.smithi084.stdout:1484: seq_num 521 ranges {756447=716733,1887942=537832} 2024-09-18T14:06:23.227 INFO:tasks.rados.rados.0.smithi084.stdout:1484: writing smithi08434663-30 from 756447 to 1473180 tid 1 2024-09-18T14:06:23.233 INFO:tasks.rados.rados.0.smithi084.stdout:1484: writing smithi08434663-30 from 1887942 to 2425774 tid 2 2024-09-18T14:06:23.233 INFO:tasks.rados.rados.0.smithi084.stdout:1485: read oid 19 snap -1 2024-09-18T14:06:23.233 INFO:tasks.rados.rados.0.smithi084.stdout:1485: expect deleted 2024-09-18T14:06:23.233 INFO:tasks.rados.rados.0.smithi084.stdout:1486: delete oid 1 current snap is 161 2024-09-18T14:06:23.233 INFO:tasks.rados.rados.0.smithi084.stdout:1483: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:06:23.347 INFO:tasks.rados.rados.0.smithi084.stdout:1484: finishing write tid 1 to smithi08434663-30 2024-09-18T14:06:23.347 INFO:tasks.rados.rados.0.smithi084.stdout:1484: finishing write tid 2 to smithi08434663-30 2024-09-18T14:06:23.347 INFO:tasks.rados.rados.0.smithi084.stdout:1484: finishing write tid 3 to smithi08434663-30 2024-09-18T14:06:23.347 INFO:tasks.rados.rados.0.smithi084.stdout:1484: finishing write tid 4 to smithi08434663-30 2024-09-18T14:06:23.348 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 552 (ObjNum 521 snap 161 seq_num 521) dirty exists 2024-09-18T14:06:23.348 INFO:tasks.rados.rados.0.smithi084.stdout:1484: left oid 30 (ObjNum 521 snap 161 seq_num 521) 2024-09-18T14:06:23.348 INFO:tasks.rados.rados.0.smithi084.stdout:1483: done (3 left) 2024-09-18T14:06:23.348 INFO:tasks.rados.rados.0.smithi084.stdout:1484: done (2 left) 2024-09-18T14:06:23.348 INFO:tasks.rados.rados.0.smithi084.stdout:1485: done (1 left) 2024-09-18T14:06:23.348 INFO:tasks.rados.rados.0.smithi084.stdout:1486: done (0 left) 2024-09-18T14:06:23.348 INFO:tasks.rados.rados.0.smithi084.stdout:1487: write oid 31 current snap is 161 2024-09-18T14:06:23.348 INFO:tasks.rados.rados.0.smithi084.stdout:1487: seq_num 523 ranges {790061=467231,2004906=541501} 2024-09-18T14:06:23.357 INFO:tasks.rados.rados.0.smithi084.stdout:1487: writing smithi08434663-31 from 790061 to 1257292 tid 1 2024-09-18T14:06:23.361 INFO:tasks.rados.rados.0.smithi084.stdout:1487: writing smithi08434663-31 from 2004906 to 2546407 tid 2 2024-09-18T14:06:23.361 INFO:tasks.rados.rados.0.smithi084.stdout:1488: rollback oid 42 current snap is 161 2024-09-18T14:06:23.361 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 42 to 153 2024-09-18T14:06:23.361 INFO:tasks.rados.rados.0.smithi084.stdout:1487: finishing write tid 1 to smithi08434663-31 2024-09-18T14:06:23.361 INFO:tasks.rados.rados.0.smithi084.stdout:1489: snap_create 2024-09-18T14:06:23.365 INFO:tasks.rados.rados.0.smithi084.stdout:1487: finishing write tid 2 to smithi08434663-31 2024-09-18T14:06:23.365 INFO:tasks.rados.rados.0.smithi084.stdout:1487: finishing write tid 3 to smithi08434663-31 2024-09-18T14:06:23.365 INFO:tasks.rados.rados.0.smithi084.stdout:1487: finishing write tid 4 to smithi08434663-31 2024-09-18T14:06:23.365 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 377 (ObjNum 523 snap 161 seq_num 523) dirty exists 2024-09-18T14:06:23.365 INFO:tasks.rados.rados.0.smithi084.stdout:1487: left oid 31 (ObjNum 523 snap 161 seq_num 523) 2024-09-18T14:06:23.366 INFO:tasks.rados.rados.0.smithi084.stdout:1488: finishing rollback tid 0 to smithi08434663-42 2024-09-18T14:06:23.366 INFO:tasks.rados.rados.0.smithi084.stdout:1488: finishing rollback tid 1 to smithi08434663-42 2024-09-18T14:06:23.366 INFO:tasks.rados.rados.0.smithi084.stdout:1488: finishing rollback tid 2 to smithi08434663-42 2024-09-18T14:06:23.366 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 727 (ObjNum 423 snap 133 seq_num 423) dirty exists 2024-09-18T14:06:24.223 INFO:tasks.rados.rados.0.smithi084.stdout:1487: done (2 left) 2024-09-18T14:06:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:1488: done (1 left) 2024-09-18T14:06:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:1489: done (0 left) 2024-09-18T14:06:24.224 INFO:tasks.rados.rados.0.smithi084.stdout:1490: delete oid 34 current snap is 162 2024-09-18T14:06:24.227 INFO:tasks.rados.rados.0.smithi084.stdout:1490: done (0 left) 2024-09-18T14:06:24.227 INFO:tasks.rados.rados.0.smithi084.stdout:1491: write oid 23 current snap is 162 2024-09-18T14:06:24.227 INFO:tasks.rados.rados.0.smithi084.stdout:1491: seq_num 525 ranges {423675=734082,1721870=545170,2717777=105898} 2024-09-18T14:06:24.233 INFO:tasks.rados.rados.0.smithi084.stdout:1491: writing smithi08434663-23 from 423675 to 1157757 tid 1 2024-09-18T14:06:24.239 INFO:tasks.rados.rados.0.smithi084.stdout:1491: writing smithi08434663-23 from 1721870 to 2267040 tid 2 2024-09-18T14:06:24.243 INFO:tasks.rados.rados.0.smithi084.stdout:1491: writing smithi08434663-23 from 2717777 to 2823675 tid 3 2024-09-18T14:06:24.243 INFO:tasks.rados.rados.0.smithi084.stdout:1492: copy_from oid 36 from oid 47 current snap is 162 2024-09-18T14:06:24.243 INFO:tasks.rados.rados.0.smithi084.stdout:1491: finishing write tid 1 to smithi08434663-23 2024-09-18T14:06:24.243 INFO:tasks.rados.rados.0.smithi084.stdout:1491: finishing write tid 2 to smithi08434663-23 2024-09-18T14:06:24.243 INFO:tasks.rados.rados.0.smithi084.stdout:1493: delete oid 10 current snap is 162 2024-09-18T14:06:24.244 INFO:tasks.rados.rados.0.smithi084.stdout:1491: finishing write tid 3 to smithi08434663-23 2024-09-18T14:06:24.244 INFO:tasks.rados.rados.0.smithi084.stdout:1491: finishing write tid 4 to smithi08434663-23 2024-09-18T14:06:24.245 INFO:tasks.rados.rados.0.smithi084.stdout:1491: finishing write tid 5 to smithi08434663-23 2024-09-18T14:06:24.245 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 569 (ObjNum 525 snap 162 seq_num 525) dirty exists 2024-09-18T14:06:24.245 INFO:tasks.rados.rados.0.smithi084.stdout:1491: left oid 23 (ObjNum 525 snap 162 seq_num 525) 2024-09-18T14:06:24.248 INFO:tasks.rados.rados.0.smithi084.stdout:1491: done (2 left) 2024-09-18T14:06:24.248 INFO:tasks.rados.rados.0.smithi084.stdout:1493: done (1 left) 2024-09-18T14:06:24.248 INFO:tasks.rados.rados.0.smithi084.stdout:1494: copy_from oid 34 from oid 9 current snap is 162 2024-09-18T14:06:24.248 INFO:tasks.rados.rados.0.smithi084.stdout:1495: copy_from oid 17 from oid 12 current snap is 162 2024-09-18T14:06:24.248 INFO:tasks.rados.rados.0.smithi084.stdout:1496: read oid 19 snap -1 2024-09-18T14:06:24.248 INFO:tasks.rados.rados.0.smithi084.stdout:1496: expect deleted 2024-09-18T14:06:24.248 INFO:tasks.rados.rados.0.smithi084.stdout:1497: write oid 15 current snap is 162 2024-09-18T14:06:24.248 INFO:tasks.rados.rados.0.smithi084.stdout:1497: seq_num 527 ranges {457289=484580,1438834=548839,2772623=84666} 2024-09-18T14:06:24.254 INFO:tasks.rados.rados.0.smithi084.stdout:1497: writing smithi08434663-15 from 457289 to 941869 tid 1 2024-09-18T14:06:24.259 INFO:tasks.rados.rados.0.smithi084.stdout:1497: writing smithi08434663-15 from 1438834 to 1987673 tid 2 2024-09-18T14:06:24.264 INFO:tasks.rados.rados.0.smithi084.stdout:1497: writing smithi08434663-15 from 2772623 to 2857289 tid 3 2024-09-18T14:06:24.264 INFO:tasks.rados.rados.0.smithi084.stdout:1498: snap_remove snap 159 2024-09-18T14:06:25.229 INFO:tasks.rados.rados.0.smithi084.stdout:1498: done (5 left) 2024-09-18T14:06:25.229 INFO:tasks.rados.rados.0.smithi084.stdout:1499: snap_create 2024-09-18T14:06:25.229 INFO:tasks.rados.rados.0.smithi084.stdout:1495: finishing copy_from to smithi08434663-17 2024-09-18T14:06:25.229 INFO:tasks.rados.rados.0.smithi084.stdout:1495: got expected ENOENT (src dne) 2024-09-18T14:06:25.229 INFO:tasks.rados.rados.0.smithi084.stdout:1495: finishing copy_from racing read to smithi08434663-17 2024-09-18T14:06:25.229 INFO:tasks.rados.rados.0.smithi084.stdout:1492: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:06:25.230 INFO:tasks.rados.rados.0.smithi084.stdout:1492: finishing copy_from to smithi08434663-36 2024-09-18T14:06:25.230 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 257 (ObjNum 373 snap 115 seq_num 373) dirty exists 2024-09-18T14:06:25.230 INFO:tasks.rados.rados.0.smithi084.stdout:1497: finishing write tid 1 to smithi08434663-15 2024-09-18T14:06:25.230 INFO:tasks.rados.rados.0.smithi084.stdout:1494: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:06:25.230 INFO:tasks.rados.rados.0.smithi084.stdout:1494: finishing copy_from to smithi08434663-34 2024-09-18T14:06:25.230 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 561 (ObjNum 429 snap 134 seq_num 429) dirty exists 2024-09-18T14:06:25.230 INFO:tasks.rados.rados.0.smithi084.stdout:1497: finishing write tid 2 to smithi08434663-15 2024-09-18T14:06:25.230 INFO:tasks.rados.rados.0.smithi084.stdout:1497: finishing write tid 3 to smithi08434663-15 2024-09-18T14:06:25.230 INFO:tasks.rados.rados.0.smithi084.stdout:1497: finishing write tid 4 to smithi08434663-15 2024-09-18T14:06:25.230 INFO:tasks.rados.rados.0.smithi084.stdout:1497: finishing write tid 5 to smithi08434663-15 2024-09-18T14:06:25.230 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 572 (ObjNum 527 snap 162 seq_num 527) dirty exists 2024-09-18T14:06:25.230 INFO:tasks.rados.rados.0.smithi084.stdout:1497: left oid 15 (ObjNum 527 snap 162 seq_num 527) 2024-09-18T14:06:26.241 INFO:tasks.rados.rados.0.smithi084.stdout:1492: done (5 left) 2024-09-18T14:06:26.242 INFO:tasks.rados.rados.0.smithi084.stdout:1494: done (4 left) 2024-09-18T14:06:26.242 INFO:tasks.rados.rados.0.smithi084.stdout:1495: done (3 left) 2024-09-18T14:06:26.242 INFO:tasks.rados.rados.0.smithi084.stdout:1496: done (2 left) 2024-09-18T14:06:26.242 INFO:tasks.rados.rados.0.smithi084.stdout:1497: done (1 left) 2024-09-18T14:06:26.242 INFO:tasks.rados.rados.0.smithi084.stdout:1499: done (0 left) 2024-09-18T14:06:26.242 INFO:tasks.rados.rados.0.smithi084.stdout:1500: snap_remove snap 160 2024-09-18T14:06:27.250 INFO:tasks.rados.rados.0.smithi084.stdout:1500: done (0 left) 2024-09-18T14:06:27.250 INFO:tasks.rados.rados.0.smithi084.stdout:1501: rollback oid 50 current snap is 163 2024-09-18T14:06:27.250 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 50 to 153 2024-09-18T14:06:27.250 INFO:tasks.rados.rados.0.smithi084.stdout:1502: write oid 32 current snap is 163 2024-09-18T14:06:27.250 INFO:tasks.rados.rados.0.smithi084.stdout:1502: seq_num 528 ranges {474096=559829,1780963=408850,2800046=74050} 2024-09-18T14:06:27.255 INFO:tasks.rados.rados.0.smithi084.stdout:1502: writing smithi08434663-32 from 474096 to 1033925 tid 1 2024-09-18T14:06:27.262 INFO:tasks.rados.rados.0.smithi084.stdout:1502: writing smithi08434663-32 from 1780963 to 2189813 tid 2 2024-09-18T14:06:27.266 INFO:tasks.rados.rados.0.smithi084.stdout:1502: writing smithi08434663-32 from 2800046 to 2874096 tid 3 2024-09-18T14:06:27.267 INFO:tasks.rados.rados.0.smithi084.stdout:1503: read oid 41 snap -1 2024-09-18T14:06:27.267 INFO:tasks.rados.rados.0.smithi084.stdout:1503: expect (ObjNum 482 snap 151 seq_num 482) 2024-09-18T14:06:27.267 INFO:tasks.rados.rados.0.smithi084.stdout:1504: copy_from oid 45 from oid 28 current snap is 163 2024-09-18T14:06:27.267 INFO:tasks.rados.rados.0.smithi084.stdout:1501: finishing rollback tid 0 to smithi08434663-50 2024-09-18T14:06:27.267 INFO:tasks.rados.rados.0.smithi084.stdout:1501: finishing rollback tid 1 to smithi08434663-50 2024-09-18T14:06:27.267 INFO:tasks.rados.rados.0.smithi084.stdout:1501: finishing rollback tid 2 to smithi08434663-50 2024-09-18T14:06:27.267 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 779 (ObjNum 386 snap 119 seq_num 386) dirty exists 2024-09-18T14:06:27.267 INFO:tasks.rados.rados.0.smithi084.stdout:1501: done (3 left) 2024-09-18T14:06:27.267 INFO:tasks.rados.rados.0.smithi084.stdout:1505: snap_create 2024-09-18T14:06:27.267 INFO:tasks.rados.rados.0.smithi084.stdout:1502: finishing write tid 1 to smithi08434663-32 2024-09-18T14:06:27.267 INFO:tasks.rados.rados.0.smithi084.stdout:1502: finishing write tid 2 to smithi08434663-32 2024-09-18T14:06:27.268 INFO:tasks.rados.rados.0.smithi084.stdout:1502: finishing write tid 3 to smithi08434663-32 2024-09-18T14:06:27.268 INFO:tasks.rados.rados.0.smithi084.stdout:1502: finishing write tid 4 to smithi08434663-32 2024-09-18T14:06:27.268 INFO:tasks.rados.rados.0.smithi084.stdout:1502: finishing write tid 5 to smithi08434663-32 2024-09-18T14:06:27.268 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 784 (ObjNum 528 snap 163 seq_num 528) dirty exists 2024-09-18T14:06:27.268 INFO:tasks.rados.rados.0.smithi084.stdout:1502: left oid 32 (ObjNum 528 snap 163 seq_num 528) 2024-09-18T14:06:27.273 INFO:tasks.rados.rados.0.smithi084.stdout:1503: expect (ObjNum 482 snap 151 seq_num 482) 2024-09-18T14:06:27.368 INFO:tasks.rados.rados.0.smithi084.stdout:1504: finishing copy_from racing read to smithi08434663-45 2024-09-18T14:06:27.368 INFO:tasks.rados.rados.0.smithi084.stdout:1504: finishing copy_from to smithi08434663-45 2024-09-18T14:06:27.368 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 336 (ObjNum 386 snap 119 seq_num 386) dirty exists 2024-09-18T14:06:28.262 INFO:tasks.rados.rados.0.smithi084.stdout:1502: done (3 left) 2024-09-18T14:06:28.263 INFO:tasks.rados.rados.0.smithi084.stdout:1503: done (2 left) 2024-09-18T14:06:28.263 INFO:tasks.rados.rados.0.smithi084.stdout:1504: done (1 left) 2024-09-18T14:06:28.263 INFO:tasks.rados.rados.0.smithi084.stdout:1505: done (0 left) 2024-09-18T14:06:28.263 INFO:tasks.rados.rados.0.smithi084.stdout:1506: delete oid 8 current snap is 164 2024-09-18T14:06:28.264 INFO:tasks.rados.rados.0.smithi084.stdout:1506: done (0 left) 2024-09-18T14:06:28.264 INFO:tasks.rados.rados.0.smithi084.stdout:1507: read oid 16 snap 153 2024-09-18T14:06:28.264 INFO:tasks.rados.rados.0.smithi084.stdout:1507: expect deleted 2024-09-18T14:06:28.264 INFO:tasks.rados.rados.0.smithi084.stdout:1508: rollback oid 25 current snap is 164 2024-09-18T14:06:28.265 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 25 to 142 2024-09-18T14:06:28.265 INFO:tasks.rados.rados.0.smithi084.stdout:1509: read oid 14 snap -1 2024-09-18T14:06:28.265 INFO:tasks.rados.rados.0.smithi084.stdout:1509: expect (ObjNum 513 snap 158 seq_num 513) 2024-09-18T14:06:28.265 INFO:tasks.rados.rados.0.smithi084.stdout:1510: snap_create 2024-09-18T14:06:28.267 INFO:tasks.rados.rados.0.smithi084.stdout:1508: finishing rollback tid 0 to smithi08434663-25 2024-09-18T14:06:28.269 INFO:tasks.rados.rados.0.smithi084.stdout:1508: finishing rollback tid 1 to smithi08434663-25 2024-09-18T14:06:28.269 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 557 (ObjNum 129 snap 0 seq_num 2495230672) dirty dne 2024-09-18T14:06:28.270 INFO:tasks.rados.rados.0.smithi084.stdout:1509: expect (ObjNum 513 snap 158 seq_num 513) 2024-09-18T14:06:29.256 INFO:tasks.rados.rados.0.smithi084.stdout:1507: done (3 left) 2024-09-18T14:06:29.257 INFO:tasks.rados.rados.0.smithi084.stdout:1508: done (2 left) 2024-09-18T14:06:29.257 INFO:tasks.rados.rados.0.smithi084.stdout:1509: done (1 left) 2024-09-18T14:06:29.257 INFO:tasks.rados.rados.0.smithi084.stdout:1510: done (0 left) 2024-09-18T14:06:29.257 INFO:tasks.rados.rados.0.smithi084.stdout:1511: snap_remove snap 164 2024-09-18T14:06:30.266 INFO:tasks.rados.rados.0.smithi084.stdout:1511: done (0 left) 2024-09-18T14:06:30.267 INFO:tasks.rados.rados.0.smithi084.stdout:1512: delete oid 27 current snap is 165 2024-09-18T14:06:30.277 INFO:tasks.rados.rados.0.smithi084.stdout:1512: done (0 left) 2024-09-18T14:06:30.277 INFO:tasks.rados.rados.0.smithi084.stdout:1513: delete oid 31 current snap is 165 2024-09-18T14:06:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:1513: done (0 left) 2024-09-18T14:06:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:1514: write oid 38 current snap is 165 2024-09-18T14:06:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:1514: seq_num 532 ranges {541324=460825,1498538=416188,2393385=547939} 2024-09-18T14:06:30.287 INFO:tasks.rados.rados.0.smithi084.stdout:1514: writing smithi08434663-38 from 541324 to 1002149 tid 1 2024-09-18T14:06:30.292 INFO:tasks.rados.rados.0.smithi084.stdout:1514: writing smithi08434663-38 from 1498538 to 1914726 tid 2 2024-09-18T14:06:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1514: writing smithi08434663-38 from 2393385 to 2941324 tid 3 2024-09-18T14:06:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1515: read oid 44 snap -1 2024-09-18T14:06:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1515: expect (ObjNum 470 snap 147 seq_num 470) 2024-09-18T14:06:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1516: write oid 22 current snap is 165 2024-09-18T14:06:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:1516: seq_num 533 ranges {558131=652427,1673373=559846,2820808=137323} 2024-09-18T14:06:30.304 INFO:tasks.rados.rados.0.smithi084.stdout:1516: writing smithi08434663-22 from 558131 to 1210558 tid 1 2024-09-18T14:06:30.310 INFO:tasks.rados.rados.0.smithi084.stdout:1516: writing smithi08434663-22 from 1673373 to 2233219 tid 2 2024-09-18T14:06:30.314 INFO:tasks.rados.rados.0.smithi084.stdout:1516: writing smithi08434663-22 from 2820808 to 2958131 tid 3 2024-09-18T14:06:30.315 INFO:tasks.rados.rados.0.smithi084.stdout:1517: snap_remove snap 153 2024-09-18T14:06:31.273 INFO:tasks.rados.rados.0.smithi084.stdout:1517: done (3 left) 2024-09-18T14:06:31.273 INFO:tasks.rados.rados.0.smithi084.stdout:1518: read oid 10 snap -1 2024-09-18T14:06:31.273 INFO:tasks.rados.rados.0.smithi084.stdout:1518: expect deleted 2024-09-18T14:06:31.273 INFO:tasks.rados.rados.0.smithi084.stdout:1519: read oid 27 snap 141 2024-09-18T14:06:31.273 INFO:tasks.rados.rados.0.smithi084.stdout:1519: expect deleted 2024-09-18T14:06:31.273 INFO:tasks.rados.rados.0.smithi084.stdout:1520: read oid 42 snap 141 2024-09-18T14:06:31.273 INFO:tasks.rados.rados.0.smithi084.stdout:1520: expect deleted 2024-09-18T14:06:31.273 INFO:tasks.rados.rados.0.smithi084.stdout:1521: snap_remove snap 149 2024-09-18T14:06:32.213 INFO:tasks.rados.rados.0.smithi084.stdout:1521: done (6 left) 2024-09-18T14:06:32.213 INFO:tasks.rados.rados.0.smithi084.stdout:1522: write (excl) oid 13 current snap is 165 2024-09-18T14:06:32.213 INFO:tasks.rados.rados.0.smithi084.stdout:1522: seq_num 534 ranges {574938=727676,1731855=703504,2848231=126707} 2024-09-18T14:06:32.220 INFO:tasks.rados.rados.0.smithi084.stdout:1522: writing smithi08434663-13 from 574938 to 1302614 tid 1 2024-09-18T14:06:32.226 INFO:tasks.rados.rados.0.smithi084.stdout:1522: writing smithi08434663-13 from 1731855 to 2435359 tid 2 2024-09-18T14:06:32.230 INFO:tasks.rados.rados.0.smithi084.stdout:1522: writing smithi08434663-13 from 2848231 to 2974938 tid 3 2024-09-18T14:06:32.230 INFO:tasks.rados.rados.0.smithi084.stdout:1523: write (excl) oid 46 current snap is 165 2024-09-18T14:06:32.230 INFO:tasks.rados.rados.0.smithi084.stdout:1514: finishing write tid 1 to smithi08434663-38 2024-09-18T14:06:32.230 INFO:tasks.rados.rados.0.smithi084.stdout:1523: seq_num 535 ranges {591745=402925,1790337=563515} 2024-09-18T14:06:32.235 INFO:tasks.rados.rados.0.smithi084.stdout:1523: writing smithi08434663-46 from 591745 to 994670 tid 1 2024-09-18T14:06:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:1523: writing smithi08434663-46 from 1790337 to 2353852 tid 2 2024-09-18T14:06:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:1524: rollback oid 7 current snap is 165 2024-09-18T14:06:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 7 to 157 2024-09-18T14:06:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:1514: finishing write tid 2 to smithi08434663-38 2024-09-18T14:06:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:1514: finishing write tid 3 to smithi08434663-38 2024-09-18T14:06:32.244 INFO:tasks.rados.rados.0.smithi084.stdout:1514: finishing write tid 4 to smithi08434663-38 2024-09-18T14:06:32.244 INFO:tasks.rados.rados.0.smithi084.stdout:1514: finishing write tid 5 to smithi08434663-38 2024-09-18T14:06:32.244 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 410 (ObjNum 532 snap 165 seq_num 532) dirty exists 2024-09-18T14:06:32.244 INFO:tasks.rados.rados.0.smithi084.stdout:1514: left oid 38 (ObjNum 532 snap 165 seq_num 532) 2024-09-18T14:06:32.244 INFO:tasks.rados.rados.0.smithi084.stdout:1515: expect (ObjNum 470 snap 147 seq_num 470) 2024-09-18T14:06:32.369 INFO:tasks.rados.rados.0.smithi084.stdout:1516: finishing write tid 1 to smithi08434663-22 2024-09-18T14:06:32.369 INFO:tasks.rados.rados.0.smithi084.stdout:1516: finishing write tid 2 to smithi08434663-22 2024-09-18T14:06:32.370 INFO:tasks.rados.rados.0.smithi084.stdout:1516: finishing write tid 3 to smithi08434663-22 2024-09-18T14:06:32.370 INFO:tasks.rados.rados.0.smithi084.stdout:1516: finishing write tid 4 to smithi08434663-22 2024-09-18T14:06:32.370 INFO:tasks.rados.rados.0.smithi084.stdout:1516: finishing write tid 5 to smithi08434663-22 2024-09-18T14:06:32.370 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 696 (ObjNum 533 snap 165 seq_num 533) dirty exists 2024-09-18T14:06:32.370 INFO:tasks.rados.rados.0.smithi084.stdout:1516: left oid 22 (ObjNum 533 snap 165 seq_num 533) 2024-09-18T14:06:32.370 INFO:tasks.rados.rados.0.smithi084.stdout:1514: done (8 left) 2024-09-18T14:06:32.370 INFO:tasks.rados.rados.0.smithi084.stdout:1515: done (7 left) 2024-09-18T14:06:32.370 INFO:tasks.rados.rados.0.smithi084.stdout:1516: done (6 left) 2024-09-18T14:06:32.370 INFO:tasks.rados.rados.0.smithi084.stdout:1525: read oid 2 snap -1 2024-09-18T14:06:32.370 INFO:tasks.rados.rados.0.smithi084.stdout:1525: expect (ObjNum 461 snap 141 seq_num 461) 2024-09-18T14:06:32.370 INFO:tasks.rados.rados.0.smithi084.stdout:1526: write oid 26 current snap is 165 2024-09-18T14:06:32.370 INFO:tasks.rados.rados.0.smithi084.stdout:1526: seq_num 536 ranges {608552=478174,1848819=707173} 2024-09-18T14:06:32.375 INFO:tasks.rados.rados.0.smithi084.stdout:1526: writing smithi08434663-26 from 608552 to 1086726 tid 1 2024-09-18T14:06:32.385 INFO:tasks.rados.rados.0.smithi084.stdout:1526: writing smithi08434663-26 from 1848819 to 2555992 tid 2 2024-09-18T14:06:32.385 INFO:tasks.rados.rados.0.smithi084.stdout:1518: done (7 left) 2024-09-18T14:06:32.385 INFO:tasks.rados.rados.0.smithi084.stdout:1519: done (6 left) 2024-09-18T14:06:32.385 INFO:tasks.rados.rados.0.smithi084.stdout:1520: done (5 left) 2024-09-18T14:06:32.385 INFO:tasks.rados.rados.0.smithi084.stdout:1527: read oid 9 snap -1 2024-09-18T14:06:32.385 INFO:tasks.rados.rados.0.smithi084.stdout:1527: expect (ObjNum 429 snap 134 seq_num 429) 2024-09-18T14:06:32.385 INFO:tasks.rados.rados.0.smithi084.stdout:1528: copy_from oid 29 from oid 5 current snap is 165 2024-09-18T14:06:32.385 INFO:tasks.rados.rados.0.smithi084.stdout:1522: finishing write tid 1 to smithi08434663-13 2024-09-18T14:06:32.385 INFO:tasks.rados.rados.0.smithi084.stdout:1522: finishing write tid 2 to smithi08434663-13 2024-09-18T14:06:32.385 INFO:tasks.rados.rados.0.smithi084.stdout:1522: finishing write tid 3 to smithi08434663-13 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:1522: finishing write tid 4 to smithi08434663-13 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:1522: finishing write tid 5 to smithi08434663-13 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 571 (ObjNum 534 snap 165 seq_num 534) dirty exists 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:1522: left oid 13 (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:1523: finishing write tid 1 to smithi08434663-46 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:1524: finishing rollback tid 0 to smithi08434663-7 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:1524: finishing rollback tid 1 to smithi08434663-7 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:1524: finishing rollback tid 2 to smithi08434663-7 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 572 (ObjNum 511 snap 157 seq_num 511) dirty exists 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:1523: finishing write tid 2 to smithi08434663-46 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:1523: finishing write tid 3 to smithi08434663-46 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:1523: finishing write tid 4 to smithi08434663-46 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 582 (ObjNum 535 snap 165 seq_num 535) dirty exists 2024-09-18T14:06:32.386 INFO:tasks.rados.rados.0.smithi084.stdout:1523: left oid 46 (ObjNum 535 snap 165 seq_num 535) 2024-09-18T14:06:32.387 INFO:tasks.rados.rados.0.smithi084.stdout:1525: expect (ObjNum 461 snap 141 seq_num 461) 2024-09-18T14:06:32.502 INFO:tasks.rados.rados.0.smithi084.stdout:1526: finishing write tid 1 to smithi08434663-26 2024-09-18T14:06:32.502 INFO:tasks.rados.rados.0.smithi084.stdout:1526: finishing write tid 2 to smithi08434663-26 2024-09-18T14:06:32.502 INFO:tasks.rados.rados.0.smithi084.stdout:1527: expect (ObjNum 429 snap 134 seq_num 429) 2024-09-18T14:06:32.633 INFO:tasks.rados.rados.0.smithi084.stdout:1526: finishing write tid 3 to smithi08434663-26 2024-09-18T14:06:32.633 INFO:tasks.rados.rados.0.smithi084.stdout:1526: finishing write tid 4 to smithi08434663-26 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 614 (ObjNum 536 snap 165 seq_num 536) dirty exists 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:1526: left oid 26 (ObjNum 536 snap 165 seq_num 536) 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:1528: finishing copy_from racing read to smithi08434663-29 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:1528: finishing copy_from to smithi08434663-29 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 214 (ObjNum 494 snap 152 seq_num 494) dirty exists 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:1522: done (6 left) 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:1523: done (5 left) 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:1524: done (4 left) 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:1525: done (3 left) 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:1526: done (2 left) 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:1527: done (1 left) 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:1528: done (0 left) 2024-09-18T14:06:32.634 INFO:tasks.rados.rados.0.smithi084.stdout:1529: snap_create 2024-09-18T14:06:33.282 INFO:tasks.rados.rados.0.smithi084.stdout:1529: done (0 left) 2024-09-18T14:06:33.343 INFO:tasks.rados.rados.0.smithi084.stdout:1530: delete oid 25 current snap is 166 2024-09-18T14:06:33.344 INFO:tasks.rados.rados.0.smithi084.stdout:1530: done (0 left) 2024-09-18T14:06:33.344 INFO:tasks.rados.rados.0.smithi084.stdout:1531: snap_create 2024-09-18T14:06:34.286 INFO:tasks.rados.rados.0.smithi084.stdout:1531: done (0 left) 2024-09-18T14:06:34.286 INFO:tasks.rados.rados.0.smithi084.stdout:1532: snap_remove snap 151 2024-09-18T14:06:35.299 INFO:tasks.rados.rados.0.smithi084.stdout:1532: done (0 left) 2024-09-18T14:06:35.299 INFO:tasks.rados.rados.0.smithi084.stdout:1533: read oid 48 snap -1 2024-09-18T14:06:35.299 INFO:tasks.rados.rados.0.smithi084.stdout:1533: expect deleted 2024-09-18T14:06:35.299 INFO:tasks.rados.rados.0.smithi084.stdout:1534: write (excl) oid 6 current snap is 167 2024-09-18T14:06:35.299 INFO:tasks.rados.rados.0.smithi084.stdout:1534: seq_num 538 ranges {642166=628672,1849430=710842} 2024-09-18T14:06:35.307 INFO:tasks.rados.rados.0.smithi084.stdout:1534: writing smithi08434663-6 from 642166 to 1270838 tid 1 2024-09-18T14:06:35.314 INFO:tasks.rados.rados.0.smithi084.stdout:1534: writing smithi08434663-6 from 1849430 to 2560272 tid 2 2024-09-18T14:06:35.314 INFO:tasks.rados.rados.0.smithi084.stdout:1535: write oid 18 current snap is 167 2024-09-18T14:06:35.314 INFO:tasks.rados.rados.0.smithi084.stdout:1535: seq_num 539 ranges {658973=703921,1907912=570853,2985346=73627} 2024-09-18T14:06:35.321 INFO:tasks.rados.rados.0.smithi084.stdout:1535: writing smithi08434663-18 from 658973 to 1362894 tid 1 2024-09-18T14:06:35.327 INFO:tasks.rados.rados.0.smithi084.stdout:1535: writing smithi08434663-18 from 1907912 to 2478765 tid 2 2024-09-18T14:06:35.331 INFO:tasks.rados.rados.0.smithi084.stdout:1535: writing smithi08434663-18 from 2985346 to 3058973 tid 3 2024-09-18T14:06:35.331 INFO:tasks.rados.rados.0.smithi084.stdout:1536: write (excl) oid 12 current snap is 167 2024-09-18T14:06:35.331 INFO:tasks.rados.rados.0.smithi084.stdout:1536: seq_num 540 ranges {675780=495523,1682747=714511,3012769=63011} 2024-09-18T14:06:35.337 INFO:tasks.rados.rados.0.smithi084.stdout:1536: writing smithi08434663-12 from 675780 to 1171303 tid 1 2024-09-18T14:06:35.343 INFO:tasks.rados.rados.0.smithi084.stdout:1536: writing smithi08434663-12 from 1682747 to 2397258 tid 2 2024-09-18T14:06:35.348 INFO:tasks.rados.rados.0.smithi084.stdout:1536: writing smithi08434663-12 from 3012769 to 3075780 tid 3 2024-09-18T14:06:35.348 INFO:tasks.rados.rados.0.smithi084.stdout:1537: write (excl) oid 3 current snap is 167 2024-09-18T14:06:35.348 INFO:tasks.rados.rados.0.smithi084.stdout:1537: seq_num 541 ranges {692587=570772,2024876=574522,3040192=52395} 2024-09-18T14:06:35.354 INFO:tasks.rados.rados.0.smithi084.stdout:1537: writing smithi08434663-3 from 692587 to 1263359 tid 1 2024-09-18T14:06:35.362 INFO:tasks.rados.rados.0.smithi084.stdout:1537: writing smithi08434663-3 from 2024876 to 2599398 tid 2 2024-09-18T14:06:35.365 INFO:tasks.rados.rados.0.smithi084.stdout:1537: writing smithi08434663-3 from 3040192 to 3092587 tid 3 2024-09-18T14:06:35.365 INFO:tasks.rados.rados.0.smithi084.stdout:1538: delete oid 20 current snap is 167 2024-09-18T14:06:35.365 INFO:tasks.rados.rados.0.smithi084.stdout:1534: finishing write tid 1 to smithi08434663-6 2024-09-18T14:06:35.365 INFO:tasks.rados.rados.0.smithi084.stdout:1534: finishing write tid 2 to smithi08434663-6 2024-09-18T14:06:35.365 INFO:tasks.rados.rados.0.smithi084.stdout:1534: finishing write tid 3 to smithi08434663-6 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:1534: finishing write tid 4 to smithi08434663-6 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 575 (ObjNum 538 snap 167 seq_num 538) dirty exists 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:1534: left oid 6 (ObjNum 538 snap 167 seq_num 538) 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:1535: finishing write tid 1 to smithi08434663-18 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:1535: finishing write tid 2 to smithi08434663-18 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:1535: finishing write tid 3 to smithi08434663-18 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:1535: finishing write tid 4 to smithi08434663-18 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:1535: finishing write tid 5 to smithi08434663-18 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 700 (ObjNum 539 snap 167 seq_num 539) dirty exists 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:1535: left oid 18 (ObjNum 539 snap 167 seq_num 539) 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:1536: finishing write tid 1 to smithi08434663-12 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:1536: finishing write tid 2 to smithi08434663-12 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:1536: finishing write tid 3 to smithi08434663-12 2024-09-18T14:06:35.366 INFO:tasks.rados.rados.0.smithi084.stdout:1536: finishing write tid 4 to smithi08434663-12 2024-09-18T14:06:35.367 INFO:tasks.rados.rados.0.smithi084.stdout:1536: finishing write tid 5 to smithi08434663-12 2024-09-18T14:06:35.367 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 812 (ObjNum 540 snap 167 seq_num 540) dirty exists 2024-09-18T14:06:35.367 INFO:tasks.rados.rados.0.smithi084.stdout:1536: left oid 12 (ObjNum 540 snap 167 seq_num 540) 2024-09-18T14:06:35.367 INFO:tasks.rados.rados.0.smithi084.stdout:1537: finishing write tid 1 to smithi08434663-3 2024-09-18T14:06:35.368 INFO:tasks.rados.rados.0.smithi084.stdout:1537: finishing write tid 2 to smithi08434663-3 2024-09-18T14:06:35.368 INFO:tasks.rados.rados.0.smithi084.stdout:1533: done (5 left) 2024-09-18T14:06:35.368 INFO:tasks.rados.rados.0.smithi084.stdout:1534: done (4 left) 2024-09-18T14:06:35.368 INFO:tasks.rados.rados.0.smithi084.stdout:1535: done (3 left) 2024-09-18T14:06:35.368 INFO:tasks.rados.rados.0.smithi084.stdout:1536: done (2 left) 2024-09-18T14:06:35.368 INFO:tasks.rados.rados.0.smithi084.stdout:1538: done (1 left) 2024-09-18T14:06:35.368 INFO:tasks.rados.rados.0.smithi084.stdout:1539: read oid 28 snap -1 2024-09-18T14:06:35.368 INFO:tasks.rados.rados.0.smithi084.stdout:1539: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:06:35.368 INFO:tasks.rados.rados.0.smithi084.stdout:1540: write oid 45 current snap is 167 2024-09-18T14:06:35.368 INFO:tasks.rados.rados.0.smithi084.stdout:1540: seq_num 543 ranges {726201=721270,2141840=578191} 2024-09-18T14:06:35.375 INFO:tasks.rados.rados.0.smithi084.stdout:1540: writing smithi08434663-45 from 726201 to 1447471 tid 1 2024-09-18T14:06:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:1540: writing smithi08434663-45 from 2141840 to 2720031 tid 2 2024-09-18T14:06:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:1541: delete oid 40 current snap is 167 2024-09-18T14:06:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:1537: finishing write tid 3 to smithi08434663-3 2024-09-18T14:06:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:1539: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:06:35.496 INFO:tasks.rados.rados.0.smithi084.stdout:1537: finishing write tid 4 to smithi08434663-3 2024-09-18T14:06:35.496 INFO:tasks.rados.rados.0.smithi084.stdout:1537: finishing write tid 5 to smithi08434663-3 2024-09-18T14:06:35.496 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 704 (ObjNum 541 snap 167 seq_num 541) dirty exists 2024-09-18T14:06:35.496 INFO:tasks.rados.rados.0.smithi084.stdout:1537: left oid 3 (ObjNum 541 snap 167 seq_num 541) 2024-09-18T14:06:35.496 INFO:tasks.rados.rados.0.smithi084.stdout:1540: finishing write tid 1 to smithi08434663-45 2024-09-18T14:06:35.496 INFO:tasks.rados.rados.0.smithi084.stdout:1537: done (3 left) 2024-09-18T14:06:35.496 INFO:tasks.rados.rados.0.smithi084.stdout:1539: done (2 left) 2024-09-18T14:06:35.496 INFO:tasks.rados.rados.0.smithi084.stdout:1541: done (1 left) 2024-09-18T14:06:35.496 INFO:tasks.rados.rados.0.smithi084.stdout:1542: write oid 46 current snap is 167 2024-09-18T14:06:35.496 INFO:tasks.rados.rados.0.smithi084.stdout:1540: finishing write tid 2 to smithi08434663-45 2024-09-18T14:06:35.496 INFO:tasks.rados.rados.0.smithi084.stdout:1540: finishing write tid 3 to smithi08434663-45 2024-09-18T14:06:35.497 INFO:tasks.rados.rados.0.smithi084.stdout:1542: seq_num 545 ranges {759815=471768,1742451=465507,2917178=242637} 2024-09-18T14:06:35.502 INFO:tasks.rados.rados.0.smithi084.stdout:1542: writing smithi08434663-46 from 759815 to 1231583 tid 1 2024-09-18T14:06:35.508 INFO:tasks.rados.rados.0.smithi084.stdout:1542: writing smithi08434663-46 from 1742451 to 2207958 tid 2 2024-09-18T14:06:35.513 INFO:tasks.rados.rados.0.smithi084.stdout:1542: writing smithi08434663-46 from 2917178 to 3159815 tid 3 2024-09-18T14:06:35.513 INFO:tasks.rados.rados.0.smithi084.stdout:1543: write (excl) oid 49 current snap is 167 2024-09-18T14:06:35.513 INFO:tasks.rados.rados.0.smithi084.stdout:1543: seq_num 546 ranges {776622=547017,1800933=725518,2944601=232021} 2024-09-18T14:06:35.520 INFO:tasks.rados.rados.0.smithi084.stdout:1543: writing smithi08434663-49 from 776622 to 1323639 tid 1 2024-09-18T14:06:35.526 INFO:tasks.rados.rados.0.smithi084.stdout:1543: writing smithi08434663-49 from 1800933 to 2526451 tid 2 2024-09-18T14:06:35.530 INFO:tasks.rados.rados.0.smithi084.stdout:1543: writing smithi08434663-49 from 2944601 to 3176622 tid 3 2024-09-18T14:06:35.531 INFO:tasks.rados.rados.0.smithi084.stdout:1544: read oid 20 snap -1 2024-09-18T14:06:35.531 INFO:tasks.rados.rados.0.smithi084.stdout:1544: expect deleted 2024-09-18T14:06:35.531 INFO:tasks.rados.rados.0.smithi084.stdout:1545: read oid 43 snap -1 2024-09-18T14:06:35.531 INFO:tasks.rados.rados.0.smithi084.stdout:1545: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:06:35.531 INFO:tasks.rados.rados.0.smithi084.stdout:1546: read oid 50 snap 141 2024-09-18T14:06:35.531 INFO:tasks.rados.rados.0.smithi084.stdout:1546: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:06:35.531 INFO:tasks.rados.rados.0.smithi084.stdout:1547: snap_remove snap 157 2024-09-18T14:06:36.312 INFO:tasks.rados.rados.0.smithi084.stdout:1547: done (6 left) 2024-09-18T14:06:36.312 INFO:tasks.rados.rados.0.smithi084.stdout:1548: rollback oid 35 current snap is 167 2024-09-18T14:06:36.312 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 35 to 165 2024-09-18T14:06:36.312 INFO:tasks.rados.rados.0.smithi084.stdout:1540: finishing write tid 4 to smithi08434663-45 2024-09-18T14:06:36.312 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 731 (ObjNum 543 snap 167 seq_num 543) dirty exists 2024-09-18T14:06:36.312 INFO:tasks.rados.rados.0.smithi084.stdout:1540: left oid 45 (ObjNum 543 snap 167 seq_num 543) 2024-09-18T14:06:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:1542: finishing write tid 1 to smithi08434663-46 2024-09-18T14:06:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:1542: finishing write tid 2 to smithi08434663-46 2024-09-18T14:06:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:1542: finishing write tid 3 to smithi08434663-46 2024-09-18T14:06:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:1542: finishing write tid 4 to smithi08434663-46 2024-09-18T14:06:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:1542: finishing write tid 5 to smithi08434663-46 2024-09-18T14:06:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 591 (ObjNum 545 snap 167 seq_num 545) dirty exists 2024-09-18T14:06:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:1542: left oid 46 (ObjNum 545 snap 167 seq_num 545) 2024-09-18T14:06:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:1543: finishing write tid 1 to smithi08434663-49 2024-09-18T14:06:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:1543: finishing write tid 2 to smithi08434663-49 2024-09-18T14:06:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:1543: finishing write tid 3 to smithi08434663-49 2024-09-18T14:06:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:1546: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:06:36.429 INFO:tasks.rados.rados.0.smithi084.stdout:1543: finishing write tid 4 to smithi08434663-49 2024-09-18T14:06:36.429 INFO:tasks.rados.rados.0.smithi084.stdout:1543: finishing write tid 5 to smithi08434663-49 2024-09-18T14:06:36.429 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 446 (ObjNum 546 snap 167 seq_num 546) dirty exists 2024-09-18T14:06:36.429 INFO:tasks.rados.rados.0.smithi084.stdout:1543: left oid 49 (ObjNum 546 snap 167 seq_num 546) 2024-09-18T14:06:36.429 INFO:tasks.rados.rados.0.smithi084.stdout:1545: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:06:36.505 INFO:tasks.rados.rados.0.smithi084.stdout:1548: finishing rollback tid 0 to smithi08434663-35 2024-09-18T14:06:36.506 INFO:tasks.rados.rados.0.smithi084.stdout:1548: finishing rollback tid 1 to smithi08434663-35 2024-09-18T14:06:36.506 INFO:tasks.rados.rados.0.smithi084.stdout:1548: finishing rollback tid 2 to smithi08434663-35 2024-09-18T14:06:36.506 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 597 (ObjNum 461 snap 141 seq_num 461) dirty exists 2024-09-18T14:06:36.506 INFO:tasks.rados.rados.0.smithi084.stdout:1540: done (6 left) 2024-09-18T14:06:36.506 INFO:tasks.rados.rados.0.smithi084.stdout:1542: done (5 left) 2024-09-18T14:06:36.506 INFO:tasks.rados.rados.0.smithi084.stdout:1543: done (4 left) 2024-09-18T14:06:36.506 INFO:tasks.rados.rados.0.smithi084.stdout:1544: done (3 left) 2024-09-18T14:06:36.506 INFO:tasks.rados.rados.0.smithi084.stdout:1545: done (2 left) 2024-09-18T14:06:36.506 INFO:tasks.rados.rados.0.smithi084.stdout:1546: done (1 left) 2024-09-18T14:06:36.506 INFO:tasks.rados.rados.0.smithi084.stdout:1548: done (0 left) 2024-09-18T14:06:36.506 INFO:tasks.rados.rados.0.smithi084.stdout:1549: read oid 21 snap -1 2024-09-18T14:06:36.506 INFO:tasks.rados.rados.0.smithi084.stdout:1549: expect (ObjNum 467 snap 145 seq_num 467) 2024-09-18T14:06:36.507 INFO:tasks.rados.rados.0.smithi084.stdout:1550: write oid 27 current snap is 167 2024-09-18T14:06:36.507 INFO:tasks.rados.rados.0.smithi084.stdout:1550: seq_num 547 ranges {793429=622266,1859415=469176,2972024=221405} 2024-09-18T14:06:36.513 INFO:tasks.rados.rados.0.smithi084.stdout:1550: writing smithi08434663-27 from 793429 to 1415695 tid 1 2024-09-18T14:06:36.518 INFO:tasks.rados.rados.0.smithi084.stdout:1550: writing smithi08434663-27 from 1859415 to 2328591 tid 2 2024-09-18T14:06:36.523 INFO:tasks.rados.rados.0.smithi084.stdout:1550: writing smithi08434663-27 from 2972024 to 3193429 tid 3 2024-09-18T14:06:36.523 INFO:tasks.rados.rados.0.smithi084.stdout:1551: copy_from oid 7 from oid 48 current snap is 167 2024-09-18T14:06:36.523 INFO:tasks.rados.rados.0.smithi084.stdout:1549: expect (ObjNum 467 snap 145 seq_num 467) 2024-09-18T14:06:36.644 INFO:tasks.rados.rados.0.smithi084.stdout:1550: finishing write tid 1 to smithi08434663-27 2024-09-18T14:06:36.644 INFO:tasks.rados.rados.0.smithi084.stdout:1551: finishing copy_from to smithi08434663-7 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:1551: got expected ENOENT (src dne) 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:1551: finishing copy_from racing read to smithi08434663-7 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:1550: finishing write tid 2 to smithi08434663-27 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:1549: done (2 left) 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:1551: done (1 left) 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:1552: read oid 40 snap -1 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:1552: expect deleted 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:1553: rollback oid 44 current snap is 167 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 44 to 155 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:1550: finishing write tid 3 to smithi08434663-27 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:1550: finishing write tid 4 to smithi08434663-27 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:1550: finishing write tid 5 to smithi08434663-27 2024-09-18T14:06:36.645 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 634 (ObjNum 547 snap 167 seq_num 547) dirty exists 2024-09-18T14:06:36.646 INFO:tasks.rados.rados.0.smithi084.stdout:1550: left oid 27 (ObjNum 547 snap 167 seq_num 547) 2024-09-18T14:06:36.646 INFO:tasks.rados.rados.0.smithi084.stdout:1550: done (2 left) 2024-09-18T14:06:36.646 INFO:tasks.rados.rados.0.smithi084.stdout:1554: read oid 11 snap 138 2024-09-18T14:06:36.646 INFO:tasks.rados.rados.0.smithi084.stdout:1554: expect (ObjNum 247 snap 57 seq_num 247) 2024-09-18T14:06:36.646 INFO:tasks.rados.rados.0.smithi084.stdout:1555: write oid 16 current snap is 167 2024-09-18T14:06:36.646 INFO:tasks.rados.rados.0.smithi084.stdout:1555: seq_num 548 ranges {410236=413868,1234250=729187,2715800=423063} 2024-09-18T14:06:36.648 INFO:tasks.rados.rados.0.smithi084.stdout:1555: writing smithi08434663-16 from 410236 to 824104 tid 1 2024-09-18T14:06:36.655 INFO:tasks.rados.rados.0.smithi084.stdout:1555: writing smithi08434663-16 from 1234250 to 1963437 tid 2 2024-09-18T14:06:36.661 INFO:tasks.rados.rados.0.smithi084.stdout:1555: writing smithi08434663-16 from 2715800 to 3138863 tid 3 2024-09-18T14:06:36.661 INFO:tasks.rados.rados.0.smithi084.stdout:1556: snap_create 2024-09-18T14:06:36.662 INFO:tasks.rados.rados.0.smithi084.stdout:1553: finishing rollback tid 0 to smithi08434663-44 2024-09-18T14:06:36.662 INFO:tasks.rados.rados.0.smithi084.stdout:1554: expect (ObjNum 247 snap 57 seq_num 247) 2024-09-18T14:06:36.811 INFO:tasks.rados.rados.0.smithi084.stdout:1553: finishing rollback tid 1 to smithi08434663-44 2024-09-18T14:06:36.811 INFO:tasks.rados.rados.0.smithi084.stdout:1553: finishing rollback tid 2 to smithi08434663-44 2024-09-18T14:06:36.811 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 228 (ObjNum 470 snap 147 seq_num 470) dirty exists 2024-09-18T14:06:36.811 INFO:tasks.rados.rados.0.smithi084.stdout:1555: finishing write tid 1 to smithi08434663-16 2024-09-18T14:06:36.811 INFO:tasks.rados.rados.0.smithi084.stdout:1555: finishing write tid 2 to smithi08434663-16 2024-09-18T14:06:36.811 INFO:tasks.rados.rados.0.smithi084.stdout:1555: finishing write tid 3 to smithi08434663-16 2024-09-18T14:06:36.811 INFO:tasks.rados.rados.0.smithi084.stdout:1555: finishing write tid 4 to smithi08434663-16 2024-09-18T14:06:36.811 INFO:tasks.rados.rados.0.smithi084.stdout:1555: finishing write tid 5 to smithi08434663-16 2024-09-18T14:06:36.811 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 602 (ObjNum 548 snap 167 seq_num 548) dirty exists 2024-09-18T14:06:36.812 INFO:tasks.rados.rados.0.smithi084.stdout:1555: left oid 16 (ObjNum 548 snap 167 seq_num 548) 2024-09-18T14:06:37.217 INFO:tasks.rados.rados.0.smithi084.stdout:1552: done (4 left) 2024-09-18T14:06:37.217 INFO:tasks.rados.rados.0.smithi084.stdout:1553: done (3 left) 2024-09-18T14:06:37.217 INFO:tasks.rados.rados.0.smithi084.stdout:1554: done (2 left) 2024-09-18T14:06:37.217 INFO:tasks.rados.rados.0.smithi084.stdout:1555: done (1 left) 2024-09-18T14:06:37.217 INFO:tasks.rados.rados.0.smithi084.stdout:1556: done (0 left) 2024-09-18T14:06:37.217 INFO:tasks.rados.rados.0.smithi084.stdout:1557: write (excl) oid 39 current snap is 168 2024-09-18T14:06:37.217 INFO:tasks.rados.rados.0.smithi084.stdout:1557: seq_num 549 ranges {427043=489117,1576379=472845,2626870=600173} 2024-09-18T14:06:37.222 INFO:tasks.rados.rados.0.smithi084.stdout:1557: writing smithi08434663-39 from 427043 to 916160 tid 1 2024-09-18T14:06:37.228 INFO:tasks.rados.rados.0.smithi084.stdout:1557: writing smithi08434663-39 from 1576379 to 2049224 tid 2 2024-09-18T14:06:37.236 INFO:tasks.rados.rados.0.smithi084.stdout:1557: writing smithi08434663-39 from 2626870 to 3227043 tid 3 2024-09-18T14:06:37.236 INFO:tasks.rados.rados.0.smithi084.stdout:1558: rollback oid 18 current snap is 168 2024-09-18T14:06:37.236 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 18 to 156 2024-09-18T14:06:37.236 INFO:tasks.rados.rados.0.smithi084.stdout:1559: copy_from oid 23 from oid 7 current snap is 168 2024-09-18T14:06:37.236 INFO:tasks.rados.rados.0.smithi084.stdout:1560: snap_remove snap 142 2024-09-18T14:06:38.223 INFO:tasks.rados.rados.0.smithi084.stdout:1560: done (3 left) 2024-09-18T14:06:38.223 INFO:tasks.rados.rados.0.smithi084.stdout:1561: write oid 37 current snap is 168 2024-09-18T14:06:38.223 INFO:tasks.rados.rados.0.smithi084.stdout:1561: seq_num 550 ranges {443850=564366,1634861=732856,2770646=473204} 2024-09-18T14:06:38.229 INFO:tasks.rados.rados.0.smithi084.stdout:1561: writing smithi08434663-37 from 443850 to 1008216 tid 1 2024-09-18T14:06:38.238 INFO:tasks.rados.rados.0.smithi084.stdout:1561: writing smithi08434663-37 from 1634861 to 2367717 tid 2 2024-09-18T14:06:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1561: writing smithi08434663-37 from 2770646 to 3243850 tid 3 2024-09-18T14:06:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1557: finishing write tid 1 to smithi08434663-39 2024-09-18T14:06:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1557: finishing write tid 2 to smithi08434663-39 2024-09-18T14:06:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1558: finishing rollback tid 0 to smithi08434663-18 2024-09-18T14:06:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1558: finishing rollback tid 1 to smithi08434663-18 2024-09-18T14:06:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1558: finishing rollback tid 2 to smithi08434663-18 2024-09-18T14:06:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 707 (ObjNum 463 snap 142 seq_num 463) dirty exists 2024-09-18T14:06:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1557: finishing write tid 3 to smithi08434663-39 2024-09-18T14:06:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1557: finishing write tid 4 to smithi08434663-39 2024-09-18T14:06:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1557: finishing write tid 5 to smithi08434663-39 2024-09-18T14:06:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 572 (ObjNum 549 snap 168 seq_num 549) dirty exists 2024-09-18T14:06:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:1557: left oid 39 (ObjNum 549 snap 168 seq_num 549) 2024-09-18T14:06:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:1559: finishing copy_from racing read to smithi08434663-23 2024-09-18T14:06:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:1559: finishing copy_from to smithi08434663-23 2024-09-18T14:06:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 572 (ObjNum 511 snap 157 seq_num 511) dirty exists 2024-09-18T14:06:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:1557: done (3 left) 2024-09-18T14:06:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:1558: done (2 left) 2024-09-18T14:06:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:1559: done (1 left) 2024-09-18T14:06:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:1562: write (excl) oid 8 current snap is 168 2024-09-18T14:06:38.245 INFO:tasks.rados.rados.0.smithi084.stdout:1562: seq_num 551 ranges {460657=639615,1693343=476514,2681716=578941} 2024-09-18T14:06:38.250 INFO:tasks.rados.rados.0.smithi084.stdout:1562: writing smithi08434663-8 from 460657 to 1100272 tid 1 2024-09-18T14:06:38.256 INFO:tasks.rados.rados.0.smithi084.stdout:1562: writing smithi08434663-8 from 1693343 to 2169857 tid 2 2024-09-18T14:06:38.262 INFO:tasks.rados.rados.0.smithi084.stdout:1562: writing smithi08434663-8 from 2681716 to 3260657 tid 3 2024-09-18T14:06:38.262 INFO:tasks.rados.rados.0.smithi084.stdout:1563: rollback oid 34 current snap is 168 2024-09-18T14:06:38.262 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 34 to 166 2024-09-18T14:06:38.262 INFO:tasks.rados.rados.0.smithi084.stdout:1561: finishing write tid 1 to smithi08434663-37 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:1561: finishing write tid 2 to smithi08434663-37 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:1561: finishing write tid 3 to smithi08434663-37 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:1561: finishing write tid 4 to smithi08434663-37 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:1561: finishing write tid 5 to smithi08434663-37 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 459 (ObjNum 550 snap 168 seq_num 550) dirty exists 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:1561: left oid 37 (ObjNum 550 snap 168 seq_num 550) 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:1562: finishing write tid 1 to smithi08434663-8 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:1562: finishing write tid 2 to smithi08434663-8 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:1561: done (2 left) 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:1564: read oid 7 snap -1 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:1564: expect (ObjNum 511 snap 157 seq_num 511) 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:1565: write (excl) oid 26 current snap is 168 2024-09-18T14:06:38.263 INFO:tasks.rados.rados.0.smithi084.stdout:1565: seq_num 552 ranges {477464=714864,1751825=736525,3225492=51972} 2024-09-18T14:06:38.268 INFO:tasks.rados.rados.0.smithi084.stdout:1565: writing smithi08434663-26 from 477464 to 1192328 tid 1 2024-09-18T14:06:38.276 INFO:tasks.rados.rados.0.smithi084.stdout:1565: writing smithi08434663-26 from 1751825 to 2488350 tid 2 2024-09-18T14:06:38.281 INFO:tasks.rados.rados.0.smithi084.stdout:1565: writing smithi08434663-26 from 3225492 to 3277464 tid 3 2024-09-18T14:06:38.281 INFO:tasks.rados.rados.0.smithi084.stdout:1566: copy_from oid 3 from oid 18 current snap is 168 2024-09-18T14:06:38.281 INFO:tasks.rados.rados.0.smithi084.stdout:1562: finishing write tid 3 to smithi08434663-8 2024-09-18T14:06:38.281 INFO:tasks.rados.rados.0.smithi084.stdout:1562: finishing write tid 4 to smithi08434663-8 2024-09-18T14:06:38.281 INFO:tasks.rados.rados.0.smithi084.stdout:1562: finishing write tid 5 to smithi08434663-8 2024-09-18T14:06:38.282 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 620 (ObjNum 551 snap 168 seq_num 551) dirty exists 2024-09-18T14:06:38.282 INFO:tasks.rados.rados.0.smithi084.stdout:1562: left oid 8 (ObjNum 551 snap 168 seq_num 551) 2024-09-18T14:06:38.282 INFO:tasks.rados.rados.0.smithi084.stdout:1563: finishing rollback tid 0 to smithi08434663-34 2024-09-18T14:06:38.282 INFO:tasks.rados.rados.0.smithi084.stdout:1563: finishing rollback tid 1 to smithi08434663-34 2024-09-18T14:06:38.282 INFO:tasks.rados.rados.0.smithi084.stdout:1563: finishing rollback tid 2 to smithi08434663-34 2024-09-18T14:06:38.282 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 734 (ObjNum 429 snap 134 seq_num 429) dirty exists 2024-09-18T14:06:38.282 INFO:tasks.rados.rados.0.smithi084.stdout:1564: expect (ObjNum 511 snap 157 seq_num 511) 2024-09-18T14:06:38.427 INFO:tasks.rados.rados.0.smithi084.stdout:1565: finishing write tid 1 to smithi08434663-26 2024-09-18T14:06:38.428 INFO:tasks.rados.rados.0.smithi084.stdout:1565: finishing write tid 2 to smithi08434663-26 2024-09-18T14:06:38.428 INFO:tasks.rados.rados.0.smithi084.stdout:1565: finishing write tid 3 to smithi08434663-26 2024-09-18T14:06:38.428 INFO:tasks.rados.rados.0.smithi084.stdout:1565: finishing write tid 4 to smithi08434663-26 2024-09-18T14:06:38.428 INFO:tasks.rados.rados.0.smithi084.stdout:1565: finishing write tid 5 to smithi08434663-26 2024-09-18T14:06:38.428 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 645 (ObjNum 552 snap 168 seq_num 552) dirty exists 2024-09-18T14:06:38.428 INFO:tasks.rados.rados.0.smithi084.stdout:1565: left oid 26 (ObjNum 552 snap 168 seq_num 552) 2024-09-18T14:06:38.428 INFO:tasks.rados.rados.0.smithi084.stdout:1566: finishing copy_from racing read to smithi08434663-3 2024-09-18T14:06:38.428 INFO:tasks.rados.rados.0.smithi084.stdout:1566: finishing copy_from to smithi08434663-3 2024-09-18T14:06:38.428 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 707 (ObjNum 463 snap 142 seq_num 463) dirty exists 2024-09-18T14:06:38.428 INFO:tasks.rados.rados.0.smithi084.stdout:1562: done (4 left) 2024-09-18T14:06:38.429 INFO:tasks.rados.rados.0.smithi084.stdout:1563: done (3 left) 2024-09-18T14:06:38.429 INFO:tasks.rados.rados.0.smithi084.stdout:1564: done (2 left) 2024-09-18T14:06:38.429 INFO:tasks.rados.rados.0.smithi084.stdout:1565: done (1 left) 2024-09-18T14:06:38.429 INFO:tasks.rados.rados.0.smithi084.stdout:1566: done (0 left) 2024-09-18T14:06:38.429 INFO:tasks.rados.rados.0.smithi084.stdout:1567: delete oid 17 current snap is 168 2024-09-18T14:06:38.430 INFO:tasks.rados.rados.0.smithi084.stdout:1567: done (0 left) 2024-09-18T14:06:38.430 INFO:tasks.rados.rados.0.smithi084.stdout:1568: delete oid 47 current snap is 168 2024-09-18T14:06:38.434 INFO:tasks.rados.rados.0.smithi084.stdout:1568: done (0 left) 2024-09-18T14:06:38.434 INFO:tasks.rados.rados.0.smithi084.stdout:1569: read oid 18 snap -1 2024-09-18T14:06:38.434 INFO:tasks.rados.rados.0.smithi084.stdout:1569: expect (ObjNum 463 snap 142 seq_num 463) 2024-09-18T14:06:38.434 INFO:tasks.rados.rados.0.smithi084.stdout:1570: read oid 32 snap -1 2024-09-18T14:06:38.434 INFO:tasks.rados.rados.0.smithi084.stdout:1570: expect (ObjNum 528 snap 163 seq_num 528) 2024-09-18T14:06:38.434 INFO:tasks.rados.rados.0.smithi084.stdout:1571: copy_from oid 33 from oid 27 current snap is 168 2024-09-18T14:06:38.434 INFO:tasks.rados.rados.0.smithi084.stdout:1572: delete oid 2 current snap is 168 2024-09-18T14:06:38.439 INFO:tasks.rados.rados.0.smithi084.stdout:1569: expect (ObjNum 463 snap 142 seq_num 463) 2024-09-18T14:06:38.559 INFO:tasks.rados.rados.0.smithi084.stdout:1570: expect (ObjNum 528 snap 163 seq_num 528) 2024-09-18T14:06:38.701 INFO:tasks.rados.rados.0.smithi084.stdout:1571: finishing copy_from racing read to smithi08434663-33 2024-09-18T14:06:38.701 INFO:tasks.rados.rados.0.smithi084.stdout:1571: finishing copy_from to smithi08434663-33 2024-09-18T14:06:38.701 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 634 (ObjNum 547 snap 167 seq_num 547) dirty exists 2024-09-18T14:06:38.702 INFO:tasks.rados.rados.0.smithi084.stdout:1569: done (3 left) 2024-09-18T14:06:38.702 INFO:tasks.rados.rados.0.smithi084.stdout:1570: done (2 left) 2024-09-18T14:06:38.702 INFO:tasks.rados.rados.0.smithi084.stdout:1571: done (1 left) 2024-09-18T14:06:38.702 INFO:tasks.rados.rados.0.smithi084.stdout:1572: done (0 left) 2024-09-18T14:06:38.702 INFO:tasks.rados.rados.0.smithi084.stdout:1573: snap_remove snap 165 2024-09-18T14:06:39.315 INFO:tasks.rados.rados.0.smithi084.stdout:1573: done (0 left) 2024-09-18T14:06:39.315 INFO:tasks.rados.rados.0.smithi084.stdout:1574: delete oid 32 current snap is 168 2024-09-18T14:06:39.320 INFO:tasks.rados.rados.0.smithi084.stdout:1574: done (0 left) 2024-09-18T14:06:39.320 INFO:tasks.rados.rados.0.smithi084.stdout:1575: copy_from oid 28 from oid 43 current snap is 168 2024-09-18T14:06:39.320 INFO:tasks.rados.rados.0.smithi084.stdout:1576: snap_remove snap 167 2024-09-18T14:06:40.321 INFO:tasks.rados.rados.0.smithi084.stdout:1576: done (1 left) 2024-09-18T14:06:40.321 INFO:tasks.rados.rados.0.smithi084.stdout:1577: snap_create 2024-09-18T14:06:40.321 INFO:tasks.rados.rados.0.smithi084.stdout:1575: finishing copy_from racing read to smithi08434663-28 2024-09-18T14:06:40.321 INFO:tasks.rados.rados.0.smithi084.stdout:1575: finishing copy_from to smithi08434663-28 2024-09-18T14:06:40.321 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 406 (ObjNum 265 snap 67 seq_num 265) dirty exists 2024-09-18T14:06:41.332 INFO:tasks.rados.rados.0.smithi084.stdout:1575: done (1 left) 2024-09-18T14:06:41.332 INFO:tasks.rados.rados.0.smithi084.stdout:1577: done (0 left) 2024-09-18T14:06:41.332 INFO:tasks.rados.rados.0.smithi084.stdout:1578: write (excl) oid 3 current snap is 169 2024-09-18T14:06:41.332 INFO:tasks.rados.rados.0.smithi084.stdout:1578: seq_num 557 ranges {561499=407462,1527882=487521,2729901=557217} 2024-09-18T14:06:41.337 INFO:tasks.rados.rados.0.smithi084.stdout:1578: writing smithi08434663-3 from 561499 to 968961 tid 1 2024-09-18T14:06:41.343 INFO:tasks.rados.rados.0.smithi084.stdout:1578: writing smithi08434663-3 from 1527882 to 2015403 tid 2 2024-09-18T14:06:41.350 INFO:tasks.rados.rados.0.smithi084.stdout:1578: writing smithi08434663-3 from 2729901 to 3287118 tid 3 2024-09-18T14:06:41.350 INFO:tasks.rados.rados.0.smithi084.stdout:1579: snap_remove snap 141 2024-09-18T14:06:42.216 INFO:tasks.rados.rados.0.smithi084.stdout:1579: done (1 left) 2024-09-18T14:06:42.216 INFO:tasks.rados.rados.0.smithi084.stdout:1580: snap_create 2024-09-18T14:06:42.216 INFO:tasks.rados.rados.0.smithi084.stdout:1578: finishing write tid 1 to smithi08434663-3 2024-09-18T14:06:42.216 INFO:tasks.rados.rados.0.smithi084.stdout:1578: finishing write tid 2 to smithi08434663-3 2024-09-18T14:06:42.216 INFO:tasks.rados.rados.0.smithi084.stdout:1578: finishing write tid 3 to smithi08434663-3 2024-09-18T14:06:42.216 INFO:tasks.rados.rados.0.smithi084.stdout:1578: finishing write tid 4 to smithi08434663-3 2024-09-18T14:06:42.216 INFO:tasks.rados.rados.0.smithi084.stdout:1578: finishing write tid 5 to smithi08434663-3 2024-09-18T14:06:42.216 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 711 (ObjNum 557 snap 169 seq_num 557) dirty exists 2024-09-18T14:06:42.216 INFO:tasks.rados.rados.0.smithi084.stdout:1578: left oid 3 (ObjNum 557 snap 169 seq_num 557) 2024-09-18T14:06:43.217 INFO:tasks.rados.rados.0.smithi084.stdout:1578: done (1 left) 2024-09-18T14:06:43.217 INFO:tasks.rados.rados.0.smithi084.stdout:1580: done (0 left) 2024-09-18T14:06:43.218 INFO:tasks.rados.rados.0.smithi084.stdout:1581: snap_remove snap 163 2024-09-18T14:06:44.222 INFO:tasks.rados.rados.0.smithi084.stdout:1581: done (0 left) 2024-09-18T14:06:44.223 INFO:tasks.rados.rados.0.smithi084.stdout:1582: snap_create 2024-09-18T14:06:45.223 INFO:tasks.rados.rados.0.smithi084.stdout:1582: done (0 left) 2024-09-18T14:06:45.223 INFO:tasks.rados.rados.0.smithi084.stdout:1583: snap_remove snap 156 2024-09-18T14:06:46.233 INFO:tasks.rados.rados.0.smithi084.stdout:1583: done (0 left) 2024-09-18T14:06:46.233 INFO:tasks.rados.rados.0.smithi084.stdout:1584: delete oid 10 current snap is 171 2024-09-18T14:06:46.238 INFO:tasks.rados.rados.0.smithi084.stdout:1584: done (0 left) 2024-09-18T14:06:46.239 INFO:tasks.rados.rados.0.smithi084.stdout:1585: write (excl) oid 27 current snap is 171 2024-09-18T14:06:46.239 INFO:tasks.rados.rados.0.smithi084.stdout:1585: seq_num 559 ranges {595113=557960,1644846=491190,2784747=610366} 2024-09-18T14:06:46.245 INFO:tasks.rados.rados.0.smithi084.stdout:1585: writing smithi08434663-27 from 595113 to 1153073 tid 1 2024-09-18T14:06:46.251 INFO:tasks.rados.rados.0.smithi084.stdout:1585: writing smithi08434663-27 from 1644846 to 2136036 tid 2 2024-09-18T14:06:46.258 INFO:tasks.rados.rados.0.smithi084.stdout:1585: writing smithi08434663-27 from 2784747 to 3395113 tid 3 2024-09-18T14:06:46.258 INFO:tasks.rados.rados.0.smithi084.stdout:1586: copy_from oid 35 from oid 48 current snap is 171 2024-09-18T14:06:46.258 INFO:tasks.rados.rados.0.smithi084.stdout:1587: copy_from oid 7 from oid 41 current snap is 171 2024-09-18T14:06:46.259 INFO:tasks.rados.rados.0.smithi084.stdout:1588: delete oid 34 current snap is 171 2024-09-18T14:06:46.259 INFO:tasks.rados.rados.0.smithi084.stdout:1585: finishing write tid 1 to smithi08434663-27 2024-09-18T14:06:46.259 INFO:tasks.rados.rados.0.smithi084.stdout:1586: finishing copy_from to smithi08434663-35 2024-09-18T14:06:46.259 INFO:tasks.rados.rados.0.smithi084.stdout:1586: got expected ENOENT (src dne) 2024-09-18T14:06:46.260 INFO:tasks.rados.rados.0.smithi084.stdout:1586: finishing copy_from racing read to smithi08434663-35 2024-09-18T14:06:46.261 INFO:tasks.rados.rados.0.smithi084.stdout:1585: finishing write tid 2 to smithi08434663-27 2024-09-18T14:06:46.262 INFO:tasks.rados.rados.0.smithi084.stdout:1586: done (3 left) 2024-09-18T14:06:46.262 INFO:tasks.rados.rados.0.smithi084.stdout:1588: done (2 left) 2024-09-18T14:06:46.262 INFO:tasks.rados.rados.0.smithi084.stdout:1589: write oid 30 current snap is 171 2024-09-18T14:06:46.262 INFO:tasks.rados.rados.0.smithi084.stdout:1589: seq_num 561 ranges {628727=708458,2045457=494859,3123240=305487} 2024-09-18T14:06:46.269 INFO:tasks.rados.rados.0.smithi084.stdout:1589: writing smithi08434663-30 from 628727 to 1337185 tid 1 2024-09-18T14:06:46.276 INFO:tasks.rados.rados.0.smithi084.stdout:1589: writing smithi08434663-30 from 2045457 to 2540316 tid 2 2024-09-18T14:06:46.281 INFO:tasks.rados.rados.0.smithi084.stdout:1589: writing smithi08434663-30 from 3123240 to 3428727 tid 3 2024-09-18T14:06:46.281 INFO:tasks.rados.rados.0.smithi084.stdout:1590: delete oid 1 current snap is 171 2024-09-18T14:06:46.281 INFO:tasks.rados.rados.0.smithi084.stdout:1585: finishing write tid 3 to smithi08434663-27 2024-09-18T14:06:46.282 INFO:tasks.rados.rados.0.smithi084.stdout:1585: finishing write tid 4 to smithi08434663-27 2024-09-18T14:06:46.282 INFO:tasks.rados.rados.0.smithi084.stdout:1585: finishing write tid 5 to smithi08434663-27 2024-09-18T14:06:46.282 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 673 (ObjNum 559 snap 171 seq_num 559) dirty exists 2024-09-18T14:06:46.282 INFO:tasks.rados.rados.0.smithi084.stdout:1585: left oid 27 (ObjNum 559 snap 171 seq_num 559) 2024-09-18T14:06:46.282 INFO:tasks.rados.rados.0.smithi084.stdout:1587: finishing copy_from racing read to smithi08434663-7 2024-09-18T14:06:46.282 INFO:tasks.rados.rados.0.smithi084.stdout:1587: finishing copy_from to smithi08434663-7 2024-09-18T14:06:46.282 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 590 (ObjNum 482 snap 151 seq_num 482) dirty exists 2024-09-18T14:06:46.282 INFO:tasks.rados.rados.0.smithi084.stdout:1589: finishing write tid 1 to smithi08434663-30 2024-09-18T14:06:46.282 INFO:tasks.rados.rados.0.smithi084.stdout:1589: finishing write tid 2 to smithi08434663-30 2024-09-18T14:06:46.283 INFO:tasks.rados.rados.0.smithi084.stdout:1585: done (3 left) 2024-09-18T14:06:46.283 INFO:tasks.rados.rados.0.smithi084.stdout:1587: done (2 left) 2024-09-18T14:06:46.283 INFO:tasks.rados.rados.0.smithi084.stdout:1590: done (1 left) 2024-09-18T14:06:46.284 INFO:tasks.rados.rados.0.smithi084.stdout:1591: rollback oid 43 current snap is 171 2024-09-18T14:06:46.284 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 170 2024-09-18T14:06:46.284 INFO:tasks.rados.rados.0.smithi084.stdout:1592: delete oid 12 current snap is 171 2024-09-18T14:06:46.284 INFO:tasks.rados.rados.0.smithi084.stdout:1589: finishing write tid 3 to smithi08434663-30 2024-09-18T14:06:46.284 INFO:tasks.rados.rados.0.smithi084.stdout:1589: finishing write tid 4 to smithi08434663-30 2024-09-18T14:06:46.284 INFO:tasks.rados.rados.0.smithi084.stdout:1589: finishing write tid 5 to smithi08434663-30 2024-09-18T14:06:46.284 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 556 (ObjNum 561 snap 171 seq_num 561) dirty exists 2024-09-18T14:06:46.284 INFO:tasks.rados.rados.0.smithi084.stdout:1589: left oid 30 (ObjNum 561 snap 171 seq_num 561) 2024-09-18T14:06:46.286 INFO:tasks.rados.rados.0.smithi084.stdout:1589: done (2 left) 2024-09-18T14:06:46.286 INFO:tasks.rados.rados.0.smithi084.stdout:1592: done (1 left) 2024-09-18T14:06:46.286 INFO:tasks.rados.rados.0.smithi084.stdout:1593: rollback oid 35 current snap is 171 2024-09-18T14:06:46.286 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 35 to 138 2024-09-18T14:06:46.286 INFO:tasks.rados.rados.0.smithi084.stdout:1594: rollback oid 49 current snap is 171 2024-09-18T14:06:46.286 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 49 to 138 2024-09-18T14:06:46.286 INFO:tasks.rados.rados.0.smithi084.stdout:1595: snap_create 2024-09-18T14:06:46.287 INFO:tasks.rados.rados.0.smithi084.stdout:1591: finishing rollback tid 0 to smithi08434663-43 2024-09-18T14:06:46.287 INFO:tasks.rados.rados.0.smithi084.stdout:1591: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:06:46.287 INFO:tasks.rados.rados.0.smithi084.stdout:1591: finishing rollback tid 2 to smithi08434663-43 2024-09-18T14:06:46.287 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 413 (ObjNum 265 snap 67 seq_num 265) dirty exists 2024-09-18T14:06:46.288 INFO:tasks.rados.rados.0.smithi084.stdout:1593: finishing rollback tid 0 to smithi08434663-35 2024-09-18T14:06:46.288 INFO:tasks.rados.rados.0.smithi084.stdout:1593: finishing rollback tid 1 to smithi08434663-35 2024-09-18T14:06:46.288 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 635 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:06:46.289 INFO:tasks.rados.rados.0.smithi084.stdout:1594: finishing rollback tid 0 to smithi08434663-49 2024-09-18T14:06:46.289 INFO:tasks.rados.rados.0.smithi084.stdout:1594: finishing rollback tid 1 to smithi08434663-49 2024-09-18T14:06:46.290 INFO:tasks.rados.rados.0.smithi084.stdout:1594: finishing rollback tid 2 to smithi08434663-49 2024-09-18T14:06:46.290 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 637 (ObjNum 397 snap 121 seq_num 397) dirty exists 2024-09-18T14:06:47.216 INFO:tasks.rados.rados.0.smithi084.stdout:1591: done (3 left) 2024-09-18T14:06:47.217 INFO:tasks.rados.rados.0.smithi084.stdout:1593: done (2 left) 2024-09-18T14:06:47.217 INFO:tasks.rados.rados.0.smithi084.stdout:1594: done (1 left) 2024-09-18T14:06:47.217 INFO:tasks.rados.rados.0.smithi084.stdout:1595: done (0 left) 2024-09-18T14:06:47.217 INFO:tasks.rados.rados.0.smithi084.stdout:1596: read oid 41 snap -1 2024-09-18T14:06:47.217 INFO:tasks.rados.rados.0.smithi084.stdout:1596: expect (ObjNum 482 snap 151 seq_num 482) 2024-09-18T14:06:47.217 INFO:tasks.rados.rados.0.smithi084.stdout:1597: read oid 20 snap 155 2024-09-18T14:06:47.217 INFO:tasks.rados.rados.0.smithi084.stdout:1597: expect (ObjNum 453 snap 139 seq_num 453) 2024-09-18T14:06:47.217 INFO:tasks.rados.rados.0.smithi084.stdout:1598: delete oid 13 current snap is 172 2024-09-18T14:06:47.223 INFO:tasks.rados.rados.0.smithi084.stdout:1596: expect (ObjNum 482 snap 151 seq_num 482) 2024-09-18T14:06:47.322 INFO:tasks.rados.rados.0.smithi084.stdout:1597: expect (ObjNum 453 snap 139 seq_num 453) 2024-09-18T14:06:47.549 INFO:tasks.rados.rados.0.smithi084.stdout:1596: done (2 left) 2024-09-18T14:06:47.550 INFO:tasks.rados.rados.0.smithi084.stdout:1597: done (1 left) 2024-09-18T14:06:47.550 INFO:tasks.rados.rados.0.smithi084.stdout:1598: done (0 left) 2024-09-18T14:06:47.550 INFO:tasks.rados.rados.0.smithi084.stdout:1599: delete oid 48 current snap is 172 2024-09-18T14:06:47.552 INFO:tasks.rados.rados.0.smithi084.stdout:1599: done (0 left) 2024-09-18T14:06:47.552 INFO:tasks.rados.rados.0.smithi084.stdout:1600: read oid 7 snap -1 2024-09-18T14:06:47.552 INFO:tasks.rados.rados.0.smithi084.stdout:1600: expect (ObjNum 482 snap 151 seq_num 482) 2024-09-18T14:06:47.552 INFO:tasks.rados.rados.0.smithi084.stdout:1601: write oid 26 current snap is 172 2024-09-18T14:06:47.552 INFO:tasks.rados.rados.0.smithi084.stdout:1601: seq_num 566 ranges {712762=401056,1537867=645855,2860355=652407} 2024-09-18T14:06:47.557 INFO:tasks.rados.rados.0.smithi084.stdout:1601: writing smithi08434663-26 from 712762 to 1113818 tid 1 2024-09-18T14:06:47.564 INFO:tasks.rados.rados.0.smithi084.stdout:1601: writing smithi08434663-26 from 1537867 to 2183722 tid 2 2024-09-18T14:06:47.571 INFO:tasks.rados.rados.0.smithi084.stdout:1601: writing smithi08434663-26 from 2860355 to 3512762 tid 3 2024-09-18T14:06:47.571 INFO:tasks.rados.rados.0.smithi084.stdout:1602: read oid 37 snap -1 2024-09-18T14:06:47.571 INFO:tasks.rados.rados.0.smithi084.stdout:1602: expect (ObjNum 550 snap 168 seq_num 550) 2024-09-18T14:06:47.571 INFO:tasks.rados.rados.0.smithi084.stdout:1603: write oid 3 current snap is 172 2024-09-18T14:06:47.571 INFO:tasks.rados.rados.0.smithi084.stdout:1603: seq_num 567 ranges {729569=476305,1996349=505866,3004131=525438} 2024-09-18T14:06:47.578 INFO:tasks.rados.rados.0.smithi084.stdout:1603: writing smithi08434663-3 from 729569 to 1205874 tid 1 2024-09-18T14:06:47.585 INFO:tasks.rados.rados.0.smithi084.stdout:1603: writing smithi08434663-3 from 1996349 to 2502215 tid 2 2024-09-18T14:06:47.590 INFO:tasks.rados.rados.0.smithi084.stdout:1603: writing smithi08434663-3 from 3004131 to 3529569 tid 3 2024-09-18T14:06:47.591 INFO:tasks.rados.rados.0.smithi084.stdout:1604: rollback oid 50 current snap is 172 2024-09-18T14:06:47.591 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 50 to 171 2024-09-18T14:06:47.591 INFO:tasks.rados.rados.0.smithi084.stdout:1600: expect (ObjNum 482 snap 151 seq_num 482) 2024-09-18T14:06:47.686 INFO:tasks.rados.rados.0.smithi084.stdout:1601: finishing write tid 1 to smithi08434663-26 2024-09-18T14:06:47.686 INFO:tasks.rados.rados.0.smithi084.stdout:1601: finishing write tid 2 to smithi08434663-26 2024-09-18T14:06:47.686 INFO:tasks.rados.rados.0.smithi084.stdout:1601: finishing write tid 3 to smithi08434663-26 2024-09-18T14:06:47.687 INFO:tasks.rados.rados.0.smithi084.stdout:1601: finishing write tid 4 to smithi08434663-26 2024-09-18T14:06:47.687 INFO:tasks.rados.rados.0.smithi084.stdout:1601: finishing write tid 5 to smithi08434663-26 2024-09-18T14:06:47.687 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 679 (ObjNum 566 snap 172 seq_num 566) dirty exists 2024-09-18T14:06:47.687 INFO:tasks.rados.rados.0.smithi084.stdout:1601: left oid 26 (ObjNum 566 snap 172 seq_num 566) 2024-09-18T14:06:47.687 INFO:tasks.rados.rados.0.smithi084.stdout:1602: expect (ObjNum 550 snap 168 seq_num 550) 2024-09-18T14:06:47.857 INFO:tasks.rados.rados.0.smithi084.stdout:1603: finishing write tid 1 to smithi08434663-3 2024-09-18T14:06:47.857 INFO:tasks.rados.rados.0.smithi084.stdout:1603: finishing write tid 2 to smithi08434663-3 2024-09-18T14:06:47.857 INFO:tasks.rados.rados.0.smithi084.stdout:1604: finishing rollback tid 0 to smithi08434663-50 2024-09-18T14:06:47.858 INFO:tasks.rados.rados.0.smithi084.stdout:1603: finishing write tid 3 to smithi08434663-3 2024-09-18T14:06:47.858 INFO:tasks.rados.rados.0.smithi084.stdout:1604: finishing rollback tid 1 to smithi08434663-50 2024-09-18T14:06:47.858 INFO:tasks.rados.rados.0.smithi084.stdout:1604: finishing rollback tid 2 to smithi08434663-50 2024-09-18T14:06:47.858 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 876 (ObjNum 386 snap 119 seq_num 386) dirty exists 2024-09-18T14:06:47.858 INFO:tasks.rados.rados.0.smithi084.stdout:1603: finishing write tid 4 to smithi08434663-3 2024-09-18T14:06:47.858 INFO:tasks.rados.rados.0.smithi084.stdout:1600: done (4 left) 2024-09-18T14:06:47.858 INFO:tasks.rados.rados.0.smithi084.stdout:1601: done (3 left) 2024-09-18T14:06:47.858 INFO:tasks.rados.rados.0.smithi084.stdout:1602: done (2 left) 2024-09-18T14:06:47.858 INFO:tasks.rados.rados.0.smithi084.stdout:1604: done (1 left) 2024-09-18T14:06:47.858 INFO:tasks.rados.rados.0.smithi084.stdout:1605: read oid 42 snap 166 2024-09-18T14:06:47.858 INFO:tasks.rados.rados.0.smithi084.stdout:1605: expect (ObjNum 423 snap 133 seq_num 423) 2024-09-18T14:06:47.858 INFO:tasks.rados.rados.0.smithi084.stdout:1606: rollback oid 38 current snap is 172 2024-09-18T14:06:47.859 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 38 to 168 2024-09-18T14:06:47.859 INFO:tasks.rados.rados.0.smithi084.stdout:1603: finishing write tid 5 to smithi08434663-3 2024-09-18T14:06:47.859 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 715 (ObjNum 567 snap 172 seq_num 567) dirty exists 2024-09-18T14:06:47.859 INFO:tasks.rados.rados.0.smithi084.stdout:1603: left oid 3 (ObjNum 567 snap 172 seq_num 567) 2024-09-18T14:06:47.859 INFO:tasks.rados.rados.0.smithi084.stdout:1603: done (2 left) 2024-09-18T14:06:47.859 INFO:tasks.rados.rados.0.smithi084.stdout:1607: snap_remove snap 138 2024-09-18T14:06:48.246 INFO:tasks.rados.rados.0.smithi084.stdout:1607: done (2 left) 2024-09-18T14:06:48.246 INFO:tasks.rados.rados.0.smithi084.stdout:1608: rollback oid 13 current snap is 172 2024-09-18T14:06:48.246 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 13 to 170 2024-09-18T14:06:48.246 INFO:tasks.rados.rados.0.smithi084.stdout:1606: finishing rollback tid 0 to smithi08434663-38 2024-09-18T14:06:48.246 INFO:tasks.rados.rados.0.smithi084.stdout:1606: finishing rollback tid 1 to smithi08434663-38 2024-09-18T14:06:48.246 INFO:tasks.rados.rados.0.smithi084.stdout:1606: finishing rollback tid 2 to smithi08434663-38 2024-09-18T14:06:48.246 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 416 (ObjNum 532 snap 165 seq_num 532) dirty exists 2024-09-18T14:06:48.246 INFO:tasks.rados.rados.0.smithi084.stdout:1605: expect (ObjNum 423 snap 133 seq_num 423) 2024-09-18T14:06:48.443 INFO:tasks.rados.rados.0.smithi084.stdout:1608: finishing rollback tid 1 to smithi08434663-13 2024-09-18T14:06:48.443 INFO:tasks.rados.rados.0.smithi084.stdout:1608: finishing rollback tid 2 to smithi08434663-13 2024-09-18T14:06:48.443 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 597 (ObjNum 534 snap 165 seq_num 534) dirty exists 2024-09-18T14:06:48.443 INFO:tasks.rados.rados.0.smithi084.stdout:1605: done (2 left) 2024-09-18T14:06:48.443 INFO:tasks.rados.rados.0.smithi084.stdout:1606: done (1 left) 2024-09-18T14:06:48.443 INFO:tasks.rados.rados.0.smithi084.stdout:1608: done (0 left) 2024-09-18T14:06:48.443 INFO:tasks.rados.rados.0.smithi084.stdout:1609: read oid 36 snap 166 2024-09-18T14:06:48.443 INFO:tasks.rados.rados.0.smithi084.stdout:1609: expect (ObjNum 373 snap 115 seq_num 373) 2024-09-18T14:06:48.443 INFO:tasks.rados.rados.0.smithi084.stdout:1610: rollback oid 40 current snap is 172 2024-09-18T14:06:48.443 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 40 to 155 2024-09-18T14:06:48.443 INFO:tasks.rados.rados.0.smithi084.stdout:1611: read oid 37 snap -1 2024-09-18T14:06:48.443 INFO:tasks.rados.rados.0.smithi084.stdout:1611: expect (ObjNum 550 snap 168 seq_num 550) 2024-09-18T14:06:48.444 INFO:tasks.rados.rados.0.smithi084.stdout:1612: snap_create 2024-09-18T14:06:48.445 INFO:tasks.rados.rados.0.smithi084.stdout:1610: finishing rollback tid 1 to smithi08434663-40 2024-09-18T14:06:48.445 INFO:tasks.rados.rados.0.smithi084.stdout:1610: finishing rollback tid 2 to smithi08434663-40 2024-09-18T14:06:48.446 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 603 (ObjNum 354 snap 109 seq_num 354) dirty exists 2024-09-18T14:06:48.453 INFO:tasks.rados.rados.0.smithi084.stdout:1609: expect (ObjNum 373 snap 115 seq_num 373) 2024-09-18T14:06:48.578 INFO:tasks.rados.rados.0.smithi084.stdout:1611: expect (ObjNum 550 snap 168 seq_num 550) 2024-09-18T14:06:49.252 INFO:tasks.rados.rados.0.smithi084.stdout:1609: done (3 left) 2024-09-18T14:06:49.252 INFO:tasks.rados.rados.0.smithi084.stdout:1610: done (2 left) 2024-09-18T14:06:49.252 INFO:tasks.rados.rados.0.smithi084.stdout:1611: done (1 left) 2024-09-18T14:06:49.252 INFO:tasks.rados.rados.0.smithi084.stdout:1612: done (0 left) 2024-09-18T14:06:49.252 INFO:tasks.rados.rados.0.smithi084.stdout:1613: read oid 35 snap 171 2024-09-18T14:06:49.252 INFO:tasks.rados.rados.0.smithi084.stdout:1613: expect deleted 2024-09-18T14:06:49.252 INFO:tasks.rados.rados.0.smithi084.stdout:1614: copy_from oid 34 from oid 15 current snap is 173 2024-09-18T14:06:49.252 INFO:tasks.rados.rados.0.smithi084.stdout:1615: copy_from oid 48 from oid 32 current snap is 173 2024-09-18T14:06:49.252 INFO:tasks.rados.rados.0.smithi084.stdout:1616: rollback oid 38 current snap is 173 2024-09-18T14:06:49.253 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 38 to 170 2024-09-18T14:06:49.253 INFO:tasks.rados.rados.0.smithi084.stdout:1617: read oid 13 snap 170 2024-09-18T14:06:49.253 INFO:tasks.rados.rados.0.smithi084.stdout:1617: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:06:49.253 INFO:tasks.rados.rados.0.smithi084.stdout:1618: copy_from oid 24 from oid 30 current snap is 173 2024-09-18T14:06:49.253 INFO:tasks.rados.rados.0.smithi084.stdout:1619: read oid 1 snap -1 2024-09-18T14:06:49.253 INFO:tasks.rados.rados.0.smithi084.stdout:1619: expect deleted 2024-09-18T14:06:49.253 INFO:tasks.rados.rados.0.smithi084.stdout:1620: snap_remove snap 172 2024-09-18T14:06:50.257 INFO:tasks.rados.rados.0.smithi084.stdout:1620: done (7 left) 2024-09-18T14:06:50.257 INFO:tasks.rados.rados.0.smithi084.stdout:1621: snap_remove snap 168 2024-09-18T14:06:51.263 INFO:tasks.rados.rados.0.smithi084.stdout:1621: done (7 left) 2024-09-18T14:06:51.263 INFO:tasks.rados.rados.0.smithi084.stdout:1622: snap_create 2024-09-18T14:06:51.263 INFO:tasks.rados.rados.0.smithi084.stdout:1615: finishing copy_from to smithi08434663-48 2024-09-18T14:06:51.263 INFO:tasks.rados.rados.0.smithi084.stdout:1615: got expected ENOENT (src dne) 2024-09-18T14:06:51.263 INFO:tasks.rados.rados.0.smithi084.stdout:1615: finishing copy_from racing read to smithi08434663-48 2024-09-18T14:06:51.263 INFO:tasks.rados.rados.0.smithi084.stdout:1614: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:06:51.263 INFO:tasks.rados.rados.0.smithi084.stdout:1614: finishing copy_from to smithi08434663-34 2024-09-18T14:06:51.263 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 572 (ObjNum 527 snap 162 seq_num 527) dirty exists 2024-09-18T14:06:51.264 INFO:tasks.rados.rados.0.smithi084.stdout:1617: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:06:51.431 INFO:tasks.rados.rados.0.smithi084.stdout:1616: finishing rollback tid 0 to smithi08434663-38 2024-09-18T14:06:51.431 INFO:tasks.rados.rados.0.smithi084.stdout:1616: finishing rollback tid 1 to smithi08434663-38 2024-09-18T14:06:51.431 INFO:tasks.rados.rados.0.smithi084.stdout:1616: finishing rollback tid 2 to smithi08434663-38 2024-09-18T14:06:51.431 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 419 (ObjNum 532 snap 165 seq_num 532) dirty exists 2024-09-18T14:06:51.431 INFO:tasks.rados.rados.0.smithi084.stdout:1618: finishing copy_from racing read to smithi08434663-24 2024-09-18T14:06:51.431 INFO:tasks.rados.rados.0.smithi084.stdout:1618: finishing copy_from to smithi08434663-24 2024-09-18T14:06:51.432 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 556 (ObjNum 561 snap 171 seq_num 561) dirty exists 2024-09-18T14:06:52.218 INFO:tasks.rados.rados.0.smithi084.stdout:1613: done (7 left) 2024-09-18T14:06:52.218 INFO:tasks.rados.rados.0.smithi084.stdout:1614: done (6 left) 2024-09-18T14:06:52.218 INFO:tasks.rados.rados.0.smithi084.stdout:1615: done (5 left) 2024-09-18T14:06:52.218 INFO:tasks.rados.rados.0.smithi084.stdout:1616: done (4 left) 2024-09-18T14:06:52.219 INFO:tasks.rados.rados.0.smithi084.stdout:1617: done (3 left) 2024-09-18T14:06:52.219 INFO:tasks.rados.rados.0.smithi084.stdout:1618: done (2 left) 2024-09-18T14:06:52.219 INFO:tasks.rados.rados.0.smithi084.stdout:1619: done (1 left) 2024-09-18T14:06:52.219 INFO:tasks.rados.rados.0.smithi084.stdout:1622: done (0 left) 2024-09-18T14:06:52.219 INFO:tasks.rados.rados.0.smithi084.stdout:1623: write (excl) oid 9 current snap is 174 2024-09-18T14:06:52.219 INFO:tasks.rados.rados.0.smithi084.stdout:1623: seq_num 568 ranges {746376=551554,2054831=649524,3315201=231175} 2024-09-18T14:06:52.225 INFO:tasks.rados.rados.0.smithi084.stdout:1623: writing smithi08434663-9 from 746376 to 1297930 tid 1 2024-09-18T14:06:52.233 INFO:tasks.rados.rados.0.smithi084.stdout:1623: writing smithi08434663-9 from 2054831 to 2704355 tid 2 2024-09-18T14:06:52.238 INFO:tasks.rados.rados.0.smithi084.stdout:1623: writing smithi08434663-9 from 3315201 to 3546376 tid 3 2024-09-18T14:06:52.238 INFO:tasks.rados.rados.0.smithi084.stdout:1624: read oid 5 snap -1 2024-09-18T14:06:52.238 INFO:tasks.rados.rados.0.smithi084.stdout:1624: expect (ObjNum 494 snap 152 seq_num 494) 2024-09-18T14:06:52.238 INFO:tasks.rados.rados.0.smithi084.stdout:1625: delete oid 26 current snap is 174 2024-09-18T14:06:52.238 INFO:tasks.rados.rados.0.smithi084.stdout:1623: finishing write tid 1 to smithi08434663-9 2024-09-18T14:06:52.242 INFO:tasks.rados.rados.0.smithi084.stdout:1623: finishing write tid 2 to smithi08434663-9 2024-09-18T14:06:52.242 INFO:tasks.rados.rados.0.smithi084.stdout:1624: expect (ObjNum 494 snap 152 seq_num 494) 2024-09-18T14:06:52.320 INFO:tasks.rados.rados.0.smithi084.stdout:1623: finishing write tid 3 to smithi08434663-9 2024-09-18T14:06:52.321 INFO:tasks.rados.rados.0.smithi084.stdout:1623: finishing write tid 4 to smithi08434663-9 2024-09-18T14:06:52.321 INFO:tasks.rados.rados.0.smithi084.stdout:1623: finishing write tid 5 to smithi08434663-9 2024-09-18T14:06:52.321 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 650 (ObjNum 568 snap 174 seq_num 568) dirty exists 2024-09-18T14:06:52.321 INFO:tasks.rados.rados.0.smithi084.stdout:1623: left oid 9 (ObjNum 568 snap 174 seq_num 568) 2024-09-18T14:06:52.321 INFO:tasks.rados.rados.0.smithi084.stdout:1623: done (2 left) 2024-09-18T14:06:52.321 INFO:tasks.rados.rados.0.smithi084.stdout:1624: done (1 left) 2024-09-18T14:06:52.321 INFO:tasks.rados.rados.0.smithi084.stdout:1625: done (0 left) 2024-09-18T14:06:52.321 INFO:tasks.rados.rados.0.smithi084.stdout:1626: read oid 50 snap -1 2024-09-18T14:06:52.321 INFO:tasks.rados.rados.0.smithi084.stdout:1626: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:06:52.321 INFO:tasks.rados.rados.0.smithi084.stdout:1627: write (excl) oid 22 current snap is 174 2024-09-18T14:06:52.321 INFO:tasks.rados.rados.0.smithi084.stdout:1627: seq_num 570 ranges {779990=702052,2055442=653193,3253694=326296} 2024-09-18T14:06:52.328 INFO:tasks.rados.rados.0.smithi084.stdout:1627: writing smithi08434663-22 from 779990 to 1482042 tid 1 2024-09-18T14:06:52.336 INFO:tasks.rados.rados.0.smithi084.stdout:1627: writing smithi08434663-22 from 2055442 to 2708635 tid 2 2024-09-18T14:06:52.341 INFO:tasks.rados.rados.0.smithi084.stdout:1627: writing smithi08434663-22 from 3253694 to 3579990 tid 3 2024-09-18T14:06:52.341 INFO:tasks.rados.rados.0.smithi084.stdout:1628: read oid 9 snap -1 2024-09-18T14:06:52.341 INFO:tasks.rados.rados.0.smithi084.stdout:1628: expect (ObjNum 568 snap 174 seq_num 568) 2024-09-18T14:06:52.341 INFO:tasks.rados.rados.0.smithi084.stdout:1629: delete oid 15 current snap is 174 2024-09-18T14:06:52.341 INFO:tasks.rados.rados.0.smithi084.stdout:1626: expect (ObjNum 386 snap 119 seq_num 386) 2024-09-18T14:06:52.454 INFO:tasks.rados.rados.0.smithi084.stdout:1627: finishing write tid 1 to smithi08434663-22 2024-09-18T14:06:52.454 INFO:tasks.rados.rados.0.smithi084.stdout:1627: finishing write tid 2 to smithi08434663-22 2024-09-18T14:06:52.454 INFO:tasks.rados.rados.0.smithi084.stdout:1626: done (3 left) 2024-09-18T14:06:52.455 INFO:tasks.rados.rados.0.smithi084.stdout:1629: done (2 left) 2024-09-18T14:06:52.455 INFO:tasks.rados.rados.0.smithi084.stdout:1630: snap_create 2024-09-18T14:06:52.455 INFO:tasks.rados.rados.0.smithi084.stdout:1627: finishing write tid 3 to smithi08434663-22 2024-09-18T14:06:52.455 INFO:tasks.rados.rados.0.smithi084.stdout:1627: finishing write tid 4 to smithi08434663-22 2024-09-18T14:06:52.455 INFO:tasks.rados.rados.0.smithi084.stdout:1627: finishing write tid 5 to smithi08434663-22 2024-09-18T14:06:52.455 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 719 (ObjNum 570 snap 174 seq_num 570) dirty exists 2024-09-18T14:06:52.455 INFO:tasks.rados.rados.0.smithi084.stdout:1627: left oid 22 (ObjNum 570 snap 174 seq_num 570) 2024-09-18T14:06:52.455 INFO:tasks.rados.rados.0.smithi084.stdout:1628: expect (ObjNum 568 snap 174 seq_num 568) 2024-09-18T14:06:53.277 INFO:tasks.rados.rados.0.smithi084.stdout:1627: done (2 left) 2024-09-18T14:06:53.277 INFO:tasks.rados.rados.0.smithi084.stdout:1628: done (1 left) 2024-09-18T14:06:53.277 INFO:tasks.rados.rados.0.smithi084.stdout:1630: done (0 left) 2024-09-18T14:06:53.278 INFO:tasks.rados.rados.0.smithi084.stdout:1631: read oid 12 snap 173 2024-09-18T14:06:53.278 INFO:tasks.rados.rados.0.smithi084.stdout:1631: expect deleted 2024-09-18T14:06:53.278 INFO:tasks.rados.rados.0.smithi084.stdout:1632: write oid 50 current snap is 175 2024-09-18T14:06:53.278 INFO:tasks.rados.rados.0.smithi084.stdout:1632: seq_num 572 ranges {413604=568903,1488759=656862,2624893=591709} 2024-09-18T14:06:53.282 INFO:tasks.rados.rados.0.smithi084.stdout:1632: writing smithi08434663-50 from 413604 to 982507 tid 1 2024-09-18T14:06:53.288 INFO:tasks.rados.rados.0.smithi084.stdout:1632: writing smithi08434663-50 from 1488759 to 2145621 tid 2 2024-09-18T14:06:53.294 INFO:tasks.rados.rados.0.smithi084.stdout:1632: writing smithi08434663-50 from 2624893 to 3216602 tid 3 2024-09-18T14:06:53.295 INFO:tasks.rados.rados.0.smithi084.stdout:1633: delete oid 5 current snap is 175 2024-09-18T14:06:53.295 INFO:tasks.rados.rados.0.smithi084.stdout:1632: finishing write tid 1 to smithi08434663-50 2024-09-18T14:06:53.295 INFO:tasks.rados.rados.0.smithi084.stdout:1632: finishing write tid 2 to smithi08434663-50 2024-09-18T14:06:53.297 INFO:tasks.rados.rados.0.smithi084.stdout:1631: done (2 left) 2024-09-18T14:06:53.298 INFO:tasks.rados.rados.0.smithi084.stdout:1633: done (1 left) 2024-09-18T14:06:53.298 INFO:tasks.rados.rados.0.smithi084.stdout:1634: write (excl) oid 21 current snap is 175 2024-09-18T14:06:53.298 INFO:tasks.rados.rados.0.smithi084.stdout:1634: seq_num 574 ranges {447218=719401,1889370=660531,2963386=614253} 2024-09-18T14:06:53.304 INFO:tasks.rados.rados.0.smithi084.stdout:1634: writing smithi08434663-21 from 447218 to 1166619 tid 1 2024-09-18T14:06:53.311 INFO:tasks.rados.rados.0.smithi084.stdout:1634: writing smithi08434663-21 from 1889370 to 2549901 tid 2 2024-09-18T14:06:53.317 INFO:tasks.rados.rados.0.smithi084.stdout:1634: writing smithi08434663-21 from 2963386 to 3577639 tid 3 2024-09-18T14:06:53.317 INFO:tasks.rados.rados.0.smithi084.stdout:1635: snap_remove snap 170 2024-09-18T14:06:53.317 INFO:tasks.rados.rados.0.smithi084.stdout:1632: finishing write tid 3 to smithi08434663-50 2024-09-18T14:06:53.317 INFO:tasks.rados.rados.0.smithi084.stdout:1632: finishing write tid 4 to smithi08434663-50 2024-09-18T14:06:53.318 INFO:tasks.rados.rados.0.smithi084.stdout:1632: finishing write tid 5 to smithi08434663-50 2024-09-18T14:06:53.318 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 897 (ObjNum 572 snap 175 seq_num 572) dirty exists 2024-09-18T14:06:53.318 INFO:tasks.rados.rados.0.smithi084.stdout:1632: left oid 50 (ObjNum 572 snap 175 seq_num 572) 2024-09-18T14:06:53.318 INFO:tasks.rados.rados.0.smithi084.stdout:1634: finishing write tid 1 to smithi08434663-21 2024-09-18T14:06:54.285 INFO:tasks.rados.rados.0.smithi084.stdout:1632: done (2 left) 2024-09-18T14:06:54.285 INFO:tasks.rados.rados.0.smithi084.stdout:1635: done (1 left) 2024-09-18T14:06:54.285 INFO:tasks.rados.rados.0.smithi084.stdout:1636: read oid 46 snap -1 2024-09-18T14:06:54.286 INFO:tasks.rados.rados.0.smithi084.stdout:1636: expect (ObjNum 545 snap 167 seq_num 545) 2024-09-18T14:06:54.286 INFO:tasks.rados.rados.0.smithi084.stdout:1637: write (excl) oid 35 current snap is 175 2024-09-18T14:06:54.286 INFO:tasks.rados.rados.0.smithi084.stdout:1637: seq_num 575 ranges {464025=794650,1947852=404189,2990809=425525} 2024-09-18T14:06:54.292 INFO:tasks.rados.rados.0.smithi084.stdout:1637: writing smithi08434663-35 from 464025 to 1258675 tid 1 2024-09-18T14:06:54.299 INFO:tasks.rados.rados.0.smithi084.stdout:1637: writing smithi08434663-35 from 1947852 to 2352041 tid 2 2024-09-18T14:06:54.305 INFO:tasks.rados.rados.0.smithi084.stdout:1637: writing smithi08434663-35 from 2990809 to 3416334 tid 3 2024-09-18T14:06:54.305 INFO:tasks.rados.rados.0.smithi084.stdout:1638: delete oid 16 current snap is 175 2024-09-18T14:06:54.306 INFO:tasks.rados.rados.0.smithi084.stdout:1634: finishing write tid 2 to smithi08434663-21 2024-09-18T14:06:54.306 INFO:tasks.rados.rados.0.smithi084.stdout:1634: finishing write tid 3 to smithi08434663-21 2024-09-18T14:06:54.306 INFO:tasks.rados.rados.0.smithi084.stdout:1634: finishing write tid 4 to smithi08434663-21 2024-09-18T14:06:54.306 INFO:tasks.rados.rados.0.smithi084.stdout:1634: finishing write tid 5 to smithi08434663-21 2024-09-18T14:06:54.306 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 561 (ObjNum 574 snap 175 seq_num 574) dirty exists 2024-09-18T14:06:54.306 INFO:tasks.rados.rados.0.smithi084.stdout:1634: left oid 21 (ObjNum 574 snap 175 seq_num 574) 2024-09-18T14:06:54.306 INFO:tasks.rados.rados.0.smithi084.stdout:1636: expect (ObjNum 545 snap 167 seq_num 545) 2024-09-18T14:06:54.508 INFO:tasks.rados.rados.0.smithi084.stdout:1637: finishing write tid 1 to smithi08434663-35 2024-09-18T14:06:54.508 INFO:tasks.rados.rados.0.smithi084.stdout:1637: finishing write tid 2 to smithi08434663-35 2024-09-18T14:06:54.508 INFO:tasks.rados.rados.0.smithi084.stdout:1637: finishing write tid 3 to smithi08434663-35 2024-09-18T14:06:54.508 INFO:tasks.rados.rados.0.smithi084.stdout:1637: finishing write tid 4 to smithi08434663-35 2024-09-18T14:06:54.508 INFO:tasks.rados.rados.0.smithi084.stdout:1637: finishing write tid 5 to smithi08434663-35 2024-09-18T14:06:54.508 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 662 (ObjNum 575 snap 175 seq_num 575) dirty exists 2024-09-18T14:06:54.508 INFO:tasks.rados.rados.0.smithi084.stdout:1637: left oid 35 (ObjNum 575 snap 175 seq_num 575) 2024-09-18T14:06:54.509 INFO:tasks.rados.rados.0.smithi084.stdout:1634: done (3 left) 2024-09-18T14:06:54.509 INFO:tasks.rados.rados.0.smithi084.stdout:1636: done (2 left) 2024-09-18T14:06:54.509 INFO:tasks.rados.rados.0.smithi084.stdout:1637: done (1 left) 2024-09-18T14:06:54.509 INFO:tasks.rados.rados.0.smithi084.stdout:1638: done (0 left) 2024-09-18T14:06:54.509 INFO:tasks.rados.rados.0.smithi084.stdout:1639: delete oid 39 current snap is 175 2024-09-18T14:06:54.510 INFO:tasks.rados.rados.0.smithi084.stdout:1639: done (0 left) 2024-09-18T14:06:54.510 INFO:tasks.rados.rados.0.smithi084.stdout:1640: delete oid 21 current snap is 175 2024-09-18T14:06:54.513 INFO:tasks.rados.rados.0.smithi084.stdout:1640: done (0 left) 2024-09-18T14:06:54.513 INFO:tasks.rados.rados.0.smithi084.stdout:1641: copy_from oid 27 from oid 31 current snap is 175 2024-09-18T14:06:54.513 INFO:tasks.rados.rados.0.smithi084.stdout:1642: read oid 26 snap -1 2024-09-18T14:06:54.513 INFO:tasks.rados.rados.0.smithi084.stdout:1642: expect deleted 2024-09-18T14:06:54.513 INFO:tasks.rados.rados.0.smithi084.stdout:1643: write (excl) oid 9 current snap is 175 2024-09-18T14:06:54.513 INFO:tasks.rados.rados.0.smithi084.stdout:1643: seq_num 579 ranges {531253=411999,1381780=411527,2300501=586966,3376974=354279} 2024-09-18T14:06:54.517 INFO:tasks.rados.rados.0.smithi084.stdout:1643: writing smithi08434663-9 from 531253 to 943252 tid 1 2024-09-18T14:06:54.522 INFO:tasks.rados.rados.0.smithi084.stdout:1643: writing smithi08434663-9 from 1381780 to 1793307 tid 2 2024-09-18T14:06:54.528 INFO:tasks.rados.rados.0.smithi084.stdout:1643: writing smithi08434663-9 from 2300501 to 2887467 tid 3 2024-09-18T14:06:54.533 INFO:tasks.rados.rados.0.smithi084.stdout:1643: writing smithi08434663-9 from 3376974 to 3731253 tid 4 2024-09-18T14:06:54.533 INFO:tasks.rados.rados.0.smithi084.stdout:1644: read oid 44 snap -1 2024-09-18T14:06:54.534 INFO:tasks.rados.rados.0.smithi084.stdout:1644: expect (ObjNum 470 snap 147 seq_num 470) 2024-09-18T14:06:54.534 INFO:tasks.rados.rados.0.smithi084.stdout:1645: copy_from oid 24 from oid 37 current snap is 175 2024-09-18T14:06:54.534 INFO:tasks.rados.rados.0.smithi084.stdout:1641: finishing copy_from to smithi08434663-27 2024-09-18T14:06:54.534 INFO:tasks.rados.rados.0.smithi084.stdout:1641: got expected ENOENT (src dne) 2024-09-18T14:06:54.534 INFO:tasks.rados.rados.0.smithi084.stdout:1641: finishing copy_from racing read to smithi08434663-27 2024-09-18T14:06:54.534 INFO:tasks.rados.rados.0.smithi084.stdout:1643: finishing write tid 1 to smithi08434663-9 2024-09-18T14:06:54.534 INFO:tasks.rados.rados.0.smithi084.stdout:1643: finishing write tid 2 to smithi08434663-9 2024-09-18T14:06:54.534 INFO:tasks.rados.rados.0.smithi084.stdout:1643: finishing write tid 3 to smithi08434663-9 2024-09-18T14:06:54.534 INFO:tasks.rados.rados.0.smithi084.stdout:1641: done (4 left) 2024-09-18T14:06:54.534 INFO:tasks.rados.rados.0.smithi084.stdout:1642: done (3 left) 2024-09-18T14:06:54.534 INFO:tasks.rados.rados.0.smithi084.stdout:1646: write oid 4 current snap is 175 2024-09-18T14:06:54.534 INFO:tasks.rados.rados.0.smithi084.stdout:1646: seq_num 580 ranges {548060=487248,1440262=671538,2844277=798238} 2024-09-18T14:06:54.539 INFO:tasks.rados.rados.0.smithi084.stdout:1646: writing smithi08434663-4 from 548060 to 1035308 tid 1 2024-09-18T14:06:54.545 INFO:tasks.rados.rados.0.smithi084.stdout:1646: writing smithi08434663-4 from 1440262 to 2111800 tid 2 2024-09-18T14:06:54.553 INFO:tasks.rados.rados.0.smithi084.stdout:1646: writing smithi08434663-4 from 2844277 to 3642515 tid 3 2024-09-18T14:06:54.553 INFO:tasks.rados.rados.0.smithi084.stdout:1647: copy_from oid 20 from oid 13 current snap is 175 2024-09-18T14:06:54.553 INFO:tasks.rados.rados.0.smithi084.stdout:1643: finishing write tid 4 to smithi08434663-9 2024-09-18T14:06:54.553 INFO:tasks.rados.rados.0.smithi084.stdout:1644: expect (ObjNum 470 snap 147 seq_num 470) 2024-09-18T14:06:54.677 INFO:tasks.rados.rados.0.smithi084.stdout:1643: finishing write tid 5 to smithi08434663-9 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:1643: finishing write tid 6 to smithi08434663-9 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 670 (ObjNum 579 snap 175 seq_num 579) dirty exists 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:1643: left oid 9 (ObjNum 579 snap 175 seq_num 579) 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:1645: finishing copy_from racing read to smithi08434663-24 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:1645: finishing copy_from to smithi08434663-24 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 459 (ObjNum 550 snap 168 seq_num 550) dirty exists 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:1646: finishing write tid 1 to smithi08434663-4 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:1646: finishing write tid 2 to smithi08434663-4 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:1646: finishing write tid 3 to smithi08434663-4 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:1646: finishing write tid 4 to smithi08434663-4 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:1643: done (4 left) 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:1644: done (3 left) 2024-09-18T14:06:54.678 INFO:tasks.rados.rados.0.smithi084.stdout:1645: done (2 left) 2024-09-18T14:06:54.679 INFO:tasks.rados.rados.0.smithi084.stdout:1648: delete oid 27 current snap is 175 2024-09-18T14:06:54.679 INFO:tasks.rados.rados.0.smithi084.stdout:1646: finishing write tid 5 to smithi08434663-4 2024-09-18T14:06:54.679 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 739 (ObjNum 580 snap 175 seq_num 580) dirty exists 2024-09-18T14:06:54.679 INFO:tasks.rados.rados.0.smithi084.stdout:1646: left oid 4 (ObjNum 580 snap 175 seq_num 580) 2024-09-18T14:06:54.679 INFO:tasks.rados.rados.0.smithi084.stdout:1647: finishing copy_from racing read to smithi08434663-20 2024-09-18T14:06:54.679 INFO:tasks.rados.rados.0.smithi084.stdout:1647: finishing copy_from to smithi08434663-20 2024-09-18T14:06:54.679 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 597 (ObjNum 534 snap 165 seq_num 534) dirty exists 2024-09-18T14:06:54.680 INFO:tasks.rados.rados.0.smithi084.stdout:1646: done (2 left) 2024-09-18T14:06:54.680 INFO:tasks.rados.rados.0.smithi084.stdout:1647: done (1 left) 2024-09-18T14:06:54.680 INFO:tasks.rados.rados.0.smithi084.stdout:1648: done (0 left) 2024-09-18T14:06:54.680 INFO:tasks.rados.rados.0.smithi084.stdout:1649: delete oid 27 current snap is 175 2024-09-18T14:06:54.681 INFO:tasks.rados.rados.0.smithi084.stdout:1649: done (0 left) 2024-09-18T14:06:54.681 INFO:tasks.rados.rados.0.smithi084.stdout:1650: write oid 38 current snap is 175 2024-09-18T14:06:54.682 INFO:tasks.rados.rados.0.smithi084.stdout:1650: seq_num 583 ranges {598481=712995,1899355=418865,3093840=704641} 2024-09-18T14:06:54.689 INFO:tasks.rados.rados.0.smithi084.stdout:1650: writing smithi08434663-38 from 598481 to 1311476 tid 1 2024-09-18T14:06:54.694 INFO:tasks.rados.rados.0.smithi084.stdout:1650: writing smithi08434663-38 from 1899355 to 2318220 tid 2 2024-09-18T14:06:54.702 INFO:tasks.rados.rados.0.smithi084.stdout:1650: writing smithi08434663-38 from 3093840 to 3798481 tid 3 2024-09-18T14:06:54.702 INFO:tasks.rados.rados.0.smithi084.stdout:1651: delete oid 5 current snap is 175 2024-09-18T14:06:54.702 INFO:tasks.rados.rados.0.smithi084.stdout:1650: finishing write tid 1 to smithi08434663-38 2024-09-18T14:06:54.702 INFO:tasks.rados.rados.0.smithi084.stdout:1650: finishing write tid 2 to smithi08434663-38 2024-09-18T14:06:54.703 INFO:tasks.rados.rados.0.smithi084.stdout:1651: done (1 left) 2024-09-18T14:06:54.703 INFO:tasks.rados.rados.0.smithi084.stdout:1652: read oid 1 snap -1 2024-09-18T14:06:54.703 INFO:tasks.rados.rados.0.smithi084.stdout:1652: expect deleted 2024-09-18T14:06:54.703 INFO:tasks.rados.rados.0.smithi084.stdout:1653: snap_remove snap 155 2024-09-18T14:06:55.298 INFO:tasks.rados.rados.0.smithi084.stdout:1653: done (2 left) 2024-09-18T14:06:55.298 INFO:tasks.rados.rados.0.smithi084.stdout:1654: write oid 33 current snap is 175 2024-09-18T14:06:55.298 INFO:tasks.rados.rados.0.smithi084.stdout:1654: seq_num 585 ranges {632095=463493,1616319=422534,2748686=487304} 2024-09-18T14:06:55.304 INFO:tasks.rados.rados.0.smithi084.stdout:1654: writing smithi08434663-33 from 632095 to 1095588 tid 1 2024-09-18T14:06:55.309 INFO:tasks.rados.rados.0.smithi084.stdout:1654: writing smithi08434663-33 from 1616319 to 2038853 tid 2 2024-09-18T14:06:55.317 INFO:tasks.rados.rados.0.smithi084.stdout:1654: writing smithi08434663-33 from 2748686 to 3235990 tid 3 2024-09-18T14:06:55.317 INFO:tasks.rados.rados.0.smithi084.stdout:1655: copy_from oid 23 from oid 28 current snap is 175 2024-09-18T14:06:55.317 INFO:tasks.rados.rados.0.smithi084.stdout:1650: finishing write tid 3 to smithi08434663-38 2024-09-18T14:06:55.317 INFO:tasks.rados.rados.0.smithi084.stdout:1650: finishing write tid 4 to smithi08434663-38 2024-09-18T14:06:55.317 INFO:tasks.rados.rados.0.smithi084.stdout:1650: finishing write tid 5 to smithi08434663-38 2024-09-18T14:06:55.317 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 423 (ObjNum 583 snap 175 seq_num 583) dirty exists 2024-09-18T14:06:55.317 INFO:tasks.rados.rados.0.smithi084.stdout:1650: left oid 38 (ObjNum 583 snap 175 seq_num 583) 2024-09-18T14:06:55.317 INFO:tasks.rados.rados.0.smithi084.stdout:1650: done (3 left) 2024-09-18T14:06:55.317 INFO:tasks.rados.rados.0.smithi084.stdout:1652: done (2 left) 2024-09-18T14:06:55.317 INFO:tasks.rados.rados.0.smithi084.stdout:1656: delete oid 45 current snap is 175 2024-09-18T14:06:55.320 INFO:tasks.rados.rados.0.smithi084.stdout:1654: finishing write tid 1 to smithi08434663-33 2024-09-18T14:06:55.320 INFO:tasks.rados.rados.0.smithi084.stdout:1656: done (2 left) 2024-09-18T14:06:55.320 INFO:tasks.rados.rados.0.smithi084.stdout:1657: snap_create 2024-09-18T14:06:55.324 INFO:tasks.rados.rados.0.smithi084.stdout:1654: finishing write tid 2 to smithi08434663-33 2024-09-18T14:06:55.324 INFO:tasks.rados.rados.0.smithi084.stdout:1654: finishing write tid 3 to smithi08434663-33 2024-09-18T14:06:55.324 INFO:tasks.rados.rados.0.smithi084.stdout:1654: finishing write tid 4 to smithi08434663-33 2024-09-18T14:06:55.324 INFO:tasks.rados.rados.0.smithi084.stdout:1654: finishing write tid 5 to smithi08434663-33 2024-09-18T14:06:55.324 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 641 (ObjNum 585 snap 175 seq_num 585) dirty exists 2024-09-18T14:06:55.324 INFO:tasks.rados.rados.0.smithi084.stdout:1654: left oid 33 (ObjNum 585 snap 175 seq_num 585) 2024-09-18T14:06:55.331 INFO:tasks.rados.rados.0.smithi084.stdout:1655: finishing copy_from racing read to smithi08434663-23 2024-09-18T14:06:55.332 INFO:tasks.rados.rados.0.smithi084.stdout:1655: finishing copy_from to smithi08434663-23 2024-09-18T14:06:55.332 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 406 (ObjNum 265 snap 67 seq_num 265) dirty exists 2024-09-18T14:06:56.299 INFO:tasks.rados.rados.0.smithi084.stdout:1654: done (2 left) 2024-09-18T14:06:56.299 INFO:tasks.rados.rados.0.smithi084.stdout:1655: done (1 left) 2024-09-18T14:06:56.299 INFO:tasks.rados.rados.0.smithi084.stdout:1657: done (0 left) 2024-09-18T14:06:56.299 INFO:tasks.rados.rados.0.smithi084.stdout:1658: read oid 45 snap -1 2024-09-18T14:06:56.299 INFO:tasks.rados.rados.0.smithi084.stdout:1658: expect deleted 2024-09-18T14:06:56.299 INFO:tasks.rados.rados.0.smithi084.stdout:1659: snap_create 2024-09-18T14:06:57.225 INFO:tasks.rados.rados.0.smithi084.stdout:1658: done (1 left) 2024-09-18T14:06:57.225 INFO:tasks.rados.rados.0.smithi084.stdout:1659: done (0 left) 2024-09-18T14:06:57.225 INFO:tasks.rados.rados.0.smithi084.stdout:1660: snap_remove snap 171 2024-09-18T14:06:58.232 INFO:tasks.rados.rados.0.smithi084.stdout:1660: done (0 left) 2024-09-18T14:06:58.233 INFO:tasks.rados.rados.0.smithi084.stdout:1661: rollback oid 9 current snap is 177 2024-09-18T14:06:58.233 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 9 to 161 2024-09-18T14:06:58.233 INFO:tasks.rados.rados.0.smithi084.stdout:1662: write (excl) oid 47 current snap is 177 2024-09-18T14:06:58.233 INFO:tasks.rados.rados.0.smithi084.stdout:1662: seq_num 587 ranges {665709=730344,2133283=426203,3203532=509848} 2024-09-18T14:06:58.241 INFO:tasks.rados.rados.0.smithi084.stdout:1662: writing smithi08434663-47 from 665709 to 1396053 tid 1 2024-09-18T14:06:58.247 INFO:tasks.rados.rados.0.smithi084.stdout:1662: writing smithi08434663-47 from 2133283 to 2559486 tid 2 2024-09-18T14:06:58.254 INFO:tasks.rados.rados.0.smithi084.stdout:1662: writing smithi08434663-47 from 3203532 to 3713380 tid 3 2024-09-18T14:06:58.254 INFO:tasks.rados.rados.0.smithi084.stdout:1663: delete oid 15 current snap is 177 2024-09-18T14:06:58.254 INFO:tasks.rados.rados.0.smithi084.stdout:1661: finishing rollback tid 0 to smithi08434663-9 2024-09-18T14:06:58.254 INFO:tasks.rados.rados.0.smithi084.stdout:1661: finishing rollback tid 1 to smithi08434663-9 2024-09-18T14:06:58.254 INFO:tasks.rados.rados.0.smithi084.stdout:1661: finishing rollback tid 2 to smithi08434663-9 2024-09-18T14:06:58.254 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 686 (ObjNum 429 snap 134 seq_num 429) dirty exists 2024-09-18T14:06:58.254 INFO:tasks.rados.rados.0.smithi084.stdout:1662: finishing write tid 1 to smithi08434663-47 2024-09-18T14:06:58.255 INFO:tasks.rados.rados.0.smithi084.stdout:1662: finishing write tid 2 to smithi08434663-47 2024-09-18T14:06:58.255 INFO:tasks.rados.rados.0.smithi084.stdout:1661: done (2 left) 2024-09-18T14:06:58.255 INFO:tasks.rados.rados.0.smithi084.stdout:1663: done (1 left) 2024-09-18T14:06:58.255 INFO:tasks.rados.rados.0.smithi084.stdout:1664: write oid 42 current snap is 177 2024-09-18T14:06:58.256 INFO:tasks.rados.rados.0.smithi084.stdout:1664: seq_num 589 ranges {699323=480842,1850247=429872,2858378=532392} 2024-09-18T14:06:58.262 INFO:tasks.rados.rados.0.smithi084.stdout:1664: writing smithi08434663-42 from 699323 to 1180165 tid 1 2024-09-18T14:06:58.268 INFO:tasks.rados.rados.0.smithi084.stdout:1664: writing smithi08434663-42 from 1850247 to 2280119 tid 2 2024-09-18T14:06:58.274 INFO:tasks.rados.rados.0.smithi084.stdout:1664: writing smithi08434663-42 from 2858378 to 3390770 tid 3 2024-09-18T14:06:58.274 INFO:tasks.rados.rados.0.smithi084.stdout:1665: read oid 21 snap -1 2024-09-18T14:06:58.275 INFO:tasks.rados.rados.0.smithi084.stdout:1665: expect deleted 2024-09-18T14:06:58.275 INFO:tasks.rados.rados.0.smithi084.stdout:1666: snap_remove snap 174 2024-09-18T14:06:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1666: done (3 left) 2024-09-18T14:06:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1667: delete oid 3 current snap is 177 2024-09-18T14:06:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1662: finishing write tid 3 to smithi08434663-47 2024-09-18T14:06:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1662: finishing write tid 4 to smithi08434663-47 2024-09-18T14:06:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1662: finishing write tid 5 to smithi08434663-47 2024-09-18T14:06:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 578 (ObjNum 587 snap 177 seq_num 587) dirty exists 2024-09-18T14:06:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1662: left oid 47 (ObjNum 587 snap 177 seq_num 587) 2024-09-18T14:06:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1664: finishing write tid 1 to smithi08434663-42 2024-09-18T14:06:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1664: finishing write tid 2 to smithi08434663-42 2024-09-18T14:06:59.234 INFO:tasks.rados.rados.0.smithi084.stdout:1664: finishing write tid 3 to smithi08434663-42 2024-09-18T14:06:59.235 INFO:tasks.rados.rados.0.smithi084.stdout:1664: finishing write tid 4 to smithi08434663-42 2024-09-18T14:06:59.235 INFO:tasks.rados.rados.0.smithi084.stdout:1664: finishing write tid 5 to smithi08434663-42 2024-09-18T14:06:59.235 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 748 (ObjNum 589 snap 177 seq_num 589) dirty exists 2024-09-18T14:06:59.235 INFO:tasks.rados.rados.0.smithi084.stdout:1664: left oid 42 (ObjNum 589 snap 177 seq_num 589) 2024-09-18T14:06:59.241 INFO:tasks.rados.rados.0.smithi084.stdout:1662: done (3 left) 2024-09-18T14:06:59.241 INFO:tasks.rados.rados.0.smithi084.stdout:1664: done (2 left) 2024-09-18T14:06:59.241 INFO:tasks.rados.rados.0.smithi084.stdout:1665: done (1 left) 2024-09-18T14:06:59.241 INFO:tasks.rados.rados.0.smithi084.stdout:1667: done (0 left) 2024-09-18T14:06:59.241 INFO:tasks.rados.rados.0.smithi084.stdout:1668: snap_create 2024-09-18T14:07:00.250 INFO:tasks.rados.rados.0.smithi084.stdout:1668: done (0 left) 2024-09-18T14:07:00.250 INFO:tasks.rados.rados.0.smithi084.stdout:1669: rollback oid 20 current snap is 178 2024-09-18T14:07:00.250 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 20 to 176 2024-09-18T14:07:00.250 INFO:tasks.rados.rados.0.smithi084.stdout:1670: rollback oid 31 current snap is 178 2024-09-18T14:07:00.250 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 31 to 166 2024-09-18T14:07:00.250 INFO:tasks.rados.rados.0.smithi084.stdout:1671: read oid 49 snap 162 2024-09-18T14:07:00.250 INFO:tasks.rados.rados.0.smithi084.stdout:1671: expect deleted 2024-09-18T14:07:00.250 INFO:tasks.rados.rados.0.smithi084.stdout:1672: write (excl) oid 40 current snap is 178 2024-09-18T14:07:00.250 INFO:tasks.rados.rados.0.smithi084.stdout:1672: seq_num 591 ranges {732937=631340,1850858=433541,2796871=671289,3888195=44742} 2024-09-18T14:07:00.257 INFO:tasks.rados.rados.0.smithi084.stdout:1672: writing smithi08434663-40 from 732937 to 1364277 tid 1 2024-09-18T14:07:00.262 INFO:tasks.rados.rados.0.smithi084.stdout:1672: writing smithi08434663-40 from 1850858 to 2284399 tid 2 2024-09-18T14:07:00.269 INFO:tasks.rados.rados.0.smithi084.stdout:1672: writing smithi08434663-40 from 2796871 to 3468160 tid 3 2024-09-18T14:07:00.272 INFO:tasks.rados.rados.0.smithi084.stdout:1672: writing smithi08434663-40 from 3888195 to 3932937 tid 4 2024-09-18T14:07:00.272 INFO:tasks.rados.rados.0.smithi084.stdout:1673: delete oid 43 current snap is 178 2024-09-18T14:07:00.272 INFO:tasks.rados.rados.0.smithi084.stdout:1670: finishing rollback tid 1 to smithi08434663-31 2024-09-18T14:07:00.272 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 378 (ObjNum 129 snap 0 seq_num 2495502640) dirty dne 2024-09-18T14:07:00.272 INFO:tasks.rados.rados.0.smithi084.stdout:1669: finishing rollback tid 0 to smithi08434663-20 2024-09-18T14:07:00.272 INFO:tasks.rados.rados.0.smithi084.stdout:1669: finishing rollback tid 1 to smithi08434663-20 2024-09-18T14:07:00.272 INFO:tasks.rados.rados.0.smithi084.stdout:1669: finishing rollback tid 2 to smithi08434663-20 2024-09-18T14:07:00.272 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 623 (ObjNum 534 snap 165 seq_num 534) dirty exists 2024-09-18T14:07:00.272 INFO:tasks.rados.rados.0.smithi084.stdout:1672: finishing write tid 1 to smithi08434663-40 2024-09-18T14:07:00.273 INFO:tasks.rados.rados.0.smithi084.stdout:1672: finishing write tid 2 to smithi08434663-40 2024-09-18T14:07:00.275 INFO:tasks.rados.rados.0.smithi084.stdout:1672: finishing write tid 3 to smithi08434663-40 2024-09-18T14:07:00.275 INFO:tasks.rados.rados.0.smithi084.stdout:1672: finishing write tid 4 to smithi08434663-40 2024-09-18T14:07:00.275 INFO:tasks.rados.rados.0.smithi084.stdout:1672: finishing write tid 5 to smithi08434663-40 2024-09-18T14:07:00.275 INFO:tasks.rados.rados.0.smithi084.stdout:1669: done (4 left) 2024-09-18T14:07:00.275 INFO:tasks.rados.rados.0.smithi084.stdout:1670: done (3 left) 2024-09-18T14:07:00.275 INFO:tasks.rados.rados.0.smithi084.stdout:1671: done (2 left) 2024-09-18T14:07:00.275 INFO:tasks.rados.rados.0.smithi084.stdout:1673: done (1 left) 2024-09-18T14:07:00.275 INFO:tasks.rados.rados.0.smithi084.stdout:1674: snap_remove snap 166 2024-09-18T14:07:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:1674: done (1 left) 2024-09-18T14:07:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:1675: rollback oid 4 current snap is 178 2024-09-18T14:07:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 4 to 161 2024-09-18T14:07:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:1672: finishing write tid 6 to smithi08434663-40 2024-09-18T14:07:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 623 (ObjNum 591 snap 178 seq_num 591) dirty exists 2024-09-18T14:07:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:1672: left oid 40 (ObjNum 591 snap 178 seq_num 591) 2024-09-18T14:07:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:1672: done (1 left) 2024-09-18T14:07:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:1676: rollback oid 46 current snap is 178 2024-09-18T14:07:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 46 to 175 2024-09-18T14:07:01.248 INFO:tasks.rados.rados.0.smithi084.stdout:1677: snap_create 2024-09-18T14:07:01.262 INFO:tasks.rados.rados.0.smithi084.stdout:1675: finishing rollback tid 0 to smithi08434663-4 2024-09-18T14:07:01.262 INFO:tasks.rados.rados.0.smithi084.stdout:1675: finishing rollback tid 1 to smithi08434663-4 2024-09-18T14:07:01.266 INFO:tasks.rados.rados.0.smithi084.stdout:1676: finishing rollback tid 0 to smithi08434663-46 2024-09-18T14:07:01.266 INFO:tasks.rados.rados.0.smithi084.stdout:1676: finishing rollback tid 1 to smithi08434663-46 2024-09-18T14:07:01.266 INFO:tasks.rados.rados.0.smithi084.stdout:1676: finishing rollback tid 2 to smithi08434663-46 2024-09-18T14:07:01.266 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 701 (ObjNum 545 snap 167 seq_num 545) dirty exists 2024-09-18T14:07:01.268 INFO:tasks.rados.rados.0.smithi084.stdout:1675: finishing rollback tid 2 to smithi08434663-4 2024-09-18T14:07:01.268 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 762 (ObjNum 518 snap 159 seq_num 518) dirty exists 2024-09-18T14:07:02.221 INFO:tasks.rados.rados.0.smithi084.stdout:1675: done (2 left) 2024-09-18T14:07:02.222 INFO:tasks.rados.rados.0.smithi084.stdout:1676: done (1 left) 2024-09-18T14:07:02.222 INFO:tasks.rados.rados.0.smithi084.stdout:1677: done (0 left) 2024-09-18T14:07:02.222 INFO:tasks.rados.rados.0.smithi084.stdout:1678: write oid 45 current snap is 179 2024-09-18T14:07:02.222 INFO:tasks.rados.rados.0.smithi084.stdout:1678: seq_num 593 ranges {766551=498191,1684175=720857,2851717=693833} 2024-09-18T14:07:02.228 INFO:tasks.rados.rados.0.smithi084.stdout:1678: writing smithi08434663-45 from 766551 to 1264742 tid 1 2024-09-18T14:07:02.234 INFO:tasks.rados.rados.0.smithi084.stdout:1678: writing smithi08434663-45 from 1684175 to 2405032 tid 2 2024-09-18T14:07:02.240 INFO:tasks.rados.rados.0.smithi084.stdout:1678: writing smithi08434663-45 from 2851717 to 3545550 tid 3 2024-09-18T14:07:02.241 INFO:tasks.rados.rados.0.smithi084.stdout:1679: snap_remove snap 162 2024-09-18T14:07:03.224 INFO:tasks.rados.rados.0.smithi084.stdout:1679: done (1 left) 2024-09-18T14:07:03.225 INFO:tasks.rados.rados.0.smithi084.stdout:1680: write (excl) oid 27 current snap is 179 2024-09-18T14:07:03.225 INFO:tasks.rados.rados.0.smithi084.stdout:1678: finishing write tid 1 to smithi08434663-45 2024-09-18T14:07:03.225 INFO:tasks.rados.rados.0.smithi084.stdout:1678: finishing write tid 2 to smithi08434663-45 2024-09-18T14:07:03.225 INFO:tasks.rados.rados.0.smithi084.stdout:1678: finishing write tid 3 to smithi08434663-45 2024-09-18T14:07:03.225 INFO:tasks.rados.rados.0.smithi084.stdout:1680: seq_num 594 ranges {783358=573440,2026304=580868,3279140=621458} 2024-09-18T14:07:03.232 INFO:tasks.rados.rados.0.smithi084.stdout:1680: writing smithi08434663-27 from 783358 to 1356798 tid 1 2024-09-18T14:07:03.239 INFO:tasks.rados.rados.0.smithi084.stdout:1680: writing smithi08434663-27 from 2026304 to 2607172 tid 2 2024-09-18T14:07:03.247 INFO:tasks.rados.rados.0.smithi084.stdout:1680: writing smithi08434663-27 from 3279140 to 3900598 tid 3 2024-09-18T14:07:03.247 INFO:tasks.rados.rados.0.smithi084.stdout:1681: read oid 32 snap -1 2024-09-18T14:07:03.247 INFO:tasks.rados.rados.0.smithi084.stdout:1681: expect deleted 2024-09-18T14:07:03.247 INFO:tasks.rados.rados.0.smithi084.stdout:1678: finishing write tid 4 to smithi08434663-45 2024-09-18T14:07:03.248 INFO:tasks.rados.rados.0.smithi084.stdout:1678: finishing write tid 5 to smithi08434663-45 2024-09-18T14:07:03.248 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 770 (ObjNum 593 snap 179 seq_num 593) dirty exists 2024-09-18T14:07:03.248 INFO:tasks.rados.rados.0.smithi084.stdout:1678: left oid 45 (ObjNum 593 snap 179 seq_num 593) 2024-09-18T14:07:03.248 INFO:tasks.rados.rados.0.smithi084.stdout:1680: finishing write tid 1 to smithi08434663-27 2024-09-18T14:07:03.248 INFO:tasks.rados.rados.0.smithi084.stdout:1680: finishing write tid 2 to smithi08434663-27 2024-09-18T14:07:03.248 INFO:tasks.rados.rados.0.smithi084.stdout:1682: read oid 43 snap -1 2024-09-18T14:07:03.248 INFO:tasks.rados.rados.0.smithi084.stdout:1682: expect deleted 2024-09-18T14:07:03.248 INFO:tasks.rados.rados.0.smithi084.stdout:1678: done (3 left) 2024-09-18T14:07:03.248 INFO:tasks.rados.rados.0.smithi084.stdout:1683: snap_remove snap 177 2024-09-18T14:07:04.226 INFO:tasks.rados.rados.0.smithi084.stdout:1683: done (3 left) 2024-09-18T14:07:04.226 INFO:tasks.rados.rados.0.smithi084.stdout:1684: write oid 28 current snap is 179 2024-09-18T14:07:04.226 INFO:tasks.rados.rados.0.smithi084.stdout:1684: seq_num 595 ranges {400165=648689,1684786=315379} 2024-09-18T14:07:04.231 INFO:tasks.rados.rados.0.smithi084.stdout:1684: writing smithi08434663-28 from 400165 to 1048854 tid 1 2024-09-18T14:07:04.237 INFO:tasks.rados.rados.0.smithi084.stdout:1684: writing smithi08434663-28 from 1684786 to 2000165 tid 2 2024-09-18T14:07:04.237 INFO:tasks.rados.rados.0.smithi084.stdout:1685: copy_from oid 15 from oid 44 current snap is 179 2024-09-18T14:07:04.237 INFO:tasks.rados.rados.0.smithi084.stdout:1680: finishing write tid 3 to smithi08434663-27 2024-09-18T14:07:04.237 INFO:tasks.rados.rados.0.smithi084.stdout:1680: finishing write tid 4 to smithi08434663-27 2024-09-18T14:07:04.237 INFO:tasks.rados.rados.0.smithi084.stdout:1680: finishing write tid 5 to smithi08434663-27 2024-09-18T14:07:04.238 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 722 (ObjNum 594 snap 179 seq_num 594) dirty exists 2024-09-18T14:07:04.238 INFO:tasks.rados.rados.0.smithi084.stdout:1680: left oid 27 (ObjNum 594 snap 179 seq_num 594) 2024-09-18T14:07:04.238 INFO:tasks.rados.rados.0.smithi084.stdout:1680: done (4 left) 2024-09-18T14:07:04.238 INFO:tasks.rados.rados.0.smithi084.stdout:1681: done (3 left) 2024-09-18T14:07:04.238 INFO:tasks.rados.rados.0.smithi084.stdout:1682: done (2 left) 2024-09-18T14:07:04.238 INFO:tasks.rados.rados.0.smithi084.stdout:1686: snap_remove snap 176 2024-09-18T14:07:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:1686: done (2 left) 2024-09-18T14:07:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:1687: write (excl) oid 35 current snap is 179 2024-09-18T14:07:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:1684: finishing write tid 1 to smithi08434663-28 2024-09-18T14:07:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:1684: finishing write tid 2 to smithi08434663-28 2024-09-18T14:07:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:1684: finishing write tid 3 to smithi08434663-28 2024-09-18T14:07:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:1684: finishing write tid 4 to smithi08434663-28 2024-09-18T14:07:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 937 (ObjNum 595 snap 179 seq_num 595) dirty exists 2024-09-18T14:07:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:1684: left oid 28 (ObjNum 595 snap 179 seq_num 595) 2024-09-18T14:07:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:1685: finishing copy_from racing read to smithi08434663-15 2024-09-18T14:07:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:1685: finishing copy_from to smithi08434663-15 2024-09-18T14:07:05.233 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 228 (ObjNum 470 snap 147 seq_num 470) dirty exists 2024-09-18T14:07:05.233 INFO:tasks.rados.rados.0.smithi084.stdout:1687: seq_num 596 ranges {416972=723938,1743268=273704} 2024-09-18T14:07:05.238 INFO:tasks.rados.rados.0.smithi084.stdout:1687: writing smithi08434663-35 from 416972 to 1140910 tid 1 2024-09-18T14:07:05.243 INFO:tasks.rados.rados.0.smithi084.stdout:1687: writing smithi08434663-35 from 1743268 to 2016972 tid 2 2024-09-18T14:07:05.244 INFO:tasks.rados.rados.0.smithi084.stdout:1684: done (2 left) 2024-09-18T14:07:05.244 INFO:tasks.rados.rados.0.smithi084.stdout:1685: done (1 left) 2024-09-18T14:07:05.244 INFO:tasks.rados.rados.0.smithi084.stdout:1688: snap_remove snap 169 2024-09-18T14:07:06.240 INFO:tasks.rados.rados.0.smithi084.stdout:1688: done (1 left) 2024-09-18T14:07:06.240 INFO:tasks.rados.rados.0.smithi084.stdout:1689: delete oid 19 current snap is 179 2024-09-18T14:07:06.240 INFO:tasks.rados.rados.0.smithi084.stdout:1687: finishing write tid 1 to smithi08434663-35 2024-09-18T14:07:06.240 INFO:tasks.rados.rados.0.smithi084.stdout:1687: finishing write tid 2 to smithi08434663-35 2024-09-18T14:07:06.241 INFO:tasks.rados.rados.0.smithi084.stdout:1687: finishing write tid 3 to smithi08434663-35 2024-09-18T14:07:06.241 INFO:tasks.rados.rados.0.smithi084.stdout:1687: finishing write tid 4 to smithi08434663-35 2024-09-18T14:07:06.241 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 728 (ObjNum 596 snap 179 seq_num 596) dirty exists 2024-09-18T14:07:06.241 INFO:tasks.rados.rados.0.smithi084.stdout:1687: left oid 35 (ObjNum 596 snap 179 seq_num 596) 2024-09-18T14:07:06.245 INFO:tasks.rados.rados.0.smithi084.stdout:1687: done (1 left) 2024-09-18T14:07:06.245 INFO:tasks.rados.rados.0.smithi084.stdout:1689: done (0 left) 2024-09-18T14:07:06.245 INFO:tasks.rados.rados.0.smithi084.stdout:1690: delete oid 19 current snap is 179 2024-09-18T14:07:06.254 INFO:tasks.rados.rados.0.smithi084.stdout:1690: done (0 left) 2024-09-18T14:07:06.254 INFO:tasks.rados.rados.0.smithi084.stdout:1691: rollback oid 22 current snap is 179 2024-09-18T14:07:06.254 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 22 to 173 2024-09-18T14:07:06.254 INFO:tasks.rados.rados.0.smithi084.stdout:1692: snap_remove snap 161 2024-09-18T14:07:07.224 INFO:tasks.rados.rados.0.smithi084.stdout:1691: finishing rollback tid 0 to smithi08434663-22 2024-09-18T14:07:07.224 INFO:tasks.rados.rados.0.smithi084.stdout:1691: finishing rollback tid 1 to smithi08434663-22 2024-09-18T14:07:07.224 INFO:tasks.rados.rados.0.smithi084.stdout:1691: finishing rollback tid 2 to smithi08434663-22 2024-09-18T14:07:07.224 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 723 (ObjNum 533 snap 165 seq_num 533) dirty exists 2024-09-18T14:07:07.224 INFO:tasks.rados.rados.0.smithi084.stdout:1691: done (1 left) 2024-09-18T14:07:07.224 INFO:tasks.rados.rados.0.smithi084.stdout:1692: done (0 left) 2024-09-18T14:07:07.224 INFO:tasks.rados.rados.0.smithi084.stdout:1693: write (excl) oid 22 current snap is 179 2024-09-18T14:07:07.224 INFO:tasks.rados.rados.0.smithi084.stdout:1693: seq_num 599 ranges {467393=549685,1802361=265032} 2024-09-18T14:07:07.229 INFO:tasks.rados.rados.0.smithi084.stdout:1693: writing smithi08434663-22 from 467393 to 1017078 tid 1 2024-09-18T14:07:07.236 INFO:tasks.rados.rados.0.smithi084.stdout:1693: writing smithi08434663-22 from 1802361 to 2067393 tid 2 2024-09-18T14:07:07.236 INFO:tasks.rados.rados.0.smithi084.stdout:1694: copy_from oid 4 from oid 5 current snap is 179 2024-09-18T14:07:07.237 INFO:tasks.rados.rados.0.smithi084.stdout:1695: read oid 36 snap -1 2024-09-18T14:07:07.238 INFO:tasks.rados.rados.0.smithi084.stdout:1695: expect (ObjNum 373 snap 115 seq_num 373) 2024-09-18T14:07:07.238 INFO:tasks.rados.rados.0.smithi084.stdout:1696: rollback oid 40 current snap is 179 2024-09-18T14:07:07.238 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 40 to 178 2024-09-18T14:07:07.238 INFO:tasks.rados.rados.0.smithi084.stdout:1697: copy_from oid 32 from oid 14 current snap is 179 2024-09-18T14:07:07.238 INFO:tasks.rados.rados.0.smithi084.stdout:1698: write oid 7 current snap is 179 2024-09-18T14:07:07.238 INFO:tasks.rados.rados.0.smithi084.stdout:1698: seq_num 600 ranges {484200=624934,1860843=223357} 2024-09-18T14:07:07.245 INFO:tasks.rados.rados.0.smithi084.stdout:1698: writing smithi08434663-7 from 484200 to 1109134 tid 1 2024-09-18T14:07:07.250 INFO:tasks.rados.rados.0.smithi084.stdout:1698: writing smithi08434663-7 from 1860843 to 2084200 tid 2 2024-09-18T14:07:07.251 INFO:tasks.rados.rados.0.smithi084.stdout:1694: finishing copy_from to smithi08434663-4 2024-09-18T14:07:07.251 INFO:tasks.rados.rados.0.smithi084.stdout:1694: got expected ENOENT (src dne) 2024-09-18T14:07:07.251 INFO:tasks.rados.rados.0.smithi084.stdout:1699: snap_remove snap 173 2024-09-18T14:07:08.258 INFO:tasks.rados.rados.0.smithi084.stdout:1699: done (6 left) 2024-09-18T14:07:08.258 INFO:tasks.rados.rados.0.smithi084.stdout:1700: copy_from oid 26 from oid 17 current snap is 179 2024-09-18T14:07:08.258 INFO:tasks.rados.rados.0.smithi084.stdout:1694: finishing copy_from racing read to smithi08434663-4 2024-09-18T14:07:08.259 INFO:tasks.rados.rados.0.smithi084.stdout:1696: finishing rollback tid 0 to smithi08434663-40 2024-09-18T14:07:08.259 INFO:tasks.rados.rados.0.smithi084.stdout:1693: finishing write tid 1 to smithi08434663-22 2024-09-18T14:07:08.259 INFO:tasks.rados.rados.0.smithi084.stdout:1697: finishing copy_from racing read to smithi08434663-32 2024-09-18T14:07:08.259 INFO:tasks.rados.rados.0.smithi084.stdout:1693: finishing write tid 2 to smithi08434663-22 2024-09-18T14:07:08.259 INFO:tasks.rados.rados.0.smithi084.stdout:1693: finishing write tid 3 to smithi08434663-22 2024-09-18T14:07:08.259 INFO:tasks.rados.rados.0.smithi084.stdout:1693: finishing write tid 4 to smithi08434663-22 2024-09-18T14:07:08.259 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 726 (ObjNum 599 snap 179 seq_num 599) dirty exists 2024-09-18T14:07:08.259 INFO:tasks.rados.rados.0.smithi084.stdout:1693: left oid 22 (ObjNum 599 snap 179 seq_num 599) 2024-09-18T14:07:08.259 INFO:tasks.rados.rados.0.smithi084.stdout:1695: expect (ObjNum 373 snap 115 seq_num 373) 2024-09-18T14:07:08.387 INFO:tasks.rados.rados.0.smithi084.stdout:1696: finishing rollback tid 1 to smithi08434663-40 2024-09-18T14:07:08.388 INFO:tasks.rados.rados.0.smithi084.stdout:1696: finishing rollback tid 2 to smithi08434663-40 2024-09-18T14:07:08.388 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 647 (ObjNum 591 snap 178 seq_num 591) dirty exists 2024-09-18T14:07:08.388 INFO:tasks.rados.rados.0.smithi084.stdout:1698: finishing write tid 1 to smithi08434663-7 2024-09-18T14:07:08.388 INFO:tasks.rados.rados.0.smithi084.stdout:1698: finishing write tid 2 to smithi08434663-7 2024-09-18T14:07:08.388 INFO:tasks.rados.rados.0.smithi084.stdout:1697: finishing copy_from to smithi08434663-32 2024-09-18T14:07:08.388 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 769 (ObjNum 513 snap 158 seq_num 513) dirty exists 2024-09-18T14:07:08.388 INFO:tasks.rados.rados.0.smithi084.stdout:1698: finishing write tid 3 to smithi08434663-7 2024-09-18T14:07:08.388 INFO:tasks.rados.rados.0.smithi084.stdout:1693: done (6 left) 2024-09-18T14:07:08.388 INFO:tasks.rados.rados.0.smithi084.stdout:1694: done (5 left) 2024-09-18T14:07:08.388 INFO:tasks.rados.rados.0.smithi084.stdout:1695: done (4 left) 2024-09-18T14:07:08.388 INFO:tasks.rados.rados.0.smithi084.stdout:1696: done (3 left) 2024-09-18T14:07:08.389 INFO:tasks.rados.rados.0.smithi084.stdout:1697: done (2 left) 2024-09-18T14:07:08.389 INFO:tasks.rados.rados.0.smithi084.stdout:1701: delete oid 31 current snap is 179 2024-09-18T14:07:08.389 INFO:tasks.rados.rados.0.smithi084.stdout:1698: finishing write tid 4 to smithi08434663-7 2024-09-18T14:07:08.389 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 593 (ObjNum 600 snap 179 seq_num 600) dirty exists 2024-09-18T14:07:08.389 INFO:tasks.rados.rados.0.smithi084.stdout:1698: left oid 7 (ObjNum 600 snap 179 seq_num 600) 2024-09-18T14:07:08.389 INFO:tasks.rados.rados.0.smithi084.stdout:1700: finishing copy_from to smithi08434663-26 2024-09-18T14:07:08.389 INFO:tasks.rados.rados.0.smithi084.stdout:1700: got expected ENOENT (src dne) 2024-09-18T14:07:08.389 INFO:tasks.rados.rados.0.smithi084.stdout:1700: finishing copy_from racing read to smithi08434663-26 2024-09-18T14:07:08.389 INFO:tasks.rados.rados.0.smithi084.stdout:1698: done (2 left) 2024-09-18T14:07:08.389 INFO:tasks.rados.rados.0.smithi084.stdout:1700: done (1 left) 2024-09-18T14:07:08.390 INFO:tasks.rados.rados.0.smithi084.stdout:1701: done (0 left) 2024-09-18T14:07:08.390 INFO:tasks.rados.rados.0.smithi084.stdout:1702: snap_create 2024-09-18T14:07:09.261 INFO:tasks.rados.rados.0.smithi084.stdout:1702: done (0 left) 2024-09-18T14:07:09.261 INFO:tasks.rados.rados.0.smithi084.stdout:1703: rollback oid 44 current snap is 180 2024-09-18T14:07:09.261 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 44 to 179 2024-09-18T14:07:09.261 INFO:tasks.rados.rados.0.smithi084.stdout:1704: write (excl) oid 11 current snap is 180 2024-09-18T14:07:09.261 INFO:tasks.rados.rados.0.smithi084.stdout:1704: seq_num 602 ranges {517814=491785,1577807=540007} 2024-09-18T14:07:09.266 INFO:tasks.rados.rados.0.smithi084.stdout:1704: writing smithi08434663-11 from 517814 to 1009599 tid 1 2024-09-18T14:07:09.273 INFO:tasks.rados.rados.0.smithi084.stdout:1704: writing smithi08434663-11 from 1577807 to 2117814 tid 2 2024-09-18T14:07:09.273 INFO:tasks.rados.rados.0.smithi084.stdout:1705: snap_create 2024-09-18T14:07:09.273 INFO:tasks.rados.rados.0.smithi084.stdout:1703: finishing rollback tid 0 to smithi08434663-44 2024-09-18T14:07:09.273 INFO:tasks.rados.rados.0.smithi084.stdout:1703: finishing rollback tid 1 to smithi08434663-44 2024-09-18T14:07:09.273 INFO:tasks.rados.rados.0.smithi084.stdout:1703: finishing rollback tid 2 to smithi08434663-44 2024-09-18T14:07:09.274 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 273 (ObjNum 470 snap 147 seq_num 470) dirty exists 2024-09-18T14:07:09.276 INFO:tasks.rados.rados.0.smithi084.stdout:1704: finishing write tid 1 to smithi08434663-11 2024-09-18T14:07:09.277 INFO:tasks.rados.rados.0.smithi084.stdout:1704: finishing write tid 2 to smithi08434663-11 2024-09-18T14:07:09.277 INFO:tasks.rados.rados.0.smithi084.stdout:1704: finishing write tid 3 to smithi08434663-11 2024-09-18T14:07:09.277 INFO:tasks.rados.rados.0.smithi084.stdout:1704: finishing write tid 4 to smithi08434663-11 2024-09-18T14:07:09.277 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 581 (ObjNum 602 snap 180 seq_num 602) dirty exists 2024-09-18T14:07:09.278 INFO:tasks.rados.rados.0.smithi084.stdout:1704: left oid 11 (ObjNum 602 snap 180 seq_num 602) 2024-09-18T14:07:10.274 INFO:tasks.rados.rados.0.smithi084.stdout:1703: done (2 left) 2024-09-18T14:07:10.275 INFO:tasks.rados.rados.0.smithi084.stdout:1704: done (1 left) 2024-09-18T14:07:10.275 INFO:tasks.rados.rados.0.smithi084.stdout:1705: done (0 left) 2024-09-18T14:07:10.275 INFO:tasks.rados.rados.0.smithi084.stdout:1706: write (excl) oid 19 current snap is 181 2024-09-18T14:07:10.275 INFO:tasks.rados.rados.0.smithi084.stdout:1706: seq_num 603 ranges {534621=567034,1636289=498332} 2024-09-18T14:07:10.280 INFO:tasks.rados.rados.0.smithi084.stdout:1706: writing smithi08434663-19 from 534621 to 1101655 tid 1 2024-09-18T14:07:10.286 INFO:tasks.rados.rados.0.smithi084.stdout:1706: writing smithi08434663-19 from 1636289 to 2134621 tid 2 2024-09-18T14:07:10.286 INFO:tasks.rados.rados.0.smithi084.stdout:1707: snap_remove snap 178 2024-09-18T14:07:11.275 INFO:tasks.rados.rados.0.smithi084.stdout:1707: done (1 left) 2024-09-18T14:07:11.276 INFO:tasks.rados.rados.0.smithi084.stdout:1708: delete oid 37 current snap is 181 2024-09-18T14:07:11.276 INFO:tasks.rados.rados.0.smithi084.stdout:1706: finishing write tid 1 to smithi08434663-19 2024-09-18T14:07:11.276 INFO:tasks.rados.rados.0.smithi084.stdout:1706: finishing write tid 2 to smithi08434663-19 2024-09-18T14:07:11.276 INFO:tasks.rados.rados.0.smithi084.stdout:1706: finishing write tid 3 to smithi08434663-19 2024-09-18T14:07:11.276 INFO:tasks.rados.rados.0.smithi084.stdout:1706: finishing write tid 4 to smithi08434663-19 2024-09-18T14:07:11.276 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 963 (ObjNum 603 snap 181 seq_num 603) dirty exists 2024-09-18T14:07:11.276 INFO:tasks.rados.rados.0.smithi084.stdout:1706: left oid 19 (ObjNum 603 snap 181 seq_num 603) 2024-09-18T14:07:11.289 INFO:tasks.rados.rados.0.smithi084.stdout:1706: done (1 left) 2024-09-18T14:07:11.289 INFO:tasks.rados.rados.0.smithi084.stdout:1708: done (0 left) 2024-09-18T14:07:11.289 INFO:tasks.rados.rados.0.smithi084.stdout:1709: read oid 39 snap -1 2024-09-18T14:07:11.289 INFO:tasks.rados.rados.0.smithi084.stdout:1709: expect deleted 2024-09-18T14:07:11.289 INFO:tasks.rados.rados.0.smithi084.stdout:1710: delete oid 42 current snap is 181 2024-09-18T14:07:11.295 INFO:tasks.rados.rados.0.smithi084.stdout:1709: done (1 left) 2024-09-18T14:07:11.295 INFO:tasks.rados.rados.0.smithi084.stdout:1710: done (0 left) 2024-09-18T14:07:11.295 INFO:tasks.rados.rados.0.smithi084.stdout:1711: write oid 40 current snap is 181 2024-09-18T14:07:11.295 INFO:tasks.rados.rados.0.smithi084.stdout:1711: seq_num 606 ranges {585042=792781,2095382=89660} 2024-09-18T14:07:11.302 INFO:tasks.rados.rados.0.smithi084.stdout:1711: writing smithi08434663-40 from 585042 to 1377823 tid 1 2024-09-18T14:07:11.307 INFO:tasks.rados.rados.0.smithi084.stdout:1711: writing smithi08434663-40 from 2095382 to 2185042 tid 2 2024-09-18T14:07:11.307 INFO:tasks.rados.rados.0.smithi084.stdout:1712: rollback oid 39 current snap is 181 2024-09-18T14:07:11.307 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 39 to 175 2024-09-18T14:07:11.307 INFO:tasks.rados.rados.0.smithi084.stdout:1713: read oid 23 snap -1 2024-09-18T14:07:11.307 INFO:tasks.rados.rados.0.smithi084.stdout:1713: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:07:11.307 INFO:tasks.rados.rados.0.smithi084.stdout:1714: read oid 4 snap -1 2024-09-18T14:07:11.308 INFO:tasks.rados.rados.0.smithi084.stdout:1714: expect (ObjNum 518 snap 159 seq_num 518) 2024-09-18T14:07:11.308 INFO:tasks.rados.rados.0.smithi084.stdout:1715: copy_from oid 45 from oid 21 current snap is 181 2024-09-18T14:07:11.308 INFO:tasks.rados.rados.0.smithi084.stdout:1716: snap_remove snap 180 2024-09-18T14:07:12.229 INFO:tasks.rados.rados.0.smithi084.stdout:1716: done (5 left) 2024-09-18T14:07:12.229 INFO:tasks.rados.rados.0.smithi084.stdout:1717: delete oid 32 current snap is 181 2024-09-18T14:07:12.229 INFO:tasks.rados.rados.0.smithi084.stdout:1712: finishing rollback tid 1 to smithi08434663-39 2024-09-18T14:07:12.230 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 0 (ObjNum 369 snap 0 seq_num 2495172224) dirty dne 2024-09-18T14:07:12.230 INFO:tasks.rados.rados.0.smithi084.stdout:1713: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:07:12.309 INFO:tasks.rados.rados.0.smithi084.stdout:1711: finishing write tid 1 to smithi08434663-40 2024-09-18T14:07:12.309 INFO:tasks.rados.rados.0.smithi084.stdout:1711: finishing write tid 2 to smithi08434663-40 2024-09-18T14:07:12.309 INFO:tasks.rados.rados.0.smithi084.stdout:1711: finishing write tid 3 to smithi08434663-40 2024-09-18T14:07:12.310 INFO:tasks.rados.rados.0.smithi084.stdout:1711: finishing write tid 4 to smithi08434663-40 2024-09-18T14:07:12.310 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 663 (ObjNum 606 snap 181 seq_num 606) dirty exists 2024-09-18T14:07:12.310 INFO:tasks.rados.rados.0.smithi084.stdout:1711: left oid 40 (ObjNum 606 snap 181 seq_num 606) 2024-09-18T14:07:12.310 INFO:tasks.rados.rados.0.smithi084.stdout:1711: done (5 left) 2024-09-18T14:07:12.310 INFO:tasks.rados.rados.0.smithi084.stdout:1712: done (4 left) 2024-09-18T14:07:12.310 INFO:tasks.rados.rados.0.smithi084.stdout:1713: done (3 left) 2024-09-18T14:07:12.310 INFO:tasks.rados.rados.0.smithi084.stdout:1717: done (2 left) 2024-09-18T14:07:12.310 INFO:tasks.rados.rados.0.smithi084.stdout:1718: rollback oid 23 current snap is 181 2024-09-18T14:07:12.310 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 23 to 175 2024-09-18T14:07:12.310 INFO:tasks.rados.rados.0.smithi084.stdout:1714: expect (ObjNum 518 snap 159 seq_num 518) 2024-09-18T14:07:12.449 INFO:tasks.rados.rados.0.smithi084.stdout:1715: finishing copy_from to smithi08434663-45 2024-09-18T14:07:12.449 INFO:tasks.rados.rados.0.smithi084.stdout:1715: got expected ENOENT (src dne) 2024-09-18T14:07:12.449 INFO:tasks.rados.rados.0.smithi084.stdout:1715: finishing copy_from racing read to smithi08434663-45 2024-09-18T14:07:12.449 INFO:tasks.rados.rados.0.smithi084.stdout:1714: done (2 left) 2024-09-18T14:07:12.450 INFO:tasks.rados.rados.0.smithi084.stdout:1715: done (1 left) 2024-09-18T14:07:12.450 INFO:tasks.rados.rados.0.smithi084.stdout:1719: copy_from oid 38 from oid 20 current snap is 181 2024-09-18T14:07:12.450 INFO:tasks.rados.rados.0.smithi084.stdout:1718: finishing rollback tid 0 to smithi08434663-23 2024-09-18T14:07:12.450 INFO:tasks.rados.rados.0.smithi084.stdout:1718: finishing rollback tid 1 to smithi08434663-23 2024-09-18T14:07:12.450 INFO:tasks.rados.rados.0.smithi084.stdout:1718: finishing rollback tid 2 to smithi08434663-23 2024-09-18T14:07:12.450 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 596 (ObjNum 265 snap 67 seq_num 265) dirty exists 2024-09-18T14:07:12.450 INFO:tasks.rados.rados.0.smithi084.stdout:1718: done (1 left) 2024-09-18T14:07:12.450 INFO:tasks.rados.rados.0.smithi084.stdout:1720: write oid 34 current snap is 181 2024-09-18T14:07:12.450 INFO:tasks.rados.rados.0.smithi084.stdout:1720: seq_num 608 ranges {618656=543279,1812346=406310} 2024-09-18T14:07:12.455 INFO:tasks.rados.rados.0.smithi084.stdout:1720: writing smithi08434663-34 from 618656 to 1161935 tid 1 2024-09-18T14:07:12.461 INFO:tasks.rados.rados.0.smithi084.stdout:1720: writing smithi08434663-34 from 1812346 to 2218656 tid 2 2024-09-18T14:07:12.461 INFO:tasks.rados.rados.0.smithi084.stdout:1721: read oid 50 snap 179 2024-09-18T14:07:12.461 INFO:tasks.rados.rados.0.smithi084.stdout:1721: expect (ObjNum 572 snap 175 seq_num 572) 2024-09-18T14:07:12.461 INFO:tasks.rados.rados.0.smithi084.stdout:1722: snap_remove snap 175 2024-09-18T14:07:13.293 INFO:tasks.rados.rados.0.smithi084.stdout:1722: done (3 left) 2024-09-18T14:07:13.294 INFO:tasks.rados.rados.0.smithi084.stdout:1723: read oid 10 snap -1 2024-09-18T14:07:13.294 INFO:tasks.rados.rados.0.smithi084.stdout:1723: expect deleted 2024-09-18T14:07:13.294 INFO:tasks.rados.rados.0.smithi084.stdout:1724: snap_create 2024-09-18T14:07:13.294 INFO:tasks.rados.rados.0.smithi084.stdout:1719: finishing copy_from racing read to smithi08434663-38 2024-09-18T14:07:13.294 INFO:tasks.rados.rados.0.smithi084.stdout:1720: finishing write tid 1 to smithi08434663-34 2024-09-18T14:07:13.294 INFO:tasks.rados.rados.0.smithi084.stdout:1719: finishing copy_from to smithi08434663-38 2024-09-18T14:07:13.294 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 623 (ObjNum 534 snap 165 seq_num 534) dirty exists 2024-09-18T14:07:13.294 INFO:tasks.rados.rados.0.smithi084.stdout:1720: finishing write tid 2 to smithi08434663-34 2024-09-18T14:07:13.294 INFO:tasks.rados.rados.0.smithi084.stdout:1720: finishing write tid 3 to smithi08434663-34 2024-09-18T14:07:13.294 INFO:tasks.rados.rados.0.smithi084.stdout:1720: finishing write tid 4 to smithi08434663-34 2024-09-18T14:07:13.294 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 814 (ObjNum 608 snap 181 seq_num 608) dirty exists 2024-09-18T14:07:13.295 INFO:tasks.rados.rados.0.smithi084.stdout:1720: left oid 34 (ObjNum 608 snap 181 seq_num 608) 2024-09-18T14:07:13.295 INFO:tasks.rados.rados.0.smithi084.stdout:1721: expect (ObjNum 572 snap 175 seq_num 572) 2024-09-18T14:07:14.297 INFO:tasks.rados.rados.0.smithi084.stdout:1719: done (4 left) 2024-09-18T14:07:14.297 INFO:tasks.rados.rados.0.smithi084.stdout:1720: done (3 left) 2024-09-18T14:07:14.297 INFO:tasks.rados.rados.0.smithi084.stdout:1721: done (2 left) 2024-09-18T14:07:14.297 INFO:tasks.rados.rados.0.smithi084.stdout:1723: done (1 left) 2024-09-18T14:07:14.297 INFO:tasks.rados.rados.0.smithi084.stdout:1724: done (0 left) 2024-09-18T14:07:14.297 INFO:tasks.rados.rados.0.smithi084.stdout:1725: write (excl) oid 4 current snap is 182 2024-09-18T14:07:14.298 INFO:tasks.rados.rados.0.smithi084.stdout:1725: seq_num 609 ranges {635463=734881,1987181=248282} 2024-09-18T14:07:14.304 INFO:tasks.rados.rados.0.smithi084.stdout:1725: writing smithi08434663-4 from 635463 to 1370344 tid 1 2024-09-18T14:07:14.309 INFO:tasks.rados.rados.0.smithi084.stdout:1725: writing smithi08434663-4 from 1987181 to 2235463 tid 2 2024-09-18T14:07:14.309 INFO:tasks.rados.rados.0.smithi084.stdout:1726: delete oid 14 current snap is 182 2024-09-18T14:07:14.309 INFO:tasks.rados.rados.0.smithi084.stdout:1725: finishing write tid 1 to smithi08434663-4 2024-09-18T14:07:14.312 INFO:tasks.rados.rados.0.smithi084.stdout:1725: finishing write tid 2 to smithi08434663-4 2024-09-18T14:07:14.312 INFO:tasks.rados.rados.0.smithi084.stdout:1725: finishing write tid 3 to smithi08434663-4 2024-09-18T14:07:14.313 INFO:tasks.rados.rados.0.smithi084.stdout:1725: finishing write tid 4 to smithi08434663-4 2024-09-18T14:07:14.313 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 824 (ObjNum 609 snap 182 seq_num 609) dirty exists 2024-09-18T14:07:14.313 INFO:tasks.rados.rados.0.smithi084.stdout:1725: left oid 4 (ObjNum 609 snap 182 seq_num 609) 2024-09-18T14:07:14.313 INFO:tasks.rados.rados.0.smithi084.stdout:1725: done (1 left) 2024-09-18T14:07:14.313 INFO:tasks.rados.rados.0.smithi084.stdout:1726: done (0 left) 2024-09-18T14:07:14.313 INFO:tasks.rados.rados.0.smithi084.stdout:1727: snap_remove snap 179 2024-09-18T14:07:15.302 INFO:tasks.rados.rados.0.smithi084.stdout:1727: done (0 left) 2024-09-18T14:07:15.302 INFO:tasks.rados.rados.0.smithi084.stdout:1728: delete oid 46 current snap is 182 2024-09-18T14:07:15.306 INFO:tasks.rados.rados.0.smithi084.stdout:1728: done (0 left) 2024-09-18T14:07:15.306 INFO:tasks.rados.rados.0.smithi084.stdout:1729: delete oid 19 current snap is 182 2024-09-18T14:07:15.310 INFO:tasks.rados.rados.0.smithi084.stdout:1729: done (0 left) 2024-09-18T14:07:15.310 INFO:tasks.rados.rados.0.smithi084.stdout:1730: snap_create 2024-09-18T14:07:16.309 INFO:tasks.rados.rados.0.smithi084.stdout:1730: done (0 left) 2024-09-18T14:07:16.310 INFO:tasks.rados.rados.0.smithi084.stdout:1731: read oid 23 snap 181 2024-09-18T14:07:16.310 INFO:tasks.rados.rados.0.smithi084.stdout:1731: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:07:16.310 INFO:tasks.rados.rados.0.smithi084.stdout:1732: delete oid 29 current snap is 183 2024-09-18T14:07:16.322 INFO:tasks.rados.rados.0.smithi084.stdout:1731: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:07:16.399 INFO:tasks.rados.rados.0.smithi084.stdout:1731: done (1 left) 2024-09-18T14:07:16.399 INFO:tasks.rados.rados.0.smithi084.stdout:1732: done (0 left) 2024-09-18T14:07:16.399 INFO:tasks.rados.rados.0.smithi084.stdout:1733: write (excl) oid 28 current snap is 183 2024-09-18T14:07:16.399 INFO:tasks.rados.rados.0.smithi084.stdout:1733: seq_num 614 ranges {719498=711126,2046885=272613} 2024-09-18T14:07:16.406 INFO:tasks.rados.rados.0.smithi084.stdout:1733: writing smithi08434663-28 from 719498 to 1430624 tid 1 2024-09-18T14:07:16.412 INFO:tasks.rados.rados.0.smithi084.stdout:1733: writing smithi08434663-28 from 2046885 to 2319498 tid 2 2024-09-18T14:07:16.412 INFO:tasks.rados.rados.0.smithi084.stdout:1734: read oid 5 snap -1 2024-09-18T14:07:16.412 INFO:tasks.rados.rados.0.smithi084.stdout:1734: expect deleted 2024-09-18T14:07:16.412 INFO:tasks.rados.rados.0.smithi084.stdout:1735: read oid 37 snap -1 2024-09-18T14:07:16.412 INFO:tasks.rados.rados.0.smithi084.stdout:1735: expect deleted 2024-09-18T14:07:16.412 INFO:tasks.rados.rados.0.smithi084.stdout:1736: read oid 8 snap -1 2024-09-18T14:07:16.413 INFO:tasks.rados.rados.0.smithi084.stdout:1736: expect (ObjNum 551 snap 168 seq_num 551) 2024-09-18T14:07:16.413 INFO:tasks.rados.rados.0.smithi084.stdout:1737: write (excl) oid 22 current snap is 183 2024-09-18T14:07:16.413 INFO:tasks.rados.rados.0.smithi084.stdout:1737: seq_num 615 ranges {736305=786375,2105367=230938} 2024-09-18T14:07:16.420 INFO:tasks.rados.rados.0.smithi084.stdout:1737: writing smithi08434663-22 from 736305 to 1522680 tid 1 2024-09-18T14:07:16.425 INFO:tasks.rados.rados.0.smithi084.stdout:1737: writing smithi08434663-22 from 2105367 to 2336305 tid 2 2024-09-18T14:07:16.425 INFO:tasks.rados.rados.0.smithi084.stdout:1738: delete oid 11 current snap is 183 2024-09-18T14:07:16.425 INFO:tasks.rados.rados.0.smithi084.stdout:1733: finishing write tid 1 to smithi08434663-28 2024-09-18T14:07:16.426 INFO:tasks.rados.rados.0.smithi084.stdout:1733: finishing write tid 2 to smithi08434663-28 2024-09-18T14:07:16.426 INFO:tasks.rados.rados.0.smithi084.stdout:1733: finishing write tid 3 to smithi08434663-28 2024-09-18T14:07:16.426 INFO:tasks.rados.rados.0.smithi084.stdout:1733: finishing write tid 4 to smithi08434663-28 2024-09-18T14:07:16.426 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 981 (ObjNum 614 snap 183 seq_num 614) dirty exists 2024-09-18T14:07:16.426 INFO:tasks.rados.rados.0.smithi084.stdout:1733: left oid 28 (ObjNum 614 snap 183 seq_num 614) 2024-09-18T14:07:16.426 INFO:tasks.rados.rados.0.smithi084.stdout:1736: expect (ObjNum 551 snap 168 seq_num 551) 2024-09-18T14:07:16.527 INFO:tasks.rados.rados.0.smithi084.stdout:1737: finishing write tid 1 to smithi08434663-22 2024-09-18T14:07:16.528 INFO:tasks.rados.rados.0.smithi084.stdout:1737: finishing write tid 2 to smithi08434663-22 2024-09-18T14:07:16.528 INFO:tasks.rados.rados.0.smithi084.stdout:1737: finishing write tid 3 to smithi08434663-22 2024-09-18T14:07:16.528 INFO:tasks.rados.rados.0.smithi084.stdout:1733: done (5 left) 2024-09-18T14:07:16.528 INFO:tasks.rados.rados.0.smithi084.stdout:1734: done (4 left) 2024-09-18T14:07:16.528 INFO:tasks.rados.rados.0.smithi084.stdout:1735: done (3 left) 2024-09-18T14:07:16.528 INFO:tasks.rados.rados.0.smithi084.stdout:1736: done (2 left) 2024-09-18T14:07:16.528 INFO:tasks.rados.rados.0.smithi084.stdout:1738: done (1 left) 2024-09-18T14:07:16.528 INFO:tasks.rados.rados.0.smithi084.stdout:1739: copy_from oid 38 from oid 14 current snap is 183 2024-09-18T14:07:16.528 INFO:tasks.rados.rados.0.smithi084.stdout:1737: finishing write tid 4 to smithi08434663-22 2024-09-18T14:07:16.528 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 729 (ObjNum 615 snap 183 seq_num 615) dirty exists 2024-09-18T14:07:16.528 INFO:tasks.rados.rados.0.smithi084.stdout:1737: left oid 22 (ObjNum 615 snap 183 seq_num 615) 2024-09-18T14:07:16.528 INFO:tasks.rados.rados.0.smithi084.stdout:1737: done (1 left) 2024-09-18T14:07:16.529 INFO:tasks.rados.rados.0.smithi084.stdout:1740: snap_remove snap 182 2024-09-18T14:07:17.227 INFO:tasks.rados.rados.0.smithi084.stdout:1740: done (1 left) 2024-09-18T14:07:17.228 INFO:tasks.rados.rados.0.smithi084.stdout:1741: read oid 20 snap -1 2024-09-18T14:07:17.228 INFO:tasks.rados.rados.0.smithi084.stdout:1741: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:17.228 INFO:tasks.rados.rados.0.smithi084.stdout:1742: write oid 40 current snap is 183 2024-09-18T14:07:17.228 INFO:tasks.rados.rados.0.smithi084.stdout:1739: finishing copy_from to smithi08434663-38 2024-09-18T14:07:17.228 INFO:tasks.rados.rados.0.smithi084.stdout:1739: got expected ENOENT (src dne) 2024-09-18T14:07:17.228 INFO:tasks.rados.rados.0.smithi084.stdout:1742: seq_num 617 ranges {769919=653226,1938684=431235} 2024-09-18T14:07:17.236 INFO:tasks.rados.rados.0.smithi084.stdout:1742: writing smithi08434663-40 from 769919 to 1423145 tid 1 2024-09-18T14:07:17.242 INFO:tasks.rados.rados.0.smithi084.stdout:1742: writing smithi08434663-40 from 1938684 to 2369919 tid 2 2024-09-18T14:07:17.242 INFO:tasks.rados.rados.0.smithi084.stdout:1743: write (excl) oid 16 current snap is 183 2024-09-18T14:07:17.242 INFO:tasks.rados.rados.0.smithi084.stdout:1739: finishing copy_from racing read to smithi08434663-38 2024-09-18T14:07:17.242 INFO:tasks.rados.rados.0.smithi084.stdout:1743: seq_num 618 ranges {786726=728475,2280813=105913} 2024-09-18T14:07:17.250 INFO:tasks.rados.rados.0.smithi084.stdout:1743: writing smithi08434663-16 from 786726 to 1515201 tid 1 2024-09-18T14:07:17.255 INFO:tasks.rados.rados.0.smithi084.stdout:1743: writing smithi08434663-16 from 2280813 to 2386726 tid 2 2024-09-18T14:07:17.255 INFO:tasks.rados.rados.0.smithi084.stdout:1739: done (3 left) 2024-09-18T14:07:17.255 INFO:tasks.rados.rados.0.smithi084.stdout:1744: rollback oid 46 current snap is 183 2024-09-18T14:07:17.255 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 46 to 181 2024-09-18T14:07:17.255 INFO:tasks.rados.rados.0.smithi084.stdout:1741: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:17.424 INFO:tasks.rados.rados.0.smithi084.stdout:1742: finishing write tid 1 to smithi08434663-40 2024-09-18T14:07:17.424 INFO:tasks.rados.rados.0.smithi084.stdout:1742: finishing write tid 2 to smithi08434663-40 2024-09-18T14:07:17.424 INFO:tasks.rados.rados.0.smithi084.stdout:1742: finishing write tid 3 to smithi08434663-40 2024-09-18T14:07:17.424 INFO:tasks.rados.rados.0.smithi084.stdout:1742: finishing write tid 4 to smithi08434663-40 2024-09-18T14:07:17.424 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 675 (ObjNum 617 snap 183 seq_num 617) dirty exists 2024-09-18T14:07:17.424 INFO:tasks.rados.rados.0.smithi084.stdout:1742: left oid 40 (ObjNum 617 snap 183 seq_num 617) 2024-09-18T14:07:17.425 INFO:tasks.rados.rados.0.smithi084.stdout:1743: finishing write tid 1 to smithi08434663-16 2024-09-18T14:07:17.425 INFO:tasks.rados.rados.0.smithi084.stdout:1743: finishing write tid 2 to smithi08434663-16 2024-09-18T14:07:17.425 INFO:tasks.rados.rados.0.smithi084.stdout:1741: done (3 left) 2024-09-18T14:07:17.425 INFO:tasks.rados.rados.0.smithi084.stdout:1742: done (2 left) 2024-09-18T14:07:17.425 INFO:tasks.rados.rados.0.smithi084.stdout:1745: copy_from oid 37 from oid 15 current snap is 183 2024-09-18T14:07:17.425 INFO:tasks.rados.rados.0.smithi084.stdout:1743: finishing write tid 3 to smithi08434663-16 2024-09-18T14:07:17.425 INFO:tasks.rados.rados.0.smithi084.stdout:1743: finishing write tid 4 to smithi08434663-16 2024-09-18T14:07:17.425 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 767 (ObjNum 618 snap 183 seq_num 618) dirty exists 2024-09-18T14:07:17.425 INFO:tasks.rados.rados.0.smithi084.stdout:1743: left oid 16 (ObjNum 618 snap 183 seq_num 618) 2024-09-18T14:07:17.425 INFO:tasks.rados.rados.0.smithi084.stdout:1744: finishing rollback tid 1 to smithi08434663-46 2024-09-18T14:07:17.425 INFO:tasks.rados.rados.0.smithi084.stdout:1744: finishing rollback tid 2 to smithi08434663-46 2024-09-18T14:07:17.425 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 769 (ObjNum 545 snap 167 seq_num 545) dirty exists 2024-09-18T14:07:17.426 INFO:tasks.rados.rados.0.smithi084.stdout:1743: done (2 left) 2024-09-18T14:07:17.426 INFO:tasks.rados.rados.0.smithi084.stdout:1744: done (1 left) 2024-09-18T14:07:17.426 INFO:tasks.rados.rados.0.smithi084.stdout:1746: delete oid 9 current snap is 183 2024-09-18T14:07:17.428 INFO:tasks.rados.rados.0.smithi084.stdout:1746: done (1 left) 2024-09-18T14:07:17.428 INFO:tasks.rados.rados.0.smithi084.stdout:1747: snap_create 2024-09-18T14:07:17.437 INFO:tasks.rados.rados.0.smithi084.stdout:1745: finishing copy_from racing read to smithi08434663-37 2024-09-18T14:07:17.441 INFO:tasks.rados.rados.0.smithi084.stdout:1745: finishing copy_from to smithi08434663-37 2024-09-18T14:07:17.441 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 228 (ObjNum 470 snap 147 seq_num 470) dirty exists 2024-09-18T14:07:18.322 INFO:tasks.rados.rados.0.smithi084.stdout:1745: done (1 left) 2024-09-18T14:07:18.322 INFO:tasks.rados.rados.0.smithi084.stdout:1747: done (0 left) 2024-09-18T14:07:18.322 INFO:tasks.rados.rados.0.smithi084.stdout:1748: read oid 3 snap -1 2024-09-18T14:07:18.322 INFO:tasks.rados.rados.0.smithi084.stdout:1748: expect deleted 2024-09-18T14:07:18.322 INFO:tasks.rados.rados.0.smithi084.stdout:1749: snap_create 2024-09-18T14:07:19.319 INFO:tasks.rados.rados.0.smithi084.stdout:1748: done (1 left) 2024-09-18T14:07:19.319 INFO:tasks.rados.rados.0.smithi084.stdout:1749: done (0 left) 2024-09-18T14:07:19.319 INFO:tasks.rados.rados.0.smithi084.stdout:1750: snap_remove snap 184 2024-09-18T14:07:20.329 INFO:tasks.rados.rados.0.smithi084.stdout:1750: done (0 left) 2024-09-18T14:07:20.329 INFO:tasks.rados.rados.0.smithi084.stdout:1751: write oid 11 current snap is 185 2024-09-18T14:07:20.329 INFO:tasks.rados.rados.0.smithi084.stdout:1751: seq_num 620 ranges {420340=478973,1597777=512212} 2024-09-18T14:07:20.333 INFO:tasks.rados.rados.0.smithi084.stdout:1751: writing smithi08434663-11 from 420340 to 899313 tid 1 2024-09-18T14:07:20.341 INFO:tasks.rados.rados.0.smithi084.stdout:1751: writing smithi08434663-11 from 1597777 to 2109989 tid 2 2024-09-18T14:07:20.341 INFO:tasks.rados.rados.0.smithi084.stdout:1752: write (excl) oid 25 current snap is 185 2024-09-18T14:07:20.341 INFO:tasks.rados.rados.0.smithi084.stdout:1752: seq_num 621 ranges {437147=554222,1656259=655870} 2024-09-18T14:07:20.346 INFO:tasks.rados.rados.0.smithi084.stdout:1752: writing smithi08434663-25 from 437147 to 991369 tid 1 2024-09-18T14:07:20.354 INFO:tasks.rados.rados.0.smithi084.stdout:1752: writing smithi08434663-25 from 1656259 to 2312129 tid 2 2024-09-18T14:07:20.354 INFO:tasks.rados.rados.0.smithi084.stdout:1753: write oid 10 current snap is 185 2024-09-18T14:07:20.354 INFO:tasks.rados.rados.0.smithi084.stdout:1751: finishing write tid 1 to smithi08434663-11 2024-09-18T14:07:20.354 INFO:tasks.rados.rados.0.smithi084.stdout:1753: seq_num 622 ranges {453954=629471,1598388=515881} 2024-09-18T14:07:20.359 INFO:tasks.rados.rados.0.smithi084.stdout:1753: writing smithi08434663-10 from 453954 to 1083425 tid 1 2024-09-18T14:07:20.365 INFO:tasks.rados.rados.0.smithi084.stdout:1753: writing smithi08434663-10 from 1598388 to 2114269 tid 2 2024-09-18T14:07:20.365 INFO:tasks.rados.rados.0.smithi084.stdout:1754: write oid 1 current snap is 185 2024-09-18T14:07:20.366 INFO:tasks.rados.rados.0.smithi084.stdout:1751: finishing write tid 2 to smithi08434663-11 2024-09-18T14:07:20.366 INFO:tasks.rados.rados.0.smithi084.stdout:1751: finishing write tid 3 to smithi08434663-11 2024-09-18T14:07:20.366 INFO:tasks.rados.rados.0.smithi084.stdout:1754: seq_num 623 ranges {470761=704720,1656870=659539} 2024-09-18T14:07:20.371 INFO:tasks.rados.rados.0.smithi084.stdout:1754: writing smithi08434663-1 from 470761 to 1175481 tid 1 2024-09-18T14:07:20.378 INFO:tasks.rados.rados.0.smithi084.stdout:1754: writing smithi08434663-1 from 1656870 to 2316409 tid 2 2024-09-18T14:07:20.378 INFO:tasks.rados.rados.0.smithi084.stdout:1755: snap_remove snap 181 2024-09-18T14:07:21.330 INFO:tasks.rados.rados.0.smithi084.stdout:1755: done (4 left) 2024-09-18T14:07:21.330 INFO:tasks.rados.rados.0.smithi084.stdout:1756: read oid 22 snap -1 2024-09-18T14:07:21.330 INFO:tasks.rados.rados.0.smithi084.stdout:1756: expect (ObjNum 615 snap 183 seq_num 615) 2024-09-18T14:07:21.331 INFO:tasks.rados.rados.0.smithi084.stdout:1757: write oid 5 current snap is 185 2024-09-18T14:07:21.331 INFO:tasks.rados.rados.0.smithi084.stdout:1757: seq_num 624 ranges {487568=496322,1431705=519550} 2024-09-18T14:07:21.337 INFO:tasks.rados.rados.0.smithi084.stdout:1757: writing smithi08434663-5 from 487568 to 983890 tid 1 2024-09-18T14:07:21.346 INFO:tasks.rados.rados.0.smithi084.stdout:1757: writing smithi08434663-5 from 1431705 to 1951255 tid 2 2024-09-18T14:07:21.346 INFO:tasks.rados.rados.0.smithi084.stdout:1758: snap_create 2024-09-18T14:07:21.346 INFO:tasks.rados.rados.0.smithi084.stdout:1751: finishing write tid 4 to smithi08434663-11 2024-09-18T14:07:21.346 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 585 (ObjNum 620 snap 185 seq_num 620) dirty exists 2024-09-18T14:07:21.346 INFO:tasks.rados.rados.0.smithi084.stdout:1751: left oid 11 (ObjNum 620 snap 185 seq_num 620) 2024-09-18T14:07:21.346 INFO:tasks.rados.rados.0.smithi084.stdout:1752: finishing write tid 1 to smithi08434663-25 2024-09-18T14:07:21.346 INFO:tasks.rados.rados.0.smithi084.stdout:1752: finishing write tid 2 to smithi08434663-25 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:1752: finishing write tid 3 to smithi08434663-25 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:1752: finishing write tid 4 to smithi08434663-25 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 565 (ObjNum 621 snap 185 seq_num 621) dirty exists 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:1752: left oid 25 (ObjNum 621 snap 185 seq_num 621) 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:1753: finishing write tid 1 to smithi08434663-10 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:1753: finishing write tid 2 to smithi08434663-10 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:1753: finishing write tid 3 to smithi08434663-10 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:1753: finishing write tid 4 to smithi08434663-10 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 739 (ObjNum 622 snap 185 seq_num 622) dirty exists 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:1753: left oid 10 (ObjNum 622 snap 185 seq_num 622) 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:1754: finishing write tid 1 to smithi08434663-1 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:1754: finishing write tid 2 to smithi08434663-1 2024-09-18T14:07:21.347 INFO:tasks.rados.rados.0.smithi084.stdout:1754: finishing write tid 3 to smithi08434663-1 2024-09-18T14:07:21.348 INFO:tasks.rados.rados.0.smithi084.stdout:1754: finishing write tid 4 to smithi08434663-1 2024-09-18T14:07:21.348 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 742 (ObjNum 623 snap 185 seq_num 623) dirty exists 2024-09-18T14:07:21.348 INFO:tasks.rados.rados.0.smithi084.stdout:1754: left oid 1 (ObjNum 623 snap 185 seq_num 623) 2024-09-18T14:07:21.348 INFO:tasks.rados.rados.0.smithi084.stdout:1756: expect (ObjNum 615 snap 183 seq_num 615) 2024-09-18T14:07:21.473 INFO:tasks.rados.rados.0.smithi084.stdout:1757: finishing write tid 1 to smithi08434663-5 2024-09-18T14:07:21.473 INFO:tasks.rados.rados.0.smithi084.stdout:1757: finishing write tid 2 to smithi08434663-5 2024-09-18T14:07:21.474 INFO:tasks.rados.rados.0.smithi084.stdout:1757: finishing write tid 3 to smithi08434663-5 2024-09-18T14:07:21.474 INFO:tasks.rados.rados.0.smithi084.stdout:1757: finishing write tid 4 to smithi08434663-5 2024-09-18T14:07:21.474 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 282 (ObjNum 624 snap 185 seq_num 624) dirty exists 2024-09-18T14:07:21.474 INFO:tasks.rados.rados.0.smithi084.stdout:1757: left oid 5 (ObjNum 624 snap 185 seq_num 624) 2024-09-18T14:07:22.229 INFO:tasks.rados.rados.0.smithi084.stdout:1751: done (6 left) 2024-09-18T14:07:22.230 INFO:tasks.rados.rados.0.smithi084.stdout:1752: done (5 left) 2024-09-18T14:07:22.230 INFO:tasks.rados.rados.0.smithi084.stdout:1753: done (4 left) 2024-09-18T14:07:22.230 INFO:tasks.rados.rados.0.smithi084.stdout:1754: done (3 left) 2024-09-18T14:07:22.230 INFO:tasks.rados.rados.0.smithi084.stdout:1756: done (2 left) 2024-09-18T14:07:22.230 INFO:tasks.rados.rados.0.smithi084.stdout:1757: done (1 left) 2024-09-18T14:07:22.230 INFO:tasks.rados.rados.0.smithi084.stdout:1758: done (0 left) 2024-09-18T14:07:22.230 INFO:tasks.rados.rados.0.smithi084.stdout:1759: rollback oid 2 current snap is 186 2024-09-18T14:07:22.230 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 2 to 183 2024-09-18T14:07:22.230 INFO:tasks.rados.rados.0.smithi084.stdout:1760: rollback oid 25 current snap is 186 2024-09-18T14:07:22.230 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 25 to 183 2024-09-18T14:07:22.230 INFO:tasks.rados.rados.0.smithi084.stdout:1761: read oid 20 snap 183 2024-09-18T14:07:22.231 INFO:tasks.rados.rados.0.smithi084.stdout:1761: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:22.231 INFO:tasks.rados.rados.0.smithi084.stdout:1762: copy_from oid 28 from oid 40 current snap is 186 2024-09-18T14:07:22.231 INFO:tasks.rados.rados.0.smithi084.stdout:1763: copy_from oid 4 from oid 46 current snap is 186 2024-09-18T14:07:22.231 INFO:tasks.rados.rados.0.smithi084.stdout:1764: rollback oid 24 current snap is 186 2024-09-18T14:07:22.231 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 24 to 183 2024-09-18T14:07:22.231 INFO:tasks.rados.rados.0.smithi084.stdout:1765: copy_from oid 31 from oid 42 current snap is 186 2024-09-18T14:07:22.231 INFO:tasks.rados.rados.0.smithi084.stdout:1766: snap_create 2024-09-18T14:07:22.231 INFO:tasks.rados.rados.0.smithi084.stdout:1759: finishing rollback tid 1 to smithi08434663-2 2024-09-18T14:07:22.231 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 0 (ObjNum 97 snap 0 seq_num 2494951472) dirty dne 2024-09-18T14:07:22.233 INFO:tasks.rados.rados.0.smithi084.stdout:1760: finishing rollback tid 0 to smithi08434663-25 2024-09-18T14:07:22.234 INFO:tasks.rados.rados.0.smithi084.stdout:1760: finishing rollback tid 1 to smithi08434663-25 2024-09-18T14:07:22.234 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 567 (ObjNum 209 snap 0 seq_num 2495530736) dirty dne 2024-09-18T14:07:22.238 INFO:tasks.rados.rados.0.smithi084.stdout:1764: finishing rollback tid 0 to smithi08434663-24 2024-09-18T14:07:22.238 INFO:tasks.rados.rados.0.smithi084.stdout:1764: finishing rollback tid 1 to smithi08434663-24 2024-09-18T14:07:22.238 INFO:tasks.rados.rados.0.smithi084.stdout:1765: finishing copy_from to smithi08434663-31 2024-09-18T14:07:22.238 INFO:tasks.rados.rados.0.smithi084.stdout:1765: got expected ENOENT (src dne) 2024-09-18T14:07:22.238 INFO:tasks.rados.rados.0.smithi084.stdout:1765: finishing copy_from racing read to smithi08434663-31 2024-09-18T14:07:22.238 INFO:tasks.rados.rados.0.smithi084.stdout:1764: finishing rollback tid 2 to smithi08434663-24 2024-09-18T14:07:22.238 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 559 (ObjNum 550 snap 168 seq_num 550) dirty exists 2024-09-18T14:07:22.241 INFO:tasks.rados.rados.0.smithi084.stdout:1761: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:22.408 INFO:tasks.rados.rados.0.smithi084.stdout:1762: finishing copy_from racing read to smithi08434663-28 2024-09-18T14:07:22.408 INFO:tasks.rados.rados.0.smithi084.stdout:1763: finishing copy_from racing read to smithi08434663-4 2024-09-18T14:07:22.408 INFO:tasks.rados.rados.0.smithi084.stdout:1763: finishing copy_from to smithi08434663-4 2024-09-18T14:07:22.408 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 769 (ObjNum 545 snap 167 seq_num 545) dirty exists 2024-09-18T14:07:22.408 INFO:tasks.rados.rados.0.smithi084.stdout:1762: finishing copy_from to smithi08434663-28 2024-09-18T14:07:22.408 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 675 (ObjNum 617 snap 183 seq_num 617) dirty exists 2024-09-18T14:07:23.229 INFO:tasks.rados.rados.0.smithi084.stdout:1759: done (7 left) 2024-09-18T14:07:23.229 INFO:tasks.rados.rados.0.smithi084.stdout:1760: done (6 left) 2024-09-18T14:07:23.229 INFO:tasks.rados.rados.0.smithi084.stdout:1761: done (5 left) 2024-09-18T14:07:23.229 INFO:tasks.rados.rados.0.smithi084.stdout:1762: done (4 left) 2024-09-18T14:07:23.229 INFO:tasks.rados.rados.0.smithi084.stdout:1763: done (3 left) 2024-09-18T14:07:23.229 INFO:tasks.rados.rados.0.smithi084.stdout:1764: done (2 left) 2024-09-18T14:07:23.229 INFO:tasks.rados.rados.0.smithi084.stdout:1765: done (1 left) 2024-09-18T14:07:23.230 INFO:tasks.rados.rados.0.smithi084.stdout:1766: done (0 left) 2024-09-18T14:07:23.230 INFO:tasks.rados.rados.0.smithi084.stdout:1767: write oid 1 current snap is 187 2024-09-18T14:07:23.230 INFO:tasks.rados.rados.0.smithi084.stdout:1767: seq_num 625 ranges {504375=571571,1490187=663208} 2024-09-18T14:07:23.234 INFO:tasks.rados.rados.0.smithi084.stdout:1767: writing smithi08434663-1 from 504375 to 1075946 tid 1 2024-09-18T14:07:23.240 INFO:tasks.rados.rados.0.smithi084.stdout:1767: writing smithi08434663-1 from 1490187 to 2153395 tid 2 2024-09-18T14:07:23.240 INFO:tasks.rados.rados.0.smithi084.stdout:1768: delete oid 35 current snap is 187 2024-09-18T14:07:23.245 INFO:tasks.rados.rados.0.smithi084.stdout:1768: done (1 left) 2024-09-18T14:07:23.245 INFO:tasks.rados.rados.0.smithi084.stdout:1769: write oid 19 current snap is 187 2024-09-18T14:07:23.245 INFO:tasks.rados.rados.0.smithi084.stdout:1769: seq_num 627 ranges {537989=722069,1890798=647191} 2024-09-18T14:07:23.252 INFO:tasks.rados.rados.0.smithi084.stdout:1769: writing smithi08434663-19 from 537989 to 1260058 tid 1 2024-09-18T14:07:23.259 INFO:tasks.rados.rados.0.smithi084.stdout:1769: writing smithi08434663-19 from 1890798 to 2537989 tid 2 2024-09-18T14:07:23.259 INFO:tasks.rados.rados.0.smithi084.stdout:1770: delete oid 37 current snap is 187 2024-09-18T14:07:23.259 INFO:tasks.rados.rados.0.smithi084.stdout:1767: finishing write tid 1 to smithi08434663-1 2024-09-18T14:07:23.259 INFO:tasks.rados.rados.0.smithi084.stdout:1767: finishing write tid 2 to smithi08434663-1 2024-09-18T14:07:23.260 INFO:tasks.rados.rados.0.smithi084.stdout:1767: finishing write tid 3 to smithi08434663-1 2024-09-18T14:07:23.260 INFO:tasks.rados.rados.0.smithi084.stdout:1767: finishing write tid 4 to smithi08434663-1 2024-09-18T14:07:23.260 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 746 (ObjNum 625 snap 187 seq_num 625) dirty exists 2024-09-18T14:07:23.260 INFO:tasks.rados.rados.0.smithi084.stdout:1767: left oid 1 (ObjNum 625 snap 187 seq_num 625) 2024-09-18T14:07:23.260 INFO:tasks.rados.rados.0.smithi084.stdout:1769: finishing write tid 1 to smithi08434663-19 2024-09-18T14:07:23.264 INFO:tasks.rados.rados.0.smithi084.stdout:1769: finishing write tid 2 to smithi08434663-19 2024-09-18T14:07:23.264 INFO:tasks.rados.rados.0.smithi084.stdout:1769: finishing write tid 3 to smithi08434663-19 2024-09-18T14:07:23.264 INFO:tasks.rados.rados.0.smithi084.stdout:1769: finishing write tid 4 to smithi08434663-19 2024-09-18T14:07:23.264 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 994 (ObjNum 627 snap 187 seq_num 627) dirty exists 2024-09-18T14:07:23.264 INFO:tasks.rados.rados.0.smithi084.stdout:1769: left oid 19 (ObjNum 627 snap 187 seq_num 627) 2024-09-18T14:07:23.265 INFO:tasks.rados.rados.0.smithi084.stdout:1767: done (2 left) 2024-09-18T14:07:23.265 INFO:tasks.rados.rados.0.smithi084.stdout:1769: done (1 left) 2024-09-18T14:07:23.265 INFO:tasks.rados.rados.0.smithi084.stdout:1770: done (0 left) 2024-09-18T14:07:23.265 INFO:tasks.rados.rados.0.smithi084.stdout:1771: snap_create 2024-09-18T14:07:24.234 INFO:tasks.rados.rados.0.smithi084.stdout:1771: done (0 left) 2024-09-18T14:07:24.234 INFO:tasks.rados.rados.0.smithi084.stdout:1772: read oid 35 snap 187 2024-09-18T14:07:24.234 INFO:tasks.rados.rados.0.smithi084.stdout:1772: expect deleted 2024-09-18T14:07:24.235 INFO:tasks.rados.rados.0.smithi084.stdout:1773: snap_remove snap 183 2024-09-18T14:07:25.236 INFO:tasks.rados.rados.0.smithi084.stdout:1773: done (1 left) 2024-09-18T14:07:25.237 INFO:tasks.rados.rados.0.smithi084.stdout:1774: rollback oid 1 current snap is 188 2024-09-18T14:07:25.237 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 1 to 186 2024-09-18T14:07:25.237 INFO:tasks.rados.rados.0.smithi084.stdout:1772: done (1 left) 2024-09-18T14:07:25.237 INFO:tasks.rados.rados.0.smithi084.stdout:1775: read oid 20 snap -1 2024-09-18T14:07:25.237 INFO:tasks.rados.rados.0.smithi084.stdout:1775: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:25.237 INFO:tasks.rados.rados.0.smithi084.stdout:1776: read oid 31 snap -1 2024-09-18T14:07:25.237 INFO:tasks.rados.rados.0.smithi084.stdout:1776: expect deleted 2024-09-18T14:07:25.237 INFO:tasks.rados.rados.0.smithi084.stdout:1777: snap_remove snap 185 2024-09-18T14:07:26.249 INFO:tasks.rados.rados.0.smithi084.stdout:1777: done (3 left) 2024-09-18T14:07:26.249 INFO:tasks.rados.rados.0.smithi084.stdout:1778: snap_remove snap 187 2024-09-18T14:07:27.230 INFO:tasks.rados.rados.0.smithi084.stdout:1778: done (3 left) 2024-09-18T14:07:27.230 INFO:tasks.rados.rados.0.smithi084.stdout:1779: read oid 34 snap -1 2024-09-18T14:07:27.230 INFO:tasks.rados.rados.0.smithi084.stdout:1779: expect (ObjNum 608 snap 181 seq_num 608) 2024-09-18T14:07:27.230 INFO:tasks.rados.rados.0.smithi084.stdout:1780: snap_create 2024-09-18T14:07:27.230 INFO:tasks.rados.rados.0.smithi084.stdout:1774: finishing rollback tid 0 to smithi08434663-1 2024-09-18T14:07:27.230 INFO:tasks.rados.rados.0.smithi084.stdout:1774: finishing rollback tid 1 to smithi08434663-1 2024-09-18T14:07:27.230 INFO:tasks.rados.rados.0.smithi084.stdout:1775: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:27.400 INFO:tasks.rados.rados.0.smithi084.stdout:1774: finishing rollback tid 2 to smithi08434663-1 2024-09-18T14:07:27.400 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 750 (ObjNum 623 snap 185 seq_num 623) dirty exists 2024-09-18T14:07:27.400 INFO:tasks.rados.rados.0.smithi084.stdout:1779: expect (ObjNum 608 snap 181 seq_num 608) 2024-09-18T14:07:28.237 INFO:tasks.rados.rados.0.smithi084.stdout:1774: done (4 left) 2024-09-18T14:07:28.237 INFO:tasks.rados.rados.0.smithi084.stdout:1775: done (3 left) 2024-09-18T14:07:28.237 INFO:tasks.rados.rados.0.smithi084.stdout:1776: done (2 left) 2024-09-18T14:07:28.237 INFO:tasks.rados.rados.0.smithi084.stdout:1779: done (1 left) 2024-09-18T14:07:28.237 INFO:tasks.rados.rados.0.smithi084.stdout:1780: done (0 left) 2024-09-18T14:07:28.237 INFO:tasks.rados.rados.0.smithi084.stdout:1781: read oid 1 snap 186 2024-09-18T14:07:28.237 INFO:tasks.rados.rados.0.smithi084.stdout:1781: expect (ObjNum 623 snap 185 seq_num 623) 2024-09-18T14:07:28.237 INFO:tasks.rados.rados.0.smithi084.stdout:1782: rollback oid 47 current snap is 189 2024-09-18T14:07:28.237 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 47 to 188 2024-09-18T14:07:28.238 INFO:tasks.rados.rados.0.smithi084.stdout:1783: write (excl) oid 26 current snap is 189 2024-09-18T14:07:28.238 INFO:tasks.rados.rados.0.smithi084.stdout:1783: seq_num 629 ranges {571603=472567,1607762=670546} 2024-09-18T14:07:28.242 INFO:tasks.rados.rados.0.smithi084.stdout:1783: writing smithi08434663-26 from 571603 to 1044170 tid 1 2024-09-18T14:07:28.250 INFO:tasks.rados.rados.0.smithi084.stdout:1783: writing smithi08434663-26 from 1607762 to 2278308 tid 2 2024-09-18T14:07:28.250 INFO:tasks.rados.rados.0.smithi084.stdout:1784: read oid 30 snap -1 2024-09-18T14:07:28.250 INFO:tasks.rados.rados.0.smithi084.stdout:1784: expect (ObjNum 561 snap 171 seq_num 561) 2024-09-18T14:07:28.251 INFO:tasks.rados.rados.0.smithi084.stdout:1785: delete oid 19 current snap is 189 2024-09-18T14:07:28.251 INFO:tasks.rados.rados.0.smithi084.stdout:1781: expect (ObjNum 623 snap 185 seq_num 623) 2024-09-18T14:07:28.328 INFO:tasks.rados.rados.0.smithi084.stdout:1782: finishing rollback tid 0 to smithi08434663-47 2024-09-18T14:07:28.329 INFO:tasks.rados.rados.0.smithi084.stdout:1782: finishing rollback tid 1 to smithi08434663-47 2024-09-18T14:07:28.329 INFO:tasks.rados.rados.0.smithi084.stdout:1782: finishing rollback tid 2 to smithi08434663-47 2024-09-18T14:07:28.329 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 588 (ObjNum 587 snap 177 seq_num 587) dirty exists 2024-09-18T14:07:28.329 INFO:tasks.rados.rados.0.smithi084.stdout:1783: finishing write tid 1 to smithi08434663-26 2024-09-18T14:07:28.329 INFO:tasks.rados.rados.0.smithi084.stdout:1781: done (4 left) 2024-09-18T14:07:28.329 INFO:tasks.rados.rados.0.smithi084.stdout:1782: done (3 left) 2024-09-18T14:07:28.329 INFO:tasks.rados.rados.0.smithi084.stdout:1785: done (2 left) 2024-09-18T14:07:28.329 INFO:tasks.rados.rados.0.smithi084.stdout:1786: delete oid 25 current snap is 189 2024-09-18T14:07:28.329 INFO:tasks.rados.rados.0.smithi084.stdout:1783: finishing write tid 2 to smithi08434663-26 2024-09-18T14:07:28.329 INFO:tasks.rados.rados.0.smithi084.stdout:1784: expect (ObjNum 561 snap 171 seq_num 561) 2024-09-18T14:07:28.673 INFO:tasks.rados.rados.0.smithi084.stdout:1783: finishing write tid 3 to smithi08434663-26 2024-09-18T14:07:28.674 INFO:tasks.rados.rados.0.smithi084.stdout:1783: finishing write tid 4 to smithi08434663-26 2024-09-18T14:07:28.674 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 757 (ObjNum 629 snap 189 seq_num 629) dirty exists 2024-09-18T14:07:28.674 INFO:tasks.rados.rados.0.smithi084.stdout:1783: left oid 26 (ObjNum 629 snap 189 seq_num 629) 2024-09-18T14:07:28.674 INFO:tasks.rados.rados.0.smithi084.stdout:1783: done (2 left) 2024-09-18T14:07:28.674 INFO:tasks.rados.rados.0.smithi084.stdout:1784: done (1 left) 2024-09-18T14:07:28.674 INFO:tasks.rados.rados.0.smithi084.stdout:1786: done (0 left) 2024-09-18T14:07:28.674 INFO:tasks.rados.rados.0.smithi084.stdout:1787: snap_remove snap 188 2024-09-18T14:07:29.253 INFO:tasks.rados.rados.0.smithi084.stdout:1787: done (0 left) 2024-09-18T14:07:29.253 INFO:tasks.rados.rados.0.smithi084.stdout:1788: delete oid 7 current snap is 189 2024-09-18T14:07:29.259 INFO:tasks.rados.rados.0.smithi084.stdout:1788: done (0 left) 2024-09-18T14:07:29.259 INFO:tasks.rados.rados.0.smithi084.stdout:1789: read oid 19 snap -1 2024-09-18T14:07:29.259 INFO:tasks.rados.rados.0.smithi084.stdout:1789: expect deleted 2024-09-18T14:07:29.260 INFO:tasks.rados.rados.0.smithi084.stdout:1790: write oid 4 current snap is 189 2024-09-18T14:07:29.260 INFO:tasks.rados.rados.0.smithi084.stdout:1790: seq_num 633 ranges {638831=489916,1841690=677884} 2024-09-18T14:07:29.265 INFO:tasks.rados.rados.0.smithi084.stdout:1790: writing smithi08434663-4 from 638831 to 1128747 tid 1 2024-09-18T14:07:29.273 INFO:tasks.rados.rados.0.smithi084.stdout:1790: writing smithi08434663-4 from 1841690 to 2519574 tid 2 2024-09-18T14:07:29.273 INFO:tasks.rados.rados.0.smithi084.stdout:1791: copy_from oid 18 from oid 49 current snap is 189 2024-09-18T14:07:29.273 INFO:tasks.rados.rados.0.smithi084.stdout:1790: finishing write tid 1 to smithi08434663-4 2024-09-18T14:07:29.273 INFO:tasks.rados.rados.0.smithi084.stdout:1789: done (2 left) 2024-09-18T14:07:29.274 INFO:tasks.rados.rados.0.smithi084.stdout:1792: read oid 32 snap 186 2024-09-18T14:07:29.274 INFO:tasks.rados.rados.0.smithi084.stdout:1792: expect deleted 2024-09-18T14:07:29.274 INFO:tasks.rados.rados.0.smithi084.stdout:1793: write oid 34 current snap is 189 2024-09-18T14:07:29.274 INFO:tasks.rados.rados.0.smithi084.stdout:1793: seq_num 634 ranges {655638=565165,1900172=421542} 2024-09-18T14:07:29.280 INFO:tasks.rados.rados.0.smithi084.stdout:1793: writing smithi08434663-34 from 655638 to 1220803 tid 1 2024-09-18T14:07:29.287 INFO:tasks.rados.rados.0.smithi084.stdout:1793: writing smithi08434663-34 from 1900172 to 2321714 tid 2 2024-09-18T14:07:29.287 INFO:tasks.rados.rados.0.smithi084.stdout:1794: read oid 11 snap -1 2024-09-18T14:07:29.287 INFO:tasks.rados.rados.0.smithi084.stdout:1794: expect (ObjNum 620 snap 185 seq_num 620) 2024-09-18T14:07:29.287 INFO:tasks.rados.rados.0.smithi084.stdout:1795: rollback oid 30 current snap is 189 2024-09-18T14:07:29.287 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 30 to 186 2024-09-18T14:07:29.287 INFO:tasks.rados.rados.0.smithi084.stdout:1790: finishing write tid 2 to smithi08434663-4 2024-09-18T14:07:29.287 INFO:tasks.rados.rados.0.smithi084.stdout:1790: finishing write tid 3 to smithi08434663-4 2024-09-18T14:07:29.287 INFO:tasks.rados.rados.0.smithi084.stdout:1790: finishing write tid 4 to smithi08434663-4 2024-09-18T14:07:29.288 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 842 (ObjNum 633 snap 189 seq_num 633) dirty exists 2024-09-18T14:07:29.288 INFO:tasks.rados.rados.0.smithi084.stdout:1790: left oid 4 (ObjNum 633 snap 189 seq_num 633) 2024-09-18T14:07:29.288 INFO:tasks.rados.rados.0.smithi084.stdout:1791: finishing copy_from racing read to smithi08434663-18 2024-09-18T14:07:29.288 INFO:tasks.rados.rados.0.smithi084.stdout:1793: finishing write tid 1 to smithi08434663-34 2024-09-18T14:07:29.288 INFO:tasks.rados.rados.0.smithi084.stdout:1791: finishing copy_from to smithi08434663-18 2024-09-18T14:07:29.288 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 637 (ObjNum 397 snap 121 seq_num 397) dirty exists 2024-09-18T14:07:29.288 INFO:tasks.rados.rados.0.smithi084.stdout:1790: done (5 left) 2024-09-18T14:07:29.288 INFO:tasks.rados.rados.0.smithi084.stdout:1791: done (4 left) 2024-09-18T14:07:29.288 INFO:tasks.rados.rados.0.smithi084.stdout:1792: done (3 left) 2024-09-18T14:07:29.288 INFO:tasks.rados.rados.0.smithi084.stdout:1796: read oid 6 snap -1 2024-09-18T14:07:29.288 INFO:tasks.rados.rados.0.smithi084.stdout:1796: expect (ObjNum 538 snap 167 seq_num 538) 2024-09-18T14:07:29.288 INFO:tasks.rados.rados.0.smithi084.stdout:1797: delete oid 25 current snap is 189 2024-09-18T14:07:29.289 INFO:tasks.rados.rados.0.smithi084.stdout:1797: done (4 left) 2024-09-18T14:07:29.289 INFO:tasks.rados.rados.0.smithi084.stdout:1798: copy_from oid 17 from oid 4 current snap is 189 2024-09-18T14:07:29.289 INFO:tasks.rados.rados.0.smithi084.stdout:1799: copy_from oid 39 from oid 19 current snap is 189 2024-09-18T14:07:29.289 INFO:tasks.rados.rados.0.smithi084.stdout:1800: write (excl) oid 35 current snap is 189 2024-09-18T14:07:29.289 INFO:tasks.rados.rados.0.smithi084.stdout:1800: seq_num 636 ranges {689252=715663,1900783=425211} 2024-09-18T14:07:29.296 INFO:tasks.rados.rados.0.smithi084.stdout:1800: writing smithi08434663-35 from 689252 to 1404915 tid 1 2024-09-18T14:07:29.302 INFO:tasks.rados.rados.0.smithi084.stdout:1800: writing smithi08434663-35 from 1900783 to 2325994 tid 2 2024-09-18T14:07:29.302 INFO:tasks.rados.rados.0.smithi084.stdout:1801: write oid 5 current snap is 189 2024-09-18T14:07:29.302 INFO:tasks.rados.rados.0.smithi084.stdout:1801: seq_num 637 ranges {706059=790912,1959265=685222} 2024-09-18T14:07:29.309 INFO:tasks.rados.rados.0.smithi084.stdout:1801: writing smithi08434663-5 from 706059 to 1496971 tid 1 2024-09-18T14:07:29.316 INFO:tasks.rados.rados.0.smithi084.stdout:1801: writing smithi08434663-5 from 1959265 to 2644487 tid 2 2024-09-18T14:07:29.316 INFO:tasks.rados.rados.0.smithi084.stdout:1802: copy_from oid 36 from oid 31 current snap is 189 2024-09-18T14:07:29.316 INFO:tasks.rados.rados.0.smithi084.stdout:1794: expect (ObjNum 620 snap 185 seq_num 620) 2024-09-18T14:07:29.395 INFO:tasks.rados.rados.0.smithi084.stdout:1799: finishing copy_from to smithi08434663-39 2024-09-18T14:07:29.395 INFO:tasks.rados.rados.0.smithi084.stdout:1799: got expected ENOENT (src dne) 2024-09-18T14:07:29.395 INFO:tasks.rados.rados.0.smithi084.stdout:1799: finishing copy_from racing read to smithi08434663-39 2024-09-18T14:07:29.395 INFO:tasks.rados.rados.0.smithi084.stdout:1795: finishing rollback tid 0 to smithi08434663-30 2024-09-18T14:07:29.395 INFO:tasks.rados.rados.0.smithi084.stdout:1795: finishing rollback tid 1 to smithi08434663-30 2024-09-18T14:07:29.395 INFO:tasks.rados.rados.0.smithi084.stdout:1795: finishing rollback tid 2 to smithi08434663-30 2024-09-18T14:07:29.396 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 559 (ObjNum 561 snap 171 seq_num 561) dirty exists 2024-09-18T14:07:29.396 INFO:tasks.rados.rados.0.smithi084.stdout:1793: finishing write tid 2 to smithi08434663-34 2024-09-18T14:07:29.396 INFO:tasks.rados.rados.0.smithi084.stdout:1793: finishing write tid 3 to smithi08434663-34 2024-09-18T14:07:29.396 INFO:tasks.rados.rados.0.smithi084.stdout:1793: finishing write tid 4 to smithi08434663-34 2024-09-18T14:07:29.396 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 846 (ObjNum 634 snap 189 seq_num 634) dirty exists 2024-09-18T14:07:29.396 INFO:tasks.rados.rados.0.smithi084.stdout:1793: left oid 34 (ObjNum 634 snap 189 seq_num 634) 2024-09-18T14:07:29.396 INFO:tasks.rados.rados.0.smithi084.stdout:1796: expect (ObjNum 538 snap 167 seq_num 538) 2024-09-18T14:07:29.516 INFO:tasks.rados.rados.0.smithi084.stdout:1798: finishing copy_from racing read to smithi08434663-17 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:1798: finishing copy_from to smithi08434663-17 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 842 (ObjNum 633 snap 189 seq_num 633) dirty exists 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:1800: finishing write tid 1 to smithi08434663-35 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:1800: finishing write tid 2 to smithi08434663-35 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:1800: finishing write tid 3 to smithi08434663-35 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:1800: finishing write tid 4 to smithi08434663-35 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 784 (ObjNum 636 snap 189 seq_num 636) dirty exists 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:1800: left oid 35 (ObjNum 636 snap 189 seq_num 636) 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:1801: finishing write tid 1 to smithi08434663-5 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:1802: finishing copy_from to smithi08434663-36 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:1802: got expected ENOENT (src dne) 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:1802: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:07:29.517 INFO:tasks.rados.rados.0.smithi084.stdout:1793: done (8 left) 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:1794: done (7 left) 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:1795: done (6 left) 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:1796: done (5 left) 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:1798: done (4 left) 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:1799: done (3 left) 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:1800: done (2 left) 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:1802: done (1 left) 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:1803: read oid 31 snap -1 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:1803: expect deleted 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:1804: rollback oid 1 current snap is 189 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 1 to 186 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:1801: finishing write tid 2 to smithi08434663-5 2024-09-18T14:07:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:1801: finishing write tid 3 to smithi08434663-5 2024-09-18T14:07:29.519 INFO:tasks.rados.rados.0.smithi084.stdout:1801: finishing write tid 4 to smithi08434663-5 2024-09-18T14:07:29.519 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 286 (ObjNum 637 snap 189 seq_num 637) dirty exists 2024-09-18T14:07:29.519 INFO:tasks.rados.rados.0.smithi084.stdout:1801: left oid 5 (ObjNum 637 snap 189 seq_num 637) 2024-09-18T14:07:29.519 INFO:tasks.rados.rados.0.smithi084.stdout:1801: done (2 left) 2024-09-18T14:07:29.519 INFO:tasks.rados.rados.0.smithi084.stdout:1805: read oid 13 snap -1 2024-09-18T14:07:29.519 INFO:tasks.rados.rados.0.smithi084.stdout:1805: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:29.519 INFO:tasks.rados.rados.0.smithi084.stdout:1806: write (excl) oid 48 current snap is 189 2024-09-18T14:07:29.519 INFO:tasks.rados.rados.0.smithi084.stdout:1806: seq_num 638 ranges {722866=466161,1617747=428880,2587634=135232} 2024-09-18T14:07:29.522 INFO:tasks.rados.rados.0.smithi084.stdout:1806: writing smithi08434663-48 from 722866 to 1189027 tid 1 2024-09-18T14:07:29.527 INFO:tasks.rados.rados.0.smithi084.stdout:1806: writing smithi08434663-48 from 1617747 to 2046627 tid 2 2024-09-18T14:07:29.531 INFO:tasks.rados.rados.0.smithi084.stdout:1806: writing smithi08434663-48 from 2587634 to 2722866 tid 3 2024-09-18T14:07:29.531 INFO:tasks.rados.rados.0.smithi084.stdout:1807: rollback oid 39 current snap is 189 2024-09-18T14:07:29.531 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 39 to 186 2024-09-18T14:07:29.531 INFO:tasks.rados.rados.0.smithi084.stdout:1804: finishing rollback tid 0 to smithi08434663-1 2024-09-18T14:07:29.531 INFO:tasks.rados.rados.0.smithi084.stdout:1804: finishing rollback tid 1 to smithi08434663-1 2024-09-18T14:07:29.532 INFO:tasks.rados.rados.0.smithi084.stdout:1804: finishing rollback tid 2 to smithi08434663-1 2024-09-18T14:07:29.532 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 760 (ObjNum 623 snap 185 seq_num 623) dirty exists 2024-09-18T14:07:29.532 INFO:tasks.rados.rados.0.smithi084.stdout:1805: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:29.696 INFO:tasks.rados.rados.0.smithi084.stdout:1806: finishing write tid 1 to smithi08434663-48 2024-09-18T14:07:29.696 INFO:tasks.rados.rados.0.smithi084.stdout:1806: finishing write tid 2 to smithi08434663-48 2024-09-18T14:07:29.696 INFO:tasks.rados.rados.0.smithi084.stdout:1807: finishing rollback tid 1 to smithi08434663-39 2024-09-18T14:07:29.696 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 0 (ObjNum 33 snap 0 seq_num 2494978944) dirty dne 2024-09-18T14:07:29.696 INFO:tasks.rados.rados.0.smithi084.stdout:1806: finishing write tid 3 to smithi08434663-48 2024-09-18T14:07:29.696 INFO:tasks.rados.rados.0.smithi084.stdout:1806: finishing write tid 4 to smithi08434663-48 2024-09-18T14:07:29.696 INFO:tasks.rados.rados.0.smithi084.stdout:1806: finishing write tid 5 to smithi08434663-48 2024-09-18T14:07:29.696 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 571 (ObjNum 638 snap 189 seq_num 638) dirty exists 2024-09-18T14:07:29.696 INFO:tasks.rados.rados.0.smithi084.stdout:1806: left oid 48 (ObjNum 638 snap 189 seq_num 638) 2024-09-18T14:07:29.696 INFO:tasks.rados.rados.0.smithi084.stdout:1803: done (4 left) 2024-09-18T14:07:29.696 INFO:tasks.rados.rados.0.smithi084.stdout:1804: done (3 left) 2024-09-18T14:07:29.696 INFO:tasks.rados.rados.0.smithi084.stdout:1805: done (2 left) 2024-09-18T14:07:29.697 INFO:tasks.rados.rados.0.smithi084.stdout:1806: done (1 left) 2024-09-18T14:07:29.697 INFO:tasks.rados.rados.0.smithi084.stdout:1807: done (0 left) 2024-09-18T14:07:29.697 INFO:tasks.rados.rados.0.smithi084.stdout:1808: delete oid 1 current snap is 189 2024-09-18T14:07:29.700 INFO:tasks.rados.rados.0.smithi084.stdout:1808: done (0 left) 2024-09-18T14:07:29.700 INFO:tasks.rados.rados.0.smithi084.stdout:1809: rollback oid 21 current snap is 189 2024-09-18T14:07:29.700 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 21 to 186 2024-09-18T14:07:29.700 INFO:tasks.rados.rados.0.smithi084.stdout:1810: rollback oid 17 current snap is 189 2024-09-18T14:07:29.700 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 17 to 186 2024-09-18T14:07:29.700 INFO:tasks.rados.rados.0.smithi084.stdout:1811: read oid 10 snap -1 2024-09-18T14:07:29.700 INFO:tasks.rados.rados.0.smithi084.stdout:1811: expect (ObjNum 622 snap 185 seq_num 622) 2024-09-18T14:07:29.700 INFO:tasks.rados.rados.0.smithi084.stdout:1812: read oid 27 snap -1 2024-09-18T14:07:29.701 INFO:tasks.rados.rados.0.smithi084.stdout:1812: expect (ObjNum 594 snap 179 seq_num 594) 2024-09-18T14:07:29.701 INFO:tasks.rados.rados.0.smithi084.stdout:1813: write oid 36 current snap is 189 2024-09-18T14:07:29.701 INFO:tasks.rados.rados.0.smithi084.stdout:1813: seq_num 640 ranges {756480=733012,2134711=432549} 2024-09-18T14:07:29.708 INFO:tasks.rados.rados.0.smithi084.stdout:1813: writing smithi08434663-36 from 756480 to 1489492 tid 1 2024-09-18T14:07:29.715 INFO:tasks.rados.rados.0.smithi084.stdout:1813: writing smithi08434663-36 from 2134711 to 2567260 tid 2 2024-09-18T14:07:29.715 INFO:tasks.rados.rados.0.smithi084.stdout:1814: rollback oid 43 current snap is 189 2024-09-18T14:07:29.715 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 186 2024-09-18T14:07:29.715 INFO:tasks.rados.rados.0.smithi084.stdout:1809: finishing rollback tid 1 to smithi08434663-21 2024-09-18T14:07:29.715 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 0 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-09-18T14:07:29.715 INFO:tasks.rados.rados.0.smithi084.stdout:1810: finishing rollback tid 0 to smithi08434663-17 2024-09-18T14:07:29.715 INFO:tasks.rados.rados.0.smithi084.stdout:1810: finishing rollback tid 1 to smithi08434663-17 2024-09-18T14:07:29.715 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 844 (ObjNum 145 snap 0 seq_num 2494941248) dirty dne 2024-09-18T14:07:29.715 INFO:tasks.rados.rados.0.smithi084.stdout:1811: expect (ObjNum 622 snap 185 seq_num 622) 2024-09-18T14:07:29.793 INFO:tasks.rados.rados.0.smithi084.stdout:1812: expect (ObjNum 594 snap 179 seq_num 594) 2024-09-18T14:07:29.920 INFO:tasks.rados.rados.0.smithi084.stdout:1814: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:07:29.920 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 0 (ObjNum 33 snap 0 seq_num 2494986944) dirty dne 2024-09-18T14:07:29.920 INFO:tasks.rados.rados.0.smithi084.stdout:1813: finishing write tid 1 to smithi08434663-36 2024-09-18T14:07:29.921 INFO:tasks.rados.rados.0.smithi084.stdout:1813: finishing write tid 2 to smithi08434663-36 2024-09-18T14:07:29.921 INFO:tasks.rados.rados.0.smithi084.stdout:1813: finishing write tid 3 to smithi08434663-36 2024-09-18T14:07:29.921 INFO:tasks.rados.rados.0.smithi084.stdout:1809: done (5 left) 2024-09-18T14:07:29.921 INFO:tasks.rados.rados.0.smithi084.stdout:1810: done (4 left) 2024-09-18T14:07:29.921 INFO:tasks.rados.rados.0.smithi084.stdout:1811: done (3 left) 2024-09-18T14:07:29.921 INFO:tasks.rados.rados.0.smithi084.stdout:1812: done (2 left) 2024-09-18T14:07:29.921 INFO:tasks.rados.rados.0.smithi084.stdout:1814: done (1 left) 2024-09-18T14:07:29.921 INFO:tasks.rados.rados.0.smithi084.stdout:1815: write (excl) oid 10 current snap is 189 2024-09-18T14:07:29.921 INFO:tasks.rados.rados.0.smithi084.stdout:1813: finishing write tid 4 to smithi08434663-36 2024-09-18T14:07:29.921 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 646 (ObjNum 640 snap 189 seq_num 640) dirty exists 2024-09-18T14:07:29.921 INFO:tasks.rados.rados.0.smithi084.stdout:1813: left oid 36 (ObjNum 640 snap 189 seq_num 640) 2024-09-18T14:07:29.921 INFO:tasks.rados.rados.0.smithi084.stdout:1815: seq_num 641 ranges {773287=408261,1793193=576207} 2024-09-18T14:07:29.926 INFO:tasks.rados.rados.0.smithi084.stdout:1815: writing smithi08434663-10 from 773287 to 1181548 tid 1 2024-09-18T14:07:29.933 INFO:tasks.rados.rados.0.smithi084.stdout:1815: writing smithi08434663-10 from 1793193 to 2369400 tid 2 2024-09-18T14:07:29.933 INFO:tasks.rados.rados.0.smithi084.stdout:1813: done (1 left) 2024-09-18T14:07:29.933 INFO:tasks.rados.rados.0.smithi084.stdout:1816: copy_from oid 4 from oid 6 current snap is 189 2024-09-18T14:07:29.933 INFO:tasks.rados.rados.0.smithi084.stdout:1815: finishing write tid 1 to smithi08434663-10 2024-09-18T14:07:29.933 INFO:tasks.rados.rados.0.smithi084.stdout:1817: write (excl) oid 2 current snap is 189 2024-09-18T14:07:29.933 INFO:tasks.rados.rados.0.smithi084.stdout:1817: seq_num 642 ranges {790094=483510,1851675=436218} 2024-09-18T14:07:29.939 INFO:tasks.rados.rados.0.smithi084.stdout:1817: writing smithi08434663-2 from 790094 to 1273604 tid 1 2024-09-18T14:07:29.945 INFO:tasks.rados.rados.0.smithi084.stdout:1817: writing smithi08434663-2 from 1851675 to 2287893 tid 2 2024-09-18T14:07:29.945 INFO:tasks.rados.rados.0.smithi084.stdout:1818: snap_remove snap 186 2024-09-18T14:07:30.264 INFO:tasks.rados.rados.0.smithi084.stdout:1818: done (3 left) 2024-09-18T14:07:30.264 INFO:tasks.rados.rados.0.smithi084.stdout:1819: rollback oid 24 current snap is 189 2024-09-18T14:07:30.264 INFO:tasks.rados.rados.0.smithi084.stdout:1819: done (3 left) 2024-09-18T14:07:30.264 INFO:tasks.rados.rados.0.smithi084.stdout:1820: write (excl) oid 25 current snap is 189 2024-09-18T14:07:30.264 INFO:tasks.rados.rados.0.smithi084.stdout:1820: seq_num 643 ranges {406901=558759,1393804=579876,2608396=198505} 2024-09-18T14:07:30.269 INFO:tasks.rados.rados.0.smithi084.stdout:1820: writing smithi08434663-25 from 406901 to 965660 tid 1 2024-09-18T14:07:30.275 INFO:tasks.rados.rados.0.smithi084.stdout:1820: writing smithi08434663-25 from 1393804 to 1973680 tid 2 2024-09-18T14:07:30.280 INFO:tasks.rados.rados.0.smithi084.stdout:1820: writing smithi08434663-25 from 2608396 to 2806901 tid 3 2024-09-18T14:07:30.281 INFO:tasks.rados.rados.0.smithi084.stdout:1821: read oid 50 snap -1 2024-09-18T14:07:30.281 INFO:tasks.rados.rados.0.smithi084.stdout:1821: expect (ObjNum 572 snap 175 seq_num 572) 2024-09-18T14:07:30.281 INFO:tasks.rados.rados.0.smithi084.stdout:1822: snap_create 2024-09-18T14:07:30.281 INFO:tasks.rados.rados.0.smithi084.stdout:1815: finishing write tid 2 to smithi08434663-10 2024-09-18T14:07:30.281 INFO:tasks.rados.rados.0.smithi084.stdout:1815: finishing write tid 3 to smithi08434663-10 2024-09-18T14:07:30.281 INFO:tasks.rados.rados.0.smithi084.stdout:1815: finishing write tid 4 to smithi08434663-10 2024-09-18T14:07:30.281 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 765 (ObjNum 641 snap 189 seq_num 641) dirty exists 2024-09-18T14:07:30.281 INFO:tasks.rados.rados.0.smithi084.stdout:1815: left oid 10 (ObjNum 641 snap 189 seq_num 641) 2024-09-18T14:07:30.281 INFO:tasks.rados.rados.0.smithi084.stdout:1816: finishing copy_from racing read to smithi08434663-4 2024-09-18T14:07:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:1816: finishing copy_from to smithi08434663-4 2024-09-18T14:07:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 575 (ObjNum 538 snap 167 seq_num 538) dirty exists 2024-09-18T14:07:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:1817: finishing write tid 1 to smithi08434663-2 2024-09-18T14:07:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:1817: finishing write tid 2 to smithi08434663-2 2024-09-18T14:07:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:1817: finishing write tid 3 to smithi08434663-2 2024-09-18T14:07:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:1817: finishing write tid 4 to smithi08434663-2 2024-09-18T14:07:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 1005 (ObjNum 642 snap 189 seq_num 642) dirty exists 2024-09-18T14:07:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:1817: left oid 2 (ObjNum 642 snap 189 seq_num 642) 2024-09-18T14:07:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:1820: finishing write tid 1 to smithi08434663-25 2024-09-18T14:07:30.282 INFO:tasks.rados.rados.0.smithi084.stdout:1820: finishing write tid 2 to smithi08434663-25 2024-09-18T14:07:30.283 INFO:tasks.rados.rados.0.smithi084.stdout:1821: expect (ObjNum 572 snap 175 seq_num 572) 2024-09-18T14:07:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:1820: finishing write tid 3 to smithi08434663-25 2024-09-18T14:07:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:1820: finishing write tid 4 to smithi08434663-25 2024-09-18T14:07:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:1820: finishing write tid 5 to smithi08434663-25 2024-09-18T14:07:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 575 (ObjNum 643 snap 189 seq_num 643) dirty exists 2024-09-18T14:07:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:1820: left oid 25 (ObjNum 643 snap 189 seq_num 643) 2024-09-18T14:07:31.259 INFO:tasks.rados.rados.0.smithi084.stdout:1815: done (5 left) 2024-09-18T14:07:31.259 INFO:tasks.rados.rados.0.smithi084.stdout:1816: done (4 left) 2024-09-18T14:07:31.259 INFO:tasks.rados.rados.0.smithi084.stdout:1817: done (3 left) 2024-09-18T14:07:31.259 INFO:tasks.rados.rados.0.smithi084.stdout:1820: done (2 left) 2024-09-18T14:07:31.259 INFO:tasks.rados.rados.0.smithi084.stdout:1821: done (1 left) 2024-09-18T14:07:31.259 INFO:tasks.rados.rados.0.smithi084.stdout:1822: done (0 left) 2024-09-18T14:07:31.259 INFO:tasks.rados.rados.0.smithi084.stdout:1823: write oid 4 current snap is 190 2024-09-18T14:07:31.260 INFO:tasks.rados.rados.0.smithi084.stdout:1823: seq_num 644 ranges {423708=634008,1852286=439887,2752172=71536} 2024-09-18T14:07:31.266 INFO:tasks.rados.rados.0.smithi084.stdout:1823: writing smithi08434663-4 from 423708 to 1057716 tid 1 2024-09-18T14:07:31.273 INFO:tasks.rados.rados.0.smithi084.stdout:1823: writing smithi08434663-4 from 1852286 to 2292173 tid 2 2024-09-18T14:07:31.276 INFO:tasks.rados.rados.0.smithi084.stdout:1823: writing smithi08434663-4 from 2752172 to 2823708 tid 3 2024-09-18T14:07:31.276 INFO:tasks.rados.rados.0.smithi084.stdout:1824: snap_remove snap 189 2024-09-18T14:07:32.232 INFO:tasks.rados.rados.0.smithi084.stdout:1824: done (1 left) 2024-09-18T14:07:32.233 INFO:tasks.rados.rados.0.smithi084.stdout:1825: read oid 28 snap -1 2024-09-18T14:07:32.233 INFO:tasks.rados.rados.0.smithi084.stdout:1825: expect (ObjNum 617 snap 183 seq_num 617) 2024-09-18T14:07:32.233 INFO:tasks.rados.rados.0.smithi084.stdout:1826: copy_from oid 22 from oid 39 current snap is 190 2024-09-18T14:07:32.233 INFO:tasks.rados.rados.0.smithi084.stdout:1823: finishing write tid 1 to smithi08434663-4 2024-09-18T14:07:32.233 INFO:tasks.rados.rados.0.smithi084.stdout:1823: finishing write tid 2 to smithi08434663-4 2024-09-18T14:07:32.233 INFO:tasks.rados.rados.0.smithi084.stdout:1823: finishing write tid 3 to smithi08434663-4 2024-09-18T14:07:32.233 INFO:tasks.rados.rados.0.smithi084.stdout:1823: finishing write tid 4 to smithi08434663-4 2024-09-18T14:07:32.233 INFO:tasks.rados.rados.0.smithi084.stdout:1823: finishing write tid 5 to smithi08434663-4 2024-09-18T14:07:32.233 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 856 (ObjNum 644 snap 190 seq_num 644) dirty exists 2024-09-18T14:07:32.233 INFO:tasks.rados.rados.0.smithi084.stdout:1823: left oid 4 (ObjNum 644 snap 190 seq_num 644) 2024-09-18T14:07:32.233 INFO:tasks.rados.rados.0.smithi084.stdout:1823: done (2 left) 2024-09-18T14:07:32.234 INFO:tasks.rados.rados.0.smithi084.stdout:1827: read oid 7 snap -1 2024-09-18T14:07:32.234 INFO:tasks.rados.rados.0.smithi084.stdout:1827: expect deleted 2024-09-18T14:07:32.234 INFO:tasks.rados.rados.0.smithi084.stdout:1828: read oid 8 snap -1 2024-09-18T14:07:32.234 INFO:tasks.rados.rados.0.smithi084.stdout:1828: expect (ObjNum 551 snap 168 seq_num 551) 2024-09-18T14:07:32.234 INFO:tasks.rados.rados.0.smithi084.stdout:1829: read oid 36 snap -1 2024-09-18T14:07:32.234 INFO:tasks.rados.rados.0.smithi084.stdout:1829: expect (ObjNum 640 snap 189 seq_num 640) 2024-09-18T14:07:32.234 INFO:tasks.rados.rados.0.smithi084.stdout:1830: write (excl) oid 5 current snap is 190 2024-09-18T14:07:32.234 INFO:tasks.rados.rados.0.smithi084.stdout:1830: seq_num 645 ranges {440515=709257,1910768=583545} 2024-09-18T14:07:32.238 INFO:tasks.rados.rados.0.smithi084.stdout:1830: writing smithi08434663-5 from 440515 to 1149772 tid 1 2024-09-18T14:07:32.246 INFO:tasks.rados.rados.0.smithi084.stdout:1830: writing smithi08434663-5 from 1910768 to 2494313 tid 2 2024-09-18T14:07:32.246 INFO:tasks.rados.rados.0.smithi084.stdout:1831: write oid 11 current snap is 190 2024-09-18T14:07:32.246 INFO:tasks.rados.rados.0.smithi084.stdout:1826: finishing copy_from to smithi08434663-22 2024-09-18T14:07:32.247 INFO:tasks.rados.rados.0.smithi084.stdout:1826: got expected ENOENT (src dne) 2024-09-18T14:07:32.247 INFO:tasks.rados.rados.0.smithi084.stdout:1826: finishing copy_from racing read to smithi08434663-22 2024-09-18T14:07:32.247 INFO:tasks.rados.rados.0.smithi084.stdout:1831: seq_num 646 ranges {457322=784506,1969250=443556} 2024-09-18T14:07:32.253 INFO:tasks.rados.rados.0.smithi084.stdout:1831: writing smithi08434663-11 from 457322 to 1241828 tid 1 2024-09-18T14:07:32.260 INFO:tasks.rados.rados.0.smithi084.stdout:1831: writing smithi08434663-11 from 1969250 to 2412806 tid 2 2024-09-18T14:07:32.260 INFO:tasks.rados.rados.0.smithi084.stdout:1826: done (6 left) 2024-09-18T14:07:32.260 INFO:tasks.rados.rados.0.smithi084.stdout:1827: done (5 left) 2024-09-18T14:07:32.260 INFO:tasks.rados.rados.0.smithi084.stdout:1832: read oid 20 snap -1 2024-09-18T14:07:32.260 INFO:tasks.rados.rados.0.smithi084.stdout:1832: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:32.260 INFO:tasks.rados.rados.0.smithi084.stdout:1833: write (excl) oid 45 current snap is 190 2024-09-18T14:07:32.260 INFO:tasks.rados.rados.0.smithi084.stdout:1833: seq_num 647 ranges {474129=576108,1627732=587214,2718088=156041} 2024-09-18T14:07:32.266 INFO:tasks.rados.rados.0.smithi084.stdout:1833: writing smithi08434663-45 from 474129 to 1050237 tid 1 2024-09-18T14:07:32.272 INFO:tasks.rados.rados.0.smithi084.stdout:1833: writing smithi08434663-45 from 1627732 to 2214946 tid 2 2024-09-18T14:07:32.276 INFO:tasks.rados.rados.0.smithi084.stdout:1833: writing smithi08434663-45 from 2718088 to 2874129 tid 3 2024-09-18T14:07:32.276 INFO:tasks.rados.rados.0.smithi084.stdout:1834: read oid 46 snap -1 2024-09-18T14:07:32.276 INFO:tasks.rados.rados.0.smithi084.stdout:1834: expect (ObjNum 545 snap 167 seq_num 545) 2024-09-18T14:07:32.276 INFO:tasks.rados.rados.0.smithi084.stdout:1835: snap_create 2024-09-18T14:07:32.277 INFO:tasks.rados.rados.0.smithi084.stdout:1825: expect (ObjNum 617 snap 183 seq_num 617) 2024-09-18T14:07:32.414 INFO:tasks.rados.rados.0.smithi084.stdout:1830: finishing write tid 1 to smithi08434663-5 2024-09-18T14:07:32.414 INFO:tasks.rados.rados.0.smithi084.stdout:1828: expect (ObjNum 551 snap 168 seq_num 551) 2024-09-18T14:07:32.514 INFO:tasks.rados.rados.0.smithi084.stdout:1829: expect (ObjNum 640 snap 189 seq_num 640) 2024-09-18T14:07:32.692 INFO:tasks.rados.rados.0.smithi084.stdout:1830: finishing write tid 2 to smithi08434663-5 2024-09-18T14:07:32.692 INFO:tasks.rados.rados.0.smithi084.stdout:1830: finishing write tid 3 to smithi08434663-5 2024-09-18T14:07:32.693 INFO:tasks.rados.rados.0.smithi084.stdout:1830: finishing write tid 4 to smithi08434663-5 2024-09-18T14:07:32.693 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 291 (ObjNum 645 snap 190 seq_num 645) dirty exists 2024-09-18T14:07:32.693 INFO:tasks.rados.rados.0.smithi084.stdout:1830: left oid 5 (ObjNum 645 snap 190 seq_num 645) 2024-09-18T14:07:32.693 INFO:tasks.rados.rados.0.smithi084.stdout:1831: finishing write tid 1 to smithi08434663-11 2024-09-18T14:07:32.693 INFO:tasks.rados.rados.0.smithi084.stdout:1831: finishing write tid 2 to smithi08434663-11 2024-09-18T14:07:32.693 INFO:tasks.rados.rados.0.smithi084.stdout:1831: finishing write tid 3 to smithi08434663-11 2024-09-18T14:07:32.693 INFO:tasks.rados.rados.0.smithi084.stdout:1831: finishing write tid 4 to smithi08434663-11 2024-09-18T14:07:32.693 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 591 (ObjNum 646 snap 190 seq_num 646) dirty exists 2024-09-18T14:07:32.693 INFO:tasks.rados.rados.0.smithi084.stdout:1831: left oid 11 (ObjNum 646 snap 190 seq_num 646) 2024-09-18T14:07:32.693 INFO:tasks.rados.rados.0.smithi084.stdout:1832: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:32.856 INFO:tasks.rados.rados.0.smithi084.stdout:1833: finishing write tid 1 to smithi08434663-45 2024-09-18T14:07:32.856 INFO:tasks.rados.rados.0.smithi084.stdout:1833: finishing write tid 2 to smithi08434663-45 2024-09-18T14:07:32.856 INFO:tasks.rados.rados.0.smithi084.stdout:1833: finishing write tid 3 to smithi08434663-45 2024-09-18T14:07:32.856 INFO:tasks.rados.rados.0.smithi084.stdout:1833: finishing write tid 4 to smithi08434663-45 2024-09-18T14:07:32.856 INFO:tasks.rados.rados.0.smithi084.stdout:1833: finishing write tid 5 to smithi08434663-45 2024-09-18T14:07:32.856 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 862 (ObjNum 647 snap 190 seq_num 647) dirty exists 2024-09-18T14:07:32.856 INFO:tasks.rados.rados.0.smithi084.stdout:1833: left oid 45 (ObjNum 647 snap 190 seq_num 647) 2024-09-18T14:07:32.856 INFO:tasks.rados.rados.0.smithi084.stdout:1834: expect (ObjNum 545 snap 167 seq_num 545) 2024-09-18T14:07:33.274 INFO:tasks.rados.rados.0.smithi084.stdout:1825: done (8 left) 2024-09-18T14:07:33.274 INFO:tasks.rados.rados.0.smithi084.stdout:1828: done (7 left) 2024-09-18T14:07:33.274 INFO:tasks.rados.rados.0.smithi084.stdout:1829: done (6 left) 2024-09-18T14:07:33.274 INFO:tasks.rados.rados.0.smithi084.stdout:1830: done (5 left) 2024-09-18T14:07:33.274 INFO:tasks.rados.rados.0.smithi084.stdout:1831: done (4 left) 2024-09-18T14:07:33.274 INFO:tasks.rados.rados.0.smithi084.stdout:1832: done (3 left) 2024-09-18T14:07:33.274 INFO:tasks.rados.rados.0.smithi084.stdout:1833: done (2 left) 2024-09-18T14:07:33.274 INFO:tasks.rados.rados.0.smithi084.stdout:1834: done (1 left) 2024-09-18T14:07:33.274 INFO:tasks.rados.rados.0.smithi084.stdout:1835: done (0 left) 2024-09-18T14:07:33.275 INFO:tasks.rados.rados.0.smithi084.stdout:1836: write oid 5 current snap is 191 2024-09-18T14:07:33.275 INFO:tasks.rados.rados.0.smithi084.stdout:1836: seq_num 648 ranges {490936=651357,1686214=447225,2861864=29072} 2024-09-18T14:07:33.279 INFO:tasks.rados.rados.0.smithi084.stdout:1836: writing smithi08434663-5 from 490936 to 1142293 tid 1 2024-09-18T14:07:33.285 INFO:tasks.rados.rados.0.smithi084.stdout:1836: writing smithi08434663-5 from 1686214 to 2133439 tid 2 2024-09-18T14:07:33.289 INFO:tasks.rados.rados.0.smithi084.stdout:1836: writing smithi08434663-5 from 2861864 to 2890936 tid 3 2024-09-18T14:07:33.290 INFO:tasks.rados.rados.0.smithi084.stdout:1837: snap_remove snap 190 2024-09-18T14:07:34.281 INFO:tasks.rados.rados.0.smithi084.stdout:1837: done (1 left) 2024-09-18T14:07:34.281 INFO:tasks.rados.rados.0.smithi084.stdout:1838: read oid 9 snap -1 2024-09-18T14:07:34.281 INFO:tasks.rados.rados.0.smithi084.stdout:1838: expect deleted 2024-09-18T14:07:34.281 INFO:tasks.rados.rados.0.smithi084.stdout:1839: copy_from oid 16 from oid 13 current snap is 191 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1836: finishing write tid 1 to smithi08434663-5 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1836: finishing write tid 2 to smithi08434663-5 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1836: finishing write tid 3 to smithi08434663-5 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1836: finishing write tid 4 to smithi08434663-5 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1836: finishing write tid 5 to smithi08434663-5 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 296 (ObjNum 648 snap 191 seq_num 648) dirty exists 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1836: left oid 5 (ObjNum 648 snap 191 seq_num 648) 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1836: done (2 left) 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1840: read oid 27 snap -1 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1840: expect (ObjNum 594 snap 179 seq_num 594) 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1841: read oid 11 snap -1 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1841: expect (ObjNum 646 snap 190 seq_num 646) 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1842: read oid 1 snap -1 2024-09-18T14:07:34.282 INFO:tasks.rados.rados.0.smithi084.stdout:1842: expect deleted 2024-09-18T14:07:34.283 INFO:tasks.rados.rados.0.smithi084.stdout:1843: write oid 49 current snap is 191 2024-09-18T14:07:34.283 INFO:tasks.rados.rados.0.smithi084.stdout:1843: seq_num 649 ranges {507743=726606,1744696=590883,2772934=134809} 2024-09-18T14:07:34.287 INFO:tasks.rados.rados.0.smithi084.stdout:1843: writing smithi08434663-49 from 507743 to 1234349 tid 1 2024-09-18T14:07:34.295 INFO:tasks.rados.rados.0.smithi084.stdout:1843: writing smithi08434663-49 from 1744696 to 2335579 tid 2 2024-09-18T14:07:34.298 INFO:tasks.rados.rados.0.smithi084.stdout:1843: writing smithi08434663-49 from 2772934 to 2907743 tid 3 2024-09-18T14:07:34.298 INFO:tasks.rados.rados.0.smithi084.stdout:1844: delete oid 17 current snap is 191 2024-09-18T14:07:34.298 INFO:tasks.rados.rados.0.smithi084.stdout:1840: expect (ObjNum 594 snap 179 seq_num 594) 2024-09-18T14:07:34.425 INFO:tasks.rados.rados.0.smithi084.stdout:1839: finishing copy_from racing read to smithi08434663-16 2024-09-18T14:07:34.425 INFO:tasks.rados.rados.0.smithi084.stdout:1841: expect (ObjNum 646 snap 190 seq_num 646) 2024-09-18T14:07:34.549 INFO:tasks.rados.rados.0.smithi084.stdout:1839: finishing copy_from to smithi08434663-16 2024-09-18T14:07:34.549 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 597 (ObjNum 534 snap 165 seq_num 534) dirty exists 2024-09-18T14:07:34.550 INFO:tasks.rados.rados.0.smithi084.stdout:1843: finishing write tid 1 to smithi08434663-49 2024-09-18T14:07:34.550 INFO:tasks.rados.rados.0.smithi084.stdout:1843: finishing write tid 2 to smithi08434663-49 2024-09-18T14:07:34.550 INFO:tasks.rados.rados.0.smithi084.stdout:1843: finishing write tid 3 to smithi08434663-49 2024-09-18T14:07:34.550 INFO:tasks.rados.rados.0.smithi084.stdout:1843: finishing write tid 4 to smithi08434663-49 2024-09-18T14:07:34.550 INFO:tasks.rados.rados.0.smithi084.stdout:1843: finishing write tid 5 to smithi08434663-49 2024-09-18T14:07:34.550 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 650 (ObjNum 649 snap 191 seq_num 649) dirty exists 2024-09-18T14:07:34.550 INFO:tasks.rados.rados.0.smithi084.stdout:1843: left oid 49 (ObjNum 649 snap 191 seq_num 649) 2024-09-18T14:07:34.550 INFO:tasks.rados.rados.0.smithi084.stdout:1838: done (6 left) 2024-09-18T14:07:34.550 INFO:tasks.rados.rados.0.smithi084.stdout:1839: done (5 left) 2024-09-18T14:07:34.551 INFO:tasks.rados.rados.0.smithi084.stdout:1840: done (4 left) 2024-09-18T14:07:34.551 INFO:tasks.rados.rados.0.smithi084.stdout:1841: done (3 left) 2024-09-18T14:07:34.551 INFO:tasks.rados.rados.0.smithi084.stdout:1842: done (2 left) 2024-09-18T14:07:34.551 INFO:tasks.rados.rados.0.smithi084.stdout:1843: done (1 left) 2024-09-18T14:07:34.551 INFO:tasks.rados.rados.0.smithi084.stdout:1844: done (0 left) 2024-09-18T14:07:34.551 INFO:tasks.rados.rados.0.smithi084.stdout:1845: copy_from oid 15 from oid 34 current snap is 191 2024-09-18T14:07:34.551 INFO:tasks.rados.rados.0.smithi084.stdout:1846: read oid 14 snap -1 2024-09-18T14:07:34.551 INFO:tasks.rados.rados.0.smithi084.stdout:1846: expect deleted 2024-09-18T14:07:34.551 INFO:tasks.rados.rados.0.smithi084.stdout:1847: delete oid 35 current snap is 191 2024-09-18T14:07:34.553 INFO:tasks.rados.rados.0.smithi084.stdout:1846: done (2 left) 2024-09-18T14:07:34.553 INFO:tasks.rados.rados.0.smithi084.stdout:1847: done (1 left) 2024-09-18T14:07:34.553 INFO:tasks.rados.rados.0.smithi084.stdout:1848: read oid 1 snap -1 2024-09-18T14:07:34.553 INFO:tasks.rados.rados.0.smithi084.stdout:1848: expect deleted 2024-09-18T14:07:34.553 INFO:tasks.rados.rados.0.smithi084.stdout:1849: read oid 40 snap -1 2024-09-18T14:07:34.553 INFO:tasks.rados.rados.0.smithi084.stdout:1849: expect (ObjNum 617 snap 183 seq_num 617) 2024-09-18T14:07:34.553 INFO:tasks.rados.rados.0.smithi084.stdout:1850: write oid 35 current snap is 191 2024-09-18T14:07:34.553 INFO:tasks.rados.rados.0.smithi084.stdout:1850: seq_num 652 ranges {558164=552353,1803789=454563,2855203=102961} 2024-09-18T14:07:34.557 INFO:tasks.rados.rados.0.smithi084.stdout:1850: writing smithi08434663-35 from 558164 to 1110517 tid 1 2024-09-18T14:07:34.564 INFO:tasks.rados.rados.0.smithi084.stdout:1850: writing smithi08434663-35 from 1803789 to 2258352 tid 2 2024-09-18T14:07:34.568 INFO:tasks.rados.rados.0.smithi084.stdout:1850: writing smithi08434663-35 from 2855203 to 2958164 tid 3 2024-09-18T14:07:34.568 INFO:tasks.rados.rados.0.smithi084.stdout:1851: write oid 5 current snap is 191 2024-09-18T14:07:34.568 INFO:tasks.rados.rados.0.smithi084.stdout:1851: seq_num 653 ranges {574971=627602,1862271=598221} 2024-09-18T14:07:34.574 INFO:tasks.rados.rados.0.smithi084.stdout:1851: writing smithi08434663-5 from 574971 to 1202573 tid 1 2024-09-18T14:07:34.581 INFO:tasks.rados.rados.0.smithi084.stdout:1851: writing smithi08434663-5 from 1862271 to 2460492 tid 2 2024-09-18T14:07:34.581 INFO:tasks.rados.rados.0.smithi084.stdout:1852: rollback oid 46 current snap is 191 2024-09-18T14:07:34.581 INFO:tasks.rados.rados.0.smithi084.stdout:1852: done (5 left) 2024-09-18T14:07:34.581 INFO:tasks.rados.rados.0.smithi084.stdout:1853: copy_from oid 21 from oid 41 current snap is 191 2024-09-18T14:07:34.582 INFO:tasks.rados.rados.0.smithi084.stdout:1849: expect (ObjNum 617 snap 183 seq_num 617) 2024-09-18T14:07:34.719 INFO:tasks.rados.rados.0.smithi084.stdout:1845: finishing copy_from racing read to smithi08434663-15 2024-09-18T14:07:34.720 INFO:tasks.rados.rados.0.smithi084.stdout:1845: finishing copy_from to smithi08434663-15 2024-09-18T14:07:34.720 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 846 (ObjNum 634 snap 189 seq_num 634) dirty exists 2024-09-18T14:07:34.720 INFO:tasks.rados.rados.0.smithi084.stdout:1850: finishing write tid 1 to smithi08434663-35 2024-09-18T14:07:34.720 INFO:tasks.rados.rados.0.smithi084.stdout:1845: done (5 left) 2024-09-18T14:07:34.720 INFO:tasks.rados.rados.0.smithi084.stdout:1848: done (4 left) 2024-09-18T14:07:34.720 INFO:tasks.rados.rados.0.smithi084.stdout:1849: done (3 left) 2024-09-18T14:07:34.720 INFO:tasks.rados.rados.0.smithi084.stdout:1854: rollback oid 27 current snap is 191 2024-09-18T14:07:34.720 INFO:tasks.rados.rados.0.smithi084.stdout:1854: done (3 left) 2024-09-18T14:07:34.720 INFO:tasks.rados.rados.0.smithi084.stdout:1855: read oid 10 snap -1 2024-09-18T14:07:34.720 INFO:tasks.rados.rados.0.smithi084.stdout:1855: expect (ObjNum 641 snap 189 seq_num 641) 2024-09-18T14:07:34.721 INFO:tasks.rados.rados.0.smithi084.stdout:1856: copy_from oid 17 from oid 48 current snap is 191 2024-09-18T14:07:34.721 INFO:tasks.rados.rados.0.smithi084.stdout:1850: finishing write tid 2 to smithi08434663-35 2024-09-18T14:07:34.721 INFO:tasks.rados.rados.0.smithi084.stdout:1850: finishing write tid 3 to smithi08434663-35 2024-09-18T14:07:34.721 INFO:tasks.rados.rados.0.smithi084.stdout:1850: finishing write tid 4 to smithi08434663-35 2024-09-18T14:07:34.721 INFO:tasks.rados.rados.0.smithi084.stdout:1850: finishing write tid 5 to smithi08434663-35 2024-09-18T14:07:34.721 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 850 (ObjNum 652 snap 191 seq_num 652) dirty exists 2024-09-18T14:07:34.721 INFO:tasks.rados.rados.0.smithi084.stdout:1850: left oid 35 (ObjNum 652 snap 191 seq_num 652) 2024-09-18T14:07:34.721 INFO:tasks.rados.rados.0.smithi084.stdout:1851: finishing write tid 1 to smithi08434663-5 2024-09-18T14:07:34.721 INFO:tasks.rados.rados.0.smithi084.stdout:1851: finishing write tid 2 to smithi08434663-5 2024-09-18T14:07:34.721 INFO:tasks.rados.rados.0.smithi084.stdout:1853: finishing copy_from racing read to smithi08434663-21 2024-09-18T14:07:34.721 INFO:tasks.rados.rados.0.smithi084.stdout:1851: finishing write tid 3 to smithi08434663-5 2024-09-18T14:07:34.721 INFO:tasks.rados.rados.0.smithi084.stdout:1851: finishing write tid 4 to smithi08434663-5 2024-09-18T14:07:34.722 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 301 (ObjNum 653 snap 191 seq_num 653) dirty exists 2024-09-18T14:07:34.722 INFO:tasks.rados.rados.0.smithi084.stdout:1851: left oid 5 (ObjNum 653 snap 191 seq_num 653) 2024-09-18T14:07:34.722 INFO:tasks.rados.rados.0.smithi084.stdout:1853: finishing copy_from to smithi08434663-21 2024-09-18T14:07:34.722 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 590 (ObjNum 482 snap 151 seq_num 482) dirty exists 2024-09-18T14:07:34.722 INFO:tasks.rados.rados.0.smithi084.stdout:1850: done (4 left) 2024-09-18T14:07:34.722 INFO:tasks.rados.rados.0.smithi084.stdout:1851: done (3 left) 2024-09-18T14:07:34.722 INFO:tasks.rados.rados.0.smithi084.stdout:1853: done (2 left) 2024-09-18T14:07:34.722 INFO:tasks.rados.rados.0.smithi084.stdout:1857: snap_create 2024-09-18T14:07:34.723 INFO:tasks.rados.rados.0.smithi084.stdout:1855: expect (ObjNum 641 snap 189 seq_num 641) 2024-09-18T14:07:34.848 INFO:tasks.rados.rados.0.smithi084.stdout:1856: finishing copy_from racing read to smithi08434663-17 2024-09-18T14:07:34.848 INFO:tasks.rados.rados.0.smithi084.stdout:1856: finishing copy_from to smithi08434663-17 2024-09-18T14:07:34.848 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 571 (ObjNum 638 snap 189 seq_num 638) dirty exists 2024-09-18T14:07:35.293 INFO:tasks.rados.rados.0.smithi084.stdout:1855: done (2 left) 2024-09-18T14:07:35.294 INFO:tasks.rados.rados.0.smithi084.stdout:1856: done (1 left) 2024-09-18T14:07:35.294 INFO:tasks.rados.rados.0.smithi084.stdout:1857: done (0 left) 2024-09-18T14:07:35.294 INFO:tasks.rados.rados.0.smithi084.stdout:1858: snap_remove snap 191 2024-09-18T14:07:36.299 INFO:tasks.rados.rados.0.smithi084.stdout:1858: done (0 left) 2024-09-18T14:07:36.299 INFO:tasks.rados.rados.0.smithi084.stdout:1859: snap_create 2024-09-18T14:07:37.234 INFO:tasks.rados.rados.0.smithi084.stdout:1859: done (0 left) 2024-09-18T14:07:37.234 INFO:tasks.rados.rados.0.smithi084.stdout:1860: snap_create 2024-09-18T14:07:38.243 INFO:tasks.rados.rados.0.smithi084.stdout:1860: done (0 left) 2024-09-18T14:07:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1861: write (excl) oid 31 current snap is 194 2024-09-18T14:07:38.244 INFO:tasks.rados.rados.0.smithi084.stdout:1861: seq_num 654 ranges {591778=419204,1637106=741879,2910049=81729} 2024-09-18T14:07:38.249 INFO:tasks.rados.rados.0.smithi084.stdout:1861: writing smithi08434663-31 from 591778 to 1010982 tid 1 2024-09-18T14:07:38.256 INFO:tasks.rados.rados.0.smithi084.stdout:1861: writing smithi08434663-31 from 1637106 to 2378985 tid 2 2024-09-18T14:07:38.259 INFO:tasks.rados.rados.0.smithi084.stdout:1861: writing smithi08434663-31 from 2910049 to 2991778 tid 3 2024-09-18T14:07:38.260 INFO:tasks.rados.rados.0.smithi084.stdout:1862: snap_remove snap 193 2024-09-18T14:07:39.245 INFO:tasks.rados.rados.0.smithi084.stdout:1862: done (1 left) 2024-09-18T14:07:39.245 INFO:tasks.rados.rados.0.smithi084.stdout:1863: snap_create 2024-09-18T14:07:39.245 INFO:tasks.rados.rados.0.smithi084.stdout:1861: finishing write tid 1 to smithi08434663-31 2024-09-18T14:07:39.245 INFO:tasks.rados.rados.0.smithi084.stdout:1861: finishing write tid 2 to smithi08434663-31 2024-09-18T14:07:39.245 INFO:tasks.rados.rados.0.smithi084.stdout:1861: finishing write tid 3 to smithi08434663-31 2024-09-18T14:07:39.245 INFO:tasks.rados.rados.0.smithi084.stdout:1861: finishing write tid 4 to smithi08434663-31 2024-09-18T14:07:39.246 INFO:tasks.rados.rados.0.smithi084.stdout:1861: finishing write tid 5 to smithi08434663-31 2024-09-18T14:07:39.246 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 563 (ObjNum 654 snap 194 seq_num 654) dirty exists 2024-09-18T14:07:39.246 INFO:tasks.rados.rados.0.smithi084.stdout:1861: left oid 31 (ObjNum 654 snap 194 seq_num 654) 2024-09-18T14:07:40.260 INFO:tasks.rados.rados.0.smithi084.stdout:1861: done (1 left) 2024-09-18T14:07:40.260 INFO:tasks.rados.rados.0.smithi084.stdout:1863: done (0 left) 2024-09-18T14:07:40.260 INFO:tasks.rados.rados.0.smithi084.stdout:1864: snap_remove snap 194 2024-09-18T14:07:41.263 INFO:tasks.rados.rados.0.smithi084.stdout:1864: done (0 left) 2024-09-18T14:07:41.263 INFO:tasks.rados.rados.0.smithi084.stdout:1865: read oid 46 snap -1 2024-09-18T14:07:41.263 INFO:tasks.rados.rados.0.smithi084.stdout:1865: expect (ObjNum 545 snap 167 seq_num 545) 2024-09-18T14:07:41.263 INFO:tasks.rados.rados.0.smithi084.stdout:1866: read oid 13 snap -1 2024-09-18T14:07:41.263 INFO:tasks.rados.rados.0.smithi084.stdout:1866: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:41.263 INFO:tasks.rados.rados.0.smithi084.stdout:1867: write oid 38 current snap is 195 2024-09-18T14:07:41.263 INFO:tasks.rados.rados.0.smithi084.stdout:1867: seq_num 655 ranges {608585=494453,1579235=601890,2821119=187466} 2024-09-18T14:07:41.270 INFO:tasks.rados.rados.0.smithi084.stdout:1867: writing smithi08434663-38 from 608585 to 1103038 tid 1 2024-09-18T14:07:41.276 INFO:tasks.rados.rados.0.smithi084.stdout:1867: writing smithi08434663-38 from 1579235 to 2181125 tid 2 2024-09-18T14:07:41.281 INFO:tasks.rados.rados.0.smithi084.stdout:1867: writing smithi08434663-38 from 2821119 to 3008585 tid 3 2024-09-18T14:07:41.281 INFO:tasks.rados.rados.0.smithi084.stdout:1868: snap_remove snap 192 2024-09-18T14:07:42.234 INFO:tasks.rados.rados.0.smithi084.stdout:1868: done (3 left) 2024-09-18T14:07:42.234 INFO:tasks.rados.rados.0.smithi084.stdout:1869: read oid 15 snap -1 2024-09-18T14:07:42.234 INFO:tasks.rados.rados.0.smithi084.stdout:1869: expect (ObjNum 634 snap 189 seq_num 634) 2024-09-18T14:07:42.234 INFO:tasks.rados.rados.0.smithi084.stdout:1870: read oid 32 snap -1 2024-09-18T14:07:42.234 INFO:tasks.rados.rados.0.smithi084.stdout:1870: expect deleted 2024-09-18T14:07:42.234 INFO:tasks.rados.rados.0.smithi084.stdout:1871: write (excl) oid 40 current snap is 195 2024-09-18T14:07:42.235 INFO:tasks.rados.rados.0.smithi084.stdout:1871: seq_num 656 ranges {625392=569702,1637717=745548,2848542=176850} 2024-09-18T14:07:42.241 INFO:tasks.rados.rados.0.smithi084.stdout:1871: writing smithi08434663-40 from 625392 to 1195094 tid 1 2024-09-18T14:07:42.247 INFO:tasks.rados.rados.0.smithi084.stdout:1871: writing smithi08434663-40 from 1637717 to 2383265 tid 2 2024-09-18T14:07:42.251 INFO:tasks.rados.rados.0.smithi084.stdout:1871: writing smithi08434663-40 from 2848542 to 3025392 tid 3 2024-09-18T14:07:42.251 INFO:tasks.rados.rados.0.smithi084.stdout:1872: write (excl) oid 44 current snap is 195 2024-09-18T14:07:42.251 INFO:tasks.rados.rados.0.smithi084.stdout:1872: seq_num 657 ranges {642199=644951,1696199=605559,2992318=49881} 2024-09-18T14:07:42.258 INFO:tasks.rados.rados.0.smithi084.stdout:1872: writing smithi08434663-44 from 642199 to 1287150 tid 1 2024-09-18T14:07:42.264 INFO:tasks.rados.rados.0.smithi084.stdout:1872: writing smithi08434663-44 from 1696199 to 2301758 tid 2 2024-09-18T14:07:42.268 INFO:tasks.rados.rados.0.smithi084.stdout:1872: writing smithi08434663-44 from 2992318 to 3042199 tid 3 2024-09-18T14:07:42.268 INFO:tasks.rados.rados.0.smithi084.stdout:1873: write oid 7 current snap is 195 2024-09-18T14:07:42.268 INFO:tasks.rados.rados.0.smithi084.stdout:1873: seq_num 658 ranges {659006=720200,2154681=749217} 2024-09-18T14:07:42.276 INFO:tasks.rados.rados.0.smithi084.stdout:1873: writing smithi08434663-7 from 659006 to 1379206 tid 1 2024-09-18T14:07:42.285 INFO:tasks.rados.rados.0.smithi084.stdout:1873: writing smithi08434663-7 from 2154681 to 2903898 tid 2 2024-09-18T14:07:42.285 INFO:tasks.rados.rados.0.smithi084.stdout:1874: rollback oid 20 current snap is 195 2024-09-18T14:07:42.285 INFO:tasks.rados.rados.0.smithi084.stdout:1874: done (8 left) 2024-09-18T14:07:42.285 INFO:tasks.rados.rados.0.smithi084.stdout:1875: rollback oid 35 current snap is 195 2024-09-18T14:07:42.285 INFO:tasks.rados.rados.0.smithi084.stdout:1875: done (8 left) 2024-09-18T14:07:42.285 INFO:tasks.rados.rados.0.smithi084.stdout:1876: read oid 47 snap -1 2024-09-18T14:07:42.285 INFO:tasks.rados.rados.0.smithi084.stdout:1876: expect (ObjNum 587 snap 177 seq_num 587) 2024-09-18T14:07:42.285 INFO:tasks.rados.rados.0.smithi084.stdout:1877: read oid 37 snap -1 2024-09-18T14:07:42.285 INFO:tasks.rados.rados.0.smithi084.stdout:1877: expect deleted 2024-09-18T14:07:42.285 INFO:tasks.rados.rados.0.smithi084.stdout:1878: read oid 23 snap -1 2024-09-18T14:07:42.285 INFO:tasks.rados.rados.0.smithi084.stdout:1878: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:07:42.286 INFO:tasks.rados.rados.0.smithi084.stdout:1879: snap_create 2024-09-18T14:07:42.286 INFO:tasks.rados.rados.0.smithi084.stdout:1865: expect (ObjNum 545 snap 167 seq_num 545) 2024-09-18T14:07:42.488 INFO:tasks.rados.rados.0.smithi084.stdout:1866: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:42.651 INFO:tasks.rados.rados.0.smithi084.stdout:1867: finishing write tid 1 to smithi08434663-38 2024-09-18T14:07:42.652 INFO:tasks.rados.rados.0.smithi084.stdout:1867: finishing write tid 2 to smithi08434663-38 2024-09-18T14:07:42.652 INFO:tasks.rados.rados.0.smithi084.stdout:1867: finishing write tid 3 to smithi08434663-38 2024-09-18T14:07:42.652 INFO:tasks.rados.rados.0.smithi084.stdout:1867: finishing write tid 4 to smithi08434663-38 2024-09-18T14:07:42.652 INFO:tasks.rados.rados.0.smithi084.stdout:1867: finishing write tid 5 to smithi08434663-38 2024-09-18T14:07:42.652 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 627 (ObjNum 655 snap 195 seq_num 655) dirty exists 2024-09-18T14:07:42.652 INFO:tasks.rados.rados.0.smithi084.stdout:1867: left oid 38 (ObjNum 655 snap 195 seq_num 655) 2024-09-18T14:07:42.652 INFO:tasks.rados.rados.0.smithi084.stdout:1869: expect (ObjNum 634 snap 189 seq_num 634) 2024-09-18T14:07:42.794 INFO:tasks.rados.rados.0.smithi084.stdout:1871: finishing write tid 1 to smithi08434663-40 2024-09-18T14:07:42.794 INFO:tasks.rados.rados.0.smithi084.stdout:1871: finishing write tid 2 to smithi08434663-40 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:1871: finishing write tid 3 to smithi08434663-40 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:1871: finishing write tid 4 to smithi08434663-40 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:1871: finishing write tid 5 to smithi08434663-40 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 848 (ObjNum 656 snap 195 seq_num 656) dirty exists 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:1871: left oid 40 (ObjNum 656 snap 195 seq_num 656) 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:1872: finishing write tid 1 to smithi08434663-44 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:1872: finishing write tid 2 to smithi08434663-44 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:1872: finishing write tid 3 to smithi08434663-44 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:1872: finishing write tid 4 to smithi08434663-44 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:1872: finishing write tid 5 to smithi08434663-44 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 305 (ObjNum 657 snap 195 seq_num 657) dirty exists 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:1872: left oid 44 (ObjNum 657 snap 195 seq_num 657) 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:1873: finishing write tid 1 to smithi08434663-7 2024-09-18T14:07:42.795 INFO:tasks.rados.rados.0.smithi084.stdout:1876: expect (ObjNum 587 snap 177 seq_num 587) 2024-09-18T14:07:42.917 INFO:tasks.rados.rados.0.smithi084.stdout:1873: finishing write tid 2 to smithi08434663-7 2024-09-18T14:07:42.917 INFO:tasks.rados.rados.0.smithi084.stdout:1873: finishing write tid 3 to smithi08434663-7 2024-09-18T14:07:42.918 INFO:tasks.rados.rados.0.smithi084.stdout:1873: finishing write tid 4 to smithi08434663-7 2024-09-18T14:07:42.918 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 600 (ObjNum 658 snap 195 seq_num 658) dirty exists 2024-09-18T14:07:42.918 INFO:tasks.rados.rados.0.smithi084.stdout:1873: left oid 7 (ObjNum 658 snap 195 seq_num 658) 2024-09-18T14:07:42.918 INFO:tasks.rados.rados.0.smithi084.stdout:1878: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:07:43.274 INFO:tasks.rados.rados.0.smithi084.stdout:1865: done (11 left) 2024-09-18T14:07:43.274 INFO:tasks.rados.rados.0.smithi084.stdout:1866: done (10 left) 2024-09-18T14:07:43.274 INFO:tasks.rados.rados.0.smithi084.stdout:1867: done (9 left) 2024-09-18T14:07:43.274 INFO:tasks.rados.rados.0.smithi084.stdout:1869: done (8 left) 2024-09-18T14:07:43.275 INFO:tasks.rados.rados.0.smithi084.stdout:1870: done (7 left) 2024-09-18T14:07:43.275 INFO:tasks.rados.rados.0.smithi084.stdout:1871: done (6 left) 2024-09-18T14:07:43.275 INFO:tasks.rados.rados.0.smithi084.stdout:1872: done (5 left) 2024-09-18T14:07:43.275 INFO:tasks.rados.rados.0.smithi084.stdout:1873: done (4 left) 2024-09-18T14:07:43.275 INFO:tasks.rados.rados.0.smithi084.stdout:1876: done (3 left) 2024-09-18T14:07:43.275 INFO:tasks.rados.rados.0.smithi084.stdout:1877: done (2 left) 2024-09-18T14:07:43.275 INFO:tasks.rados.rados.0.smithi084.stdout:1878: done (1 left) 2024-09-18T14:07:43.275 INFO:tasks.rados.rados.0.smithi084.stdout:1879: done (0 left) 2024-09-18T14:07:43.275 INFO:tasks.rados.rados.0.smithi084.stdout:1880: read oid 29 snap 195 2024-09-18T14:07:43.275 INFO:tasks.rados.rados.0.smithi084.stdout:1880: expect deleted 2024-09-18T14:07:43.275 INFO:tasks.rados.rados.0.smithi084.stdout:1881: write oid 38 current snap is 196 2024-09-18T14:07:43.275 INFO:tasks.rados.rados.0.smithi084.stdout:1881: seq_num 659 ranges {675813=795449,2096810=609228} 2024-09-18T14:07:43.282 INFO:tasks.rados.rados.0.smithi084.stdout:1881: writing smithi08434663-38 from 675813 to 1471262 tid 1 2024-09-18T14:07:43.289 INFO:tasks.rados.rados.0.smithi084.stdout:1881: writing smithi08434663-38 from 2096810 to 2706038 tid 2 2024-09-18T14:07:43.289 INFO:tasks.rados.rados.0.smithi084.stdout:1882: write oid 31 current snap is 196 2024-09-18T14:07:43.289 INFO:tasks.rados.rados.0.smithi084.stdout:1882: seq_num 660 ranges {692620=470698,1755292=752886} 2024-09-18T14:07:43.295 INFO:tasks.rados.rados.0.smithi084.stdout:1882: writing smithi08434663-31 from 692620 to 1163318 tid 1 2024-09-18T14:07:43.302 INFO:tasks.rados.rados.0.smithi084.stdout:1882: writing smithi08434663-31 from 1755292 to 2508178 tid 2 2024-09-18T14:07:43.302 INFO:tasks.rados.rados.0.smithi084.stdout:1883: delete oid 5 current snap is 196 2024-09-18T14:07:43.303 INFO:tasks.rados.rados.0.smithi084.stdout:1881: finishing write tid 1 to smithi08434663-38 2024-09-18T14:07:43.303 INFO:tasks.rados.rados.0.smithi084.stdout:1881: finishing write tid 2 to smithi08434663-38 2024-09-18T14:07:43.303 INFO:tasks.rados.rados.0.smithi084.stdout:1881: finishing write tid 3 to smithi08434663-38 2024-09-18T14:07:43.303 INFO:tasks.rados.rados.0.smithi084.stdout:1881: finishing write tid 4 to smithi08434663-38 2024-09-18T14:07:43.303 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 630 (ObjNum 659 snap 196 seq_num 659) dirty exists 2024-09-18T14:07:43.304 INFO:tasks.rados.rados.0.smithi084.stdout:1881: left oid 38 (ObjNum 659 snap 196 seq_num 659) 2024-09-18T14:07:43.304 INFO:tasks.rados.rados.0.smithi084.stdout:1882: finishing write tid 1 to smithi08434663-31 2024-09-18T14:07:43.305 INFO:tasks.rados.rados.0.smithi084.stdout:1880: done (3 left) 2024-09-18T14:07:43.305 INFO:tasks.rados.rados.0.smithi084.stdout:1881: done (2 left) 2024-09-18T14:07:43.305 INFO:tasks.rados.rados.0.smithi084.stdout:1883: done (1 left) 2024-09-18T14:07:43.306 INFO:tasks.rados.rados.0.smithi084.stdout:1884: snap_remove snap 195 2024-09-18T14:07:44.288 INFO:tasks.rados.rados.0.smithi084.stdout:1884: done (1 left) 2024-09-18T14:07:44.288 INFO:tasks.rados.rados.0.smithi084.stdout:1885: copy_from oid 9 from oid 20 current snap is 196 2024-09-18T14:07:44.289 INFO:tasks.rados.rados.0.smithi084.stdout:1882: finishing write tid 2 to smithi08434663-31 2024-09-18T14:07:44.289 INFO:tasks.rados.rados.0.smithi084.stdout:1882: finishing write tid 3 to smithi08434663-31 2024-09-18T14:07:44.289 INFO:tasks.rados.rados.0.smithi084.stdout:1882: finishing write tid 4 to smithi08434663-31 2024-09-18T14:07:44.289 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 566 (ObjNum 660 snap 196 seq_num 660) dirty exists 2024-09-18T14:07:44.289 INFO:tasks.rados.rados.0.smithi084.stdout:1882: left oid 31 (ObjNum 660 snap 196 seq_num 660) 2024-09-18T14:07:44.289 INFO:tasks.rados.rados.0.smithi084.stdout:1882: done (1 left) 2024-09-18T14:07:44.289 INFO:tasks.rados.rados.0.smithi084.stdout:1886: write oid 3 current snap is 196 2024-09-18T14:07:44.289 INFO:tasks.rados.rados.0.smithi084.stdout:1886: seq_num 662 ranges {726234=737549,1988609=756555} 2024-09-18T14:07:44.296 INFO:tasks.rados.rados.0.smithi084.stdout:1886: writing smithi08434663-3 from 726234 to 1463783 tid 1 2024-09-18T14:07:44.303 INFO:tasks.rados.rados.0.smithi084.stdout:1886: writing smithi08434663-3 from 1988609 to 2745164 tid 2 2024-09-18T14:07:44.303 INFO:tasks.rados.rados.0.smithi084.stdout:1887: snap_create 2024-09-18T14:07:44.303 INFO:tasks.rados.rados.0.smithi084.stdout:1885: finishing copy_from racing read to smithi08434663-9 2024-09-18T14:07:44.304 INFO:tasks.rados.rados.0.smithi084.stdout:1885: finishing copy_from to smithi08434663-9 2024-09-18T14:07:44.304 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 623 (ObjNum 534 snap 165 seq_num 534) dirty exists 2024-09-18T14:07:44.304 INFO:tasks.rados.rados.0.smithi084.stdout:1886: finishing write tid 1 to smithi08434663-3 2024-09-18T14:07:44.309 INFO:tasks.rados.rados.0.smithi084.stdout:1886: finishing write tid 2 to smithi08434663-3 2024-09-18T14:07:44.310 INFO:tasks.rados.rados.0.smithi084.stdout:1886: finishing write tid 3 to smithi08434663-3 2024-09-18T14:07:44.310 INFO:tasks.rados.rados.0.smithi084.stdout:1886: finishing write tid 4 to smithi08434663-3 2024-09-18T14:07:44.310 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 732 (ObjNum 662 snap 196 seq_num 662) dirty exists 2024-09-18T14:07:44.310 INFO:tasks.rados.rados.0.smithi084.stdout:1886: left oid 3 (ObjNum 662 snap 196 seq_num 662) 2024-09-18T14:07:45.290 INFO:tasks.rados.rados.0.smithi084.stdout:1885: done (2 left) 2024-09-18T14:07:45.290 INFO:tasks.rados.rados.0.smithi084.stdout:1886: done (1 left) 2024-09-18T14:07:45.290 INFO:tasks.rados.rados.0.smithi084.stdout:1887: done (0 left) 2024-09-18T14:07:45.290 INFO:tasks.rados.rados.0.smithi084.stdout:1888: snap_create 2024-09-18T14:07:46.304 INFO:tasks.rados.rados.0.smithi084.stdout:1888: done (0 left) 2024-09-18T14:07:46.304 INFO:tasks.rados.rados.0.smithi084.stdout:1889: snap_remove snap 196 2024-09-18T14:07:47.236 INFO:tasks.rados.rados.0.smithi084.stdout:1889: done (0 left) 2024-09-18T14:07:47.236 INFO:tasks.rados.rados.0.smithi084.stdout:1890: read oid 34 snap -1 2024-09-18T14:07:47.236 INFO:tasks.rados.rados.0.smithi084.stdout:1890: expect (ObjNum 634 snap 189 seq_num 634) 2024-09-18T14:07:47.236 INFO:tasks.rados.rados.0.smithi084.stdout:1891: delete oid 1 current snap is 198 2024-09-18T14:07:47.243 INFO:tasks.rados.rados.0.smithi084.stdout:1891: done (1 left) 2024-09-18T14:07:47.243 INFO:tasks.rados.rados.0.smithi084.stdout:1892: read oid 29 snap 197 2024-09-18T14:07:47.243 INFO:tasks.rados.rados.0.smithi084.stdout:1892: expect deleted 2024-09-18T14:07:47.243 INFO:tasks.rados.rados.0.smithi084.stdout:1893: delete oid 41 current snap is 198 2024-09-18T14:07:47.249 INFO:tasks.rados.rados.0.smithi084.stdout:1890: expect (ObjNum 634 snap 189 seq_num 634) 2024-09-18T14:07:47.395 INFO:tasks.rados.rados.0.smithi084.stdout:1890: done (2 left) 2024-09-18T14:07:47.395 INFO:tasks.rados.rados.0.smithi084.stdout:1892: done (1 left) 2024-09-18T14:07:47.395 INFO:tasks.rados.rados.0.smithi084.stdout:1893: done (0 left) 2024-09-18T14:07:47.396 INFO:tasks.rados.rados.0.smithi084.stdout:1894: write oid 41 current snap is 198 2024-09-18T14:07:47.396 INFO:tasks.rados.rados.0.smithi084.stdout:1894: seq_num 665 ranges {776655=563296,2047702=620235,3095349=81306} 2024-09-18T14:07:47.402 INFO:tasks.rados.rados.0.smithi084.stdout:1894: writing smithi08434663-41 from 776655 to 1339951 tid 1 2024-09-18T14:07:47.409 INFO:tasks.rados.rados.0.smithi084.stdout:1894: writing smithi08434663-41 from 2047702 to 2667937 tid 2 2024-09-18T14:07:47.412 INFO:tasks.rados.rados.0.smithi084.stdout:1894: writing smithi08434663-41 from 3095349 to 3176655 tid 3 2024-09-18T14:07:47.412 INFO:tasks.rados.rados.0.smithi084.stdout:1895: write oid 7 current snap is 198 2024-09-18T14:07:47.412 INFO:tasks.rados.rados.0.smithi084.stdout:1895: seq_num 666 ranges {793462=638545,2106184=763893} 2024-09-18T14:07:47.420 INFO:tasks.rados.rados.0.smithi084.stdout:1895: writing smithi08434663-7 from 793462 to 1432007 tid 1 2024-09-18T14:07:47.427 INFO:tasks.rados.rados.0.smithi084.stdout:1895: writing smithi08434663-7 from 2106184 to 2870077 tid 2 2024-09-18T14:07:47.428 INFO:tasks.rados.rados.0.smithi084.stdout:1896: snap_create 2024-09-18T14:07:47.428 INFO:tasks.rados.rados.0.smithi084.stdout:1895: finishing write tid 1 to smithi08434663-7 2024-09-18T14:07:47.431 INFO:tasks.rados.rados.0.smithi084.stdout:1895: finishing write tid 2 to smithi08434663-7 2024-09-18T14:07:47.431 INFO:tasks.rados.rados.0.smithi084.stdout:1895: finishing write tid 3 to smithi08434663-7 2024-09-18T14:07:47.431 INFO:tasks.rados.rados.0.smithi084.stdout:1895: finishing write tid 4 to smithi08434663-7 2024-09-18T14:07:47.432 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 603 (ObjNum 666 snap 198 seq_num 666) dirty exists 2024-09-18T14:07:47.432 INFO:tasks.rados.rados.0.smithi084.stdout:1895: left oid 7 (ObjNum 666 snap 198 seq_num 666) 2024-09-18T14:07:47.609 INFO:tasks.rados.rados.0.smithi084.stdout:1894: finishing write tid 1 to smithi08434663-41 2024-09-18T14:07:47.610 INFO:tasks.rados.rados.0.smithi084.stdout:1894: finishing write tid 2 to smithi08434663-41 2024-09-18T14:07:47.610 INFO:tasks.rados.rados.0.smithi084.stdout:1894: finishing write tid 3 to smithi08434663-41 2024-09-18T14:07:47.610 INFO:tasks.rados.rados.0.smithi084.stdout:1894: finishing write tid 4 to smithi08434663-41 2024-09-18T14:07:47.610 INFO:tasks.rados.rados.0.smithi084.stdout:1894: finishing write tid 5 to smithi08434663-41 2024-09-18T14:07:47.610 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 868 (ObjNum 665 snap 198 seq_num 665) dirty exists 2024-09-18T14:07:47.610 INFO:tasks.rados.rados.0.smithi084.stdout:1894: left oid 41 (ObjNum 665 snap 198 seq_num 665) 2024-09-18T14:07:48.307 INFO:tasks.rados.rados.0.smithi084.stdout:1894: done (2 left) 2024-09-18T14:07:48.307 INFO:tasks.rados.rados.0.smithi084.stdout:1895: done (1 left) 2024-09-18T14:07:48.307 INFO:tasks.rados.rados.0.smithi084.stdout:1896: done (0 left) 2024-09-18T14:07:48.307 INFO:tasks.rados.rados.0.smithi084.stdout:1897: read oid 27 snap -1 2024-09-18T14:07:48.307 INFO:tasks.rados.rados.0.smithi084.stdout:1897: expect (ObjNum 594 snap 179 seq_num 594) 2024-09-18T14:07:48.307 INFO:tasks.rados.rados.0.smithi084.stdout:1898: write (excl) oid 30 current snap is 199 2024-09-18T14:07:48.307 INFO:tasks.rados.rados.0.smithi084.stdout:1898: seq_num 667 ranges {410269=713794,1648313=507551,2917489=292780} 2024-09-18T14:07:48.313 INFO:tasks.rados.rados.0.smithi084.stdout:1898: writing smithi08434663-30 from 410269 to 1124063 tid 1 2024-09-18T14:07:48.318 INFO:tasks.rados.rados.0.smithi084.stdout:1898: writing smithi08434663-30 from 1648313 to 2155864 tid 2 2024-09-18T14:07:48.325 INFO:tasks.rados.rados.0.smithi084.stdout:1898: writing smithi08434663-30 from 2917489 to 3210269 tid 3 2024-09-18T14:07:48.325 INFO:tasks.rados.rados.0.smithi084.stdout:1899: write oid 45 current snap is 199 2024-09-18T14:07:48.325 INFO:tasks.rados.rados.0.smithi084.stdout:1899: seq_num 668 ranges {427076=789043,1706795=767562,2944912=282164} 2024-09-18T14:07:48.331 INFO:tasks.rados.rados.0.smithi084.stdout:1899: writing smithi08434663-45 from 427076 to 1216119 tid 1 2024-09-18T14:07:48.338 INFO:tasks.rados.rados.0.smithi084.stdout:1899: writing smithi08434663-45 from 1706795 to 2474357 tid 2 2024-09-18T14:07:48.342 INFO:tasks.rados.rados.0.smithi084.stdout:1899: writing smithi08434663-45 from 2944912 to 3227076 tid 3 2024-09-18T14:07:48.342 INFO:tasks.rados.rados.0.smithi084.stdout:1900: copy_from oid 13 from oid 22 current snap is 199 2024-09-18T14:07:48.342 INFO:tasks.rados.rados.0.smithi084.stdout:1897: expect (ObjNum 594 snap 179 seq_num 594) 2024-09-18T14:07:48.469 INFO:tasks.rados.rados.0.smithi084.stdout:1898: finishing write tid 1 to smithi08434663-30 2024-09-18T14:07:48.470 INFO:tasks.rados.rados.0.smithi084.stdout:1898: finishing write tid 2 to smithi08434663-30 2024-09-18T14:07:48.470 INFO:tasks.rados.rados.0.smithi084.stdout:1898: finishing write tid 3 to smithi08434663-30 2024-09-18T14:07:48.470 INFO:tasks.rados.rados.0.smithi084.stdout:1898: finishing write tid 4 to smithi08434663-30 2024-09-18T14:07:48.470 INFO:tasks.rados.rados.0.smithi084.stdout:1898: finishing write tid 5 to smithi08434663-30 2024-09-18T14:07:48.470 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 563 (ObjNum 667 snap 199 seq_num 667) dirty exists 2024-09-18T14:07:48.470 INFO:tasks.rados.rados.0.smithi084.stdout:1898: left oid 30 (ObjNum 667 snap 199 seq_num 667) 2024-09-18T14:07:48.470 INFO:tasks.rados.rados.0.smithi084.stdout:1899: finishing write tid 1 to smithi08434663-45 2024-09-18T14:07:48.470 INFO:tasks.rados.rados.0.smithi084.stdout:1897: done (3 left) 2024-09-18T14:07:48.470 INFO:tasks.rados.rados.0.smithi084.stdout:1898: done (2 left) 2024-09-18T14:07:48.470 INFO:tasks.rados.rados.0.smithi084.stdout:1901: delete oid 48 current snap is 199 2024-09-18T14:07:48.470 INFO:tasks.rados.rados.0.smithi084.stdout:1899: finishing write tid 2 to smithi08434663-45 2024-09-18T14:07:48.471 INFO:tasks.rados.rados.0.smithi084.stdout:1899: finishing write tid 3 to smithi08434663-45 2024-09-18T14:07:48.471 INFO:tasks.rados.rados.0.smithi084.stdout:1899: finishing write tid 4 to smithi08434663-45 2024-09-18T14:07:48.471 INFO:tasks.rados.rados.0.smithi084.stdout:1899: finishing write tid 5 to smithi08434663-45 2024-09-18T14:07:48.471 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 873 (ObjNum 668 snap 199 seq_num 668) dirty exists 2024-09-18T14:07:48.471 INFO:tasks.rados.rados.0.smithi084.stdout:1899: left oid 45 (ObjNum 668 snap 199 seq_num 668) 2024-09-18T14:07:48.471 INFO:tasks.rados.rados.0.smithi084.stdout:1900: finishing copy_from racing read to smithi08434663-13 2024-09-18T14:07:48.471 INFO:tasks.rados.rados.0.smithi084.stdout:1900: finishing copy_from to smithi08434663-13 2024-09-18T14:07:48.471 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 729 (ObjNum 615 snap 183 seq_num 615) dirty exists 2024-09-18T14:07:48.472 INFO:tasks.rados.rados.0.smithi084.stdout:1899: done (2 left) 2024-09-18T14:07:48.472 INFO:tasks.rados.rados.0.smithi084.stdout:1900: done (1 left) 2024-09-18T14:07:48.472 INFO:tasks.rados.rados.0.smithi084.stdout:1901: done (0 left) 2024-09-18T14:07:48.472 INFO:tasks.rados.rados.0.smithi084.stdout:1902: write oid 21 current snap is 199 2024-09-18T14:07:48.472 INFO:tasks.rados.rados.0.smithi084.stdout:1902: seq_num 670 ranges {460690=655894,1540112=771231,2716111=539778} 2024-09-18T14:07:48.478 INFO:tasks.rados.rados.0.smithi084.stdout:1902: writing smithi08434663-21 from 460690 to 1116584 tid 1 2024-09-18T14:07:48.484 INFO:tasks.rados.rados.0.smithi084.stdout:1902: writing smithi08434663-21 from 1540112 to 2311343 tid 2 2024-09-18T14:07:48.490 INFO:tasks.rados.rados.0.smithi084.stdout:1902: writing smithi08434663-21 from 2716111 to 3255889 tid 3 2024-09-18T14:07:48.490 INFO:tasks.rados.rados.0.smithi084.stdout:1903: copy_from oid 45 from oid 35 current snap is 199 2024-09-18T14:07:48.491 INFO:tasks.rados.rados.0.smithi084.stdout:1902: finishing write tid 1 to smithi08434663-21 2024-09-18T14:07:48.491 INFO:tasks.rados.rados.0.smithi084.stdout:1902: finishing write tid 2 to smithi08434663-21 2024-09-18T14:07:48.491 INFO:tasks.rados.rados.0.smithi084.stdout:1904: copy_from oid 27 from oid 3 current snap is 199 2024-09-18T14:07:48.491 INFO:tasks.rados.rados.0.smithi084.stdout:1905: snap_create 2024-09-18T14:07:48.493 INFO:tasks.rados.rados.0.smithi084.stdout:1902: finishing write tid 3 to smithi08434663-21 2024-09-18T14:07:48.493 INFO:tasks.rados.rados.0.smithi084.stdout:1902: finishing write tid 4 to smithi08434663-21 2024-09-18T14:07:48.493 INFO:tasks.rados.rados.0.smithi084.stdout:1902: finishing write tid 5 to smithi08434663-21 2024-09-18T14:07:48.493 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 595 (ObjNum 670 snap 199 seq_num 670) dirty exists 2024-09-18T14:07:48.493 INFO:tasks.rados.rados.0.smithi084.stdout:1902: left oid 21 (ObjNum 670 snap 199 seq_num 670) 2024-09-18T14:07:48.502 INFO:tasks.rados.rados.0.smithi084.stdout:1903: finishing copy_from racing read to smithi08434663-45 2024-09-18T14:07:48.502 INFO:tasks.rados.rados.0.smithi084.stdout:1904: finishing copy_from racing read to smithi08434663-27 2024-09-18T14:07:48.503 INFO:tasks.rados.rados.0.smithi084.stdout:1903: finishing copy_from to smithi08434663-45 2024-09-18T14:07:48.503 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 850 (ObjNum 652 snap 191 seq_num 652) dirty exists 2024-09-18T14:07:48.505 INFO:tasks.rados.rados.0.smithi084.stdout:1904: finishing copy_from to smithi08434663-27 2024-09-18T14:07:48.505 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 732 (ObjNum 662 snap 196 seq_num 662) dirty exists 2024-09-18T14:07:49.316 INFO:tasks.rados.rados.0.smithi084.stdout:1902: done (3 left) 2024-09-18T14:07:49.316 INFO:tasks.rados.rados.0.smithi084.stdout:1903: done (2 left) 2024-09-18T14:07:49.316 INFO:tasks.rados.rados.0.smithi084.stdout:1904: done (1 left) 2024-09-18T14:07:49.317 INFO:tasks.rados.rados.0.smithi084.stdout:1905: done (0 left) 2024-09-18T14:07:49.317 INFO:tasks.rados.rados.0.smithi084.stdout:1906: write oid 7 current snap is 200 2024-09-18T14:07:49.317 INFO:tasks.rados.rados.0.smithi084.stdout:1906: seq_num 671 ranges {477497=731143,1998594=514889,3143534=133963} 2024-09-18T14:07:49.322 INFO:tasks.rados.rados.0.smithi084.stdout:1906: writing smithi08434663-7 from 477497 to 1208640 tid 1 2024-09-18T14:07:49.329 INFO:tasks.rados.rados.0.smithi084.stdout:1906: writing smithi08434663-7 from 1998594 to 2513483 tid 2 2024-09-18T14:07:49.334 INFO:tasks.rados.rados.0.smithi084.stdout:1906: writing smithi08434663-7 from 3143534 to 3277497 tid 3 2024-09-18T14:07:49.334 INFO:tasks.rados.rados.0.smithi084.stdout:1907: read oid 26 snap -1 2024-09-18T14:07:49.334 INFO:tasks.rados.rados.0.smithi084.stdout:1907: expect (ObjNum 629 snap 189 seq_num 629) 2024-09-18T14:07:49.334 INFO:tasks.rados.rados.0.smithi084.stdout:1908: read oid 43 snap 197 2024-09-18T14:07:49.334 INFO:tasks.rados.rados.0.smithi084.stdout:1908: expect deleted 2024-09-18T14:07:49.334 INFO:tasks.rados.rados.0.smithi084.stdout:1909: snap_remove snap 199 2024-09-18T14:07:50.324 INFO:tasks.rados.rados.0.smithi084.stdout:1909: done (3 left) 2024-09-18T14:07:50.325 INFO:tasks.rados.rados.0.smithi084.stdout:1910: snap_remove snap 198 2024-09-18T14:07:51.330 INFO:tasks.rados.rados.0.smithi084.stdout:1910: done (3 left) 2024-09-18T14:07:51.330 INFO:tasks.rados.rados.0.smithi084.stdout:1911: write (excl) oid 25 current snap is 200 2024-09-18T14:07:51.330 INFO:tasks.rados.rados.0.smithi084.stdout:1906: finishing write tid 1 to smithi08434663-7 2024-09-18T14:07:51.330 INFO:tasks.rados.rados.0.smithi084.stdout:1911: seq_num 672 ranges {494304=406392,1540723=774900,2770957=523347} 2024-09-18T14:07:51.335 INFO:tasks.rados.rados.0.smithi084.stdout:1911: writing smithi08434663-25 from 494304 to 900696 tid 1 2024-09-18T14:07:51.342 INFO:tasks.rados.rados.0.smithi084.stdout:1911: writing smithi08434663-25 from 1540723 to 2315623 tid 2 2024-09-18T14:07:51.348 INFO:tasks.rados.rados.0.smithi084.stdout:1911: writing smithi08434663-25 from 2770957 to 3294304 tid 3 2024-09-18T14:07:51.348 INFO:tasks.rados.rados.0.smithi084.stdout:1912: write (excl) oid 44 current snap is 200 2024-09-18T14:07:51.348 INFO:tasks.rados.rados.0.smithi084.stdout:1912: seq_num 673 ranges {511111=481641,1599205=518558,2682027=489947} 2024-09-18T14:07:51.353 INFO:tasks.rados.rados.0.smithi084.stdout:1912: writing smithi08434663-44 from 511111 to 992752 tid 1 2024-09-18T14:07:51.360 INFO:tasks.rados.rados.0.smithi084.stdout:1912: writing smithi08434663-44 from 1599205 to 2117763 tid 2 2024-09-18T14:07:51.366 INFO:tasks.rados.rados.0.smithi084.stdout:1912: writing smithi08434663-44 from 2682027 to 3171974 tid 3 2024-09-18T14:07:51.367 INFO:tasks.rados.rados.0.smithi084.stdout:1913: read oid 47 snap -1 2024-09-18T14:07:51.367 INFO:tasks.rados.rados.0.smithi084.stdout:1913: expect (ObjNum 587 snap 177 seq_num 587) 2024-09-18T14:07:51.367 INFO:tasks.rados.rados.0.smithi084.stdout:1914: copy_from oid 37 from oid 5 current snap is 200 2024-09-18T14:07:51.367 INFO:tasks.rados.rados.0.smithi084.stdout:1906: finishing write tid 2 to smithi08434663-7 2024-09-18T14:07:51.367 INFO:tasks.rados.rados.0.smithi084.stdout:1906: finishing write tid 3 to smithi08434663-7 2024-09-18T14:07:51.367 INFO:tasks.rados.rados.0.smithi084.stdout:1906: finishing write tid 4 to smithi08434663-7 2024-09-18T14:07:51.367 INFO:tasks.rados.rados.0.smithi084.stdout:1906: finishing write tid 5 to smithi08434663-7 2024-09-18T14:07:51.367 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 607 (ObjNum 671 snap 200 seq_num 671) dirty exists 2024-09-18T14:07:51.367 INFO:tasks.rados.rados.0.smithi084.stdout:1906: left oid 7 (ObjNum 671 snap 200 seq_num 671) 2024-09-18T14:07:51.367 INFO:tasks.rados.rados.0.smithi084.stdout:1907: expect (ObjNum 629 snap 189 seq_num 629) 2024-09-18T14:07:51.450 INFO:tasks.rados.rados.0.smithi084.stdout:1911: finishing write tid 1 to smithi08434663-25 2024-09-18T14:07:51.451 INFO:tasks.rados.rados.0.smithi084.stdout:1911: finishing write tid 2 to smithi08434663-25 2024-09-18T14:07:51.451 INFO:tasks.rados.rados.0.smithi084.stdout:1911: finishing write tid 3 to smithi08434663-25 2024-09-18T14:07:51.451 INFO:tasks.rados.rados.0.smithi084.stdout:1911: finishing write tid 4 to smithi08434663-25 2024-09-18T14:07:51.451 INFO:tasks.rados.rados.0.smithi084.stdout:1911: finishing write tid 5 to smithi08434663-25 2024-09-18T14:07:51.451 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 599 (ObjNum 672 snap 200 seq_num 672) dirty exists 2024-09-18T14:07:51.451 INFO:tasks.rados.rados.0.smithi084.stdout:1911: left oid 25 (ObjNum 672 snap 200 seq_num 672) 2024-09-18T14:07:51.451 INFO:tasks.rados.rados.0.smithi084.stdout:1912: finishing write tid 1 to smithi08434663-44 2024-09-18T14:07:51.452 INFO:tasks.rados.rados.0.smithi084.stdout:1906: done (6 left) 2024-09-18T14:07:51.452 INFO:tasks.rados.rados.0.smithi084.stdout:1907: done (5 left) 2024-09-18T14:07:51.452 INFO:tasks.rados.rados.0.smithi084.stdout:1908: done (4 left) 2024-09-18T14:07:51.452 INFO:tasks.rados.rados.0.smithi084.stdout:1911: done (3 left) 2024-09-18T14:07:51.452 INFO:tasks.rados.rados.0.smithi084.stdout:1915: write (excl) oid 11 current snap is 200 2024-09-18T14:07:51.452 INFO:tasks.rados.rados.0.smithi084.stdout:1912: finishing write tid 2 to smithi08434663-44 2024-09-18T14:07:51.452 INFO:tasks.rados.rados.0.smithi084.stdout:1914: finishing copy_from to smithi08434663-37 2024-09-18T14:07:51.452 INFO:tasks.rados.rados.0.smithi084.stdout:1914: got expected ENOENT (src dne) 2024-09-18T14:07:51.452 INFO:tasks.rados.rados.0.smithi084.stdout:1914: finishing copy_from racing read to smithi08434663-37 2024-09-18T14:07:51.452 INFO:tasks.rados.rados.0.smithi084.stdout:1915: seq_num 674 ranges {527918=556890,1657687=778569,3225803=102115} 2024-09-18T14:07:51.456 INFO:tasks.rados.rados.0.smithi084.stdout:1915: writing smithi08434663-11 from 527918 to 1084808 tid 1 2024-09-18T14:07:51.463 INFO:tasks.rados.rados.0.smithi084.stdout:1915: writing smithi08434663-11 from 1657687 to 2436256 tid 2 2024-09-18T14:07:51.469 INFO:tasks.rados.rados.0.smithi084.stdout:1915: writing smithi08434663-11 from 3225803 to 3327918 tid 3 2024-09-18T14:07:51.469 INFO:tasks.rados.rados.0.smithi084.stdout:1914: done (3 left) 2024-09-18T14:07:51.469 INFO:tasks.rados.rados.0.smithi084.stdout:1916: rollback oid 17 current snap is 200 2024-09-18T14:07:51.469 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 17 to 197 2024-09-18T14:07:51.469 INFO:tasks.rados.rados.0.smithi084.stdout:1912: finishing write tid 3 to smithi08434663-44 2024-09-18T14:07:51.470 INFO:tasks.rados.rados.0.smithi084.stdout:1912: finishing write tid 4 to smithi08434663-44 2024-09-18T14:07:51.470 INFO:tasks.rados.rados.0.smithi084.stdout:1912: finishing write tid 5 to smithi08434663-44 2024-09-18T14:07:51.470 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 314 (ObjNum 673 snap 200 seq_num 673) dirty exists 2024-09-18T14:07:51.470 INFO:tasks.rados.rados.0.smithi084.stdout:1912: left oid 44 (ObjNum 673 snap 200 seq_num 673) 2024-09-18T14:07:51.470 INFO:tasks.rados.rados.0.smithi084.stdout:1913: expect (ObjNum 587 snap 177 seq_num 587) 2024-09-18T14:07:51.592 INFO:tasks.rados.rados.0.smithi084.stdout:1915: finishing write tid 1 to smithi08434663-11 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:1915: finishing write tid 2 to smithi08434663-11 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:1912: done (3 left) 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:1913: done (2 left) 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:1917: snap_create 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:1915: finishing write tid 3 to smithi08434663-11 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:1915: finishing write tid 4 to smithi08434663-11 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:1915: finishing write tid 5 to smithi08434663-11 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 595 (ObjNum 674 snap 200 seq_num 674) dirty exists 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:1915: left oid 11 (ObjNum 674 snap 200 seq_num 674) 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:1916: finishing rollback tid 0 to smithi08434663-17 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:1916: finishing rollback tid 1 to smithi08434663-17 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:1916: finishing rollback tid 2 to smithi08434663-17 2024-09-18T14:07:51.593 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 851 (ObjNum 638 snap 189 seq_num 638) dirty exists 2024-09-18T14:07:52.235 INFO:tasks.rados.rados.0.smithi084.stdout:1915: done (2 left) 2024-09-18T14:07:52.236 INFO:tasks.rados.rados.0.smithi084.stdout:1916: done (1 left) 2024-09-18T14:07:52.236 INFO:tasks.rados.rados.0.smithi084.stdout:1917: done (0 left) 2024-09-18T14:07:52.236 INFO:tasks.rados.rados.0.smithi084.stdout:1918: delete oid 12 current snap is 201 2024-09-18T14:07:52.240 INFO:tasks.rados.rados.0.smithi084.stdout:1918: done (0 left) 2024-09-18T14:07:52.240 INFO:tasks.rados.rados.0.smithi084.stdout:1919: read oid 11 snap -1 2024-09-18T14:07:52.241 INFO:tasks.rados.rados.0.smithi084.stdout:1919: expect (ObjNum 674 snap 200 seq_num 674) 2024-09-18T14:07:52.241 INFO:tasks.rados.rados.0.smithi084.stdout:1920: copy_from oid 44 from oid 15 current snap is 201 2024-09-18T14:07:52.241 INFO:tasks.rados.rados.0.smithi084.stdout:1921: copy_from oid 7 from oid 24 current snap is 201 2024-09-18T14:07:52.241 INFO:tasks.rados.rados.0.smithi084.stdout:1922: rollback oid 49 current snap is 201 2024-09-18T14:07:52.241 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 49 to 197 2024-09-18T14:07:52.241 INFO:tasks.rados.rados.0.smithi084.stdout:1923: read oid 32 snap -1 2024-09-18T14:07:52.241 INFO:tasks.rados.rados.0.smithi084.stdout:1923: expect deleted 2024-09-18T14:07:52.241 INFO:tasks.rados.rados.0.smithi084.stdout:1924: copy_from oid 1 from oid 6 current snap is 201 2024-09-18T14:07:52.241 INFO:tasks.rados.rados.0.smithi084.stdout:1925: rollback oid 43 current snap is 201 2024-09-18T14:07:52.241 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 200 2024-09-18T14:07:52.241 INFO:tasks.rados.rados.0.smithi084.stdout:1926: read oid 48 snap -1 2024-09-18T14:07:52.241 INFO:tasks.rados.rados.0.smithi084.stdout:1926: expect deleted 2024-09-18T14:07:52.242 INFO:tasks.rados.rados.0.smithi084.stdout:1927: read oid 3 snap 197 2024-09-18T14:07:52.242 INFO:tasks.rados.rados.0.smithi084.stdout:1927: expect (ObjNum 662 snap 196 seq_num 662) 2024-09-18T14:07:52.242 INFO:tasks.rados.rados.0.smithi084.stdout:1928: write (excl) oid 36 current snap is 201 2024-09-18T14:07:52.242 INFO:tasks.rados.rados.0.smithi084.stdout:1928: seq_num 676 ranges {561532=707388,2058298=782238} 2024-09-18T14:07:52.248 INFO:tasks.rados.rados.0.smithi084.stdout:1928: writing smithi08434663-36 from 561532 to 1268920 tid 1 2024-09-18T14:07:52.257 INFO:tasks.rados.rados.0.smithi084.stdout:1928: writing smithi08434663-36 from 2058298 to 2840536 tid 2 2024-09-18T14:07:52.257 INFO:tasks.rados.rados.0.smithi084.stdout:1929: write oid 10 current snap is 201 2024-09-18T14:07:52.257 INFO:tasks.rados.rados.0.smithi084.stdout:1929: seq_num 677 ranges {578339=498990,1833133=525896,2791719=586620} 2024-09-18T14:07:52.263 INFO:tasks.rados.rados.0.smithi084.stdout:1929: writing smithi08434663-10 from 578339 to 1077329 tid 1 2024-09-18T14:07:52.270 INFO:tasks.rados.rados.0.smithi084.stdout:1929: writing smithi08434663-10 from 1833133 to 2359029 tid 2 2024-09-18T14:07:52.276 INFO:tasks.rados.rados.0.smithi084.stdout:1929: writing smithi08434663-10 from 2791719 to 3378339 tid 3 2024-09-18T14:07:52.276 INFO:tasks.rados.rados.0.smithi084.stdout:1930: read oid 41 snap -1 2024-09-18T14:07:52.276 INFO:tasks.rados.rados.0.smithi084.stdout:1930: expect (ObjNum 665 snap 198 seq_num 665) 2024-09-18T14:07:52.276 INFO:tasks.rados.rados.0.smithi084.stdout:1931: read oid 9 snap -1 2024-09-18T14:07:52.276 INFO:tasks.rados.rados.0.smithi084.stdout:1931: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:52.276 INFO:tasks.rados.rados.0.smithi084.stdout:1932: write oid 5 current snap is 201 2024-09-18T14:07:52.277 INFO:tasks.rados.rados.0.smithi084.stdout:1932: seq_num 678 ranges {595146=574239,1891615=669554,3219142=176004} 2024-09-18T14:07:52.282 INFO:tasks.rados.rados.0.smithi084.stdout:1932: writing smithi08434663-5 from 595146 to 1169385 tid 1 2024-09-18T14:07:52.290 INFO:tasks.rados.rados.0.smithi084.stdout:1932: writing smithi08434663-5 from 1891615 to 2561169 tid 2 2024-09-18T14:07:52.295 INFO:tasks.rados.rados.0.smithi084.stdout:1932: writing smithi08434663-5 from 3219142 to 3395146 tid 3 2024-09-18T14:07:52.295 INFO:tasks.rados.rados.0.smithi084.stdout:1933: write (excl) oid 34 current snap is 201 2024-09-18T14:07:52.295 INFO:tasks.rados.rados.0.smithi084.stdout:1933: seq_num 679 ranges {611953=649488,1950097=529565,3246565=165388} 2024-09-18T14:07:52.302 INFO:tasks.rados.rados.0.smithi084.stdout:1933: writing smithi08434663-34 from 611953 to 1261441 tid 1 2024-09-18T14:07:52.308 INFO:tasks.rados.rados.0.smithi084.stdout:1933: writing smithi08434663-34 from 1950097 to 2479662 tid 2 2024-09-18T14:07:52.314 INFO:tasks.rados.rados.0.smithi084.stdout:1933: writing smithi08434663-34 from 3246565 to 3411953 tid 3 2024-09-18T14:07:52.314 INFO:tasks.rados.rados.0.smithi084.stdout:1923: done (14 left) 2024-09-18T14:07:52.314 INFO:tasks.rados.rados.0.smithi084.stdout:1934: read oid 45 snap -1 2024-09-18T14:07:52.314 INFO:tasks.rados.rados.0.smithi084.stdout:1934: expect (ObjNum 652 snap 191 seq_num 652) 2024-09-18T14:07:52.314 INFO:tasks.rados.rados.0.smithi084.stdout:1935: snap_create 2024-09-18T14:07:52.314 INFO:tasks.rados.rados.0.smithi084.stdout:1925: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:07:52.314 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 0 (ObjNum 113 snap 0 seq_num 0) dirty dne 2024-09-18T14:07:52.314 INFO:tasks.rados.rados.0.smithi084.stdout:1922: finishing rollback tid 0 to smithi08434663-49 2024-09-18T14:07:52.314 INFO:tasks.rados.rados.0.smithi084.stdout:1919: expect (ObjNum 674 snap 200 seq_num 674) 2024-09-18T14:07:52.476 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-18T14:07:52.477 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-18T14:07:52.477 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T14:07:52.477 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T14:07:52.481 INFO:tasks.ceph.mon.c:Started 2024-09-18T14:07:52.481 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-18T14:07:52.481 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-18T14:07:52.481 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-18T14:07:52.481 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-18T14:07:52.483 INFO:tasks.ceph.mon.f:Started 2024-09-18T14:07:52.484 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:07:52.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:07:52.500 INFO:tasks.rados.rados.0.smithi084.stdout:1922: finishing rollback tid 1 to smithi08434663-49 2024-09-18T14:07:52.500 INFO:tasks.rados.rados.0.smithi084.stdout:1922: finishing rollback tid 2 to smithi08434663-49 2024-09-18T14:07:52.500 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 653 (ObjNum 649 snap 191 seq_num 649) dirty exists 2024-09-18T14:07:52.500 INFO:tasks.rados.rados.0.smithi084.stdout:1920: finishing copy_from racing read to smithi08434663-44 2024-09-18T14:07:52.501 INFO:tasks.rados.rados.0.smithi084.stdout:1927: expect (ObjNum 662 snap 196 seq_num 662) 2024-09-18T14:07:52.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.582+0000 7ff01891d640 10 monclient: get_monmap_and_config 2024-09-18T14:07:52.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.582+0000 7ff01891d640 10 monclient: build_initial_monmap 2024-09-18T14:07:52.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.582+0000 7ff01891d640 10 monclient: monmap: 2024-09-18T14:07:52.593 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:07:52.593 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:07:52.593 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:07:52.583270+0000 2024-09-18T14:07:52.593 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:07:52.583270+0000 2024-09-18T14:07:52.593 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:07:52.593 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:07:52.593 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:07:52.593 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:07:52.593 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:07:52.593 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:07:52.593 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.582+0000 7ff01891d640 1 Processor -- start 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.582+0000 7ff01891d640 1 -- start start 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.582+0000 7ff01891d640 10 monclient: init 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff01891d640 10 monclient: _reopen_session rank -1 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff01891d640 10 monclient: _add_conns ranks=[0,3,1,7,8,5,4,2,6] 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff01891d640 10 monclient(hunting): picked mon.noname-a con 0x7ff0140a41c0 addr v1:172.21.15.45:6789/0 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff01891d640 10 monclient(hunting): picked mon.noname-g con 0x7ff01406e480 addr v1:172.21.15.84:6790/0 2024-09-18T14:07:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff01891d640 10 monclient(hunting): picked mon.noname-f con 0x7ff01406e850 addr v1:172.21.15.84:6789/0 2024-09-18T14:07:52.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff01891d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01405ed70 con 0x7ff0140a41c0 2024-09-18T14:07:52.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff01891d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01409ffd0 con 0x7ff01406e850 2024-09-18T14:07:52.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff01891d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff014133340 con 0x7ff01406e480 2024-09-18T14:07:52.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff01891d640 10 monclient(hunting): _renew_subs 2024-09-18T14:07:52.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff01891d640 10 monclient(hunting): authenticate will time out at 1287.921559s 2024-09-18T14:07:52.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff01259c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff0140a41c0 0x7ff014132470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52860/0 (socket says 172.21.15.45:52860) 2024-09-18T14:07:52.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff01259c640 1 -- 172.21.15.45:0/2282271155 learned_addr learned my addr 172.21.15.45:0/2282271155 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:07:52.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff011d9b640 1 -- 172.21.15.45:0/2282271155 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194821158 0 0) 0x7ff01405ed70 con 0x7ff0140a41c0 2024-09-18T14:07:52.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff011d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:07:52.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff011d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:07:52.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff011d9b640 10 monclient(hunting): my global_id is 5292 2024-09-18T14:07:52.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff011d9b640 1 -- 172.21.15.45:0/2282271155 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0000034e0 con 0x7ff0140a41c0 2024-09-18T14:07:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff011d9b640 1 -- 172.21.15.45:0/2282271155 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179150018 0 0) 0x7ff014133340 con 0x7ff01406e480 2024-09-18T14:07:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff011d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:07:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff011d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:07:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff011d9b640 10 monclient(hunting): my global_id is 154290 2024-09-18T14:07:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.583+0000 7ff011d9b640 1 -- 172.21.15.45:0/2282271155 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff01405ed70 con 0x7ff01406e480 2024-09-18T14:07:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 1 -- 172.21.15.45:0/2282271155 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4058336225 0 0) 0x7ff0000034e0 con 0x7ff0140a41c0 2024-09-18T14:07:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 1 -- 172.21.15.45:0/2282271155 >> v1:172.21.15.84:6790/0 conn(0x7ff01406e480 legacy=0x7ff0141377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:07:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 1 -- 172.21.15.45:0/2282271155 >> v1:172.21.15.84:6789/0 conn(0x7ff01406e850 legacy=0x7ff01413aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:07:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:07:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 1 monclient: found mon.noname-a 2024-09-18T14:07:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:07:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 1 -- 172.21.15.45:0/2282271155 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff014069030 con 0x7ff0140a41c0 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 10 monclient: _finish_auth 0 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 1 -- 172.21.15.45:0/2282271155 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff004001830 con 0x7ff0140a41c0 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff01891d640 5 monclient: authenticate success, global_id 5292 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff01891d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 10 monclient: dump: 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:07:52.597 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff01891d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 1 -- 172.21.15.45:0/2282271155 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff004001a10 con 0x7ff0140a41c0 2024-09-18T14:07:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff01891d640 10 monclient: get_monmap_and_config success 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 1 -- 172.21.15.45:0/2282271155 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff004004ba0 con 0x7ff0140a41c0 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff01891d640 4 set_mon_vals no callback set 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff011d9b640 10 monclient: dump: 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:07:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff01891d640 10 monclient: shutdown 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff01891d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff01891d640 1 -- 172.21.15.45:0/2282271155 >> v1:172.21.15.45:6789/0 conn(0x7ff0140a41c0 legacy=0x7ff014132470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff01891d640 1 -- 172.21.15.45:0/2282271155 shutdown_connections 2024-09-18T14:07:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff01891d640 1 -- 172.21.15.45:0/2282271155 >> 172.21.15.45:0/2282271155 conn(0x7ff0140636e0 msgr2=0x7ff0140a3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.584+0000 7ff01891d640 1 -- 172.21.15.45:0/2282271155 shutdown_connections 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.585+0000 7ff01891d640 1 -- 172.21.15.45:0/2282271155 wait complete. 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.585+0000 7ff01891d640 10 monclient: build_initial_monmap 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.585+0000 7ff01891d640 10 monclient: monmap: 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:07:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.585+0000 7ff01891d640 1 Processor -- start 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01891d640 1 -- start start 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01891d640 10 monclient: init 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01891d640 10 monclient: _reopen_session rank -1 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01891d640 10 monclient: _add_conns ranks=[8,6,7,3,5,2,4,0,1] 2024-09-18T14:07:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01891d640 10 monclient(hunting): picked mon.noname-e con 0x7ff01406e480 addr v1:172.21.15.45:6793/0 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01891d640 10 monclient(hunting): picked mon.noname-d con 0x7ff01406e850 addr v1:172.21.15.45:6792/0 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01891d640 10 monclient(hunting): picked mon.noname-i con 0x7ff0140a41c0 addr v1:172.21.15.84:6792/0 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01891d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff014068720 con 0x7ff01406e850 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01891d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01411e3f0 con 0x7ff0140a41c0 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01891d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0140683f0 con 0x7ff01406e480 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01891d640 10 monclient(hunting): _renew_subs 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01891d640 10 monclient(hunting): authenticate will time out at 1287.924505s 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01259c640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7ff01406e480 0x7ff0141312d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:44108/0 (socket says 172.21.15.45:44108) 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff01259c640 1 -- 172.21.15.45:0/259777587 learned_addr learned my addr 172.21.15.45:0/259777587 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.586+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707647572 0 0) 0x7ff01405ed70 con 0x7ff01406e850 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:07:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): my global_id is 5163 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feff40034e0 con 0x7ff01406e850 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215928540 0 0) 0x7ff0140683f0 con 0x7ff01406e480 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): my global_id is 5246 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feff4004860 con 0x7ff01406e480 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37200729 0 0) 0x7ff01411e3f0 con 0x7ff0140a41c0 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): my global_id is 5299 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0140683f0 con 0x7ff0140a41c0 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 133936579 0 0) 0x7feff4004860 con 0x7ff01406e480 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff01411e3f0 con 0x7ff01406e480 2024-09-18T14:07:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff0040018b0 con 0x7ff01406e480 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): dump: 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:07:52.605 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 962742291 0 0) 0x7ff0140683f0 con 0x7ff0140a41c0 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feff4004860 con 0x7ff0140a41c0 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3624884279 0 0) 0x7feff40034e0 con 0x7ff01406e850 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff0140683f0 con 0x7ff01406e850 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff00c002c40 con 0x7ff0140a41c0 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:07:52.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): dump: 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:07:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feffc001c30 con 0x7ff01406e850 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient(hunting): dump: 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:07:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 272869454 0 0) 0x7ff0140683f0 con 0x7ff01406e850 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 >> v1:172.21.15.45:6793/0 conn(0x7ff01406e480 legacy=0x7ff0141312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 >> v1:172.21.15.84:6792/0 conn(0x7ff0140a41c0 legacy=0x7ff014149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient: _finish_hunting 0 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 monclient: found mon.d 2024-09-18T14:07:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff014132070 con 0x7ff01406e850 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 10 monclient: _finish_auth 0 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff00affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff01891d640 5 monclient: authenticate success, global_id 5163 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff01891d640 10 monclient: _renew_subs 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff01891d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.587+0000 7ff01891d640 1 -- 172.21.15.45:0/259777587 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01411d490 con 0x7ff01406e850 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.588+0000 7ff01891d640 10 monclient: _renew_subs 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.588+0000 7ff01891d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.588+0000 7ff01891d640 1 -- 172.21.15.45:0/259777587 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff014136a50 con 0x7ff01406e850 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.589+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feffc001000 con 0x7ff01406e850 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.589+0000 7ff00affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.589+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feffc0024c0 con 0x7ff01406e850 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.589+0000 7ff00affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:07:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.589+0000 7ff00affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.589+0000 7ff01159a640 4 set_mon_vals no callback set 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.589+0000 7ff00affd640 10 monclient: dump: 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:07:52.611 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:07:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:07:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:07:52.612 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:07:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.589+0000 7ff01891d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:07:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.590+0000 7ff01891d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:07:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.590+0000 7ff01891d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:07:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.590+0000 7ff01891d640 1 -- 172.21.15.45:0/259777587 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff8005010 con 0x7ff01406e850 2024-09-18T14:07:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.590+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7feffc0026a0 con 0x7ff01406e850 2024-09-18T14:07:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.590+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(421..421 src has 126..421) v4 ==== 3943+0+0 (unknown 177230416 0 0) 0x7feffc043ef0 con 0x7ff01406e850 2024-09-18T14:07:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.590+0000 7ff01259c640 10 monclient: get_auth_request con 0x7feff403ab00 auth_method 0 2024-09-18T14:07:52.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.595+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7feffc002b90 con 0x7ff01406e850 2024-09-18T14:07:52.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.595+0000 7ff00affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:07:52.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.595+0000 7ff00affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:07:52.613 INFO:tasks.rados.rados.0.smithi084.stdout:1924: finishing copy_from racing read to smithi08434663-1 2024-09-18T14:07:52.613 INFO:tasks.rados.rados.0.smithi084.stdout:1921: finishing copy_from racing read to smithi08434663-7 2024-09-18T14:07:52.613 INFO:tasks.rados.rados.0.smithi084.stdout:1920: finishing copy_from to smithi08434663-44 2024-09-18T14:07:52.613 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 846 (ObjNum 634 snap 189 seq_num 634) dirty exists 2024-09-18T14:07:52.613 INFO:tasks.rados.rados.0.smithi084.stdout:1921: finishing copy_from to smithi08434663-7 2024-09-18T14:07:52.613 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 559 (ObjNum 550 snap 168 seq_num 550) dirty exists 2024-09-18T14:07:52.614 INFO:tasks.rados.rados.0.smithi084.stdout:1924: finishing copy_from to smithi08434663-1 2024-09-18T14:07:52.614 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 575 (ObjNum 538 snap 167 seq_num 538) dirty exists 2024-09-18T14:07:52.614 INFO:tasks.rados.rados.0.smithi084.stdout:1928: finishing write tid 1 to smithi08434663-36 2024-09-18T14:07:52.614 INFO:tasks.rados.rados.0.smithi084.stdout:1928: finishing write tid 2 to smithi08434663-36 2024-09-18T14:07:52.614 INFO:tasks.rados.rados.0.smithi084.stdout:1928: finishing write tid 3 to smithi08434663-36 2024-09-18T14:07:52.614 INFO:tasks.rados.rados.0.smithi084.stdout:1928: finishing write tid 4 to smithi08434663-36 2024-09-18T14:07:52.614 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 656 (ObjNum 676 snap 201 seq_num 676) dirty exists 2024-09-18T14:07:52.614 INFO:tasks.rados.rados.0.smithi084.stdout:1928: left oid 36 (ObjNum 676 snap 201 seq_num 676) 2024-09-18T14:07:52.614 INFO:tasks.rados.rados.0.smithi084.stdout:1929: finishing write tid 1 to smithi08434663-10 2024-09-18T14:07:52.614 INFO:tasks.rados.rados.0.smithi084.stdout:1929: finishing write tid 2 to smithi08434663-10 2024-09-18T14:07:52.614 INFO:tasks.rados.rados.0.smithi084.stdout:1931: expect (ObjNum 534 snap 165 seq_num 534) 2024-09-18T14:07:52.771 INFO:tasks.rados.rados.0.smithi084.stdout:1929: finishing write tid 3 to smithi08434663-10 2024-09-18T14:07:52.771 INFO:tasks.rados.rados.0.smithi084.stdout:1929: finishing write tid 4 to smithi08434663-10 2024-09-18T14:07:52.771 INFO:tasks.rados.rados.0.smithi084.stdout:1929: finishing write tid 5 to smithi08434663-10 2024-09-18T14:07:52.771 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 780 (ObjNum 677 snap 201 seq_num 677) dirty exists 2024-09-18T14:07:52.771 INFO:tasks.rados.rados.0.smithi084.stdout:1929: left oid 10 (ObjNum 677 snap 201 seq_num 677) 2024-09-18T14:07:52.771 INFO:tasks.rados.rados.0.smithi084.stdout:1930: expect (ObjNum 665 snap 198 seq_num 665) 2024-09-18T14:07:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.822+0000 7ff01891d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:07:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.822+0000 7ff01891d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:07:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.822+0000 7ff01891d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:07:52.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:52.822+0000 7ff01891d640 1 -- 172.21.15.45:0/259777587 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7feff8005840 con 0x7ff01406e850 2024-09-18T14:07:52.877 INFO:tasks.rados.rados.0.smithi084.stdout:1932: finishing write tid 1 to smithi08434663-5 2024-09-18T14:07:52.877 INFO:tasks.rados.rados.0.smithi084.stdout:1932: finishing write tid 2 to smithi08434663-5 2024-09-18T14:07:52.877 INFO:tasks.rados.rados.0.smithi084.stdout:1932: finishing write tid 3 to smithi08434663-5 2024-09-18T14:07:52.877 INFO:tasks.rados.rados.0.smithi084.stdout:1932: finishing write tid 4 to smithi08434663-5 2024-09-18T14:07:52.877 INFO:tasks.rados.rados.0.smithi084.stdout:1932: finishing write tid 5 to smithi08434663-5 2024-09-18T14:07:52.877 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 850 (ObjNum 678 snap 201 seq_num 678) dirty exists 2024-09-18T14:07:52.877 INFO:tasks.rados.rados.0.smithi084.stdout:1932: left oid 5 (ObjNum 678 snap 201 seq_num 678) 2024-09-18T14:07:52.878 INFO:tasks.rados.rados.0.smithi084.stdout:1933: finishing write tid 1 to smithi08434663-34 2024-09-18T14:07:52.878 INFO:tasks.rados.rados.0.smithi084.stdout:1933: finishing write tid 2 to smithi08434663-34 2024-09-18T14:07:52.878 INFO:tasks.rados.rados.0.smithi084.stdout:1933: finishing write tid 3 to smithi08434663-34 2024-09-18T14:07:52.878 INFO:tasks.rados.rados.0.smithi084.stdout:1933: finishing write tid 4 to smithi08434663-34 2024-09-18T14:07:52.878 INFO:tasks.rados.rados.0.smithi084.stdout:1933: finishing write tid 5 to smithi08434663-34 2024-09-18T14:07:52.878 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 881 (ObjNum 679 snap 201 seq_num 679) dirty exists 2024-09-18T14:07:52.878 INFO:tasks.rados.rados.0.smithi084.stdout:1933: left oid 34 (ObjNum 679 snap 201 seq_num 679) 2024-09-18T14:07:52.878 INFO:tasks.rados.rados.0.smithi084.stdout:1934: expect (ObjNum 652 snap 191 seq_num 652) 2024-09-18T14:07:55.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:55.586+0000 7ff00959a640 10 monclient: tick 2024-09-18T14:07:55.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:55.586+0000 7ff00959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:07:55.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:55.586+0000 7ff00959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:07:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:56.585+0000 7ff00959a640 10 monclient: tick 2024-09-18T14:07:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:56.585+0000 7ff00959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:07:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.586+0000 7ff00959a640 10 monclient: tick 2024-09-18T14:07:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.586+0000 7ff00959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:07:57.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.589+0000 7ff00affd640 1 -- 172.21.15.45:0/259777587 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2682 (unknown 254746657 0 704200466) 0x7feffc0121b0 con 0x7ff01406e850 2024-09-18T14:07:57.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.589+0000 7ff00affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:07:57.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.589+0000 7ff00affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:07:57.590 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":82,"quorum":[0,1,2,3,5,6,7,8],"quorum_names":["a","f","b","g","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:07:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.591+0000 7ff01891d640 1 -- 172.21.15.45:0/259777587 >> v1:172.21.15.84:6812/31478 conn(0x7feff403ab00 legacy=0x7feff403cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:07:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.591+0000 7ff01891d640 10 monclient: shutdown 2024-09-18T14:07:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.591+0000 7ff01891d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:07:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.591+0000 7ff01891d640 1 -- 172.21.15.45:0/259777587 >> v1:172.21.15.45:6792/0 conn(0x7ff01406e850 legacy=0x7ff014146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:07:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.591+0000 7ff01891d640 1 -- 172.21.15.45:0/259777587 shutdown_connections 2024-09-18T14:07:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.591+0000 7ff01891d640 1 -- 172.21.15.45:0/259777587 >> 172.21.15.45:0/259777587 conn(0x7ff0140636e0 msgr2=0x7ff0140a3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:07:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.592+0000 7ff01891d640 1 -- 172.21.15.45:0/259777587 shutdown_connections 2024-09-18T14:07:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:07:57.592+0000 7ff01891d640 1 -- 172.21.15.45:0/259777587 wait complete. 2024-09-18T14:07:57.665 INFO:tasks.rados.rados.0.smithi084.stdout:1919: done (15 left) 2024-09-18T14:07:57.665 INFO:tasks.rados.rados.0.smithi084.stdout:1920: done (14 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1921: done (13 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1922: done (12 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1924: done (11 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1925: done (10 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1926: done (9 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1927: done (8 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1928: done (7 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1929: done (6 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1930: done (5 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1931: done (4 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1932: done (3 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1933: done (2 left) 2024-09-18T14:07:57.666 INFO:tasks.rados.rados.0.smithi084.stdout:1934: done (1 left) 2024-09-18T14:07:57.667 INFO:tasks.rados.rados.0.smithi084.stdout:1935: done (0 left) 2024-09-18T14:07:57.667 INFO:tasks.rados.rados.0.smithi084.stdout:1936: write oid 20 current snap is 202 2024-09-18T14:07:57.667 INFO:tasks.rados.rados.0.smithi084.stdout:1936: seq_num 680 ranges {628760=724737,1892226=673223,3157635=271125} 2024-09-18T14:07:57.672 INFO:tasks.rados.rados.0.smithi084.stdout:1936: writing smithi08434663-20 from 628760 to 1353497 tid 1 2024-09-18T14:07:57.679 INFO:tasks.rados.rados.0.smithi084.stdout:1936: writing smithi08434663-20 from 1892226 to 2565449 tid 2 2024-09-18T14:07:57.684 INFO:tasks.rados.rados.0.smithi084.stdout:1936: writing smithi08434663-20 from 3157635 to 3428760 tid 3 2024-09-18T14:07:57.685 INFO:tasks.rados.rados.0.smithi084.stdout:1937: snap_remove snap 201 2024-09-18T14:07:58.679 INFO:tasks.rados.rados.0.smithi084.stdout:1937: done (1 left) 2024-09-18T14:07:58.680 INFO:tasks.rados.rados.0.smithi084.stdout:1938: write (excl) oid 42 current snap is 202 2024-09-18T14:07:58.680 INFO:tasks.rados.rados.0.smithi084.stdout:1936: finishing write tid 1 to smithi08434663-20 2024-09-18T14:07:58.680 INFO:tasks.rados.rados.0.smithi084.stdout:1938: seq_num 681 ranges {645567=799986,1950708=533234,3185058=260509} 2024-09-18T14:07:58.688 INFO:tasks.rados.rados.0.smithi084.stdout:1938: writing smithi08434663-42 from 645567 to 1445553 tid 1 2024-09-18T14:07:58.693 INFO:tasks.rados.rados.0.smithi084.stdout:1938: writing smithi08434663-42 from 1950708 to 2483942 tid 2 2024-09-18T14:07:58.699 INFO:tasks.rados.rados.0.smithi084.stdout:1938: writing smithi08434663-42 from 3185058 to 3445567 tid 3 2024-09-18T14:07:58.699 INFO:tasks.rados.rados.0.smithi084.stdout:1939: snap_remove snap 197 2024-09-18T14:07:59.686 INFO:tasks.rados.rados.0.smithi084.stdout:1939: done (2 left) 2024-09-18T14:07:59.686 INFO:tasks.rados.rados.0.smithi084.stdout:1940: snap_remove snap 200 2024-09-18T14:08:00.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:08:00.688 INFO:tasks.rados.rados.0.smithi084.stdout:1940: done (2 left) 2024-09-18T14:08:00.689 INFO:tasks.rados.rados.0.smithi084.stdout:1941: snap_create 2024-09-18T14:08:00.689 INFO:tasks.rados.rados.0.smithi084.stdout:1936: finishing write tid 2 to smithi08434663-20 2024-09-18T14:08:00.689 INFO:tasks.rados.rados.0.smithi084.stdout:1936: finishing write tid 3 to smithi08434663-20 2024-09-18T14:08:00.689 INFO:tasks.rados.rados.0.smithi084.stdout:1936: finishing write tid 4 to smithi08434663-20 2024-09-18T14:08:00.689 INFO:tasks.rados.rados.0.smithi084.stdout:1936: finishing write tid 5 to smithi08434663-20 2024-09-18T14:08:00.689 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 627 (ObjNum 680 snap 202 seq_num 680) dirty exists 2024-09-18T14:08:00.689 INFO:tasks.rados.rados.0.smithi084.stdout:1936: left oid 20 (ObjNum 680 snap 202 seq_num 680) 2024-09-18T14:08:00.689 INFO:tasks.rados.rados.0.smithi084.stdout:1938: finishing write tid 1 to smithi08434663-42 2024-09-18T14:08:00.689 INFO:tasks.rados.rados.0.smithi084.stdout:1938: finishing write tid 2 to smithi08434663-42 2024-09-18T14:08:00.689 INFO:tasks.rados.rados.0.smithi084.stdout:1938: finishing write tid 3 to smithi08434663-42 2024-09-18T14:08:00.689 INFO:tasks.rados.rados.0.smithi084.stdout:1938: finishing write tid 4 to smithi08434663-42 2024-09-18T14:08:00.689 INFO:tasks.rados.rados.0.smithi084.stdout:1938: finishing write tid 5 to smithi08434663-42 2024-09-18T14:08:00.690 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 885 (ObjNum 681 snap 202 seq_num 681) dirty exists 2024-09-18T14:08:00.690 INFO:tasks.rados.rados.0.smithi084.stdout:1938: left oid 42 (ObjNum 681 snap 202 seq_num 681) 2024-09-18T14:08:00.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.725+0000 7fe6b9b5d640 10 monclient: get_monmap_and_config 2024-09-18T14:08:00.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.725+0000 7fe6b9b5d640 10 monclient: build_initial_monmap 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.725+0000 7fe6b9b5d640 10 monclient: monmap: 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:00.726344+0000 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:00.726344+0000 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:00.727 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:00.727 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:00.727 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 1 Processor -- start 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 1 -- start start 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 10 monclient: init 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 10 monclient: _add_conns ranks=[4,1,0,3,8,6,2,7,5] 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 10 monclient(hunting): picked mon.noname-c con 0x7fe6b409f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 10 monclient(hunting): picked mon.noname-f con 0x7fe6b409fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 10 monclient(hunting): picked mon.noname-a con 0x7fe6b409ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b405ed70 con 0x7fe6b409ff70 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b409dfc0 con 0x7fe6b409fba0 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b409dce0 con 0x7fe6b409f7d0 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b9b5d640 10 monclient(hunting): authenticate will time out at 1296.064724s 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b37fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fe6b409f7d0 0x7fe6b4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:50352/0 (socket says 172.21.15.45:50352) 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b37fe640 1 -- 172.21.15.45:0/3409365206 learned_addr learned my addr 172.21.15.45:0/3409365206 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.726+0000 7fe6b3fff640 1 --1- 172.21.15.45:0/3409365206 >> v1:172.21.15.45:6789/0 conn(0x7fe6b409ff70 0x7fe6b413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54884/0 (socket says 172.21.15.45:54884) 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 1 -- 172.21.15.45:0/3409365206 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988093358 0 0) 0x7fe6b405ed70 con 0x7fe6b409ff70 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient(hunting): my global_id is 5301 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 1 -- 172.21.15.45:0/3409365206 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe69c0034e0 con 0x7fe6b409ff70 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 1 -- 172.21.15.45:0/3409365206 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 39585813 0 0) 0x7fe69c0034e0 con 0x7fe6b409ff70 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 1 -- 172.21.15.45:0/3409365206 >> v1:172.21.15.45:6791/0 conn(0x7fe6b409f7d0 legacy=0x7fe6b4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 1 -- 172.21.15.45:0/3409365206 >> v1:172.21.15.84:6789/0 conn(0x7fe6b409fba0 legacy=0x7fe6b4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient: _finish_hunting 0 2024-09-18T14:08:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 1 monclient: found mon.noname-a 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 1 -- 172.21.15.45:0/3409365206 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40a1c00 con 0x7fe6b409ff70 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient: _finish_auth 0 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 1 -- 172.21.15.45:0/3409365206 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe6ac001bb0 con 0x7fe6b409ff70 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient: dump: 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b9b5d640 5 monclient: authenticate success, global_id 5301 2024-09-18T14:08:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b9b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 1 -- 172.21.15.45:0/3409365206 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe6ac001d90 con 0x7fe6b409ff70 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 1 -- 172.21.15.45:0/3409365206 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe6ac004e60 con 0x7fe6b409ff70 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b9b5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b9b5d640 4 set_mon_vals no callback set 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.727+0000 7fe6b27fc640 10 monclient: dump: 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.728+0000 7fe6b9b5d640 10 monclient: shutdown 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.728+0000 7fe6b9b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.728+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3409365206 >> v1:172.21.15.45:6789/0 conn(0x7fe6b409ff70 legacy=0x7fe6b413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.728+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3409365206 shutdown_connections 2024-09-18T14:08:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.728+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3409365206 >> 172.21.15.45:0/3409365206 conn(0x7fe6b40630b0 msgr2=0x7fe6b40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.728+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3409365206 shutdown_connections 2024-09-18T14:08:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.728+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3409365206 wait complete. 2024-09-18T14:08:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.729+0000 7fe6b9b5d640 10 monclient: build_initial_monmap 2024-09-18T14:08:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.729+0000 7fe6b9b5d640 10 monclient: monmap: 2024-09-18T14:08:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.729+0000 7fe6b9b5d640 1 Processor -- start 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.729+0000 7fe6b9b5d640 1 -- start start 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.729+0000 7fe6b9b5d640 10 monclient: init 2024-09-18T14:08:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.730+0000 7fe6b9b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.730+0000 7fe6b9b5d640 10 monclient: _add_conns ranks=[8,0,1,2,6,7,3,4,5] 2024-09-18T14:08:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.730+0000 7fe6b9b5d640 10 monclient(hunting): picked mon.noname-e con 0x7fe6b409f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.730+0000 7fe6b9b5d640 10 monclient(hunting): picked mon.noname-a con 0x7fe6b409fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.730+0000 7fe6b9b5d640 10 monclient(hunting): picked mon.noname-f con 0x7fe6b409ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.730+0000 7fe6b9b5d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b41318c0 con 0x7fe6b409fba0 2024-09-18T14:08:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.730+0000 7fe6b9b5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b40a2560 con 0x7fe6b409ff70 2024-09-18T14:08:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.730+0000 7fe6b9b5d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b409d310 con 0x7fe6b409f7d0 2024-09-18T14:08:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.730+0000 7fe6b9b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.730+0000 7fe6b9b5d640 10 monclient(hunting): authenticate will time out at 1296.068521s 2024-09-18T14:08:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.730+0000 7fe6b2ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe6b409fba0 0x7fe6b409a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54896/0 (socket says 172.21.15.45:54896) 2024-09-18T14:08:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.730+0000 7fe6b2ffd640 1 -- 172.21.15.45:0/3007380266 learned_addr learned my addr 172.21.15.45:0/3007380266 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.731+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988955285 0 0) 0x7fe6b41318c0 con 0x7fe6b409fba0 2024-09-18T14:08:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.731+0000 7fe69b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.731+0000 7fe69b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.731+0000 7fe69b7fe640 10 monclient(hunting): my global_id is 5310 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.731+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a00034e0 con 0x7fe6b409fba0 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.731+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1415035098 0 0) 0x7fe6a00034e0 con 0x7fe6b409fba0 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.731+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe6b41318c0 con 0x7fe6b409fba0 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.731+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe6a8001be0 con 0x7fe6b409fba0 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.731+0000 7fe69b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.731+0000 7fe69b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.731+0000 7fe69b7fe640 10 monclient(hunting): dump: 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4072592188 0 0) 0x7fe6b409d310 con 0x7fe6b409f7d0 2024-09-18T14:08:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient(hunting): my global_id is 5255 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a00034e0 con 0x7fe6b409f7d0 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3792509200 0 0) 0x7fe6a00034e0 con 0x7fe6b409f7d0 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe6b409d310 con 0x7fe6b409f7d0 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe6a4001930 con 0x7fe6b409f7d0 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient(hunting): dump: 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3467870811 0 0) 0x7fe6b409d310 con 0x7fe6b409f7d0 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 >> v1:172.21.15.84:6789/0 conn(0x7fe6b409ff70 legacy=0x7fe6b409aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 >> v1:172.21.15.45:6789/0 conn(0x7fe6b409fba0 legacy=0x7fe6b409a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 1 monclient: found mon.e 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b409c090 con 0x7fe6b409f7d0 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient: _finish_auth 0 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe6b9b5d640 5 monclient: authenticate success, global_id 5255 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe6b9b5d640 10 monclient: _renew_subs 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe6b9b5d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3007380266 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b4130e80 con 0x7fe6b409f7d0 2024-09-18T14:08:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe6b9b5d640 10 monclient: _renew_subs 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe6b9b5d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3007380266 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b409bb80 con 0x7fe6b409f7d0 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe6a40047a0 con 0x7fe6b409f7d0 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe6a4004c50 con 0x7fe6b409f7d0 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe6b1ffb640 4 set_mon_vals no callback set 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.732+0000 7fe69b7fe640 10 monclient: dump: 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.733+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fe6a400fc20 con 0x7fe6b409f7d0 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.733+0000 7fe6b9b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.733+0000 7fe6b9b5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.733+0000 7fe6b9b5d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:08:00.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.733+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3007380266 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe694005010 con 0x7fe6b409f7d0 2024-09-18T14:08:00.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.734+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(425..425 src has 126..425) v4 ==== 3987+0+0 (unknown 1936645747 0 0) 0x7fe6a40469a0 con 0x7fe6b409f7d0 2024-09-18T14:08:00.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.734+0000 7fe6b2ffd640 10 monclient: get_auth_request con 0x7fe6a003a540 auth_method 0 2024-09-18T14:08:00.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.740+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fe6a4014c50 con 0x7fe6b409f7d0 2024-09-18T14:08:00.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.740+0000 7fe69b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:00.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.740+0000 7fe69b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:00.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.967+0000 7fe6b9b5d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:08:00.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.967+0000 7fe6b9b5d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:08:00.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.967+0000 7fe6b9b5d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:08:00.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.967+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3007380266 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe694005840 con 0x7fe6b409f7d0 2024-09-18T14:08:00.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.967+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3007380266 <== mon.8 v1:172.21.15.45:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 3567806574) 0x7fe6a4014a70 con 0x7fe6b409f7d0 2024-09-18T14:08:00.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.967+0000 7fe69b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:08:00.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.967+0000 7fe69b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:00.968 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":84,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":2,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:08:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.969+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3007380266 >> v1:172.21.15.84:6812/31478 conn(0x7fe6a003a540 legacy=0x7fe6a003c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.969+0000 7fe6b9b5d640 10 monclient: shutdown 2024-09-18T14:08:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.969+0000 7fe6b9b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.969+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3007380266 >> v1:172.21.15.45:6793/0 conn(0x7fe6b409f7d0 legacy=0x7fe6b409a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.969+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3007380266 shutdown_connections 2024-09-18T14:08:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.969+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3007380266 >> 172.21.15.45:0/3007380266 conn(0x7fe6b40630b0 msgr2=0x7fe6b406dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.969+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3007380266 shutdown_connections 2024-09-18T14:08:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:00.969+0000 7fe6b9b5d640 1 -- 172.21.15.45:0/3007380266 wait complete. 2024-09-18T14:08:00.980 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:08:00.980 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:08:00.980 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:08:01.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.077+0000 7f4d4ae9a640 10 monclient: get_monmap_and_config 2024-09-18T14:08:01.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.077+0000 7f4d4ae9a640 10 monclient: build_initial_monmap 2024-09-18T14:08:01.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.077+0000 7f4d4ae9a640 10 monclient: monmap: 2024-09-18T14:08:01.078 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:01.078 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.078 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:01.078611+0000 2024-09-18T14:08:01.078 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:01.078611+0000 2024-09-18T14:08:01.078 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:01.078 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.078 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:01.078 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:01.079 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:01.079 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:01.079 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:01.079 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:01.079 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:01.079 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:01.079 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:01.079 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 1 Processor -- start 2024-09-18T14:08:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 1 -- start start 2024-09-18T14:08:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 10 monclient: init 2024-09-18T14:08:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 10 monclient: _add_conns ranks=[0,8,3,4,1,2,7,6,5] 2024-09-18T14:08:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 10 monclient(hunting): picked mon.noname-a con 0x7f4d4409f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 10 monclient(hunting): picked mon.noname-e con 0x7f4d4409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 10 monclient(hunting): picked mon.noname-g con 0x7f4d4409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4405ed70 con 0x7f4d4409f7d0 2024-09-18T14:08:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4409dfc0 con 0x7f4d4409ff70 2024-09-18T14:08:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4409dce0 con 0x7f4d4409fba0 2024-09-18T14:08:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.078+0000 7f4d4ae9a640 10 monclient(hunting): authenticate will time out at 1296.416933s 2024-09-18T14:08:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d48c36640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4d4409f7d0 0x7f4d44132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54906/0 (socket says 172.21.15.45:54906) 2024-09-18T14:08:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d48c36640 1 -- 172.21.15.45:0/2963026732 learned_addr learned my addr 172.21.15.45:0/2963026732 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d43fff640 1 --1- 172.21.15.45:0/2963026732 >> v1:172.21.15.45:6793/0 conn(0x7f4d4409fba0 0x7f4d44137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:53266/0 (socket says 172.21.15.45:53266) 2024-09-18T14:08:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092339307 0 0) 0x7f4d4405ed70 con 0x7f4d4409f7d0 2024-09-18T14:08:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient(hunting): my global_id is 5319 2024-09-18T14:08:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d340034e0 con 0x7f4d4409f7d0 2024-09-18T14:08:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476824460 0 0) 0x7f4d4409dce0 con 0x7f4d4409fba0 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient(hunting): my global_id is 5264 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d4405ed70 con 0x7f4d4409fba0 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058924922 0 0) 0x7f4d4409dfc0 con 0x7f4d4409ff70 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient(hunting): my global_id is 154299 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d4409dce0 con 0x7f4d4409ff70 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4102524463 0 0) 0x7f4d4405ed70 con 0x7f4d4409fba0 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 >> v1:172.21.15.84:6790/0 conn(0x7f4d4409ff70 legacy=0x7f4d4413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 >> v1:172.21.15.45:6789/0 conn(0x7f4d4409f7d0 legacy=0x7f4d44132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient: _finish_hunting 0 2024-09-18T14:08:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 monclient: found mon.noname-e 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d440a1c00 con 0x7f4d4409fba0 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient: _finish_auth 0 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d30001740 con 0x7f4d4409fba0 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d4ae9a640 5 monclient: authenticate success, global_id 5264 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d4ae9a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient: dump: 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.092 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d4ae9a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4d300018a0 con 0x7f4d4409fba0 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d4ae9a640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 1 -- 172.21.15.45:0/2963026732 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d300047a0 con 0x7f4d4409fba0 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d4ae9a640 4 set_mon_vals no callback set 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.079+0000 7f4d437fe640 10 monclient: dump: 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.093 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.080+0000 7f4d4ae9a640 10 monclient: shutdown 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.080+0000 7f4d4ae9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:01.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.080+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/2963026732 >> v1:172.21.15.45:6793/0 conn(0x7f4d4409fba0 legacy=0x7f4d44137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.080+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/2963026732 shutdown_connections 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.080+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/2963026732 >> 172.21.15.45:0/2963026732 conn(0x7f4d440630b0 msgr2=0x7f4d440634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.080+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/2963026732 shutdown_connections 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.080+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/2963026732 wait complete. 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.080+0000 7f4d4ae9a640 10 monclient: build_initial_monmap 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.080+0000 7f4d4ae9a640 10 monclient: monmap: 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:01.095 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 1 Processor -- start 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 1 -- start start 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 10 monclient: init 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 10 monclient: _add_conns ranks=[7,4,0,2,8,5,3,6,1] 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 10 monclient(hunting): picked mon.noname-i con 0x7f4d4409f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 10 monclient(hunting): picked mon.noname-c con 0x7f4d4409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 10 monclient(hunting): picked mon.noname-a con 0x7f4d4409ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4409d310 con 0x7f4d4409ff70 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d440a2560 con 0x7f4d4409fba0 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4409be50 con 0x7f4d4409f7d0 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d4ae9a640 10 monclient(hunting): authenticate will time out at 1296.419870s 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d49437640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4d4409ff70 0x7f4d4414a470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54922/0 (socket says 172.21.15.45:54922) 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.081+0000 7f4d49437640 1 -- 172.21.15.45:0/367813572 learned_addr learned my addr 172.21.15.45:0/367813572 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280533644 0 0) 0x7f4d4409d310 con 0x7f4d4409ff70 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): my global_id is 5328 2024-09-18T14:08:01.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d280034e0 con 0x7f4d4409ff70 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815046870 0 0) 0x7f4d440a2560 con 0x7f4d4409fba0 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): my global_id is 164101 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d4409d310 con 0x7f4d4409fba0 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2902278895 0 0) 0x7f4d280034e0 con 0x7f4d4409ff70 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d440a2560 con 0x7f4d4409ff70 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3753299218 0 0) 0x7f4d4409be50 con 0x7f4d4409f7d0 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): my global_id is 5308 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d280034e0 con 0x7f4d4409f7d0 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3794801597 0 0) 0x7f4d4409d310 con 0x7f4d4409fba0 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d4409be50 con 0x7f4d4409fba0 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d3c002d40 con 0x7f4d4409ff70 2024-09-18T14:08:01.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): dump: 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.099 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d30004820 con 0x7f4d4409fba0 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient(hunting): dump: 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3238922844 0 0) 0x7f4d440a2560 con 0x7f4d4409ff70 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 >> v1:172.21.15.84:6792/0 conn(0x7f4d4409f7d0 legacy=0x7f4d44131720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 >> v1:172.21.15.45:6791/0 conn(0x7f4d4409fba0 legacy=0x7f4d44146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 monclient: found mon.a 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4409b940 con 0x7f4d4409ff70 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient: _finish_auth 0 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d4ae9a640 5 monclient: authenticate success, global_id 5328 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d4ae9a640 10 monclient: _renew_subs 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d4ae9a640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d44136ea0 con 0x7f4d4409ff70 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d4ae9a640 10 monclient: _renew_subs 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d4ae9a640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4411d8e0 con 0x7f4d4409ff70 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4d3c002ea0 con 0x7f4d4409ff70 2024-09-18T14:08:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d3c0019f0 con 0x7f4d4409ff70 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d42ffd640 4 set_mon_vals no callback set 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.082+0000 7f4d40ff9640 10 monclient: dump: 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.103 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.083+0000 7f4d4ae9a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.083+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4d24006580 con 0x7f4d24002830 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.083+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4d3c001bd0 con 0x7f4d4409ff70 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.083+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(425..425 src has 126..425) v4 ==== 3987+0+0 (unknown 1936645747 0 0) 0x7f4d3c046d10 con 0x7f4d4409ff70 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.083+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870772915 0 0) 0x7f4d24006580 con 0x7f4d24002830 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.083+0000 7f4d40ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.083+0000 7f4d40ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.083+0000 7f4d40ff9640 10 monclient(hunting): my global_id is 5337 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.083+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d440a2560 con 0x7f4d24002830 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d43fff640 10 monclient: get_auth_request con 0x7f4d2803a670 auth_method 0 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3228041312 0 0) 0x7f4d440a2560 con 0x7f4d24002830 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d24006580 con 0x7f4d24002830 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d38001da0 con 0x7f4d24002830 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 10 monclient: dump: 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2894075614 0 0) 0x7f4d24006580 con 0x7f4d24002830 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d2400a0a0 con 0x7f4d24002830 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f4d380017f0 con 0x7f4d24002830 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 >> v1:172.21.15.45:6789/0 conn(0x7f4d24002830 legacy=0x7f4d24004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.084+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 >> v1:172.21.15.45:6789/0 conn(0x7f4d24002830 legacy=0x7f4d24004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d4ae9a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4d24007540 con 0x7f4d240068f0 2024-09-18T14:08:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419282793 0 0) 0x7f4d24007540 con 0x7f4d240068f0 2024-09-18T14:08:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 10 monclient(hunting): my global_id is 5346 2024-09-18T14:08:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d24006580 con 0x7f4d240068f0 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 563436308 0 0) 0x7f4d24006580 con 0x7f4d240068f0 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d24007540 con 0x7f4d240068f0 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d38004eb0 con 0x7f4d240068f0 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 10 monclient: dump: 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 870959717 0 0) 0x7f4d24007540 con 0x7f4d240068f0 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.090+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4d240087a0 con 0x7f4d240068f0 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 852217845) 0x7f4d38007110 con 0x7f4d240068f0 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d40ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d40ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 >> v1:172.21.15.45:6789/0 conn(0x7f4d240068f0 legacy=0x7f4d24006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d40ff9640 1 -- 172.21.15.45:0/367813572 >> v1:172.21.15.45:6789/0 conn(0x7f4d240068f0 legacy=0x7f4d24006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:01.111 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 2, 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:01.122 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.123 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:08:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:01.130 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/367813572 >> v1:172.21.15.84:6812/31478 conn(0x7f4d2803a670 legacy=0x7f4d2803cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d4ae9a640 10 monclient: shutdown 2024-09-18T14:08:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d4ae9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/367813572 >> v1:172.21.15.45:6789/0 conn(0x7f4d4409ff70 legacy=0x7f4d4414a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d49437640 1 -- 172.21.15.45:0/367813572 reap_dead start 2024-09-18T14:08:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d49437640 1 -- 172.21.15.45:0/367813572 reap_dead start 2024-09-18T14:08:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/367813572 shutdown_connections 2024-09-18T14:08:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/367813572 >> 172.21.15.45:0/367813572 conn(0x7f4d440630b0 msgr2=0x7f4d440a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/367813572 shutdown_connections 2024-09-18T14:08:01.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.091+0000 7f4d4ae9a640 1 -- 172.21.15.45:0/367813572 wait complete. 2024-09-18T14:08:01.132 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:01.133 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:08:01.133 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:08:01.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.230+0000 7f1fd411d640 10 monclient: get_monmap_and_config 2024-09-18T14:08:01.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.230+0000 7f1fd411d640 10 monclient: build_initial_monmap 2024-09-18T14:08:01.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.230+0000 7f1fd411d640 10 monclient: monmap: 2024-09-18T14:08:01.230 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:01.230 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.230 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:01.231132+0000 2024-09-18T14:08:01.230 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:01.231132+0000 2024-09-18T14:08:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:01.231 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.230+0000 7f1fd411d640 1 Processor -- start 2024-09-18T14:08:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.230+0000 7f1fd411d640 1 -- start start 2024-09-18T14:08:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.230+0000 7f1fd411d640 10 monclient: init 2024-09-18T14:08:01.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd411d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:01.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd411d640 10 monclient: _add_conns ranks=[6,0,5,7,4,8,1,2,3] 2024-09-18T14:08:01.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd411d640 10 monclient(hunting): picked mon.noname-d con 0x7f1fcc06e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:01.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd411d640 10 monclient(hunting): picked mon.noname-a con 0x7f1fcc0a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:01.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd411d640 10 monclient(hunting): picked mon.noname-h con 0x7f1fcc0a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:01.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd411d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc05ed70 con 0x7f1fcc0a0cb0 2024-09-18T14:08:01.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd411d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc097620 con 0x7f1fcc0a1080 2024-09-18T14:08:01.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd411d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc097340 con 0x7f1fcc06e810 2024-09-18T14:08:01.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd411d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd411d640 10 monclient(hunting): authenticate will time out at 1296.569504s 2024-09-18T14:08:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd1eb9640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f1fcc06e810 0x7f1fcc0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40840/0 (socket says 172.21.15.45:40840) 2024-09-18T14:08:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd16b8640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1fcc0a0cb0 0x7f1fcc137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54930/0 (socket says 172.21.15.45:54930) 2024-09-18T14:08:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd1eb9640 1 -- 172.21.15.45:0/424101074 learned_addr learned my addr 172.21.15.45:0/424101074 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd0eb7640 1 -- 172.21.15.45:0/424101074 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021931829 0 0) 0x7f1fcc05ed70 con 0x7f1fcc0a0cb0 2024-09-18T14:08:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd0eb7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd0eb7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd0eb7640 10 monclient(hunting): my global_id is 5355 2024-09-18T14:08:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd0eb7640 1 -- 172.21.15.45:0/424101074 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fb80034e0 con 0x7f1fcc0a0cb0 2024-09-18T14:08:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd0eb7640 1 -- 172.21.15.45:0/424101074 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665881176 0 0) 0x7f1fcc097340 con 0x7f1fcc06e810 2024-09-18T14:08:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd0eb7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd0eb7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd0eb7640 10 monclient(hunting): my global_id is 5172 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.231+0000 7f1fd0eb7640 1 -- 172.21.15.45:0/424101074 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fcc05ed70 con 0x7f1fcc06e810 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 1 -- 172.21.15.45:0/424101074 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3496850834 0 0) 0x7f1fcc05ed70 con 0x7f1fcc06e810 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 1 -- 172.21.15.45:0/424101074 >> v1:172.21.15.84:6791/0 conn(0x7f1fcc0a1080 legacy=0x7f1fcc13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 1 -- 172.21.15.45:0/424101074 >> v1:172.21.15.45:6789/0 conn(0x7f1fcc0a0cb0 legacy=0x7f1fcc137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 10 monclient: _finish_hunting 0 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 1 monclient: found mon.noname-d 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 1 -- 172.21.15.45:0/424101074 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc0a1660 con 0x7f1fcc06e810 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 10 monclient: _finish_auth 0 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 1 -- 172.21.15.45:0/424101074 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1fc0001810 con 0x7f1fcc06e810 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd411d640 5 monclient: authenticate success, global_id 5172 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd411d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 10 monclient: dump: 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd411d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 1 -- 172.21.15.45:0/424101074 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1fc0001970 con 0x7f1fcc06e810 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 1 -- 172.21.15.45:0/424101074 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1fc0004860 con 0x7f1fcc06e810 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd0eb7640 10 monclient: dump: 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.244 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd411d640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:01.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd411d640 4 set_mon_vals no callback set 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd411d640 10 monclient: shutdown 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd411d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd411d640 1 -- 172.21.15.45:0/424101074 >> v1:172.21.15.45:6792/0 conn(0x7f1fcc06e810 legacy=0x7f1fcc0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd411d640 1 -- 172.21.15.45:0/424101074 shutdown_connections 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.232+0000 7f1fd411d640 1 -- 172.21.15.45:0/424101074 >> 172.21.15.45:0/424101074 conn(0x7f1fcc0a5bb0 msgr2=0x7f1fcc0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.233+0000 7f1fd411d640 1 -- 172.21.15.45:0/424101074 shutdown_connections 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.233+0000 7f1fd411d640 1 -- 172.21.15.45:0/424101074 wait complete. 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.233+0000 7f1fd411d640 10 monclient: build_initial_monmap 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.233+0000 7f1fd411d640 10 monclient: monmap: 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.233+0000 7f1fd411d640 1 Processor -- start 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.233+0000 7f1fd411d640 1 -- start start 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.233+0000 7f1fd411d640 10 monclient: init 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fd411d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fd411d640 10 monclient: _add_conns ranks=[2,3,5,7,0,4,1,8,6] 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fd411d640 10 monclient(hunting): picked mon.noname-b con 0x7f1fcc06e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fd411d640 10 monclient(hunting): picked mon.noname-g con 0x7f1fcc0a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fd411d640 10 monclient(hunting): picked mon.noname-h con 0x7f1fcc0a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fd411d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc09fa10 con 0x7f1fcc06e810 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fd411d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc0a2270 con 0x7f1fcc0a0cb0 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fd411d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc11da00 con 0x7f1fcc0a1080 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fd411d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fd411d640 10 monclient(hunting): authenticate will time out at 1296.572294s 2024-09-18T14:08:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fd1eb9640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f1fcc06e810 0x7f1fcc11c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47106/0 (socket says 172.21.15.45:47106) 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fd1eb9640 1 -- 172.21.15.45:0/181580748 learned_addr learned my addr 172.21.15.45:0/181580748 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222325793 0 0) 0x7f1fcc09fa10 con 0x7f1fcc06e810 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): my global_id is 144596 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fb00034e0 con 0x7f1fcc06e810 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078841013 0 0) 0x7f1fcc0a2270 con 0x7f1fcc0a0cb0 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): my global_id is 154308 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fcc09fa10 con 0x7f1fcc0a0cb0 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024218598 0 0) 0x7f1fcc11da00 con 0x7f1fcc0a1080 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): my global_id is 5126 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fcc0a2270 con 0x7f1fcc0a1080 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1207599799 0 0) 0x7f1fb00034e0 con 0x7f1fcc06e810 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1fcc11da00 con 0x7f1fcc06e810 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1fc00017d0 con 0x7f1fcc06e810 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): dump: 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 630876979 0 0) 0x7f1fcc09fa10 con 0x7f1fcc0a0cb0 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1fb00034e0 con 0x7f1fcc0a0cb0 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1fc80015b0 con 0x7f1fcc0a0cb0 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.234+0000 7f1fbdffb640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 10 monclient(hunting): dump: 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4147236322 0 0) 0x7f1fcc11da00 con 0x7f1fcc06e810 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 >> v1:172.21.15.84:6791/0 conn(0x7f1fcc0a1080 legacy=0x7f1fcc149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 >> v1:172.21.15.84:6790/0 conn(0x7f1fcc0a0cb0 legacy=0x7f1fcc11c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 10 monclient: _finish_hunting 0 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 1 monclient: found mon.b 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc11d030 con 0x7f1fcc06e810 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 10 monclient: _finish_auth 0 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:01.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fd411d640 5 monclient: authenticate success, global_id 144596 2024-09-18T14:08:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fd411d640 10 monclient: _renew_subs 2024-09-18T14:08:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fd411d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:08:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fd411d640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc11e0b0 con 0x7f1fcc06e810 2024-09-18T14:08:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1fc0004dd0 con 0x7f1fcc06e810 2024-09-18T14:08:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1fc0005280 con 0x7f1fcc06e810 2024-09-18T14:08:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbdffb640 10 monclient: dump: 2024-09-18T14:08:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fbffff640 4 set_mon_vals no callback set 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fd411d640 10 monclient: _renew_subs 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fd411d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fd411d640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc131050 con 0x7f1fcc06e810 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.235+0000 7f1fd411d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1fc0010250 con 0x7f1fcc06e810 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fd411d640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1fcc04c4a0 con 0x7f1fcc09a580 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(425..425 src has 126..425) v4 ==== 3987+0+0 (unknown 1936645747 0 0) 0x7f1fc0046c80 con 0x7f1fcc06e810 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079460440 0 0) 0x7f1fcc04c4a0 con 0x7f1fcc09a580 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fd26ba640 10 monclient: get_auth_request con 0x7f1fb003a5f0 auth_method 0 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 10 monclient(hunting): my global_id is 144605 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fcc11da00 con 0x7f1fcc09a580 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 878569500 0 0) 0x7f1fcc11da00 con 0x7f1fcc09a580 2024-09-18T14:08:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1fcc04c4a0 con 0x7f1fcc09a580 2024-09-18T14:08:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1fc8004ae0 con 0x7f1fcc09a580 2024-09-18T14:08:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 10 monclient: dump: 2024-09-18T14:08:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4118529055 0 0) 0x7f1fcc04c4a0 con 0x7f1fcc09a580 2024-09-18T14:08:01.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.236+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1fcc11efa0 con 0x7f1fcc09a580 2024-09-18T14:08:01.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.237+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1fc8005010 con 0x7f1fcc09a580 2024-09-18T14:08:01.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.237+0000 7f1fbdffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:01.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.237+0000 7f1fbdffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:01.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.237+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 >> v1:172.21.15.45:6790/0 conn(0x7f1fcc09a580 legacy=0x7f1fcc09a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.237+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 >> v1:172.21.15.45:6790/0 conn(0x7f1fcc09a580 legacy=0x7f1fcc09a950 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:01.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.242+0000 7f1fd411d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:01.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.242+0000 7f1fd411d640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1fcc11f410 con 0x7f1fcc09b060 2024-09-18T14:08:01.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628202346 0 0) 0x7f1fcc11f410 con 0x7f1fcc09b060 2024-09-18T14:08:01.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 10 monclient(hunting): my global_id is 144614 2024-09-18T14:08:01.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fcc04c4a0 con 0x7f1fcc09b060 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 927059699 0 0) 0x7f1fcc04c4a0 con 0x7f1fcc09b060 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1fcc11f410 con 0x7f1fcc09b060 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1fc80056f0 con 0x7f1fcc09b060 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 10 monclient: dump: 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.261 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 252803215 0 0) 0x7f1fcc11f410 con 0x7f1fcc09b060 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1fcc04b720 con 0x7f1fcc09b060 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 354888641) 0x7f1fc8006ab0 con 0x7f1fcc09b060 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:01.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 >> v1:172.21.15.45:6790/0 conn(0x7f1fcc09b060 legacy=0x7f1fcc14efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.243+0000 7f1fbdffb640 1 -- 172.21.15.45:0/181580748 >> v1:172.21.15.45:6790/0 conn(0x7f1fcc09b060 legacy=0x7f1fcc14efd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:01.263 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 2, 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:01.283 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:01.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.244+0000 7f1fd411d640 1 -- 172.21.15.45:0/181580748 >> v1:172.21.15.84:6812/31478 conn(0x7f1fb003a5f0 legacy=0x7f1fb003caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.244+0000 7f1fd411d640 10 monclient: shutdown 2024-09-18T14:08:01.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.244+0000 7f1fd411d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:01.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.244+0000 7f1fd411d640 1 -- 172.21.15.45:0/181580748 >> v1:172.21.15.45:6790/0 conn(0x7f1fcc06e810 legacy=0x7f1fcc11c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.244+0000 7f1fd26ba640 1 -- 172.21.15.45:0/181580748 reap_dead start 2024-09-18T14:08:01.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.244+0000 7f1fd26ba640 1 -- 172.21.15.45:0/181580748 reap_dead start 2024-09-18T14:08:01.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.244+0000 7f1fd411d640 1 -- 172.21.15.45:0/181580748 shutdown_connections 2024-09-18T14:08:01.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.244+0000 7f1fd411d640 1 -- 172.21.15.45:0/181580748 >> 172.21.15.45:0/181580748 conn(0x7f1fcc0a5bb0 msgr2=0x7f1fcc132b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:01.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.244+0000 7f1fd411d640 1 -- 172.21.15.45:0/181580748 shutdown_connections 2024-09-18T14:08:01.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.244+0000 7f1fd411d640 1 -- 172.21.15.45:0/181580748 wait complete. 2024-09-18T14:08:01.285 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:01.285 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:08:01.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:08:01.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.382+0000 7f5aa4709640 10 monclient: get_monmap_and_config 2024-09-18T14:08:01.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.382+0000 7f5aa4709640 10 monclient: build_initial_monmap 2024-09-18T14:08:01.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.382+0000 7f5aa4709640 10 monclient: monmap: 2024-09-18T14:08:01.382 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:01.383056+0000 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:01.383056+0000 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:01.384 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.382+0000 7f5aa4709640 1 Processor -- start 2024-09-18T14:08:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.382+0000 7f5aa4709640 1 -- start start 2024-09-18T14:08:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.382+0000 7f5aa4709640 10 monclient: init 2024-09-18T14:08:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa4709640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa4709640 10 monclient: _add_conns ranks=[5,4,1,3,0,8,6,7,2] 2024-09-18T14:08:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa4709640 10 monclient(hunting): picked mon.noname-h con 0x7f5a9c06e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa4709640 10 monclient(hunting): picked mon.noname-c con 0x7f5a9c0a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa4709640 10 monclient(hunting): picked mon.noname-f con 0x7f5a9c0a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa4709640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c05ed70 con 0x7f5a9c0a1080 2024-09-18T14:08:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa4709640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c097620 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa4709640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c097340 con 0x7f5a9c06e810 2024-09-18T14:08:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa4709640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa4709640 10 monclient(hunting): authenticate will time out at 1296.721532s 2024-09-18T14:08:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa1ca4640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f5a9c0a0cb0 0x7f5a9c137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:50378/0 (socket says 172.21.15.45:50378) 2024-09-18T14:08:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa1ca4640 1 -- 172.21.15.45:0/292648973 learned_addr learned my addr 172.21.15.45:0/292648973 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa14a3640 1 -- 172.21.15.45:0/292648973 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642273193 0 0) 0x7f5a9c097620 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa14a3640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa14a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa14a3640 10 monclient(hunting): my global_id is 164110 2024-09-18T14:08:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.383+0000 7f5aa14a3640 1 -- 172.21.15.45:0/292648973 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a880034e0 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 1 -- 172.21.15.45:0/292648973 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110460487 0 0) 0x7f5a9c097340 con 0x7f5a9c06e810 2024-09-18T14:08:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient(hunting): my global_id is 5135 2024-09-18T14:08:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 1 -- 172.21.15.45:0/292648973 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a9c097620 con 0x7f5a9c06e810 2024-09-18T14:08:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 1 -- 172.21.15.45:0/292648973 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2573672751 0 0) 0x7f5a880034e0 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 1 -- 172.21.15.45:0/292648973 >> v1:172.21.15.84:6791/0 conn(0x7f5a9c06e810 legacy=0x7f5a9c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 1 -- 172.21.15.45:0/292648973 >> v1:172.21.15.84:6789/0 conn(0x7f5a9c0a1080 legacy=0x7f5a9c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient: _finish_hunting 0 2024-09-18T14:08:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 1 monclient: found mon.noname-c 2024-09-18T14:08:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:08:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 1 -- 172.21.15.45:0/292648973 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c0a1660 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient: _finish_auth 0 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 1 -- 172.21.15.45:0/292648973 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5a8c001830 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa4709640 5 monclient: authenticate success, global_id 164110 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa4709640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient: dump: 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.397 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa4709640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 1 -- 172.21.15.45:0/292648973 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5a8c001a10 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa4709640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 1 -- 172.21.15.45:0/292648973 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5a8c004ba0 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa4709640 4 set_mon_vals no callback set 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa14a3640 10 monclient: dump: 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa4709640 10 monclient: shutdown 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa4709640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa4709640 1 -- 172.21.15.45:0/292648973 >> v1:172.21.15.45:6791/0 conn(0x7f5a9c0a0cb0 legacy=0x7f5a9c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa4709640 1 -- 172.21.15.45:0/292648973 shutdown_connections 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa4709640 1 -- 172.21.15.45:0/292648973 >> 172.21.15.45:0/292648973 conn(0x7f5a9c0a5bb0 msgr2=0x7f5a9c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:01.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.384+0000 7f5aa4709640 1 -- 172.21.15.45:0/292648973 shutdown_connections 2024-09-18T14:08:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.385+0000 7f5aa4709640 1 -- 172.21.15.45:0/292648973 wait complete. 2024-09-18T14:08:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.385+0000 7f5aa4709640 10 monclient: build_initial_monmap 2024-09-18T14:08:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.385+0000 7f5aa4709640 10 monclient: monmap: 2024-09-18T14:08:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:01.401 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.385+0000 7f5aa4709640 1 Processor -- start 2024-09-18T14:08:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.385+0000 7f5aa4709640 1 -- start start 2024-09-18T14:08:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.385+0000 7f5aa4709640 10 monclient: init 2024-09-18T14:08:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5aa4709640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5aa4709640 10 monclient: _add_conns ranks=[7,4,8,2,6,0,5,1,3] 2024-09-18T14:08:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5aa4709640 10 monclient(hunting): picked mon.noname-i con 0x7f5a9c06e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5aa4709640 10 monclient(hunting): picked mon.noname-c con 0x7f5a9c0a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5aa4709640 10 monclient(hunting): picked mon.noname-e con 0x7f5a9c0a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5aa4709640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c122d90 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5aa4709640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c13db80 con 0x7f5a9c06e810 2024-09-18T14:08:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5aa4709640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c1227d0 con 0x7f5a9c0a1080 2024-09-18T14:08:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5aa4709640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5aa4709640 10 monclient(hunting): authenticate will time out at 1296.724373s 2024-09-18T14:08:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5aa1ca4640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f5a9c0a0cb0 0x7f5a9c1466a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:50388/0 (socket says 172.21.15.45:50388) 2024-09-18T14:08:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5aa1ca4640 1 -- 172.21.15.45:0/2147652553 learned_addr learned my addr 172.21.15.45:0/2147652553 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548912533 0 0) 0x7f5a9c122d90 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): my global_id is 164119 2024-09-18T14:08:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a800034e0 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388321011 0 0) 0x7f5a9c1227d0 con 0x7f5a9c0a1080 2024-09-18T14:08:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): my global_id is 5273 2024-09-18T14:08:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a9c122d90 con 0x7f5a9c0a1080 2024-09-18T14:08:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119345983 0 0) 0x7f5a9c13db80 con 0x7f5a9c06e810 2024-09-18T14:08:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): my global_id is 5317 2024-09-18T14:08:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a9c1227d0 con 0x7f5a9c06e810 2024-09-18T14:08:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3056476570 0 0) 0x7f5a800034e0 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5a9c13db80 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5a8c0047a0 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): dump: 2024-09-18T14:08:01.404 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1152981597 0 0) 0x7f5a9c122d90 con 0x7f5a9c0a1080 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5a800034e0 con 0x7f5a9c0a1080 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5a94001c20 con 0x7f5a9c0a1080 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.386+0000 7f5a927fc640 10 monclient(hunting): dump: 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.407 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 768933600 0 0) 0x7f5a9c1227d0 con 0x7f5a9c06e810 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5a9c122d90 con 0x7f5a9c06e810 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5a98001bf0 con 0x7f5a9c06e810 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 10 monclient(hunting): dump: 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.409 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.409 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.409 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.409 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.409 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.409 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.409 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.409 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.409 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3103117978 0 0) 0x7f5a9c13db80 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 >> v1:172.21.15.45:6793/0 conn(0x7f5a9c0a1080 legacy=0x7f5a9c149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 >> v1:172.21.15.84:6792/0 conn(0x7f5a9c06e810 legacy=0x7f5a9c13d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 10 monclient: _finish_hunting 0 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 1 monclient: found mon.c 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c120290 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 10 monclient: _finish_auth 0 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5aa4709640 5 monclient: authenticate success, global_id 164119 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5aa4709640 10 monclient: _renew_subs 2024-09-18T14:08:01.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5aa4709640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:01.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5aa4709640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c1224f0 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5aa4709640 10 monclient: _renew_subs 2024-09-18T14:08:01.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5aa4709640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:01.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5aa4709640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c121620 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5a8c001210 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:01.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5a8c004bb0 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5aa0ca2640 4 set_mon_vals no callback set 2024-09-18T14:08:01.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:01.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5a927fc640 10 monclient: dump: 2024-09-18T14:08:01.411 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.412 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5aa4709640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.387+0000 7f5aa4709640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5a84006580 con 0x7f5a84002830 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f5a8c010050 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(425..425 src has 126..425) v4 ==== 3987+0+0 (unknown 1936645747 0 0) 0x7f5a8c046ce0 con 0x7f5a9c0a0cb0 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914597803 0 0) 0x7f5a84006580 con 0x7f5a84002830 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 10 monclient(hunting): my global_id is 164128 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a9c13db80 con 0x7f5a84002830 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5aa2ca6640 10 monclient: get_auth_request con 0x7f5a8003aa40 auth_method 0 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1249870883 0 0) 0x7f5a9c13db80 con 0x7f5a84002830 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5a84006580 con 0x7f5a84002830 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5a98002340 con 0x7f5a84002830 2024-09-18T14:08:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 10 monclient: dump: 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.414 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.415 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.415 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.415 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1006771680 0 0) 0x7f5a84006580 con 0x7f5a84002830 2024-09-18T14:08:01.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.388+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a8400a0a0 con 0x7f5a84002830 2024-09-18T14:08:01.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.389+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5a98002500 con 0x7f5a84002830 2024-09-18T14:08:01.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.389+0000 7f5a927fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:01.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.389+0000 7f5a927fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:01.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.389+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 >> v1:172.21.15.45:6791/0 conn(0x7f5a84002830 legacy=0x7f5a84004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.389+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 >> v1:172.21.15.45:6791/0 conn(0x7f5a84002830 legacy=0x7f5a84004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:01.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.394+0000 7f5aa4709640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.394+0000 7f5aa4709640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5a84009660 con 0x7f5a84009040 2024-09-18T14:08:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888412094 0 0) 0x7f5a84009660 con 0x7f5a84009040 2024-09-18T14:08:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 10 monclient(hunting): my global_id is 164137 2024-09-18T14:08:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a84006580 con 0x7f5a84009040 2024-09-18T14:08:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2135918662 0 0) 0x7f5a84006580 con 0x7f5a84009040 2024-09-18T14:08:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5a84009660 con 0x7f5a84009040 2024-09-18T14:08:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5a98002da0 con 0x7f5a84009040 2024-09-18T14:08:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 10 monclient: dump: 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.419 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.419 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.419 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2426419266 0 0) 0x7f5a84009660 con 0x7f5a84009040 2024-09-18T14:08:01.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5a84009920 con 0x7f5a84009040 2024-09-18T14:08:01.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 4045530462) 0x7f5a98003230 con 0x7f5a84009040 2024-09-18T14:08:01.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:01.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:01.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 >> v1:172.21.15.45:6791/0 conn(0x7f5a84009040 legacy=0x7f5a84008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.395+0000 7f5a927fc640 1 -- 172.21.15.45:0/2147652553 >> v1:172.21.15.45:6791/0 conn(0x7f5a84009040 legacy=0x7f5a84008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:01.419 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:01.420 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:01.421 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:01.421 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.421 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 2, 2024-09-18T14:08:01.421 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:01.421 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:01.421 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:01.421 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.421 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.421 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.421 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.421 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.422 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:01.423 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.424 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:01.425 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:01.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.429 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:01.430 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.431 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.396+0000 7f5aa4709640 1 -- 172.21.15.45:0/2147652553 >> v1:172.21.15.84:6812/31478 conn(0x7f5a8003aa40 legacy=0x7f5a8003cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.396+0000 7f5aa4709640 10 monclient: shutdown 2024-09-18T14:08:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.396+0000 7f5aa4709640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.396+0000 7f5aa4709640 1 -- 172.21.15.45:0/2147652553 >> v1:172.21.15.45:6791/0 conn(0x7f5a9c0a0cb0 legacy=0x7f5a9c1466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.396+0000 7f5aa2ca6640 1 -- 172.21.15.45:0/2147652553 reap_dead start 2024-09-18T14:08:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.396+0000 7f5aa2ca6640 1 -- 172.21.15.45:0/2147652553 reap_dead start 2024-09-18T14:08:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.396+0000 7f5aa4709640 1 -- 172.21.15.45:0/2147652553 shutdown_connections 2024-09-18T14:08:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.396+0000 7f5aa4709640 1 -- 172.21.15.45:0/2147652553 >> 172.21.15.45:0/2147652553 conn(0x7f5a9c0a5bb0 msgr2=0x7f5a9c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.396+0000 7f5aa4709640 1 -- 172.21.15.45:0/2147652553 shutdown_connections 2024-09-18T14:08:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.396+0000 7f5aa4709640 1 -- 172.21.15.45:0/2147652553 wait complete. 2024-09-18T14:08:01.440 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:01.440 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:08:01.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:08:01.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.537+0000 7f3730094640 10 monclient: get_monmap_and_config 2024-09-18T14:08:01.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.537+0000 7f3730094640 10 monclient: build_initial_monmap 2024-09-18T14:08:01.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.537+0000 7f3730094640 10 monclient: monmap: 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:01.538527+0000 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:01.538527+0000 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:01.539 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:01.539 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 1 Processor -- start 2024-09-18T14:08:01.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 1 -- start start 2024-09-18T14:08:01.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 10 monclient: init 2024-09-18T14:08:01.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:01.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 10 monclient: _add_conns ranks=[6,0,4,8,1,5,7,3,2] 2024-09-18T14:08:01.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 10 monclient(hunting): picked mon.noname-d con 0x7f372809f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:01.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 10 monclient(hunting): picked mon.noname-a con 0x7f372809fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 10 monclient(hunting): picked mon.noname-c con 0x7f372809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f372805ed70 con 0x7f372809fba0 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f372809dfc0 con 0x7f372809ff70 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f372809dce0 con 0x7f372809f7d0 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f3730094640 10 monclient(hunting): authenticate will time out at 1296.876860s 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f372de30640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f372809f7d0 0x7f3728132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40852/0 (socket says 172.21.15.45:40852) 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372e631640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f372809ff70 0x7f372813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:50414/0 (socket says 172.21.15.45:50414) 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.538+0000 7f372d62f640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f372809fba0 0x7f3728137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54932/0 (socket says 172.21.15.45:54932) 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372de30640 1 -- 172.21.15.45:0/524558162 learned_addr learned my addr 172.21.15.45:0/524558162 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816376445 0 0) 0x7f372809dce0 con 0x7f372809f7d0 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient(hunting): my global_id is 5181 2024-09-18T14:08:01.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37140034e0 con 0x7f372809f7d0 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674042166 0 0) 0x7f372805ed70 con 0x7f372809fba0 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient(hunting): my global_id is 5364 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f372809dce0 con 0x7f372809fba0 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224018502 0 0) 0x7f372809dfc0 con 0x7f372809ff70 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient(hunting): my global_id is 164146 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f372805ed70 con 0x7f372809ff70 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3022959906 0 0) 0x7f372809dce0 con 0x7f372809fba0 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 >> v1:172.21.15.45:6792/0 conn(0x7f372809f7d0 legacy=0x7f3728132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 >> v1:172.21.15.45:6791/0 conn(0x7f372809ff70 legacy=0x7f372813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient: _finish_hunting 0 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 monclient: found mon.noname-a 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37280a1c00 con 0x7f372809fba0 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient: _finish_auth 0 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f3730094640 5 monclient: authenticate success, global_id 5364 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3718001790 con 0x7f372809fba0 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f3730094640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient: dump: 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.550 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f3730094640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f37180018f0 con 0x7f372809fba0 2024-09-18T14:08:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 1 -- 172.21.15.45:0/524558162 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f37180047a0 con 0x7f372809fba0 2024-09-18T14:08:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f3730094640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f3730094640 4 set_mon_vals no callback set 2024-09-18T14:08:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.539+0000 7f372ce2e640 10 monclient: dump: 2024-09-18T14:08:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.552 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.540+0000 7f3730094640 10 monclient: shutdown 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.540+0000 7f3730094640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.540+0000 7f3730094640 1 -- 172.21.15.45:0/524558162 >> v1:172.21.15.45:6789/0 conn(0x7f372809fba0 legacy=0x7f3728137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.540+0000 7f3730094640 1 -- 172.21.15.45:0/524558162 shutdown_connections 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.540+0000 7f3730094640 1 -- 172.21.15.45:0/524558162 >> 172.21.15.45:0/524558162 conn(0x7f37280630b0 msgr2=0x7f37280634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.540+0000 7f3730094640 1 -- 172.21.15.45:0/524558162 shutdown_connections 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.540+0000 7f3730094640 1 -- 172.21.15.45:0/524558162 wait complete. 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.540+0000 7f3730094640 10 monclient: build_initial_monmap 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.540+0000 7f3730094640 10 monclient: monmap: 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:01.553 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:01.554 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 1 Processor -- start 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 1 -- start start 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 10 monclient: init 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 10 monclient: _add_conns ranks=[0,2,1,7,6,8,5,4,3] 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 10 monclient(hunting): picked mon.noname-a con 0x7f372809f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 10 monclient(hunting): picked mon.noname-b con 0x7f372809fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 10 monclient(hunting): picked mon.noname-f con 0x7f372809ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f372809bf70 con 0x7f372809f7d0 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f372809ba20 con 0x7f372809ff70 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3728099e80 con 0x7f372809fba0 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f3730094640 10 monclient(hunting): authenticate will time out at 1296.879699s 2024-09-18T14:08:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f372de30640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f372809f7d0 0x7f3728098560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54938/0 (socket says 172.21.15.45:54938) 2024-09-18T14:08:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f372de30640 1 -- 172.21.15.45:0/970643740 learned_addr learned my addr 172.21.15.45:0/970643740 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f372d62f640 1 --1- 172.21.15.45:0/970643740 >> v1:172.21.15.45:6790/0 conn(0x7f372809fba0 0x7f3728098c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47136/0 (socket says 172.21.15.45:47136) 2024-09-18T14:08:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775057744 0 0) 0x7f372809bf70 con 0x7f372809f7d0 2024-09-18T14:08:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f371dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f371dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f371dffb640 10 monclient(hunting): my global_id is 5373 2024-09-18T14:08:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.541+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f370c0034e0 con 0x7f372809f7d0 2024-09-18T14:08:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593798839 0 0) 0x7f3728099e80 con 0x7f372809fba0 2024-09-18T14:08:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): my global_id is 144623 2024-09-18T14:08:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f372809bf70 con 0x7f372809fba0 2024-09-18T14:08:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1705349024 0 0) 0x7f370c0034e0 con 0x7f372809f7d0 2024-09-18T14:08:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3728099e80 con 0x7f372809f7d0 2024-09-18T14:08:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3724001650 con 0x7f372809f7d0 2024-09-18T14:08:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:08:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): dump: 2024-09-18T14:08:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.562 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397467239 0 0) 0x7f372809ba20 con 0x7f372809ff70 2024-09-18T14:08:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): my global_id is 164107 2024-09-18T14:08:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f370c0034e0 con 0x7f372809ff70 2024-09-18T14:08:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2172669523 0 0) 0x7f372809bf70 con 0x7f372809fba0 2024-09-18T14:08:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f372809ba20 con 0x7f372809fba0 2024-09-18T14:08:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f37180016d0 con 0x7f372809fba0 2024-09-18T14:08:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient(hunting): dump: 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.564 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1567519646 0 0) 0x7f3728099e80 con 0x7f372809f7d0 2024-09-18T14:08:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 >> v1:172.21.15.45:6790/0 conn(0x7f372809fba0 legacy=0x7f3728098c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 >> v1:172.21.15.84:6789/0 conn(0x7f372809ff70 legacy=0x7f3728149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient: _finish_hunting 0 2024-09-18T14:08:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 monclient: found mon.a 2024-09-18T14:08:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372809bc90 con 0x7f372809f7d0 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient: _finish_auth 0 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f3730094640 5 monclient: authenticate success, global_id 5373 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f3730094640 10 monclient: _renew_subs 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f3730094640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f3730094640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372809c250 con 0x7f372809f7d0 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3724004a50 con 0x7f372809f7d0 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3724004f00 con 0x7f372809f7d0 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371dffb640 10 monclient: dump: 2024-09-18T14:08:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f371ffff640 4 set_mon_vals no callback set 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f3730094640 10 monclient: _renew_subs 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f3730094640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.542+0000 7f3730094640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f372809c670 con 0x7f372809f7d0 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.543+0000 7f3730094640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.543+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f372400fed0 con 0x7f372809f7d0 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.543+0000 7f3730094640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f372804c4a0 con 0x7f3728099380 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.543+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(425..425 src has 126..425) v4 ==== 3987+0+0 (unknown 1936645747 0 0) 0x7f3724046ba0 con 0x7f372809f7d0 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.543+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199025255 0 0) 0x7f372804c4a0 con 0x7f3728099380 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.543+0000 7f371dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.543+0000 7f371dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.543+0000 7f371dffb640 10 monclient(hunting): my global_id is 5190 2024-09-18T14:08:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.543+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3728099e80 con 0x7f3728099380 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.543+0000 7f372e631640 10 monclient: get_auth_request con 0x7f370c03a6f0 auth_method 0 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2914548352 0 0) 0x7f3728099e80 con 0x7f3728099380 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f372804c4a0 con 0x7f3728099380 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f37180011b0 con 0x7f3728099380 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 10 monclient: dump: 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4006179770 0 0) 0x7f372804c4a0 con 0x7f3728099380 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f372809abf0 con 0x7f3728099380 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3718005180 con 0x7f3728099380 2024-09-18T14:08:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:01.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:01.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 >> v1:172.21.15.45:6792/0 conn(0x7f3728099380 legacy=0x7f37281334c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.544+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 >> v1:172.21.15.45:6792/0 conn(0x7f3728099380 legacy=0x7f37281334c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:01.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f3730094640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:01.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f3730094640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f372804b720 con 0x7f372802a8d0 2024-09-18T14:08:01.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232646348 0 0) 0x7f372804b720 con 0x7f372802a8d0 2024-09-18T14:08:01.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f371dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f371dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f371dffb640 10 monclient(hunting): my global_id is 5199 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f372804c4a0 con 0x7f372802a8d0 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 740815948 0 0) 0x7f372804c4a0 con 0x7f372802a8d0 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f372804b720 con 0x7f372802a8d0 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3718005a40 con 0x7f372802a8d0 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f371dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f371dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.550+0000 7f371dffb640 10 monclient: dump: 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4103784085 0 0) 0x7f372804b720 con 0x7f372802a8d0 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3728099750 con 0x7f372802a8d0 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 509351398) 0x7f3718008170 con 0x7f372802a8d0 2024-09-18T14:08:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f371dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:01.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f371dffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:01.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 >> v1:172.21.15.45:6792/0 conn(0x7f372802a8d0 legacy=0x7f3728133bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.574 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:01.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:01.574 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:01.574 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:01.574 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:01.574 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:01.574 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:01.574 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:01.575 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:01.575 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:01.575 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:01.575 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:01.575 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:01.575 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:01.575 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:01.575 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.575 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 2, 2024-09-18T14:08:01.575 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:01.576 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:01.576 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:01.576 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.576 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.576 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.576 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.576 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.576 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.576 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.576 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.576 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.576 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:01.578 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.579 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.580 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.581 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:01.581 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:01.581 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.581 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.581 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.581 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.582 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.583 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.584 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:01.584 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:01.584 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.584 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.584 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.584 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:01.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:01.584 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.584 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:01.586 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.587 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.588 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:01.590 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:01.591 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.592 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.592 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.592 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:01.592 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:01.592 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.592 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.592 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.592 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.592 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.592 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.594 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.594 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:08:01.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.594 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.594 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.594 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:01.594 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.594 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.594 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:01.594 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f371dffb640 1 -- 172.21.15.45:0/970643740 >> v1:172.21.15.45:6792/0 conn(0x7f372802a8d0 legacy=0x7f3728133bd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f3730094640 1 -- 172.21.15.45:0/970643740 >> v1:172.21.15.84:6812/31478 conn(0x7f370c03a6f0 legacy=0x7f370c03cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f3730094640 10 monclient: shutdown 2024-09-18T14:08:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f3730094640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f3730094640 1 -- 172.21.15.45:0/970643740 >> v1:172.21.15.45:6789/0 conn(0x7f372809f7d0 legacy=0x7f3728098560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f372e631640 1 -- 172.21.15.45:0/970643740 reap_dead start 2024-09-18T14:08:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f372e631640 1 -- 172.21.15.45:0/970643740 reap_dead start 2024-09-18T14:08:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f3730094640 1 -- 172.21.15.45:0/970643740 shutdown_connections 2024-09-18T14:08:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f3730094640 1 -- 172.21.15.45:0/970643740 >> 172.21.15.45:0/970643740 conn(0x7f37280630b0 msgr2=0x7f3728130ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:01.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f3730094640 1 -- 172.21.15.45:0/970643740 shutdown_connections 2024-09-18T14:08:01.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.551+0000 7f3730094640 1 -- 172.21.15.45:0/970643740 wait complete. 2024-09-18T14:08:01.597 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:01.597 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:08:01.597 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:08:01.693 INFO:tasks.rados.rados.0.smithi084.stdout:1936: done (2 left) 2024-09-18T14:08:01.693 INFO:tasks.rados.rados.0.smithi084.stdout:1938: done (1 left) 2024-09-18T14:08:01.693 INFO:tasks.rados.rados.0.smithi084.stdout:1941: done (0 left) 2024-09-18T14:08:01.693 INFO:tasks.rados.rados.0.smithi084.stdout:1942: delete oid 30 current snap is 203 2024-09-18T14:08:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.695+0000 7f25a0b5d640 10 monclient: get_monmap_and_config 2024-09-18T14:08:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.695+0000 7f25a0b5d640 10 monclient: build_initial_monmap 2024-09-18T14:08:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.695+0000 7f25a0b5d640 10 monclient: monmap: 2024-09-18T14:08:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:01.696817+0000 2024-09-18T14:08:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:01.696817+0000 2024-09-18T14:08:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:01.698 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.699 INFO:tasks.rados.rados.0.smithi084.stdout:1942: done (0 left) 2024-09-18T14:08:01.699 INFO:tasks.rados.rados.0.smithi084.stdout:1943: read oid 14 snap -1 2024-09-18T14:08:01.700 INFO:tasks.rados.rados.0.smithi084.stdout:1943: expect deleted 2024-09-18T14:08:01.700 INFO:tasks.rados.rados.0.smithi084.stdout:1944: write oid 22 current snap is 203 2024-09-18T14:08:01.700 INFO:tasks.rados.rados.0.smithi084.stdout:1944: seq_num 683 ranges {679181=550484,1667672=536903,2839904=435373} 2024-09-18T14:08:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.696+0000 7f25a0b5d640 1 Processor -- start 2024-09-18T14:08:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.696+0000 7f25a0b5d640 1 -- start start 2024-09-18T14:08:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.696+0000 7f25a0b5d640 10 monclient: init 2024-09-18T14:08:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 10 monclient: _add_conns ranks=[2,1,6,8,7,0,3,5,4] 2024-09-18T14:08:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 10 monclient(hunting): picked mon.noname-b con 0x7f259c06e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 10 monclient(hunting): picked mon.noname-f con 0x7f259c0a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 10 monclient(hunting): picked mon.noname-d con 0x7f259c0a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c05ed70 con 0x7f259c0a0cb0 2024-09-18T14:08:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c097620 con 0x7f259c06e810 2024-09-18T14:08:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c097340 con 0x7f259c0a1080 2024-09-18T14:08:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 10 monclient(hunting): authenticate will time out at 1297.035258s 2024-09-18T14:08:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259a59c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f259c06e810 0x7f259c0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47152/0 (socket says 172.21.15.45:47152) 2024-09-18T14:08:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259a59c640 1 -- 172.21.15.45:0/3517237256 learned_addr learned my addr 172.21.15.45:0/3517237256 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005533441 0 0) 0x7f259c097340 con 0x7f259c0a1080 2024-09-18T14:08:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient(hunting): my global_id is 5208 2024-09-18T14:08:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25840034e0 con 0x7f259c0a1080 2024-09-18T14:08:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307599904 0 0) 0x7f259c097620 con 0x7f259c06e810 2024-09-18T14:08:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient(hunting): my global_id is 144632 2024-09-18T14:08:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f259c097340 con 0x7f259c06e810 2024-09-18T14:08:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155791974 0 0) 0x7f259c05ed70 con 0x7f259c0a0cb0 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient(hunting): my global_id is 164116 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f259c097620 con 0x7f259c0a0cb0 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1123898044 0 0) 0x7f25840034e0 con 0x7f259c0a1080 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 >> v1:172.21.15.45:6790/0 conn(0x7f259c06e810 legacy=0x7f259c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 >> v1:172.21.15.84:6789/0 conn(0x7f259c0a0cb0 legacy=0x7f259c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient: _finish_hunting 0 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 1 monclient: found mon.noname-d 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c0a1660 con 0x7f259c0a1080 2024-09-18T14:08:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient: _finish_auth 0 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 5 monclient: authenticate success, global_id 5208 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2594001bb0 con 0x7f259c0a1080 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f259959a640 10 monclient: dump: 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.712 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.713 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.697+0000 7f25a0b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2594001d90 con 0x7f259c0a1080 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f259959a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f259959a640 1 -- 172.21.15.45:0/3517237256 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f25940022c0 con 0x7f259c0a1080 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f25a0b5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f25a0b5d640 4 set_mon_vals no callback set 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f259959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f259959a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f259959a640 10 monclient: dump: 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f25a0b5d640 10 monclient: shutdown 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f25a0b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f25a0b5d640 1 -- 172.21.15.45:0/3517237256 >> v1:172.21.15.45:6792/0 conn(0x7f259c0a1080 legacy=0x7f259c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f25a0b5d640 1 -- 172.21.15.45:0/3517237256 shutdown_connections 2024-09-18T14:08:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f25a0b5d640 1 -- 172.21.15.45:0/3517237256 >> 172.21.15.45:0/3517237256 conn(0x7f259c0a5bb0 msgr2=0x7f259c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f25a0b5d640 1 -- 172.21.15.45:0/3517237256 shutdown_connections 2024-09-18T14:08:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.698+0000 7f25a0b5d640 1 -- 172.21.15.45:0/3517237256 wait complete. 2024-09-18T14:08:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 10 monclient: build_initial_monmap 2024-09-18T14:08:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 10 monclient: monmap: 2024-09-18T14:08:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 1 Processor -- start 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 1 -- start start 2024-09-18T14:08:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 10 monclient: init 2024-09-18T14:08:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 10 monclient: _add_conns ranks=[1,2,4,5,0,8,7,3,6] 2024-09-18T14:08:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 10 monclient(hunting): picked mon.noname-f con 0x7f259c06e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 10 monclient(hunting): picked mon.noname-b con 0x7f259c0a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 10 monclient(hunting): picked mon.noname-c con 0x7f259c0a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c122d80 con 0x7f259c06e810 2024-09-18T14:08:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c13db70 con 0x7f259c0a0cb0 2024-09-18T14:08:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c1227c0 con 0x7f259c0a1080 2024-09-18T14:08:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.699+0000 7f25a0b5d640 10 monclient(hunting): authenticate will time out at 1297.037979s 2024-09-18T14:08:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f2599d9b640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f259c0a0cb0 0x7f259c146690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47164/0 (socket says 172.21.15.45:47164) 2024-09-18T14:08:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f2599d9b640 1 -- 172.21.15.45:0/1026024867 learned_addr learned my addr 172.21.15.45:0/1026024867 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930771871 0 0) 0x7f259c13db70 con 0x7f259c0a0cb0 2024-09-18T14:08:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): my global_id is 144641 2024-09-18T14:08:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f257c0034e0 con 0x7f259c0a0cb0 2024-09-18T14:08:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916641496 0 0) 0x7f259c122d80 con 0x7f259c06e810 2024-09-18T14:08:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): my global_id is 164125 2024-09-18T14:08:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f259c13db70 con 0x7f259c06e810 2024-09-18T14:08:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983326902 0 0) 0x7f259c1227c0 con 0x7f259c0a1080 2024-09-18T14:08:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): my global_id is 164155 2024-09-18T14:08:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f259c122d80 con 0x7f259c0a1080 2024-09-18T14:08:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3610937641 0 0) 0x7f259c13db70 con 0x7f259c06e810 2024-09-18T14:08:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f259c1227c0 con 0x7f259c06e810 2024-09-18T14:08:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f258c001480 con 0x7f259c06e810 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): dump: 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.722 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4289348725 0 0) 0x7f257c0034e0 con 0x7f259c0a0cb0 2024-09-18T14:08:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f259c13db70 con 0x7f259c0a0cb0 2024-09-18T14:08:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2590002b70 con 0x7f259c0a0cb0 2024-09-18T14:08:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): dump: 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.724 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1355586655 0 0) 0x7f259c122d80 con 0x7f259c0a1080 2024-09-18T14:08:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f257c0034e0 con 0x7f259c0a1080 2024-09-18T14:08:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f25940013a0 con 0x7f259c0a1080 2024-09-18T14:08:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient(hunting): dump: 2024-09-18T14:08:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.725 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3580158055 0 0) 0x7f259c1227c0 con 0x7f259c06e810 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 >> v1:172.21.15.45:6791/0 conn(0x7f259c0a1080 legacy=0x7f259c149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 >> v1:172.21.15.45:6790/0 conn(0x7f259c0a0cb0 legacy=0x7f259c146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 monclient: found mon.f 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c120310 con 0x7f259c06e810 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f25a0b5d640 5 monclient: authenticate success, global_id 164125 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f25a0b5d640 10 monclient: _renew_subs 2024-09-18T14:08:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f25a0b5d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f25a0b5d640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c1224e0 con 0x7f259c06e810 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f25a0b5d640 10 monclient: _renew_subs 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f25a0b5d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f25a0b5d640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c121610 con 0x7f259c06e810 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f258c0017b0 con 0x7f259c06e810 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f258c004e10 con 0x7f259c06e810 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f258a7fc640 10 monclient: dump: 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.700+0000 7f2598d99640 4 set_mon_vals no callback set 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.701+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f258c004ff0 con 0x7f259c06e810 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.701+0000 7f25a0b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.701+0000 7f25a0b5d640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f259c04c4a0 con 0x7f259c13e710 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.701+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(426..426 src has 126..426) v4 ==== 3959+0+0 (unknown 169183747 0 0) 0x7f258c046b40 con 0x7f259c06e810 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f259ad9d640 10 monclient: get_auth_request con 0x7f257c03aa20 auth_method 0 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128408628 0 0) 0x7f259c04c4a0 con 0x7f259c13e710 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 10 monclient(hunting): my global_id is 5282 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f259c1227c0 con 0x7f259c13e710 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3670418971 0 0) 0x7f259c1227c0 con 0x7f259c13e710 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f259c04c4a0 con 0x7f259c13e710 2024-09-18T14:08:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2590001a20 con 0x7f259c13e710 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 10 monclient: dump: 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1863464058 0 0) 0x7f259c04c4a0 con 0x7f259c13e710 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f259c04b720 con 0x7f259c13e710 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2590001c50 con 0x7f259c13e710 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 >> v1:172.21.15.45:6793/0 conn(0x7f259c13e710 legacy=0x7f259c09bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.702+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 >> v1:172.21.15.45:6793/0 conn(0x7f259c13e710 legacy=0x7f259c09bb80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:01.735 INFO:tasks.rados.rados.0.smithi084.stdout:1944: writing smithi08434663-22 from 679181 to 1229665 tid 1 2024-09-18T14:08:01.735 INFO:tasks.rados.rados.0.smithi084.stdout:1944: writing smithi08434663-22 from 1667672 to 2204575 tid 2 2024-09-18T14:08:01.735 INFO:tasks.rados.rados.0.smithi084.stdout:1944: writing smithi08434663-22 from 2839904 to 3275277 tid 3 2024-09-18T14:08:01.735 INFO:tasks.rados.rados.0.smithi084.stdout:1945: read oid 10 snap 202 2024-09-18T14:08:01.735 INFO:tasks.rados.rados.0.smithi084.stdout:1945: expect (ObjNum 677 snap 201 seq_num 677) 2024-09-18T14:08:01.735 INFO:tasks.rados.rados.0.smithi084.stdout:1946: delete oid 42 current snap is 203 2024-09-18T14:08:01.735 INFO:tasks.rados.rados.0.smithi084.stdout:1944: finishing write tid 1 to smithi08434663-22 2024-09-18T14:08:01.735 INFO:tasks.rados.rados.0.smithi084.stdout:1944: finishing write tid 2 to smithi08434663-22 2024-09-18T14:08:01.735 INFO:tasks.rados.rados.0.smithi084.stdout:1944: finishing write tid 3 to smithi08434663-22 2024-09-18T14:08:01.736 INFO:tasks.rados.rados.0.smithi084.stdout:1943: done (3 left) 2024-09-18T14:08:01.736 INFO:tasks.rados.rados.0.smithi084.stdout:1946: done (2 left) 2024-09-18T14:08:01.736 INFO:tasks.rados.rados.0.smithi084.stdout:1947: read oid 5 snap -1 2024-09-18T14:08:01.736 INFO:tasks.rados.rados.0.smithi084.stdout:1947: expect (ObjNum 678 snap 201 seq_num 678) 2024-09-18T14:08:01.736 INFO:tasks.rados.rados.0.smithi084.stdout:1948: snap_create 2024-09-18T14:08:01.736 INFO:tasks.rados.rados.0.smithi084.stdout:1944: finishing write tid 4 to smithi08434663-22 2024-09-18T14:08:01.736 INFO:tasks.rados.rados.0.smithi084.stdout:1944: finishing write tid 5 to smithi08434663-22 2024-09-18T14:08:01.736 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 736 (ObjNum 683 snap 203 seq_num 683) dirty exists 2024-09-18T14:08:01.736 INFO:tasks.rados.rados.0.smithi084.stdout:1944: left oid 22 (ObjNum 683 snap 203 seq_num 683) 2024-09-18T14:08:01.736 INFO:tasks.rados.rados.0.smithi084.stdout:1945: expect (ObjNum 677 snap 201 seq_num 677) 2024-09-18T14:08:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.708+0000 7f25a0b5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.708+0000 7f25a0b5d640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f259c13e160 con 0x7f259c13dd90 2024-09-18T14:08:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304386046 0 0) 0x7f259c13e160 con 0x7f259c13dd90 2024-09-18T14:08:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 10 monclient(hunting): my global_id is 5291 2024-09-18T14:08:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f259c04c4a0 con 0x7f259c13dd90 2024-09-18T14:08:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2056437831 0 0) 0x7f259c04c4a0 con 0x7f259c13dd90 2024-09-18T14:08:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f259c13e160 con 0x7f259c13dd90 2024-09-18T14:08:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2590002150 con 0x7f259c13dd90 2024-09-18T14:08:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 10 monclient: dump: 2024-09-18T14:08:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.738 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.739 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2992600507 0 0) 0x7f259c13e160 con 0x7f259c13dd90 2024-09-18T14:08:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.709+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f259c062dd0 con 0x7f259c13dd90 2024-09-18T14:08:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7558 (unknown 0 0 3471080835) 0x7f2590007d90 con 0x7f259c13dd90 2024-09-18T14:08:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f258a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f258a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 >> v1:172.21.15.45:6793/0 conn(0x7f259c13dd90 legacy=0x7f259c09c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f258a7fc640 1 -- 172.21.15.45:0/1026024867 >> v1:172.21.15.45:6793/0 conn(0x7f259c13dd90 legacy=0x7f259c09c290 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:01.740 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:01.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:01.740 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:01.740 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:08:01.740 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:01.740 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 2, 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:01.741 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:01.742 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.742 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.742 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.742 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.742 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.742 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.742 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.742 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.742 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.742 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:01.744 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:01.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:01.745 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:01.745 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:01.745 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:01.745 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.745 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.745 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.745 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.745 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.745 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.745 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.745 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.746 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:01.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:01.748 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.748 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.748 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:01.748 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.748 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.748 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.748 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:01.748 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:01.748 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.750 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.752 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:01.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:01.752 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.752 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.752 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:01.752 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.752 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.753 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:01.753 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:01.753 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.753 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.753 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.753 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:01.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:01.753 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.753 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.754 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:01.754 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.754 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.754 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:01.754 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:01.754 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.754 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.754 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.755 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:01.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:01.755 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.755 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.755 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.755 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:01.755 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.755 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.756 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.757 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:01.757 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.757 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.757 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.757 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:01.757 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:01.757 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.757 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.757 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.759 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.760 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.761 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:08:01.761 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.761 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.764 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:01.764 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:01.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f25a0b5d640 1 -- 172.21.15.45:0/1026024867 >> v1:172.21.15.84:6812/31478 conn(0x7f257c03aa20 legacy=0x7f257c03ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f25a0b5d640 10 monclient: shutdown 2024-09-18T14:08:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f25a0b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f25a0b5d640 1 -- 172.21.15.45:0/1026024867 >> v1:172.21.15.84:6789/0 conn(0x7f259c06e810 legacy=0x7f259c13d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f259ad9d640 1 -- 172.21.15.45:0/1026024867 reap_dead start 2024-09-18T14:08:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f259ad9d640 1 -- 172.21.15.45:0/1026024867 reap_dead start 2024-09-18T14:08:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f25a0b5d640 1 -- 172.21.15.45:0/1026024867 shutdown_connections 2024-09-18T14:08:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f25a0b5d640 1 -- 172.21.15.45:0/1026024867 >> 172.21.15.45:0/1026024867 conn(0x7f259c0a5bb0 msgr2=0x7f259c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f25a0b5d640 1 -- 172.21.15.45:0/1026024867 shutdown_connections 2024-09-18T14:08:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.710+0000 7f25a0b5d640 1 -- 172.21.15.45:0/1026024867 wait complete. 2024-09-18T14:08:01.767 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:01.767 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:08:01.767 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:08:01.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.864+0000 7f20b3900640 10 monclient: get_monmap_and_config 2024-09-18T14:08:01.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.864+0000 7f20b3900640 10 monclient: build_initial_monmap 2024-09-18T14:08:01.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.864+0000 7f20b3900640 10 monclient: monmap: 2024-09-18T14:08:01.865 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:01.865 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.865 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:01.865044+0000 2024-09-18T14:08:01.865 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:01.865044+0000 2024-09-18T14:08:01.865 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:01.866 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.864+0000 7f20b3900640 1 Processor -- start 2024-09-18T14:08:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.864+0000 7f20b3900640 1 -- start start 2024-09-18T14:08:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.864+0000 7f20b3900640 10 monclient: init 2024-09-18T14:08:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20b3900640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20b3900640 10 monclient: _add_conns ranks=[7,5,2,6,3,4,8,0,1] 2024-09-18T14:08:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20b3900640 10 monclient(hunting): picked mon.noname-i con 0x7f20ac09f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20b3900640 10 monclient(hunting): picked mon.noname-h con 0x7f20ac09fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20b3900640 10 monclient(hunting): picked mon.noname-b con 0x7f20ac09ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20b3900640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac05ed70 con 0x7f20ac09ff70 2024-09-18T14:08:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20b3900640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac09dfc0 con 0x7f20ac09fba0 2024-09-18T14:08:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20b3900640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac09dce0 con 0x7f20ac09f7d0 2024-09-18T14:08:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20b3900640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20b3900640 10 monclient(hunting): authenticate will time out at 1297.203389s 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20b1e9d640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f20ac09ff70 0x7f20ac13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47170/0 (socket says 172.21.15.45:47170) 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20b1e9d640 1 -- 172.21.15.45:0/2238490688 learned_addr learned my addr 172.21.15.45:0/2238490688 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187332197 0 0) 0x7f20ac05ed70 con 0x7f20ac09ff70 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 10 monclient(hunting): my global_id is 144650 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20980034e0 con 0x7f20ac09ff70 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601649648 0 0) 0x7f20ac09dce0 con 0x7f20ac09f7d0 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 10 monclient(hunting): my global_id is 5326 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac05ed70 con 0x7f20ac09f7d0 2024-09-18T14:08:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320365106 0 0) 0x7f20ac09dfc0 con 0x7f20ac09fba0 2024-09-18T14:08:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 10 monclient(hunting): my global_id is 5144 2024-09-18T14:08:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.865+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac09dce0 con 0x7f20ac09fba0 2024-09-18T14:08:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4054898065 0 0) 0x7f20980034e0 con 0x7f20ac09ff70 2024-09-18T14:08:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 >> v1:172.21.15.84:6792/0 conn(0x7f20ac09f7d0 legacy=0x7f20ac132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 >> v1:172.21.15.84:6791/0 conn(0x7f20ac09fba0 legacy=0x7f20ac137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 10 monclient: _finish_hunting 0 2024-09-18T14:08:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 1 monclient: found mon.noname-b 2024-09-18T14:08:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:08:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac0a1c00 con 0x7f20ac09ff70 2024-09-18T14:08:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 10 monclient: _finish_auth 0 2024-09-18T14:08:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20b3900640 5 monclient: authenticate success, global_id 144650 2024-09-18T14:08:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20b3900640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f20a4001830 con 0x7f20ac09ff70 2024-09-18T14:08:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 10 monclient: dump: 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20b3900640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f20a4001a10 con 0x7f20ac09ff70 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 1 -- 172.21.15.45:0/2238490688 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f20a4004ba0 con 0x7f20ac09ff70 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20b3900640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20b3900640 4 set_mon_vals no callback set 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20a3fff640 10 monclient: dump: 2024-09-18T14:08:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.886 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20b3900640 10 monclient: shutdown 2024-09-18T14:08:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20b3900640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20b3900640 1 -- 172.21.15.45:0/2238490688 >> v1:172.21.15.45:6790/0 conn(0x7f20ac09ff70 legacy=0x7f20ac13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20b3900640 1 -- 172.21.15.45:0/2238490688 shutdown_connections 2024-09-18T14:08:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20b3900640 1 -- 172.21.15.45:0/2238490688 >> 172.21.15.45:0/2238490688 conn(0x7f20ac0630b0 msgr2=0x7f20ac0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20b3900640 1 -- 172.21.15.45:0/2238490688 shutdown_connections 2024-09-18T14:08:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.866+0000 7f20b3900640 1 -- 172.21.15.45:0/2238490688 wait complete. 2024-09-18T14:08:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.867+0000 7f20b3900640 10 monclient: build_initial_monmap 2024-09-18T14:08:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.867+0000 7f20b3900640 10 monclient: monmap: 2024-09-18T14:08:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:01.888 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.867+0000 7f20b3900640 1 Processor -- start 2024-09-18T14:08:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.867+0000 7f20b3900640 1 -- start start 2024-09-18T14:08:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.867+0000 7f20b3900640 10 monclient: init 2024-09-18T14:08:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20b3900640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20b3900640 10 monclient: _add_conns ranks=[4,3,0,7,6,1,2,5,8] 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20b3900640 10 monclient(hunting): picked mon.noname-c con 0x7f20ac09f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20b3900640 10 monclient(hunting): picked mon.noname-g con 0x7f20ac09fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20b3900640 10 monclient(hunting): picked mon.noname-a con 0x7f20ac09ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20b3900640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac09d310 con 0x7f20ac09ff70 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20b3900640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac0a2560 con 0x7f20ac09fba0 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20b3900640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac121e90 con 0x7f20ac09f7d0 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20b3900640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20b3900640 10 monclient(hunting): authenticate will time out at 1297.206414s 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20b169c640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f20ac09f7d0 0x7f20ac141990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:50432/0 (socket says 172.21.15.45:50432) 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20b169c640 1 -- 172.21.15.45:0/579832682 learned_addr learned my addr 172.21.15.45:0/579832682 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141765793 0 0) 0x7f20ac121e90 con 0x7f20ac09f7d0 2024-09-18T14:08:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 10 monclient(hunting): my global_id is 164164 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20900034e0 con 0x7f20ac09f7d0 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933107585 0 0) 0x7f20ac09d310 con 0x7f20ac09ff70 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 10 monclient(hunting): my global_id is 5382 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac121e90 con 0x7f20ac09ff70 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847277764 0 0) 0x7f20ac0a2560 con 0x7f20ac09fba0 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 10 monclient(hunting): my global_id is 154317 2024-09-18T14:08:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac09d310 con 0x7f20ac09fba0 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 251212472 0 0) 0x7f20900034e0 con 0x7f20ac09f7d0 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.868+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20ac0a2560 con 0x7f20ac09f7d0 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f20a8002f90 con 0x7f20ac09f7d0 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient(hunting): dump: 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 177303216 0 0) 0x7f20ac121e90 con 0x7f20ac09ff70 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20900034e0 con 0x7f20ac09ff70 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1237019689 0 0) 0x7f20ac09d310 con 0x7f20ac09fba0 2024-09-18T14:08:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20ac121e90 con 0x7f20ac09fba0 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f20a4001770 con 0x7f20ac09ff70 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient(hunting): dump: 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f209c001920 con 0x7f20ac09fba0 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient(hunting): dump: 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3142200913 0 0) 0x7f20ac0a2560 con 0x7f20ac09f7d0 2024-09-18T14:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 >> v1:172.21.15.84:6790/0 conn(0x7f20ac09fba0 legacy=0x7f20ac1319d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 >> v1:172.21.15.45:6789/0 conn(0x7f20ac09ff70 legacy=0x7f20ac149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient: _finish_hunting 0 2024-09-18T14:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 monclient: found mon.c 2024-09-18T14:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac120060 con 0x7f20ac09f7d0 2024-09-18T14:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient: _finish_auth 0 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20b3900640 5 monclient: authenticate success, global_id 164164 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20b3900640 10 monclient: _renew_subs 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20b3900640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20b3900640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ac121b20 con 0x7f20ac09f7d0 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20b3900640 10 monclient: _renew_subs 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20b3900640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20b3900640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ac1420a0 con 0x7f20ac09f7d0 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f20a8001b20 con 0x7f20ac09f7d0 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f20a8002010 con 0x7f20ac09f7d0 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a37fe640 4 set_mon_vals no callback set 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.869+0000 7f20a17fa640 10 monclient: dump: 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f20a80021f0 con 0x7f20ac09f7d0 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20b3900640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20b3900640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f208c006580 con 0x7f208c002830 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(426..426 src has 126..426) v4 ==== 3959+0+0 (unknown 169183747 0 0) 0x7f20a8046f90 con 0x7f20ac09f7d0 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20b1e9d640 10 monclient: get_auth_request con 0x7f209003aa50 auth_method 0 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927001277 0 0) 0x7f208c006580 con 0x7f208c002830 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 10 monclient(hunting): my global_id is 164134 2024-09-18T14:08:01.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac0a2560 con 0x7f208c002830 2024-09-18T14:08:01.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2188349041 0 0) 0x7f20ac0a2560 con 0x7f208c002830 2024-09-18T14:08:01.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f208c006580 con 0x7f208c002830 2024-09-18T14:08:01.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f209c002070 con 0x7f208c002830 2024-09-18T14:08:01.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:08:01.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.870+0000 7f20a17fa640 10 monclient: dump: 2024-09-18T14:08:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.871+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2428743269 0 0) 0x7f208c006580 con 0x7f208c002830 2024-09-18T14:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.871+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f208c00a0a0 con 0x7f208c002830 2024-09-18T14:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.871+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f209c004350 con 0x7f208c002830 2024-09-18T14:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.871+0000 7f20a17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.871+0000 7f20a17fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.871+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 >> v1:172.21.15.84:6789/0 conn(0x7f208c002830 legacy=0x7f208c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.871+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 >> v1:172.21.15.84:6789/0 conn(0x7f208c002830 legacy=0x7f208c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20b3900640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20b3900640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f208c0074f0 con 0x7f208c0068a0 2024-09-18T14:08:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955870222 0 0) 0x7f208c0074f0 con 0x7f208c0068a0 2024-09-18T14:08:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 10 monclient(hunting): my global_id is 164143 2024-09-18T14:08:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f208c006580 con 0x7f208c0068a0 2024-09-18T14:08:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2309607292 0 0) 0x7f208c006580 con 0x7f208c0068a0 2024-09-18T14:08:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f208c0074f0 con 0x7f208c0068a0 2024-09-18T14:08:01.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f209c002b30 con 0x7f208c0068a0 2024-09-18T14:08:01.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:01.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:08:01.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 10 monclient: dump: 2024-09-18T14:08:01.905 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:01.905 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:01.905 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1559869897 0 0) 0x7f208c0074f0 con 0x7f208c0068a0 2024-09-18T14:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.877+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f208c008750 con 0x7f208c0068a0 2024-09-18T14:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 1843965333) 0x7f209c003f60 con 0x7f208c0068a0 2024-09-18T14:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20a17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20a17fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 >> v1:172.21.15.84:6789/0 conn(0x7f208c0068a0 legacy=0x7f208c006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20a17fa640 1 -- 172.21.15.45:0/579832682 >> v1:172.21.15.84:6789/0 conn(0x7f208c0068a0 legacy=0x7f208c006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:01.908 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:01.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:01.909 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:01.909 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:01.909 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:01.909 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:01.909 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:01.909 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:01.909 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:01.910 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:01.910 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:01.910 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:01.910 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:01.910 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:01.910 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:01.910 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.910 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:08:01.911 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:01.911 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:01.911 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:01.911 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.911 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.911 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.911 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.911 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.912 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.912 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.912 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.912 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.912 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.912 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:01.912 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:01.912 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.913 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.913 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.913 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.913 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.913 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.913 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.913 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.914 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.914 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.914 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:01.914 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:01.914 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:01.914 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:01.914 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:01.915 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:01.915 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:01.915 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:01.915 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:01.915 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:01.915 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:01.915 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:01.916 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:01.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:01.916 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:01.916 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:01.916 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:01.916 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:01.916 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:01.916 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:01.916 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:01.916 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:01.916 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:01.917 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:01.917 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:01.917 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:01.917 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.917 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:01.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.917 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:01.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.917 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:01.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:01.917 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:01.918 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.918 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.918 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:01.918 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:01.918 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.918 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.919 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.919 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:01.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:01.919 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:01.919 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.920 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.920 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:01.920 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:01.920 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.920 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.920 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.920 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:01.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:01.925 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:01.926 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.926 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.926 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.926 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:01.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:01.926 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.926 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.926 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:01.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:01.928 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.928 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.928 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:01.928 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.928 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.928 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:01.928 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:01.928 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.929 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.929 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.929 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:01.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:01.929 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:01.929 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:01.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.931 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:01.931 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:01.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.931 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.931 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.931 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:01.931 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.931 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:01.931 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:01.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:01.931 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:01.931 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:01.932 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:01.932 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:01.932 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:01.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:01.932 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:01.932 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:01.933 INFO:tasks.rados.rados.0.smithi084.stdout:1947: expect (ObjNum 678 snap 201 seq_num 678) 2024-09-18T14:08:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20b3900640 1 -- 172.21.15.45:0/579832682 >> v1:172.21.15.84:6812/31478 conn(0x7f209003aa50 legacy=0x7f209003cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20b3900640 10 monclient: shutdown 2024-09-18T14:08:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20b3900640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20b3900640 1 -- 172.21.15.45:0/579832682 >> v1:172.21.15.45:6791/0 conn(0x7f20ac09f7d0 legacy=0x7f20ac141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20b1e9d640 1 -- 172.21.15.45:0/579832682 reap_dead start 2024-09-18T14:08:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20b1e9d640 1 -- 172.21.15.45:0/579832682 reap_dead start 2024-09-18T14:08:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20b3900640 1 -- 172.21.15.45:0/579832682 shutdown_connections 2024-09-18T14:08:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20b3900640 1 -- 172.21.15.45:0/579832682 >> 172.21.15.45:0/579832682 conn(0x7f20ac0630b0 msgr2=0x7f20ac0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20b3900640 1 -- 172.21.15.45:0/579832682 shutdown_connections 2024-09-18T14:08:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:01.878+0000 7f20b3900640 1 -- 172.21.15.45:0/579832682 wait complete. 2024-09-18T14:08:01.935 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:01.935 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:08:01.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:08:02.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.032+0000 7fef6b0ef640 10 monclient: get_monmap_and_config 2024-09-18T14:08:02.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.032+0000 7fef6b0ef640 10 monclient: build_initial_monmap 2024-09-18T14:08:02.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.032+0000 7fef6b0ef640 10 monclient: monmap: 2024-09-18T14:08:02.033 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:02.033 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.033 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:02.033361+0000 2024-09-18T14:08:02.033 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:02.033361+0000 2024-09-18T14:08:02.033 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:02.033 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.033 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:02.033 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:02.034 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:02.034 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:02.034 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:02.034 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:02.034 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:02.034 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:02.034 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:02.034 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 1 Processor -- start 2024-09-18T14:08:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 1 -- start start 2024-09-18T14:08:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 10 monclient: init 2024-09-18T14:08:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 10 monclient: _add_conns ranks=[6,0,2,8,7,3,5,1,4] 2024-09-18T14:08:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 10 monclient(hunting): picked mon.noname-d con 0x7fef6409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 10 monclient(hunting): picked mon.noname-a con 0x7fef6409fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 10 monclient(hunting): picked mon.noname-b con 0x7fef6409ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6405ed70 con 0x7fef6409fba0 2024-09-18T14:08:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6409dfc0 con 0x7fef6409ff70 2024-09-18T14:08:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6409dce0 con 0x7fef6409f7d0 2024-09-18T14:08:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6b0ef640 10 monclient(hunting): authenticate will time out at 1297.371726s 2024-09-18T14:08:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef68e8b640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fef6409f7d0 0x7fef64132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40874/0 (socket says 172.21.15.45:40874) 2024-09-18T14:08:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef63fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fef6409fba0 0x7fef64137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54956/0 (socket says 172.21.15.45:54956) 2024-09-18T14:08:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef68e8b640 1 -- 172.21.15.45:0/3303257891 learned_addr learned my addr 172.21.15.45:0/3303257891 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.033+0000 7fef6968c640 1 --1- 172.21.15.45:0/3303257891 >> v1:172.21.15.45:6790/0 conn(0x7fef6409ff70 0x7fef6413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47174/0 (socket says 172.21.15.45:47174) 2024-09-18T14:08:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700124557 0 0) 0x7fef6405ed70 con 0x7fef6409fba0 2024-09-18T14:08:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient(hunting): my global_id is 5391 2024-09-18T14:08:02.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef540034e0 con 0x7fef6409fba0 2024-09-18T14:08:02.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737198717 0 0) 0x7fef6409dfc0 con 0x7fef6409ff70 2024-09-18T14:08:02.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient(hunting): my global_id is 144659 2024-09-18T14:08:02.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef6405ed70 con 0x7fef6409ff70 2024-09-18T14:08:02.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959978370 0 0) 0x7fef6409dce0 con 0x7fef6409f7d0 2024-09-18T14:08:02.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient(hunting): my global_id is 5217 2024-09-18T14:08:02.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef6409dfc0 con 0x7fef6409f7d0 2024-09-18T14:08:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3340779186 0 0) 0x7fef6409dfc0 con 0x7fef6409f7d0 2024-09-18T14:08:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 >> v1:172.21.15.45:6790/0 conn(0x7fef6409ff70 legacy=0x7fef6413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 >> v1:172.21.15.45:6789/0 conn(0x7fef6409fba0 legacy=0x7fef64137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient: _finish_hunting 0 2024-09-18T14:08:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 monclient: found mon.noname-d 2024-09-18T14:08:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:08:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef640a1c00 con 0x7fef6409f7d0 2024-09-18T14:08:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient: _finish_auth 0 2024-09-18T14:08:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fef58001600 con 0x7fef6409f7d0 2024-09-18T14:08:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:08:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient: dump: 2024-09-18T14:08:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.054 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.054 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.054 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.054 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.054 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.054 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.055 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.055 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.055 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.055 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.055 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.055 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.056 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.056 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.056 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef6b0ef640 5 monclient: authenticate success, global_id 5217 2024-09-18T14:08:02.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef6b0ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:02.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fef58001760 con 0x7fef6409f7d0 2024-09-18T14:08:02.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:02.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef6b0ef640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:02.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 1 -- 172.21.15.45:0/3303257891 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fef58001d10 con 0x7fef6409f7d0 2024-09-18T14:08:02.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef6b0ef640 4 set_mon_vals no callback set 2024-09-18T14:08:02.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:02.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.034+0000 7fef637fe640 10 monclient: dump: 2024-09-18T14:08:02.058 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.058 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.058 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.058 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.058 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.060 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.060 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.060 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.060 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.060 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.060 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.061 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.061 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.061 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.061 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.061 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.035+0000 7fef6b0ef640 10 monclient: shutdown 2024-09-18T14:08:02.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.035+0000 7fef6b0ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:02.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.035+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3303257891 >> v1:172.21.15.45:6792/0 conn(0x7fef6409f7d0 legacy=0x7fef64132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.035+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3303257891 shutdown_connections 2024-09-18T14:08:02.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.035+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3303257891 >> 172.21.15.45:0/3303257891 conn(0x7fef640630b0 msgr2=0x7fef640634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:02.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.035+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3303257891 shutdown_connections 2024-09-18T14:08:02.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.035+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3303257891 wait complete. 2024-09-18T14:08:02.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.035+0000 7fef6b0ef640 10 monclient: build_initial_monmap 2024-09-18T14:08:02.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.035+0000 7fef6b0ef640 10 monclient: monmap: 2024-09-18T14:08:02.063 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:02.063 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:02.064 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:02.064 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:02.064 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:02.064 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:02.064 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:02.065 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:02.065 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:02.065 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:02.065 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:02.065 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:02.065 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:02.066 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:02.066 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:02.066 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 1 Processor -- start 2024-09-18T14:08:02.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 1 -- start start 2024-09-18T14:08:02.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 10 monclient: init 2024-09-18T14:08:02.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:02.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 10 monclient: _add_conns ranks=[6,3,8,4,5,1,7,0,2] 2024-09-18T14:08:02.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 10 monclient(hunting): picked mon.noname-d con 0x7fef6409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:02.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 10 monclient(hunting): picked mon.noname-g con 0x7fef6409fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:02.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 10 monclient(hunting): picked mon.noname-e con 0x7fef6409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:02.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6409d310 con 0x7fef6409fba0 2024-09-18T14:08:02.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef640a2560 con 0x7fef6409f7d0 2024-09-18T14:08:02.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef641425c0 con 0x7fef6409ff70 2024-09-18T14:08:02.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:02.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef6b0ef640 10 monclient(hunting): authenticate will time out at 1297.374796s 2024-09-18T14:08:02.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef68e8b640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fef6409f7d0 0x7fef64131660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40878/0 (socket says 172.21.15.45:40878) 2024-09-18T14:08:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.036+0000 7fef68e8b640 1 -- 172.21.15.45:0/3148828930 learned_addr learned my addr 172.21.15.45:0/3148828930 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717987214 0 0) 0x7fef641425c0 con 0x7fef6409ff70 2024-09-18T14:08:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): my global_id is 5300 2024-09-18T14:08:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef480034e0 con 0x7fef6409ff70 2024-09-18T14:08:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 587637039 0 0) 0x7fef640a2560 con 0x7fef6409f7d0 2024-09-18T14:08:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): my global_id is 5226 2024-09-18T14:08:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef641425c0 con 0x7fef6409f7d0 2024-09-18T14:08:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1012742845 0 0) 0x7fef480034e0 con 0x7fef6409ff70 2024-09-18T14:08:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fef640a2560 con 0x7fef6409ff70 2024-09-18T14:08:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fef5c001bf0 con 0x7fef6409ff70 2024-09-18T14:08:02.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:08:02.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): dump: 2024-09-18T14:08:02.071 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.071 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.071 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.071 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.071 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.071 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.072 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.072 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.072 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.072 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.072 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.072 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.072 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.072 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.073 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2467721443 0 0) 0x7fef641425c0 con 0x7fef6409f7d0 2024-09-18T14:08:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fef480034e0 con 0x7fef6409f7d0 2024-09-18T14:08:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fef58001c10 con 0x7fef6409f7d0 2024-09-18T14:08:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): dump: 2024-09-18T14:08:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.075 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2517616047 0 0) 0x7fef6409d310 con 0x7fef6409fba0 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient(hunting): my global_id is 154326 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef641425c0 con 0x7fef6409fba0 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1001662118 0 0) 0x7fef640a2560 con 0x7fef6409ff70 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 >> v1:172.21.15.45:6792/0 conn(0x7fef6409f7d0 legacy=0x7fef64131660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 >> v1:172.21.15.84:6790/0 conn(0x7fef6409fba0 legacy=0x7fef64146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 monclient: found mon.e 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:08:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef641208c0 con 0x7fef6409ff70 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient: _finish_auth 0 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef6b0ef640 5 monclient: authenticate success, global_id 5300 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef6b0ef640 10 monclient: _renew_subs 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef6b0ef640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef641217b0 con 0x7fef6409ff70 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef6b0ef640 10 monclient: _renew_subs 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef6b0ef640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fef64133bf0 con 0x7fef6409ff70 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fef5c004b20 con 0x7fef6409ff70 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fef5c004fd0 con 0x7fef6409ff70 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef62ffd640 4 set_mon_vals no callback set 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.037+0000 7fef60ff9640 10 monclient: dump: 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.038+0000 7fef6b0ef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.038+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fef44006580 con 0x7fef44002830 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.038+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fef5c00ffa0 con 0x7fef6409ff70 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.038+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(426..426 src has 126..426) v4 ==== 3959+0+0 (unknown 169183747 0 0) 0x7fef5c046b90 con 0x7fef6409ff70 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422961840 0 0) 0x7fef44006580 con 0x7fef44002830 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 10 monclient(hunting): my global_id is 154335 2024-09-18T14:08:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef640a2560 con 0x7fef44002830 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef63fff640 10 monclient: get_auth_request con 0x7fef4803a5c0 auth_method 0 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3911948901 0 0) 0x7fef640a2560 con 0x7fef44002830 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fef44006580 con 0x7fef44002830 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fef58000fa0 con 0x7fef44002830 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 10 monclient: dump: 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3966991792 0 0) 0x7fef44006580 con 0x7fef44002830 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef4400a0a0 con 0x7fef44002830 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fef58007160 con 0x7fef44002830 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:02.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 >> v1:172.21.15.84:6790/0 conn(0x7fef44002830 legacy=0x7fef44004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.039+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 >> v1:172.21.15.84:6790/0 conn(0x7fef44002830 legacy=0x7fef44004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:02.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef6b0ef640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:02.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fef44009e70 con 0x7fef440068f0 2024-09-18T14:08:02.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357543904 0 0) 0x7fef44009e70 con 0x7fef440068f0 2024-09-18T14:08:02.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 10 monclient(hunting): my global_id is 154344 2024-09-18T14:08:02.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef44006580 con 0x7fef440068f0 2024-09-18T14:08:02.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 766181334 0 0) 0x7fef44006580 con 0x7fef440068f0 2024-09-18T14:08:02.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fef44009e70 con 0x7fef440068f0 2024-09-18T14:08:02.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fef58005a40 con 0x7fef440068f0 2024-09-18T14:08:02.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:08:02.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 10 monclient: dump: 2024-09-18T14:08:02.084 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.084 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.084 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.085 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 721139279 0 0) 0x7fef44009e70 con 0x7fef440068f0 2024-09-18T14:08:02.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fef44008750 con 0x7fef440068f0 2024-09-18T14:08:02.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 496931927) 0x7fef58006e90 con 0x7fef440068f0 2024-09-18T14:08:02.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:02.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:02.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.046+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 >> v1:172.21.15.84:6790/0 conn(0x7fef440068f0 legacy=0x7fef44006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.047+0000 7fef60ff9640 1 -- 172.21.15.45:0/3148828930 >> v1:172.21.15.84:6790/0 conn(0x7fef440068f0 legacy=0x7fef44006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:02.086 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:02.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:02.086 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.087 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:02.088 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:02.090 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:02.091 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.092 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.094 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.094 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:02.094 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.094 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.094 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.094 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:02.094 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:02.094 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.094 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.094 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.095 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:02.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:02.095 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.095 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.095 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.095 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:02.095 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.095 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.095 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.096 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:02.096 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:02.096 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.096 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.096 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.096 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:02.096 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:02.097 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.097 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.097 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.097 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:02.097 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.097 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.097 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.097 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:02.098 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.100 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.100 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:02.100 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.100 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.100 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.100 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:02.100 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:02.100 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.101 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.103 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:02.103 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.103 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.103 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.103 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:02.103 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:02.103 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.104 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.105 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.105 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:02.105 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:02.105 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.105 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.105 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.105 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.105 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:02.105 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.106 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.107 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:02.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.107 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:02.107 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:02.107 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:02.107 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.107 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.107 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:02.107 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.047+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3148828930 >> v1:172.21.15.84:6812/31478 conn(0x7fef4803a5c0 legacy=0x7fef4803ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.047+0000 7fef6b0ef640 10 monclient: shutdown 2024-09-18T14:08:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.047+0000 7fef6b0ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.047+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3148828930 >> v1:172.21.15.45:6793/0 conn(0x7fef6409ff70 legacy=0x7fef6414a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.047+0000 7fef6968c640 1 -- 172.21.15.45:0/3148828930 reap_dead start 2024-09-18T14:08:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.047+0000 7fef6968c640 1 -- 172.21.15.45:0/3148828930 reap_dead start 2024-09-18T14:08:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.047+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3148828930 shutdown_connections 2024-09-18T14:08:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.047+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3148828930 >> 172.21.15.45:0/3148828930 conn(0x7fef640630b0 msgr2=0x7fef640a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.047+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3148828930 shutdown_connections 2024-09-18T14:08:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.047+0000 7fef6b0ef640 1 -- 172.21.15.45:0/3148828930 wait complete. 2024-09-18T14:08:02.110 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:02.110 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:08:02.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:08:02.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.207+0000 7f35d0657640 10 monclient: get_monmap_and_config 2024-09-18T14:08:02.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.207+0000 7f35d0657640 10 monclient: build_initial_monmap 2024-09-18T14:08:02.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.207+0000 7f35d0657640 10 monclient: monmap: 2024-09-18T14:08:02.208 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:02.208 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.208 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:02.208064+0000 2024-09-18T14:08:02.208 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:02.208064+0000 2024-09-18T14:08:02.208 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:02.208 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.208 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:02.208 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:02.208 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:02.209 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:02.209 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:02.209 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:02.209 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:02.209 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:02.209 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:02.209 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.207+0000 7f35d0657640 1 Processor -- start 2024-09-18T14:08:02.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.207+0000 7f35d0657640 1 -- start start 2024-09-18T14:08:02.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.207+0000 7f35d0657640 10 monclient: init 2024-09-18T14:08:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35d0657640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35d0657640 10 monclient: _add_conns ranks=[5,1,0,7,2,6,8,4,3] 2024-09-18T14:08:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35d0657640 10 monclient(hunting): picked mon.noname-h con 0x7f35c806e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35d0657640 10 monclient(hunting): picked mon.noname-f con 0x7f35c80a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35d0657640 10 monclient(hunting): picked mon.noname-a con 0x7f35c80a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35d0657640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c805ed70 con 0x7f35c80a1080 2024-09-18T14:08:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35d0657640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c8097620 con 0x7f35c80a0cb0 2024-09-18T14:08:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35d0657640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c8097340 con 0x7f35c806e810 2024-09-18T14:08:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35d0657640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35d0657640 10 monclient(hunting): authenticate will time out at 1297.546594s 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cebf4640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f35c80a1080 0x7f35c813af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54964/0 (socket says 172.21.15.45:54964) 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cebf4640 1 -- 172.21.15.45:0/3689843944 learned_addr learned my addr 172.21.15.45:0/3689843944 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35ce3f3640 1 --1- 172.21.15.45:0/3689843944 >> v1:172.21.15.84:6791/0 conn(0x7f35c806e810 0x7f35c80a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:57236/0 (socket says 172.21.15.45:57236) 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929980882 0 0) 0x7f35c805ed70 con 0x7f35c80a1080 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 10 monclient(hunting): my global_id is 5400 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35b40034e0 con 0x7f35c80a1080 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822042794 0 0) 0x7f35c8097340 con 0x7f35c806e810 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 10 monclient(hunting): my global_id is 5153 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c805ed70 con 0x7f35c806e810 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056369940 0 0) 0x7f35c8097620 con 0x7f35c80a0cb0 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 10 monclient(hunting): my global_id is 164152 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.208+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c8097340 con 0x7f35c80a0cb0 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1995058683 0 0) 0x7f35b40034e0 con 0x7f35c80a1080 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 >> v1:172.21.15.84:6791/0 conn(0x7f35c806e810 legacy=0x7f35c80a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 >> v1:172.21.15.84:6789/0 conn(0x7f35c80a0cb0 legacy=0x7f35c8137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 10 monclient: _finish_hunting 0 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 1 monclient: found mon.noname-a 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c80a1660 con 0x7f35c80a1080 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 10 monclient: _finish_auth 0 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35d0657640 5 monclient: authenticate success, global_id 5400 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f35c0001bb0 con 0x7f35c80a1080 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 10 monclient: dump: 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35d0657640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f35c0001d90 con 0x7f35c80a1080 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35d0657640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 1 -- 172.21.15.45:0/3689843944 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f35c0004f20 con 0x7f35c80a1080 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35d0657640 4 set_mon_vals no callback set 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35cd3f1640 10 monclient: dump: 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35d0657640 10 monclient: shutdown 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35d0657640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35d0657640 1 -- 172.21.15.45:0/3689843944 >> v1:172.21.15.45:6789/0 conn(0x7f35c80a1080 legacy=0x7f35c813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35d0657640 1 -- 172.21.15.45:0/3689843944 shutdown_connections 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35d0657640 1 -- 172.21.15.45:0/3689843944 >> 172.21.15.45:0/3689843944 conn(0x7f35c80a5bb0 msgr2=0x7f35c80a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35d0657640 1 -- 172.21.15.45:0/3689843944 shutdown_connections 2024-09-18T14:08:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.209+0000 7f35d0657640 1 -- 172.21.15.45:0/3689843944 wait complete. 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.210+0000 7f35d0657640 10 monclient: build_initial_monmap 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.210+0000 7f35d0657640 10 monclient: monmap: 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.210+0000 7f35d0657640 1 Processor -- start 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.210+0000 7f35d0657640 1 -- start start 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.210+0000 7f35d0657640 10 monclient: init 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35d0657640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35d0657640 10 monclient: _add_conns ranks=[8,6,4,5,7,1,0,3,2] 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35d0657640 10 monclient(hunting): picked mon.noname-e con 0x7f35c806e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35d0657640 10 monclient(hunting): picked mon.noname-d con 0x7f35c80a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35d0657640 10 monclient(hunting): picked mon.noname-c con 0x7f35c80a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35d0657640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c8122680 con 0x7f35c80a1080 2024-09-18T14:08:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35d0657640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c813e770 con 0x7f35c80a0cb0 2024-09-18T14:08:02.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35d0657640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c813da10 con 0x7f35c806e810 2024-09-18T14:08:02.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35d0657640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:02.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35d0657640 10 monclient(hunting): authenticate will time out at 1297.549324s 2024-09-18T14:08:02.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ce3f3640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f35c806e810 0x7f35c813d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:53302/0 (socket says 172.21.15.45:53302) 2024-09-18T14:08:02.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ce3f3640 1 -- 172.21.15.45:0/3959645449 learned_addr learned my addr 172.21.15.45:0/3959645449 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:02.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375393623 0 0) 0x7f35c813da10 con 0x7f35c806e810 2024-09-18T14:08:02.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): my global_id is 5309 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35ac0034e0 con 0x7f35c806e810 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148657381 0 0) 0x7f35c8122680 con 0x7f35c80a1080 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): my global_id is 164173 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c813da10 con 0x7f35c80a1080 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736330850 0 0) 0x7f35c813e770 con 0x7f35c80a0cb0 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): my global_id is 5235 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c8122680 con 0x7f35c80a0cb0 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1078898624 0 0) 0x7f35c813da10 con 0x7f35c80a1080 2024-09-18T14:08:02.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35c813e770 con 0x7f35c80a1080 2024-09-18T14:08:02.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f35c0001b70 con 0x7f35c80a1080 2024-09-18T14:08:02.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:02.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): dump: 2024-09-18T14:08:02.229 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.229 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.229 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.229 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.229 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.229 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.229 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.211+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2740808303 0 0) 0x7f35ac0034e0 con 0x7f35c806e810 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35c813da10 con 0x7f35c806e810 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2811678433 0 0) 0x7f35c8122680 con 0x7f35c80a0cb0 2024-09-18T14:08:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35ac0034e0 con 0x7f35c80a0cb0 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f35c40015d0 con 0x7f35c806e810 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient(hunting): dump: 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.231 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f35bc001000 con 0x7f35c80a0cb0 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient(hunting): dump: 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.232 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2625696076 0 0) 0x7f35c813e770 con 0x7f35c80a1080 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 >> v1:172.21.15.45:6793/0 conn(0x7f35c806e810 legacy=0x7f35c813d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 >> v1:172.21.15.45:6792/0 conn(0x7f35c80a0cb0 legacy=0x7f35c8146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 1 monclient: found mon.c 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c81223a0 con 0x7f35c80a1080 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient: _finish_auth 0 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35d0657640 5 monclient: authenticate success, global_id 164173 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35d0657640 10 monclient: _renew_subs 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35d0657640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35d0657640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c8122960 con 0x7f35c80a1080 2024-09-18T14:08:02.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35d0657640 10 monclient: _renew_subs 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35d0657640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35d0657640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c8121670 con 0x7f35c80a1080 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f35c0001520 con 0x7f35c80a1080 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f35c0005110 con 0x7f35c80a1080 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ccbf0640 4 set_mon_vals no callback set 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.212+0000 7f35ba7fc640 10 monclient: dump: 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.235 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35d0657640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35d0657640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f35b0006580 con 0x7f35b0002830 2024-09-18T14:08:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f35c00100e0 con 0x7f35c80a1080 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(426..426 src has 126..426) v4 ==== 3959+0+0 (unknown 169183747 0 0) 0x7f35c0046d60 con 0x7f35c80a1080 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35cdbf2640 10 monclient: get_auth_request con 0x7f35ac03abd0 auth_method 0 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678421920 0 0) 0x7f35b0006580 con 0x7f35b0002830 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 10 monclient(hunting): my global_id is 5162 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c813e770 con 0x7f35b0002830 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 239359031 0 0) 0x7f35c813e770 con 0x7f35b0002830 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35b0006580 con 0x7f35b0002830 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f35c4001770 con 0x7f35b0002830 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.213+0000 7f35ba7fc640 10 monclient: dump: 2024-09-18T14:08:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.239 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.214+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2521056476 0 0) 0x7f35b0006580 con 0x7f35b0002830 2024-09-18T14:08:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.214+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35b000a0a0 con 0x7f35b0002830 2024-09-18T14:08:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.214+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f35c40050c0 con 0x7f35b0002830 2024-09-18T14:08:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.214+0000 7f35ba7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.214+0000 7f35ba7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.214+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 >> v1:172.21.15.84:6791/0 conn(0x7f35b0002830 legacy=0x7f35b0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.214+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 >> v1:172.21.15.84:6791/0 conn(0x7f35b0002830 legacy=0x7f35b0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:02.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35d0657640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:02.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35d0657640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f35b0009610 con 0x7f35b0008ff0 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877719454 0 0) 0x7f35b0009610 con 0x7f35b0008ff0 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 10 monclient(hunting): my global_id is 5171 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35b0006580 con 0x7f35b0008ff0 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3434356913 0 0) 0x7f35b0006580 con 0x7f35b0008ff0 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35b0009610 con 0x7f35b0008ff0 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f35c4005690 con 0x7f35b0008ff0 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 10 monclient: dump: 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.242 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2032992625 0 0) 0x7f35b0009610 con 0x7f35b0008ff0 2024-09-18T14:08:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.220+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f35b00098d0 con 0x7f35b0008ff0 2024-09-18T14:08:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.221+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 478777105) 0x7f35c4006ae0 con 0x7f35b0008ff0 2024-09-18T14:08:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.221+0000 7f35ba7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.221+0000 7f35ba7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.221+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 >> v1:172.21.15.84:6791/0 conn(0x7f35b0008ff0 legacy=0x7f35b0008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.221+0000 7f35ba7fc640 1 -- 172.21.15.45:0/3959645449 >> v1:172.21.15.84:6791/0 conn(0x7f35b0008ff0 legacy=0x7f35b0008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:02.244 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:02.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:02.244 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:02.244 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:02.244 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:02.244 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:02.244 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:02.244 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:02.244 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:02.244 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:02.244 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:02.244 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:02.245 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:02.246 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:02.248 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.250 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.251 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.252 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.252 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:02.252 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:02.252 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.252 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.252 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:02.252 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.252 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:02.255 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.256 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:02.260 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:02.261 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.262 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:02.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.263 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:02.263 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:02.263 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:02.263 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.263 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.263 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:02.263 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.221+0000 7f35d0657640 1 -- 172.21.15.45:0/3959645449 >> v1:172.21.15.84:6812/31478 conn(0x7f35ac03abd0 legacy=0x7f35ac03d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.221+0000 7f35d0657640 10 monclient: shutdown 2024-09-18T14:08:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.221+0000 7f35d0657640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.221+0000 7f35d0657640 1 -- 172.21.15.45:0/3959645449 >> v1:172.21.15.45:6791/0 conn(0x7f35c80a1080 legacy=0x7f35c8149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.221+0000 7f35cebf4640 1 -- 172.21.15.45:0/3959645449 reap_dead start 2024-09-18T14:08:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.221+0000 7f35cebf4640 1 -- 172.21.15.45:0/3959645449 reap_dead start 2024-09-18T14:08:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.222+0000 7f35d0657640 1 -- 172.21.15.45:0/3959645449 shutdown_connections 2024-09-18T14:08:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.222+0000 7f35d0657640 1 -- 172.21.15.45:0/3959645449 >> 172.21.15.45:0/3959645449 conn(0x7f35c80a5bb0 msgr2=0x7f35c80a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.222+0000 7f35d0657640 1 -- 172.21.15.45:0/3959645449 shutdown_connections 2024-09-18T14:08:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.222+0000 7f35d0657640 1 -- 172.21.15.45:0/3959645449 wait complete. 2024-09-18T14:08:02.265 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:02.265 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:08:02.266 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:08:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1944: done (3 left) 2024-09-18T14:08:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1945: done (2 left) 2024-09-18T14:08:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1947: done (1 left) 2024-09-18T14:08:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1948: done (0 left) 2024-09-18T14:08:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1949: write (excl) oid 28 current snap is 204 2024-09-18T14:08:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1949: seq_num 685 ranges {712795=417335,1784636=540572,2894750=457917} 2024-09-18T14:08:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1949: writing smithi08434663-28 from 712795 to 1130130 tid 1 2024-09-18T14:08:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1949: writing smithi08434663-28 from 1784636 to 2325208 tid 2 2024-09-18T14:08:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1949: writing smithi08434663-28 from 2894750 to 3352667 tid 3 2024-09-18T14:08:02.267 INFO:tasks.rados.rados.0.smithi084.stdout:1950: read oid 22 snap -1 2024-09-18T14:08:02.268 INFO:tasks.rados.rados.0.smithi084.stdout:1950: expect (ObjNum 683 snap 203 seq_num 683) 2024-09-18T14:08:02.268 INFO:tasks.rados.rados.0.smithi084.stdout:1951: write oid 47 current snap is 204 2024-09-18T14:08:02.268 INFO:tasks.rados.rados.0.smithi084.stdout:1951: seq_num 686 ranges {729602=492584,1843118=684230,3322173=207429} 2024-09-18T14:08:02.268 INFO:tasks.rados.rados.0.smithi084.stdout:1951: writing smithi08434663-47 from 729602 to 1222186 tid 1 2024-09-18T14:08:02.275 INFO:tasks.rados.rados.0.smithi084.stdout:1951: writing smithi08434663-47 from 1843118 to 2527348 tid 2 2024-09-18T14:08:02.282 INFO:tasks.rados.rados.0.smithi084.stdout:1951: writing smithi08434663-47 from 3322173 to 3529602 tid 3 2024-09-18T14:08:02.282 INFO:tasks.rados.rados.0.smithi084.stdout:1952: read oid 38 snap -1 2024-09-18T14:08:02.282 INFO:tasks.rados.rados.0.smithi084.stdout:1952: expect (ObjNum 659 snap 196 seq_num 659) 2024-09-18T14:08:02.282 INFO:tasks.rados.rados.0.smithi084.stdout:1953: copy_from oid 19 from oid 15 current snap is 204 2024-09-18T14:08:02.282 INFO:tasks.rados.rados.0.smithi084.stdout:1949: finishing write tid 1 to smithi08434663-28 2024-09-18T14:08:02.283 INFO:tasks.rados.rados.0.smithi084.stdout:1949: finishing write tid 2 to smithi08434663-28 2024-09-18T14:08:02.283 INFO:tasks.rados.rados.0.smithi084.stdout:1949: finishing write tid 3 to smithi08434663-28 2024-09-18T14:08:02.283 INFO:tasks.rados.rados.0.smithi084.stdout:1949: finishing write tid 4 to smithi08434663-28 2024-09-18T14:08:02.283 INFO:tasks.rados.rados.0.smithi084.stdout:1949: finishing write tid 5 to smithi08434663-28 2024-09-18T14:08:02.283 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1011 (ObjNum 685 snap 204 seq_num 685) dirty exists 2024-09-18T14:08:02.283 INFO:tasks.rados.rados.0.smithi084.stdout:1949: left oid 28 (ObjNum 685 snap 204 seq_num 685) 2024-09-18T14:08:02.283 INFO:tasks.rados.rados.0.smithi084.stdout:1950: expect (ObjNum 683 snap 203 seq_num 683) 2024-09-18T14:08:02.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.366+0000 7fbdcc0a8640 10 monclient: get_monmap_and_config 2024-09-18T14:08:02.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.366+0000 7fbdcc0a8640 10 monclient: build_initial_monmap 2024-09-18T14:08:02.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.366+0000 7fbdcc0a8640 10 monclient: monmap: 2024-09-18T14:08:02.367 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:02.367 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.367 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:02.367131+0000 2024-09-18T14:08:02.367 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:02.367131+0000 2024-09-18T14:08:02.367 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:02.367 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:02.368 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.366+0000 7fbdcc0a8640 1 Processor -- start 2024-09-18T14:08:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.366+0000 7fbdcc0a8640 1 -- start start 2024-09-18T14:08:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.366+0000 7fbdcc0a8640 10 monclient: init 2024-09-18T14:08:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdcc0a8640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdcc0a8640 10 monclient: _add_conns ranks=[1,3,4,8,0,2,6,7,5] 2024-09-18T14:08:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdcc0a8640 10 monclient(hunting): picked mon.noname-f con 0x7fbdc406e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdcc0a8640 10 monclient(hunting): picked mon.noname-g con 0x7fbdc40a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdcc0a8640 10 monclient(hunting): picked mon.noname-c con 0x7fbdc40a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdcc0a8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc405ed70 con 0x7fbdc406e810 2024-09-18T14:08:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdcc0a8640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc4097620 con 0x7fbdc40a0cb0 2024-09-18T14:08:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdcc0a8640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc4097340 con 0x7fbdc40a1080 2024-09-18T14:08:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdcc0a8640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdcc0a8640 10 monclient(hunting): authenticate will time out at 1297.705497s 2024-09-18T14:08:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdca645640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fbdc40a1080 0x7fbdc413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:50442/0 (socket says 172.21.15.45:50442) 2024-09-18T14:08:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdca645640 1 -- 172.21.15.45:0/2664589761 learned_addr learned my addr 172.21.15.45:0/2664589761 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474247255 0 0) 0x7fbdc4097340 con 0x7fbdc40a1080 2024-09-18T14:08:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 10 monclient(hunting): my global_id is 164182 2024-09-18T14:08:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdac0034e0 con 0x7fbdc40a1080 2024-09-18T14:08:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536131647 0 0) 0x7fbdc405ed70 con 0x7fbdc406e810 2024-09-18T14:08:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 10 monclient(hunting): my global_id is 164161 2024-09-18T14:08:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc4097340 con 0x7fbdc406e810 2024-09-18T14:08:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420621721 0 0) 0x7fbdc4097620 con 0x7fbdc40a0cb0 2024-09-18T14:08:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 10 monclient(hunting): my global_id is 154353 2024-09-18T14:08:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.367+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc405ed70 con 0x7fbdc40a0cb0 2024-09-18T14:08:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2462336323 0 0) 0x7fbdac0034e0 con 0x7fbdc40a1080 2024-09-18T14:08:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 >> v1:172.21.15.84:6790/0 conn(0x7fbdc40a0cb0 legacy=0x7fbdc4137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 >> v1:172.21.15.84:6789/0 conn(0x7fbdc406e810 legacy=0x7fbdc40a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 10 monclient: _finish_hunting 0 2024-09-18T14:08:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 1 monclient: found mon.noname-c 2024-09-18T14:08:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:08:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc40a1660 con 0x7fbdc40a1080 2024-09-18T14:08:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 10 monclient: _finish_auth 0 2024-09-18T14:08:02.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:02.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbdbc001830 con 0x7fbdc40a1080 2024-09-18T14:08:02.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:02.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 10 monclient: dump: 2024-09-18T14:08:02.396 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.396 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.396 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.396 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.396 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.396 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.398 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdcc0a8640 5 monclient: authenticate success, global_id 164182 2024-09-18T14:08:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdcc0a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:02.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbdbc001a10 con 0x7fbdc40a1080 2024-09-18T14:08:02.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:02.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdcc0a8640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:02.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 1 -- 172.21.15.45:0/2664589761 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbdbc004ba0 con 0x7fbdc40a1080 2024-09-18T14:08:02.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdcc0a8640 4 set_mon_vals no callback set 2024-09-18T14:08:02.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:02.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdc8e42640 10 monclient: dump: 2024-09-18T14:08:02.400 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.400 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.400 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.400 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.400 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.401 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.401 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.401 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.401 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.401 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.401 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.402 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.402 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.402 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.402 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.402 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdcc0a8640 10 monclient: shutdown 2024-09-18T14:08:02.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdcc0a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:02.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/2664589761 >> v1:172.21.15.45:6791/0 conn(0x7fbdc40a1080 legacy=0x7fbdc413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/2664589761 shutdown_connections 2024-09-18T14:08:02.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/2664589761 >> 172.21.15.45:0/2664589761 conn(0x7fbdc40a5bb0 msgr2=0x7fbdc40a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:02.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/2664589761 shutdown_connections 2024-09-18T14:08:02.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.368+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/2664589761 wait complete. 2024-09-18T14:08:02.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.369+0000 7fbdcc0a8640 10 monclient: build_initial_monmap 2024-09-18T14:08:02.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.369+0000 7fbdcc0a8640 10 monclient: monmap: 2024-09-18T14:08:02.404 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:02.404 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:02.404 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:02.404 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:02.405 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:02.405 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:02.405 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:02.405 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:02.405 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:02.405 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:02.405 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:02.406 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:02.406 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:02.406 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:02.406 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:02.406 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.369+0000 7fbdcc0a8640 1 Processor -- start 2024-09-18T14:08:02.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.369+0000 7fbdcc0a8640 1 -- start start 2024-09-18T14:08:02.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.369+0000 7fbdcc0a8640 10 monclient: init 2024-09-18T14:08:02.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdcc0a8640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:02.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdcc0a8640 10 monclient: _add_conns ranks=[2,4,3,5,1,8,6,7,0] 2024-09-18T14:08:02.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdcc0a8640 10 monclient(hunting): picked mon.noname-b con 0x7fbdc406e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:02.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdcc0a8640 10 monclient(hunting): picked mon.noname-c con 0x7fbdc40a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:02.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdcc0a8640 10 monclient(hunting): picked mon.noname-g con 0x7fbdc40a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:02.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdcc0a8640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc4122da0 con 0x7fbdc406e810 2024-09-18T14:08:02.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdcc0a8640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc413db90 con 0x7fbdc40a1080 2024-09-18T14:08:02.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdcc0a8640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc41227e0 con 0x7fbdc40a0cb0 2024-09-18T14:08:02.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdcc0a8640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:02.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdcc0a8640 10 monclient(hunting): authenticate will time out at 1297.708392s 2024-09-18T14:08:02.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc9e44640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fbdc406e810 0x7fbdc413d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47186/0 (socket says 172.21.15.45:47186) 2024-09-18T14:08:02.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc9e44640 1 -- 172.21.15.45:0/443284718 learned_addr learned my addr 172.21.15.45:0/443284718 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:02.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc9643640 1 --1- 172.21.15.45:0/443284718 >> v1:172.21.15.45:6791/0 conn(0x7fbdc40a0cb0 0x7fbdc41466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:50450/0 (socket says 172.21.15.45:50450) 2024-09-18T14:08:02.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354087510 0 0) 0x7fbdc413db90 con 0x7fbdc40a1080 2024-09-18T14:08:02.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): my global_id is 154362 2024-09-18T14:08:02.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb00034e0 con 0x7fbdc40a1080 2024-09-18T14:08:02.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811881864 0 0) 0x7fbdc4122da0 con 0x7fbdc406e810 2024-09-18T14:08:02.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): my global_id is 144668 2024-09-18T14:08:02.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc413db90 con 0x7fbdc406e810 2024-09-18T14:08:02.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31000220 0 0) 0x7fbdc41227e0 con 0x7fbdc40a0cb0 2024-09-18T14:08:02.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): my global_id is 164191 2024-09-18T14:08:02.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc4122da0 con 0x7fbdc40a0cb0 2024-09-18T14:08:02.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 977946302 0 0) 0x7fbdb00034e0 con 0x7fbdc40a1080 2024-09-18T14:08:02.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbdc41227e0 con 0x7fbdc40a1080 2024-09-18T14:08:02.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbdbc0017f0 con 0x7fbdc40a1080 2024-09-18T14:08:02.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:08:02.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): dump: 2024-09-18T14:08:02.413 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.413 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.413 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.413 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.413 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.414 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.414 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.414 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.414 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.414 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.414 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.414 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.415 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.415 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.415 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.415 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:02.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3338259890 0 0) 0x7fbdc4122da0 con 0x7fbdc40a0cb0 2024-09-18T14:08:02.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.370+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbdb00034e0 con 0x7fbdc40a0cb0 2024-09-18T14:08:02.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3658805632 0 0) 0x7fbdc413db90 con 0x7fbdc406e810 2024-09-18T14:08:02.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbdc4122da0 con 0x7fbdc406e810 2024-09-18T14:08:02.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbdb8001aa0 con 0x7fbdc40a0cb0 2024-09-18T14:08:02.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:02.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient(hunting): dump: 2024-09-18T14:08:02.417 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.417 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.420 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbdb4002040 con 0x7fbdc406e810 2024-09-18T14:08:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient(hunting): dump: 2024-09-18T14:08:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.422 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.422 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.422 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.422 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.422 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.422 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.422 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.423 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1671478410 0 0) 0x7fbdc41227e0 con 0x7fbdc40a1080 2024-09-18T14:08:02.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 >> v1:172.21.15.45:6791/0 conn(0x7fbdc40a0cb0 legacy=0x7fbdc41466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 >> v1:172.21.15.45:6790/0 conn(0x7fbdc406e810 legacy=0x7fbdc413d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:08:02.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 1 monclient: found mon.g 2024-09-18T14:08:02.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:02.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:08:02.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc41202f0 con 0x7fbdc40a1080 2024-09-18T14:08:02.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient: _finish_auth 0 2024-09-18T14:08:02.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:02.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdcc0a8640 5 monclient: authenticate success, global_id 154362 2024-09-18T14:08:02.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdcc0a8640 10 monclient: _renew_subs 2024-09-18T14:08:02.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdcc0a8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:08:02.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc4122500 con 0x7fbdc40a1080 2024-09-18T14:08:02.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbdbc004b50 con 0x7fbdc40a1080 2024-09-18T14:08:02.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:02.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbdbc005080 con 0x7fbdc40a1080 2024-09-18T14:08:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:08:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc1ffb640 10 monclient: dump: 2024-09-18T14:08:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.427 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.427 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.427 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.427 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.427 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.427 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.428 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.428 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.428 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.428 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.428 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.428 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdcc0a8640 10 monclient: _renew_subs 2024-09-18T14:08:02.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdcc0a8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:08:02.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc4121630 con 0x7fbdc40a1080 2024-09-18T14:08:02.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.371+0000 7fbdc3fff640 4 set_mon_vals no callback set 2024-09-18T14:08:02.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fbdbc010050 con 0x7fbdc40a1080 2024-09-18T14:08:02.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdcc0a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:02.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbdc404c4a0 con 0x7fbdc413e730 2024-09-18T14:08:02.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(427..427 src has 126..427) v4 ==== 3987+0+0 (unknown 361319633 0 0) 0x7fbdbc046a90 con 0x7fbdc40a1080 2024-09-18T14:08:02.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc9643640 10 monclient: get_auth_request con 0x7fbdb003aa40 auth_method 0 2024-09-18T14:08:02.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 535597291 0 0) 0x7fbdc404c4a0 con 0x7fbdc413e730 2024-09-18T14:08:02.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 10 monclient(hunting): my global_id is 5335 2024-09-18T14:08:02.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc41227e0 con 0x7fbdc413e730 2024-09-18T14:08:02.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4235005158 0 0) 0x7fbdc41227e0 con 0x7fbdc413e730 2024-09-18T14:08:02.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbdc404c4a0 con 0x7fbdc413e730 2024-09-18T14:08:02.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbdb4002810 con 0x7fbdc413e730 2024-09-18T14:08:02.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:08:02.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.372+0000 7fbdc1ffb640 10 monclient: dump: 2024-09-18T14:08:02.432 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.432 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.432 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.432 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.432 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.433 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.433 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.433 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.433 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.433 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.433 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.434 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.434 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.434 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.434 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.434 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.373+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 850718040 0 0) 0x7fbdc404c4a0 con 0x7fbdc413e730 2024-09-18T14:08:02.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.373+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbdc404b720 con 0x7fbdc413e730 2024-09-18T14:08:02.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.373+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fbdb40029f0 con 0x7fbdc413e730 2024-09-18T14:08:02.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.373+0000 7fbdc1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:02.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.373+0000 7fbdc1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:02.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.373+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 >> v1:172.21.15.84:6792/0 conn(0x7fbdc413e730 legacy=0x7fbdc405e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.373+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 >> v1:172.21.15.84:6792/0 conn(0x7fbdc413e730 legacy=0x7fbdc405e310 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:02.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdcc0a8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:02.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbdc4062dd0 con 0x7fbdc402aa40 2024-09-18T14:08:02.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179330466 0 0) 0x7fbdc4062dd0 con 0x7fbdc402aa40 2024-09-18T14:08:02.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 10 monclient(hunting): my global_id is 5344 2024-09-18T14:08:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc404c4a0 con 0x7fbdc402aa40 2024-09-18T14:08:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 474165844 0 0) 0x7fbdc404c4a0 con 0x7fbdc402aa40 2024-09-18T14:08:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbdc4062dd0 con 0x7fbdc402aa40 2024-09-18T14:08:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbdb4005a90 con 0x7fbdc402aa40 2024-09-18T14:08:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:08:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 10 monclient: dump: 2024-09-18T14:08:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 70188765 0 0) 0x7fbdc4062dd0 con 0x7fbdc402aa40 2024-09-18T14:08:02.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.379+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbdc402ae10 con 0x7fbdc402aa40 2024-09-18T14:08:02.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 1879889869) 0x7fbdb4007cf0 con 0x7fbdc402aa40 2024-09-18T14:08:02.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdc1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:02.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdc1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:02.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 >> v1:172.21.15.84:6792/0 conn(0x7fbdc402aa40 legacy=0x7fbdc409bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdc1ffb640 1 -- 172.21.15.45:0/443284718 >> v1:172.21.15.84:6792/0 conn(0x7fbdc402aa40 legacy=0x7fbdc409bba0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:02.438 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:02.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:02.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:02.439 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:02.440 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:02.441 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:02.442 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:02.443 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.444 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:02.445 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:02.447 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:02.449 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.450 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:02.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:02.454 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.454 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.454 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:02.454 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.454 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.454 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:02.455 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:02.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:02.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:02.458 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:02.458 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:02.459 INFO:tasks.rados.rados.0.smithi084.stdout:1951: finishing write tid 1 to smithi08434663-47 2024-09-18T14:08:02.459 INFO:tasks.rados.rados.0.smithi084.stdout:1951: finishing write tid 2 to smithi08434663-47 2024-09-18T14:08:02.459 INFO:tasks.rados.rados.0.smithi084.stdout:1949: done (4 left) 2024-09-18T14:08:02.459 INFO:tasks.rados.rados.0.smithi084.stdout:1950: done (3 left) 2024-09-18T14:08:02.459 INFO:tasks.rados.rados.0.smithi084.stdout:1954: delete oid 33 current snap is 204 2024-09-18T14:08:02.460 INFO:tasks.rados.rados.0.smithi084.stdout:1951: finishing write tid 3 to smithi08434663-47 2024-09-18T14:08:02.460 INFO:tasks.rados.rados.0.smithi084.stdout:1951: finishing write tid 4 to smithi08434663-47 2024-09-18T14:08:02.460 INFO:tasks.rados.rados.0.smithi084.stdout:1951: finishing write tid 5 to smithi08434663-47 2024-09-18T14:08:02.460 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 599 (ObjNum 686 snap 204 seq_num 686) dirty exists 2024-09-18T14:08:02.460 INFO:tasks.rados.rados.0.smithi084.stdout:1951: left oid 47 (ObjNum 686 snap 204 seq_num 686) 2024-09-18T14:08:02.460 INFO:tasks.rados.rados.0.smithi084.stdout:1953: finishing copy_from racing read to smithi08434663-19 2024-09-18T14:08:02.460 INFO:tasks.rados.rados.0.smithi084.stdout:1953: finishing copy_from to smithi08434663-19 2024-09-18T14:08:02.460 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 846 (ObjNum 634 snap 189 seq_num 634) dirty exists 2024-09-18T14:08:02.460 INFO:tasks.rados.rados.0.smithi084.stdout:1952: expect (ObjNum 659 snap 196 seq_num 659) 2024-09-18T14:08:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/443284718 >> v1:172.21.15.84:6812/31478 conn(0x7fbdb003aa40 legacy=0x7fbdb003cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdcc0a8640 10 monclient: shutdown 2024-09-18T14:08:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdcc0a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/443284718 >> v1:172.21.15.84:6790/0 conn(0x7fbdc40a1080 legacy=0x7fbdc4149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdca645640 1 -- 172.21.15.45:0/443284718 reap_dead start 2024-09-18T14:08:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdca645640 1 -- 172.21.15.45:0/443284718 reap_dead start 2024-09-18T14:08:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/443284718 shutdown_connections 2024-09-18T14:08:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/443284718 >> 172.21.15.45:0/443284718 conn(0x7fbdc40a5bb0 msgr2=0x7fbdc4096b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/443284718 shutdown_connections 2024-09-18T14:08:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.380+0000 7fbdcc0a8640 1 -- 172.21.15.45:0/443284718 wait complete. 2024-09-18T14:08:02.461 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:02.462 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-18T14:08:02.462 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-18T14:08:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.557+0000 7f818fb5d640 10 monclient: get_monmap_and_config 2024-09-18T14:08:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.557+0000 7f818fb5d640 10 monclient: build_initial_monmap 2024-09-18T14:08:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.557+0000 7f818fb5d640 10 monclient: monmap: 2024-09-18T14:08:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:02.558642+0000 2024-09-18T14:08:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:02.558642+0000 2024-09-18T14:08:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:02.559 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.558+0000 7f818fb5d640 1 Processor -- start 2024-09-18T14:08:02.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.558+0000 7f818fb5d640 1 -- start start 2024-09-18T14:08:02.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.558+0000 7f818fb5d640 10 monclient: init 2024-09-18T14:08:02.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.558+0000 7f818fb5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:02.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.558+0000 7f818fb5d640 10 monclient: _add_conns ranks=[4,5,8,2,0,6,1,3,7] 2024-09-18T14:08:02.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818fb5d640 10 monclient(hunting): picked mon.noname-c con 0x7f818809f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:02.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818fb5d640 10 monclient(hunting): picked mon.noname-h con 0x7f818809fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:02.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818fb5d640 10 monclient(hunting): picked mon.noname-e con 0x7f818809ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818fb5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818805ed70 con 0x7f818809f7d0 2024-09-18T14:08:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818fb5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818809dfc0 con 0x7f818809fba0 2024-09-18T14:08:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818fb5d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818809dce0 con 0x7f818809ff70 2024-09-18T14:08:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818fb5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818fb5d640 10 monclient(hunting): authenticate will time out at 1297.897136s 2024-09-18T14:08:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818d8f9640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f818809f7d0 0x7f8188132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:50462/0 (socket says 172.21.15.45:50462) 2024-09-18T14:08:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818d8f9640 1 -- 172.21.15.45:0/3489450560 learned_addr learned my addr 172.21.15.45:0/3489450560 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952946596 0 0) 0x7f818805ed70 con 0x7f818809f7d0 2024-09-18T14:08:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient(hunting): my global_id is 164200 2024-09-18T14:08:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81740034e0 con 0x7f818809f7d0 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358429191 0 0) 0x7f818809dce0 con 0x7f818809ff70 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient(hunting): my global_id is 5318 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f818805ed70 con 0x7f818809ff70 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80724478 0 0) 0x7f818809dfc0 con 0x7f818809fba0 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient(hunting): my global_id is 5180 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f818809dce0 con 0x7f818809fba0 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3916051785 0 0) 0x7f81740034e0 con 0x7f818809f7d0 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 >> v1:172.21.15.45:6793/0 conn(0x7f818809ff70 legacy=0x7f818813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 >> v1:172.21.15.84:6791/0 conn(0x7f818809fba0 legacy=0x7f8188137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient: _finish_hunting 0 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 monclient: found mon.noname-c 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880a1c00 con 0x7f818809f7d0 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient: _finish_auth 0 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818fb5d640 5 monclient: authenticate success, global_id 164200 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8184001bb0 con 0x7f818809f7d0 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient: dump: 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818fb5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8184001d10 con 0x7f818809f7d0 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 1 -- 172.21.15.45:0/3489450560 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8184004f20 con 0x7f818809f7d0 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818fb5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818fb5d640 4 set_mon_vals no callback set 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.559+0000 7f818c8f7640 10 monclient: dump: 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.560+0000 7f818fb5d640 10 monclient: shutdown 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.560+0000 7f818fb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.560+0000 7f818fb5d640 1 -- 172.21.15.45:0/3489450560 >> v1:172.21.15.45:6791/0 conn(0x7f818809f7d0 legacy=0x7f8188132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.560+0000 7f818fb5d640 1 -- 172.21.15.45:0/3489450560 shutdown_connections 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.560+0000 7f818fb5d640 1 -- 172.21.15.45:0/3489450560 >> 172.21.15.45:0/3489450560 conn(0x7f81880630b0 msgr2=0x7f81880634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.560+0000 7f818fb5d640 1 -- 172.21.15.45:0/3489450560 shutdown_connections 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.560+0000 7f818fb5d640 1 -- 172.21.15.45:0/3489450560 wait complete. 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.560+0000 7f818fb5d640 10 monclient: build_initial_monmap 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.560+0000 7f818fb5d640 10 monclient: monmap: 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 1 Processor -- start 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 1 -- start start 2024-09-18T14:08:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 10 monclient: init 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 10 monclient: _add_conns ranks=[6,5,4,8,7,1,0,2,3] 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 10 monclient(hunting): picked mon.noname-d con 0x7f818809f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 10 monclient(hunting): picked mon.noname-h con 0x7f818809fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 10 monclient(hunting): picked mon.noname-c con 0x7f818809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818809d310 con 0x7f818809ff70 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81880a2560 con 0x7f818809fba0 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8188120a60 con 0x7f818809f7d0 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818fb5d640 10 monclient(hunting): authenticate will time out at 1297.899859s 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818d8f9640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f818809f7d0 0x7f8188141bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40904/0 (socket says 172.21.15.45:40904) 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.561+0000 7f818d8f9640 1 -- 172.21.15.45:0/2922254054 learned_addr learned my addr 172.21.15.45:0/2922254054 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18480004 0 0) 0x7f818809d310 con 0x7f818809ff70 2024-09-18T14:08:02.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): my global_id is 164209 2024-09-18T14:08:02.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f816c0034e0 con 0x7f818809ff70 2024-09-18T14:08:02.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980111025 0 0) 0x7f81880a2560 con 0x7f818809fba0 2024-09-18T14:08:02.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): my global_id is 5189 2024-09-18T14:08:02.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f818809d310 con 0x7f818809fba0 2024-09-18T14:08:02.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702995107 0 0) 0x7f8188120a60 con 0x7f818809f7d0 2024-09-18T14:08:02.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:02.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:02.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): my global_id is 5244 2024-09-18T14:08:02.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81880a2560 con 0x7f818809f7d0 2024-09-18T14:08:02.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 853797573 0 0) 0x7f816c0034e0 con 0x7f818809ff70 2024-09-18T14:08:02.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8188120a60 con 0x7f818809ff70 2024-09-18T14:08:02.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8180001890 con 0x7f818809ff70 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): dump: 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.581 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2346446892 0 0) 0x7f818809d310 con 0x7f818809fba0 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f816c0034e0 con 0x7f818809fba0 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4217041708 0 0) 0x7f81880a2560 con 0x7f818809f7d0 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f818809d310 con 0x7f818809f7d0 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8178001680 con 0x7f818809fba0 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:08:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): dump: 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.583 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8184001af0 con 0x7f818809f7d0 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient(hunting): dump: 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.584 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 391490999 0 0) 0x7f8188120a60 con 0x7f818809ff70 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 >> v1:172.21.15.45:6792/0 conn(0x7f818809f7d0 legacy=0x7f8188141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 >> v1:172.21.15.84:6791/0 conn(0x7f818809fba0 legacy=0x7f8188142300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient: _finish_hunting 0 2024-09-18T14:08:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 monclient: found mon.c 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8188121710 con 0x7f818809ff70 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient: _finish_auth 0 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f818fb5d640 5 monclient: authenticate success, global_id 164209 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f818fb5d640 10 monclient: _renew_subs 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f818fb5d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f818fb5d640 1 -- 172.21.15.45:0/2922254054 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8188142a10 con 0x7f818809ff70 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f81800019f0 con 0x7f818809ff70 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8180001f20 con 0x7f818809ff70 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817dffb640 10 monclient: dump: 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f818fb5d640 10 monclient: _renew_subs 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f817ffff640 4 set_mon_vals no callback set 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f818fb5d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.562+0000 7f818fb5d640 1 -- 172.21.15.45:0/2922254054 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f818811e970 con 0x7f818809ff70 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.563+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f8180002080 con 0x7f818809ff70 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.563+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(427..427 src has 126..427) v4 ==== 3987+0+0 (unknown 361319633 0 0) 0x7f8180043fc0 con 0x7f818809ff70 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.563+0000 7f818fb5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.563+0000 7f818fb5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.563+0000 7f818fb5d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.563+0000 7f818fb5d640 1 -- 172.21.15.45:0/2922254054 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818804c4a0 con 0x7f818809ff70 2024-09-18T14:08:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.563+0000 7f818d8f9640 10 monclient: get_auth_request con 0x7f816c03a9c0 auth_method 0 2024-09-18T14:08:02.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.568+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.4 v1:172.21.15.45:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f81800127b0 con 0x7f818809ff70 2024-09-18T14:08:02.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.568+0000 7f817dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:02.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.568+0000 7f817dffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:02.636 INFO:tasks.rados.rados.0.smithi084.stdout:1951: done (3 left) 2024-09-18T14:08:02.636 INFO:tasks.rados.rados.0.smithi084.stdout:1952: done (2 left) 2024-09-18T14:08:02.636 INFO:tasks.rados.rados.0.smithi084.stdout:1953: done (1 left) 2024-09-18T14:08:02.636 INFO:tasks.rados.rados.0.smithi084.stdout:1954: done (0 left) 2024-09-18T14:08:02.636 INFO:tasks.rados.rados.0.smithi084.stdout:1955: copy_from oid 10 from oid 6 current snap is 204 2024-09-18T14:08:02.637 INFO:tasks.rados.rados.0.smithi084.stdout:1956: write oid 12 current snap is 204 2024-09-18T14:08:02.637 INFO:tasks.rados.rados.0.smithi084.stdout:1956: seq_num 688 ranges {763216=643082,1843729=687899,3260666=302550} 2024-09-18T14:08:02.643 INFO:tasks.rados.rados.0.smithi084.stdout:1956: writing smithi08434663-12 from 763216 to 1406298 tid 1 2024-09-18T14:08:02.650 INFO:tasks.rados.rados.0.smithi084.stdout:1956: writing smithi08434663-12 from 1843729 to 2531628 tid 2 2024-09-18T14:08:02.656 INFO:tasks.rados.rados.0.smithi084.stdout:1956: writing smithi08434663-12 from 3260666 to 3563216 tid 3 2024-09-18T14:08:02.656 INFO:tasks.rados.rados.0.smithi084.stdout:1957: write oid 47 current snap is 204 2024-09-18T14:08:02.656 INFO:tasks.rados.rados.0.smithi084.stdout:1955: finishing copy_from racing read to smithi08434663-10 2024-09-18T14:08:02.656 INFO:tasks.rados.rados.0.smithi084.stdout:1956: finishing write tid 1 to smithi08434663-12 2024-09-18T14:08:02.656 INFO:tasks.rados.rados.0.smithi084.stdout:1955: finishing copy_from to smithi08434663-10 2024-09-18T14:08:02.656 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 575 (ObjNum 538 snap 167 seq_num 538) dirty exists 2024-09-18T14:08:02.656 INFO:tasks.rados.rados.0.smithi084.stdout:1956: finishing write tid 2 to smithi08434663-12 2024-09-18T14:08:02.656 INFO:tasks.rados.rados.0.smithi084.stdout:1957: seq_num 689 ranges {780023=718331,1902211=547910,3004442=575581} 2024-09-18T14:08:02.664 INFO:tasks.rados.rados.0.smithi084.stdout:1957: writing smithi08434663-47 from 780023 to 1498354 tid 1 2024-09-18T14:08:02.669 INFO:tasks.rados.rados.0.smithi084.stdout:1957: writing smithi08434663-47 from 1902211 to 2450121 tid 2 2024-09-18T14:08:02.676 INFO:tasks.rados.rados.0.smithi084.stdout:1957: writing smithi08434663-47 from 3004442 to 3580023 tid 3 2024-09-18T14:08:02.676 INFO:tasks.rados.rados.0.smithi084.stdout:1955: done (2 left) 2024-09-18T14:08:02.676 INFO:tasks.rados.rados.0.smithi084.stdout:1958: read oid 41 snap -1 2024-09-18T14:08:02.676 INFO:tasks.rados.rados.0.smithi084.stdout:1958: expect (ObjNum 665 snap 198 seq_num 665) 2024-09-18T14:08:02.676 INFO:tasks.rados.rados.0.smithi084.stdout:1959: snap_remove snap 203 2024-09-18T14:08:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.803+0000 7f818fb5d640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-18T14:08:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.803+0000 7f818fb5d640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-18T14:08:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.803+0000 7f818fb5d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.803+0000 7f818fb5d640 1 -- 172.21.15.45:0/2922254054 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f81881206a0 con 0x7f818809ff70 2024-09-18T14:08:02.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=1714710239,config=3140514009,health=134227264,kv=594742435,logm=1613640398}) 2024-09-18T14:08:02.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=11,kv=8,logm=55} crc {auth=1304107639,config=55734535,health=2794291964,kv=594742435,logm=3959552952}) 2024-09-18T14:08:02.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=1714710239,config=3140514009,health=134227264,kv=594742435,logm=1613640398}) 2024-09-18T14:08:02.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=2,health=12,kv=8,logm=55} crc {auth=566878099,config=3140514009,health=134227264,kv=594742435,logm=1762599997}) 2024-09-18T14:08:02.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f817dffb640 1 -- 172.21.15.45:0/2922254054 <== mon.4 v1:172.21.15.45:6791/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f81800121b0 con 0x7f818809ff70 2024-09-18T14:08:02.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f817dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-18T14:08:02.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f817dffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:02.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=1714710239,config=3140514009,health=134227264,kv=594742435,logm=1613640398}) 2024-09-18T14:08:02.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=23,config=2,health=12,kv=8,logm=55} crc {auth=1834292400,config=3140514009,health=134227264,kv=594742435,logm=1391164226}) 2024-09-18T14:08:02.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=1714710239,config=3140514009,health=134227264,kv=594742435,logm=1613640398}) 2024-09-18T14:08:02.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=24,config=2,health=12,kv=8,logm=54} crc {auth=3157162466,config=3140514009,health=134227264,kv=1780534418,logm=3705026636}) 2024-09-18T14:08:02.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=1714710239,config=3140514009,health=134227264,kv=594742435,logm=1613640398}) 2024-09-18T14:08:02.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=1714710239,config=3140514009,health=134227264,kv=594742435,logm=981144808}) 2024-09-18T14:08:02.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=1714710239,config=3140514009,health=134227264,kv=594742435,logm=1613640398}) 2024-09-18T14:08:02.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=23,config=2,health=11,kv=7,logm=57} crc {auth=1649004547,config=3140514009,health=3951138712,kv=960027993,logm=2797390992}) 2024-09-18T14:08:02.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=1714710239,config=3140514009,health=134227264,kv=594742435,logm=1613640398}) 2024-09-18T14:08:02.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=1935673365,config=3140514009,health=134227264,kv=594742435,logm=167115457}) 2024-09-18T14:08:02.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=1714710239,config=3140514009,health=134227264,kv=594742435,logm=1613640398}) 2024-09-18T14:08:02.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.805+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=24,config=2,health=11,kv=8,logm=55} crc {auth=3538178346,config=3140514009,health=2869851841,kv=594742435,logm=4031893834}) 2024-09-18T14:08:02.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3029747151}) 2024-09-18T14:08:02.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3170242803}) 2024-09-18T14:08:02.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3029747151}) 2024-09-18T14:08:02.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3491597454}) 2024-09-18T14:08:02.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3029747151}) 2024-09-18T14:08:02.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1993625540}) 2024-09-18T14:08:02.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3029747151}) 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=872613745}) 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3029747151}) 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3691367188}) 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3029747151}) 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3571664631}) 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3029747151}) 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=945042810}) 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.806+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3029747151}) 2024-09-18T14:08:02.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.807+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3797368007}) 2024-09-18T14:08:02.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4039100673}) 2024-09-18T14:08:02.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2346645018}) 2024-09-18T14:08:02.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4039100673}) 2024-09-18T14:08:02.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3796939210}) 2024-09-18T14:08:02.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f818fb5d640 1 -- 172.21.15.45:0/2922254054 >> v1:172.21.15.84:6812/31478 conn(0x7f816c03a9c0 legacy=0x7f816c03ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f818fb5d640 10 monclient: shutdown 2024-09-18T14:08:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f818fb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f818fb5d640 1 -- 172.21.15.45:0/2922254054 >> v1:172.21.15.45:6791/0 conn(0x7f818809ff70 legacy=0x7f8188149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:02.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4039100673}) 2024-09-18T14:08:02.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2959014220}) 2024-09-18T14:08:02.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f818fb5d640 1 -- 172.21.15.45:0/2922254054 shutdown_connections 2024-09-18T14:08:02.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4039100673}) 2024-09-18T14:08:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f818fb5d640 1 -- 172.21.15.45:0/2922254054 >> 172.21.15.45:0/2922254054 conn(0x7f81880630b0 msgr2=0x7f81880a45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:02.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4114271720}) 2024-09-18T14:08:02.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4039100673}) 2024-09-18T14:08:02.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3533993065}) 2024-09-18T14:08:02.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4039100673}) 2024-09-18T14:08:02.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3676111621}) 2024-09-18T14:08:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f818fb5d640 1 -- 172.21.15.45:0/2922254054 shutdown_connections 2024-09-18T14:08:02.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4039100673}) 2024-09-18T14:08:02.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4196755624}) 2024-09-18T14:08:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f818fb5d640 1 -- 172.21.15.45:0/2922254054 wait complete. 2024-09-18T14:08:02.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4039100673}) 2024-09-18T14:08:02.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.808+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3001143784}) 2024-09-18T14:08:02.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3568547408}) 2024-09-18T14:08:02.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3332339303}) 2024-09-18T14:08:02.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3568547408}) 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3449760795}) 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3568547408}) 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2382163346}) 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3568547408}) 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1272988421}) 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3568547408}) 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3203251080}) 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3568547408}) 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3323989641}) 2024-09-18T14:08:02.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3568547408}) 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3348122294}) 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3568547408}) 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.809+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2215470238}) 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3372129017}) 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813128834}) 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3372129017}) 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=720284429}) 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3372129017}) 2024-09-18T14:08:02.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3512915907}) 2024-09-18T14:08:02.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3372129017}) 2024-09-18T14:08:02.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2446892187}) 2024-09-18T14:08:02.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3372129017}) 2024-09-18T14:08:02.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=410944914}) 2024-09-18T14:08:02.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3372129017}) 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3948621434}) 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3372129017}) 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3703215610}) 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3372129017}) 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.811+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4120151622}) 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380460554}) 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2406000739}) 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380460554}) 2024-09-18T14:08:02.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1036710457}) 2024-09-18T14:08:02.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380460554}) 2024-09-18T14:08:02.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2833313532}) 2024-09-18T14:08:02.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380460554}) 2024-09-18T14:08:02.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=972953548}) 2024-09-18T14:08:02.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380460554}) 2024-09-18T14:08:02.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1011697123}) 2024-09-18T14:08:02.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380460554}) 2024-09-18T14:08:02.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3511805647}) 2024-09-18T14:08:02.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380460554}) 2024-09-18T14:08:02.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4020608538}) 2024-09-18T14:08:02.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380460554}) 2024-09-18T14:08:02.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.812+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3018027243}) 2024-09-18T14:08:02.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4204371972}) 2024-09-18T14:08:02.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3169055968}) 2024-09-18T14:08:02.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4204371972}) 2024-09-18T14:08:02.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=834108166}) 2024-09-18T14:08:02.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4204371972}) 2024-09-18T14:08:02.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2541629235}) 2024-09-18T14:08:02.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4204371972}) 2024-09-18T14:08:02.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2020597474}) 2024-09-18T14:08:02.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4204371972}) 2024-09-18T14:08:02.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=247433690}) 2024-09-18T14:08:02.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4204371972}) 2024-09-18T14:08:02.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=451843607}) 2024-09-18T14:08:02.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4204371972}) 2024-09-18T14:08:02.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1426876530}) 2024-09-18T14:08:02.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4204371972}) 2024-09-18T14:08:02.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.814+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3498197515}) 2024-09-18T14:08:02.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2508912804}) 2024-09-18T14:08:02.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2910401905}) 2024-09-18T14:08:02.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2508912804}) 2024-09-18T14:08:02.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2634505166}) 2024-09-18T14:08:02.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2508912804}) 2024-09-18T14:08:02.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2124662897}) 2024-09-18T14:08:02.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2508912804}) 2024-09-18T14:08:02.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3610476432}) 2024-09-18T14:08:02.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2508912804}) 2024-09-18T14:08:02.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4009950275}) 2024-09-18T14:08:02.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2508912804}) 2024-09-18T14:08:02.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4006136403}) 2024-09-18T14:08:02.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2508912804}) 2024-09-18T14:08:02.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3121222299}) 2024-09-18T14:08:02.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2508912804}) 2024-09-18T14:08:02.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.815+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=612962101}) 2024-09-18T14:08:02.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16383078}) 2024-09-18T14:08:02.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=703644068}) 2024-09-18T14:08:02.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16383078}) 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1389124605}) 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16383078}) 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=208961171}) 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16383078}) 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=630829063}) 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16383078}) 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3281250669}) 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16383078}) 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3491426284}) 2024-09-18T14:08:02.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16383078}) 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2134452773}) 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16383078}) 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.816+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2333162463}) 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=450191999}) 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1938985993}) 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=450191999}) 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3356972552}) 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=450191999}) 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2632500625}) 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=450191999}) 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3856855307}) 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=450191999}) 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2395386734}) 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=450191999}) 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1547250368}) 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=450191999}) 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1074243985}) 2024-09-18T14:08:02.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=450191999}) 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.817+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4127368519}) 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2822882335}) 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176945194}) 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2822882335}) 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3899202799}) 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2822882335}) 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4013436218}) 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2822882335}) 2024-09-18T14:08:02.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=992529600}) 2024-09-18T14:08:02.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2822882335}) 2024-09-18T14:08:02.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1261599442}) 2024-09-18T14:08:02.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2822882335}) 2024-09-18T14:08:02.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3499261920}) 2024-09-18T14:08:02.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2822882335}) 2024-09-18T14:08:02.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1679224702}) 2024-09-18T14:08:02.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2822882335}) 2024-09-18T14:08:02.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.818+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3622381933}) 2024-09-18T14:08:02.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3314176151}) 2024-09-18T14:08:02.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3972080686}) 2024-09-18T14:08:02.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3314176151}) 2024-09-18T14:08:02.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2295377946}) 2024-09-18T14:08:02.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3314176151}) 2024-09-18T14:08:02.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=637142384}) 2024-09-18T14:08:02.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3314176151}) 2024-09-18T14:08:02.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=155431962}) 2024-09-18T14:08:02.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3314176151}) 2024-09-18T14:08:02.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2720072032}) 2024-09-18T14:08:02.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3314176151}) 2024-09-18T14:08:02.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1977234015}) 2024-09-18T14:08:02.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3314176151}) 2024-09-18T14:08:02.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3551541519}) 2024-09-18T14:08:02.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3314176151}) 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.819+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2509341001}) 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=710405962}) 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3495527863}) 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=710405962}) 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2742181056}) 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=710405962}) 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=944372388}) 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=710405962}) 2024-09-18T14:08:02.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2141583150}) 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=710405962}) 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3404583519}) 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=710405962}) 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2409504756}) 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=710405962}) 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1832541199}) 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=710405962}) 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.821+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2348798933}) 2024-09-18T14:08:02.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3802996042}) 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3388225036}) 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3802996042}) 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1477771807}) 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3802996042}) 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=338029087}) 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3802996042}) 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=135603361}) 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3802996042}) 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1891001330}) 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3802996042}) 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=608907152}) 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3802996042}) 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2969767899}) 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3802996042}) 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.822+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=918928990}) 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3428617482}) 2024-09-18T14:08:02.851 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545472117}) 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3428617482}) 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3651060039}) 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3428617482}) 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2711434035}) 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3428617482}) 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3214219700}) 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3428617482}) 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3992331374}) 2024-09-18T14:08:02.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3428617482}) 2024-09-18T14:08:02.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3679780887}) 2024-09-18T14:08:02.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3428617482}) 2024-09-18T14:08:02.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1146210895}) 2024-09-18T14:08:02.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3428617482}) 2024-09-18T14:08:02.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.823+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1180061633}) 2024-09-18T14:08:02.854 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-18T14:08:02.882 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1497206336}) 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2839261223}) 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1497206336}) 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2805506140}) 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1497206336}) 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1492920007}) 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1497206336}) 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3556588245}) 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.883 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1497206336}) 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=412916520}) 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1497206336}) 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1075477631}) 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1497206336}) 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4160486}) 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1497206336}) 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.824+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=81851209}) 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3386733723}) 2024-09-18T14:08:02.884 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874601011}) 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3386733723}) 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4121559777}) 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3386733723}) 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3018981263}) 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3386733723}) 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3993053351}) 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3386733723}) 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3597319544}) 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.885 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3386733723}) 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2313512181}) 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3386733723}) 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=769630936}) 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3386733723}) 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.825+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4090043523}) 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4026666359}) 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1355689489}) 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4026666359}) 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=148238730}) 2024-09-18T14:08:02.886 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4026666359}) 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1140902661}) 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4026666359}) 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3650700023}) 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4026666359}) 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=156205586}) 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4026666359}) 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=315033405}) 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4026666359}) 2024-09-18T14:08:02.887 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2390522162}) 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4026666359}) 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.827+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=806602000}) 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710134998}) 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1399154281}) 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710134998}) 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=841498596}) 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710134998}) 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3721457982}) 2024-09-18T14:08:02.888 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710134998}) 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3678661083}) 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710134998}) 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=955869396}) 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710134998}) 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3723998265}) 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710134998}) 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1453920307}) 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710134998}) 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.828+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1018928851}) 2024-09-18T14:08:02.889 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.829+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.829+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2191013418}) 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.829+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3172675948}) 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.829+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.829+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2191013418}) 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3465381904}) 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2191013418}) 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1235652469}) 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2191013418}) 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2227854494}) 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2191013418}) 2024-09-18T14:08:02.890 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1370346524}) 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2191013418}) 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2558612212}) 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2191013418}) 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3793219114}) 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2191013418}) 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.830+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3430438815}) 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=370398145}) 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4214891885}) 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.891 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=370398145}) 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1693859207}) 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=370398145}) 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2663648871}) 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=370398145}) 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4257610227}) 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=370398145}) 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2809274247}) 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=370398145}) 2024-09-18T14:08:02.892 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3528938947}) 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=370398145}) 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2386688618}) 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=370398145}) 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.831+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2404998138}) 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=848352852}) 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2915092087}) 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=848352852}) 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3914884711}) 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.893 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=848352852}) 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=418461532}) 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=848352852}) 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2175199974}) 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=848352852}) 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3612023275}) 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=848352852}) 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3217413195}) 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=848352852}) 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2961440644}) 2024-09-18T14:08:02.894 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=848352852}) 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.832+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4003745550}) 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=449357499}) 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1032166850}) 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=449357499}) 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878519905}) 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=449357499}) 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1720105143}) 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=449357499}) 2024-09-18T14:08:02.895 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2123891038}) 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=449357499}) 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4277479416}) 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=449357499}) 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1703827283}) 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=449357499}) 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=88231248}) 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=449357499}) 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.833+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1200069104}) 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.896 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2948122509}) 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1074370995}) 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2948122509}) 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2580128500}) 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2948122509}) 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2567615577}) 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2948122509}) 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=393957688}) 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2948122509}) 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2641309529}) 2024-09-18T14:08:02.897 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2948122509}) 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1861327419}) 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.834+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.835+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2948122509}) 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.835+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2951746865}) 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.835+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.835+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2948122509}) 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.835+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2712518951}) 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.835+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1194632825}) 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1579879837}) 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1194632825}) 2024-09-18T14:08:02.898 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2684235484}) 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1194632825}) 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=227948808}) 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1194632825}) 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=897403737}) 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1194632825}) 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1987492866}) 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1194632825}) 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1538589279}) 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.899 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1194632825}) 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2107019257}) 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1194632825}) 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.836+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2595050086}) 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3556031320}) 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2340453438}) 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3556031320}) 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3058611713}) 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3556031320}) 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291700752}) 2024-09-18T14:08:02.900 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3556031320}) 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1552404469}) 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3556031320}) 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=332909011}) 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3556031320}) 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1799017387}) 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3556031320}) 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3986979480}) 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.901 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3556031320}) 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.837+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1015151983}) 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1731989444}) 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2546648406}) 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1731989444}) 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3547672050}) 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1731989444}) 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1782053659}) 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1731989444}) 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2105273217}) 2024-09-18T14:08:02.902 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1731989444}) 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4046317740}) 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1731989444}) 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3253470562}) 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1731989444}) 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=281928529}) 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1731989444}) 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.838+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2340963252}) 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596466546}) 2024-09-18T14:08:02.903 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3098541567}) 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596466546}) 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1563382923}) 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596466546}) 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1529780551}) 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596466546}) 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=542075606}) 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596466546}) 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3153658278}) 2024-09-18T14:08:02.904 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596466546}) 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1969648472}) 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596466546}) 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1161194495}) 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596466546}) 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.839+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2495804191}) 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982369918}) 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1666006566}) 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982369918}) 2024-09-18T14:08:02.905 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1839325653}) 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982369918}) 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2616962037}) 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982369918}) 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4047208070}) 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982369918}) 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4126423599}) 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982369918}) 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2564462567}) 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.906 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982369918}) 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3450174429}) 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982369918}) 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.840+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3834291790}) 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=23,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=42} crc {logm=2444087580,mdsmap=1531151339,mgr=3670438315,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=110469805,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=4061567604,osd_snap=2602928669}) 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=26,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=5,mon_config_key=4,monmap=3,osd_metadata=4,osd_pg_creating=1,osd_snap=45} crc {logm=1151126888,mdsmap=1531151339,mgr=1301466042,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1092725837,mon_config_key=1254902222,monmap=455718034,osd_metadata=3878874755,osd_pg_creating=4061567604,osd_snap=3909607954}) 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=23,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=42} crc {logm=2444087580,mdsmap=1531151339,mgr=3670438315,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=110469805,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=4061567604,osd_snap=2602928669}) 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=25,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39} crc {logm=1857971819,mdsmap=3140514009,mgr=2734874852,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1101483713,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=4061567604,osd_snap=14723567}) 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=23,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=42} crc {logm=2444087580,mdsmap=1531151339,mgr=3670438315,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=110469805,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=4061567604,osd_snap=2602928669}) 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=24,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=4,osd_pg_creating=1,osd_snap=41} crc {logm=936668852,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1101483713,mon_config_key=1810658514,monmap=455718034,osd_metadata=376733959,osd_pg_creating=4061567604,osd_snap=176307061}) 2024-09-18T14:08:02.907 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=23,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=42} crc {logm=2444087580,mdsmap=1531151339,mgr=3670438315,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=110469805,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=4061567604,osd_snap=2602928669}) 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=22,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=43} crc {logm=1529050302,mdsmap=1531151339,mgr=2394081750,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=4021528587,mon_config_key=2010730987,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=4061567604,osd_snap=1575550113}) 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=23,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=42} crc {logm=2444087580,mdsmap=1531151339,mgr=3670438315,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=110469805,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=4061567604,osd_snap=2602928669}) 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=25,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=39} crc {logm=257634025,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737493,mgr_metadata=270970526,mgrstat=4002840774,mon_config_key=1810658514,monmap=455718034,osd_metadata=3328057993,osd_pg_creating=4061567604,osd_snap=3714571179}) 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=23,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=42} crc {logm=2444087580,mdsmap=1531151339,mgr=3670438315,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=110469805,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=4061567604,osd_snap=2602928669}) 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=23,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=41} crc {logm=3019898426,mdsmap=1531151339,mgr=1259143633,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1101483713,mon_config_key=1810658514,monmap=455718034,osd_metadata=774608592,osd_pg_creating=4061567604,osd_snap=2080486437}) 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=23,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=42} crc {logm=2444087580,mdsmap=1531151339,mgr=3670438315,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=110469805,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=4061567604,osd_snap=2602928669}) 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=26,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=39} crc {logm=1151126888,mdsmap=1531151339,mgr=2734874852,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3028821863,mon_config_key=1810658514,monmap=3140514009,osd_metadata=2800182873,osd_pg_creating=4061567604,osd_snap=1792422050}) 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.908 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=23,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=42} crc {logm=2444087580,mdsmap=1531151339,mgr=3670438315,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=110469805,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=4061567604,osd_snap=2602928669}) 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.842+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=24,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39} crc {logm=4226816572,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1101483713,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=4061567604,osd_snap=245699290}) 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=1097886740,osdmap=724062592}) 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=20,osdmap=80} crc {osd_snap=2317562418,osdmap=1704792402}) 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=1097886740,osdmap=724062592}) 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=1320003504,osdmap=2199228579}) 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=1097886740,osdmap=724062592}) 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=20,osdmap=80} crc {osd_snap=2317562418,osdmap=1612584043}) 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=1097886740,osdmap=724062592}) 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=3581327324,osdmap=4225768996}) 2024-09-18T14:08:02.909 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.843+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.844+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=1097886740,osdmap=724062592}) 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.844+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=1320003504,osdmap=1395946696}) 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.844+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.844+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=1097886740,osdmap=724062592}) 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.844+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=3864076619,osdmap=2367789649}) 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.844+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.844+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=1097886740,osdmap=724062592}) 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.844+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=115730090,osdmap=2420121464}) 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.844+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.844+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=1097886740,osdmap=724062592}) 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.844+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=1441797386,osdmap=3012429491}) 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=244826265}) 2024-09-18T14:08:02.910 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=323948883}) 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=244826265}) 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3381701423}) 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=244826265}) 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1634478517}) 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=244826265}) 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3463172159}) 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=244826265}) 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=359462528}) 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.911 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=244826265}) 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2762470106}) 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=244826265}) 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4214240484}) 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=244826265}) 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.845+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=860804561}) 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2929346623}) 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2191527291}) 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2929346623}) 2024-09-18T14:08:02.912 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1431619728}) 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2929346623}) 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3815661864}) 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2929346623}) 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1927380294}) 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2929346623}) 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3208638420}) 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2929346623}) 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1510843203}) 2024-09-18T14:08:02.913 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2929346623}) 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2136097844}) 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2929346623}) 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.846+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1382271859}) 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1281285335}) 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2328702219}) 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1281285335}) 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2262900828}) 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1281285335}) 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=383359310}) 2024-09-18T14:08:02.914 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1281285335}) 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2479126062}) 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1281285335}) 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2666652923}) 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1281285335}) 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4089839165}) 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1281285335}) 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3951367188}) 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1281285335}) 2024-09-18T14:08:02.915 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.848+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1210195356}) 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=66} crc {osdmap=296743262}) 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=66} crc {osdmap=2968936344}) 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=66} crc {osdmap=296743262}) 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=65} crc {osdmap=3411379760}) 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=66} crc {osdmap=296743262}) 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=67} crc {osdmap=3479702853}) 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=66} crc {osdmap=296743262}) 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=66} crc {osdmap=3953503932}) 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.916 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=66} crc {osdmap=296743262}) 2024-09-18T14:08:02.917 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=68} crc {osdmap=2624231434}) 2024-09-18T14:08:02.917 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.917 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=66} crc {osdmap=296743262}) 2024-09-18T14:08:02.917 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=67} crc {osdmap=1733274169}) 2024-09-18T14:08:02.917 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.917 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=66} crc {osdmap=296743262}) 2024-09-18T14:08:02.917 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=69} crc {osdmap=4251494737}) 2024-09-18T14:08:02.917 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:08:02.917 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=66} crc {osdmap=296743262}) 2024-09-18T14:08:02.917 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:08:02.850+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=68} crc {osdmap=1723809029}) 2024-09-18T14:08:03.247 INFO:tasks.rados.rados.0.smithi084.stdout:1959: done (3 left) 2024-09-18T14:08:03.247 INFO:tasks.rados.rados.0.smithi084.stdout:1960: read oid 3 snap 202 2024-09-18T14:08:03.247 INFO:tasks.rados.rados.0.smithi084.stdout:1960: expect (ObjNum 662 snap 196 seq_num 662) 2024-09-18T14:08:03.248 INFO:tasks.rados.rados.0.smithi084.stdout:1961: write (excl) oid 39 current snap is 204 2024-09-18T14:08:03.248 INFO:tasks.rados.rados.0.smithi084.stdout:1961: seq_num 690 ranges {796830=793580,2360693=691568} 2024-09-18T14:08:03.255 INFO:tasks.rados.rados.0.smithi084.stdout:1961: writing smithi08434663-39 from 796830 to 1590410 tid 1 2024-09-18T14:08:03.264 INFO:tasks.rados.rados.0.smithi084.stdout:1961: writing smithi08434663-39 from 2360693 to 3052261 tid 2 2024-09-18T14:08:03.264 INFO:tasks.rados.rados.0.smithi084.stdout:1962: write oid 34 current snap is 204 2024-09-18T14:08:03.264 INFO:tasks.rados.rados.0.smithi084.stdout:1956: finishing write tid 3 to smithi08434663-12 2024-09-18T14:08:03.264 INFO:tasks.rados.rados.0.smithi084.stdout:1962: seq_num 691 ranges {413637=468829,1619175=435226,2542935=758255} 2024-09-18T14:08:03.269 INFO:tasks.rados.rados.0.smithi084.stdout:1962: writing smithi08434663-34 from 413637 to 882466 tid 1 2024-09-18T14:08:03.275 INFO:tasks.rados.rados.0.smithi084.stdout:1962: writing smithi08434663-34 from 1619175 to 2054401 tid 2 2024-09-18T14:08:03.282 INFO:tasks.rados.rados.0.smithi084.stdout:1962: writing smithi08434663-34 from 2542935 to 3301190 tid 3 2024-09-18T14:08:03.282 INFO:tasks.rados.rados.0.smithi084.stdout:1963: write (excl) oid 36 current snap is 204 2024-09-18T14:08:03.282 INFO:tasks.rados.rados.0.smithi084.stdout:1963: seq_num 692 ranges {430444=660431,1677657=695237,2970358=569527} 2024-09-18T14:08:03.287 INFO:tasks.rados.rados.0.smithi084.stdout:1963: writing smithi08434663-36 from 430444 to 1090875 tid 1 2024-09-18T14:08:03.294 INFO:tasks.rados.rados.0.smithi084.stdout:1963: writing smithi08434663-36 from 1677657 to 2372894 tid 2 2024-09-18T14:08:03.301 INFO:tasks.rados.rados.0.smithi084.stdout:1963: writing smithi08434663-36 from 2970358 to 3539885 tid 3 2024-09-18T14:08:03.302 INFO:tasks.rados.rados.0.smithi084.stdout:1964: delete oid 16 current snap is 204 2024-09-18T14:08:03.302 INFO:tasks.rados.rados.0.smithi084.stdout:1956: finishing write tid 4 to smithi08434663-12 2024-09-18T14:08:03.302 INFO:tasks.rados.rados.0.smithi084.stdout:1956: finishing write tid 5 to smithi08434663-12 2024-09-18T14:08:03.302 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 1016 (ObjNum 688 snap 204 seq_num 688) dirty exists 2024-09-18T14:08:03.302 INFO:tasks.rados.rados.0.smithi084.stdout:1956: left oid 12 (ObjNum 688 snap 204 seq_num 688) 2024-09-18T14:08:03.302 INFO:tasks.rados.rados.0.smithi084.stdout:1957: finishing write tid 1 to smithi08434663-47 2024-09-18T14:08:03.302 INFO:tasks.rados.rados.0.smithi084.stdout:1957: finishing write tid 2 to smithi08434663-47 2024-09-18T14:08:03.302 INFO:tasks.rados.rados.0.smithi084.stdout:1958: expect (ObjNum 665 snap 198 seq_num 665) 2024-09-18T14:08:03.405 INFO:tasks.rados.rados.0.smithi084.stdout:1957: finishing write tid 3 to smithi08434663-47 2024-09-18T14:08:03.405 INFO:tasks.rados.rados.0.smithi084.stdout:1957: finishing write tid 4 to smithi08434663-47 2024-09-18T14:08:03.405 INFO:tasks.rados.rados.0.smithi084.stdout:1957: finishing write tid 5 to smithi08434663-47 2024-09-18T14:08:03.405 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 603 (ObjNum 689 snap 204 seq_num 689) dirty exists 2024-09-18T14:08:03.405 INFO:tasks.rados.rados.0.smithi084.stdout:1957: left oid 47 (ObjNum 689 snap 204 seq_num 689) 2024-09-18T14:08:03.405 INFO:tasks.rados.rados.0.smithi084.stdout:1960: expect (ObjNum 662 snap 196 seq_num 662) 2024-09-18T14:08:03.506 INFO:tasks.rados.rados.0.smithi084.stdout:1961: finishing write tid 1 to smithi08434663-39 2024-09-18T14:08:03.506 INFO:tasks.rados.rados.0.smithi084.stdout:1961: finishing write tid 2 to smithi08434663-39 2024-09-18T14:08:03.506 INFO:tasks.rados.rados.0.smithi084.stdout:1961: finishing write tid 3 to smithi08434663-39 2024-09-18T14:08:03.506 INFO:tasks.rados.rados.0.smithi084.stdout:1956: done (7 left) 2024-09-18T14:08:03.506 INFO:tasks.rados.rados.0.smithi084.stdout:1957: done (6 left) 2024-09-18T14:08:03.506 INFO:tasks.rados.rados.0.smithi084.stdout:1958: done (5 left) 2024-09-18T14:08:03.506 INFO:tasks.rados.rados.0.smithi084.stdout:1960: done (4 left) 2024-09-18T14:08:03.506 INFO:tasks.rados.rados.0.smithi084.stdout:1964: done (3 left) 2024-09-18T14:08:03.506 INFO:tasks.rados.rados.0.smithi084.stdout:1965: rollback oid 17 current snap is 204 2024-09-18T14:08:03.506 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 17 to 202 2024-09-18T14:08:03.506 INFO:tasks.rados.rados.0.smithi084.stdout:1961: finishing write tid 4 to smithi08434663-39 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 606 (ObjNum 690 snap 204 seq_num 690) dirty exists 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:1961: left oid 39 (ObjNum 690 snap 204 seq_num 690) 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:1962: finishing write tid 1 to smithi08434663-34 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:1962: finishing write tid 2 to smithi08434663-34 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:1962: finishing write tid 3 to smithi08434663-34 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:1962: finishing write tid 4 to smithi08434663-34 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:1962: finishing write tid 5 to smithi08434663-34 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 900 (ObjNum 691 snap 204 seq_num 691) dirty exists 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:1962: left oid 34 (ObjNum 691 snap 204 seq_num 691) 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:1963: finishing write tid 1 to smithi08434663-36 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:1963: finishing write tid 2 to smithi08434663-36 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:1963: finishing write tid 3 to smithi08434663-36 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:1963: finishing write tid 4 to smithi08434663-36 2024-09-18T14:08:03.507 INFO:tasks.rados.rados.0.smithi084.stdout:1963: finishing write tid 5 to smithi08434663-36 2024-09-18T14:08:03.508 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 661 (ObjNum 692 snap 204 seq_num 692) dirty exists 2024-09-18T14:08:03.508 INFO:tasks.rados.rados.0.smithi084.stdout:1963: left oid 36 (ObjNum 692 snap 204 seq_num 692) 2024-09-18T14:08:03.508 INFO:tasks.rados.rados.0.smithi084.stdout:1961: done (3 left) 2024-09-18T14:08:03.508 INFO:tasks.rados.rados.0.smithi084.stdout:1962: done (2 left) 2024-09-18T14:08:03.508 INFO:tasks.rados.rados.0.smithi084.stdout:1963: done (1 left) 2024-09-18T14:08:03.508 INFO:tasks.rados.rados.0.smithi084.stdout:1966: snap_create 2024-09-18T14:08:03.508 INFO:tasks.rados.rados.0.smithi084.stdout:1965: finishing rollback tid 0 to smithi08434663-17 2024-09-18T14:08:03.509 INFO:tasks.rados.rados.0.smithi084.stdout:1965: finishing rollback tid 1 to smithi08434663-17 2024-09-18T14:08:03.509 INFO:tasks.rados.rados.0.smithi084.stdout:1965: finishing rollback tid 2 to smithi08434663-17 2024-09-18T14:08:03.509 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 854 (ObjNum 638 snap 189 seq_num 638) dirty exists 2024-09-18T14:08:03.855 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:08:03.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:08:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.955+0000 7f05246dd640 10 monclient: get_monmap_and_config 2024-09-18T14:08:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.955+0000 7f05246dd640 10 monclient: build_initial_monmap 2024-09-18T14:08:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.955+0000 7f05246dd640 10 monclient: monmap: 2024-09-18T14:08:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:03.956081+0000 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:03.956081+0000 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:03.957 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:03.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.955+0000 7f05246dd640 1 Processor -- start 2024-09-18T14:08:03.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.955+0000 7f05246dd640 1 -- start start 2024-09-18T14:08:03.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.955+0000 7f05246dd640 10 monclient: init 2024-09-18T14:08:03.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f05246dd640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:03.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f05246dd640 10 monclient: _add_conns ranks=[7,5,6,0,2,1,4,3,8] 2024-09-18T14:08:03.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f05246dd640 10 monclient(hunting): picked mon.noname-i con 0x7f051c06e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:03.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f05246dd640 10 monclient(hunting): picked mon.noname-h con 0x7f051c0a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:03.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f05246dd640 10 monclient(hunting): picked mon.noname-d con 0x7f051c0a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:03.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f05246dd640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051c05ed70 con 0x7f051c0a0cb0 2024-09-18T14:08:03.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f05246dd640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051c097620 con 0x7f051c0a1080 2024-09-18T14:08:03.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f05246dd640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051c097340 con 0x7f051c06e810 2024-09-18T14:08:03.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f05246dd640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:03.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f05246dd640 10 monclient(hunting): authenticate will time out at 1299.294451s 2024-09-18T14:08:03.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f0522c7a640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f051c0a1080 0x7f051c13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40906/0 (socket says 172.21.15.45:40906) 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f0522c7a640 1 -- 172.21.15.45:0/6361467 learned_addr learned my addr 172.21.15.45:0/6361467 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133018830 0 0) 0x7f051c097620 con 0x7f051c0a1080 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f0521477640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f0521477640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f0521477640 10 monclient(hunting): my global_id is 5253 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.956+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05080034e0 con 0x7f051c0a1080 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394990313 0 0) 0x7f051c05ed70 con 0x7f051c0a0cb0 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient(hunting): my global_id is 5198 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f051c097620 con 0x7f051c0a0cb0 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246355546 0 0) 0x7f051c097340 con 0x7f051c06e810 2024-09-18T14:08:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient(hunting): my global_id is 5353 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f051c05ed70 con 0x7f051c06e810 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1938046044 0 0) 0x7f05080034e0 con 0x7f051c0a1080 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 >> v1:172.21.15.84:6792/0 conn(0x7f051c06e810 legacy=0x7f051c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 >> v1:172.21.15.84:6791/0 conn(0x7f051c0a0cb0 legacy=0x7f051c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient: _finish_hunting 0 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 1 monclient: found mon.noname-d 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c0a1660 con 0x7f051c0a1080 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient: _finish_auth 0 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:03.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0514001bb0 con 0x7f051c0a1080 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient: dump: 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:03.967 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f05246dd640 5 monclient: authenticate success, global_id 5253 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f05246dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0514001d90 con 0x7f051c0a1080 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 1 -- 172.21.15.45:0/6361467 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0514004f20 con 0x7f051c0a1080 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f0521477640 10 monclient: dump: 2024-09-18T14:08:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f05246dd640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f05246dd640 4 set_mon_vals no callback set 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f05246dd640 10 monclient: shutdown 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f05246dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f05246dd640 1 -- 172.21.15.45:0/6361467 >> v1:172.21.15.45:6792/0 conn(0x7f051c0a1080 legacy=0x7f051c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f05246dd640 1 -- 172.21.15.45:0/6361467 shutdown_connections 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f05246dd640 1 -- 172.21.15.45:0/6361467 >> 172.21.15.45:0/6361467 conn(0x7f051c0a5bb0 msgr2=0x7f051c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.957+0000 7f05246dd640 1 -- 172.21.15.45:0/6361467 shutdown_connections 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.958+0000 7f05246dd640 1 -- 172.21.15.45:0/6361467 wait complete. 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.958+0000 7f05246dd640 10 monclient: build_initial_monmap 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.958+0000 7f05246dd640 10 monclient: monmap: 2024-09-18T14:08:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.958+0000 7f05246dd640 1 Processor -- start 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.958+0000 7f05246dd640 1 -- start start 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.958+0000 7f05246dd640 10 monclient: init 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05246dd640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05246dd640 10 monclient: _add_conns ranks=[4,8,1,3,5,7,6,0,2] 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05246dd640 10 monclient(hunting): picked mon.noname-c con 0x7f051c06e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05246dd640 10 monclient(hunting): picked mon.noname-e con 0x7f051c0a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05246dd640 10 monclient(hunting): picked mon.noname-f con 0x7f051c0a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05246dd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051c09fa10 con 0x7f051c0a1080 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05246dd640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051c0a2270 con 0x7f051c06e810 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05246dd640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051c1215e0 con 0x7f051c0a0cb0 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05246dd640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05246dd640 10 monclient(hunting): authenticate will time out at 1299.297424s 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f0522479640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f051c06e810 0x7f051c1353b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:50478/0 (socket says 172.21.15.45:50478) 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f0522479640 1 -- 172.21.15.45:0/1741517815 learned_addr learned my addr 172.21.15.45:0/1741517815 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185374560 0 0) 0x7f051c0a2270 con 0x7f051c06e810 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 10 monclient(hunting): my global_id is 164218 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05000034e0 con 0x7f051c06e810 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750837122 0 0) 0x7f051c1215e0 con 0x7f051c0a0cb0 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 10 monclient(hunting): my global_id is 5327 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f051c0a2270 con 0x7f051c0a0cb0 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123535974 0 0) 0x7f051c09fa10 con 0x7f051c0a1080 2024-09-18T14:08:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 10 monclient(hunting): my global_id is 164170 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f051c1215e0 con 0x7f051c0a1080 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.959+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2441388931 0 0) 0x7f051c1215e0 con 0x7f051c0a1080 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f051c09fa10 con 0x7f051c0a1080 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0514001b70 con 0x7f051c0a1080 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient(hunting): dump: 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:03.974 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1458943716 0 0) 0x7f05000034e0 con 0x7f051c06e810 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f051c1215e0 con 0x7f051c06e810 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1968508493 0 0) 0x7f051c0a2270 con 0x7f051c0a0cb0 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05000034e0 con 0x7f051c0a0cb0 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0518001aa0 con 0x7f051c06e810 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient(hunting): dump: 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f050c001a60 con 0x7f051c0a0cb0 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient(hunting): dump: 2024-09-18T14:08:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:03.978 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1946263161 0 0) 0x7f051c09fa10 con 0x7f051c0a1080 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 >> v1:172.21.15.45:6793/0 conn(0x7f051c0a0cb0 legacy=0x7f051c1468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 >> v1:172.21.15.45:6791/0 conn(0x7f051c06e810 legacy=0x7f051c1353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient: _finish_hunting 0 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 monclient: found mon.f 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c142440 con 0x7f051c0a1080 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient: _finish_auth 0 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05246dd640 5 monclient: authenticate success, global_id 164170 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05246dd640 10 monclient: _renew_subs 2024-09-18T14:08:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05246dd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05246dd640 1 -- 172.21.15.45:0/1741517815 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051c1224d0 con 0x7f051c0a1080 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05246dd640 10 monclient: _renew_subs 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05246dd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05246dd640 1 -- 172.21.15.45:0/1741517815 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f051c136ad0 con 0x7f051c0a1080 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0514001560 con 0x7f051c0a1080 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0514005190 con 0x7f051c0a1080 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f0520c76640 4 set_mon_vals no callback set 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.960+0000 7f05127fc640 10 monclient: dump: 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:03.980 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:03.981 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.961+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f05140101e0 con 0x7f051c0a1080 2024-09-18T14:08:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.961+0000 7f05246dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.961+0000 7f05246dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.961+0000 7f05246dd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:08:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.961+0000 7f05246dd640 1 -- 172.21.15.45:0/1741517815 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0504005010 con 0x7f051c0a1080 2024-09-18T14:08:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.961+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(428..428 src has 126..428) v4 ==== 3955+0+0 (unknown 2604472031 0 0) 0x7f0514046ec0 con 0x7f051c0a1080 2024-09-18T14:08:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.961+0000 7f0522479640 10 monclient: get_auth_request con 0x7f050003aaa0 auth_method 0 2024-09-18T14:08:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.966+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.1 v1:172.21.15.84:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f0514011780 con 0x7f051c0a1080 2024-09-18T14:08:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.966+0000 7f05127fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:03.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:03.966+0000 7f05127fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.188+0000 7f05246dd640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:08:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.188+0000 7f05246dd640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:08:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.188+0000 7f05246dd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:08:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.188+0000 7f05246dd640 1 -- 172.21.15.45:0/1741517815 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0504005840 con 0x7f051c0a1080 2024-09-18T14:08:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.189+0000 7f05127fc640 1 -- 172.21.15.45:0/1741517815 <== mon.1 v1:172.21.15.84:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 1220724865) 0x7f05140150e0 con 0x7f051c0a1080 2024-09-18T14:08:04.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.189+0000 7f05127fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:08:04.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.189+0000 7f05127fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:04.190 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":84,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":5,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:08:04.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.191+0000 7f05246dd640 1 -- 172.21.15.45:0/1741517815 >> v1:172.21.15.84:6812/31478 conn(0x7f050003aaa0 legacy=0x7f050003cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.191+0000 7f05246dd640 10 monclient: shutdown 2024-09-18T14:08:04.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.191+0000 7f05246dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:04.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.191+0000 7f05246dd640 1 -- 172.21.15.45:0/1741517815 >> v1:172.21.15.84:6789/0 conn(0x7f051c0a1080 legacy=0x7f051c149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.191+0000 7f05246dd640 1 -- 172.21.15.45:0/1741517815 shutdown_connections 2024-09-18T14:08:04.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.191+0000 7f05246dd640 1 -- 172.21.15.45:0/1741517815 >> 172.21.15.45:0/1741517815 conn(0x7f051c0a5bb0 msgr2=0x7f051c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:04.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.191+0000 7f05246dd640 1 -- 172.21.15.45:0/1741517815 shutdown_connections 2024-09-18T14:08:04.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.191+0000 7f05246dd640 1 -- 172.21.15.45:0/1741517815 wait complete. 2024-09-18T14:08:04.202 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:08:04.202 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-18T14:08:04.202 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:08:04.202 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:08:04.248 INFO:tasks.rados.rados.0.smithi084.stdout:1965: done (1 left) 2024-09-18T14:08:04.248 INFO:tasks.rados.rados.0.smithi084.stdout:1966: done (0 left) 2024-09-18T14:08:04.249 INFO:tasks.rados.rados.0.smithi084.stdout:1967: delete oid 15 current snap is 205 2024-09-18T14:08:04.251 INFO:tasks.rados.rados.0.smithi084.stdout:1967: done (0 left) 2024-09-18T14:08:04.251 INFO:tasks.rados.rados.0.smithi084.stdout:1968: write (excl) oid 21 current snap is 205 2024-09-18T14:08:04.251 INFO:tasks.rados.rados.0.smithi084.stdout:1968: seq_num 695 ranges {480865=486178,1453103=442564,2652627=519696} 2024-09-18T14:08:04.256 INFO:tasks.rados.rados.0.smithi084.stdout:1968: writing smithi08434663-21 from 480865 to 967043 tid 1 2024-09-18T14:08:04.261 INFO:tasks.rados.rados.0.smithi084.stdout:1968: writing smithi08434663-21 from 1453103 to 1895667 tid 2 2024-09-18T14:08:04.269 INFO:tasks.rados.rados.0.smithi084.stdout:1968: writing smithi08434663-21 from 2652627 to 3172323 tid 3 2024-09-18T14:08:04.269 INFO:tasks.rados.rados.0.smithi084.stdout:1969: rollback oid 22 current snap is 205 2024-09-18T14:08:04.269 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 22 to 202 2024-09-18T14:08:04.269 INFO:tasks.rados.rados.0.smithi084.stdout:1968: finishing write tid 1 to smithi08434663-21 2024-09-18T14:08:04.269 INFO:tasks.rados.rados.0.smithi084.stdout:1968: finishing write tid 2 to smithi08434663-21 2024-09-18T14:08:04.269 INFO:tasks.rados.rados.0.smithi084.stdout:1970: copy_from oid 30 from oid 44 current snap is 205 2024-09-18T14:08:04.269 INFO:tasks.rados.rados.0.smithi084.stdout:1971: read oid 34 snap 204 2024-09-18T14:08:04.269 INFO:tasks.rados.rados.0.smithi084.stdout:1971: expect (ObjNum 691 snap 204 seq_num 691) 2024-09-18T14:08:04.269 INFO:tasks.rados.rados.0.smithi084.stdout:1972: write oid 38 current snap is 205 2024-09-18T14:08:04.269 INFO:tasks.rados.rados.0.smithi084.stdout:1972: seq_num 696 ranges {497672=561427,1795232=702575,2963697=730968} 2024-09-18T14:08:04.274 INFO:tasks.rados.rados.0.smithi084.stdout:1972: writing smithi08434663-38 from 497672 to 1059099 tid 1 2024-09-18T14:08:04.282 INFO:tasks.rados.rados.0.smithi084.stdout:1972: writing smithi08434663-38 from 1795232 to 2497807 tid 2 2024-09-18T14:08:04.290 INFO:tasks.rados.rados.0.smithi084.stdout:1972: writing smithi08434663-38 from 2963697 to 3694665 tid 3 2024-09-18T14:08:04.290 INFO:tasks.rados.rados.0.smithi084.stdout:1973: read oid 45 snap 204 2024-09-18T14:08:04.290 INFO:tasks.rados.rados.0.smithi084.stdout:1973: expect (ObjNum 652 snap 191 seq_num 652) 2024-09-18T14:08:04.290 INFO:tasks.rados.rados.0.smithi084.stdout:1974: read oid 4 snap 204 2024-09-18T14:08:04.290 INFO:tasks.rados.rados.0.smithi084.stdout:1974: expect (ObjNum 644 snap 190 seq_num 644) 2024-09-18T14:08:04.290 INFO:tasks.rados.rados.0.smithi084.stdout:1975: write oid 12 current snap is 205 2024-09-18T14:08:04.290 INFO:tasks.rados.rados.0.smithi084.stdout:1968: finishing write tid 3 to smithi08434663-21 2024-09-18T14:08:04.290 INFO:tasks.rados.rados.0.smithi084.stdout:1968: finishing write tid 4 to smithi08434663-21 2024-09-18T14:08:04.290 INFO:tasks.rados.rados.0.smithi084.stdout:1968: finishing write tid 5 to smithi08434663-21 2024-09-18T14:08:04.290 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 603 (ObjNum 695 snap 205 seq_num 695) dirty exists 2024-09-18T14:08:04.290 INFO:tasks.rados.rados.0.smithi084.stdout:1968: left oid 21 (ObjNum 695 snap 205 seq_num 695) 2024-09-18T14:08:04.291 INFO:tasks.rados.rados.0.smithi084.stdout:1969: finishing rollback tid 0 to smithi08434663-22 2024-09-18T14:08:04.291 INFO:tasks.rados.rados.0.smithi084.stdout:1969: finishing rollback tid 1 to smithi08434663-22 2024-09-18T14:08:04.291 INFO:tasks.rados.rados.0.smithi084.stdout:1969: finishing rollback tid 2 to smithi08434663-22 2024-09-18T14:08:04.291 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 739 (ObjNum 615 snap 183 seq_num 615) dirty exists 2024-09-18T14:08:04.291 INFO:tasks.rados.rados.0.smithi084.stdout:1970: finishing copy_from racing read to smithi08434663-30 2024-09-18T14:08:04.291 INFO:tasks.rados.rados.0.smithi084.stdout:1972: finishing write tid 1 to smithi08434663-38 2024-09-18T14:08:04.291 INFO:tasks.rados.rados.0.smithi084.stdout:1975: seq_num 697 ranges {514479=636676,1853714=446233,2991120=542240} 2024-09-18T14:08:04.295 INFO:tasks.rados.rados.0.smithi084.stdout:1975: writing smithi08434663-12 from 514479 to 1151155 tid 1 2024-09-18T14:08:04.301 INFO:tasks.rados.rados.0.smithi084.stdout:1975: writing smithi08434663-12 from 1853714 to 2299947 tid 2 2024-09-18T14:08:04.308 INFO:tasks.rados.rados.0.smithi084.stdout:1975: writing smithi08434663-12 from 2991120 to 3533360 tid 3 2024-09-18T14:08:04.309 INFO:tasks.rados.rados.0.smithi084.stdout:1968: done (7 left) 2024-09-18T14:08:04.309 INFO:tasks.rados.rados.0.smithi084.stdout:1969: done (6 left) 2024-09-18T14:08:04.309 INFO:tasks.rados.rados.0.smithi084.stdout:1976: write (excl) oid 11 current snap is 205 2024-09-18T14:08:04.309 INFO:tasks.rados.rados.0.smithi084.stdout:1971: expect (ObjNum 691 snap 204 seq_num 691) 2024-09-18T14:08:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.310+0000 7f5888300640 10 monclient: get_monmap_and_config 2024-09-18T14:08:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.310+0000 7f5888300640 10 monclient: build_initial_monmap 2024-09-18T14:08:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.310+0000 7f5888300640 10 monclient: monmap: 2024-09-18T14:08:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:04.311007+0000 2024-09-18T14:08:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:04.311007+0000 2024-09-18T14:08:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:04.311 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.310+0000 7f5888300640 1 Processor -- start 2024-09-18T14:08:04.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.310+0000 7f5888300640 1 -- start start 2024-09-18T14:08:04.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.310+0000 7f5888300640 10 monclient: init 2024-09-18T14:08:04.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f5888300640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:04.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f5888300640 10 monclient: _add_conns ranks=[5,8,0,3,6,1,2,7,4] 2024-09-18T14:08:04.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f5888300640 10 monclient(hunting): picked mon.noname-h con 0x7f588006e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:04.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f5888300640 10 monclient(hunting): picked mon.noname-e con 0x7f58800a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:04.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f5888300640 10 monclient(hunting): picked mon.noname-a con 0x7f58800a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:04.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f5888300640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588005ed70 con 0x7f58800a1080 2024-09-18T14:08:04.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f5888300640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5880097620 con 0x7f588006e810 2024-09-18T14:08:04.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f5888300640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5880097340 con 0x7f58800a0cb0 2024-09-18T14:08:04.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f5888300640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:04.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f5888300640 10 monclient(hunting): authenticate will time out at 1299.649507s 2024-09-18T14:08:04.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588589b640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f58800a0cb0 0x7f5880137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:53324/0 (socket says 172.21.15.45:53324) 2024-09-18T14:08:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588589b640 1 -- 172.21.15.45:0/3037140015 learned_addr learned my addr 172.21.15.45:0/3037140015 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452082192 0 0) 0x7f588005ed70 con 0x7f58800a1080 2024-09-18T14:08:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 10 monclient(hunting): my global_id is 5409 2024-09-18T14:08:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f586c0034e0 con 0x7f58800a1080 2024-09-18T14:08:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707226326 0 0) 0x7f5880097340 con 0x7f58800a0cb0 2024-09-18T14:08:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 10 monclient(hunting): my global_id is 5336 2024-09-18T14:08:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f588005ed70 con 0x7f58800a0cb0 2024-09-18T14:08:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 535090074 0 0) 0x7f5880097620 con 0x7f588006e810 2024-09-18T14:08:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.311+0000 7f588509a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 10 monclient(hunting): my global_id is 5207 2024-09-18T14:08:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5880097340 con 0x7f588006e810 2024-09-18T14:08:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3357909205 0 0) 0x7f586c0034e0 con 0x7f58800a1080 2024-09-18T14:08:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 >> v1:172.21.15.45:6793/0 conn(0x7f58800a0cb0 legacy=0x7f5880137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 >> v1:172.21.15.84:6791/0 conn(0x7f588006e810 legacy=0x7f58800a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 10 monclient: _finish_hunting 0 2024-09-18T14:08:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 1 monclient: found mon.noname-a 2024-09-18T14:08:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:08:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58800a1660 con 0x7f58800a1080 2024-09-18T14:08:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 10 monclient: _finish_auth 0 2024-09-18T14:08:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f5888300640 5 monclient: authenticate success, global_id 5409 2024-09-18T14:08:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5878001bb0 con 0x7f58800a1080 2024-09-18T14:08:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f5888300640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:08:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 10 monclient: dump: 2024-09-18T14:08:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.331 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f5888300640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5878001d10 con 0x7f58800a1080 2024-09-18T14:08:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f5888300640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 1 -- 172.21.15.45:0/3037140015 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5878002240 con 0x7f58800a1080 2024-09-18T14:08:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f5888300640 4 set_mon_vals no callback set 2024-09-18T14:08:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f588509a640 10 monclient: dump: 2024-09-18T14:08:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.333 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f5888300640 10 monclient: shutdown 2024-09-18T14:08:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f5888300640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.312+0000 7f5888300640 1 -- 172.21.15.45:0/3037140015 >> v1:172.21.15.45:6789/0 conn(0x7f58800a1080 legacy=0x7f588013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.313+0000 7f5888300640 1 -- 172.21.15.45:0/3037140015 shutdown_connections 2024-09-18T14:08:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.313+0000 7f5888300640 1 -- 172.21.15.45:0/3037140015 >> 172.21.15.45:0/3037140015 conn(0x7f58800a5bb0 msgr2=0x7f58800a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.313+0000 7f5888300640 1 -- 172.21.15.45:0/3037140015 shutdown_connections 2024-09-18T14:08:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.313+0000 7f5888300640 1 -- 172.21.15.45:0/3037140015 wait complete. 2024-09-18T14:08:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.313+0000 7f5888300640 10 monclient: build_initial_monmap 2024-09-18T14:08:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.313+0000 7f5888300640 10 monclient: monmap: 2024-09-18T14:08:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:04.335 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.313+0000 7f5888300640 1 Processor -- start 2024-09-18T14:08:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f5888300640 1 -- start start 2024-09-18T14:08:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f5888300640 10 monclient: init 2024-09-18T14:08:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f5888300640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f5888300640 10 monclient: _add_conns ranks=[5,6,3,0,2,8,4,7,1] 2024-09-18T14:08:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f5888300640 10 monclient(hunting): picked mon.noname-h con 0x7f588006e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f5888300640 10 monclient(hunting): picked mon.noname-d con 0x7f58800a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f5888300640 10 monclient(hunting): picked mon.noname-g con 0x7f58800a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f5888300640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5880122da0 con 0x7f58800a1080 2024-09-18T14:08:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f5888300640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588013db90 con 0x7f588006e810 2024-09-18T14:08:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f5888300640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58801227e0 con 0x7f58800a0cb0 2024-09-18T14:08:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f5888300640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f5888300640 10 monclient(hunting): authenticate will time out at 1299.652547s 2024-09-18T14:08:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f588589b640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f58800a0cb0 0x7f58801466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40918/0 (socket says 172.21.15.45:40918) 2024-09-18T14:08:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f588589b640 1 -- 172.21.15.45:0/4032636836 learned_addr learned my addr 172.21.15.45:0/4032636836 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768576074 0 0) 0x7f58801227e0 con 0x7f58800a0cb0 2024-09-18T14:08:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 10 monclient(hunting): my global_id is 5262 2024-09-18T14:08:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58640034e0 con 0x7f58800a0cb0 2024-09-18T14:08:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404591719 0 0) 0x7f588013db90 con 0x7f588006e810 2024-09-18T14:08:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 10 monclient(hunting): my global_id is 5216 2024-09-18T14:08:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58801227e0 con 0x7f588006e810 2024-09-18T14:08:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299168628 0 0) 0x7f5880122da0 con 0x7f58800a1080 2024-09-18T14:08:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 10 monclient(hunting): my global_id is 154371 2024-09-18T14:08:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.314+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f588013db90 con 0x7f58800a1080 2024-09-18T14:08:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1850381882 0 0) 0x7f58640034e0 con 0x7f58800a0cb0 2024-09-18T14:08:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5880122da0 con 0x7f58800a0cb0 2024-09-18T14:08:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5870001690 con 0x7f58800a0cb0 2024-09-18T14:08:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:08:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient(hunting): dump: 2024-09-18T14:08:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.340 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.340 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.340 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.340 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.340 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.340 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.340 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.340 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.340 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.340 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:04.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3145568696 0 0) 0x7f58801227e0 con 0x7f588006e810 2024-09-18T14:08:04.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f58640034e0 con 0x7f588006e810 2024-09-18T14:08:04.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f587c001aa0 con 0x7f588006e810 2024-09-18T14:08:04.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:08:04.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient(hunting): dump: 2024-09-18T14:08:04.341 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.341 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.341 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.341 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.341 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.341 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.341 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.342 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.342 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.342 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.342 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.342 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.342 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.342 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.342 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.342 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2289500296 0 0) 0x7f588013db90 con 0x7f58800a1080 2024-09-18T14:08:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f58801227e0 con 0x7f58800a1080 2024-09-18T14:08:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f58780013a0 con 0x7f58800a1080 2024-09-18T14:08:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:08:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient(hunting): dump: 2024-09-18T14:08:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.344 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4235715989 0 0) 0x7f5880122da0 con 0x7f58800a0cb0 2024-09-18T14:08:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 >> v1:172.21.15.84:6791/0 conn(0x7f588006e810 legacy=0x7f588013d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 >> v1:172.21.15.84:6790/0 conn(0x7f58800a1080 legacy=0x7f5880149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient: _finish_hunting 0 2024-09-18T14:08:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 monclient: found mon.d 2024-09-18T14:08:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:08:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58801202a0 con 0x7f58800a0cb0 2024-09-18T14:08:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient: _finish_auth 0 2024-09-18T14:08:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f5888300640 5 monclient: authenticate success, global_id 5262 2024-09-18T14:08:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f5888300640 10 monclient: _renew_subs 2024-09-18T14:08:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f5888300640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:08:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f5888300640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5880122500 con 0x7f58800a0cb0 2024-09-18T14:08:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f5888300640 10 monclient: _renew_subs 2024-09-18T14:08:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f5888300640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:08:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f5888300640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5880121630 con 0x7f58800a0cb0 2024-09-18T14:08:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5870004a50 con 0x7f58800a0cb0 2024-09-18T14:08:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5870004f00 con 0x7f58800a0cb0 2024-09-18T14:08:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f5884899640 4 set_mon_vals no callback set 2024-09-18T14:08:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.315+0000 7f58767fc640 10 monclient: dump: 2024-09-18T14:08:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.348 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f5888300640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f5888300640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5868006580 con 0x7f5868002830 2024-09-18T14:08:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f587000fed0 con 0x7f58800a0cb0 2024-09-18T14:08:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(429..429 src has 126..429) v4 ==== 3927+0+0 (unknown 1058389548 0 0) 0x7f5870046ae0 con 0x7f58800a0cb0 2024-09-18T14:08:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378485679 0 0) 0x7f5868006580 con 0x7f5868002830 2024-09-18T14:08:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f58767fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f58767fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f58767fc640 10 monclient(hunting): my global_id is 5418 2024-09-18T14:08:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5880122da0 con 0x7f5868002830 2024-09-18T14:08:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f588689d640 10 monclient: get_auth_request con 0x7f586403aa50 auth_method 0 2024-09-18T14:08:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3670448002 0 0) 0x7f5880122da0 con 0x7f5868002830 2024-09-18T14:08:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5868006580 con 0x7f5868002830 2024-09-18T14:08:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f587c0021b0 con 0x7f5868002830 2024-09-18T14:08:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.316+0000 7f58767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.317+0000 7f58767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.317+0000 7f58767fc640 10 monclient: dump: 2024-09-18T14:08:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.350 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.350 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.350 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.350 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.350 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.350 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.350 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.350 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.350 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.350 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.350 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.317+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1034724954 0 0) 0x7f5868006580 con 0x7f5868002830 2024-09-18T14:08:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.317+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f586800a0a0 con 0x7f5868002830 2024-09-18T14:08:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.317+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f587c002690 con 0x7f5868002830 2024-09-18T14:08:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.317+0000 7f58767fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.317+0000 7f58767fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.317+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 >> v1:172.21.15.45:6789/0 conn(0x7f5868002830 legacy=0x7f5868004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.317+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 >> v1:172.21.15.45:6789/0 conn(0x7f5868002830 legacy=0x7f5868004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f5888300640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f5888300640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f58680074d0 con 0x7f58680068a0 2024-09-18T14:08:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787254124 0 0) 0x7f58680074d0 con 0x7f58680068a0 2024-09-18T14:08:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 10 monclient(hunting): my global_id is 5427 2024-09-18T14:08:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5868006580 con 0x7f58680068a0 2024-09-18T14:08:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1093386918 0 0) 0x7f5868006580 con 0x7f58680068a0 2024-09-18T14:08:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f58680074d0 con 0x7f58680068a0 2024-09-18T14:08:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f587c003000 con 0x7f58680068a0 2024-09-18T14:08:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 10 monclient: dump: 2024-09-18T14:08:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.354 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 469873781 0 0) 0x7f58680074d0 con 0x7f58680068a0 2024-09-18T14:08:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.323+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5868008750 con 0x7f58680068a0 2024-09-18T14:08:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3778518515) 0x7f587c003310 con 0x7f58680068a0 2024-09-18T14:08:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f58767fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f58767fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 >> v1:172.21.15.45:6789/0 conn(0x7f58680068a0 legacy=0x7f5868006d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f58767fc640 1 -- 172.21.15.45:0/4032636836 >> v1:172.21.15.45:6789/0 conn(0x7f58680068a0 legacy=0x7f5868006d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:04.355 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:04.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:04.355 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:04.355 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:04.355 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:04.355 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:04.356 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:04.356 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:04.356 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:04.356 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:04.356 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:04.356 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:04.356 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:04.356 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:04.356 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:04.356 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.356 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 5, 2024-09-18T14:08:04.357 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:04.357 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:04.357 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:04.357 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.357 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.357 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.357 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.357 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.357 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.359 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.359 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.359 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.359 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.359 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.359 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.359 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:04.359 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:04.360 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:04.360 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:04.360 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:04.360 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:04.360 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:04.360 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:04.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:04.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:04.360 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:04.360 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:04.360 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:04.361 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:04.361 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:04.361 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:04.361 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:04.361 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.361 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.361 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.361 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.361 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.361 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.361 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.362 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.362 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.362 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.362 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:04.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.362 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:04.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.362 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:04.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:04.362 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.362 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:04.363 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.363 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.363 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:04.363 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:04.363 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.364 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.364 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.364 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:04.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:04.364 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.364 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.364 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:04.365 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.365 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.365 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.365 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:04.365 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:04.365 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.365 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.365 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.366 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:04.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:04.366 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.366 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:04.366 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.366 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:04.367 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:04.367 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.367 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.367 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.367 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:04.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:04.367 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.369 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:04.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:04.369 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.369 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.369 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.369 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:04.369 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.370 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.370 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:04.370 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:04.370 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.370 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.370 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.371 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:04.371 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:04.371 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.371 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.371 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:04.371 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.371 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.371 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.372 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:04.372 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:04.372 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.372 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.372 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.372 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:04.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:04.372 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.373 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.373 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:04.373 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.373 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.373 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.373 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:04.373 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:04.374 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.374 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.374 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.374 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:04.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:04.374 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.374 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.374 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:04.375 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.375 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.375 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:04.375 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:04.375 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.375 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.375 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.376 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:04.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:04.376 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.376 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.376 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:04.376 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.377 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.377 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.377 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:04.377 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:04.377 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.377 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.377 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.377 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.377 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.377 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:04.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:04.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.378 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:04.378 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:04.378 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:04.378 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.378 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.378 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:04.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.378 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:04.378 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:04.379 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:08:04.379 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.379 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.379 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:04.379 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f5888300640 1 -- 172.21.15.45:0/4032636836 >> v1:172.21.15.84:6812/31478 conn(0x7f586403aa50 legacy=0x7f586403cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f5888300640 10 monclient: shutdown 2024-09-18T14:08:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f5888300640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f5888300640 1 -- 172.21.15.45:0/4032636836 >> v1:172.21.15.45:6792/0 conn(0x7f58800a0cb0 legacy=0x7f58801466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f588689d640 1 -- 172.21.15.45:0/4032636836 reap_dead start 2024-09-18T14:08:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f588689d640 1 -- 172.21.15.45:0/4032636836 reap_dead start 2024-09-18T14:08:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f5888300640 1 -- 172.21.15.45:0/4032636836 shutdown_connections 2024-09-18T14:08:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f5888300640 1 -- 172.21.15.45:0/4032636836 >> 172.21.15.45:0/4032636836 conn(0x7f58800a5bb0 msgr2=0x7f58800a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f5888300640 1 -- 172.21.15.45:0/4032636836 shutdown_connections 2024-09-18T14:08:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.324+0000 7f5888300640 1 -- 172.21.15.45:0/4032636836 wait complete. 2024-09-18T14:08:04.382 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:04.382 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:08:04.382 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:08:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.479+0000 7f102399d640 10 monclient: get_monmap_and_config 2024-09-18T14:08:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.479+0000 7f102399d640 10 monclient: build_initial_monmap 2024-09-18T14:08:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.479+0000 7f102399d640 10 monclient: monmap: 2024-09-18T14:08:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:04.480382+0000 2024-09-18T14:08:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:04.480382+0000 2024-09-18T14:08:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 1 Processor -- start 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 1 -- start start 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 10 monclient: init 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 10 monclient: _add_conns ranks=[5,7,2,6,1,4,0,3,8] 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 10 monclient(hunting): picked mon.noname-h con 0x7f101c09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 10 monclient(hunting): picked mon.noname-i con 0x7f101c09fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 10 monclient(hunting): picked mon.noname-b con 0x7f101c09ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c05ed70 con 0x7f101c09ff70 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c09dfc0 con 0x7f101c09f7d0 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c09dce0 con 0x7f101c09fba0 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f102399d640 10 monclient(hunting): authenticate will time out at 1299.818863s 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f1021f3a640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f101c09ff70 0x7f101c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47194/0 (socket says 172.21.15.45:47194) 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.480+0000 7f1021f3a640 1 -- 172.21.15.45:0/3345372144 learned_addr learned my addr 172.21.15.45:0/3345372144 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157203568 0 0) 0x7f101c05ed70 con 0x7f101c09ff70 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient(hunting): my global_id is 144677 2024-09-18T14:08:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10080034e0 con 0x7f101c09ff70 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3671365982 0 0) 0x7f101c09dce0 con 0x7f101c09fba0 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient(hunting): my global_id is 5362 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f101c05ed70 con 0x7f101c09fba0 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031549576 0 0) 0x7f101c09dfc0 con 0x7f101c09f7d0 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient(hunting): my global_id is 5225 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f101c09dce0 con 0x7f101c09f7d0 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 425339507 0 0) 0x7f10080034e0 con 0x7f101c09ff70 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 >> v1:172.21.15.84:6792/0 conn(0x7f101c09fba0 legacy=0x7f101c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 >> v1:172.21.15.84:6791/0 conn(0x7f101c09f7d0 legacy=0x7f101c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient: _finish_hunting 0 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 monclient: found mon.noname-b 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c0a1c00 con 0x7f101c09ff70 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient: _finish_auth 0 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1014001bb0 con 0x7f101c09ff70 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f102399d640 5 monclient: authenticate success, global_id 144677 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f102399d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:08:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient: dump: 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f102399d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1014001d90 con 0x7f101c09ff70 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 1 -- 172.21.15.45:0/3345372144 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1014004f20 con 0x7f101c09ff70 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f102399d640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f102399d640 4 set_mon_vals no callback set 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.481+0000 7f1013fff640 10 monclient: dump: 2024-09-18T14:08:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.482+0000 7f102399d640 10 monclient: shutdown 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.482+0000 7f102399d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.482+0000 7f102399d640 1 -- 172.21.15.45:0/3345372144 >> v1:172.21.15.45:6790/0 conn(0x7f101c09ff70 legacy=0x7f101c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.482+0000 7f102399d640 1 -- 172.21.15.45:0/3345372144 shutdown_connections 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.482+0000 7f102399d640 1 -- 172.21.15.45:0/3345372144 >> 172.21.15.45:0/3345372144 conn(0x7f101c0630b0 msgr2=0x7f101c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.482+0000 7f102399d640 1 -- 172.21.15.45:0/3345372144 shutdown_connections 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.482+0000 7f102399d640 1 -- 172.21.15.45:0/3345372144 wait complete. 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.482+0000 7f102399d640 10 monclient: build_initial_monmap 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.482+0000 7f102399d640 10 monclient: monmap: 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 1 Processor -- start 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 1 -- start start 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 10 monclient: init 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 10 monclient: _add_conns ranks=[5,0,8,4,6,1,3,7,2] 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 10 monclient(hunting): picked mon.noname-h con 0x7f101c09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 10 monclient(hunting): picked mon.noname-a con 0x7f101c09fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 10 monclient(hunting): picked mon.noname-e con 0x7f101c09ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c0a2560 con 0x7f101c09fba0 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c120740 con 0x7f101c09f7d0 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c11e930 con 0x7f101c09ff70 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f102399d640 10 monclient(hunting): authenticate will time out at 1299.821716s 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f1020f38640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f101c09fba0 0x7f101c11d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55010/0 (socket says 172.21.15.45:55010) 2024-09-18T14:08:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.483+0000 7f1020f38640 1 -- 172.21.15.45:0/499839039 learned_addr learned my addr 172.21.15.45:0/499839039 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280717032 0 0) 0x7f101c11e930 con 0x7f101c09ff70 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): my global_id is 5345 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10000034e0 con 0x7f101c09ff70 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602404919 0 0) 0x7f101c120740 con 0x7f101c09f7d0 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): my global_id is 5234 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f101c11e930 con 0x7f101c09f7d0 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295524675 0 0) 0x7f101c0a2560 con 0x7f101c09fba0 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): my global_id is 5436 2024-09-18T14:08:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f101c120740 con 0x7f101c09fba0 2024-09-18T14:08:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 209891745 0 0) 0x7f101c11e930 con 0x7f101c09f7d0 2024-09-18T14:08:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f101c0a2560 con 0x7f101c09f7d0 2024-09-18T14:08:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3991965002 0 0) 0x7f10000034e0 con 0x7f101c09ff70 2024-09-18T14:08:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f101c11e930 con 0x7f101c09ff70 2024-09-18T14:08:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f100c001640 con 0x7f101c09f7d0 2024-09-18T14:08:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:08:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): dump: 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.503 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2322602350 0 0) 0x7f101c120740 con 0x7f101c09fba0 2024-09-18T14:08:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10000034e0 con 0x7f101c09fba0 2024-09-18T14:08:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1014001af0 con 0x7f101c09ff70 2024-09-18T14:08:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): dump: 2024-09-18T14:08:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1018001a50 con 0x7f101c09fba0 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient(hunting): dump: 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1631782264 0 0) 0x7f101c0a2560 con 0x7f101c09f7d0 2024-09-18T14:08:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 >> v1:172.21.15.45:6793/0 conn(0x7f101c09ff70 legacy=0x7f101c149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 >> v1:172.21.15.45:6789/0 conn(0x7f101c09fba0 legacy=0x7f101c11d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient: _finish_hunting 0 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 monclient: found mon.h 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c120460 con 0x7f101c09f7d0 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient: _finish_auth 0 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f102399d640 5 monclient: authenticate success, global_id 5234 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f102399d640 10 monclient: _renew_subs 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f102399d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f102399d640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.84:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c120150 con 0x7f101c09f7d0 2024-09-18T14:08:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.5 v1:172.21.15.84:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f100c002000 con 0x7f101c09f7d0 2024-09-18T14:08:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f102399d640 10 monclient: _renew_subs 2024-09-18T14:08:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f102399d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:08:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f102399d640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.84:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c11ddc0 con 0x7f101c09f7d0 2024-09-18T14:08:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.5 v1:172.21.15.84:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f100c0024b0 con 0x7f101c09f7d0 2024-09-18T14:08:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:08:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10117fa640 10 monclient: dump: 2024-09-18T14:08:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.509 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.484+0000 7f10137fe640 4 set_mon_vals no callback set 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.485+0000 7f102399d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.485+0000 7f102399d640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f101c04c4a0 con 0x7f101c02aa40 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.485+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.5 v1:172.21.15.84:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f100c00d500 con 0x7f101c09f7d0 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.485+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.5 v1:172.21.15.84:6791/0 8 ==== osd_map(429..429 src has 126..429) v4 ==== 3927+0+0 (unknown 1058389548 0 0) 0x7f100c044240 con 0x7f101c09f7d0 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.485+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302844272 0 0) 0x7f101c04c4a0 con 0x7f101c02aa40 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.485+0000 7f10117fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.485+0000 7f10117fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.485+0000 7f10117fa640 10 monclient(hunting): my global_id is 144686 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.485+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f101c0a2560 con 0x7f101c02aa40 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.485+0000 7f1021f3a640 10 monclient: get_auth_request con 0x7f100003a9d0 auth_method 0 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2776842513 0 0) 0x7f101c0a2560 con 0x7f101c02aa40 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f101c04c4a0 con 0x7f101c02aa40 2024-09-18T14:08:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10180021a0 con 0x7f101c02aa40 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 10 monclient: dump: 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.512 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3495274669 0 0) 0x7f101c04c4a0 con 0x7f101c02aa40 2024-09-18T14:08:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f101c02ae10 con 0x7f101c02aa40 2024-09-18T14:08:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1018002680 con 0x7f101c02aa40 2024-09-18T14:08:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 >> v1:172.21.15.45:6790/0 conn(0x7f101c02aa40 legacy=0x7f101c05e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.486+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 >> v1:172.21.15.45:6790/0 conn(0x7f101c02aa40 legacy=0x7f101c05e310 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f102399d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f102399d640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f101c11f6a0 con 0x7f101c099a40 2024-09-18T14:08:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394530797 0 0) 0x7f10000034e0 con 0x7f101c099a40 2024-09-18T14:08:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f10117fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f10117fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f10117fa640 10 monclient(hunting): my global_id is 144695 2024-09-18T14:08:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f101c04c4a0 con 0x7f101c099a40 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1246295193 0 0) 0x7f101c04c4a0 con 0x7f101c099a40 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10000034e0 con 0x7f101c099a40 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10180058e0 con 0x7f101c099a40 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f10117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f10117fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.492+0000 7f10117fa640 10 monclient: dump: 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 294028536 0 0) 0x7f10000034e0 con 0x7f101c099a40 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f101c04b720 con 0x7f101c099a40 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 1384422318) 0x7f10180082a0 con 0x7f101c099a40 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f10117fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f10117fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 >> v1:172.21.15.45:6790/0 conn(0x7f101c099a40 legacy=0x7f101c14f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f10117fa640 1 -- 172.21.15.45:0/499839039 >> v1:172.21.15.45:6790/0 conn(0x7f101c099a40 legacy=0x7f101c14f310 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:04.517 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:04.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:04.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:04.517 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:04.517 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:04.517 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:04.517 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:04.517 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 5, 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.518 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.519 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:04.520 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.521 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:04.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.523 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.526 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:04.526 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.526 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.526 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:04.526 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:04.526 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.526 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.526 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:04.527 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:04.528 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.528 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.528 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:04.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:04.528 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.528 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.528 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:04.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.530 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.532 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:04.532 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:04.532 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.532 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.532 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:04.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:04.532 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.532 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:04.533 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.533 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.533 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:04.533 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:04.533 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.534 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.534 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:04.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:04.534 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.534 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.535 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:04.535 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.535 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.535 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.535 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:04.535 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:04.535 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.535 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.536 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:04.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.537 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:04.537 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:04.537 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:04.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.537 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.537 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:04.537 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f102399d640 1 -- 172.21.15.45:0/499839039 >> v1:172.21.15.84:6812/31478 conn(0x7f100003a9d0 legacy=0x7f100003ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f102399d640 10 monclient: shutdown 2024-09-18T14:08:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f102399d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f102399d640 1 -- 172.21.15.45:0/499839039 >> v1:172.21.15.84:6791/0 conn(0x7f101c09f7d0 legacy=0x7f101c11cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f1021f3a640 1 -- 172.21.15.45:0/499839039 reap_dead start 2024-09-18T14:08:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f1021f3a640 1 -- 172.21.15.45:0/499839039 reap_dead start 2024-09-18T14:08:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f102399d640 1 -- 172.21.15.45:0/499839039 shutdown_connections 2024-09-18T14:08:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f102399d640 1 -- 172.21.15.45:0/499839039 >> 172.21.15.45:0/499839039 conn(0x7f101c0630b0 msgr2=0x7f101c0a44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f102399d640 1 -- 172.21.15.45:0/499839039 shutdown_connections 2024-09-18T14:08:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.493+0000 7f102399d640 1 -- 172.21.15.45:0/499839039 wait complete. 2024-09-18T14:08:04.539 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:04.540 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:08:04.540 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:08:04.540 INFO:tasks.rados.rados.0.smithi084.stdout:1970: finishing copy_from to smithi08434663-30 2024-09-18T14:08:04.540 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 846 (ObjNum 634 snap 189 seq_num 634) dirty exists 2024-09-18T14:08:04.540 INFO:tasks.rados.rados.0.smithi084.stdout:1972: finishing write tid 2 to smithi08434663-38 2024-09-18T14:08:04.540 INFO:tasks.rados.rados.0.smithi084.stdout:1973: expect (ObjNum 652 snap 191 seq_num 652) 2024-09-18T14:08:04.626 INFO:tasks.rados.rados.0.smithi084.stdout:1972: finishing write tid 3 to smithi08434663-38 2024-09-18T14:08:04.626 INFO:tasks.rados.rados.0.smithi084.stdout:1972: finishing write tid 4 to smithi08434663-38 2024-09-18T14:08:04.626 INFO:tasks.rados.rados.0.smithi084.stdout:1972: finishing write tid 5 to smithi08434663-38 2024-09-18T14:08:04.627 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 634 (ObjNum 696 snap 205 seq_num 696) dirty exists 2024-09-18T14:08:04.627 INFO:tasks.rados.rados.0.smithi084.stdout:1972: left oid 38 (ObjNum 696 snap 205 seq_num 696) 2024-09-18T14:08:04.627 INFO:tasks.rados.rados.0.smithi084.stdout:1974: expect (ObjNum 644 snap 190 seq_num 644) 2024-09-18T14:08:04.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.637+0000 7f53cd49f640 10 monclient: get_monmap_and_config 2024-09-18T14:08:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.637+0000 7f53cd49f640 10 monclient: build_initial_monmap 2024-09-18T14:08:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.637+0000 7f53cd49f640 10 monclient: monmap: 2024-09-18T14:08:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:04.638695+0000 2024-09-18T14:08:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:04.638695+0000 2024-09-18T14:08:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:04.639 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:04.639 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:04.639 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:04.639 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:04.639 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:04.639 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 1 Processor -- start 2024-09-18T14:08:04.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 1 -- start start 2024-09-18T14:08:04.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 10 monclient: init 2024-09-18T14:08:04.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:04.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 10 monclient: _add_conns ranks=[5,2,1,7,4,0,6,8,3] 2024-09-18T14:08:04.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 10 monclient(hunting): picked mon.noname-h con 0x7f53c809f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:04.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 10 monclient(hunting): picked mon.noname-b con 0x7f53c809fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:04.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 10 monclient(hunting): picked mon.noname-f con 0x7f53c809ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:04.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c805ed70 con 0x7f53c809ff70 2024-09-18T14:08:04.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c809dfc0 con 0x7f53c809fba0 2024-09-18T14:08:04.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c809dce0 con 0x7f53c809f7d0 2024-09-18T14:08:04.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:04.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.638+0000 7f53cd49f640 10 monclient(hunting): authenticate will time out at 1299.976894s 2024-09-18T14:08:04.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c67fc640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f53c809fba0 0x7f53c8137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47220/0 (socket says 172.21.15.45:47220) 2024-09-18T14:08:04.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c67fc640 1 -- 172.21.15.45:0/3286835863 learned_addr learned my addr 172.21.15.45:0/3286835863 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:04.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078643327 0 0) 0x7f53c809dfc0 con 0x7f53c809fba0 2024-09-18T14:08:04.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient(hunting): my global_id is 144704 2024-09-18T14:08:04.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b00034e0 con 0x7f53c809fba0 2024-09-18T14:08:04.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740399863 0 0) 0x7f53c805ed70 con 0x7f53c809ff70 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient(hunting): my global_id is 164179 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c809dfc0 con 0x7f53c809ff70 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693941092 0 0) 0x7f53c809dce0 con 0x7f53c809f7d0 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient(hunting): my global_id is 5243 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c805ed70 con 0x7f53c809f7d0 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3018075769 0 0) 0x7f53c809dfc0 con 0x7f53c809ff70 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 >> v1:172.21.15.84:6791/0 conn(0x7f53c809f7d0 legacy=0x7f53c8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 >> v1:172.21.15.45:6790/0 conn(0x7f53c809fba0 legacy=0x7f53c8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 monclient: found mon.noname-f 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c80a1c00 con 0x7f53c809ff70 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient: _finish_auth 0 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f53c0001b90 con 0x7f53c809ff70 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53cd49f640 5 monclient: authenticate success, global_id 164179 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53cd49f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient: dump: 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.651 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53cd49f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f53c0001cf0 con 0x7f53c809ff70 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 1 -- 172.21.15.45:0/3286835863 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f53c00022a0 con 0x7f53c809ff70 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53cd49f640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53cd49f640 4 set_mon_vals no callback set 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.639+0000 7f53c5ffb640 10 monclient: dump: 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.652 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.640+0000 7f53cd49f640 10 monclient: shutdown 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.640+0000 7f53cd49f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.640+0000 7f53cd49f640 1 -- 172.21.15.45:0/3286835863 >> v1:172.21.15.84:6789/0 conn(0x7f53c809ff70 legacy=0x7f53c813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.640+0000 7f53cd49f640 1 -- 172.21.15.45:0/3286835863 shutdown_connections 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.640+0000 7f53cd49f640 1 -- 172.21.15.45:0/3286835863 >> 172.21.15.45:0/3286835863 conn(0x7f53c80630b0 msgr2=0x7f53c80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.640+0000 7f53cd49f640 1 -- 172.21.15.45:0/3286835863 shutdown_connections 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.640+0000 7f53cd49f640 1 -- 172.21.15.45:0/3286835863 wait complete. 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.640+0000 7f53cd49f640 10 monclient: build_initial_monmap 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.640+0000 7f53cd49f640 10 monclient: monmap: 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 1 Processor -- start 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 1 -- start start 2024-09-18T14:08:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 10 monclient: init 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 10 monclient: _add_conns ranks=[7,6,3,0,1,2,4,5,8] 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 10 monclient(hunting): picked mon.noname-i con 0x7f53c809f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 10 monclient(hunting): picked mon.noname-d con 0x7f53c809fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 10 monclient(hunting): picked mon.noname-g con 0x7f53c809ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c809d310 con 0x7f53c809ff70 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c80a2560 con 0x7f53c809fba0 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c8141f20 con 0x7f53c809f7d0 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53cd49f640 10 monclient(hunting): authenticate will time out at 1299.979622s 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53c67fc640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f53c809fba0 0x7f53c8131830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40922/0 (socket says 172.21.15.45:40922) 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53c67fc640 1 -- 172.21.15.45:0/2060406191 learned_addr learned my addr 172.21.15.45:0/2060406191 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145200800 0 0) 0x7f53c80a2560 con 0x7f53c809fba0 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53aeffd640 10 monclient(hunting): my global_id is 5271 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b40034e0 con 0x7f53c809fba0 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711489962 0 0) 0x7f53c8141f20 con 0x7f53c809f7d0 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53aeffd640 10 monclient(hunting): my global_id is 5371 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.641+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c80a2560 con 0x7f53c809f7d0 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328307207 0 0) 0x7f53c809d310 con 0x7f53c809ff70 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): my global_id is 154380 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c8141f20 con 0x7f53c809ff70 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2741832494 0 0) 0x7f53b40034e0 con 0x7f53c809fba0 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53c809d310 con 0x7f53c809fba0 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f53bc001670 con 0x7f53c809fba0 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): dump: 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.659 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3155152342 0 0) 0x7f53c80a2560 con 0x7f53c809f7d0 2024-09-18T14:08:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53b40034e0 con 0x7f53c809f7d0 2024-09-18T14:08:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f53b8001560 con 0x7f53c809f7d0 2024-09-18T14:08:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:08:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): dump: 2024-09-18T14:08:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1817374029 0 0) 0x7f53c8141f20 con 0x7f53c809ff70 2024-09-18T14:08:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53c80a2560 con 0x7f53c809ff70 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f53c0001430 con 0x7f53c809ff70 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient(hunting): dump: 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2552929967 0 0) 0x7f53c809d310 con 0x7f53c809fba0 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 >> v1:172.21.15.84:6792/0 conn(0x7f53c809f7d0 legacy=0x7f53c8141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 >> v1:172.21.15.84:6790/0 conn(0x7f53c809ff70 legacy=0x7f53c8149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient: _finish_hunting 0 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 monclient: found mon.d 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c8121980 con 0x7f53c809fba0 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient: _finish_auth 0 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53cd49f640 5 monclient: authenticate success, global_id 5271 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53cd49f640 10 monclient: _renew_subs 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53cd49f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53cd49f640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c81425f0 con 0x7f53c809fba0 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53cd49f640 10 monclient: _renew_subs 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53cd49f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53cd49f640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c81320e0 con 0x7f53c809fba0 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f53bc004a50 con 0x7f53c809fba0 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f53bc004f00 con 0x7f53c809fba0 2024-09-18T14:08:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53c57fa640 4 set_mon_vals no callback set 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.642+0000 7f53aeffd640 10 monclient: dump: 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f53bc00fed0 con 0x7f53c809fba0 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53cd49f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53cd49f640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f53a8006580 con 0x7f53a8002830 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(429..429 src has 126..429) v4 ==== 3927+0+0 (unknown 1058389548 0 0) 0x7f53bc046af0 con 0x7f53c809fba0 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53c6ffd640 10 monclient: get_auth_request con 0x7f53b403aa40 auth_method 0 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263577880 0 0) 0x7f53a8006580 con 0x7f53a8002830 2024-09-18T14:08:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53aeffd640 10 monclient(hunting): my global_id is 164227 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c809d310 con 0x7f53a8002830 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4050409806 0 0) 0x7f53c809d310 con 0x7f53a8002830 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53a8006580 con 0x7f53a8002830 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f53c00026d0 con 0x7f53a8002830 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.643+0000 7f53aeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.644+0000 7f53aeffd640 10 monclient: dump: 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.667 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.644+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3613745791 0 0) 0x7f53a8006580 con 0x7f53a8002830 2024-09-18T14:08:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.644+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53a800a0a0 con 0x7f53a8002830 2024-09-18T14:08:04.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.644+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f53c00048b0 con 0x7f53a8002830 2024-09-18T14:08:04.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.644+0000 7f53aeffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:04.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.644+0000 7f53aeffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:04.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.644+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 >> v1:172.21.15.45:6791/0 conn(0x7f53a8002830 legacy=0x7f53a8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.644+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 >> v1:172.21.15.45:6791/0 conn(0x7f53a8002830 legacy=0x7f53a8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:04.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53cd49f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:04.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53cd49f640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f53a8009660 con 0x7f53a8009040 2024-09-18T14:08:04.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480606239 0 0) 0x7f53a8009660 con 0x7f53a8009040 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 10 monclient(hunting): my global_id is 164236 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a8006580 con 0x7f53a8009040 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2471675135 0 0) 0x7f53a8006580 con 0x7f53a8009040 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53a8009660 con 0x7f53a8009040 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f53c0003190 con 0x7f53a8009040 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 10 monclient: dump: 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.673 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 181533277 0 0) 0x7f53a8009660 con 0x7f53a8009040 2024-09-18T14:08:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.650+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f53a8009920 con 0x7f53a8009040 2024-09-18T14:08:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3061994801) 0x7f53c0004b60 con 0x7f53a8009040 2024-09-18T14:08:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53aeffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53aeffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 >> v1:172.21.15.45:6791/0 conn(0x7f53a8009040 legacy=0x7f53a8008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53aeffd640 1 -- 172.21.15.45:0/2060406191 >> v1:172.21.15.45:6791/0 conn(0x7f53a8009040 legacy=0x7f53a8008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:04.674 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 5, 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.690 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:04.690 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:04.690 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.690 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.690 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:04.690 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.690 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.690 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.690 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.691 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:04.692 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.693 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.694 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:04.694 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53cd49f640 1 -- 172.21.15.45:0/2060406191 >> v1:172.21.15.84:6812/31478 conn(0x7f53b403aa40 legacy=0x7f53b403cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53cd49f640 10 monclient: shutdown 2024-09-18T14:08:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53cd49f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53cd49f640 1 -- 172.21.15.45:0/2060406191 >> v1:172.21.15.45:6792/0 conn(0x7f53c809fba0 legacy=0x7f53c8131830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53c77fe640 1 -- 172.21.15.45:0/2060406191 reap_dead start 2024-09-18T14:08:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53c77fe640 1 -- 172.21.15.45:0/2060406191 reap_dead start 2024-09-18T14:08:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53cd49f640 1 -- 172.21.15.45:0/2060406191 shutdown_connections 2024-09-18T14:08:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53cd49f640 1 -- 172.21.15.45:0/2060406191 >> 172.21.15.45:0/2060406191 conn(0x7f53c80630b0 msgr2=0x7f53c80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53cd49f640 1 -- 172.21.15.45:0/2060406191 shutdown_connections 2024-09-18T14:08:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.651+0000 7f53cd49f640 1 -- 172.21.15.45:0/2060406191 wait complete. 2024-09-18T14:08:04.696 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:04.696 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:08:04.696 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:08:04.781 INFO:tasks.rados.rados.0.smithi084.stdout:1975: finishing write tid 1 to smithi08434663-12 2024-09-18T14:08:04.781 INFO:tasks.rados.rados.0.smithi084.stdout:1975: finishing write tid 2 to smithi08434663-12 2024-09-18T14:08:04.781 INFO:tasks.rados.rados.0.smithi084.stdout:1975: finishing write tid 3 to smithi08434663-12 2024-09-18T14:08:04.781 INFO:tasks.rados.rados.0.smithi084.stdout:1975: finishing write tid 4 to smithi08434663-12 2024-09-18T14:08:04.781 INFO:tasks.rados.rados.0.smithi084.stdout:1976: seq_num 698 ranges {531286=711925,1912196=706244,3018543=712743} 2024-09-18T14:08:04.787 INFO:tasks.rados.rados.0.smithi084.stdout:1976: writing smithi08434663-11 from 531286 to 1243211 tid 1 2024-09-18T14:08:04.795 INFO:tasks.rados.rados.0.smithi084.stdout:1976: writing smithi08434663-11 from 1912196 to 2618440 tid 2 2024-09-18T14:08:04.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.796+0000 7f2b1af9d640 10 monclient: get_monmap_and_config 2024-09-18T14:08:04.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.796+0000 7f2b1af9d640 10 monclient: build_initial_monmap 2024-09-18T14:08:04.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.796+0000 7f2b1af9d640 10 monclient: monmap: 2024-09-18T14:08:04.796 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:04.796 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.796 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:04.797088+0000 2024-09-18T14:08:04.796 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:04.797088+0000 2024-09-18T14:08:04.797 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:04.797 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.797 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:04.797 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:04.797 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:04.797 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:04.797 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:04.797 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:04.797 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:04.797 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:04.797 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:04.797 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.807 INFO:tasks.rados.rados.0.smithi084.stdout:1976: writing smithi08434663-11 from 3018543 to 3731286 tid 3 2024-09-18T14:08:04.807 INFO:tasks.rados.rados.0.smithi084.stdout:1970: done (6 left) 2024-09-18T14:08:04.807 INFO:tasks.rados.rados.0.smithi084.stdout:1971: done (5 left) 2024-09-18T14:08:04.807 INFO:tasks.rados.rados.0.smithi084.stdout:1972: done (4 left) 2024-09-18T14:08:04.807 INFO:tasks.rados.rados.0.smithi084.stdout:1973: done (3 left) 2024-09-18T14:08:04.807 INFO:tasks.rados.rados.0.smithi084.stdout:1974: done (2 left) 2024-09-18T14:08:04.807 INFO:tasks.rados.rados.0.smithi084.stdout:1977: read oid 49 snap -1 2024-09-18T14:08:04.807 INFO:tasks.rados.rados.0.smithi084.stdout:1977: expect (ObjNum 649 snap 191 seq_num 649) 2024-09-18T14:08:04.807 INFO:tasks.rados.rados.0.smithi084.stdout:1978: snap_remove snap 204 2024-09-18T14:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.796+0000 7f2b1af9d640 1 Processor -- start 2024-09-18T14:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.796+0000 7f2b1af9d640 1 -- start start 2024-09-18T14:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.796+0000 7f2b1af9d640 10 monclient: init 2024-09-18T14:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b1af9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b1af9d640 10 monclient: _add_conns ranks=[7,8,0,4,1,5,2,3,6] 2024-09-18T14:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b1af9d640 10 monclient(hunting): picked mon.noname-i con 0x7f2b1406e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b1af9d640 10 monclient(hunting): picked mon.noname-e con 0x7f2b140a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b1af9d640 10 monclient(hunting): picked mon.noname-a con 0x7f2b140a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b1af9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1405ed70 con 0x7f2b140a1080 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b1af9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b14097620 con 0x7f2b1406e810 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b1af9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b14097340 con 0x7f2b140a0cb0 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b1af9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b1af9d640 10 monclient(hunting): authenticate will time out at 1300.135561s 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b13fff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f2b140a0cb0 0x7f2b14137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:53338/0 (socket says 172.21.15.45:53338) 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b13fff640 1 -- 172.21.15.45:0/3891288724 learned_addr learned my addr 172.21.15.45:0/3891288724 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b1953a640 1 --1- 172.21.15.45:0/3891288724 >> v1:172.21.15.45:6789/0 conn(0x7f2b140a1080 0x7f2b1413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55026/0 (socket says 172.21.15.45:55026) 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049853838 0 0) 0x7f2b1405ed70 con 0x7f2b140a1080 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 10 monclient(hunting): my global_id is 5445 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b000034e0 con 0x7f2b140a1080 2024-09-18T14:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017262901 0 0) 0x7f2b14097340 con 0x7f2b140a0cb0 2024-09-18T14:08:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 10 monclient(hunting): my global_id is 5354 2024-09-18T14:08:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b1405ed70 con 0x7f2b140a0cb0 2024-09-18T14:08:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001749594 0 0) 0x7f2b14097620 con 0x7f2b1406e810 2024-09-18T14:08:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 10 monclient(hunting): my global_id is 5380 2024-09-18T14:08:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.797+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b14097340 con 0x7f2b1406e810 2024-09-18T14:08:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 997832639 0 0) 0x7f2b000034e0 con 0x7f2b140a1080 2024-09-18T14:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 >> v1:172.21.15.45:6793/0 conn(0x7f2b140a0cb0 legacy=0x7f2b14137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 >> v1:172.21.15.84:6792/0 conn(0x7f2b1406e810 legacy=0x7f2b140a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 10 monclient: _finish_hunting 0 2024-09-18T14:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 1 monclient: found mon.noname-a 2024-09-18T14:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b140a1660 con 0x7f2b140a1080 2024-09-18T14:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 10 monclient: _finish_auth 0 2024-09-18T14:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2b0c001ae0 con 0x7f2b140a1080 2024-09-18T14:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b1af9d640 5 monclient: authenticate success, global_id 5445 2024-09-18T14:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 10 monclient: dump: 2024-09-18T14:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b1af9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2b0c001cc0 con 0x7f2b140a1080 2024-09-18T14:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 1 -- 172.21.15.45:0/3891288724 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2b0c004b20 con 0x7f2b140a1080 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b1af9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b1af9d640 4 set_mon_vals no callback set 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b137fe640 10 monclient: dump: 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b1af9d640 10 monclient: shutdown 2024-09-18T14:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b1af9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b1af9d640 1 -- 172.21.15.45:0/3891288724 >> v1:172.21.15.45:6789/0 conn(0x7f2b140a1080 legacy=0x7f2b1413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b1af9d640 1 -- 172.21.15.45:0/3891288724 shutdown_connections 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b1af9d640 1 -- 172.21.15.45:0/3891288724 >> 172.21.15.45:0/3891288724 conn(0x7f2b140a5bb0 msgr2=0x7f2b140a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b1af9d640 1 -- 172.21.15.45:0/3891288724 shutdown_connections 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.798+0000 7f2b1af9d640 1 -- 172.21.15.45:0/3891288724 wait complete. 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.799+0000 7f2b1af9d640 10 monclient: build_initial_monmap 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.799+0000 7f2b1af9d640 10 monclient: monmap: 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.799+0000 7f2b1af9d640 1 Processor -- start 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.799+0000 7f2b1af9d640 1 -- start start 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.799+0000 7f2b1af9d640 10 monclient: init 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b1af9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b1af9d640 10 monclient: _add_conns ranks=[7,6,8,5,1,4,2,0,3] 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b1af9d640 10 monclient(hunting): picked mon.noname-i con 0x7f2b1406e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b1af9d640 10 monclient(hunting): picked mon.noname-d con 0x7f2b140a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b1af9d640 10 monclient(hunting): picked mon.noname-e con 0x7f2b140a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b1af9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b14122690 con 0x7f2b140a0cb0 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b1af9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1413da20 con 0x7f2b1406e810 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b1af9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1413e6e0 con 0x7f2b140a1080 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b1af9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b1af9d640 10 monclient(hunting): authenticate will time out at 1300.138494s 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b1953a640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f2b140a1080 0x7f2b14149c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:53348/0 (socket says 172.21.15.45:53348) 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b13fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f2b140a0cb0 0x7f2b14146560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40932/0 (socket says 172.21.15.45:40932) 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b18d39640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f2b1406e810 0x7f2b1413d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:46670/0 (socket says 172.21.15.45:46670) 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b1953a640 1 -- 172.21.15.45:0/1637848135 learned_addr learned my addr 172.21.15.45:0/1637848135 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215409013 0 0) 0x7f2b1413da20 con 0x7f2b1406e810 2024-09-18T14:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 10 monclient(hunting): my global_id is 5389 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2af40034e0 con 0x7f2b1406e810 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275626464 0 0) 0x7f2b1413e6e0 con 0x7f2b140a1080 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 10 monclient(hunting): my global_id is 5363 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b1413da20 con 0x7f2b140a1080 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544228678 0 0) 0x7f2b14122690 con 0x7f2b140a0cb0 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 10 monclient(hunting): my global_id is 5280 2024-09-18T14:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.800+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b1413e6e0 con 0x7f2b140a0cb0 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 644014394 0 0) 0x7f2af40034e0 con 0x7f2b1406e810 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2b14122690 con 0x7f2b1406e810 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2b04001c50 con 0x7f2b1406e810 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient(hunting): dump: 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 655416157 0 0) 0x7f2b1413da20 con 0x7f2b140a1080 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2af40034e0 con 0x7f2b140a1080 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2b0c001aa0 con 0x7f2b140a1080 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient(hunting): dump: 2024-09-18T14:08:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.823 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2459078669 0 0) 0x7f2b1413e6e0 con 0x7f2b140a0cb0 2024-09-18T14:08:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2b1413da20 con 0x7f2b140a0cb0 2024-09-18T14:08:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2afc001550 con 0x7f2b140a0cb0 2024-09-18T14:08:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient(hunting): dump: 2024-09-18T14:08:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.826 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.826 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.826 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.826 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.826 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 782544292 0 0) 0x7f2b14122690 con 0x7f2b1406e810 2024-09-18T14:08:04.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 >> v1:172.21.15.45:6793/0 conn(0x7f2b140a1080 legacy=0x7f2b14149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 >> v1:172.21.15.45:6792/0 conn(0x7f2b140a0cb0 legacy=0x7f2b14146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:08:04.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 monclient: found mon.i 2024-09-18T14:08:04.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:04.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:08:04.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b141223b0 con 0x7f2b1406e810 2024-09-18T14:08:04.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient: _finish_auth 0 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b1af9d640 5 monclient: authenticate success, global_id 5389 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b1af9d640 10 monclient: _renew_subs 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b1af9d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b1af9d640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b14122970 con 0x7f2b1406e810 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b1af9d640 10 monclient: _renew_subs 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b1af9d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b1af9d640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1413dec0 con 0x7f2b1406e810 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2b040020e0 con 0x7f2b1406e810 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2b04002650 con 0x7f2b1406e810 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b10ff9640 10 monclient: dump: 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.801+0000 7f2b12ffd640 4 set_mon_vals no callback set 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.802+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2b0400ffe0 con 0x7f2b1406e810 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.802+0000 7f2b1af9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.802+0000 7f2b1af9d640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2b1404c4a0 con 0x7f2b141214e0 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.802+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(429..429 src has 126..429) v4 ==== 3927+0+0 (unknown 1058389548 0 0) 0x7f2b040469b0 con 0x7f2b1406e810 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.802+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115808194 0 0) 0x7f2b1404c4a0 con 0x7f2b141214e0 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.802+0000 7f2b10ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.802+0000 7f2b10ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.802+0000 7f2b10ff9640 10 monclient(hunting): my global_id is 5289 2024-09-18T14:08:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.802+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b14122690 con 0x7f2b141214e0 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.802+0000 7f2b1953a640 10 monclient: get_auth_request con 0x7f2af403aa60 auth_method 0 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2592985699 0 0) 0x7f2b14122690 con 0x7f2b141214e0 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2b1404c4a0 con 0x7f2b141214e0 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2afc004ae0 con 0x7f2b141214e0 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 10 monclient: dump: 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.834 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 264887060 0 0) 0x7f2b1404c4a0 con 0x7f2b141214e0 2024-09-18T14:08:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b1404b720 con 0x7f2b141214e0 2024-09-18T14:08:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2afc006fc0 con 0x7f2b141214e0 2024-09-18T14:08:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 >> v1:172.21.15.45:6792/0 conn(0x7f2b141214e0 legacy=0x7f2b1409b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.803+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 >> v1:172.21.15.45:6792/0 conn(0x7f2b141214e0 legacy=0x7f2b1409b980 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.811+0000 7f2b1af9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.811+0000 7f2b1af9d640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2b14062dd0 con 0x7f2b1411fdd0 2024-09-18T14:08:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.811+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788916927 0 0) 0x7f2b14062dd0 con 0x7f2b1411fdd0 2024-09-18T14:08:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.811+0000 7f2b10ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.811+0000 7f2b10ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.811+0000 7f2b10ff9640 10 monclient(hunting): my global_id is 5298 2024-09-18T14:08:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.811+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b1404c4a0 con 0x7f2b1411fdd0 2024-09-18T14:08:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 567690313 0 0) 0x7f2b1404c4a0 con 0x7f2b1411fdd0 2024-09-18T14:08:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2b14062dd0 con 0x7f2b1411fdd0 2024-09-18T14:08:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2afc0055f0 con 0x7f2b1411fdd0 2024-09-18T14:08:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 10 monclient: dump: 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.839 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3034600494 0 0) 0x7f2b14062dd0 con 0x7f2b1411fdd0 2024-09-18T14:08:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2b140670e0 con 0x7f2b1411fdd0 2024-09-18T14:08:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 359776505) 0x7f2afc006a40 con 0x7f2b1411fdd0 2024-09-18T14:08:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 >> v1:172.21.15.45:6792/0 conn(0x7f2b1411fdd0 legacy=0x7f2b1409c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.812+0000 7f2b10ff9640 1 -- 172.21.15.45:0/1637848135 >> v1:172.21.15.45:6792/0 conn(0x7f2b1411fdd0 legacy=0x7f2b1409c090 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:04.840 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:04.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:04.840 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:04.840 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:04.840 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:04.840 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:04.840 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:04.840 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:04.840 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:04.840 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:04.840 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:04.841 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:04.841 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:04.841 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:04.841 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:04.841 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.841 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:08:04.841 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:04.841 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:04.841 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:04.841 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.841 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.841 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.842 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.842 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.842 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.842 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.842 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.842 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.842 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.842 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:04.842 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:04.842 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.842 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.843 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.843 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.843 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.843 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.843 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.843 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.843 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.843 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.843 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:04.843 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:04.843 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:04.844 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:04.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.847 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:04.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.849 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.850 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.852 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:04.853 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.854 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.856 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:04.857 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:04.858 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.860 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:04.860 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.813+0000 7f2b1af9d640 1 -- 172.21.15.45:0/1637848135 >> v1:172.21.15.84:6812/31478 conn(0x7f2af403aa60 legacy=0x7f2af403cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.813+0000 7f2b1af9d640 10 monclient: shutdown 2024-09-18T14:08:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.813+0000 7f2b1af9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.813+0000 7f2b1af9d640 1 -- 172.21.15.45:0/1637848135 >> v1:172.21.15.84:6792/0 conn(0x7f2b1406e810 legacy=0x7f2b1413d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.813+0000 7f2b1953a640 1 -- 172.21.15.45:0/1637848135 reap_dead start 2024-09-18T14:08:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.813+0000 7f2b1953a640 1 -- 172.21.15.45:0/1637848135 reap_dead start 2024-09-18T14:08:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.813+0000 7f2b1af9d640 1 -- 172.21.15.45:0/1637848135 shutdown_connections 2024-09-18T14:08:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.813+0000 7f2b1af9d640 1 -- 172.21.15.45:0/1637848135 >> 172.21.15.45:0/1637848135 conn(0x7f2b140a5bb0 msgr2=0x7f2b140a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.813+0000 7f2b1af9d640 1 -- 172.21.15.45:0/1637848135 shutdown_connections 2024-09-18T14:08:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.813+0000 7f2b1af9d640 1 -- 172.21.15.45:0/1637848135 wait complete. 2024-09-18T14:08:04.862 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:04.862 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:08:04.863 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:08:04.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.960+0000 7f7de964f640 10 monclient: get_monmap_and_config 2024-09-18T14:08:04.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.960+0000 7f7de964f640 10 monclient: build_initial_monmap 2024-09-18T14:08:04.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.960+0000 7f7de964f640 10 monclient: monmap: 2024-09-18T14:08:04.960 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:04.960 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:04.961043+0000 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:04.961043+0000 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:04.961 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.960+0000 7f7de964f640 1 Processor -- start 2024-09-18T14:08:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.960+0000 7f7de964f640 1 -- start start 2024-09-18T14:08:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.960+0000 7f7de964f640 10 monclient: init 2024-09-18T14:08:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de964f640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de964f640 10 monclient: _add_conns ranks=[7,8,2,0,4,6,3,5,1] 2024-09-18T14:08:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de964f640 10 monclient(hunting): picked mon.noname-i con 0x7f7de406e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de964f640 10 monclient(hunting): picked mon.noname-e con 0x7f7de40a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de964f640 10 monclient(hunting): picked mon.noname-b con 0x7f7de40a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de964f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de405ed70 con 0x7f7de40a1080 2024-09-18T14:08:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de964f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de4097620 con 0x7f7de406e810 2024-09-18T14:08:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de964f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de4097340 con 0x7f7de40a0cb0 2024-09-18T14:08:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de964f640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de964f640 10 monclient(hunting): authenticate will time out at 1300.299443s 2024-09-18T14:08:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de37fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f7de40a1080 0x7f7de413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47222/0 (socket says 172.21.15.45:47222) 2024-09-18T14:08:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de27fc640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f7de40a0cb0 0x7f7de4137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:53360/0 (socket says 172.21.15.45:53360) 2024-09-18T14:08:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de37fe640 1 -- 172.21.15.45:0/3639251623 learned_addr learned my addr 172.21.15.45:0/3639251623 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de1ffb640 1 -- 172.21.15.45:0/3639251623 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860398149 0 0) 0x7f7de4097340 con 0x7f7de40a0cb0 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de1ffb640 10 monclient(hunting): my global_id is 5372 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de1ffb640 1 -- 172.21.15.45:0/3639251623 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dd00034e0 con 0x7f7de40a0cb0 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de1ffb640 1 -- 172.21.15.45:0/3639251623 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301828270 0 0) 0x7f7de405ed70 con 0x7f7de40a1080 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de1ffb640 10 monclient(hunting): my global_id is 144713 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.961+0000 7f7de1ffb640 1 -- 172.21.15.45:0/3639251623 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de4097340 con 0x7f7de40a1080 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 1 -- 172.21.15.45:0/3639251623 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1236696455 0 0) 0x7f7dd00034e0 con 0x7f7de40a0cb0 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 1 -- 172.21.15.45:0/3639251623 >> v1:172.21.15.84:6792/0 conn(0x7f7de406e810 legacy=0x7f7de40a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 1 -- 172.21.15.45:0/3639251623 >> v1:172.21.15.45:6790/0 conn(0x7f7de40a1080 legacy=0x7f7de413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:08:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 1 monclient: found mon.noname-e 2024-09-18T14:08:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:08:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 1 -- 172.21.15.45:0/3639251623 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de40a1660 con 0x7f7de40a0cb0 2024-09-18T14:08:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 10 monclient: _finish_auth 0 2024-09-18T14:08:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de964f640 5 monclient: authenticate success, global_id 5372 2024-09-18T14:08:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 1 -- 172.21.15.45:0/3639251623 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7dcc001830 con 0x7f7de40a0cb0 2024-09-18T14:08:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de964f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:08:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 10 monclient: dump: 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.973 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de964f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 1 -- 172.21.15.45:0/3639251623 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7dcc001990 con 0x7f7de40a0cb0 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 1 -- 172.21.15.45:0/3639251623 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7dcc0047a0 con 0x7f7de40a0cb0 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de964f640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de964f640 4 set_mon_vals no callback set 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de1ffb640 10 monclient: dump: 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.974 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.975 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.976 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de964f640 10 monclient: shutdown 2024-09-18T14:08:04.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de964f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:04.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de964f640 1 -- 172.21.15.45:0/3639251623 >> v1:172.21.15.45:6793/0 conn(0x7f7de40a0cb0 legacy=0x7f7de4137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de964f640 1 -- 172.21.15.45:0/3639251623 shutdown_connections 2024-09-18T14:08:04.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de964f640 1 -- 172.21.15.45:0/3639251623 >> 172.21.15.45:0/3639251623 conn(0x7f7de40a5bb0 msgr2=0x7f7de40a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:04.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de964f640 1 -- 172.21.15.45:0/3639251623 shutdown_connections 2024-09-18T14:08:04.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.962+0000 7f7de964f640 1 -- 172.21.15.45:0/3639251623 wait complete. 2024-09-18T14:08:04.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.963+0000 7f7de964f640 10 monclient: build_initial_monmap 2024-09-18T14:08:04.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.963+0000 7f7de964f640 10 monclient: monmap: 2024-09-18T14:08:04.976 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:04.976 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:04.977 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:04.977 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:04.977 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:04.977 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:04.977 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:04.977 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:04.977 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:04.977 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:04.977 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:04.977 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:04.977 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:04.977 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.963+0000 7f7de964f640 1 Processor -- start 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.963+0000 7f7de964f640 1 -- start start 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.963+0000 7f7de964f640 10 monclient: init 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7de964f640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7de964f640 10 monclient: _add_conns ranks=[6,5,7,2,4,3,0,8,1] 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7de964f640 10 monclient(hunting): picked mon.noname-d con 0x7f7de406e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7de964f640 10 monclient(hunting): picked mon.noname-h con 0x7f7de40a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7de964f640 10 monclient(hunting): picked mon.noname-i con 0x7f7de40a1080 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7de964f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de4122cb0 con 0x7f7de40a0cb0 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7de964f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de413daa0 con 0x7f7de406e810 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7de964f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de41226f0 con 0x7f7de40a1080 2024-09-18T14:08:04.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7de964f640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7de964f640 10 monclient(hunting): authenticate will time out at 1300.302457s 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7de2ffd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f7de406e810 0x7f7de413d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40954/0 (socket says 172.21.15.45:40954) 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7de2ffd640 1 -- 172.21.15.45:0/3986750440 learned_addr learned my addr 172.21.15.45:0/3986750440 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199650771 0 0) 0x7f7de413daa0 con 0x7f7de406e810 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 10 monclient(hunting): my global_id is 5307 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc40034e0 con 0x7f7de406e810 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179897790 0 0) 0x7f7de4122cb0 con 0x7f7de40a0cb0 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 10 monclient(hunting): my global_id is 5252 2024-09-18T14:08:04.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de413daa0 con 0x7f7de40a0cb0 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3748872142 0 0) 0x7f7dc40034e0 con 0x7f7de406e810 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7de4122cb0 con 0x7f7de406e810 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900331841 0 0) 0x7f7de41226f0 con 0x7f7de40a1080 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 10 monclient(hunting): my global_id is 5407 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc40034e0 con 0x7f7de40a1080 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.964+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7dd4001f20 con 0x7f7de406e810 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 10 monclient(hunting): dump: 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.980 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.981 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.982 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:04.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3070858712 0 0) 0x7f7de4122cb0 con 0x7f7de406e810 2024-09-18T14:08:04.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 >> v1:172.21.15.84:6792/0 conn(0x7f7de40a1080 legacy=0x7f7de4149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 >> v1:172.21.15.84:6791/0 conn(0x7f7de40a0cb0 legacy=0x7f7de41465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 10 monclient: _finish_hunting 0 2024-09-18T14:08:04.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 1 monclient: found mon.d 2024-09-18T14:08:04.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:04.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:08:04.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de4120200 con 0x7f7de406e810 2024-09-18T14:08:04.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 10 monclient: _finish_auth 0 2024-09-18T14:08:04.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7de964f640 5 monclient: authenticate success, global_id 5307 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7de964f640 10 monclient: _renew_subs 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7de964f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7de964f640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de4122410 con 0x7f7de406e810 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7de964f640 10 monclient: _renew_subs 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7de964f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7de964f640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de4121540 con 0x7f7de406e810 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7dd4002080 con 0x7f7de406e810 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7dd4002970 con 0x7f7de406e810 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7de17fa640 4 set_mon_vals no callback set 2024-09-18T14:08:04.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 10 monclient: dump: 2024-09-18T14:08:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.984 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.965+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7dd4002b70 con 0x7f7de406e810 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7de964f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7de964f640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7dc8006580 con 0x7f7dc8002830 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(429..429 src has 126..429) v4 ==== 3927+0+0 (unknown 1058389548 0 0) 0x7f7dd4046eb0 con 0x7f7de406e810 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7de27fc640 10 monclient: get_auth_request con 0x7f7dc4039f90 auth_method 0 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869566656 0 0) 0x7f7dc8006580 con 0x7f7dc8002830 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 10 monclient(hunting): my global_id is 5381 2024-09-18T14:08:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de4122cb0 con 0x7f7dc8002830 2024-09-18T14:08:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2603830827 0 0) 0x7f7de4122cb0 con 0x7f7dc8002830 2024-09-18T14:08:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7dc8006580 con 0x7f7dc8002830 2024-09-18T14:08:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7ddc001800 con 0x7f7dc8002830 2024-09-18T14:08:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 10 monclient: dump: 2024-09-18T14:08:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.988 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 84727653 0 0) 0x7f7dc8006580 con 0x7f7dc8002830 2024-09-18T14:08:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.966+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7dc800a0a0 con 0x7f7dc8002830 2024-09-18T14:08:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.967+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7ddc001d40 con 0x7f7dc8002830 2024-09-18T14:08:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.967+0000 7f7ddaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.967+0000 7f7ddaffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.967+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 >> v1:172.21.15.45:6793/0 conn(0x7f7dc8002830 legacy=0x7f7dc8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.967+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 >> v1:172.21.15.45:6793/0 conn(0x7f7dc8002830 legacy=0x7f7dc8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.972+0000 7f7de964f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.972+0000 7f7de964f640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7dc80074f0 con 0x7f7dc80068a0 2024-09-18T14:08:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306556253 0 0) 0x7f7dc80074f0 con 0x7f7dc80068a0 2024-09-18T14:08:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 10 monclient(hunting): my global_id is 5390 2024-09-18T14:08:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc8006580 con 0x7f7dc80068a0 2024-09-18T14:08:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1042233833 0 0) 0x7f7dc8006580 con 0x7f7dc80068a0 2024-09-18T14:08:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7dc80074f0 con 0x7f7dc80068a0 2024-09-18T14:08:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7ddc004d70 con 0x7f7dc80068a0 2024-09-18T14:08:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 10 monclient: dump: 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:04.992 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3005628171 0 0) 0x7f7dc80074f0 con 0x7f7dc80068a0 2024-09-18T14:08:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7dc8008750 con 0x7f7dc80068a0 2024-09-18T14:08:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7558 (unknown 0 0 3318327452) 0x7f7ddc005110 con 0x7f7dc80068a0 2024-09-18T14:08:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 >> v1:172.21.15.45:6793/0 conn(0x7f7dc80068a0 legacy=0x7f7dc8006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.973+0000 7f7ddaffd640 1 -- 172.21.15.45:0/3986750440 >> v1:172.21.15.45:6793/0 conn(0x7f7dc80068a0 legacy=0x7f7dc8006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:04.993 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:04.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:04.993 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:04.993 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:08:04.993 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:04.993 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:04.993 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:04.993 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:04.993 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:04.993 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:04.993 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:04.993 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:04.994 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:04.994 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:04.994 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:04.994 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.994 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:08:04.994 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:04.994 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:04.994 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:04.994 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.994 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.994 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.994 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:04.996 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:04.998 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:04.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.000 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:05.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:05.000 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:05.000 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.000 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.000 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:05.001 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:05.001 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.001 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.001 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.001 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:05.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:05.001 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.001 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.002 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:05.002 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.002 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.002 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:05.002 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:05.002 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.002 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.002 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:05.003 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:05.004 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.004 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.004 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.004 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:05.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:05.004 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.004 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.004 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:05.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.006 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.007 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:05.010 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.011 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:08:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:05.013 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.974+0000 7f7de964f640 1 -- 172.21.15.45:0/3986750440 >> v1:172.21.15.84:6812/31478 conn(0x7f7dc4039f90 legacy=0x7f7dc403c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.974+0000 7f7de964f640 10 monclient: shutdown 2024-09-18T14:08:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.974+0000 7f7de964f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.974+0000 7f7de964f640 1 -- 172.21.15.45:0/3986750440 >> v1:172.21.15.45:6792/0 conn(0x7f7de406e810 legacy=0x7f7de413d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.974+0000 7f7de37fe640 1 -- 172.21.15.45:0/3986750440 reap_dead start 2024-09-18T14:08:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.974+0000 7f7de37fe640 1 -- 172.21.15.45:0/3986750440 reap_dead start 2024-09-18T14:08:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.974+0000 7f7de964f640 1 -- 172.21.15.45:0/3986750440 shutdown_connections 2024-09-18T14:08:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.974+0000 7f7de964f640 1 -- 172.21.15.45:0/3986750440 >> 172.21.15.45:0/3986750440 conn(0x7f7de40a5bb0 msgr2=0x7f7de40a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.974+0000 7f7de964f640 1 -- 172.21.15.45:0/3986750440 shutdown_connections 2024-09-18T14:08:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:04.974+0000 7f7de964f640 1 -- 172.21.15.45:0/3986750440 wait complete. 2024-09-18T14:08:05.015 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:05.016 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:08:05.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:08:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.112+0000 7f7046b1d640 10 monclient: get_monmap_and_config 2024-09-18T14:08:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.112+0000 7f7046b1d640 10 monclient: build_initial_monmap 2024-09-18T14:08:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.112+0000 7f7046b1d640 10 monclient: monmap: 2024-09-18T14:08:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:05.113599+0000 2024-09-18T14:08:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:05.113599+0000 2024-09-18T14:08:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:05.114 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:05.114 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:05.114 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:05.114 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:05.114 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:05.114 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:05.114 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:05.114 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 1 Processor -- start 2024-09-18T14:08:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 1 -- start start 2024-09-18T14:08:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 10 monclient: init 2024-09-18T14:08:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 10 monclient: _add_conns ranks=[4,2,3,8,5,6,1,0,7] 2024-09-18T14:08:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 10 monclient(hunting): picked mon.noname-c con 0x7f704009f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 10 monclient(hunting): picked mon.noname-b con 0x7f704009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 10 monclient(hunting): picked mon.noname-g con 0x7f704009ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704005ed70 con 0x7f704009fba0 2024-09-18T14:08:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704009dfc0 con 0x7f704009ff70 2024-09-18T14:08:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704009dce0 con 0x7f704009f7d0 2024-09-18T14:08:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.113+0000 7f7046b1d640 10 monclient(hunting): authenticate will time out at 1300.451909s 2024-09-18T14:08:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703ffff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f704009fba0 0x7f7040137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47226/0 (socket says 172.21.15.45:47226) 2024-09-18T14:08:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703ffff640 1 -- 172.21.15.45:0/2678341310 learned_addr learned my addr 172.21.15.45:0/2678341310 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f70448b9640 1 --1- 172.21.15.45:0/2678341310 >> v1:172.21.15.45:6791/0 conn(0x7f704009f7d0 0x7f7040132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:50516/0 (socket says 172.21.15.45:50516) 2024-09-18T14:08:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198959284 0 0) 0x7f704005ed70 con 0x7f704009fba0 2024-09-18T14:08:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient(hunting): my global_id is 144722 2024-09-18T14:08:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70300034e0 con 0x7f704009fba0 2024-09-18T14:08:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694701281 0 0) 0x7f704009dce0 con 0x7f704009f7d0 2024-09-18T14:08:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient(hunting): my global_id is 164245 2024-09-18T14:08:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f704005ed70 con 0x7f704009f7d0 2024-09-18T14:08:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483162298 0 0) 0x7f704009dfc0 con 0x7f704009ff70 2024-09-18T14:08:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient(hunting): my global_id is 154389 2024-09-18T14:08:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f704009dce0 con 0x7f704009ff70 2024-09-18T14:08:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 42809449 0 0) 0x7f70300034e0 con 0x7f704009fba0 2024-09-18T14:08:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 >> v1:172.21.15.45:6791/0 conn(0x7f704009f7d0 legacy=0x7f7040132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 >> v1:172.21.15.84:6790/0 conn(0x7f704009ff70 legacy=0x7f704013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:08:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 monclient: found mon.noname-b 2024-09-18T14:08:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:08:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70400a1c00 con 0x7f704009fba0 2024-09-18T14:08:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient: _finish_auth 0 2024-09-18T14:08:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f702c001760 con 0x7f704009fba0 2024-09-18T14:08:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f7046b1d640 5 monclient: authenticate success, global_id 144722 2024-09-18T14:08:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f7046b1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:08:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient: dump: 2024-09-18T14:08:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.129 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f7046b1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f702c0018c0 con 0x7f704009fba0 2024-09-18T14:08:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f7046b1d640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 1 -- 172.21.15.45:0/2678341310 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f702c004860 con 0x7f704009fba0 2024-09-18T14:08:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f7046b1d640 4 set_mon_vals no callback set 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.114+0000 7f703f7fe640 10 monclient: dump: 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.115+0000 7f7046b1d640 10 monclient: shutdown 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.115+0000 7f7046b1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.115+0000 7f7046b1d640 1 -- 172.21.15.45:0/2678341310 >> v1:172.21.15.45:6790/0 conn(0x7f704009fba0 legacy=0x7f7040137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.115+0000 7f7046b1d640 1 -- 172.21.15.45:0/2678341310 shutdown_connections 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.115+0000 7f7046b1d640 1 -- 172.21.15.45:0/2678341310 >> 172.21.15.45:0/2678341310 conn(0x7f70400630b0 msgr2=0x7f70400634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.115+0000 7f7046b1d640 1 -- 172.21.15.45:0/2678341310 shutdown_connections 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.115+0000 7f7046b1d640 1 -- 172.21.15.45:0/2678341310 wait complete. 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.115+0000 7f7046b1d640 10 monclient: build_initial_monmap 2024-09-18T14:08:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.115+0000 7f7046b1d640 10 monclient: monmap: 2024-09-18T14:08:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 1 Processor -- start 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 1 -- start start 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 10 monclient: init 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 10 monclient: _add_conns ranks=[0,6,7,3,1,8,4,2,5] 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 10 monclient(hunting): picked mon.noname-a con 0x7f704009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 10 monclient(hunting): picked mon.noname-d con 0x7f704009fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 10 monclient(hunting): picked mon.noname-i con 0x7f704009ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704009d310 con 0x7f704009f7d0 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70400a2560 con 0x7f704009fba0 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7040121b70 con 0x7f704009ff70 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f7046b1d640 10 monclient(hunting): authenticate will time out at 1300.454792s 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f70448b9640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f704009f7d0 0x7f7040131740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55030/0 (socket says 172.21.15.45:55030) 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.116+0000 7f70448b9640 1 -- 172.21.15.45:0/2580373651 learned_addr learned my addr 172.21.15.45:0/2580373651 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878300390 0 0) 0x7f704009d310 con 0x7f704009f7d0 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): my global_id is 5454 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70240034e0 con 0x7f704009f7d0 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510634151 0 0) 0x7f70400a2560 con 0x7f704009fba0 2024-09-18T14:08:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): my global_id is 5316 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f704009d310 con 0x7f704009fba0 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223976409 0 0) 0x7f7040121b70 con 0x7f704009ff70 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): my global_id is 5416 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70400a2560 con 0x7f704009ff70 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3193254851 0 0) 0x7f70240034e0 con 0x7f704009f7d0 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7040121b70 con 0x7f704009f7d0 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7034001610 con 0x7f704009f7d0 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:08:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): dump: 2024-09-18T14:08:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1067081759 0 0) 0x7f704009d310 con 0x7f704009fba0 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f70240034e0 con 0x7f704009fba0 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f702c0016a0 con 0x7f704009fba0 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): dump: 2024-09-18T14:08:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1169168824 0 0) 0x7f70400a2560 con 0x7f704009ff70 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f704009d310 con 0x7f704009ff70 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7038001c00 con 0x7f704009ff70 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient(hunting): dump: 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 456721302 0 0) 0x7f7040121b70 con 0x7f704009f7d0 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 >> v1:172.21.15.84:6792/0 conn(0x7f704009ff70 legacy=0x7f704014a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 >> v1:172.21.15.45:6792/0 conn(0x7f704009fba0 legacy=0x7f7040146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient: _finish_hunting 0 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 monclient: found mon.a 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704011fd60 con 0x7f704009f7d0 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient: _finish_auth 0 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f7046b1d640 5 monclient: authenticate success, global_id 5454 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f7046b1d640 10 monclient: _renew_subs 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f7046b1d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f7046b1d640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7040121890 con 0x7f704009f7d0 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f7046b1d640 10 monclient: _renew_subs 2024-09-18T14:08:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f7046b1d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f7046b1d640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7040131e50 con 0x7f704009f7d0 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7034004a50 con 0x7f704009f7d0 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7034004f00 con 0x7f704009f7d0 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703cff9640 10 monclient: dump: 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.117+0000 7f703effd640 4 set_mon_vals no callback set 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.118+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f703400fed0 con 0x7f704009f7d0 2024-09-18T14:08:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.118+0000 7f7046b1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.118+0000 7f7046b1d640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f704004c4a0 con 0x7f7040133c60 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.118+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(429..429 src has 126..429) v4 ==== 3927+0+0 (unknown 1058389548 0 0) 0x7f7034046b20 con 0x7f704009f7d0 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.118+0000 7f70450ba640 10 monclient: get_auth_request con 0x7f702403a9a0 auth_method 0 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467052083 0 0) 0x7f704004c4a0 con 0x7f7040133c60 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 10 monclient(hunting): my global_id is 164188 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7040121b70 con 0x7f7040133c60 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3194596471 0 0) 0x7f7040121b70 con 0x7f7040133c60 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f704004c4a0 con 0x7f7040133c60 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f702c001180 con 0x7f7040133c60 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 10 monclient: dump: 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.146 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3214560054 0 0) 0x7f704004c4a0 con 0x7f7040133c60 2024-09-18T14:08:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7040120ad0 con 0x7f7040133c60 2024-09-18T14:08:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f702c0071d0 con 0x7f7040133c60 2024-09-18T14:08:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 >> v1:172.21.15.84:6789/0 conn(0x7f7040133c60 legacy=0x7f7040099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.119+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 >> v1:172.21.15.84:6789/0 conn(0x7f7040133c60 legacy=0x7f7040099a40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.125+0000 7f7046b1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.125+0000 7f7046b1d640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f704011f780 con 0x7f704011f3b0 2024-09-18T14:08:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059069648 0 0) 0x7f704011f780 con 0x7f704011f3b0 2024-09-18T14:08:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 10 monclient(hunting): my global_id is 164197 2024-09-18T14:08:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f704004c4a0 con 0x7f704011f3b0 2024-09-18T14:08:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3855976006 0 0) 0x7f704004c4a0 con 0x7f704011f3b0 2024-09-18T14:08:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f704011f780 con 0x7f704011f3b0 2024-09-18T14:08:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f702c0059f0 con 0x7f704011f3b0 2024-09-18T14:08:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 10 monclient: dump: 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.150 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1007845999 0 0) 0x7f704011f780 con 0x7f704011f3b0 2024-09-18T14:08:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f704004b720 con 0x7f704011f3b0 2024-09-18T14:08:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3355708766) 0x7f702c00a1c0 con 0x7f704011f3b0 2024-09-18T14:08:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 >> v1:172.21.15.84:6789/0 conn(0x7f704011f3b0 legacy=0x7f704009a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.126+0000 7f703cff9640 1 -- 172.21.15.45:0/2580373651 >> v1:172.21.15.84:6789/0 conn(0x7f704011f3b0 legacy=0x7f704009a150 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:05.150 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:05.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:05.151 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:05.151 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:05.151 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:05.151 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:05.151 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:05.151 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:05.151 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:05.151 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:05.151 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:05.151 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:05.151 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:05.152 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:05.152 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:05.152 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.152 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:08:05.152 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:05.152 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:05.152 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:05.152 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:05.152 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:05.152 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:05.152 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:05.152 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:05.153 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:05.153 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:05.153 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:05.153 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:05.153 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.153 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:05.153 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:05.153 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:05.153 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:05.153 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:05.153 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:05.153 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:05.154 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:05.154 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:05.154 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:05.154 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:05.154 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.154 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:05.154 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:05.154 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:05.154 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:05.154 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:05.154 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:05.155 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:05.155 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:05.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:05.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:05.155 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:05.155 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:05.155 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:05.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:05.155 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:05.155 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:05.155 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:05.155 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:05.156 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:05.156 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:05.156 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:05.156 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:05.156 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:05.156 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:05.156 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:05.156 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:05.156 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.156 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:05.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.156 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:05.159 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.160 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:05.160 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.160 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.160 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:05.160 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:05.160 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.161 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:05.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:05.165 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.166 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.169 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.169 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:05.169 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:05.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.169 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.169 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.169 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:05.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.169 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.169 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:05.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.170 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.170 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.170 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:05.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.170 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.170 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:05.170 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.127+0000 7f7046b1d640 1 -- 172.21.15.45:0/2580373651 >> v1:172.21.15.84:6812/31478 conn(0x7f702403a9a0 legacy=0x7f702403ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.127+0000 7f7046b1d640 10 monclient: shutdown 2024-09-18T14:08:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.127+0000 7f7046b1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.127+0000 7f7046b1d640 1 -- 172.21.15.45:0/2580373651 >> v1:172.21.15.45:6789/0 conn(0x7f704009f7d0 legacy=0x7f7040131740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.127+0000 7f70450ba640 1 -- 172.21.15.45:0/2580373651 reap_dead start 2024-09-18T14:08:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.127+0000 7f70450ba640 1 -- 172.21.15.45:0/2580373651 reap_dead start 2024-09-18T14:08:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.127+0000 7f7046b1d640 1 -- 172.21.15.45:0/2580373651 shutdown_connections 2024-09-18T14:08:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.127+0000 7f7046b1d640 1 -- 172.21.15.45:0/2580373651 >> 172.21.15.45:0/2580373651 conn(0x7f70400630b0 msgr2=0x7f70400a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.127+0000 7f7046b1d640 1 -- 172.21.15.45:0/2580373651 shutdown_connections 2024-09-18T14:08:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.127+0000 7f7046b1d640 1 -- 172.21.15.45:0/2580373651 wait complete. 2024-09-18T14:08:05.173 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:05.173 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:08:05.173 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:08:05.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.270+0000 7f261955d640 10 monclient: get_monmap_and_config 2024-09-18T14:08:05.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.270+0000 7f261955d640 10 monclient: build_initial_monmap 2024-09-18T14:08:05.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.270+0000 7f261955d640 10 monclient: monmap: 2024-09-18T14:08:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:05.271387+0000 2024-09-18T14:08:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:05.271387+0000 2024-09-18T14:08:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:05.272 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:05.272 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:05.272 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:05.272 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 1 Processor -- start 2024-09-18T14:08:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 1 -- start start 2024-09-18T14:08:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 10 monclient: init 2024-09-18T14:08:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 10 monclient: _add_conns ranks=[5,1,7,2,8,3,0,6,4] 2024-09-18T14:08:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 10 monclient(hunting): picked mon.noname-h con 0x7f26140a2610 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 10 monclient(hunting): picked mon.noname-f con 0x7f26141330b0 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 10 monclient(hunting): picked mon.noname-i con 0x7f2614096a20 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261405ed70 con 0x7f26141330b0 2024-09-18T14:08:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140a1740 con 0x7f26140a2610 2024-09-18T14:08:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140a29e0 con 0x7f2614096a20 2024-09-18T14:08:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f261955d640 10 monclient(hunting): authenticate will time out at 1300.609755s 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f26127fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f26141330b0 0x7f2614137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:38578/0 (socket says 172.21.15.45:38578) 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f26127fc640 1 -- 172.21.15.45:0/1795543932 learned_addr learned my addr 172.21.15.45:0/1795543932 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f2612ffd640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f26140a2610 0x7f26141329a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:57382/0 (socket says 172.21.15.45:57382) 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.271+0000 7f26137fe640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f2614096a20 0x7f261413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:46718/0 (socket says 172.21.15.45:46718) 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496414685 0 0) 0x7f26140a29e0 con 0x7f2614096a20 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient(hunting): my global_id is 5425 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26000034e0 con 0x7f2614096a20 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041859040 0 0) 0x7f261405ed70 con 0x7f26141330b0 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient(hunting): my global_id is 164206 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26140a29e0 con 0x7f26141330b0 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127573148 0 0) 0x7f26140a1740 con 0x7f26140a2610 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient(hunting): my global_id is 5261 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f261405ed70 con 0x7f26140a2610 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3781485542 0 0) 0x7f26000034e0 con 0x7f2614096a20 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 >> v1:172.21.15.84:6791/0 conn(0x7f26140a2610 legacy=0x7f26141329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 >> v1:172.21.15.84:6789/0 conn(0x7f26141330b0 legacy=0x7f2614137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 monclient: found mon.noname-i 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140970a0 con 0x7f2614096a20 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient: _finish_auth 0 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f261955d640 5 monclient: authenticate success, global_id 5425 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f260c001760 con 0x7f2614096a20 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f261955d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient: dump: 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.284 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f261955d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f260c001940 con 0x7f2614096a20 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 1 -- 172.21.15.45:0/1795543932 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f260c0047a0 con 0x7f2614096a20 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f261955d640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f261955d640 4 set_mon_vals no callback set 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.272+0000 7f2611ffb640 10 monclient: dump: 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.273+0000 7f261955d640 10 monclient: shutdown 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.273+0000 7f261955d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.273+0000 7f261955d640 1 -- 172.21.15.45:0/1795543932 >> v1:172.21.15.84:6792/0 conn(0x7f2614096a20 legacy=0x7f261413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.273+0000 7f261955d640 1 -- 172.21.15.45:0/1795543932 shutdown_connections 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.273+0000 7f261955d640 1 -- 172.21.15.45:0/1795543932 >> 172.21.15.45:0/1795543932 conn(0x7f26140630f0 msgr2=0x7f26140634f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.273+0000 7f261955d640 1 -- 172.21.15.45:0/1795543932 shutdown_connections 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.273+0000 7f261955d640 1 -- 172.21.15.45:0/1795543932 wait complete. 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.273+0000 7f261955d640 10 monclient: build_initial_monmap 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.273+0000 7f261955d640 10 monclient: monmap: 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 1 Processor -- start 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 1 -- start start 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 10 monclient: init 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 10 monclient: _add_conns ranks=[0,1,7,8,6,5,2,4,3] 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 10 monclient(hunting): picked mon.noname-a con 0x7f2614096a20 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 10 monclient(hunting): picked mon.noname-f con 0x7f26140a2610 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 10 monclient(hunting): picked mon.noname-i con 0x7f26141330b0 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26141339a0 con 0x7f2614096a20 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2614120190 con 0x7f26140a2610 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140a2350 con 0x7f26141330b0 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f261955d640 10 monclient(hunting): authenticate will time out at 1300.612711s 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f2612ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2614096a20 0x7f2614131340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55046/0 (socket says 172.21.15.45:55046) 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.274+0000 7f2612ffd640 1 -- 172.21.15.45:0/1451102640 learned_addr learned my addr 172.21.15.45:0/1451102640 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146936104 0 0) 0x7f26141339a0 con 0x7f2614096a20 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): my global_id is 5463 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25f40034e0 con 0x7f2614096a20 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741262710 0 0) 0x7f26140a2350 con 0x7f26141330b0 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): my global_id is 5434 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26141339a0 con 0x7f26141330b0 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400195484 0 0) 0x7f2614120190 con 0x7f26140a2610 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): my global_id is 164215 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26140a2350 con 0x7f26140a2610 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3517314225 0 0) 0x7f26141339a0 con 0x7f26141330b0 2024-09-18T14:08:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2614120190 con 0x7f26141330b0 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f260c0016a0 con 0x7f26141330b0 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): dump: 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 132377499 0 0) 0x7f26140a2350 con 0x7f26140a2610 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f26141339a0 con 0x7f26140a2610 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2715714319 0 0) 0x7f25f40034e0 con 0x7f2614096a20 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f26140a2350 con 0x7f2614096a20 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f25fc001980 con 0x7f26140a2610 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): dump: 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2604001dc0 con 0x7f2614096a20 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient(hunting): dump: 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 668872645 0 0) 0x7f2614120190 con 0x7f26141330b0 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 >> v1:172.21.15.84:6789/0 conn(0x7f26140a2610 legacy=0x7f26141467f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 >> v1:172.21.15.45:6789/0 conn(0x7f2614096a20 legacy=0x7f2614131340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient: _finish_hunting 0 2024-09-18T14:08:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 monclient: found mon.i 2024-09-18T14:08:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:08:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26141205a0 con 0x7f26141330b0 2024-09-18T14:08:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient: _finish_auth 0 2024-09-18T14:08:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f261955d640 5 monclient: authenticate success, global_id 5434 2024-09-18T14:08:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f261955d640 10 monclient: _renew_subs 2024-09-18T14:08:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f261955d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:08:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f261955d640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2614121490 con 0x7f26141330b0 2024-09-18T14:08:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f261955d640 10 monclient: _renew_subs 2024-09-18T14:08:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f261955d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f261955d640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2614142320 con 0x7f26141330b0 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f260c0047a0 con 0x7f26141330b0 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f260c005090 con 0x7f26141330b0 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f26117fa640 4 set_mon_vals no callback set 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.275+0000 7f260affd640 10 monclient: dump: 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.276+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f260c0100e0 con 0x7f26141330b0 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.276+0000 7f261955d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.276+0000 7f261955d640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f25f8006580 con 0x7f25f8002830 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.276+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(429..429 src has 126..429) v4 ==== 3927+0+0 (unknown 1058389548 0 0) 0x7f260c046b50 con 0x7f26141330b0 2024-09-18T14:08:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.276+0000 7f26127fc640 10 monclient: get_auth_request con 0x7f25f403ab10 auth_method 0 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.276+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80136895 0 0) 0x7f25f8006580 con 0x7f25f8002830 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.276+0000 7f260affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.276+0000 7f260affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.276+0000 7f260affd640 10 monclient(hunting): my global_id is 154398 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.276+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2614120190 con 0x7f25f8002830 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3206133398 0 0) 0x7f2614120190 con 0x7f25f8002830 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f25f8006580 con 0x7f25f8002830 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2604004e60 con 0x7f25f8002830 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 10 monclient: dump: 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 463315191 0 0) 0x7f25f8006580 con 0x7f25f8002830 2024-09-18T14:08:05.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25f800a0a0 con 0x7f25f8002830 2024-09-18T14:08:05.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2604009fc0 con 0x7f25f8002830 2024-09-18T14:08:05.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:05.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:05.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 >> v1:172.21.15.84:6790/0 conn(0x7f25f8002830 legacy=0x7f25f8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.277+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 >> v1:172.21.15.84:6790/0 conn(0x7f25f8002830 legacy=0x7f25f8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:05.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f261955d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:05.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f261955d640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f25f80074d0 con 0x7f25f80068a0 2024-09-18T14:08:05.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544033204 0 0) 0x7f25f80074d0 con 0x7f25f80068a0 2024-09-18T14:08:05.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 10 monclient(hunting): my global_id is 154407 2024-09-18T14:08:05.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25f8006580 con 0x7f25f80068a0 2024-09-18T14:08:05.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1503436974 0 0) 0x7f25f8006580 con 0x7f25f80068a0 2024-09-18T14:08:05.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f25f80074d0 con 0x7f25f80068a0 2024-09-18T14:08:05.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2604005a20 con 0x7f25f80068a0 2024-09-18T14:08:05.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:08:05.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 10 monclient: dump: 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.304 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.305 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 969030903 0 0) 0x7f25f80074d0 con 0x7f25f80068a0 2024-09-18T14:08:05.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.283+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f25f8008750 con 0x7f25f80068a0 2024-09-18T14:08:05.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 3623162571) 0x7f26040071e0 con 0x7f25f80068a0 2024-09-18T14:08:05.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f260affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:05.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f260affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:05.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 >> v1:172.21.15.84:6790/0 conn(0x7f25f80068a0 legacy=0x7f25f8006d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f260affd640 1 -- 172.21.15.45:0/1451102640 >> v1:172.21.15.84:6790/0 conn(0x7f25f80068a0 legacy=0x7f25f8006d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:05.305 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:05.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.326 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:05.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.326 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.326 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:05.326 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f261955d640 1 -- 172.21.15.45:0/1451102640 >> v1:172.21.15.84:6812/31478 conn(0x7f25f403ab10 legacy=0x7f25f403cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f261955d640 10 monclient: shutdown 2024-09-18T14:08:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f261955d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f261955d640 1 -- 172.21.15.45:0/1451102640 >> v1:172.21.15.84:6792/0 conn(0x7f26141330b0 legacy=0x7f2614149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f26137fe640 1 -- 172.21.15.45:0/1451102640 reap_dead start 2024-09-18T14:08:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f26137fe640 1 -- 172.21.15.45:0/1451102640 reap_dead start 2024-09-18T14:08:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f261955d640 1 -- 172.21.15.45:0/1451102640 shutdown_connections 2024-09-18T14:08:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f261955d640 1 -- 172.21.15.45:0/1451102640 >> 172.21.15.45:0/1451102640 conn(0x7f26140630f0 msgr2=0x7f26140a04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f261955d640 1 -- 172.21.15.45:0/1451102640 shutdown_connections 2024-09-18T14:08:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.284+0000 7f261955d640 1 -- 172.21.15.45:0/1451102640 wait complete. 2024-09-18T14:08:05.328 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:05.329 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:08:05.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:08:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.425+0000 7f9f92d09640 10 monclient: get_monmap_and_config 2024-09-18T14:08:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.425+0000 7f9f92d09640 10 monclient: build_initial_monmap 2024-09-18T14:08:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.425+0000 7f9f92d09640 10 monclient: monmap: 2024-09-18T14:08:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:05.426515+0000 2024-09-18T14:08:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:05.426515+0000 2024-09-18T14:08:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:05.427 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 1 Processor -- start 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 1 -- start start 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 10 monclient: init 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 10 monclient: _add_conns ranks=[3,2,4,1,5,8,6,7,0] 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 10 monclient(hunting): picked mon.noname-g con 0x7f9f8c09f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 10 monclient(hunting): picked mon.noname-b con 0x7f9f8c09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 10 monclient(hunting): picked mon.noname-c con 0x7f9f8c09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c05ed70 con 0x7f9f8c09fba0 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c09dfc0 con 0x7f9f8c09f7d0 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c09dce0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f92d09640 10 monclient(hunting): authenticate will time out at 1300.764837s 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f8bfff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f9f8c09fba0 0x7f9f8c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47236/0 (socket says 172.21.15.45:47236) 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.426+0000 7f9f8bfff640 1 -- 172.21.15.45:0/3032506195 learned_addr learned my addr 172.21.15.45:0/3032506195 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406470296 0 0) 0x7f9f8c09dce0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient(hunting): my global_id is 164254 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f7c0034e0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314044099 0 0) 0x7f9f8c05ed70 con 0x7f9f8c09fba0 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient(hunting): my global_id is 144731 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8c09dce0 con 0x7f9f8c09fba0 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466669435 0 0) 0x7f9f8c09dfc0 con 0x7f9f8c09f7d0 2024-09-18T14:08:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient(hunting): my global_id is 154416 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8c05ed70 con 0x7f9f8c09f7d0 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1289850886 0 0) 0x7f9f7c0034e0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 >> v1:172.21.15.84:6790/0 conn(0x7f9f8c09f7d0 legacy=0x7f9f8c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 >> v1:172.21.15.45:6790/0 conn(0x7f9f8c09fba0 legacy=0x7f9f8c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 monclient: found mon.noname-c 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c0a1c00 con 0x7f9f8c09ff70 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient: _finish_auth 0 2024-09-18T14:08:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f84001bb0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f92d09640 5 monclient: authenticate success, global_id 164254 2024-09-18T14:08:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f92d09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient: dump: 2024-09-18T14:08:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f92d09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9f84001d10 con 0x7f9f8c09ff70 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 1 -- 172.21.15.45:0/3032506195 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f840022c0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f92d09640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f92d09640 4 set_mon_vals no callback set 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.427+0000 7f9f8b7fe640 10 monclient: dump: 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.428+0000 7f9f92d09640 10 monclient: shutdown 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.428+0000 7f9f92d09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.428+0000 7f9f92d09640 1 -- 172.21.15.45:0/3032506195 >> v1:172.21.15.45:6791/0 conn(0x7f9f8c09ff70 legacy=0x7f9f8c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.428+0000 7f9f92d09640 1 -- 172.21.15.45:0/3032506195 shutdown_connections 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.428+0000 7f9f92d09640 1 -- 172.21.15.45:0/3032506195 >> 172.21.15.45:0/3032506195 conn(0x7f9f8c0630b0 msgr2=0x7f9f8c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.428+0000 7f9f92d09640 1 -- 172.21.15.45:0/3032506195 shutdown_connections 2024-09-18T14:08:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.428+0000 7f9f92d09640 1 -- 172.21.15.45:0/3032506195 wait complete. 2024-09-18T14:08:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.428+0000 7f9f92d09640 10 monclient: build_initial_monmap 2024-09-18T14:08:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.428+0000 7f9f92d09640 10 monclient: monmap: 2024-09-18T14:08:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 1 Processor -- start 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 1 -- start start 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 10 monclient: init 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 10 monclient: _add_conns ranks=[0,6,3,2,4,7,5,1,8] 2024-09-18T14:08:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 10 monclient(hunting): picked mon.noname-a con 0x7f9f8c09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 10 monclient(hunting): picked mon.noname-d con 0x7f9f8c09fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 10 monclient(hunting): picked mon.noname-g con 0x7f9f8c09ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c1208b0 con 0x7f9f8c09f7d0 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c121ee0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f90aa5640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9f8c09f7d0 0x7f9f8c1418b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55060/0 (socket says 172.21.15.45:55060) 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f90aa5640 1 -- 172.21.15.45:0/464003086 learned_addr learned my addr 172.21.15.45:0/464003086 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c09d310 con 0x7f9f8c09fba0 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.429+0000 7f9f92d09640 10 monclient(hunting): authenticate will time out at 1300.767911s 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232066948 0 0) 0x7f9f8c09d310 con 0x7f9f8c09fba0 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): my global_id is 5325 2024-09-18T14:08:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f700034e0 con 0x7f9f8c09fba0 2024-09-18T14:08:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810760913 0 0) 0x7f9f8c1208b0 con 0x7f9f8c09f7d0 2024-09-18T14:08:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): my global_id is 5472 2024-09-18T14:08:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8c09d310 con 0x7f9f8c09f7d0 2024-09-18T14:08:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873442054 0 0) 0x7f9f8c121ee0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): my global_id is 154425 2024-09-18T14:08:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8c1208b0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 992590995 0 0) 0x7f9f8c1208b0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f8c121ee0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f840013a0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): dump: 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 805048826 0 0) 0x7f9f700034e0 con 0x7f9f8c09fba0 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f8c1208b0 con 0x7f9f8c09fba0 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3879032605 0 0) 0x7f9f8c09d310 con 0x7f9f8c09f7d0 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f700034e0 con 0x7f9f8c09f7d0 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f78001470 con 0x7f9f8c09fba0 2024-09-18T14:08:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): dump: 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f80001b20 con 0x7f9f8c09f7d0 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient(hunting): dump: 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 761297899 0 0) 0x7f9f8c121ee0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 >> v1:172.21.15.45:6792/0 conn(0x7f9f8c09fba0 legacy=0x7f9f8c131ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 >> v1:172.21.15.45:6789/0 conn(0x7f9f8c09f7d0 legacy=0x7f9f8c1418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 monclient: found mon.g 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c142750 con 0x7f9f8c09ff70 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient: _finish_auth 0 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f92d09640 5 monclient: authenticate success, global_id 154425 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f92d09640 10 monclient: _renew_subs 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f92d09640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f92d09640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8c121c00 con 0x7f9f8c09ff70 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f92d09640 10 monclient: _renew_subs 2024-09-18T14:08:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f92d09640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f92d09640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c1200d0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9f84002470 con 0x7f9f8c09ff70 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f840029a0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f8affd640 4 set_mon_vals no callback set 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.430+0000 7f9f88ff9640 10 monclient: dump: 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.431+0000 7f9f92d09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.431+0000 7f9f92d09640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9f6c006580 con 0x7f9f6c002830 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.431+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f9f8400d970 con 0x7f9f8c09ff70 2024-09-18T14:08:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.431+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(429..429 src has 126..429) v4 ==== 3927+0+0 (unknown 1058389548 0 0) 0x7f9f840445b0 con 0x7f9f8c09ff70 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f8bfff640 10 monclient: get_auth_request con 0x7f9f7003aa60 auth_method 0 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149705432 0 0) 0x7f9f6c006580 con 0x7f9f6c002830 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 10 monclient(hunting): my global_id is 5270 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8c121ee0 con 0x7f9f6c002830 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4046157749 0 0) 0x7f9f8c121ee0 con 0x7f9f6c002830 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f6c006580 con 0x7f9f6c002830 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f80002270 con 0x7f9f6c002830 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 10 monclient: dump: 2024-09-18T14:08:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.458 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3611771922 0 0) 0x7f9f6c006580 con 0x7f9f6c002830 2024-09-18T14:08:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f6c00a0a0 con 0x7f9f6c002830 2024-09-18T14:08:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9f800026e0 con 0x7f9f6c002830 2024-09-18T14:08:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 >> v1:172.21.15.84:6791/0 conn(0x7f9f6c002830 legacy=0x7f9f6c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.432+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 >> v1:172.21.15.84:6791/0 conn(0x7f9f6c002830 legacy=0x7f9f6c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.438+0000 7f9f92d09640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.438+0000 7f9f92d09640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9f6c009660 con 0x7f9f6c009040 2024-09-18T14:08:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.438+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432276122 0 0) 0x7f9f6c009660 con 0x7f9f6c009040 2024-09-18T14:08:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.438+0000 7f9f88ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.438+0000 7f9f88ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.438+0000 7f9f88ff9640 10 monclient(hunting): my global_id is 5279 2024-09-18T14:08:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.438+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f6c006580 con 0x7f9f6c009040 2024-09-18T14:08:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4169166983 0 0) 0x7f9f6c006580 con 0x7f9f6c009040 2024-09-18T14:08:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f6c009660 con 0x7f9f6c009040 2024-09-18T14:08:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f80002c60 con 0x7f9f6c009040 2024-09-18T14:08:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:08:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 10 monclient: dump: 2024-09-18T14:08:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.462 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1020426771 0 0) 0x7f9f6c009660 con 0x7f9f6c009040 2024-09-18T14:08:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9f6c009920 con 0x7f9f6c009040 2024-09-18T14:08:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 3605137805) 0x7f9f80007d90 con 0x7f9f6c009040 2024-09-18T14:08:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 >> v1:172.21.15.84:6791/0 conn(0x7f9f6c009040 legacy=0x7f9f6c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f88ff9640 1 -- 172.21.15.45:0/464003086 >> v1:172.21.15.84:6791/0 conn(0x7f9f6c009040 legacy=0x7f9f6c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:05.463 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:05.464 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:05.464 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:05.464 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.464 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:08:05.464 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:05.464 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:05.464 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:05.464 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:05.464 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:05.464 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:05.464 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:05.464 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:05.465 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:05.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:05.467 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.471 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.474 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.475 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:05.478 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.479 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:05.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:05.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.482 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.482 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.482 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:05.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.482 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.482 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:05.482 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:05.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f92d09640 1 -- 172.21.15.45:0/464003086 >> v1:172.21.15.84:6812/31478 conn(0x7f9f7003aa60 legacy=0x7f9f7003cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f92d09640 10 monclient: shutdown 2024-09-18T14:08:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f92d09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f92d09640 1 -- 172.21.15.45:0/464003086 >> v1:172.21.15.84:6790/0 conn(0x7f9f8c09ff70 legacy=0x7f9f8c149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f912a6640 1 -- 172.21.15.45:0/464003086 reap_dead start 2024-09-18T14:08:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f912a6640 1 -- 172.21.15.45:0/464003086 reap_dead start 2024-09-18T14:08:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f92d09640 1 -- 172.21.15.45:0/464003086 shutdown_connections 2024-09-18T14:08:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.439+0000 7f9f92d09640 1 -- 172.21.15.45:0/464003086 >> 172.21.15.45:0/464003086 conn(0x7f9f8c0630b0 msgr2=0x7f9f8c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.440+0000 7f9f92d09640 1 -- 172.21.15.45:0/464003086 shutdown_connections 2024-09-18T14:08:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.440+0000 7f9f92d09640 1 -- 172.21.15.45:0/464003086 wait complete. 2024-09-18T14:08:05.484 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:05.485 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:08:05.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:08:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.581+0000 7f2a4655d640 10 monclient: get_monmap_and_config 2024-09-18T14:08:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.581+0000 7f2a4655d640 10 monclient: build_initial_monmap 2024-09-18T14:08:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.581+0000 7f2a4655d640 10 monclient: monmap: 2024-09-18T14:08:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:05.582616+0000 2024-09-18T14:08:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:05.582616+0000 2024-09-18T14:08:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:05.583 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.582+0000 7f2a4655d640 1 Processor -- start 2024-09-18T14:08:05.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.582+0000 7f2a4655d640 1 -- start start 2024-09-18T14:08:05.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.582+0000 7f2a4655d640 10 monclient: init 2024-09-18T14:08:05.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.582+0000 7f2a4655d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:05.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.582+0000 7f2a4655d640 10 monclient: _add_conns ranks=[7,3,4,1,2,6,5,8,0] 2024-09-18T14:08:05.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.582+0000 7f2a4655d640 10 monclient(hunting): picked mon.noname-i con 0x7f2a4009f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:05.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.582+0000 7f2a4655d640 10 monclient(hunting): picked mon.noname-g con 0x7f2a4009fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:05.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.582+0000 7f2a4655d640 10 monclient(hunting): picked mon.noname-c con 0x7f2a4009ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.582+0000 7f2a4655d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4005ed70 con 0x7f2a4009fba0 2024-09-18T14:08:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a4655d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4009dfc0 con 0x7f2a4009ff70 2024-09-18T14:08:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a4655d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4009dce0 con 0x7f2a4009f7d0 2024-09-18T14:08:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a4655d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a4655d640 10 monclient(hunting): authenticate will time out at 1300.921030s 2024-09-18T14:08:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a44afa640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f2a4009ff70 0x7f2a4013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:50526/0 (socket says 172.21.15.45:50526) 2024-09-18T14:08:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a44afa640 1 -- 172.21.15.45:0/3813278585 learned_addr learned my addr 172.21.15.45:0/3813278585 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584194393 0 0) 0x7f2a4009dfc0 con 0x7f2a4009ff70 2024-09-18T14:08:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient(hunting): my global_id is 164263 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a280034e0 con 0x7f2a4009ff70 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489368491 0 0) 0x7f2a4009dce0 con 0x7f2a4009f7d0 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient(hunting): my global_id is 5443 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a4009dfc0 con 0x7f2a4009f7d0 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993861753 0 0) 0x7f2a4005ed70 con 0x7f2a4009fba0 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient(hunting): my global_id is 154434 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a4009dce0 con 0x7f2a4009fba0 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1019444239 0 0) 0x7f2a4009dfc0 con 0x7f2a4009f7d0 2024-09-18T14:08:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 >> v1:172.21.15.45:6791/0 conn(0x7f2a4009ff70 legacy=0x7f2a4013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 >> v1:172.21.15.84:6790/0 conn(0x7f2a4009fba0 legacy=0x7f2a40137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient: _finish_hunting 0 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 monclient: found mon.noname-i 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a400a1c00 con 0x7f2a4009f7d0 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient: _finish_auth 0 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a30001810 con 0x7f2a4009f7d0 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a4655d640 5 monclient: authenticate success, global_id 5443 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a4655d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:08:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient: dump: 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a4655d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2a30001970 con 0x7f2a4009f7d0 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 1 -- 172.21.15.45:0/3813278585 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a30001f20 con 0x7f2a4009f7d0 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a4655d640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a4655d640 4 set_mon_vals no callback set 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.583+0000 7f2a3effd640 10 monclient: dump: 2024-09-18T14:08:05.596 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.597 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.584+0000 7f2a4655d640 10 monclient: shutdown 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.584+0000 7f2a4655d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.584+0000 7f2a4655d640 1 -- 172.21.15.45:0/3813278585 >> v1:172.21.15.84:6792/0 conn(0x7f2a4009f7d0 legacy=0x7f2a40132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.584+0000 7f2a4655d640 1 -- 172.21.15.45:0/3813278585 shutdown_connections 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.584+0000 7f2a4655d640 1 -- 172.21.15.45:0/3813278585 >> 172.21.15.45:0/3813278585 conn(0x7f2a400630b0 msgr2=0x7f2a400634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.584+0000 7f2a4655d640 1 -- 172.21.15.45:0/3813278585 shutdown_connections 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.584+0000 7f2a4655d640 1 -- 172.21.15.45:0/3813278585 wait complete. 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.584+0000 7f2a4655d640 10 monclient: build_initial_monmap 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.584+0000 7f2a4655d640 10 monclient: monmap: 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 1 Processor -- start 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 1 -- start start 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 10 monclient: init 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 10 monclient: _add_conns ranks=[3,6,4,8,2,5,1,0,7] 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 10 monclient(hunting): picked mon.noname-g con 0x7f2a4009f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 10 monclient(hunting): picked mon.noname-d con 0x7f2a4009fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 10 monclient(hunting): picked mon.noname-c con 0x7f2a4009ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a400a2560 con 0x7f2a4009f7d0 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a40120d30 con 0x7f2a4009ff70 2024-09-18T14:08:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a40121c90 con 0x7f2a4009fba0 2024-09-18T14:08:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a4655d640 10 monclient(hunting): authenticate will time out at 1300.923807s 2024-09-18T14:08:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a44afa640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f2a4009ff70 0x7f2a40149d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:50532/0 (socket says 172.21.15.45:50532) 2024-09-18T14:08:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a44afa640 1 -- 172.21.15.45:0/3278535064 learned_addr learned my addr 172.21.15.45:0/3278535064 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.585+0000 7f2a3f7fe640 1 --1- 172.21.15.45:0/3278535064 >> v1:172.21.15.45:6792/0 conn(0x7f2a4009fba0 0x7f2a40131b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:41002/0 (socket says 172.21.15.45:41002) 2024-09-18T14:08:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412662965 0 0) 0x7f2a40120d30 con 0x7f2a4009ff70 2024-09-18T14:08:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): my global_id is 164272 2024-09-18T14:08:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a2c0034e0 con 0x7f2a4009ff70 2024-09-18T14:08:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31406082 0 0) 0x7f2a40121c90 con 0x7f2a4009fba0 2024-09-18T14:08:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): my global_id is 5334 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a40120d30 con 0x7f2a4009fba0 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788717846 0 0) 0x7f2a400a2560 con 0x7f2a4009f7d0 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): my global_id is 154443 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a40121c90 con 0x7f2a4009f7d0 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3135779510 0 0) 0x7f2a2c0034e0 con 0x7f2a4009ff70 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a400a2560 con 0x7f2a4009ff70 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a38001de0 con 0x7f2a4009ff70 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): dump: 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.604 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2912757549 0 0) 0x7f2a40121c90 con 0x7f2a4009f7d0 2024-09-18T14:08:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a2c0034e0 con 0x7f2a4009f7d0 2024-09-18T14:08:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3630097300 0 0) 0x7f2a40120d30 con 0x7f2a4009fba0 2024-09-18T14:08:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a40121c90 con 0x7f2a4009fba0 2024-09-18T14:08:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a30001e20 con 0x7f2a4009f7d0 2024-09-18T14:08:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:08:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): dump: 2024-09-18T14:08:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a34001950 con 0x7f2a4009fba0 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient(hunting): dump: 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1605357553 0 0) 0x7f2a400a2560 con 0x7f2a4009ff70 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 >> v1:172.21.15.45:6792/0 conn(0x7f2a4009fba0 legacy=0x7f2a40131b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 >> v1:172.21.15.84:6790/0 conn(0x7f2a4009f7d0 legacy=0x7f2a40141a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient: _finish_hunting 0 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 monclient: found mon.c 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a40120160 con 0x7f2a4009ff70 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient: _finish_auth 0 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a4655d640 5 monclient: authenticate success, global_id 164272 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a4655d640 10 monclient: _renew_subs 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a4655d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a4655d640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a40132250 con 0x7f2a4009ff70 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a4655d640 10 monclient: _renew_subs 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a4655d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:05.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a4655d640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4014b630 con 0x7f2a4009ff70 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2a38002060 con 0x7f2a4009ff70 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a38005110 con 0x7f2a4009ff70 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a3e7fc640 4 set_mon_vals no callback set 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.586+0000 7f2a27fff640 10 monclient: dump: 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.609 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.587+0000 7f2a4655d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.587+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2a380100e0 con 0x7f2a4009ff70 2024-09-18T14:08:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.587+0000 7f2a4655d640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2a20006580 con 0x7f2a20002830 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.587+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(429..429 src has 126..429) v4 ==== 3927+0+0 (unknown 1058389548 0 0) 0x7f2a38046e00 con 0x7f2a4009ff70 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.587+0000 7f2a3f7fe640 10 monclient: get_auth_request con 0x7f2a2c03aac0 auth_method 0 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.587+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889808773 0 0) 0x7f2a20006580 con 0x7f2a20002830 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.587+0000 7f2a27fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.587+0000 7f2a27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.587+0000 7f2a27fff640 10 monclient(hunting): my global_id is 5452 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a400a2560 con 0x7f2a20002830 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 590609450 0 0) 0x7f2a400a2560 con 0x7f2a20002830 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a20006580 con 0x7f2a20002830 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a30002790 con 0x7f2a20002830 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 10 monclient: dump: 2024-09-18T14:08:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.612 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.612 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.612 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.612 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.612 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.612 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.612 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.612 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.612 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.612 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.612 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.612 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.613 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.613 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.613 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1280033489 0 0) 0x7f2a20006580 con 0x7f2a20002830 2024-09-18T14:08:05.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a2000a0a0 con 0x7f2a20002830 2024-09-18T14:08:05.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2a300049b0 con 0x7f2a20002830 2024-09-18T14:08:05.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:05.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:05.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 >> v1:172.21.15.84:6792/0 conn(0x7f2a20002830 legacy=0x7f2a20004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.588+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 >> v1:172.21.15.84:6792/0 conn(0x7f2a20002830 legacy=0x7f2a20004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.594+0000 7f2a4655d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.594+0000 7f2a4655d640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2a20009660 con 0x7f2a20009040 2024-09-18T14:08:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31979923 0 0) 0x7f2a20009660 con 0x7f2a20009040 2024-09-18T14:08:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 10 monclient(hunting): my global_id is 5461 2024-09-18T14:08:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a20006580 con 0x7f2a20009040 2024-09-18T14:08:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1876684264 0 0) 0x7f2a20006580 con 0x7f2a20009040 2024-09-18T14:08:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a20009660 con 0x7f2a20009040 2024-09-18T14:08:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2a30003210 con 0x7f2a20009040 2024-09-18T14:08:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 10 monclient: dump: 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:05.617 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1494526352 0 0) 0x7f2a20009660 con 0x7f2a20009040 2024-09-18T14:08:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2a20009920 con 0x7f2a20009040 2024-09-18T14:08:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3588257542) 0x7f2a300035b0 con 0x7f2a20009040 2024-09-18T14:08:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 >> v1:172.21.15.84:6792/0 conn(0x7f2a20009040 legacy=0x7f2a20008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.595+0000 7f2a27fff640 1 -- 172.21.15.45:0/3278535064 >> v1:172.21.15.84:6792/0 conn(0x7f2a20009040 legacy=0x7f2a20008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:05.618 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:05.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:05.618 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:05.618 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:05.618 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 84, 2024-09-18T14:08:05.618 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:05.618 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:05.619 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:05.620 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:05.621 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:05.622 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.623 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:05.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.624 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:05.624 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:05.624 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.624 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.624 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.624 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:05.624 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.624 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.624 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.625 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:05.625 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:05.625 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.625 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.625 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.625 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:05.625 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:05.625 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.625 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:05.627 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:05.628 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:05.629 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.631 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.632 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:05.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.634 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.635 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:05.636 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:05.636 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:05.636 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:05.636 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:05.636 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.636 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.636 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:05.636 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:05.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.636 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.636 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.637 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:05.637 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.637 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:05.637 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:05.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:05.637 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:05.637 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:05.637 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:08:05.637 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:05.637 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:05.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:05.637 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:05.638 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.596+0000 7f2a4655d640 1 -- 172.21.15.45:0/3278535064 >> v1:172.21.15.84:6812/31478 conn(0x7f2a2c03aac0 legacy=0x7f2a2c03cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.596+0000 7f2a4655d640 10 monclient: shutdown 2024-09-18T14:08:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.596+0000 7f2a4655d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.596+0000 7f2a4655d640 1 -- 172.21.15.45:0/3278535064 >> v1:172.21.15.45:6791/0 conn(0x7f2a4009ff70 legacy=0x7f2a40149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.596+0000 7f2a44afa640 1 -- 172.21.15.45:0/3278535064 reap_dead start 2024-09-18T14:08:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.596+0000 7f2a44afa640 1 -- 172.21.15.45:0/3278535064 reap_dead start 2024-09-18T14:08:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.596+0000 7f2a4655d640 1 -- 172.21.15.45:0/3278535064 shutdown_connections 2024-09-18T14:08:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.596+0000 7f2a4655d640 1 -- 172.21.15.45:0/3278535064 >> 172.21.15.45:0/3278535064 conn(0x7f2a400630b0 msgr2=0x7f2a400a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.596+0000 7f2a4655d640 1 -- 172.21.15.45:0/3278535064 shutdown_connections 2024-09-18T14:08:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.596+0000 7f2a4655d640 1 -- 172.21.15.45:0/3278535064 wait complete. 2024-09-18T14:08:05.640 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:08:05.640 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['g', 'i', 'h', 'd'] 2024-09-18T14:08:05.640 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-18T14:08:05.640 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-18T14:08:05.640 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-18T14:08:05.641 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-18T14:08:05.641 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:08:05.651 INFO:tasks.ceph.mon.g:Stopped 2024-09-18T14:08:05.651 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-18T14:08:05.651 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-18T14:08:05.652 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-18T14:08:05.652 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:08:05.704 INFO:tasks.ceph.mon.i:Stopped 2024-09-18T14:08:05.769 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h 2024-09-18T14:08:05.769 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-09-18T14:08:05.769 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-09-18T14:08:05.769 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:08:05.782 INFO:tasks.ceph.mon.h:Stopped 2024-09-18T14:08:05.782 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d 2024-09-18T14:08:05.782 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-18T14:08:05.782 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-09-18T14:08:05.783 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:08:05.796 INFO:tasks.ceph.mon.d:Stopped 2024-09-18T14:08:05.796 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 5 2024-09-18T14:08:05.796 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:08:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.931+0000 7fd152d9d640 10 monclient: get_monmap_and_config 2024-09-18T14:08:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.931+0000 7fd152d9d640 10 monclient: build_initial_monmap 2024-09-18T14:08:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.932+0000 7fd152d9d640 10 monclient: monmap: 2024-09-18T14:08:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:05.932918+0000 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:05.932918+0000 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:05.933 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.932+0000 7fd152d9d640 1 Processor -- start 2024-09-18T14:08:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.932+0000 7fd152d9d640 1 -- start start 2024-09-18T14:08:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.932+0000 7fd152d9d640 10 monclient: init 2024-09-18T14:08:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd152d9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd152d9d640 10 monclient: _add_conns ranks=[1,0,2,4,7,8,6,3,5] 2024-09-18T14:08:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd152d9d640 10 monclient(hunting): picked mon.noname-f con 0x7fd14c09f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd152d9d640 10 monclient(hunting): picked mon.noname-a con 0x7fd14c09fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd152d9d640 10 monclient(hunting): picked mon.noname-b con 0x7fd14c09ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd152d9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c05ed70 con 0x7fd14c09fba0 2024-09-18T14:08:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd152d9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c09dfc0 con 0x7fd14c09f7d0 2024-09-18T14:08:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd152d9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c09dce0 con 0x7fd14c09ff70 2024-09-18T14:08:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd152d9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd152d9d640 10 monclient(hunting): authenticate will time out at 1301.271272s 2024-09-18T14:08:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd14bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd14c09fba0 0x7fd14c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55078/0 (socket says 172.21.15.45:55078) 2024-09-18T14:08:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd14bfff640 1 -- 172.21.15.45:0/585269430 learned_addr learned my addr 172.21.15.45:0/585269430 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:05.933+0000 7fd15133a640 1 --1- 172.21.15.45:0/585269430 >> v1:172.21.15.45:6790/0 conn(0x7fd14c09ff70 0x7fd14c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47272/0 (socket says 172.21.15.45:47272) 2024-09-18T14:08:08.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 10 monclient(hunting): tick 2024-09-18T14:08:08.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 1 monclient(hunting): continuing hunt 2024-09-18T14:08:08.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 1 -- 172.21.15.45:0/585269430 >> v1:172.21.15.45:6790/0 conn(0x7fd14c09ff70 legacy=0x7fd14c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 1 -- 172.21.15.45:0/585269430 >> v1:172.21.15.84:6789/0 conn(0x7fd14c09f7d0 legacy=0x7fd14c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 1 -- 172.21.15.45:0/585269430 >> v1:172.21.15.45:6789/0 conn(0x7fd14c09fba0 legacy=0x7fd14c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 10 monclient: _add_conns ranks=[6,8,5,3,4,7] 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 10 monclient(hunting): picked mon.noname-d con 0x7fd1380037a0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 10 monclient(hunting): picked mon.noname-e con 0x7fd138007380 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd150b39640 1 -- 172.21.15.45:0/585269430 >> v1:172.21.15.45:6792/0 conn(0x7fd1380037a0 legacy=0x7fd138005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 10 monclient(hunting): picked mon.noname-h con 0x7fd13800b0a0 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 1 -- 172.21.15.45:0/585269430 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c0a1c00 con 0x7fd13800b0a0 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 1 -- 172.21.15.45:0/585269430 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13800ff60 con 0x7fd1380037a0 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 1 -- 172.21.15.45:0/585269430 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1380111c0 con 0x7fd138007380 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd14a7fc640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.933+0000 7fd15133a640 1 -- 172.21.15.45:0/585269430 >> v1:172.21.15.84:6791/0 conn(0x7fd13800b0a0 legacy=0x7fd13800d550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 1 -- 172.21.15.45:0/585269430 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035988579 0 0) 0x7fd1380111c0 con 0x7fd138007380 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient(hunting): my global_id is 5399 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 1 -- 172.21.15.45:0/585269430 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1340034e0 con 0x7fd138007380 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 1 -- 172.21.15.45:0/585269430 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 698519249 0 0) 0x7fd1340034e0 con 0x7fd138007380 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 1 -- 172.21.15.45:0/585269430 >> v1:172.21.15.45:6792/0 conn(0x7fd1380037a0 legacy=0x7fd138005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 1 -- 172.21.15.45:0/585269430 >> v1:172.21.15.84:6791/0 conn(0x7fd13800b0a0 legacy=0x7fd13800d550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 1 monclient: found mon.noname-e 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:08:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd15133a640 1 -- 172.21.15.45:0/585269430 reap_dead start 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 1 -- 172.21.15.45:0/585269430 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd138012420 con 0x7fd138007380 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient: _finish_auth 0 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 1 -- 172.21.15.45:0/585269430 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd140001870 con 0x7fd138007380 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd152d9d640 5 monclient: authenticate success, global_id 5399 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd152d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient: dump: 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd152d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 1 -- 172.21.15.45:0/585269430 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd140001a40 con 0x7fd138007380 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 1 -- 172.21.15.45:0/585269430 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd140004ba0 con 0x7fd138007380 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd152d9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd152d9d640 4 set_mon_vals no callback set 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.934+0000 7fd14b7fe640 10 monclient: dump: 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.935+0000 7fd152d9d640 10 monclient: shutdown 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.935+0000 7fd152d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.935+0000 7fd152d9d640 1 -- 172.21.15.45:0/585269430 >> v1:172.21.15.45:6793/0 conn(0x7fd138007380 legacy=0x7fd1380097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.935+0000 7fd152d9d640 1 -- 172.21.15.45:0/585269430 shutdown_connections 2024-09-18T14:08:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.935+0000 7fd152d9d640 1 -- 172.21.15.45:0/585269430 >> 172.21.15.45:0/585269430 conn(0x7fd14c0630b0 msgr2=0x7fd14c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.935+0000 7fd152d9d640 1 -- 172.21.15.45:0/585269430 shutdown_connections 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.935+0000 7fd152d9d640 1 -- 172.21.15.45:0/585269430 wait complete. 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.936+0000 7fd152d9d640 10 monclient: build_initial_monmap 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.936+0000 7fd152d9d640 10 monclient: monmap: 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:08.944 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.936+0000 7fd152d9d640 1 Processor -- start 2024-09-18T14:08:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd152d9d640 1 -- start start 2024-09-18T14:08:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd152d9d640 10 monclient: init 2024-09-18T14:08:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd152d9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd152d9d640 10 monclient: _add_conns ranks=[7,5,3,2,8,4,1,0,6] 2024-09-18T14:08:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd152d9d640 10 monclient(hunting): picked mon.noname-i con 0x7fd138007380 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd152d9d640 10 monclient(hunting): picked mon.noname-h con 0x7fd14c09b150 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd152d9d640 10 monclient(hunting): picked mon.noname-g con 0x7fd14c1484a0 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd152d9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c11f160 con 0x7fd14c1484a0 2024-09-18T14:08:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd152d9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c0a2560 con 0x7fd14c09b150 2024-09-18T14:08:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd152d9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c09d310 con 0x7fd138007380 2024-09-18T14:08:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd152d9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd152d9d640 10 monclient(hunting): authenticate will time out at 1304.275594s 2024-09-18T14:08:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd15133a640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7fd14c1484a0 legacy=0x7fd14c14a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd150b39640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7fd138007380 legacy=0x7fd14c131780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:08.937+0000 7fd14bfff640 1 -- >> v1:172.21.15.84:6791/0 conn(0x7fd14c09b150 legacy=0x7fd14c146d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:09.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:09.138+0000 7fd15133a640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7fd14c1484a0 legacy=0x7fd14c14a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:09.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:09.138+0000 7fd150b39640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7fd138007380 legacy=0x7fd14c131780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:09.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:09.138+0000 7fd14bfff640 1 -- >> v1:172.21.15.84:6791/0 conn(0x7fd14c09b150 legacy=0x7fd14c146d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:09.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:09.539+0000 7fd150b39640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7fd138007380 legacy=0x7fd14c131780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:09.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:09.539+0000 7fd14bfff640 1 -- >> v1:172.21.15.84:6791/0 conn(0x7fd14c09b150 legacy=0x7fd14c146d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:09.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:09.539+0000 7fd15133a640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7fd14c1484a0 legacy=0x7fd14c14a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:10.340+0000 7fd14bfff640 1 -- >> v1:172.21.15.84:6791/0 conn(0x7fd14c09b150 legacy=0x7fd14c146d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:10.340+0000 7fd15133a640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7fd14c1484a0 legacy=0x7fd14c14a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:10.340+0000 7fd150b39640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7fd138007380 legacy=0x7fd14c131780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:11.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 10 monclient(hunting): tick 2024-09-18T14:08:11.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 1 monclient(hunting): continuing hunt 2024-09-18T14:08:11.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:08:11.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7fd138007380 legacy=0x7fd14c131780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 1 -- >> v1:172.21.15.84:6791/0 conn(0x7fd14c09b150 legacy=0x7fd14c146d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7fd14c1484a0 legacy=0x7fd14c14a880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 10 monclient: _add_conns ranks=[1,0,4,6,2,8] 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 10 monclient(hunting): picked mon.noname-f con 0x7fd12c003740 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 10 monclient(hunting): picked mon.noname-a con 0x7fd12c0073c0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 10 monclient(hunting): picked mon.noname-c con 0x7fd12c00b140 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c09b520 con 0x7fd12c0073c0 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c11f160 con 0x7fd12c003740 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c0a2560 con 0x7fd12c00b140 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.937+0000 7fd1337fe640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.938+0000 7fd15133a640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fd12c00b140 0x7fd12c00d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:45092/0 (socket says 172.21.15.45:45092) 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.938+0000 7fd15133a640 1 -- 172.21.15.45:0/2011797464 learned_addr learned my addr 172.21.15.45:0/2011797464 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:11.938+0000 7fd14bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd12c0073c0 0x7fd12c009870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48856/0 (socket says 172.21.15.45:48856) 2024-09-18T14:08:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 10 monclient(hunting): tick 2024-09-18T14:08:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 1 monclient(hunting): continuing hunt 2024-09-18T14:08:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:08:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6791/0 conn(0x7fd12c00b140 legacy=0x7fd12c00d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.84:6789/0 conn(0x7fd12c003740 legacy=0x7fd12c005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6789/0 conn(0x7fd12c0073c0 legacy=0x7fd12c009870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 10 monclient: _add_conns ranks=[8,6,2] 2024-09-18T14:08:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd15133a640 1 -- 172.21.15.45:0/2011797464 reap_dead start 2024-09-18T14:08:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 10 monclient(hunting): picked mon.noname-e con 0x7fd12c011180 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd15133a640 1 -- 172.21.15.45:0/2011797464 reap_dead start 2024-09-18T14:08:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 10 monclient(hunting): picked mon.noname-d con 0x7fd12c003740 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 10 monclient(hunting): picked mon.noname-b con 0x7fd12c00b140 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c09d310 con 0x7fd12c00b140 2024-09-18T14:08:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12c00df90 con 0x7fd12c003740 2024-09-18T14:08:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12c00e1e0 con 0x7fd12c011180 2024-09-18T14:08:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd1337fe640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:14.938+0000 7fd14bfff640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6792/0 conn(0x7fd12c003740 legacy=0x7fd12c009870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:15.139+0000 7fd14bfff640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6792/0 conn(0x7fd12c003740 legacy=0x7fd12c009870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:15.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:15.540+0000 7fd14bfff640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6792/0 conn(0x7fd12c003740 legacy=0x7fd12c009870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:16.341+0000 7fd14bfff640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6792/0 conn(0x7fd12c003740 legacy=0x7fd12c009870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd1337fe640 10 monclient(hunting): tick 2024-09-18T14:08:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd1337fe640 1 monclient(hunting): continuing hunt 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd1337fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd1337fe640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6793/0 conn(0x7fd12c011180 legacy=0x7fd12c0115b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd1337fe640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6792/0 conn(0x7fd12c003740 legacy=0x7fd12c009870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd1337fe640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6790/0 conn(0x7fd12c00b140 legacy=0x7fd12c005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd1337fe640 10 monclient: _add_conns ranks=[5,6,0,3,2,1,7,8,4] 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd1337fe640 10 monclient(hunting): picked mon.noname-h con 0x7fd12c00d8f0 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd1337fe640 10 monclient(hunting): picked mon.noname-d con 0x7fd12c003740 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd14bfff640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6792/0 conn(0x7fd12c003740 legacy=0x7fd12c009870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd1337fe640 10 monclient(hunting): picked mon.noname-a con 0x7fd12c0140a0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd150b39640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.84:6791/0 conn(0x7fd12c00d8f0 legacy=0x7fd12c00e370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd1337fe640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12c00d640 con 0x7fd12c0140a0 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.938+0000 7fd1337fe640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12c00df90 con 0x7fd12c00d8f0 2024-09-18T14:08:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.939+0000 7fd1337fe640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12c019bc0 con 0x7fd12c003740 2024-09-18T14:08:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:17.939+0000 7fd1337fe640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:18.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:18.139+0000 7fd14bfff640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6792/0 conn(0x7fd12c003740 legacy=0x7fd12c009870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:18.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:18.139+0000 7fd150b39640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.84:6791/0 conn(0x7fd12c00d8f0 legacy=0x7fd12c00e370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:18.540+0000 7fd14bfff640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6792/0 conn(0x7fd12c003740 legacy=0x7fd12c009870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:18.540+0000 7fd150b39640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.84:6791/0 conn(0x7fd12c00d8f0 legacy=0x7fd12c00e370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:19.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.341+0000 7fd14bfff640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6792/0 conn(0x7fd12c003740 legacy=0x7fd12c009870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:19.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.341+0000 7fd150b39640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.84:6791/0 conn(0x7fd12c00d8f0 legacy=0x7fd12c00e370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:19.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.778+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866261611 0 0) 0x7fd12c00d640 con 0x7fd12c0140a0 2024-09-18T14:08:19.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.778+0000 7fd148ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:19.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.778+0000 7fd148ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:19.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.778+0000 7fd148ff9640 10 monclient(hunting): my global_id is 5481 2024-09-18T14:08:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.778+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1380104d0 con 0x7fd12c0140a0 2024-09-18T14:08:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.779+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3119088949 0 0) 0x7fd1380104d0 con 0x7fd12c0140a0 2024-09-18T14:08:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.779+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd12c00d640 con 0x7fd12c0140a0 2024-09-18T14:08:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.779+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd144001bd0 con 0x7fd12c0140a0 2024-09-18T14:08:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.779+0000 7fd148ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.779+0000 7fd148ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:08:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.779+0000 7fd148ff9640 10 monclient(hunting): dump: 2024-09-18T14:08:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:19.781 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.779+0000 7fd148ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1024017698 0 0) 0x7fd12c00d640 con 0x7fd12c0140a0 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6792/0 conn(0x7fd12c003740 legacy=0x7fd12c009870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.84:6791/0 conn(0x7fd12c00d8f0 legacy=0x7fd12c00e370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd148ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd148ff9640 1 monclient: found mon.a 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd148ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd148ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c01ad60 con 0x7fd12c0140a0 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd148ff9640 10 monclient: _finish_auth 0 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd148ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd152d9d640 5 monclient: authenticate success, global_id 5481 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd152d9d640 10 monclient: _renew_subs 2024-09-18T14:08:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd152d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd152d9d640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c09bd50 con 0x7fd12c0140a0 2024-09-18T14:08:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd152d9d640 10 monclient: _renew_subs 2024-09-18T14:08:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd152d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.785+0000 7fd152d9d640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c11d9d0 con 0x7fd12c0140a0 2024-09-18T14:08:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.786+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd1440020e0 con 0x7fd12c0140a0 2024-09-18T14:08:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.786+0000 7fd148ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.786+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd144004e60 con 0x7fd12c0140a0 2024-09-18T14:08:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.786+0000 7fd148ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.786+0000 7fd148ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.786+0000 7fd14affd640 4 set_mon_vals no callback set 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.786+0000 7fd148ff9640 10 monclient: dump: 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:19.789 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.787+0000 7fd152d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.787+0000 7fd152d9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.787+0000 7fd152d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:19.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.787+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd144004fc0 con 0x7fd12c0140a0 2024-09-18T14:08:19.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.787+0000 7fd152d9d640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd124005010 con 0x7fd12c0140a0 2024-09-18T14:08:19.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.787+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(430..430 src has 126..430) v4 ==== 3971+0+0 (unknown 2346877366 0 0) 0x7fd144046aa0 con 0x7fd12c0140a0 2024-09-18T14:08:19.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.789+0000 7fd150b39640 10 monclient: get_auth_request con 0x7fd13803a880 auth_method 0 2024-09-18T14:08:19.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.799+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fd144014e20 con 0x7fd12c0140a0 2024-09-18T14:08:19.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.799+0000 7fd148ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:19.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:19.799+0000 7fd148ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:19.800 INFO:tasks.rados.rados.0.smithi084.stdout:1978: done (3 left) 2024-09-18T14:08:19.800 INFO:tasks.rados.rados.0.smithi084.stdout:1979: rollback oid 34 current snap is 205 2024-09-18T14:08:19.800 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 34 to 202 2024-09-18T14:08:19.800 INFO:tasks.rados.rados.0.smithi084.stdout:1975: finishing write tid 5 to smithi08434663-12 2024-09-18T14:08:19.800 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 1023 (ObjNum 697 snap 205 seq_num 697) dirty exists 2024-09-18T14:08:19.800 INFO:tasks.rados.rados.0.smithi084.stdout:1975: left oid 12 (ObjNum 697 snap 205 seq_num 697) 2024-09-18T14:08:19.800 INFO:tasks.rados.rados.0.smithi084.stdout:1976: finishing write tid 1 to smithi08434663-11 2024-09-18T14:08:19.800 INFO:tasks.rados.rados.0.smithi084.stdout:1976: finishing write tid 2 to smithi08434663-11 2024-09-18T14:08:19.800 INFO:tasks.rados.rados.0.smithi084.stdout:1976: finishing write tid 3 to smithi08434663-11 2024-09-18T14:08:19.800 INFO:tasks.rados.rados.0.smithi084.stdout:1976: finishing write tid 4 to smithi08434663-11 2024-09-18T14:08:19.800 INFO:tasks.rados.rados.0.smithi084.stdout:1976: finishing write tid 5 to smithi08434663-11 2024-09-18T14:08:19.801 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 610 (ObjNum 698 snap 205 seq_num 698) dirty exists 2024-09-18T14:08:19.801 INFO:tasks.rados.rados.0.smithi084.stdout:1976: left oid 11 (ObjNum 698 snap 205 seq_num 698) 2024-09-18T14:08:19.801 INFO:tasks.rados.rados.0.smithi084.stdout:1977: expect (ObjNum 649 snap 191 seq_num 649) 2024-09-18T14:08:19.929 INFO:tasks.rados.rados.0.smithi084.stdout:1979: finishing rollback tid 0 to smithi08434663-34 2024-09-18T14:08:19.929 INFO:tasks.rados.rados.0.smithi084.stdout:1979: finishing rollback tid 1 to smithi08434663-34 2024-09-18T14:08:19.929 INFO:tasks.rados.rados.0.smithi084.stdout:1979: finishing rollback tid 2 to smithi08434663-34 2024-09-18T14:08:19.929 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 903 (ObjNum 679 snap 201 seq_num 679) dirty exists 2024-09-18T14:08:19.930 INFO:tasks.rados.rados.0.smithi084.stdout:1975: done (3 left) 2024-09-18T14:08:19.930 INFO:tasks.rados.rados.0.smithi084.stdout:1976: done (2 left) 2024-09-18T14:08:19.930 INFO:tasks.rados.rados.0.smithi084.stdout:1977: done (1 left) 2024-09-18T14:08:19.930 INFO:tasks.rados.rados.0.smithi084.stdout:1979: done (0 left) 2024-09-18T14:08:19.930 INFO:tasks.rados.rados.0.smithi084.stdout:1980: write (excl) oid 15 current snap is 205 2024-09-18T14:08:19.930 INFO:tasks.rados.rados.0.smithi084.stdout:1980: seq_num 699 ranges {548093=787174,1970678=449902,3045966=681137} 2024-09-18T14:08:19.936 INFO:tasks.rados.rados.0.smithi084.stdout:1980: writing smithi08434663-15 from 548093 to 1335267 tid 1 2024-09-18T14:08:19.942 INFO:tasks.rados.rados.0.smithi084.stdout:1980: writing smithi08434663-15 from 1970678 to 2420580 tid 2 2024-09-18T14:08:19.949 INFO:tasks.rados.rados.0.smithi084.stdout:1980: writing smithi08434663-15 from 3045966 to 3727103 tid 3 2024-09-18T14:08:19.950 INFO:tasks.rados.rados.0.smithi084.stdout:1981: rollback oid 35 current snap is 205 2024-09-18T14:08:19.950 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 35 to 202 2024-09-18T14:08:19.950 INFO:tasks.rados.rados.0.smithi084.stdout:1980: finishing write tid 1 to smithi08434663-15 2024-09-18T14:08:19.950 INFO:tasks.rados.rados.0.smithi084.stdout:1980: finishing write tid 2 to smithi08434663-15 2024-09-18T14:08:19.950 INFO:tasks.rados.rados.0.smithi084.stdout:1982: read oid 21 snap -1 2024-09-18T14:08:19.950 INFO:tasks.rados.rados.0.smithi084.stdout:1982: expect (ObjNum 695 snap 205 seq_num 695) 2024-09-18T14:08:19.950 INFO:tasks.rados.rados.0.smithi084.stdout:1983: read oid 23 snap -1 2024-09-18T14:08:19.950 INFO:tasks.rados.rados.0.smithi084.stdout:1983: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:08:19.950 INFO:tasks.rados.rados.0.smithi084.stdout:1984: write (excl) oid 28 current snap is 205 2024-09-18T14:08:19.950 INFO:tasks.rados.rados.0.smithi084.stdout:1984: seq_num 700 ranges {564900=578776,1629160=709913,3073389=492409} 2024-09-18T14:08:19.956 INFO:tasks.rados.rados.0.smithi084.stdout:1984: writing smithi08434663-28 from 564900 to 1143676 tid 1 2024-09-18T14:08:19.963 INFO:tasks.rados.rados.0.smithi084.stdout:1984: writing smithi08434663-28 from 1629160 to 2339073 tid 2 2024-09-18T14:08:19.970 INFO:tasks.rados.rados.0.smithi084.stdout:1984: writing smithi08434663-28 from 3073389 to 3565798 tid 3 2024-09-18T14:08:19.970 INFO:tasks.rados.rados.0.smithi084.stdout:1985: write (excl) oid 42 current snap is 205 2024-09-18T14:08:19.970 INFO:tasks.rados.rados.0.smithi084.stdout:1985: seq_num 701 ranges {581707=654025,1687642=453571,2700812=703681} 2024-09-18T14:08:19.976 INFO:tasks.rados.rados.0.smithi084.stdout:1985: writing smithi08434663-42 from 581707 to 1235732 tid 1 2024-09-18T14:08:19.981 INFO:tasks.rados.rados.0.smithi084.stdout:1985: writing smithi08434663-42 from 1687642 to 2141213 tid 2 2024-09-18T14:08:19.988 INFO:tasks.rados.rados.0.smithi084.stdout:1985: writing smithi08434663-42 from 2700812 to 3404493 tid 3 2024-09-18T14:08:19.989 INFO:tasks.rados.rados.0.smithi084.stdout:1986: copy_from oid 13 from oid 22 current snap is 205 2024-09-18T14:08:19.989 INFO:tasks.rados.rados.0.smithi084.stdout:1983: expect (ObjNum 265 snap 67 seq_num 265) 2024-09-18T14:08:20.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.038+0000 7fd152d9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:08:20.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.038+0000 7fd152d9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:08:20.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.038+0000 7fd152d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:20.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.038+0000 7fd152d9d640 1 -- 172.21.15.45:0/2011797464 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd124005840 con 0x7fd12c0140a0 2024-09-18T14:08:20.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.038+0000 7fd148ff9640 1 -- 172.21.15.45:0/2011797464 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2664 (unknown 254746657 0 975055310) 0x7fd144014c40 con 0x7fd12c0140a0 2024-09-18T14:08:20.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.038+0000 7fd148ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:08:20.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.038+0000 7fd148ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:20.039 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":88,"quorum":[0,1,2,4,8],"quorum_names":["a","f","b","c","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:08:20.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.040+0000 7fd152d9d640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.84:6812/31478 conn(0x7fd13803a880 legacy=0x7fd13803cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:20.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.040+0000 7fd152d9d640 10 monclient: shutdown 2024-09-18T14:08:20.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.040+0000 7fd152d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:20.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.040+0000 7fd152d9d640 1 -- 172.21.15.45:0/2011797464 >> v1:172.21.15.45:6789/0 conn(0x7fd12c0140a0 legacy=0x7fd12c016480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:20.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.040+0000 7fd15133a640 1 -- 172.21.15.45:0/2011797464 reap_dead start 2024-09-18T14:08:20.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.040+0000 7fd15133a640 1 -- 172.21.15.45:0/2011797464 reap_dead start 2024-09-18T14:08:20.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.040+0000 7fd152d9d640 1 -- 172.21.15.45:0/2011797464 shutdown_connections 2024-09-18T14:08:20.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.040+0000 7fd152d9d640 1 -- 172.21.15.45:0/2011797464 >> 172.21.15.45:0/2011797464 conn(0x7fd14c0630b0 msgr2=0x7fd14c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:20.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.040+0000 7fd152d9d640 1 -- 172.21.15.45:0/2011797464 shutdown_connections 2024-09-18T14:08:20.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.040+0000 7fd152d9d640 1 -- 172.21.15.45:0/2011797464 wait complete. 2024-09-18T14:08:20.051 INFO:tasks.mon_thrash.ceph_manager:quorum is size 5 2024-09-18T14:08:20.051 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon a 2024-09-18T14:08:20.051 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:08:20.066 INFO:tasks.rados.rados.0.smithi084.stdout:1982: expect (ObjNum 695 snap 205 seq_num 695) 2024-09-18T14:08:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.148+0000 7fd641b9d640 10 monclient: get_monmap_and_config 2024-09-18T14:08:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.148+0000 7fd641b9d640 10 monclient: build_initial_monmap 2024-09-18T14:08:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.148+0000 7fd641b9d640 10 monclient: monmap: 2024-09-18T14:08:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:20.149681+0000 2024-09-18T14:08:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:20.149681+0000 2024-09-18T14:08:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:20.150 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.149+0000 7fd641b9d640 1 Processor -- start 2024-09-18T14:08:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.149+0000 7fd641b9d640 1 -- start start 2024-09-18T14:08:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.149+0000 7fd641b9d640 10 monclient: init 2024-09-18T14:08:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.149+0000 7fd641b9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.149+0000 7fd641b9d640 10 monclient: _add_conns ranks=[3,8,5,7,4,1,0,6,2] 2024-09-18T14:08:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.149+0000 7fd641b9d640 10 monclient(hunting): picked mon.noname-g con 0x7fd63c0a2610 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.149+0000 7fd641b9d640 10 monclient(hunting): picked mon.noname-e con 0x7fd63c1330b0 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.149+0000 7fd641b9d640 10 monclient(hunting): picked mon.noname-h con 0x7fd63c096a20 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd641b9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c05ed70 con 0x7fd63c0a2610 2024-09-18T14:08:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd641b9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0a1740 con 0x7fd63c096a20 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd641b9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0a29e0 con 0x7fd63c1330b0 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd641b9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd641b9d640 10 monclient(hunting): authenticate will time out at 1315.488046s 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63bfff640 1 -- >> v1:172.21.15.84:6791/0 conn(0x7fd63c096a20 legacy=0x7fd63c13af40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63b7fe640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7fd63c0a2610 legacy=0x7fd63c1329a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63affd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fd63c1330b0 0x7fd63c137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:48760/0 (socket says 172.21.15.45:48760) 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63affd640 1 -- 172.21.15.45:0/3242973038 learned_addr learned my addr 172.21.15.45:0/3242973038 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 1 -- 172.21.15.45:0/3242973038 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778548706 0 0) 0x7fd63c0a29e0 con 0x7fd63c1330b0 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient(hunting): my global_id is 5408 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 1 -- 172.21.15.45:0/3242973038 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6240034e0 con 0x7fd63c1330b0 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 1 -- 172.21.15.45:0/3242973038 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1566443296 0 0) 0x7fd6240034e0 con 0x7fd63c1330b0 2024-09-18T14:08:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 1 -- 172.21.15.45:0/3242973038 >> v1:172.21.15.84:6791/0 conn(0x7fd63c096a20 legacy=0x7fd63c13af40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 1 -- 172.21.15.45:0/3242973038 >> v1:172.21.15.84:6790/0 conn(0x7fd63c0a2610 legacy=0x7fd63c1329a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 1 monclient: found mon.noname-e 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 1 -- 172.21.15.45:0/3242973038 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0970a0 con 0x7fd63c1330b0 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd641b9d640 5 monclient: authenticate success, global_id 5408 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 1 -- 172.21.15.45:0/3242973038 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd630001830 con 0x7fd63c1330b0 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd641b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:08:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient: dump: 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd641b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 1 -- 172.21.15.45:0/3242973038 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd630001a10 con 0x7fd63c1330b0 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 1 -- 172.21.15.45:0/3242973038 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd630004ba0 con 0x7fd63c1330b0 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd641b9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd641b9d640 4 set_mon_vals no callback set 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.150+0000 7fd63a7fc640 10 monclient: dump: 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.151+0000 7fd641b9d640 10 monclient: shutdown 2024-09-18T14:08:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.151+0000 7fd641b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.151+0000 7fd641b9d640 1 -- 172.21.15.45:0/3242973038 >> v1:172.21.15.45:6793/0 conn(0x7fd63c1330b0 legacy=0x7fd63c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.151+0000 7fd641b9d640 1 -- 172.21.15.45:0/3242973038 shutdown_connections 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.151+0000 7fd641b9d640 1 -- 172.21.15.45:0/3242973038 >> 172.21.15.45:0/3242973038 conn(0x7fd63c0630f0 msgr2=0x7fd63c0634f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.151+0000 7fd641b9d640 1 -- 172.21.15.45:0/3242973038 shutdown_connections 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.151+0000 7fd641b9d640 1 -- 172.21.15.45:0/3242973038 wait complete. 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.151+0000 7fd641b9d640 10 monclient: build_initial_monmap 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.151+0000 7fd641b9d640 10 monclient: monmap: 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 1 Processor -- start 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 1 -- start start 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 10 monclient: init 2024-09-18T14:08:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 10 monclient: _add_conns ranks=[7,0,4,1,6,5,2,3,8] 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 10 monclient(hunting): picked mon.noname-i con 0x7fd63c096a20 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 10 monclient(hunting): picked mon.noname-a con 0x7fd63c0a2610 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 10 monclient(hunting): picked mon.noname-c con 0x7fd63c1330b0 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0a2350 con 0x7fd63c0a2610 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c09e250 con 0x7fd63c1330b0 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c09dc90 con 0x7fd63c096a20 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd641b9d640 10 monclient(hunting): authenticate will time out at 1315.490836s 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd63b7fe640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7fd63c096a20 legacy=0x7fd63c09a910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd63affd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd63c0a2610 0x7fd63c09b020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47974/0 (socket says 172.21.15.45:47974) 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.152+0000 7fd63affd640 1 -- 172.21.15.45:0/4157822967 learned_addr learned my addr 172.21.15.45:0/4157822967 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd63affd640 1 -- 172.21.15.45:0/4157822967 >> v1:172.21.15.45:6789/0 conn(0x7fd63c0a2610 legacy=0x7fd63c09b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-18T14:08:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd63affd640 1 -- 172.21.15.45:0/4157822967 >> v1:172.21.15.45:6789/0 conn(0x7fd63c0a2610 legacy=0x7fd63c09b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:08:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd63affd640 1 --1- 172.21.15.45:0/4157822967 >> v1:172.21.15.45:6789/0 conn(0x7fd63c0a2610 0x7fd63c09b020 :-1 s=OPENED pgs=405 cs=1 l=1).handle_message read tag failed 2024-09-18T14:08:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd63affd640 1 --1- 172.21.15.45:0/4157822967 >> v1:172.21.15.45:6789/0 conn(0x7fd63c0a2610 0x7fd63c09b020 :-1 s=OPENED pgs=405 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:08:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.45:6789/0 2024-09-18T14:08:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353773437 0 0) 0x7fd63c09e250 con 0x7fd63c1330b0 2024-09-18T14:08:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient(hunting): my global_id is 164281 2024-09-18T14:08:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6280034e0 con 0x7fd63c1330b0 2024-09-18T14:08:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2812294285 0 0) 0x7fd6280034e0 con 0x7fd63c1330b0 2024-09-18T14:08:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd63c09e250 con 0x7fd63c1330b0 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd6340045a0 con 0x7fd63c1330b0 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient(hunting): dump: 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3554277797 0 0) 0x7fd63c09e250 con 0x7fd63c1330b0 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 >> v1:172.21.15.84:6792/0 conn(0x7fd63c096a20 legacy=0x7fd63c09a910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 >> v1:172.21.15.45:6789/0 conn(0x7fd63c0a2610 legacy=0x7fd63c09b020 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient: _finish_hunting 0 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 1 monclient: found mon.c 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c09df70 con 0x7fd63c1330b0 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient: _finish_auth 0 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd641b9d640 5 monclient: authenticate success, global_id 164281 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd641b9d640 10 monclient: _renew_subs 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd641b9d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd641b9d640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c12d030 con 0x7fd63c1330b0 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd641b9d640 10 monclient: _renew_subs 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd641b9d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd641b9d640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c09f9d0 con 0x7fd63c1330b0 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd634004780 con 0x7fd63c1330b0 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd634004cf0 con 0x7fd63c1330b0 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6397fa640 4 set_mon_vals no callback set 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.153+0000 7fd6237fe640 10 monclient: dump: 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.154+0000 7fd641b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.154+0000 7fd641b9d640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd61c006580 con 0x7fd61c002830 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.154+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd634004ed0 con 0x7fd63c1330b0 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.154+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(430..430 src has 126..430) v4 ==== 3971+0+0 (unknown 2346877366 0 0) 0x7fd634046eb0 con 0x7fd63c1330b0 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.154+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376073518 0 0) 0x7fd61c006580 con 0x7fd61c002830 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.154+0000 7fd6237fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.154+0000 7fd6237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.154+0000 7fd6237fe640 10 monclient(hunting): my global_id is 5490 2024-09-18T14:08:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.154+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd63c09e250 con 0x7fd61c002830 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.154+0000 7fd63affd640 10 monclient: get_auth_request con 0x7fd628039b70 auth_method 0 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1743200245 0 0) 0x7fd63c09e250 con 0x7fd61c002830 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd61c006580 con 0x7fd61c002830 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd62c004040 con 0x7fd61c002830 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 10 monclient: dump: 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 712849291 0 0) 0x7fd61c006580 con 0x7fd61c002830 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd61c00a0a0 con 0x7fd61c002830 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd62c0062e0 con 0x7fd61c002830 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 >> v1:172.21.15.45:6789/0 conn(0x7fd61c002830 legacy=0x7fd61c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.155+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 >> v1:172.21.15.45:6789/0 conn(0x7fd61c002830 legacy=0x7fd61c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd641b9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd641b9d640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd61c009660 con 0x7fd61c009040 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682517407 0 0) 0x7fd61c009660 con 0x7fd61c009040 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 10 monclient(hunting): my global_id is 5499 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd61c006580 con 0x7fd61c009040 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3890237641 0 0) 0x7fd61c006580 con 0x7fd61c009040 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd61c009660 con 0x7fd61c009040 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd62c004ac0 con 0x7fd61c009040 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 10 monclient: dump: 2024-09-18T14:08:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:20.179 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3033475954 0 0) 0x7fd61c009660 con 0x7fd61c009040 2024-09-18T14:08:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.161+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd61c009920 con 0x7fd61c009040 2024-09-18T14:08:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7512 (unknown 0 0 2559241086) 0x7fd62c005fd0 con 0x7fd61c009040 2024-09-18T14:08:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd6237fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd6237fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 >> v1:172.21.15.45:6789/0 conn(0x7fd61c009040 legacy=0x7fd61c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd6237fe640 1 -- 172.21.15.45:0/4157822967 >> v1:172.21.15.45:6789/0 conn(0x7fd61c009040 legacy=0x7fd61c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:20.179 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:20.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:20.179 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 88, 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:08:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:20.200 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd641b9d640 1 -- 172.21.15.45:0/4157822967 >> v1:172.21.15.84:6812/31478 conn(0x7fd628039b70 legacy=0x7fd62803c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd641b9d640 10 monclient: shutdown 2024-09-18T14:08:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd641b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:20.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd641b9d640 1 -- 172.21.15.45:0/4157822967 >> v1:172.21.15.45:6791/0 conn(0x7fd63c1330b0 legacy=0x7fd63c149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:20.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd63bfff640 1 -- 172.21.15.45:0/4157822967 reap_dead start 2024-09-18T14:08:20.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd63bfff640 1 -- 172.21.15.45:0/4157822967 reap_dead start 2024-09-18T14:08:20.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd641b9d640 1 -- 172.21.15.45:0/4157822967 shutdown_connections 2024-09-18T14:08:20.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd641b9d640 1 -- 172.21.15.45:0/4157822967 >> 172.21.15.45:0/4157822967 conn(0x7fd63c0630f0 msgr2=0x7fd63c133c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:20.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd641b9d640 1 -- 172.21.15.45:0/4157822967 shutdown_connections 2024-09-18T14:08:20.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.162+0000 7fd641b9d640 1 -- 172.21.15.45:0/4157822967 wait complete. 2024-09-18T14:08:20.202 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-18T14:08:20.202 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon b 2024-09-18T14:08:20.203 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:08:20.295 INFO:tasks.rados.rados.0.smithi084.stdout:1980: finishing write tid 3 to smithi08434663-15 2024-09-18T14:08:20.296 INFO:tasks.rados.rados.0.smithi084.stdout:1980: finishing write tid 4 to smithi08434663-15 2024-09-18T14:08:20.296 INFO:tasks.rados.rados.0.smithi084.stdout:1980: finishing write tid 5 to smithi08434663-15 2024-09-18T14:08:20.296 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 856 (ObjNum 699 snap 205 seq_num 699) dirty exists 2024-09-18T14:08:20.296 INFO:tasks.rados.rados.0.smithi084.stdout:1980: left oid 15 (ObjNum 699 snap 205 seq_num 699) 2024-09-18T14:08:20.296 INFO:tasks.rados.rados.0.smithi084.stdout:1981: finishing rollback tid 0 to smithi08434663-35 2024-09-18T14:08:20.296 INFO:tasks.rados.rados.0.smithi084.stdout:1980: done (6 left) 2024-09-18T14:08:20.296 INFO:tasks.rados.rados.0.smithi084.stdout:1982: done (5 left) 2024-09-18T14:08:20.296 INFO:tasks.rados.rados.0.smithi084.stdout:1983: done (4 left) 2024-09-18T14:08:20.296 INFO:tasks.rados.rados.0.smithi084.stdout:1987: read oid 49 snap -1 2024-09-18T14:08:20.296 INFO:tasks.rados.rados.0.smithi084.stdout:1987: expect (ObjNum 649 snap 191 seq_num 649) 2024-09-18T14:08:20.296 INFO:tasks.rados.rados.0.smithi084.stdout:1988: delete oid 38 current snap is 205 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:1981: finishing rollback tid 1 to smithi08434663-35 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:1981: finishing rollback tid 2 to smithi08434663-35 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 859 (ObjNum 652 snap 191 seq_num 652) dirty exists 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:1984: finishing write tid 1 to smithi08434663-28 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:1984: finishing write tid 2 to smithi08434663-28 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:1984: finishing write tid 3 to smithi08434663-28 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:1984: finishing write tid 4 to smithi08434663-28 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:1984: finishing write tid 5 to smithi08434663-28 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1029 (ObjNum 700 snap 205 seq_num 700) dirty exists 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:1984: left oid 28 (ObjNum 700 snap 205 seq_num 700) 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:1985: finishing write tid 1 to smithi08434663-42 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:1985: finishing write tid 2 to smithi08434663-42 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:1985: finishing write tid 3 to smithi08434663-42 2024-09-18T14:08:20.297 INFO:tasks.rados.rados.0.smithi084.stdout:1985: finishing write tid 4 to smithi08434663-42 2024-09-18T14:08:20.298 INFO:tasks.rados.rados.0.smithi084.stdout:1985: finishing write tid 5 to smithi08434663-42 2024-09-18T14:08:20.298 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 907 (ObjNum 701 snap 205 seq_num 701) dirty exists 2024-09-18T14:08:20.298 INFO:tasks.rados.rados.0.smithi084.stdout:1985: left oid 42 (ObjNum 701 snap 205 seq_num 701) 2024-09-18T14:08:20.298 INFO:tasks.rados.rados.0.smithi084.stdout:1986: finishing copy_from racing read to smithi08434663-13 2024-09-18T14:08:20.298 INFO:tasks.rados.rados.0.smithi084.stdout:1986: finishing copy_from to smithi08434663-13 2024-09-18T14:08:20.298 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 739 (ObjNum 615 snap 183 seq_num 615) dirty exists 2024-09-18T14:08:20.298 INFO:tasks.rados.rados.0.smithi084.stdout:1981: done (5 left) 2024-09-18T14:08:20.299 INFO:tasks.rados.rados.0.smithi084.stdout:1984: done (4 left) 2024-09-18T14:08:20.299 INFO:tasks.rados.rados.0.smithi084.stdout:1985: done (3 left) 2024-09-18T14:08:20.299 INFO:tasks.rados.rados.0.smithi084.stdout:1986: done (2 left) 2024-09-18T14:08:20.299 INFO:tasks.rados.rados.0.smithi084.stdout:1988: done (1 left) 2024-09-18T14:08:20.299 INFO:tasks.rados.rados.0.smithi084.stdout:1989: write (excl) oid 48 current snap is 205 2024-09-18T14:08:20.299 INFO:tasks.rados.rados.0.smithi084.stdout:1989: seq_num 703 ranges {615321=404523,1804606=457240,2755658=726225} 2024-09-18T14:08:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.301+0000 7f7a72d5d640 10 monclient: get_monmap_and_config 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.301+0000 7f7a72d5d640 10 monclient: build_initial_monmap 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.301+0000 7f7a72d5d640 10 monclient: monmap: 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:20.302547+0000 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:20.302547+0000 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:20.303 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:20.303 INFO:tasks.rados.rados.0.smithi084.stdout:1989: writing smithi08434663-48 from 615321 to 1019844 tid 1 2024-09-18T14:08:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.302+0000 7f7a72d5d640 1 Processor -- start 2024-09-18T14:08:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.302+0000 7f7a72d5d640 1 -- start start 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.302+0000 7f7a72d5d640 10 monclient: init 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a72d5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a72d5d640 10 monclient: _add_conns ranks=[6,3,7,8,0,5,1,4,2] 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a72d5d640 10 monclient(hunting): picked mon.noname-d con 0x7f7a6c06e330 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a72d5d640 10 monclient(hunting): picked mon.noname-g con 0x7f7a6c06e700 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a72d5d640 10 monclient(hunting): picked mon.noname-i con 0x7f7a6c06ead0 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a72d5d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c05ed70 con 0x7f7a6c06e700 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a72d5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c069de0 con 0x7f7a6c06e330 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a72d5d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c069790 con 0x7f7a6c06ead0 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a72d5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a72d5d640 10 monclient(hunting): authenticate will time out at 1315.641176s 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a70af9640 1 -- >> v1:172.21.15.45:6792/0 conn(0x7f7a6c06e330 legacy=0x7f7a6c132cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a712fa640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7f7a6c06ead0 legacy=0x7f7a6c13afa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.303+0000 7f7a6bfff640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7f7a6c06e700 legacy=0x7f7a6c137870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:20.309 INFO:tasks.rados.rados.0.smithi084.stdout:1989: writing smithi08434663-48 from 1804606 to 2261846 tid 2 2024-09-18T14:08:20.316 INFO:tasks.rados.rados.0.smithi084.stdout:1989: writing smithi08434663-48 from 2755658 to 3481883 tid 3 2024-09-18T14:08:20.316 INFO:tasks.rados.rados.0.smithi084.stdout:1990: snap_remove snap 202 2024-09-18T14:08:20.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.504+0000 7f7a70af9640 1 -- >> v1:172.21.15.45:6792/0 conn(0x7f7a6c06e330 legacy=0x7f7a6c132cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:20.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.504+0000 7f7a712fa640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7f7a6c06ead0 legacy=0x7f7a6c13afa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:20.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.504+0000 7f7a6bfff640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7f7a6c06e700 legacy=0x7f7a6c137870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:20.792 INFO:tasks.rados.rados.0.smithi084.stdout:1987: expect (ObjNum 649 snap 191 seq_num 649) 2024-09-18T14:08:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.905+0000 7f7a70af9640 1 -- >> v1:172.21.15.45:6792/0 conn(0x7f7a6c06e330 legacy=0x7f7a6c132cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.905+0000 7f7a6bfff640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7f7a6c06e700 legacy=0x7f7a6c137870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:20.905+0000 7f7a712fa640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7f7a6c06ead0 legacy=0x7f7a6c13afa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:20.923 INFO:tasks.rados.rados.0.smithi084.stdout:1989: finishing write tid 1 to smithi08434663-48 2024-09-18T14:08:20.923 INFO:tasks.rados.rados.0.smithi084.stdout:1989: finishing write tid 2 to smithi08434663-48 2024-09-18T14:08:20.923 INFO:tasks.rados.rados.0.smithi084.stdout:1989: finishing write tid 3 to smithi08434663-48 2024-09-18T14:08:20.923 INFO:tasks.rados.rados.0.smithi084.stdout:1989: finishing write tid 4 to smithi08434663-48 2024-09-18T14:08:20.923 INFO:tasks.rados.rados.0.smithi084.stdout:1989: finishing write tid 5 to smithi08434663-48 2024-09-18T14:08:20.924 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 607 (ObjNum 703 snap 205 seq_num 703) dirty exists 2024-09-18T14:08:20.924 INFO:tasks.rados.rados.0.smithi084.stdout:1989: left oid 48 (ObjNum 703 snap 205 seq_num 703) 2024-09-18T14:08:20.924 INFO:tasks.rados.rados.0.smithi084.stdout:1987: done (2 left) 2024-09-18T14:08:20.924 INFO:tasks.rados.rados.0.smithi084.stdout:1989: done (1 left) 2024-09-18T14:08:20.924 INFO:tasks.rados.rados.0.smithi084.stdout:1990: done (0 left) 2024-09-18T14:08:20.924 INFO:tasks.rados.rados.0.smithi084.stdout:1991: write (excl) oid 29 current snap is 205 2024-09-18T14:08:20.924 INFO:tasks.rados.rados.0.smithi084.stdout:1991: seq_num 704 ranges {632128=479772,1746735=600898,3066728=653850} 2024-09-18T14:08:20.929 INFO:tasks.rados.rados.0.smithi084.stdout:1991: writing smithi08434663-29 from 632128 to 1111900 tid 1 2024-09-18T14:08:20.935 INFO:tasks.rados.rados.0.smithi084.stdout:1991: writing smithi08434663-29 from 1746735 to 2347633 tid 2 2024-09-18T14:08:20.943 INFO:tasks.rados.rados.0.smithi084.stdout:1991: writing smithi08434663-29 from 3066728 to 3720578 tid 3 2024-09-18T14:08:20.943 INFO:tasks.rados.rados.0.smithi084.stdout:1992: rollback oid 35 current snap is 205 2024-09-18T14:08:20.943 INFO:tasks.rados.rados.0.smithi084.stdout:1992: done (1 left) 2024-09-18T14:08:20.943 INFO:tasks.rados.rados.0.smithi084.stdout:1993: delete oid 45 current snap is 205 2024-09-18T14:08:20.943 INFO:tasks.rados.rados.0.smithi084.stdout:1991: finishing write tid 1 to smithi08434663-29 2024-09-18T14:08:20.943 INFO:tasks.rados.rados.0.smithi084.stdout:1991: finishing write tid 2 to smithi08434663-29 2024-09-18T14:08:20.946 INFO:tasks.rados.rados.0.smithi084.stdout:1993: done (1 left) 2024-09-18T14:08:20.946 INFO:tasks.rados.rados.0.smithi084.stdout:1994: write oid 16 current snap is 205 2024-09-18T14:08:20.946 INFO:tasks.rados.rados.0.smithi084.stdout:1994: seq_num 706 ranges {665742=630270,1863699=604567,3121574=676394} 2024-09-18T14:08:20.953 INFO:tasks.rados.rados.0.smithi084.stdout:1994: writing smithi08434663-16 from 665742 to 1296012 tid 1 2024-09-18T14:08:20.959 INFO:tasks.rados.rados.0.smithi084.stdout:1994: writing smithi08434663-16 from 1863699 to 2468266 tid 2 2024-09-18T14:08:20.967 INFO:tasks.rados.rados.0.smithi084.stdout:1994: writing smithi08434663-16 from 3121574 to 3797968 tid 3 2024-09-18T14:08:20.967 INFO:tasks.rados.rados.0.smithi084.stdout:1995: write oid 3 current snap is 205 2024-09-18T14:08:20.967 INFO:tasks.rados.rados.0.smithi084.stdout:1991: finishing write tid 3 to smithi08434663-29 2024-09-18T14:08:20.967 INFO:tasks.rados.rados.0.smithi084.stdout:1991: finishing write tid 4 to smithi08434663-29 2024-09-18T14:08:20.967 INFO:tasks.rados.rados.0.smithi084.stdout:1991: finishing write tid 5 to smithi08434663-29 2024-09-18T14:08:20.968 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 858 (ObjNum 704 snap 205 seq_num 704) dirty exists 2024-09-18T14:08:20.968 INFO:tasks.rados.rados.0.smithi084.stdout:1991: left oid 29 (ObjNum 704 snap 205 seq_num 704) 2024-09-18T14:08:20.968 INFO:tasks.rados.rados.0.smithi084.stdout:1994: finishing write tid 1 to smithi08434663-16 2024-09-18T14:08:20.968 INFO:tasks.rados.rados.0.smithi084.stdout:1994: finishing write tid 2 to smithi08434663-16 2024-09-18T14:08:20.968 INFO:tasks.rados.rados.0.smithi084.stdout:1995: seq_num 707 ranges {682549=705519,1922181=464578,3148997=487666} 2024-09-18T14:08:20.974 INFO:tasks.rados.rados.0.smithi084.stdout:1995: writing smithi08434663-3 from 682549 to 1388068 tid 1 2024-09-18T14:08:20.979 INFO:tasks.rados.rados.0.smithi084.stdout:1995: writing smithi08434663-3 from 1922181 to 2386759 tid 2 2024-09-18T14:08:20.987 INFO:tasks.rados.rados.0.smithi084.stdout:1995: writing smithi08434663-3 from 3148997 to 3636663 tid 3 2024-09-18T14:08:20.988 INFO:tasks.rados.rados.0.smithi084.stdout:1991: done (2 left) 2024-09-18T14:08:20.988 INFO:tasks.rados.rados.0.smithi084.stdout:1996: read oid 34 snap -1 2024-09-18T14:08:20.988 INFO:tasks.rados.rados.0.smithi084.stdout:1996: expect (ObjNum 679 snap 201 seq_num 679) 2024-09-18T14:08:20.988 INFO:tasks.rados.rados.0.smithi084.stdout:1997: read oid 5 snap -1 2024-09-18T14:08:20.988 INFO:tasks.rados.rados.0.smithi084.stdout:1997: expect (ObjNum 678 snap 201 seq_num 678) 2024-09-18T14:08:20.988 INFO:tasks.rados.rados.0.smithi084.stdout:1998: copy_from oid 50 from oid 17 current snap is 205 2024-09-18T14:08:20.988 INFO:tasks.rados.rados.0.smithi084.stdout:1994: finishing write tid 3 to smithi08434663-16 2024-09-18T14:08:20.988 INFO:tasks.rados.rados.0.smithi084.stdout:1994: finishing write tid 4 to smithi08434663-16 2024-09-18T14:08:20.988 INFO:tasks.rados.rados.0.smithi084.stdout:1994: finishing write tid 5 to smithi08434663-16 2024-09-18T14:08:20.988 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 863 (ObjNum 706 snap 205 seq_num 706) dirty exists 2024-09-18T14:08:20.988 INFO:tasks.rados.rados.0.smithi084.stdout:1994: left oid 16 (ObjNum 706 snap 205 seq_num 706) 2024-09-18T14:08:20.988 INFO:tasks.rados.rados.0.smithi084.stdout:1995: finishing write tid 1 to smithi08434663-3 2024-09-18T14:08:20.989 INFO:tasks.rados.rados.0.smithi084.stdout:1995: finishing write tid 2 to smithi08434663-3 2024-09-18T14:08:20.989 INFO:tasks.rados.rados.0.smithi084.stdout:1994: done (4 left) 2024-09-18T14:08:20.989 INFO:tasks.rados.rados.0.smithi084.stdout:1999: copy_from oid 39 from oid 28 current snap is 205 2024-09-18T14:08:20.989 INFO:tasks.rados.rados.0.smithi084.stdout:2000: copy_from oid 49 from oid 25 current snap is 205 2024-09-18T14:08:20.989 INFO:tasks.rados.rados.0.smithi084.stdout:2001: read oid 37 snap -1 2024-09-18T14:08:20.989 INFO:tasks.rados.rados.0.smithi084.stdout:2001: expect deleted 2024-09-18T14:08:20.989 INFO:tasks.rados.rados.0.smithi084.stdout:2002: copy_from oid 8 from oid 7 current snap is 205 2024-09-18T14:08:20.989 INFO:tasks.rados.rados.0.smithi084.stdout:2003: snap_create 2024-09-18T14:08:20.991 INFO:tasks.rados.rados.0.smithi084.stdout:1997: expect (ObjNum 678 snap 201 seq_num 678) 2024-09-18T14:08:21.104 INFO:tasks.rados.rados.0.smithi084.stdout:1995: finishing write tid 3 to smithi08434663-3 2024-09-18T14:08:21.105 INFO:tasks.rados.rados.0.smithi084.stdout:1995: finishing write tid 4 to smithi08434663-3 2024-09-18T14:08:21.105 INFO:tasks.rados.rados.0.smithi084.stdout:1995: finishing write tid 5 to smithi08434663-3 2024-09-18T14:08:21.105 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 743 (ObjNum 707 snap 205 seq_num 707) dirty exists 2024-09-18T14:08:21.105 INFO:tasks.rados.rados.0.smithi084.stdout:1995: left oid 3 (ObjNum 707 snap 205 seq_num 707) 2024-09-18T14:08:21.105 INFO:tasks.rados.rados.0.smithi084.stdout:1999: finishing copy_from racing read to smithi08434663-39 2024-09-18T14:08:21.105 INFO:tasks.rados.rados.0.smithi084.stdout:1998: finishing copy_from racing read to smithi08434663-50 2024-09-18T14:08:21.105 INFO:tasks.rados.rados.0.smithi084.stdout:2000: finishing copy_from racing read to smithi08434663-49 2024-09-18T14:08:21.105 INFO:tasks.rados.rados.0.smithi084.stdout:1996: expect (ObjNum 679 snap 201 seq_num 679) 2024-09-18T14:08:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:2002: finishing copy_from racing read to smithi08434663-8 2024-09-18T14:08:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:2002: finishing copy_from to smithi08434663-8 2024-09-18T14:08:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 559 (ObjNum 550 snap 168 seq_num 550) dirty exists 2024-09-18T14:08:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:1999: finishing copy_from to smithi08434663-39 2024-09-18T14:08:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 1029 (ObjNum 700 snap 205 seq_num 700) dirty exists 2024-09-18T14:08:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:1998: finishing copy_from to smithi08434663-50 2024-09-18T14:08:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 854 (ObjNum 638 snap 189 seq_num 638) dirty exists 2024-09-18T14:08:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:2000: finishing copy_from to smithi08434663-49 2024-09-18T14:08:21.316 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 599 (ObjNum 672 snap 200 seq_num 672) dirty exists 2024-09-18T14:08:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:21.706+0000 7f7a70af9640 1 -- >> v1:172.21.15.45:6792/0 conn(0x7f7a6c06e330 legacy=0x7f7a6c132cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:21.706+0000 7f7a6bfff640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7f7a6c06e700 legacy=0x7f7a6c137870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:21.706+0000 7f7a712fa640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7f7a6c06ead0 legacy=0x7f7a6c13afa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:21.793 INFO:tasks.rados.rados.0.smithi084.stdout:1995: done (8 left) 2024-09-18T14:08:21.793 INFO:tasks.rados.rados.0.smithi084.stdout:1996: done (7 left) 2024-09-18T14:08:21.794 INFO:tasks.rados.rados.0.smithi084.stdout:1997: done (6 left) 2024-09-18T14:08:21.794 INFO:tasks.rados.rados.0.smithi084.stdout:1998: done (5 left) 2024-09-18T14:08:21.794 INFO:tasks.rados.rados.0.smithi084.stdout:1999: done (4 left) 2024-09-18T14:08:21.794 INFO:tasks.rados.rados.0.smithi084.stdout:2000: done (3 left) 2024-09-18T14:08:21.794 INFO:tasks.rados.rados.0.smithi084.stdout:2001: done (2 left) 2024-09-18T14:08:21.794 INFO:tasks.rados.rados.0.smithi084.stdout:2002: done (1 left) 2024-09-18T14:08:21.794 INFO:tasks.rados.rados.0.smithi084.stdout:2003: done (0 left) 2024-09-18T14:08:21.794 INFO:tasks.rados.rados.0.smithi084.stdout:2004: snap_create 2024-09-18T14:08:22.241 INFO:tasks.rados.rados.0.smithi084.stdout:2004: done (0 left) 2024-09-18T14:08:22.241 INFO:tasks.rados.rados.0.smithi084.stdout:2005: write oid 41 current snap is 207 2024-09-18T14:08:22.241 INFO:tasks.rados.rados.0.smithi084.stdout:2005: seq_num 708 ranges {699356=497121,1980663=608236,3176420=415291} 2024-09-18T14:08:22.247 INFO:tasks.rados.rados.0.smithi084.stdout:2005: writing smithi08434663-41 from 699356 to 1196477 tid 1 2024-09-18T14:08:22.254 INFO:tasks.rados.rados.0.smithi084.stdout:2005: writing smithi08434663-41 from 1980663 to 2588899 tid 2 2024-09-18T14:08:22.260 INFO:tasks.rados.rados.0.smithi084.stdout:2005: writing smithi08434663-41 from 3176420 to 3591711 tid 3 2024-09-18T14:08:22.260 INFO:tasks.rados.rados.0.smithi084.stdout:2006: read oid 44 snap 206 2024-09-18T14:08:22.260 INFO:tasks.rados.rados.0.smithi084.stdout:2006: expect (ObjNum 634 snap 189 seq_num 634) 2024-09-18T14:08:22.260 INFO:tasks.rados.rados.0.smithi084.stdout:2007: snap_remove snap 205 2024-09-18T14:08:23.243 INFO:tasks.rados.rados.0.smithi084.stdout:2007: done (2 left) 2024-09-18T14:08:23.244 INFO:tasks.rados.rados.0.smithi084.stdout:2008: delete oid 40 current snap is 207 2024-09-18T14:08:23.244 INFO:tasks.rados.rados.0.smithi084.stdout:2005: finishing write tid 1 to smithi08434663-41 2024-09-18T14:08:23.244 INFO:tasks.rados.rados.0.smithi084.stdout:2005: finishing write tid 2 to smithi08434663-41 2024-09-18T14:08:23.244 INFO:tasks.rados.rados.0.smithi084.stdout:2005: finishing write tid 3 to smithi08434663-41 2024-09-18T14:08:23.244 INFO:tasks.rados.rados.0.smithi084.stdout:2005: finishing write tid 4 to smithi08434663-41 2024-09-18T14:08:23.244 INFO:tasks.rados.rados.0.smithi084.stdout:2005: finishing write tid 5 to smithi08434663-41 2024-09-18T14:08:23.244 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 917 (ObjNum 708 snap 207 seq_num 708) dirty exists 2024-09-18T14:08:23.244 INFO:tasks.rados.rados.0.smithi084.stdout:2005: left oid 41 (ObjNum 708 snap 207 seq_num 708) 2024-09-18T14:08:23.245 INFO:tasks.rados.rados.0.smithi084.stdout:2006: expect (ObjNum 634 snap 189 seq_num 634) 2024-09-18T14:08:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 10 monclient(hunting): tick 2024-09-18T14:08:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 1 monclient(hunting): continuing hunt 2024-09-18T14:08:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:08:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7f7a6c06ead0 legacy=0x7f7a6c13afa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:08:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 1 -- >> v1:172.21.15.45:6792/0 conn(0x7f7a6c06e330 legacy=0x7f7a6c132cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:08:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7f7a6c06e700 legacy=0x7f7a6c137870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:08:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 10 monclient: _add_conns ranks=[1,2,0,4,5,8] 2024-09-18T14:08:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 10 monclient(hunting): picked mon.noname-f con 0x7f7a50003740 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 10 monclient(hunting): picked mon.noname-b con 0x7f7a50007320 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 10 monclient(hunting): picked mon.noname-a con 0x7f7a5000b0a0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c0694b0 con 0x7f7a5000b0a0 2024-09-18T14:08:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c05ed70 con 0x7f7a50003740 2024-09-18T14:08:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c069de0 con 0x7f7a50007320 2024-09-18T14:08:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6a7fc640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6bfff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f7a50007320 0x7f7a500097d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55122/0 (socket says 172.21.15.45:55122) 2024-09-18T14:08:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6bfff640 1 -- 172.21.15.45:0/201521026 learned_addr learned my addr 172.21.15.45:0/201521026 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374085572 0 0) 0x7f7a6c069de0 con 0x7f7a50007320 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 10 monclient(hunting): my global_id is 144740 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a5c0034e0 con 0x7f7a50007320 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329524616 0 0) 0x7f7a6c0694b0 con 0x7f7a5000b0a0 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 10 monclient(hunting): my global_id is 5508 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a6c069de0 con 0x7f7a5000b0a0 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237038309 0 0) 0x7f7a6c05ed70 con 0x7f7a50003740 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 10 monclient(hunting): my global_id is 164224 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.303+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a6c0694b0 con 0x7f7a50003740 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4143246808 0 0) 0x7f7a5c0034e0 con 0x7f7a50007320 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 >> v1:172.21.15.84:6789/0 conn(0x7f7a50003740 legacy=0x7f7a50005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 >> v1:172.21.15.45:6789/0 conn(0x7f7a5000b0a0 legacy=0x7f7a5000d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 1 monclient: found mon.noname-b 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a712fa640 1 -- 172.21.15.45:0/201521026 reap_dead start 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c069790 con 0x7f7a50007320 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 10 monclient: _finish_auth 0 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a580041e0 con 0x7f7a50007320 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 10 monclient: dump: 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a72d5d640 5 monclient: authenticate success, global_id 144740 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a72d5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7a58004340 con 0x7f7a50007320 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a72d5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 1 -- 172.21.15.45:0/201521026 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a58004930 con 0x7f7a50007320 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a72d5d640 4 set_mon_vals no callback set 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.304+0000 7f7a6b7fe640 10 monclient: dump: 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:23.312 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.305+0000 7f7a72d5d640 10 monclient: shutdown 2024-09-18T14:08:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.305+0000 7f7a72d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.305+0000 7f7a72d5d640 1 -- 172.21.15.45:0/201521026 >> v1:172.21.15.45:6790/0 conn(0x7f7a50007320 legacy=0x7f7a500097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.305+0000 7f7a72d5d640 1 -- 172.21.15.45:0/201521026 shutdown_connections 2024-09-18T14:08:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.305+0000 7f7a72d5d640 1 -- 172.21.15.45:0/201521026 >> 172.21.15.45:0/201521026 conn(0x7f7a6c0a4530 msgr2=0x7f7a6c0a0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.305+0000 7f7a72d5d640 1 -- 172.21.15.45:0/201521026 shutdown_connections 2024-09-18T14:08:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.305+0000 7f7a72d5d640 1 -- 172.21.15.45:0/201521026 wait complete. 2024-09-18T14:08:23.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.305+0000 7f7a72d5d640 10 monclient: build_initial_monmap 2024-09-18T14:08:23.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.306+0000 7f7a72d5d640 10 monclient: monmap: 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:23.318 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.306+0000 7f7a72d5d640 1 Processor -- start 2024-09-18T14:08:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.306+0000 7f7a72d5d640 1 -- start start 2024-09-18T14:08:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.306+0000 7f7a72d5d640 10 monclient: init 2024-09-18T14:08:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a72d5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a72d5d640 10 monclient: _add_conns ranks=[5,4,7,2,6,8,3,1,0] 2024-09-18T14:08:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a72d5d640 10 monclient(hunting): picked mon.noname-h con 0x7f7a50007320 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a72d5d640 10 monclient(hunting): picked mon.noname-c con 0x7f7a6c11f1d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a72d5d640 10 monclient(hunting): picked mon.noname-i con 0x7f7a6c1480d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a72d5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c068800 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a70af9640 1 -- >> v1:172.21.15.84:6791/0 conn(0x7f7a50007320 legacy=0x7f7a6c131650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a72d5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c121a80 con 0x7f7a50007320 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a72d5d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c11f5a0 con 0x7f7a6c1480d0 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a72d5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a712fa640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7f7a6c1480d0 legacy=0x7f7a6c14a4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a72d5d640 10 monclient(hunting): authenticate will time out at 1318.645384s 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a6bfff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f7a6c11f1d0 0x7f7a6c1469b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58786/0 (socket says 172.21.15.45:58786) 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a6bfff640 1 -- 172.21.15.45:0/432351079 learned_addr learned my addr 172.21.15.45:0/432351079 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053689493 0 0) 0x7f7a6c068800 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a68ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a68ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a68ff9640 10 monclient(hunting): my global_id is 164290 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a50008f70 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.307+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2199054121 0 0) 0x7f7a50008f70 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a6c068800 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a580047f0 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 10 monclient(hunting): dump: 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:23.321 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:23.321 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:23.321 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:23.321 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:23.321 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:23.321 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:23.322 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:23.322 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:23.322 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:23.322 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:23.322 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:23.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:23.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1548552333 0 0) 0x7f7a6c068800 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 >> v1:172.21.15.84:6792/0 conn(0x7f7a6c1480d0 legacy=0x7f7a6c14a4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:08:23.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 >> v1:172.21.15.84:6791/0 conn(0x7f7a50007320 legacy=0x7f7a6c131650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:08:23.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:08:23.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 1 monclient: found mon.c 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c11fc70 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 10 monclient: _finish_auth 0 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a72d5d640 5 monclient: authenticate success, global_id 164290 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a72d5d640 10 monclient: _renew_subs 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a72d5d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a72d5d640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6c1217a0 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a72d5d640 10 monclient: _renew_subs 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a72d5d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a72d5d640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6c133ab0 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7a58004d00 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a580051f0 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a6a7fc640 4 set_mon_vals no callback set 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.308+0000 7f7a68ff9640 10 monclient: dump: 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.309+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7a58005350 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.309+0000 7f7a72d5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.309+0000 7f7a72d5d640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7a4c006580 con 0x7f7a4c002830 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.309+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(434..434 src has 126..434) v4 ==== 3955+0+0 (unknown 4169200971 0 0) 0x7f7a58046d90 con 0x7f7a6c11f1d0 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.309+0000 7f7a712fa640 10 monclient: get_auth_request con 0x7f7a5003abe0 auth_method 0 2024-09-18T14:08:23.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.309+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220800755 0 0) 0x7f7a4c006580 con 0x7f7a4c002830 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.309+0000 7f7a68ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.309+0000 7f7a68ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.309+0000 7f7a68ff9640 10 monclient(hunting): my global_id is 144749 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.309+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a6c068800 con 0x7f7a4c002830 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2447827393 0 0) 0x7f7a6c068800 con 0x7f7a4c002830 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a4c006580 con 0x7f7a4c002830 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a60003c60 con 0x7f7a4c002830 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 10 monclient: dump: 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:23.326 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1340515695 0 0) 0x7f7a4c006580 con 0x7f7a4c002830 2024-09-18T14:08:23.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a4c00a0a0 con 0x7f7a4c002830 2024-09-18T14:08:23.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7a60006510 con 0x7f7a4c002830 2024-09-18T14:08:23.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:23.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:23.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 >> v1:172.21.15.45:6790/0 conn(0x7f7a4c002830 legacy=0x7f7a4c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:23.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.310+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 >> v1:172.21.15.45:6790/0 conn(0x7f7a4c002830 legacy=0x7f7a4c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:23.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.316+0000 7f7a72d5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:23.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.316+0000 7f7a72d5d640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7a4c007540 con 0x7f7a4c0068f0 2024-09-18T14:08:23.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214657531 0 0) 0x7f7a4c007540 con 0x7f7a4c0068f0 2024-09-18T14:08:23.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:23.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 10 monclient(hunting): my global_id is 144758 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a4c006580 con 0x7f7a4c0068f0 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 814363732 0 0) 0x7f7a4c006580 con 0x7f7a4c0068f0 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a4c007540 con 0x7f7a4c0068f0 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a60004e30 con 0x7f7a4c0068f0 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 10 monclient: dump: 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:23.330 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1998596097 0 0) 0x7f7a4c007540 con 0x7f7a4c0068f0 2024-09-18T14:08:23.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7a4c0087a0 con 0x7f7a4c0068f0 2024-09-18T14:08:23.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7512 (unknown 0 0 1693368834) 0x7f7a60006260 con 0x7f7a4c0068f0 2024-09-18T14:08:23.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:23.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:23.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 >> v1:172.21.15.45:6790/0 conn(0x7f7a4c0068f0 legacy=0x7f7a4c006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:23.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.317+0000 7f7a68ff9640 1 -- 172.21.15.45:0/432351079 >> v1:172.21.15.45:6790/0 conn(0x7f7a4c0068f0 legacy=0x7f7a4c006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:23.332 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 88, 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:23.352 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:23.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.318+0000 7f7a72d5d640 1 -- 172.21.15.45:0/432351079 >> v1:172.21.15.84:6812/31478 conn(0x7f7a5003abe0 legacy=0x7f7a5003d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:23.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.318+0000 7f7a72d5d640 10 monclient: shutdown 2024-09-18T14:08:23.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.318+0000 7f7a72d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:23.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.318+0000 7f7a72d5d640 1 -- 172.21.15.45:0/432351079 >> v1:172.21.15.45:6791/0 conn(0x7f7a6c11f1d0 legacy=0x7f7a6c1469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:23.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.318+0000 7f7a712fa640 1 -- 172.21.15.45:0/432351079 reap_dead start 2024-09-18T14:08:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.318+0000 7f7a712fa640 1 -- 172.21.15.45:0/432351079 reap_dead start 2024-09-18T14:08:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.318+0000 7f7a72d5d640 1 -- 172.21.15.45:0/432351079 shutdown_connections 2024-09-18T14:08:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.318+0000 7f7a72d5d640 1 -- 172.21.15.45:0/432351079 >> 172.21.15.45:0/432351079 conn(0x7f7a6c0a4530 msgr2=0x7f7a6c0a18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.318+0000 7f7a72d5d640 1 -- 172.21.15.45:0/432351079 shutdown_connections 2024-09-18T14:08:23.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.318+0000 7f7a72d5d640 1 -- 172.21.15.45:0/432351079 wait complete. 2024-09-18T14:08:23.354 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-18T14:08:23.354 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon c 2024-09-18T14:08:23.355 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:08:23.390 INFO:tasks.rados.rados.0.smithi084.stdout:2005: done (2 left) 2024-09-18T14:08:23.390 INFO:tasks.rados.rados.0.smithi084.stdout:2006: done (1 left) 2024-09-18T14:08:23.390 INFO:tasks.rados.rados.0.smithi084.stdout:2008: done (0 left) 2024-09-18T14:08:23.390 INFO:tasks.rados.rados.0.smithi084.stdout:2009: snap_remove snap 206 2024-09-18T14:08:23.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.457+0000 7ff19d80f640 10 monclient: get_monmap_and_config 2024-09-18T14:08:23.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.457+0000 7ff19d80f640 10 monclient: build_initial_monmap 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.457+0000 7ff19d80f640 10 monclient: monmap: 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:23.458341+0000 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:23.458341+0000 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:23.458 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:23.459 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:23.459 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:23.459 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 1 Processor -- start 2024-09-18T14:08:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 1 -- start start 2024-09-18T14:08:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 10 monclient: init 2024-09-18T14:08:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 10 monclient: _add_conns ranks=[3,6,5,1,8,0,7,4,2] 2024-09-18T14:08:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 10 monclient(hunting): picked mon.noname-g con 0x7ff19809f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:08:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 10 monclient(hunting): picked mon.noname-d con 0x7ff19809fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:08:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 10 monclient(hunting): picked mon.noname-h con 0x7ff19809ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:08:23.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19805ed70 con 0x7ff19809f7d0 2024-09-18T14:08:23.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19809dfc0 con 0x7ff19809ff70 2024-09-18T14:08:23.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff18ffff640 1 -- >> v1:172.21.15.45:6792/0 conn(0x7ff19809fba0 legacy=0x7ff198137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:23.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19809dce0 con 0x7ff19809fba0 2024-09-18T14:08:23.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:23.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff19d80f640 10 monclient(hunting): authenticate will time out at 1318.796811s 2024-09-18T14:08:23.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff1977fe640 1 -- >> v1:172.21.15.84:6791/0 conn(0x7ff19809ff70 legacy=0x7ff19813b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:23.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.458+0000 7ff196ffd640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7ff19809f7d0 legacy=0x7ff198132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.659+0000 7ff18ffff640 1 -- >> v1:172.21.15.45:6792/0 conn(0x7ff19809fba0 legacy=0x7ff198137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.659+0000 7ff196ffd640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7ff19809f7d0 legacy=0x7ff198132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:23.659+0000 7ff1977fe640 1 -- >> v1:172.21.15.84:6791/0 conn(0x7ff19809ff70 legacy=0x7ff19813b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:24.060+0000 7ff18ffff640 1 -- >> v1:172.21.15.45:6792/0 conn(0x7ff19809fba0 legacy=0x7ff198137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:24.060+0000 7ff1977fe640 1 -- >> v1:172.21.15.84:6791/0 conn(0x7ff19809ff70 legacy=0x7ff19813b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:24.060+0000 7ff196ffd640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7ff19809f7d0 legacy=0x7ff198132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:24.252 INFO:tasks.rados.rados.0.smithi084.stdout:2009: done (0 left) 2024-09-18T14:08:24.253 INFO:tasks.rados.rados.0.smithi084.stdout:2010: read oid 22 snap -1 2024-09-18T14:08:24.253 INFO:tasks.rados.rados.0.smithi084.stdout:2010: expect (ObjNum 615 snap 183 seq_num 615) 2024-09-18T14:08:24.253 INFO:tasks.rados.rados.0.smithi084.stdout:2011: rollback oid 21 current snap is 207 2024-09-18T14:08:24.253 INFO:tasks.rados.rados.0.smithi084.stdout:2011: done (1 left) 2024-09-18T14:08:24.253 INFO:tasks.rados.rados.0.smithi084.stdout:2012: delete oid 44 current snap is 207 2024-09-18T14:08:24.257 INFO:tasks.rados.rados.0.smithi084.stdout:2012: done (1 left) 2024-09-18T14:08:24.257 INFO:tasks.rados.rados.0.smithi084.stdout:2013: read oid 2 snap -1 2024-09-18T14:08:24.257 INFO:tasks.rados.rados.0.smithi084.stdout:2013: expect (ObjNum 642 snap 189 seq_num 642) 2024-09-18T14:08:24.257 INFO:tasks.rados.rados.0.smithi084.stdout:2014: delete oid 23 current snap is 207 2024-09-18T14:08:24.260 INFO:tasks.rados.rados.0.smithi084.stdout:2010: expect (ObjNum 615 snap 183 seq_num 615) 2024-09-18T14:08:24.389 INFO:tasks.rados.rados.0.smithi084.stdout:2013: expect (ObjNum 642 snap 189 seq_num 642) 2024-09-18T14:08:24.482 INFO:tasks.rados.rados.0.smithi084.stdout:2010: done (2 left) 2024-09-18T14:08:24.482 INFO:tasks.rados.rados.0.smithi084.stdout:2013: done (1 left) 2024-09-18T14:08:24.482 INFO:tasks.rados.rados.0.smithi084.stdout:2014: done (0 left) 2024-09-18T14:08:24.482 INFO:tasks.rados.rados.0.smithi084.stdout:2015: rollback oid 11 current snap is 207 2024-09-18T14:08:24.482 INFO:tasks.rados.rados.0.smithi084.stdout:2015: done (0 left) 2024-09-18T14:08:24.482 INFO:tasks.rados.rados.0.smithi084.stdout:2016: write oid 50 current snap is 207 2024-09-18T14:08:24.483 INFO:tasks.rados.rados.0.smithi084.stdout:2016: seq_num 712 ranges {766584=798117,2098238=615574,3169759=460379} 2024-09-18T14:08:24.490 INFO:tasks.rados.rados.0.smithi084.stdout:2016: writing smithi08434663-50 from 766584 to 1564701 tid 1 2024-09-18T14:08:24.496 INFO:tasks.rados.rados.0.smithi084.stdout:2016: writing smithi08434663-50 from 2098238 to 2713812 tid 2 2024-09-18T14:08:24.505 INFO:tasks.rados.rados.0.smithi084.stdout:2016: writing smithi08434663-50 from 3169759 to 3630138 tid 3 2024-09-18T14:08:24.505 INFO:tasks.rados.rados.0.smithi084.stdout:2017: write (excl) oid 31 current snap is 207 2024-09-18T14:08:24.505 INFO:tasks.rados.rados.0.smithi084.stdout:2016: finishing write tid 1 to smithi08434663-50 2024-09-18T14:08:24.505 INFO:tasks.rados.rados.0.smithi084.stdout:2016: finishing write tid 2 to smithi08434663-50 2024-09-18T14:08:24.505 INFO:tasks.rados.rados.0.smithi084.stdout:2017: seq_num 713 ranges {783391=473366,1756720=475585,2797182=788004} 2024-09-18T14:08:24.508 INFO:tasks.rados.rados.0.smithi084.stdout:2017: writing smithi08434663-31 from 783391 to 1256757 tid 1 2024-09-18T14:08:24.513 INFO:tasks.rados.rados.0.smithi084.stdout:2017: writing smithi08434663-31 from 1756720 to 2232305 tid 2 2024-09-18T14:08:24.521 INFO:tasks.rados.rados.0.smithi084.stdout:2017: writing smithi08434663-31 from 2797182 to 3585186 tid 3 2024-09-18T14:08:24.522 INFO:tasks.rados.rados.0.smithi084.stdout:2018: read oid 49 snap -1 2024-09-18T14:08:24.522 INFO:tasks.rados.rados.0.smithi084.stdout:2018: expect (ObjNum 672 snap 200 seq_num 672) 2024-09-18T14:08:24.522 INFO:tasks.rados.rados.0.smithi084.stdout:2019: copy_from oid 7 from oid 13 current snap is 207 2024-09-18T14:08:24.522 INFO:tasks.rados.rados.0.smithi084.stdout:2016: finishing write tid 3 to smithi08434663-50 2024-09-18T14:08:24.522 INFO:tasks.rados.rados.0.smithi084.stdout:2016: finishing write tid 4 to smithi08434663-50 2024-09-18T14:08:24.522 INFO:tasks.rados.rados.0.smithi084.stdout:2016: finishing write tid 5 to smithi08434663-50 2024-09-18T14:08:24.522 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 1036 (ObjNum 712 snap 207 seq_num 712) dirty exists 2024-09-18T14:08:24.522 INFO:tasks.rados.rados.0.smithi084.stdout:2016: left oid 50 (ObjNum 712 snap 207 seq_num 712) 2024-09-18T14:08:24.522 INFO:tasks.rados.rados.0.smithi084.stdout:2017: finishing write tid 1 to smithi08434663-31 2024-09-18T14:08:24.522 INFO:tasks.rados.rados.0.smithi084.stdout:2017: finishing write tid 2 to smithi08434663-31 2024-09-18T14:08:24.522 INFO:tasks.rados.rados.0.smithi084.stdout:2016: done (3 left) 2024-09-18T14:08:24.522 INFO:tasks.rados.rados.0.smithi084.stdout:2020: copy_from oid 39 from oid 1 current snap is 207 2024-09-18T14:08:24.523 INFO:tasks.rados.rados.0.smithi084.stdout:2021: read oid 16 snap -1 2024-09-18T14:08:24.523 INFO:tasks.rados.rados.0.smithi084.stdout:2021: expect (ObjNum 706 snap 205 seq_num 706) 2024-09-18T14:08:24.523 INFO:tasks.rados.rados.0.smithi084.stdout:2022: copy_from oid 21 from oid 18 current snap is 207 2024-09-18T14:08:24.523 INFO:tasks.rados.rados.0.smithi084.stdout:2023: write (excl) oid 42 current snap is 207 2024-09-18T14:08:24.523 INFO:tasks.rados.rados.0.smithi084.stdout:2023: seq_num 714 ranges {400198=548615,1415202=584996} 2024-09-18T14:08:24.526 INFO:tasks.rados.rados.0.smithi084.stdout:2023: writing smithi08434663-42 from 400198 to 948813 tid 1 2024-09-18T14:08:24.533 INFO:tasks.rados.rados.0.smithi084.stdout:2023: writing smithi08434663-42 from 1415202 to 2000198 tid 2 2024-09-18T14:08:24.533 INFO:tasks.rados.rados.0.smithi084.stdout:2024: write oid 24 current snap is 207 2024-09-18T14:08:24.533 INFO:tasks.rados.rados.0.smithi084.stdout:2024: seq_num 715 ranges {417005=740217,1590037=426968} 2024-09-18T14:08:24.539 INFO:tasks.rados.rados.0.smithi084.stdout:2024: writing smithi08434663-24 from 417005 to 1157222 tid 1 2024-09-18T14:08:24.544 INFO:tasks.rados.rados.0.smithi084.stdout:2024: writing smithi08434663-24 from 1590037 to 2017005 tid 2 2024-09-18T14:08:24.544 INFO:tasks.rados.rados.0.smithi084.stdout:2025: copy_from oid 19 from oid 15 current snap is 207 2024-09-18T14:08:24.544 INFO:tasks.rados.rados.0.smithi084.stdout:2021: expect (ObjNum 706 snap 205 seq_num 706) 2024-09-18T14:08:24.668 INFO:tasks.rados.rados.0.smithi084.stdout:2017: finishing write tid 3 to smithi08434663-31 2024-09-18T14:08:24.668 INFO:tasks.rados.rados.0.smithi084.stdout:2017: finishing write tid 4 to smithi08434663-31 2024-09-18T14:08:24.668 INFO:tasks.rados.rados.0.smithi084.stdout:2023: finishing write tid 1 to smithi08434663-42 2024-09-18T14:08:24.668 INFO:tasks.rados.rados.0.smithi084.stdout:2022: finishing copy_from racing read to smithi08434663-21 2024-09-18T14:08:24.668 INFO:tasks.rados.rados.0.smithi084.stdout:2018: expect (ObjNum 672 snap 200 seq_num 672) 2024-09-18T14:08:24.808 INFO:tasks.rados.rados.0.smithi084.stdout:2017: finishing write tid 5 to smithi08434663-31 2024-09-18T14:08:24.808 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 570 (ObjNum 713 snap 207 seq_num 713) dirty exists 2024-09-18T14:08:24.808 INFO:tasks.rados.rados.0.smithi084.stdout:2017: left oid 31 (ObjNum 713 snap 207 seq_num 713) 2024-09-18T14:08:24.808 INFO:tasks.rados.rados.0.smithi084.stdout:2019: finishing copy_from racing read to smithi08434663-7 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:2020: finishing copy_from racing read to smithi08434663-39 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:2019: finishing copy_from to smithi08434663-7 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 739 (ObjNum 615 snap 183 seq_num 615) dirty exists 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:2023: finishing write tid 2 to smithi08434663-42 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:2023: finishing write tid 3 to smithi08434663-42 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:2020: finishing copy_from to smithi08434663-39 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 575 (ObjNum 538 snap 167 seq_num 538) dirty exists 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:2023: finishing write tid 4 to smithi08434663-42 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 924 (ObjNum 714 snap 207 seq_num 714) dirty exists 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:2023: left oid 42 (ObjNum 714 snap 207 seq_num 714) 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:2022: finishing copy_from to smithi08434663-21 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 637 (ObjNum 397 snap 121 seq_num 397) dirty exists 2024-09-18T14:08:24.809 INFO:tasks.rados.rados.0.smithi084.stdout:2024: finishing write tid 1 to smithi08434663-24 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2017: done (8 left) 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2018: done (7 left) 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2019: done (6 left) 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2020: done (5 left) 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2021: done (4 left) 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2022: done (3 left) 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2023: done (2 left) 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2026: read oid 45 snap -1 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2026: expect deleted 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2027: snap_create 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2024: finishing write tid 2 to smithi08434663-24 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2024: finishing write tid 3 to smithi08434663-24 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:2024: finishing write tid 4 to smithi08434663-24 2024-09-18T14:08:24.810 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 573 (ObjNum 715 snap 207 seq_num 715) dirty exists 2024-09-18T14:08:24.811 INFO:tasks.rados.rados.0.smithi084.stdout:2024: left oid 24 (ObjNum 715 snap 207 seq_num 715) 2024-09-18T14:08:24.811 INFO:tasks.rados.rados.0.smithi084.stdout:2025: finishing copy_from racing read to smithi08434663-19 2024-09-18T14:08:24.811 INFO:tasks.rados.rados.0.smithi084.stdout:2025: finishing copy_from to smithi08434663-19 2024-09-18T14:08:24.811 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 856 (ObjNum 699 snap 205 seq_num 699) dirty exists 2024-09-18T14:08:24.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:24.861+0000 7ff18ffff640 1 -- >> v1:172.21.15.45:6792/0 conn(0x7ff19809fba0 legacy=0x7ff198137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6792/0 2024-09-18T14:08:24.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:24.861+0000 7ff196ffd640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7ff19809f7d0 legacy=0x7ff198132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:08:24.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:24.861+0000 7ff1977fe640 1 -- >> v1:172.21.15.84:6791/0 conn(0x7ff19809ff70 legacy=0x7ff19813b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6791/0 2024-09-18T14:08:25.253 INFO:tasks.rados.rados.0.smithi084.stdout:2024: done (3 left) 2024-09-18T14:08:25.253 INFO:tasks.rados.rados.0.smithi084.stdout:2025: done (2 left) 2024-09-18T14:08:25.253 INFO:tasks.rados.rados.0.smithi084.stdout:2026: done (1 left) 2024-09-18T14:08:25.253 INFO:tasks.rados.rados.0.smithi084.stdout:2027: done (0 left) 2024-09-18T14:08:25.253 INFO:tasks.rados.rados.0.smithi084.stdout:2028: delete oid 50 current snap is 208 2024-09-18T14:08:25.261 INFO:tasks.rados.rados.0.smithi084.stdout:2028: done (0 left) 2024-09-18T14:08:25.261 INFO:tasks.rados.rados.0.smithi084.stdout:2029: read oid 49 snap -1 2024-09-18T14:08:25.261 INFO:tasks.rados.rados.0.smithi084.stdout:2029: expect (ObjNum 672 snap 200 seq_num 672) 2024-09-18T14:08:25.261 INFO:tasks.rados.rados.0.smithi084.stdout:2030: copy_from oid 6 from oid 41 current snap is 208 2024-09-18T14:08:25.261 INFO:tasks.rados.rados.0.smithi084.stdout:2031: write oid 2 current snap is 208 2024-09-18T14:08:25.261 INFO:tasks.rados.rados.0.smithi084.stdout:2031: seq_num 717 ranges {450619=490715,1590648=459971} 2024-09-18T14:08:25.264 INFO:tasks.rados.rados.0.smithi084.stdout:2031: writing smithi08434663-2 from 450619 to 941334 tid 1 2024-09-18T14:08:25.270 INFO:tasks.rados.rados.0.smithi084.stdout:2031: writing smithi08434663-2 from 1590648 to 2050619 tid 2 2024-09-18T14:08:25.270 INFO:tasks.rados.rados.0.smithi084.stdout:2032: copy_from oid 3 from oid 1 current snap is 208 2024-09-18T14:08:25.270 INFO:tasks.rados.rados.0.smithi084.stdout:2029: expect (ObjNum 672 snap 200 seq_num 672) 2024-09-18T14:08:25.413 INFO:tasks.rados.rados.0.smithi084.stdout:2031: finishing write tid 1 to smithi08434663-2 2024-09-18T14:08:25.413 INFO:tasks.rados.rados.0.smithi084.stdout:2029: done (3 left) 2024-09-18T14:08:25.413 INFO:tasks.rados.rados.0.smithi084.stdout:2033: write (excl) oid 34 current snap is 208 2024-09-18T14:08:25.414 INFO:tasks.rados.rados.0.smithi084.stdout:2031: finishing write tid 2 to smithi08434663-2 2024-09-18T14:08:25.414 INFO:tasks.rados.rados.0.smithi084.stdout:2031: finishing write tid 3 to smithi08434663-2 2024-09-18T14:08:25.414 INFO:tasks.rados.rados.0.smithi084.stdout:2031: finishing write tid 4 to smithi08434663-2 2024-09-18T14:08:25.414 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 1043 (ObjNum 717 snap 208 seq_num 717) dirty exists 2024-09-18T14:08:25.414 INFO:tasks.rados.rados.0.smithi084.stdout:2031: left oid 2 (ObjNum 717 snap 208 seq_num 717) 2024-09-18T14:08:25.414 INFO:tasks.rados.rados.0.smithi084.stdout:2030: finishing copy_from racing read to smithi08434663-6 2024-09-18T14:08:25.414 INFO:tasks.rados.rados.0.smithi084.stdout:2030: finishing copy_from to smithi08434663-6 2024-09-18T14:08:25.414 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 917 (ObjNum 708 snap 207 seq_num 708) dirty exists 2024-09-18T14:08:25.414 INFO:tasks.rados.rados.0.smithi084.stdout:2032: finishing copy_from racing read to smithi08434663-3 2024-09-18T14:08:25.414 INFO:tasks.rados.rados.0.smithi084.stdout:2032: finishing copy_from to smithi08434663-3 2024-09-18T14:08:25.414 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 575 (ObjNum 538 snap 167 seq_num 538) dirty exists 2024-09-18T14:08:25.414 INFO:tasks.rados.rados.0.smithi084.stdout:2033: seq_num 718 ranges {467426=565964,1649130=418296} 2024-09-18T14:08:25.418 INFO:tasks.rados.rados.0.smithi084.stdout:2033: writing smithi08434663-34 from 467426 to 1033390 tid 1 2024-09-18T14:08:25.424 INFO:tasks.rados.rados.0.smithi084.stdout:2033: writing smithi08434663-34 from 1649130 to 2067426 tid 2 2024-09-18T14:08:25.424 INFO:tasks.rados.rados.0.smithi084.stdout:2030: done (3 left) 2024-09-18T14:08:25.424 INFO:tasks.rados.rados.0.smithi084.stdout:2031: done (2 left) 2024-09-18T14:08:25.424 INFO:tasks.rados.rados.0.smithi084.stdout:2032: done (1 left) 2024-09-18T14:08:25.424 INFO:tasks.rados.rados.0.smithi084.stdout:2034: delete oid 24 current snap is 208 2024-09-18T14:08:25.424 INFO:tasks.rados.rados.0.smithi084.stdout:2033: finishing write tid 1 to smithi08434663-34 2024-09-18T14:08:25.428 INFO:tasks.rados.rados.0.smithi084.stdout:2034: done (1 left) 2024-09-18T14:08:25.428 INFO:tasks.rados.rados.0.smithi084.stdout:2035: delete oid 35 current snap is 208 2024-09-18T14:08:25.429 INFO:tasks.rados.rados.0.smithi084.stdout:2033: finishing write tid 2 to smithi08434663-34 2024-09-18T14:08:25.429 INFO:tasks.rados.rados.0.smithi084.stdout:2033: finishing write tid 3 to smithi08434663-34 2024-09-18T14:08:25.429 INFO:tasks.rados.rados.0.smithi084.stdout:2033: finishing write tid 4 to smithi08434663-34 2024-09-18T14:08:25.429 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 928 (ObjNum 718 snap 208 seq_num 718) dirty exists 2024-09-18T14:08:25.429 INFO:tasks.rados.rados.0.smithi084.stdout:2033: left oid 34 (ObjNum 718 snap 208 seq_num 718) 2024-09-18T14:08:25.431 INFO:tasks.rados.rados.0.smithi084.stdout:2033: done (1 left) 2024-09-18T14:08:25.431 INFO:tasks.rados.rados.0.smithi084.stdout:2035: done (0 left) 2024-09-18T14:08:25.431 INFO:tasks.rados.rados.0.smithi084.stdout:2036: read oid 42 snap -1 2024-09-18T14:08:25.431 INFO:tasks.rados.rados.0.smithi084.stdout:2036: expect (ObjNum 714 snap 207 seq_num 714) 2024-09-18T14:08:25.431 INFO:tasks.rados.rados.0.smithi084.stdout:2037: copy_from oid 5 from oid 34 current snap is 208 2024-09-18T14:08:25.431 INFO:tasks.rados.rados.0.smithi084.stdout:2038: delete oid 36 current snap is 208 2024-09-18T14:08:25.435 INFO:tasks.rados.rados.0.smithi084.stdout:2038: done (2 left) 2024-09-18T14:08:25.435 INFO:tasks.rados.rados.0.smithi084.stdout:2039: snap_remove snap 207 2024-09-18T14:08:26.265 INFO:tasks.rados.rados.0.smithi084.stdout:2039: done (2 left) 2024-09-18T14:08:26.265 INFO:tasks.rados.rados.0.smithi084.stdout:2040: write oid 6 current snap is 208 2024-09-18T14:08:26.265 INFO:tasks.rados.rados.0.smithi084.stdout:2040: seq_num 722 ranges {534654=466960,1766705=367949} 2024-09-18T14:08:26.270 INFO:tasks.rados.rados.0.smithi084.stdout:2040: writing smithi08434663-6 from 534654 to 1001614 tid 1 2024-09-18T14:08:26.276 INFO:tasks.rados.rados.0.smithi084.stdout:2040: writing smithi08434663-6 from 1766705 to 2134654 tid 2 2024-09-18T14:08:26.276 INFO:tasks.rados.rados.0.smithi084.stdout:2041: snap_create 2024-09-18T14:08:26.277 INFO:tasks.rados.rados.0.smithi084.stdout:2036: expect (ObjNum 714 snap 207 seq_num 714) 2024-09-18T14:08:26.354 INFO:tasks.rados.rados.0.smithi084.stdout:2037: finishing copy_from racing read to smithi08434663-5 2024-09-18T14:08:26.354 INFO:tasks.rados.rados.0.smithi084.stdout:2037: finishing copy_from to smithi08434663-5 2024-09-18T14:08:26.354 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 928 (ObjNum 718 snap 208 seq_num 718) dirty exists 2024-09-18T14:08:26.354 INFO:tasks.rados.rados.0.smithi084.stdout:2040: finishing write tid 1 to smithi08434663-6 2024-09-18T14:08:26.354 INFO:tasks.rados.rados.0.smithi084.stdout:2040: finishing write tid 2 to smithi08434663-6 2024-09-18T14:08:26.354 INFO:tasks.rados.rados.0.smithi084.stdout:2040: finishing write tid 3 to smithi08434663-6 2024-09-18T14:08:26.354 INFO:tasks.rados.rados.0.smithi084.stdout:2040: finishing write tid 4 to smithi08434663-6 2024-09-18T14:08:26.354 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 920 (ObjNum 722 snap 208 seq_num 722) dirty exists 2024-09-18T14:08:26.355 INFO:tasks.rados.rados.0.smithi084.stdout:2040: left oid 6 (ObjNum 722 snap 208 seq_num 722) 2024-09-18T14:08:26.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.458+0000 7ff1957fa640 10 monclient(hunting): tick 2024-09-18T14:08:26.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.458+0000 7ff1957fa640 1 monclient(hunting): continuing hunt 2024-09-18T14:08:26.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.458+0000 7ff1957fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:08:26.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.458+0000 7ff1957fa640 1 -- >> v1:172.21.15.45:6792/0 conn(0x7ff19809fba0 legacy=0x7ff198137a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:08:26.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.458+0000 7ff1957fa640 1 -- >> v1:172.21.15.84:6791/0 conn(0x7ff19809ff70 legacy=0x7ff19813b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:08:26.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.458+0000 7ff1957fa640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7ff19809f7d0 legacy=0x7ff198132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:08:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.458+0000 7ff1957fa640 10 monclient: _add_conns ranks=[7,4,0,1,8,2] 2024-09-18T14:08:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1957fa640 10 monclient(hunting): picked mon.noname-i con 0x7ff178003740 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1957fa640 10 monclient(hunting): picked mon.noname-c con 0x7ff178007320 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1957fa640 10 monclient(hunting): picked mon.noname-a con 0x7ff17800b0a0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1957fa640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1980a1c00 con 0x7ff17800b0a0 2024-09-18T14:08:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1957fa640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19805ed70 con 0x7ff178007320 2024-09-18T14:08:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1957fa640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19809dfc0 con 0x7ff178003740 2024-09-18T14:08:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1957fa640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:26.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff196ffd640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7ff178003740 legacy=0x7ff178005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:26.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff18ffff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7ff178007320 0x7ff1780097d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58800/0 (socket says 172.21.15.45:58800) 2024-09-18T14:08:26.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff18ffff640 1 -- 172.21.15.45:0/3686100950 learned_addr learned my addr 172.21.15.45:0/3686100950 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:26.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 1 -- 172.21.15.45:0/3686100950 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330053400 0 0) 0x7ff19805ed70 con 0x7ff178007320 2024-09-18T14:08:26.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 10 monclient(hunting): my global_id is 164299 2024-09-18T14:08:26.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 1 -- 172.21.15.45:0/3686100950 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1840034e0 con 0x7ff178007320 2024-09-18T14:08:26.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 1 -- 172.21.15.45:0/3686100950 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221617399 0 0) 0x7ff1980a1c00 con 0x7ff17800b0a0 2024-09-18T14:08:26.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 10 monclient(hunting): my global_id is 5517 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 1 -- 172.21.15.45:0/3686100950 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff19805ed70 con 0x7ff17800b0a0 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 1 -- 172.21.15.45:0/3686100950 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4220440333 0 0) 0x7ff1840034e0 con 0x7ff178007320 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 1 -- 172.21.15.45:0/3686100950 >> v1:172.21.15.84:6792/0 conn(0x7ff178003740 legacy=0x7ff178005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 1 -- 172.21.15.45:0/3686100950 >> v1:172.21.15.45:6789/0 conn(0x7ff17800b0a0 legacy=0x7ff17800d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 10 monclient: _finish_hunting 0 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 1 monclient: found mon.noname-c 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1977fe640 1 -- 172.21.15.45:0/3686100950 reap_dead start 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 1 -- 172.21.15.45:0/3686100950 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19809dce0 con 0x7ff178007320 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 10 monclient: _finish_auth 0 2024-09-18T14:08:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 1 -- 172.21.15.45:0/3686100950 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff180004380 con 0x7ff178007320 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff1967fc640 10 monclient: dump: 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff19d80f640 5 monclient: authenticate success, global_id 164299 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.459+0000 7ff19d80f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff1967fc640 1 -- 172.21.15.45:0/3686100950 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff1800044e0 con 0x7ff178007320 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff1967fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff1967fc640 1 -- 172.21.15.45:0/3686100950 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff180004a50 con 0x7ff178007320 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff1967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff1967fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff1967fc640 10 monclient: dump: 2024-09-18T14:08:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff19d80f640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff19d80f640 4 set_mon_vals no callback set 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff19d80f640 10 monclient: shutdown 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff19d80f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff19d80f640 1 -- 172.21.15.45:0/3686100950 >> v1:172.21.15.45:6791/0 conn(0x7ff178007320 legacy=0x7ff1780097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff19d80f640 1 -- 172.21.15.45:0/3686100950 shutdown_connections 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff19d80f640 1 -- 172.21.15.45:0/3686100950 >> 172.21.15.45:0/3686100950 conn(0x7ff1980630b0 msgr2=0x7ff1980634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.460+0000 7ff19d80f640 1 -- 172.21.15.45:0/3686100950 shutdown_connections 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.461+0000 7ff19d80f640 1 -- 172.21.15.45:0/3686100950 wait complete. 2024-09-18T14:08:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.461+0000 7ff19d80f640 10 monclient: build_initial_monmap 2024-09-18T14:08:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.461+0000 7ff19d80f640 10 monclient: monmap: 2024-09-18T14:08:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:26.480 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.461+0000 7ff19d80f640 1 Processor -- start 2024-09-18T14:08:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.461+0000 7ff19d80f640 1 -- start start 2024-09-18T14:08:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff19d80f640 10 monclient: init 2024-09-18T14:08:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff19d80f640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff19d80f640 10 monclient: _add_conns ranks=[7,4,2,6,5,1,3,8,0] 2024-09-18T14:08:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff19d80f640 10 monclient(hunting): picked mon.noname-i con 0x7ff178007320 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff19d80f640 10 monclient(hunting): picked mon.noname-c con 0x7ff198136e90 addr v1:172.21.15.45:6791/0 2024-09-18T14:08:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff19d80f640 10 monclient(hunting): picked mon.noname-b con 0x7ff19809b110 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff19d80f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff198142960 con 0x7ff19809b110 2024-09-18T14:08:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff19d80f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1980a2560 con 0x7ff198136e90 2024-09-18T14:08:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff19d80f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19811eba0 con 0x7ff178007320 2024-09-18T14:08:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff19d80f640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff19d80f640 10 monclient(hunting): authenticate will time out at 1321.800532s 2024-09-18T14:08:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff196ffd640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7ff178007320 legacy=0x7ff198131780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff1977fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7ff19809b110 0x7ff19814a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55144/0 (socket says 172.21.15.45:55144) 2024-09-18T14:08:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff1977fe640 1 -- 172.21.15.45:0/3653625815 learned_addr learned my addr 172.21.15.45:0/3653625815 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639329529 0 0) 0x7ff19805ed70 con 0x7ff19809b110 2024-09-18T14:08:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff18effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff18effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff18effd640 10 monclient(hunting): my global_id is 144767 2024-09-18T14:08:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff178008f70 con 0x7ff19809b110 2024-09-18T14:08:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893875125 0 0) 0x7ff1980a2560 con 0x7ff198136e90 2024-09-18T14:08:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff18effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff18effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff18effd640 10 monclient(hunting): my global_id is 164308 2024-09-18T14:08:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.462+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff19805ed70 con 0x7ff198136e90 2024-09-18T14:08:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3764073190 0 0) 0x7ff19805ed70 con 0x7ff198136e90 2024-09-18T14:08:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1980a2560 con 0x7ff198136e90 2024-09-18T14:08:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff180004990 con 0x7ff198136e90 2024-09-18T14:08:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:08:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient(hunting): dump: 2024-09-18T14:08:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.485 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 915296856 0 0) 0x7ff178008f70 con 0x7ff19809b110 2024-09-18T14:08:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff19805ed70 con 0x7ff19809b110 2024-09-18T14:08:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff190004570 con 0x7ff19809b110 2024-09-18T14:08:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient(hunting): dump: 2024-09-18T14:08:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.487 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1516045038 0 0) 0x7ff1980a2560 con 0x7ff198136e90 2024-09-18T14:08:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 >> v1:172.21.15.84:6792/0 conn(0x7ff178007320 legacy=0x7ff198131780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:08:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 >> v1:172.21.15.45:6790/0 conn(0x7ff19809b110 legacy=0x7ff19814a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient: _finish_hunting 0 2024-09-18T14:08:26.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 monclient: found mon.c 2024-09-18T14:08:26.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:26.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:26.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19809b530 con 0x7ff198136e90 2024-09-18T14:08:26.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient: _finish_auth 0 2024-09-18T14:08:26.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:26.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff19d80f640 5 monclient: authenticate success, global_id 164308 2024-09-18T14:08:26.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff19d80f640 10 monclient: _renew_subs 2024-09-18T14:08:26.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff19d80f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff19d80f640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19811e8c0 con 0x7ff198136e90 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff19d80f640 10 monclient: _renew_subs 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff19d80f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff19d80f640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff198131e90 con 0x7ff198136e90 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff180004370 con 0x7ff198136e90 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff180005160 con 0x7ff198136e90 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff18effd640 10 monclient: dump: 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.463+0000 7ff195ffb640 4 set_mon_vals no callback set 2024-09-18T14:08:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.464+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff1800101b0 con 0x7ff198136e90 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.464+0000 7ff19d80f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.464+0000 7ff19d80f640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff19804c4a0 con 0x7ff198096a20 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(437..437 src has 126..437) v4 ==== 3971+0+0 (unknown 923945606 0 0) 0x7ff180046bf0 con 0x7ff198136e90 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258691254 0 0) 0x7ff19804c4a0 con 0x7ff198096a20 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 10 monclient(hunting): my global_id is 164317 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1980a2560 con 0x7ff198096a20 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff1977fe640 10 monclient: get_auth_request con 0x7ff17803b300 auth_method 0 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 755405616 0 0) 0x7ff1980a2560 con 0x7ff198096a20 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff19804c4a0 con 0x7ff198096a20 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff188004140 con 0x7ff198096a20 2024-09-18T14:08:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 10 monclient: dump: 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.493 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3824970499 0 0) 0x7ff19804c4a0 con 0x7ff198096a20 2024-09-18T14:08:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.465+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff19811db70 con 0x7ff198096a20 2024-09-18T14:08:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.466+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff1880063c0 con 0x7ff198096a20 2024-09-18T14:08:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.466+0000 7ff18effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.466+0000 7ff18effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.466+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 >> v1:172.21.15.45:6791/0 conn(0x7ff198096a20 legacy=0x7ff198096df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.466+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 >> v1:172.21.15.45:6791/0 conn(0x7ff198096a20 legacy=0x7ff198096df0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff19d80f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff19d80f640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff19809bc30 con 0x7ff198097500 2024-09-18T14:08:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358600582 0 0) 0x7ff188006630 con 0x7ff198097500 2024-09-18T14:08:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff18effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff18effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff18effd640 10 monclient(hunting): my global_id is 164326 2024-09-18T14:08:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff19804c4a0 con 0x7ff198097500 2024-09-18T14:08:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 613318978 0 0) 0x7ff19804c4a0 con 0x7ff198097500 2024-09-18T14:08:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff188006630 con 0x7ff198097500 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff188004bc0 con 0x7ff198097500 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff18effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff18effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.472+0000 7ff18effd640 10 monclient: dump: 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.473+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 926883156 0 0) 0x7ff188006630 con 0x7ff198097500 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.473+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff19809d380 con 0x7ff198097500 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.473+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7342 (unknown 0 0 2352400472) 0x7ff188006010 con 0x7ff198097500 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.473+0000 7ff18effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.473+0000 7ff18effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.473+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 >> v1:172.21.15.45:6791/0 conn(0x7ff198097500 legacy=0x7ff19814fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.473+0000 7ff18effd640 1 -- 172.21.15.45:0/3653625815 >> v1:172.21.15.45:6791/0 conn(0x7ff198097500 legacy=0x7ff19814fc20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:26.498 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 88, 2024-09-18T14:08:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:08:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:08:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:26.519 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:26.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.474+0000 7ff19d80f640 1 -- 172.21.15.45:0/3653625815 >> v1:172.21.15.84:6812/31478 conn(0x7ff17803b300 legacy=0x7ff17803d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.474+0000 7ff19d80f640 10 monclient: shutdown 2024-09-18T14:08:26.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.474+0000 7ff19d80f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:26.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.474+0000 7ff19d80f640 1 -- 172.21.15.45:0/3653625815 >> v1:172.21.15.45:6791/0 conn(0x7ff198136e90 legacy=0x7ff198146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.474+0000 7ff1977fe640 1 -- 172.21.15.45:0/3653625815 reap_dead start 2024-09-18T14:08:26.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.474+0000 7ff1977fe640 1 -- 172.21.15.45:0/3653625815 reap_dead start 2024-09-18T14:08:26.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.474+0000 7ff19d80f640 1 -- 172.21.15.45:0/3653625815 shutdown_connections 2024-09-18T14:08:26.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.474+0000 7ff19d80f640 1 -- 172.21.15.45:0/3653625815 >> 172.21.15.45:0/3653625815 conn(0x7ff1980630b0 msgr2=0x7ff1980a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:26.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.474+0000 7ff19d80f640 1 -- 172.21.15.45:0/3653625815 shutdown_connections 2024-09-18T14:08:26.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.474+0000 7ff19d80f640 1 -- 172.21.15.45:0/3653625815 wait complete. 2024-09-18T14:08:26.521 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-18T14:08:26.521 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon e 2024-09-18T14:08:26.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:08:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.625+0000 7f7a618e8640 10 monclient: get_monmap_and_config 2024-09-18T14:08:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.625+0000 7f7a618e8640 10 monclient: build_initial_monmap 2024-09-18T14:08:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.625+0000 7f7a618e8640 10 monclient: monmap: 2024-09-18T14:08:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:26.626368+0000 2024-09-18T14:08:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:26.626368+0000 2024-09-18T14:08:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:26.627 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.625+0000 7f7a618e8640 1 Processor -- start 2024-09-18T14:08:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a618e8640 1 -- start start 2024-09-18T14:08:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a618e8640 10 monclient: init 2024-09-18T14:08:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a618e8640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a618e8640 10 monclient: _add_conns ranks=[2,7,0,3,5,4,1,6,8] 2024-09-18T14:08:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a618e8640 10 monclient(hunting): picked mon.noname-b con 0x7f7a5c0a2610 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a618e8640 10 monclient(hunting): picked mon.noname-i con 0x7f7a5c1330b0 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a618e8640 10 monclient(hunting): picked mon.noname-a con 0x7f7a5c096a20 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a618e8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c05ed70 con 0x7f7a5c096a20 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a618e8640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c0a1740 con 0x7f7a5c0a2610 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a618e8640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c0a29e0 con 0x7f7a5c1330b0 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a618e8640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a618e8640 10 monclient(hunting): authenticate will time out at 1321.964716s 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a5a7fc640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7f7a5c1330b0 legacy=0x7f7a5c137810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a5affd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f7a5c0a2610 0x7f7a5c1329a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55150/0 (socket says 172.21.15.45:55150) 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.626+0000 7f7a5affd640 1 -- 172.21.15.45:0/2587253432 learned_addr learned my addr 172.21.15.45:0/2587253432 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 1 -- 172.21.15.45:0/2587253432 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372635878 0 0) 0x7f7a5c0a1740 con 0x7f7a5c0a2610 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient(hunting): my global_id is 144776 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 1 -- 172.21.15.45:0/2587253432 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a440034e0 con 0x7f7a5c0a2610 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 1 -- 172.21.15.45:0/2587253432 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980991345 0 0) 0x7f7a5c05ed70 con 0x7f7a5c096a20 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient(hunting): my global_id is 5526 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 1 -- 172.21.15.45:0/2587253432 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a5c0a1740 con 0x7f7a5c096a20 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 1 -- 172.21.15.45:0/2587253432 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 102393278 0 0) 0x7f7a440034e0 con 0x7f7a5c0a2610 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 1 -- 172.21.15.45:0/2587253432 >> v1:172.21.15.84:6792/0 conn(0x7f7a5c1330b0 legacy=0x7f7a5c137810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 1 -- 172.21.15.45:0/2587253432 >> v1:172.21.15.45:6789/0 conn(0x7f7a5c096a20 legacy=0x7f7a5c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 1 monclient: found mon.noname-b 2024-09-18T14:08:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 1 -- 172.21.15.45:0/2587253432 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c0970a0 con 0x7f7a5c0a2610 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient: _finish_auth 0 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a618e8640 5 monclient: authenticate success, global_id 144776 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 1 -- 172.21.15.45:0/2587253432 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a4c001830 con 0x7f7a5c0a2610 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a618e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient: dump: 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a618e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 1 -- 172.21.15.45:0/2587253432 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7a4c001a10 con 0x7f7a5c0a2610 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 1 -- 172.21.15.45:0/2587253432 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a4c004ba0 con 0x7f7a5c0a2610 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a618e8640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a618e8640 4 set_mon_vals no callback set 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a59ffb640 10 monclient: dump: 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a618e8640 10 monclient: shutdown 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a618e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a618e8640 1 -- 172.21.15.45:0/2587253432 >> v1:172.21.15.45:6790/0 conn(0x7f7a5c0a2610 legacy=0x7f7a5c1329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a618e8640 1 -- 172.21.15.45:0/2587253432 shutdown_connections 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.627+0000 7f7a618e8640 1 -- 172.21.15.45:0/2587253432 >> 172.21.15.45:0/2587253432 conn(0x7f7a5c0630f0 msgr2=0x7f7a5c0634f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.628+0000 7f7a618e8640 1 -- 172.21.15.45:0/2587253432 shutdown_connections 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.628+0000 7f7a618e8640 1 -- 172.21.15.45:0/2587253432 wait complete. 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.628+0000 7f7a618e8640 10 monclient: build_initial_monmap 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.628+0000 7f7a618e8640 10 monclient: monmap: 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.628+0000 7f7a618e8640 1 Processor -- start 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a618e8640 1 -- start start 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a618e8640 10 monclient: init 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a618e8640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a618e8640 10 monclient: _add_conns ranks=[0,2,7,3,4,8,1,6,5] 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a618e8640 10 monclient(hunting): picked mon.noname-a con 0x7f7a5c096a20 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a618e8640 10 monclient(hunting): picked mon.noname-b con 0x7f7a5c0a2610 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a618e8640 10 monclient(hunting): picked mon.noname-i con 0x7f7a5c1330b0 addr v1:172.21.15.84:6792/0 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a618e8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c09e620 con 0x7f7a5c096a20 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a618e8640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c11ce50 con 0x7f7a5c0a2610 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a618e8640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c0a2350 con 0x7f7a5c1330b0 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a618e8640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a618e8640 10 monclient(hunting): authenticate will time out at 1321.967493s 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a5a7fc640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f7a5c0a2610 0x7f7a5c09ccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55160/0 (socket says 172.21.15.45:55160) 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a5affd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7a5c096a20 0x7f7a5c09c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48040/0 (socket says 172.21.15.45:48040) 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a5a7fc640 1 -- 172.21.15.45:0/3383048776 learned_addr learned my addr 172.21.15.45:0/3383048776 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a5b7fe640 1 -- 172.21.15.45:0/3383048776 >> v1:172.21.15.84:6792/0 conn(0x7f7a5c1330b0 legacy=0x7f7a5c09d3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:08:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908738341 0 0) 0x7f7a5c09e620 con 0x7f7a5c096a20 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a42ffd640 10 monclient(hunting): my global_id is 5535 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a480034e0 con 0x7f7a5c096a20 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466017452 0 0) 0x7f7a5c11ce50 con 0x7f7a5c0a2610 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a42ffd640 10 monclient(hunting): my global_id is 144785 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.629+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a5c09e620 con 0x7f7a5c0a2610 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 607819694 0 0) 0x7f7a480034e0 con 0x7f7a5c096a20 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a5c11ce50 con 0x7f7a5c096a20 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a4c004ba0 con 0x7f7a5c096a20 2024-09-18T14:08:26.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient(hunting): dump: 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.647 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3023481575 0 0) 0x7f7a5c09e620 con 0x7f7a5c0a2610 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a480034e0 con 0x7f7a5c0a2610 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a50004380 con 0x7f7a5c0a2610 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient(hunting): dump: 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.648 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.649 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3012843042 0 0) 0x7f7a5c11ce50 con 0x7f7a5c096a20 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 >> v1:172.21.15.84:6792/0 conn(0x7f7a5c1330b0 legacy=0x7f7a5c09d3c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 >> v1:172.21.15.45:6790/0 conn(0x7f7a5c0a2610 legacy=0x7f7a5c09ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 monclient: found mon.a 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c09ec70 con 0x7f7a5c096a20 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient: _finish_auth 0 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a618e8640 5 monclient: authenticate success, global_id 5535 2024-09-18T14:08:26.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a618e8640 10 monclient: _renew_subs 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a618e8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a618e8640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c11d130 con 0x7f7a5c096a20 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a618e8640 10 monclient: _renew_subs 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a618e8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a618e8640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c14a8a0 con 0x7f7a5c096a20 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7a4c001840 con 0x7f7a5c096a20 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a4c001250 con 0x7f7a5c096a20 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a597fa640 4 set_mon_vals no callback set 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.630+0000 7f7a42ffd640 10 monclient: dump: 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.651 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.652 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.652 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.652 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.652 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.652 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.652 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.652 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.652 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.652 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.652 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a618e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a618e8640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7a3c006580 con 0x7f7a3c002830 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7a4c010050 con 0x7f7a5c096a20 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(437..437 src has 126..437) v4 ==== 3971+0+0 (unknown 923945606 0 0) 0x7f7a4c046cc0 con 0x7f7a5c096a20 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994669622 0 0) 0x7f7a3c006580 con 0x7f7a3c002830 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 10 monclient(hunting): my global_id is 5417 2024-09-18T14:08:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a5c11ce50 con 0x7f7a3c002830 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a5b7fe640 10 monclient: get_auth_request con 0x7f7a4803a2e0 auth_method 0 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2614648241 0 0) 0x7f7a5c11ce50 con 0x7f7a3c002830 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a3c006580 con 0x7f7a3c002830 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a50004b90 con 0x7f7a3c002830 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 10 monclient: dump: 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.654 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1068656642 0 0) 0x7f7a3c006580 con 0x7f7a3c002830 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.631+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a3c00a0a0 con 0x7f7a3c002830 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.632+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7a500050c0 con 0x7f7a3c002830 2024-09-18T14:08:26.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.632+0000 7f7a42ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.632+0000 7f7a42ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.632+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 >> v1:172.21.15.45:6793/0 conn(0x7f7a3c002830 legacy=0x7f7a3c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.632+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 >> v1:172.21.15.45:6793/0 conn(0x7f7a3c002830 legacy=0x7f7a3c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:26.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a618e8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a618e8640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7a3c007540 con 0x7f7a3c0068f0 2024-09-18T14:08:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786226591 0 0) 0x7f7a3c007540 con 0x7f7a3c0068f0 2024-09-18T14:08:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 10 monclient(hunting): my global_id is 5426 2024-09-18T14:08:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a3c006580 con 0x7f7a3c0068f0 2024-09-18T14:08:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3995282161 0 0) 0x7f7a3c006580 con 0x7f7a3c0068f0 2024-09-18T14:08:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a3c007540 con 0x7f7a3c0068f0 2024-09-18T14:08:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7a50005630 con 0x7f7a3c0068f0 2024-09-18T14:08:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:08:26.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 10 monclient: dump: 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.660 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2612706627 0 0) 0x7f7a3c007540 con 0x7f7a3c0068f0 2024-09-18T14:08:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.638+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7a3c0087a0 con 0x7f7a3c0068f0 2024-09-18T14:08:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7684 (unknown 0 0 1936776994) 0x7f7a50006ab0 con 0x7f7a3c0068f0 2024-09-18T14:08:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a42ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a42ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 >> v1:172.21.15.45:6793/0 conn(0x7f7a3c0068f0 legacy=0x7f7a3c006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a42ffd640 1 -- 172.21.15.45:0/3383048776 >> v1:172.21.15.45:6793/0 conn(0x7f7a3c0068f0 legacy=0x7f7a3c006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:26.660 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:08:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 88, 2024-09-18T14:08:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 6, 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:26.681 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a618e8640 1 -- 172.21.15.45:0/3383048776 >> v1:172.21.15.84:6812/31478 conn(0x7f7a4803a2e0 legacy=0x7f7a4803c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a618e8640 10 monclient: shutdown 2024-09-18T14:08:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a618e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a618e8640 1 -- 172.21.15.45:0/3383048776 >> v1:172.21.15.45:6789/0 conn(0x7f7a5c096a20 legacy=0x7f7a5c09c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a5b7fe640 1 -- 172.21.15.45:0/3383048776 reap_dead start 2024-09-18T14:08:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a5b7fe640 1 -- 172.21.15.45:0/3383048776 reap_dead start 2024-09-18T14:08:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a618e8640 1 -- 172.21.15.45:0/3383048776 shutdown_connections 2024-09-18T14:08:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a618e8640 1 -- 172.21.15.45:0/3383048776 >> 172.21.15.45:0/3383048776 conn(0x7f7a5c0630f0 msgr2=0x7f7a5c133b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a618e8640 1 -- 172.21.15.45:0/3383048776 shutdown_connections 2024-09-18T14:08:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.639+0000 7f7a618e8640 1 -- 172.21.15.45:0/3383048776 wait complete. 2024-09-18T14:08:26.684 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-18T14:08:26.684 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon f 2024-09-18T14:08:26.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:08:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.781+0000 7fd294d00640 10 monclient: get_monmap_and_config 2024-09-18T14:08:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.781+0000 7fd294d00640 10 monclient: build_initial_monmap 2024-09-18T14:08:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.781+0000 7fd294d00640 10 monclient: monmap: 2024-09-18T14:08:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:08:26.782543+0000 2024-09-18T14:08:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:08:26.782543+0000 2024-09-18T14:08:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:26.783 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:26.783 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:26.783 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:26.783 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:26.783 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:26.783 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:26.783 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:26.783 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 1 Processor -- start 2024-09-18T14:08:26.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 1 -- start start 2024-09-18T14:08:26.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 10 monclient: init 2024-09-18T14:08:26.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:26.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 10 monclient: _add_conns ranks=[8,0,1,4,6,5,2,7,3] 2024-09-18T14:08:26.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 10 monclient(hunting): picked mon.noname-e con 0x7fd29009f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:08:26.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 10 monclient(hunting): picked mon.noname-a con 0x7fd29009fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:26.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 10 monclient(hunting): picked mon.noname-f con 0x7fd29009ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:26.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29005ed70 con 0x7fd29009fba0 2024-09-18T14:08:26.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29009dfc0 con 0x7fd29009ff70 2024-09-18T14:08:26.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29009dce0 con 0x7fd29009f7d0 2024-09-18T14:08:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.782+0000 7fd294d00640 10 monclient(hunting): authenticate will time out at 1322.120961s 2024-09-18T14:08:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28e59c640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fd29009f7d0 0x7fd290132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:48802/0 (socket says 172.21.15.45:48802) 2024-09-18T14:08:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28e59c640 1 -- 172.21.15.45:0/816300334 learned_addr learned my addr 172.21.15.45:0/816300334 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd287fff640 1 --1- 172.21.15.45:0/816300334 >> v1:172.21.15.45:6789/0 conn(0x7fd29009fba0 0x7fd290137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48050/0 (socket says 172.21.15.45:48050) 2024-09-18T14:08:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977240634 0 0) 0x7fd29005ed70 con 0x7fd29009fba0 2024-09-18T14:08:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient(hunting): my global_id is 5544 2024-09-18T14:08:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd27c0034e0 con 0x7fd29009fba0 2024-09-18T14:08:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 109554066 0 0) 0x7fd29009dce0 con 0x7fd29009f7d0 2024-09-18T14:08:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient(hunting): my global_id is 5435 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd29005ed70 con 0x7fd29009f7d0 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703281197 0 0) 0x7fd29009dfc0 con 0x7fd29009ff70 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient(hunting): my global_id is 164233 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd29009dce0 con 0x7fd29009ff70 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 879248982 0 0) 0x7fd27c0034e0 con 0x7fd29009fba0 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 >> v1:172.21.15.45:6793/0 conn(0x7fd29009f7d0 legacy=0x7fd290132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 >> v1:172.21.15.84:6789/0 conn(0x7fd29009ff70 legacy=0x7fd29013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient: _finish_hunting 0 2024-09-18T14:08:26.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 monclient: found mon.noname-a 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900a1c00 con 0x7fd29009fba0 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient: _finish_auth 0 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd294d00640 5 monclient: authenticate success, global_id 5544 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd278001830 con 0x7fd29009fba0 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient: dump: 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd294d00640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd278001a10 con 0x7fd29009fba0 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 1 -- 172.21.15.45:0/816300334 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd278004ae0 con 0x7fd29009fba0 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd294d00640 10 monclient: get_monmap_and_config success 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd294d00640 4 set_mon_vals no callback set 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.783+0000 7fd28dd9b640 10 monclient: dump: 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.784+0000 7fd294d00640 10 monclient: shutdown 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.784+0000 7fd294d00640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.784+0000 7fd294d00640 1 -- 172.21.15.45:0/816300334 >> v1:172.21.15.45:6789/0 conn(0x7fd29009fba0 legacy=0x7fd290137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.784+0000 7fd294d00640 1 -- 172.21.15.45:0/816300334 shutdown_connections 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.784+0000 7fd294d00640 1 -- 172.21.15.45:0/816300334 >> 172.21.15.45:0/816300334 conn(0x7fd2900630b0 msgr2=0x7fd2900634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.784+0000 7fd294d00640 1 -- 172.21.15.45:0/816300334 shutdown_connections 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.784+0000 7fd294d00640 1 -- 172.21.15.45:0/816300334 wait complete. 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.784+0000 7fd294d00640 10 monclient: build_initial_monmap 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.784+0000 7fd294d00640 10 monclient: monmap: 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:08:26.799 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 1 Processor -- start 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 1 -- start start 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 10 monclient: init 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 10 monclient: _reopen_session rank -1 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 10 monclient: _add_conns ranks=[1,2,0,5,6,3,7,8,4] 2024-09-18T14:08:26.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 10 monclient(hunting): picked mon.noname-f con 0x7fd29009f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 10 monclient(hunting): picked mon.noname-b con 0x7fd29009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 10 monclient(hunting): picked mon.noname-a con 0x7fd29009ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29009d310 con 0x7fd29009ff70 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2900a2560 con 0x7fd29009f7d0 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd290121b70 con 0x7fd29009fba0 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 10 monclient(hunting): _renew_subs 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd294d00640 10 monclient(hunting): authenticate will time out at 1322.123708s 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd287fff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fd29009fba0 0x7fd290146b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55166/0 (socket says 172.21.15.45:55166) 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd287fff640 1 -- 172.21.15.45:0/791933117 learned_addr learned my addr 172.21.15.45:0/791933117 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203179781 0 0) 0x7fd290121b70 con 0x7fd29009fba0 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd2867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd2867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.785+0000 7fd2867fc640 10 monclient(hunting): my global_id is 144794 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2700034e0 con 0x7fd29009fba0 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438920396 0 0) 0x7fd29009d310 con 0x7fd29009ff70 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): my global_id is 5553 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd290121b70 con 0x7fd29009ff70 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1781943473 0 0) 0x7fd2900a2560 con 0x7fd29009f7d0 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): my global_id is 164242 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd29009d310 con 0x7fd29009f7d0 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3188673630 0 0) 0x7fd2700034e0 con 0x7fd29009fba0 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2900a2560 con 0x7fd29009fba0 2024-09-18T14:08:26.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd278001430 con 0x7fd29009fba0 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): dump: 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2737488766 0 0) 0x7fd290121b70 con 0x7fd29009ff70 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2700034e0 con 0x7fd29009ff70 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3275175287 0 0) 0x7fd29009d310 con 0x7fd29009f7d0 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd290121b70 con 0x7fd29009f7d0 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd288001d00 con 0x7fd29009ff70 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:08:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): dump: 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.805 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd280001550 con 0x7fd29009f7d0 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient(hunting): dump: 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.806 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1302424397 0 0) 0x7fd2900a2560 con 0x7fd29009fba0 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 >> v1:172.21.15.84:6789/0 conn(0x7fd29009f7d0 legacy=0x7fd290131740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 >> v1:172.21.15.45:6789/0 conn(0x7fd29009ff70 legacy=0x7fd29014a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient: _finish_hunting 0 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 monclient: found mon.b 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:08:26.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29011fd60 con 0x7fd29009fba0 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient: _finish_auth 0 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd294d00640 5 monclient: authenticate success, global_id 144794 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd294d00640 10 monclient: _renew_subs 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd294d00640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd294d00640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd290121890 con 0x7fd29009fba0 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd294d00640 10 monclient: _renew_subs 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd294d00640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd294d00640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd290131e50 con 0x7fd29009fba0 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd278001230 con 0x7fd29009fba0 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd2780050a0 con 0x7fd29009fba0 2024-09-18T14:08:26.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd28d59a640 4 set_mon_vals no callback set 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.786+0000 7fd2867fc640 10 monclient: dump: 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.787+0000 7fd294d00640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.787+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd278010070 con 0x7fd29009fba0 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.787+0000 7fd294d00640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd274006580 con 0x7fd274002830 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.787+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(437..437 src has 126..437) v4 ==== 3971+0+0 (unknown 923945606 0 0) 0x7fd278046d00 con 0x7fd29009fba0 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.787+0000 7fd28ed9d640 10 monclient: get_auth_request con 0x7fd27003a9d0 auth_method 0 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.787+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386553252 0 0) 0x7fd274006580 con 0x7fd274002830 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.787+0000 7fd2867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.787+0000 7fd2867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.787+0000 7fd2867fc640 10 monclient(hunting): my global_id is 164251 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.787+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2900a2560 con 0x7fd274002830 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1621460360 0 0) 0x7fd2900a2560 con 0x7fd274002830 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd274006580 con 0x7fd274002830 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd280004ae0 con 0x7fd274002830 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 10 monclient: dump: 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3205562334 0 0) 0x7fd274006580 con 0x7fd274002830 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd27400a0a0 con 0x7fd274002830 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd280006fc0 con 0x7fd274002830 2024-09-18T14:08:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:08:26.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:08:26.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 >> v1:172.21.15.84:6789/0 conn(0x7fd274002830 legacy=0x7fd274004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.788+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 >> v1:172.21.15.84:6789/0 conn(0x7fd274002830 legacy=0x7fd274004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:26.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.794+0000 7fd294d00640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:08:26.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.794+0000 7fd294d00640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd2740074f0 con 0x7fd2740068a0 2024-09-18T14:08:26.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169007413 0 0) 0x7fd2740074f0 con 0x7fd2740068a0 2024-09-18T14:08:26.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:08:26.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:08:26.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 10 monclient(hunting): my global_id is 164260 2024-09-18T14:08:26.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd274006580 con 0x7fd2740068a0 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3910575437 0 0) 0x7fd274006580 con 0x7fd2740068a0 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2740074f0 con 0x7fd2740068a0 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd2800055f0 con 0x7fd2740068a0 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 10 monclient: dump: 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:08:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2026572630 0 0) 0x7fd2740074f0 con 0x7fd2740068a0 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd274008750 con 0x7fd2740068a0 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7342 (unknown 0 0 3805289718) 0x7fd280007250 con 0x7fd2740068a0 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 >> v1:172.21.15.84:6789/0 conn(0x7fd2740068a0 legacy=0x7fd274006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.795+0000 7fd2867fc640 1 -- 172.21.15.45:0/791933117 >> v1:172.21.15.84:6789/0 conn(0x7fd2740068a0 legacy=0x7fd274006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:08:26.817 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:08:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:08:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 88, 2024-09-18T14:08:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:08:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:08:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:08:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:08:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:08:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 7, 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:08:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:08:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:08:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:08:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:08:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:08:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:08:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:08:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:08:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:08:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:08:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:08:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:08:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:08:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:08:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:08:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:08:26.835 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:08:26.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.796+0000 7fd294d00640 1 -- 172.21.15.45:0/791933117 >> v1:172.21.15.84:6812/31478 conn(0x7fd27003a9d0 legacy=0x7fd27003ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.796+0000 7fd294d00640 10 monclient: shutdown 2024-09-18T14:08:26.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.796+0000 7fd294d00640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:08:26.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.796+0000 7fd294d00640 1 -- 172.21.15.45:0/791933117 >> v1:172.21.15.45:6790/0 conn(0x7fd29009fba0 legacy=0x7fd290146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:08:26.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.796+0000 7fd28ed9d640 1 -- 172.21.15.45:0/791933117 reap_dead start 2024-09-18T14:08:26.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.796+0000 7fd28ed9d640 1 -- 172.21.15.45:0/791933117 reap_dead start 2024-09-18T14:08:26.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.796+0000 7fd294d00640 1 -- 172.21.15.45:0/791933117 shutdown_connections 2024-09-18T14:08:26.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.796+0000 7fd294d00640 1 -- 172.21.15.45:0/791933117 >> 172.21.15.45:0/791933117 conn(0x7fd2900630b0 msgr2=0x7fd2900a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:08:26.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.796+0000 7fd294d00640 1 -- 172.21.15.45:0/791933117 shutdown_connections 2024-09-18T14:08:26.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:08:26.796+0000 7fd294d00640 1 -- 172.21.15.45:0/791933117 wait complete. 2024-09-18T14:08:26.837 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-18T14:08:26.837 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-18T14:08:27.241 INFO:tasks.rados.rados.0.smithi084.stdout:2036: done (3 left) 2024-09-18T14:08:27.241 INFO:tasks.rados.rados.0.smithi084.stdout:2037: done (2 left) 2024-09-18T14:08:27.241 INFO:tasks.rados.rados.0.smithi084.stdout:2040: done (1 left) 2024-09-18T14:08:27.241 INFO:tasks.rados.rados.0.smithi084.stdout:2041: done (0 left) 2024-09-18T14:08:27.241 INFO:tasks.rados.rados.0.smithi084.stdout:2042: write (excl) oid 27 current snap is 209 2024-09-18T14:08:27.241 INFO:tasks.rados.rados.0.smithi084.stdout:2042: seq_num 723 ranges {551461=658562,1941540=209921} 2024-09-18T14:08:27.247 INFO:tasks.rados.rados.0.smithi084.stdout:2042: writing smithi08434663-27 from 551461 to 1210023 tid 1 2024-09-18T14:08:27.253 INFO:tasks.rados.rados.0.smithi084.stdout:2042: writing smithi08434663-27 from 1941540 to 2151461 tid 2 2024-09-18T14:08:27.253 INFO:tasks.rados.rados.0.smithi084.stdout:2043: snap_create 2024-09-18T14:08:27.258 INFO:tasks.rados.rados.0.smithi084.stdout:2042: finishing write tid 1 to smithi08434663-27 2024-09-18T14:08:27.261 INFO:tasks.rados.rados.0.smithi084.stdout:2042: finishing write tid 2 to smithi08434663-27 2024-09-18T14:08:27.261 INFO:tasks.rados.rados.0.smithi084.stdout:2042: finishing write tid 3 to smithi08434663-27 2024-09-18T14:08:27.261 INFO:tasks.rados.rados.0.smithi084.stdout:2042: finishing write tid 4 to smithi08434663-27 2024-09-18T14:08:27.261 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 796 (ObjNum 723 snap 209 seq_num 723) dirty exists 2024-09-18T14:08:27.261 INFO:tasks.rados.rados.0.smithi084.stdout:2042: left oid 27 (ObjNum 723 snap 209 seq_num 723) 2024-09-18T14:08:28.252 INFO:tasks.rados.rados.0.smithi084.stdout:2042: done (1 left) 2024-09-18T14:08:28.252 INFO:tasks.rados.rados.0.smithi084.stdout:2043: done (0 left) 2024-09-18T14:08:28.252 INFO:tasks.rados.rados.0.smithi084.stdout:2044: read oid 31 snap -1 2024-09-18T14:08:28.252 INFO:tasks.rados.rados.0.smithi084.stdout:2044: expect (ObjNum 713 snap 207 seq_num 713) 2024-09-18T14:08:28.252 INFO:tasks.rados.rados.0.smithi084.stdout:2045: read oid 21 snap -1 2024-09-18T14:08:28.252 INFO:tasks.rados.rados.0.smithi084.stdout:2045: expect (ObjNum 397 snap 121 seq_num 397) 2024-09-18T14:08:28.252 INFO:tasks.rados.rados.0.smithi084.stdout:2046: delete oid 11 current snap is 210 2024-09-18T14:08:28.255 INFO:tasks.rados.rados.0.smithi084.stdout:2046: done (2 left) 2024-09-18T14:08:28.255 INFO:tasks.rados.rados.0.smithi084.stdout:2047: write (excl) oid 30 current snap is 210 2024-09-18T14:08:28.255 INFO:tasks.rados.rados.0.smithi084.stdout:2047: seq_num 725 ranges {585075=409060,1542151=642924} 2024-09-18T14:08:28.261 INFO:tasks.rados.rados.0.smithi084.stdout:2047: writing smithi08434663-30 from 585075 to 994135 tid 1 2024-09-18T14:08:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2047: writing smithi08434663-30 from 1542151 to 2185075 tid 2 2024-09-18T14:08:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2048: write oid 25 current snap is 210 2024-09-18T14:08:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2048: seq_num 726 ranges {601882=484309,1600633=524904} 2024-09-18T14:08:28.274 INFO:tasks.rados.rados.0.smithi084.stdout:2048: writing smithi08434663-25 from 601882 to 1086191 tid 1 2024-09-18T14:08:28.280 INFO:tasks.rados.rados.0.smithi084.stdout:2048: writing smithi08434663-25 from 1600633 to 2125537 tid 2 2024-09-18T14:08:28.280 INFO:tasks.rados.rados.0.smithi084.stdout:2049: snap_create 2024-09-18T14:08:28.280 INFO:tasks.rados.rados.0.smithi084.stdout:2045: expect (ObjNum 397 snap 121 seq_num 397) 2024-09-18T14:08:28.365 INFO:tasks.rados.rados.0.smithi084.stdout:2044: expect (ObjNum 713 snap 207 seq_num 713) 2024-09-18T14:08:28.567 INFO:tasks.rados.rados.0.smithi084.stdout:2047: finishing write tid 1 to smithi08434663-30 2024-09-18T14:08:28.567 INFO:tasks.rados.rados.0.smithi084.stdout:2047: finishing write tid 2 to smithi08434663-30 2024-09-18T14:08:28.567 INFO:tasks.rados.rados.0.smithi084.stdout:2047: finishing write tid 3 to smithi08434663-30 2024-09-18T14:08:28.567 INFO:tasks.rados.rados.0.smithi084.stdout:2047: finishing write tid 4 to smithi08434663-30 2024-09-18T14:08:28.568 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 849 (ObjNum 725 snap 210 seq_num 725) dirty exists 2024-09-18T14:08:28.568 INFO:tasks.rados.rados.0.smithi084.stdout:2047: left oid 30 (ObjNum 725 snap 210 seq_num 725) 2024-09-18T14:08:28.568 INFO:tasks.rados.rados.0.smithi084.stdout:2048: finishing write tid 1 to smithi08434663-25 2024-09-18T14:08:28.568 INFO:tasks.rados.rados.0.smithi084.stdout:2048: finishing write tid 2 to smithi08434663-25 2024-09-18T14:08:28.568 INFO:tasks.rados.rados.0.smithi084.stdout:2048: finishing write tid 3 to smithi08434663-25 2024-09-18T14:08:28.568 INFO:tasks.rados.rados.0.smithi084.stdout:2048: finishing write tid 4 to smithi08434663-25 2024-09-18T14:08:28.568 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 640 (ObjNum 726 snap 210 seq_num 726) dirty exists 2024-09-18T14:08:28.568 INFO:tasks.rados.rados.0.smithi084.stdout:2048: left oid 25 (ObjNum 726 snap 210 seq_num 726) 2024-09-18T14:08:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2044: done (4 left) 2024-09-18T14:08:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2045: done (3 left) 2024-09-18T14:08:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2047: done (2 left) 2024-09-18T14:08:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2048: done (1 left) 2024-09-18T14:08:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2049: done (0 left) 2024-09-18T14:08:29.272 INFO:tasks.rados.rados.0.smithi084.stdout:2050: rollback oid 2 current snap is 211 2024-09-18T14:08:29.272 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 2 to 210 2024-09-18T14:08:29.272 INFO:tasks.rados.rados.0.smithi084.stdout:2051: snap_remove snap 208 2024-09-18T14:08:30.274 INFO:tasks.rados.rados.0.smithi084.stdout:2051: done (1 left) 2024-09-18T14:08:30.275 INFO:tasks.rados.rados.0.smithi084.stdout:2052: write (excl) oid 6 current snap is 211 2024-09-18T14:08:30.275 INFO:tasks.rados.rados.0.smithi084.stdout:2052: seq_num 727 ranges {618689=559558,1659115=559574} 2024-09-18T14:08:30.281 INFO:tasks.rados.rados.0.smithi084.stdout:2052: writing smithi08434663-6 from 618689 to 1178247 tid 1 2024-09-18T14:08:30.287 INFO:tasks.rados.rados.0.smithi084.stdout:2052: writing smithi08434663-6 from 1659115 to 2218689 tid 2 2024-09-18T14:08:30.287 INFO:tasks.rados.rados.0.smithi084.stdout:2053: read oid 23 snap -1 2024-09-18T14:08:30.287 INFO:tasks.rados.rados.0.smithi084.stdout:2053: expect deleted 2024-09-18T14:08:30.287 INFO:tasks.rados.rados.0.smithi084.stdout:2054: snap_create 2024-09-18T14:08:30.287 INFO:tasks.rados.rados.0.smithi084.stdout:2050: finishing rollback tid 0 to smithi08434663-2 2024-09-18T14:08:30.287 INFO:tasks.rados.rados.0.smithi084.stdout:2050: finishing rollback tid 1 to smithi08434663-2 2024-09-18T14:08:30.287 INFO:tasks.rados.rados.0.smithi084.stdout:2050: finishing rollback tid 2 to smithi08434663-2 2024-09-18T14:08:30.287 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 1051 (ObjNum 717 snap 208 seq_num 717) dirty exists 2024-09-18T14:08:30.287 INFO:tasks.rados.rados.0.smithi084.stdout:2052: finishing write tid 1 to smithi08434663-6 2024-09-18T14:08:30.290 INFO:tasks.rados.rados.0.smithi084.stdout:2052: finishing write tid 2 to smithi08434663-6 2024-09-18T14:08:30.290 INFO:tasks.rados.rados.0.smithi084.stdout:2052: finishing write tid 3 to smithi08434663-6 2024-09-18T14:08:30.290 INFO:tasks.rados.rados.0.smithi084.stdout:2052: finishing write tid 4 to smithi08434663-6 2024-09-18T14:08:30.290 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 923 (ObjNum 727 snap 211 seq_num 727) dirty exists 2024-09-18T14:08:30.291 INFO:tasks.rados.rados.0.smithi084.stdout:2052: left oid 6 (ObjNum 727 snap 211 seq_num 727) 2024-09-18T14:08:31.280 INFO:tasks.rados.rados.0.smithi084.stdout:2050: done (3 left) 2024-09-18T14:08:31.280 INFO:tasks.rados.rados.0.smithi084.stdout:2052: done (2 left) 2024-09-18T14:08:31.280 INFO:tasks.rados.rados.0.smithi084.stdout:2053: done (1 left) 2024-09-18T14:08:31.281 INFO:tasks.rados.rados.0.smithi084.stdout:2054: done (0 left) 2024-09-18T14:08:31.281 INFO:tasks.rados.rados.0.smithi084.stdout:2055: rollback oid 6 current snap is 212 2024-09-18T14:08:31.281 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 209 2024-09-18T14:08:31.281 INFO:tasks.rados.rados.0.smithi084.stdout:2056: read oid 33 snap -1 2024-09-18T14:08:31.281 INFO:tasks.rados.rados.0.smithi084.stdout:2056: expect deleted 2024-09-18T14:08:31.281 INFO:tasks.rados.rados.0.smithi084.stdout:2057: delete oid 42 current snap is 212 2024-09-18T14:08:31.285 INFO:tasks.rados.rados.0.smithi084.stdout:2056: done (2 left) 2024-09-18T14:08:31.285 INFO:tasks.rados.rados.0.smithi084.stdout:2057: done (1 left) 2024-09-18T14:08:31.285 INFO:tasks.rados.rados.0.smithi084.stdout:2058: delete oid 29 current snap is 212 2024-09-18T14:08:31.287 INFO:tasks.rados.rados.0.smithi084.stdout:2055: finishing rollback tid 0 to smithi08434663-6 2024-09-18T14:08:31.288 INFO:tasks.rados.rados.0.smithi084.stdout:2055: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:08:31.288 INFO:tasks.rados.rados.0.smithi084.stdout:2055: finishing rollback tid 2 to smithi08434663-6 2024-09-18T14:08:31.288 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 926 (ObjNum 722 snap 208 seq_num 722) dirty exists 2024-09-18T14:08:31.289 INFO:tasks.rados.rados.0.smithi084.stdout:2055: done (1 left) 2024-09-18T14:08:31.289 INFO:tasks.rados.rados.0.smithi084.stdout:2058: done (0 left) 2024-09-18T14:08:31.289 INFO:tasks.rados.rados.0.smithi084.stdout:2059: delete oid 2 current snap is 212 2024-09-18T14:08:31.293 INFO:tasks.rados.rados.0.smithi084.stdout:2059: done (0 left) 2024-09-18T14:08:31.293 INFO:tasks.rados.rados.0.smithi084.stdout:2060: write (excl) oid 45 current snap is 212 2024-09-18T14:08:31.293 INFO:tasks.rados.rados.0.smithi084.stdout:2060: seq_num 731 ranges {685917=576907,1893043=392874} 2024-09-18T14:08:31.300 INFO:tasks.rados.rados.0.smithi084.stdout:2060: writing smithi08434663-45 from 685917 to 1262824 tid 1 2024-09-18T14:08:31.306 INFO:tasks.rados.rados.0.smithi084.stdout:2060: writing smithi08434663-45 from 1893043 to 2285917 tid 2 2024-09-18T14:08:31.306 INFO:tasks.rados.rados.0.smithi084.stdout:2061: read oid 23 snap -1 2024-09-18T14:08:31.306 INFO:tasks.rados.rados.0.smithi084.stdout:2061: expect deleted 2024-09-18T14:08:31.306 INFO:tasks.rados.rados.0.smithi084.stdout:2062: write oid 3 current snap is 212 2024-09-18T14:08:31.306 INFO:tasks.rados.rados.0.smithi084.stdout:2060: finishing write tid 1 to smithi08434663-45 2024-09-18T14:08:31.306 INFO:tasks.rados.rados.0.smithi084.stdout:2062: seq_num 732 ranges {702724=652156,1951525=351199} 2024-09-18T14:08:31.313 INFO:tasks.rados.rados.0.smithi084.stdout:2062: writing smithi08434663-3 from 702724 to 1354880 tid 1 2024-09-18T14:08:31.318 INFO:tasks.rados.rados.0.smithi084.stdout:2062: writing smithi08434663-3 from 1951525 to 2302724 tid 2 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:2063: rollback oid 34 current snap is 212 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 34 to 211 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:2060: finishing write tid 2 to smithi08434663-45 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:2060: finishing write tid 3 to smithi08434663-45 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:2060: finishing write tid 4 to smithi08434663-45 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 935 (ObjNum 731 snap 212 seq_num 731) dirty exists 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:2060: left oid 45 (ObjNum 731 snap 212 seq_num 731) 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:2062: finishing write tid 1 to smithi08434663-3 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:2060: done (3 left) 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:2061: done (2 left) 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:2064: rollback oid 27 current snap is 212 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 27 to 211 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:2065: read oid 21 snap 210 2024-09-18T14:08:31.319 INFO:tasks.rados.rados.0.smithi084.stdout:2065: expect (ObjNum 397 snap 121 seq_num 397) 2024-09-18T14:08:31.320 INFO:tasks.rados.rados.0.smithi084.stdout:2066: snap_create 2024-09-18T14:08:31.322 INFO:tasks.rados.rados.0.smithi084.stdout:2063: finishing rollback tid 0 to smithi08434663-34 2024-09-18T14:08:31.323 INFO:tasks.rados.rados.0.smithi084.stdout:2063: finishing rollback tid 1 to smithi08434663-34 2024-09-18T14:08:31.323 INFO:tasks.rados.rados.0.smithi084.stdout:2063: finishing rollback tid 2 to smithi08434663-34 2024-09-18T14:08:31.323 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 939 (ObjNum 718 snap 208 seq_num 718) dirty exists 2024-09-18T14:08:31.324 INFO:tasks.rados.rados.0.smithi084.stdout:2064: finishing rollback tid 0 to smithi08434663-27 2024-09-18T14:08:31.325 INFO:tasks.rados.rados.0.smithi084.stdout:2064: finishing rollback tid 1 to smithi08434663-27 2024-09-18T14:08:31.325 INFO:tasks.rados.rados.0.smithi084.stdout:2062: finishing write tid 2 to smithi08434663-3 2024-09-18T14:08:31.325 INFO:tasks.rados.rados.0.smithi084.stdout:2062: finishing write tid 3 to smithi08434663-3 2024-09-18T14:08:31.325 INFO:tasks.rados.rados.0.smithi084.stdout:2062: finishing write tid 4 to smithi08434663-3 2024-09-18T14:08:31.325 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 746 (ObjNum 732 snap 212 seq_num 732) dirty exists 2024-09-18T14:08:31.325 INFO:tasks.rados.rados.0.smithi084.stdout:2062: left oid 3 (ObjNum 732 snap 212 seq_num 732) 2024-09-18T14:08:31.325 INFO:tasks.rados.rados.0.smithi084.stdout:2064: finishing rollback tid 2 to smithi08434663-27 2024-09-18T14:08:31.325 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 802 (ObjNum 723 snap 209 seq_num 723) dirty exists 2024-09-18T14:08:31.325 INFO:tasks.rados.rados.0.smithi084.stdout:2065: expect (ObjNum 397 snap 121 seq_num 397) 2024-09-18T14:08:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:2062: done (4 left) 2024-09-18T14:08:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:2063: done (3 left) 2024-09-18T14:08:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:2064: done (2 left) 2024-09-18T14:08:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:2065: done (1 left) 2024-09-18T14:08:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:2066: done (0 left) 2024-09-18T14:08:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:2067: rollback oid 15 current snap is 213 2024-09-18T14:08:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 15 to 210 2024-09-18T14:08:32.243 INFO:tasks.rados.rados.0.smithi084.stdout:2068: delete oid 7 current snap is 213 2024-09-18T14:08:32.247 INFO:tasks.rados.rados.0.smithi084.stdout:2068: done (1 left) 2024-09-18T14:08:32.248 INFO:tasks.rados.rados.0.smithi084.stdout:2069: snap_remove snap 209 2024-09-18T14:08:33.245 INFO:tasks.rados.rados.0.smithi084.stdout:2069: done (1 left) 2024-09-18T14:08:33.245 INFO:tasks.rados.rados.0.smithi084.stdout:2070: write oid 18 current snap is 213 2024-09-18T14:08:33.245 INFO:tasks.rados.rados.0.smithi084.stdout:2070: seq_num 734 ranges {736338=402654,1552136=539580} 2024-09-18T14:08:33.251 INFO:tasks.rados.rados.0.smithi084.stdout:2070: writing smithi08434663-18 from 736338 to 1138992 tid 1 2024-09-18T14:08:33.257 INFO:tasks.rados.rados.0.smithi084.stdout:2070: writing smithi08434663-18 from 1552136 to 2091716 tid 2 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:2071: copy_from oid 13 from oid 23 current snap is 213 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:2067: finishing rollback tid 0 to smithi08434663-15 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:2067: finishing rollback tid 1 to smithi08434663-15 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:2067: finishing rollback tid 2 to smithi08434663-15 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 867 (ObjNum 699 snap 205 seq_num 699) dirty exists 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:2070: finishing write tid 1 to smithi08434663-18 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:2067: done (2 left) 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:2072: read oid 3 snap -1 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:2072: expect (ObjNum 732 snap 212 seq_num 732) 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:2073: rollback oid 43 current snap is 213 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 210 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:2074: write (excl) oid 8 current snap is 213 2024-09-18T14:08:33.258 INFO:tasks.rados.rados.0.smithi084.stdout:2074: seq_num 735 ranges {753145=477903,2010618=342527} 2024-09-18T14:08:33.264 INFO:tasks.rados.rados.0.smithi084.stdout:2074: writing smithi08434663-8 from 753145 to 1231048 tid 1 2024-09-18T14:08:33.270 INFO:tasks.rados.rados.0.smithi084.stdout:2074: writing smithi08434663-8 from 2010618 to 2353145 tid 2 2024-09-18T14:08:33.271 INFO:tasks.rados.rados.0.smithi084.stdout:2075: read oid 29 snap -1 2024-09-18T14:08:33.271 INFO:tasks.rados.rados.0.smithi084.stdout:2075: expect deleted 2024-09-18T14:08:33.271 INFO:tasks.rados.rados.0.smithi084.stdout:2076: read oid 6 snap 212 2024-09-18T14:08:33.271 INFO:tasks.rados.rados.0.smithi084.stdout:2076: expect (ObjNum 722 snap 208 seq_num 722) 2024-09-18T14:08:33.271 INFO:tasks.rados.rados.0.smithi084.stdout:2077: read oid 28 snap -1 2024-09-18T14:08:33.271 INFO:tasks.rados.rados.0.smithi084.stdout:2077: expect (ObjNum 700 snap 205 seq_num 700) 2024-09-18T14:08:33.271 INFO:tasks.rados.rados.0.smithi084.stdout:2078: read oid 30 snap -1 2024-09-18T14:08:33.271 INFO:tasks.rados.rados.0.smithi084.stdout:2078: expect (ObjNum 725 snap 210 seq_num 725) 2024-09-18T14:08:33.271 INFO:tasks.rados.rados.0.smithi084.stdout:2079: snap_remove snap 211 2024-09-18T14:08:34.246 INFO:tasks.rados.rados.0.smithi084.stdout:2079: done (9 left) 2024-09-18T14:08:34.246 INFO:tasks.rados.rados.0.smithi084.stdout:2080: write oid 11 current snap is 213 2024-09-18T14:08:34.246 INFO:tasks.rados.rados.0.smithi084.stdout:2080: seq_num 736 ranges {769952=553152,2069100=300852} 2024-09-18T14:08:34.253 INFO:tasks.rados.rados.0.smithi084.stdout:2080: writing smithi08434663-11 from 769952 to 1323104 tid 1 2024-09-18T14:08:34.260 INFO:tasks.rados.rados.0.smithi084.stdout:2080: writing smithi08434663-11 from 2069100 to 2369952 tid 2 2024-09-18T14:08:34.260 INFO:tasks.rados.rados.0.smithi084.stdout:2081: rollback oid 36 current snap is 213 2024-09-18T14:08:34.260 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 36 to 212 2024-09-18T14:08:34.260 INFO:tasks.rados.rados.0.smithi084.stdout:2071: finishing copy_from to smithi08434663-13 2024-09-18T14:08:34.260 INFO:tasks.rados.rados.0.smithi084.stdout:2071: got expected ENOENT (src dne) 2024-09-18T14:08:34.260 INFO:tasks.rados.rados.0.smithi084.stdout:2071: finishing copy_from racing read to smithi08434663-13 2024-09-18T14:08:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:2073: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:08:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-18T14:08:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:2070: finishing write tid 2 to smithi08434663-18 2024-09-18T14:08:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:2070: finishing write tid 3 to smithi08434663-18 2024-09-18T14:08:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:2071: done (10 left) 2024-09-18T14:08:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:2073: done (9 left) 2024-09-18T14:08:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:2082: write (excl) oid 23 current snap is 213 2024-09-18T14:08:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:2070: finishing write tid 4 to smithi08434663-18 2024-09-18T14:08:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 749 (ObjNum 734 snap 213 seq_num 734) dirty exists 2024-09-18T14:08:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:2070: left oid 18 (ObjNum 734 snap 213 seq_num 734) 2024-09-18T14:08:34.261 INFO:tasks.rados.rados.0.smithi084.stdout:2072: expect (ObjNum 732 snap 212 seq_num 732) 2024-09-18T14:08:34.387 INFO:tasks.rados.rados.0.smithi084.stdout:2074: finishing write tid 1 to smithi08434663-8 2024-09-18T14:08:34.387 INFO:tasks.rados.rados.0.smithi084.stdout:2074: finishing write tid 2 to smithi08434663-8 2024-09-18T14:08:34.387 INFO:tasks.rados.rados.0.smithi084.stdout:2074: finishing write tid 3 to smithi08434663-8 2024-09-18T14:08:34.387 INFO:tasks.rados.rados.0.smithi084.stdout:2074: finishing write tid 4 to smithi08434663-8 2024-09-18T14:08:34.387 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 630 (ObjNum 735 snap 213 seq_num 735) dirty exists 2024-09-18T14:08:34.388 INFO:tasks.rados.rados.0.smithi084.stdout:2074: left oid 8 (ObjNum 735 snap 213 seq_num 735) 2024-09-18T14:08:34.388 INFO:tasks.rados.rados.0.smithi084.stdout:2077: expect (ObjNum 700 snap 205 seq_num 700) 2024-09-18T14:08:34.613 INFO:tasks.rados.rados.0.smithi084.stdout:2078: expect (ObjNum 725 snap 210 seq_num 725) 2024-09-18T14:08:34.718 INFO:tasks.rados.rados.0.smithi084.stdout:2076: expect (ObjNum 722 snap 208 seq_num 722) 2024-09-18T14:08:34.824 INFO:tasks.rados.rados.0.smithi084.stdout:2080: finishing write tid 1 to smithi08434663-11 2024-09-18T14:08:34.824 INFO:tasks.rados.rados.0.smithi084.stdout:2082: seq_num 737 ranges {786759=628401,2011229=375530} 2024-09-18T14:08:34.831 INFO:tasks.rados.rados.0.smithi084.stdout:2082: writing smithi08434663-23 from 786759 to 1415160 tid 1 2024-09-18T14:08:34.836 INFO:tasks.rados.rados.0.smithi084.stdout:2082: writing smithi08434663-23 from 2011229 to 2386759 tid 2 2024-09-18T14:08:34.837 INFO:tasks.rados.rados.0.smithi084.stdout:2070: done (9 left) 2024-09-18T14:08:34.837 INFO:tasks.rados.rados.0.smithi084.stdout:2072: done (8 left) 2024-09-18T14:08:34.837 INFO:tasks.rados.rados.0.smithi084.stdout:2074: done (7 left) 2024-09-18T14:08:34.837 INFO:tasks.rados.rados.0.smithi084.stdout:2075: done (6 left) 2024-09-18T14:08:34.837 INFO:tasks.rados.rados.0.smithi084.stdout:2076: done (5 left) 2024-09-18T14:08:34.837 INFO:tasks.rados.rados.0.smithi084.stdout:2077: done (4 left) 2024-09-18T14:08:34.837 INFO:tasks.rados.rados.0.smithi084.stdout:2078: done (3 left) 2024-09-18T14:08:34.837 INFO:tasks.rados.rados.0.smithi084.stdout:2083: snap_remove snap 210 2024-09-18T14:08:35.294 INFO:tasks.rados.rados.0.smithi084.stdout:2083: done (3 left) 2024-09-18T14:08:35.295 INFO:tasks.rados.rados.0.smithi084.stdout:2084: copy_from oid 21 from oid 7 current snap is 213 2024-09-18T14:08:35.295 INFO:tasks.rados.rados.0.smithi084.stdout:2081: finishing rollback tid 1 to smithi08434663-36 2024-09-18T14:08:35.295 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 0 (ObjNum 657 snap 0 seq_num 2495040592) dirty dne 2024-09-18T14:08:35.295 INFO:tasks.rados.rados.0.smithi084.stdout:2080: finishing write tid 2 to smithi08434663-11 2024-09-18T14:08:35.295 INFO:tasks.rados.rados.0.smithi084.stdout:2080: finishing write tid 3 to smithi08434663-11 2024-09-18T14:08:35.295 INFO:tasks.rados.rados.0.smithi084.stdout:2080: finishing write tid 4 to smithi08434663-11 2024-09-18T14:08:35.295 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1033 (ObjNum 736 snap 213 seq_num 736) dirty exists 2024-09-18T14:08:35.295 INFO:tasks.rados.rados.0.smithi084.stdout:2080: left oid 11 (ObjNum 736 snap 213 seq_num 736) 2024-09-18T14:08:35.295 INFO:tasks.rados.rados.0.smithi084.stdout:2082: finishing write tid 1 to smithi08434663-23 2024-09-18T14:08:35.295 INFO:tasks.rados.rados.0.smithi084.stdout:2082: finishing write tid 2 to smithi08434663-23 2024-09-18T14:08:35.296 INFO:tasks.rados.rados.0.smithi084.stdout:2082: finishing write tid 3 to smithi08434663-23 2024-09-18T14:08:35.296 INFO:tasks.rados.rados.0.smithi084.stdout:2082: finishing write tid 4 to smithi08434663-23 2024-09-18T14:08:35.296 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 930 (ObjNum 737 snap 213 seq_num 737) dirty exists 2024-09-18T14:08:35.296 INFO:tasks.rados.rados.0.smithi084.stdout:2082: left oid 23 (ObjNum 737 snap 213 seq_num 737) 2024-09-18T14:08:35.296 INFO:tasks.rados.rados.0.smithi084.stdout:2080: done (3 left) 2024-09-18T14:08:35.296 INFO:tasks.rados.rados.0.smithi084.stdout:2081: done (2 left) 2024-09-18T14:08:35.296 INFO:tasks.rados.rados.0.smithi084.stdout:2082: done (1 left) 2024-09-18T14:08:35.296 INFO:tasks.rados.rados.0.smithi084.stdout:2085: rollback oid 9 current snap is 213 2024-09-18T14:08:35.296 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 9 to 212 2024-09-18T14:08:35.296 INFO:tasks.rados.rados.0.smithi084.stdout:2086: copy_from oid 34 from oid 26 current snap is 213 2024-09-18T14:08:35.296 INFO:tasks.rados.rados.0.smithi084.stdout:2087: read oid 4 snap -1 2024-09-18T14:08:35.296 INFO:tasks.rados.rados.0.smithi084.stdout:2087: expect (ObjNum 644 snap 190 seq_num 644) 2024-09-18T14:08:35.297 INFO:tasks.rados.rados.0.smithi084.stdout:2088: read oid 1 snap -1 2024-09-18T14:08:35.297 INFO:tasks.rados.rados.0.smithi084.stdout:2088: expect (ObjNum 538 snap 167 seq_num 538) 2024-09-18T14:08:35.297 INFO:tasks.rados.rados.0.smithi084.stdout:2089: copy_from oid 11 from oid 25 current snap is 213 2024-09-18T14:08:35.297 INFO:tasks.rados.rados.0.smithi084.stdout:2090: write oid 19 current snap is 213 2024-09-18T14:08:35.297 INFO:tasks.rados.rados.0.smithi084.stdout:2090: seq_num 738 ranges {403566=420003,1386064=546918} 2024-09-18T14:08:35.299 INFO:tasks.rados.rados.0.smithi084.stdout:2090: writing smithi08434663-19 from 403566 to 823569 tid 1 2024-09-18T14:08:35.306 INFO:tasks.rados.rados.0.smithi084.stdout:2090: writing smithi08434663-19 from 1386064 to 1932982 tid 2 2024-09-18T14:08:35.306 INFO:tasks.rados.rados.0.smithi084.stdout:2091: delete oid 44 current snap is 213 2024-09-18T14:08:35.307 INFO:tasks.rados.rados.0.smithi084.stdout:2088: expect (ObjNum 538 snap 167 seq_num 538) 2024-09-18T14:08:35.429 INFO:tasks.rados.rados.0.smithi084.stdout:2084: finishing copy_from to smithi08434663-21 2024-09-18T14:08:35.430 INFO:tasks.rados.rados.0.smithi084.stdout:2084: got expected ENOENT (src dne) 2024-09-18T14:08:35.430 INFO:tasks.rados.rados.0.smithi084.stdout:2084: finishing copy_from racing read to smithi08434663-21 2024-09-18T14:08:35.430 INFO:tasks.rados.rados.0.smithi084.stdout:2084: done (7 left) 2024-09-18T14:08:35.430 INFO:tasks.rados.rados.0.smithi084.stdout:2088: done (6 left) 2024-09-18T14:08:35.430 INFO:tasks.rados.rados.0.smithi084.stdout:2091: done (5 left) 2024-09-18T14:08:35.430 INFO:tasks.rados.rados.0.smithi084.stdout:2092: delete oid 43 current snap is 213 2024-09-18T14:08:35.430 INFO:tasks.rados.rados.0.smithi084.stdout:2090: finishing write tid 1 to smithi08434663-19 2024-09-18T14:08:35.430 INFO:tasks.rados.rados.0.smithi084.stdout:2089: finishing copy_from racing read to smithi08434663-11 2024-09-18T14:08:35.430 INFO:tasks.rados.rados.0.smithi084.stdout:2087: expect (ObjNum 644 snap 190 seq_num 644) 2024-09-18T14:08:35.581 INFO:tasks.rados.rados.0.smithi084.stdout:2085: finishing rollback tid 0 to smithi08434663-9 2024-09-18T14:08:35.581 INFO:tasks.rados.rados.0.smithi084.stdout:2085: finishing rollback tid 1 to smithi08434663-9 2024-09-18T14:08:35.581 INFO:tasks.rados.rados.0.smithi084.stdout:2085: finishing rollback tid 2 to smithi08434663-9 2024-09-18T14:08:35.581 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 870 (ObjNum 534 snap 165 seq_num 534) dirty exists 2024-09-18T14:08:35.581 INFO:tasks.rados.rados.0.smithi084.stdout:2086: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:08:35.581 INFO:tasks.rados.rados.0.smithi084.stdout:2086: finishing copy_from to smithi08434663-34 2024-09-18T14:08:35.581 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 757 (ObjNum 629 snap 189 seq_num 629) dirty exists 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:2090: finishing write tid 2 to smithi08434663-19 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:2090: finishing write tid 3 to smithi08434663-19 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:2090: finishing write tid 4 to smithi08434663-19 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1065 (ObjNum 738 snap 213 seq_num 738) dirty exists 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:2090: left oid 19 (ObjNum 738 snap 213 seq_num 738) 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:2089: finishing copy_from to smithi08434663-11 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 640 (ObjNum 726 snap 210 seq_num 726) dirty exists 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:2085: done (5 left) 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:2086: done (4 left) 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:2087: done (3 left) 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:2089: done (2 left) 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:2090: done (1 left) 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:2092: done (0 left) 2024-09-18T14:08:35.582 INFO:tasks.rados.rados.0.smithi084.stdout:2093: snap_create 2024-09-18T14:08:36.299 INFO:tasks.rados.rados.0.smithi084.stdout:2093: done (0 left) 2024-09-18T14:08:36.300 INFO:tasks.rados.rados.0.smithi084.stdout:2094: write oid 9 current snap is 214 2024-09-18T14:08:36.300 INFO:tasks.rados.rados.0.smithi084.stdout:2094: seq_num 741 ranges {453987=645750,1845157=608830} 2024-09-18T14:08:36.305 INFO:tasks.rados.rados.0.smithi084.stdout:2094: writing smithi08434663-9 from 453987 to 1099737 tid 1 2024-09-18T14:08:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:2094: writing smithi08434663-9 from 1845157 to 2453987 tid 2 2024-09-18T14:08:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:2095: write (excl) oid 25 current snap is 214 2024-09-18T14:08:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:2094: finishing write tid 1 to smithi08434663-9 2024-09-18T14:08:36.313 INFO:tasks.rados.rados.0.smithi084.stdout:2095: seq_num 742 ranges {470794=720999,1903639=554256} 2024-09-18T14:08:36.319 INFO:tasks.rados.rados.0.smithi084.stdout:2095: writing smithi08434663-25 from 470794 to 1191793 tid 1 2024-09-18T14:08:36.327 INFO:tasks.rados.rados.0.smithi084.stdout:2095: writing smithi08434663-25 from 1903639 to 2457895 tid 2 2024-09-18T14:08:36.327 INFO:tasks.rados.rados.0.smithi084.stdout:2096: snap_create 2024-09-18T14:08:36.327 INFO:tasks.rados.rados.0.smithi084.stdout:2094: finishing write tid 2 to smithi08434663-9 2024-09-18T14:08:36.327 INFO:tasks.rados.rados.0.smithi084.stdout:2094: finishing write tid 3 to smithi08434663-9 2024-09-18T14:08:36.327 INFO:tasks.rados.rados.0.smithi084.stdout:2094: finishing write tid 4 to smithi08434663-9 2024-09-18T14:08:36.327 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 873 (ObjNum 741 snap 214 seq_num 741) dirty exists 2024-09-18T14:08:36.328 INFO:tasks.rados.rados.0.smithi084.stdout:2094: left oid 9 (ObjNum 741 snap 214 seq_num 741) 2024-09-18T14:08:36.328 INFO:tasks.rados.rados.0.smithi084.stdout:2095: finishing write tid 1 to smithi08434663-25 2024-09-18T14:08:36.330 INFO:tasks.rados.rados.0.smithi084.stdout:2095: finishing write tid 2 to smithi08434663-25 2024-09-18T14:08:36.330 INFO:tasks.rados.rados.0.smithi084.stdout:2095: finishing write tid 3 to smithi08434663-25 2024-09-18T14:08:36.330 INFO:tasks.rados.rados.0.smithi084.stdout:2095: finishing write tid 4 to smithi08434663-25 2024-09-18T14:08:36.330 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 643 (ObjNum 742 snap 214 seq_num 742) dirty exists 2024-09-18T14:08:36.330 INFO:tasks.rados.rados.0.smithi084.stdout:2095: left oid 25 (ObjNum 742 snap 214 seq_num 742) 2024-09-18T14:08:37.245 INFO:tasks.rados.rados.0.smithi084.stdout:2094: done (2 left) 2024-09-18T14:08:37.245 INFO:tasks.rados.rados.0.smithi084.stdout:2095: done (1 left) 2024-09-18T14:08:37.245 INFO:tasks.rados.rados.0.smithi084.stdout:2096: done (0 left) 2024-09-18T14:08:37.245 INFO:tasks.rados.rados.0.smithi084.stdout:2097: read oid 12 snap 213 2024-09-18T14:08:37.246 INFO:tasks.rados.rados.0.smithi084.stdout:2097: expect (ObjNum 697 snap 205 seq_num 697) 2024-09-18T14:08:37.246 INFO:tasks.rados.rados.0.smithi084.stdout:2098: delete oid 5 current snap is 215 2024-09-18T14:08:37.260 INFO:tasks.rados.rados.0.smithi084.stdout:2098: done (1 left) 2024-09-18T14:08:37.260 INFO:tasks.rados.rados.0.smithi084.stdout:2099: read oid 13 snap -1 2024-09-18T14:08:37.261 INFO:tasks.rados.rados.0.smithi084.stdout:2099: expect (ObjNum 615 snap 183 seq_num 615) 2024-09-18T14:08:37.261 INFO:tasks.rados.rados.0.smithi084.stdout:2100: snap_create 2024-09-18T14:08:37.263 INFO:tasks.rados.rados.0.smithi084.stdout:2097: expect (ObjNum 697 snap 205 seq_num 697) 2024-09-18T14:08:37.432 INFO:tasks.rados.rados.0.smithi084.stdout:2099: expect (ObjNum 615 snap 183 seq_num 615) 2024-09-18T14:08:38.255 INFO:tasks.rados.rados.0.smithi084.stdout:2097: done (2 left) 2024-09-18T14:08:38.256 INFO:tasks.rados.rados.0.smithi084.stdout:2099: done (1 left) 2024-09-18T14:08:38.256 INFO:tasks.rados.rados.0.smithi084.stdout:2100: done (0 left) 2024-09-18T14:08:38.256 INFO:tasks.rados.rados.0.smithi084.stdout:2101: snap_remove snap 215 2024-09-18T14:08:39.259 INFO:tasks.rados.rados.0.smithi084.stdout:2101: done (0 left) 2024-09-18T14:08:39.259 INFO:tasks.rados.rados.0.smithi084.stdout:2102: write (excl) oid 11 current snap is 216 2024-09-18T14:08:39.259 INFO:tasks.rados.rados.0.smithi084.stdout:2102: seq_num 744 ranges {504408=471497,1620603=557925} 2024-09-18T14:08:39.264 INFO:tasks.rados.rados.0.smithi084.stdout:2102: writing smithi08434663-11 from 504408 to 975905 tid 1 2024-09-18T14:08:39.271 INFO:tasks.rados.rados.0.smithi084.stdout:2102: writing smithi08434663-11 from 1620603 to 2178528 tid 2 2024-09-18T14:08:39.271 INFO:tasks.rados.rados.0.smithi084.stdout:2103: copy_from oid 25 from oid 5 current snap is 216 2024-09-18T14:08:39.272 INFO:tasks.rados.rados.0.smithi084.stdout:2104: rollback oid 16 current snap is 216 2024-09-18T14:08:39.272 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 16 to 214 2024-09-18T14:08:39.272 INFO:tasks.rados.rados.0.smithi084.stdout:2105: read oid 10 snap -1 2024-09-18T14:08:39.272 INFO:tasks.rados.rados.0.smithi084.stdout:2105: expect (ObjNum 538 snap 167 seq_num 538) 2024-09-18T14:08:39.272 INFO:tasks.rados.rados.0.smithi084.stdout:2106: delete oid 32 current snap is 216 2024-09-18T14:08:39.277 INFO:tasks.rados.rados.0.smithi084.stdout:2106: done (4 left) 2024-09-18T14:08:39.277 INFO:tasks.rados.rados.0.smithi084.stdout:2107: snap_remove snap 212 2024-09-18T14:08:40.268 INFO:tasks.rados.rados.0.smithi084.stdout:2107: done (4 left) 2024-09-18T14:08:40.268 INFO:tasks.rados.rados.0.smithi084.stdout:2108: write oid 4 current snap is 216 2024-09-18T14:08:40.268 INFO:tasks.rados.rados.0.smithi084.stdout:2108: seq_num 746 ranges {538022=738348,1737567=561594} 2024-09-18T14:08:40.275 INFO:tasks.rados.rados.0.smithi084.stdout:2108: writing smithi08434663-4 from 538022 to 1276370 tid 1 2024-09-18T14:08:40.281 INFO:tasks.rados.rados.0.smithi084.stdout:2108: writing smithi08434663-4 from 1737567 to 2299161 tid 2 2024-09-18T14:08:40.282 INFO:tasks.rados.rados.0.smithi084.stdout:2109: snap_create 2024-09-18T14:08:40.282 INFO:tasks.rados.rados.0.smithi084.stdout:2103: finishing copy_from to smithi08434663-25 2024-09-18T14:08:40.282 INFO:tasks.rados.rados.0.smithi084.stdout:2103: got expected ENOENT (src dne) 2024-09-18T14:08:40.282 INFO:tasks.rados.rados.0.smithi084.stdout:2103: finishing copy_from racing read to smithi08434663-25 2024-09-18T14:08:40.282 INFO:tasks.rados.rados.0.smithi084.stdout:2105: expect (ObjNum 538 snap 167 seq_num 538) 2024-09-18T14:08:40.404 INFO:tasks.rados.rados.0.smithi084.stdout:2102: finishing write tid 1 to smithi08434663-11 2024-09-18T14:08:40.404 INFO:tasks.rados.rados.0.smithi084.stdout:2102: finishing write tid 2 to smithi08434663-11 2024-09-18T14:08:40.404 INFO:tasks.rados.rados.0.smithi084.stdout:2102: finishing write tid 3 to smithi08434663-11 2024-09-18T14:08:40.404 INFO:tasks.rados.rados.0.smithi084.stdout:2104: finishing rollback tid 0 to smithi08434663-16 2024-09-18T14:08:40.404 INFO:tasks.rados.rados.0.smithi084.stdout:2102: finishing write tid 4 to smithi08434663-11 2024-09-18T14:08:40.404 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1036 (ObjNum 744 snap 216 seq_num 744) dirty exists 2024-09-18T14:08:40.404 INFO:tasks.rados.rados.0.smithi084.stdout:2102: left oid 11 (ObjNum 744 snap 216 seq_num 744) 2024-09-18T14:08:40.404 INFO:tasks.rados.rados.0.smithi084.stdout:2104: finishing rollback tid 1 to smithi08434663-16 2024-09-18T14:08:40.404 INFO:tasks.rados.rados.0.smithi084.stdout:2104: finishing rollback tid 2 to smithi08434663-16 2024-09-18T14:08:40.404 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 876 (ObjNum 706 snap 205 seq_num 706) dirty exists 2024-09-18T14:08:40.405 INFO:tasks.rados.rados.0.smithi084.stdout:2108: finishing write tid 1 to smithi08434663-4 2024-09-18T14:08:40.405 INFO:tasks.rados.rados.0.smithi084.stdout:2108: finishing write tid 2 to smithi08434663-4 2024-09-18T14:08:40.405 INFO:tasks.rados.rados.0.smithi084.stdout:2108: finishing write tid 3 to smithi08434663-4 2024-09-18T14:08:40.405 INFO:tasks.rados.rados.0.smithi084.stdout:2108: finishing write tid 4 to smithi08434663-4 2024-09-18T14:08:40.405 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 959 (ObjNum 746 snap 216 seq_num 746) dirty exists 2024-09-18T14:08:40.405 INFO:tasks.rados.rados.0.smithi084.stdout:2108: left oid 4 (ObjNum 746 snap 216 seq_num 746) 2024-09-18T14:08:41.269 INFO:tasks.rados.rados.0.smithi084.stdout:2102: done (5 left) 2024-09-18T14:08:41.269 INFO:tasks.rados.rados.0.smithi084.stdout:2103: done (4 left) 2024-09-18T14:08:41.269 INFO:tasks.rados.rados.0.smithi084.stdout:2104: done (3 left) 2024-09-18T14:08:41.269 INFO:tasks.rados.rados.0.smithi084.stdout:2105: done (2 left) 2024-09-18T14:08:41.270 INFO:tasks.rados.rados.0.smithi084.stdout:2108: done (1 left) 2024-09-18T14:08:41.270 INFO:tasks.rados.rados.0.smithi084.stdout:2109: done (0 left) 2024-09-18T14:08:41.270 INFO:tasks.rados.rados.0.smithi084.stdout:2110: snap_create 2024-09-18T14:08:42.245 INFO:tasks.rados.rados.0.smithi084.stdout:2110: done (0 left) 2024-09-18T14:08:42.246 INFO:tasks.rados.rados.0.smithi084.stdout:2111: snap_remove snap 214 2024-09-18T14:08:43.251 INFO:tasks.rados.rados.0.smithi084.stdout:2111: done (0 left) 2024-09-18T14:08:43.251 INFO:tasks.rados.rados.0.smithi084.stdout:2112: write oid 34 current snap is 218 2024-09-18T14:08:43.252 INFO:tasks.rados.rados.0.smithi084.stdout:2112: seq_num 747 ranges {554829=413597,1396049=705252} 2024-09-18T14:08:43.256 INFO:tasks.rados.rados.0.smithi084.stdout:2112: writing smithi08434663-34 from 554829 to 968426 tid 1 2024-09-18T14:08:43.263 INFO:tasks.rados.rados.0.smithi084.stdout:2112: writing smithi08434663-34 from 1396049 to 2101301 tid 2 2024-09-18T14:08:43.263 INFO:tasks.rados.rados.0.smithi084.stdout:2113: read oid 3 snap -1 2024-09-18T14:08:43.263 INFO:tasks.rados.rados.0.smithi084.stdout:2113: expect (ObjNum 732 snap 212 seq_num 732) 2024-09-18T14:08:43.263 INFO:tasks.rados.rados.0.smithi084.stdout:2114: read oid 30 snap 217 2024-09-18T14:08:43.263 INFO:tasks.rados.rados.0.smithi084.stdout:2114: expect (ObjNum 725 snap 210 seq_num 725) 2024-09-18T14:08:43.263 INFO:tasks.rados.rados.0.smithi084.stdout:2115: delete oid 14 current snap is 218 2024-09-18T14:08:43.265 INFO:tasks.rados.rados.0.smithi084.stdout:2115: done (3 left) 2024-09-18T14:08:43.265 INFO:tasks.rados.rados.0.smithi084.stdout:2116: snap_create 2024-09-18T14:08:43.266 INFO:tasks.rados.rados.0.smithi084.stdout:2114: expect (ObjNum 725 snap 210 seq_num 725) 2024-09-18T14:08:43.373 INFO:tasks.rados.rados.0.smithi084.stdout:2112: finishing write tid 1 to smithi08434663-34 2024-09-18T14:08:43.374 INFO:tasks.rados.rados.0.smithi084.stdout:2112: finishing write tid 2 to smithi08434663-34 2024-09-18T14:08:43.374 INFO:tasks.rados.rados.0.smithi084.stdout:2112: finishing write tid 3 to smithi08434663-34 2024-09-18T14:08:43.374 INFO:tasks.rados.rados.0.smithi084.stdout:2112: finishing write tid 4 to smithi08434663-34 2024-09-18T14:08:43.374 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 965 (ObjNum 747 snap 218 seq_num 747) dirty exists 2024-09-18T14:08:43.374 INFO:tasks.rados.rados.0.smithi084.stdout:2112: left oid 34 (ObjNum 747 snap 218 seq_num 747) 2024-09-18T14:08:43.374 INFO:tasks.rados.rados.0.smithi084.stdout:2113: expect (ObjNum 732 snap 212 seq_num 732) 2024-09-18T14:08:44.252 INFO:tasks.rados.rados.0.smithi084.stdout:2112: done (3 left) 2024-09-18T14:08:44.252 INFO:tasks.rados.rados.0.smithi084.stdout:2113: done (2 left) 2024-09-18T14:08:44.253 INFO:tasks.rados.rados.0.smithi084.stdout:2114: done (1 left) 2024-09-18T14:08:44.253 INFO:tasks.rados.rados.0.smithi084.stdout:2116: done (0 left) 2024-09-18T14:08:44.253 INFO:tasks.rados.rados.0.smithi084.stdout:2117: write (excl) oid 22 current snap is 219 2024-09-18T14:08:44.253 INFO:tasks.rados.rados.0.smithi084.stdout:2117: seq_num 749 ranges {588443=564095,1796660=708921} 2024-09-18T14:08:44.258 INFO:tasks.rados.rados.0.smithi084.stdout:2117: writing smithi08434663-22 from 588443 to 1152538 tid 1 2024-09-18T14:08:44.266 INFO:tasks.rados.rados.0.smithi084.stdout:2117: writing smithi08434663-22 from 1796660 to 2505581 tid 2 2024-09-18T14:08:44.266 INFO:tasks.rados.rados.0.smithi084.stdout:2118: write oid 40 current snap is 219 2024-09-18T14:08:44.266 INFO:tasks.rados.rados.0.smithi084.stdout:2117: finishing write tid 1 to smithi08434663-22 2024-09-18T14:08:44.267 INFO:tasks.rados.rados.0.smithi084.stdout:2118: seq_num 750 ranges {605250=639344,1855142=452579} 2024-09-18T14:08:44.273 INFO:tasks.rados.rados.0.smithi084.stdout:2118: writing smithi08434663-40 from 605250 to 1244594 tid 1 2024-09-18T14:08:44.279 INFO:tasks.rados.rados.0.smithi084.stdout:2118: writing smithi08434663-40 from 1855142 to 2307721 tid 2 2024-09-18T14:08:44.279 INFO:tasks.rados.rados.0.smithi084.stdout:2119: read oid 6 snap 218 2024-09-18T14:08:44.279 INFO:tasks.rados.rados.0.smithi084.stdout:2119: expect (ObjNum 722 snap 208 seq_num 722) 2024-09-18T14:08:44.280 INFO:tasks.rados.rados.0.smithi084.stdout:2120: snap_remove snap 213 2024-09-18T14:08:45.283 INFO:tasks.rados.rados.0.smithi084.stdout:2120: done (3 left) 2024-09-18T14:08:45.283 INFO:tasks.rados.rados.0.smithi084.stdout:2121: delete oid 15 current snap is 219 2024-09-18T14:08:45.283 INFO:tasks.rados.rados.0.smithi084.stdout:2117: finishing write tid 2 to smithi08434663-22 2024-09-18T14:08:45.283 INFO:tasks.rados.rados.0.smithi084.stdout:2117: finishing write tid 3 to smithi08434663-22 2024-09-18T14:08:45.283 INFO:tasks.rados.rados.0.smithi084.stdout:2117: finishing write tid 4 to smithi08434663-22 2024-09-18T14:08:45.283 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 752 (ObjNum 749 snap 219 seq_num 749) dirty exists 2024-09-18T14:08:45.283 INFO:tasks.rados.rados.0.smithi084.stdout:2117: left oid 22 (ObjNum 749 snap 219 seq_num 749) 2024-09-18T14:08:45.283 INFO:tasks.rados.rados.0.smithi084.stdout:2118: finishing write tid 1 to smithi08434663-40 2024-09-18T14:08:45.283 INFO:tasks.rados.rados.0.smithi084.stdout:2118: finishing write tid 2 to smithi08434663-40 2024-09-18T14:08:45.283 INFO:tasks.rados.rados.0.smithi084.stdout:2118: finishing write tid 3 to smithi08434663-40 2024-09-18T14:08:45.283 INFO:tasks.rados.rados.0.smithi084.stdout:2119: expect (ObjNum 722 snap 208 seq_num 722) 2024-09-18T14:08:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2118: finishing write tid 4 to smithi08434663-40 2024-09-18T14:08:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 863 (ObjNum 750 snap 219 seq_num 750) dirty exists 2024-09-18T14:08:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2118: left oid 40 (ObjNum 750 snap 219 seq_num 750) 2024-09-18T14:08:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2117: done (3 left) 2024-09-18T14:08:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2118: done (2 left) 2024-09-18T14:08:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2119: done (1 left) 2024-09-18T14:08:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2121: done (0 left) 2024-09-18T14:08:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2122: rollback oid 12 current snap is 219 2024-09-18T14:08:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 12 to 218 2024-09-18T14:08:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2123: read oid 14 snap 217 2024-09-18T14:08:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2123: expect deleted 2024-09-18T14:08:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2124: delete oid 7 current snap is 219 2024-09-18T14:08:45.392 INFO:tasks.rados.rados.0.smithi084.stdout:2124: done (2 left) 2024-09-18T14:08:45.393 INFO:tasks.rados.rados.0.smithi084.stdout:2125: write (excl) oid 30 current snap is 219 2024-09-18T14:08:45.393 INFO:tasks.rados.rados.0.smithi084.stdout:2125: seq_num 753 ranges {655671=581444,2030588=625083} 2024-09-18T14:08:45.399 INFO:tasks.rados.rados.0.smithi084.stdout:2125: writing smithi08434663-30 from 655671 to 1237115 tid 1 2024-09-18T14:08:45.407 INFO:tasks.rados.rados.0.smithi084.stdout:2125: writing smithi08434663-30 from 2030588 to 2655671 tid 2 2024-09-18T14:08:45.407 INFO:tasks.rados.rados.0.smithi084.stdout:2126: write oid 13 current snap is 219 2024-09-18T14:08:45.407 INFO:tasks.rados.rados.0.smithi084.stdout:2126: seq_num 754 ranges {672478=656693,2089070=459917} 2024-09-18T14:08:45.414 INFO:tasks.rados.rados.0.smithi084.stdout:2126: writing smithi08434663-13 from 672478 to 1329171 tid 1 2024-09-18T14:08:45.421 INFO:tasks.rados.rados.0.smithi084.stdout:2126: writing smithi08434663-13 from 2089070 to 2548987 tid 2 2024-09-18T14:08:45.421 INFO:tasks.rados.rados.0.smithi084.stdout:2127: rollback oid 40 current snap is 219 2024-09-18T14:08:45.421 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 40 to 216 2024-09-18T14:08:45.421 INFO:tasks.rados.rados.0.smithi084.stdout:2122: finishing rollback tid 0 to smithi08434663-12 2024-09-18T14:08:45.421 INFO:tasks.rados.rados.0.smithi084.stdout:2122: finishing rollback tid 1 to smithi08434663-12 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:2122: finishing rollback tid 2 to smithi08434663-12 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 1073 (ObjNum 697 snap 205 seq_num 697) dirty exists 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:2125: finishing write tid 1 to smithi08434663-30 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:2125: finishing write tid 2 to smithi08434663-30 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:2125: finishing write tid 3 to smithi08434663-30 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:2125: finishing write tid 4 to smithi08434663-30 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 852 (ObjNum 753 snap 219 seq_num 753) dirty exists 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:2125: left oid 30 (ObjNum 753 snap 219 seq_num 753) 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:2126: finishing write tid 1 to smithi08434663-13 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:2122: done (4 left) 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:2123: done (3 left) 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:2125: done (2 left) 2024-09-18T14:08:45.422 INFO:tasks.rados.rados.0.smithi084.stdout:2128: snap_create 2024-09-18T14:08:45.426 INFO:tasks.rados.rados.0.smithi084.stdout:2126: finishing write tid 2 to smithi08434663-13 2024-09-18T14:08:45.426 INFO:tasks.rados.rados.0.smithi084.stdout:2126: finishing write tid 3 to smithi08434663-13 2024-09-18T14:08:45.426 INFO:tasks.rados.rados.0.smithi084.stdout:2126: finishing write tid 4 to smithi08434663-13 2024-09-18T14:08:45.426 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 866 (ObjNum 754 snap 219 seq_num 754) dirty exists 2024-09-18T14:08:45.426 INFO:tasks.rados.rados.0.smithi084.stdout:2126: left oid 13 (ObjNum 754 snap 219 seq_num 754) 2024-09-18T14:08:45.426 INFO:tasks.rados.rados.0.smithi084.stdout:2127: finishing rollback tid 0 to smithi08434663-40 2024-09-18T14:08:45.426 INFO:tasks.rados.rados.0.smithi084.stdout:2127: finishing rollback tid 1 to smithi08434663-40 2024-09-18T14:08:45.426 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 868 (ObjNum 3985 snap 0 seq_num 3269436144) dirty dne 2024-09-18T14:08:46.288 INFO:tasks.rados.rados.0.smithi084.stdout:2126: done (2 left) 2024-09-18T14:08:46.288 INFO:tasks.rados.rados.0.smithi084.stdout:2127: done (1 left) 2024-09-18T14:08:46.288 INFO:tasks.rados.rados.0.smithi084.stdout:2128: done (0 left) 2024-09-18T14:08:46.288 INFO:tasks.rados.rados.0.smithi084.stdout:2129: snap_remove snap 217 2024-09-18T14:08:47.249 INFO:tasks.rados.rados.0.smithi084.stdout:2129: done (0 left) 2024-09-18T14:08:47.249 INFO:tasks.rados.rados.0.smithi084.stdout:2130: delete oid 28 current snap is 220 2024-09-18T14:08:47.267 INFO:tasks.rados.rados.0.smithi084.stdout:2130: done (0 left) 2024-09-18T14:08:47.267 INFO:tasks.rados.rados.0.smithi084.stdout:2131: read oid 7 snap -1 2024-09-18T14:08:47.267 INFO:tasks.rados.rados.0.smithi084.stdout:2131: expect deleted 2024-09-18T14:08:47.267 INFO:tasks.rados.rados.0.smithi084.stdout:2132: write oid 13 current snap is 220 2024-09-18T14:08:47.267 INFO:tasks.rados.rados.0.smithi084.stdout:2132: seq_num 756 ranges {706092=407191,1806034=463586} 2024-09-18T14:08:47.273 INFO:tasks.rados.rados.0.smithi084.stdout:2132: writing smithi08434663-13 from 706092 to 1113283 tid 1 2024-09-18T14:08:47.279 INFO:tasks.rados.rados.0.smithi084.stdout:2132: writing smithi08434663-13 from 1806034 to 2269620 tid 2 2024-09-18T14:08:47.280 INFO:tasks.rados.rados.0.smithi084.stdout:2133: read oid 3 snap 218 2024-09-18T14:08:47.280 INFO:tasks.rados.rados.0.smithi084.stdout:2133: expect (ObjNum 732 snap 212 seq_num 732) 2024-09-18T14:08:47.280 INFO:tasks.rados.rados.0.smithi084.stdout:2134: read oid 44 snap -1 2024-09-18T14:08:47.280 INFO:tasks.rados.rados.0.smithi084.stdout:2134: expect deleted 2024-09-18T14:08:47.280 INFO:tasks.rados.rados.0.smithi084.stdout:2135: snap_create 2024-09-18T14:08:47.280 INFO:tasks.rados.rados.0.smithi084.stdout:2132: finishing write tid 1 to smithi08434663-13 2024-09-18T14:08:47.285 INFO:tasks.rados.rados.0.smithi084.stdout:2133: expect (ObjNum 732 snap 212 seq_num 732) 2024-09-18T14:08:47.412 INFO:tasks.rados.rados.0.smithi084.stdout:2132: finishing write tid 2 to smithi08434663-13 2024-09-18T14:08:47.413 INFO:tasks.rados.rados.0.smithi084.stdout:2132: finishing write tid 3 to smithi08434663-13 2024-09-18T14:08:47.413 INFO:tasks.rados.rados.0.smithi084.stdout:2132: finishing write tid 4 to smithi08434663-13 2024-09-18T14:08:47.413 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 871 (ObjNum 756 snap 220 seq_num 756) dirty exists 2024-09-18T14:08:47.413 INFO:tasks.rados.rados.0.smithi084.stdout:2132: left oid 13 (ObjNum 756 snap 220 seq_num 756) 2024-09-18T14:08:48.255 INFO:tasks.rados.rados.0.smithi084.stdout:2131: done (4 left) 2024-09-18T14:08:48.255 INFO:tasks.rados.rados.0.smithi084.stdout:2132: done (3 left) 2024-09-18T14:08:48.255 INFO:tasks.rados.rados.0.smithi084.stdout:2133: done (2 left) 2024-09-18T14:08:48.256 INFO:tasks.rados.rados.0.smithi084.stdout:2134: done (1 left) 2024-09-18T14:08:48.256 INFO:tasks.rados.rados.0.smithi084.stdout:2135: done (0 left) 2024-09-18T14:08:48.256 INFO:tasks.rados.rados.0.smithi084.stdout:2136: read oid 39 snap 219 2024-09-18T14:08:48.256 INFO:tasks.rados.rados.0.smithi084.stdout:2136: expect (ObjNum 538 snap 167 seq_num 538) 2024-09-18T14:08:48.256 INFO:tasks.rados.rados.0.smithi084.stdout:2137: read oid 18 snap -1 2024-09-18T14:08:48.256 INFO:tasks.rados.rados.0.smithi084.stdout:2137: expect (ObjNum 734 snap 213 seq_num 734) 2024-09-18T14:08:48.256 INFO:tasks.rados.rados.0.smithi084.stdout:2138: read oid 31 snap -1 2024-09-18T14:08:48.256 INFO:tasks.rados.rados.0.smithi084.stdout:2138: expect (ObjNum 713 snap 207 seq_num 713) 2024-09-18T14:08:48.256 INFO:tasks.rados.rados.0.smithi084.stdout:2139: delete oid 7 current snap is 221 2024-09-18T14:08:48.258 INFO:tasks.rados.rados.0.smithi084.stdout:2139: done (3 left) 2024-09-18T14:08:48.258 INFO:tasks.rados.rados.0.smithi084.stdout:2140: write oid 44 current snap is 221 2024-09-18T14:08:48.258 INFO:tasks.rados.rados.0.smithi084.stdout:2140: seq_num 758 ranges {739706=557689,1806645=467255} 2024-09-18T14:08:48.265 INFO:tasks.rados.rados.0.smithi084.stdout:2140: writing smithi08434663-44 from 739706 to 1297395 tid 1 2024-09-18T14:08:48.272 INFO:tasks.rados.rados.0.smithi084.stdout:2140: writing smithi08434663-44 from 1806645 to 2273900 tid 2 2024-09-18T14:08:48.272 INFO:tasks.rados.rados.0.smithi084.stdout:2141: read oid 43 snap -1 2024-09-18T14:08:48.272 INFO:tasks.rados.rados.0.smithi084.stdout:2141: expect deleted 2024-09-18T14:08:48.272 INFO:tasks.rados.rados.0.smithi084.stdout:2142: rollback oid 40 current snap is 221 2024-09-18T14:08:48.272 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 40 to 219 2024-09-18T14:08:48.272 INFO:tasks.rados.rados.0.smithi084.stdout:2136: expect (ObjNum 538 snap 167 seq_num 538) 2024-09-18T14:08:48.395 INFO:tasks.rados.rados.0.smithi084.stdout:2138: expect (ObjNum 713 snap 207 seq_num 713) 2024-09-18T14:08:48.597 INFO:tasks.rados.rados.0.smithi084.stdout:2137: expect (ObjNum 734 snap 213 seq_num 734) 2024-09-18T14:08:48.701 INFO:tasks.rados.rados.0.smithi084.stdout:2142: finishing rollback tid 1 to smithi08434663-40 2024-09-18T14:08:48.701 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 0 (ObjNum 241 snap 0 seq_num 2495221232) dirty dne 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:2140: finishing write tid 1 to smithi08434663-44 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:2140: finishing write tid 2 to smithi08434663-44 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:2140: finishing write tid 3 to smithi08434663-44 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:2136: done (5 left) 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:2137: done (4 left) 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:2138: done (3 left) 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:2141: done (2 left) 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:2142: done (1 left) 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:2143: write (excl) oid 19 current snap is 221 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:2140: finishing write tid 4 to smithi08434663-44 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 932 (ObjNum 758 snap 221 seq_num 758) dirty exists 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:2140: left oid 44 (ObjNum 758 snap 221 seq_num 758) 2024-09-18T14:08:48.702 INFO:tasks.rados.rados.0.smithi084.stdout:2143: seq_num 759 ranges {756513=632938,1865127=727266} 2024-09-18T14:08:48.708 INFO:tasks.rados.rados.0.smithi084.stdout:2143: writing smithi08434663-19 from 756513 to 1389451 tid 1 2024-09-18T14:08:48.715 INFO:tasks.rados.rados.0.smithi084.stdout:2143: writing smithi08434663-19 from 1865127 to 2592393 tid 2 2024-09-18T14:08:48.715 INFO:tasks.rados.rados.0.smithi084.stdout:2140: done (1 left) 2024-09-18T14:08:48.715 INFO:tasks.rados.rados.0.smithi084.stdout:2144: delete oid 8 current snap is 221 2024-09-18T14:08:48.715 INFO:tasks.rados.rados.0.smithi084.stdout:2143: finishing write tid 1 to smithi08434663-19 2024-09-18T14:08:48.718 INFO:tasks.rados.rados.0.smithi084.stdout:2144: done (1 left) 2024-09-18T14:08:48.718 INFO:tasks.rados.rados.0.smithi084.stdout:2145: copy_from oid 45 from oid 14 current snap is 221 2024-09-18T14:08:48.718 INFO:tasks.rados.rados.0.smithi084.stdout:2146: write oid 39 current snap is 221 2024-09-18T14:08:48.718 INFO:tasks.rados.rados.0.smithi084.stdout:2146: seq_num 761 ranges {790127=499789,1698444=730935} 2024-09-18T14:08:48.725 INFO:tasks.rados.rados.0.smithi084.stdout:2146: writing smithi08434663-39 from 790127 to 1289916 tid 1 2024-09-18T14:08:48.732 INFO:tasks.rados.rados.0.smithi084.stdout:2146: writing smithi08434663-39 from 1698444 to 2429379 tid 2 2024-09-18T14:08:48.732 INFO:tasks.rados.rados.0.smithi084.stdout:2147: snap_create 2024-09-18T14:08:48.732 INFO:tasks.rados.rados.0.smithi084.stdout:2143: finishing write tid 2 to smithi08434663-19 2024-09-18T14:08:48.732 INFO:tasks.rados.rados.0.smithi084.stdout:2143: finishing write tid 3 to smithi08434663-19 2024-09-18T14:08:48.732 INFO:tasks.rados.rados.0.smithi084.stdout:2145: finishing copy_from to smithi08434663-45 2024-09-18T14:08:48.732 INFO:tasks.rados.rados.0.smithi084.stdout:2145: got expected ENOENT (src dne) 2024-09-18T14:08:48.732 INFO:tasks.rados.rados.0.smithi084.stdout:2143: finishing write tid 4 to smithi08434663-19 2024-09-18T14:08:48.733 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1081 (ObjNum 759 snap 221 seq_num 759) dirty exists 2024-09-18T14:08:48.733 INFO:tasks.rados.rados.0.smithi084.stdout:2143: left oid 19 (ObjNum 759 snap 221 seq_num 759) 2024-09-18T14:08:48.733 INFO:tasks.rados.rados.0.smithi084.stdout:2145: finishing copy_from racing read to smithi08434663-45 2024-09-18T14:08:48.733 INFO:tasks.rados.rados.0.smithi084.stdout:2146: finishing write tid 1 to smithi08434663-39 2024-09-18T14:08:48.735 INFO:tasks.rados.rados.0.smithi084.stdout:2146: finishing write tid 2 to smithi08434663-39 2024-09-18T14:08:48.735 INFO:tasks.rados.rados.0.smithi084.stdout:2146: finishing write tid 3 to smithi08434663-39 2024-09-18T14:08:48.735 INFO:tasks.rados.rados.0.smithi084.stdout:2146: finishing write tid 4 to smithi08434663-39 2024-09-18T14:08:48.735 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 1039 (ObjNum 761 snap 221 seq_num 761) dirty exists 2024-09-18T14:08:48.735 INFO:tasks.rados.rados.0.smithi084.stdout:2146: left oid 39 (ObjNum 761 snap 221 seq_num 761) 2024-09-18T14:08:49.292 INFO:tasks.rados.rados.0.smithi084.stdout:2143: done (3 left) 2024-09-18T14:08:49.292 INFO:tasks.rados.rados.0.smithi084.stdout:2145: done (2 left) 2024-09-18T14:08:49.292 INFO:tasks.rados.rados.0.smithi084.stdout:2146: done (1 left) 2024-09-18T14:08:49.292 INFO:tasks.rados.rados.0.smithi084.stdout:2147: done (0 left) 2024-09-18T14:08:49.292 INFO:tasks.rados.rados.0.smithi084.stdout:2148: write (excl) oid 18 current snap is 222 2024-09-18T14:08:49.292 INFO:tasks.rados.rados.0.smithi084.stdout:2148: seq_num 762 ranges {406934=575038,1640573=474593,2642791=164143} 2024-09-18T14:08:49.297 INFO:tasks.rados.rados.0.smithi084.stdout:2148: writing smithi08434663-18 from 406934 to 981972 tid 1 2024-09-18T14:08:49.303 INFO:tasks.rados.rados.0.smithi084.stdout:2148: writing smithi08434663-18 from 1640573 to 2115166 tid 2 2024-09-18T14:08:49.308 INFO:tasks.rados.rados.0.smithi084.stdout:2148: writing smithi08434663-18 from 2642791 to 2806934 tid 3 2024-09-18T14:08:49.308 INFO:tasks.rados.rados.0.smithi084.stdout:2149: read oid 17 snap -1 2024-09-18T14:08:49.308 INFO:tasks.rados.rados.0.smithi084.stdout:2149: expect (ObjNum 638 snap 189 seq_num 638) 2024-09-18T14:08:49.308 INFO:tasks.rados.rados.0.smithi084.stdout:2150: write oid 8 current snap is 222 2024-09-18T14:08:49.308 INFO:tasks.rados.rados.0.smithi084.stdout:2148: finishing write tid 1 to smithi08434663-18 2024-09-18T14:08:49.308 INFO:tasks.rados.rados.0.smithi084.stdout:2148: finishing write tid 2 to smithi08434663-18 2024-09-18T14:08:49.308 INFO:tasks.rados.rados.0.smithi084.stdout:2150: seq_num 763 ranges {423741=650287,1699055=618251} 2024-09-18T14:08:49.313 INFO:tasks.rados.rados.0.smithi084.stdout:2150: writing smithi08434663-8 from 423741 to 1074028 tid 1 2024-09-18T14:08:49.320 INFO:tasks.rados.rados.0.smithi084.stdout:2150: writing smithi08434663-8 from 1699055 to 2317306 tid 2 2024-09-18T14:08:49.321 INFO:tasks.rados.rados.0.smithi084.stdout:2151: read oid 48 snap -1 2024-09-18T14:08:49.321 INFO:tasks.rados.rados.0.smithi084.stdout:2151: expect (ObjNum 703 snap 205 seq_num 703) 2024-09-18T14:08:49.321 INFO:tasks.rados.rados.0.smithi084.stdout:2152: snap_create 2024-09-18T14:08:49.321 INFO:tasks.rados.rados.0.smithi084.stdout:2148: finishing write tid 3 to smithi08434663-18 2024-09-18T14:08:49.321 INFO:tasks.rados.rados.0.smithi084.stdout:2148: finishing write tid 4 to smithi08434663-18 2024-09-18T14:08:49.321 INFO:tasks.rados.rados.0.smithi084.stdout:2148: finishing write tid 5 to smithi08434663-18 2024-09-18T14:08:49.321 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 756 (ObjNum 762 snap 222 seq_num 762) dirty exists 2024-09-18T14:08:49.321 INFO:tasks.rados.rados.0.smithi084.stdout:2148: left oid 18 (ObjNum 762 snap 222 seq_num 762) 2024-09-18T14:08:49.321 INFO:tasks.rados.rados.0.smithi084.stdout:2149: expect (ObjNum 638 snap 189 seq_num 638) 2024-09-18T14:08:49.410 INFO:tasks.rados.rados.0.smithi084.stdout:2150: finishing write tid 1 to smithi08434663-8 2024-09-18T14:08:49.410 INFO:tasks.rados.rados.0.smithi084.stdout:2150: finishing write tid 2 to smithi08434663-8 2024-09-18T14:08:49.411 INFO:tasks.rados.rados.0.smithi084.stdout:2150: finishing write tid 3 to smithi08434663-8 2024-09-18T14:08:49.411 INFO:tasks.rados.rados.0.smithi084.stdout:2150: finishing write tid 4 to smithi08434663-8 2024-09-18T14:08:49.411 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 634 (ObjNum 763 snap 222 seq_num 763) dirty exists 2024-09-18T14:08:49.411 INFO:tasks.rados.rados.0.smithi084.stdout:2150: left oid 8 (ObjNum 763 snap 222 seq_num 763) 2024-09-18T14:08:49.411 INFO:tasks.rados.rados.0.smithi084.stdout:2151: expect (ObjNum 703 snap 205 seq_num 703) 2024-09-18T14:08:50.297 INFO:tasks.rados.rados.0.smithi084.stdout:2148: done (4 left) 2024-09-18T14:08:50.298 INFO:tasks.rados.rados.0.smithi084.stdout:2149: done (3 left) 2024-09-18T14:08:50.298 INFO:tasks.rados.rados.0.smithi084.stdout:2150: done (2 left) 2024-09-18T14:08:50.298 INFO:tasks.rados.rados.0.smithi084.stdout:2151: done (1 left) 2024-09-18T14:08:50.298 INFO:tasks.rados.rados.0.smithi084.stdout:2152: done (0 left) 2024-09-18T14:08:50.298 INFO:tasks.rados.rados.0.smithi084.stdout:2153: delete oid 15 current snap is 223 2024-09-18T14:08:50.301 INFO:tasks.rados.rados.0.smithi084.stdout:2153: done (0 left) 2024-09-18T14:08:50.301 INFO:tasks.rados.rados.0.smithi084.stdout:2154: write oid 28 current snap is 223 2024-09-18T14:08:50.301 INFO:tasks.rados.rados.0.smithi084.stdout:2154: seq_num 765 ranges {457355=400785,1416019=621920,2725060=132295} 2024-09-18T14:08:50.305 INFO:tasks.rados.rados.0.smithi084.stdout:2154: writing smithi08434663-28 from 457355 to 858140 tid 1 2024-09-18T14:08:50.312 INFO:tasks.rados.rados.0.smithi084.stdout:2154: writing smithi08434663-28 from 1416019 to 2037939 tid 2 2024-09-18T14:08:50.317 INFO:tasks.rados.rados.0.smithi084.stdout:2154: writing smithi08434663-28 from 2725060 to 2857355 tid 3 2024-09-18T14:08:50.317 INFO:tasks.rados.rados.0.smithi084.stdout:2155: delete oid 41 current snap is 223 2024-09-18T14:08:50.317 INFO:tasks.rados.rados.0.smithi084.stdout:2154: finishing write tid 1 to smithi08434663-28 2024-09-18T14:08:50.317 INFO:tasks.rados.rados.0.smithi084.stdout:2154: finishing write tid 2 to smithi08434663-28 2024-09-18T14:08:50.319 INFO:tasks.rados.rados.0.smithi084.stdout:2154: finishing write tid 3 to smithi08434663-28 2024-09-18T14:08:50.319 INFO:tasks.rados.rados.0.smithi084.stdout:2154: finishing write tid 4 to smithi08434663-28 2024-09-18T14:08:50.319 INFO:tasks.rados.rados.0.smithi084.stdout:2154: finishing write tid 5 to smithi08434663-28 2024-09-18T14:08:50.319 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1085 (ObjNum 765 snap 223 seq_num 765) dirty exists 2024-09-18T14:08:50.320 INFO:tasks.rados.rados.0.smithi084.stdout:2154: left oid 28 (ObjNum 765 snap 223 seq_num 765) 2024-09-18T14:08:50.320 INFO:tasks.rados.rados.0.smithi084.stdout:2154: done (1 left) 2024-09-18T14:08:50.320 INFO:tasks.rados.rados.0.smithi084.stdout:2155: done (0 left) 2024-09-18T14:08:50.321 INFO:tasks.rados.rados.0.smithi084.stdout:2156: snap_remove snap 216 2024-09-18T14:08:51.304 INFO:tasks.rados.rados.0.smithi084.stdout:2156: done (0 left) 2024-09-18T14:08:51.304 INFO:tasks.rados.rados.0.smithi084.stdout:2157: rollback oid 30 current snap is 223 2024-09-18T14:08:51.304 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 30 to 219 2024-09-18T14:08:51.304 INFO:tasks.rados.rados.0.smithi084.stdout:2158: read oid 23 snap -1 2024-09-18T14:08:51.304 INFO:tasks.rados.rados.0.smithi084.stdout:2158: expect (ObjNum 737 snap 213 seq_num 737) 2024-09-18T14:08:51.304 INFO:tasks.rados.rados.0.smithi084.stdout:2159: write (excl) oid 22 current snap is 223 2024-09-18T14:08:51.305 INFO:tasks.rados.rados.0.smithi084.stdout:2159: seq_num 767 ranges {490969=551283,1816630=625589} 2024-09-18T14:08:51.310 INFO:tasks.rados.rados.0.smithi084.stdout:2159: writing smithi08434663-22 from 490969 to 1042252 tid 1 2024-09-18T14:08:51.318 INFO:tasks.rados.rados.0.smithi084.stdout:2159: writing smithi08434663-22 from 1816630 to 2442219 tid 2 2024-09-18T14:08:51.318 INFO:tasks.rados.rados.0.smithi084.stdout:2160: read oid 16 snap -1 2024-09-18T14:08:51.318 INFO:tasks.rados.rados.0.smithi084.stdout:2160: expect (ObjNum 706 snap 205 seq_num 706) 2024-09-18T14:08:51.318 INFO:tasks.rados.rados.0.smithi084.stdout:2161: write (excl) oid 44 current snap is 223 2024-09-18T14:08:51.318 INFO:tasks.rados.rados.0.smithi084.stdout:2157: finishing rollback tid 0 to smithi08434663-30 2024-09-18T14:08:51.320 INFO:tasks.rados.rados.0.smithi084.stdout:2157: finishing rollback tid 1 to smithi08434663-30 2024-09-18T14:08:51.320 INFO:tasks.rados.rados.0.smithi084.stdout:2157: finishing rollback tid 2 to smithi08434663-30 2024-09-18T14:08:51.320 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 855 (ObjNum 753 snap 219 seq_num 753) dirty exists 2024-09-18T14:08:51.320 INFO:tasks.rados.rados.0.smithi084.stdout:2161: seq_num 768 ranges {507776=742885,1991465=485600} 2024-09-18T14:08:51.325 INFO:tasks.rados.rados.0.smithi084.stdout:2161: writing smithi08434663-44 from 507776 to 1250661 tid 1 2024-09-18T14:08:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:2161: writing smithi08434663-44 from 1991465 to 2477065 tid 2 2024-09-18T14:08:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:2157: done (4 left) 2024-09-18T14:08:51.332 INFO:tasks.rados.rados.0.smithi084.stdout:2162: copy_from oid 6 from oid 48 current snap is 223 2024-09-18T14:08:51.333 INFO:tasks.rados.rados.0.smithi084.stdout:2158: expect (ObjNum 737 snap 213 seq_num 737) 2024-09-18T14:08:51.411 INFO:tasks.rados.rados.0.smithi084.stdout:2159: finishing write tid 1 to smithi08434663-22 2024-09-18T14:08:51.411 INFO:tasks.rados.rados.0.smithi084.stdout:2159: finishing write tid 2 to smithi08434663-22 2024-09-18T14:08:51.411 INFO:tasks.rados.rados.0.smithi084.stdout:2159: finishing write tid 3 to smithi08434663-22 2024-09-18T14:08:51.411 INFO:tasks.rados.rados.0.smithi084.stdout:2159: finishing write tid 4 to smithi08434663-22 2024-09-18T14:08:51.411 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 759 (ObjNum 767 snap 223 seq_num 767) dirty exists 2024-09-18T14:08:51.411 INFO:tasks.rados.rados.0.smithi084.stdout:2159: left oid 22 (ObjNum 767 snap 223 seq_num 767) 2024-09-18T14:08:51.411 INFO:tasks.rados.rados.0.smithi084.stdout:2158: done (4 left) 2024-09-18T14:08:51.411 INFO:tasks.rados.rados.0.smithi084.stdout:2159: done (3 left) 2024-09-18T14:08:51.411 INFO:tasks.rados.rados.0.smithi084.stdout:2163: snap_remove snap 219 2024-09-18T14:08:52.249 INFO:tasks.rados.rados.0.smithi084.stdout:2163: done (3 left) 2024-09-18T14:08:52.250 INFO:tasks.rados.rados.0.smithi084.stdout:2164: copy_from oid 31 from oid 40 current snap is 223 2024-09-18T14:08:52.250 INFO:tasks.rados.rados.0.smithi084.stdout:2160: expect (ObjNum 706 snap 205 seq_num 706) 2024-09-18T14:08:52.374 INFO:tasks.rados.rados.0.smithi084.stdout:2161: finishing write tid 1 to smithi08434663-44 2024-09-18T14:08:52.374 INFO:tasks.rados.rados.0.smithi084.stdout:2160: done (3 left) 2024-09-18T14:08:52.374 INFO:tasks.rados.rados.0.smithi084.stdout:2165: copy_from oid 35 from oid 50 current snap is 223 2024-09-18T14:08:52.374 INFO:tasks.rados.rados.0.smithi084.stdout:2161: finishing write tid 2 to smithi08434663-44 2024-09-18T14:08:52.374 INFO:tasks.rados.rados.0.smithi084.stdout:2161: finishing write tid 3 to smithi08434663-44 2024-09-18T14:08:52.374 INFO:tasks.rados.rados.0.smithi084.stdout:2161: finishing write tid 4 to smithi08434663-44 2024-09-18T14:08:52.374 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 935 (ObjNum 768 snap 223 seq_num 768) dirty exists 2024-09-18T14:08:52.374 INFO:tasks.rados.rados.0.smithi084.stdout:2161: left oid 44 (ObjNum 768 snap 223 seq_num 768) 2024-09-18T14:08:52.374 INFO:tasks.rados.rados.0.smithi084.stdout:2162: finishing copy_from racing read to smithi08434663-6 2024-09-18T14:08:52.374 INFO:tasks.rados.rados.0.smithi084.stdout:2162: finishing copy_from to smithi08434663-6 2024-09-18T14:08:52.375 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 607 (ObjNum 703 snap 205 seq_num 703) dirty exists 2024-09-18T14:08:52.375 INFO:tasks.rados.rados.0.smithi084.stdout:2164: finishing copy_from to smithi08434663-31 2024-09-18T14:08:52.375 INFO:tasks.rados.rados.0.smithi084.stdout:2164: got expected ENOENT (src dne) 2024-09-18T14:08:52.375 INFO:tasks.rados.rados.0.smithi084.stdout:2164: finishing copy_from racing read to smithi08434663-31 2024-09-18T14:08:52.375 INFO:tasks.rados.rados.0.smithi084.stdout:2161: done (3 left) 2024-09-18T14:08:52.375 INFO:tasks.rados.rados.0.smithi084.stdout:2162: done (2 left) 2024-09-18T14:08:52.375 INFO:tasks.rados.rados.0.smithi084.stdout:2164: done (1 left) 2024-09-18T14:08:52.375 INFO:tasks.rados.rados.0.smithi084.stdout:2166: copy_from oid 10 from oid 46 current snap is 223 2024-09-18T14:08:52.375 INFO:tasks.rados.rados.0.smithi084.stdout:2167: write (excl) oid 17 current snap is 223 2024-09-18T14:08:52.375 INFO:tasks.rados.rados.0.smithi084.stdout:2167: seq_num 769 ranges {524583=418134,1649947=629258,2834752=89831} 2024-09-18T14:08:52.378 INFO:tasks.rados.rados.0.smithi084.stdout:2167: writing smithi08434663-17 from 524583 to 942717 tid 1 2024-09-18T14:08:52.386 INFO:tasks.rados.rados.0.smithi084.stdout:2167: writing smithi08434663-17 from 1649947 to 2279205 tid 2 2024-09-18T14:08:52.390 INFO:tasks.rados.rados.0.smithi084.stdout:2167: writing smithi08434663-17 from 2834752 to 2924583 tid 3 2024-09-18T14:08:52.390 INFO:tasks.rados.rados.0.smithi084.stdout:2168: write (excl) oid 3 current snap is 223 2024-09-18T14:08:52.391 INFO:tasks.rados.rados.0.smithi084.stdout:2167: finishing write tid 1 to smithi08434663-17 2024-09-18T14:08:52.391 INFO:tasks.rados.rados.0.smithi084.stdout:2165: finishing copy_from to smithi08434663-35 2024-09-18T14:08:52.391 INFO:tasks.rados.rados.0.smithi084.stdout:2165: got expected ENOENT (src dne) 2024-09-18T14:08:52.391 INFO:tasks.rados.rados.0.smithi084.stdout:2165: finishing copy_from racing read to smithi08434663-35 2024-09-18T14:08:52.391 INFO:tasks.rados.rados.0.smithi084.stdout:2167: finishing write tid 2 to smithi08434663-17 2024-09-18T14:08:52.391 INFO:tasks.rados.rados.0.smithi084.stdout:2168: seq_num 770 ranges {541390=493383,1592076=489269,2862175=79215} 2024-09-18T14:08:52.396 INFO:tasks.rados.rados.0.smithi084.stdout:2168: writing smithi08434663-3 from 541390 to 1034773 tid 1 2024-09-18T14:08:52.401 INFO:tasks.rados.rados.0.smithi084.stdout:2168: writing smithi08434663-3 from 1592076 to 2081345 tid 2 2024-09-18T14:08:52.407 INFO:tasks.rados.rados.0.smithi084.stdout:2168: writing smithi08434663-3 from 2862175 to 2941390 tid 3 2024-09-18T14:08:52.407 INFO:tasks.rados.rados.0.smithi084.stdout:2165: done (3 left) 2024-09-18T14:08:52.407 INFO:tasks.rados.rados.0.smithi084.stdout:2169: write oid 44 current snap is 223 2024-09-18T14:08:52.407 INFO:tasks.rados.rados.0.smithi084.stdout:2166: finishing copy_from racing read to smithi08434663-10 2024-09-18T14:08:52.407 INFO:tasks.rados.rados.0.smithi084.stdout:2169: seq_num 771 ranges {558197=568632,1650558=632927,2773245=184952} 2024-09-18T14:08:52.412 INFO:tasks.rados.rados.0.smithi084.stdout:2169: writing smithi08434663-44 from 558197 to 1126829 tid 1 2024-09-18T14:08:52.419 INFO:tasks.rados.rados.0.smithi084.stdout:2169: writing smithi08434663-44 from 1650558 to 2283485 tid 2 2024-09-18T14:08:52.423 INFO:tasks.rados.rados.0.smithi084.stdout:2169: writing smithi08434663-44 from 2773245 to 2958197 tid 3 2024-09-18T14:08:52.423 INFO:tasks.rados.rados.0.smithi084.stdout:2170: read oid 22 snap 220 2024-09-18T14:08:52.423 INFO:tasks.rados.rados.0.smithi084.stdout:2170: expect (ObjNum 749 snap 219 seq_num 749) 2024-09-18T14:08:52.423 INFO:tasks.rados.rados.0.smithi084.stdout:2171: delete oid 37 current snap is 223 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:2167: finishing write tid 3 to smithi08434663-17 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:2167: finishing write tid 4 to smithi08434663-17 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:2167: finishing write tid 5 to smithi08434663-17 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 875 (ObjNum 769 snap 223 seq_num 769) dirty exists 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:2167: left oid 17 (ObjNum 769 snap 223 seq_num 769) 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:2166: finishing copy_from to smithi08434663-10 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 769 (ObjNum 545 snap 167 seq_num 545) dirty exists 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:2168: finishing write tid 1 to smithi08434663-3 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:2168: finishing write tid 2 to smithi08434663-3 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:2168: finishing write tid 3 to smithi08434663-3 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:2168: finishing write tid 4 to smithi08434663-3 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:2168: finishing write tid 5 to smithi08434663-3 2024-09-18T14:08:52.424 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 763 (ObjNum 770 snap 223 seq_num 770) dirty exists 2024-09-18T14:08:52.425 INFO:tasks.rados.rados.0.smithi084.stdout:2168: left oid 3 (ObjNum 770 snap 223 seq_num 770) 2024-09-18T14:08:52.425 INFO:tasks.rados.rados.0.smithi084.stdout:2169: finishing write tid 1 to smithi08434663-44 2024-09-18T14:08:52.425 INFO:tasks.rados.rados.0.smithi084.stdout:2166: done (5 left) 2024-09-18T14:08:52.425 INFO:tasks.rados.rados.0.smithi084.stdout:2167: done (4 left) 2024-09-18T14:08:52.426 INFO:tasks.rados.rados.0.smithi084.stdout:2168: done (3 left) 2024-09-18T14:08:52.426 INFO:tasks.rados.rados.0.smithi084.stdout:2171: done (2 left) 2024-09-18T14:08:52.426 INFO:tasks.rados.rados.0.smithi084.stdout:2172: copy_from oid 4 from oid 27 current snap is 223 2024-09-18T14:08:52.426 INFO:tasks.rados.rados.0.smithi084.stdout:2169: finishing write tid 2 to smithi08434663-44 2024-09-18T14:08:52.426 INFO:tasks.rados.rados.0.smithi084.stdout:2173: read oid 14 snap -1 2024-09-18T14:08:52.426 INFO:tasks.rados.rados.0.smithi084.stdout:2173: expect deleted 2024-09-18T14:08:52.426 INFO:tasks.rados.rados.0.smithi084.stdout:2174: snap_remove snap 218 2024-09-18T14:08:53.308 INFO:tasks.rados.rados.0.smithi084.stdout:2174: done (4 left) 2024-09-18T14:08:53.308 INFO:tasks.rados.rados.0.smithi084.stdout:2175: delete oid 5 current snap is 223 2024-09-18T14:08:53.309 INFO:tasks.rados.rados.0.smithi084.stdout:2169: finishing write tid 3 to smithi08434663-44 2024-09-18T14:08:53.309 INFO:tasks.rados.rados.0.smithi084.stdout:2169: finishing write tid 4 to smithi08434663-44 2024-09-18T14:08:53.309 INFO:tasks.rados.rados.0.smithi084.stdout:2169: finishing write tid 5 to smithi08434663-44 2024-09-18T14:08:53.309 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 939 (ObjNum 771 snap 223 seq_num 771) dirty exists 2024-09-18T14:08:53.309 INFO:tasks.rados.rados.0.smithi084.stdout:2169: left oid 44 (ObjNum 771 snap 223 seq_num 771) 2024-09-18T14:08:53.309 INFO:tasks.rados.rados.0.smithi084.stdout:2170: expect (ObjNum 749 snap 219 seq_num 749) 2024-09-18T14:08:53.460 INFO:tasks.rados.rados.0.smithi084.stdout:2172: finishing copy_from racing read to smithi08434663-4 2024-09-18T14:08:53.460 INFO:tasks.rados.rados.0.smithi084.stdout:2172: finishing copy_from to smithi08434663-4 2024-09-18T14:08:53.460 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 802 (ObjNum 723 snap 209 seq_num 723) dirty exists 2024-09-18T14:08:53.460 INFO:tasks.rados.rados.0.smithi084.stdout:2169: done (4 left) 2024-09-18T14:08:53.460 INFO:tasks.rados.rados.0.smithi084.stdout:2170: done (3 left) 2024-09-18T14:08:53.461 INFO:tasks.rados.rados.0.smithi084.stdout:2172: done (2 left) 2024-09-18T14:08:53.461 INFO:tasks.rados.rados.0.smithi084.stdout:2173: done (1 left) 2024-09-18T14:08:53.461 INFO:tasks.rados.rados.0.smithi084.stdout:2175: done (0 left) 2024-09-18T14:08:53.461 INFO:tasks.rados.rados.0.smithi084.stdout:2176: snap_remove snap 222 2024-09-18T14:08:54.314 INFO:tasks.rados.rados.0.smithi084.stdout:2176: done (0 left) 2024-09-18T14:08:54.314 INFO:tasks.rados.rados.0.smithi084.stdout:2177: copy_from oid 7 from oid 9 current snap is 223 2024-09-18T14:08:54.314 INFO:tasks.rados.rados.0.smithi084.stdout:2178: read oid 5 snap 220 2024-09-18T14:08:54.315 INFO:tasks.rados.rados.0.smithi084.stdout:2178: expect deleted 2024-09-18T14:08:54.315 INFO:tasks.rados.rados.0.smithi084.stdout:2179: snap_remove snap 221 2024-09-18T14:08:55.315 INFO:tasks.rados.rados.0.smithi084.stdout:2179: done (2 left) 2024-09-18T14:08:55.316 INFO:tasks.rados.rados.0.smithi084.stdout:2180: read oid 28 snap -1 2024-09-18T14:08:55.316 INFO:tasks.rados.rados.0.smithi084.stdout:2180: expect (ObjNum 765 snap 223 seq_num 765) 2024-09-18T14:08:55.316 INFO:tasks.rados.rados.0.smithi084.stdout:2181: read oid 45 snap -1 2024-09-18T14:08:55.316 INFO:tasks.rados.rados.0.smithi084.stdout:2181: expect (ObjNum 731 snap 212 seq_num 731) 2024-09-18T14:08:55.316 INFO:tasks.rados.rados.0.smithi084.stdout:2182: rollback oid 23 current snap is 223 2024-09-18T14:08:55.316 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 23 to 220 2024-09-18T14:08:55.316 INFO:tasks.rados.rados.0.smithi084.stdout:2177: finishing copy_from racing read to smithi08434663-7 2024-09-18T14:08:55.316 INFO:tasks.rados.rados.0.smithi084.stdout:2177: finishing copy_from to smithi08434663-7 2024-09-18T14:08:55.316 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 873 (ObjNum 741 snap 214 seq_num 741) dirty exists 2024-09-18T14:08:55.316 INFO:tasks.rados.rados.0.smithi084.stdout:2177: done (4 left) 2024-09-18T14:08:55.316 INFO:tasks.rados.rados.0.smithi084.stdout:2178: done (3 left) 2024-09-18T14:08:55.316 INFO:tasks.rados.rados.0.smithi084.stdout:2183: write oid 25 current snap is 223 2024-09-18T14:08:55.317 INFO:tasks.rados.rados.0.smithi084.stdout:2183: seq_num 774 ranges {608618=794379,2109651=780254} 2024-09-18T14:08:55.323 INFO:tasks.rados.rados.0.smithi084.stdout:2183: writing smithi08434663-25 from 608618 to 1402997 tid 1 2024-09-18T14:08:55.333 INFO:tasks.rados.rados.0.smithi084.stdout:2183: writing smithi08434663-25 from 2109651 to 2889905 tid 2 2024-09-18T14:08:55.333 INFO:tasks.rados.rados.0.smithi084.stdout:2184: rollback oid 10 current snap is 223 2024-09-18T14:08:55.333 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 10 to 220 2024-09-18T14:08:55.333 INFO:tasks.rados.rados.0.smithi084.stdout:2182: finishing rollback tid 0 to smithi08434663-23 2024-09-18T14:08:55.333 INFO:tasks.rados.rados.0.smithi084.stdout:2182: finishing rollback tid 1 to smithi08434663-23 2024-09-18T14:08:55.333 INFO:tasks.rados.rados.0.smithi084.stdout:2182: finishing rollback tid 2 to smithi08434663-23 2024-09-18T14:08:55.333 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 933 (ObjNum 737 snap 213 seq_num 737) dirty exists 2024-09-18T14:08:55.333 INFO:tasks.rados.rados.0.smithi084.stdout:2181: expect (ObjNum 731 snap 212 seq_num 731) 2024-09-18T14:08:55.408 INFO:tasks.rados.rados.0.smithi084.stdout:2180: expect (ObjNum 765 snap 223 seq_num 765) 2024-09-18T14:08:55.501 INFO:tasks.rados.rados.0.smithi084.stdout:2183: finishing write tid 1 to smithi08434663-25 2024-09-18T14:08:55.501 INFO:tasks.rados.rados.0.smithi084.stdout:2183: finishing write tid 2 to smithi08434663-25 2024-09-18T14:08:55.501 INFO:tasks.rados.rados.0.smithi084.stdout:2184: finishing rollback tid 0 to smithi08434663-10 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:2183: finishing write tid 3 to smithi08434663-25 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:2183: finishing write tid 4 to smithi08434663-25 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 646 (ObjNum 774 snap 223 seq_num 774) dirty exists 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:2183: left oid 25 (ObjNum 774 snap 223 seq_num 774) 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:2184: finishing rollback tid 1 to smithi08434663-10 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:2184: finishing rollback tid 2 to smithi08434663-10 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 819 (ObjNum 538 snap 167 seq_num 538) dirty exists 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:2180: done (4 left) 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:2181: done (3 left) 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:2182: done (2 left) 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:2183: done (1 left) 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:2184: done (0 left) 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:2185: write (excl) oid 41 current snap is 223 2024-09-18T14:08:55.502 INFO:tasks.rados.rados.0.smithi084.stdout:2185: seq_num 775 ranges {625425=469628,1768133=640265} 2024-09-18T14:08:55.506 INFO:tasks.rados.rados.0.smithi084.stdout:2185: writing smithi08434663-41 from 625425 to 1095053 tid 1 2024-09-18T14:08:55.514 INFO:tasks.rados.rados.0.smithi084.stdout:2185: writing smithi08434663-41 from 1768133 to 2408398 tid 2 2024-09-18T14:08:55.514 INFO:tasks.rados.rados.0.smithi084.stdout:2186: delete oid 21 current snap is 223 2024-09-18T14:08:55.514 INFO:tasks.rados.rados.0.smithi084.stdout:2185: finishing write tid 1 to smithi08434663-41 2024-09-18T14:08:55.518 INFO:tasks.rados.rados.0.smithi084.stdout:2185: finishing write tid 2 to smithi08434663-41 2024-09-18T14:08:55.518 INFO:tasks.rados.rados.0.smithi084.stdout:2185: finishing write tid 3 to smithi08434663-41 2024-09-18T14:08:55.519 INFO:tasks.rados.rados.0.smithi084.stdout:2185: finishing write tid 4 to smithi08434663-41 2024-09-18T14:08:55.519 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 996 (ObjNum 775 snap 223 seq_num 775) dirty exists 2024-09-18T14:08:55.519 INFO:tasks.rados.rados.0.smithi084.stdout:2185: left oid 41 (ObjNum 775 snap 223 seq_num 775) 2024-09-18T14:08:55.520 INFO:tasks.rados.rados.0.smithi084.stdout:2185: done (1 left) 2024-09-18T14:08:55.520 INFO:tasks.rados.rados.0.smithi084.stdout:2186: done (0 left) 2024-09-18T14:08:55.520 INFO:tasks.rados.rados.0.smithi084.stdout:2187: write oid 6 current snap is 223 2024-09-18T14:08:55.520 INFO:tasks.rados.rados.0.smithi084.stdout:2187: seq_num 777 ranges {659039=736479,2001450=643934} 2024-09-18T14:08:55.527 INFO:tasks.rados.rados.0.smithi084.stdout:2187: writing smithi08434663-6 from 659039 to 1395518 tid 1 2024-09-18T14:08:55.534 INFO:tasks.rados.rados.0.smithi084.stdout:2187: writing smithi08434663-6 from 2001450 to 2645384 tid 2 2024-09-18T14:08:55.534 INFO:tasks.rados.rados.0.smithi084.stdout:2188: read oid 8 snap 220 2024-09-18T14:08:55.534 INFO:tasks.rados.rados.0.smithi084.stdout:2188: expect (ObjNum 735 snap 213 seq_num 735) 2024-09-18T14:08:55.535 INFO:tasks.rados.rados.0.smithi084.stdout:2189: write (excl) oid 9 current snap is 223 2024-09-18T14:08:55.535 INFO:tasks.rados.rados.0.smithi084.stdout:2187: finishing write tid 1 to smithi08434663-6 2024-09-18T14:08:55.535 INFO:tasks.rados.rados.0.smithi084.stdout:2189: seq_num 778 ranges {675846=411728,1543579=787592,2848853=226993} 2024-09-18T14:08:55.540 INFO:tasks.rados.rados.0.smithi084.stdout:2189: writing smithi08434663-9 from 675846 to 1087574 tid 1 2024-09-18T14:08:55.547 INFO:tasks.rados.rados.0.smithi084.stdout:2189: writing smithi08434663-9 from 1543579 to 2331171 tid 2 2024-09-18T14:08:55.551 INFO:tasks.rados.rados.0.smithi084.stdout:2189: writing smithi08434663-9 from 2848853 to 3075846 tid 3 2024-09-18T14:08:55.551 INFO:tasks.rados.rados.0.smithi084.stdout:2190: snap_create 2024-09-18T14:08:55.552 INFO:tasks.rados.rados.0.smithi084.stdout:2187: finishing write tid 2 to smithi08434663-6 2024-09-18T14:08:55.552 INFO:tasks.rados.rados.0.smithi084.stdout:2187: finishing write tid 3 to smithi08434663-6 2024-09-18T14:08:55.552 INFO:tasks.rados.rados.0.smithi084.stdout:2187: finishing write tid 4 to smithi08434663-6 2024-09-18T14:08:55.552 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 936 (ObjNum 777 snap 223 seq_num 777) dirty exists 2024-09-18T14:08:55.552 INFO:tasks.rados.rados.0.smithi084.stdout:2187: left oid 6 (ObjNum 777 snap 223 seq_num 777) 2024-09-18T14:08:55.552 INFO:tasks.rados.rados.0.smithi084.stdout:2188: expect (ObjNum 735 snap 213 seq_num 735) 2024-09-18T14:08:55.696 INFO:tasks.rados.rados.0.smithi084.stdout:2189: finishing write tid 1 to smithi08434663-9 2024-09-18T14:08:55.696 INFO:tasks.rados.rados.0.smithi084.stdout:2189: finishing write tid 2 to smithi08434663-9 2024-09-18T14:08:55.696 INFO:tasks.rados.rados.0.smithi084.stdout:2189: finishing write tid 3 to smithi08434663-9 2024-09-18T14:08:55.696 INFO:tasks.rados.rados.0.smithi084.stdout:2189: finishing write tid 4 to smithi08434663-9 2024-09-18T14:08:55.696 INFO:tasks.rados.rados.0.smithi084.stdout:2189: finishing write tid 5 to smithi08434663-9 2024-09-18T14:08:55.697 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 881 (ObjNum 778 snap 223 seq_num 778) dirty exists 2024-09-18T14:08:55.697 INFO:tasks.rados.rados.0.smithi084.stdout:2189: left oid 9 (ObjNum 778 snap 223 seq_num 778) 2024-09-18T14:08:56.327 INFO:tasks.rados.rados.0.smithi084.stdout:2187: done (3 left) 2024-09-18T14:08:56.327 INFO:tasks.rados.rados.0.smithi084.stdout:2188: done (2 left) 2024-09-18T14:08:56.327 INFO:tasks.rados.rados.0.smithi084.stdout:2189: done (1 left) 2024-09-18T14:08:56.327 INFO:tasks.rados.rados.0.smithi084.stdout:2190: done (0 left) 2024-09-18T14:08:56.327 INFO:tasks.rados.rados.0.smithi084.stdout:2191: write (excl) oid 37 current snap is 224 2024-09-18T14:08:56.327 INFO:tasks.rados.rados.0.smithi084.stdout:2191: seq_num 779 ranges {692653=486977,1602061=647603,2992629=100024} 2024-09-18T14:08:56.333 INFO:tasks.rados.rados.0.smithi084.stdout:2191: writing smithi08434663-37 from 692653 to 1179630 tid 1 2024-09-18T14:08:56.339 INFO:tasks.rados.rados.0.smithi084.stdout:2191: writing smithi08434663-37 from 1602061 to 2249664 tid 2 2024-09-18T14:08:56.344 INFO:tasks.rados.rados.0.smithi084.stdout:2191: writing smithi08434663-37 from 2992629 to 3092653 tid 3 2024-09-18T14:08:56.344 INFO:tasks.rados.rados.0.smithi084.stdout:2192: rollback oid 49 current snap is 224 2024-09-18T14:08:56.344 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 49 to 220 2024-09-18T14:08:56.344 INFO:tasks.rados.rados.0.smithi084.stdout:2191: finishing write tid 1 to smithi08434663-37 2024-09-18T14:08:56.345 INFO:tasks.rados.rados.0.smithi084.stdout:2191: finishing write tid 2 to smithi08434663-37 2024-09-18T14:08:56.345 INFO:tasks.rados.rados.0.smithi084.stdout:2193: write (excl) oid 26 current snap is 224 2024-09-18T14:08:56.345 INFO:tasks.rados.rados.0.smithi084.stdout:2193: seq_num 780 ranges {709460=562226,2060543=791261} 2024-09-18T14:08:56.351 INFO:tasks.rados.rados.0.smithi084.stdout:2193: writing smithi08434663-26 from 709460 to 1271686 tid 1 2024-09-18T14:08:56.361 INFO:tasks.rados.rados.0.smithi084.stdout:2193: writing smithi08434663-26 from 2060543 to 2851804 tid 2 2024-09-18T14:08:56.362 INFO:tasks.rados.rados.0.smithi084.stdout:2194: copy_from oid 33 from oid 41 current snap is 224 2024-09-18T14:08:56.362 INFO:tasks.rados.rados.0.smithi084.stdout:2191: finishing write tid 3 to smithi08434663-37 2024-09-18T14:08:56.362 INFO:tasks.rados.rados.0.smithi084.stdout:2191: finishing write tid 4 to smithi08434663-37 2024-09-18T14:08:56.362 INFO:tasks.rados.rados.0.smithi084.stdout:2191: finishing write tid 5 to smithi08434663-37 2024-09-18T14:08:56.362 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 666 (ObjNum 779 snap 224 seq_num 779) dirty exists 2024-09-18T14:08:56.363 INFO:tasks.rados.rados.0.smithi084.stdout:2191: left oid 37 (ObjNum 779 snap 224 seq_num 779) 2024-09-18T14:08:56.363 INFO:tasks.rados.rados.0.smithi084.stdout:2192: finishing rollback tid 0 to smithi08434663-49 2024-09-18T14:08:56.363 INFO:tasks.rados.rados.0.smithi084.stdout:2192: finishing rollback tid 1 to smithi08434663-49 2024-09-18T14:08:56.363 INFO:tasks.rados.rados.0.smithi084.stdout:2192: finishing rollback tid 2 to smithi08434663-49 2024-09-18T14:08:56.363 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 669 (ObjNum 672 snap 200 seq_num 672) dirty exists 2024-09-18T14:08:56.363 INFO:tasks.rados.rados.0.smithi084.stdout:2193: finishing write tid 1 to smithi08434663-26 2024-09-18T14:08:56.364 INFO:tasks.rados.rados.0.smithi084.stdout:2191: done (3 left) 2024-09-18T14:08:56.364 INFO:tasks.rados.rados.0.smithi084.stdout:2192: done (2 left) 2024-09-18T14:08:56.364 INFO:tasks.rados.rados.0.smithi084.stdout:2195: snap_remove snap 223 2024-09-18T14:08:57.250 INFO:tasks.rados.rados.0.smithi084.stdout:2195: done (2 left) 2024-09-18T14:08:57.250 INFO:tasks.rados.rados.0.smithi084.stdout:2196: write (excl) oid 21 current snap is 224 2024-09-18T14:08:57.250 INFO:tasks.rados.rados.0.smithi084.stdout:2193: finishing write tid 2 to smithi08434663-26 2024-09-18T14:08:57.250 INFO:tasks.rados.rados.0.smithi084.stdout:2193: finishing write tid 3 to smithi08434663-26 2024-09-18T14:08:57.250 INFO:tasks.rados.rados.0.smithi084.stdout:2193: finishing write tid 4 to smithi08434663-26 2024-09-18T14:08:57.250 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 823 (ObjNum 780 snap 224 seq_num 780) dirty exists 2024-09-18T14:08:57.250 INFO:tasks.rados.rados.0.smithi084.stdout:2193: left oid 26 (ObjNum 780 snap 224 seq_num 780) 2024-09-18T14:08:57.251 INFO:tasks.rados.rados.0.smithi084.stdout:2194: finishing copy_from racing read to smithi08434663-33 2024-09-18T14:08:57.251 INFO:tasks.rados.rados.0.smithi084.stdout:2194: finishing copy_from to smithi08434663-33 2024-09-18T14:08:57.251 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 996 (ObjNum 775 snap 223 seq_num 775) dirty exists 2024-09-18T14:08:57.251 INFO:tasks.rados.rados.0.smithi084.stdout:2196: seq_num 781 ranges {726267=637475,2119025=651272} 2024-09-18T14:08:57.258 INFO:tasks.rados.rados.0.smithi084.stdout:2196: writing smithi08434663-21 from 726267 to 1363742 tid 1 2024-09-18T14:08:57.268 INFO:tasks.rados.rados.0.smithi084.stdout:2196: writing smithi08434663-21 from 2119025 to 2770297 tid 2 2024-09-18T14:08:57.268 INFO:tasks.rados.rados.0.smithi084.stdout:2193: done (2 left) 2024-09-18T14:08:57.269 INFO:tasks.rados.rados.0.smithi084.stdout:2194: done (1 left) 2024-09-18T14:08:57.269 INFO:tasks.rados.rados.0.smithi084.stdout:2197: snap_remove snap 220 2024-09-18T14:08:58.257 INFO:tasks.rados.rados.0.smithi084.stdout:2197: done (1 left) 2024-09-18T14:08:58.257 INFO:tasks.rados.rados.0.smithi084.stdout:2198: read oid 20 snap -1 2024-09-18T14:08:58.257 INFO:tasks.rados.rados.0.smithi084.stdout:2198: expect (ObjNum 680 snap 202 seq_num 680) 2024-09-18T14:08:58.257 INFO:tasks.rados.rados.0.smithi084.stdout:2199: rollback oid 22 current snap is 224 2024-09-18T14:08:58.257 INFO:tasks.rados.rados.0.smithi084.stdout:2199: done (2 left) 2024-09-18T14:08:58.257 INFO:tasks.rados.rados.0.smithi084.stdout:2200: snap_create 2024-09-18T14:08:58.257 INFO:tasks.rados.rados.0.smithi084.stdout:2196: finishing write tid 1 to smithi08434663-21 2024-09-18T14:08:58.257 INFO:tasks.rados.rados.0.smithi084.stdout:2196: finishing write tid 2 to smithi08434663-21 2024-09-18T14:08:58.257 INFO:tasks.rados.rados.0.smithi084.stdout:2196: finishing write tid 3 to smithi08434663-21 2024-09-18T14:08:58.257 INFO:tasks.rados.rados.0.smithi084.stdout:2196: finishing write tid 4 to smithi08434663-21 2024-09-18T14:08:58.257 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 650 (ObjNum 781 snap 224 seq_num 781) dirty exists 2024-09-18T14:08:58.258 INFO:tasks.rados.rados.0.smithi084.stdout:2196: left oid 21 (ObjNum 781 snap 224 seq_num 781) 2024-09-18T14:08:58.272 INFO:tasks.rados.rados.0.smithi084.stdout:2198: expect (ObjNum 680 snap 202 seq_num 680) 2024-09-18T14:08:59.257 INFO:tasks.rados.rados.0.smithi084.stdout:2196: done (2 left) 2024-09-18T14:08:59.258 INFO:tasks.rados.rados.0.smithi084.stdout:2198: done (1 left) 2024-09-18T14:08:59.258 INFO:tasks.rados.rados.0.smithi084.stdout:2200: done (0 left) 2024-09-18T14:08:59.258 INFO:tasks.rados.rados.0.smithi084.stdout:2201: delete oid 23 current snap is 225 2024-09-18T14:08:59.260 INFO:tasks.rados.rados.0.smithi084.stdout:2201: done (0 left) 2024-09-18T14:08:59.260 INFO:tasks.rados.rados.0.smithi084.stdout:2202: read oid 5 snap 224 2024-09-18T14:08:59.260 INFO:tasks.rados.rados.0.smithi084.stdout:2202: expect deleted 2024-09-18T14:08:59.260 INFO:tasks.rados.rados.0.smithi084.stdout:2203: read oid 41 snap -1 2024-09-18T14:08:59.260 INFO:tasks.rados.rados.0.smithi084.stdout:2203: expect (ObjNum 775 snap 223 seq_num 775) 2024-09-18T14:08:59.260 INFO:tasks.rados.rados.0.smithi084.stdout:2204: snap_create 2024-09-18T14:08:59.268 INFO:tasks.rados.rados.0.smithi084.stdout:2203: expect (ObjNum 775 snap 223 seq_num 775) 2024-09-18T14:09:00.268 INFO:tasks.rados.rados.0.smithi084.stdout:2202: done (2 left) 2024-09-18T14:09:00.268 INFO:tasks.rados.rados.0.smithi084.stdout:2203: done (1 left) 2024-09-18T14:09:00.268 INFO:tasks.rados.rados.0.smithi084.stdout:2204: done (0 left) 2024-09-18T14:09:00.268 INFO:tasks.rados.rados.0.smithi084.stdout:2205: read oid 4 snap -1 2024-09-18T14:09:00.268 INFO:tasks.rados.rados.0.smithi084.stdout:2205: expect (ObjNum 723 snap 209 seq_num 723) 2024-09-18T14:09:00.268 INFO:tasks.rados.rados.0.smithi084.stdout:2206: snap_create 2024-09-18T14:09:00.274 INFO:tasks.rados.rados.0.smithi084.stdout:2205: expect (ObjNum 723 snap 209 seq_num 723) 2024-09-18T14:09:01.269 INFO:tasks.rados.rados.0.smithi084.stdout:2205: done (1 left) 2024-09-18T14:09:01.269 INFO:tasks.rados.rados.0.smithi084.stdout:2206: done (0 left) 2024-09-18T14:09:01.270 INFO:tasks.rados.rados.0.smithi084.stdout:2207: read oid 38 snap -1 2024-09-18T14:09:01.270 INFO:tasks.rados.rados.0.smithi084.stdout:2207: expect deleted 2024-09-18T14:09:01.270 INFO:tasks.rados.rados.0.smithi084.stdout:2208: delete oid 14 current snap is 227 2024-09-18T14:09:01.278 INFO:tasks.rados.rados.0.smithi084.stdout:2207: done (1 left) 2024-09-18T14:09:01.278 INFO:tasks.rados.rados.0.smithi084.stdout:2208: done (0 left) 2024-09-18T14:09:01.278 INFO:tasks.rados.rados.0.smithi084.stdout:2209: read oid 40 snap -1 2024-09-18T14:09:01.278 INFO:tasks.rados.rados.0.smithi084.stdout:2209: expect deleted 2024-09-18T14:09:01.278 INFO:tasks.rados.rados.0.smithi084.stdout:2210: read oid 15 snap -1 2024-09-18T14:09:01.278 INFO:tasks.rados.rados.0.smithi084.stdout:2210: expect deleted 2024-09-18T14:09:01.278 INFO:tasks.rados.rados.0.smithi084.stdout:2211: write (excl) oid 11 current snap is 227 2024-09-18T14:09:01.279 INFO:tasks.rados.rados.0.smithi084.stdout:2211: seq_num 784 ranges {776688=579575,1894471=798599} 2024-09-18T14:09:01.285 INFO:tasks.rados.rados.0.smithi084.stdout:2211: writing smithi08434663-11 from 776688 to 1356263 tid 1 2024-09-18T14:09:01.293 INFO:tasks.rados.rados.0.smithi084.stdout:2211: writing smithi08434663-11 from 1894471 to 2693070 tid 2 2024-09-18T14:09:01.293 INFO:tasks.rados.rados.0.smithi084.stdout:2212: snap_create 2024-09-18T14:09:01.293 INFO:tasks.rados.rados.0.smithi084.stdout:2211: finishing write tid 1 to smithi08434663-11 2024-09-18T14:09:01.299 INFO:tasks.rados.rados.0.smithi084.stdout:2211: finishing write tid 2 to smithi08434663-11 2024-09-18T14:09:01.302 INFO:tasks.rados.rados.0.smithi084.stdout:2211: finishing write tid 3 to smithi08434663-11 2024-09-18T14:09:01.302 INFO:tasks.rados.rados.0.smithi084.stdout:2211: finishing write tid 4 to smithi08434663-11 2024-09-18T14:09:01.302 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1042 (ObjNum 784 snap 227 seq_num 784) dirty exists 2024-09-18T14:09:01.302 INFO:tasks.rados.rados.0.smithi084.stdout:2211: left oid 11 (ObjNum 784 snap 227 seq_num 784) 2024-09-18T14:09:02.251 INFO:tasks.rados.rados.0.smithi084.stdout:2209: done (3 left) 2024-09-18T14:09:02.251 INFO:tasks.rados.rados.0.smithi084.stdout:2210: done (2 left) 2024-09-18T14:09:02.251 INFO:tasks.rados.rados.0.smithi084.stdout:2211: done (1 left) 2024-09-18T14:09:02.251 INFO:tasks.rados.rados.0.smithi084.stdout:2212: done (0 left) 2024-09-18T14:09:02.251 INFO:tasks.rados.rados.0.smithi084.stdout:2213: snap_create 2024-09-18T14:09:03.252 INFO:tasks.rados.rados.0.smithi084.stdout:2213: done (0 left) 2024-09-18T14:09:03.252 INFO:tasks.rados.rados.0.smithi084.stdout:2214: snap_create 2024-09-18T14:09:04.253 INFO:tasks.rados.rados.0.smithi084.stdout:2214: done (0 left) 2024-09-18T14:09:04.253 INFO:tasks.rados.rados.0.smithi084.stdout:2215: read oid 7 snap 227 2024-09-18T14:09:04.253 INFO:tasks.rados.rados.0.smithi084.stdout:2215: expect (ObjNum 741 snap 214 seq_num 741) 2024-09-18T14:09:04.254 INFO:tasks.rados.rados.0.smithi084.stdout:2216: snap_remove snap 225 2024-09-18T14:09:05.258 INFO:tasks.rados.rados.0.smithi084.stdout:2216: done (1 left) 2024-09-18T14:09:05.259 INFO:tasks.rados.rados.0.smithi084.stdout:2217: snap_create 2024-09-18T14:09:05.259 INFO:tasks.rados.rados.0.smithi084.stdout:2215: expect (ObjNum 741 snap 214 seq_num 741) 2024-09-18T14:09:06.260 INFO:tasks.rados.rados.0.smithi084.stdout:2215: done (1 left) 2024-09-18T14:09:06.260 INFO:tasks.rados.rados.0.smithi084.stdout:2217: done (0 left) 2024-09-18T14:09:06.261 INFO:tasks.rados.rados.0.smithi084.stdout:2218: snap_remove snap 228 2024-09-18T14:09:07.254 INFO:tasks.rados.rados.0.smithi084.stdout:2218: done (0 left) 2024-09-18T14:09:07.254 INFO:tasks.rados.rados.0.smithi084.stdout:2219: snap_remove snap 230 2024-09-18T14:09:08.260 INFO:tasks.rados.rados.0.smithi084.stdout:2219: done (0 left) 2024-09-18T14:09:08.261 INFO:tasks.rados.rados.0.smithi084.stdout:2220: copy_from oid 1 from oid 50 current snap is 231 2024-09-18T14:09:08.261 INFO:tasks.rados.rados.0.smithi084.stdout:2221: snap_remove snap 226 2024-09-18T14:09:09.262 INFO:tasks.rados.rados.0.smithi084.stdout:2221: done (1 left) 2024-09-18T14:09:09.263 INFO:tasks.rados.rados.0.smithi084.stdout:2222: read oid 20 snap -1 2024-09-18T14:09:09.263 INFO:tasks.rados.rados.0.smithi084.stdout:2222: expect (ObjNum 680 snap 202 seq_num 680) 2024-09-18T14:09:09.263 INFO:tasks.rados.rados.0.smithi084.stdout:2223: delete oid 5 current snap is 231 2024-09-18T14:09:09.263 INFO:tasks.rados.rados.0.smithi084.stdout:2220: finishing copy_from to smithi08434663-1 2024-09-18T14:09:09.263 INFO:tasks.rados.rados.0.smithi084.stdout:2220: got expected ENOENT (src dne) 2024-09-18T14:09:09.263 INFO:tasks.rados.rados.0.smithi084.stdout:2220: finishing copy_from racing read to smithi08434663-1 2024-09-18T14:09:09.270 INFO:tasks.rados.rados.0.smithi084.stdout:2220: done (2 left) 2024-09-18T14:09:09.270 INFO:tasks.rados.rados.0.smithi084.stdout:2223: done (1 left) 2024-09-18T14:09:09.270 INFO:tasks.rados.rados.0.smithi084.stdout:2224: write (excl) oid 45 current snap is 231 2024-09-18T14:09:09.270 INFO:tasks.rados.rados.0.smithi084.stdout:2224: seq_num 786 ranges {410302=730073,1895082=402268,2951884=258418} 2024-09-18T14:09:09.277 INFO:tasks.rados.rados.0.smithi084.stdout:2224: writing smithi08434663-45 from 410302 to 1140375 tid 1 2024-09-18T14:09:09.283 INFO:tasks.rados.rados.0.smithi084.stdout:2224: writing smithi08434663-45 from 1895082 to 2297350 tid 2 2024-09-18T14:09:09.288 INFO:tasks.rados.rados.0.smithi084.stdout:2224: writing smithi08434663-45 from 2951884 to 3210302 tid 3 2024-09-18T14:09:09.288 INFO:tasks.rados.rados.0.smithi084.stdout:2225: write (excl) oid 48 current snap is 231 2024-09-18T14:09:09.288 INFO:tasks.rados.rados.0.smithi084.stdout:2225: seq_num 787 ranges {427109=405322,1553564=545926,2579307=647802} 2024-09-18T14:09:09.293 INFO:tasks.rados.rados.0.smithi084.stdout:2225: writing smithi08434663-48 from 427109 to 832431 tid 1 2024-09-18T14:09:09.300 INFO:tasks.rados.rados.0.smithi084.stdout:2225: writing smithi08434663-48 from 1553564 to 2099490 tid 2 2024-09-18T14:09:09.306 INFO:tasks.rados.rados.0.smithi084.stdout:2225: writing smithi08434663-48 from 2579307 to 3227109 tid 3 2024-09-18T14:09:09.306 INFO:tasks.rados.rados.0.smithi084.stdout:2226: read oid 18 snap -1 2024-09-18T14:09:09.306 INFO:tasks.rados.rados.0.smithi084.stdout:2226: expect (ObjNum 762 snap 222 seq_num 762) 2024-09-18T14:09:09.306 INFO:tasks.rados.rados.0.smithi084.stdout:2227: read oid 24 snap -1 2024-09-18T14:09:09.306 INFO:tasks.rados.rados.0.smithi084.stdout:2227: expect deleted 2024-09-18T14:09:09.306 INFO:tasks.rados.rados.0.smithi084.stdout:2228: write oid 22 current snap is 231 2024-09-18T14:09:09.306 INFO:tasks.rados.rados.0.smithi084.stdout:2228: seq_num 788 ranges {443916=480571,1612046=405937,2606730=495052} 2024-09-18T14:09:09.311 INFO:tasks.rados.rados.0.smithi084.stdout:2228: writing smithi08434663-22 from 443916 to 924487 tid 1 2024-09-18T14:09:09.318 INFO:tasks.rados.rados.0.smithi084.stdout:2228: writing smithi08434663-22 from 1612046 to 2017983 tid 2 2024-09-18T14:09:09.324 INFO:tasks.rados.rados.0.smithi084.stdout:2228: writing smithi08434663-22 from 2606730 to 3101782 tid 3 2024-09-18T14:09:09.324 INFO:tasks.rados.rados.0.smithi084.stdout:2229: snap_create 2024-09-18T14:09:09.324 INFO:tasks.rados.rados.0.smithi084.stdout:2222: expect (ObjNum 680 snap 202 seq_num 680) 2024-09-18T14:09:09.511 INFO:tasks.rados.rados.0.smithi084.stdout:2224: finishing write tid 1 to smithi08434663-45 2024-09-18T14:09:09.511 INFO:tasks.rados.rados.0.smithi084.stdout:2224: finishing write tid 2 to smithi08434663-45 2024-09-18T14:09:09.511 INFO:tasks.rados.rados.0.smithi084.stdout:2224: finishing write tid 3 to smithi08434663-45 2024-09-18T14:09:09.511 INFO:tasks.rados.rados.0.smithi084.stdout:2224: finishing write tid 4 to smithi08434663-45 2024-09-18T14:09:09.512 INFO:tasks.rados.rados.0.smithi084.stdout:2224: finishing write tid 5 to smithi08434663-45 2024-09-18T14:09:09.512 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 1005 (ObjNum 786 snap 231 seq_num 786) dirty exists 2024-09-18T14:09:09.512 INFO:tasks.rados.rados.0.smithi084.stdout:2224: left oid 45 (ObjNum 786 snap 231 seq_num 786) 2024-09-18T14:09:09.512 INFO:tasks.rados.rados.0.smithi084.stdout:2225: finishing write tid 1 to smithi08434663-48 2024-09-18T14:09:09.512 INFO:tasks.rados.rados.0.smithi084.stdout:2225: finishing write tid 2 to smithi08434663-48 2024-09-18T14:09:09.512 INFO:tasks.rados.rados.0.smithi084.stdout:2225: finishing write tid 3 to smithi08434663-48 2024-09-18T14:09:09.512 INFO:tasks.rados.rados.0.smithi084.stdout:2225: finishing write tid 4 to smithi08434663-48 2024-09-18T14:09:09.512 INFO:tasks.rados.rados.0.smithi084.stdout:2226: expect (ObjNum 762 snap 222 seq_num 762) 2024-09-18T14:09:09.664 INFO:tasks.rados.rados.0.smithi084.stdout:2225: finishing write tid 5 to smithi08434663-48 2024-09-18T14:09:09.665 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 654 (ObjNum 787 snap 231 seq_num 787) dirty exists 2024-09-18T14:09:09.665 INFO:tasks.rados.rados.0.smithi084.stdout:2225: left oid 48 (ObjNum 787 snap 231 seq_num 787) 2024-09-18T14:09:09.665 INFO:tasks.rados.rados.0.smithi084.stdout:2228: finishing write tid 1 to smithi08434663-22 2024-09-18T14:09:09.665 INFO:tasks.rados.rados.0.smithi084.stdout:2228: finishing write tid 2 to smithi08434663-22 2024-09-18T14:09:09.665 INFO:tasks.rados.rados.0.smithi084.stdout:2228: finishing write tid 3 to smithi08434663-22 2024-09-18T14:09:09.665 INFO:tasks.rados.rados.0.smithi084.stdout:2228: finishing write tid 4 to smithi08434663-22 2024-09-18T14:09:09.665 INFO:tasks.rados.rados.0.smithi084.stdout:2228: finishing write tid 5 to smithi08434663-22 2024-09-18T14:09:09.665 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 767 (ObjNum 788 snap 231 seq_num 788) dirty exists 2024-09-18T14:09:09.665 INFO:tasks.rados.rados.0.smithi084.stdout:2228: left oid 22 (ObjNum 788 snap 231 seq_num 788) 2024-09-18T14:09:10.271 INFO:tasks.rados.rados.0.smithi084.stdout:2222: done (6 left) 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2224: done (5 left) 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2225: done (4 left) 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2226: done (3 left) 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2227: done (2 left) 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2228: done (1 left) 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2229: done (0 left) 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2230: read oid 27 snap 227 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2230: expect (ObjNum 723 snap 209 seq_num 723) 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2231: rollback oid 32 current snap is 232 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 32 to 231 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2232: read oid 8 snap -1 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2232: expect (ObjNum 763 snap 222 seq_num 763) 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2233: copy_from oid 20 from oid 34 current snap is 232 2024-09-18T14:09:10.272 INFO:tasks.rados.rados.0.smithi084.stdout:2234: snap_create 2024-09-18T14:09:10.273 INFO:tasks.rados.rados.0.smithi084.stdout:2231: finishing rollback tid 1 to smithi08434663-32 2024-09-18T14:09:10.273 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-18T14:09:10.275 INFO:tasks.rados.rados.0.smithi084.stdout:2230: expect (ObjNum 723 snap 209 seq_num 723) 2024-09-18T14:09:10.365 INFO:tasks.rados.rados.0.smithi084.stdout:2232: expect (ObjNum 763 snap 222 seq_num 763) 2024-09-18T14:09:10.454 INFO:tasks.rados.rados.0.smithi084.stdout:2233: finishing copy_from racing read to smithi08434663-20 2024-09-18T14:09:10.454 INFO:tasks.rados.rados.0.smithi084.stdout:2233: finishing copy_from to smithi08434663-20 2024-09-18T14:09:10.454 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 965 (ObjNum 747 snap 218 seq_num 747) dirty exists 2024-09-18T14:09:11.274 INFO:tasks.rados.rados.0.smithi084.stdout:2230: done (4 left) 2024-09-18T14:09:11.274 INFO:tasks.rados.rados.0.smithi084.stdout:2231: done (3 left) 2024-09-18T14:09:11.274 INFO:tasks.rados.rados.0.smithi084.stdout:2232: done (2 left) 2024-09-18T14:09:11.275 INFO:tasks.rados.rados.0.smithi084.stdout:2233: done (1 left) 2024-09-18T14:09:11.275 INFO:tasks.rados.rados.0.smithi084.stdout:2234: done (0 left) 2024-09-18T14:09:11.275 INFO:tasks.rados.rados.0.smithi084.stdout:2235: read oid 26 snap -1 2024-09-18T14:09:11.275 INFO:tasks.rados.rados.0.smithi084.stdout:2235: expect (ObjNum 780 snap 224 seq_num 780) 2024-09-18T14:09:11.275 INFO:tasks.rados.rados.0.smithi084.stdout:2236: copy_from oid 45 from oid 49 current snap is 233 2024-09-18T14:09:11.275 INFO:tasks.rados.rados.0.smithi084.stdout:2237: copy_from oid 1 from oid 32 current snap is 233 2024-09-18T14:09:11.275 INFO:tasks.rados.rados.0.smithi084.stdout:2238: copy_from oid 34 from oid 22 current snap is 233 2024-09-18T14:09:11.275 INFO:tasks.rados.rados.0.smithi084.stdout:2239: write oid 38 current snap is 233 2024-09-18T14:09:11.275 INFO:tasks.rados.rados.0.smithi084.stdout:2239: seq_num 789 ranges {460723=555820,1670528=549595,2634153=626570} 2024-09-18T14:09:11.279 INFO:tasks.rados.rados.0.smithi084.stdout:2239: writing smithi08434663-38 from 460723 to 1016543 tid 1 2024-09-18T14:09:11.286 INFO:tasks.rados.rados.0.smithi084.stdout:2239: writing smithi08434663-38 from 1670528 to 2220123 tid 2 2024-09-18T14:09:11.292 INFO:tasks.rados.rados.0.smithi084.stdout:2239: writing smithi08434663-38 from 2634153 to 3260723 tid 3 2024-09-18T14:09:11.292 INFO:tasks.rados.rados.0.smithi084.stdout:2240: copy_from oid 2 from oid 20 current snap is 233 2024-09-18T14:09:11.292 INFO:tasks.rados.rados.0.smithi084.stdout:2235: expect (ObjNum 780 snap 224 seq_num 780) 2024-09-18T14:09:11.430 INFO:tasks.rados.rados.0.smithi084.stdout:2237: finishing copy_from to smithi08434663-1 2024-09-18T14:09:11.431 INFO:tasks.rados.rados.0.smithi084.stdout:2237: got expected ENOENT (src dne) 2024-09-18T14:09:11.431 INFO:tasks.rados.rados.0.smithi084.stdout:2237: finishing copy_from racing read to smithi08434663-1 2024-09-18T14:09:11.431 INFO:tasks.rados.rados.0.smithi084.stdout:2239: finishing write tid 1 to smithi08434663-38 2024-09-18T14:09:11.431 INFO:tasks.rados.rados.0.smithi084.stdout:2239: finishing write tid 2 to smithi08434663-38 2024-09-18T14:09:11.431 INFO:tasks.rados.rados.0.smithi084.stdout:2239: finishing write tid 3 to smithi08434663-38 2024-09-18T14:09:11.431 INFO:tasks.rados.rados.0.smithi084.stdout:2239: finishing write tid 4 to smithi08434663-38 2024-09-18T14:09:11.431 INFO:tasks.rados.rados.0.smithi084.stdout:2239: finishing write tid 5 to smithi08434663-38 2024-09-18T14:09:11.431 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 639 (ObjNum 789 snap 233 seq_num 789) dirty exists 2024-09-18T14:09:11.431 INFO:tasks.rados.rados.0.smithi084.stdout:2239: left oid 38 (ObjNum 789 snap 233 seq_num 789) 2024-09-18T14:09:11.431 INFO:tasks.rados.rados.0.smithi084.stdout:2238: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:09:11.431 INFO:tasks.rados.rados.0.smithi084.stdout:2235: done (5 left) 2024-09-18T14:09:11.431 INFO:tasks.rados.rados.0.smithi084.stdout:2237: done (4 left) 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:2239: done (3 left) 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:2241: copy_from oid 36 from oid 13 current snap is 233 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:2236: finishing copy_from racing read to smithi08434663-45 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:2236: finishing copy_from to smithi08434663-45 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 669 (ObjNum 672 snap 200 seq_num 672) dirty exists 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:2240: finishing copy_from racing read to smithi08434663-2 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:2238: finishing copy_from to smithi08434663-34 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 767 (ObjNum 788 snap 231 seq_num 788) dirty exists 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:2240: finishing copy_from to smithi08434663-2 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 965 (ObjNum 747 snap 218 seq_num 747) dirty exists 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:2236: done (3 left) 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:2238: done (2 left) 2024-09-18T14:09:11.432 INFO:tasks.rados.rados.0.smithi084.stdout:2240: done (1 left) 2024-09-18T14:09:11.433 INFO:tasks.rados.rados.0.smithi084.stdout:2242: copy_from oid 16 from oid 37 current snap is 233 2024-09-18T14:09:11.433 INFO:tasks.rados.rados.0.smithi084.stdout:2243: read oid 17 snap -1 2024-09-18T14:09:11.433 INFO:tasks.rados.rados.0.smithi084.stdout:2243: expect (ObjNum 769 snap 223 seq_num 769) 2024-09-18T14:09:11.433 INFO:tasks.rados.rados.0.smithi084.stdout:2244: snap_create 2024-09-18T14:09:11.444 INFO:tasks.rados.rados.0.smithi084.stdout:2241: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:09:11.446 INFO:tasks.rados.rados.0.smithi084.stdout:2243: expect (ObjNum 769 snap 223 seq_num 769) 2024-09-18T14:09:11.580 INFO:tasks.rados.rados.0.smithi084.stdout:2242: finishing copy_from racing read to smithi08434663-16 2024-09-18T14:09:11.580 INFO:tasks.rados.rados.0.smithi084.stdout:2241: finishing copy_from to smithi08434663-36 2024-09-18T14:09:11.580 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 871 (ObjNum 756 snap 220 seq_num 756) dirty exists 2024-09-18T14:09:11.581 INFO:tasks.rados.rados.0.smithi084.stdout:2242: finishing copy_from to smithi08434663-16 2024-09-18T14:09:11.581 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 666 (ObjNum 779 snap 224 seq_num 779) dirty exists 2024-09-18T14:09:12.253 INFO:tasks.rados.rados.0.smithi084.stdout:2241: done (3 left) 2024-09-18T14:09:12.253 INFO:tasks.rados.rados.0.smithi084.stdout:2242: done (2 left) 2024-09-18T14:09:12.253 INFO:tasks.rados.rados.0.smithi084.stdout:2243: done (1 left) 2024-09-18T14:09:12.254 INFO:tasks.rados.rados.0.smithi084.stdout:2244: done (0 left) 2024-09-18T14:09:12.254 INFO:tasks.rados.rados.0.smithi084.stdout:2245: write oid 1 current snap is 234 2024-09-18T14:09:12.254 INFO:tasks.rados.rados.0.smithi084.stdout:2245: seq_num 790 ranges {477530=631069,1612657=409606,2545223=517596} 2024-09-18T14:09:12.259 INFO:tasks.rados.rados.0.smithi084.stdout:2245: writing smithi08434663-1 from 477530 to 1108599 tid 1 2024-09-18T14:09:12.264 INFO:tasks.rados.rados.0.smithi084.stdout:2245: writing smithi08434663-1 from 1612657 to 2022263 tid 2 2024-09-18T14:09:12.271 INFO:tasks.rados.rados.0.smithi084.stdout:2245: writing smithi08434663-1 from 2545223 to 3062819 tid 3 2024-09-18T14:09:12.271 INFO:tasks.rados.rados.0.smithi084.stdout:2246: delete oid 31 current snap is 234 2024-09-18T14:09:12.271 INFO:tasks.rados.rados.0.smithi084.stdout:2245: finishing write tid 1 to smithi08434663-1 2024-09-18T14:09:12.271 INFO:tasks.rados.rados.0.smithi084.stdout:2245: finishing write tid 2 to smithi08434663-1 2024-09-18T14:09:12.274 INFO:tasks.rados.rados.0.smithi084.stdout:2246: done (1 left) 2024-09-18T14:09:12.275 INFO:tasks.rados.rados.0.smithi084.stdout:2247: snap_remove snap 233 2024-09-18T14:09:13.282 INFO:tasks.rados.rados.0.smithi084.stdout:2247: done (1 left) 2024-09-18T14:09:13.283 INFO:tasks.rados.rados.0.smithi084.stdout:2248: read oid 16 snap 231 2024-09-18T14:09:13.283 INFO:tasks.rados.rados.0.smithi084.stdout:2248: expect (ObjNum 706 snap 205 seq_num 706) 2024-09-18T14:09:13.283 INFO:tasks.rados.rados.0.smithi084.stdout:2249: delete oid 7 current snap is 234 2024-09-18T14:09:13.283 INFO:tasks.rados.rados.0.smithi084.stdout:2245: finishing write tid 3 to smithi08434663-1 2024-09-18T14:09:13.283 INFO:tasks.rados.rados.0.smithi084.stdout:2245: finishing write tid 4 to smithi08434663-1 2024-09-18T14:09:13.283 INFO:tasks.rados.rados.0.smithi084.stdout:2245: finishing write tid 5 to smithi08434663-1 2024-09-18T14:09:13.283 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 834 (ObjNum 790 snap 234 seq_num 790) dirty exists 2024-09-18T14:09:13.283 INFO:tasks.rados.rados.0.smithi084.stdout:2245: left oid 1 (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:09:13.285 INFO:tasks.rados.rados.0.smithi084.stdout:2245: done (2 left) 2024-09-18T14:09:13.286 INFO:tasks.rados.rados.0.smithi084.stdout:2249: done (1 left) 2024-09-18T14:09:13.286 INFO:tasks.rados.rados.0.smithi084.stdout:2250: delete oid 22 current snap is 234 2024-09-18T14:09:13.288 INFO:tasks.rados.rados.0.smithi084.stdout:2248: expect (ObjNum 706 snap 205 seq_num 706) 2024-09-18T14:09:13.409 INFO:tasks.rados.rados.0.smithi084.stdout:2248: done (1 left) 2024-09-18T14:09:13.409 INFO:tasks.rados.rados.0.smithi084.stdout:2250: done (0 left) 2024-09-18T14:09:13.409 INFO:tasks.rados.rados.0.smithi084.stdout:2251: read oid 24 snap -1 2024-09-18T14:09:13.409 INFO:tasks.rados.rados.0.smithi084.stdout:2251: expect deleted 2024-09-18T14:09:13.409 INFO:tasks.rados.rados.0.smithi084.stdout:2252: read oid 47 snap 229 2024-09-18T14:09:13.409 INFO:tasks.rados.rados.0.smithi084.stdout:2252: expect (ObjNum 689 snap 204 seq_num 689) 2024-09-18T14:09:13.409 INFO:tasks.rados.rados.0.smithi084.stdout:2253: snap_create 2024-09-18T14:09:13.414 INFO:tasks.rados.rados.0.smithi084.stdout:2252: expect (ObjNum 689 snap 204 seq_num 689) 2024-09-18T14:09:14.292 INFO:tasks.rados.rados.0.smithi084.stdout:2251: done (2 left) 2024-09-18T14:09:14.293 INFO:tasks.rados.rados.0.smithi084.stdout:2252: done (1 left) 2024-09-18T14:09:14.293 INFO:tasks.rados.rados.0.smithi084.stdout:2253: done (0 left) 2024-09-18T14:09:14.293 INFO:tasks.rados.rados.0.smithi084.stdout:2254: snap_create 2024-09-18T14:09:15.296 INFO:tasks.rados.rados.0.smithi084.stdout:2254: done (0 left) 2024-09-18T14:09:15.297 INFO:tasks.rados.rados.0.smithi084.stdout:2255: copy_from oid 40 from oid 26 current snap is 236 2024-09-18T14:09:15.297 INFO:tasks.rados.rados.0.smithi084.stdout:2256: write oid 24 current snap is 236 2024-09-18T14:09:15.297 INFO:tasks.rados.rados.0.smithi084.stdout:2256: seq_num 794 ranges {544758=648418,1846585=416944,2771268=573490} 2024-09-18T14:09:15.303 INFO:tasks.rados.rados.0.smithi084.stdout:2256: writing smithi08434663-24 from 544758 to 1193176 tid 1 2024-09-18T14:09:15.309 INFO:tasks.rados.rados.0.smithi084.stdout:2256: writing smithi08434663-24 from 1846585 to 2263529 tid 2 2024-09-18T14:09:15.315 INFO:tasks.rados.rados.0.smithi084.stdout:2256: writing smithi08434663-24 from 2771268 to 3344758 tid 3 2024-09-18T14:09:15.315 INFO:tasks.rados.rados.0.smithi084.stdout:2257: copy_from oid 28 from oid 1 current snap is 236 2024-09-18T14:09:15.315 INFO:tasks.rados.rados.0.smithi084.stdout:2256: finishing write tid 1 to smithi08434663-24 2024-09-18T14:09:15.316 INFO:tasks.rados.rados.0.smithi084.stdout:2255: finishing copy_from racing read to smithi08434663-40 2024-09-18T14:09:15.316 INFO:tasks.rados.rados.0.smithi084.stdout:2255: finishing copy_from to smithi08434663-40 2024-09-18T14:09:15.316 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 823 (ObjNum 780 snap 224 seq_num 780) dirty exists 2024-09-18T14:09:15.316 INFO:tasks.rados.rados.0.smithi084.stdout:2256: finishing write tid 2 to smithi08434663-24 2024-09-18T14:09:15.316 INFO:tasks.rados.rados.0.smithi084.stdout:2255: done (2 left) 2024-09-18T14:09:15.316 INFO:tasks.rados.rados.0.smithi084.stdout:2258: read oid 29 snap -1 2024-09-18T14:09:15.316 INFO:tasks.rados.rados.0.smithi084.stdout:2258: expect deleted 2024-09-18T14:09:15.316 INFO:tasks.rados.rados.0.smithi084.stdout:2259: delete oid 8 current snap is 236 2024-09-18T14:09:15.318 INFO:tasks.rados.rados.0.smithi084.stdout:2258: done (3 left) 2024-09-18T14:09:15.318 INFO:tasks.rados.rados.0.smithi084.stdout:2259: done (2 left) 2024-09-18T14:09:15.318 INFO:tasks.rados.rados.0.smithi084.stdout:2260: write oid 47 current snap is 236 2024-09-18T14:09:15.319 INFO:tasks.rados.rados.0.smithi084.stdout:2260: seq_num 796 ranges {578372=798916,1963549=420613,2826114=417934} 2024-09-18T14:09:15.326 INFO:tasks.rados.rados.0.smithi084.stdout:2260: writing smithi08434663-47 from 578372 to 1377288 tid 1 2024-09-18T14:09:15.332 INFO:tasks.rados.rados.0.smithi084.stdout:2260: writing smithi08434663-47 from 1963549 to 2384162 tid 2 2024-09-18T14:09:15.337 INFO:tasks.rados.rados.0.smithi084.stdout:2260: writing smithi08434663-47 from 2826114 to 3244048 tid 3 2024-09-18T14:09:15.337 INFO:tasks.rados.rados.0.smithi084.stdout:2261: write (excl) oid 4 current snap is 236 2024-09-18T14:09:15.337 INFO:tasks.rados.rados.0.smithi084.stdout:2256: finishing write tid 3 to smithi08434663-24 2024-09-18T14:09:15.337 INFO:tasks.rados.rados.0.smithi084.stdout:2261: seq_num 797 ranges {595179=474165,1622031=564271,2737184=629206} 2024-09-18T14:09:15.342 INFO:tasks.rados.rados.0.smithi084.stdout:2261: writing smithi08434663-4 from 595179 to 1069344 tid 1 2024-09-18T14:09:15.349 INFO:tasks.rados.rados.0.smithi084.stdout:2261: writing smithi08434663-4 from 1622031 to 2186302 tid 2 2024-09-18T14:09:15.355 INFO:tasks.rados.rados.0.smithi084.stdout:2261: writing smithi08434663-4 from 2737184 to 3366390 tid 3 2024-09-18T14:09:15.355 INFO:tasks.rados.rados.0.smithi084.stdout:2262: read oid 11 snap -1 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:2262: expect (ObjNum 784 snap 227 seq_num 784) 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:2263: copy_from oid 9 from oid 45 current snap is 236 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:2256: finishing write tid 4 to smithi08434663-24 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:2256: finishing write tid 5 to smithi08434663-24 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 579 (ObjNum 794 snap 236 seq_num 794) dirty exists 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:2256: left oid 24 (ObjNum 794 snap 236 seq_num 794) 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:2257: finishing copy_from racing read to smithi08434663-28 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:2257: finishing copy_from to smithi08434663-28 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 834 (ObjNum 790 snap 234 seq_num 790) dirty exists 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:2260: finishing write tid 1 to smithi08434663-47 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:2260: finishing write tid 2 to smithi08434663-47 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:2260: finishing write tid 3 to smithi08434663-47 2024-09-18T14:09:15.356 INFO:tasks.rados.rados.0.smithi084.stdout:2260: finishing write tid 4 to smithi08434663-47 2024-09-18T14:09:15.357 INFO:tasks.rados.rados.0.smithi084.stdout:2260: finishing write tid 5 to smithi08434663-47 2024-09-18T14:09:15.357 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 1046 (ObjNum 796 snap 236 seq_num 796) dirty exists 2024-09-18T14:09:15.357 INFO:tasks.rados.rados.0.smithi084.stdout:2260: left oid 47 (ObjNum 796 snap 236 seq_num 796) 2024-09-18T14:09:15.357 INFO:tasks.rados.rados.0.smithi084.stdout:2256: done (5 left) 2024-09-18T14:09:15.357 INFO:tasks.rados.rados.0.smithi084.stdout:2257: done (4 left) 2024-09-18T14:09:15.357 INFO:tasks.rados.rados.0.smithi084.stdout:2260: done (3 left) 2024-09-18T14:09:15.357 INFO:tasks.rados.rados.0.smithi084.stdout:2264: delete oid 8 current snap is 236 2024-09-18T14:09:15.357 INFO:tasks.rados.rados.0.smithi084.stdout:2261: finishing write tid 1 to smithi08434663-4 2024-09-18T14:09:15.357 INFO:tasks.rados.rados.0.smithi084.stdout:2261: finishing write tid 2 to smithi08434663-4 2024-09-18T14:09:15.357 INFO:tasks.rados.rados.0.smithi084.stdout:2264: done (3 left) 2024-09-18T14:09:15.358 INFO:tasks.rados.rados.0.smithi084.stdout:2265: snap_create 2024-09-18T14:09:15.361 INFO:tasks.rados.rados.0.smithi084.stdout:2261: finishing write tid 3 to smithi08434663-4 2024-09-18T14:09:15.361 INFO:tasks.rados.rados.0.smithi084.stdout:2262: expect (ObjNum 784 snap 227 seq_num 784) 2024-09-18T14:09:15.569 INFO:tasks.rados.rados.0.smithi084.stdout:2261: finishing write tid 4 to smithi08434663-4 2024-09-18T14:09:15.570 INFO:tasks.rados.rados.0.smithi084.stdout:2261: finishing write tid 5 to smithi08434663-4 2024-09-18T14:09:15.570 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1014 (ObjNum 797 snap 236 seq_num 797) dirty exists 2024-09-18T14:09:15.570 INFO:tasks.rados.rados.0.smithi084.stdout:2261: left oid 4 (ObjNum 797 snap 236 seq_num 797) 2024-09-18T14:09:15.570 INFO:tasks.rados.rados.0.smithi084.stdout:2263: finishing copy_from racing read to smithi08434663-9 2024-09-18T14:09:15.570 INFO:tasks.rados.rados.0.smithi084.stdout:2263: finishing copy_from to smithi08434663-9 2024-09-18T14:09:15.570 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 669 (ObjNum 672 snap 200 seq_num 672) dirty exists 2024-09-18T14:09:16.304 INFO:tasks.rados.rados.0.smithi084.stdout:2261: done (3 left) 2024-09-18T14:09:16.304 INFO:tasks.rados.rados.0.smithi084.stdout:2262: done (2 left) 2024-09-18T14:09:16.304 INFO:tasks.rados.rados.0.smithi084.stdout:2263: done (1 left) 2024-09-18T14:09:16.304 INFO:tasks.rados.rados.0.smithi084.stdout:2265: done (0 left) 2024-09-18T14:09:16.304 INFO:tasks.rados.rados.0.smithi084.stdout:2266: delete oid 3 current snap is 237 2024-09-18T14:09:16.312 INFO:tasks.rados.rados.0.smithi084.stdout:2266: done (0 left) 2024-09-18T14:09:16.312 INFO:tasks.rados.rados.0.smithi084.stdout:2267: snap_create 2024-09-18T14:09:17.255 INFO:tasks.rados.rados.0.smithi084.stdout:2267: done (0 left) 2024-09-18T14:09:17.256 INFO:tasks.rados.rados.0.smithi084.stdout:2268: read oid 14 snap -1 2024-09-18T14:09:17.256 INFO:tasks.rados.rados.0.smithi084.stdout:2268: expect deleted 2024-09-18T14:09:17.256 INFO:tasks.rados.rados.0.smithi084.stdout:2269: delete oid 39 current snap is 238 2024-09-18T14:09:17.259 INFO:tasks.rados.rados.0.smithi084.stdout:2269: done (1 left) 2024-09-18T14:09:17.259 INFO:tasks.rados.rados.0.smithi084.stdout:2270: delete oid 2 current snap is 238 2024-09-18T14:09:17.268 INFO:tasks.rados.rados.0.smithi084.stdout:2268: done (1 left) 2024-09-18T14:09:17.268 INFO:tasks.rados.rados.0.smithi084.stdout:2270: done (0 left) 2024-09-18T14:09:17.268 INFO:tasks.rados.rados.0.smithi084.stdout:2271: write oid 5 current snap is 238 2024-09-18T14:09:17.268 INFO:tasks.rados.rados.0.smithi084.stdout:2271: seq_num 802 ranges {679214=566763,1798088=715267,3157946=321268} 2024-09-18T14:09:17.275 INFO:tasks.rados.rados.0.smithi084.stdout:2271: writing smithi08434663-5 from 679214 to 1245977 tid 1 2024-09-18T14:09:17.282 INFO:tasks.rados.rados.0.smithi084.stdout:2271: writing smithi08434663-5 from 1798088 to 2513355 tid 2 2024-09-18T14:09:17.288 INFO:tasks.rados.rados.0.smithi084.stdout:2271: writing smithi08434663-5 from 3157946 to 3479214 tid 3 2024-09-18T14:09:17.288 INFO:tasks.rados.rados.0.smithi084.stdout:2272: read oid 42 snap -1 2024-09-18T14:09:17.288 INFO:tasks.rados.rados.0.smithi084.stdout:2272: expect deleted 2024-09-18T14:09:17.288 INFO:tasks.rados.rados.0.smithi084.stdout:2273: rollback oid 39 current snap is 238 2024-09-18T14:09:17.288 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 39 to 231 2024-09-18T14:09:17.288 INFO:tasks.rados.rados.0.smithi084.stdout:2271: finishing write tid 1 to smithi08434663-5 2024-09-18T14:09:17.288 INFO:tasks.rados.rados.0.smithi084.stdout:2271: finishing write tid 2 to smithi08434663-5 2024-09-18T14:09:17.288 INFO:tasks.rados.rados.0.smithi084.stdout:2274: snap_remove snap 232 2024-09-18T14:09:18.263 INFO:tasks.rados.rados.0.smithi084.stdout:2274: done (3 left) 2024-09-18T14:09:18.263 INFO:tasks.rados.rados.0.smithi084.stdout:2275: copy_from oid 21 from oid 1 current snap is 238 2024-09-18T14:09:18.263 INFO:tasks.rados.rados.0.smithi084.stdout:2271: finishing write tid 3 to smithi08434663-5 2024-09-18T14:09:18.263 INFO:tasks.rados.rados.0.smithi084.stdout:2273: finishing rollback tid 1 to smithi08434663-39 2024-09-18T14:09:18.263 INFO:tasks.rados.rados.0.smithi084.stdout:2271: finishing write tid 4 to smithi08434663-5 2024-09-18T14:09:18.263 INFO:tasks.rados.rados.0.smithi084.stdout:2273: finishing rollback tid 2 to smithi08434663-39 2024-09-18T14:09:18.263 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 1049 (ObjNum 761 snap 221 seq_num 761) dirty exists 2024-09-18T14:09:18.263 INFO:tasks.rados.rados.0.smithi084.stdout:2271: finishing write tid 5 to smithi08434663-5 2024-09-18T14:09:18.264 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 943 (ObjNum 802 snap 238 seq_num 802) dirty exists 2024-09-18T14:09:18.264 INFO:tasks.rados.rados.0.smithi084.stdout:2271: left oid 5 (ObjNum 802 snap 238 seq_num 802) 2024-09-18T14:09:18.264 INFO:tasks.rados.rados.0.smithi084.stdout:2271: done (3 left) 2024-09-18T14:09:18.264 INFO:tasks.rados.rados.0.smithi084.stdout:2272: done (2 left) 2024-09-18T14:09:18.264 INFO:tasks.rados.rados.0.smithi084.stdout:2273: done (1 left) 2024-09-18T14:09:18.264 INFO:tasks.rados.rados.0.smithi084.stdout:2276: write (excl) oid 15 current snap is 238 2024-09-18T14:09:18.264 INFO:tasks.rados.rados.0.smithi084.stdout:2276: seq_num 803 ranges {696021=642012,1856570=575278,3185369=310652} 2024-09-18T14:09:18.270 INFO:tasks.rados.rados.0.smithi084.stdout:2276: writing smithi08434663-15 from 696021 to 1338033 tid 1 2024-09-18T14:09:18.276 INFO:tasks.rados.rados.0.smithi084.stdout:2276: writing smithi08434663-15 from 1856570 to 2431848 tid 2 2024-09-18T14:09:18.282 INFO:tasks.rados.rados.0.smithi084.stdout:2276: writing smithi08434663-15 from 3185369 to 3496021 tid 3 2024-09-18T14:09:18.282 INFO:tasks.rados.rados.0.smithi084.stdout:2277: snap_remove snap 229 2024-09-18T14:09:19.264 INFO:tasks.rados.rados.0.smithi084.stdout:2277: done (2 left) 2024-09-18T14:09:19.264 INFO:tasks.rados.rados.0.smithi084.stdout:2278: snap_create 2024-09-18T14:09:19.264 INFO:tasks.rados.rados.0.smithi084.stdout:2275: finishing copy_from racing read to smithi08434663-21 2024-09-18T14:09:19.264 INFO:tasks.rados.rados.0.smithi084.stdout:2276: finishing write tid 1 to smithi08434663-15 2024-09-18T14:09:19.265 INFO:tasks.rados.rados.0.smithi084.stdout:2275: finishing copy_from to smithi08434663-21 2024-09-18T14:09:19.265 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 834 (ObjNum 790 snap 234 seq_num 790) dirty exists 2024-09-18T14:09:19.265 INFO:tasks.rados.rados.0.smithi084.stdout:2276: finishing write tid 2 to smithi08434663-15 2024-09-18T14:09:19.265 INFO:tasks.rados.rados.0.smithi084.stdout:2276: finishing write tid 3 to smithi08434663-15 2024-09-18T14:09:19.265 INFO:tasks.rados.rados.0.smithi084.stdout:2276: finishing write tid 4 to smithi08434663-15 2024-09-18T14:09:19.265 INFO:tasks.rados.rados.0.smithi084.stdout:2276: finishing write tid 5 to smithi08434663-15 2024-09-18T14:09:19.265 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 885 (ObjNum 803 snap 238 seq_num 803) dirty exists 2024-09-18T14:09:19.265 INFO:tasks.rados.rados.0.smithi084.stdout:2276: left oid 15 (ObjNum 803 snap 238 seq_num 803) 2024-09-18T14:09:20.273 INFO:tasks.rados.rados.0.smithi084.stdout:2275: done (2 left) 2024-09-18T14:09:20.273 INFO:tasks.rados.rados.0.smithi084.stdout:2276: done (1 left) 2024-09-18T14:09:20.273 INFO:tasks.rados.rados.0.smithi084.stdout:2278: done (0 left) 2024-09-18T14:09:20.273 INFO:tasks.rados.rados.0.smithi084.stdout:2279: snap_remove snap 238 2024-09-18T14:09:21.275 INFO:tasks.rados.rados.0.smithi084.stdout:2279: done (0 left) 2024-09-18T14:09:21.275 INFO:tasks.rados.rados.0.smithi084.stdout:2280: snap_remove snap 237 2024-09-18T14:09:22.257 INFO:tasks.rados.rados.0.smithi084.stdout:2280: done (0 left) 2024-09-18T14:09:22.257 INFO:tasks.rados.rados.0.smithi084.stdout:2281: snap_create 2024-09-18T14:09:23.258 INFO:tasks.rados.rados.0.smithi084.stdout:2281: done (0 left) 2024-09-18T14:09:23.258 INFO:tasks.rados.rados.0.smithi084.stdout:2282: snap_remove snap 235 2024-09-18T14:09:24.259 INFO:tasks.rados.rados.0.smithi084.stdout:2282: done (0 left) 2024-09-18T14:09:24.260 INFO:tasks.rados.rados.0.smithi084.stdout:2283: read oid 46 snap 239 2024-09-18T14:09:24.260 INFO:tasks.rados.rados.0.smithi084.stdout:2283: expect (ObjNum 545 snap 167 seq_num 545) 2024-09-18T14:09:24.260 INFO:tasks.rados.rados.0.smithi084.stdout:2284: write (excl) oid 9 current snap is 240 2024-09-18T14:09:24.260 INFO:tasks.rados.rados.0.smithi084.stdout:2284: seq_num 804 ranges {712828=717261,1915052=718936,3212792=300036} 2024-09-18T14:09:24.268 INFO:tasks.rados.rados.0.smithi084.stdout:2284: writing smithi08434663-9 from 712828 to 1430089 tid 1 2024-09-18T14:09:24.275 INFO:tasks.rados.rados.0.smithi084.stdout:2284: writing smithi08434663-9 from 1915052 to 2633988 tid 2 2024-09-18T14:09:24.280 INFO:tasks.rados.rados.0.smithi084.stdout:2284: writing smithi08434663-9 from 3212792 to 3512828 tid 3 2024-09-18T14:09:24.280 INFO:tasks.rados.rados.0.smithi084.stdout:2285: rollback oid 35 current snap is 240 2024-09-18T14:09:24.280 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 35 to 224 2024-09-18T14:09:24.280 INFO:tasks.rados.rados.0.smithi084.stdout:2283: expect (ObjNum 545 snap 167 seq_num 545) 2024-09-18T14:09:24.483 INFO:tasks.rados.rados.0.smithi084.stdout:2284: finishing write tid 1 to smithi08434663-9 2024-09-18T14:09:24.483 INFO:tasks.rados.rados.0.smithi084.stdout:2285: finishing rollback tid 1 to smithi08434663-35 2024-09-18T14:09:24.483 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 0 (ObjNum 33 snap 0 seq_num 2495332064) dirty dne 2024-09-18T14:09:24.483 INFO:tasks.rados.rados.0.smithi084.stdout:2284: finishing write tid 2 to smithi08434663-9 2024-09-18T14:09:24.483 INFO:tasks.rados.rados.0.smithi084.stdout:2284: finishing write tid 3 to smithi08434663-9 2024-09-18T14:09:24.483 INFO:tasks.rados.rados.0.smithi084.stdout:2284: finishing write tid 4 to smithi08434663-9 2024-09-18T14:09:24.483 INFO:tasks.rados.rados.0.smithi084.stdout:2284: finishing write tid 5 to smithi08434663-9 2024-09-18T14:09:24.484 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 895 (ObjNum 804 snap 240 seq_num 804) dirty exists 2024-09-18T14:09:24.484 INFO:tasks.rados.rados.0.smithi084.stdout:2284: left oid 9 (ObjNum 804 snap 240 seq_num 804) 2024-09-18T14:09:24.484 INFO:tasks.rados.rados.0.smithi084.stdout:2283: done (2 left) 2024-09-18T14:09:24.484 INFO:tasks.rados.rados.0.smithi084.stdout:2284: done (1 left) 2024-09-18T14:09:24.484 INFO:tasks.rados.rados.0.smithi084.stdout:2285: done (0 left) 2024-09-18T14:09:24.484 INFO:tasks.rados.rados.0.smithi084.stdout:2286: write (excl) oid 10 current snap is 240 2024-09-18T14:09:24.484 INFO:tasks.rados.rados.0.smithi084.stdout:2286: seq_num 805 ranges {729635=792510,1973534=578947,3240215=289420} 2024-09-18T14:09:24.490 INFO:tasks.rados.rados.0.smithi084.stdout:2286: writing smithi08434663-10 from 729635 to 1522145 tid 1 2024-09-18T14:09:24.496 INFO:tasks.rados.rados.0.smithi084.stdout:2286: writing smithi08434663-10 from 1973534 to 2552481 tid 2 2024-09-18T14:09:24.502 INFO:tasks.rados.rados.0.smithi084.stdout:2286: writing smithi08434663-10 from 3240215 to 3529635 tid 3 2024-09-18T14:09:24.502 INFO:tasks.rados.rados.0.smithi084.stdout:2287: snap_create 2024-09-18T14:09:24.502 INFO:tasks.rados.rados.0.smithi084.stdout:2286: finishing write tid 1 to smithi08434663-10 2024-09-18T14:09:24.502 INFO:tasks.rados.rados.0.smithi084.stdout:2286: finishing write tid 2 to smithi08434663-10 2024-09-18T14:09:24.506 INFO:tasks.rados.rados.0.smithi084.stdout:2286: finishing write tid 3 to smithi08434663-10 2024-09-18T14:09:24.512 INFO:tasks.rados.rados.0.smithi084.stdout:2286: finishing write tid 4 to smithi08434663-10 2024-09-18T14:09:24.513 INFO:tasks.rados.rados.0.smithi084.stdout:2286: finishing write tid 5 to smithi08434663-10 2024-09-18T14:09:24.513 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 845 (ObjNum 805 snap 240 seq_num 805) dirty exists 2024-09-18T14:09:24.513 INFO:tasks.rados.rados.0.smithi084.stdout:2286: left oid 10 (ObjNum 805 snap 240 seq_num 805) 2024-09-18T14:09:25.287 INFO:tasks.rados.rados.0.smithi084.stdout:2286: done (1 left) 2024-09-18T14:09:25.287 INFO:tasks.rados.rados.0.smithi084.stdout:2287: done (0 left) 2024-09-18T14:09:25.287 INFO:tasks.rados.rados.0.smithi084.stdout:2288: snap_remove snap 236 2024-09-18T14:09:26.294 INFO:tasks.rados.rados.0.smithi084.stdout:2288: done (0 left) 2024-09-18T14:09:26.294 INFO:tasks.rados.rados.0.smithi084.stdout:2289: snap_create 2024-09-18T14:09:27.258 INFO:tasks.rados.rados.0.smithi084.stdout:2289: done (0 left) 2024-09-18T14:09:27.258 INFO:tasks.rados.rados.0.smithi084.stdout:2290: write oid 34 current snap is 242 2024-09-18T14:09:27.258 INFO:tasks.rados.rados.0.smithi084.stdout:2290: seq_num 806 ranges {746442=584112,1748369=722605,2983991=562451} 2024-09-18T14:09:27.265 INFO:tasks.rados.rados.0.smithi084.stdout:2290: writing smithi08434663-34 from 746442 to 1330554 tid 1 2024-09-18T14:09:27.271 INFO:tasks.rados.rados.0.smithi084.stdout:2290: writing smithi08434663-34 from 1748369 to 2470974 tid 2 2024-09-18T14:09:27.278 INFO:tasks.rados.rados.0.smithi084.stdout:2290: writing smithi08434663-34 from 2983991 to 3546442 tid 3 2024-09-18T14:09:27.278 INFO:tasks.rados.rados.0.smithi084.stdout:2291: snap_remove snap 241 2024-09-18T14:09:28.267 INFO:tasks.rados.rados.0.smithi084.stdout:2291: done (1 left) 2024-09-18T14:09:28.267 INFO:tasks.rados.rados.0.smithi084.stdout:2292: read oid 23 snap -1 2024-09-18T14:09:28.267 INFO:tasks.rados.rados.0.smithi084.stdout:2292: expect deleted 2024-09-18T14:09:28.267 INFO:tasks.rados.rados.0.smithi084.stdout:2293: read oid 24 snap -1 2024-09-18T14:09:28.267 INFO:tasks.rados.rados.0.smithi084.stdout:2293: expect (ObjNum 794 snap 236 seq_num 794) 2024-09-18T14:09:28.267 INFO:tasks.rados.rados.0.smithi084.stdout:2294: copy_from oid 2 from oid 50 current snap is 242 2024-09-18T14:09:28.267 INFO:tasks.rados.rados.0.smithi084.stdout:2290: finishing write tid 1 to smithi08434663-34 2024-09-18T14:09:28.267 INFO:tasks.rados.rados.0.smithi084.stdout:2290: finishing write tid 2 to smithi08434663-34 2024-09-18T14:09:28.267 INFO:tasks.rados.rados.0.smithi084.stdout:2290: finishing write tid 3 to smithi08434663-34 2024-09-18T14:09:28.267 INFO:tasks.rados.rados.0.smithi084.stdout:2290: finishing write tid 4 to smithi08434663-34 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2290: finishing write tid 5 to smithi08434663-34 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1033 (ObjNum 806 snap 242 seq_num 806) dirty exists 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2290: left oid 34 (ObjNum 806 snap 242 seq_num 806) 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2290: done (3 left) 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2295: rollback oid 21 current snap is 242 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 21 to 239 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2296: copy_from oid 6 from oid 22 current snap is 242 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2297: read oid 3 snap -1 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2297: expect deleted 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2298: read oid 1 snap 239 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2298: expect (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2299: snap_create 2024-09-18T14:09:28.268 INFO:tasks.rados.rados.0.smithi084.stdout:2296: finishing copy_from to smithi08434663-6 2024-09-18T14:09:28.269 INFO:tasks.rados.rados.0.smithi084.stdout:2296: got expected ENOENT (src dne) 2024-09-18T14:09:28.269 INFO:tasks.rados.rados.0.smithi084.stdout:2296: finishing copy_from racing read to smithi08434663-6 2024-09-18T14:09:28.269 INFO:tasks.rados.rados.0.smithi084.stdout:2294: finishing copy_from to smithi08434663-2 2024-09-18T14:09:28.269 INFO:tasks.rados.rados.0.smithi084.stdout:2294: got expected ENOENT (src dne) 2024-09-18T14:09:28.269 INFO:tasks.rados.rados.0.smithi084.stdout:2294: finishing copy_from racing read to smithi08434663-2 2024-09-18T14:09:28.272 INFO:tasks.rados.rados.0.smithi084.stdout:2295: finishing rollback tid 0 to smithi08434663-21 2024-09-18T14:09:28.272 INFO:tasks.rados.rados.0.smithi084.stdout:2298: expect (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:09:28.440 INFO:tasks.rados.rados.0.smithi084.stdout:2293: expect (ObjNum 794 snap 236 seq_num 794) 2024-09-18T14:09:28.545 INFO:tasks.rados.rados.0.smithi084.stdout:2295: finishing rollback tid 1 to smithi08434663-21 2024-09-18T14:09:28.545 INFO:tasks.rados.rados.0.smithi084.stdout:2295: finishing rollback tid 2 to smithi08434663-21 2024-09-18T14:09:28.545 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 837 (ObjNum 790 snap 234 seq_num 790) dirty exists 2024-09-18T14:09:29.270 INFO:tasks.rados.rados.0.smithi084.stdout:2292: done (7 left) 2024-09-18T14:09:29.270 INFO:tasks.rados.rados.0.smithi084.stdout:2293: done (6 left) 2024-09-18T14:09:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2294: done (5 left) 2024-09-18T14:09:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2295: done (4 left) 2024-09-18T14:09:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2296: done (3 left) 2024-09-18T14:09:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2297: done (2 left) 2024-09-18T14:09:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2298: done (1 left) 2024-09-18T14:09:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2299: done (0 left) 2024-09-18T14:09:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2300: copy_from oid 46 from oid 34 current snap is 243 2024-09-18T14:09:29.271 INFO:tasks.rados.rados.0.smithi084.stdout:2301: snap_create 2024-09-18T14:09:29.290 INFO:tasks.rados.rados.0.smithi084.stdout:2300: finishing copy_from racing read to smithi08434663-46 2024-09-18T14:09:29.295 INFO:tasks.rados.rados.0.smithi084.stdout:2300: finishing copy_from to smithi08434663-46 2024-09-18T14:09:29.295 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1033 (ObjNum 806 snap 242 seq_num 806) dirty exists 2024-09-18T14:09:30.275 INFO:tasks.rados.rados.0.smithi084.stdout:2300: done (1 left) 2024-09-18T14:09:30.275 INFO:tasks.rados.rados.0.smithi084.stdout:2301: done (0 left) 2024-09-18T14:09:30.276 INFO:tasks.rados.rados.0.smithi084.stdout:2302: snap_remove snap 240 2024-09-18T14:09:31.277 INFO:tasks.rados.rados.0.smithi084.stdout:2302: done (0 left) 2024-09-18T14:09:31.278 INFO:tasks.rados.rados.0.smithi084.stdout:2303: write oid 39 current snap is 244 2024-09-18T14:09:31.278 INFO:tasks.rados.rados.0.smithi084.stdout:2303: seq_num 807 ranges {763249=659361,2090498=582616,3295061=268188} 2024-09-18T14:09:31.285 INFO:tasks.rados.rados.0.smithi084.stdout:2303: writing smithi08434663-39 from 763249 to 1422610 tid 1 2024-09-18T14:09:31.292 INFO:tasks.rados.rados.0.smithi084.stdout:2303: writing smithi08434663-39 from 2090498 to 2673114 tid 2 2024-09-18T14:09:31.297 INFO:tasks.rados.rados.0.smithi084.stdout:2303: writing smithi08434663-39 from 3295061 to 3563249 tid 3 2024-09-18T14:09:31.297 INFO:tasks.rados.rados.0.smithi084.stdout:2304: write oid 37 current snap is 244 2024-09-18T14:09:31.298 INFO:tasks.rados.rados.0.smithi084.stdout:2304: seq_num 808 ranges {780056=734610,2148980=726274,3322484=257572} 2024-09-18T14:09:31.305 INFO:tasks.rados.rados.0.smithi084.stdout:2304: writing smithi08434663-37 from 780056 to 1514666 tid 1 2024-09-18T14:09:31.313 INFO:tasks.rados.rados.0.smithi084.stdout:2304: writing smithi08434663-37 from 2148980 to 2875254 tid 2 2024-09-18T14:09:31.318 INFO:tasks.rados.rados.0.smithi084.stdout:2304: writing smithi08434663-37 from 3322484 to 3580056 tid 3 2024-09-18T14:09:31.318 INFO:tasks.rados.rados.0.smithi084.stdout:2305: write oid 46 current snap is 244 2024-09-18T14:09:31.318 INFO:tasks.rados.rados.0.smithi084.stdout:2303: finishing write tid 1 to smithi08434663-39 2024-09-18T14:09:31.318 INFO:tasks.rados.rados.0.smithi084.stdout:2303: finishing write tid 2 to smithi08434663-39 2024-09-18T14:09:31.318 INFO:tasks.rados.rados.0.smithi084.stdout:2303: finishing write tid 3 to smithi08434663-39 2024-09-18T14:09:31.318 INFO:tasks.rados.rados.0.smithi084.stdout:2303: finishing write tid 4 to smithi08434663-39 2024-09-18T14:09:31.318 INFO:tasks.rados.rados.0.smithi084.stdout:2303: finishing write tid 5 to smithi08434663-39 2024-09-18T14:09:31.318 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 1053 (ObjNum 807 snap 244 seq_num 807) dirty exists 2024-09-18T14:09:31.318 INFO:tasks.rados.rados.0.smithi084.stdout:2303: left oid 39 (ObjNum 807 snap 244 seq_num 807) 2024-09-18T14:09:31.318 INFO:tasks.rados.rados.0.smithi084.stdout:2304: finishing write tid 1 to smithi08434663-37 2024-09-18T14:09:31.318 INFO:tasks.rados.rados.0.smithi084.stdout:2305: seq_num 809 ranges {796863=409859,1807462=586285,3066260=530603} 2024-09-18T14:09:31.324 INFO:tasks.rados.rados.0.smithi084.stdout:2305: writing smithi08434663-46 from 796863 to 1206722 tid 1 2024-09-18T14:09:31.330 INFO:tasks.rados.rados.0.smithi084.stdout:2305: writing smithi08434663-46 from 1807462 to 2393747 tid 2 2024-09-18T14:09:31.337 INFO:tasks.rados.rados.0.smithi084.stdout:2305: writing smithi08434663-46 from 3066260 to 3596863 tid 3 2024-09-18T14:09:31.337 INFO:tasks.rados.rados.0.smithi084.stdout:2303: done (2 left) 2024-09-18T14:09:31.337 INFO:tasks.rados.rados.0.smithi084.stdout:2306: write oid 11 current snap is 244 2024-09-18T14:09:31.337 INFO:tasks.rados.rados.0.smithi084.stdout:2304: finishing write tid 2 to smithi08434663-37 2024-09-18T14:09:31.337 INFO:tasks.rados.rados.0.smithi084.stdout:2306: seq_num 810 ranges {413670=485108,1465944=729943,2977330=524801} 2024-09-18T14:09:31.342 INFO:tasks.rados.rados.0.smithi084.stdout:2306: writing smithi08434663-11 from 413670 to 898778 tid 1 2024-09-18T14:09:31.349 INFO:tasks.rados.rados.0.smithi084.stdout:2306: writing smithi08434663-11 from 1465944 to 2195887 tid 2 2024-09-18T14:09:31.356 INFO:tasks.rados.rados.0.smithi084.stdout:2306: writing smithi08434663-11 from 2977330 to 3502131 tid 3 2024-09-18T14:09:31.356 INFO:tasks.rados.rados.0.smithi084.stdout:2307: snap_remove snap 239 2024-09-18T14:09:32.261 INFO:tasks.rados.rados.0.smithi084.stdout:2307: done (3 left) 2024-09-18T14:09:32.261 INFO:tasks.rados.rados.0.smithi084.stdout:2308: snap_create 2024-09-18T14:09:32.261 INFO:tasks.rados.rados.0.smithi084.stdout:2304: finishing write tid 3 to smithi08434663-37 2024-09-18T14:09:32.261 INFO:tasks.rados.rados.0.smithi084.stdout:2304: finishing write tid 4 to smithi08434663-37 2024-09-18T14:09:32.261 INFO:tasks.rados.rados.0.smithi084.stdout:2304: finishing write tid 5 to smithi08434663-37 2024-09-18T14:09:32.261 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 1000 (ObjNum 808 snap 244 seq_num 808) dirty exists 2024-09-18T14:09:32.261 INFO:tasks.rados.rados.0.smithi084.stdout:2304: left oid 37 (ObjNum 808 snap 244 seq_num 808) 2024-09-18T14:09:32.261 INFO:tasks.rados.rados.0.smithi084.stdout:2305: finishing write tid 1 to smithi08434663-46 2024-09-18T14:09:32.261 INFO:tasks.rados.rados.0.smithi084.stdout:2305: finishing write tid 2 to smithi08434663-46 2024-09-18T14:09:32.262 INFO:tasks.rados.rados.0.smithi084.stdout:2305: finishing write tid 3 to smithi08434663-46 2024-09-18T14:09:32.262 INFO:tasks.rados.rados.0.smithi084.stdout:2305: finishing write tid 4 to smithi08434663-46 2024-09-18T14:09:32.262 INFO:tasks.rados.rados.0.smithi084.stdout:2305: finishing write tid 5 to smithi08434663-46 2024-09-18T14:09:32.262 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1037 (ObjNum 809 snap 244 seq_num 809) dirty exists 2024-09-18T14:09:32.262 INFO:tasks.rados.rados.0.smithi084.stdout:2305: left oid 46 (ObjNum 809 snap 244 seq_num 809) 2024-09-18T14:09:32.262 INFO:tasks.rados.rados.0.smithi084.stdout:2306: finishing write tid 1 to smithi08434663-11 2024-09-18T14:09:32.262 INFO:tasks.rados.rados.0.smithi084.stdout:2306: finishing write tid 2 to smithi08434663-11 2024-09-18T14:09:32.262 INFO:tasks.rados.rados.0.smithi084.stdout:2306: finishing write tid 3 to smithi08434663-11 2024-09-18T14:09:32.262 INFO:tasks.rados.rados.0.smithi084.stdout:2306: finishing write tid 4 to smithi08434663-11 2024-09-18T14:09:32.262 INFO:tasks.rados.rados.0.smithi084.stdout:2306: finishing write tid 5 to smithi08434663-11 2024-09-18T14:09:32.262 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1057 (ObjNum 810 snap 244 seq_num 810) dirty exists 2024-09-18T14:09:32.262 INFO:tasks.rados.rados.0.smithi084.stdout:2306: left oid 11 (ObjNum 810 snap 244 seq_num 810) 2024-09-18T14:09:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:2304: done (3 left) 2024-09-18T14:09:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:2305: done (2 left) 2024-09-18T14:09:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:2306: done (1 left) 2024-09-18T14:09:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:2308: done (0 left) 2024-09-18T14:09:33.260 INFO:tasks.rados.rados.0.smithi084.stdout:2309: snap_remove snap 242 2024-09-18T14:09:34.262 INFO:tasks.rados.rados.0.smithi084.stdout:2309: done (0 left) 2024-09-18T14:09:34.263 INFO:tasks.rados.rados.0.smithi084.stdout:2310: read oid 16 snap -1 2024-09-18T14:09:34.263 INFO:tasks.rados.rados.0.smithi084.stdout:2310: expect (ObjNum 779 snap 224 seq_num 779) 2024-09-18T14:09:34.263 INFO:tasks.rados.rados.0.smithi084.stdout:2311: rollback oid 19 current snap is 245 2024-09-18T14:09:34.263 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 19 to 243 2024-09-18T14:09:34.263 INFO:tasks.rados.rados.0.smithi084.stdout:2312: read oid 49 snap 227 2024-09-18T14:09:34.263 INFO:tasks.rados.rados.0.smithi084.stdout:2312: expect (ObjNum 672 snap 200 seq_num 672) 2024-09-18T14:09:34.263 INFO:tasks.rados.rados.0.smithi084.stdout:2313: read oid 32 snap -1 2024-09-18T14:09:34.263 INFO:tasks.rados.rados.0.smithi084.stdout:2313: expect deleted 2024-09-18T14:09:34.263 INFO:tasks.rados.rados.0.smithi084.stdout:2314: snap_remove snap 231 2024-09-18T14:09:35.268 INFO:tasks.rados.rados.0.smithi084.stdout:2314: done (4 left) 2024-09-18T14:09:35.268 INFO:tasks.rados.rados.0.smithi084.stdout:2315: snap_create 2024-09-18T14:09:35.268 INFO:tasks.rados.rados.0.smithi084.stdout:2311: finishing rollback tid 0 to smithi08434663-19 2024-09-18T14:09:35.268 INFO:tasks.rados.rados.0.smithi084.stdout:2311: finishing rollback tid 1 to smithi08434663-19 2024-09-18T14:09:35.268 INFO:tasks.rados.rados.0.smithi084.stdout:2311: finishing rollback tid 2 to smithi08434663-19 2024-09-18T14:09:35.268 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1125 (ObjNum 759 snap 221 seq_num 759) dirty exists 2024-09-18T14:09:35.268 INFO:tasks.rados.rados.0.smithi084.stdout:2312: expect (ObjNum 672 snap 200 seq_num 672) 2024-09-18T14:09:35.410 INFO:tasks.rados.rados.0.smithi084.stdout:2310: expect (ObjNum 779 snap 224 seq_num 779) 2024-09-18T14:09:36.274 INFO:tasks.rados.rados.0.smithi084.stdout:2310: done (4 left) 2024-09-18T14:09:36.274 INFO:tasks.rados.rados.0.smithi084.stdout:2311: done (3 left) 2024-09-18T14:09:36.274 INFO:tasks.rados.rados.0.smithi084.stdout:2312: done (2 left) 2024-09-18T14:09:36.275 INFO:tasks.rados.rados.0.smithi084.stdout:2313: done (1 left) 2024-09-18T14:09:36.275 INFO:tasks.rados.rados.0.smithi084.stdout:2315: done (0 left) 2024-09-18T14:09:36.275 INFO:tasks.rados.rados.0.smithi084.stdout:2316: write oid 3 current snap is 246 2024-09-18T14:09:36.275 INFO:tasks.rados.rados.0.smithi084.stdout:2316: seq_num 811 ranges {430477=560357,1408073=473601,2488400=736073} 2024-09-18T14:09:36.279 INFO:tasks.rados.rados.0.smithi084.stdout:2316: writing smithi08434663-3 from 430477 to 990834 tid 1 2024-09-18T14:09:36.284 INFO:tasks.rados.rados.0.smithi084.stdout:2316: writing smithi08434663-3 from 1408073 to 1881674 tid 2 2024-09-18T14:09:36.292 INFO:tasks.rados.rados.0.smithi084.stdout:2316: writing smithi08434663-3 from 2488400 to 3224473 tid 3 2024-09-18T14:09:36.292 INFO:tasks.rados.rados.0.smithi084.stdout:2317: delete oid 41 current snap is 246 2024-09-18T14:09:36.292 INFO:tasks.rados.rados.0.smithi084.stdout:2316: finishing write tid 1 to smithi08434663-3 2024-09-18T14:09:36.292 INFO:tasks.rados.rados.0.smithi084.stdout:2316: finishing write tid 2 to smithi08434663-3 2024-09-18T14:09:36.296 INFO:tasks.rados.rados.0.smithi084.stdout:2316: finishing write tid 3 to smithi08434663-3 2024-09-18T14:09:36.296 INFO:tasks.rados.rados.0.smithi084.stdout:2316: finishing write tid 4 to smithi08434663-3 2024-09-18T14:09:36.296 INFO:tasks.rados.rados.0.smithi084.stdout:2316: finishing write tid 5 to smithi08434663-3 2024-09-18T14:09:36.296 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 773 (ObjNum 811 snap 246 seq_num 811) dirty exists 2024-09-18T14:09:36.296 INFO:tasks.rados.rados.0.smithi084.stdout:2316: left oid 3 (ObjNum 811 snap 246 seq_num 811) 2024-09-18T14:09:36.297 INFO:tasks.rados.rados.0.smithi084.stdout:2316: done (1 left) 2024-09-18T14:09:36.297 INFO:tasks.rados.rados.0.smithi084.stdout:2317: done (0 left) 2024-09-18T14:09:36.297 INFO:tasks.rados.rados.0.smithi084.stdout:2318: rollback oid 24 current snap is 246 2024-09-18T14:09:36.297 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 24 to 244 2024-09-18T14:09:36.297 INFO:tasks.rados.rados.0.smithi084.stdout:2319: write (excl) oid 29 current snap is 246 2024-09-18T14:09:36.297 INFO:tasks.rados.rados.0.smithi084.stdout:2319: seq_num 813 ranges {464091=710855,1925037=477270,2943246=474970} 2024-09-18T14:09:36.303 INFO:tasks.rados.rados.0.smithi084.stdout:2319: writing smithi08434663-29 from 464091 to 1174946 tid 1 2024-09-18T14:09:36.309 INFO:tasks.rados.rados.0.smithi084.stdout:2319: writing smithi08434663-29 from 1925037 to 2402307 tid 2 2024-09-18T14:09:36.315 INFO:tasks.rados.rados.0.smithi084.stdout:2319: writing smithi08434663-29 from 2943246 to 3418216 tid 3 2024-09-18T14:09:36.315 INFO:tasks.rados.rados.0.smithi084.stdout:2320: snap_create 2024-09-18T14:09:36.315 INFO:tasks.rados.rados.0.smithi084.stdout:2318: finishing rollback tid 0 to smithi08434663-24 2024-09-18T14:09:36.315 INFO:tasks.rados.rados.0.smithi084.stdout:2318: finishing rollback tid 1 to smithi08434663-24 2024-09-18T14:09:36.315 INFO:tasks.rados.rados.0.smithi084.stdout:2318: finishing rollback tid 2 to smithi08434663-24 2024-09-18T14:09:36.315 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 582 (ObjNum 794 snap 236 seq_num 794) dirty exists 2024-09-18T14:09:36.316 INFO:tasks.rados.rados.0.smithi084.stdout:2319: finishing write tid 1 to smithi08434663-29 2024-09-18T14:09:36.316 INFO:tasks.rados.rados.0.smithi084.stdout:2319: finishing write tid 2 to smithi08434663-29 2024-09-18T14:09:36.318 INFO:tasks.rados.rados.0.smithi084.stdout:2319: finishing write tid 3 to smithi08434663-29 2024-09-18T14:09:36.318 INFO:tasks.rados.rados.0.smithi084.stdout:2319: finishing write tid 4 to smithi08434663-29 2024-09-18T14:09:36.318 INFO:tasks.rados.rados.0.smithi084.stdout:2319: finishing write tid 5 to smithi08434663-29 2024-09-18T14:09:36.318 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 879 (ObjNum 813 snap 246 seq_num 813) dirty exists 2024-09-18T14:09:36.318 INFO:tasks.rados.rados.0.smithi084.stdout:2319: left oid 29 (ObjNum 813 snap 246 seq_num 813) 2024-09-18T14:09:37.261 INFO:tasks.rados.rados.0.smithi084.stdout:2318: done (2 left) 2024-09-18T14:09:37.261 INFO:tasks.rados.rados.0.smithi084.stdout:2319: done (1 left) 2024-09-18T14:09:37.261 INFO:tasks.rados.rados.0.smithi084.stdout:2320: done (0 left) 2024-09-18T14:09:37.261 INFO:tasks.rados.rados.0.smithi084.stdout:2321: read oid 35 snap -1 2024-09-18T14:09:37.261 INFO:tasks.rados.rados.0.smithi084.stdout:2321: expect deleted 2024-09-18T14:09:37.261 INFO:tasks.rados.rados.0.smithi084.stdout:2322: write oid 16 current snap is 247 2024-09-18T14:09:37.261 INFO:tasks.rados.rados.0.smithi084.stdout:2322: seq_num 814 ranges {480898=502457,1699872=737281,3087022=593876} 2024-09-18T14:09:37.266 INFO:tasks.rados.rados.0.smithi084.stdout:2322: writing smithi08434663-16 from 480898 to 983355 tid 1 2024-09-18T14:09:37.274 INFO:tasks.rados.rados.0.smithi084.stdout:2322: writing smithi08434663-16 from 1699872 to 2437153 tid 2 2024-09-18T14:09:37.281 INFO:tasks.rados.rados.0.smithi084.stdout:2322: writing smithi08434663-16 from 3087022 to 3680898 tid 3 2024-09-18T14:09:37.281 INFO:tasks.rados.rados.0.smithi084.stdout:2323: delete oid 30 current snap is 247 2024-09-18T14:09:37.281 INFO:tasks.rados.rados.0.smithi084.stdout:2322: finishing write tid 1 to smithi08434663-16 2024-09-18T14:09:37.282 INFO:tasks.rados.rados.0.smithi084.stdout:2322: finishing write tid 2 to smithi08434663-16 2024-09-18T14:09:37.284 INFO:tasks.rados.rados.0.smithi084.stdout:2321: done (2 left) 2024-09-18T14:09:37.284 INFO:tasks.rados.rados.0.smithi084.stdout:2323: done (1 left) 2024-09-18T14:09:37.284 INFO:tasks.rados.rados.0.smithi084.stdout:2324: write oid 23 current snap is 247 2024-09-18T14:09:37.284 INFO:tasks.rados.rados.0.smithi084.stdout:2324: seq_num 816 ranges {514512=652955,1700483=740950,3025515=688997} 2024-09-18T14:09:37.290 INFO:tasks.rados.rados.0.smithi084.stdout:2324: writing smithi08434663-23 from 514512 to 1167467 tid 1 2024-09-18T14:09:37.297 INFO:tasks.rados.rados.0.smithi084.stdout:2324: writing smithi08434663-23 from 1700483 to 2441433 tid 2 2024-09-18T14:09:37.305 INFO:tasks.rados.rados.0.smithi084.stdout:2324: writing smithi08434663-23 from 3025515 to 3714512 tid 3 2024-09-18T14:09:37.305 INFO:tasks.rados.rados.0.smithi084.stdout:2325: snap_create 2024-09-18T14:09:37.305 INFO:tasks.rados.rados.0.smithi084.stdout:2322: finishing write tid 3 to smithi08434663-16 2024-09-18T14:09:37.305 INFO:tasks.rados.rados.0.smithi084.stdout:2322: finishing write tid 4 to smithi08434663-16 2024-09-18T14:09:37.305 INFO:tasks.rados.rados.0.smithi084.stdout:2322: finishing write tid 5 to smithi08434663-16 2024-09-18T14:09:37.305 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1041 (ObjNum 814 snap 247 seq_num 814) dirty exists 2024-09-18T14:09:37.305 INFO:tasks.rados.rados.0.smithi084.stdout:2322: left oid 16 (ObjNum 814 snap 247 seq_num 814) 2024-09-18T14:09:37.305 INFO:tasks.rados.rados.0.smithi084.stdout:2324: finishing write tid 1 to smithi08434663-23 2024-09-18T14:09:37.305 INFO:tasks.rados.rados.0.smithi084.stdout:2324: finishing write tid 2 to smithi08434663-23 2024-09-18T14:09:37.308 INFO:tasks.rados.rados.0.smithi084.stdout:2324: finishing write tid 3 to smithi08434663-23 2024-09-18T14:09:37.308 INFO:tasks.rados.rados.0.smithi084.stdout:2324: finishing write tid 4 to smithi08434663-23 2024-09-18T14:09:37.308 INFO:tasks.rados.rados.0.smithi084.stdout:2324: finishing write tid 5 to smithi08434663-23 2024-09-18T14:09:37.308 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 942 (ObjNum 816 snap 247 seq_num 816) dirty exists 2024-09-18T14:09:37.308 INFO:tasks.rados.rados.0.smithi084.stdout:2324: left oid 23 (ObjNum 816 snap 247 seq_num 816) 2024-09-18T14:09:38.276 INFO:tasks.rados.rados.0.smithi084.stdout:2322: done (2 left) 2024-09-18T14:09:38.276 INFO:tasks.rados.rados.0.smithi084.stdout:2324: done (1 left) 2024-09-18T14:09:38.276 INFO:tasks.rados.rados.0.smithi084.stdout:2325: done (0 left) 2024-09-18T14:09:38.276 INFO:tasks.rados.rados.0.smithi084.stdout:2326: snap_create 2024-09-18T14:09:39.281 INFO:tasks.rados.rados.0.smithi084.stdout:2326: done (0 left) 2024-09-18T14:09:39.281 INFO:tasks.rados.rados.0.smithi084.stdout:2327: read oid 3 snap -1 2024-09-18T14:09:39.281 INFO:tasks.rados.rados.0.smithi084.stdout:2327: expect (ObjNum 811 snap 246 seq_num 811) 2024-09-18T14:09:39.281 INFO:tasks.rados.rados.0.smithi084.stdout:2328: read oid 11 snap -1 2024-09-18T14:09:39.282 INFO:tasks.rados.rados.0.smithi084.stdout:2328: expect (ObjNum 810 snap 244 seq_num 810) 2024-09-18T14:09:39.282 INFO:tasks.rados.rados.0.smithi084.stdout:2329: snap_remove snap 248 2024-09-18T14:09:40.288 INFO:tasks.rados.rados.0.smithi084.stdout:2329: done (2 left) 2024-09-18T14:09:40.288 INFO:tasks.rados.rados.0.smithi084.stdout:2330: read oid 32 snap 224 2024-09-18T14:09:40.288 INFO:tasks.rados.rados.0.smithi084.stdout:2330: expect deleted 2024-09-18T14:09:40.288 INFO:tasks.rados.rados.0.smithi084.stdout:2331: read oid 47 snap 243 2024-09-18T14:09:40.288 INFO:tasks.rados.rados.0.smithi084.stdout:2331: expect (ObjNum 796 snap 236 seq_num 796) 2024-09-18T14:09:40.288 INFO:tasks.rados.rados.0.smithi084.stdout:2332: snap_create 2024-09-18T14:09:40.288 INFO:tasks.rados.rados.0.smithi084.stdout:2328: expect (ObjNum 810 snap 244 seq_num 810) 2024-09-18T14:09:40.510 INFO:tasks.rados.rados.0.smithi084.stdout:2327: expect (ObjNum 811 snap 246 seq_num 811) 2024-09-18T14:09:40.623 INFO:tasks.rados.rados.0.smithi084.stdout:2331: expect (ObjNum 796 snap 236 seq_num 796) 2024-09-18T14:09:41.288 INFO:tasks.rados.rados.0.smithi084.stdout:2327: done (4 left) 2024-09-18T14:09:41.288 INFO:tasks.rados.rados.0.smithi084.stdout:2328: done (3 left) 2024-09-18T14:09:41.288 INFO:tasks.rados.rados.0.smithi084.stdout:2330: done (2 left) 2024-09-18T14:09:41.288 INFO:tasks.rados.rados.0.smithi084.stdout:2331: done (1 left) 2024-09-18T14:09:41.288 INFO:tasks.rados.rados.0.smithi084.stdout:2332: done (0 left) 2024-09-18T14:09:41.288 INFO:tasks.rados.rados.0.smithi084.stdout:2333: write oid 41 current snap is 250 2024-09-18T14:09:41.288 INFO:tasks.rados.rados.0.smithi084.stdout:2333: seq_num 817 ranges {531319=728204,1758965=484608,2652938=520058} 2024-09-18T14:09:41.294 INFO:tasks.rados.rados.0.smithi084.stdout:2333: writing smithi08434663-41 from 531319 to 1259523 tid 1 2024-09-18T14:09:41.300 INFO:tasks.rados.rados.0.smithi084.stdout:2333: writing smithi08434663-41 from 1758965 to 2243573 tid 2 2024-09-18T14:09:41.305 INFO:tasks.rados.rados.0.smithi084.stdout:2333: writing smithi08434663-41 from 2652938 to 3172996 tid 3 2024-09-18T14:09:41.305 INFO:tasks.rados.rados.0.smithi084.stdout:2334: write oid 7 current snap is 250 2024-09-18T14:09:41.305 INFO:tasks.rados.rados.0.smithi084.stdout:2334: seq_num 818 ranges {548126=403453,1417447=744619,2680361=447683,3680450=67676} 2024-09-18T14:09:41.310 INFO:tasks.rados.rados.0.smithi084.stdout:2334: writing smithi08434663-7 from 548126 to 951579 tid 1 2024-09-18T14:09:41.316 INFO:tasks.rados.rados.0.smithi084.stdout:2334: writing smithi08434663-7 from 1417447 to 2162066 tid 2 2024-09-18T14:09:41.321 INFO:tasks.rados.rados.0.smithi084.stdout:2334: writing smithi08434663-7 from 2680361 to 3128044 tid 3 2024-09-18T14:09:41.325 INFO:tasks.rados.rados.0.smithi084.stdout:2334: writing smithi08434663-7 from 3680450 to 3748126 tid 4 2024-09-18T14:09:41.325 INFO:tasks.rados.rados.0.smithi084.stdout:2335: write (excl) oid 2 current snap is 250 2024-09-18T14:09:41.325 INFO:tasks.rados.rados.0.smithi084.stdout:2335: seq_num 819 ranges {564933=478702,1759576=488277,2991431=658955} 2024-09-18T14:09:41.330 INFO:tasks.rados.rados.0.smithi084.stdout:2335: writing smithi08434663-2 from 564933 to 1043635 tid 1 2024-09-18T14:09:41.337 INFO:tasks.rados.rados.0.smithi084.stdout:2335: writing smithi08434663-2 from 1759576 to 2247853 tid 2 2024-09-18T14:09:41.346 INFO:tasks.rados.rados.0.smithi084.stdout:2335: writing smithi08434663-2 from 2991431 to 3650386 tid 3 2024-09-18T14:09:41.346 INFO:tasks.rados.rados.0.smithi084.stdout:2336: rollback oid 3 current snap is 250 2024-09-18T14:09:41.346 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 3 to 234 2024-09-18T14:09:41.346 INFO:tasks.rados.rados.0.smithi084.stdout:2333: finishing write tid 1 to smithi08434663-41 2024-09-18T14:09:41.346 INFO:tasks.rados.rados.0.smithi084.stdout:2333: finishing write tid 2 to smithi08434663-41 2024-09-18T14:09:41.346 INFO:tasks.rados.rados.0.smithi084.stdout:2333: finishing write tid 3 to smithi08434663-41 2024-09-18T14:09:41.346 INFO:tasks.rados.rados.0.smithi084.stdout:2333: finishing write tid 4 to smithi08434663-41 2024-09-18T14:09:41.346 INFO:tasks.rados.rados.0.smithi084.stdout:2333: finishing write tid 5 to smithi08434663-41 2024-09-18T14:09:41.346 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 1051 (ObjNum 817 snap 250 seq_num 817) dirty exists 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:2333: left oid 41 (ObjNum 817 snap 250 seq_num 817) 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:2334: finishing write tid 1 to smithi08434663-7 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:2334: finishing write tid 2 to smithi08434663-7 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:2333: done (3 left) 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:2337: delete oid 50 current snap is 250 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:2334: finishing write tid 3 to smithi08434663-7 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:2334: finishing write tid 4 to smithi08434663-7 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:2334: finishing write tid 5 to smithi08434663-7 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:2334: finishing write tid 6 to smithi08434663-7 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 947 (ObjNum 818 snap 250 seq_num 818) dirty exists 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:2334: left oid 7 (ObjNum 818 snap 250 seq_num 818) 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:2335: finishing write tid 1 to smithi08434663-2 2024-09-18T14:09:41.347 INFO:tasks.rados.rados.0.smithi084.stdout:2335: finishing write tid 2 to smithi08434663-2 2024-09-18T14:09:41.349 INFO:tasks.rados.rados.0.smithi084.stdout:2336: finishing rollback tid 0 to smithi08434663-3 2024-09-18T14:09:41.350 INFO:tasks.rados.rados.0.smithi084.stdout:2335: finishing write tid 3 to smithi08434663-2 2024-09-18T14:09:41.350 INFO:tasks.rados.rados.0.smithi084.stdout:2335: finishing write tid 4 to smithi08434663-2 2024-09-18T14:09:41.350 INFO:tasks.rados.rados.0.smithi084.stdout:2336: finishing rollback tid 1 to smithi08434663-3 2024-09-18T14:09:41.350 INFO:tasks.rados.rados.0.smithi084.stdout:2335: finishing write tid 5 to smithi08434663-2 2024-09-18T14:09:41.350 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 1133 (ObjNum 819 snap 250 seq_num 819) dirty exists 2024-09-18T14:09:41.351 INFO:tasks.rados.rados.0.smithi084.stdout:2335: left oid 2 (ObjNum 819 snap 250 seq_num 819) 2024-09-18T14:09:41.351 INFO:tasks.rados.rados.0.smithi084.stdout:2334: done (3 left) 2024-09-18T14:09:41.351 INFO:tasks.rados.rados.0.smithi084.stdout:2335: done (2 left) 2024-09-18T14:09:41.351 INFO:tasks.rados.rados.0.smithi084.stdout:2337: done (1 left) 2024-09-18T14:09:41.351 INFO:tasks.rados.rados.0.smithi084.stdout:2338: snap_remove snap 249 2024-09-18T14:09:42.263 INFO:tasks.rados.rados.0.smithi084.stdout:2338: done (1 left) 2024-09-18T14:09:42.263 INFO:tasks.rados.rados.0.smithi084.stdout:2339: snap_create 2024-09-18T14:09:42.263 INFO:tasks.rados.rados.0.smithi084.stdout:2336: finishing rollback tid 2 to smithi08434663-3 2024-09-18T14:09:42.263 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 776 (ObjNum 770 snap 223 seq_num 770) dirty exists 2024-09-18T14:09:43.263 INFO:tasks.rados.rados.0.smithi084.stdout:2336: done (1 left) 2024-09-18T14:09:43.263 INFO:tasks.rados.rados.0.smithi084.stdout:2339: done (0 left) 2024-09-18T14:09:43.263 INFO:tasks.rados.rados.0.smithi084.stdout:2340: delete oid 28 current snap is 251 2024-09-18T14:09:43.268 INFO:tasks.rados.rados.0.smithi084.stdout:2340: done (0 left) 2024-09-18T14:09:43.268 INFO:tasks.rados.rados.0.smithi084.stdout:2341: read oid 29 snap -1 2024-09-18T14:09:43.268 INFO:tasks.rados.rados.0.smithi084.stdout:2341: expect (ObjNum 813 snap 246 seq_num 813) 2024-09-18T14:09:43.268 INFO:tasks.rados.rados.0.smithi084.stdout:2342: read oid 14 snap 227 2024-09-18T14:09:43.268 INFO:tasks.rados.rados.0.smithi084.stdout:2342: expect deleted 2024-09-18T14:09:43.269 INFO:tasks.rados.rados.0.smithi084.stdout:2343: write (excl) oid 15 current snap is 251 2024-09-18T14:09:43.269 INFO:tasks.rados.rados.0.smithi084.stdout:2343: seq_num 822 ranges {615354=420802,1651375=751957,3190053=492771} 2024-09-18T14:09:43.280 INFO:tasks.rados.rados.0.smithi084.stdout:2343: writing smithi08434663-15 from 615354 to 1036156 tid 1 2024-09-18T14:09:43.283 INFO:tasks.rados.rados.0.smithi084.stdout:2343: writing smithi08434663-15 from 1651375 to 2403332 tid 2 2024-09-18T14:09:43.289 INFO:tasks.rados.rados.0.smithi084.stdout:2343: writing smithi08434663-15 from 3190053 to 3682824 tid 3 2024-09-18T14:09:43.289 INFO:tasks.rados.rados.0.smithi084.stdout:2344: write oid 2 current snap is 251 2024-09-18T14:09:43.290 INFO:tasks.rados.rados.0.smithi084.stdout:2344: seq_num 823 ranges {632161=496051,1593504=495615,2701123=420396,3611645=220516} 2024-09-18T14:09:43.296 INFO:tasks.rados.rados.0.smithi084.stdout:2344: writing smithi08434663-2 from 632161 to 1128212 tid 1 2024-09-18T14:09:43.301 INFO:tasks.rados.rados.0.smithi084.stdout:2344: writing smithi08434663-2 from 1593504 to 2089119 tid 2 2024-09-18T14:09:43.306 INFO:tasks.rados.rados.0.smithi084.stdout:2344: writing smithi08434663-2 from 2701123 to 3121519 tid 3 2024-09-18T14:09:43.311 INFO:tasks.rados.rados.0.smithi084.stdout:2344: writing smithi08434663-2 from 3611645 to 3832161 tid 4 2024-09-18T14:09:43.311 INFO:tasks.rados.rados.0.smithi084.stdout:2345: read oid 39 snap -1 2024-09-18T14:09:43.311 INFO:tasks.rados.rados.0.smithi084.stdout:2345: expect (ObjNum 807 snap 244 seq_num 807) 2024-09-18T14:09:43.311 INFO:tasks.rados.rados.0.smithi084.stdout:2346: delete oid 40 current snap is 251 2024-09-18T14:09:43.311 INFO:tasks.rados.rados.0.smithi084.stdout:2343: finishing write tid 1 to smithi08434663-15 2024-09-18T14:09:43.311 INFO:tasks.rados.rados.0.smithi084.stdout:2341: expect (ObjNum 813 snap 246 seq_num 813) 2024-09-18T14:09:43.429 INFO:tasks.rados.rados.0.smithi084.stdout:2343: finishing write tid 2 to smithi08434663-15 2024-09-18T14:09:43.429 INFO:tasks.rados.rados.0.smithi084.stdout:2343: finishing write tid 3 to smithi08434663-15 2024-09-18T14:09:43.429 INFO:tasks.rados.rados.0.smithi084.stdout:2343: finishing write tid 4 to smithi08434663-15 2024-09-18T14:09:43.429 INFO:tasks.rados.rados.0.smithi084.stdout:2343: finishing write tid 5 to smithi08434663-15 2024-09-18T14:09:43.429 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 1045 (ObjNum 822 snap 251 seq_num 822) dirty exists 2024-09-18T14:09:43.429 INFO:tasks.rados.rados.0.smithi084.stdout:2343: left oid 15 (ObjNum 822 snap 251 seq_num 822) 2024-09-18T14:09:43.429 INFO:tasks.rados.rados.0.smithi084.stdout:2344: finishing write tid 1 to smithi08434663-2 2024-09-18T14:09:43.429 INFO:tasks.rados.rados.0.smithi084.stdout:2341: done (5 left) 2024-09-18T14:09:43.429 INFO:tasks.rados.rados.0.smithi084.stdout:2342: done (4 left) 2024-09-18T14:09:43.429 INFO:tasks.rados.rados.0.smithi084.stdout:2343: done (3 left) 2024-09-18T14:09:43.429 INFO:tasks.rados.rados.0.smithi084.stdout:2346: done (2 left) 2024-09-18T14:09:43.429 INFO:tasks.rados.rados.0.smithi084.stdout:2347: write oid 3 current snap is 251 2024-09-18T14:09:43.430 INFO:tasks.rados.rados.0.smithi084.stdout:2344: finishing write tid 2 to smithi08434663-2 2024-09-18T14:09:43.430 INFO:tasks.rados.rados.0.smithi084.stdout:2344: finishing write tid 3 to smithi08434663-2 2024-09-18T14:09:43.430 INFO:tasks.rados.rados.0.smithi084.stdout:2344: finishing write tid 4 to smithi08434663-2 2024-09-18T14:09:43.430 INFO:tasks.rados.rados.0.smithi084.stdout:2344: finishing write tid 5 to smithi08434663-2 2024-09-18T14:09:43.430 INFO:tasks.rados.rados.0.smithi084.stdout:2344: finishing write tid 6 to smithi08434663-2 2024-09-18T14:09:43.430 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 1142 (ObjNum 823 snap 251 seq_num 823) dirty exists 2024-09-18T14:09:43.430 INFO:tasks.rados.rados.0.smithi084.stdout:2344: left oid 2 (ObjNum 823 snap 251 seq_num 823) 2024-09-18T14:09:43.430 INFO:tasks.rados.rados.0.smithi084.stdout:2347: seq_num 825 ranges {665775=646549,2110468=499284,3155969=442940} 2024-09-18T14:09:43.435 INFO:tasks.rados.rados.0.smithi084.stdout:2347: writing smithi08434663-3 from 665775 to 1312324 tid 1 2024-09-18T14:09:43.442 INFO:tasks.rados.rados.0.smithi084.stdout:2347: writing smithi08434663-3 from 2110468 to 2609752 tid 2 2024-09-18T14:09:43.448 INFO:tasks.rados.rados.0.smithi084.stdout:2347: writing smithi08434663-3 from 3155969 to 3598909 tid 3 2024-09-18T14:09:43.448 INFO:tasks.rados.rados.0.smithi084.stdout:2344: done (2 left) 2024-09-18T14:09:43.448 INFO:tasks.rados.rados.0.smithi084.stdout:2348: read oid 26 snap 244 2024-09-18T14:09:43.448 INFO:tasks.rados.rados.0.smithi084.stdout:2348: expect (ObjNum 780 snap 224 seq_num 780) 2024-09-18T14:09:43.448 INFO:tasks.rados.rados.0.smithi084.stdout:2349: write oid 28 current snap is 251 2024-09-18T14:09:43.448 INFO:tasks.rados.rados.0.smithi084.stdout:2345: expect (ObjNum 807 snap 244 seq_num 807) 2024-09-18T14:09:43.637 INFO:tasks.rados.rados.0.smithi084.stdout:2347: finishing write tid 1 to smithi08434663-3 2024-09-18T14:09:43.637 INFO:tasks.rados.rados.0.smithi084.stdout:2347: finishing write tid 2 to smithi08434663-3 2024-09-18T14:09:43.637 INFO:tasks.rados.rados.0.smithi084.stdout:2348: expect (ObjNum 780 snap 224 seq_num 780) 2024-09-18T14:09:43.773 INFO:tasks.rados.rados.0.smithi084.stdout:2347: finishing write tid 3 to smithi08434663-3 2024-09-18T14:09:43.773 INFO:tasks.rados.rados.0.smithi084.stdout:2347: finishing write tid 4 to smithi08434663-3 2024-09-18T14:09:43.773 INFO:tasks.rados.rados.0.smithi084.stdout:2347: finishing write tid 5 to smithi08434663-3 2024-09-18T14:09:43.773 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 780 (ObjNum 825 snap 251 seq_num 825) dirty exists 2024-09-18T14:09:43.773 INFO:tasks.rados.rados.0.smithi084.stdout:2347: left oid 3 (ObjNum 825 snap 251 seq_num 825) 2024-09-18T14:09:43.773 INFO:tasks.rados.rados.0.smithi084.stdout:2349: seq_num 826 ranges {682582=721798,2168950=642942,3583392=299190} 2024-09-18T14:09:43.780 INFO:tasks.rados.rados.0.smithi084.stdout:2349: writing smithi08434663-28 from 682582 to 1404380 tid 1 2024-09-18T14:09:43.788 INFO:tasks.rados.rados.0.smithi084.stdout:2349: writing smithi08434663-28 from 2168950 to 2811892 tid 2 2024-09-18T14:09:43.794 INFO:tasks.rados.rados.0.smithi084.stdout:2349: writing smithi08434663-28 from 3583392 to 3882582 tid 3 2024-09-18T14:09:43.794 INFO:tasks.rados.rados.0.smithi084.stdout:2345: done (3 left) 2024-09-18T14:09:43.794 INFO:tasks.rados.rados.0.smithi084.stdout:2347: done (2 left) 2024-09-18T14:09:43.794 INFO:tasks.rados.rados.0.smithi084.stdout:2348: done (1 left) 2024-09-18T14:09:43.794 INFO:tasks.rados.rados.0.smithi084.stdout:2350: write (excl) oid 23 current snap is 251 2024-09-18T14:09:43.794 INFO:tasks.rados.rados.0.smithi084.stdout:2349: finishing write tid 1 to smithi08434663-28 2024-09-18T14:09:43.795 INFO:tasks.rados.rados.0.smithi084.stdout:2349: finishing write tid 2 to smithi08434663-28 2024-09-18T14:09:43.795 INFO:tasks.rados.rados.0.smithi084.stdout:2350: seq_num 827 ranges {699389=797047,2111079=502953,3094462=581837} 2024-09-18T14:09:43.802 INFO:tasks.rados.rados.0.smithi084.stdout:2350: writing smithi08434663-23 from 699389 to 1496436 tid 1 2024-09-18T14:09:43.808 INFO:tasks.rados.rados.0.smithi084.stdout:2350: writing smithi08434663-23 from 2111079 to 2614032 tid 2 2024-09-18T14:09:43.814 INFO:tasks.rados.rados.0.smithi084.stdout:2350: writing smithi08434663-23 from 3094462 to 3676299 tid 3 2024-09-18T14:09:43.814 INFO:tasks.rados.rados.0.smithi084.stdout:2351: read oid 30 snap 244 2024-09-18T14:09:43.814 INFO:tasks.rados.rados.0.smithi084.stdout:2351: expect (ObjNum 753 snap 219 seq_num 753) 2024-09-18T14:09:43.814 INFO:tasks.rados.rados.0.smithi084.stdout:2352: write (excl) oid 29 current snap is 251 2024-09-18T14:09:43.814 INFO:tasks.rados.rados.0.smithi084.stdout:2349: finishing write tid 3 to smithi08434663-28 2024-09-18T14:09:43.814 INFO:tasks.rados.rados.0.smithi084.stdout:2349: finishing write tid 4 to smithi08434663-28 2024-09-18T14:09:43.814 INFO:tasks.rados.rados.0.smithi084.stdout:2349: finishing write tid 5 to smithi08434663-28 2024-09-18T14:09:43.814 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1146 (ObjNum 826 snap 251 seq_num 826) dirty exists 2024-09-18T14:09:43.815 INFO:tasks.rados.rados.0.smithi084.stdout:2349: left oid 28 (ObjNum 826 snap 251 seq_num 826) 2024-09-18T14:09:43.815 INFO:tasks.rados.rados.0.smithi084.stdout:2350: finishing write tid 1 to smithi08434663-23 2024-09-18T14:09:43.815 INFO:tasks.rados.rados.0.smithi084.stdout:2350: finishing write tid 2 to smithi08434663-23 2024-09-18T14:09:43.815 INFO:tasks.rados.rados.0.smithi084.stdout:2352: seq_num 828 ranges {716196=472296,1769561=646611,3121885=793109} 2024-09-18T14:09:43.820 INFO:tasks.rados.rados.0.smithi084.stdout:2352: writing smithi08434663-29 from 716196 to 1188492 tid 1 2024-09-18T14:09:43.826 INFO:tasks.rados.rados.0.smithi084.stdout:2352: writing smithi08434663-29 from 1769561 to 2416172 tid 2 2024-09-18T14:09:43.835 INFO:tasks.rados.rados.0.smithi084.stdout:2352: writing smithi08434663-29 from 3121885 to 3914994 tid 3 2024-09-18T14:09:43.835 INFO:tasks.rados.rados.0.smithi084.stdout:2349: done (3 left) 2024-09-18T14:09:43.835 INFO:tasks.rados.rados.0.smithi084.stdout:2353: copy_from oid 20 from oid 16 current snap is 251 2024-09-18T14:09:43.835 INFO:tasks.rados.rados.0.smithi084.stdout:2350: finishing write tid 3 to smithi08434663-23 2024-09-18T14:09:43.835 INFO:tasks.rados.rados.0.smithi084.stdout:2351: expect (ObjNum 753 snap 219 seq_num 753) 2024-09-18T14:09:43.965 INFO:tasks.rados.rados.0.smithi084.stdout:2350: finishing write tid 4 to smithi08434663-23 2024-09-18T14:09:43.966 INFO:tasks.rados.rados.0.smithi084.stdout:2350: finishing write tid 5 to smithi08434663-23 2024-09-18T14:09:43.966 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 951 (ObjNum 827 snap 251 seq_num 827) dirty exists 2024-09-18T14:09:43.966 INFO:tasks.rados.rados.0.smithi084.stdout:2350: left oid 23 (ObjNum 827 snap 251 seq_num 827) 2024-09-18T14:09:43.966 INFO:tasks.rados.rados.0.smithi084.stdout:2352: finishing write tid 1 to smithi08434663-29 2024-09-18T14:09:43.966 INFO:tasks.rados.rados.0.smithi084.stdout:2352: finishing write tid 2 to smithi08434663-29 2024-09-18T14:09:43.966 INFO:tasks.rados.rados.0.smithi084.stdout:2352: finishing write tid 3 to smithi08434663-29 2024-09-18T14:09:43.966 INFO:tasks.rados.rados.0.smithi084.stdout:2352: finishing write tid 4 to smithi08434663-29 2024-09-18T14:09:43.966 INFO:tasks.rados.rados.0.smithi084.stdout:2352: finishing write tid 5 to smithi08434663-29 2024-09-18T14:09:43.966 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 884 (ObjNum 828 snap 251 seq_num 828) dirty exists 2024-09-18T14:09:43.966 INFO:tasks.rados.rados.0.smithi084.stdout:2352: left oid 29 (ObjNum 828 snap 251 seq_num 828) 2024-09-18T14:09:43.966 INFO:tasks.rados.rados.0.smithi084.stdout:2353: finishing copy_from racing read to smithi08434663-20 2024-09-18T14:09:43.967 INFO:tasks.rados.rados.0.smithi084.stdout:2353: finishing copy_from to smithi08434663-20 2024-09-18T14:09:43.967 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 1041 (ObjNum 814 snap 247 seq_num 814) dirty exists 2024-09-18T14:09:43.967 INFO:tasks.rados.rados.0.smithi084.stdout:2350: done (3 left) 2024-09-18T14:09:43.967 INFO:tasks.rados.rados.0.smithi084.stdout:2351: done (2 left) 2024-09-18T14:09:43.967 INFO:tasks.rados.rados.0.smithi084.stdout:2352: done (1 left) 2024-09-18T14:09:43.967 INFO:tasks.rados.rados.0.smithi084.stdout:2353: done (0 left) 2024-09-18T14:09:43.967 INFO:tasks.rados.rados.0.smithi084.stdout:2354: delete oid 47 current snap is 251 2024-09-18T14:09:43.969 INFO:tasks.rados.rados.0.smithi084.stdout:2354: done (0 left) 2024-09-18T14:09:43.969 INFO:tasks.rados.rados.0.smithi084.stdout:2355: rollback oid 37 current snap is 251 2024-09-18T14:09:43.969 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 37 to 245 2024-09-18T14:09:43.969 INFO:tasks.rados.rados.0.smithi084.stdout:2356: rollback oid 40 current snap is 251 2024-09-18T14:09:43.969 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 40 to 247 2024-09-18T14:09:43.969 INFO:tasks.rados.rados.0.smithi084.stdout:2357: snap_create 2024-09-18T14:09:43.971 INFO:tasks.rados.rados.0.smithi084.stdout:2356: finishing rollback tid 1 to smithi08434663-40 2024-09-18T14:09:43.972 INFO:tasks.rados.rados.0.smithi084.stdout:2356: finishing rollback tid 2 to smithi08434663-40 2024-09-18T14:09:43.972 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 886 (ObjNum 780 snap 224 seq_num 780) dirty exists 2024-09-18T14:09:43.973 INFO:tasks.rados.rados.0.smithi084.stdout:2355: finishing rollback tid 0 to smithi08434663-37 2024-09-18T14:09:43.973 INFO:tasks.rados.rados.0.smithi084.stdout:2355: finishing rollback tid 1 to smithi08434663-37 2024-09-18T14:09:43.976 INFO:tasks.rados.rados.0.smithi084.stdout:2355: finishing rollback tid 2 to smithi08434663-37 2024-09-18T14:09:43.976 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 1003 (ObjNum 808 snap 244 seq_num 808) dirty exists 2024-09-18T14:09:44.303 INFO:tasks.rados.rados.0.smithi084.stdout:2355: done (2 left) 2024-09-18T14:09:44.303 INFO:tasks.rados.rados.0.smithi084.stdout:2356: done (1 left) 2024-09-18T14:09:44.303 INFO:tasks.rados.rados.0.smithi084.stdout:2357: done (0 left) 2024-09-18T14:09:44.303 INFO:tasks.rados.rados.0.smithi084.stdout:2358: snap_remove snap 247 2024-09-18T14:09:45.306 INFO:tasks.rados.rados.0.smithi084.stdout:2358: done (0 left) 2024-09-18T14:09:45.307 INFO:tasks.rados.rados.0.smithi084.stdout:2359: snap_remove snap 246 2024-09-18T14:09:46.314 INFO:tasks.rados.rados.0.smithi084.stdout:2359: done (0 left) 2024-09-18T14:09:46.314 INFO:tasks.rados.rados.0.smithi084.stdout:2360: delete oid 44 current snap is 252 2024-09-18T14:09:46.324 INFO:tasks.rados.rados.0.smithi084.stdout:2360: done (0 left) 2024-09-18T14:09:46.324 INFO:tasks.rados.rados.0.smithi084.stdout:2361: delete oid 30 current snap is 252 2024-09-18T14:09:46.327 INFO:tasks.rados.rados.0.smithi084.stdout:2361: done (0 left) 2024-09-18T14:09:46.327 INFO:tasks.rados.rados.0.smithi084.stdout:2362: read oid 15 snap -1 2024-09-18T14:09:46.327 INFO:tasks.rados.rados.0.smithi084.stdout:2362: expect (ObjNum 822 snap 251 seq_num 822) 2024-09-18T14:09:46.327 INFO:tasks.rados.rados.0.smithi084.stdout:2363: delete oid 46 current snap is 252 2024-09-18T14:09:46.335 INFO:tasks.rados.rados.0.smithi084.stdout:2362: expect (ObjNum 822 snap 251 seq_num 822) 2024-09-18T14:09:46.510 INFO:tasks.rados.rados.0.smithi084.stdout:2362: done (1 left) 2024-09-18T14:09:46.510 INFO:tasks.rados.rados.0.smithi084.stdout:2363: done (0 left) 2024-09-18T14:09:46.510 INFO:tasks.rados.rados.0.smithi084.stdout:2364: rollback oid 32 current snap is 252 2024-09-18T14:09:46.510 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 32 to 234 2024-09-18T14:09:46.510 INFO:tasks.rados.rados.0.smithi084.stdout:2365: read oid 45 snap -1 2024-09-18T14:09:46.510 INFO:tasks.rados.rados.0.smithi084.stdout:2365: expect (ObjNum 672 snap 200 seq_num 672) 2024-09-18T14:09:46.510 INFO:tasks.rados.rados.0.smithi084.stdout:2366: read oid 13 snap -1 2024-09-18T14:09:46.510 INFO:tasks.rados.rados.0.smithi084.stdout:2366: expect (ObjNum 756 snap 220 seq_num 756) 2024-09-18T14:09:46.511 INFO:tasks.rados.rados.0.smithi084.stdout:2367: snap_remove snap 245 2024-09-18T14:09:47.269 INFO:tasks.rados.rados.0.smithi084.stdout:2367: done (3 left) 2024-09-18T14:09:47.269 INFO:tasks.rados.rados.0.smithi084.stdout:2368: read oid 30 snap -1 2024-09-18T14:09:47.269 INFO:tasks.rados.rados.0.smithi084.stdout:2368: expect deleted 2024-09-18T14:09:47.269 INFO:tasks.rados.rados.0.smithi084.stdout:2369: read oid 41 snap -1 2024-09-18T14:09:47.269 INFO:tasks.rados.rados.0.smithi084.stdout:2369: expect (ObjNum 817 snap 250 seq_num 817) 2024-09-18T14:09:47.269 INFO:tasks.rados.rados.0.smithi084.stdout:2370: read oid 21 snap -1 2024-09-18T14:09:47.270 INFO:tasks.rados.rados.0.smithi084.stdout:2370: expect (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:09:47.270 INFO:tasks.rados.rados.0.smithi084.stdout:2371: write oid 20 current snap is 252 2024-09-18T14:09:47.270 INFO:tasks.rados.rados.0.smithi084.stdout:2364: finishing rollback tid 1 to smithi08434663-32 2024-09-18T14:09:47.270 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-18T14:09:47.270 INFO:tasks.rados.rados.0.smithi084.stdout:2371: seq_num 833 ranges {400231=564894,1661971=338260} 2024-09-18T14:09:47.271 INFO:tasks.rados.rados.0.smithi084.stdout:2371: writing smithi08434663-20 from 400231 to 965125 tid 1 2024-09-18T14:09:47.277 INFO:tasks.rados.rados.0.smithi084.stdout:2371: writing smithi08434663-20 from 1661971 to 2000231 tid 2 2024-09-18T14:09:47.277 INFO:tasks.rados.rados.0.smithi084.stdout:2366: expect (ObjNum 756 snap 220 seq_num 756) 2024-09-18T14:09:47.496 INFO:tasks.rados.rados.0.smithi084.stdout:2365: expect (ObjNum 672 snap 200 seq_num 672) 2024-09-18T14:09:47.636 INFO:tasks.rados.rados.0.smithi084.stdout:2370: expect (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:09:47.802 INFO:tasks.rados.rados.0.smithi084.stdout:2369: expect (ObjNum 817 snap 250 seq_num 817) 2024-09-18T14:09:47.921 INFO:tasks.rados.rados.0.smithi084.stdout:2371: finishing write tid 1 to smithi08434663-20 2024-09-18T14:09:47.921 INFO:tasks.rados.rados.0.smithi084.stdout:2371: finishing write tid 2 to smithi08434663-20 2024-09-18T14:09:47.921 INFO:tasks.rados.rados.0.smithi084.stdout:2371: finishing write tid 3 to smithi08434663-20 2024-09-18T14:09:47.921 INFO:tasks.rados.rados.0.smithi084.stdout:2371: finishing write tid 4 to smithi08434663-20 2024-09-18T14:09:47.921 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 1044 (ObjNum 833 snap 252 seq_num 833) dirty exists 2024-09-18T14:09:47.921 INFO:tasks.rados.rados.0.smithi084.stdout:2371: left oid 20 (ObjNum 833 snap 252 seq_num 833) 2024-09-18T14:09:47.921 INFO:tasks.rados.rados.0.smithi084.stdout:2364: done (6 left) 2024-09-18T14:09:47.921 INFO:tasks.rados.rados.0.smithi084.stdout:2365: done (5 left) 2024-09-18T14:09:47.921 INFO:tasks.rados.rados.0.smithi084.stdout:2366: done (4 left) 2024-09-18T14:09:47.921 INFO:tasks.rados.rados.0.smithi084.stdout:2368: done (3 left) 2024-09-18T14:09:47.922 INFO:tasks.rados.rados.0.smithi084.stdout:2369: done (2 left) 2024-09-18T14:09:47.922 INFO:tasks.rados.rados.0.smithi084.stdout:2370: done (1 left) 2024-09-18T14:09:47.922 INFO:tasks.rados.rados.0.smithi084.stdout:2371: done (0 left) 2024-09-18T14:09:47.922 INFO:tasks.rados.rados.0.smithi084.stdout:2372: read oid 24 snap -1 2024-09-18T14:09:47.922 INFO:tasks.rados.rados.0.smithi084.stdout:2372: expect (ObjNum 794 snap 236 seq_num 794) 2024-09-18T14:09:47.922 INFO:tasks.rados.rados.0.smithi084.stdout:2373: read oid 26 snap -1 2024-09-18T14:09:47.922 INFO:tasks.rados.rados.0.smithi084.stdout:2373: expect (ObjNum 780 snap 224 seq_num 780) 2024-09-18T14:09:47.922 INFO:tasks.rados.rados.0.smithi084.stdout:2374: write oid 5 current snap is 252 2024-09-18T14:09:47.922 INFO:tasks.rados.rados.0.smithi084.stdout:2374: seq_num 834 ranges {417038=640143,1720453=296585} 2024-09-18T14:09:47.926 INFO:tasks.rados.rados.0.smithi084.stdout:2374: writing smithi08434663-5 from 417038 to 1057181 tid 1 2024-09-18T14:09:47.932 INFO:tasks.rados.rados.0.smithi084.stdout:2374: writing smithi08434663-5 from 1720453 to 2017038 tid 2 2024-09-18T14:09:47.932 INFO:tasks.rados.rados.0.smithi084.stdout:2375: write oid 40 current snap is 252 2024-09-18T14:09:47.932 INFO:tasks.rados.rados.0.smithi084.stdout:2375: seq_num 835 ranges {433845=715392,1662582=371263} 2024-09-18T14:09:47.938 INFO:tasks.rados.rados.0.smithi084.stdout:2375: writing smithi08434663-40 from 433845 to 1149237 tid 1 2024-09-18T14:09:47.943 INFO:tasks.rados.rados.0.smithi084.stdout:2375: writing smithi08434663-40 from 1662582 to 2033845 tid 2 2024-09-18T14:09:47.943 INFO:tasks.rados.rados.0.smithi084.stdout:2376: write oid 49 current snap is 252 2024-09-18T14:09:47.943 INFO:tasks.rados.rados.0.smithi084.stdout:2376: seq_num 836 ranges {450652=790641,1721064=329588} 2024-09-18T14:09:47.950 INFO:tasks.rados.rados.0.smithi084.stdout:2376: writing smithi08434663-49 from 450652 to 1241293 tid 1 2024-09-18T14:09:47.955 INFO:tasks.rados.rados.0.smithi084.stdout:2376: writing smithi08434663-49 from 1721064 to 2050652 tid 2 2024-09-18T14:09:47.955 INFO:tasks.rados.rados.0.smithi084.stdout:2377: delete oid 8 current snap is 252 2024-09-18T14:09:47.955 INFO:tasks.rados.rados.0.smithi084.stdout:2373: expect (ObjNum 780 snap 224 seq_num 780) 2024-09-18T14:09:48.091 INFO:tasks.rados.rados.0.smithi084.stdout:2372: expect (ObjNum 794 snap 236 seq_num 794) 2024-09-18T14:09:48.196 INFO:tasks.rados.rados.0.smithi084.stdout:2374: finishing write tid 1 to smithi08434663-5 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:2374: finishing write tid 2 to smithi08434663-5 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:2374: finishing write tid 3 to smithi08434663-5 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:2374: finishing write tid 4 to smithi08434663-5 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 947 (ObjNum 834 snap 252 seq_num 834) dirty exists 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:2374: left oid 5 (ObjNum 834 snap 252 seq_num 834) 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:2375: finishing write tid 1 to smithi08434663-40 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:2375: finishing write tid 2 to smithi08434663-40 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:2375: finishing write tid 3 to smithi08434663-40 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:2375: finishing write tid 4 to smithi08434663-40 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 889 (ObjNum 835 snap 252 seq_num 835) dirty exists 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:2375: left oid 40 (ObjNum 835 snap 252 seq_num 835) 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:2376: finishing write tid 1 to smithi08434663-49 2024-09-18T14:09:48.197 INFO:tasks.rados.rados.0.smithi084.stdout:2372: done (5 left) 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:2373: done (4 left) 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:2374: done (3 left) 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:2375: done (2 left) 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:2377: done (1 left) 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:2378: read oid 16 snap -1 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:2378: expect (ObjNum 814 snap 247 seq_num 814) 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:2379: write oid 40 current snap is 252 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:2376: finishing write tid 2 to smithi08434663-49 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:2376: finishing write tid 3 to smithi08434663-49 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:2376: finishing write tid 4 to smithi08434663-49 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 1006 (ObjNum 836 snap 252 seq_num 836) dirty exists 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:2376: left oid 49 (ObjNum 836 snap 252 seq_num 836) 2024-09-18T14:09:48.198 INFO:tasks.rados.rados.0.smithi084.stdout:2379: seq_num 838 ranges {484266=657492,1554381=529885} 2024-09-18T14:09:48.202 INFO:tasks.rados.rados.0.smithi084.stdout:2379: writing smithi08434663-40 from 484266 to 1141758 tid 1 2024-09-18T14:09:48.208 INFO:tasks.rados.rados.0.smithi084.stdout:2379: writing smithi08434663-40 from 1554381 to 2084266 tid 2 2024-09-18T14:09:48.208 INFO:tasks.rados.rados.0.smithi084.stdout:2376: done (2 left) 2024-09-18T14:09:48.208 INFO:tasks.rados.rados.0.smithi084.stdout:2380: write oid 18 current snap is 252 2024-09-18T14:09:48.208 INFO:tasks.rados.rados.0.smithi084.stdout:2380: seq_num 839 ranges {501073=732741,1896510=204563} 2024-09-18T14:09:48.214 INFO:tasks.rados.rados.0.smithi084.stdout:2380: writing smithi08434663-18 from 501073 to 1233814 tid 1 2024-09-18T14:09:48.219 INFO:tasks.rados.rados.0.smithi084.stdout:2380: writing smithi08434663-18 from 1896510 to 2101073 tid 2 2024-09-18T14:09:48.219 INFO:tasks.rados.rados.0.smithi084.stdout:2381: read oid 38 snap -1 2024-09-18T14:09:48.219 INFO:tasks.rados.rados.0.smithi084.stdout:2381: expect (ObjNum 789 snap 233 seq_num 789) 2024-09-18T14:09:48.219 INFO:tasks.rados.rados.0.smithi084.stdout:2382: copy_from oid 50 from oid 21 current snap is 252 2024-09-18T14:09:48.220 INFO:tasks.rados.rados.0.smithi084.stdout:2378: expect (ObjNum 814 snap 247 seq_num 814) 2024-09-18T14:09:48.380 INFO:tasks.rados.rados.0.smithi084.stdout:2379: finishing write tid 1 to smithi08434663-40 2024-09-18T14:09:48.380 INFO:tasks.rados.rados.0.smithi084.stdout:2379: finishing write tid 2 to smithi08434663-40 2024-09-18T14:09:48.380 INFO:tasks.rados.rados.0.smithi084.stdout:2379: finishing write tid 3 to smithi08434663-40 2024-09-18T14:09:48.380 INFO:tasks.rados.rados.0.smithi084.stdout:2379: finishing write tid 4 to smithi08434663-40 2024-09-18T14:09:48.380 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 892 (ObjNum 838 snap 252 seq_num 838) dirty exists 2024-09-18T14:09:48.380 INFO:tasks.rados.rados.0.smithi084.stdout:2379: left oid 40 (ObjNum 838 snap 252 seq_num 838) 2024-09-18T14:09:48.380 INFO:tasks.rados.rados.0.smithi084.stdout:2378: done (4 left) 2024-09-18T14:09:48.381 INFO:tasks.rados.rados.0.smithi084.stdout:2379: done (3 left) 2024-09-18T14:09:48.381 INFO:tasks.rados.rados.0.smithi084.stdout:2383: snap_remove snap 250 2024-09-18T14:09:49.321 INFO:tasks.rados.rados.0.smithi084.stdout:2383: done (3 left) 2024-09-18T14:09:49.321 INFO:tasks.rados.rados.0.smithi084.stdout:2384: read oid 47 snap -1 2024-09-18T14:09:49.322 INFO:tasks.rados.rados.0.smithi084.stdout:2384: expect deleted 2024-09-18T14:09:49.322 INFO:tasks.rados.rados.0.smithi084.stdout:2385: write (excl) oid 42 current snap is 252 2024-09-18T14:09:49.322 INFO:tasks.rados.rados.0.smithi084.stdout:2385: seq_num 840 ranges {517880=407990,1554992=562888} 2024-09-18T14:09:49.327 INFO:tasks.rados.rados.0.smithi084.stdout:2385: writing smithi08434663-42 from 517880 to 925870 tid 1 2024-09-18T14:09:49.334 INFO:tasks.rados.rados.0.smithi084.stdout:2385: writing smithi08434663-42 from 1554992 to 2117880 tid 2 2024-09-18T14:09:49.334 INFO:tasks.rados.rados.0.smithi084.stdout:2386: read oid 43 snap -1 2024-09-18T14:09:49.334 INFO:tasks.rados.rados.0.smithi084.stdout:2386: expect deleted 2024-09-18T14:09:49.334 INFO:tasks.rados.rados.0.smithi084.stdout:2387: snap_remove snap 234 2024-09-18T14:09:50.326 INFO:tasks.rados.rados.0.smithi084.stdout:2387: done (6 left) 2024-09-18T14:09:50.326 INFO:tasks.rados.rados.0.smithi084.stdout:2388: read oid 19 snap -1 2024-09-18T14:09:50.326 INFO:tasks.rados.rados.0.smithi084.stdout:2388: expect (ObjNum 759 snap 221 seq_num 759) 2024-09-18T14:09:50.326 INFO:tasks.rados.rados.0.smithi084.stdout:2389: read oid 17 snap 251 2024-09-18T14:09:50.326 INFO:tasks.rados.rados.0.smithi084.stdout:2389: expect (ObjNum 769 snap 223 seq_num 769) 2024-09-18T14:09:50.327 INFO:tasks.rados.rados.0.smithi084.stdout:2390: read oid 30 snap -1 2024-09-18T14:09:50.327 INFO:tasks.rados.rados.0.smithi084.stdout:2390: expect deleted 2024-09-18T14:09:50.327 INFO:tasks.rados.rados.0.smithi084.stdout:2391: rollback oid 2 current snap is 252 2024-09-18T14:09:50.327 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 2 to 243 2024-09-18T14:09:50.327 INFO:tasks.rados.rados.0.smithi084.stdout:2380: finishing write tid 1 to smithi08434663-18 2024-09-18T14:09:50.327 INFO:tasks.rados.rados.0.smithi084.stdout:2380: finishing write tid 2 to smithi08434663-18 2024-09-18T14:09:50.327 INFO:tasks.rados.rados.0.smithi084.stdout:2380: finishing write tid 3 to smithi08434663-18 2024-09-18T14:09:50.327 INFO:tasks.rados.rados.0.smithi084.stdout:2380: finishing write tid 4 to smithi08434663-18 2024-09-18T14:09:50.327 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 783 (ObjNum 839 snap 252 seq_num 839) dirty exists 2024-09-18T14:09:50.327 INFO:tasks.rados.rados.0.smithi084.stdout:2380: left oid 18 (ObjNum 839 snap 252 seq_num 839) 2024-09-18T14:09:50.327 INFO:tasks.rados.rados.0.smithi084.stdout:2382: finishing copy_from racing read to smithi08434663-50 2024-09-18T14:09:50.327 INFO:tasks.rados.rados.0.smithi084.stdout:2381: expect (ObjNum 789 snap 233 seq_num 789) 2024-09-18T14:09:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:2382: finishing copy_from to smithi08434663-50 2024-09-18T14:09:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 837 (ObjNum 790 snap 234 seq_num 790) dirty exists 2024-09-18T14:09:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:2385: finishing write tid 1 to smithi08434663-42 2024-09-18T14:09:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:2380: done (9 left) 2024-09-18T14:09:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:2381: done (8 left) 2024-09-18T14:09:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:2382: done (7 left) 2024-09-18T14:09:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:2384: done (6 left) 2024-09-18T14:09:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:2392: snap_create 2024-09-18T14:09:50.435 INFO:tasks.rados.rados.0.smithi084.stdout:2385: finishing write tid 2 to smithi08434663-42 2024-09-18T14:09:50.435 INFO:tasks.rados.rados.0.smithi084.stdout:2385: finishing write tid 3 to smithi08434663-42 2024-09-18T14:09:50.435 INFO:tasks.rados.rados.0.smithi084.stdout:2385: finishing write tid 4 to smithi08434663-42 2024-09-18T14:09:50.435 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1056 (ObjNum 840 snap 252 seq_num 840) dirty exists 2024-09-18T14:09:50.435 INFO:tasks.rados.rados.0.smithi084.stdout:2385: left oid 42 (ObjNum 840 snap 252 seq_num 840) 2024-09-18T14:09:50.435 INFO:tasks.rados.rados.0.smithi084.stdout:2388: expect (ObjNum 759 snap 221 seq_num 759) 2024-09-18T14:09:50.550 INFO:tasks.rados.rados.0.smithi084.stdout:2389: expect (ObjNum 769 snap 223 seq_num 769) 2024-09-18T14:09:50.683 INFO:tasks.rados.rados.0.smithi084.stdout:2391: finishing rollback tid 0 to smithi08434663-2 2024-09-18T14:09:50.683 INFO:tasks.rados.rados.0.smithi084.stdout:2391: finishing rollback tid 1 to smithi08434663-2 2024-09-18T14:09:50.683 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 1165 (ObjNum 129 snap 0 seq_num 2495448696) dirty dne 2024-09-18T14:09:51.330 INFO:tasks.rados.rados.0.smithi084.stdout:2385: done (6 left) 2024-09-18T14:09:51.330 INFO:tasks.rados.rados.0.smithi084.stdout:2386: done (5 left) 2024-09-18T14:09:51.330 INFO:tasks.rados.rados.0.smithi084.stdout:2388: done (4 left) 2024-09-18T14:09:51.331 INFO:tasks.rados.rados.0.smithi084.stdout:2389: done (3 left) 2024-09-18T14:09:51.331 INFO:tasks.rados.rados.0.smithi084.stdout:2390: done (2 left) 2024-09-18T14:09:51.331 INFO:tasks.rados.rados.0.smithi084.stdout:2391: done (1 left) 2024-09-18T14:09:51.331 INFO:tasks.rados.rados.0.smithi084.stdout:2392: done (0 left) 2024-09-18T14:09:51.331 INFO:tasks.rados.rados.0.smithi084.stdout:2393: snap_remove snap 244 2024-09-18T14:09:52.337 INFO:tasks.rados.rados.0.smithi084.stdout:2393: done (0 left) 2024-09-18T14:09:52.337 INFO:tasks.rados.rados.0.smithi084.stdout:2394: snap_remove snap 243 2024-09-18T14:09:53.339 INFO:tasks.rados.rados.0.smithi084.stdout:2394: done (0 left) 2024-09-18T14:09:53.340 INFO:tasks.rados.rados.0.smithi084.stdout:2395: copy_from oid 50 from oid 44 current snap is 253 2024-09-18T14:09:53.340 INFO:tasks.rados.rados.0.smithi084.stdout:2396: snap_create 2024-09-18T14:09:53.347 INFO:tasks.rados.rados.0.smithi084.stdout:2395: finishing copy_from to smithi08434663-50 2024-09-18T14:09:53.347 INFO:tasks.rados.rados.0.smithi084.stdout:2395: got expected ENOENT (src dne) 2024-09-18T14:09:53.347 INFO:tasks.rados.rados.0.smithi084.stdout:2395: finishing copy_from racing read to smithi08434663-50 2024-09-18T14:09:54.349 INFO:tasks.rados.rados.0.smithi084.stdout:2395: done (1 left) 2024-09-18T14:09:54.349 INFO:tasks.rados.rados.0.smithi084.stdout:2396: done (0 left) 2024-09-18T14:09:54.349 INFO:tasks.rados.rados.0.smithi084.stdout:2397: copy_from oid 45 from oid 20 current snap is 254 2024-09-18T14:09:54.349 INFO:tasks.rados.rados.0.smithi084.stdout:2398: rollback oid 28 current snap is 254 2024-09-18T14:09:54.349 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 28 to 224 2024-09-18T14:09:54.349 INFO:tasks.rados.rados.0.smithi084.stdout:2399: read oid 44 snap -1 2024-09-18T14:09:54.349 INFO:tasks.rados.rados.0.smithi084.stdout:2399: expect deleted 2024-09-18T14:09:54.349 INFO:tasks.rados.rados.0.smithi084.stdout:2400: snap_create 2024-09-18T14:09:54.357 INFO:tasks.rados.rados.0.smithi084.stdout:2398: finishing rollback tid 0 to smithi08434663-28 2024-09-18T14:09:54.357 INFO:tasks.rados.rados.0.smithi084.stdout:2398: finishing rollback tid 1 to smithi08434663-28 2024-09-18T14:09:54.358 INFO:tasks.rados.rados.0.smithi084.stdout:2398: finishing rollback tid 2 to smithi08434663-28 2024-09-18T14:09:54.358 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1179 (ObjNum 765 snap 223 seq_num 765) dirty exists 2024-09-18T14:09:54.361 INFO:tasks.rados.rados.0.smithi084.stdout:2397: finishing copy_from racing read to smithi08434663-45 2024-09-18T14:09:54.362 INFO:tasks.rados.rados.0.smithi084.stdout:2397: finishing copy_from to smithi08434663-45 2024-09-18T14:09:54.362 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 1044 (ObjNum 833 snap 252 seq_num 833) dirty exists 2024-09-18T14:09:55.353 INFO:tasks.rados.rados.0.smithi084.stdout:2397: done (3 left) 2024-09-18T14:09:55.353 INFO:tasks.rados.rados.0.smithi084.stdout:2398: done (2 left) 2024-09-18T14:09:55.353 INFO:tasks.rados.rados.0.smithi084.stdout:2399: done (1 left) 2024-09-18T14:09:55.353 INFO:tasks.rados.rados.0.smithi084.stdout:2400: done (0 left) 2024-09-18T14:09:55.353 INFO:tasks.rados.rados.0.smithi084.stdout:2401: write (excl) oid 26 current snap is 255 2024-09-18T14:09:55.353 INFO:tasks.rados.rados.0.smithi084.stdout:2401: seq_num 841 ranges {534687=483239,1613474=412283} 2024-09-18T14:09:55.358 INFO:tasks.rados.rados.0.smithi084.stdout:2401: writing smithi08434663-26 from 534687 to 1017926 tid 1 2024-09-18T14:09:55.364 INFO:tasks.rados.rados.0.smithi084.stdout:2401: writing smithi08434663-26 from 1613474 to 2025757 tid 2 2024-09-18T14:09:55.364 INFO:tasks.rados.rados.0.smithi084.stdout:2402: delete oid 4 current snap is 255 2024-09-18T14:09:55.364 INFO:tasks.rados.rados.0.smithi084.stdout:2401: finishing write tid 1 to smithi08434663-26 2024-09-18T14:09:55.367 INFO:tasks.rados.rados.0.smithi084.stdout:2401: finishing write tid 2 to smithi08434663-26 2024-09-18T14:09:55.369 INFO:tasks.rados.rados.0.smithi084.stdout:2401: finishing write tid 3 to smithi08434663-26 2024-09-18T14:09:55.369 INFO:tasks.rados.rados.0.smithi084.stdout:2401: finishing write tid 4 to smithi08434663-26 2024-09-18T14:09:55.369 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 859 (ObjNum 841 snap 255 seq_num 841) dirty exists 2024-09-18T14:09:55.369 INFO:tasks.rados.rados.0.smithi084.stdout:2401: left oid 26 (ObjNum 841 snap 255 seq_num 841) 2024-09-18T14:09:55.372 INFO:tasks.rados.rados.0.smithi084.stdout:2401: done (1 left) 2024-09-18T14:09:55.372 INFO:tasks.rados.rados.0.smithi084.stdout:2402: done (0 left) 2024-09-18T14:09:55.372 INFO:tasks.rados.rados.0.smithi084.stdout:2403: write (excl) oid 35 current snap is 255 2024-09-18T14:09:55.372 INFO:tasks.rados.rados.0.smithi084.stdout:2403: seq_num 843 ranges {568301=633737,1614085=415952} 2024-09-18T14:09:55.378 INFO:tasks.rados.rados.0.smithi084.stdout:2403: writing smithi08434663-35 from 568301 to 1202038 tid 1 2024-09-18T14:09:55.384 INFO:tasks.rados.rados.0.smithi084.stdout:2403: writing smithi08434663-35 from 1614085 to 2030037 tid 2 2024-09-18T14:09:55.384 INFO:tasks.rados.rados.0.smithi084.stdout:2404: read oid 7 snap 252 2024-09-18T14:09:55.384 INFO:tasks.rados.rados.0.smithi084.stdout:2404: expect (ObjNum 818 snap 250 seq_num 818) 2024-09-18T14:09:55.384 INFO:tasks.rados.rados.0.smithi084.stdout:2405: read oid 13 snap -1 2024-09-18T14:09:55.384 INFO:tasks.rados.rados.0.smithi084.stdout:2405: expect (ObjNum 756 snap 220 seq_num 756) 2024-09-18T14:09:55.384 INFO:tasks.rados.rados.0.smithi084.stdout:2406: delete oid 34 current snap is 255 2024-09-18T14:09:55.384 INFO:tasks.rados.rados.0.smithi084.stdout:2403: finishing write tid 1 to smithi08434663-35 2024-09-18T14:09:55.387 INFO:tasks.rados.rados.0.smithi084.stdout:2403: finishing write tid 2 to smithi08434663-35 2024-09-18T14:09:55.388 INFO:tasks.rados.rados.0.smithi084.stdout:2403: finishing write tid 3 to smithi08434663-35 2024-09-18T14:09:55.389 INFO:tasks.rados.rados.0.smithi084.stdout:2403: finishing write tid 4 to smithi08434663-35 2024-09-18T14:09:55.389 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 1049 (ObjNum 843 snap 255 seq_num 843) dirty exists 2024-09-18T14:09:55.389 INFO:tasks.rados.rados.0.smithi084.stdout:2403: left oid 35 (ObjNum 843 snap 255 seq_num 843) 2024-09-18T14:09:55.389 INFO:tasks.rados.rados.0.smithi084.stdout:2403: done (3 left) 2024-09-18T14:09:55.389 INFO:tasks.rados.rados.0.smithi084.stdout:2406: done (2 left) 2024-09-18T14:09:55.390 INFO:tasks.rados.rados.0.smithi084.stdout:2407: rollback oid 1 current snap is 255 2024-09-18T14:09:55.390 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 1 to 254 2024-09-18T14:09:55.390 INFO:tasks.rados.rados.0.smithi084.stdout:2408: write (excl) oid 5 current snap is 255 2024-09-18T14:09:55.390 INFO:tasks.rados.rados.0.smithi084.stdout:2408: seq_num 845 ranges {601915=500588,1847402=354513} 2024-09-18T14:09:55.395 INFO:tasks.rados.rados.0.smithi084.stdout:2408: writing smithi08434663-5 from 601915 to 1102503 tid 1 2024-09-18T14:09:55.402 INFO:tasks.rados.rados.0.smithi084.stdout:2408: writing smithi08434663-5 from 1847402 to 2201915 tid 2 2024-09-18T14:09:55.402 INFO:tasks.rados.rados.0.smithi084.stdout:2409: read oid 27 snap -1 2024-09-18T14:09:55.402 INFO:tasks.rados.rados.0.smithi084.stdout:2409: expect (ObjNum 723 snap 209 seq_num 723) 2024-09-18T14:09:55.402 INFO:tasks.rados.rados.0.smithi084.stdout:2410: write oid 32 current snap is 255 2024-09-18T14:09:55.402 INFO:tasks.rados.rados.0.smithi084.stdout:2410: seq_num 846 ranges {618722=575837,1905884=312838} 2024-09-18T14:09:55.408 INFO:tasks.rados.rados.0.smithi084.stdout:2410: writing smithi08434663-32 from 618722 to 1194559 tid 1 2024-09-18T14:09:55.414 INFO:tasks.rados.rados.0.smithi084.stdout:2410: writing smithi08434663-32 from 1905884 to 2218722 tid 2 2024-09-18T14:09:55.415 INFO:tasks.rados.rados.0.smithi084.stdout:2411: read oid 29 snap -1 2024-09-18T14:09:55.415 INFO:tasks.rados.rados.0.smithi084.stdout:2411: expect (ObjNum 828 snap 251 seq_num 828) 2024-09-18T14:09:55.415 INFO:tasks.rados.rados.0.smithi084.stdout:2412: snap_remove snap 253 2024-09-18T14:09:56.361 INFO:tasks.rados.rados.0.smithi084.stdout:2405: expect (ObjNum 756 snap 220 seq_num 756) 2024-09-18T14:09:56.578 INFO:tasks.rados.rados.0.smithi084.stdout:2407: finishing rollback tid 0 to smithi08434663-1 2024-09-18T14:09:56.578 INFO:tasks.rados.rados.0.smithi084.stdout:2407: finishing rollback tid 1 to smithi08434663-1 2024-09-18T14:09:56.579 INFO:tasks.rados.rados.0.smithi084.stdout:2407: finishing rollback tid 2 to smithi08434663-1 2024-09-18T14:09:56.579 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 863 (ObjNum 790 snap 234 seq_num 790) dirty exists 2024-09-18T14:09:56.579 INFO:tasks.rados.rados.0.smithi084.stdout:2404: expect (ObjNum 818 snap 250 seq_num 818) 2024-09-18T14:09:56.697 INFO:tasks.rados.rados.0.smithi084.stdout:2408: finishing write tid 1 to smithi08434663-5 2024-09-18T14:09:56.697 INFO:tasks.rados.rados.0.smithi084.stdout:2409: expect (ObjNum 723 snap 209 seq_num 723) 2024-09-18T14:09:56.787 INFO:tasks.rados.rados.0.smithi084.stdout:2408: finishing write tid 2 to smithi08434663-5 2024-09-18T14:09:56.787 INFO:tasks.rados.rados.0.smithi084.stdout:2408: finishing write tid 3 to smithi08434663-5 2024-09-18T14:09:56.787 INFO:tasks.rados.rados.0.smithi084.stdout:2408: finishing write tid 4 to smithi08434663-5 2024-09-18T14:09:56.787 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 950 (ObjNum 845 snap 255 seq_num 845) dirty exists 2024-09-18T14:09:56.787 INFO:tasks.rados.rados.0.smithi084.stdout:2408: left oid 5 (ObjNum 845 snap 255 seq_num 845) 2024-09-18T14:09:56.787 INFO:tasks.rados.rados.0.smithi084.stdout:2404: done (7 left) 2024-09-18T14:09:56.787 INFO:tasks.rados.rados.0.smithi084.stdout:2405: done (6 left) 2024-09-18T14:09:56.787 INFO:tasks.rados.rados.0.smithi084.stdout:2407: done (5 left) 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2408: done (4 left) 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2409: done (3 left) 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2412: done (2 left) 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2413: read oid 20 snap -1 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2413: expect (ObjNum 833 snap 252 seq_num 833) 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2414: read oid 46 snap -1 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2414: expect deleted 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2415: read oid 2 snap -1 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2415: expect deleted 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2416: read oid 30 snap -1 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2416: expect deleted 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2417: write (excl) oid 33 current snap is 255 2024-09-18T14:09:56.788 INFO:tasks.rados.rados.0.smithi084.stdout:2410: finishing write tid 1 to smithi08434663-32 2024-09-18T14:09:56.789 INFO:tasks.rados.rados.0.smithi084.stdout:2410: finishing write tid 2 to smithi08434663-32 2024-09-18T14:09:56.789 INFO:tasks.rados.rados.0.smithi084.stdout:2410: finishing write tid 3 to smithi08434663-32 2024-09-18T14:09:56.789 INFO:tasks.rados.rados.0.smithi084.stdout:2410: finishing write tid 4 to smithi08434663-32 2024-09-18T14:09:56.789 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 1182 (ObjNum 846 snap 255 seq_num 846) dirty exists 2024-09-18T14:09:56.789 INFO:tasks.rados.rados.0.smithi084.stdout:2410: left oid 32 (ObjNum 846 snap 255 seq_num 846) 2024-09-18T14:09:56.789 INFO:tasks.rados.rados.0.smithi084.stdout:2411: expect (ObjNum 828 snap 251 seq_num 828) 2024-09-18T14:09:56.839 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-18T14:09:56.839 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-18T14:09:56.839 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-18T14:09:56.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-18T14:09:56.842 INFO:tasks.ceph.mon.g:Started 2024-09-18T14:09:56.842 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-18T14:09:56.842 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-18T14:09:56.842 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-18T14:09:56.842 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-18T14:09:56.844 INFO:tasks.ceph.mon.i:Started 2024-09-18T14:09:56.845 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-09-18T14:09:56.845 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.h 2024-09-18T14:09:56.845 INFO:tasks.ceph.mon.h:Restarting daemon 2024-09-18T14:09:56.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-09-18T14:09:56.847 INFO:tasks.ceph.mon.h:Started 2024-09-18T14:09:56.847 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-18T14:09:56.847 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.d 2024-09-18T14:09:56.847 INFO:tasks.ceph.mon.d:Restarting daemon 2024-09-18T14:09:56.847 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-09-18T14:09:56.850 INFO:tasks.ceph.mon.d:Started 2024-09-18T14:09:56.850 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:09:56.850 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:09:56.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.969+0000 7f936e91d640 10 monclient: get_monmap_and_config 2024-09-18T14:09:56.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.969+0000 7f936e91d640 10 monclient: build_initial_monmap 2024-09-18T14:09:56.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.969+0000 7f936e91d640 10 monclient: monmap: 2024-09-18T14:09:56.970 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:09:56.970 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:09:56.970 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:09:56.970636+0000 2024-09-18T14:09:56.970 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:09:56.970636+0000 2024-09-18T14:09:56.970 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:09:56.970 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:09:56.970 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:09:56.970 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:09:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:09:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:09:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:09:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:09:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:09:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:09:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:09:56.971 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:09:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.970+0000 7f936e91d640 1 Processor -- start 2024-09-18T14:09:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.970+0000 7f936e91d640 1 -- start start 2024-09-18T14:09:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.970+0000 7f936e91d640 10 monclient: init 2024-09-18T14:09:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f936e91d640 10 monclient: _reopen_session rank -1 2024-09-18T14:09:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f936e91d640 10 monclient: _add_conns ranks=[2,8,3,4,0,7,5,6,1] 2024-09-18T14:09:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f936e91d640 10 monclient(hunting): picked mon.noname-b con 0x7f936806e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:09:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f936e91d640 10 monclient(hunting): picked mon.noname-e con 0x7f93680a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:09:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f936e91d640 10 monclient(hunting): picked mon.noname-g con 0x7f93680a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:09:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f936e91d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936805ed70 con 0x7f936806e810 2024-09-18T14:09:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f936e91d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9368097620 con 0x7f93680a1080 2024-09-18T14:09:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f936e91d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9368097340 con 0x7f93680a0cb0 2024-09-18T14:09:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f936e91d640 10 monclient(hunting): _renew_subs 2024-09-18T14:09:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f936e91d640 10 monclient(hunting): authenticate will time out at 1412.309114s 2024-09-18T14:09:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f9367fff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f936806e810 0x7f93680a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:43754/0 (socket says 172.21.15.45:43754) 2024-09-18T14:09:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f9367fff640 1 -- 172.21.15.45:0/3010741436 learned_addr learned my addr 172.21.15.45:0/3010741436 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:09:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:56.971+0000 7f93677fe640 1 --1- 172.21.15.45:0/3010741436 >> v1:172.21.15.45:6793/0 conn(0x7f93680a0cb0 0x7f9368137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:38014/0 (socket says 172.21.15.45:38014) 2024-09-18T14:09:56.983 INFO:tasks.rados.rados.0.smithi084.stdout:2417: seq_num 847 ranges {635529=651086,1848013=387516} 2024-09-18T14:09:56.990 INFO:tasks.rados.rados.0.smithi084.stdout:2417: writing smithi08434663-33 from 635529 to 1286615 tid 1 2024-09-18T14:09:56.996 INFO:tasks.rados.rados.0.smithi084.stdout:2417: writing smithi08434663-33 from 1848013 to 2235529 tid 2 2024-09-18T14:09:56.996 INFO:tasks.rados.rados.0.smithi084.stdout:2410: done (6 left) 2024-09-18T14:09:56.996 INFO:tasks.rados.rados.0.smithi084.stdout:2411: done (5 left) 2024-09-18T14:09:56.996 INFO:tasks.rados.rados.0.smithi084.stdout:2418: snap_remove snap 252 2024-09-18T14:09:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 10 monclient(hunting): tick 2024-09-18T14:09:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 1 monclient(hunting): continuing hunt 2024-09-18T14:09:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:09:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 1 -- 172.21.15.45:0/3010741436 >> v1:172.21.15.45:6793/0 conn(0x7f93680a0cb0 legacy=0x7f9368137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:09:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 1 -- 172.21.15.45:0/3010741436 >> v1:172.21.15.84:6790/0 conn(0x7f93680a1080 legacy=0x7f936813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:09:59.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 1 -- 172.21.15.45:0/3010741436 >> v1:172.21.15.45:6790/0 conn(0x7f936806e810 legacy=0x7f93680a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:09:59.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 10 monclient: _add_conns ranks=[7,4,0,1,5,6] 2024-09-18T14:09:59.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 10 monclient(hunting): picked mon.noname-i con 0x7f93540037a0 addr v1:172.21.15.84:6792/0 2024-09-18T14:09:59.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 10 monclient(hunting): picked mon.noname-c con 0x7f9354007420 addr v1:172.21.15.45:6791/0 2024-09-18T14:09:59.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 10 monclient(hunting): picked mon.noname-a con 0x7f935400b140 addr v1:172.21.15.45:6789/0 2024-09-18T14:09:59.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 1 -- 172.21.15.45:0/3010741436 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93680a1660 con 0x7f935400b140 2024-09-18T14:09:59.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 1 -- 172.21.15.45:0/3010741436 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935400ff70 con 0x7f9354007420 2024-09-18T14:09:59.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 1 -- 172.21.15.45:0/3010741436 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93540111d0 con 0x7f93540037a0 2024-09-18T14:09:59.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:09:59.971+0000 7f9365ffb640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 1 -- 172.21.15.45:0/3010741436 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550806380 0 0) 0x7f935400ff70 con 0x7f9354007420 2024-09-18T14:10:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 10 monclient(hunting): my global_id is 164335 2024-09-18T14:10:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 1 -- 172.21.15.45:0/3010741436 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93500034e0 con 0x7f9354007420 2024-09-18T14:10:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 1 -- 172.21.15.45:0/3010741436 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245675477 0 0) 0x7f93680a1660 con 0x7f935400b140 2024-09-18T14:10:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 10 monclient(hunting): my global_id is 5562 2024-09-18T14:10:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 1 -- 172.21.15.45:0/3010741436 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f935400ff70 con 0x7f935400b140 2024-09-18T14:10:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 1 -- 172.21.15.45:0/3010741436 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2633466751 0 0) 0x7f935400ff70 con 0x7f935400b140 2024-09-18T14:10:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 1 -- 172.21.15.45:0/3010741436 >> v1:172.21.15.84:6792/0 conn(0x7f93540037a0 legacy=0x7f9354005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.973+0000 7f9366ffd640 1 -- 172.21.15.45:0/3010741436 >> v1:172.21.15.45:6791/0 conn(0x7f9354007420 legacy=0x7f9354009890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 1 monclient: found mon.noname-a 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 1 -- 172.21.15.45:0/3010741436 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9354012490 con 0x7f935400b140 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 10 monclient: _finish_auth 0 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f936ceba640 1 -- 172.21.15.45:0/3010741436 reap_dead start 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 1 -- 172.21.15.45:0/3010741436 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9360001a60 con 0x7f935400b140 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 10 monclient: dump: 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f936e91d640 5 monclient: authenticate success, global_id 5562 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f936e91d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 1 -- 172.21.15.45:0/3010741436 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9360002df0 con 0x7f935400b140 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 1 -- 172.21.15.45:0/3010741436 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9360002fe0 con 0x7f935400b140 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f9366ffd640 10 monclient: dump: 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f936e91d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.974+0000 7f936e91d640 4 set_mon_vals no callback set 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.975+0000 7f936e91d640 10 monclient: shutdown 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.975+0000 7f936e91d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.975+0000 7f936e91d640 1 -- 172.21.15.45:0/3010741436 >> v1:172.21.15.45:6789/0 conn(0x7f935400b140 legacy=0x7f935400d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.975+0000 7f936e91d640 1 -- 172.21.15.45:0/3010741436 shutdown_connections 2024-09-18T14:10:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.975+0000 7f936e91d640 1 -- 172.21.15.45:0/3010741436 >> 172.21.15.45:0/3010741436 conn(0x7f93680a5bb0 msgr2=0x7f93680a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.975+0000 7f936e91d640 1 -- 172.21.15.45:0/3010741436 shutdown_connections 2024-09-18T14:10:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.975+0000 7f936e91d640 1 -- 172.21.15.45:0/3010741436 wait complete. 2024-09-18T14:10:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.976+0000 7f936e91d640 10 monclient: build_initial_monmap 2024-09-18T14:10:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.976+0000 7f936e91d640 10 monclient: monmap: 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.976+0000 7f936e91d640 1 Processor -- start 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.977+0000 7f936e91d640 1 -- start start 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.977+0000 7f936e91d640 10 monclient: init 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.977+0000 7f936e91d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.977+0000 7f936e91d640 10 monclient: _add_conns ranks=[6,7,4,3,8,0,5,2,1] 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.978+0000 7f936e91d640 10 monclient(hunting): picked mon.noname-d con 0x7f935400b140 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.978+0000 7f936e91d640 10 monclient(hunting): picked mon.noname-i con 0x7f93681218e0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.978+0000 7f936e91d640 10 monclient(hunting): picked mon.noname-c con 0x7f936813e060 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.978+0000 7f936e91d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9368122d70 con 0x7f936813e060 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.978+0000 7f9367fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f935400b140 0x7f936813d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34724/0 (socket says 172.21.15.45:34724) 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.978+0000 7f9367fff640 1 -- 172.21.15.45:0/608825299 learned_addr learned my addr 172.21.15.45:0/608825299 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.978+0000 7f936e91d640 1 -- 172.21.15.45:0/608825299 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936813de20 con 0x7f935400b140 2024-09-18T14:10:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.978+0000 7f936e91d640 1 -- 172.21.15.45:0/608825299 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9368120d80 con 0x7f93681218e0 2024-09-18T14:10:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.978+0000 7f936e91d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.978+0000 7f936e91d640 10 monclient(hunting): authenticate will time out at 1417.316403s 2024-09-18T14:10:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.982+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495947346 0 0) 0x7f9368122d70 con 0x7f936813e060 2024-09-18T14:10:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.982+0000 7f934ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.982+0000 7f934ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.982+0000 7f934ffff640 10 monclient(hunting): my global_id is 164344 2024-09-18T14:10:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.982+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9354011440 con 0x7f936813e060 2024-09-18T14:10:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.983+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3761942161 0 0) 0x7f9354011440 con 0x7f936813e060 2024-09-18T14:10:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.983+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9368122d70 con 0x7f936813e060 2024-09-18T14:10:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.983+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9360004980 con 0x7f936813e060 2024-09-18T14:10:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.983+0000 7f934ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.983+0000 7f934ffff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:10:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.983+0000 7f934ffff640 10 monclient(hunting): dump: 2024-09-18T14:10:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:01.983+0000 7f934ffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:02.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191922653 0 0) 0x7f936813de20 con 0x7f935400b140 2024-09-18T14:10:02.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 10 monclient(hunting): my global_id is 184110 2024-09-18T14:10:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9354011440 con 0x7f935400b140 2024-09-18T14:10:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 227942527 0 0) 0x7f9368122d70 con 0x7f936813e060 2024-09-18T14:10:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 >> v1:172.21.15.84:6792/0 conn(0x7f93681218e0 legacy=0x7f9368146960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 >> v1:172.21.15.45:6792/0 conn(0x7f935400b140 legacy=0x7f936813d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 10 monclient: _finish_hunting 0 2024-09-18T14:10:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 1 monclient: found mon.c 2024-09-18T14:10:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93681227b0 con 0x7f936813e060 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 10 monclient: _finish_auth 0 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f936e91d640 5 monclient: authenticate success, global_id 164344 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f936e91d640 10 monclient: _renew_subs 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f936e91d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f936e91d640 1 -- 172.21.15.45:0/608825299 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9368122a90 con 0x7f936813e060 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9360004e90 con 0x7f936813e060 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f936e91d640 10 monclient: _renew_subs 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f936e91d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9360005400 con 0x7f936813e060 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f93667fc640 4 set_mon_vals no callback set 2024-09-18T14:10:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f936e91d640 1 -- 172.21.15.45:0/608825299 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f936813ea70 con 0x7f936813e060 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.004+0000 7f934ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.005+0000 7f934ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.005+0000 7f934ffff640 10 monclient: dump: 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.009 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.006+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f93600055e0 con 0x7f936813e060 2024-09-18T14:10:02.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.006+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(526..526 src has 344..526) v4 ==== 4019+0+0 (unknown 398333645 0 0) 0x7f9360046de0 con 0x7f936813e060 2024-09-18T14:10:02.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.006+0000 7f936e91d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:02.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.006+0000 7f936e91d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:02.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.006+0000 7f936e91d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:02.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.006+0000 7f936e91d640 1 -- 172.21.15.45:0/608825299 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936804c4a0 con 0x7f936813e060 2024-09-18T14:10:02.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.006+0000 7f9367fff640 10 monclient: get_auth_request con 0x7f935403b3b0 auth_method 0 2024-09-18T14:10:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.015+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 <== mon.4 v1:172.21.15.45:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f9360015610 con 0x7f936813e060 2024-09-18T14:10:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.015+0000 7f934ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.015+0000 7f934ffff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.242+0000 7f936e91d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:10:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.242+0000 7f936e91d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:10:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.242+0000 7f936e91d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.242+0000 7f936e91d640 1 -- 172.21.15.45:0/608825299 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f93681206c0 con 0x7f936813e060 2024-09-18T14:10:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.242+0000 7f934ffff640 1 -- 172.21.15.45:0/608825299 <== mon.4 v1:172.21.15.45:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 47867129) 0x7f9360015010 con 0x7f936813e060 2024-09-18T14:10:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.242+0000 7f934ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:10:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.242+0000 7f934ffff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:02.243 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":92,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:10:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.244+0000 7f936e91d640 1 -- 172.21.15.45:0/608825299 >> v1:172.21.15.84:6812/31478 conn(0x7f935403b3b0 legacy=0x7f935403d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.244+0000 7f936e91d640 10 monclient: shutdown 2024-09-18T14:10:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.244+0000 7f936e91d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.244+0000 7f936e91d640 1 -- 172.21.15.45:0/608825299 >> v1:172.21.15.45:6791/0 conn(0x7f936813e060 legacy=0x7f936814a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.244+0000 7f936e91d640 1 -- 172.21.15.45:0/608825299 shutdown_connections 2024-09-18T14:10:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.244+0000 7f936e91d640 1 -- 172.21.15.45:0/608825299 >> 172.21.15.45:0/608825299 conn(0x7f93680a5bb0 msgr2=0x7f93680a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.244+0000 7f936e91d640 1 -- 172.21.15.45:0/608825299 shutdown_connections 2024-09-18T14:10:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.244+0000 7f936e91d640 1 -- 172.21.15.45:0/608825299 wait complete. 2024-09-18T14:10:02.255 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:10:02.255 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:10:02.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:10:02.271 INFO:tasks.rados.rados.0.smithi084.stdout:2418: done (5 left) 2024-09-18T14:10:02.271 INFO:tasks.rados.rados.0.smithi084.stdout:2419: snap_remove snap 251 2024-09-18T14:10:02.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.362+0000 7f6d76754640 10 monclient: get_monmap_and_config 2024-09-18T14:10:02.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.362+0000 7f6d76754640 10 monclient: build_initial_monmap 2024-09-18T14:10:02.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.362+0000 7f6d76754640 10 monclient: monmap: 2024-09-18T14:10:02.363 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:02.363 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.363 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:02.363686+0000 2024-09-18T14:10:02.363 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:02.363686+0000 2024-09-18T14:10:02.363 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:02.363 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.363 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:02.364 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:02.364 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:02.364 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:02.364 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:02.364 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:02.364 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:02.364 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:02.364 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:02.364 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.363+0000 7f6d76754640 1 Processor -- start 2024-09-18T14:10:02.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.363+0000 7f6d76754640 1 -- start start 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.363+0000 7f6d76754640 10 monclient: init 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.363+0000 7f6d76754640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.363+0000 7f6d76754640 10 monclient: _add_conns ranks=[2,1,5,4,0,7,8,3,6] 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.363+0000 7f6d76754640 10 monclient(hunting): picked mon.noname-b con 0x7f6d7009f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d76754640 10 monclient(hunting): picked mon.noname-f con 0x7f6d7009fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d76754640 10 monclient(hunting): picked mon.noname-h con 0x7f6d7009ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d76754640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7005ed70 con 0x7f6d7009fba0 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d76754640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7009dfc0 con 0x7f6d7009f7d0 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d76754640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7009dce0 con 0x7f6d7009ff70 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d76754640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d76754640 10 monclient(hunting): authenticate will time out at 1417.702057s 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6ffff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f6d7009f7d0 0x7f6d70132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33852/0 (socket says 172.21.15.45:33852) 2024-09-18T14:10:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6ffff640 1 -- 172.21.15.45:0/1325593801 learned_addr learned my addr 172.21.15.45:0/1325593801 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:02.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 1 -- 172.21.15.45:0/1325593801 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419329314 0 0) 0x7f6d7009dfc0 con 0x7f6d7009f7d0 2024-09-18T14:10:02.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient(hunting): my global_id is 144803 2024-09-18T14:10:02.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 1 -- 172.21.15.45:0/1325593801 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d580034e0 con 0x7f6d7009f7d0 2024-09-18T14:10:02.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 1 -- 172.21.15.45:0/1325593801 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1607908483 0 0) 0x7f6d580034e0 con 0x7f6d7009f7d0 2024-09-18T14:10:02.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 1 -- 172.21.15.45:0/1325593801 >> v1:172.21.15.84:6791/0 conn(0x7f6d7009ff70 legacy=0x7f6d7013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 1 -- 172.21.15.45:0/1325593801 >> v1:172.21.15.84:6789/0 conn(0x7f6d7009fba0 legacy=0x7f6d70137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient: _finish_hunting 0 2024-09-18T14:10:02.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 1 monclient: found mon.noname-b 2024-09-18T14:10:02.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:02.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 1 -- 172.21.15.45:0/1325593801 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d700a1c00 con 0x7f6d7009f7d0 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient: _finish_auth 0 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 1 -- 172.21.15.45:0/1325593801 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d60001830 con 0x7f6d7009f7d0 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d76754640 5 monclient: authenticate success, global_id 144803 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d76754640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient: dump: 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.376 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d76754640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 1 -- 172.21.15.45:0/1325593801 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6d60001a10 con 0x7f6d7009f7d0 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d76754640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 1 -- 172.21.15.45:0/1325593801 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d60004ba0 con 0x7f6d7009f7d0 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d76754640 4 set_mon_vals no callback set 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.364+0000 7f6d6effd640 10 monclient: dump: 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.365+0000 7f6d76754640 10 monclient: shutdown 2024-09-18T14:10:02.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.365+0000 7f6d76754640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.365+0000 7f6d76754640 1 -- 172.21.15.45:0/1325593801 >> v1:172.21.15.45:6790/0 conn(0x7f6d7009f7d0 legacy=0x7f6d70132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.365+0000 7f6d76754640 1 -- 172.21.15.45:0/1325593801 shutdown_connections 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.365+0000 7f6d76754640 1 -- 172.21.15.45:0/1325593801 >> 172.21.15.45:0/1325593801 conn(0x7f6d700630b0 msgr2=0x7f6d700634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.365+0000 7f6d76754640 1 -- 172.21.15.45:0/1325593801 shutdown_connections 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.365+0000 7f6d76754640 1 -- 172.21.15.45:0/1325593801 wait complete. 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.365+0000 7f6d76754640 10 monclient: build_initial_monmap 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.365+0000 7f6d76754640 10 monclient: monmap: 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:02.380 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 1 Processor -- start 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 1 -- start start 2024-09-18T14:10:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 10 monclient: init 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 10 monclient: _add_conns ranks=[8,4,6,0,5,1,2,7,3] 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 10 monclient(hunting): picked mon.noname-e con 0x7f6d7009f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 10 monclient(hunting): picked mon.noname-c con 0x7f6d7009fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 10 monclient(hunting): picked mon.noname-d con 0x7f6d7009ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d701207f0 con 0x7f6d7009fba0 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d700a2560 con 0x7f6d7009ff70 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d70121eb0 con 0x7f6d7009f7d0 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d76754640 10 monclient(hunting): authenticate will time out at 1417.704814s 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d6ffff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f6d7009f7d0 0x7f6d701417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:44810/0 (socket says 172.21.15.45:44810) 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d74cf1640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f6d7009ff70 0x7f6d70149b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34734/0 (socket says 172.21.15.45:34734) 2024-09-18T14:10:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.366+0000 7f6d6ffff640 1 -- 172.21.15.45:0/480891010 learned_addr learned my addr 172.21.15.45:0/480891010 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891195344 0 0) 0x7f6d701207f0 con 0x7f6d7009fba0 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): my global_id is 164353 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d5c0034e0 con 0x7f6d7009fba0 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528210855 0 0) 0x7f6d700a2560 con 0x7f6d7009ff70 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): my global_id is 184119 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d701207f0 con 0x7f6d7009ff70 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999462190 0 0) 0x7f6d70121eb0 con 0x7f6d7009f7d0 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): my global_id is 5444 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d700a2560 con 0x7f6d7009f7d0 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2803907233 0 0) 0x7f6d5c0034e0 con 0x7f6d7009fba0 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6d70121eb0 con 0x7f6d7009fba0 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d64001a60 con 0x7f6d7009fba0 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): dump: 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2557243810 0 0) 0x7f6d701207f0 con 0x7f6d7009ff70 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6d5c0034e0 con 0x7f6d7009ff70 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d68001d40 con 0x7f6d7009ff70 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): dump: 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3514097574 0 0) 0x7f6d700a2560 con 0x7f6d7009f7d0 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6d701207f0 con 0x7f6d7009f7d0 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d60001770 con 0x7f6d7009f7d0 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:02.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient(hunting): dump: 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.388 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.389 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2802181324 0 0) 0x7f6d70121eb0 con 0x7f6d7009fba0 2024-09-18T14:10:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 >> v1:172.21.15.45:6793/0 conn(0x7f6d7009f7d0 legacy=0x7f6d701417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 >> v1:172.21.15.45:6792/0 conn(0x7f6d7009ff70 legacy=0x7f6d70149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient: _finish_hunting 0 2024-09-18T14:10:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 monclient: found mon.c 2024-09-18T14:10:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d70120010 con 0x7f6d7009fba0 2024-09-18T14:10:02.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 10 monclient: _finish_auth 0 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d76754640 5 monclient: authenticate success, global_id 164353 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d76754640 10 monclient: _renew_subs 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d76754640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d76754640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d70121b40 con 0x7f6d7009fba0 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d76754640 10 monclient: _renew_subs 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d76754640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d76754640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d70141eb0 con 0x7f6d7009fba0 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.367+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6d64002d30 con 0x7f6d7009fba0 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.368+0000 7f6d57fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.368+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d640022f0 con 0x7f6d7009fba0 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.368+0000 7f6d57fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.368+0000 7f6d6e7fc640 4 set_mon_vals no callback set 2024-09-18T14:10:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.368+0000 7f6d57fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.368+0000 7f6d57fff640 10 monclient: dump: 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.368+0000 7f6d76754640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.368+0000 7f6d76754640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6d50006580 con 0x7f6d50002830 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.368+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f6d64002450 con 0x7f6d7009fba0 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(527..527 src has 344..527) v4 ==== 4047+0+0 (unknown 3136580331 0 0) 0x7f6d64044270 con 0x7f6d7009fba0 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318291734 0 0) 0x7f6d50006580 con 0x7f6d50002830 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 10 monclient(hunting): my global_id is 5571 2024-09-18T14:10:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d70121eb0 con 0x7f6d50002830 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d74cf1640 10 monclient: get_auth_request con 0x7f6d5c03a9d0 auth_method 0 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4017281872 0 0) 0x7f6d70121eb0 con 0x7f6d50002830 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6d50006580 con 0x7f6d50002830 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d60004ae0 con 0x7f6d50002830 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 10 monclient: dump: 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3989555840 0 0) 0x7f6d50006580 con 0x7f6d50002830 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d5000a0a0 con 0x7f6d50002830 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6d60006fc0 con 0x7f6d50002830 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 >> v1:172.21.15.45:6789/0 conn(0x7f6d50002830 legacy=0x7f6d50004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.369+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 >> v1:172.21.15.45:6789/0 conn(0x7f6d50002830 legacy=0x7f6d50004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:02.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.375+0000 7f6d76754640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:02.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.375+0000 7f6d76754640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6d50009660 con 0x7f6d50009040 2024-09-18T14:10:02.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.375+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904021176 0 0) 0x7f6d50009660 con 0x7f6d50009040 2024-09-18T14:10:02.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.375+0000 7f6d57fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.375+0000 7f6d57fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.375+0000 7f6d57fff640 10 monclient(hunting): my global_id is 5580 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.375+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d50006580 con 0x7f6d50009040 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3731461335 0 0) 0x7f6d50006580 con 0x7f6d50009040 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6d50009660 con 0x7f6d50009040 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6d60005630 con 0x7f6d50009040 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 10 monclient: dump: 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.397 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2229739111 0 0) 0x7f6d50009660 con 0x7f6d50009040 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6d50009920 con 0x7f6d50009040 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 1842961442) 0x7f6d60006c00 con 0x7f6d50009040 2024-09-18T14:10:02.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:02.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:02.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 >> v1:172.21.15.45:6789/0 conn(0x7f6d50009040 legacy=0x7f6d50008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d57fff640 1 -- 172.21.15.45:0/480891010 >> v1:172.21.15.45:6789/0 conn(0x7f6d50009040 legacy=0x7f6d50008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:02.399 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:02.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:02.399 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:02.399 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:02.399 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:02.399 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:02.400 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:02.400 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:02.400 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:02.400 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:02.400 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:02.400 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:02.400 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:02.400 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:02.400 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:02.400 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.400 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:10:02.400 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.401 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.402 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:02.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:02.404 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.405 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.408 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:02.409 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:02.410 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.412 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:02.413 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:02.414 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.416 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:02.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:02.418 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:10:02.419 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.419 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.419 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:02.419 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d76754640 1 -- 172.21.15.45:0/480891010 >> v1:172.21.15.84:6812/31478 conn(0x7f6d5c03a9d0 legacy=0x7f6d5c03ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d76754640 10 monclient: shutdown 2024-09-18T14:10:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d76754640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d76754640 1 -- 172.21.15.45:0/480891010 >> v1:172.21.15.45:6791/0 conn(0x7f6d7009fba0 legacy=0x7f6d701319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d74cf1640 1 -- 172.21.15.45:0/480891010 reap_dead start 2024-09-18T14:10:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d74cf1640 1 -- 172.21.15.45:0/480891010 reap_dead start 2024-09-18T14:10:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d76754640 1 -- 172.21.15.45:0/480891010 shutdown_connections 2024-09-18T14:10:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.376+0000 7f6d76754640 1 -- 172.21.15.45:0/480891010 >> 172.21.15.45:0/480891010 conn(0x7f6d700630b0 msgr2=0x7f6d700a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.377+0000 7f6d76754640 1 -- 172.21.15.45:0/480891010 shutdown_connections 2024-09-18T14:10:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.377+0000 7f6d76754640 1 -- 172.21.15.45:0/480891010 wait complete. 2024-09-18T14:10:02.421 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:02.421 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:10:02.421 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:10:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.518+0000 7f752f68b640 10 monclient: get_monmap_and_config 2024-09-18T14:10:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.518+0000 7f752f68b640 10 monclient: build_initial_monmap 2024-09-18T14:10:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.518+0000 7f752f68b640 10 monclient: monmap: 2024-09-18T14:10:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:02.519138+0000 2024-09-18T14:10:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:02.519138+0000 2024-09-18T14:10:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:02.520 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.518+0000 7f752f68b640 1 Processor -- start 2024-09-18T14:10:02.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752f68b640 1 -- start start 2024-09-18T14:10:02.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752f68b640 10 monclient: init 2024-09-18T14:10:02.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752f68b640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:02.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752f68b640 10 monclient: _add_conns ranks=[2,3,8,0,6,1,7,5,4] 2024-09-18T14:10:02.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752f68b640 10 monclient(hunting): picked mon.noname-b con 0x7f752806e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:02.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752f68b640 10 monclient(hunting): picked mon.noname-g con 0x7f75280a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:02.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752f68b640 10 monclient(hunting): picked mon.noname-e con 0x7f75280a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:02.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752f68b640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752805ed70 con 0x7f752806e810 2024-09-18T14:10:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752f68b640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7528097620 con 0x7f75280a0cb0 2024-09-18T14:10:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752f68b640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7528097340 con 0x7f75280a1080 2024-09-18T14:10:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752f68b640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752f68b640 10 monclient(hunting): authenticate will time out at 1417.857633s 2024-09-18T14:10:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752d427640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f752806e810 0x7f75280a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33858/0 (socket says 172.21.15.45:33858) 2024-09-18T14:10:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f752d427640 1 -- 172.21.15.45:0/3367857893 learned_addr learned my addr 172.21.15.45:0/3367857893 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932740911 0 0) 0x7f752805ed70 con 0x7f752806e810 2024-09-18T14:10:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f751ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f751ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.519+0000 7f751ffff640 10 monclient(hunting): my global_id is 144812 2024-09-18T14:10:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75140034e0 con 0x7f752806e810 2024-09-18T14:10:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802575910 0 0) 0x7f7528097340 con 0x7f75280a1080 2024-09-18T14:10:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient(hunting): my global_id is 5453 2024-09-18T14:10:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f752805ed70 con 0x7f75280a1080 2024-09-18T14:10:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677373975 0 0) 0x7f7528097620 con 0x7f75280a0cb0 2024-09-18T14:10:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient(hunting): my global_id is 184107 2024-09-18T14:10:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7528097340 con 0x7f75280a0cb0 2024-09-18T14:10:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2869518079 0 0) 0x7f75140034e0 con 0x7f752806e810 2024-09-18T14:10:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 >> v1:172.21.15.45:6793/0 conn(0x7f75280a1080 legacy=0x7f752813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 >> v1:172.21.15.84:6790/0 conn(0x7f75280a0cb0 legacy=0x7f7528137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient: _finish_hunting 0 2024-09-18T14:10:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 monclient: found mon.noname-b 2024-09-18T14:10:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:10:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75280a1660 con 0x7f752806e810 2024-09-18T14:10:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient: _finish_auth 0 2024-09-18T14:10:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7524001830 con 0x7f752806e810 2024-09-18T14:10:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f752f68b640 5 monclient: authenticate success, global_id 144812 2024-09-18T14:10:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:10:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient: dump: 2024-09-18T14:10:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.535 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f752f68b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7524001a10 con 0x7f752806e810 2024-09-18T14:10:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f752f68b640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 1 -- 172.21.15.45:0/3367857893 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7524004ba0 con 0x7f752806e810 2024-09-18T14:10:02.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f752f68b640 4 set_mon_vals no callback set 2024-09-18T14:10:02.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:02.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f751ffff640 10 monclient: dump: 2024-09-18T14:10:02.536 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.536 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.536 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.536 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.537 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.538 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f752f68b640 10 monclient: shutdown 2024-09-18T14:10:02.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f752f68b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:02.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f752f68b640 1 -- 172.21.15.45:0/3367857893 >> v1:172.21.15.45:6790/0 conn(0x7f752806e810 legacy=0x7f75280a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f752f68b640 1 -- 172.21.15.45:0/3367857893 shutdown_connections 2024-09-18T14:10:02.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.520+0000 7f752f68b640 1 -- 172.21.15.45:0/3367857893 >> 172.21.15.45:0/3367857893 conn(0x7f75280a5bb0 msgr2=0x7f75280a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:02.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.521+0000 7f752f68b640 1 -- 172.21.15.45:0/3367857893 shutdown_connections 2024-09-18T14:10:02.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.521+0000 7f752f68b640 1 -- 172.21.15.45:0/3367857893 wait complete. 2024-09-18T14:10:02.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.521+0000 7f752f68b640 10 monclient: build_initial_monmap 2024-09-18T14:10:02.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.521+0000 7f752f68b640 10 monclient: monmap: 2024-09-18T14:10:02.538 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:02.540 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.521+0000 7f752f68b640 1 Processor -- start 2024-09-18T14:10:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.521+0000 7f752f68b640 1 -- start start 2024-09-18T14:10:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.521+0000 7f752f68b640 10 monclient: init 2024-09-18T14:10:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f752f68b640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f752f68b640 10 monclient: _add_conns ranks=[6,2,5,8,4,7,3,1,0] 2024-09-18T14:10:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f752f68b640 10 monclient(hunting): picked mon.noname-d con 0x7f752806e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f752f68b640 10 monclient(hunting): picked mon.noname-b con 0x7f75280a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f752f68b640 10 monclient(hunting): picked mon.noname-h con 0x7f75280a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f752f68b640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7528122fe0 con 0x7f75280a0cb0 2024-09-18T14:10:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f752f68b640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752813ddd0 con 0x7f75280a1080 2024-09-18T14:10:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f752f68b640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7528122a20 con 0x7f752806e810 2024-09-18T14:10:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f752f68b640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f752f68b640 10 monclient(hunting): authenticate will time out at 1417.860621s 2024-09-18T14:10:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f752d427640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f752806e810 0x7f752813d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34740/0 (socket says 172.21.15.45:34740) 2024-09-18T14:10:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f752d427640 1 -- 172.21.15.45:0/111835146 learned_addr learned my addr 172.21.15.45:0/111835146 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311695623 0 0) 0x7f7528122fe0 con 0x7f75280a0cb0 2024-09-18T14:10:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f751d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f751d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f751d7fa640 10 monclient(hunting): my global_id is 144821 2024-09-18T14:10:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f750c0034e0 con 0x7f75280a0cb0 2024-09-18T14:10:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3670538070 0 0) 0x7f7528122a20 con 0x7f752806e810 2024-09-18T14:10:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f751d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f751d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f751d7fa640 10 monclient(hunting): my global_id is 184128 2024-09-18T14:10:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.522+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7528122fe0 con 0x7f752806e810 2024-09-18T14:10:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590194330 0 0) 0x7f752813ddd0 con 0x7f75280a1080 2024-09-18T14:10:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): my global_id is 184118 2024-09-18T14:10:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7528122a20 con 0x7f75280a1080 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1481470316 0 0) 0x7f750c0034e0 con 0x7f75280a0cb0 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f752813ddd0 con 0x7f75280a0cb0 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7518001ae0 con 0x7f75280a0cb0 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): dump: 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.545 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 574382752 0 0) 0x7f7528122fe0 con 0x7f752806e810 2024-09-18T14:10:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f750c0034e0 con 0x7f752806e810 2024-09-18T14:10:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f75240017f0 con 0x7f752806e810 2024-09-18T14:10:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): dump: 2024-09-18T14:10:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.547 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2318002744 0 0) 0x7f7528122a20 con 0x7f75280a1080 2024-09-18T14:10:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7528122fe0 con 0x7f75280a1080 2024-09-18T14:10:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7520001c80 con 0x7f75280a1080 2024-09-18T14:10:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient(hunting): dump: 2024-09-18T14:10:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.548 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.548 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.548 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.548 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.548 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.548 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.548 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.549 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3028151025 0 0) 0x7f752813ddd0 con 0x7f75280a0cb0 2024-09-18T14:10:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 >> v1:172.21.15.45:6792/0 conn(0x7f752806e810 legacy=0x7f752813d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 >> v1:172.21.15.84:6791/0 conn(0x7f75280a1080 legacy=0x7f752814a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient: _finish_hunting 0 2024-09-18T14:10:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 monclient: found mon.b 2024-09-18T14:10:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75281204e0 con 0x7f75280a0cb0 2024-09-18T14:10:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient: _finish_auth 0 2024-09-18T14:10:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f752f68b640 5 monclient: authenticate success, global_id 144821 2024-09-18T14:10:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f752f68b640 10 monclient: _renew_subs 2024-09-18T14:10:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f752f68b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f752f68b640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7528122740 con 0x7f75280a0cb0 2024-09-18T14:10:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f752f68b640 10 monclient: _renew_subs 2024-09-18T14:10:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f752f68b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f752f68b640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7528121870 con 0x7f75280a0cb0 2024-09-18T14:10:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7518002070 con 0x7f75280a0cb0 2024-09-18T14:10:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f75180025a0 con 0x7f75280a0cb0 2024-09-18T14:10:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751f7fe640 4 set_mon_vals no callback set 2024-09-18T14:10:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.523+0000 7f751d7fa640 10 monclient: dump: 2024-09-18T14:10:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.553 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f752f68b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7518002780 con 0x7f75280a0cb0 2024-09-18T14:10:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f752f68b640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7508006580 con 0x7f7508002830 2024-09-18T14:10:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(527..527 src has 344..527) v4 ==== 4047+0+0 (unknown 3136580331 0 0) 0x7f7518044380 con 0x7f75280a0cb0 2024-09-18T14:10:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934728355 0 0) 0x7f7508006580 con 0x7f7508002830 2024-09-18T14:10:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 10 monclient(hunting): my global_id is 144830 2024-09-18T14:10:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f752813ddd0 con 0x7f7508002830 2024-09-18T14:10:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f752dc28640 10 monclient: get_auth_request con 0x7f750c03a8d0 auth_method 0 2024-09-18T14:10:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 215128222 0 0) 0x7f752813ddd0 con 0x7f7508002830 2024-09-18T14:10:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7508006580 con 0x7f7508002830 2024-09-18T14:10:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f75240011f0 con 0x7f7508002830 2024-09-18T14:10:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.524+0000 7f751d7fa640 10 monclient: dump: 2024-09-18T14:10:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.555 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.525+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 974915696 0 0) 0x7f7508006580 con 0x7f7508002830 2024-09-18T14:10:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.525+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f750800a0a0 con 0x7f7508002830 2024-09-18T14:10:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.525+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7524005160 con 0x7f7508002830 2024-09-18T14:10:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.525+0000 7f751d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.525+0000 7f751d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.525+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 >> v1:172.21.15.45:6790/0 conn(0x7f7508002830 legacy=0x7f7508004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.525+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 >> v1:172.21.15.45:6790/0 conn(0x7f7508002830 legacy=0x7f7508004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:02.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f752f68b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:02.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f752f68b640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7508007540 con 0x7f75080068f0 2024-09-18T14:10:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569762972 0 0) 0x7f7508007540 con 0x7f75080068f0 2024-09-18T14:10:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 10 monclient(hunting): my global_id is 144839 2024-09-18T14:10:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7508006580 con 0x7f75080068f0 2024-09-18T14:10:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2103204622 0 0) 0x7f7508006580 con 0x7f75080068f0 2024-09-18T14:10:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7508007540 con 0x7f75080068f0 2024-09-18T14:10:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7524005940 con 0x7f75080068f0 2024-09-18T14:10:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 10 monclient: dump: 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.560 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 825314127 0 0) 0x7f7508007540 con 0x7f75080068f0 2024-09-18T14:10:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.531+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f75080087a0 con 0x7f75080068f0 2024-09-18T14:10:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 512746919) 0x7f7524008150 con 0x7f75080068f0 2024-09-18T14:10:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f751d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f751d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 >> v1:172.21.15.45:6790/0 conn(0x7f75080068f0 legacy=0x7f7508006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f751d7fa640 1 -- 172.21.15.45:0/111835146 >> v1:172.21.15.45:6790/0 conn(0x7f75080068f0 legacy=0x7f7508006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:02.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:02.561 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:02.561 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:02.561 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:02.561 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:02.561 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:02.561 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:02.561 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:02.562 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:02.562 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:02.562 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:02.562 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:02.562 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:02.562 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:02.562 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.562 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:10:02.562 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:02.562 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:02.562 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:02.563 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:02.563 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:02.563 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:02.563 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:02.563 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:02.563 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:02.563 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:02.563 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:02.563 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:02.563 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.563 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:02.564 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:02.564 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:02.564 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:02.564 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:02.564 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:02.564 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:02.564 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:02.564 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:02.564 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:02.564 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:02.565 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:02.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:02.566 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:02.566 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:02.566 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:02.566 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:02.566 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:02.566 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:02.566 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:02.566 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:02.566 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:02.566 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:02.567 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:02.567 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:02.567 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:02.567 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:02.567 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:02.567 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:02.567 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.567 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:02.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.567 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:02.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.568 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:02.568 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:02.568 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.568 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.568 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:02.568 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.568 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.568 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.569 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:02.569 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:02.569 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.569 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.569 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.569 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:02.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:02.569 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.570 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.570 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:02.570 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.570 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.570 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.570 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:02.570 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:02.570 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.570 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.571 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.571 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:02.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:02.571 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.571 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.571 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:02.571 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:02.572 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.573 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.574 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.575 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.576 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.577 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:02.578 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:02.580 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.582 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:02.582 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:02.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.582 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:02.582 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:02.582 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:02.582 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.582 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.582 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:10:02.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.582 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:02.582 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:02.583 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:02.583 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.583 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.583 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:02.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.583 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:02.583 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:02.583 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:10:02.583 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.583 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.584 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:02.584 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f752f68b640 1 -- 172.21.15.45:0/111835146 >> v1:172.21.15.84:6812/31478 conn(0x7f750c03a8d0 legacy=0x7f750c03cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f752f68b640 10 monclient: shutdown 2024-09-18T14:10:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f752f68b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f752f68b640 1 -- 172.21.15.45:0/111835146 >> v1:172.21.15.45:6790/0 conn(0x7f75280a0cb0 legacy=0x7f75281468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f752dc28640 1 -- 172.21.15.45:0/111835146 reap_dead start 2024-09-18T14:10:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f752dc28640 1 -- 172.21.15.45:0/111835146 reap_dead start 2024-09-18T14:10:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f752f68b640 1 -- 172.21.15.45:0/111835146 shutdown_connections 2024-09-18T14:10:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f752f68b640 1 -- 172.21.15.45:0/111835146 >> 172.21.15.45:0/111835146 conn(0x7f75280a5bb0 msgr2=0x7f75280a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f752f68b640 1 -- 172.21.15.45:0/111835146 shutdown_connections 2024-09-18T14:10:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.532+0000 7f752f68b640 1 -- 172.21.15.45:0/111835146 wait complete. 2024-09-18T14:10:02.586 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:02.586 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:10:02.587 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:10:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.683+0000 7fe130d9d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.683+0000 7fe130d9d640 10 monclient: build_initial_monmap 2024-09-18T14:10:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.683+0000 7fe130d9d640 10 monclient: monmap: 2024-09-18T14:10:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:02.684385+0000 2024-09-18T14:10:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:02.684385+0000 2024-09-18T14:10:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:02.685 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 1 Processor -- start 2024-09-18T14:10:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 1 -- start start 2024-09-18T14:10:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 10 monclient: init 2024-09-18T14:10:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 10 monclient: _add_conns ranks=[7,2,1,4,8,0,6,3,5] 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 10 monclient(hunting): picked mon.noname-i con 0x7fe12c09f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 10 monclient(hunting): picked mon.noname-b con 0x7fe12c09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 10 monclient(hunting): picked mon.noname-f con 0x7fe12c09ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c05ed70 con 0x7fe12c09ff70 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c09dfc0 con 0x7fe12c09fba0 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c09dce0 con 0x7fe12c09f7d0 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe130d9d640 10 monclient(hunting): authenticate will time out at 1418.022730s 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe129d9b640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fe12c09fba0 0x7fe12c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33886/0 (socket says 172.21.15.45:33886) 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.684+0000 7fe129d9b640 1 -- 172.21.15.45:0/751383484 learned_addr learned my addr 172.21.15.45:0/751383484 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 1 -- 172.21.15.45:0/751383484 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585061045 0 0) 0x7fe12c09dfc0 con 0x7fe12c09fba0 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient(hunting): my global_id is 144848 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 1 -- 172.21.15.45:0/751383484 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1140034e0 con 0x7fe12c09fba0 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 1 -- 172.21.15.45:0/751383484 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1345663220 0 0) 0x7fe1140034e0 con 0x7fe12c09fba0 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 1 -- 172.21.15.45:0/751383484 >> v1:172.21.15.84:6792/0 conn(0x7fe12c09f7d0 legacy=0x7fe12c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 1 -- 172.21.15.45:0/751383484 >> v1:172.21.15.84:6789/0 conn(0x7fe12c09ff70 legacy=0x7fe12c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient: _finish_hunting 0 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 1 monclient: found mon.noname-b 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 1 -- 172.21.15.45:0/751383484 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c0a1c00 con 0x7fe12c09fba0 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient: _finish_auth 0 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 1 -- 172.21.15.45:0/751383484 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe120001830 con 0x7fe12c09fba0 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient: dump: 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe130d9d640 5 monclient: authenticate success, global_id 144848 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe130d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 1 -- 172.21.15.45:0/751383484 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe120001a10 con 0x7fe12c09fba0 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 1 -- 172.21.15.45:0/751383484 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe120004ba0 con 0x7fe12c09fba0 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe130d9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe130d9d640 4 set_mon_vals no callback set 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe12959a640 10 monclient: dump: 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe130d9d640 10 monclient: shutdown 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe130d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.685+0000 7fe130d9d640 1 -- 172.21.15.45:0/751383484 >> v1:172.21.15.45:6790/0 conn(0x7fe12c09fba0 legacy=0x7fe12c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.686+0000 7fe130d9d640 1 -- 172.21.15.45:0/751383484 shutdown_connections 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.686+0000 7fe130d9d640 1 -- 172.21.15.45:0/751383484 >> 172.21.15.45:0/751383484 conn(0x7fe12c0630b0 msgr2=0x7fe12c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.686+0000 7fe130d9d640 1 -- 172.21.15.45:0/751383484 shutdown_connections 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.686+0000 7fe130d9d640 1 -- 172.21.15.45:0/751383484 wait complete. 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.686+0000 7fe130d9d640 10 monclient: build_initial_monmap 2024-09-18T14:10:02.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.686+0000 7fe130d9d640 10 monclient: monmap: 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:02.701 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.686+0000 7fe130d9d640 1 Processor -- start 2024-09-18T14:10:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.686+0000 7fe130d9d640 1 -- start start 2024-09-18T14:10:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.686+0000 7fe130d9d640 10 monclient: init 2024-09-18T14:10:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe130d9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe130d9d640 10 monclient: _add_conns ranks=[6,8,2,4,3,1,7,0,5] 2024-09-18T14:10:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe130d9d640 10 monclient(hunting): picked mon.noname-d con 0x7fe12c09f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe130d9d640 10 monclient(hunting): picked mon.noname-e con 0x7fe12c09fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe130d9d640 10 monclient(hunting): picked mon.noname-b con 0x7fe12c09ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe130d9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c09d310 con 0x7fe12c09ff70 2024-09-18T14:10:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe130d9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c0a2560 con 0x7fe12c09f7d0 2024-09-18T14:10:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe130d9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c121ef0 con 0x7fe12c09fba0 2024-09-18T14:10:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe130d9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe130d9d640 10 monclient(hunting): authenticate will time out at 1418.025434s 2024-09-18T14:10:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe12ad9d640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fe12c09ff70 0x7fe12c149c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33892/0 (socket says 172.21.15.45:33892) 2024-09-18T14:10:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe12ad9d640 1 -- 172.21.15.45:0/1641195638 learned_addr learned my addr 172.21.15.45:0/1641195638 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe12a59c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fe12c09f7d0 0x7fe12c141920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34750/0 (socket says 172.21.15.45:34750) 2024-09-18T14:10:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091431797 0 0) 0x7fe12c09d310 con 0x7fe12c09ff70 2024-09-18T14:10:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 10 monclient(hunting): my global_id is 144857 2024-09-18T14:10:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10c0034e0 con 0x7fe12c09ff70 2024-09-18T14:10:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532700610 0 0) 0x7fe12c121ef0 con 0x7fe12c09fba0 2024-09-18T14:10:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 10 monclient(hunting): my global_id is 5462 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe12c09d310 con 0x7fe12c09fba0 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686730680 0 0) 0x7fe12c0a2560 con 0x7fe12c09f7d0 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 10 monclient(hunting): my global_id is 184137 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.687+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe12c121ef0 con 0x7fe12c09f7d0 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 857698478 0 0) 0x7fe10c0034e0 con 0x7fe12c09ff70 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe12c0a2560 con 0x7fe12c09ff70 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe124001e40 con 0x7fe12c09ff70 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient(hunting): dump: 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.704 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1516510545 0 0) 0x7fe12c121ef0 con 0x7fe12c09f7d0 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe10c0034e0 con 0x7fe12c09f7d0 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3388455893 0 0) 0x7fe12c09d310 con 0x7fe12c09fba0 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe12c121ef0 con 0x7fe12c09fba0 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe11c0019a0 con 0x7fe12c09f7d0 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient(hunting): dump: 2024-09-18T14:10:02.705 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe120004ae0 con 0x7fe12c09fba0 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient(hunting): dump: 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 752156645 0 0) 0x7fe12c0a2560 con 0x7fe12c09ff70 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 >> v1:172.21.15.45:6793/0 conn(0x7fe12c09fba0 legacy=0x7fe12c131a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 >> v1:172.21.15.45:6792/0 conn(0x7fe12c09f7d0 legacy=0x7fe12c141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 monclient: found mon.b 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c120050 con 0x7fe12c09ff70 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe130d9d640 5 monclient: authenticate success, global_id 144857 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe130d9d640 10 monclient: _renew_subs 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe130d9d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe130d9d640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12c121b80 con 0x7fe12c09ff70 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe130d9d640 10 monclient: _renew_subs 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe130d9d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe130d9d640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12c142030 con 0x7fe12c09ff70 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe124002080 con 0x7fe12c09ff70 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe124002970 con 0x7fe12c09ff70 2024-09-18T14:10:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe128d99640 4 set_mon_vals no callback set 2024-09-18T14:10:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.688+0000 7fe11a7fc640 10 monclient: dump: 2024-09-18T14:10:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe130d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe130d9d640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe110006580 con 0x7fe110002830 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fe124002ad0 con 0x7fe12c09ff70 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(527..527 src has 344..527) v4 ==== 4047+0+0 (unknown 3136580331 0 0) 0x7fe124046e40 con 0x7fe12c09ff70 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973900986 0 0) 0x7fe110006580 con 0x7fe110002830 2024-09-18T14:10:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 10 monclient(hunting): my global_id is 164362 2024-09-18T14:10:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe12c0a2560 con 0x7fe110002830 2024-09-18T14:10:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe129d9b640 10 monclient: get_auth_request con 0x7fe10c03aa30 auth_method 0 2024-09-18T14:10:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3921847663 0 0) 0x7fe12c0a2560 con 0x7fe110002830 2024-09-18T14:10:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe110006580 con 0x7fe110002830 2024-09-18T14:10:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe11c0024a0 con 0x7fe110002830 2024-09-18T14:10:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 10 monclient: dump: 2024-09-18T14:10:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.714 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2379781338 0 0) 0x7fe110006580 con 0x7fe110002830 2024-09-18T14:10:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.689+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe11000a0a0 con 0x7fe110002830 2024-09-18T14:10:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.690+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe11c002980 con 0x7fe110002830 2024-09-18T14:10:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.690+0000 7fe11a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.690+0000 7fe11a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.690+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 >> v1:172.21.15.45:6791/0 conn(0x7fe110002830 legacy=0x7fe110004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.690+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 >> v1:172.21.15.45:6791/0 conn(0x7fe110002830 legacy=0x7fe110004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe130d9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe130d9d640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe110007520 con 0x7fe1100068d0 2024-09-18T14:10:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482216396 0 0) 0x7fe110007520 con 0x7fe1100068d0 2024-09-18T14:10:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 10 monclient(hunting): my global_id is 164371 2024-09-18T14:10:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe110006580 con 0x7fe1100068d0 2024-09-18T14:10:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2581442468 0 0) 0x7fe110006580 con 0x7fe1100068d0 2024-09-18T14:10:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe110007520 con 0x7fe1100068d0 2024-09-18T14:10:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe11c005ac0 con 0x7fe1100068d0 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 10 monclient: dump: 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 737741616 0 0) 0x7fe110007520 con 0x7fe1100068d0 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe1100087a0 con 0x7fe1100068d0 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3137760572) 0x7fe11c007cf0 con 0x7fe1100068d0 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.696+0000 7fe11a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.697+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 >> v1:172.21.15.45:6791/0 conn(0x7fe1100068d0 legacy=0x7fe110006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.697+0000 7fe11a7fc640 1 -- 172.21.15.45:0/1641195638 >> v1:172.21.15.45:6791/0 conn(0x7fe1100068d0 legacy=0x7fe110006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:02.719 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:02.719 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:02.719 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:02.719 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:02.719 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:02.719 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:02.719 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:02.719 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:02.719 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:02.719 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:02.719 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:02.720 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:02.720 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:02.720 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:02.720 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:02.720 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.720 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:10:02.720 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:02.720 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:02.720 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:02.720 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:02.720 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:02.720 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:02.721 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:02.722 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:02.723 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:02.725 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.725 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.725 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.725 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:02.725 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.725 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.725 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.725 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:02.725 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:02.725 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.726 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.727 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.727 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:02.727 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:02.727 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.727 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.727 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.727 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:02.727 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:02.727 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.728 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.730 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:02.731 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.731 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.731 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.731 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:02.731 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:02.731 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.731 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.731 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.733 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:02.733 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:02.733 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.733 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.733 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.733 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:02.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:02.733 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.733 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.734 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.735 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:02.737 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:02.738 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.697+0000 7fe130d9d640 1 -- 172.21.15.45:0/1641195638 >> v1:172.21.15.84:6812/31478 conn(0x7fe10c03aa30 legacy=0x7fe10c03cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.697+0000 7fe130d9d640 10 monclient: shutdown 2024-09-18T14:10:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.697+0000 7fe130d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.697+0000 7fe130d9d640 1 -- 172.21.15.45:0/1641195638 >> v1:172.21.15.45:6790/0 conn(0x7fe12c09ff70 legacy=0x7fe12c149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.697+0000 7fe12ad9d640 1 -- 172.21.15.45:0/1641195638 reap_dead start 2024-09-18T14:10:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.697+0000 7fe12ad9d640 1 -- 172.21.15.45:0/1641195638 reap_dead start 2024-09-18T14:10:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.697+0000 7fe130d9d640 1 -- 172.21.15.45:0/1641195638 shutdown_connections 2024-09-18T14:10:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.697+0000 7fe130d9d640 1 -- 172.21.15.45:0/1641195638 >> 172.21.15.45:0/1641195638 conn(0x7fe12c0630b0 msgr2=0x7fe12c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.697+0000 7fe130d9d640 1 -- 172.21.15.45:0/1641195638 shutdown_connections 2024-09-18T14:10:02.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.697+0000 7fe130d9d640 1 -- 172.21.15.45:0/1641195638 wait complete. 2024-09-18T14:10:02.741 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:02.741 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:10:02.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:10:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.838+0000 7f1ee36c0640 10 monclient: get_monmap_and_config 2024-09-18T14:10:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.838+0000 7f1ee36c0640 10 monclient: build_initial_monmap 2024-09-18T14:10:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.838+0000 7f1ee36c0640 10 monclient: monmap: 2024-09-18T14:10:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:02.839080+0000 2024-09-18T14:10:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:02.839080+0000 2024-09-18T14:10:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:02.840 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.838+0000 7f1ee36c0640 1 Processor -- start 2024-09-18T14:10:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.838+0000 7f1ee36c0640 1 -- start start 2024-09-18T14:10:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.838+0000 7f1ee36c0640 10 monclient: init 2024-09-18T14:10:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.839+0000 7f1ee36c0640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.839+0000 7f1ee36c0640 10 monclient: _add_conns ranks=[5,1,7,0,3,6,2,8,4] 2024-09-18T14:10:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.839+0000 7f1ee36c0640 10 monclient(hunting): picked mon.noname-h con 0x7f1edc06e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.839+0000 7f1ee36c0640 10 monclient(hunting): picked mon.noname-f con 0x7f1edc0a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.839+0000 7f1ee36c0640 10 monclient(hunting): picked mon.noname-i con 0x7f1edc0a1080 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.839+0000 7f1ee36c0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc05ed70 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.839+0000 7f1ee36c0640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc097620 con 0x7f1edc06e810 2024-09-18T14:10:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.839+0000 7f1ee36c0640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc097340 con 0x7f1edc0a1080 2024-09-18T14:10:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.839+0000 7f1ee36c0640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:02.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.839+0000 7f1ee36c0640 10 monclient(hunting): authenticate will time out at 1418.177546s 2024-09-18T14:10:02.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.839+0000 7f1ee0c5b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1edc0a0cb0 0x7f1edc137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:46996/0 (socket says 172.21.15.45:46996) 2024-09-18T14:10:02.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.839+0000 7f1ee0c5b640 1 -- 172.21.15.45:0/499866774 learned_addr learned my addr 172.21.15.45:0/499866774 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:02.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243858962 0 0) 0x7f1edc05ed70 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient(hunting): my global_id is 164287 2024-09-18T14:10:02.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ec80034e0 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157927550 0 0) 0x7f1edc097620 con 0x7f1edc06e810 2024-09-18T14:10:02.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient(hunting): my global_id is 184127 2024-09-18T14:10:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1edc05ed70 con 0x7f1edc06e810 2024-09-18T14:10:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682219102 0 0) 0x7f1edc097340 con 0x7f1edc0a1080 2024-09-18T14:10:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient(hunting): my global_id is 184129 2024-09-18T14:10:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1edc097620 con 0x7f1edc0a1080 2024-09-18T14:10:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2979844240 0 0) 0x7f1ec80034e0 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 >> v1:172.21.15.84:6792/0 conn(0x7f1edc0a1080 legacy=0x7f1edc13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 >> v1:172.21.15.84:6791/0 conn(0x7f1edc06e810 legacy=0x7f1edc0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient: _finish_hunting 0 2024-09-18T14:10:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 monclient: found mon.noname-f 2024-09-18T14:10:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc0a1660 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient: _finish_auth 0 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ee36c0640 5 monclient: authenticate success, global_id 164287 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ecc001830 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ee36c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient: dump: 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.855 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ee36c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1ecc001a10 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ee36c0640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 1 -- 172.21.15.45:0/499866774 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ecc004ba0 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ee36c0640 4 set_mon_vals no callback set 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.840+0000 7f1ed3fff640 10 monclient: dump: 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.841+0000 7f1ee36c0640 10 monclient: shutdown 2024-09-18T14:10:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.841+0000 7f1ee36c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.841+0000 7f1ee36c0640 1 -- 172.21.15.45:0/499866774 >> v1:172.21.15.84:6789/0 conn(0x7f1edc0a0cb0 legacy=0x7f1edc137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.841+0000 7f1ee36c0640 1 -- 172.21.15.45:0/499866774 shutdown_connections 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.841+0000 7f1ee36c0640 1 -- 172.21.15.45:0/499866774 >> 172.21.15.45:0/499866774 conn(0x7f1edc0a5bb0 msgr2=0x7f1edc0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.841+0000 7f1ee36c0640 1 -- 172.21.15.45:0/499866774 shutdown_connections 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.841+0000 7f1ee36c0640 1 -- 172.21.15.45:0/499866774 wait complete. 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.841+0000 7f1ee36c0640 10 monclient: build_initial_monmap 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.841+0000 7f1ee36c0640 10 monclient: monmap: 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.841+0000 7f1ee36c0640 1 Processor -- start 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee36c0640 1 -- start start 2024-09-18T14:10:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee36c0640 10 monclient: init 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee36c0640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee36c0640 10 monclient: _add_conns ranks=[0,6,2,7,4,3,5,1,8] 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee36c0640 10 monclient(hunting): picked mon.noname-a con 0x7f1edc06e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee36c0640 10 monclient(hunting): picked mon.noname-d con 0x7f1edc0a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee36c0640 10 monclient(hunting): picked mon.noname-b con 0x7f1edc0a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee36c0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc09fa10 con 0x7f1edc06e810 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee36c0640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc13db60 con 0x7f1edc0a1080 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee36c0640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc0a2270 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee36c0640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee36c0640 10 monclient(hunting): authenticate will time out at 1418.180620s 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee145c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1edc06e810 0x7f1edc13d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51070/0 (socket says 172.21.15.45:51070) 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee0c5b640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f1edc0a0cb0 0x7f1edc146660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34762/0 (socket says 172.21.15.45:34762) 2024-09-18T14:10:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ee145c640 1 -- 172.21.15.45:0/3864121559 learned_addr learned my addr 172.21.15.45:0/3864121559 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.842+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406789076 0 0) 0x7f1edc09fa10 con 0x7f1edc06e810 2024-09-18T14:10:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): my global_id is 5589 2024-09-18T14:10:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ec00034e0 con 0x7f1edc06e810 2024-09-18T14:10:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876483598 0 0) 0x7f1edc0a2270 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): my global_id is 184146 2024-09-18T14:10:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1edc09fa10 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907440886 0 0) 0x7f1edc13db60 con 0x7f1edc0a1080 2024-09-18T14:10:02.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): my global_id is 144866 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1edc0a2270 con 0x7f1edc0a1080 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 398254116 0 0) 0x7f1edc09fa10 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1edc13db60 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3327171605 0 0) 0x7f1ec00034e0 con 0x7f1edc06e810 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1edc09fa10 con 0x7f1edc06e810 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ecc0017f0 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): dump: 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.862 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.863 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1658098967 0 0) 0x7f1edc0a2270 con 0x7f1edc0a1080 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ec00034e0 con 0x7f1edc0a1080 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ed8002110 con 0x7f1edc06e810 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): dump: 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ed4001980 con 0x7f1edc0a1080 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient(hunting): dump: 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4040589196 0 0) 0x7f1edc13db60 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 >> v1:172.21.15.45:6790/0 conn(0x7f1edc0a1080 legacy=0x7f1edc149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 >> v1:172.21.15.45:6789/0 conn(0x7f1edc06e810 legacy=0x7f1edc13d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient: _finish_hunting 0 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 monclient: found mon.d 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc13e970 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient: _finish_auth 0 2024-09-18T14:10:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ee36c0640 5 monclient: authenticate success, global_id 184146 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ee36c0640 10 monclient: _renew_subs 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ee36c0640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ee36c0640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1edc1224f0 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ee36c0640 10 monclient: _renew_subs 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ee36c0640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ee36c0640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1edc120190 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1ecc004a50 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ecc005160 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed37fe640 4 set_mon_vals no callback set 2024-09-18T14:10:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.843+0000 7f1ed17fa640 10 monclient: dump: 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.844+0000 7f1ee36c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.844+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1ecc001100 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.844+0000 7f1ee36c0640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1ebc006580 con 0x7f1ebc002830 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.844+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(527..527 src has 344..527) v4 ==== 4047+0+0 (unknown 3136580331 0 0) 0x7f1ecc046b20 con 0x7f1edc0a0cb0 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.844+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498721137 0 0) 0x7f1ebc006580 con 0x7f1ebc002830 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.844+0000 7f1ed17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.844+0000 7f1ed17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.844+0000 7f1ed17fa640 10 monclient(hunting): my global_id is 184155 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.844+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1edc13db60 con 0x7f1ebc002830 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.844+0000 7f1ee1c5d640 10 monclient: get_auth_request con 0x7f1ec003ab40 auth_method 0 2024-09-18T14:10:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4072660854 0 0) 0x7f1edc13db60 con 0x7f1ebc002830 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ebc006580 con 0x7f1ebc002830 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ed80025b0 con 0x7f1ebc002830 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 10 monclient: dump: 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1161219439 0 0) 0x7f1ebc006580 con 0x7f1ebc002830 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ebc00a0a0 con 0x7f1ebc002830 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1ed8002a40 con 0x7f1ebc002830 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 >> v1:172.21.15.45:6792/0 conn(0x7f1ebc002830 legacy=0x7f1ebc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.845+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 >> v1:172.21.15.45:6792/0 conn(0x7f1ebc002830 legacy=0x7f1ebc004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ee36c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ee36c0640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1ebc007520 con 0x7f1ebc0068d0 2024-09-18T14:10:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50738348 0 0) 0x7f1ebc007520 con 0x7f1ebc0068d0 2024-09-18T14:10:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 10 monclient(hunting): my global_id is 184164 2024-09-18T14:10:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ebc006580 con 0x7f1ebc0068d0 2024-09-18T14:10:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3547951711 0 0) 0x7f1ebc006580 con 0x7f1ebc0068d0 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ebc007520 con 0x7f1ebc0068d0 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ed8005b60 con 0x7f1ebc0068d0 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 10 monclient: dump: 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2148921996 0 0) 0x7f1ebc007520 con 0x7f1ebc0068d0 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.851+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1ebc008750 con 0x7f1ebc0068d0 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3623920007) 0x7f1ed8005f00 con 0x7f1ebc0068d0 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ed17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ed17fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 >> v1:172.21.15.45:6792/0 conn(0x7f1ebc0068d0 legacy=0x7f1ebc006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ed17fa640 1 -- 172.21.15.45:0/3864121559 >> v1:172.21.15.45:6792/0 conn(0x7f1ebc0068d0 legacy=0x7f1ebc006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:02.877 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:02.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:02.877 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:02.877 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:02.877 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:02.877 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:02.877 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:02.877 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:02.879 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:02.880 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:02.881 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:02.881 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:02.881 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:02.881 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:02.881 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:02.881 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:02.881 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:02.881 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:02.881 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:02.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:02.881 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:02.881 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.883 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.884 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:02.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.886 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.887 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:02.890 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:02.891 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:02.892 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.893 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:02.895 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:02.896 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:02.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ee36c0640 1 -- 172.21.15.45:0/3864121559 >> v1:172.21.15.84:6812/31478 conn(0x7f1ec003ab40 legacy=0x7f1ec003cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ee36c0640 10 monclient: shutdown 2024-09-18T14:10:02.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ee36c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:02.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ee36c0640 1 -- 172.21.15.45:0/3864121559 >> v1:172.21.15.45:6792/0 conn(0x7f1edc0a0cb0 legacy=0x7f1edc146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:02.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ee1c5d640 1 -- 172.21.15.45:0/3864121559 reap_dead start 2024-09-18T14:10:02.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ee1c5d640 1 -- 172.21.15.45:0/3864121559 reap_dead start 2024-09-18T14:10:02.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ee36c0640 1 -- 172.21.15.45:0/3864121559 shutdown_connections 2024-09-18T14:10:02.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ee36c0640 1 -- 172.21.15.45:0/3864121559 >> 172.21.15.45:0/3864121559 conn(0x7f1edc0a5bb0 msgr2=0x7f1edc0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:02.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ee36c0640 1 -- 172.21.15.45:0/3864121559 shutdown_connections 2024-09-18T14:10:02.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.852+0000 7f1ee36c0640 1 -- 172.21.15.45:0/3864121559 wait complete. 2024-09-18T14:10:02.899 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:02.899 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:10:02.899 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:10:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.997+0000 7f3318b9d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.997+0000 7f3318b9d640 10 monclient: build_initial_monmap 2024-09-18T14:10:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.997+0000 7f3318b9d640 10 monclient: monmap: 2024-09-18T14:10:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:02.998272+0000 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:02.998272+0000 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:02.999 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.997+0000 7f3318b9d640 1 Processor -- start 2024-09-18T14:10:03.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.997+0000 7f3318b9d640 1 -- start start 2024-09-18T14:10:03.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.997+0000 7f3318b9d640 10 monclient: init 2024-09-18T14:10:03.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f3318b9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:03.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f3318b9d640 10 monclient: _add_conns ranks=[5,7,3,4,6,0,2,1,8] 2024-09-18T14:10:03.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f3318b9d640 10 monclient(hunting): picked mon.noname-h con 0x7f331409f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:03.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f3318b9d640 10 monclient(hunting): picked mon.noname-i con 0x7f331409fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:03.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f3318b9d640 10 monclient(hunting): picked mon.noname-g con 0x7f331409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:03.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f3318b9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331405ed70 con 0x7f331409ff70 2024-09-18T14:10:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f3318b9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331409dfc0 con 0x7f331409f7d0 2024-09-18T14:10:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f3318b9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331409dce0 con 0x7f331409fba0 2024-09-18T14:10:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f3318b9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f3318b9d640 10 monclient(hunting): authenticate will time out at 1418.336716s 2024-09-18T14:10:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f331259c640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f331409f7d0 0x7f3314132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:48252/0 (socket says 172.21.15.45:48252) 2024-09-18T14:10:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f3312d9d640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7f331409ff70 0x7f331413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:51158/0 (socket says 172.21.15.45:51158) 2024-09-18T14:10:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.998+0000 7f331259c640 1 -- 172.21.15.45:0/3111012731 learned_addr learned my addr 172.21.15.45:0/3111012731 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144145040 0 0) 0x7f331405ed70 con 0x7f331409ff70 2024-09-18T14:10:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient(hunting): my global_id is 184116 2024-09-18T14:10:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32fc0034e0 con 0x7f331409ff70 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585477802 0 0) 0x7f331409dfc0 con 0x7f331409f7d0 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient(hunting): my global_id is 184136 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f331405ed70 con 0x7f331409f7d0 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869483879 0 0) 0x7f331409dce0 con 0x7f331409fba0 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient(hunting): my global_id is 184138 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f331409dfc0 con 0x7f331409fba0 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2830611621 0 0) 0x7f32fc0034e0 con 0x7f331409ff70 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 >> v1:172.21.15.84:6792/0 conn(0x7f331409fba0 legacy=0x7f3314137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 >> v1:172.21.15.84:6791/0 conn(0x7f331409f7d0 legacy=0x7f3314132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient: _finish_hunting 0 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 monclient: found mon.noname-g 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33140a1c00 con 0x7f331409ff70 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient: _finish_auth 0 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f3318b9d640 5 monclient: authenticate success, global_id 184116 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f330c001bb0 con 0x7f331409ff70 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f3318b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient: dump: 2024-09-18T14:10:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f3318b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f330c001d90 con 0x7f331409ff70 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f3318b9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f3318b9d640 4 set_mon_vals no callback set 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 1 -- 172.21.15.45:0/3111012731 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f330c004e60 con 0x7f331409ff70 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:02.999+0000 7f331159a640 10 monclient: dump: 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.009 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.000+0000 7f3318b9d640 10 monclient: shutdown 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.000+0000 7f3318b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.000+0000 7f3318b9d640 1 -- 172.21.15.45:0/3111012731 >> v1:172.21.15.84:6790/0 conn(0x7f331409ff70 legacy=0x7f331413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.000+0000 7f3318b9d640 1 -- 172.21.15.45:0/3111012731 shutdown_connections 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.000+0000 7f3318b9d640 1 -- 172.21.15.45:0/3111012731 >> 172.21.15.45:0/3111012731 conn(0x7f33140630b0 msgr2=0x7f33140634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.000+0000 7f3318b9d640 1 -- 172.21.15.45:0/3111012731 shutdown_connections 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.000+0000 7f3318b9d640 1 -- 172.21.15.45:0/3111012731 wait complete. 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.000+0000 7f3318b9d640 10 monclient: build_initial_monmap 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.000+0000 7f3318b9d640 10 monclient: monmap: 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 1 Processor -- start 2024-09-18T14:10:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 1 -- start start 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 10 monclient: init 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 10 monclient: _add_conns ranks=[3,5,1,6,4,7,2,0,8] 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 10 monclient(hunting): picked mon.noname-g con 0x7f331409f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 10 monclient(hunting): picked mon.noname-h con 0x7f331409fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 10 monclient(hunting): picked mon.noname-f con 0x7f331409ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331409d310 con 0x7f331409ff70 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33140a2560 con 0x7f331409f7d0 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3314142940 con 0x7f331409fba0 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3318b9d640 10 monclient(hunting): authenticate will time out at 1418.339840s 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3312d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f331409ff70 0x7f3314149e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:47008/0 (socket says 172.21.15.45:47008) 2024-09-18T14:10:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.001+0000 7f3312d9d640 1 -- 172.21.15.45:0/2967029390 learned_addr learned my addr 172.21.15.45:0/2967029390 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:03.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721931115 0 0) 0x7f33140a2560 con 0x7f331409f7d0 2024-09-18T14:10:03.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 10 monclient(hunting): my global_id is 184125 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f40034e0 con 0x7f331409f7d0 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257689480 0 0) 0x7f3314142940 con 0x7f331409fba0 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 10 monclient(hunting): my global_id is 184145 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33140a2560 con 0x7f331409fba0 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597556841 0 0) 0x7f331409d310 con 0x7f331409ff70 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 10 monclient(hunting): my global_id is 164296 2024-09-18T14:10:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.007+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3314142940 con 0x7f331409ff70 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4018069044 0 0) 0x7f3314142940 con 0x7f331409ff70 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f331409d310 con 0x7f331409ff70 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f330c001af0 con 0x7f331409ff70 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 10 monclient(hunting): dump: 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3243435122 0 0) 0x7f32f40034e0 con 0x7f331409f7d0 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3314142940 con 0x7f331409f7d0 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f33040019d0 con 0x7f331409f7d0 2024-09-18T14:10:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 10 monclient(hunting): dump: 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2777978201 0 0) 0x7f331409d310 con 0x7f331409ff70 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 >> v1:172.21.15.84:6791/0 conn(0x7f331409fba0 legacy=0x7f3314131aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 >> v1:172.21.15.84:6790/0 conn(0x7f331409f7d0 legacy=0x7f3314141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 10 monclient: _finish_hunting 0 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 1 monclient: found mon.f 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:10:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3314120880 con 0x7f331409ff70 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 10 monclient: _finish_auth 0 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f33027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f3318b9d640 5 monclient: authenticate success, global_id 164296 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f3318b9d640 10 monclient: _renew_subs 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f3318b9d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.011+0000 7f3318b9d640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3314120c90 con 0x7f331409ff70 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.012+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f330c004dd0 con 0x7f331409ff70 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.012+0000 7f33027fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.012+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f330c005280 con 0x7f331409ff70 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.012+0000 7f33027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.012+0000 7f33027fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.012+0000 7f33027fc640 10 monclient: dump: 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.023 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.012+0000 7f3318b9d640 10 monclient: _renew_subs 2024-09-18T14:10:03.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.012+0000 7f3318b9d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:10:03.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.012+0000 7f3318b9d640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f33141200c0 con 0x7f331409ff70 2024-09-18T14:10:03.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.012+0000 7f3303fff640 4 set_mon_vals no callback set 2024-09-18T14:10:03.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.014+0000 7f3318b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.014+0000 7f3318b9d640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f331404c4a0 con 0x7f3314099a40 2024-09-18T14:10:03.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.015+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f330c010250 con 0x7f331409ff70 2024-09-18T14:10:03.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.015+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(527..527 src has 344..527) v4 ==== 4047+0+0 (unknown 3136580331 0 0) 0x7f330c046db0 con 0x7f331409ff70 2024-09-18T14:10:03.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.016+0000 7f3311d9b640 10 monclient: get_auth_request con 0x7f32f403a630 auth_method 0 2024-09-18T14:10:03.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.018+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411385845 0 0) 0x7f331404c4a0 con 0x7f3314099a40 2024-09-18T14:10:03.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.018+0000 7f33027fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.018+0000 7f33027fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.018+0000 7f33027fc640 10 monclient(hunting): my global_id is 5471 2024-09-18T14:10:03.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.018+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f331409d310 con 0x7f3314099a40 2024-09-18T14:10:03.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.018+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 969419073 0 0) 0x7f331409d310 con 0x7f3314099a40 2024-09-18T14:10:03.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.019+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f331404c4a0 con 0x7f3314099a40 2024-09-18T14:10:03.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.019+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3304004ae0 con 0x7f3314099a40 2024-09-18T14:10:03.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.019+0000 7f33027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.019+0000 7f33027fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:03.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.019+0000 7f33027fc640 10 monclient: dump: 2024-09-18T14:10:03.026 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.026 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.027 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.019+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1247995852 0 0) 0x7f331404c4a0 con 0x7f3314099a40 2024-09-18T14:10:03.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.019+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3314142180 con 0x7f3314099a40 2024-09-18T14:10:03.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.019+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3304005890 con 0x7f3314099a40 2024-09-18T14:10:03.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.019+0000 7f33027fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.019+0000 7f33027fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:03.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.019+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 >> v1:172.21.15.45:6793/0 conn(0x7f3314099a40 legacy=0x7f3314099e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.019+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 >> v1:172.21.15.45:6793/0 conn(0x7f3314099a40 legacy=0x7f3314099e10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:03.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f3318b9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f3318b9d640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f331404b720 con 0x7f331409a520 2024-09-18T14:10:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686680040 0 0) 0x7f331404b720 con 0x7f331409a520 2024-09-18T14:10:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 10 monclient(hunting): my global_id is 5480 2024-09-18T14:10:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f331404c4a0 con 0x7f331409a520 2024-09-18T14:10:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1185748113 0 0) 0x7f331404c4a0 con 0x7f331409a520 2024-09-18T14:10:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f331404b720 con 0x7f331409a520 2024-09-18T14:10:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f33040054d0 con 0x7f331409a520 2024-09-18T14:10:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 10 monclient: dump: 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.032 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.032 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1423605154 0 0) 0x7f331404b720 con 0x7f331409a520 2024-09-18T14:10:03.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.026+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3314071550 con 0x7f331409a520 2024-09-18T14:10:03.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7728 (unknown 0 0 3445500025) 0x7f3304007e40 con 0x7f331409a520 2024-09-18T14:10:03.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f33027fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:03.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f33027fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:03.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 >> v1:172.21.15.45:6793/0 conn(0x7f331409a520 legacy=0x7f331414f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f33027fc640 1 -- 172.21.15.45:0/2967029390 >> v1:172.21.15.45:6793/0 conn(0x7f331409a520 legacy=0x7f331414f540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:03.032 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:03.033 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.034 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.035 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:03.036 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.037 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:03.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:03.040 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.040 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.040 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:03.040 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.040 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.040 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.040 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:03.040 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:03.040 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.041 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:03.044 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:03.045 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.045 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.045 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.045 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:03.045 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:03.045 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.045 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.048 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:03.048 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:03.049 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.050 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.052 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:03.052 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:03.052 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.052 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.052 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.052 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:03.052 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:03.052 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.052 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.054 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.055 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.056 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:03.056 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.056 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.056 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:03.056 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:03.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f3318b9d640 1 -- 172.21.15.45:0/2967029390 >> v1:172.21.15.84:6812/31478 conn(0x7f32f403a630 legacy=0x7f32f403cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f3318b9d640 10 monclient: shutdown 2024-09-18T14:10:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f3318b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f3318b9d640 1 -- 172.21.15.45:0/2967029390 >> v1:172.21.15.84:6789/0 conn(0x7f331409ff70 legacy=0x7f3314149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f3312d9d640 1 -- 172.21.15.45:0/2967029390 reap_dead start 2024-09-18T14:10:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f3312d9d640 1 -- 172.21.15.45:0/2967029390 reap_dead start 2024-09-18T14:10:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f3318b9d640 1 -- 172.21.15.45:0/2967029390 shutdown_connections 2024-09-18T14:10:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f3318b9d640 1 -- 172.21.15.45:0/2967029390 >> 172.21.15.45:0/2967029390 conn(0x7f33140630b0 msgr2=0x7f33140a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f3318b9d640 1 -- 172.21.15.45:0/2967029390 shutdown_connections 2024-09-18T14:10:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.027+0000 7f3318b9d640 1 -- 172.21.15.45:0/2967029390 wait complete. 2024-09-18T14:10:03.058 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:03.059 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:10:03.059 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:10:03.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.155+0000 7ff2e8c17640 10 monclient: get_monmap_and_config 2024-09-18T14:10:03.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.155+0000 7ff2e8c17640 10 monclient: build_initial_monmap 2024-09-18T14:10:03.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.155+0000 7ff2e8c17640 10 monclient: monmap: 2024-09-18T14:10:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:03.156258+0000 2024-09-18T14:10:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:03.156258+0000 2024-09-18T14:10:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:03.157 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:03.157 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:03.157 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:03.157 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:03.157 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.155+0000 7ff2e8c17640 1 Processor -- start 2024-09-18T14:10:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.155+0000 7ff2e8c17640 1 -- start start 2024-09-18T14:10:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.155+0000 7ff2e8c17640 10 monclient: init 2024-09-18T14:10:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e8c17640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e8c17640 10 monclient: _add_conns ranks=[7,0,8,3,4,6,2,5,1] 2024-09-18T14:10:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e8c17640 10 monclient(hunting): picked mon.noname-i con 0x7ff2e409f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e8c17640 10 monclient(hunting): picked mon.noname-a con 0x7ff2e409fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e8c17640 10 monclient(hunting): picked mon.noname-e con 0x7ff2e409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e8c17640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e405ed70 con 0x7ff2e409fba0 2024-09-18T14:10:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e8c17640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e409dfc0 con 0x7ff2e409f7d0 2024-09-18T14:10:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e8c17640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e409dce0 con 0x7ff2e409ff70 2024-09-18T14:10:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e8c17640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e8c17640 10 monclient(hunting): authenticate will time out at 1418.494571s 2024-09-18T14:10:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e2d9d640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7ff2e409ff70 0x7ff2e413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:44848/0 (socket says 172.21.15.45:44848) 2024-09-18T14:10:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e2d9d640 1 -- 172.21.15.45:0/2275605865 learned_addr learned my addr 172.21.15.45:0/2275605865 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e1d9b640 1 --1- 172.21.15.45:0/2275605865 >> v1:172.21.15.45:6789/0 conn(0x7ff2e409fba0 0x7ff2e4137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51080/0 (socket says 172.21.15.45:51080) 2024-09-18T14:10:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951190769 0 0) 0x7ff2e405ed70 con 0x7ff2e409fba0 2024-09-18T14:10:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.156+0000 7ff2e159a640 10 monclient(hunting): my global_id is 5598 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2cc0034e0 con 0x7ff2e409fba0 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2061343045 0 0) 0x7ff2e409dce0 con 0x7ff2e409ff70 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient(hunting): my global_id is 5489 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e405ed70 con 0x7ff2e409ff70 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1234607741 0 0) 0x7ff2e409dfc0 con 0x7ff2e409f7d0 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient(hunting): my global_id is 184147 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e409dce0 con 0x7ff2e409f7d0 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 950857803 0 0) 0x7ff2cc0034e0 con 0x7ff2e409fba0 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 >> v1:172.21.15.45:6793/0 conn(0x7ff2e409ff70 legacy=0x7ff2e413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 >> v1:172.21.15.84:6792/0 conn(0x7ff2e409f7d0 legacy=0x7ff2e4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient: _finish_hunting 0 2024-09-18T14:10:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 monclient: found mon.noname-a 2024-09-18T14:10:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:10:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e40a1c00 con 0x7ff2e409fba0 2024-09-18T14:10:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient: _finish_auth 0 2024-09-18T14:10:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff2d8001830 con 0x7ff2e409fba0 2024-09-18T14:10:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e8c17640 5 monclient: authenticate success, global_id 5598 2024-09-18T14:10:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e8c17640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient: dump: 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.169 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e8c17640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff2d8001a10 con 0x7ff2e409fba0 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e8c17640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 1 -- 172.21.15.45:0/2275605865 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff2d8004ae0 con 0x7ff2e409fba0 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e8c17640 4 set_mon_vals no callback set 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:03.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e159a640 10 monclient: dump: 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.171 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.172 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e8c17640 10 monclient: shutdown 2024-09-18T14:10:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e8c17640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.157+0000 7ff2e8c17640 1 -- 172.21.15.45:0/2275605865 >> v1:172.21.15.45:6789/0 conn(0x7ff2e409fba0 legacy=0x7ff2e4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.158+0000 7ff2e8c17640 1 -- 172.21.15.45:0/2275605865 shutdown_connections 2024-09-18T14:10:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.158+0000 7ff2e8c17640 1 -- 172.21.15.45:0/2275605865 >> 172.21.15.45:0/2275605865 conn(0x7ff2e40630b0 msgr2=0x7ff2e40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.158+0000 7ff2e8c17640 1 -- 172.21.15.45:0/2275605865 shutdown_connections 2024-09-18T14:10:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.158+0000 7ff2e8c17640 1 -- 172.21.15.45:0/2275605865 wait complete. 2024-09-18T14:10:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.158+0000 7ff2e8c17640 10 monclient: build_initial_monmap 2024-09-18T14:10:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.158+0000 7ff2e8c17640 10 monclient: monmap: 2024-09-18T14:10:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.158+0000 7ff2e8c17640 1 Processor -- start 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.158+0000 7ff2e8c17640 1 -- start start 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.158+0000 7ff2e8c17640 10 monclient: init 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2e8c17640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2e8c17640 10 monclient: _add_conns ranks=[5,0,4,8,2,7,3,1,6] 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2e8c17640 10 monclient(hunting): picked mon.noname-h con 0x7ff2e409f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2e8c17640 10 monclient(hunting): picked mon.noname-a con 0x7ff2e409fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2e8c17640 10 monclient(hunting): picked mon.noname-c con 0x7ff2e409ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2e8c17640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e409d310 con 0x7ff2e409fba0 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2e8c17640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e40a2560 con 0x7ff2e409ff70 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2e8c17640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e4121e90 con 0x7ff2e409f7d0 2024-09-18T14:10:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2e8c17640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2e8c17640 10 monclient(hunting): authenticate will time out at 1418.497328s 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2e1d9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff2e409fba0 0x7ff2e4131a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51084/0 (socket says 172.21.15.45:51084) 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2e1d9b640 1 -- 172.21.15.45:0/598289706 learned_addr learned my addr 172.21.15.45:0/598289706 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110890141 0 0) 0x7ff2e409d310 con 0x7ff2e409fba0 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): my global_id is 5607 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c40034e0 con 0x7ff2e409fba0 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629548017 0 0) 0x7ff2e40a2560 con 0x7ff2e409ff70 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): my global_id is 164380 2024-09-18T14:10:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e409d310 con 0x7ff2e409ff70 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286382247 0 0) 0x7ff2e4121e90 con 0x7ff2e409f7d0 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): my global_id is 184154 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e40a2560 con 0x7ff2e409f7d0 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 367463629 0 0) 0x7ff2c40034e0 con 0x7ff2e409fba0 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff2e4121e90 con 0x7ff2e409fba0 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff2d8001430 con 0x7ff2e409fba0 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): dump: 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3103021853 0 0) 0x7ff2e409d310 con 0x7ff2e409ff70 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff2c40034e0 con 0x7ff2e409ff70 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff2dc001d00 con 0x7ff2e409ff70 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.159+0000 7ff2d27fc640 10 monclient(hunting): dump: 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.179 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2422333833 0 0) 0x7ff2e40a2560 con 0x7ff2e409f7d0 2024-09-18T14:10:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff2e409d310 con 0x7ff2e409f7d0 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff2d4001aa0 con 0x7ff2e409f7d0 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 10 monclient(hunting): dump: 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 40157828 0 0) 0x7ff2e4121e90 con 0x7ff2e409fba0 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 >> v1:172.21.15.84:6791/0 conn(0x7ff2e409f7d0 legacy=0x7ff2e41417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 >> v1:172.21.15.45:6791/0 conn(0x7ff2e409ff70 legacy=0x7ff2e4149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 10 monclient: _finish_hunting 0 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 1 monclient: found mon.a 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e4142660 con 0x7ff2e409fba0 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 10 monclient: _finish_auth 0 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2e8c17640 5 monclient: authenticate success, global_id 5607 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2e8c17640 10 monclient: _renew_subs 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2e8c17640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2e8c17640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e4141eb0 con 0x7ff2e409fba0 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2e8c17640 10 monclient: _renew_subs 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2e8c17640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2e8c17640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e4121b90 con 0x7ff2e409fba0 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff2d80018a0 con 0x7ff2e409fba0 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff2d80050a0 con 0x7ff2e409fba0 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2e0d99640 4 set_mon_vals no callback set 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.160+0000 7ff2d27fc640 10 monclient: dump: 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2e8c17640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2e8c17640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff2c8006580 con 0x7ff2c8002830 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff2d8010070 con 0x7ff2e409fba0 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(527..527 src has 344..527) v4 ==== 4047+0+0 (unknown 3136580331 0 0) 0x7ff2d8046b90 con 0x7ff2e409fba0 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2e2d9d640 10 monclient: get_auth_request con 0x7ff2c403aab0 auth_method 0 2024-09-18T14:10:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708007802 0 0) 0x7ff2c8006580 con 0x7ff2c8002830 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 10 monclient(hunting): my global_id is 164305 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e4121e90 con 0x7ff2c8002830 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3041786673 0 0) 0x7ff2e4121e90 con 0x7ff2c8002830 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff2c8006580 con 0x7ff2c8002830 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff2d4002230 con 0x7ff2c8002830 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.161+0000 7ff2d27fc640 10 monclient: dump: 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.186 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.162+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1098985484 0 0) 0x7ff2c8006580 con 0x7ff2c8002830 2024-09-18T14:10:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.162+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2c800a0a0 con 0x7ff2c8002830 2024-09-18T14:10:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.162+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff2d4002700 con 0x7ff2c8002830 2024-09-18T14:10:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.162+0000 7ff2d27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.162+0000 7ff2d27fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.162+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 >> v1:172.21.15.84:6789/0 conn(0x7ff2c8002830 legacy=0x7ff2c8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.162+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 >> v1:172.21.15.84:6789/0 conn(0x7ff2c8002830 legacy=0x7ff2c8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2e8c17640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2e8c17640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff2c80074f0 con 0x7ff2c80068a0 2024-09-18T14:10:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397774040 0 0) 0x7ff2c80074f0 con 0x7ff2c80068a0 2024-09-18T14:10:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 10 monclient(hunting): my global_id is 164314 2024-09-18T14:10:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c8006580 con 0x7ff2c80068a0 2024-09-18T14:10:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2327273331 0 0) 0x7ff2c8006580 con 0x7ff2c80068a0 2024-09-18T14:10:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff2c80074f0 con 0x7ff2c80068a0 2024-09-18T14:10:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff2d4003000 con 0x7ff2c80068a0 2024-09-18T14:10:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:10:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 10 monclient: dump: 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.191 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 531539322 0 0) 0x7ff2c80074f0 con 0x7ff2c80068a0 2024-09-18T14:10:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.168+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff2c8008750 con 0x7ff2c80068a0 2024-09-18T14:10:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 1873141270) 0x7ff2d4003390 con 0x7ff2c80068a0 2024-09-18T14:10:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2d27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2d27fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 >> v1:172.21.15.84:6789/0 conn(0x7ff2c80068a0 legacy=0x7ff2c8006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2d27fc640 1 -- 172.21.15.45:0/598289706 >> v1:172.21.15.84:6789/0 conn(0x7ff2c80068a0 legacy=0x7ff2c8006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:03.192 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.209 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.210 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:03.211 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:03.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2e8c17640 1 -- 172.21.15.45:0/598289706 >> v1:172.21.15.84:6812/31478 conn(0x7ff2c403aab0 legacy=0x7ff2c403cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2e8c17640 10 monclient: shutdown 2024-09-18T14:10:03.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2e8c17640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:03.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2e8c17640 1 -- 172.21.15.45:0/598289706 >> v1:172.21.15.45:6789/0 conn(0x7ff2e409fba0 legacy=0x7ff2e4131a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2e2d9d640 1 -- 172.21.15.45:0/598289706 reap_dead start 2024-09-18T14:10:03.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2e2d9d640 1 -- 172.21.15.45:0/598289706 reap_dead start 2024-09-18T14:10:03.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2e8c17640 1 -- 172.21.15.45:0/598289706 shutdown_connections 2024-09-18T14:10:03.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2e8c17640 1 -- 172.21.15.45:0/598289706 >> 172.21.15.45:0/598289706 conn(0x7ff2e40630b0 msgr2=0x7ff2e40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:03.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2e8c17640 1 -- 172.21.15.45:0/598289706 shutdown_connections 2024-09-18T14:10:03.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.169+0000 7ff2e8c17640 1 -- 172.21.15.45:0/598289706 wait complete. 2024-09-18T14:10:03.214 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:03.214 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:10:03.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:10:03.279 INFO:tasks.rados.rados.0.smithi084.stdout:2419: done (5 left) 2024-09-18T14:10:03.279 INFO:tasks.rados.rados.0.smithi084.stdout:2420: write oid 17 current snap is 255 2024-09-18T14:10:03.279 INFO:tasks.rados.rados.0.smithi084.stdout:2420: seq_num 848 ranges {652336=726335,1906495=345841} 2024-09-18T14:10:03.287 INFO:tasks.rados.rados.0.smithi084.stdout:2420: writing smithi08434663-17 from 652336 to 1378671 tid 1 2024-09-18T14:10:03.292 INFO:tasks.rados.rados.0.smithi084.stdout:2420: writing smithi08434663-17 from 1906495 to 2252336 tid 2 2024-09-18T14:10:03.292 INFO:tasks.rados.rados.0.smithi084.stdout:2413: expect (ObjNum 833 snap 252 seq_num 833) 2024-09-18T14:10:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.319+0000 7f4ec7354640 10 monclient: get_monmap_and_config 2024-09-18T14:10:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.319+0000 7f4ec7354640 10 monclient: build_initial_monmap 2024-09-18T14:10:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.319+0000 7f4ec7354640 10 monclient: monmap: 2024-09-18T14:10:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:03.320019+0000 2024-09-18T14:10:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:03.320019+0000 2024-09-18T14:10:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:03.320 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.319+0000 7f4ec7354640 1 Processor -- start 2024-09-18T14:10:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.319+0000 7f4ec7354640 1 -- start start 2024-09-18T14:10:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.319+0000 7f4ec7354640 10 monclient: init 2024-09-18T14:10:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ec7354640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ec7354640 10 monclient: _add_conns ranks=[5,3,2,6,8,7,1,4,0] 2024-09-18T14:10:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ec7354640 10 monclient(hunting): picked mon.noname-h con 0x7f4ec006e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ec7354640 10 monclient(hunting): picked mon.noname-g con 0x7f4ec00a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ec7354640 10 monclient(hunting): picked mon.noname-b con 0x7f4ec00a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ec7354640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec005ed70 con 0x7f4ec00a1080 2024-09-18T14:10:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ec7354640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec0097620 con 0x7f4ec00a0cb0 2024-09-18T14:10:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ec7354640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec0097340 con 0x7f4ec006e810 2024-09-18T14:10:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ec7354640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ec7354640 10 monclient(hunting): authenticate will time out at 1418.658341s 2024-09-18T14:10:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ec58f1640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f4ec00a1080 0x7f4ec013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33908/0 (socket says 172.21.15.45:33908) 2024-09-18T14:10:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ec58f1640 1 -- 172.21.15.45:0/3043277420 learned_addr learned my addr 172.21.15.45:0/3043277420 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776921106 0 0) 0x7f4ec005ed70 con 0x7f4ec00a1080 2024-09-18T14:10:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 10 monclient(hunting): my global_id is 144875 2024-09-18T14:10:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea80034e0 con 0x7f4ec00a1080 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050722104 0 0) 0x7f4ec0097340 con 0x7f4ec006e810 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 10 monclient(hunting): my global_id is 184163 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec005ed70 con 0x7f4ec006e810 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367517534 0 0) 0x7f4ec0097620 con 0x7f4ec00a0cb0 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 10 monclient(hunting): my global_id is 184134 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec0097340 con 0x7f4ec00a0cb0 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3431465100 0 0) 0x7f4ea80034e0 con 0x7f4ec00a1080 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 >> v1:172.21.15.84:6791/0 conn(0x7f4ec006e810 legacy=0x7f4ec00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 >> v1:172.21.15.84:6790/0 conn(0x7f4ec00a0cb0 legacy=0x7f4ec0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 10 monclient: _finish_hunting 0 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 1 monclient: found mon.noname-b 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.320+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec00a1660 con 0x7f4ec00a1080 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ebffff640 10 monclient: _finish_auth 0 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ebffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4eb8001830 con 0x7f4ec00a1080 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ec7354640 5 monclient: authenticate success, global_id 144875 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ec7354640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ebffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ebffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ebffff640 10 monclient: dump: 2024-09-18T14:10:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ec7354640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4eb8001a10 con 0x7f4ec00a1080 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ebffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ebffff640 1 -- 172.21.15.45:0/3043277420 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4eb8004ba0 con 0x7f4ec00a1080 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ec7354640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ec7354640 4 set_mon_vals no callback set 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ebffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ebffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ebffff640 10 monclient: dump: 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.335 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ec7354640 10 monclient: shutdown 2024-09-18T14:10:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ec7354640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ec7354640 1 -- 172.21.15.45:0/3043277420 >> v1:172.21.15.45:6790/0 conn(0x7f4ec00a1080 legacy=0x7f4ec013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ec7354640 1 -- 172.21.15.45:0/3043277420 shutdown_connections 2024-09-18T14:10:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ec7354640 1 -- 172.21.15.45:0/3043277420 >> 172.21.15.45:0/3043277420 conn(0x7f4ec00a5bb0 msgr2=0x7f4ec00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ec7354640 1 -- 172.21.15.45:0/3043277420 shutdown_connections 2024-09-18T14:10:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.321+0000 7f4ec7354640 1 -- 172.21.15.45:0/3043277420 wait complete. 2024-09-18T14:10:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.322+0000 7f4ec7354640 10 monclient: build_initial_monmap 2024-09-18T14:10:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.322+0000 7f4ec7354640 10 monclient: monmap: 2024-09-18T14:10:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:03.337 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.322+0000 7f4ec7354640 1 Processor -- start 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.322+0000 7f4ec7354640 1 -- start start 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.322+0000 7f4ec7354640 10 monclient: init 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ec7354640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ec7354640 10 monclient: _add_conns ranks=[4,7,5,3,6,1,2,8,0] 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ec7354640 10 monclient(hunting): picked mon.noname-c con 0x7f4ec006e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ec7354640 10 monclient(hunting): picked mon.noname-i con 0x7f4ec00a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ec7354640 10 monclient(hunting): picked mon.noname-h con 0x7f4ec00a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ec7354640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec009fa10 con 0x7f4ec006e810 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ec7354640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec00a2270 con 0x7f4ec00a1080 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ec7354640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec0136a90 con 0x7f4ec00a0cb0 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ec7354640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ec7354640 10 monclient(hunting): authenticate will time out at 1418.661190s 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ec50f0640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f4ec006e810 0x7f4ec01353f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:56782/0 (socket says 172.21.15.45:56782) 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ec50f0640 1 -- 172.21.15.45:0/287617021 learned_addr learned my addr 172.21.15.45:0/287617021 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721859157 0 0) 0x7f4ec009fa10 con 0x7f4ec006e810 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): my global_id is 164389 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eac0034e0 con 0x7f4ec006e810 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493059003 0 0) 0x7f4ec0136a90 con 0x7f4ec00a0cb0 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): my global_id is 184156 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec009fa10 con 0x7f4ec00a0cb0 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831839822 0 0) 0x7f4ec00a2270 con 0x7f4ec00a1080 2024-09-18T14:10:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): my global_id is 184172 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec0136a90 con 0x7f4ec00a1080 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3239852917 0 0) 0x7f4eac0034e0 con 0x7f4ec006e810 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4ec00a2270 con 0x7f4ec006e810 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4eb0002f90 con 0x7f4ec006e810 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): dump: 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2847142969 0 0) 0x7f4ec0136a90 con 0x7f4ec00a1080 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4eac0034e0 con 0x7f4ec00a1080 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4eb80017f0 con 0x7f4ec00a1080 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): dump: 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3266943342 0 0) 0x7f4ec009fa10 con 0x7f4ec00a0cb0 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4ec0136a90 con 0x7f4ec00a0cb0 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4eb4001a60 con 0x7f4ec00a0cb0 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.323+0000 7f4ebd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 10 monclient(hunting): dump: 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3801516851 0 0) 0x7f4ec00a2270 con 0x7f4ec006e810 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 >> v1:172.21.15.84:6792/0 conn(0x7f4ec00a0cb0 legacy=0x7f4ec01468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 >> v1:172.21.15.84:6791/0 conn(0x7f4ec00a1080 legacy=0x7f4ec0149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 10 monclient: _finish_hunting 0 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 1 monclient: found mon.c 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec01424a0 con 0x7f4ec006e810 2024-09-18T14:10:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 10 monclient: _finish_auth 0 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ec7354640 5 monclient: authenticate success, global_id 164389 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ec7354640 10 monclient: _renew_subs 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ec7354640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ec7354640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec0136190 con 0x7f4ec006e810 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ec7354640 10 monclient: _renew_subs 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ec7354640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ec7354640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec0122ad0 con 0x7f4ec006e810 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4eb0001a20 con 0x7f4ec006e810 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4eb0001f50 con 0x7f4ec006e810 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebf7fe640 4 set_mon_vals no callback set 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 10 monclient: dump: 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4eb00020b0 con 0x7f4ec006e810 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ec7354640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.324+0000 7f4ec7354640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4ea0006580 con 0x7f4ea0002830 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(528..528 src has 344..528) v4 ==== 3987+0+0 (unknown 4243422052 0 0) 0x7f4eb0046f80 con 0x7f4ec006e810 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ec58f1640 10 monclient: get_auth_request con 0x7f4eac03a840 auth_method 0 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355831322 0 0) 0x7f4ea0006580 con 0x7f4ea0002830 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 10 monclient(hunting): my global_id is 184143 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec00a2270 con 0x7f4ea0002830 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1658591246 0 0) 0x7f4ec00a2270 con 0x7f4ea0002830 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4ea0006580 con 0x7f4ea0002830 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4eb40021b0 con 0x7f4ea0002830 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 10 monclient: dump: 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2941715993 0 0) 0x7f4ea0006580 con 0x7f4ea0002830 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.325+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ea000a0a0 con 0x7f4ea0002830 2024-09-18T14:10:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.326+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f4eb40044b0 con 0x7f4ea0002830 2024-09-18T14:10:03.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.326+0000 7f4ebd7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.326+0000 7f4ebd7fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:03.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.326+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 >> v1:172.21.15.84:6790/0 conn(0x7f4ea0002830 legacy=0x7f4ea0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.326+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 >> v1:172.21.15.84:6790/0 conn(0x7f4ea0002830 legacy=0x7f4ea0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:03.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.331+0000 7f4ec7354640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:03.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.331+0000 7f4ec7354640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4ea00074f0 con 0x7f4ea00068a0 2024-09-18T14:10:03.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781353503 0 0) 0x7f4ea00074f0 con 0x7f4ea00068a0 2024-09-18T14:10:03.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 10 monclient(hunting): my global_id is 184152 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea0006580 con 0x7f4ea00068a0 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3613286788 0 0) 0x7f4ea0006580 con 0x7f4ea00068a0 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4ea00074f0 con 0x7f4ea00068a0 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4eb4002c70 con 0x7f4ea00068a0 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 10 monclient: dump: 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.354 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3663271164 0 0) 0x7f4ea00074f0 con 0x7f4ea00068a0 2024-09-18T14:10:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4ea0008750 con 0x7f4ea00068a0 2024-09-18T14:10:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 4230399705) 0x7f4eb40040a0 con 0x7f4ea00068a0 2024-09-18T14:10:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 >> v1:172.21.15.84:6790/0 conn(0x7f4ea00068a0 legacy=0x7f4ea0006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.332+0000 7f4ebd7fa640 1 -- 172.21.15.45:0/287617021 >> v1:172.21.15.84:6790/0 conn(0x7f4ea00068a0 legacy=0x7f4ea0006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:03.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:03.355 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:03.355 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.356 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:10:03.357 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:03.357 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:03.357 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:03.357 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.357 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.357 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.357 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.357 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.357 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.357 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.357 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.357 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.358 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:03.359 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:03.360 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:03.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:03.360 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:03.360 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:03.360 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:03.360 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.360 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.360 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.360 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.360 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.360 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.360 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.362 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:03.363 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.364 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.364 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.364 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:03.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:03.364 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.364 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.364 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:03.364 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:03.365 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.369 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:03.370 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.371 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.373 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.374 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:03.375 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.375 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.375 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:03.375 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:03.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.333+0000 7f4ec7354640 1 -- 172.21.15.45:0/287617021 >> v1:172.21.15.84:6812/31478 conn(0x7f4eac03a840 legacy=0x7f4eac03ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.333+0000 7f4ec7354640 10 monclient: shutdown 2024-09-18T14:10:03.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.333+0000 7f4ec7354640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.333+0000 7f4ec7354640 1 -- 172.21.15.45:0/287617021 >> v1:172.21.15.45:6791/0 conn(0x7f4ec006e810 legacy=0x7f4ec01353f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.333+0000 7f4ec58f1640 1 -- 172.21.15.45:0/287617021 reap_dead start 2024-09-18T14:10:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.333+0000 7f4ec58f1640 1 -- 172.21.15.45:0/287617021 reap_dead start 2024-09-18T14:10:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.333+0000 7f4ec7354640 1 -- 172.21.15.45:0/287617021 shutdown_connections 2024-09-18T14:10:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.333+0000 7f4ec7354640 1 -- 172.21.15.45:0/287617021 >> 172.21.15.45:0/287617021 conn(0x7f4ec00a5bb0 msgr2=0x7f4ec00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.333+0000 7f4ec7354640 1 -- 172.21.15.45:0/287617021 shutdown_connections 2024-09-18T14:10:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.333+0000 7f4ec7354640 1 -- 172.21.15.45:0/287617021 wait complete. 2024-09-18T14:10:03.377 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:03.377 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:10:03.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:10:03.397 INFO:tasks.rados.rados.0.smithi084.stdout:2417: finishing write tid 1 to smithi08434663-33 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:2417: finishing write tid 2 to smithi08434663-33 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:2417: finishing write tid 3 to smithi08434663-33 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:2417: finishing write tid 4 to smithi08434663-33 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 1009 (ObjNum 847 snap 255 seq_num 847) dirty exists 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:2417: left oid 33 (ObjNum 847 snap 255 seq_num 847) 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:2420: finishing write tid 1 to smithi08434663-17 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:2420: finishing write tid 2 to smithi08434663-17 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:2420: finishing write tid 3 to smithi08434663-17 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:2420: finishing write tid 4 to smithi08434663-17 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 895 (ObjNum 848 snap 255 seq_num 848) dirty exists 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:2420: left oid 17 (ObjNum 848 snap 255 seq_num 848) 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:2421: read oid 10 snap 227 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:2421: expect (ObjNum 538 snap 167 seq_num 538) 2024-09-18T14:10:03.398 INFO:tasks.rados.rados.0.smithi084.stdout:2413: done (6 left) 2024-09-18T14:10:03.399 INFO:tasks.rados.rados.0.smithi084.stdout:2414: done (5 left) 2024-09-18T14:10:03.399 INFO:tasks.rados.rados.0.smithi084.stdout:2415: done (4 left) 2024-09-18T14:10:03.399 INFO:tasks.rados.rados.0.smithi084.stdout:2416: done (3 left) 2024-09-18T14:10:03.399 INFO:tasks.rados.rados.0.smithi084.stdout:2417: done (2 left) 2024-09-18T14:10:03.399 INFO:tasks.rados.rados.0.smithi084.stdout:2420: done (1 left) 2024-09-18T14:10:03.399 INFO:tasks.rados.rados.0.smithi084.stdout:2422: write oid 6 current snap is 255 2024-09-18T14:10:03.399 INFO:tasks.rados.rados.0.smithi084.stdout:2422: seq_num 849 ranges {669143=401584,1564977=426959} 2024-09-18T14:10:03.403 INFO:tasks.rados.rados.0.smithi084.stdout:2422: writing smithi08434663-6 from 669143 to 1070727 tid 1 2024-09-18T14:10:03.408 INFO:tasks.rados.rados.0.smithi084.stdout:2422: writing smithi08434663-6 from 1564977 to 1991936 tid 2 2024-09-18T14:10:03.408 INFO:tasks.rados.rados.0.smithi084.stdout:2423: snap_remove snap 254 2024-09-18T14:10:03.408 INFO:tasks.rados.rados.0.smithi084.stdout:2421: expect (ObjNum 538 snap 167 seq_num 538) 2024-09-18T14:10:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.476+0000 7f6cbd39d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.476+0000 7f6cbd39d640 10 monclient: build_initial_monmap 2024-09-18T14:10:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.476+0000 7f6cbd39d640 10 monclient: monmap: 2024-09-18T14:10:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:03.477513+0000 2024-09-18T14:10:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:03.477513+0000 2024-09-18T14:10:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:03.478 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:03.478 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:03.478 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:03.478 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:03.478 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:03.478 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:03.478 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:03.478 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 1 Processor -- start 2024-09-18T14:10:03.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 1 -- start start 2024-09-18T14:10:03.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 10 monclient: init 2024-09-18T14:10:03.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:03.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 10 monclient: _add_conns ranks=[7,5,6,0,4,3,2,8,1] 2024-09-18T14:10:03.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 10 monclient(hunting): picked mon.noname-i con 0x7f6cb809f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:03.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 10 monclient(hunting): picked mon.noname-h con 0x7f6cb809fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:03.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 10 monclient(hunting): picked mon.noname-d con 0x7f6cb809ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:03.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb805ed70 con 0x7f6cb809fba0 2024-09-18T14:10:03.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb809dfc0 con 0x7f6cb809ff70 2024-09-18T14:10:03.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb809dce0 con 0x7f6cb809f7d0 2024-09-18T14:10:03.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:03.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cbd39d640 10 monclient(hunting): authenticate will time out at 1418.815699s 2024-09-18T14:10:03.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cb77fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f6cb809ff70 0x7f6cb813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34796/0 (socket says 172.21.15.45:34796) 2024-09-18T14:10:03.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.477+0000 7f6cb77fe640 1 -- 172.21.15.45:0/3202332879 learned_addr learned my addr 172.21.15.45:0/3202332879 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:03.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664643611 0 0) 0x7f6cb809dfc0 con 0x7f6cb809ff70 2024-09-18T14:10:03.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient(hunting): my global_id is 184173 2024-09-18T14:10:03.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca00034e0 con 0x7f6cb809ff70 2024-09-18T14:10:03.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943387326 0 0) 0x7f6cb809dce0 con 0x7f6cb809f7d0 2024-09-18T14:10:03.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient(hunting): my global_id is 184165 2024-09-18T14:10:03.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb809dfc0 con 0x7f6cb809f7d0 2024-09-18T14:10:03.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410193908 0 0) 0x7f6cb805ed70 con 0x7f6cb809fba0 2024-09-18T14:10:03.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient(hunting): my global_id is 184181 2024-09-18T14:10:03.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb809dce0 con 0x7f6cb809fba0 2024-09-18T14:10:03.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1999478733 0 0) 0x7f6ca00034e0 con 0x7f6cb809ff70 2024-09-18T14:10:03.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 >> v1:172.21.15.84:6792/0 conn(0x7f6cb809f7d0 legacy=0x7f6cb8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 >> v1:172.21.15.84:6791/0 conn(0x7f6cb809fba0 legacy=0x7f6cb8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 monclient: found mon.noname-d 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb80a1c00 con 0x7f6cb809ff70 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient: _finish_auth 0 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6cb0001bb0 con 0x7f6cb809ff70 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cbd39d640 5 monclient: authenticate success, global_id 184173 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cbd39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient: dump: 2024-09-18T14:10:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.491 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cbd39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6cb0001d90 con 0x7f6cb809ff70 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 1 -- 172.21.15.45:0/3202332879 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6cb0004f20 con 0x7f6cb809ff70 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cbd39d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cbd39d640 4 set_mon_vals no callback set 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cb5ffb640 10 monclient: dump: 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.492 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cbd39d640 10 monclient: shutdown 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cbd39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.478+0000 7f6cbd39d640 1 -- 172.21.15.45:0/3202332879 >> v1:172.21.15.45:6792/0 conn(0x7f6cb809ff70 legacy=0x7f6cb813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.479+0000 7f6cbd39d640 1 -- 172.21.15.45:0/3202332879 shutdown_connections 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.479+0000 7f6cbd39d640 1 -- 172.21.15.45:0/3202332879 >> 172.21.15.45:0/3202332879 conn(0x7f6cb80630b0 msgr2=0x7f6cb80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.479+0000 7f6cbd39d640 1 -- 172.21.15.45:0/3202332879 shutdown_connections 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.479+0000 7f6cbd39d640 1 -- 172.21.15.45:0/3202332879 wait complete. 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.479+0000 7f6cbd39d640 10 monclient: build_initial_monmap 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.479+0000 7f6cbd39d640 10 monclient: monmap: 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:03.494 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 1 Processor -- start 2024-09-18T14:10:03.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 1 -- start start 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 10 monclient: init 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 10 monclient: _add_conns ranks=[4,3,8,2,7,1,5,6,0] 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 10 monclient(hunting): picked mon.noname-c con 0x7f6cb809f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 10 monclient(hunting): picked mon.noname-g con 0x7f6cb809fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 10 monclient(hunting): picked mon.noname-e con 0x7f6cb809ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb809d310 con 0x7f6cb809fba0 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb80a2560 con 0x7f6cb809f7d0 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb8121a80 con 0x7f6cb809ff70 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cbd39d640 10 monclient(hunting): authenticate will time out at 1418.818707s 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cb6ffd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f6cb809f7d0 0x7f6cb8131650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:56784/0 (socket says 172.21.15.45:56784) 2024-09-18T14:10:03.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6cb6ffd640 1 -- 172.21.15.45:0/2794590724 learned_addr learned my addr 172.21.15.45:0/2794590724 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477043079 0 0) 0x7f6cb80a2560 con 0x7f6cb809f7d0 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6c9effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6c9effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.480+0000 7f6c9effd640 10 monclient(hunting): my global_id is 164398 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca40034e0 con 0x7f6cb809f7d0 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904202718 0 0) 0x7f6cb809d310 con 0x7f6cb809fba0 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): my global_id is 184161 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb80a2560 con 0x7f6cb809fba0 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476191326 0 0) 0x7f6cb8121a80 con 0x7f6cb809ff70 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): my global_id is 5498 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb809d310 con 0x7f6cb809ff70 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2740034357 0 0) 0x7f6ca40034e0 con 0x7f6cb809f7d0 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6cb8121a80 con 0x7f6cb809f7d0 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2316002136 0 0) 0x7f6cb80a2560 con 0x7f6cb809fba0 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ca40034e0 con 0x7f6cb809fba0 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6ca8001d90 con 0x7f6cb809f7d0 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): dump: 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.498 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6cac001640 con 0x7f6cb809fba0 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): dump: 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.500 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 573366614 0 0) 0x7f6cb809d310 con 0x7f6cb809ff70 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6cb80a2560 con 0x7f6cb809ff70 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6cb0001af0 con 0x7f6cb809ff70 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient(hunting): dump: 2024-09-18T14:10:03.501 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.502 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2379208029 0 0) 0x7f6cb8121a80 con 0x7f6cb809f7d0 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 >> v1:172.21.15.45:6793/0 conn(0x7f6cb809ff70 legacy=0x7f6cb814a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 >> v1:172.21.15.84:6790/0 conn(0x7f6cb809fba0 legacy=0x7f6cb8146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient: _finish_hunting 0 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 monclient: found mon.c 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb811fce0 con 0x7f6cb809f7d0 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient: _finish_auth 0 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:03.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6cbd39d640 5 monclient: authenticate success, global_id 164398 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6cbd39d640 10 monclient: _renew_subs 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6cbd39d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6cbd39d640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb81217a0 con 0x7f6cb809f7d0 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6cbd39d640 10 monclient: _renew_subs 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6cbd39d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6cbd39d640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb8131d60 con 0x7f6cb809f7d0 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6ca80017c0 con 0x7f6cb809f7d0 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6ca8002680 con 0x7f6cb809f7d0 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6cb57fa640 4 set_mon_vals no callback set 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.481+0000 7f6c9effd640 10 monclient: dump: 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.482+0000 7f6cbd39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.482+0000 7f6cbd39d640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6c98006580 con 0x7f6c98002830 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.482+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f6ca8010120 con 0x7f6cb809f7d0 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.482+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(528..528 src has 344..528) v4 ==== 3987+0+0 (unknown 4243422052 0 0) 0x7f6ca8046be0 con 0x7f6cb809f7d0 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.482+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430706932 0 0) 0x7f6c98006580 con 0x7f6c98002830 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.482+0000 7f6c9effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.482+0000 7f6c9effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.482+0000 7f6c9effd640 10 monclient(hunting): my global_id is 184190 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.482+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb8121a80 con 0x7f6c98002830 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.482+0000 7f6cb77fe640 10 monclient: get_auth_request con 0x7f6ca403a920 auth_method 0 2024-09-18T14:10:03.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.482+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3498655549 0 0) 0x7f6cb8121a80 con 0x7f6c98002830 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.483+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c98006580 con 0x7f6c98002830 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.483+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6cac002530 con 0x7f6c98002830 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.483+0000 7f6c9effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.483+0000 7f6c9effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.483+0000 7f6c9effd640 10 monclient: dump: 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.507 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.483+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1408800381 0 0) 0x7f6c98006580 con 0x7f6c98002830 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.483+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c9800a0a0 con 0x7f6c98002830 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.483+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6cac004770 con 0x7f6c98002830 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.483+0000 7f6c9effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.483+0000 7f6c9effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:03.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.483+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 >> v1:172.21.15.84:6791/0 conn(0x7f6c98002830 legacy=0x7f6c98004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.483+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 >> v1:172.21.15.84:6791/0 conn(0x7f6c98002830 legacy=0x7f6c98004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:03.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6cbd39d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:03.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6cbd39d640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6c98009e70 con 0x7f6c980068f0 2024-09-18T14:10:03.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669798424 0 0) 0x7f6c98009e70 con 0x7f6c980068f0 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6c9effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6c9effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6c9effd640 10 monclient(hunting): my global_id is 184199 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c98006580 con 0x7f6c980068f0 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1060878286 0 0) 0x7f6c98006580 con 0x7f6c980068f0 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c98009e70 con 0x7f6c980068f0 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6cac002fb0 con 0x7f6c980068f0 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6c9effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6c9effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6c9effd640 10 monclient: dump: 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.511 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.512 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.512 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.512 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.512 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.512 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.512 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.512 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.512 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.512 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.512 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.512 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.512 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.513 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.489+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 386109182 0 0) 0x7f6c98009e70 con 0x7f6c980068f0 2024-09-18T14:10:03.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6c98008750 con 0x7f6c980068f0 2024-09-18T14:10:03.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 71877870) 0x7f6cac004400 con 0x7f6c980068f0 2024-09-18T14:10:03.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6c9effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:03.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6c9effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:03.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 >> v1:172.21.15.84:6791/0 conn(0x7f6c980068f0 legacy=0x7f6c98006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6c9effd640 1 -- 172.21.15.45:0/2794590724 >> v1:172.21.15.84:6791/0 conn(0x7f6c980068f0 legacy=0x7f6c98006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:03.513 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:03.514 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.515 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.516 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.516 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.516 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.516 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:03.516 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:03.516 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.516 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.516 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.516 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.516 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.516 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.516 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:03.517 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:03.521 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:03.522 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:03.523 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:03.533 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:03.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6cbd39d640 1 -- 172.21.15.45:0/2794590724 >> v1:172.21.15.84:6812/31478 conn(0x7f6ca403a920 legacy=0x7f6ca403cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6cbd39d640 10 monclient: shutdown 2024-09-18T14:10:03.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6cbd39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:03.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6cbd39d640 1 -- 172.21.15.45:0/2794590724 >> v1:172.21.15.45:6791/0 conn(0x7f6cb809f7d0 legacy=0x7f6cb8131650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6cb77fe640 1 -- 172.21.15.45:0/2794590724 reap_dead start 2024-09-18T14:10:03.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6cb77fe640 1 -- 172.21.15.45:0/2794590724 reap_dead start 2024-09-18T14:10:03.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6cbd39d640 1 -- 172.21.15.45:0/2794590724 shutdown_connections 2024-09-18T14:10:03.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6cbd39d640 1 -- 172.21.15.45:0/2794590724 >> 172.21.15.45:0/2794590724 conn(0x7f6cb80630b0 msgr2=0x7f6cb80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:03.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6cbd39d640 1 -- 172.21.15.45:0/2794590724 shutdown_connections 2024-09-18T14:10:03.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.490+0000 7f6cbd39d640 1 -- 172.21.15.45:0/2794590724 wait complete. 2024-09-18T14:10:03.535 INFO:tasks.rados.rados.0.smithi084.stdout:2422: finishing write tid 1 to smithi08434663-6 2024-09-18T14:10:03.535 INFO:tasks.rados.rados.0.smithi084.stdout:2422: finishing write tid 2 to smithi08434663-6 2024-09-18T14:10:03.535 INFO:tasks.rados.rados.0.smithi084.stdout:2422: finishing write tid 3 to smithi08434663-6 2024-09-18T14:10:03.535 INFO:tasks.rados.rados.0.smithi084.stdout:2422: finishing write tid 4 to smithi08434663-6 2024-09-18T14:10:03.535 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 954 (ObjNum 849 snap 255 seq_num 849) dirty exists 2024-09-18T14:10:03.536 INFO:tasks.rados.rados.0.smithi084.stdout:2422: left oid 6 (ObjNum 849 snap 255 seq_num 849) 2024-09-18T14:10:03.536 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:03.536 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:10:03.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:10:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.633+0000 7f62ccf1d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.633+0000 7f62ccf1d640 10 monclient: build_initial_monmap 2024-09-18T14:10:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.633+0000 7f62ccf1d640 10 monclient: monmap: 2024-09-18T14:10:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:03.634609+0000 2024-09-18T14:10:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:03.634609+0000 2024-09-18T14:10:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.635 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:03.635 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:03.635 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:03.635 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:03.635 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:03.635 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:03.635 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:03.635 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:03.635 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:03.635 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 1 Processor -- start 2024-09-18T14:10:03.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 1 -- start start 2024-09-18T14:10:03.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 10 monclient: init 2024-09-18T14:10:03.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:03.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 10 monclient: _add_conns ranks=[8,5,3,4,0,6,7,1,2] 2024-09-18T14:10:03.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 10 monclient(hunting): picked mon.noname-e con 0x7f62c809f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 10 monclient(hunting): picked mon.noname-h con 0x7f62c809fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 10 monclient(hunting): picked mon.noname-g con 0x7f62c809ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c805ed70 con 0x7f62c809ff70 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c809dfc0 con 0x7f62c809fba0 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c809dce0 con 0x7f62c809f7d0 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.634+0000 7f62ccf1d640 10 monclient(hunting): authenticate will time out at 1418.972919s 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c659c640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f62c809f7d0 0x7f62c8132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:44866/0 (socket says 172.21.15.45:44866) 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c659c640 1 -- 172.21.15.45:0/3654558616 learned_addr learned my addr 172.21.15.45:0/3654558616 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305991854 0 0) 0x7f62c809dce0 con 0x7f62c809f7d0 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient(hunting): my global_id is 5507 2024-09-18T14:10:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62b40034e0 con 0x7f62c809f7d0 2024-09-18T14:10:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2116166870 0 0) 0x7f62c805ed70 con 0x7f62c809ff70 2024-09-18T14:10:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient(hunting): my global_id is 184170 2024-09-18T14:10:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c809dce0 con 0x7f62c809ff70 2024-09-18T14:10:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040416773 0 0) 0x7f62c809dfc0 con 0x7f62c809fba0 2024-09-18T14:10:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient(hunting): my global_id is 184208 2024-09-18T14:10:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c805ed70 con 0x7f62c809fba0 2024-09-18T14:10:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3919644991 0 0) 0x7f62b40034e0 con 0x7f62c809f7d0 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 >> v1:172.21.15.84:6791/0 conn(0x7f62c809fba0 legacy=0x7f62c8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 >> v1:172.21.15.84:6790/0 conn(0x7f62c809ff70 legacy=0x7f62c813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 monclient: found mon.noname-e 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c80a1c00 con 0x7f62c809f7d0 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient: _finish_auth 0 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f62b8001830 con 0x7f62c809f7d0 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:10:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient: dump: 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62ccf1d640 5 monclient: authenticate success, global_id 5507 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62ccf1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f62b8001a10 con 0x7f62c809f7d0 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 1 -- 172.21.15.45:0/3654558616 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f62b8004ba0 con 0x7f62c809f7d0 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62c5d9b640 10 monclient: dump: 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62ccf1d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:03.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.635+0000 7f62ccf1d640 4 set_mon_vals no callback set 2024-09-18T14:10:03.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.636+0000 7f62ccf1d640 10 monclient: shutdown 2024-09-18T14:10:03.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.636+0000 7f62ccf1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:03.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.636+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3654558616 >> v1:172.21.15.45:6793/0 conn(0x7f62c809f7d0 legacy=0x7f62c8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.636+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3654558616 shutdown_connections 2024-09-18T14:10:03.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.636+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3654558616 >> 172.21.15.45:0/3654558616 conn(0x7f62c80630b0 msgr2=0x7f62c80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:03.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.636+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3654558616 shutdown_connections 2024-09-18T14:10:03.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.636+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3654558616 wait complete. 2024-09-18T14:10:03.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.636+0000 7f62ccf1d640 10 monclient: build_initial_monmap 2024-09-18T14:10:03.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.636+0000 7f62ccf1d640 10 monclient: monmap: 2024-09-18T14:10:03.650 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:03.650 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 1 Processor -- start 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 1 -- start start 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 10 monclient: init 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 10 monclient: _add_conns ranks=[1,0,6,3,8,4,7,2,5] 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 10 monclient(hunting): picked mon.noname-f con 0x7f62c809f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 10 monclient(hunting): picked mon.noname-a con 0x7f62c809fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 10 monclient(hunting): picked mon.noname-d con 0x7f62c809ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c809d310 con 0x7f62c809fba0 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c80a2560 con 0x7f62c809f7d0 2024-09-18T14:10:03.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c811d510 con 0x7f62c809ff70 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62ccf1d640 10 monclient(hunting): authenticate will time out at 1418.975596s 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62c659c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f62c809f7d0 0x7f62c81316f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:47042/0 (socket says 172.21.15.45:47042) 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62c659c640 1 -- 172.21.15.45:0/3599379358 learned_addr learned my addr 172.21.15.45:0/3599379358 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62c6d9d640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f62c809ff70 0x7f62c814a470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34812/0 (socket says 172.21.15.45:34812) 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454286200 0 0) 0x7f62c809d310 con 0x7f62c809fba0 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 10 monclient(hunting): my global_id is 5616 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62a80034e0 con 0x7f62c809fba0 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937244686 0 0) 0x7f62c811d510 con 0x7f62c809ff70 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 10 monclient(hunting): my global_id is 184182 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c809d310 con 0x7f62c809ff70 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845324370 0 0) 0x7f62c80a2560 con 0x7f62c809f7d0 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 10 monclient(hunting): my global_id is 164323 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.637+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c811d510 con 0x7f62c809f7d0 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2403431134 0 0) 0x7f62a80034e0 con 0x7f62c809fba0 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62c80a2560 con 0x7f62c809fba0 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f62b0002a90 con 0x7f62c809fba0 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient(hunting): dump: 2024-09-18T14:10:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.655 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 284185565 0 0) 0x7f62c811d510 con 0x7f62c809f7d0 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62a80034e0 con 0x7f62c809f7d0 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1323201616 0 0) 0x7f62c809d310 con 0x7f62c809ff70 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62c811d510 con 0x7f62c809ff70 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f62b8004ae0 con 0x7f62c809f7d0 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient(hunting): dump: 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.657 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f62c0001cc0 con 0x7f62c809ff70 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient(hunting): dump: 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.659 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1694553456 0 0) 0x7f62c80a2560 con 0x7f62c809fba0 2024-09-18T14:10:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 >> v1:172.21.15.45:6792/0 conn(0x7f62c809ff70 legacy=0x7f62c814a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 >> v1:172.21.15.84:6789/0 conn(0x7f62c809f7d0 legacy=0x7f62c81316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:10:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 monclient: found mon.a 2024-09-18T14:10:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c81428a0 con 0x7f62c809fba0 2024-09-18T14:10:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient: _finish_auth 0 2024-09-18T14:10:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62ccf1d640 5 monclient: authenticate success, global_id 5616 2024-09-18T14:10:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62ccf1d640 10 monclient: _renew_subs 2024-09-18T14:10:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62ccf1d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c811d920 con 0x7f62c809fba0 2024-09-18T14:10:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62ccf1d640 10 monclient: _renew_subs 2024-09-18T14:10:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62ccf1d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c8136ee0 con 0x7f62c809fba0 2024-09-18T14:10:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f62b0002cd0 con 0x7f62c809fba0 2024-09-18T14:10:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f62b0001920 con 0x7f62c809fba0 2024-09-18T14:10:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62c4d99640 4 set_mon_vals no callback set 2024-09-18T14:10:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.638+0000 7f62be7fc640 10 monclient: dump: 2024-09-18T14:10:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.662 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62ccf1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f62ac006580 con 0x7f62ac002830 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f62b0001a80 con 0x7f62c809fba0 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(528..528 src has 344..528) v4 ==== 3987+0+0 (unknown 4243422052 0 0) 0x7f62b0044220 con 0x7f62c809fba0 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62c6d9d640 10 monclient: get_auth_request con 0x7f62a803aa40 auth_method 0 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104727886 0 0) 0x7f62ac006580 con 0x7f62ac002830 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 10 monclient(hunting): my global_id is 184174 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c80a2560 con 0x7f62ac002830 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1953388721 0 0) 0x7f62c80a2560 con 0x7f62ac002830 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62ac006580 con 0x7f62ac002830 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f62b80017c0 con 0x7f62ac002830 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.639+0000 7f62be7fc640 10 monclient: dump: 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.665 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.640+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2123176093 0 0) 0x7f62ac006580 con 0x7f62ac002830 2024-09-18T14:10:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.640+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62ac00a0a0 con 0x7f62ac002830 2024-09-18T14:10:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.640+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f62b80052d0 con 0x7f62ac002830 2024-09-18T14:10:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.640+0000 7f62be7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.640+0000 7f62be7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.640+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 >> v1:172.21.15.84:6792/0 conn(0x7f62ac002830 legacy=0x7f62ac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.640+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 >> v1:172.21.15.84:6792/0 conn(0x7f62ac002830 legacy=0x7f62ac004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62ccf1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f62ac009660 con 0x7f62ac009040 2024-09-18T14:10:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225253512 0 0) 0x7f62ac009660 con 0x7f62ac009040 2024-09-18T14:10:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 10 monclient(hunting): my global_id is 184183 2024-09-18T14:10:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62ac006580 con 0x7f62ac009040 2024-09-18T14:10:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2220928133 0 0) 0x7f62ac006580 con 0x7f62ac009040 2024-09-18T14:10:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62ac009660 con 0x7f62ac009040 2024-09-18T14:10:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f62b8005ad0 con 0x7f62ac009040 2024-09-18T14:10:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 10 monclient: dump: 2024-09-18T14:10:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.669 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3039097916 0 0) 0x7f62ac009660 con 0x7f62ac009040 2024-09-18T14:10:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.646+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f62ac009920 con 0x7f62ac009040 2024-09-18T14:10:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 2200375269) 0x7f62b8007d90 con 0x7f62ac009040 2024-09-18T14:10:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62be7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62be7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 >> v1:172.21.15.84:6792/0 conn(0x7f62ac009040 legacy=0x7f62ac008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62be7fc640 1 -- 172.21.15.45:0/3599379358 >> v1:172.21.15.84:6792/0 conn(0x7f62ac009040 legacy=0x7f62ac008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:03.670 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:10:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:03.689 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:03.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3599379358 >> v1:172.21.15.84:6812/31478 conn(0x7f62a803aa40 legacy=0x7f62a803cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62ccf1d640 10 monclient: shutdown 2024-09-18T14:10:03.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62ccf1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:03.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3599379358 >> v1:172.21.15.45:6789/0 conn(0x7f62c809fba0 legacy=0x7f62c8146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62c6d9d640 1 -- 172.21.15.45:0/3599379358 reap_dead start 2024-09-18T14:10:03.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62c6d9d640 1 -- 172.21.15.45:0/3599379358 reap_dead start 2024-09-18T14:10:03.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3599379358 shutdown_connections 2024-09-18T14:10:03.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3599379358 >> 172.21.15.45:0/3599379358 conn(0x7f62c80630b0 msgr2=0x7f62c80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:03.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3599379358 shutdown_connections 2024-09-18T14:10:03.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.647+0000 7f62ccf1d640 1 -- 172.21.15.45:0/3599379358 wait complete. 2024-09-18T14:10:03.691 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:03.691 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-18T14:10:03.692 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-18T14:10:03.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.787+0000 7fc5be8dd640 10 monclient: get_monmap_and_config 2024-09-18T14:10:03.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.787+0000 7fc5be8dd640 10 monclient: build_initial_monmap 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.787+0000 7fc5be8dd640 10 monclient: monmap: 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:03.788352+0000 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:03.788352+0000 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:03.847 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 1 Processor -- start 2024-09-18T14:10:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 1 -- start start 2024-09-18T14:10:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 10 monclient: init 2024-09-18T14:10:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 10 monclient: _add_conns ranks=[0,6,5,1,4,7,2,8,3] 2024-09-18T14:10:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 10 monclient(hunting): picked mon.noname-a con 0x7fc5b80a42a0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 10 monclient(hunting): picked mon.noname-d con 0x7fc5b80a1d10 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 10 monclient(hunting): picked mon.noname-h con 0x7fc5b8139060 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b805ed70 con 0x7fc5b80a42a0 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b80a35f0 con 0x7fc5b8139060 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b80a4670 con 0x7fc5b80a1d10 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5be8dd640 10 monclient(hunting): authenticate will time out at 1419.126679s 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5b7fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc5b80a42a0 0x7fc5b806e410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51100/0 (socket says 172.21.15.45:51100) 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5b7fff640 1 -- 172.21.15.45:0/1519498206 learned_addr learned my addr 172.21.15.45:0/1519498206 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.788+0000 7fc5b77fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fc5b80a1d10 0x7fc5b8137940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34826/0 (socket says 172.21.15.45:34826) 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807597046 0 0) 0x7fc5b80a4670 con 0x7fc5b80a1d10 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient(hunting): my global_id is 184191 2024-09-18T14:10:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a00034e0 con 0x7fc5b80a1d10 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614268982 0 0) 0x7fc5b805ed70 con 0x7fc5b80a42a0 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient(hunting): my global_id is 5625 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b80a4670 con 0x7fc5b80a42a0 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471313474 0 0) 0x7fc5b80a35f0 con 0x7fc5b8139060 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient(hunting): my global_id is 184217 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b805ed70 con 0x7fc5b8139060 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1054217791 0 0) 0x7fc5a00034e0 con 0x7fc5b80a1d10 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 >> v1:172.21.15.84:6791/0 conn(0x7fc5b8139060 legacy=0x7fc5b813b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b7fff640 1 -- 172.21.15.45:0/1519498206 >> v1:172.21.15.45:6789/0 conn(0x7fc5b80a42a0 legacy=0x7fc5b806e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-18T14:10:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 >> v1:172.21.15.45:6789/0 conn(0x7fc5b80a42a0 legacy=0x7fc5b806e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b7fff640 1 -- 172.21.15.45:0/1519498206 >> v1:172.21.15.45:6789/0 conn(0x7fc5b80a42a0 legacy=0x7fc5b806e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b7fff640 1 --1- 172.21.15.45:0/1519498206 >> v1:172.21.15.45:6789/0 conn(0x7fc5b80a42a0 0x7fc5b806e410 :-1 s=OPENED pgs=466 cs=1 l=1).handle_message read tag failed 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b7fff640 1 --1- 172.21.15.45:0/1519498206 >> v1:172.21.15.45:6789/0 conn(0x7fc5b80a42a0 0x7fc5b806e410 :-1 s=OPENED pgs=466 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 monclient: found mon.noname-d 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b80a27b0 con 0x7fc5b80a1d10 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient: _finish_auth 0 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient: ms_handle_reset stray mon v1:172.21.15.45:6789/0 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc5ac001830 con 0x7fc5b80a1d10 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient: dump: 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.860 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5be8dd640 5 monclient: authenticate success, global_id 184191 2024-09-18T14:10:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5be8dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc5ac001990 con 0x7fc5b80a1d10 2024-09-18T14:10:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 1 -- 172.21.15.45:0/1519498206 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc5ac004ae0 con 0x7fc5b80a1d10 2024-09-18T14:10:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5be8dd640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5be8dd640 4 set_mon_vals no callback set 2024-09-18T14:10:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5b6ffd640 10 monclient: dump: 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5be8dd640 10 monclient: shutdown 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5be8dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.789+0000 7fc5be8dd640 1 -- 172.21.15.45:0/1519498206 >> v1:172.21.15.45:6792/0 conn(0x7fc5b80a1d10 legacy=0x7fc5b8137940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.790+0000 7fc5be8dd640 1 -- 172.21.15.45:0/1519498206 shutdown_connections 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.790+0000 7fc5be8dd640 1 -- 172.21.15.45:0/1519498206 >> 172.21.15.45:0/1519498206 conn(0x7fc5b8063720 msgr2=0x7fc5b80a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.790+0000 7fc5be8dd640 1 -- 172.21.15.45:0/1519498206 shutdown_connections 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.790+0000 7fc5be8dd640 1 -- 172.21.15.45:0/1519498206 wait complete. 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.790+0000 7fc5be8dd640 10 monclient: build_initial_monmap 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.790+0000 7fc5be8dd640 10 monclient: monmap: 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.790+0000 7fc5be8dd640 1 Processor -- start 2024-09-18T14:10:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5be8dd640 1 -- start start 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5be8dd640 10 monclient: init 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5be8dd640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5be8dd640 10 monclient: _add_conns ranks=[1,5,7,8,6,4,2,0,3] 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5be8dd640 10 monclient(hunting): picked mon.noname-f con 0x7fc5b80a1d10 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5be8dd640 10 monclient(hunting): picked mon.noname-h con 0x7fc5b80a42a0 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5be8dd640 10 monclient(hunting): picked mon.noname-i con 0x7fc5b8139060 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5be8dd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b8139900 con 0x7fc5b80a1d10 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5be8dd640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b8136580 con 0x7fc5b80a42a0 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5be8dd640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b8068550 con 0x7fc5b8139060 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5be8dd640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5be8dd640 10 monclient(hunting): authenticate will time out at 1419.129533s 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5b7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc5b80a1d10 0x7fc5b8141b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:47044/0 (socket says 172.21.15.45:47044) 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.791+0000 7fc5b7fff640 1 -- 172.21.15.45:0/4264415132 learned_addr learned my addr 172.21.15.45:0/4264415132 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628370328 0 0) 0x7fc5b8139900 con 0x7fc5b80a1d10 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): my global_id is 164332 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a40034e0 con 0x7fc5b80a1d10 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081726896 0 0) 0x7fc5b8136580 con 0x7fc5b80a42a0 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): my global_id is 184226 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b8139900 con 0x7fc5b80a42a0 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941495328 0 0) 0x7fc5b8068550 con 0x7fc5b8139060 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): my global_id is 184192 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b8136580 con 0x7fc5b8139060 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1769666132 0 0) 0x7fc5a40034e0 con 0x7fc5b80a1d10 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5b8068550 con 0x7fc5b80a1d10 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc5a8001ac0 con 0x7fc5b80a1d10 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): dump: 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2155602270 0 0) 0x7fc5b8139900 con 0x7fc5b80a42a0 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5a40034e0 con 0x7fc5b80a42a0 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc5ac004ae0 con 0x7fc5b80a42a0 2024-09-18T14:10:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): dump: 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.869 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.869 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.869 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.872 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.872 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2924441919 0 0) 0x7fc5b8136580 con 0x7fc5b8139060 2024-09-18T14:10:03.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5b8139900 con 0x7fc5b8139060 2024-09-18T14:10:03.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc5b0001c40 con 0x7fc5b8139060 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient(hunting): dump: 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2741512853 0 0) 0x7fc5b8068550 con 0x7fc5b80a1d10 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 >> v1:172.21.15.84:6792/0 conn(0x7fc5b8139060 legacy=0x7fc5b8149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 >> v1:172.21.15.84:6791/0 conn(0x7fc5b80a42a0 legacy=0x7fc5b81466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient: _finish_hunting 0 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 monclient: found mon.f 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b8068ad0 con 0x7fc5b80a1d10 2024-09-18T14:10:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient: _finish_auth 0 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc5be8dd640 5 monclient: authenticate success, global_id 164332 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc5be8dd640 10 monclient: _renew_subs 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc5be8dd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc5be8dd640 1 -- 172.21.15.45:0/4264415132 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b8137ac0 con 0x7fc5b80a1d10 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc5be8dd640 10 monclient: _renew_subs 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc5be8dd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc5be8dd640 1 -- 172.21.15.45:0/4264415132 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b8142220 con 0x7fc5b80a1d10 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc5a8001d80 con 0x7fc5b80a1d10 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc5a8004e90 con 0x7fc5b80a1d10 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc5b67fc640 4 set_mon_vals no callback set 2024-09-18T14:10:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.792+0000 7fc59ffff640 10 monclient: dump: 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.793+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc5a8004ff0 con 0x7fc5b80a1d10 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.793+0000 7fc5be8dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.793+0000 7fc5be8dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.793+0000 7fc5be8dd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.793+0000 7fc5be8dd640 1 -- 172.21.15.45:0/4264415132 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc598005010 con 0x7fc5b80a1d10 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.793+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(528..528 src has 344..528) v4 ==== 3987+0+0 (unknown 4243422052 0 0) 0x7fc5a8046da0 con 0x7fc5b80a1d10 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.793+0000 7fc5b77fe640 10 monclient: get_auth_request con 0x7fc5a403a8e0 auth_method 0 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.798+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.1 v1:172.21.15.84:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fc5a80154d0 con 0x7fc5b80a1d10 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.798+0000 7fc59ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:03.798+0000 7fc59ffff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:04.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.025+0000 7fc5be8dd640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-18T14:10:04.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.025+0000 7fc5be8dd640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-18T14:10:04.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.025+0000 7fc5be8dd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:10:04.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.025+0000 7fc5be8dd640 1 -- 172.21.15.45:0/4264415132 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fc598005b90 con 0x7fc5b80a1d10 2024-09-18T14:10:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.027+0000 7fc59ffff640 1 -- 172.21.15.45:0/4264415132 <== mon.1 v1:172.21.15.84:6789/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7fc5a8014ed0 con 0x7fc5b80a1d10 2024-09-18T14:10:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.027+0000 7fc59ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-18T14:10:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.027+0000 7fc59ffff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:04.032 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=1659687362,config=3140514009,health=1477600393,kv=594742435,logm=4279542220}) 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=2,health=10,kv=7,logm=54} crc {auth=1843918487,config=3140514009,health=3868936392,kv=594742435,logm=1206826869}) 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=1659687362,config=3140514009,health=1477600393,kv=594742435,logm=4279542220}) 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=2295940798,config=3140514009,health=223668505,kv=594742435,logm=3489206544}) 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=1659687362,config=3140514009,health=1477600393,kv=594742435,logm=4279542220}) 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=27,config=1,health=12,kv=8,logm=52} crc {auth=4055546634,config=1228700967,health=223668505,kv=594742435,logm=683678623}) 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=1659687362,config=3140514009,health=1477600393,kv=594742435,logm=4279542220}) 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=4195289087,config=55734535,health=223668505,kv=594742435,logm=2432522340}) 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.033 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=1659687362,config=3140514009,health=1477600393,kv=594742435,logm=4279542220}) 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=1553513746,config=3140514009,health=223668505,kv=681570228,logm=982271095}) 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=1659687362,config=3140514009,health=1477600393,kv=594742435,logm=4279542220}) 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=10,kv=8,logm=54} crc {auth=693003259,config=3140514009,health=3960245248,kv=594742435,logm=1412862416}) 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=1659687362,config=3140514009,health=1477600393,kv=594742435,logm=4279542220}) 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=27,config=2,health=11,kv=7,logm=53} crc {auth=1720385713,config=3140514009,health=4264109200,kv=4171427160,logm=3406365984}) 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=1659687362,config=3140514009,health=1477600393,kv=594742435,logm=4279542220}) 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=26,config=2,health=12,kv=6,logm=54} crc {auth=3211957677,config=3140514009,health=223668505,kv=2609631587,logm=664217302}) 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235168324}) 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=849509603}) 2024-09-18T14:10:04.034 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235168324}) 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=711906926}) 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235168324}) 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=735180779}) 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235168324}) 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2706068884}) 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235168324}) 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=529485602}) 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235168324}) 2024-09-18T14:10:04.035 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=611683793}) 2024-09-18T14:10:04.036 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.036 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235168324}) 2024-09-18T14:10:04.036 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3979803620}) 2024-09-18T14:10:04.036 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.036 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235168324}) 2024-09-18T14:10:04.036 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3015004685}) 2024-09-18T14:10:04.036 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.036 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4190979296}) 2024-09-18T14:10:04.036 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307250533}) 2024-09-18T14:10:04.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.030+0000 7fc5be8dd640 1 -- 172.21.15.45:0/4264415132 >> v1:172.21.15.84:6812/31478 conn(0x7fc5a403a8e0 legacy=0x7fc5a403cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.030+0000 7fc5be8dd640 10 monclient: shutdown 2024-09-18T14:10:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.030+0000 7fc5be8dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.030+0000 7fc5be8dd640 1 -- 172.21.15.45:0/4264415132 >> v1:172.21.15.84:6789/0 conn(0x7fc5b80a1d10 legacy=0x7fc5b8141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:04.037 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.037 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4190979296}) 2024-09-18T14:10:04.037 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2954896823}) 2024-09-18T14:10:04.037 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.030+0000 7fc5be8dd640 1 -- 172.21.15.45:0/4264415132 shutdown_connections 2024-09-18T14:10:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.030+0000 7fc5be8dd640 1 -- 172.21.15.45:0/4264415132 >> 172.21.15.45:0/4264415132 conn(0x7fc5b8063720 msgr2=0x7fc5b809d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:04.039 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4190979296}) 2024-09-18T14:10:04.039 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=578187283}) 2024-09-18T14:10:04.039 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.039 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4190979296}) 2024-09-18T14:10:04.039 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3204672456}) 2024-09-18T14:10:04.039 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.039 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4190979296}) 2024-09-18T14:10:04.039 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2459026773}) 2024-09-18T14:10:04.039 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.039 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4190979296}) 2024-09-18T14:10:04.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.030+0000 7fc5be8dd640 1 -- 172.21.15.45:0/4264415132 shutdown_connections 2024-09-18T14:10:04.040 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=258742262}) 2024-09-18T14:10:04.040 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:04.030+0000 7fc5be8dd640 1 -- 172.21.15.45:0/4264415132 wait complete. 2024-09-18T14:10:04.052 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4190979296}) 2024-09-18T14:10:04.052 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1026799678}) 2024-09-18T14:10:04.052 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.052 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4190979296}) 2024-09-18T14:10:04.052 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1784923696}) 2024-09-18T14:10:04.052 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.052 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1584750704}) 2024-09-18T14:10:04.052 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1282797528}) 2024-09-18T14:10:04.052 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.052 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1584750704}) 2024-09-18T14:10:04.052 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1095523220}) 2024-09-18T14:10:04.052 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.053 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1584750704}) 2024-09-18T14:10:04.053 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=712603753}) 2024-09-18T14:10:04.053 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.053 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1584750704}) 2024-09-18T14:10:04.053 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1419370953}) 2024-09-18T14:10:04.053 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.053 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1584750704}) 2024-09-18T14:10:04.053 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=973828169}) 2024-09-18T14:10:04.053 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.053 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1584750704}) 2024-09-18T14:10:04.053 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=145643751}) 2024-09-18T14:10:04.053 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1584750704}) 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3413554743}) 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1584750704}) 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.032+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3080934119}) 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906982105}) 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4236305781}) 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906982105}) 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2190999416}) 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906982105}) 2024-09-18T14:10:04.054 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=725353850}) 2024-09-18T14:10:04.055 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.055 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906982105}) 2024-09-18T14:10:04.055 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3541831499}) 2024-09-18T14:10:04.055 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.055 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906982105}) 2024-09-18T14:10:04.055 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1632254405}) 2024-09-18T14:10:04.055 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.055 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906982105}) 2024-09-18T14:10:04.055 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3048327009}) 2024-09-18T14:10:04.055 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906982105}) 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2235801416}) 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906982105}) 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2145967178}) 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=454358088}) 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2027924063}) 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=454358088}) 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3302796800}) 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.056 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=454358088}) 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=905981668}) 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=454358088}) 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3525658621}) 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=454358088}) 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3885144987}) 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=454358088}) 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2367189666}) 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=454358088}) 2024-09-18T14:10:04.057 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=75906597}) 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=454358088}) 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3302556414}) 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1769414275}) 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1303715905}) 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1769414275}) 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2900978239}) 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1769414275}) 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2810627403}) 2024-09-18T14:10:04.058 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1769414275}) 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2961278933}) 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1769414275}) 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1347622248}) 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1769414275}) 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1629913897}) 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1769414275}) 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4027593937}) 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.059 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1769414275}) 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.035+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2042945233}) 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4134265936}) 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2384094087}) 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4134265936}) 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2955869966}) 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4134265936}) 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1359429881}) 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4134265936}) 2024-09-18T14:10:04.060 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2244353588}) 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4134265936}) 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=482570432}) 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4134265936}) 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2463316429}) 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4134265936}) 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=869828600}) 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4134265936}) 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3817797460}) 2024-09-18T14:10:04.061 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3093401853}) 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2732889112}) 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3093401853}) 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2711387387}) 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3093401853}) 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2789935648}) 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3093401853}) 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=33172100}) 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3093401853}) 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4187478908}) 2024-09-18T14:10:04.062 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3093401853}) 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=376017405}) 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3093401853}) 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1129370785}) 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3093401853}) 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2236581531}) 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2202892983}) 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1793351823}) 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2202892983}) 2024-09-18T14:10:04.063 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4023916352}) 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2202892983}) 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3130812018}) 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2202892983}) 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=811231777}) 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2202892983}) 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=615782650}) 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2202892983}) 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2259314171}) 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2202892983}) 2024-09-18T14:10:04.064 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1704028401}) 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2202892983}) 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3310504230}) 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1533115329}) 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=75073622}) 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1533115329}) 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2318633752}) 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1533115329}) 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=556568142}) 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.065 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1533115329}) 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3285953504}) 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1533115329}) 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1434457765}) 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1533115329}) 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1955949681}) 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1533115329}) 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3175963154}) 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1533115329}) 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3524970182}) 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.066 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078506908}) 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3186266836}) 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078506908}) 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3660159797}) 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078506908}) 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3836394570}) 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078506908}) 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1968932476}) 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078506908}) 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1908285404}) 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.067 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078506908}) 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=165411246}) 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078506908}) 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4278260274}) 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078506908}) 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3965631065}) 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=327418875}) 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1476759040}) 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=327418875}) 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=450576086}) 2024-09-18T14:10:04.068 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=327418875}) 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2503716116}) 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=327418875}) 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2848667571}) 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=327418875}) 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4227969871}) 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=327418875}) 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2386991847}) 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=327418875}) 2024-09-18T14:10:04.069 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=331043103}) 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=327418875}) 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2070140989}) 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339258237}) 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2048071073}) 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339258237}) 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3207513722}) 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339258237}) 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=460838919}) 2024-09-18T14:10:04.070 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339258237}) 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3283483446}) 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339258237}) 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3005803135}) 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339258237}) 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=292565751}) 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339258237}) 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1467973114}) 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339258237}) 2024-09-18T14:10:04.071 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2643836302}) 2024-09-18T14:10:04.072 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-18T14:10:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4284177660}) 2024-09-18T14:10:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4177683665}) 2024-09-18T14:10:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4284177660}) 2024-09-18T14:10:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2203712951}) 2024-09-18T14:10:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4284177660}) 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3837539770}) 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4284177660}) 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2942570555}) 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4284177660}) 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=511757985}) 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4284177660}) 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3396128238}) 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4284177660}) 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1317098392}) 2024-09-18T14:10:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4284177660}) 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1684087757}) 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2683205672}) 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578457546}) 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2683205672}) 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=565969986}) 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2683205672}) 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1853682448}) 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2683205672}) 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=575320344}) 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2683205672}) 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=250864056}) 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2683205672}) 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2349283957}) 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2683205672}) 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4254832293}) 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2683205672}) 2024-09-18T14:10:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2064365180}) 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380669777}) 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2880407745}) 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380669777}) 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3518160121}) 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380669777}) 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2257362359}) 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380669777}) 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1301223212}) 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380669777}) 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1705541970}) 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380669777}) 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1665219345}) 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380669777}) 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3488394613}) 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2380669777}) 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3194061101}) 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452953469}) 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1890429903}) 2024-09-18T14:10:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452953469}) 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3102913850}) 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452953469}) 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=551968406}) 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452953469}) 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3069304819}) 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452953469}) 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1976332556}) 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452953469}) 2024-09-18T14:10:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2413596870}) 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452953469}) 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3477575840}) 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452953469}) 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2036355015}) 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=85342462}) 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3077329801}) 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=85342462}) 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3371818817}) 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=85342462}) 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=414763348}) 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=85342462}) 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1962409351}) 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=85342462}) 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=724183064}) 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=85342462}) 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519116915}) 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=85342462}) 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4024026772}) 2024-09-18T14:10:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=85342462}) 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3565703905}) 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=518008885}) 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3848938498}) 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=518008885}) 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2635391254}) 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=518008885}) 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2563500048}) 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=518008885}) 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4292622665}) 2024-09-18T14:10:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=518008885}) 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2567784558}) 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=518008885}) 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3453458621}) 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=518008885}) 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=318586117}) 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=518008885}) 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.051+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2422592708}) 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423843657}) 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2301848945}) 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423843657}) 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3244062977}) 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423843657}) 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=689460853}) 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423843657}) 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3373910760}) 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423843657}) 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1338019425}) 2024-09-18T14:10:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423843657}) 2024-09-18T14:10:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=326377418}) 2024-09-18T14:10:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423843657}) 2024-09-18T14:10:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3133138171}) 2024-09-18T14:10:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423843657}) 2024-09-18T14:10:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2731025588}) 2024-09-18T14:10:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3553770330}) 2024-09-18T14:10:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2748298518}) 2024-09-18T14:10:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3553770330}) 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2767788705}) 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3553770330}) 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3685069084}) 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3553770330}) 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=836788897}) 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3553770330}) 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1836944608}) 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3553770330}) 2024-09-18T14:10:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=526248741}) 2024-09-18T14:10:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3553770330}) 2024-09-18T14:10:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=641055736}) 2024-09-18T14:10:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3553770330}) 2024-09-18T14:10:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=109825265}) 2024-09-18T14:10:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2801808065}) 2024-09-18T14:10:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3258861634}) 2024-09-18T14:10:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2801808065}) 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4194314358}) 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2801808065}) 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3275131782}) 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2801808065}) 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=137550095}) 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2801808065}) 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2967024275}) 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2801808065}) 2024-09-18T14:10:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3310166493}) 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2801808065}) 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3986628103}) 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2801808065}) 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1971694040}) 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3407993106}) 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3591365118}) 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3407993106}) 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=284347893}) 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3407993106}) 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2952922626}) 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3407993106}) 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=664834605}) 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3407993106}) 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2970363241}) 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3407993106}) 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=373308390}) 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3407993106}) 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3550542360}) 2024-09-18T14:10:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3407993106}) 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2064605677}) 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1057777998}) 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3744074954}) 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1057777998}) 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=496762494}) 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1057777998}) 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2708220686}) 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1057777998}) 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2803423981}) 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1057777998}) 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=789329006}) 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1057777998}) 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2727584971}) 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1057777998}) 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1042450774}) 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1057777998}) 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2524728771}) 2024-09-18T14:10:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1852259041}) 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123132728}) 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1852259041}) 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3015846176}) 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1852259041}) 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2319600869}) 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1852259041}) 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2713516339}) 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1852259041}) 2024-09-18T14:10:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4019125553}) 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1852259041}) 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2050949358}) 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1852259041}) 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3505751015}) 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1852259041}) 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3481396480}) 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3369334348}) 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1369031079}) 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3369334348}) 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=567115560}) 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3369334348}) 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2528234826}) 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3369334348}) 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2600362415}) 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3369334348}) 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=542477870}) 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3369334348}) 2024-09-18T14:10:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3128222731}) 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3369334348}) 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=202359571}) 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3369334348}) 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3806282887}) 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2408990796}) 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=802952239}) 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2408990796}) 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1611120483}) 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2408990796}) 2024-09-18T14:10:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3509488965}) 2024-09-18T14:10:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2408990796}) 2024-09-18T14:10:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1122130002}) 2024-09-18T14:10:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2408990796}) 2024-09-18T14:10:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4030637361}) 2024-09-18T14:10:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2408990796}) 2024-09-18T14:10:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4290710515}) 2024-09-18T14:10:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2408990796}) 2024-09-18T14:10:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3945188313}) 2024-09-18T14:10:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2408990796}) 2024-09-18T14:10:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1089956826}) 2024-09-18T14:10:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=242504700}) 2024-09-18T14:10:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3869716711}) 2024-09-18T14:10:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=242504700}) 2024-09-18T14:10:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=542124029}) 2024-09-18T14:10:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=242504700}) 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2292731537}) 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=242504700}) 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=683408142}) 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=242504700}) 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2925126306}) 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=242504700}) 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3608155959}) 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=242504700}) 2024-09-18T14:10:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2151397605}) 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=242504700}) 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=509897474}) 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883625577}) 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2498231498}) 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883625577}) 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=245682564}) 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883625577}) 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1021060018}) 2024-09-18T14:10:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883625577}) 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2764232789}) 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883625577}) 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2778136401}) 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883625577}) 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=711831051}) 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883625577}) 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3619885689}) 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883625577}) 2024-09-18T14:10:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1510598584}) 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3860623670}) 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2295722486}) 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3860623670}) 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4275399211}) 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3860623670}) 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2564771451}) 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3860623670}) 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4124975596}) 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3860623670}) 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2869240873}) 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3860623670}) 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2671788859}) 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3860623670}) 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1401031179}) 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3860623670}) 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2772174893}) 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2570819847}) 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1201569781}) 2024-09-18T14:10:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2570819847}) 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=498544788}) 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2570819847}) 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3707605353}) 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2570819847}) 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2484518066}) 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2570819847}) 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=455534610}) 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2570819847}) 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1822014238}) 2024-09-18T14:10:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2570819847}) 2024-09-18T14:10:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1597242116}) 2024-09-18T14:10:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2570819847}) 2024-09-18T14:10:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=11949489}) 2024-09-18T14:10:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749838286}) 2024-09-18T14:10:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=444555196}) 2024-09-18T14:10:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749838286}) 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1291670833}) 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749838286}) 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49093490}) 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749838286}) 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3416949226}) 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749838286}) 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3065211556}) 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749838286}) 2024-09-18T14:10:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=271436263}) 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749838286}) 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4237991708}) 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749838286}) 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3271996250}) 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=959924219}) 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1809218798}) 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=959924219}) 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3549568778}) 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=959924219}) 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1048858998}) 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=959924219}) 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=151875670}) 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=959924219}) 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=822626179}) 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=959924219}) 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3871770513}) 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=959924219}) 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3457707692}) 2024-09-18T14:10:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=959924219}) 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3467033573}) 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3337843444}) 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1054512189}) 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3337843444}) 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3813552576}) 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3337843444}) 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3905284906}) 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3337843444}) 2024-09-18T14:10:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1216909225}) 2024-09-18T14:10:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3337843444}) 2024-09-18T14:10:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1836588940}) 2024-09-18T14:10:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3337843444}) 2024-09-18T14:10:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583985340}) 2024-09-18T14:10:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3337843444}) 2024-09-18T14:10:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=938604730}) 2024-09-18T14:10:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3337843444}) 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=877421726}) 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3875137661}) 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1147519708}) 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3875137661}) 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1092504214}) 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3875137661}) 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1558200763}) 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3875137661}) 2024-09-18T14:10:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2397588872}) 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3875137661}) 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2844989611}) 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3875137661}) 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1233431853}) 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3875137661}) 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2456909956}) 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3875137661}) 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2521506158}) 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.071+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.071+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=714321261}) 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1680332567}) 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=714321261}) 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4180355755}) 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=714321261}) 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1051303003}) 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=714321261}) 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=999966369}) 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=714321261}) 2024-09-18T14:10:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1156370427}) 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=714321261}) 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=321778177}) 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=714321261}) 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3375777682}) 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=714321261}) 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1423704895}) 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2225307247}) 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=868305525}) 2024-09-18T14:10:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2225307247}) 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1838544577}) 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2225307247}) 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3130298539}) 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2225307247}) 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3001140213}) 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2225307247}) 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2022944642}) 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2225307247}) 2024-09-18T14:10:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3079214841}) 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2225307247}) 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2440496377}) 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2225307247}) 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3897907591}) 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3892066390}) 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4077393646}) 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3892066390}) 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3296093291}) 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3892066390}) 2024-09-18T14:10:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3140331335}) 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3892066390}) 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=116740813}) 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3892066390}) 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1998122416}) 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3892066390}) 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1545452527}) 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3892066390}) 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2911835335}) 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3892066390}) 2024-09-18T14:10:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1958423101}) 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3498133408}) 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366133568}) 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3498133408}) 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3190250877}) 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3498133408}) 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2092936506}) 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3498133408}) 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2198948465}) 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3498133408}) 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4195362962}) 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3498133408}) 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3419184264}) 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3498133408}) 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=23367672}) 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3498133408}) 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=304590890}) 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=39,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=25} crc {logm=3769211269,mdsmap=1531151339,mgr=2831785783,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3452503532,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=2089822439}) 2024-09-18T14:10:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=24} crc {logm=376214143,mdsmap=1531151339,mgr=1359932698,mgr_metadata=270970526,mgrstat=2230679755,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=483569634}) 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=39,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=25} crc {logm=3769211269,mdsmap=1531151339,mgr=2831785783,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3452503532,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=2089822439}) 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=42,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=22} crc {logm=929146846,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2230679755,mon_config_key=3155120151,monmap=455718034,osd_metadata=774608592,osd_pg_creating=3758990646,osd_snap=573443477}) 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=39,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=25} crc {logm=3769211269,mdsmap=1531151339,mgr=2831785783,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3452503532,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=2089822439}) 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=41,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=22} crc {logm=3510955302,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2230679755,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=3999525990}) 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=39,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=25} crc {logm=3769211269,mdsmap=1531151339,mgr=2831785783,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3452503532,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=2089822439}) 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=42,mdsmap=2,mgr=6,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=26} crc {logm=3573339358,mdsmap=3140514009,mgr=1582877503,mgr_metadata=270970526,mgrstat=2230679755,mon_config_key=1810658514,monmap=2358078507,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=3889655484}) 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=39,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=25} crc {logm=3769211269,mdsmap=1531151339,mgr=2831785783,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3452503532,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=2089822439}) 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=39,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=26} crc {logm=3139942725,mdsmap=1531151339,mgr=3987142098,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2230679755,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=26419106}) 2024-09-18T14:10:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=39,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=25} crc {logm=3769211269,mdsmap=1531151339,mgr=2831785783,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3452503532,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=2089822439}) 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=39,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27} crc {logm=970974487,mdsmap=3585761405,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=821953492,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=30924456}) 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=39,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=25} crc {logm=3769211269,mdsmap=1531151339,mgr=2831785783,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3452503532,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=2089822439}) 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=39,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=1,osd_metadata=5,osd_pg_creating=1,osd_snap=28} crc {logm=4188883728,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2230679755,mon_config_key=1045651642,monmap=1228700967,osd_metadata=3702629295,osd_pg_creating=3758990646,osd_snap=2489366209}) 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=39,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=25} crc {logm=3769211269,mdsmap=1531151339,mgr=2831785783,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3452503532,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=2089822439}) 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.077+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=39,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=26} crc {logm=1606852114,mdsmap=3585761405,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1606366372,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3758990646,osd_snap=189297268}) 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=50,osdmap=50} crc {osd_snap=1646590110,osdmap=3924167137}) 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=54,osdmap=46} crc {osd_snap=556618186,osdmap=1879267682}) 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=50,osdmap=50} crc {osd_snap=1646590110,osdmap=3924167137}) 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=51,osdmap=49} crc {osd_snap=309209153,osdmap=3103849619}) 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=50,osdmap=50} crc {osd_snap=1646590110,osdmap=3924167137}) 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=54,osdmap=46} crc {osd_snap=90634574,osdmap=1129683124}) 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=50,osdmap=50} crc {osd_snap=1646590110,osdmap=3924167137}) 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=53,osdmap=47} crc {osd_snap=3586998515,osdmap=967960759}) 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=50,osdmap=50} crc {osd_snap=1646590110,osdmap=3924167137}) 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=55,osdmap=45} crc {osd_snap=888268192,osdmap=3910834648}) 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=50,osdmap=50} crc {osd_snap=1646590110,osdmap=3924167137}) 2024-09-18T14:10:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=51,osdmap=49} crc {osd_snap=1780976443,osdmap=2111248615}) 2024-09-18T14:10:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=50,osdmap=50} crc {osd_snap=1646590110,osdmap=3924167137}) 2024-09-18T14:10:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=53,osdmap=47} crc {osd_snap=1622661295,osdmap=425599823}) 2024-09-18T14:10:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=50,osdmap=50} crc {osd_snap=1646590110,osdmap=3924167137}) 2024-09-18T14:10:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=52,osdmap=48} crc {osd_snap=4270514076,osdmap=254072778}) 2024-09-18T14:10:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2016207285}) 2024-09-18T14:10:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1434389255}) 2024-09-18T14:10:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2016207285}) 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2891353980}) 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2016207285}) 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=40701346}) 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2016207285}) 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=994915837}) 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2016207285}) 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=778976361}) 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2016207285}) 2024-09-18T14:10:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3809534973}) 2024-09-18T14:10:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2016207285}) 2024-09-18T14:10:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.080+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2609991935}) 2024-09-18T14:10:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.080+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.080+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2016207285}) 2024-09-18T14:10:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.080+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1588250613}) 2024-09-18T14:10:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.081+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.081+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4211938746}) 2024-09-18T14:10:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.081+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3925196206}) 2024-09-18T14:10:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.081+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.081+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4211938746}) 2024-09-18T14:10:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.081+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3745382576}) 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.081+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.081+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4211938746}) 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.081+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3759809221}) 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.081+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.081+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4211938746}) 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=609688322}) 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4211938746}) 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2103912902}) 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4211938746}) 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2187363175}) 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4211938746}) 2024-09-18T14:10:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2521049786}) 2024-09-18T14:10:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4211938746}) 2024-09-18T14:10:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.082+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2614852120}) 2024-09-18T14:10:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=31} crc {osdmap=2008288217}) 2024-09-18T14:10:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=33} crc {osdmap=3388858022}) 2024-09-18T14:10:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=31} crc {osdmap=2008288217}) 2024-09-18T14:10:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=32} crc {osdmap=911276258}) 2024-09-18T14:10:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=31} crc {osdmap=2008288217}) 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=31} crc {osdmap=1266802104}) 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=31} crc {osdmap=2008288217}) 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=33} crc {osdmap=1213374724}) 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=31} crc {osdmap=2008288217}) 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=33} crc {osdmap=1346180721}) 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=31} crc {osdmap=2008288217}) 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.083+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=29} crc {osdmap=2269286845}) 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.084+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.084+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=31} crc {osdmap=2008288217}) 2024-09-18T14:10:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.084+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=33} crc {osdmap=196964983}) 2024-09-18T14:10:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.084+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:10:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.084+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=31} crc {osdmap=2008288217}) 2024-09-18T14:10:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:10:04.084+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=32} crc {osdmap=3382462806}) 2024-09-18T14:10:04.281 INFO:tasks.rados.rados.0.smithi084.stdout:2421: done (2 left) 2024-09-18T14:10:04.281 INFO:tasks.rados.rados.0.smithi084.stdout:2422: done (1 left) 2024-09-18T14:10:04.281 INFO:tasks.rados.rados.0.smithi084.stdout:2423: done (0 left) 2024-09-18T14:10:04.281 INFO:tasks.rados.rados.0.smithi084.stdout:2424: delete oid 36 current snap is 255 2024-09-18T14:10:04.284 INFO:tasks.rados.rados.0.smithi084.stdout:2424: done (0 left) 2024-09-18T14:10:04.284 INFO:tasks.rados.rados.0.smithi084.stdout:2425: write (excl) oid 19 current snap is 255 2024-09-18T14:10:04.285 INFO:tasks.rados.rados.0.smithi084.stdout:2425: seq_num 851 ranges {702757=552082,1681941=430628} 2024-09-18T14:10:04.290 INFO:tasks.rados.rados.0.smithi084.stdout:2425: writing smithi08434663-19 from 702757 to 1254839 tid 1 2024-09-18T14:10:04.295 INFO:tasks.rados.rados.0.smithi084.stdout:2425: writing smithi08434663-19 from 1681941 to 2112569 tid 2 2024-09-18T14:10:04.295 INFO:tasks.rados.rados.0.smithi084.stdout:2426: read oid 27 snap -1 2024-09-18T14:10:04.295 INFO:tasks.rados.rados.0.smithi084.stdout:2426: expect (ObjNum 723 snap 209 seq_num 723) 2024-09-18T14:10:04.295 INFO:tasks.rados.rados.0.smithi084.stdout:2427: write (excl) oid 16 current snap is 255 2024-09-18T14:10:04.296 INFO:tasks.rados.rados.0.smithi084.stdout:2425: finishing write tid 1 to smithi08434663-19 2024-09-18T14:10:04.296 INFO:tasks.rados.rados.0.smithi084.stdout:2427: seq_num 852 ranges {719564=743684,2140423=179141} 2024-09-18T14:10:04.303 INFO:tasks.rados.rados.0.smithi084.stdout:2427: writing smithi08434663-16 from 719564 to 1463248 tid 1 2024-09-18T14:10:04.308 INFO:tasks.rados.rados.0.smithi084.stdout:2427: writing smithi08434663-16 from 2140423 to 2319564 tid 2 2024-09-18T14:10:04.308 INFO:tasks.rados.rados.0.smithi084.stdout:2428: snap_create 2024-09-18T14:10:04.308 INFO:tasks.rados.rados.0.smithi084.stdout:2426: expect (ObjNum 723 snap 209 seq_num 723) 2024-09-18T14:10:04.397 INFO:tasks.rados.rados.0.smithi084.stdout:2425: finishing write tid 2 to smithi08434663-19 2024-09-18T14:10:04.397 INFO:tasks.rados.rados.0.smithi084.stdout:2425: finishing write tid 3 to smithi08434663-19 2024-09-18T14:10:04.397 INFO:tasks.rados.rados.0.smithi084.stdout:2425: finishing write tid 4 to smithi08434663-19 2024-09-18T14:10:04.397 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1193 (ObjNum 851 snap 255 seq_num 851) dirty exists 2024-09-18T14:10:04.398 INFO:tasks.rados.rados.0.smithi084.stdout:2425: left oid 19 (ObjNum 851 snap 255 seq_num 851) 2024-09-18T14:10:04.398 INFO:tasks.rados.rados.0.smithi084.stdout:2427: finishing write tid 1 to smithi08434663-16 2024-09-18T14:10:04.398 INFO:tasks.rados.rados.0.smithi084.stdout:2427: finishing write tid 2 to smithi08434663-16 2024-09-18T14:10:04.398 INFO:tasks.rados.rados.0.smithi084.stdout:2427: finishing write tid 3 to smithi08434663-16 2024-09-18T14:10:04.398 INFO:tasks.rados.rados.0.smithi084.stdout:2427: finishing write tid 4 to smithi08434663-16 2024-09-18T14:10:04.398 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1052 (ObjNum 852 snap 255 seq_num 852) dirty exists 2024-09-18T14:10:04.398 INFO:tasks.rados.rados.0.smithi084.stdout:2427: left oid 16 (ObjNum 852 snap 255 seq_num 852) 2024-09-18T14:10:05.074 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:10:05.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:10:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.171+0000 7f71c831d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.171+0000 7f71c831d640 10 monclient: build_initial_monmap 2024-09-18T14:10:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.171+0000 7f71c831d640 10 monclient: monmap: 2024-09-18T14:10:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:05.173836+0000 2024-09-18T14:10:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:05.173836+0000 2024-09-18T14:10:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:05.175 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.172+0000 7f71c831d640 1 Processor -- start 2024-09-18T14:10:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.172+0000 7f71c831d640 1 -- start start 2024-09-18T14:10:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.172+0000 7f71c831d640 10 monclient: init 2024-09-18T14:10:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c831d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c831d640 10 monclient: _add_conns ranks=[0,2,6,4,3,8,1,7,5] 2024-09-18T14:10:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c831d640 10 monclient(hunting): picked mon.noname-a con 0x7f71c009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c831d640 10 monclient(hunting): picked mon.noname-b con 0x7f71c009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c831d640 10 monclient(hunting): picked mon.noname-d con 0x7f71c009ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c831d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c005ed70 con 0x7f71c009f7d0 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c831d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c009dfc0 con 0x7f71c009fba0 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c831d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c009dce0 con 0x7f71c009ff70 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c831d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c831d640 10 monclient(hunting): authenticate will time out at 1420.512226s 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c60b9640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f71c009f7d0 0x7f71c0132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51118/0 (socket says 172.21.15.45:51118) 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c58b8640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f71c009fba0 0x7f71c0137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33924/0 (socket says 172.21.15.45:33924) 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c60b9640 1 -- 172.21.15.45:0/1408791447 learned_addr learned my addr 172.21.15.45:0/1408791447 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994525101 0 0) 0x7f71c005ed70 con 0x7f71c009f7d0 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient(hunting): my global_id is 5634 2024-09-18T14:10:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71ac0034e0 con 0x7f71c009f7d0 2024-09-18T14:10:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4318895 0 0) 0x7f71c009dce0 con 0x7f71c009ff70 2024-09-18T14:10:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient(hunting): my global_id is 184200 2024-09-18T14:10:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71c005ed70 con 0x7f71c009ff70 2024-09-18T14:10:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780406786 0 0) 0x7f71c009dfc0 con 0x7f71c009fba0 2024-09-18T14:10:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient(hunting): my global_id is 144884 2024-09-18T14:10:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71c009dce0 con 0x7f71c009fba0 2024-09-18T14:10:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1430600642 0 0) 0x7f71ac0034e0 con 0x7f71c009f7d0 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 >> v1:172.21.15.45:6792/0 conn(0x7f71c009ff70 legacy=0x7f71c013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 >> v1:172.21.15.45:6790/0 conn(0x7f71c009fba0 legacy=0x7f71c0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient: _finish_hunting 0 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 1 monclient: found mon.noname-a 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c00a1c00 con 0x7f71c009f7d0 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient: _finish_auth 0 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f71b0001830 con 0x7f71c009f7d0 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.173+0000 7f71c50b7640 10 monclient: dump: 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.187 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c831d640 5 monclient: authenticate success, global_id 5634 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c831d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f71b0001a10 con 0x7f71c009f7d0 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c50b7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c50b7640 1 -- 172.21.15.45:0/1408791447 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f71b0004ba0 con 0x7f71c009f7d0 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c50b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c50b7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c50b7640 10 monclient: dump: 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c831d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c831d640 4 set_mon_vals no callback set 2024-09-18T14:10:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c831d640 10 monclient: shutdown 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c831d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c831d640 1 -- 172.21.15.45:0/1408791447 >> v1:172.21.15.45:6789/0 conn(0x7f71c009f7d0 legacy=0x7f71c0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c831d640 1 -- 172.21.15.45:0/1408791447 shutdown_connections 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c831d640 1 -- 172.21.15.45:0/1408791447 >> 172.21.15.45:0/1408791447 conn(0x7f71c00630b0 msgr2=0x7f71c00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c831d640 1 -- 172.21.15.45:0/1408791447 shutdown_connections 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.174+0000 7f71c831d640 1 -- 172.21.15.45:0/1408791447 wait complete. 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.175+0000 7f71c831d640 10 monclient: build_initial_monmap 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.175+0000 7f71c831d640 10 monclient: monmap: 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.175+0000 7f71c831d640 1 Processor -- start 2024-09-18T14:10:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.175+0000 7f71c831d640 1 -- start start 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.175+0000 7f71c831d640 10 monclient: init 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71c831d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71c831d640 10 monclient: _add_conns ranks=[7,0,6,8,5,3,4,2,1] 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71c831d640 10 monclient(hunting): picked mon.noname-i con 0x7f71c009f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71c831d640 10 monclient(hunting): picked mon.noname-a con 0x7f71c009fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71c831d640 10 monclient(hunting): picked mon.noname-d con 0x7f71c009ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71c831d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c009d310 con 0x7f71c009fba0 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71c831d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c00a2560 con 0x7f71c009ff70 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71c831d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c009b7a0 con 0x7f71c009f7d0 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71c831d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71c831d640 10 monclient(hunting): authenticate will time out at 1420.515172s 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71c58b8640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f71c009fba0 0x7f71c0146cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51124/0 (socket says 172.21.15.45:51124) 2024-09-18T14:10:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71c58b8640 1 -- 172.21.15.45:0/988780401 learned_addr learned my addr 172.21.15.45:0/988780401 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387540940 0 0) 0x7f71c009d310 con 0x7f71c009fba0 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): my global_id is 5643 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71a40034e0 con 0x7f71c009fba0 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912181840 0 0) 0x7f71c00a2560 con 0x7f71c009ff70 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): my global_id is 184209 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71c009d310 con 0x7f71c009ff70 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166050785 0 0) 0x7f71c009b7a0 con 0x7f71c009f7d0 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): my global_id is 184201 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71c00a2560 con 0x7f71c009f7d0 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3906987199 0 0) 0x7f71a40034e0 con 0x7f71c009fba0 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f71c009b7a0 con 0x7f71c009fba0 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f71bc001b90 con 0x7f71c009fba0 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): dump: 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2430247622 0 0) 0x7f71c009d310 con 0x7f71c009ff70 2024-09-18T14:10:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f71a40034e0 con 0x7f71c009ff70 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2731233895 0 0) 0x7f71c00a2560 con 0x7f71c009f7d0 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f71c009d310 con 0x7f71c009f7d0 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f71b8001b70 con 0x7f71c009ff70 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): dump: 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f71b0001770 con 0x7f71c009f7d0 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient(hunting): dump: 2024-09-18T14:10:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.198 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 767970373 0 0) 0x7f71c009b7a0 con 0x7f71c009fba0 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 >> v1:172.21.15.84:6792/0 conn(0x7f71c009f7d0 legacy=0x7f71c0131640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 >> v1:172.21.15.45:6792/0 conn(0x7f71c009ff70 legacy=0x7f71c014a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient: _finish_hunting 0 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 monclient: found mon.a 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c011e760 con 0x7f71c009fba0 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 10 monclient: _finish_auth 0 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.176+0000 7f71b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71c831d640 5 monclient: authenticate success, global_id 5643 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71c831d640 10 monclient: _renew_subs 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71c831d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71c831d640 1 -- 172.21.15.45:0/988780401 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c011ea40 con 0x7f71c009fba0 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71c831d640 10 monclient: _renew_subs 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71c831d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71c831d640 1 -- 172.21.15.45:0/988780401 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c0136ee0 con 0x7f71c009fba0 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f71bc004b50 con 0x7f71c009fba0 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71b67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71c48b6640 4 set_mon_vals no callback set 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f71bc005080 con 0x7f71c009fba0 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71b67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71b67fc640 10 monclient: dump: 2024-09-18T14:10:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.202 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f71bc0051e0 con 0x7f71c009fba0 2024-09-18T14:10:05.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71c831d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:05.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71c831d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:05.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71c831d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:05.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.177+0000 7f71c831d640 1 -- 172.21.15.45:0/988780401 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a8005010 con 0x7f71c009fba0 2024-09-18T14:10:05.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.178+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(529..529 src has 344..529) v4 ==== 3987+0+0 (unknown 3483284021 0 0) 0x7f71bc046c10 con 0x7f71c009fba0 2024-09-18T14:10:05.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.178+0000 7f71c60b9640 10 monclient: get_auth_request con 0x7f71a403aa20 auth_method 0 2024-09-18T14:10:05.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.182+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f71bc0153a0 con 0x7f71c009fba0 2024-09-18T14:10:05.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.182+0000 7f71b67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:05.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.182+0000 7f71b67fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:05.285 INFO:tasks.rados.rados.0.smithi084.stdout:2425: done (3 left) 2024-09-18T14:10:05.285 INFO:tasks.rados.rados.0.smithi084.stdout:2426: done (2 left) 2024-09-18T14:10:05.285 INFO:tasks.rados.rados.0.smithi084.stdout:2427: done (1 left) 2024-09-18T14:10:05.285 INFO:tasks.rados.rados.0.smithi084.stdout:2428: done (0 left) 2024-09-18T14:10:05.285 INFO:tasks.rados.rados.0.smithi084.stdout:2429: rollback oid 40 current snap is 256 2024-09-18T14:10:05.285 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 40 to 255 2024-09-18T14:10:05.285 INFO:tasks.rados.rados.0.smithi084.stdout:2430: snap_remove snap 224 2024-09-18T14:10:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.412+0000 7f71c831d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:10:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.412+0000 7f71c831d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:10:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.412+0000 7f71c831d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.412+0000 7f71c831d640 1 -- 172.21.15.45:0/988780401 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f71a8005840 con 0x7f71c009fba0 2024-09-18T14:10:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.413+0000 7f71b67fc640 1 -- 172.21.15.45:0/988780401 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 3780750383) 0x7f71bc014da0 con 0x7f71c009fba0 2024-09-18T14:10:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.413+0000 7f71b67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:10:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.413+0000 7f71b67fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:05.415 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":92,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:10:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.415+0000 7f71c831d640 1 -- 172.21.15.45:0/988780401 >> v1:172.21.15.84:6812/31478 conn(0x7f71a403aa20 legacy=0x7f71a403ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.415+0000 7f71c831d640 10 monclient: shutdown 2024-09-18T14:10:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.415+0000 7f71c831d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.415+0000 7f71c831d640 1 -- 172.21.15.45:0/988780401 >> v1:172.21.15.45:6789/0 conn(0x7f71c009fba0 legacy=0x7f71c0146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.415+0000 7f71c831d640 1 -- 172.21.15.45:0/988780401 shutdown_connections 2024-09-18T14:10:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.415+0000 7f71c831d640 1 -- 172.21.15.45:0/988780401 >> 172.21.15.45:0/988780401 conn(0x7f71c00630b0 msgr2=0x7f71c00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.415+0000 7f71c831d640 1 -- 172.21.15.45:0/988780401 shutdown_connections 2024-09-18T14:10:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.415+0000 7f71c831d640 1 -- 172.21.15.45:0/988780401 wait complete. 2024-09-18T14:10:05.426 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:10:05.426 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-18T14:10:05.426 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:10:05.427 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:10:05.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.522+0000 7f838919d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:05.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.522+0000 7f838919d640 10 monclient: build_initial_monmap 2024-09-18T14:10:05.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.522+0000 7f838919d640 10 monclient: monmap: 2024-09-18T14:10:05.524 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:05.524 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.524 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:05.524197+0000 2024-09-18T14:10:05.524 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:05.524197+0000 2024-09-18T14:10:05.524 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:05.524 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.524 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:05.524 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:05.524 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:05.525 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:05.525 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:05.525 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:05.525 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:05.525 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:05.525 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:05.525 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.522+0000 7f838919d640 1 Processor -- start 2024-09-18T14:10:05.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.522+0000 7f838919d640 1 -- start start 2024-09-18T14:10:05.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.522+0000 7f838919d640 10 monclient: init 2024-09-18T14:10:05.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f838919d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:05.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f838919d640 10 monclient: _add_conns ranks=[2,3,8,5,6,0,4,7,1] 2024-09-18T14:10:05.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f838919d640 10 monclient(hunting): picked mon.noname-b con 0x7f838406e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:05.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f838919d640 10 monclient(hunting): picked mon.noname-g con 0x7f83840a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f838919d640 10 monclient(hunting): picked mon.noname-e con 0x7f83840a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f838919d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838405ed70 con 0x7f838406e810 2024-09-18T14:10:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f838919d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8384097620 con 0x7f83840a0cb0 2024-09-18T14:10:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f838919d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8384097340 con 0x7f83840a1080 2024-09-18T14:10:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f838919d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f838919d640 10 monclient(hunting): authenticate will time out at 1420.862389s 2024-09-18T14:10:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8382d9d640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f838406e810 0x7f83840a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33940/0 (socket says 172.21.15.45:33940) 2024-09-18T14:10:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f838359e640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f83840a1080 0x7f838413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:44908/0 (socket says 172.21.15.45:44908) 2024-09-18T14:10:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8382d9d640 1 -- 172.21.15.45:0/3659554303 learned_addr learned my addr 172.21.15.45:0/3659554303 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315830768 0 0) 0x7f838405ed70 con 0x7f838406e810 2024-09-18T14:10:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 10 monclient(hunting): my global_id is 144893 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f836c0034e0 con 0x7f838406e810 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737254983 0 0) 0x7f8384097340 con 0x7f83840a1080 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 10 monclient(hunting): my global_id is 5516 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f838405ed70 con 0x7f83840a1080 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038899723 0 0) 0x7f8384097620 con 0x7f83840a0cb0 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 10 monclient(hunting): my global_id is 184179 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.523+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8384097340 con 0x7f83840a0cb0 2024-09-18T14:10:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2228634072 0 0) 0x7f838405ed70 con 0x7f83840a1080 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 >> v1:172.21.15.84:6790/0 conn(0x7f83840a0cb0 legacy=0x7f8384137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 >> v1:172.21.15.45:6790/0 conn(0x7f838406e810 legacy=0x7f83840a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 1 monclient: found mon.noname-e 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83840a1660 con 0x7f83840a1080 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 10 monclient: _finish_auth 0 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f837c001b10 con 0x7f83840a1080 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f838919d640 5 monclient: authenticate success, global_id 5516 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f838919d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 10 monclient: dump: 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f838919d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f837c001cf0 con 0x7f83840a1080 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 1 -- 172.21.15.45:0/3659554303 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f837c004b20 con 0x7f83840a1080 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f838919d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f838919d640 4 set_mon_vals no callback set 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f8381d9b640 10 monclient: dump: 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.541 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f838919d640 10 monclient: shutdown 2024-09-18T14:10:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f838919d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f838919d640 1 -- 172.21.15.45:0/3659554303 >> v1:172.21.15.45:6793/0 conn(0x7f83840a1080 legacy=0x7f838413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f838919d640 1 -- 172.21.15.45:0/3659554303 shutdown_connections 2024-09-18T14:10:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f838919d640 1 -- 172.21.15.45:0/3659554303 >> 172.21.15.45:0/3659554303 conn(0x7f83840a5bb0 msgr2=0x7f83840a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f838919d640 1 -- 172.21.15.45:0/3659554303 shutdown_connections 2024-09-18T14:10:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.524+0000 7f838919d640 1 -- 172.21.15.45:0/3659554303 wait complete. 2024-09-18T14:10:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.525+0000 7f838919d640 10 monclient: build_initial_monmap 2024-09-18T14:10:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.525+0000 7f838919d640 10 monclient: monmap: 2024-09-18T14:10:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.525+0000 7f838919d640 1 Processor -- start 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.525+0000 7f838919d640 1 -- start start 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.525+0000 7f838919d640 10 monclient: init 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f838919d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f838919d640 10 monclient: _add_conns ranks=[7,6,0,8,4,1,3,5,2] 2024-09-18T14:10:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f838919d640 10 monclient(hunting): picked mon.noname-i con 0x7f838406e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f838919d640 10 monclient(hunting): picked mon.noname-d con 0x7f83840a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f838919d640 10 monclient(hunting): picked mon.noname-a con 0x7f83840a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f838919d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8384122d20 con 0x7f83840a1080 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f838919d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838413db10 con 0x7f83840a0cb0 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f838919d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8384122760 con 0x7f838406e810 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f838919d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f838919d640 10 monclient(hunting): authenticate will time out at 1420.865407s 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f838259c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f83840a0cb0 0x7f8384146630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34848/0 (socket says 172.21.15.45:34848) 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f838259c640 1 -- 172.21.15.45:0/2450861639 learned_addr learned my addr 172.21.15.45:0/2450861639 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043253099 0 0) 0x7f8384122d20 con 0x7f83840a1080 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 10 monclient(hunting): my global_id is 5652 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83700034e0 con 0x7f83840a1080 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450572984 0 0) 0x7f838413db10 con 0x7f83840a0cb0 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 10 monclient(hunting): my global_id is 184218 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8384122d20 con 0x7f83840a0cb0 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550219317 0 0) 0x7f8384122760 con 0x7f838406e810 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 10 monclient(hunting): my global_id is 184210 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f838413db10 con 0x7f838406e810 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.526+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 509393189 0 0) 0x7f8384122d20 con 0x7f83840a0cb0 2024-09-18T14:10:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8384122760 con 0x7f83840a0cb0 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8378001ba0 con 0x7f83840a0cb0 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient(hunting): dump: 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1075830776 0 0) 0x7f838413db10 con 0x7f838406e810 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8384122d20 con 0x7f838406e810 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1268046690 0 0) 0x7f83700034e0 con 0x7f83840a1080 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f838413db10 con 0x7f83840a1080 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f83740014e0 con 0x7f838406e810 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:05.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient(hunting): dump: 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.549 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f837c001ad0 con 0x7f83840a1080 2024-09-18T14:10:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient(hunting): dump: 2024-09-18T14:10:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1012984833 0 0) 0x7f8384122760 con 0x7f83840a0cb0 2024-09-18T14:10:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 >> v1:172.21.15.84:6792/0 conn(0x7f838406e810 legacy=0x7f838413d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 >> v1:172.21.15.45:6789/0 conn(0x7f83840a1080 legacy=0x7f8384149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient: _finish_hunting 0 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 monclient: found mon.d 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8384120270 con 0x7f83840a0cb0 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient: _finish_auth 0 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f838919d640 5 monclient: authenticate success, global_id 184218 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f838919d640 10 monclient: _renew_subs 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f838919d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f838919d640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8384122480 con 0x7f83840a0cb0 2024-09-18T14:10:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f838919d640 10 monclient: _renew_subs 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f838919d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f838919d640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f83841215b0 con 0x7f83840a0cb0 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8378002130 con 0x7f83840a0cb0 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f838159a640 4 set_mon_vals no callback set 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f83780025e0 con 0x7f83840a0cb0 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.527+0000 7f836affd640 10 monclient: dump: 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f838919d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f8378002740 con 0x7f83840a0cb0 2024-09-18T14:10:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f838919d640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8364006580 con 0x7f8364002830 2024-09-18T14:10:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(530..530 src has 344..530) v4 ==== 3959+0+0 (unknown 688093774 0 0) 0x7f8378046ae0 con 0x7f83840a0cb0 2024-09-18T14:10:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125729609 0 0) 0x7f8364006580 con 0x7f8364002830 2024-09-18T14:10:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 10 monclient(hunting): my global_id is 5661 2024-09-18T14:10:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8384122760 con 0x7f8364002830 2024-09-18T14:10:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1820328074 0 0) 0x7f8384122760 con 0x7f8364002830 2024-09-18T14:10:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8364006580 con 0x7f8364002830 2024-09-18T14:10:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8374001d90 con 0x7f8364002830 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f838359e640 10 monclient: get_auth_request con 0x7f837003ab00 auth_method 0 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 10 monclient: dump: 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.556 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.556 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.556 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.556 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.556 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2018133624 0 0) 0x7f8364006580 con 0x7f8364002830 2024-09-18T14:10:05.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.528+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f836400a0a0 con 0x7f8364002830 2024-09-18T14:10:05.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.529+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8374004ed0 con 0x7f8364002830 2024-09-18T14:10:05.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.529+0000 7f836affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:05.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.529+0000 7f836affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:05.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.529+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 >> v1:172.21.15.45:6789/0 conn(0x7f8364002830 legacy=0x7f8364004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.529+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 >> v1:172.21.15.45:6789/0 conn(0x7f8364002830 legacy=0x7f8364004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.534+0000 7f838919d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.534+0000 7f838919d640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f83640074f0 con 0x7f83640068a0 2024-09-18T14:10:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958593313 0 0) 0x7f83640074f0 con 0x7f83640068a0 2024-09-18T14:10:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 10 monclient(hunting): my global_id is 5670 2024-09-18T14:10:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8364006580 con 0x7f83640068a0 2024-09-18T14:10:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 71510228 0 0) 0x7f8364006580 con 0x7f83640068a0 2024-09-18T14:10:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f83640074f0 con 0x7f83640068a0 2024-09-18T14:10:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f83740056b0 con 0x7f83640068a0 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 10 monclient: dump: 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.560 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 526361908 0 0) 0x7f83640074f0 con 0x7f83640068a0 2024-09-18T14:10:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8364008750 con 0x7f83640068a0 2024-09-18T14:10:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 738188886) 0x7f8374005810 con 0x7f83640068a0 2024-09-18T14:10:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 >> v1:172.21.15.45:6789/0 conn(0x7f83640068a0 legacy=0x7f8364006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.535+0000 7f836affd640 1 -- 172.21.15.45:0/2450861639 >> v1:172.21.15.45:6789/0 conn(0x7f83640068a0 legacy=0x7f8364006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:05.561 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:05.561 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:05.562 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:05.563 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:05.564 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:05.565 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.566 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:05.568 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.569 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.569 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.569 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:05.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:05.569 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.569 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.569 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:05.569 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:05.570 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.571 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.572 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:05.572 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:05.572 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.572 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.572 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.572 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:05.572 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.572 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.573 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:05.575 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:05.576 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.576 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.576 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.576 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:05.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:05.576 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.576 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.576 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:05.577 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.578 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.578 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:05.578 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.578 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.578 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.578 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:05.578 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:05.578 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.578 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:10:05.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.581 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:05.581 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.536+0000 7f838919d640 1 -- 172.21.15.45:0/2450861639 >> v1:172.21.15.84:6812/31478 conn(0x7f837003ab00 legacy=0x7f837003cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.536+0000 7f838919d640 10 monclient: shutdown 2024-09-18T14:10:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.536+0000 7f838919d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.536+0000 7f838919d640 1 -- 172.21.15.45:0/2450861639 >> v1:172.21.15.45:6792/0 conn(0x7f83840a0cb0 legacy=0x7f8384146630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.536+0000 7f838359e640 1 -- 172.21.15.45:0/2450861639 reap_dead start 2024-09-18T14:10:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.536+0000 7f838359e640 1 -- 172.21.15.45:0/2450861639 reap_dead start 2024-09-18T14:10:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.536+0000 7f838919d640 1 -- 172.21.15.45:0/2450861639 shutdown_connections 2024-09-18T14:10:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.536+0000 7f838919d640 1 -- 172.21.15.45:0/2450861639 >> 172.21.15.45:0/2450861639 conn(0x7f83840a5bb0 msgr2=0x7f8384096a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.536+0000 7f838919d640 1 -- 172.21.15.45:0/2450861639 shutdown_connections 2024-09-18T14:10:05.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.536+0000 7f838919d640 1 -- 172.21.15.45:0/2450861639 wait complete. 2024-09-18T14:10:05.583 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:05.583 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:10:05.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:10:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.679+0000 7f125586f640 10 monclient: get_monmap_and_config 2024-09-18T14:10:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.679+0000 7f125586f640 10 monclient: build_initial_monmap 2024-09-18T14:10:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.679+0000 7f125586f640 10 monclient: monmap: 2024-09-18T14:10:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:05.681288+0000 2024-09-18T14:10:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:05.681288+0000 2024-09-18T14:10:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.679+0000 7f125586f640 1 Processor -- start 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.679+0000 7f125586f640 1 -- start start 2024-09-18T14:10:05.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.679+0000 7f125586f640 10 monclient: init 2024-09-18T14:10:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f125586f640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f125586f640 10 monclient: _add_conns ranks=[0,3,7,1,6,4,5,2,8] 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f125586f640 10 monclient(hunting): picked mon.noname-a con 0x7f125009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f125586f640 10 monclient(hunting): picked mon.noname-g con 0x7f125009fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f125586f640 10 monclient(hunting): picked mon.noname-i con 0x7f125009ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f125586f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125005ed70 con 0x7f125009f7d0 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f125586f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125009dfc0 con 0x7f125009fba0 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f125586f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125009dce0 con 0x7f125009ff70 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f125586f640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f125586f640 10 monclient(hunting): authenticate will time out at 1421.019498s 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f124effd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f125009f7d0 0x7f1250132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51162/0 (socket says 172.21.15.45:51162) 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f124effd640 1 -- 172.21.15.45:0/3054200567 learned_addr learned my addr 172.21.15.45:0/3054200567 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f124dffb640 1 -- 172.21.15.45:0/3054200567 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097038828 0 0) 0x7f125005ed70 con 0x7f125009f7d0 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f124dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f124dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f124dffb640 10 monclient(hunting): my global_id is 5679 2024-09-18T14:10:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.680+0000 7f124dffb640 1 -- 172.21.15.45:0/3054200567 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12380034e0 con 0x7f125009f7d0 2024-09-18T14:10:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 1 -- 172.21.15.45:0/3054200567 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969036417 0 0) 0x7f125009dfc0 con 0x7f125009fba0 2024-09-18T14:10:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient(hunting): my global_id is 184188 2024-09-18T14:10:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 1 -- 172.21.15.45:0/3054200567 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f125005ed70 con 0x7f125009fba0 2024-09-18T14:10:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 1 -- 172.21.15.45:0/3054200567 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 515454746 0 0) 0x7f12380034e0 con 0x7f125009f7d0 2024-09-18T14:10:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 1 -- 172.21.15.45:0/3054200567 >> v1:172.21.15.84:6792/0 conn(0x7f125009ff70 legacy=0x7f125013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 1 -- 172.21.15.45:0/3054200567 >> v1:172.21.15.84:6790/0 conn(0x7f125009fba0 legacy=0x7f1250137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient: _finish_hunting 0 2024-09-18T14:10:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 1 monclient: found mon.noname-a 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 1 -- 172.21.15.45:0/3054200567 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12500a1c00 con 0x7f125009f7d0 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient: _finish_auth 0 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 1 -- 172.21.15.45:0/3054200567 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1240001830 con 0x7f125009f7d0 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient: dump: 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f125586f640 5 monclient: authenticate success, global_id 5679 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f125586f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 1 -- 172.21.15.45:0/3054200567 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1240001a10 con 0x7f125009f7d0 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 1 -- 172.21.15.45:0/3054200567 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1240004ba0 con 0x7f125009f7d0 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f124dffb640 10 monclient: dump: 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f125586f640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f125586f640 4 set_mon_vals no callback set 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f125586f640 10 monclient: shutdown 2024-09-18T14:10:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f125586f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f125586f640 1 -- 172.21.15.45:0/3054200567 >> v1:172.21.15.45:6789/0 conn(0x7f125009f7d0 legacy=0x7f1250132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f125586f640 1 -- 172.21.15.45:0/3054200567 shutdown_connections 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f125586f640 1 -- 172.21.15.45:0/3054200567 >> 172.21.15.45:0/3054200567 conn(0x7f12500630b0 msgr2=0x7f12500634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f125586f640 1 -- 172.21.15.45:0/3054200567 shutdown_connections 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.681+0000 7f125586f640 1 -- 172.21.15.45:0/3054200567 wait complete. 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.682+0000 7f125586f640 10 monclient: build_initial_monmap 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.682+0000 7f125586f640 10 monclient: monmap: 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.682+0000 7f125586f640 1 Processor -- start 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.682+0000 7f125586f640 1 -- start start 2024-09-18T14:10:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.682+0000 7f125586f640 10 monclient: init 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f125586f640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f125586f640 10 monclient: _add_conns ranks=[6,5,4,1,8,3,2,0,7] 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f125586f640 10 monclient(hunting): picked mon.noname-d con 0x7f125009f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f125586f640 10 monclient(hunting): picked mon.noname-h con 0x7f125009fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f125586f640 10 monclient(hunting): picked mon.noname-c con 0x7f125009ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f125586f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125009bbd0 con 0x7f125009ff70 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f125586f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12500a2560 con 0x7f125009fba0 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f125586f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125009d310 con 0x7f125009f7d0 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f125586f640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f125586f640 10 monclient(hunting): authenticate will time out at 1421.022416s 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f124effd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f125009f7d0 0x7f12501315d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34854/0 (socket says 172.21.15.45:34854) 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f124effd640 1 -- 172.21.15.45:0/2549172429 learned_addr learned my addr 172.21.15.45:0/2549172429 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972663870 0 0) 0x7f125009d310 con 0x7f125009f7d0 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 10 monclient(hunting): my global_id is 184227 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f123c003510 con 0x7f125009f7d0 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838505990 0 0) 0x7f125009bbd0 con 0x7f125009ff70 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 10 monclient(hunting): my global_id is 164407 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f125009d310 con 0x7f125009ff70 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140247173 0 0) 0x7f12500a2560 con 0x7f125009fba0 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 10 monclient(hunting): my global_id is 184235 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f125009bbd0 con 0x7f125009fba0 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 850740131 0 0) 0x7f123c003510 con 0x7f125009f7d0 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12500a2560 con 0x7f125009f7d0 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1240004c20 con 0x7f125009f7d0 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.683+0000 7f1236ffd640 10 monclient(hunting): dump: 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1141594837 0 0) 0x7f125009d310 con 0x7f125009ff70 2024-09-18T14:10:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f123c003510 con 0x7f125009ff70 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1248001c20 con 0x7f125009ff70 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient(hunting): dump: 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.704 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 543156233 0 0) 0x7f125009bbd0 con 0x7f125009fba0 2024-09-18T14:10:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f125009d310 con 0x7f125009fba0 2024-09-18T14:10:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1244001bf0 con 0x7f125009fba0 2024-09-18T14:10:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient(hunting): dump: 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.705 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2108377525 0 0) 0x7f12500a2560 con 0x7f125009f7d0 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 >> v1:172.21.15.84:6791/0 conn(0x7f125009fba0 legacy=0x7f1250146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 >> v1:172.21.15.45:6791/0 conn(0x7f125009ff70 legacy=0x7f125014a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 monclient: found mon.d 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125011ef90 con 0x7f125009f7d0 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient: _finish_auth 0 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f125586f640 5 monclient: authenticate success, global_id 184227 2024-09-18T14:10:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f125586f640 10 monclient: _renew_subs 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f125586f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f125586f640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125011e6f0 con 0x7f125009f7d0 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f125586f640 10 monclient: _renew_subs 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f125586f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f125586f640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f125011d790 con 0x7f125009f7d0 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1240001820 con 0x7f125009f7d0 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1240005160 con 0x7f125009f7d0 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f124d7fa640 4 set_mon_vals no callback set 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 10 monclient: dump: 2024-09-18T14:10:05.707 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.684+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1240010130 con 0x7f125009f7d0 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f125586f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f125586f640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1230006580 con 0x7f1230002830 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(530..530 src has 344..530) v4 ==== 3959+0+0 (unknown 688093774 0 0) 0x7f1240046bf0 con 0x7f125009f7d0 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f124f7fe640 10 monclient: get_auth_request con 0x7f123c03ab20 auth_method 0 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905408909 0 0) 0x7f1230006580 con 0x7f1230002830 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f1236ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f1236ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f1236ffd640 10 monclient(hunting): my global_id is 144902 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12500a2560 con 0x7f1230002830 2024-09-18T14:10:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2005621009 0 0) 0x7f12500a2560 con 0x7f1230002830 2024-09-18T14:10:05.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1230006580 con 0x7f1230002830 2024-09-18T14:10:05.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1244002340 con 0x7f1230002830 2024-09-18T14:10:05.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f1236ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f1236ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:05.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.685+0000 7f1236ffd640 10 monclient: dump: 2024-09-18T14:10:05.710 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.710 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.710 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.710 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.710 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.710 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.686+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2183577788 0 0) 0x7f1230006580 con 0x7f1230002830 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.686+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f123000a0a0 con 0x7f1230002830 2024-09-18T14:10:05.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.686+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f12440027e0 con 0x7f1230002830 2024-09-18T14:10:05.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.686+0000 7f1236ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:05.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.686+0000 7f1236ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:05.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.686+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 >> v1:172.21.15.45:6790/0 conn(0x7f1230002830 legacy=0x7f1230004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.686+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 >> v1:172.21.15.45:6790/0 conn(0x7f1230002830 legacy=0x7f1230004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:05.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f125586f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:05.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f125586f640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1230009610 con 0x7f1230008ff0 2024-09-18T14:10:05.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718185453 0 0) 0x7f1230009610 con 0x7f1230008ff0 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 10 monclient(hunting): my global_id is 144911 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1230006580 con 0x7f1230008ff0 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2919500372 0 0) 0x7f1230006580 con 0x7f1230008ff0 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1230009610 con 0x7f1230008ff0 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1244000ed0 con 0x7f1230008ff0 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 10 monclient: dump: 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.714 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3548434174 0 0) 0x7f1230009610 con 0x7f1230008ff0 2024-09-18T14:10:05.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.692+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f12300098d0 con 0x7f1230008ff0 2024-09-18T14:10:05.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 70209526) 0x7f1244003340 con 0x7f1230008ff0 2024-09-18T14:10:05.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f1236ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:05.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f1236ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:05.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 >> v1:172.21.15.45:6790/0 conn(0x7f1230008ff0 legacy=0x7f1230008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f1236ffd640 1 -- 172.21.15.45:0/2549172429 >> v1:172.21.15.45:6790/0 conn(0x7f1230008ff0 legacy=0x7f1230008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:05.716 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:05.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:05.716 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.717 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:05.718 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.719 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:05.720 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:05.720 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:05.720 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:05.720 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:05.720 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:05.720 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:05.720 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:05.720 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:05.720 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:05.720 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:05.720 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:05.720 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:05.721 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:05.722 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.722 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:05.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.727 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:05.728 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.729 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.730 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:05.730 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:05.730 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.730 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.730 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.730 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:05.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:05.730 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.730 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.732 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:05.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:05.732 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.732 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.732 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.732 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:05.732 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.732 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:05.735 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.736 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.737 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.737 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:05.737 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:05.737 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.737 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.737 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.737 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.737 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.737 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:05.737 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.738 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.739 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:05.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.739 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:05.739 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:05.739 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:05.739 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.739 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.739 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:05.739 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:05.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f125586f640 1 -- 172.21.15.45:0/2549172429 >> v1:172.21.15.84:6812/31478 conn(0x7f123c03ab20 legacy=0x7f123c03cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f125586f640 10 monclient: shutdown 2024-09-18T14:10:05.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f125586f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:05.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f125586f640 1 -- 172.21.15.45:0/2549172429 >> v1:172.21.15.45:6792/0 conn(0x7f125009f7d0 legacy=0x7f12501315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f124f7fe640 1 -- 172.21.15.45:0/2549172429 reap_dead start 2024-09-18T14:10:05.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f124f7fe640 1 -- 172.21.15.45:0/2549172429 reap_dead start 2024-09-18T14:10:05.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f125586f640 1 -- 172.21.15.45:0/2549172429 shutdown_connections 2024-09-18T14:10:05.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f125586f640 1 -- 172.21.15.45:0/2549172429 >> 172.21.15.45:0/2549172429 conn(0x7f12500630b0 msgr2=0x7f12500a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:05.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f125586f640 1 -- 172.21.15.45:0/2549172429 shutdown_connections 2024-09-18T14:10:05.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.693+0000 7f125586f640 1 -- 172.21.15.45:0/2549172429 wait complete. 2024-09-18T14:10:05.742 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:05.742 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:10:05.742 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:10:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.837+0000 7efe4b634640 10 monclient: get_monmap_and_config 2024-09-18T14:10:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.837+0000 7efe4b634640 10 monclient: build_initial_monmap 2024-09-18T14:10:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.837+0000 7efe4b634640 10 monclient: monmap: 2024-09-18T14:10:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:05.839891+0000 2024-09-18T14:10:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:05.839891+0000 2024-09-18T14:10:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:05.839 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:05.840 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.838+0000 7efe4b634640 1 Processor -- start 2024-09-18T14:10:05.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.838+0000 7efe4b634640 1 -- start start 2024-09-18T14:10:05.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.838+0000 7efe4b634640 10 monclient: init 2024-09-18T14:10:05.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe4b634640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:05.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe4b634640 10 monclient: _add_conns ranks=[0,2,4,6,5,3,1,7,8] 2024-09-18T14:10:05.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe4b634640 10 monclient(hunting): picked mon.noname-a con 0x7efe4409f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:05.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe4b634640 10 monclient(hunting): picked mon.noname-b con 0x7efe4409fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe4b634640 10 monclient(hunting): picked mon.noname-c con 0x7efe4409ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe4b634640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4405ed70 con 0x7efe4409f7d0 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe4b634640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4409dfc0 con 0x7efe4409fba0 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe4b634640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4409dce0 con 0x7efe4409ff70 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe4b634640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe4b634640 10 monclient(hunting): authenticate will time out at 1421.178316s 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe493d0640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efe4409f7d0 0x7efe44132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51172/0 (socket says 172.21.15.45:51172) 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe493d0640 1 -- 172.21.15.45:0/3630756829 learned_addr learned my addr 172.21.15.45:0/3630756829 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe48bcf640 1 --1- 172.21.15.45:0/3630756829 >> v1:172.21.15.45:6790/0 conn(0x7efe4409fba0 0x7efe44137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33972/0 (socket says 172.21.15.45:33972) 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe49bd1640 1 --1- 172.21.15.45:0/3630756829 >> v1:172.21.15.45:6791/0 conn(0x7efe4409ff70 0x7efe4413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:56794/0 (socket says 172.21.15.45:56794) 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779585754 0 0) 0x7efe4405ed70 con 0x7efe4409f7d0 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 10 monclient(hunting): my global_id is 5688 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe300034e0 con 0x7efe4409f7d0 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525778535 0 0) 0x7efe4409dfc0 con 0x7efe4409fba0 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 10 monclient(hunting): my global_id is 144920 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe4405ed70 con 0x7efe4409fba0 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646866501 0 0) 0x7efe4409dce0 con 0x7efe4409ff70 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 10 monclient(hunting): my global_id is 164416 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe4409dfc0 con 0x7efe4409ff70 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3262603281 0 0) 0x7efe300034e0 con 0x7efe4409f7d0 2024-09-18T14:10:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 >> v1:172.21.15.45:6791/0 conn(0x7efe4409ff70 legacy=0x7efe4413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 >> v1:172.21.15.45:6790/0 conn(0x7efe4409fba0 legacy=0x7efe44137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 1 monclient: found mon.noname-a 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe440a1c00 con 0x7efe4409f7d0 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 10 monclient: _finish_auth 0 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.839+0000 7efe3bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe4b634640 5 monclient: authenticate success, global_id 5688 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe40001760 con 0x7efe4409f7d0 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe4b634640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe3bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe3bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe3bfff640 10 monclient: dump: 2024-09-18T14:10:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.852 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe4b634640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efe400018c0 con 0x7efe4409f7d0 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe3bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe3bfff640 1 -- 172.21.15.45:0/3630756829 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe40004860 con 0x7efe4409f7d0 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe4b634640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe4b634640 4 set_mon_vals no callback set 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe3bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe3bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe3bfff640 10 monclient: dump: 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe4b634640 10 monclient: shutdown 2024-09-18T14:10:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe4b634640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe4b634640 1 -- 172.21.15.45:0/3630756829 >> v1:172.21.15.45:6789/0 conn(0x7efe4409f7d0 legacy=0x7efe44132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe4b634640 1 -- 172.21.15.45:0/3630756829 shutdown_connections 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe4b634640 1 -- 172.21.15.45:0/3630756829 >> 172.21.15.45:0/3630756829 conn(0x7efe440630b0 msgr2=0x7efe440634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe4b634640 1 -- 172.21.15.45:0/3630756829 shutdown_connections 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.840+0000 7efe4b634640 1 -- 172.21.15.45:0/3630756829 wait complete. 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.841+0000 7efe4b634640 10 monclient: build_initial_monmap 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.841+0000 7efe4b634640 10 monclient: monmap: 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:05.855 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.841+0000 7efe4b634640 1 Processor -- start 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.841+0000 7efe4b634640 1 -- start start 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.841+0000 7efe4b634640 10 monclient: init 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe4b634640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe4b634640 10 monclient: _add_conns ranks=[4,1,2,6,5,3,7,8,0] 2024-09-18T14:10:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe4b634640 10 monclient(hunting): picked mon.noname-c con 0x7efe4409f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe4b634640 10 monclient(hunting): picked mon.noname-f con 0x7efe4409fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe4b634640 10 monclient(hunting): picked mon.noname-b con 0x7efe4409ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe4b634640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe44121600 con 0x7efe4409fba0 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe4b634640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe440a2560 con 0x7efe4409ff70 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe4b634640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4409d310 con 0x7efe4409f7d0 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe4b634640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe4b634640 10 monclient(hunting): authenticate will time out at 1421.181287s 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe493d0640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7efe4409f7d0 0x7efe4411d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:56804/0 (socket says 172.21.15.45:56804) 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe493d0640 1 -- 172.21.15.45:0/2789849493 learned_addr learned my addr 172.21.15.45:0/2789849493 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880888492 0 0) 0x7efe440a2560 con 0x7efe4409ff70 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): my global_id is 144929 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe280034e0 con 0x7efe4409ff70 2024-09-18T14:10:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699050448 0 0) 0x7efe4409d310 con 0x7efe4409f7d0 2024-09-18T14:10:05.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): my global_id is 164425 2024-09-18T14:10:05.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe440a2560 con 0x7efe4409f7d0 2024-09-18T14:10:05.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167106813 0 0) 0x7efe44121600 con 0x7efe4409fba0 2024-09-18T14:10:05.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): my global_id is 164341 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe4409d310 con 0x7efe4409fba0 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1362967217 0 0) 0x7efe280034e0 con 0x7efe4409ff70 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe44121600 con 0x7efe4409ff70 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe3c001d00 con 0x7efe4409ff70 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): dump: 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.859 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 144021614 0 0) 0x7efe440a2560 con 0x7efe4409f7d0 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe280034e0 con 0x7efe4409f7d0 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe400016a0 con 0x7efe4409f7d0 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.842+0000 7efe397fa640 10 monclient(hunting): dump: 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.862 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 700906872 0 0) 0x7efe44121600 con 0x7efe4409ff70 2024-09-18T14:10:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 >> v1:172.21.15.45:6791/0 conn(0x7efe4409f7d0 legacy=0x7efe4411d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 >> v1:172.21.15.84:6789/0 conn(0x7efe4409fba0 legacy=0x7efe4411d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 10 monclient: _finish_hunting 0 2024-09-18T14:10:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 1 monclient: found mon.b 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe441206c0 con 0x7efe4409ff70 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 10 monclient: _finish_auth 0 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe4b634640 5 monclient: authenticate success, global_id 144929 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe4b634640 10 monclient: _renew_subs 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe4b634640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe4b634640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4411e8b0 con 0x7efe4409ff70 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe4b634640 10 monclient: _renew_subs 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe4b634640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe4b634640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4411f480 con 0x7efe4409ff70 2024-09-18T14:10:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efe3c004b20 con 0x7efe4409ff70 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe3c004fd0 con 0x7efe4409ff70 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe3b7fe640 4 set_mon_vals no callback set 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 10 monclient: dump: 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7efe3c00ffa0 con 0x7efe4409ff70 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.843+0000 7efe4b634640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe4b634640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7efe24006580 con 0x7efe24002830 2024-09-18T14:10:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(530..530 src has 344..530) v4 ==== 3959+0+0 (unknown 688093774 0 0) 0x7efe3c046d80 con 0x7efe4409ff70 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586097169 0 0) 0x7efe24006580 con 0x7efe24002830 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 10 monclient(hunting): my global_id is 164434 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe44121600 con 0x7efe24002830 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe48bcf640 10 monclient: get_auth_request con 0x7efe2803a610 auth_method 0 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3320145909 0 0) 0x7efe44121600 con 0x7efe24002830 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe24006580 con 0x7efe24002830 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe40001180 con 0x7efe24002830 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 10 monclient: dump: 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 263409562 0 0) 0x7efe24006580 con 0x7efe24002830 2024-09-18T14:10:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.844+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe2400a0a0 con 0x7efe24002830 2024-09-18T14:10:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.845+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7efe400051d0 con 0x7efe24002830 2024-09-18T14:10:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.845+0000 7efe397fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.845+0000 7efe397fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.845+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 >> v1:172.21.15.45:6791/0 conn(0x7efe24002830 legacy=0x7efe24004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.845+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 >> v1:172.21.15.45:6791/0 conn(0x7efe24002830 legacy=0x7efe24004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.850+0000 7efe4b634640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.850+0000 7efe4b634640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7efe24009610 con 0x7efe24008ff0 2024-09-18T14:10:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615886154 0 0) 0x7efe24009610 con 0x7efe24008ff0 2024-09-18T14:10:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 10 monclient(hunting): my global_id is 164443 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe24006580 con 0x7efe24008ff0 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2069394667 0 0) 0x7efe24006580 con 0x7efe24008ff0 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe24009610 con 0x7efe24008ff0 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe40005970 con 0x7efe24008ff0 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 10 monclient: dump: 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2331838353 0 0) 0x7efe24009610 con 0x7efe24008ff0 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efe240098d0 con 0x7efe24008ff0 2024-09-18T14:10:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 3630035589) 0x7efe40006d40 con 0x7efe24008ff0 2024-09-18T14:10:05.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:05.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:05.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 >> v1:172.21.15.45:6791/0 conn(0x7efe24008ff0 legacy=0x7efe24008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.851+0000 7efe397fa640 1 -- 172.21.15.45:0/2789849493 >> v1:172.21.15.45:6791/0 conn(0x7efe24008ff0 legacy=0x7efe24008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:05.872 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:05.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:05.872 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:05.872 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:05.872 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:05.873 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:05.874 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:05.874 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:05.874 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:05.874 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:05.874 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:05.874 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:05.874 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:05.874 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:05.874 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:05.874 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:05.874 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.874 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:05.875 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:05.876 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:05.878 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:05.879 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.879 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.879 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.879 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:05.879 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:05.879 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.879 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.880 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:05.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:05.881 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.881 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.881 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:05.881 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.881 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.881 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.881 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:05.881 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.884 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.885 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.886 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:05.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.891 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:05.891 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:05.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:05.891 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:05.891 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:05.891 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:05.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:05.891 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:05.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:05.891 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:05.891 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.852+0000 7efe4b634640 1 -- 172.21.15.45:0/2789849493 >> v1:172.21.15.84:6812/31478 conn(0x7efe2803a610 legacy=0x7efe2803cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.852+0000 7efe4b634640 10 monclient: shutdown 2024-09-18T14:10:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.852+0000 7efe4b634640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.852+0000 7efe4b634640 1 -- 172.21.15.45:0/2789849493 >> v1:172.21.15.45:6790/0 conn(0x7efe4409ff70 legacy=0x7efe44149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.852+0000 7efe49bd1640 1 -- 172.21.15.45:0/2789849493 reap_dead start 2024-09-18T14:10:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.852+0000 7efe49bd1640 1 -- 172.21.15.45:0/2789849493 reap_dead start 2024-09-18T14:10:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.852+0000 7efe4b634640 1 -- 172.21.15.45:0/2789849493 shutdown_connections 2024-09-18T14:10:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.852+0000 7efe4b634640 1 -- 172.21.15.45:0/2789849493 >> 172.21.15.45:0/2789849493 conn(0x7efe440630b0 msgr2=0x7efe440a4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.852+0000 7efe4b634640 1 -- 172.21.15.45:0/2789849493 shutdown_connections 2024-09-18T14:10:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.852+0000 7efe4b634640 1 -- 172.21.15.45:0/2789849493 wait complete. 2024-09-18T14:10:05.894 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:05.894 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:10:05.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:10:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.991+0000 7f38b755d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.991+0000 7f38b755d640 10 monclient: build_initial_monmap 2024-09-18T14:10:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.991+0000 7f38b755d640 10 monclient: monmap: 2024-09-18T14:10:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:05.993462+0000 2024-09-18T14:10:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:05.993462+0000 2024-09-18T14:10:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:05.994 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 1 Processor -- start 2024-09-18T14:10:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 1 -- start start 2024-09-18T14:10:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 10 monclient: init 2024-09-18T14:10:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 10 monclient: _add_conns ranks=[3,1,8,4,0,2,7,6,5] 2024-09-18T14:10:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 10 monclient(hunting): picked mon.noname-g con 0x7f38b0098300 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 10 monclient(hunting): picked mon.noname-f con 0x7f38b00986d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 10 monclient(hunting): picked mon.noname-e con 0x7f38b0098b00 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b005ed70 con 0x7f38b00986d0 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b0133160 con 0x7f38b0098300 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b00965e0 con 0x7f38b0098b00 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b755d640 10 monclient(hunting): authenticate will time out at 1421.331768s 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b5afa640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f38b0098b00 0x7f38b013aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:44910/0 (socket says 172.21.15.45:44910) 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.992+0000 7f38b5afa640 1 -- 172.21.15.45:0/1064281632 learned_addr learned my addr 172.21.15.45:0/1064281632 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 1 -- 172.21.15.45:0/1064281632 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932901092 0 0) 0x7f38b00965e0 con 0x7f38b0098b00 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient(hunting): my global_id is 5525 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 1 -- 172.21.15.45:0/1064281632 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38980034e0 con 0x7f38b0098b00 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 1 -- 172.21.15.45:0/1064281632 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3437367228 0 0) 0x7f38980034e0 con 0x7f38b0098b00 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 1 -- 172.21.15.45:0/1064281632 >> v1:172.21.15.84:6790/0 conn(0x7f38b0098300 legacy=0x7f38b0132a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 1 -- 172.21.15.45:0/1064281632 >> v1:172.21.15.84:6789/0 conn(0x7f38b00986d0 legacy=0x7f38b0137770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient: _finish_hunting 0 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 1 monclient: found mon.noname-e 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 1 -- 172.21.15.45:0/1064281632 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b00968c0 con 0x7f38b0098b00 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient: _finish_auth 0 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 1 -- 172.21.15.45:0/1064281632 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f38a0001830 con 0x7f38b0098b00 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38b755d640 5 monclient: authenticate success, global_id 5525 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38b755d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient: dump: 2024-09-18T14:10:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38b755d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 1 -- 172.21.15.45:0/1064281632 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f38a0001990 con 0x7f38b0098b00 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 1 -- 172.21.15.45:0/1064281632 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f38a0004ba0 con 0x7f38b0098b00 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38b755d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38b755d640 4 set_mon_vals no callback set 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38affff640 10 monclient: dump: 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.006 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38b755d640 10 monclient: shutdown 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38b755d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:06.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38b755d640 1 -- 172.21.15.45:0/1064281632 >> v1:172.21.15.45:6793/0 conn(0x7f38b0098b00 legacy=0x7f38b013aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38b755d640 1 -- 172.21.15.45:0/1064281632 shutdown_connections 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.993+0000 7f38b755d640 1 -- 172.21.15.45:0/1064281632 >> 172.21.15.45:0/1064281632 conn(0x7f38b0067b20 msgr2=0x7f38b00a2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.994+0000 7f38b755d640 1 -- 172.21.15.45:0/1064281632 shutdown_connections 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.994+0000 7f38b755d640 1 -- 172.21.15.45:0/1064281632 wait complete. 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.994+0000 7f38b755d640 10 monclient: build_initial_monmap 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.994+0000 7f38b755d640 10 monclient: monmap: 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.994+0000 7f38b755d640 1 Processor -- start 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.994+0000 7f38b755d640 1 -- start start 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.994+0000 7f38b755d640 10 monclient: init 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38b755d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38b755d640 10 monclient: _add_conns ranks=[6,7,2,0,4,5,8,1,3] 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38b755d640 10 monclient(hunting): picked mon.noname-d con 0x7f38b0098300 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:06.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38b755d640 10 monclient(hunting): picked mon.noname-i con 0x7f38b00986d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38b755d640 10 monclient(hunting): picked mon.noname-b con 0x7f38b0098b00 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38b755d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b0099040 con 0x7f38b0098b00 2024-09-18T14:10:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38b755d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b012dc50 con 0x7f38b0098300 2024-09-18T14:10:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38b755d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b012f720 con 0x7f38b00986d0 2024-09-18T14:10:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38b52f9640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f38b0098300 0x7f38b0141ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34856/0 (socket says 172.21.15.45:34856) 2024-09-18T14:10:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38b52f9640 1 -- 172.21.15.45:0/4026835238 learned_addr learned my addr 172.21.15.45:0/4026835238 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38b755d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38b755d640 10 monclient(hunting): authenticate will time out at 1421.334681s 2024-09-18T14:10:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005732993 0 0) 0x7f38b012dc50 con 0x7f38b0098300 2024-09-18T14:10:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38ad7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38ad7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38ad7fa640 10 monclient(hunting): my global_id is 184236 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f389c0034e0 con 0x7f38b0098300 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684201671 0 0) 0x7f38b0099040 con 0x7f38b0098b00 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38ad7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38ad7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38ad7fa640 10 monclient(hunting): my global_id is 144938 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.995+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38b012dc50 con 0x7f38b0098b00 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799551310 0 0) 0x7f38b012f720 con 0x7f38b00986d0 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): my global_id is 184228 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38b0099040 con 0x7f38b00986d0 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4073348794 0 0) 0x7f389c0034e0 con 0x7f38b0098300 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f38b012f720 con 0x7f38b0098300 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f38a8001ad0 con 0x7f38b0098300 2024-09-18T14:10:06.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): dump: 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.013 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.013 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.013 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:06.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 56932807 0 0) 0x7f38b012dc50 con 0x7f38b0098b00 2024-09-18T14:10:06.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f389c0034e0 con 0x7f38b0098b00 2024-09-18T14:10:06.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f38a0001770 con 0x7f38b0098b00 2024-09-18T14:10:06.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:06.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): dump: 2024-09-18T14:10:06.013 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.013 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.014 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2936327887 0 0) 0x7f38b0099040 con 0x7f38b00986d0 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f38b012dc50 con 0x7f38b00986d0 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f38a4001de0 con 0x7f38b00986d0 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient(hunting): dump: 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.016 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1541538715 0 0) 0x7f38b012f720 con 0x7f38b0098300 2024-09-18T14:10:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 >> v1:172.21.15.84:6792/0 conn(0x7f38b00986d0 legacy=0x7f38b01468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 >> v1:172.21.15.45:6790/0 conn(0x7f38b0098b00 legacy=0x7f38b014a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient: _finish_hunting 0 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 monclient: found mon.d 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b012e7c0 con 0x7f38b0098300 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient: _finish_auth 0 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38b755d640 5 monclient: authenticate success, global_id 184236 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38b755d640 10 monclient: _renew_subs 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38b755d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38b755d640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b01421e0 con 0x7f38b0098300 2024-09-18T14:10:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38b755d640 10 monclient: _renew_subs 2024-09-18T14:10:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38b755d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38b755d640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b0130b70 con 0x7f38b0098300 2024-09-18T14:10:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f38a8001d10 con 0x7f38b0098300 2024-09-18T14:10:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f38a80025f0 con 0x7f38b0098300 2024-09-18T14:10:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38af7fe640 4 set_mon_vals no callback set 2024-09-18T14:10:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.996+0000 7f38ad7fa640 10 monclient: dump: 2024-09-18T14:10:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.019 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.020 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.997+0000 7f38b755d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.997+0000 7f38b755d640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3890006580 con 0x7f3890002830 2024-09-18T14:10:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.997+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f38a80027d0 con 0x7f38b0098300 2024-09-18T14:10:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.997+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(530..530 src has 344..530) v4 ==== 3959+0+0 (unknown 688093774 0 0) 0x7f38a8046b40 con 0x7f38b0098300 2024-09-18T14:10:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.997+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070065690 0 0) 0x7f3890006580 con 0x7f3890002830 2024-09-18T14:10:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.997+0000 7f38ad7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.997+0000 7f38ad7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.997+0000 7f38ad7fa640 10 monclient(hunting): my global_id is 184245 2024-09-18T14:10:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.997+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38b012f720 con 0x7f3890002830 2024-09-18T14:10:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.997+0000 7f38b5afa640 10 monclient: get_auth_request con 0x7f389c03aa00 auth_method 0 2024-09-18T14:10:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.997+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2948018556 0 0) 0x7f38b012f720 con 0x7f3890002830 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.998+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3890006580 con 0x7f3890002830 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.998+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f38a4002630 con 0x7f3890002830 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.998+0000 7f38ad7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.998+0000 7f38ad7fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.998+0000 7f38ad7fa640 10 monclient: dump: 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.998+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1039004746 0 0) 0x7f3890006580 con 0x7f3890002830 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.998+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f389000a0a0 con 0x7f3890002830 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.998+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f38a4002930 con 0x7f3890002830 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.998+0000 7f38ad7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.998+0000 7f38ad7fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:06.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.998+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 >> v1:172.21.15.45:6792/0 conn(0x7f3890002830 legacy=0x7f3890004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:05.998+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 >> v1:172.21.15.45:6792/0 conn(0x7f3890002830 legacy=0x7f3890004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:06.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38b755d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:06.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38b755d640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3890007540 con 0x7f38900068f0 2024-09-18T14:10:06.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181243652 0 0) 0x7f3890007540 con 0x7f38900068f0 2024-09-18T14:10:06.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38ad7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38ad7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38ad7fa640 10 monclient(hunting): my global_id is 184254 2024-09-18T14:10:06.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3890006580 con 0x7f38900068f0 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2262481225 0 0) 0x7f3890006580 con 0x7f38900068f0 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3890007540 con 0x7f38900068f0 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f38a4003150 con 0x7f38900068f0 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38ad7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38ad7fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.004+0000 7f38ad7fa640 10 monclient: dump: 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.025 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2291267779 0 0) 0x7f3890007540 con 0x7f38900068f0 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f38900087a0 con 0x7f38900068f0 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 2692654708) 0x7f38a4005370 con 0x7f38900068f0 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38ad7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38ad7fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 >> v1:172.21.15.45:6792/0 conn(0x7f38900068f0 legacy=0x7f3890006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38ad7fa640 1 -- 172.21.15.45:0/4026835238 >> v1:172.21.15.45:6792/0 conn(0x7f38900068f0 legacy=0x7f3890006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:06.027 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:06.027 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:06.027 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:06.027 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:06.027 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:06.027 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:06.027 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:06.027 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:06.028 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:06.030 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.031 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.035 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.036 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:06.038 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.039 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.040 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:06.040 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:06.040 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.040 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.040 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.040 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:06.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:06.041 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.041 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.041 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.041 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:06.041 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.041 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.041 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.041 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:06.041 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.042 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.043 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.044 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.045 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:10:06.046 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.046 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.046 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:06.046 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38b755d640 1 -- 172.21.15.45:0/4026835238 >> v1:172.21.15.84:6812/31478 conn(0x7f389c03aa00 legacy=0x7f389c03ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38b755d640 10 monclient: shutdown 2024-09-18T14:10:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38b755d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38b755d640 1 -- 172.21.15.45:0/4026835238 >> v1:172.21.15.45:6792/0 conn(0x7f38b0098300 legacy=0x7f38b0141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38b5afa640 1 -- 172.21.15.45:0/4026835238 reap_dead start 2024-09-18T14:10:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38b5afa640 1 -- 172.21.15.45:0/4026835238 reap_dead start 2024-09-18T14:10:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38b755d640 1 -- 172.21.15.45:0/4026835238 shutdown_connections 2024-09-18T14:10:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38b755d640 1 -- 172.21.15.45:0/4026835238 >> 172.21.15.45:0/4026835238 conn(0x7f38b0067b20 msgr2=0x7f38b009ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38b755d640 1 -- 172.21.15.45:0/4026835238 shutdown_connections 2024-09-18T14:10:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.005+0000 7f38b755d640 1 -- 172.21.15.45:0/4026835238 wait complete. 2024-09-18T14:10:06.048 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:06.048 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:10:06.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:10:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.144+0000 7fae75e78640 10 monclient: get_monmap_and_config 2024-09-18T14:10:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.144+0000 7fae75e78640 10 monclient: build_initial_monmap 2024-09-18T14:10:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.144+0000 7fae75e78640 10 monclient: monmap: 2024-09-18T14:10:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:06.146169+0000 2024-09-18T14:10:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:06.146169+0000 2024-09-18T14:10:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:06.149 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.144+0000 7fae75e78640 1 Processor -- start 2024-09-18T14:10:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.144+0000 7fae75e78640 1 -- start start 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae75e78640 10 monclient: init 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae75e78640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae75e78640 10 monclient: _add_conns ranks=[3,7,1,4,6,0,8,2,5] 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae75e78640 10 monclient(hunting): picked mon.noname-g con 0x7fae7006e810 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae75e78640 10 monclient(hunting): picked mon.noname-i con 0x7fae700a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae75e78640 10 monclient(hunting): picked mon.noname-f con 0x7fae700a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae75e78640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7005ed70 con 0x7fae700a1080 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae75e78640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae70097620 con 0x7fae7006e810 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae75e78640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae70097340 con 0x7fae700a0cb0 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae75e78640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae75e78640 10 monclient(hunting): authenticate will time out at 1421.484572s 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae6ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fae700a1080 0x7fae7013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:47066/0 (socket says 172.21.15.45:47066) 2024-09-18T14:10:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae6f7fe640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7fae7006e810 0x7fae700a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:51246/0 (socket says 172.21.15.45:51246) 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.145+0000 7fae6ffff640 1 -- 172.21.15.45:0/3036265327 learned_addr learned my addr 172.21.15.45:0/3036265327 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591263559 0 0) 0x7fae70097340 con 0x7fae700a0cb0 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient(hunting): my global_id is 184237 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae5c0034e0 con 0x7fae700a0cb0 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241508195 0 0) 0x7fae7005ed70 con 0x7fae700a1080 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient(hunting): my global_id is 164359 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae70097340 con 0x7fae700a1080 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588507682 0 0) 0x7fae70097620 con 0x7fae7006e810 2024-09-18T14:10:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient(hunting): my global_id is 184206 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae7005ed70 con 0x7fae7006e810 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2783802583 0 0) 0x7fae70097340 con 0x7fae700a1080 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 >> v1:172.21.15.84:6792/0 conn(0x7fae700a0cb0 legacy=0x7fae70137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 >> v1:172.21.15.84:6790/0 conn(0x7fae7006e810 legacy=0x7fae700a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 monclient: found mon.noname-f 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae700a1660 con 0x7fae700a1080 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae68001b90 con 0x7fae700a1080 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient: dump: 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.162 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae75e78640 5 monclient: authenticate success, global_id 164359 2024-09-18T14:10:06.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae75e78640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fae68001cf0 con 0x7fae700a1080 2024-09-18T14:10:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae75e78640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae75e78640 4 set_mon_vals no callback set 2024-09-18T14:10:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 1 -- 172.21.15.45:0/3036265327 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae68004be0 con 0x7fae700a1080 2024-09-18T14:10:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:10:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae6e7fc640 10 monclient: dump: 2024-09-18T14:10:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae75e78640 10 monclient: shutdown 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae75e78640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.146+0000 7fae75e78640 1 -- 172.21.15.45:0/3036265327 >> v1:172.21.15.84:6789/0 conn(0x7fae700a1080 legacy=0x7fae7013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.147+0000 7fae75e78640 1 -- 172.21.15.45:0/3036265327 shutdown_connections 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.147+0000 7fae75e78640 1 -- 172.21.15.45:0/3036265327 >> 172.21.15.45:0/3036265327 conn(0x7fae700a5bb0 msgr2=0x7fae700a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.147+0000 7fae75e78640 1 -- 172.21.15.45:0/3036265327 shutdown_connections 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.147+0000 7fae75e78640 1 -- 172.21.15.45:0/3036265327 wait complete. 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.147+0000 7fae75e78640 10 monclient: build_initial_monmap 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.147+0000 7fae75e78640 10 monclient: monmap: 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 1 Processor -- start 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 1 -- start start 2024-09-18T14:10:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 10 monclient: init 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 10 monclient: _add_conns ranks=[1,4,2,5,7,6,8,0,3] 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 10 monclient(hunting): picked mon.noname-f con 0x7fae7006e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 10 monclient(hunting): picked mon.noname-c con 0x7fae700a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 10 monclient(hunting): picked mon.noname-b con 0x7fae700a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae70122690 con 0x7fae7006e810 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7013da20 con 0x7fae700a1080 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7013e6e0 con 0x7fae700a0cb0 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae75e78640 10 monclient(hunting): authenticate will time out at 1421.487619s 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae6f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fae7006e810 0x7fae7013d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:47080/0 (socket says 172.21.15.45:47080) 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae6f7fe640 1 -- 172.21.15.45:0/356981102 learned_addr learned my addr 172.21.15.45:0/356981102 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467514534 0 0) 0x7fae7013da20 con 0x7fae700a1080 2024-09-18T14:10:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae677fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae677fe640 10 monclient(hunting): my global_id is 144947 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae500034e0 con 0x7fae700a1080 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310497501 0 0) 0x7fae7013e6e0 con 0x7fae700a0cb0 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae677fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.148+0000 7fae677fe640 10 monclient(hunting): my global_id is 164452 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae7013da20 con 0x7fae700a0cb0 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280668092 0 0) 0x7fae70122690 con 0x7fae7006e810 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): my global_id is 164368 2024-09-18T14:10:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae7013e6e0 con 0x7fae7006e810 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2389888932 0 0) 0x7fae500034e0 con 0x7fae700a1080 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fae70122690 con 0x7fae700a1080 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4145934258 0 0) 0x7fae7013da20 con 0x7fae700a0cb0 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fae500034e0 con 0x7fae700a0cb0 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae68001b50 con 0x7fae700a1080 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): dump: 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.170 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4123355544 0 0) 0x7fae7013e6e0 con 0x7fae7006e810 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fae7013da20 con 0x7fae7006e810 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae58001af0 con 0x7fae700a0cb0 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): dump: 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae60001250 con 0x7fae7006e810 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient(hunting): dump: 2024-09-18T14:10:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.174 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.174 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2111058132 0 0) 0x7fae70122690 con 0x7fae700a1080 2024-09-18T14:10:06.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 >> v1:172.21.15.45:6791/0 conn(0x7fae700a0cb0 legacy=0x7fae70146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 >> v1:172.21.15.84:6789/0 conn(0x7fae7006e810 legacy=0x7fae7013d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient: _finish_hunting 0 2024-09-18T14:10:06.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 monclient: found mon.b 2024-09-18T14:10:06.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:06.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:06.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae701223b0 con 0x7fae700a1080 2024-09-18T14:10:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient: _finish_auth 0 2024-09-18T14:10:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae75e78640 5 monclient: authenticate success, global_id 144947 2024-09-18T14:10:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae75e78640 10 monclient: _renew_subs 2024-09-18T14:10:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae75e78640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae75e78640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae70122970 con 0x7fae700a1080 2024-09-18T14:10:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae75e78640 10 monclient: _renew_subs 2024-09-18T14:10:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae75e78640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae75e78640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7013dec0 con 0x7fae700a1080 2024-09-18T14:10:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fae68004ec0 con 0x7fae700a1080 2024-09-18T14:10:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae680053f0 con 0x7fae700a1080 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae6dffb640 4 set_mon_vals no callback set 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.149+0000 7fae677fe640 10 monclient: dump: 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.150+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fae68005550 con 0x7fae700a1080 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.150+0000 7fae75e78640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.150+0000 7fae75e78640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fae54006580 con 0x7fae54002830 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.150+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(530..530 src has 344..530) v4 ==== 3959+0+0 (unknown 688093774 0 0) 0x7fae68046e40 con 0x7fae700a1080 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.150+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909478625 0 0) 0x7fae54006580 con 0x7fae54002830 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.150+0000 7fae677fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.150+0000 7fae677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.150+0000 7fae677fe640 10 monclient(hunting): my global_id is 5534 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.150+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae70122690 con 0x7fae54002830 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.150+0000 7fae6effd640 10 monclient: get_auth_request con 0x7fae5003a9f0 auth_method 0 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.150+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3032409599 0 0) 0x7fae70122690 con 0x7fae54002830 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.150+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fae54006580 con 0x7fae54002830 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.151+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae60004ae0 con 0x7fae54002830 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.151+0000 7fae677fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.151+0000 7fae677fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.151+0000 7fae677fe640 10 monclient: dump: 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.151+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 638506627 0 0) 0x7fae54006580 con 0x7fae54002830 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.151+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae5400a0a0 con 0x7fae54002830 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.151+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fae60006fc0 con 0x7fae54002830 2024-09-18T14:10:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.151+0000 7fae677fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.151+0000 7fae677fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.151+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 >> v1:172.21.15.45:6793/0 conn(0x7fae54002830 legacy=0x7fae54004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.151+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 >> v1:172.21.15.45:6793/0 conn(0x7fae54002830 legacy=0x7fae54004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae75e78640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae75e78640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fae540074f0 con 0x7fae540068a0 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1518992329 0 0) 0x7fae540074f0 con 0x7fae540068a0 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 10 monclient(hunting): my global_id is 5543 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae54006580 con 0x7fae540068a0 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1245024514 0 0) 0x7fae54006580 con 0x7fae540068a0 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fae540074f0 con 0x7fae540068a0 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fae60005630 con 0x7fae540068a0 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 10 monclient: dump: 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1685646519 0 0) 0x7fae540074f0 con 0x7fae540068a0 2024-09-18T14:10:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.157+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fae54008750 con 0x7fae540068a0 2024-09-18T14:10:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7728 (unknown 0 0 3908967088) 0x7fae60006b30 con 0x7fae540068a0 2024-09-18T14:10:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae677fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae677fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 >> v1:172.21.15.45:6793/0 conn(0x7fae540068a0 legacy=0x7fae54006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae677fe640 1 -- 172.21.15.45:0/356981102 >> v1:172.21.15.45:6793/0 conn(0x7fae540068a0 legacy=0x7fae54006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:06.184 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:10:06.185 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:06.186 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:06.187 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:06.188 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.189 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:06.193 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.194 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.197 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:06.199 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:06.200 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.201 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.202 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.203 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.204 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.204 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:06.204 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.204 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.204 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:06.204 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae75e78640 1 -- 172.21.15.45:0/356981102 >> v1:172.21.15.84:6812/31478 conn(0x7fae5003a9f0 legacy=0x7fae5003cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae75e78640 10 monclient: shutdown 2024-09-18T14:10:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae75e78640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae75e78640 1 -- 172.21.15.45:0/356981102 >> v1:172.21.15.45:6790/0 conn(0x7fae700a1080 legacy=0x7fae70149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae6ffff640 1 -- 172.21.15.45:0/356981102 reap_dead start 2024-09-18T14:10:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae6ffff640 1 -- 172.21.15.45:0/356981102 reap_dead start 2024-09-18T14:10:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae75e78640 1 -- 172.21.15.45:0/356981102 shutdown_connections 2024-09-18T14:10:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae75e78640 1 -- 172.21.15.45:0/356981102 >> 172.21.15.45:0/356981102 conn(0x7fae700a5bb0 msgr2=0x7fae700a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:06.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae75e78640 1 -- 172.21.15.45:0/356981102 shutdown_connections 2024-09-18T14:10:06.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.158+0000 7fae75e78640 1 -- 172.21.15.45:0/356981102 wait complete. 2024-09-18T14:10:06.207 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:06.207 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:10:06.207 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:10:06.294 INFO:tasks.rados.rados.0.smithi084.stdout:2430: done (1 left) 2024-09-18T14:10:06.295 INFO:tasks.rados.rados.0.smithi084.stdout:2431: read oid 7 snap -1 2024-09-18T14:10:06.295 INFO:tasks.rados.rados.0.smithi084.stdout:2431: expect (ObjNum 818 snap 250 seq_num 818) 2024-09-18T14:10:06.295 INFO:tasks.rados.rados.0.smithi084.stdout:2432: read oid 4 snap -1 2024-09-18T14:10:06.295 INFO:tasks.rados.rados.0.smithi084.stdout:2432: expect deleted 2024-09-18T14:10:06.295 INFO:tasks.rados.rados.0.smithi084.stdout:2433: snap_remove snap 227 2024-09-18T14:10:06.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.313+0000 7fc766b9d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:06.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.313+0000 7fc766b9d640 10 monclient: build_initial_monmap 2024-09-18T14:10:06.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.313+0000 7fc766b9d640 10 monclient: monmap: 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:06.315435+0000 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:06.315435+0000 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:06.316 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:06.316 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.314+0000 7fc766b9d640 1 Processor -- start 2024-09-18T14:10:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.314+0000 7fc766b9d640 1 -- start start 2024-09-18T14:10:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.314+0000 7fc766b9d640 10 monclient: init 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.314+0000 7fc766b9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.314+0000 7fc766b9d640 10 monclient: _add_conns ranks=[0,6,5,4,8,2,3,7,1] 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.314+0000 7fc766b9d640 10 monclient(hunting): picked mon.noname-a con 0x7fc76009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.314+0000 7fc766b9d640 10 monclient(hunting): picked mon.noname-d con 0x7fc76009fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.314+0000 7fc766b9d640 10 monclient(hunting): picked mon.noname-h con 0x7fc76009ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.314+0000 7fc766b9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76005ed70 con 0x7fc76009f7d0 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.314+0000 7fc766b9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76009dfc0 con 0x7fc76009ff70 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.314+0000 7fc766b9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76009dce0 con 0x7fc76009fba0 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc766b9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc766b9d640 10 monclient(hunting): authenticate will time out at 1421.654016s 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc764939640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc76009f7d0 0x7fc760132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51184/0 (socket says 172.21.15.45:51184) 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc764939640 1 -- 172.21.15.45:0/3904604235 learned_addr learned my addr 172.21.15.45:0/3904604235 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592784033 0 0) 0x7fc76005ed70 con 0x7fc76009f7d0 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient(hunting): my global_id is 5697 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7500034e0 con 0x7fc76009f7d0 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156376463 0 0) 0x7fc76009dce0 con 0x7fc76009fba0 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient(hunting): my global_id is 184263 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc76005ed70 con 0x7fc76009fba0 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876149288 0 0) 0x7fc76009dfc0 con 0x7fc76009ff70 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient(hunting): my global_id is 184244 2024-09-18T14:10:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc76009dce0 con 0x7fc76009ff70 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4222621770 0 0) 0x7fc76005ed70 con 0x7fc76009fba0 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 >> v1:172.21.15.84:6791/0 conn(0x7fc76009ff70 legacy=0x7fc76013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 >> v1:172.21.15.45:6789/0 conn(0x7fc76009f7d0 legacy=0x7fc760132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 monclient: found mon.noname-d 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7600a1c00 con 0x7fc76009fba0 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient: _finish_auth 0 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc766b9d640 5 monclient: authenticate success, global_id 184263 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc74c001810 con 0x7fc76009fba0 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc766b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient: dump: 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc766b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc74c001970 con 0x7fc76009fba0 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 1 -- 172.21.15.45:0/3904604235 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc74c001f20 con 0x7fc76009fba0 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc766b9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc766b9d640 4 set_mon_vals no callback set 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.315+0000 7fc75f7fe640 10 monclient: dump: 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.316+0000 7fc766b9d640 10 monclient: shutdown 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.316+0000 7fc766b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.316+0000 7fc766b9d640 1 -- 172.21.15.45:0/3904604235 >> v1:172.21.15.45:6792/0 conn(0x7fc76009fba0 legacy=0x7fc760137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.316+0000 7fc766b9d640 1 -- 172.21.15.45:0/3904604235 shutdown_connections 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.316+0000 7fc766b9d640 1 -- 172.21.15.45:0/3904604235 >> 172.21.15.45:0/3904604235 conn(0x7fc7600630b0 msgr2=0x7fc7600634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.316+0000 7fc766b9d640 1 -- 172.21.15.45:0/3904604235 shutdown_connections 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.316+0000 7fc766b9d640 1 -- 172.21.15.45:0/3904604235 wait complete. 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.316+0000 7fc766b9d640 10 monclient: build_initial_monmap 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.317+0000 7fc766b9d640 10 monclient: monmap: 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.317+0000 7fc766b9d640 1 Processor -- start 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.317+0000 7fc766b9d640 1 -- start start 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.317+0000 7fc766b9d640 10 monclient: init 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.317+0000 7fc766b9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.317+0000 7fc766b9d640 10 monclient: _add_conns ranks=[1,3,2,6,0,5,8,4,7] 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.317+0000 7fc766b9d640 10 monclient(hunting): picked mon.noname-f con 0x7fc76009f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.317+0000 7fc766b9d640 10 monclient(hunting): picked mon.noname-g con 0x7fc76009fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.317+0000 7fc766b9d640 10 monclient(hunting): picked mon.noname-b con 0x7fc76009ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc766b9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76009d310 con 0x7fc76009f7d0 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc766b9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7600a2560 con 0x7fc76009ff70 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc766b9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc760142690 con 0x7fc76009fba0 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc766b9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc766b9d640 10 monclient(hunting): authenticate will time out at 1421.657078s 2024-09-18T14:10:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc76513a640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fc76009ff70 0x7fc760149b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33994/0 (socket says 172.21.15.45:33994) 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc76513a640 1 -- 172.21.15.45:0/4217662977 learned_addr learned my addr 172.21.15.45:0/4217662977 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158352987 0 0) 0x7fc7600a2560 con 0x7fc76009ff70 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): my global_id is 144956 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7440034e0 con 0x7fc76009ff70 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455374669 0 0) 0x7fc760142690 con 0x7fc76009fba0 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): my global_id is 184215 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7600a2560 con 0x7fc76009fba0 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905413631 0 0) 0x7fc76009d310 con 0x7fc76009f7d0 2024-09-18T14:10:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): my global_id is 164377 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc760142690 con 0x7fc76009f7d0 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2692936669 0 0) 0x7fc7440034e0 con 0x7fc76009ff70 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc76009d310 con 0x7fc76009ff70 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc758002e50 con 0x7fc76009ff70 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): dump: 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.336 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:06.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3969818492 0 0) 0x7fc7600a2560 con 0x7fc76009fba0 2024-09-18T14:10:06.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc7440034e0 con 0x7fc76009fba0 2024-09-18T14:10:06.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2109086293 0 0) 0x7fc760142690 con 0x7fc76009f7d0 2024-09-18T14:10:06.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc7600a2560 con 0x7fc76009f7d0 2024-09-18T14:10:06.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc74c001e20 con 0x7fc76009fba0 2024-09-18T14:10:06.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:06.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): dump: 2024-09-18T14:10:06.337 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.337 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.337 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.340 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc7540015b0 con 0x7fc76009f7d0 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient(hunting): dump: 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.341 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3953576831 0 0) 0x7fc76009d310 con 0x7fc76009ff70 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 >> v1:172.21.15.84:6790/0 conn(0x7fc76009fba0 legacy=0x7fc7601319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 >> v1:172.21.15.84:6789/0 conn(0x7fc76009f7d0 legacy=0x7fc760141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient: _finish_hunting 0 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 monclient: found mon.b 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc760120080 con 0x7fc76009ff70 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient: _finish_auth 0 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc766b9d640 5 monclient: authenticate success, global_id 144956 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc766b9d640 10 monclient: _renew_subs 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc766b9d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc766b9d640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7601207d0 con 0x7fc76009ff70 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc766b9d640 10 monclient: _renew_subs 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc766b9d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc766b9d640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc760132100 con 0x7fc76009ff70 2024-09-18T14:10:06.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc758003110 con 0x7fc76009ff70 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc758001a70 con 0x7fc76009ff70 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.319+0000 7fc75cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.319+0000 7fc75cff9640 10 monclient: dump: 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.344 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.318+0000 7fc75effd640 4 set_mon_vals no callback set 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.319+0000 7fc766b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.319+0000 7fc766b9d640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc76004c4a0 con 0x7fc760099a40 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.319+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc758001c50 con 0x7fc76009ff70 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.319+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(531..531 src has 344..531) v4 ==== 4003+0+0 (unknown 1057628888 0 0) 0x7fc758046e80 con 0x7fc76009ff70 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901909287 0 0) 0x7fc76004c4a0 con 0x7fc760099a40 2024-09-18T14:10:06.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 10 monclient(hunting): my global_id is 164386 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc76009d310 con 0x7fc760099a40 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75ffff640 10 monclient: get_auth_request con 0x7fc74403aa50 auth_method 0 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3041165827 0 0) 0x7fc76009d310 con 0x7fc760099a40 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc76004c4a0 con 0x7fc760099a40 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc7540017d0 con 0x7fc760099a40 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 10 monclient: dump: 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.346 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1655782949 0 0) 0x7fc76004c4a0 con 0x7fc760099a40 2024-09-18T14:10:06.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc760122170 con 0x7fc760099a40 2024-09-18T14:10:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc754006ba0 con 0x7fc760099a40 2024-09-18T14:10:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 >> v1:172.21.15.84:6789/0 conn(0x7fc760099a40 legacy=0x7fc760099e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.320+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 >> v1:172.21.15.84:6789/0 conn(0x7fc760099a40 legacy=0x7fc760099e10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.326+0000 7fc766b9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.326+0000 7fc766b9d640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc760141f50 con 0x7fc76009a520 2024-09-18T14:10:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443620840 0 0) 0x7fc760141f50 con 0x7fc76009a520 2024-09-18T14:10:06.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 10 monclient(hunting): my global_id is 164395 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc76004c4a0 con 0x7fc76009a520 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 106345888 0 0) 0x7fc76004c4a0 con 0x7fc76009a520 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc760141f50 con 0x7fc76009a520 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc754005480 con 0x7fc76009a520 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 10 monclient: dump: 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.350 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2712501665 0 0) 0x7fc760141f50 con 0x7fc76009a520 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc760121c40 con 0x7fc76009a520 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.327+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7556 (unknown 0 0 2781763722) 0x7fc754005820 con 0x7fc76009a520 2024-09-18T14:10:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc75cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc75cff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 >> v1:172.21.15.84:6789/0 conn(0x7fc76009a520 legacy=0x7fc76014f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc75cff9640 1 -- 172.21.15.45:0/4217662977 >> v1:172.21.15.84:6789/0 conn(0x7fc76009a520 legacy=0x7fc76014f2d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:06.352 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:06.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:06.352 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:06.352 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:06.352 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:06.352 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:06.352 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:06.353 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.354 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:06.355 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.356 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:06.357 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:06.359 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.360 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.362 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.363 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.364 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.365 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:06.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.367 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:06.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:10:06.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.371 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.371 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:06.371 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc766b9d640 1 -- 172.21.15.45:0/4217662977 >> v1:172.21.15.84:6812/31478 conn(0x7fc74403aa50 legacy=0x7fc74403cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc766b9d640 10 monclient: shutdown 2024-09-18T14:10:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc766b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc766b9d640 1 -- 172.21.15.45:0/4217662977 >> v1:172.21.15.45:6790/0 conn(0x7fc76009ff70 legacy=0x7fc760149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc76513a640 1 -- 172.21.15.45:0/4217662977 reap_dead start 2024-09-18T14:10:06.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc76513a640 1 -- 172.21.15.45:0/4217662977 reap_dead start 2024-09-18T14:10:06.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc766b9d640 1 -- 172.21.15.45:0/4217662977 shutdown_connections 2024-09-18T14:10:06.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc766b9d640 1 -- 172.21.15.45:0/4217662977 >> 172.21.15.45:0/4217662977 conn(0x7fc7600630b0 msgr2=0x7fc7600a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:06.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc766b9d640 1 -- 172.21.15.45:0/4217662977 shutdown_connections 2024-09-18T14:10:06.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.328+0000 7fc766b9d640 1 -- 172.21.15.45:0/4217662977 wait complete. 2024-09-18T14:10:06.373 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:06.373 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:10:06.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:10:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.469+0000 7f2369cc0640 10 monclient: get_monmap_and_config 2024-09-18T14:10:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.469+0000 7f2369cc0640 10 monclient: build_initial_monmap 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.469+0000 7f2369cc0640 10 monclient: monmap: 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:06.471165+0000 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:06.471165+0000 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:06.472 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.469+0000 7f2369cc0640 1 Processor -- start 2024-09-18T14:10:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f2369cc0640 1 -- start start 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f2369cc0640 10 monclient: init 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f2369cc0640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f2369cc0640 10 monclient: _add_conns ranks=[0,1,2,5,7,4,8,6,3] 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f2369cc0640 10 monclient(hunting): picked mon.noname-a con 0x7f236406e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f2369cc0640 10 monclient(hunting): picked mon.noname-f con 0x7f23640a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f2369cc0640 10 monclient(hunting): picked mon.noname-b con 0x7f23640a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f2369cc0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236405ed70 con 0x7f236406e810 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f2369cc0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2364097620 con 0x7f23640a0cb0 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f2369cc0640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2364097340 con 0x7f23640a1080 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f2369cc0640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f2369cc0640 10 monclient(hunting): authenticate will time out at 1421.809509s 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f23637fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f236406e810 0x7f23640a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51188/0 (socket says 172.21.15.45:51188) 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f23637fe640 1 -- 172.21.15.45:0/1255491806 learned_addr learned my addr 172.21.15.45:0/1255491806 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435651109 0 0) 0x7f236405ed70 con 0x7f236406e810 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f23627fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f23627fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f23627fc640 10 monclient(hunting): my global_id is 5706 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f234c0034e0 con 0x7f236406e810 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127443360 0 0) 0x7f2364097340 con 0x7f23640a1080 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f23627fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f23627fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f23627fc640 10 monclient(hunting): my global_id is 144965 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.470+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f236405ed70 con 0x7f23640a1080 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425159204 0 0) 0x7f2364097620 con 0x7f23640a0cb0 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient(hunting): my global_id is 164404 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2364097340 con 0x7f23640a0cb0 2024-09-18T14:10:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 456188320 0 0) 0x7f234c0034e0 con 0x7f236406e810 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 >> v1:172.21.15.45:6790/0 conn(0x7f23640a1080 legacy=0x7f236413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 >> v1:172.21.15.84:6789/0 conn(0x7f23640a0cb0 legacy=0x7f2364137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient: _finish_hunting 0 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 1 monclient: found mon.noname-a 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23640a1660 con 0x7f236406e810 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient: _finish_auth 0 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f2369cc0640 5 monclient: authenticate success, global_id 5706 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2354001830 con 0x7f236406e810 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f2369cc0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient: dump: 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f2369cc0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2354001a10 con 0x7f236406e810 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f2369cc0640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 1 -- 172.21.15.45:0/1255491806 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2354004ba0 con 0x7f236406e810 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f2369cc0640 4 set_mon_vals no callback set 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f23627fc640 10 monclient: dump: 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f2369cc0640 10 monclient: shutdown 2024-09-18T14:10:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f2369cc0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f2369cc0640 1 -- 172.21.15.45:0/1255491806 >> v1:172.21.15.45:6789/0 conn(0x7f236406e810 legacy=0x7f23640a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f2369cc0640 1 -- 172.21.15.45:0/1255491806 shutdown_connections 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f2369cc0640 1 -- 172.21.15.45:0/1255491806 >> 172.21.15.45:0/1255491806 conn(0x7f23640a5bb0 msgr2=0x7f23640a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f2369cc0640 1 -- 172.21.15.45:0/1255491806 shutdown_connections 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.471+0000 7f2369cc0640 1 -- 172.21.15.45:0/1255491806 wait complete. 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.472+0000 7f2369cc0640 10 monclient: build_initial_monmap 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.472+0000 7f2369cc0640 10 monclient: monmap: 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.472+0000 7f2369cc0640 1 Processor -- start 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.472+0000 7f2369cc0640 1 -- start start 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.472+0000 7f2369cc0640 10 monclient: init 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f2369cc0640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f2369cc0640 10 monclient: _add_conns ranks=[7,0,8,3,5,6,4,2,1] 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f2369cc0640 10 monclient(hunting): picked mon.noname-i con 0x7f236406e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f2369cc0640 10 monclient(hunting): picked mon.noname-a con 0x7f23640a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f2369cc0640 10 monclient(hunting): picked mon.noname-e con 0x7f23640a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f2369cc0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2364122e90 con 0x7f23640a0cb0 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f2369cc0640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236413dc80 con 0x7f236406e810 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f2369cc0640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23641228d0 con 0x7f23640a1080 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f2369cc0640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f2369cc0640 10 monclient(hunting): authenticate will time out at 1421.812414s 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f2362ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f23640a0cb0 0x7f23641467a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51202/0 (socket says 172.21.15.45:51202) 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f2362ffd640 1 -- 172.21.15.45:0/588830586 learned_addr learned my addr 172.21.15.45:0/588830586 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568767488 0 0) 0x7f23641228d0 con 0x7f23640a1080 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 10 monclient(hunting): my global_id is 5552 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23500034e0 con 0x7f23640a1080 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792073440 0 0) 0x7f2364122e90 con 0x7f23640a0cb0 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 10 monclient(hunting): my global_id is 5715 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23641228d0 con 0x7f23640a0cb0 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192637489 0 0) 0x7f236413dc80 con 0x7f236406e810 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 10 monclient(hunting): my global_id is 184246 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.473+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2364122e90 con 0x7f236406e810 2024-09-18T14:10:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2145799870 0 0) 0x7f23641228d0 con 0x7f23640a0cb0 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f236413dc80 con 0x7f23640a0cb0 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2358001b20 con 0x7f23640a0cb0 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient(hunting): dump: 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1015046615 0 0) 0x7f23500034e0 con 0x7f23640a1080 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f23641228d0 con 0x7f23640a1080 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1006894537 0 0) 0x7f2364122e90 con 0x7f236406e810 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f23500034e0 con 0x7f236406e810 2024-09-18T14:10:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f235c001cf0 con 0x7f23640a1080 2024-09-18T14:10:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient(hunting): dump: 2024-09-18T14:10:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.493 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f23540017f0 con 0x7f236406e810 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient(hunting): dump: 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.494 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.495 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3736117591 0 0) 0x7f236413dc80 con 0x7f23640a0cb0 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 >> v1:172.21.15.45:6793/0 conn(0x7f23640a1080 legacy=0x7f2364149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 >> v1:172.21.15.84:6792/0 conn(0x7f236406e810 legacy=0x7f236413d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 monclient: found mon.a 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23641225f0 con 0x7f23640a0cb0 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient: _finish_auth 0 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f2369cc0640 5 monclient: authenticate success, global_id 5715 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f2369cc0640 10 monclient: _renew_subs 2024-09-18T14:10:06.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f2369cc0640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f2369cc0640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2364122bb0 con 0x7f23640a0cb0 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f2369cc0640 10 monclient: _renew_subs 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f2369cc0640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f2369cc0640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2364121720 con 0x7f23640a0cb0 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2358002d70 con 0x7f23640a0cb0 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2358002370 con 0x7f23640a0cb0 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f2361ffb640 4 set_mon_vals no callback set 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.474+0000 7f234b7fe640 10 monclient: dump: 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.497 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.475+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2358002550 con 0x7f23640a0cb0 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.475+0000 7f2369cc0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:06.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.475+0000 7f2369cc0640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2344006580 con 0x7f2344002830 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.475+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(531..531 src has 344..531) v4 ==== 4003+0+0 (unknown 1057628888 0 0) 0x7f2358046ba0 con 0x7f23640a0cb0 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.475+0000 7f2363fff640 10 monclient: get_auth_request con 0x7f235003a8a0 auth_method 0 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.475+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151025384 0 0) 0x7f2344006580 con 0x7f2344002830 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.475+0000 7f234b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.475+0000 7f234b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 10 monclient(hunting): my global_id is 184224 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f236413dc80 con 0x7f2344002830 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1653937971 0 0) 0x7f236413dc80 con 0x7f2344002830 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2344006580 con 0x7f2344002830 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f23540010e0 con 0x7f2344002830 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 10 monclient: dump: 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.499 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.500 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3761947988 0 0) 0x7f2344006580 con 0x7f2344002830 2024-09-18T14:10:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f234400a0a0 con 0x7f2344002830 2024-09-18T14:10:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f23540071c0 con 0x7f2344002830 2024-09-18T14:10:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 >> v1:172.21.15.84:6790/0 conn(0x7f2344002830 legacy=0x7f2344004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.476+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 >> v1:172.21.15.84:6790/0 conn(0x7f2344002830 legacy=0x7f2344004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:06.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.482+0000 7f2369cc0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:06.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.482+0000 7f2369cc0640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2344009660 con 0x7f2344009040 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167737869 0 0) 0x7f2344009660 con 0x7f2344009040 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 10 monclient(hunting): my global_id is 184233 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2344006580 con 0x7f2344009040 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 719074183 0 0) 0x7f2344006580 con 0x7f2344009040 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2344009660 con 0x7f2344009040 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2354005810 con 0x7f2344009040 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 10 monclient: dump: 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3757912135 0 0) 0x7f2344009660 con 0x7f2344009040 2024-09-18T14:10:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2344009920 con 0x7f2344009040 2024-09-18T14:10:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 1506436626) 0x7f235400a1b0 con 0x7f2344009040 2024-09-18T14:10:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 >> v1:172.21.15.84:6790/0 conn(0x7f2344009040 legacy=0x7f2344008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.483+0000 7f234b7fe640 1 -- 172.21.15.45:0/588830586 >> v1:172.21.15.84:6790/0 conn(0x7f2344009040 legacy=0x7f2344008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:06.505 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:06.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:06.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.506 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:06.507 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:06.508 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:06.509 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:06.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:06.512 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:06.512 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.512 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.512 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:06.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:06.512 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:06.514 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:06.515 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.516 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.517 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:06.517 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.517 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.517 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.517 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:06.517 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:06.517 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.519 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.521 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.522 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.523 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:06.524 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.484+0000 7f2369cc0640 1 -- 172.21.15.45:0/588830586 >> v1:172.21.15.84:6812/31478 conn(0x7f235003a8a0 legacy=0x7f235003cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.484+0000 7f2369cc0640 10 monclient: shutdown 2024-09-18T14:10:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.484+0000 7f2369cc0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.484+0000 7f2369cc0640 1 -- 172.21.15.45:0/588830586 >> v1:172.21.15.45:6789/0 conn(0x7f23640a0cb0 legacy=0x7f23641467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.484+0000 7f2363fff640 1 -- 172.21.15.45:0/588830586 reap_dead start 2024-09-18T14:10:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.484+0000 7f2363fff640 1 -- 172.21.15.45:0/588830586 reap_dead start 2024-09-18T14:10:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.484+0000 7f2369cc0640 1 -- 172.21.15.45:0/588830586 shutdown_connections 2024-09-18T14:10:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.484+0000 7f2369cc0640 1 -- 172.21.15.45:0/588830586 >> 172.21.15.45:0/588830586 conn(0x7f23640a5bb0 msgr2=0x7f23640a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.484+0000 7f2369cc0640 1 -- 172.21.15.45:0/588830586 shutdown_connections 2024-09-18T14:10:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.484+0000 7f2369cc0640 1 -- 172.21.15.45:0/588830586 wait complete. 2024-09-18T14:10:06.527 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:06.527 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:10:06.527 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:10:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.623+0000 7fe1118e8640 10 monclient: get_monmap_and_config 2024-09-18T14:10:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.623+0000 7fe1118e8640 10 monclient: build_initial_monmap 2024-09-18T14:10:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.623+0000 7fe1118e8640 10 monclient: monmap: 2024-09-18T14:10:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:06.625846+0000 2024-09-18T14:10:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:06.625846+0000 2024-09-18T14:10:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:06.626 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.624+0000 7fe1118e8640 1 Processor -- start 2024-09-18T14:10:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.624+0000 7fe1118e8640 1 -- start start 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.624+0000 7fe1118e8640 10 monclient: init 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.624+0000 7fe1118e8640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.624+0000 7fe1118e8640 10 monclient: _add_conns ranks=[3,0,1,8,5,4,7,2,6] 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe1118e8640 10 monclient(hunting): picked mon.noname-g con 0x7fe10c09f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe1118e8640 10 monclient(hunting): picked mon.noname-a con 0x7fe10c09fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe1118e8640 10 monclient(hunting): picked mon.noname-f con 0x7fe10c09ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe1118e8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10c05ed70 con 0x7fe10c09fba0 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe1118e8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10c09dfc0 con 0x7fe10c09ff70 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe1118e8640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10c09dce0 con 0x7fe10c09f7d0 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe1118e8640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe1118e8640 10 monclient(hunting): authenticate will time out at 1421.964075s 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe10a7fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe10c09fba0 0x7fe10c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51204/0 (socket says 172.21.15.45:51204) 2024-09-18T14:10:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe10a7fc640 1 -- 172.21.15.45:0/4239183089 learned_addr learned my addr 172.21.15.45:0/4239183089 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229674334 0 0) 0x7fe10c05ed70 con 0x7fe10c09fba0 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient(hunting): my global_id is 5724 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f80034e0 con 0x7fe10c09fba0 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897471310 0 0) 0x7fe10c09dfc0 con 0x7fe10c09ff70 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient(hunting): my global_id is 164413 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10c05ed70 con 0x7fe10c09ff70 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680645700 0 0) 0x7fe10c09dce0 con 0x7fe10c09f7d0 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient(hunting): my global_id is 184242 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10c09dfc0 con 0x7fe10c09f7d0 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4251111789 0 0) 0x7fe0f80034e0 con 0x7fe10c09fba0 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 >> v1:172.21.15.84:6790/0 conn(0x7fe10c09f7d0 legacy=0x7fe10c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 >> v1:172.21.15.84:6789/0 conn(0x7fe10c09ff70 legacy=0x7fe10c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 monclient: found mon.noname-a 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c0a1c00 con 0x7fe10c09fba0 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient: _finish_auth 0 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe0f4001830 con 0x7fe10c09fba0 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient: dump: 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe1118e8640 5 monclient: authenticate success, global_id 5724 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe1118e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe0f4001990 con 0x7fe10c09fba0 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 1 -- 172.21.15.45:0/4239183089 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe0f4004ba0 con 0x7fe10c09fba0 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe1118e8640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe1118e8640 4 set_mon_vals no callback set 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.625+0000 7fe109ffb640 10 monclient: dump: 2024-09-18T14:10:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.639 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.626+0000 7fe1118e8640 10 monclient: shutdown 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.626+0000 7fe1118e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.626+0000 7fe1118e8640 1 -- 172.21.15.45:0/4239183089 >> v1:172.21.15.45:6789/0 conn(0x7fe10c09fba0 legacy=0x7fe10c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.626+0000 7fe1118e8640 1 -- 172.21.15.45:0/4239183089 shutdown_connections 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.626+0000 7fe1118e8640 1 -- 172.21.15.45:0/4239183089 >> 172.21.15.45:0/4239183089 conn(0x7fe10c0630b0 msgr2=0x7fe10c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.626+0000 7fe1118e8640 1 -- 172.21.15.45:0/4239183089 shutdown_connections 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.626+0000 7fe1118e8640 1 -- 172.21.15.45:0/4239183089 wait complete. 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 10 monclient: build_initial_monmap 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 10 monclient: monmap: 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 1 Processor -- start 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 1 -- start start 2024-09-18T14:10:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 10 monclient: init 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 10 monclient: _add_conns ranks=[1,7,5,3,6,8,2,4,0] 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 10 monclient(hunting): picked mon.noname-f con 0x7fe10c09f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 10 monclient(hunting): picked mon.noname-i con 0x7fe10c09fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 10 monclient(hunting): picked mon.noname-h con 0x7fe10c09ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10c09baf0 con 0x7fe10c09f7d0 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10c11ea10 con 0x7fe10c09ff70 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10c11ecf0 con 0x7fe10c09fba0 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe1118e8640 10 monclient(hunting): authenticate will time out at 1421.966865s 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.627+0000 7fe10affd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe10c09f7d0 0x7fe10c1318f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:47144/0 (socket says 172.21.15.45:47144) 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe10affd640 1 -- 172.21.15.45:0/1637968764 learned_addr learned my addr 172.21.15.45:0/1637968764 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779965401 0 0) 0x7fe10c09baf0 con 0x7fe10c09f7d0 2024-09-18T14:10:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): my global_id is 164422 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0ec0034e0 con 0x7fe10c09f7d0 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941616377 0 0) 0x7fe10c11ea10 con 0x7fe10c09ff70 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): my global_id is 184253 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10c09baf0 con 0x7fe10c09ff70 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186452752 0 0) 0x7fe10c11ecf0 con 0x7fe10c09fba0 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): my global_id is 184255 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10c11ea10 con 0x7fe10c09fba0 2024-09-18T14:10:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3533374565 0 0) 0x7fe10c09baf0 con 0x7fe10c09ff70 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe10c11ecf0 con 0x7fe10c09ff70 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe104001980 con 0x7fe10c09ff70 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): dump: 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2197316883 0 0) 0x7fe0ec0034e0 con 0x7fe10c09f7d0 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe10c09baf0 con 0x7fe10c09f7d0 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe0fc001f60 con 0x7fe10c09f7d0 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): dump: 2024-09-18T14:10:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 576002950 0 0) 0x7fe10c11ea10 con 0x7fe10c09fba0 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0ec0034e0 con 0x7fe10c09fba0 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe0f4001770 con 0x7fe10c09fba0 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient(hunting): dump: 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4162919504 0 0) 0x7fe10c11ecf0 con 0x7fe10c09ff70 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 >> v1:172.21.15.84:6792/0 conn(0x7fe10c09fba0 legacy=0x7fe10c146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 >> v1:172.21.15.84:6789/0 conn(0x7fe10c09f7d0 legacy=0x7fe10c1318f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:10:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 monclient: found mon.h 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c11dab0 con 0x7fe10c09ff70 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 10 monclient: _finish_auth 0 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe102ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe1118e8640 5 monclient: authenticate success, global_id 184253 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe1118e8640 10 monclient: _renew_subs 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe1118e8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe1118e8640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10c11d510 con 0x7fe10c09ff70 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe1118e8640 10 monclient: _renew_subs 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe1118e8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.628+0000 7fe1118e8640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10c1321a0 con 0x7fe10c09ff70 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.629+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe104001f10 con 0x7fe10c09ff70 2024-09-18T14:10:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.629+0000 7fe102ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.629+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe104002500 con 0x7fe10c09ff70 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.629+0000 7fe102ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.629+0000 7fe1097fa640 4 set_mon_vals no callback set 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.629+0000 7fe102ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.629+0000 7fe102ffd640 10 monclient: dump: 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.629+0000 7fe1118e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.629+0000 7fe1118e8640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe0f0006580 con 0x7fe0f0002830 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.629+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fe1040026e0 con 0x7fe10c09ff70 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 8 ==== osd_map(531..531 src has 344..531) v4 ==== 4003+0+0 (unknown 1057628888 0 0) 0x7fe104044230 con 0x7fe10c09ff70 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302085095 0 0) 0x7fe0f0006580 con 0x7fe0f0002830 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 10 monclient(hunting): my global_id is 184262 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10c11ecf0 con 0x7fe0f0002830 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe10a7fc640 10 monclient: get_auth_request con 0x7fe0ec03aa40 auth_method 0 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2768033659 0 0) 0x7fe10c11ecf0 con 0x7fe0f0002830 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0f0006580 con 0x7fe0f0002830 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe0fc0028b0 con 0x7fe0f0002830 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 10 monclient: dump: 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3024646296 0 0) 0x7fe0f0006580 con 0x7fe0f0002830 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0f000a0a0 con 0x7fe0f0002830 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe0fc005b00 con 0x7fe0f0002830 2024-09-18T14:10:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:06.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:06.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 >> v1:172.21.15.84:6791/0 conn(0x7fe0f0002830 legacy=0x7fe0f0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.630+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 >> v1:172.21.15.84:6791/0 conn(0x7fe0f0002830 legacy=0x7fe0f0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.636+0000 7fe1118e8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.636+0000 7fe1118e8640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe0f00095f0 con 0x7fe0f0008ff0 2024-09-18T14:10:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178999201 0 0) 0x7fe0f00095f0 con 0x7fe0f0008ff0 2024-09-18T14:10:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 10 monclient(hunting): my global_id is 184271 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f0006580 con 0x7fe0f0008ff0 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2523398081 0 0) 0x7fe0f0006580 con 0x7fe0f0008ff0 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0f00095f0 con 0x7fe0f0008ff0 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe0fc003130 con 0x7fe0f0008ff0 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 10 monclient: dump: 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2476070065 0 0) 0x7fe0f00095f0 con 0x7fe0f0008ff0 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.637+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe0f00098b0 con 0x7fe0f0008ff0 2024-09-18T14:10:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 4246883328) 0x7fe0fc007cf0 con 0x7fe0f0008ff0 2024-09-18T14:10:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe102ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe102ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 >> v1:172.21.15.84:6791/0 conn(0x7fe0f0008ff0 legacy=0x7fe0f0008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe102ffd640 1 -- 172.21.15.45:0/1637968764 >> v1:172.21.15.84:6791/0 conn(0x7fe0f0008ff0 legacy=0x7fe0f0008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:06.658 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:06.658 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:06.658 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:06.658 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:06.658 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:06.658 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:06.658 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:06.659 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.660 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:06.661 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.662 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.665 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.665 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:06.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:06.665 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.665 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.665 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:06.668 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.669 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.671 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.672 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.673 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:10:06.676 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.677 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.677 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:06.677 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:06.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe1118e8640 1 -- 172.21.15.45:0/1637968764 >> v1:172.21.15.84:6812/31478 conn(0x7fe0ec03aa40 legacy=0x7fe0ec03cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe1118e8640 10 monclient: shutdown 2024-09-18T14:10:06.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe1118e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:06.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe1118e8640 1 -- 172.21.15.45:0/1637968764 >> v1:172.21.15.84:6791/0 conn(0x7fe10c09ff70 legacy=0x7fe10c14a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe10b7fe640 1 -- 172.21.15.45:0/1637968764 reap_dead start 2024-09-18T14:10:06.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe10b7fe640 1 -- 172.21.15.45:0/1637968764 reap_dead start 2024-09-18T14:10:06.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe1118e8640 1 -- 172.21.15.45:0/1637968764 shutdown_connections 2024-09-18T14:10:06.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe1118e8640 1 -- 172.21.15.45:0/1637968764 >> 172.21.15.45:0/1637968764 conn(0x7fe10c0630b0 msgr2=0x7fe10c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:06.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe1118e8640 1 -- 172.21.15.45:0/1637968764 shutdown_connections 2024-09-18T14:10:06.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.638+0000 7fe1118e8640 1 -- 172.21.15.45:0/1637968764 wait complete. 2024-09-18T14:10:06.679 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:06.679 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:10:06.679 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:10:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.774+0000 7f0e6355d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.774+0000 7f0e6355d640 10 monclient: build_initial_monmap 2024-09-18T14:10:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.775+0000 7f0e6355d640 10 monclient: monmap: 2024-09-18T14:10:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:06.776932+0000 2024-09-18T14:10:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:06.776932+0000 2024-09-18T14:10:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:06.778 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.775+0000 7f0e6355d640 1 Processor -- start 2024-09-18T14:10:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.775+0000 7f0e6355d640 1 -- start start 2024-09-18T14:10:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.775+0000 7f0e6355d640 10 monclient: init 2024-09-18T14:10:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e6355d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e6355d640 10 monclient: _add_conns ranks=[5,2,6,4,0,3,7,1,8] 2024-09-18T14:10:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e6355d640 10 monclient(hunting): picked mon.noname-h con 0x7f0e5c09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e6355d640 10 monclient(hunting): picked mon.noname-b con 0x7f0e5c09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e6355d640 10 monclient(hunting): picked mon.noname-d con 0x7f0e5c09ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e6355d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5c05ed70 con 0x7f0e5c09fba0 2024-09-18T14:10:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e6355d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5c09dfc0 con 0x7f0e5c09f7d0 2024-09-18T14:10:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e6355d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5c09dce0 con 0x7f0e5c09ff70 2024-09-18T14:10:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e6355d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e6355d640 10 monclient(hunting): authenticate will time out at 1422.115287s 2024-09-18T14:10:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e61afa640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f0e5c09ff70 0x7f0e5c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:34900/0 (socket says 172.21.15.45:34900) 2024-09-18T14:10:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e61afa640 1 -- 172.21.15.45:0/3432440630 learned_addr learned my addr 172.21.15.45:0/3432440630 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456236514 0 0) 0x7f0e5c05ed70 con 0x7f0e5c09fba0 2024-09-18T14:10:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient(hunting): my global_id is 144974 2024-09-18T14:10:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e480034e0 con 0x7f0e5c09fba0 2024-09-18T14:10:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916114545 0 0) 0x7f0e5c09dce0 con 0x7f0e5c09ff70 2024-09-18T14:10:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient(hunting): my global_id is 184272 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5c05ed70 con 0x7f0e5c09ff70 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762279513 0 0) 0x7f0e5c09dfc0 con 0x7f0e5c09f7d0 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient(hunting): my global_id is 184280 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5c09dce0 con 0x7f0e5c09f7d0 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 75600924 0 0) 0x7f0e5c05ed70 con 0x7f0e5c09ff70 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 >> v1:172.21.15.84:6791/0 conn(0x7f0e5c09f7d0 legacy=0x7f0e5c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 >> v1:172.21.15.45:6790/0 conn(0x7f0e5c09fba0 legacy=0x7f0e5c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient: _finish_hunting 0 2024-09-18T14:10:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 1 monclient: found mon.noname-d 2024-09-18T14:10:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:10:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c0a1c00 con 0x7f0e5c09ff70 2024-09-18T14:10:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient: _finish_auth 0 2024-09-18T14:10:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e6355d640 5 monclient: authenticate success, global_id 184272 2024-09-18T14:10:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e54001b10 con 0x7f0e5c09ff70 2024-09-18T14:10:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e6355d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.776+0000 7f0e4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e4ffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:10:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e4ffff640 10 monclient: dump: 2024-09-18T14:10:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.792 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e6355d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:06.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0e54001c70 con 0x7f0e5c09ff70 2024-09-18T14:10:06.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e4ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:06.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e6355d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:06.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e4ffff640 1 -- 172.21.15.45:0/3432440630 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e54004be0 con 0x7f0e5c09ff70 2024-09-18T14:10:06.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e6355d640 4 set_mon_vals no callback set 2024-09-18T14:10:06.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e4ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:06.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e4ffff640 10 monclient: dump: 2024-09-18T14:10:06.792 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.793 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.793 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.793 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.793 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.793 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.793 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.793 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.793 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.793 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.793 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.793 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.793 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e6355d640 10 monclient: shutdown 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e6355d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e6355d640 1 -- 172.21.15.45:0/3432440630 >> v1:172.21.15.45:6792/0 conn(0x7f0e5c09ff70 legacy=0x7f0e5c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e6355d640 1 -- 172.21.15.45:0/3432440630 shutdown_connections 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e6355d640 1 -- 172.21.15.45:0/3432440630 >> 172.21.15.45:0/3432440630 conn(0x7f0e5c0630b0 msgr2=0x7f0e5c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e6355d640 1 -- 172.21.15.45:0/3432440630 shutdown_connections 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.777+0000 7f0e6355d640 1 -- 172.21.15.45:0/3432440630 wait complete. 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.778+0000 7f0e6355d640 10 monclient: build_initial_monmap 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.778+0000 7f0e6355d640 10 monclient: monmap: 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.778+0000 7f0e6355d640 1 Processor -- start 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.778+0000 7f0e6355d640 1 -- start start 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.778+0000 7f0e6355d640 10 monclient: init 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e6355d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e6355d640 10 monclient: _add_conns ranks=[3,8,2,1,6,4,0,7,5] 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e6355d640 10 monclient(hunting): picked mon.noname-g con 0x7f0e5c09f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e6355d640 10 monclient(hunting): picked mon.noname-e con 0x7f0e5c09fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e6355d640 10 monclient(hunting): picked mon.noname-b con 0x7f0e5c09ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e6355d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5c1208b0 con 0x7f0e5c09ff70 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e6355d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5c121ee0 con 0x7f0e5c09f7d0 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e6355d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5c09d310 con 0x7f0e5c09fba0 2024-09-18T14:10:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e6355d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e6355d640 10 monclient(hunting): authenticate will time out at 1422.118273s 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e60af8640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f0e5c09fba0 0x7f0e5c131ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:44950/0 (socket says 172.21.15.45:44950) 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e61afa640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f0e5c09ff70 0x7f0e5c149c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:34000/0 (socket says 172.21.15.45:34000) 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e60af8640 1 -- 172.21.15.45:0/3463549464 learned_addr learned my addr 172.21.15.45:0/3463549464 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709173791 0 0) 0x7f0e5c1208b0 con 0x7f0e5c09ff70 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): my global_id is 144983 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e400034e0 con 0x7f0e5c09ff70 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644651042 0 0) 0x7f0e5c09d310 con 0x7f0e5c09fba0 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): my global_id is 5561 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5c1208b0 con 0x7f0e5c09fba0 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285507595 0 0) 0x7f0e5c121ee0 con 0x7f0e5c09f7d0 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): my global_id is 184251 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5c09d310 con 0x7f0e5c09f7d0 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 502716555 0 0) 0x7f0e400034e0 con 0x7f0e5c09ff70 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e5c121ee0 con 0x7f0e5c09ff70 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e54001a50 con 0x7f0e5c09ff70 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): dump: 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.799 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 291652501 0 0) 0x7f0e5c1208b0 con 0x7f0e5c09fba0 2024-09-18T14:10:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e400034e0 con 0x7f0e5c09fba0 2024-09-18T14:10:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e58001e30 con 0x7f0e5c09fba0 2024-09-18T14:10:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): dump: 2024-09-18T14:10:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.801 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.801 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.801 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.801 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.801 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.801 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.801 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.801 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.801 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.801 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 818712471 0 0) 0x7f0e5c09d310 con 0x7f0e5c09f7d0 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e5c1208b0 con 0x7f0e5c09f7d0 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e50001aa0 con 0x7f0e5c09f7d0 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.779+0000 7f0e4d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 10 monclient(hunting): dump: 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.802 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1669390995 0 0) 0x7f0e5c121ee0 con 0x7f0e5c09ff70 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 >> v1:172.21.15.45:6793/0 conn(0x7f0e5c09fba0 legacy=0x7f0e5c131ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 >> v1:172.21.15.84:6790/0 conn(0x7f0e5c09f7d0 legacy=0x7f0e5c1418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 10 monclient: _finish_hunting 0 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 1 monclient: found mon.b 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c142750 con 0x7f0e5c09ff70 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 10 monclient: _finish_auth 0 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e6355d640 5 monclient: authenticate success, global_id 144983 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e6355d640 10 monclient: _renew_subs 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e6355d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e6355d640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c121c00 con 0x7f0e5c09ff70 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e6355d640 10 monclient: _renew_subs 2024-09-18T14:10:06.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e6355d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:06.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e6355d640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5c1200d0 con 0x7f0e5c09ff70 2024-09-18T14:10:06.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0e54004dd0 con 0x7f0e5c09ff70 2024-09-18T14:10:06.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:06.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e54005380 con 0x7f0e5c09ff70 2024-09-18T14:10:06.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4f7fe640 4 set_mon_vals no callback set 2024-09-18T14:10:06.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:06.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 10 monclient: dump: 2024-09-18T14:10:06.805 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.805 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.805 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.805 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e6355d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:06.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e6355d640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0e3c006580 con 0x7f0e3c002830 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.780+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f0e54005560 con 0x7f0e5c09ff70 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(531..531 src has 344..531) v4 ==== 4003+0+0 (unknown 1057628888 0 0) 0x7f0e54046e60 con 0x7f0e5c09ff70 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e60af8640 10 monclient: get_auth_request con 0x7f0e4003a990 auth_method 0 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261317672 0 0) 0x7f0e3c006580 con 0x7f0e3c002830 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 10 monclient(hunting): my global_id is 184264 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5c121ee0 con 0x7f0e3c002830 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3929550893 0 0) 0x7f0e5c121ee0 con 0x7f0e3c002830 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e3c006580 con 0x7f0e3c002830 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e50002230 con 0x7f0e3c002830 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 10 monclient: dump: 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.808 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.809 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3389487852 0 0) 0x7f0e3c006580 con 0x7f0e3c002830 2024-09-18T14:10:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.781+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0e3c00a0a0 con 0x7f0e3c002830 2024-09-18T14:10:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.782+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0e50002680 con 0x7f0e3c002830 2024-09-18T14:10:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.782+0000 7f0e4d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.782+0000 7f0e4d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.782+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 >> v1:172.21.15.84:6792/0 conn(0x7f0e3c002830 legacy=0x7f0e3c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.782+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 >> v1:172.21.15.84:6792/0 conn(0x7f0e3c002830 legacy=0x7f0e3c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.787+0000 7f0e6355d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.787+0000 7f0e6355d640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0e3c007510 con 0x7f0e3c0068c0 2024-09-18T14:10:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226608960 0 0) 0x7f0e3c007510 con 0x7f0e3c0068c0 2024-09-18T14:10:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 10 monclient(hunting): my global_id is 184273 2024-09-18T14:10:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e3c006580 con 0x7f0e3c0068c0 2024-09-18T14:10:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 147109082 0 0) 0x7f0e3c006580 con 0x7f0e3c0068c0 2024-09-18T14:10:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e3c007510 con 0x7f0e3c0068c0 2024-09-18T14:10:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0e50003000 con 0x7f0e3c0068c0 2024-09-18T14:10:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 10 monclient: dump: 2024-09-18T14:10:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:06.812 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 980004849 0 0) 0x7f0e3c007510 con 0x7f0e3c0068c0 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.788+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0e3c008750 con 0x7f0e3c0068c0 2024-09-18T14:10:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7386 (unknown 0 0 650882862) 0x7f0e50003310 con 0x7f0e3c0068c0 2024-09-18T14:10:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e4d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e4d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 >> v1:172.21.15.84:6792/0 conn(0x7f0e3c0068c0 legacy=0x7f0e3c006d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e4d7fa640 1 -- 172.21.15.45:0/3463549464 >> v1:172.21.15.84:6792/0 conn(0x7f0e3c0068c0 legacy=0x7f0e3c006d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:06.814 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:06.814 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:06.814 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:06.814 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:06.814 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 92, 2024-09-18T14:10:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:10:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.816 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:06.818 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.820 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:06.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.820 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:06.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.820 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:06.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:06.820 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.820 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.821 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:06.821 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.821 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.822 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:06.822 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:06.822 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.822 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.822 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.823 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:06.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:06.823 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.823 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.823 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:06.823 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.823 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.826 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:06.827 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:06.828 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.829 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.829 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:06.829 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.829 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.829 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.829 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:06.829 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:06.829 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.829 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.830 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.830 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:06.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:06.830 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.830 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.831 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:06.831 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.831 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.831 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.831 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.832 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:06.833 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.833 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.833 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:06.833 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:06.833 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.833 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.833 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.833 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:06.834 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:06.835 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:06.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:06.836 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:06.836 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:06.836 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:06.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:06.836 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:06.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:06.836 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:06.836 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e6355d640 1 -- 172.21.15.45:0/3463549464 >> v1:172.21.15.84:6812/31478 conn(0x7f0e4003a990 legacy=0x7f0e4003ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e6355d640 10 monclient: shutdown 2024-09-18T14:10:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e6355d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e6355d640 1 -- 172.21.15.45:0/3463549464 >> v1:172.21.15.45:6790/0 conn(0x7f0e5c09ff70 legacy=0x7f0e5c149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e61afa640 1 -- 172.21.15.45:0/3463549464 reap_dead start 2024-09-18T14:10:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e61afa640 1 -- 172.21.15.45:0/3463549464 reap_dead start 2024-09-18T14:10:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e6355d640 1 -- 172.21.15.45:0/3463549464 shutdown_connections 2024-09-18T14:10:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e6355d640 1 -- 172.21.15.45:0/3463549464 >> 172.21.15.45:0/3463549464 conn(0x7f0e5c0630b0 msgr2=0x7f0e5c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e6355d640 1 -- 172.21.15.45:0/3463549464 shutdown_connections 2024-09-18T14:10:06.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:06.789+0000 7f0e6355d640 1 -- 172.21.15.45:0/3463549464 wait complete. 2024-09-18T14:10:06.839 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:10:06.839 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['f'] 2024-09-18T14:10:06.840 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a', 'b'] 2024-09-18T14:10:06.840 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-18T14:10:06.840 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-18T14:10:06.840 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-18T14:10:06.840 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:10:06.853 INFO:tasks.ceph.mon.f:Stopped 2024-09-18T14:10:06.853 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-09-18T14:10:06.854 INFO:tasks.ceph.mon.a:Sent signal 19 2024-09-18T14:10:06.854 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-18T14:10:06.854 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-18T14:10:06.854 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-18T14:10:21.855 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-09-18T14:10:21.856 INFO:tasks.ceph.mon.a:Sent signal 18 2024-09-18T14:10:21.856 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-18T14:10:21.856 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-18T14:10:21.856 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-09-18T14:10:21.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:10:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.966+0000 7f68deeaf640 10 monclient: get_monmap_and_config 2024-09-18T14:10:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.966+0000 7f68deeaf640 10 monclient: build_initial_monmap 2024-09-18T14:10:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.966+0000 7f68deeaf640 10 monclient: monmap: 2024-09-18T14:10:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:21.967651+0000 2024-09-18T14:10:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:21.967651+0000 2024-09-18T14:10:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:21.968 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.967+0000 7f68deeaf640 1 Processor -- start 2024-09-18T14:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.967+0000 7f68deeaf640 1 -- start start 2024-09-18T14:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.967+0000 7f68deeaf640 10 monclient: init 2024-09-18T14:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.968+0000 7f68deeaf640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.968+0000 7f68deeaf640 10 monclient: _add_conns ranks=[3,7,8,0,2,4,5,1,6] 2024-09-18T14:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.968+0000 7f68deeaf640 10 monclient(hunting): picked mon.noname-g con 0x7f68d809f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.968+0000 7f68deeaf640 10 monclient(hunting): picked mon.noname-i con 0x7f68d809fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.968+0000 7f68deeaf640 10 monclient(hunting): picked mon.noname-e con 0x7f68d809ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.968+0000 7f68deeaf640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d805ed70 con 0x7f68d809f7d0 2024-09-18T14:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.968+0000 7f68deeaf640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d809dfc0 con 0x7f68d809fba0 2024-09-18T14:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.968+0000 7f68deeaf640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d809dce0 con 0x7f68d809ff70 2024-09-18T14:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.968+0000 7f68deeaf640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.968+0000 7f68deeaf640 10 monclient(hunting): authenticate will time out at 1437.306148s 2024-09-18T14:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.968+0000 7f68dd44c640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f68d809ff70 0x7f68d813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:44364/0 (socket says 172.21.15.45:44364) 2024-09-18T14:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:21.968+0000 7f68dd44c640 1 -- 172.21.15.45:0/579377113 learned_addr learned my addr 172.21.15.45:0/579377113 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:24.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 10 monclient(hunting): tick 2024-09-18T14:10:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 1 monclient(hunting): continuing hunt 2024-09-18T14:10:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:10:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 1 -- 172.21.15.45:0/579377113 >> v1:172.21.15.45:6793/0 conn(0x7f68d809ff70 legacy=0x7f68d813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 1 -- 172.21.15.45:0/579377113 >> v1:172.21.15.84:6792/0 conn(0x7f68d809fba0 legacy=0x7f68d8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 1 -- 172.21.15.45:0/579377113 >> v1:172.21.15.84:6790/0 conn(0x7f68d809f7d0 legacy=0x7f68d8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 10 monclient: _add_conns ranks=[5,2,6,4,1,0] 2024-09-18T14:10:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 10 monclient(hunting): picked mon.noname-h con 0x7f68bc0037a0 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 10 monclient(hunting): picked mon.noname-b con 0x7f68bc007380 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 10 monclient(hunting): picked mon.noname-d con 0x7f68bc00b100 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 1 -- 172.21.15.45:0/579377113 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d80a1c00 con 0x7f68bc007380 2024-09-18T14:10:24.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 1 -- 172.21.15.45:0/579377113 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc00ffa0 con 0x7f68bc0037a0 2024-09-18T14:10:24.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 1 -- 172.21.15.45:0/579377113 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc0111e0 con 0x7f68bc00b100 2024-09-18T14:10:24.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:24.968+0000 7f68d67fc640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:26.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.866+0000 7f68d77fe640 1 -- 172.21.15.45:0/579377113 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504288294 0 0) 0x7f68bc00ffa0 con 0x7f68bc0037a0 2024-09-18T14:10:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 10 monclient(hunting): my global_id is 184289 2024-09-18T14:10:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 1 -- 172.21.15.45:0/579377113 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68c80034e0 con 0x7f68bc0037a0 2024-09-18T14:10:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 1 -- 172.21.15.45:0/579377113 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 436974389 0 0) 0x7f68c80034e0 con 0x7f68bc0037a0 2024-09-18T14:10:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 1 -- 172.21.15.45:0/579377113 >> v1:172.21.15.45:6792/0 conn(0x7f68bc00b100 legacy=0x7f68bc00d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 1 -- 172.21.15.45:0/579377113 >> v1:172.21.15.45:6790/0 conn(0x7f68bc007380 legacy=0x7f68bc009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 10 monclient: _finish_hunting 0 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 1 monclient: found mon.noname-h 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 1 -- 172.21.15.45:0/579377113 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc012420 con 0x7f68bc0037a0 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 10 monclient: _finish_auth 0 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68dd44c640 1 -- 172.21.15.45:0/579377113 reap_dead start 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68deeaf640 5 monclient: authenticate success, global_id 184289 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68deeaf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 1 -- 172.21.15.45:0/579377113 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68cc001830 con 0x7f68bc0037a0 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:10:26.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68d77fe640 10 monclient: dump: 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.867+0000 7f68deeaf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.868+0000 7f68d77fe640 1 -- 172.21.15.45:0/579377113 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f68cc001a00 con 0x7f68bc0037a0 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.868+0000 7f68d77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.868+0000 7f68d77fe640 1 -- 172.21.15.45:0/579377113 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68cc001ff0 con 0x7f68bc0037a0 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.868+0000 7f68d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.868+0000 7f68d77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.868+0000 7f68d77fe640 10 monclient: dump: 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:26.875 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.868+0000 7f68deeaf640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.868+0000 7f68deeaf640 4 set_mon_vals no callback set 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.869+0000 7f68deeaf640 10 monclient: shutdown 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.869+0000 7f68deeaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.869+0000 7f68deeaf640 1 -- 172.21.15.45:0/579377113 >> v1:172.21.15.84:6791/0 conn(0x7f68bc0037a0 legacy=0x7f68bc005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.869+0000 7f68deeaf640 1 -- 172.21.15.45:0/579377113 shutdown_connections 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.869+0000 7f68deeaf640 1 -- 172.21.15.45:0/579377113 >> 172.21.15.45:0/579377113 conn(0x7f68d80630b0 msgr2=0x7f68d80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.869+0000 7f68deeaf640 1 -- 172.21.15.45:0/579377113 shutdown_connections 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.869+0000 7f68deeaf640 1 -- 172.21.15.45:0/579377113 wait complete. 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.870+0000 7f68deeaf640 10 monclient: build_initial_monmap 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.870+0000 7f68deeaf640 10 monclient: monmap: 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:26.878 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.870+0000 7f68deeaf640 1 Processor -- start 2024-09-18T14:10:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.871+0000 7f68deeaf640 1 -- start start 2024-09-18T14:10:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.871+0000 7f68deeaf640 10 monclient: init 2024-09-18T14:10:26.879 INFO:tasks.rados.rados.0.smithi084.stdout:2433: done (3 left) 2024-09-18T14:10:26.879 INFO:tasks.rados.rados.0.smithi084.stdout:2434: rollback oid 12 current snap is 256 2024-09-18T14:10:26.879 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 12 to 255 2024-09-18T14:10:26.879 INFO:tasks.rados.rados.0.smithi084.stdout:2429: finishing rollback tid 0 to smithi08434663-40 2024-09-18T14:10:26.879 INFO:tasks.rados.rados.0.smithi084.stdout:2429: finishing rollback tid 1 to smithi08434663-40 2024-09-18T14:10:26.879 INFO:tasks.rados.rados.0.smithi084.stdout:2429: finishing rollback tid 2 to smithi08434663-40 2024-09-18T14:10:26.879 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 898 (ObjNum 838 snap 252 seq_num 838) dirty exists 2024-09-18T14:10:26.879 INFO:tasks.rados.rados.0.smithi084.stdout:2431: expect (ObjNum 818 snap 250 seq_num 818) 2024-09-18T14:10:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.871+0000 7f68deeaf640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.871+0000 7f68deeaf640 10 monclient: _add_conns ranks=[2,0,6,3,4,8,5,1,7] 2024-09-18T14:10:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.871+0000 7f68deeaf640 10 monclient(hunting): picked mon.noname-b con 0x7f68bc0037a0 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.872+0000 7f68deeaf640 10 monclient(hunting): picked mon.noname-a con 0x7f68d809b310 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.872+0000 7f68deeaf640 10 monclient(hunting): picked mon.noname-d con 0x7f68d81485e0 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.872+0000 7f68deeaf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d809b700 con 0x7f68d809b310 2024-09-18T14:10:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.872+0000 7f68deeaf640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d811e930 con 0x7f68bc0037a0 2024-09-18T14:10:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.872+0000 7f68deeaf640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d81429f0 con 0x7f68d81485e0 2024-09-18T14:10:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.872+0000 7f68deeaf640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.872+0000 7f68deeaf640 10 monclient(hunting): authenticate will time out at 1442.210116s 2024-09-18T14:10:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.872+0000 7f68dcc4b640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f68bc0037a0 0x7f68d8131810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:56236/0 (socket says 172.21.15.45:56236) 2024-09-18T14:10:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.872+0000 7f68dcc4b640 1 -- 172.21.15.45:0/2926518916 learned_addr learned my addr 172.21.15.45:0/2926518916 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:26.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262163803 0 0) 0x7f68d811e930 con 0x7f68bc0037a0 2024-09-18T14:10:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): my global_id is 144992 2024-09-18T14:10:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68bc005380 con 0x7f68bc0037a0 2024-09-18T14:10:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211985367 0 0) 0x7f68d81429f0 con 0x7f68d81485e0 2024-09-18T14:10:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): my global_id is 184290 2024-09-18T14:10:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d811e930 con 0x7f68d81485e0 2024-09-18T14:10:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602387872 0 0) 0x7f68d809b700 con 0x7f68d809b310 2024-09-18T14:10:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:26.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): my global_id is 5733 2024-09-18T14:10:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d81429f0 con 0x7f68d809b310 2024-09-18T14:10:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 527900561 0 0) 0x7f68bc005380 con 0x7f68bc0037a0 2024-09-18T14:10:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68d809b700 con 0x7f68bc0037a0 2024-09-18T14:10:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68cc001eb0 con 0x7f68bc0037a0 2024-09-18T14:10:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:10:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): dump: 2024-09-18T14:10:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:26.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1746442031 0 0) 0x7f68d811e930 con 0x7f68d81485e0 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68bc005380 con 0x7f68d81485e0 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3404455066 0 0) 0x7f68d81429f0 con 0x7f68d809b310 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68d811e930 con 0x7f68d809b310 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68d0001d60 con 0x7f68d81485e0 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): dump: 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68c4001a60 con 0x7f68d809b310 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.885+0000 7f68d4ff9640 10 monclient(hunting): dump: 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2591686568 0 0) 0x7f68d809b700 con 0x7f68bc0037a0 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 >> v1:172.21.15.45:6792/0 conn(0x7f68d81485e0 legacy=0x7f68d814a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 >> v1:172.21.15.45:6789/0 conn(0x7f68d809b310 legacy=0x7f68d8146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 1 monclient: found mon.b 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d811ec10 con 0x7f68bc0037a0 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 10 monclient: _finish_auth 0 2024-09-18T14:10:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68deeaf640 5 monclient: authenticate success, global_id 144992 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68deeaf640 10 monclient: _renew_subs 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68deeaf640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68deeaf640 1 -- 172.21.15.45:0/2926518916 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d811eef0 con 0x7f68bc0037a0 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68deeaf640 10 monclient: _renew_subs 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68deeaf640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68deeaf640 1 -- 172.21.15.45:0/2926518916 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d81320c0 con 0x7f68bc0037a0 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f68cc002340 con 0x7f68bc0037a0 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68cc0028b0 con 0x7f68bc0037a0 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:26.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d6ffd640 4 set_mon_vals no callback set 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.886+0000 7f68d4ff9640 10 monclient: dump: 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:26.897 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.887+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f68cc010240 con 0x7f68bc0037a0 2024-09-18T14:10:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.887+0000 7f68deeaf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.887+0000 7f68deeaf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.887+0000 7f68deeaf640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.887+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(532..532 src has 344..532) v4 ==== 4047+0+0 (unknown 3301093510 0 0) 0x7f68cc046be0 con 0x7f68bc0037a0 2024-09-18T14:10:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.887+0000 7f68deeaf640 1 -- 172.21.15.45:0/2926518916 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b8005010 con 0x7f68bc0037a0 2024-09-18T14:10:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.887+0000 7f68d7fff640 10 monclient: get_auth_request con 0x7f68bc03bfe0 auth_method 0 2024-09-18T14:10:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.892+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f68cc014fe0 con 0x7f68bc0037a0 2024-09-18T14:10:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.892+0000 7f68d4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:26.892+0000 7f68d4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:27.001 INFO:tasks.rados.rados.0.smithi084.stdout:2434: finishing rollback tid 0 to smithi08434663-12 2024-09-18T14:10:27.001 INFO:tasks.rados.rados.0.smithi084.stdout:2434: finishing rollback tid 1 to smithi08434663-12 2024-09-18T14:10:27.001 INFO:tasks.rados.rados.0.smithi084.stdout:2434: finishing rollback tid 2 to smithi08434663-12 2024-09-18T14:10:27.001 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 1205 (ObjNum 697 snap 205 seq_num 697) dirty exists 2024-09-18T14:10:27.002 INFO:tasks.rados.rados.0.smithi084.stdout:2429: done (3 left) 2024-09-18T14:10:27.002 INFO:tasks.rados.rados.0.smithi084.stdout:2431: done (2 left) 2024-09-18T14:10:27.002 INFO:tasks.rados.rados.0.smithi084.stdout:2432: done (1 left) 2024-09-18T14:10:27.002 INFO:tasks.rados.rados.0.smithi084.stdout:2434: done (0 left) 2024-09-18T14:10:27.002 INFO:tasks.rados.rados.0.smithi084.stdout:2435: delete oid 5 current snap is 256 2024-09-18T14:10:27.003 INFO:tasks.rados.rados.0.smithi084.stdout:2435: done (0 left) 2024-09-18T14:10:27.003 INFO:tasks.rados.rados.0.smithi084.stdout:2436: copy_from oid 33 from oid 14 current snap is 256 2024-09-18T14:10:27.003 INFO:tasks.rados.rados.0.smithi084.stdout:2437: delete oid 32 current snap is 256 2024-09-18T14:10:27.005 INFO:tasks.rados.rados.0.smithi084.stdout:2436: finishing copy_from to smithi08434663-33 2024-09-18T14:10:27.005 INFO:tasks.rados.rados.0.smithi084.stdout:2436: got expected ENOENT (src dne) 2024-09-18T14:10:27.005 INFO:tasks.rados.rados.0.smithi084.stdout:2436: finishing copy_from racing read to smithi08434663-33 2024-09-18T14:10:27.006 INFO:tasks.rados.rados.0.smithi084.stdout:2436: done (1 left) 2024-09-18T14:10:27.006 INFO:tasks.rados.rados.0.smithi084.stdout:2437: done (0 left) 2024-09-18T14:10:27.006 INFO:tasks.rados.rados.0.smithi084.stdout:2438: rollback oid 19 current snap is 256 2024-09-18T14:10:27.006 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 19 to 255 2024-09-18T14:10:27.006 INFO:tasks.rados.rados.0.smithi084.stdout:2439: rollback oid 23 current snap is 256 2024-09-18T14:10:27.006 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 23 to 255 2024-09-18T14:10:27.006 INFO:tasks.rados.rados.0.smithi084.stdout:2440: read oid 3 snap -1 2024-09-18T14:10:27.006 INFO:tasks.rados.rados.0.smithi084.stdout:2440: expect (ObjNum 825 snap 251 seq_num 825) 2024-09-18T14:10:27.006 INFO:tasks.rados.rados.0.smithi084.stdout:2441: delete oid 8 current snap is 256 2024-09-18T14:10:27.010 INFO:tasks.rados.rados.0.smithi084.stdout:2438: finishing rollback tid 0 to smithi08434663-19 2024-09-18T14:10:27.010 INFO:tasks.rados.rados.0.smithi084.stdout:2438: finishing rollback tid 1 to smithi08434663-19 2024-09-18T14:10:27.010 INFO:tasks.rados.rados.0.smithi084.stdout:2441: done (3 left) 2024-09-18T14:10:27.011 INFO:tasks.rados.rados.0.smithi084.stdout:2442: write oid 7 current snap is 256 2024-09-18T14:10:27.011 INFO:tasks.rados.rados.0.smithi084.stdout:2442: seq_num 856 ranges {786792=644680,1857998=528794} 2024-09-18T14:10:27.018 INFO:tasks.rados.rados.0.smithi084.stdout:2442: writing smithi08434663-7 from 786792 to 1431472 tid 1 2024-09-18T14:10:27.024 INFO:tasks.rados.rados.0.smithi084.stdout:2442: writing smithi08434663-7 from 1857998 to 2386792 tid 2 2024-09-18T14:10:27.024 INFO:tasks.rados.rados.0.smithi084.stdout:2443: read oid 32 snap -1 2024-09-18T14:10:27.024 INFO:tasks.rados.rados.0.smithi084.stdout:2443: expect deleted 2024-09-18T14:10:27.024 INFO:tasks.rados.rados.0.smithi084.stdout:2444: read oid 28 snap -1 2024-09-18T14:10:27.024 INFO:tasks.rados.rados.0.smithi084.stdout:2444: expect (ObjNum 765 snap 223 seq_num 765) 2024-09-18T14:10:27.024 INFO:tasks.rados.rados.0.smithi084.stdout:2445: snap_create 2024-09-18T14:10:27.024 INFO:tasks.rados.rados.0.smithi084.stdout:2438: finishing rollback tid 2 to smithi08434663-19 2024-09-18T14:10:27.024 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1211 (ObjNum 851 snap 255 seq_num 851) dirty exists 2024-09-18T14:10:27.024 INFO:tasks.rados.rados.0.smithi084.stdout:2439: finishing rollback tid 0 to smithi08434663-23 2024-09-18T14:10:27.024 INFO:tasks.rados.rados.0.smithi084.stdout:2439: finishing rollback tid 1 to smithi08434663-23 2024-09-18T14:10:27.025 INFO:tasks.rados.rados.0.smithi084.stdout:2440: expect (ObjNum 825 snap 251 seq_num 825) 2024-09-18T14:10:27.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.132+0000 7f68deeaf640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:10:27.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.132+0000 7f68deeaf640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:10:27.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.132+0000 7f68deeaf640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:10:27.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.132+0000 7f68deeaf640 1 -- 172.21.15.45:0/2926518916 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f68b8005840 con 0x7f68bc0037a0 2024-09-18T14:10:27.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.132+0000 7f68d4ff9640 1 -- 172.21.15.45:0/2926518916 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2682 (unknown 254746657 0 1480168316) 0x7f68cc014e00 con 0x7f68bc0037a0 2024-09-18T14:10:27.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.132+0000 7f68d4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:10:27.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.132+0000 7f68d4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:27.133 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":98,"quorum":[0,2,3,4,5,6,7,8],"quorum_names":["a","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:10:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.134+0000 7f68deeaf640 1 -- 172.21.15.45:0/2926518916 >> v1:172.21.15.84:6812/31478 conn(0x7f68bc03bfe0 legacy=0x7f68bc03e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.134+0000 7f68deeaf640 10 monclient: shutdown 2024-09-18T14:10:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.134+0000 7f68deeaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.134+0000 7f68deeaf640 1 -- 172.21.15.45:0/2926518916 >> v1:172.21.15.45:6790/0 conn(0x7f68bc0037a0 legacy=0x7f68d8131810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.134+0000 7f68deeaf640 1 -- 172.21.15.45:0/2926518916 shutdown_connections 2024-09-18T14:10:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.134+0000 7f68deeaf640 1 -- 172.21.15.45:0/2926518916 >> 172.21.15.45:0/2926518916 conn(0x7f68d80630b0 msgr2=0x7f68d80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.134+0000 7f68deeaf640 1 -- 172.21.15.45:0/2926518916 shutdown_connections 2024-09-18T14:10:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.134+0000 7f68deeaf640 1 -- 172.21.15.45:0/2926518916 wait complete. 2024-09-18T14:10:27.145 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-09-18T14:10:27.145 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-09-18T14:10:27.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:10:27.235 INFO:tasks.rados.rados.0.smithi084.stdout:2439: finishing rollback tid 2 to smithi08434663-23 2024-09-18T14:10:27.235 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 957 (ObjNum 827 snap 251 seq_num 827) dirty exists 2024-09-18T14:10:27.235 INFO:tasks.rados.rados.0.smithi084.stdout:2442: finishing write tid 1 to smithi08434663-7 2024-09-18T14:10:27.235 INFO:tasks.rados.rados.0.smithi084.stdout:2442: finishing write tid 2 to smithi08434663-7 2024-09-18T14:10:27.235 INFO:tasks.rados.rados.0.smithi084.stdout:2442: finishing write tid 3 to smithi08434663-7 2024-09-18T14:10:27.235 INFO:tasks.rados.rados.0.smithi084.stdout:2442: finishing write tid 4 to smithi08434663-7 2024-09-18T14:10:27.236 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 960 (ObjNum 856 snap 256 seq_num 856) dirty exists 2024-09-18T14:10:27.236 INFO:tasks.rados.rados.0.smithi084.stdout:2442: left oid 7 (ObjNum 856 snap 256 seq_num 856) 2024-09-18T14:10:27.236 INFO:tasks.rados.rados.0.smithi084.stdout:2444: expect (ObjNum 765 snap 223 seq_num 765) 2024-09-18T14:10:27.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.242+0000 7eff2875d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.242+0000 7eff2875d640 10 monclient: build_initial_monmap 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.242+0000 7eff2875d640 10 monclient: monmap: 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:27.243411+0000 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:27.243411+0000 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:27.255 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 1 Processor -- start 2024-09-18T14:10:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 1 -- start start 2024-09-18T14:10:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 10 monclient: init 2024-09-18T14:10:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 10 monclient: _add_conns ranks=[8,1,0,3,2,6,7,5,4] 2024-09-18T14:10:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 10 monclient(hunting): picked mon.noname-e con 0x7eff2009f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 10 monclient(hunting): picked mon.noname-f con 0x7eff2009fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 10 monclient(hunting): picked mon.noname-a con 0x7eff2009ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2005ed70 con 0x7eff2009ff70 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2009dfc0 con 0x7eff2009fba0 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2009dce0 con 0x7eff2009f7d0 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff2875d640 10 monclient(hunting): authenticate will time out at 1442.581736s 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff25cf8640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7eff2009fba0 legacy=0x7eff20137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff264f9640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7eff2009f7d0 0x7eff20132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:44374/0 (socket says 172.21.15.45:44374) 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff264f9640 1 -- 172.21.15.45:0/954215242 learned_addr learned my addr 172.21.15.45:0/954215242 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.243+0000 7eff26cfa640 1 --1- 172.21.15.45:0/954215242 >> v1:172.21.15.45:6789/0 conn(0x7eff2009ff70 0x7eff2013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60868/0 (socket says 172.21.15.45:60868) 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 1 -- 172.21.15.45:0/954215242 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859666105 0 0) 0x7eff2005ed70 con 0x7eff2009ff70 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient(hunting): my global_id is 5742 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 1 -- 172.21.15.45:0/954215242 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff0c0034e0 con 0x7eff2009ff70 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 1 -- 172.21.15.45:0/954215242 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446901885 0 0) 0x7eff2009dce0 con 0x7eff2009f7d0 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient(hunting): my global_id is 5570 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 1 -- 172.21.15.45:0/954215242 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff2005ed70 con 0x7eff2009f7d0 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 1 -- 172.21.15.45:0/954215242 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 584967480 0 0) 0x7eff0c0034e0 con 0x7eff2009ff70 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 1 -- 172.21.15.45:0/954215242 >> v1:172.21.15.45:6793/0 conn(0x7eff2009f7d0 legacy=0x7eff20132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 1 -- 172.21.15.45:0/954215242 >> v1:172.21.15.84:6789/0 conn(0x7eff2009fba0 legacy=0x7eff20137a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient: _finish_hunting 0 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 1 monclient: found mon.noname-a 2024-09-18T14:10:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 1 -- 172.21.15.45:0/954215242 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff200a1c00 con 0x7eff2009ff70 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient: _finish_auth 0 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff2875d640 5 monclient: authenticate success, global_id 5742 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 1 -- 172.21.15.45:0/954215242 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7eff18001bb0 con 0x7eff2009ff70 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff2875d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient: dump: 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.258 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff2875d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:27.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 1 -- 172.21.15.45:0/954215242 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7eff18001d90 con 0x7eff2009ff70 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff2875d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 1 -- 172.21.15.45:0/954215242 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7eff18004e60 con 0x7eff2009ff70 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff2875d640 4 set_mon_vals no callback set 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff254f7640 10 monclient: dump: 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff2875d640 10 monclient: shutdown 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff2875d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:27.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.244+0000 7eff2875d640 1 -- 172.21.15.45:0/954215242 >> v1:172.21.15.45:6789/0 conn(0x7eff2009ff70 legacy=0x7eff2013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.245+0000 7eff2875d640 1 -- 172.21.15.45:0/954215242 shutdown_connections 2024-09-18T14:10:27.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.245+0000 7eff2875d640 1 -- 172.21.15.45:0/954215242 >> 172.21.15.45:0/954215242 conn(0x7eff200630b0 msgr2=0x7eff200634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:27.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.245+0000 7eff2875d640 1 -- 172.21.15.45:0/954215242 shutdown_connections 2024-09-18T14:10:27.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.245+0000 7eff2875d640 1 -- 172.21.15.45:0/954215242 wait complete. 2024-09-18T14:10:27.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.245+0000 7eff2875d640 10 monclient: build_initial_monmap 2024-09-18T14:10:27.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.245+0000 7eff2875d640 10 monclient: monmap: 2024-09-18T14:10:27.262 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:27.262 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:27.262 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:27.262 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.245+0000 7eff2875d640 1 Processor -- start 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.245+0000 7eff2875d640 1 -- start start 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.245+0000 7eff2875d640 10 monclient: init 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff2875d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff2875d640 10 monclient: _add_conns ranks=[3,2,0,6,4,1,5,8,7] 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff2875d640 10 monclient(hunting): picked mon.noname-g con 0x7eff2009f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff2875d640 10 monclient(hunting): picked mon.noname-b con 0x7eff2009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff2875d640 10 monclient(hunting): picked mon.noname-a con 0x7eff2009ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff2875d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff20120920 con 0x7eff2009ff70 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff2875d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2009d310 con 0x7eff2009fba0 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff2875d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff200a2560 con 0x7eff2009f7d0 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff2875d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff2875d640 10 monclient(hunting): authenticate will time out at 1442.584435s 2024-09-18T14:10:27.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff26cfa640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7eff2009ff70 0x7eff20149c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60884/0 (socket says 172.21.15.45:60884) 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff26cfa640 1 -- 172.21.15.45:0/2331274647 learned_addr learned my addr 172.21.15.45:0/2331274647 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934481685 0 0) 0x7eff200a1c00 con 0x7eff2009ff70 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 10 monclient(hunting): my global_id is 5751 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff040034e0 con 0x7eff2009ff70 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372112943 0 0) 0x7eff2009d310 con 0x7eff2009fba0 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 10 monclient(hunting): my global_id is 145001 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff200a1c00 con 0x7eff2009fba0 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009753595 0 0) 0x7eff200a2560 con 0x7eff2009f7d0 2024-09-18T14:10:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 10 monclient(hunting): my global_id is 184260 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff2009d310 con 0x7eff2009f7d0 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3828974484 0 0) 0x7eff040034e0 con 0x7eff2009ff70 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7eff200a2560 con 0x7eff2009ff70 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.246+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 630983342 0 0) 0x7eff200a1c00 con 0x7eff2009fba0 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7eff040034e0 con 0x7eff2009fba0 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7eff18001bf0 con 0x7eff2009ff70 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient(hunting): dump: 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.267 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7eff10004220 con 0x7eff2009fba0 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient(hunting): dump: 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 58072875 0 0) 0x7eff2009d310 con 0x7eff2009f7d0 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7eff200a1c00 con 0x7eff2009f7d0 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7eff1c001ae0 con 0x7eff2009f7d0 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient(hunting): dump: 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2500487124 0 0) 0x7eff200a2560 con 0x7eff2009ff70 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 >> v1:172.21.15.84:6790/0 conn(0x7eff2009f7d0 legacy=0x7eff20141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 >> v1:172.21.15.45:6790/0 conn(0x7eff2009fba0 legacy=0x7eff20131b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient: _finish_hunting 0 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 monclient: found mon.a 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff201427e0 con 0x7eff2009ff70 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient: _finish_auth 0 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff2875d640 5 monclient: authenticate success, global_id 5751 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff2875d640 10 monclient: _renew_subs 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff2875d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff2875d640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff20121c60 con 0x7eff2009ff70 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff2875d640 10 monclient: _renew_subs 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff2875d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff2875d640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7eff20120d00 con 0x7eff2009ff70 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7eff18004f40 con 0x7eff2009ff70 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7eff180053f0 con 0x7eff2009ff70 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff24cf6640 4 set_mon_vals no callback set 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.247+0000 7eff167fc640 10 monclient: dump: 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff2875d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff2875d640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7eff08006580 con 0x7eff08002830 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7eff18005550 con 0x7eff2009ff70 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(532..532 src has 344..532) v4 ==== 4047+0+0 (unknown 3301093510 0 0) 0x7eff18046d60 con 0x7eff2009ff70 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623578779 0 0) 0x7eff08006580 con 0x7eff08002830 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 10 monclient(hunting): my global_id is 5760 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff200a2560 con 0x7eff08002830 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff25cf8640 10 monclient: get_auth_request con 0x7eff0403a8f0 auth_method 0 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1106170871 0 0) 0x7eff200a2560 con 0x7eff08002830 2024-09-18T14:10:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7eff08006580 con 0x7eff08002830 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7eff1c004ae0 con 0x7eff08002830 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 10 monclient: dump: 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1389334582 0 0) 0x7eff08006580 con 0x7eff08002830 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.248+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff0800a0a0 con 0x7eff08002830 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.249+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7eff1c005890 con 0x7eff08002830 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.249+0000 7eff167fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.249+0000 7eff167fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.249+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 >> v1:172.21.15.45:6789/0 conn(0x7eff08002830 legacy=0x7eff08004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.249+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 >> v1:172.21.15.45:6789/0 conn(0x7eff08002830 legacy=0x7eff08004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff2875d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff2875d640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7eff08009610 con 0x7eff08008ff0 2024-09-18T14:10:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040483577 0 0) 0x7eff08009610 con 0x7eff08008ff0 2024-09-18T14:10:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 10 monclient(hunting): my global_id is 5769 2024-09-18T14:10:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff08006580 con 0x7eff08008ff0 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 338849751 0 0) 0x7eff08006580 con 0x7eff08008ff0 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7eff08009610 con 0x7eff08008ff0 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7eff1c005550 con 0x7eff08008ff0 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 10 monclient: dump: 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 807662706 0 0) 0x7eff08009610 con 0x7eff08008ff0 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7eff080098d0 con 0x7eff08008ff0 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7545 (unknown 0 0 3286880641) 0x7eff1c005b20 con 0x7eff08008ff0 2024-09-18T14:10:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:27.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:27.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 >> v1:172.21.15.45:6789/0 conn(0x7eff08008ff0 legacy=0x7eff08008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.255+0000 7eff167fc640 1 -- 172.21.15.45:0/2331274647 >> v1:172.21.15.45:6789/0 conn(0x7eff08008ff0 legacy=0x7eff08008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:27.281 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 98, 2024-09-18T14:10:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 5 2024-09-18T14:10:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:27.300 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:27.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.256+0000 7eff2875d640 1 -- 172.21.15.45:0/2331274647 >> v1:172.21.15.84:6812/31478 conn(0x7eff0403a8f0 legacy=0x7eff0403cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.256+0000 7eff2875d640 10 monclient: shutdown 2024-09-18T14:10:27.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.256+0000 7eff2875d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:27.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.256+0000 7eff2875d640 1 -- 172.21.15.45:0/2331274647 >> v1:172.21.15.45:6789/0 conn(0x7eff2009ff70 legacy=0x7eff20149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.256+0000 7eff26cfa640 1 -- 172.21.15.45:0/2331274647 reap_dead start 2024-09-18T14:10:27.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.256+0000 7eff26cfa640 1 -- 172.21.15.45:0/2331274647 reap_dead start 2024-09-18T14:10:27.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.256+0000 7eff2875d640 1 -- 172.21.15.45:0/2331274647 shutdown_connections 2024-09-18T14:10:27.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.256+0000 7eff2875d640 1 -- 172.21.15.45:0/2331274647 >> 172.21.15.45:0/2331274647 conn(0x7eff200630b0 msgr2=0x7eff200a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:27.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.256+0000 7eff2875d640 1 -- 172.21.15.45:0/2331274647 shutdown_connections 2024-09-18T14:10:27.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.256+0000 7eff2875d640 1 -- 172.21.15.45:0/2331274647 wait complete. 2024-09-18T14:10:27.302 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:10:27.302 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-09-18T14:10:27.302 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:10:27.335 INFO:tasks.rados.rados.0.smithi084.stdout:2438: done (6 left) 2024-09-18T14:10:27.336 INFO:tasks.rados.rados.0.smithi084.stdout:2439: done (5 left) 2024-09-18T14:10:27.336 INFO:tasks.rados.rados.0.smithi084.stdout:2440: done (4 left) 2024-09-18T14:10:27.336 INFO:tasks.rados.rados.0.smithi084.stdout:2442: done (3 left) 2024-09-18T14:10:27.336 INFO:tasks.rados.rados.0.smithi084.stdout:2443: done (2 left) 2024-09-18T14:10:27.336 INFO:tasks.rados.rados.0.smithi084.stdout:2444: done (1 left) 2024-09-18T14:10:27.336 INFO:tasks.rados.rados.0.smithi084.stdout:2445: done (0 left) 2024-09-18T14:10:27.336 INFO:tasks.rados.rados.0.smithi084.stdout:2446: copy_from oid 20 from oid 12 current snap is 257 2024-09-18T14:10:27.336 INFO:tasks.rados.rados.0.smithi084.stdout:2447: read oid 31 snap -1 2024-09-18T14:10:27.336 INFO:tasks.rados.rados.0.smithi084.stdout:2447: expect deleted 2024-09-18T14:10:27.336 INFO:tasks.rados.rados.0.smithi084.stdout:2448: read oid 35 snap -1 2024-09-18T14:10:27.336 INFO:tasks.rados.rados.0.smithi084.stdout:2448: expect (ObjNum 843 snap 255 seq_num 843) 2024-09-18T14:10:27.336 INFO:tasks.rados.rados.0.smithi084.stdout:2449: snap_remove snap 256 2024-09-18T14:10:27.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.404+0000 7fe7b782b640 10 monclient: get_monmap_and_config 2024-09-18T14:10:27.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.404+0000 7fe7b782b640 10 monclient: build_initial_monmap 2024-09-18T14:10:27.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.404+0000 7fe7b782b640 10 monclient: monmap: 2024-09-18T14:10:27.404 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:27.404 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.404 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:27.405073+0000 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:27.405073+0000 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:27.405 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.404+0000 7fe7b782b640 1 Processor -- start 2024-09-18T14:10:27.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.404+0000 7fe7b782b640 1 -- start start 2024-09-18T14:10:27.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.404+0000 7fe7b782b640 10 monclient: init 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b782b640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b782b640 10 monclient: _add_conns ranks=[4,0,1,5,8,7,2,6,3] 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b782b640 10 monclient(hunting): picked mon.noname-c con 0x7fe7b006e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b782b640 10 monclient(hunting): picked mon.noname-a con 0x7fe7b00a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b782b640 10 monclient(hunting): picked mon.noname-f con 0x7fe7b00a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b782b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b005ed70 con 0x7fe7b00a0cb0 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b782b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b0097620 con 0x7fe7b00a1080 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b782b640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b0097340 con 0x7fe7b006e810 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b782b640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b782b640 10 monclient(hunting): authenticate will time out at 1442.743475s 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b55c7640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fe7b006e810 0x7fe7b00a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:47674/0 (socket says 172.21.15.45:47674) 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b55c7640 1 -- 172.21.15.45:0/3186440085 learned_addr learned my addr 172.21.15.45:0/3186440085 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:27.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7b5dc8640 1 -- 172.21.15.45:0/3186440085 >> v1:172.21.15.84:6789/0 conn(0x7fe7b00a1080 legacy=0x7fe7b013af40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7a7fff640 1 -- 172.21.15.45:0/3186440085 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807504155 0 0) 0x7fe7b0097340 con 0x7fe7b006e810 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7a7fff640 10 monclient(hunting): my global_id is 164461 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7a7fff640 1 -- 172.21.15.45:0/3186440085 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe79c0034e0 con 0x7fe7b006e810 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7a7fff640 1 -- 172.21.15.45:0/3186440085 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599638597 0 0) 0x7fe7b005ed70 con 0x7fe7b00a0cb0 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7a7fff640 10 monclient(hunting): my global_id is 5778 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7a7fff640 1 -- 172.21.15.45:0/3186440085 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b0097340 con 0x7fe7b00a0cb0 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.405+0000 7fe7a7fff640 1 -- 172.21.15.45:0/3186440085 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3042138066 0 0) 0x7fe79c0034e0 con 0x7fe7b006e810 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 1 -- 172.21.15.45:0/3186440085 >> v1:172.21.15.84:6789/0 conn(0x7fe7b00a1080 legacy=0x7fe7b013af40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:10:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 1 -- 172.21.15.45:0/3186440085 >> v1:172.21.15.45:6789/0 conn(0x7fe7b00a0cb0 legacy=0x7fe7b0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 10 monclient: _finish_hunting 0 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 1 monclient: found mon.noname-c 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 1 -- 172.21.15.45:0/3186440085 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b00a1660 con 0x7fe7b006e810 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 10 monclient: _finish_auth 0 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 1 -- 172.21.15.45:0/3186440085 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe7ac001760 con 0x7fe7b006e810 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7b782b640 5 monclient: authenticate success, global_id 164461 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7b782b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 10 monclient: dump: 2024-09-18T14:10:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.419 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7b782b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:27.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 1 -- 172.21.15.45:0/3186440085 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe7ac0018c0 con 0x7fe7b006e810 2024-09-18T14:10:27.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:27.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 1 -- 172.21.15.45:0/3186440085 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe7ac004860 con 0x7fe7b006e810 2024-09-18T14:10:27.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7b782b640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:27.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7b782b640 4 set_mon_vals no callback set 2024-09-18T14:10:27.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:27.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7a7fff640 10 monclient: dump: 2024-09-18T14:10:27.420 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.420 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.420 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.420 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.420 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.420 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.420 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.420 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.420 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.420 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7b782b640 10 monclient: shutdown 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7b782b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7b782b640 1 -- 172.21.15.45:0/3186440085 >> v1:172.21.15.45:6791/0 conn(0x7fe7b006e810 legacy=0x7fe7b00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7b782b640 1 -- 172.21.15.45:0/3186440085 shutdown_connections 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7b782b640 1 -- 172.21.15.45:0/3186440085 >> 172.21.15.45:0/3186440085 conn(0x7fe7b00a5bb0 msgr2=0x7fe7b00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7b782b640 1 -- 172.21.15.45:0/3186440085 shutdown_connections 2024-09-18T14:10:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.406+0000 7fe7b782b640 1 -- 172.21.15.45:0/3186440085 wait complete. 2024-09-18T14:10:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.407+0000 7fe7b782b640 10 monclient: build_initial_monmap 2024-09-18T14:10:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.407+0000 7fe7b782b640 10 monclient: monmap: 2024-09-18T14:10:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.407+0000 7fe7b782b640 1 Processor -- start 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.407+0000 7fe7b782b640 1 -- start start 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.407+0000 7fe7b782b640 10 monclient: init 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7b782b640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7b782b640 10 monclient: _add_conns ranks=[0,5,3,8,1,6,7,4,2] 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7b782b640 10 monclient(hunting): picked mon.noname-a con 0x7fe7b006e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7b782b640 10 monclient(hunting): picked mon.noname-h con 0x7fe7b00a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7b782b640 10 monclient(hunting): picked mon.noname-g con 0x7fe7b00a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7b782b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b00a2270 con 0x7fe7b006e810 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7b782b640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b013da30 con 0x7fe7b00a1080 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7b782b640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b0120940 con 0x7fe7b00a0cb0 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7b782b640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7b55c7640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe7b006e810 0x7fe7b013d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60926/0 (socket says 172.21.15.45:60926) 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7b55c7640 1 -- 172.21.15.45:0/1605274599 learned_addr learned my addr 172.21.15.45:0/1605274599 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7b782b640 10 monclient(hunting): authenticate will time out at 1442.746587s 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037704482 0 0) 0x7fe7b00a2270 con 0x7fe7b006e810 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7a57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7a57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7a57fa640 10 monclient(hunting): my global_id is 5787 2024-09-18T14:10:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7940034e0 con 0x7fe7b006e810 2024-09-18T14:10:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167141323 0 0) 0x7fe7b0120940 con 0x7fe7b00a0cb0 2024-09-18T14:10:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7a57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7a57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.408+0000 7fe7a57fa640 10 monclient(hunting): my global_id is 184298 2024-09-18T14:10:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b00a2270 con 0x7fe7b00a0cb0 2024-09-18T14:10:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866702659 0 0) 0x7fe7b013da30 con 0x7fe7b00a1080 2024-09-18T14:10:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient(hunting): my global_id is 184269 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b0120940 con 0x7fe7b00a1080 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3229074684 0 0) 0x7fe7940034e0 con 0x7fe7b006e810 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe7b013da30 con 0x7fe7b006e810 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe7ac001720 con 0x7fe7b006e810 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient(hunting): dump: 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2044044761 0 0) 0x7fe7b00a2270 con 0x7fe7b00a0cb0 2024-09-18T14:10:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe7940034e0 con 0x7fe7b00a0cb0 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe7a0001ad0 con 0x7fe7b00a0cb0 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient(hunting): dump: 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3387752948 0 0) 0x7fe7b0120940 con 0x7fe7b00a1080 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe7b00a2270 con 0x7fe7b00a1080 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4144212586 0 0) 0x7fe7b013da30 con 0x7fe7b006e810 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 >> v1:172.21.15.84:6791/0 conn(0x7fe7b00a0cb0 legacy=0x7fe7b0146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 >> v1:172.21.15.84:6790/0 conn(0x7fe7b00a1080 legacy=0x7fe7b0149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient: _finish_hunting 0 2024-09-18T14:10:27.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 monclient: found mon.a 2024-09-18T14:10:27.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:27.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:27.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b013e7d0 con 0x7fe7b006e810 2024-09-18T14:10:27.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient: _finish_auth 0 2024-09-18T14:10:27.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:27.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7b782b640 5 monclient: authenticate success, global_id 5787 2024-09-18T14:10:27.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7b782b640 10 monclient: _renew_subs 2024-09-18T14:10:27.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7b782b640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:27.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7b782b640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b01223c0 con 0x7fe7b006e810 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7b782b640 10 monclient: _renew_subs 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7b782b640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe7ac0048c0 con 0x7fe7b006e810 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7b782b640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b0120060 con 0x7fe7b006e810 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe7ac0050d0 con 0x7fe7b006e810 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a77fe640 4 set_mon_vals no callback set 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.409+0000 7fe7a57fa640 10 monclient: dump: 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.431 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.432 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.410+0000 7fe7b782b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.410+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fe7ac0052b0 con 0x7fe7b006e810 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.410+0000 7fe7b782b640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe7b004c4a0 con 0x7fe7b01214a0 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.410+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(533..533 src has 344..533) v4 ==== 3959+0+0 (unknown 756524739 0 0) 0x7fe7ac046d10 con 0x7fe7b006e810 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.410+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937032979 0 0) 0x7fe7b004c4a0 con 0x7fe7b01214a0 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.410+0000 7fe7a57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.410+0000 7fe7a57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.410+0000 7fe7a57fa640 10 monclient(hunting): my global_id is 145010 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.410+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b013da30 con 0x7fe7b01214a0 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.410+0000 7fe7b5dc8640 10 monclient: get_auth_request con 0x7fe79403a8a0 auth_method 0 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.410+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 938367144 0 0) 0x7fe7b013da30 con 0x7fe7b01214a0 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.411+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe7b004c4a0 con 0x7fe7b01214a0 2024-09-18T14:10:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.411+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe7a0002220 con 0x7fe7b01214a0 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.411+0000 7fe7a57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.411+0000 7fe7a57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.411+0000 7fe7a57fa640 10 monclient: dump: 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.411+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 378999629 0 0) 0x7fe7b004c4a0 con 0x7fe7b01214a0 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.411+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7b004b720 con 0x7fe7b01214a0 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.411+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe7a00044a0 con 0x7fe7b01214a0 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.411+0000 7fe7a57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.411+0000 7fe7a57fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.411+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 >> v1:172.21.15.45:6790/0 conn(0x7fe7b01214a0 legacy=0x7fe7b009b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.411+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 >> v1:172.21.15.45:6790/0 conn(0x7fe7b01214a0 legacy=0x7fe7b009b860 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7b782b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7b782b640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe7b013e040 con 0x7fe7b013dc70 2024-09-18T14:10:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092522387 0 0) 0x7fe7b013e040 con 0x7fe7b013dc70 2024-09-18T14:10:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 10 monclient(hunting): my global_id is 145019 2024-09-18T14:10:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b004c4a0 con 0x7fe7b013dc70 2024-09-18T14:10:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3203759087 0 0) 0x7fe7b004c4a0 con 0x7fe7b013dc70 2024-09-18T14:10:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe7b013e040 con 0x7fe7b013dc70 2024-09-18T14:10:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe7a0002c30 con 0x7fe7b013dc70 2024-09-18T14:10:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 10 monclient: dump: 2024-09-18T14:10:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 154247435 0 0) 0x7fe7b013e040 con 0x7fe7b013dc70 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.417+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe7b0062dd0 con 0x7fe7b013dc70 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7545 (unknown 0 0 1412708438) 0x7fe7a0004730 con 0x7fe7b013dc70 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7a57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7a57fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:27.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 >> v1:172.21.15.45:6790/0 conn(0x7fe7b013dc70 legacy=0x7fe7b009bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7a57fa640 1 -- 172.21.15.45:0/1605274599 >> v1:172.21.15.45:6790/0 conn(0x7fe7b013dc70 legacy=0x7fe7b009bf70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 98, 2024-09-18T14:10:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:10:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:27.463 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7b782b640 1 -- 172.21.15.45:0/1605274599 >> v1:172.21.15.84:6812/31478 conn(0x7fe79403a8a0 legacy=0x7fe79403cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7b782b640 10 monclient: shutdown 2024-09-18T14:10:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7b782b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7b782b640 1 -- 172.21.15.45:0/1605274599 >> v1:172.21.15.45:6789/0 conn(0x7fe7b006e810 legacy=0x7fe7b013d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7b5dc8640 1 -- 172.21.15.45:0/1605274599 reap_dead start 2024-09-18T14:10:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7b5dc8640 1 -- 172.21.15.45:0/1605274599 reap_dead start 2024-09-18T14:10:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7b782b640 1 -- 172.21.15.45:0/1605274599 shutdown_connections 2024-09-18T14:10:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7b782b640 1 -- 172.21.15.45:0/1605274599 >> 172.21.15.45:0/1605274599 conn(0x7fe7b00a5bb0 msgr2=0x7fe7b00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7b782b640 1 -- 172.21.15.45:0/1605274599 shutdown_connections 2024-09-18T14:10:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.418+0000 7fe7b782b640 1 -- 172.21.15.45:0/1605274599 wait complete. 2024-09-18T14:10:27.466 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:10:27.466 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-09-18T14:10:27.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:10:27.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.563+0000 7f3fcda46640 10 monclient: get_monmap_and_config 2024-09-18T14:10:27.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.563+0000 7f3fcda46640 10 monclient: build_initial_monmap 2024-09-18T14:10:27.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.563+0000 7f3fcda46640 10 monclient: monmap: 2024-09-18T14:10:27.564 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:27.564 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.564 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:27.564133+0000 2024-09-18T14:10:27.564 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:27.564133+0000 2024-09-18T14:10:27.564 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:27.564 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.564 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:27.564 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:27.564 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:27.564 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:27.565 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:27.565 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:27.565 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:27.565 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:27.565 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:27.565 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.563+0000 7f3fcda46640 1 Processor -- start 2024-09-18T14:10:27.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.563+0000 7f3fcda46640 1 -- start start 2024-09-18T14:10:27.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.563+0000 7f3fcda46640 10 monclient: init 2024-09-18T14:10:27.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fcda46640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:27.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fcda46640 10 monclient: _add_conns ranks=[7,6,4,2,0,8,3,1,5] 2024-09-18T14:10:27.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fcda46640 10 monclient(hunting): picked mon.noname-i con 0x7f3fc806e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:27.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fcda46640 10 monclient(hunting): picked mon.noname-d con 0x7f3fc80a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:27.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fcda46640 10 monclient(hunting): picked mon.noname-c con 0x7f3fc80a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:27.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fcda46640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc805ed70 con 0x7f3fc80a1080 2024-09-18T14:10:27.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fcda46640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc8097620 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fcda46640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc8097340 con 0x7f3fc806e810 2024-09-18T14:10:27.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fcda46640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:27.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fcda46640 10 monclient(hunting): authenticate will time out at 1442.902323s 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc67fc640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f3fc80a0cb0 0x7f3fc8137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:59694/0 (socket says 172.21.15.45:59694) 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc67fc640 1 -- 172.21.15.45:0/997241034 learned_addr learned my addr 172.21.15.45:0/997241034 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749737623 0 0) 0x7f3fc805ed70 con 0x7f3fc80a1080 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 10 monclient(hunting): my global_id is 164470 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fb00034e0 con 0x7f3fc80a1080 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607386272 0 0) 0x7f3fc8097620 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 10 monclient(hunting): my global_id is 184299 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc805ed70 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2718042780 0 0) 0x7f3fc8097340 con 0x7f3fc806e810 2024-09-18T14:10:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 10 monclient(hunting): my global_id is 184282 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc8097620 con 0x7f3fc806e810 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3330084771 0 0) 0x7f3fc805ed70 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 >> v1:172.21.15.84:6792/0 conn(0x7f3fc806e810 legacy=0x7f3fc80a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 >> v1:172.21.15.45:6791/0 conn(0x7f3fc80a1080 legacy=0x7f3fc813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 1 monclient: found mon.noname-d 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc80a1660 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.564+0000 7f3fc5ffb640 10 monclient: _finish_auth 0 2024-09-18T14:10:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fc5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3fbc001b90 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fcda46640 5 monclient: authenticate success, global_id 184299 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fcda46640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fc5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fc5ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fc5ffb640 10 monclient: dump: 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.575 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fcda46640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3fbc001cf0 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fc5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fc5ffb640 1 -- 172.21.15.45:0/997241034 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3fbc004be0 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fcda46640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fcda46640 4 set_mon_vals no callback set 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fc5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fc5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fc5ffb640 10 monclient: dump: 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fcda46640 10 monclient: shutdown 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fcda46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fcda46640 1 -- 172.21.15.45:0/997241034 >> v1:172.21.15.45:6792/0 conn(0x7f3fc80a0cb0 legacy=0x7f3fc8137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fcda46640 1 -- 172.21.15.45:0/997241034 shutdown_connections 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fcda46640 1 -- 172.21.15.45:0/997241034 >> 172.21.15.45:0/997241034 conn(0x7f3fc80a5bb0 msgr2=0x7f3fc80a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fcda46640 1 -- 172.21.15.45:0/997241034 shutdown_connections 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.565+0000 7f3fcda46640 1 -- 172.21.15.45:0/997241034 wait complete. 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.566+0000 7f3fcda46640 10 monclient: build_initial_monmap 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.566+0000 7f3fcda46640 10 monclient: monmap: 2024-09-18T14:10:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:27.579 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.566+0000 7f3fcda46640 1 Processor -- start 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.566+0000 7f3fcda46640 1 -- start start 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.566+0000 7f3fcda46640 10 monclient: init 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fcda46640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fcda46640 10 monclient: _add_conns ranks=[1,6,4,2,0,3,7,5,8] 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fcda46640 10 monclient(hunting): picked mon.noname-f con 0x7f3fc806e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fcda46640 10 monclient(hunting): picked mon.noname-d con 0x7f3fc80a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fcda46640 10 monclient(hunting): picked mon.noname-c con 0x7f3fc80a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fcda46640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc8122d10 con 0x7f3fc806e810 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fcda46640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc813db00 con 0x7f3fc80a1080 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fcda46640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc8122750 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fcda46640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fcda46640 10 monclient(hunting): authenticate will time out at 1442.905369s 2024-09-18T14:10:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fc6ffd640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7f3fc806e810 legacy=0x7f3fc813d3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:10:27.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fc67fc640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f3fc80a0cb0 0x7f3fc8146620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:59708/0 (socket says 172.21.15.45:59708) 2024-09-18T14:10:27.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3fc67fc640 1 -- 172.21.15.45:0/527475379 learned_addr learned my addr 172.21.15.45:0/527475379 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:27.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294292415 0 0) 0x7f3fc8122750 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): my global_id is 184308 2024-09-18T14:10:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fb40034e0 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157285982 0 0) 0x7f3fc813db00 con 0x7f3fc80a1080 2024-09-18T14:10:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): my global_id is 164479 2024-09-18T14:10:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc8122750 con 0x7f3fc80a1080 2024-09-18T14:10:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1337825635 0 0) 0x7f3fb40034e0 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3fc813db00 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3fbc001a50 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): dump: 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.587 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.587 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.587 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.587 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.587 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.587 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.587 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.587 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:27.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1010104139 0 0) 0x7f3fc8122750 con 0x7f3fc80a1080 2024-09-18T14:10:27.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3fb40034e0 con 0x7f3fc80a1080 2024-09-18T14:10:27.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3fc00017d0 con 0x7f3fc80a1080 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.567+0000 7f3faeffd640 10 monclient(hunting): dump: 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2825686780 0 0) 0x7f3fc813db00 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 >> v1:172.21.15.45:6791/0 conn(0x7f3fc80a1080 legacy=0x7f3fc8149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 >> v1:172.21.15.84:6789/0 conn(0x7f3fc806e810 legacy=0x7f3fc813d3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 10 monclient: _finish_hunting 0 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 1 monclient: found mon.d 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8120260 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 10 monclient: _finish_auth 0 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3fcda46640 5 monclient: authenticate success, global_id 184308 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3fcda46640 10 monclient: _renew_subs 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3fcda46640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3fcda46640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc8122470 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3fcda46640 10 monclient: _renew_subs 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3fcda46640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3fcda46640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc81215a0 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3fbc005110 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3fbc0055c0 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3fc57fa640 4 set_mon_vals no callback set 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3faeffd640 10 monclient: dump: 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.568+0000 7f3fcda46640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f3fbc010590 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3fcda46640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3fa8006580 con 0x7f3fa8002830 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(533..533 src has 344..533) v4 ==== 3959+0+0 (unknown 756524739 0 0) 0x7f3fbc0470f0 con 0x7f3fc80a0cb0 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077702815 0 0) 0x7f3fa8006580 con 0x7f3fa8002830 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 10 monclient(hunting): my global_id is 164488 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc813db00 con 0x7f3fa8002830 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3fc77fe640 10 monclient: get_auth_request con 0x7f3fb403a3e0 auth_method 0 2024-09-18T14:10:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4014809283 0 0) 0x7f3fc813db00 con 0x7f3fa8002830 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3fa8006580 con 0x7f3fa8002830 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3fb8002cb0 con 0x7f3fa8002830 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 10 monclient: dump: 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3646193871 0 0) 0x7f3fa8006580 con 0x7f3fa8002830 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.569+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3fa800a0a0 con 0x7f3fa8002830 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.570+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3fb8006450 con 0x7f3fa8002830 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.570+0000 7f3faeffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.570+0000 7f3faeffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.570+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 >> v1:172.21.15.45:6791/0 conn(0x7f3fa8002830 legacy=0x7f3fa8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.570+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 >> v1:172.21.15.45:6791/0 conn(0x7f3fa8002830 legacy=0x7f3fa8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.575+0000 7f3fcda46640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.575+0000 7f3fcda46640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3fa8007540 con 0x7f3fa80068f0 2024-09-18T14:10:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800350442 0 0) 0x7f3fa8007540 con 0x7f3fa80068f0 2024-09-18T14:10:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 10 monclient(hunting): my global_id is 164497 2024-09-18T14:10:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa8006580 con 0x7f3fa80068f0 2024-09-18T14:10:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2813591063 0 0) 0x7f3fa8006580 con 0x7f3fa80068f0 2024-09-18T14:10:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3fa8007540 con 0x7f3fa80068f0 2024-09-18T14:10:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3fb8004d10 con 0x7f3fa80068f0 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 10 monclient: dump: 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.598 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1795405504 0 0) 0x7f3fa8007540 con 0x7f3fa80068f0 2024-09-18T14:10:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3fa80087a0 con 0x7f3fa80068f0 2024-09-18T14:10:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7375 (unknown 0 0 3594826239) 0x7f3fb80065c0 con 0x7f3fa80068f0 2024-09-18T14:10:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 >> v1:172.21.15.45:6791/0 conn(0x7f3fa80068f0 legacy=0x7f3fa8006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.576+0000 7f3faeffd640 1 -- 172.21.15.45:0/527475379 >> v1:172.21.15.45:6791/0 conn(0x7f3fa80068f0 legacy=0x7f3fa8006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:27.599 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 98, 2024-09-18T14:10:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:27.618 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:27.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.577+0000 7f3fcda46640 1 -- 172.21.15.45:0/527475379 >> v1:172.21.15.84:6812/31478 conn(0x7f3fb403a3e0 legacy=0x7f3fb403c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.577+0000 7f3fcda46640 10 monclient: shutdown 2024-09-18T14:10:27.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.577+0000 7f3fcda46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:27.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.577+0000 7f3fcda46640 1 -- 172.21.15.45:0/527475379 >> v1:172.21.15.45:6792/0 conn(0x7f3fc80a0cb0 legacy=0x7f3fc8146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.577+0000 7f3fc77fe640 1 -- 172.21.15.45:0/527475379 reap_dead start 2024-09-18T14:10:27.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.577+0000 7f3fc77fe640 1 -- 172.21.15.45:0/527475379 reap_dead start 2024-09-18T14:10:27.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.577+0000 7f3fcda46640 1 -- 172.21.15.45:0/527475379 shutdown_connections 2024-09-18T14:10:27.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.577+0000 7f3fcda46640 1 -- 172.21.15.45:0/527475379 >> 172.21.15.45:0/527475379 conn(0x7f3fc80a5bb0 msgr2=0x7f3fc80a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:27.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.577+0000 7f3fcda46640 1 -- 172.21.15.45:0/527475379 shutdown_connections 2024-09-18T14:10:27.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.577+0000 7f3fcda46640 1 -- 172.21.15.45:0/527475379 wait complete. 2024-09-18T14:10:27.620 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:10:27.620 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-09-18T14:10:27.620 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:10:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.717+0000 7ffaa408b640 10 monclient: get_monmap_and_config 2024-09-18T14:10:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.717+0000 7ffaa408b640 10 monclient: build_initial_monmap 2024-09-18T14:10:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.717+0000 7ffaa408b640 10 monclient: monmap: 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:27.718128+0000 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:27.718128+0000 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:27.719 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.717+0000 7ffaa408b640 1 Processor -- start 2024-09-18T14:10:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.717+0000 7ffaa408b640 1 -- start start 2024-09-18T14:10:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.717+0000 7ffaa408b640 10 monclient: init 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa408b640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa408b640 10 monclient: _add_conns ranks=[7,1,8,0,3,2,5,6,4] 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa408b640 10 monclient(hunting): picked mon.noname-i con 0x7ffa9c0a2610 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa408b640 10 monclient(hunting): picked mon.noname-f con 0x7ffa9c1330b0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa408b640 10 monclient(hunting): picked mon.noname-e con 0x7ffa9c096a20 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa408b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c05ed70 con 0x7ffa9c1330b0 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa408b640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c0a1740 con 0x7ffa9c0a2610 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa408b640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c0a29e0 con 0x7ffa9c096a20 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa408b640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa408b640 10 monclient(hunting): authenticate will time out at 1443.056494s 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa1626640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7ffa9c1330b0 legacy=0x7ffa9c137810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa2628640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7ffa9c096a20 0x7ffa9c13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:48260/0 (socket says 172.21.15.45:48260) 2024-09-18T14:10:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa2628640 1 -- 172.21.15.45:0/2112407146 learned_addr learned my addr 172.21.15.45:0/2112407146 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa0e25640 1 -- 172.21.15.45:0/2112407146 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919656905 0 0) 0x7ffa9c0a29e0 con 0x7ffa9c096a20 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa0e25640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa0e25640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa0e25640 10 monclient(hunting): my global_id is 5579 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.718+0000 7ffaa0e25640 1 -- 172.21.15.45:0/2112407146 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa880034e0 con 0x7ffa9c096a20 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 1 -- 172.21.15.45:0/2112407146 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 22319334 0 0) 0x7ffa880034e0 con 0x7ffa9c096a20 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 1 -- 172.21.15.45:0/2112407146 >> v1:172.21.15.84:6792/0 conn(0x7ffa9c0a2610 legacy=0x7ffa9c1329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 1 -- 172.21.15.45:0/2112407146 >> v1:172.21.15.84:6789/0 conn(0x7ffa9c1330b0 legacy=0x7ffa9c137810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 10 monclient: _finish_hunting 0 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 1 monclient: found mon.noname-e 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 1 -- 172.21.15.45:0/2112407146 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c0970a0 con 0x7ffa9c096a20 2024-09-18T14:10:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 10 monclient: _finish_auth 0 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa408b640 5 monclient: authenticate success, global_id 5579 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa408b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 1 -- 172.21.15.45:0/2112407146 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ffa94001bb0 con 0x7ffa9c096a20 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 10 monclient: dump: 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa408b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 1 -- 172.21.15.45:0/2112407146 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffa94001d10 con 0x7ffa9c096a20 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa408b640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 1 -- 172.21.15.45:0/2112407146 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ffa94004f20 con 0x7ffa9c096a20 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa408b640 4 set_mon_vals no callback set 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa0e25640 10 monclient: dump: 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.732 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa408b640 10 monclient: shutdown 2024-09-18T14:10:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa408b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa408b640 1 -- 172.21.15.45:0/2112407146 >> v1:172.21.15.45:6793/0 conn(0x7ffa9c096a20 legacy=0x7ffa9c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa408b640 1 -- 172.21.15.45:0/2112407146 shutdown_connections 2024-09-18T14:10:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa408b640 1 -- 172.21.15.45:0/2112407146 >> 172.21.15.45:0/2112407146 conn(0x7ffa9c0630f0 msgr2=0x7ffa9c0634f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa408b640 1 -- 172.21.15.45:0/2112407146 shutdown_connections 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.719+0000 7ffaa408b640 1 -- 172.21.15.45:0/2112407146 wait complete. 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.720+0000 7ffaa408b640 10 monclient: build_initial_monmap 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.720+0000 7ffaa408b640 10 monclient: monmap: 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.720+0000 7ffaa408b640 1 Processor -- start 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.720+0000 7ffaa408b640 1 -- start start 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.720+0000 7ffaa408b640 10 monclient: init 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa408b640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa408b640 10 monclient: _add_conns ranks=[4,6,5,0,2,1,7,8,3] 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa408b640 10 monclient(hunting): picked mon.noname-c con 0x7ffa9c096a20 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa408b640 10 monclient(hunting): picked mon.noname-d con 0x7ffa9c0a2610 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa408b640 10 monclient(hunting): picked mon.noname-h con 0x7ffa9c1330b0 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa408b640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c0a2350 con 0x7ffa9c096a20 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa408b640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c11d490 con 0x7ffa9c1330b0 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa408b640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c1423d0 con 0x7ffa9c0a2610 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa408b640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa408b640 10 monclient(hunting): authenticate will time out at 1443.059137s 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa1626640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7ffa9c0a2610 0x7ffa9c141cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:59716/0 (socket says 172.21.15.45:59716) 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa1626640 1 -- 172.21.15.45:0/2440555972 learned_addr learned my addr 172.21.15.45:0/2440555972 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffaa1e27640 1 --1- 172.21.15.45:0/2440555972 >> v1:172.21.15.45:6791/0 conn(0x7ffa9c096a20 0x7ffa9c1415b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:54618/0 (socket says 172.21.15.45:54618) 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633729643 0 0) 0x7ffa9c1423d0 con 0x7ffa9c0a2610 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): my global_id is 184317 2024-09-18T14:10:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa800034e0 con 0x7ffa9c0a2610 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298230715 0 0) 0x7ffa9c0a2350 con 0x7ffa9c096a20 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): my global_id is 164506 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa9c1423d0 con 0x7ffa9c096a20 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75910320 0 0) 0x7ffa9c11d490 con 0x7ffa9c1330b0 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): my global_id is 184307 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa9c0a2350 con 0x7ffa9c1330b0 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 740126246 0 0) 0x7ffa800034e0 con 0x7ffa9c0a2610 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa9c11d490 con 0x7ffa9c0a2610 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ffa8c004370 con 0x7ffa9c0a2610 2024-09-18T14:10:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): dump: 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4290060615 0 0) 0x7ffa9c1423d0 con 0x7ffa9c096a20 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa800034e0 con 0x7ffa9c096a20 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ffa980019a0 con 0x7ffa9c096a20 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): dump: 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.739 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2740180520 0 0) 0x7ffa9c0a2350 con 0x7ffa9c1330b0 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa9c1423d0 con 0x7ffa9c1330b0 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ffa94001af0 con 0x7ffa9c1330b0 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 10 monclient(hunting): dump: 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.741 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3990935907 0 0) 0x7ffa9c11d490 con 0x7ffa9c0a2610 2024-09-18T14:10:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.721+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 >> v1:172.21.15.84:6791/0 conn(0x7ffa9c1330b0 legacy=0x7ffa9c149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 >> v1:172.21.15.45:6791/0 conn(0x7ffa9c096a20 legacy=0x7ffa9c1415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 1 monclient: found mon.d 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c11c2c0 con 0x7ffa9c0a2610 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 10 monclient: _finish_auth 0 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffaa408b640 5 monclient: authenticate success, global_id 184317 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffaa408b640 10 monclient: _renew_subs 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffaa408b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffaa408b640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9c11d1b0 con 0x7ffa9c0a2610 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffa8c0044d0 con 0x7ffa9c0a2610 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ffa8c004a00 con 0x7ffa9c0a2610 2024-09-18T14:10:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa91ffb640 10 monclient: dump: 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffa93fff640 4 set_mon_vals no callback set 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffaa408b640 10 monclient: _renew_subs 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffaa408b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.722+0000 7ffaa408b640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c11e470 con 0x7ffa9c0a2610 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ffa8c004b60 con 0x7ffa9c0a2610 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffaa408b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffaa408b640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ffa9c04c4a0 con 0x7ffa9c09c740 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(533..533 src has 344..533) v4 ==== 3959+0+0 (unknown 756524739 0 0) 0x7ffa8c046a50 con 0x7ffa9c0a2610 2024-09-18T14:10:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895741321 0 0) 0x7ffa9c04c4a0 con 0x7ffa9c09c740 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 10 monclient(hunting): my global_id is 184326 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa9c11d490 con 0x7ffa9c09c740 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffaa2628640 10 monclient: get_auth_request con 0x7ffa8003aa20 auth_method 0 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1759026680 0 0) 0x7ffa9c11d490 con 0x7ffa9c09c740 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa9c04c4a0 con 0x7ffa9c09c740 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ffa98002570 con 0x7ffa9c09c740 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 10 monclient: dump: 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.747 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1596044355 0 0) 0x7ffa9c04c4a0 con 0x7ffa9c09c740 2024-09-18T14:10:27.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.723+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa9c11d7b0 con 0x7ffa9c09c740 2024-09-18T14:10:27.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.724+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ffa98002990 con 0x7ffa9c09c740 2024-09-18T14:10:27.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.724+0000 7ffa91ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:27.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.724+0000 7ffa91ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:27.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.724+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 >> v1:172.21.15.45:6792/0 conn(0x7ffa9c09c740 legacy=0x7ffa9c097e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.724+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 >> v1:172.21.15.45:6792/0 conn(0x7ffa9c09c740 legacy=0x7ffa9c097e40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:27.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.729+0000 7ffaa408b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:27.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffaa408b640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ffa9c11d920 con 0x7ffa9c02a8d0 2024-09-18T14:10:27.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364494347 0 0) 0x7ffa98002c20 con 0x7ffa9c02a8d0 2024-09-18T14:10:27.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 10 monclient(hunting): my global_id is 184335 2024-09-18T14:10:27.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa9c04c4a0 con 0x7ffa9c02a8d0 2024-09-18T14:10:27.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 398295375 0 0) 0x7ffa9c04c4a0 con 0x7ffa9c02a8d0 2024-09-18T14:10:27.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa98002c20 con 0x7ffa9c02a8d0 2024-09-18T14:10:27.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ffa98002db0 con 0x7ffa9c02a8d0 2024-09-18T14:10:27.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 10 monclient: dump: 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.750 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.751 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.751 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.751 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.751 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.751 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4029197624 0 0) 0x7ffa98002c20 con 0x7ffa9c02a8d0 2024-09-18T14:10:27.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffa9c04b720 con 0x7ffa9c02a8d0 2024-09-18T14:10:27.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7375 (unknown 0 0 4100087371) 0x7ffa98007ce0 con 0x7ffa9c02a8d0 2024-09-18T14:10:27.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:27.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:27.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 >> v1:172.21.15.45:6792/0 conn(0x7ffa9c02a8d0 legacy=0x7ffa9c098550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.730+0000 7ffa91ffb640 1 -- 172.21.15.45:0/2440555972 >> v1:172.21.15.45:6792/0 conn(0x7ffa9c02a8d0 legacy=0x7ffa9c098550 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:27.752 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 98, 2024-09-18T14:10:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:27.771 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.731+0000 7ffaa408b640 1 -- 172.21.15.45:0/2440555972 >> v1:172.21.15.84:6812/31478 conn(0x7ffa8003aa20 legacy=0x7ffa8003ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.731+0000 7ffaa408b640 10 monclient: shutdown 2024-09-18T14:10:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.731+0000 7ffaa408b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.731+0000 7ffaa408b640 1 -- 172.21.15.45:0/2440555972 >> v1:172.21.15.45:6792/0 conn(0x7ffa9c0a2610 legacy=0x7ffa9c141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.731+0000 7ffaa2628640 1 -- 172.21.15.45:0/2440555972 reap_dead start 2024-09-18T14:10:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.731+0000 7ffaa2628640 1 -- 172.21.15.45:0/2440555972 reap_dead start 2024-09-18T14:10:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.731+0000 7ffaa408b640 1 -- 172.21.15.45:0/2440555972 shutdown_connections 2024-09-18T14:10:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.731+0000 7ffaa408b640 1 -- 172.21.15.45:0/2440555972 >> 172.21.15.45:0/2440555972 conn(0x7ffa9c0630f0 msgr2=0x7ffa9c09e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.731+0000 7ffaa408b640 1 -- 172.21.15.45:0/2440555972 shutdown_connections 2024-09-18T14:10:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.731+0000 7ffaa408b640 1 -- 172.21.15.45:0/2440555972 wait complete. 2024-09-18T14:10:27.773 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:10:27.773 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-09-18T14:10:27.773 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:10:27.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.870+0000 7febda39d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:27.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.870+0000 7febda39d640 10 monclient: build_initial_monmap 2024-09-18T14:10:27.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.870+0000 7febda39d640 10 monclient: monmap: 2024-09-18T14:10:27.871 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:27.871 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.871 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:27.871781+0000 2024-09-18T14:10:27.871 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:27.871781+0000 2024-09-18T14:10:27.872 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:27.872 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.872 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:27.872 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:27.872 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:27.872 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:27.872 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:27.872 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:27.872 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:27.872 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:27.872 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:27.872 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.871+0000 7febda39d640 1 Processor -- start 2024-09-18T14:10:27.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.871+0000 7febda39d640 1 -- start start 2024-09-18T14:10:27.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.871+0000 7febda39d640 10 monclient: init 2024-09-18T14:10:27.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febda39d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:27.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febda39d640 10 monclient: _add_conns ranks=[6,1,8,4,3,5,7,2,0] 2024-09-18T14:10:27.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febda39d640 10 monclient(hunting): picked mon.noname-d con 0x7febd409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:27.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febda39d640 10 monclient(hunting): picked mon.noname-f con 0x7febd409fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:27.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febda39d640 10 monclient(hunting): picked mon.noname-e con 0x7febd409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:27.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febda39d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd405ed70 con 0x7febd409fba0 2024-09-18T14:10:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febda39d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd409dfc0 con 0x7febd409f7d0 2024-09-18T14:10:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febda39d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd409dce0 con 0x7febd409ff70 2024-09-18T14:10:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febda39d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febda39d640 10 monclient(hunting): authenticate will time out at 1443.210349s 2024-09-18T14:10:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd3fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7febd409f7d0 0x7febd4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:59744/0 (socket says 172.21.15.45:59744) 2024-09-18T14:10:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd3fff640 1 -- 172.21.15.45:0/2402787225 learned_addr learned my addr 172.21.15.45:0/2402787225 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd893a640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7febd409ff70 0x7febd413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:48274/0 (socket says 172.21.15.45:48274) 2024-09-18T14:10:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febcbfff640 1 -- 172.21.15.45:0/2402787225 >> v1:172.21.15.84:6789/0 conn(0x7febd409fba0 legacy=0x7febd4137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:10:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 1 -- 172.21.15.45:0/2402787225 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101930151 0 0) 0x7febd409dfc0 con 0x7febd409f7d0 2024-09-18T14:10:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 10 monclient(hunting): my global_id is 184344 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 1 -- 172.21.15.45:0/2402787225 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febc00034e0 con 0x7febd409f7d0 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 1 -- 172.21.15.45:0/2402787225 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62977468 0 0) 0x7febd409dce0 con 0x7febd409ff70 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 10 monclient(hunting): my global_id is 5588 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 1 -- 172.21.15.45:0/2402787225 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febd409dfc0 con 0x7febd409ff70 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 1 -- 172.21.15.45:0/2402787225 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1323294716 0 0) 0x7febd409dfc0 con 0x7febd409ff70 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 1 -- 172.21.15.45:0/2402787225 >> v1:172.21.15.45:6792/0 conn(0x7febd409f7d0 legacy=0x7febd4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 1 -- 172.21.15.45:0/2402787225 >> v1:172.21.15.84:6789/0 conn(0x7febd409fba0 legacy=0x7febd4137a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 10 monclient: _finish_hunting 0 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 1 monclient: found mon.noname-e 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 1 -- 172.21.15.45:0/2402787225 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd40a1c00 con 0x7febd409ff70 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 10 monclient: _finish_auth 0 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 1 -- 172.21.15.45:0/2402787225 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7febcc001b90 con 0x7febd409ff70 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febda39d640 5 monclient: authenticate success, global_id 5588 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febda39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.872+0000 7febd37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febd37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febd37fe640 10 monclient: dump: 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febda39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febd37fe640 1 -- 172.21.15.45:0/2402787225 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7febcc001cf0 con 0x7febd409ff70 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febd37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febd37fe640 1 -- 172.21.15.45:0/2402787225 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7febcc004e60 con 0x7febd409ff70 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febda39d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febda39d640 4 set_mon_vals no callback set 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febd37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febd37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febd37fe640 10 monclient: dump: 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febda39d640 10 monclient: shutdown 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febda39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febda39d640 1 -- 172.21.15.45:0/2402787225 >> v1:172.21.15.45:6793/0 conn(0x7febd409ff70 legacy=0x7febd413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febda39d640 1 -- 172.21.15.45:0/2402787225 shutdown_connections 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febda39d640 1 -- 172.21.15.45:0/2402787225 >> 172.21.15.45:0/2402787225 conn(0x7febd40630b0 msgr2=0x7febd40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febda39d640 1 -- 172.21.15.45:0/2402787225 shutdown_connections 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.873+0000 7febda39d640 1 -- 172.21.15.45:0/2402787225 wait complete. 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.874+0000 7febda39d640 10 monclient: build_initial_monmap 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.874+0000 7febda39d640 10 monclient: monmap: 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:27.888 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.874+0000 7febda39d640 1 Processor -- start 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.874+0000 7febda39d640 1 -- start start 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.874+0000 7febda39d640 10 monclient: init 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.874+0000 7febda39d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 10 monclient: _add_conns ranks=[1,8,2,5,3,7,0,6,4] 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 10 monclient(hunting): picked mon.noname-f con 0x7febd409f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 10 monclient(hunting): picked mon.noname-e con 0x7febd409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 10 monclient(hunting): picked mon.noname-b con 0x7febd409ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd409d310 con 0x7febd409f7d0 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd40a2560 con 0x7febd409ff70 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd41209f0 con 0x7febd409fba0 2024-09-18T14:10:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 10 monclient(hunting): authenticate will time out at 1443.213167s 2024-09-18T14:10:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd3fff640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7febd409f7d0 legacy=0x7febd41305c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:10:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febcbfff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7febd409fba0 0x7febd4146ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:48278/0 (socket says 172.21.15.45:48278) 2024-09-18T14:10:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febcbfff640 1 -- 172.21.15.45:0/1575738217 learned_addr learned my addr 172.21.15.45:0/1575738217 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224127444 0 0) 0x7febd41209f0 con 0x7febd409fba0 2024-09-18T14:10:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): my global_id is 5597 2024-09-18T14:10:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febb40034e0 con 0x7febd409fba0 2024-09-18T14:10:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009499008 0 0) 0x7febd40a2560 con 0x7febd409ff70 2024-09-18T14:10:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): my global_id is 145028 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febd41209f0 con 0x7febd409ff70 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1067254679 0 0) 0x7febb40034e0 con 0x7febd409fba0 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7febd40a2560 con 0x7febd409fba0 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7febbc004370 con 0x7febd409fba0 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): dump: 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.891 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:27.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1604972 0 0) 0x7febd41209f0 con 0x7febd409ff70 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7febb40034e0 con 0x7febd409ff70 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7febcc001ad0 con 0x7febd409ff70 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient(hunting): dump: 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.893 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3197427631 0 0) 0x7febd40a2560 con 0x7febd409fba0 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 >> v1:172.21.15.45:6790/0 conn(0x7febd409ff70 legacy=0x7febd414a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 >> v1:172.21.15.84:6789/0 conn(0x7febd409f7d0 legacy=0x7febd41305c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 monclient: found mon.e 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:27.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd41426c0 con 0x7febd409fba0 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 10 monclient: _finish_auth 0 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febd0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 5 monclient: authenticate success, global_id 5597 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 10 monclient: _renew_subs 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd411ecb0 con 0x7febd409fba0 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 10 monclient: _renew_subs 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.875+0000 7febda39d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.876+0000 7febda39d640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7febd411f820 con 0x7febd409fba0 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.876+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7febbc004550 con 0x7febd409fba0 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.876+0000 7febd0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:27.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.876+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7febbc004b00 con 0x7febd409fba0 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.876+0000 7febd0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.876+0000 7febd0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.876+0000 7febd2ffd640 4 set_mon_vals no callback set 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.876+0000 7febd0ff9640 10 monclient: dump: 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.896 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.876+0000 7febda39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.876+0000 7febda39d640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7febb8006580 con 0x7febb8002830 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.876+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7febbc004c60 con 0x7febd409fba0 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(533..533 src has 344..533) v4 ==== 3959+0+0 (unknown 756524739 0 0) 0x7febbc046ae0 con 0x7febd409fba0 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3980823251 0 0) 0x7febb8006580 con 0x7febb8002830 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 10 monclient(hunting): my global_id is 5606 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febd40a2560 con 0x7febb8002830 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd893a640 10 monclient: get_auth_request con 0x7febb403a400 auth_method 0 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2441480118 0 0) 0x7febd40a2560 con 0x7febb8002830 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7febb8006580 con 0x7febb8002830 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7febc4001770 con 0x7febb8002830 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 10 monclient: dump: 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2675680334 0 0) 0x7febb8006580 con 0x7febb8002830 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7febb800a0a0 con 0x7febb8002830 2024-09-18T14:10:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7febc4001d50 con 0x7febb8002830 2024-09-18T14:10:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 >> v1:172.21.15.45:6793/0 conn(0x7febb8002830 legacy=0x7febb8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.877+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 >> v1:172.21.15.45:6793/0 conn(0x7febb8002830 legacy=0x7febb8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.883+0000 7febda39d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.883+0000 7febda39d640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7febb8009660 con 0x7febb8009040 2024-09-18T14:10:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.883+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384399883 0 0) 0x7febb8009660 con 0x7febb8009040 2024-09-18T14:10:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.883+0000 7febd0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.883+0000 7febd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.883+0000 7febd0ff9640 10 monclient(hunting): my global_id is 5615 2024-09-18T14:10:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.883+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febb8006580 con 0x7febb8009040 2024-09-18T14:10:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4217389040 0 0) 0x7febb8006580 con 0x7febb8009040 2024-09-18T14:10:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7febb8009660 con 0x7febb8009040 2024-09-18T14:10:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7febc4004f70 con 0x7febb8009040 2024-09-18T14:10:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 10 monclient: dump: 2024-09-18T14:10:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:27.904 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 926661406 0 0) 0x7febb8009660 con 0x7febb8009040 2024-09-18T14:10:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7febb8009920 con 0x7febb8009040 2024-09-18T14:10:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7717 (unknown 0 0 1606200080) 0x7febc4007190 con 0x7febb8009040 2024-09-18T14:10:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 >> v1:172.21.15.45:6793/0 conn(0x7febb8009040 legacy=0x7febb8008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd0ff9640 1 -- 172.21.15.45:0/1575738217 >> v1:172.21.15.45:6793/0 conn(0x7febb8009040 legacy=0x7febb8008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 98, 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:10:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:10:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:10:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:27.924 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febda39d640 1 -- 172.21.15.45:0/1575738217 >> v1:172.21.15.84:6812/31478 conn(0x7febb403a400 legacy=0x7febb403c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febda39d640 10 monclient: shutdown 2024-09-18T14:10:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febda39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febda39d640 1 -- 172.21.15.45:0/1575738217 >> v1:172.21.15.45:6793/0 conn(0x7febd409fba0 legacy=0x7febd4146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd893a640 1 -- 172.21.15.45:0/1575738217 reap_dead start 2024-09-18T14:10:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febd893a640 1 -- 172.21.15.45:0/1575738217 reap_dead start 2024-09-18T14:10:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febda39d640 1 -- 172.21.15.45:0/1575738217 shutdown_connections 2024-09-18T14:10:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febda39d640 1 -- 172.21.15.45:0/1575738217 >> 172.21.15.45:0/1575738217 conn(0x7febd40630b0 msgr2=0x7febd40a4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.884+0000 7febda39d640 1 -- 172.21.15.45:0/1575738217 shutdown_connections 2024-09-18T14:10:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:27.885+0000 7febda39d640 1 -- 172.21.15.45:0/1575738217 wait complete. 2024-09-18T14:10:27.927 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:10:27.927 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-09-18T14:10:27.927 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:10:28.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.025+0000 7f91e771d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:28.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.025+0000 7f91e771d640 10 monclient: build_initial_monmap 2024-09-18T14:10:28.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.025+0000 7f91e771d640 10 monclient: monmap: 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:28.026270+0000 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:28.026270+0000 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:28.027 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:28.027 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.025+0000 7f91e771d640 1 Processor -- start 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e771d640 1 -- start start 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e771d640 10 monclient: init 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e771d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e771d640 10 monclient: _add_conns ranks=[7,2,4,6,8,1,0,5,3] 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e771d640 10 monclient(hunting): picked mon.noname-i con 0x7f91e006e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e771d640 10 monclient(hunting): picked mon.noname-b con 0x7f91e00a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e771d640 10 monclient(hunting): picked mon.noname-c con 0x7f91e00a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e771d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e005ed70 con 0x7f91e00a0cb0 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e771d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e0097620 con 0x7f91e00a1080 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e771d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e0097340 con 0x7f91e006e810 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e771d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e771d640 10 monclient(hunting): authenticate will time out at 1443.364642s 2024-09-18T14:10:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e4cb8640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f91e00a0cb0 0x7f91e0137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:56844/0 (socket says 172.21.15.45:56844) 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e5cba640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f91e00a1080 0x7f91e013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:54624/0 (socket says 172.21.15.45:54624) 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.026+0000 7f91e4cb8640 1 -- 172.21.15.45:0/4179398700 learned_addr learned my addr 172.21.15.45:0/4179398700 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173677812 0 0) 0x7f91e005ed70 con 0x7f91e00a0cb0 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient(hunting): my global_id is 145037 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91cc0034e0 con 0x7f91e00a0cb0 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268213852 0 0) 0x7f91e0097620 con 0x7f91e00a1080 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient(hunting): my global_id is 164515 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e005ed70 con 0x7f91e00a1080 2024-09-18T14:10:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286875863 0 0) 0x7f91e0097340 con 0x7f91e006e810 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient(hunting): my global_id is 184300 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e0097620 con 0x7f91e006e810 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4119818137 0 0) 0x7f91e005ed70 con 0x7f91e00a1080 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 >> v1:172.21.15.84:6792/0 conn(0x7f91e006e810 legacy=0x7f91e00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 >> v1:172.21.15.45:6790/0 conn(0x7f91e00a0cb0 legacy=0x7f91e0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient: _finish_hunting 0 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 monclient: found mon.noname-c 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00a1660 con 0x7f91e00a1080 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient: _finish_auth 0 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91d8001b90 con 0x7f91e00a1080 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91e771d640 5 monclient: authenticate success, global_id 164515 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91e771d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient: dump: 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91e771d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f91d8001d70 con 0x7f91e00a1080 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 1 -- 172.21.15.45:0/4179398700 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91d8004e60 con 0x7f91e00a1080 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91e771d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91e771d640 4 set_mon_vals no callback set 2024-09-18T14:10:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91d7fff640 10 monclient: dump: 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91e771d640 10 monclient: shutdown 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91e771d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91e771d640 1 -- 172.21.15.45:0/4179398700 >> v1:172.21.15.45:6791/0 conn(0x7f91e00a1080 legacy=0x7f91e013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91e771d640 1 -- 172.21.15.45:0/4179398700 shutdown_connections 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.027+0000 7f91e771d640 1 -- 172.21.15.45:0/4179398700 >> 172.21.15.45:0/4179398700 conn(0x7f91e00a5bb0 msgr2=0x7f91e00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.028+0000 7f91e771d640 1 -- 172.21.15.45:0/4179398700 shutdown_connections 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.028+0000 7f91e771d640 1 -- 172.21.15.45:0/4179398700 wait complete. 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.028+0000 7f91e771d640 10 monclient: build_initial_monmap 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.028+0000 7f91e771d640 10 monclient: monmap: 2024-09-18T14:10:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.028+0000 7f91e771d640 1 Processor -- start 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.028+0000 7f91e771d640 1 -- start start 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.028+0000 7f91e771d640 10 monclient: init 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91e771d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91e771d640 10 monclient: _add_conns ranks=[5,7,2,3,0,6,1,8,4] 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91e771d640 10 monclient(hunting): picked mon.noname-h con 0x7f91e006e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91e771d640 10 monclient(hunting): picked mon.noname-i con 0x7f91e00a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91e771d640 10 monclient(hunting): picked mon.noname-b con 0x7f91e00a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91e771d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e0122da0 con 0x7f91e00a1080 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91e771d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e013db90 con 0x7f91e006e810 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91e771d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e01227e0 con 0x7f91e00a0cb0 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91e771d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91e771d640 10 monclient(hunting): authenticate will time out at 1443.367682s 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91e5cba640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f91e00a1080 0x7f91e0149de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:56852/0 (socket says 172.21.15.45:56852) 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91e5cba640 1 -- 172.21.15.45:0/2535906552 learned_addr learned my addr 172.21.15.45:0/2535906552 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463697001 0 0) 0x7f91e0122da0 con 0x7f91e00a1080 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91d57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91d57fa640 10 monclient(hunting): my global_id is 145046 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c40034e0 con 0x7f91e00a1080 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317055155 0 0) 0x7f91e01227e0 con 0x7f91e00a0cb0 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91d57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91d57fa640 10 monclient(hunting): my global_id is 184309 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e0122da0 con 0x7f91e00a0cb0 2024-09-18T14:10:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.029+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993911557 0 0) 0x7f91e013db90 con 0x7f91e006e810 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): my global_id is 184316 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e01227e0 con 0x7f91e006e810 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3068605784 0 0) 0x7f91e0122da0 con 0x7f91e00a0cb0 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91e013db90 con 0x7f91e00a0cb0 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91d0001410 con 0x7f91e00a0cb0 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): dump: 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.046 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2535490502 0 0) 0x7f91c40034e0 con 0x7f91e00a1080 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91e0122da0 con 0x7f91e00a1080 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 729547929 0 0) 0x7f91e01227e0 con 0x7f91e006e810 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91c40034e0 con 0x7f91e006e810 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91d8001b50 con 0x7f91e00a1080 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): dump: 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91dc001ab0 con 0x7f91e006e810 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient(hunting): dump: 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 95738378 0 0) 0x7f91e013db90 con 0x7f91e00a0cb0 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 >> v1:172.21.15.84:6791/0 conn(0x7f91e006e810 legacy=0x7f91e013d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 >> v1:172.21.15.45:6790/0 conn(0x7f91e00a1080 legacy=0x7f91e0149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient: _finish_hunting 0 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 monclient: found mon.i 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e01202f0 con 0x7f91e00a0cb0 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient: _finish_auth 0 2024-09-18T14:10:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91e771d640 5 monclient: authenticate success, global_id 184309 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91e771d640 10 monclient: _renew_subs 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91e771d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91e771d640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e0122500 con 0x7f91e00a0cb0 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91e771d640 10 monclient: _renew_subs 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91e771d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91e771d640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e0121630 con 0x7f91e00a0cb0 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f91d00047a0 con 0x7f91e00a0cb0 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91d0004c50 con 0x7f91e00a0cb0 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d6ffd640 4 set_mon_vals no callback set 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.030+0000 7f91d57fa640 10 monclient: dump: 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.053 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.031+0000 7f91e771d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.031+0000 7f91e771d640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f91c0006580 con 0x7f91c0002830 2024-09-18T14:10:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.031+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f91d000fca0 con 0x7f91e00a0cb0 2024-09-18T14:10:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.031+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(533..533 src has 344..533) v4 ==== 3959+0+0 (unknown 756524739 0 0) 0x7f91d0046ab0 con 0x7f91e00a0cb0 2024-09-18T14:10:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.031+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346230776 0 0) 0x7f91c0006580 con 0x7f91c0002830 2024-09-18T14:10:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.031+0000 7f91d57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.031+0000 7f91d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.031+0000 7f91d57fa640 10 monclient(hunting): my global_id is 184278 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.031+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e013db90 con 0x7f91c0002830 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.031+0000 7f91e5cba640 10 monclient: get_auth_request con 0x7f91c403ab90 auth_method 0 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1278014809 0 0) 0x7f91e013db90 con 0x7f91c0002830 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91c0006580 con 0x7f91c0002830 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91dc000f50 con 0x7f91c0002830 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 10 monclient: dump: 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4030584835 0 0) 0x7f91c0006580 con 0x7f91c0002830 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91c000a0a0 con 0x7f91c0002830 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f91dc004240 con 0x7f91c0002830 2024-09-18T14:10:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 >> v1:172.21.15.84:6790/0 conn(0x7f91c0002830 legacy=0x7f91c0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.032+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 >> v1:172.21.15.84:6790/0 conn(0x7f91c0002830 legacy=0x7f91c0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.038+0000 7f91e771d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.038+0000 7f91e771d640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f91c0009e70 con 0x7f91c00068f0 2024-09-18T14:10:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427235259 0 0) 0x7f91c0009e70 con 0x7f91c00068f0 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 10 monclient(hunting): my global_id is 184287 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c0006580 con 0x7f91c00068f0 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2138520741 0 0) 0x7f91c0006580 con 0x7f91c00068f0 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91c0009e70 con 0x7f91c00068f0 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91dc002b20 con 0x7f91c00068f0 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 10 monclient: dump: 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1677779439 0 0) 0x7f91c0009e70 con 0x7f91c00068f0 2024-09-18T14:10:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f91c0008750 con 0x7f91c00068f0 2024-09-18T14:10:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7545 (unknown 0 0 2869086279) 0x7f91dc003f10 con 0x7f91c00068f0 2024-09-18T14:10:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 >> v1:172.21.15.84:6790/0 conn(0x7f91c00068f0 legacy=0x7f91c0006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.039+0000 7f91d57fa640 1 -- 172.21.15.45:0/2535906552 >> v1:172.21.15.84:6790/0 conn(0x7f91c00068f0 legacy=0x7f91c0006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:28.060 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:28.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:28.060 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:28.060 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 98, 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:10:28.061 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:28.062 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:28.063 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:28.064 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.065 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.068 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.068 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.068 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:28.068 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:28.068 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.068 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.068 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.068 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:28.068 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.069 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.072 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:28.072 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.072 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.072 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.072 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:28.072 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:28.072 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.072 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.072 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:28.073 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:28.074 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:28.077 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:28.078 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.078 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.078 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.078 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.078 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.078 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:28.078 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:28.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.078 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:28.078 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:28.078 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:28.079 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:28.080 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.080 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.080 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:28.080 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.040+0000 7f91e771d640 1 -- 172.21.15.45:0/2535906552 >> v1:172.21.15.84:6812/31478 conn(0x7f91c403ab90 legacy=0x7f91c403d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.040+0000 7f91e771d640 10 monclient: shutdown 2024-09-18T14:10:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.040+0000 7f91e771d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.040+0000 7f91e771d640 1 -- 172.21.15.45:0/2535906552 >> v1:172.21.15.84:6792/0 conn(0x7f91e00a0cb0 legacy=0x7f91e01466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.040+0000 7f91e5cba640 1 -- 172.21.15.45:0/2535906552 reap_dead start 2024-09-18T14:10:28.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.040+0000 7f91e5cba640 1 -- 172.21.15.45:0/2535906552 reap_dead start 2024-09-18T14:10:28.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.040+0000 7f91e771d640 1 -- 172.21.15.45:0/2535906552 shutdown_connections 2024-09-18T14:10:28.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.040+0000 7f91e771d640 1 -- 172.21.15.45:0/2535906552 >> 172.21.15.45:0/2535906552 conn(0x7f91e00a5bb0 msgr2=0x7f91e00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:28.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.040+0000 7f91e771d640 1 -- 172.21.15.45:0/2535906552 shutdown_connections 2024-09-18T14:10:28.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.040+0000 7f91e771d640 1 -- 172.21.15.45:0/2535906552 wait complete. 2024-09-18T14:10:28.082 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:10:28.082 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-09-18T14:10:28.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:10:28.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.178+0000 7f2cdaf9d640 10 monclient: get_monmap_and_config 2024-09-18T14:10:28.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.178+0000 7f2cdaf9d640 10 monclient: build_initial_monmap 2024-09-18T14:10:28.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.178+0000 7f2cdaf9d640 10 monclient: monmap: 2024-09-18T14:10:28.179 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:28.179 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.179 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:28.179851+0000 2024-09-18T14:10:28.179 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:28.179851+0000 2024-09-18T14:10:28.179 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:28.179 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.180 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:28.180 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:28.180 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:28.180 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:28.180 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:28.180 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:28.180 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:28.180 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:28.180 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:28.180 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.179+0000 7f2cdaf9d640 1 Processor -- start 2024-09-18T14:10:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.179+0000 7f2cdaf9d640 1 -- start start 2024-09-18T14:10:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.179+0000 7f2cdaf9d640 10 monclient: init 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cdaf9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cdaf9d640 10 monclient: _add_conns ranks=[6,0,3,4,8,1,7,2,5] 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cdaf9d640 10 monclient(hunting): picked mon.noname-d con 0x7f2cd409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cdaf9d640 10 monclient(hunting): picked mon.noname-a con 0x7f2cd409fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cdaf9d640 10 monclient(hunting): picked mon.noname-g con 0x7f2cd409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cdaf9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd405ed70 con 0x7f2cd409fba0 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cdaf9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd409dfc0 con 0x7f2cd409ff70 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cdaf9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd409dce0 con 0x7f2cd409f7d0 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cdaf9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cdaf9d640 10 monclient(hunting): authenticate will time out at 1443.518214s 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd8d39640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f2cd409f7d0 0x7f2cd4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:59750/0 (socket says 172.21.15.45:59750) 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd8d39640 1 -- 172.21.15.45:0/49378545 learned_addr learned my addr 172.21.15.45:0/49378545 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd3fff640 1 --1- 172.21.15.45:0/49378545 >> v1:172.21.15.45:6789/0 conn(0x7f2cd409fba0 0x7f2cd4137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50242/0 (socket says 172.21.15.45:50242) 2024-09-18T14:10:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666843642 0 0) 0x7f2cd409dce0 con 0x7f2cd409f7d0 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient(hunting): my global_id is 184353 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cbc0034e0 con 0x7f2cd409f7d0 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181235974 0 0) 0x7f2cd405ed70 con 0x7f2cd409fba0 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient(hunting): my global_id is 5796 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd409dce0 con 0x7f2cd409fba0 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591128820 0 0) 0x7f2cd409dfc0 con 0x7f2cd409ff70 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient(hunting): my global_id is 184296 2024-09-18T14:10:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd405ed70 con 0x7f2cd409ff70 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2305179298 0 0) 0x7f2cbc0034e0 con 0x7f2cd409f7d0 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 >> v1:172.21.15.84:6790/0 conn(0x7f2cd409ff70 legacy=0x7f2cd413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 >> v1:172.21.15.45:6789/0 conn(0x7f2cd409fba0 legacy=0x7f2cd4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient: _finish_hunting 0 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 1 monclient: found mon.noname-d 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40a1c00 con 0x7f2cd409f7d0 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient: _finish_auth 0 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2cc4001830 con 0x7f2cd409f7d0 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cdaf9d640 5 monclient: authenticate success, global_id 184353 2024-09-18T14:10:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cd37fe640 10 monclient: dump: 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.180+0000 7f2cdaf9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2cc4001a10 con 0x7f2cd409f7d0 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cd37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cdaf9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cd37fe640 1 -- 172.21.15.45:0/49378545 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2cc4004ba0 con 0x7f2cd409f7d0 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cdaf9d640 4 set_mon_vals no callback set 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cd37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cd37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cd37fe640 10 monclient: dump: 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.196 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cdaf9d640 10 monclient: shutdown 2024-09-18T14:10:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cdaf9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/49378545 >> v1:172.21.15.45:6792/0 conn(0x7f2cd409f7d0 legacy=0x7f2cd4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/49378545 shutdown_connections 2024-09-18T14:10:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/49378545 >> 172.21.15.45:0/49378545 conn(0x7f2cd40630b0 msgr2=0x7f2cd40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/49378545 shutdown_connections 2024-09-18T14:10:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.181+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/49378545 wait complete. 2024-09-18T14:10:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.182+0000 7f2cdaf9d640 10 monclient: build_initial_monmap 2024-09-18T14:10:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.182+0000 7f2cdaf9d640 10 monclient: monmap: 2024-09-18T14:10:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.182+0000 7f2cdaf9d640 1 Processor -- start 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.182+0000 7f2cdaf9d640 1 -- start start 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.182+0000 7f2cdaf9d640 10 monclient: init 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.182+0000 7f2cdaf9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.182+0000 7f2cdaf9d640 10 monclient: _add_conns ranks=[7,3,8,4,1,5,0,6,2] 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.182+0000 7f2cdaf9d640 10 monclient(hunting): picked mon.noname-i con 0x7f2cd409f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 10 monclient(hunting): picked mon.noname-g con 0x7f2cd409fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 10 monclient(hunting): picked mon.noname-e con 0x7f2cd409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd409d310 con 0x7f2cd409fba0 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd40a2560 con 0x7f2cd409f7d0 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd4122480 con 0x7f2cd409ff70 2024-09-18T14:10:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 10 monclient(hunting): authenticate will time out at 1443.521086s 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd953a640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f2cd409ff70 0x7f2cd4149b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:48304/0 (socket says 172.21.15.45:48304) 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd953a640 1 -- 172.21.15.45:0/4017115159 learned_addr learned my addr 172.21.15.45:0/4017115159 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635924048 0 0) 0x7f2cd405ed70 con 0x7f2cd409ff70 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): my global_id is 5624 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cc00034e0 con 0x7f2cd409ff70 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217669211 0 0) 0x7f2cd40a2560 con 0x7f2cd409f7d0 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): my global_id is 184318 2024-09-18T14:10:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd405ed70 con 0x7f2cd409f7d0 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107078023 0 0) 0x7f2cd409d310 con 0x7f2cd409fba0 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): my global_id is 184305 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd40a2560 con 0x7f2cd409fba0 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 941558800 0 0) 0x7f2cc00034e0 con 0x7f2cd409ff70 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2cd409d310 con 0x7f2cd409ff70 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2ccc002e50 con 0x7f2cd409ff70 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): dump: 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.201 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 623271425 0 0) 0x7f2cd405ed70 con 0x7f2cd409f7d0 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2cc00034e0 con 0x7f2cd409f7d0 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2cc4004b20 con 0x7f2cd409f7d0 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): dump: 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 879024169 0 0) 0x7f2cd40a2560 con 0x7f2cd409fba0 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2cd405ed70 con 0x7f2cd409fba0 2024-09-18T14:10:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2cc8001550 con 0x7f2cd409fba0 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient(hunting): dump: 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1864529200 0 0) 0x7f2cd409d310 con 0x7f2cd409ff70 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 >> v1:172.21.15.84:6792/0 conn(0x7f2cd409f7d0 legacy=0x7f2cd4141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 >> v1:172.21.15.84:6790/0 conn(0x7f2cd409fba0 legacy=0x7f2cd4131a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 monclient: found mon.e 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:10:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd4121e70 con 0x7f2cd409ff70 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 10 monclient: _finish_auth 0 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cd0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 5 monclient: authenticate success, global_id 5624 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 10 monclient: _renew_subs 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd4121b70 con 0x7f2cd409ff70 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 10 monclient: _renew_subs 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.183+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd4120c10 con 0x7f2cd409ff70 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.184+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2ccc003110 con 0x7f2cd409ff70 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.184+0000 7f2cd0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.184+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2ccc001d00 con 0x7f2cd409ff70 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.184+0000 7f2cd0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.184+0000 7f2cd2ffd640 4 set_mon_vals no callback set 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.184+0000 7f2cd0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.184+0000 7f2cd0ff9640 10 monclient: dump: 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.184+0000 7f2cdaf9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.184+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2cb4006580 con 0x7f2cb4002830 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.184+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2ccc001ee0 con 0x7f2cd409ff70 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(533..533 src has 344..533) v4 ==== 3959+0+0 (unknown 756524739 0 0) 0x7f2ccc046d90 con 0x7f2cd409ff70 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752846042 0 0) 0x7f2cb4006580 con 0x7f2cb4002830 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 10 monclient(hunting): my global_id is 184325 2024-09-18T14:10:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd409d310 con 0x7f2cb4002830 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd3fff640 10 monclient: get_auth_request con 0x7f2cc003a8d0 auth_method 0 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 477459948 0 0) 0x7f2cd409d310 con 0x7f2cb4002830 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2cb4006580 con 0x7f2cb4002830 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2cc40016e0 con 0x7f2cb4002830 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 10 monclient: dump: 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.209 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.210 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.210 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.210 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.210 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.210 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.210 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.210 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.210 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.210 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2213262202 0 0) 0x7f2cb4006580 con 0x7f2cb4002830 2024-09-18T14:10:28.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.185+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2cb400a0a0 con 0x7f2cb4002830 2024-09-18T14:10:28.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.186+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2cc4005160 con 0x7f2cb4002830 2024-09-18T14:10:28.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.186+0000 7f2cd0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:28.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.186+0000 7f2cd0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:28.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.186+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 >> v1:172.21.15.84:6791/0 conn(0x7f2cb4002830 legacy=0x7f2cb4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.186+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 >> v1:172.21.15.84:6791/0 conn(0x7f2cb4002830 legacy=0x7f2cb4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:28.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.191+0000 7f2cdaf9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:28.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.191+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2cb4009660 con 0x7f2cb4009040 2024-09-18T14:10:28.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1902174691 0 0) 0x7f2cb4009660 con 0x7f2cb4009040 2024-09-18T14:10:28.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 10 monclient(hunting): my global_id is 184334 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cb4006580 con 0x7f2cb4009040 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2939539832 0 0) 0x7f2cb4006580 con 0x7f2cb4009040 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2cb4009660 con 0x7f2cb4009040 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2cc40018b0 con 0x7f2cb4009040 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 10 monclient: dump: 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3898949914 0 0) 0x7f2cb4009660 con 0x7f2cb4009040 2024-09-18T14:10:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2cb4009920 con 0x7f2cb4009040 2024-09-18T14:10:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7375 (unknown 0 0 1427833705) 0x7f2cc4007c40 con 0x7f2cb4009040 2024-09-18T14:10:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 >> v1:172.21.15.84:6791/0 conn(0x7f2cb4009040 legacy=0x7f2cb4008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.192+0000 7f2cd0ff9640 1 -- 172.21.15.45:0/4017115159 >> v1:172.21.15.84:6791/0 conn(0x7f2cb4009040 legacy=0x7f2cb4008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:28.215 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:28.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:28.215 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 98, 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.216 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.217 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.218 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:28.219 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.220 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.222 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:28.225 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:28.227 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:28.228 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.229 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.231 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.233 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.234 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:28.234 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.193+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/4017115159 >> v1:172.21.15.84:6812/31478 conn(0x7f2cc003a8d0 legacy=0x7f2cc003cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.193+0000 7f2cdaf9d640 10 monclient: shutdown 2024-09-18T14:10:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.193+0000 7f2cdaf9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.193+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/4017115159 >> v1:172.21.15.45:6793/0 conn(0x7f2cd409ff70 legacy=0x7f2cd4149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.193+0000 7f2cd953a640 1 -- 172.21.15.45:0/4017115159 reap_dead start 2024-09-18T14:10:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.193+0000 7f2cd953a640 1 -- 172.21.15.45:0/4017115159 reap_dead start 2024-09-18T14:10:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.193+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/4017115159 shutdown_connections 2024-09-18T14:10:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.193+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/4017115159 >> 172.21.15.45:0/4017115159 conn(0x7f2cd40630b0 msgr2=0x7f2cd40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.193+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/4017115159 shutdown_connections 2024-09-18T14:10:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.193+0000 7f2cdaf9d640 1 -- 172.21.15.45:0/4017115159 wait complete. 2024-09-18T14:10:28.236 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:10:28.236 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-09-18T14:10:28.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:10:28.282 INFO:tasks.rados.rados.0.smithi084.stdout:2449: done (3 left) 2024-09-18T14:10:28.282 INFO:tasks.rados.rados.0.smithi084.stdout:2450: write (excl) oid 37 current snap is 257 2024-09-18T14:10:28.282 INFO:tasks.rados.rados.0.smithi084.stdout:2450: seq_num 857 ranges {403599=719929,1916480=441635} 2024-09-18T14:10:28.288 INFO:tasks.rados.rados.0.smithi084.stdout:2450: writing smithi08434663-37 from 403599 to 1123528 tid 1 2024-09-18T14:10:28.295 INFO:tasks.rados.rados.0.smithi084.stdout:2450: writing smithi08434663-37 from 1916480 to 2358115 tid 2 2024-09-18T14:10:28.296 INFO:tasks.rados.rados.0.smithi084.stdout:2451: rollback oid 17 current snap is 257 2024-09-18T14:10:28.296 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 17 to 255 2024-09-18T14:10:28.296 INFO:tasks.rados.rados.0.smithi084.stdout:2448: expect (ObjNum 843 snap 255 seq_num 843) 2024-09-18T14:10:28.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.339+0000 7f812902b640 10 monclient: get_monmap_and_config 2024-09-18T14:10:28.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.339+0000 7f812902b640 10 monclient: build_initial_monmap 2024-09-18T14:10:28.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.339+0000 7f812902b640 10 monclient: monmap: 2024-09-18T14:10:28.339 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:10:28.340427+0000 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:10:28.340427+0000 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:28.341 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:28.341 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 1 Processor -- start 2024-09-18T14:10:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 1 -- start start 2024-09-18T14:10:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 10 monclient: init 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 10 monclient: _add_conns ranks=[8,4,1,6,7,2,0,3,5] 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 10 monclient(hunting): picked mon.noname-e con 0x7f812409f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 10 monclient(hunting): picked mon.noname-c con 0x7f812409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 10 monclient(hunting): picked mon.noname-f con 0x7f812409ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812405ed70 con 0x7f812409ff70 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812409dfc0 con 0x7f812409fba0 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812409dce0 con 0x7f812409f7d0 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812902b640 10 monclient(hunting): authenticate will time out at 1443.678614s 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812359e640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7f812409ff70 legacy=0x7f812413b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f8122d9d640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f812409f7d0 0x7f8124132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:48306/0 (socket says 172.21.15.45:48306) 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f812259c640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f812409fba0 0x7f8124137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:54632/0 (socket says 172.21.15.45:54632) 2024-09-18T14:10:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f8122d9d640 1 -- 172.21.15.45:0/3996815927 learned_addr learned my addr 172.21.15.45:0/3996815927 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.340+0000 7f8121d9b640 1 -- 172.21.15.45:0/3996815927 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697092587 0 0) 0x7f812409dfc0 con 0x7f812409fba0 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient(hunting): my global_id is 164524 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 1 -- 172.21.15.45:0/3996815927 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f810c0034e0 con 0x7f812409fba0 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 1 -- 172.21.15.45:0/3996815927 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106637064 0 0) 0x7f812409dce0 con 0x7f812409f7d0 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient(hunting): my global_id is 5633 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 1 -- 172.21.15.45:0/3996815927 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f812409dfc0 con 0x7f812409f7d0 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 1 -- 172.21.15.45:0/3996815927 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3542605958 0 0) 0x7f810c0034e0 con 0x7f812409fba0 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 1 -- 172.21.15.45:0/3996815927 >> v1:172.21.15.45:6793/0 conn(0x7f812409f7d0 legacy=0x7f8124132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 1 -- 172.21.15.45:0/3996815927 >> v1:172.21.15.84:6789/0 conn(0x7f812409ff70 legacy=0x7f812413b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 1 monclient: found mon.noname-c 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 1 -- 172.21.15.45:0/3996815927 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81240a1c00 con 0x7f812409fba0 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient: _finish_auth 0 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 1 -- 172.21.15.45:0/3996815927 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8118001830 con 0x7f812409fba0 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f812902b640 5 monclient: authenticate success, global_id 164524 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f812902b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:10:28.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient: dump: 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.352 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.353 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.353 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.353 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f812902b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:10:28.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 1 -- 172.21.15.45:0/3996815927 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8118001a10 con 0x7f812409fba0 2024-09-18T14:10:28.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:28.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 1 -- 172.21.15.45:0/3996815927 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8118004ae0 con 0x7f812409fba0 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f812902b640 10 monclient: get_monmap_and_config success 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f812902b640 4 set_mon_vals no callback set 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f8121d9b640 10 monclient: dump: 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.354 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f812902b640 10 monclient: shutdown 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f812902b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f812902b640 1 -- 172.21.15.45:0/3996815927 >> v1:172.21.15.45:6791/0 conn(0x7f812409fba0 legacy=0x7f8124137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f812902b640 1 -- 172.21.15.45:0/3996815927 shutdown_connections 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.341+0000 7f812902b640 1 -- 172.21.15.45:0/3996815927 >> 172.21.15.45:0/3996815927 conn(0x7f81240630b0 msgr2=0x7f81240634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:28.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.342+0000 7f812902b640 1 -- 172.21.15.45:0/3996815927 shutdown_connections 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.342+0000 7f812902b640 1 -- 172.21.15.45:0/3996815927 wait complete. 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.342+0000 7f812902b640 10 monclient: build_initial_monmap 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.342+0000 7f812902b640 10 monclient: monmap: 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:10:28.356 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.342+0000 7f812902b640 1 Processor -- start 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.342+0000 7f812902b640 1 -- start start 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.342+0000 7f812902b640 10 monclient: init 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f812902b640 10 monclient: _reopen_session rank -1 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f812902b640 10 monclient: _add_conns ranks=[1,4,7,8,5,2,6,0,3] 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f812902b640 10 monclient(hunting): picked mon.noname-f con 0x7f812409f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f812902b640 10 monclient(hunting): picked mon.noname-c con 0x7f812409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:10:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f812902b640 10 monclient(hunting): picked mon.noname-i con 0x7f812409ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f812902b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812409d310 con 0x7f812409f7d0 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f812902b640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81240a2560 con 0x7f812409fba0 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f812902b640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812411fc70 con 0x7f812409ff70 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f812902b640 10 monclient(hunting): _renew_subs 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f812902b640 10 monclient(hunting): authenticate will time out at 1443.681444s 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f8122d9d640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7f812409f7d0 legacy=0x7f8124131650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f812259c640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f812409fba0 0x7f8124146a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:54642/0 (socket says 172.21.15.45:54642) 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f812259c640 1 -- 172.21.15.45:0/3740227725 learned_addr learned my addr 172.21.15.45:0/3740227725 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037061968 0 0) 0x7f81240a2560 con 0x7f812409fba0 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 10 monclient(hunting): my global_id is 164533 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8110003510 con 0x7f812409fba0 2024-09-18T14:10:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828571676 0 0) 0x7f812411fc70 con 0x7f812409ff70 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 10 monclient(hunting): my global_id is 184327 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81240a2560 con 0x7f812409ff70 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1733269217 0 0) 0x7f8110003510 con 0x7f812409fba0 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f812411fc70 con 0x7f812409fba0 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8118001770 con 0x7f812409fba0 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 10 monclient(hunting): dump: 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.359 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.343+0000 7f810affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2339128292 0 0) 0x7f81240a2560 con 0x7f812409ff70 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8110003510 con 0x7f812409ff70 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4233099716 0 0) 0x7f812411fc70 con 0x7f812409fba0 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 >> v1:172.21.15.84:6792/0 conn(0x7f812409ff70 legacy=0x7f812414a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 >> v1:172.21.15.84:6789/0 conn(0x7f812409f7d0 legacy=0x7f8124131650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 10 monclient: _finish_hunting 0 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 1 monclient: found mon.c 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81241203c0 con 0x7f812409fba0 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 10 monclient: _finish_auth 0 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f812902b640 5 monclient: authenticate success, global_id 164533 2024-09-18T14:10:28.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f812902b640 10 monclient: _renew_subs 2024-09-18T14:10:28.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f812902b640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:28.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f812902b640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81241217a0 con 0x7f812409fba0 2024-09-18T14:10:28.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f812902b640 10 monclient: _renew_subs 2024-09-18T14:10:28.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f812902b640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:10:28.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f812902b640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8124133c00 con 0x7f812409fba0 2024-09-18T14:10:28.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8118004d60 con 0x7f812409fba0 2024-09-18T14:10:28.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:10:28.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8118005290 con 0x7f812409fba0 2024-09-18T14:10:28.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:10:28.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 10 monclient: dump: 2024-09-18T14:10:28.362 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.363 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f812159a640 4 set_mon_vals no callback set 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.344+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f81180102e0 con 0x7f812409fba0 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f812902b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f812902b640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f812404c4a0 con 0x7f812411f220 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(534..534 src has 344..534) v4 ==== 3923+0+0 (unknown 1749669119 0 0) 0x7f8118046ed0 con 0x7f812409fba0 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f8122d9d640 10 monclient: get_auth_request con 0x7f811003a000 auth_method 0 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209206889 0 0) 0x7f812404c4a0 con 0x7f812411f220 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 10 monclient(hunting): my global_id is 184336 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f812411fc70 con 0x7f812411f220 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3227020222 0 0) 0x7f812411fc70 con 0x7f812411f220 2024-09-18T14:10:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f812404c4a0 con 0x7f812411f220 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f811c004ca0 con 0x7f812411f220 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 10 monclient: dump: 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.365 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1271678611 0 0) 0x7f812404c4a0 con 0x7f812411f220 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.345+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f812404b720 con 0x7f812411f220 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.346+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f811c0051f0 con 0x7f812411f220 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.346+0000 7f810affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.346+0000 7f810affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.346+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 >> v1:172.21.15.84:6792/0 conn(0x7f812411f220 legacy=0x7f8124099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.346+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 >> v1:172.21.15.84:6792/0 conn(0x7f812411f220 legacy=0x7f8124099a40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:28.368 INFO:tasks.rados.rados.0.smithi084.stdout:2446: finishing copy_from racing read to smithi08434663-20 2024-09-18T14:10:28.368 INFO:tasks.rados.rados.0.smithi084.stdout:2446: finishing copy_from to smithi08434663-20 2024-09-18T14:10:28.368 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 1205 (ObjNum 697 snap 205 seq_num 697) dirty exists 2024-09-18T14:10:28.368 INFO:tasks.rados.rados.0.smithi084.stdout:2450: finishing write tid 1 to smithi08434663-37 2024-09-18T14:10:28.368 INFO:tasks.rados.rados.0.smithi084.stdout:2451: finishing rollback tid 0 to smithi08434663-17 2024-09-18T14:10:28.368 INFO:tasks.rados.rados.0.smithi084.stdout:2451: finishing rollback tid 1 to smithi08434663-17 2024-09-18T14:10:28.368 INFO:tasks.rados.rados.0.smithi084.stdout:2451: finishing rollback tid 2 to smithi08434663-17 2024-09-18T14:10:28.369 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 901 (ObjNum 848 snap 255 seq_num 848) dirty exists 2024-09-18T14:10:28.369 INFO:tasks.rados.rados.0.smithi084.stdout:2450: finishing write tid 2 to smithi08434663-37 2024-09-18T14:10:28.369 INFO:tasks.rados.rados.0.smithi084.stdout:2450: finishing write tid 3 to smithi08434663-37 2024-09-18T14:10:28.369 INFO:tasks.rados.rados.0.smithi084.stdout:2450: finishing write tid 4 to smithi08434663-37 2024-09-18T14:10:28.369 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 1013 (ObjNum 857 snap 257 seq_num 857) dirty exists 2024-09-18T14:10:28.369 INFO:tasks.rados.rados.0.smithi084.stdout:2450: left oid 37 (ObjNum 857 snap 257 seq_num 857) 2024-09-18T14:10:28.369 INFO:tasks.rados.rados.0.smithi084.stdout:2446: done (4 left) 2024-09-18T14:10:28.369 INFO:tasks.rados.rados.0.smithi084.stdout:2447: done (3 left) 2024-09-18T14:10:28.369 INFO:tasks.rados.rados.0.smithi084.stdout:2448: done (2 left) 2024-09-18T14:10:28.369 INFO:tasks.rados.rados.0.smithi084.stdout:2450: done (1 left) 2024-09-18T14:10:28.369 INFO:tasks.rados.rados.0.smithi084.stdout:2451: done (0 left) 2024-09-18T14:10:28.369 INFO:tasks.rados.rados.0.smithi084.stdout:2452: snap_remove snap 255 2024-09-18T14:10:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f812902b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:10:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f812902b640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8124071550 con 0x7f812402a8d0 2024-09-18T14:10:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723055866 0 0) 0x7f8124071550 con 0x7f812402a8d0 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 10 monclient(hunting): my global_id is 184345 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f812404c4a0 con 0x7f812402a8d0 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2504779739 0 0) 0x7f812404c4a0 con 0x7f812402a8d0 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8124071550 con 0x7f812402a8d0 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f811c0059d0 con 0x7f812402a8d0 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 10 monclient: dump: 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1088836286 0 0) 0x7f8124071550 con 0x7f812402a8d0 2024-09-18T14:10:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.352+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f812409d670 con 0x7f812402a8d0 2024-09-18T14:10:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7375 (unknown 0 0 1439781331) 0x7f811c005d70 con 0x7f812402a8d0 2024-09-18T14:10:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f810affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:10:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f810affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:10:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 >> v1:172.21.15.84:6792/0 conn(0x7f812402a8d0 legacy=0x7f812409a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f810affd640 1 -- 172.21.15.45:0/3740227725 >> v1:172.21.15.84:6792/0 conn(0x7f812402a8d0 legacy=0x7f812409a150 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:10:28.373 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:10:28.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:28.373 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 98, 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.374 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.375 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:10:28.376 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:10:28.376 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:28.376 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:28.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:28.376 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:28.376 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:28.376 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:28.376 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:28.376 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:28.376 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:28.376 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:10:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:10:28.378 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.379 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.381 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:10:28.381 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.381 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.381 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.381 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:10:28.381 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:10:28.381 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.381 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.381 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:10:28.382 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:10:28.383 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.384 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.385 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:10:28.385 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.385 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.385 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.385 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:10:28.385 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:10:28.385 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.385 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.385 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.386 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.387 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:10:28.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:28.391 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:10:28.392 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:10:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f812902b640 1 -- 172.21.15.45:0/3740227725 >> v1:172.21.15.84:6812/31478 conn(0x7f811003a000 legacy=0x7f811003c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f812902b640 10 monclient: shutdown 2024-09-18T14:10:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f812902b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:10:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f812902b640 1 -- 172.21.15.45:0/3740227725 >> v1:172.21.15.45:6791/0 conn(0x7f812409fba0 legacy=0x7f8124146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:10:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f812359e640 1 -- 172.21.15.45:0/3740227725 reap_dead start 2024-09-18T14:10:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f812902b640 1 -- 172.21.15.45:0/3740227725 shutdown_connections 2024-09-18T14:10:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f812902b640 1 -- 172.21.15.45:0/3740227725 >> 172.21.15.45:0/3740227725 conn(0x7f81240630b0 msgr2=0x7f81240a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:10:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f812902b640 1 -- 172.21.15.45:0/3740227725 shutdown_connections 2024-09-18T14:10:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:10:28.353+0000 7f812902b640 1 -- 172.21.15.45:0/3740227725 wait complete. 2024-09-18T14:10:28.394 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:10:28.395 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-18T14:10:29.285 INFO:tasks.rados.rados.0.smithi084.stdout:2452: done (0 left) 2024-09-18T14:10:29.285 INFO:tasks.rados.rados.0.smithi084.stdout:2453: delete oid 18 current snap is 257 2024-09-18T14:10:29.290 INFO:tasks.rados.rados.0.smithi084.stdout:2453: done (0 left) 2024-09-18T14:10:29.291 INFO:tasks.rados.rados.0.smithi084.stdout:2454: read oid 20 snap -1 2024-09-18T14:10:29.291 INFO:tasks.rados.rados.0.smithi084.stdout:2454: expect (ObjNum 697 snap 205 seq_num 697) 2024-09-18T14:10:29.291 INFO:tasks.rados.rados.0.smithi084.stdout:2455: read oid 47 snap -1 2024-09-18T14:10:29.291 INFO:tasks.rados.rados.0.smithi084.stdout:2455: expect deleted 2024-09-18T14:10:29.291 INFO:tasks.rados.rados.0.smithi084.stdout:2456: write (excl) oid 10 current snap is 257 2024-09-18T14:10:29.291 INFO:tasks.rados.rados.0.smithi084.stdout:2456: seq_num 859 ranges {437213=470427,1633444=728951} 2024-09-18T14:10:29.295 INFO:tasks.rados.rados.0.smithi084.stdout:2456: writing smithi08434663-10 from 437213 to 907640 tid 1 2024-09-18T14:10:29.304 INFO:tasks.rados.rados.0.smithi084.stdout:2456: writing smithi08434663-10 from 1633444 to 2362395 tid 2 2024-09-18T14:10:29.304 INFO:tasks.rados.rados.0.smithi084.stdout:2457: write (excl) oid 18 current snap is 257 2024-09-18T14:10:29.304 INFO:tasks.rados.rados.0.smithi084.stdout:2457: seq_num 860 ranges {454020=662029,1691926=588962} 2024-09-18T14:10:29.310 INFO:tasks.rados.rados.0.smithi084.stdout:2457: writing smithi08434663-18 from 454020 to 1116049 tid 1 2024-09-18T14:10:29.316 INFO:tasks.rados.rados.0.smithi084.stdout:2457: writing smithi08434663-18 from 1691926 to 2280888 tid 2 2024-09-18T14:10:29.317 INFO:tasks.rados.rados.0.smithi084.stdout:2458: rollback oid 17 current snap is 257 2024-09-18T14:10:29.317 INFO:tasks.rados.rados.0.smithi084.stdout:2458: done (4 left) 2024-09-18T14:10:29.317 INFO:tasks.rados.rados.0.smithi084.stdout:2459: read oid 5 snap -1 2024-09-18T14:10:29.317 INFO:tasks.rados.rados.0.smithi084.stdout:2459: expect deleted 2024-09-18T14:10:29.317 INFO:tasks.rados.rados.0.smithi084.stdout:2460: snap_create 2024-09-18T14:10:29.317 INFO:tasks.rados.rados.0.smithi084.stdout:2456: finishing write tid 1 to smithi08434663-10 2024-09-18T14:10:29.317 INFO:tasks.rados.rados.0.smithi084.stdout:2454: expect (ObjNum 697 snap 205 seq_num 697) 2024-09-18T14:10:29.484 INFO:tasks.rados.rados.0.smithi084.stdout:2456: finishing write tid 2 to smithi08434663-10 2024-09-18T14:10:29.484 INFO:tasks.rados.rados.0.smithi084.stdout:2456: finishing write tid 3 to smithi08434663-10 2024-09-18T14:10:29.484 INFO:tasks.rados.rados.0.smithi084.stdout:2456: finishing write tid 4 to smithi08434663-10 2024-09-18T14:10:29.485 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 894 (ObjNum 859 snap 257 seq_num 859) dirty exists 2024-09-18T14:10:29.485 INFO:tasks.rados.rados.0.smithi084.stdout:2456: left oid 10 (ObjNum 859 snap 257 seq_num 859) 2024-09-18T14:10:29.485 INFO:tasks.rados.rados.0.smithi084.stdout:2457: finishing write tid 1 to smithi08434663-18 2024-09-18T14:10:29.485 INFO:tasks.rados.rados.0.smithi084.stdout:2457: finishing write tid 2 to smithi08434663-18 2024-09-18T14:10:29.485 INFO:tasks.rados.rados.0.smithi084.stdout:2457: finishing write tid 3 to smithi08434663-18 2024-09-18T14:10:29.485 INFO:tasks.rados.rados.0.smithi084.stdout:2457: finishing write tid 4 to smithi08434663-18 2024-09-18T14:10:29.485 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 787 (ObjNum 860 snap 257 seq_num 860) dirty exists 2024-09-18T14:10:29.485 INFO:tasks.rados.rados.0.smithi084.stdout:2457: left oid 18 (ObjNum 860 snap 257 seq_num 860) 2024-09-18T14:10:30.290 INFO:tasks.rados.rados.0.smithi084.stdout:2454: done (5 left) 2024-09-18T14:10:30.291 INFO:tasks.rados.rados.0.smithi084.stdout:2455: done (4 left) 2024-09-18T14:10:30.291 INFO:tasks.rados.rados.0.smithi084.stdout:2456: done (3 left) 2024-09-18T14:10:30.291 INFO:tasks.rados.rados.0.smithi084.stdout:2457: done (2 left) 2024-09-18T14:10:30.291 INFO:tasks.rados.rados.0.smithi084.stdout:2459: done (1 left) 2024-09-18T14:10:30.291 INFO:tasks.rados.rados.0.smithi084.stdout:2460: done (0 left) 2024-09-18T14:10:30.291 INFO:tasks.rados.rados.0.smithi084.stdout:2461: read oid 50 snap 257 2024-09-18T14:10:30.291 INFO:tasks.rados.rados.0.smithi084.stdout:2461: expect (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:10:30.291 INFO:tasks.rados.rados.0.smithi084.stdout:2462: copy_from oid 16 from oid 37 current snap is 258 2024-09-18T14:10:30.291 INFO:tasks.rados.rados.0.smithi084.stdout:2463: write (excl) oid 22 current snap is 258 2024-09-18T14:10:30.291 INFO:tasks.rados.rados.0.smithi084.stdout:2463: seq_num 861 ranges {470827=737278,1750408=720419} 2024-09-18T14:10:30.298 INFO:tasks.rados.rados.0.smithi084.stdout:2463: writing smithi08434663-22 from 470827 to 1208105 tid 1 2024-09-18T14:10:30.306 INFO:tasks.rados.rados.0.smithi084.stdout:2463: writing smithi08434663-22 from 1750408 to 2470827 tid 2 2024-09-18T14:10:30.306 INFO:tasks.rados.rados.0.smithi084.stdout:2464: delete oid 24 current snap is 258 2024-09-18T14:10:30.306 INFO:tasks.rados.rados.0.smithi084.stdout:2461: expect (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:10:30.474 INFO:tasks.rados.rados.0.smithi084.stdout:2463: finishing write tid 1 to smithi08434663-22 2024-09-18T14:10:30.474 INFO:tasks.rados.rados.0.smithi084.stdout:2462: finishing copy_from racing read to smithi08434663-16 2024-09-18T14:10:30.474 INFO:tasks.rados.rados.0.smithi084.stdout:2462: finishing copy_from to smithi08434663-16 2024-09-18T14:10:30.474 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1013 (ObjNum 857 snap 257 seq_num 857) dirty exists 2024-09-18T14:10:30.474 INFO:tasks.rados.rados.0.smithi084.stdout:2463: finishing write tid 2 to smithi08434663-22 2024-09-18T14:10:30.474 INFO:tasks.rados.rados.0.smithi084.stdout:2463: finishing write tid 3 to smithi08434663-22 2024-09-18T14:10:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:2463: finishing write tid 4 to smithi08434663-22 2024-09-18T14:10:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 790 (ObjNum 861 snap 258 seq_num 861) dirty exists 2024-09-18T14:10:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:2463: left oid 22 (ObjNum 861 snap 258 seq_num 861) 2024-09-18T14:10:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:2461: done (3 left) 2024-09-18T14:10:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:2462: done (2 left) 2024-09-18T14:10:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:2463: done (1 left) 2024-09-18T14:10:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:2464: done (0 left) 2024-09-18T14:10:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:2465: read oid 44 snap 257 2024-09-18T14:10:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:2465: expect deleted 2024-09-18T14:10:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:2466: read oid 1 snap -1 2024-09-18T14:10:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:2466: expect (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:10:30.475 INFO:tasks.rados.rados.0.smithi084.stdout:2467: rollback oid 8 current snap is 258 2024-09-18T14:10:30.476 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 8 to 257 2024-09-18T14:10:30.476 INFO:tasks.rados.rados.0.smithi084.stdout:2468: write (excl) oid 34 current snap is 258 2024-09-18T14:10:30.476 INFO:tasks.rados.rados.0.smithi084.stdout:2468: seq_num 863 ranges {504441=487776,1467372=736289} 2024-09-18T14:10:30.479 INFO:tasks.rados.rados.0.smithi084.stdout:2468: writing smithi08434663-34 from 504441 to 992217 tid 1 2024-09-18T14:10:30.485 INFO:tasks.rados.rados.0.smithi084.stdout:2468: writing smithi08434663-34 from 1467372 to 2203661 tid 2 2024-09-18T14:10:30.485 INFO:tasks.rados.rados.0.smithi084.stdout:2469: write oid 19 current snap is 258 2024-09-18T14:10:30.485 INFO:tasks.rados.rados.0.smithi084.stdout:2469: seq_num 864 ranges {521248=563025,1809501=596300} 2024-09-18T14:10:30.491 INFO:tasks.rados.rados.0.smithi084.stdout:2469: writing smithi08434663-19 from 521248 to 1084273 tid 1 2024-09-18T14:10:30.498 INFO:tasks.rados.rados.0.smithi084.stdout:2469: writing smithi08434663-19 from 1809501 to 2405801 tid 2 2024-09-18T14:10:30.499 INFO:tasks.rados.rados.0.smithi084.stdout:2470: rollback oid 14 current snap is 258 2024-09-18T14:10:30.499 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 14 to 257 2024-09-18T14:10:30.499 INFO:tasks.rados.rados.0.smithi084.stdout:2467: finishing rollback tid 1 to smithi08434663-8 2024-09-18T14:10:30.499 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:10:30.499 INFO:tasks.rados.rados.0.smithi084.stdout:2466: expect (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:10:30.667 INFO:tasks.rados.rados.0.smithi084.stdout:2468: finishing write tid 1 to smithi08434663-34 2024-09-18T14:10:30.667 INFO:tasks.rados.rados.0.smithi084.stdout:2468: finishing write tid 2 to smithi08434663-34 2024-09-18T14:10:30.667 INFO:tasks.rados.rados.0.smithi084.stdout:2468: finishing write tid 3 to smithi08434663-34 2024-09-18T14:10:30.667 INFO:tasks.rados.rados.0.smithi084.stdout:2468: finishing write tid 4 to smithi08434663-34 2024-09-18T14:10:30.667 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1113 (ObjNum 863 snap 258 seq_num 863) dirty exists 2024-09-18T14:10:30.667 INFO:tasks.rados.rados.0.smithi084.stdout:2468: left oid 34 (ObjNum 863 snap 258 seq_num 863) 2024-09-18T14:10:30.667 INFO:tasks.rados.rados.0.smithi084.stdout:2469: finishing write tid 1 to smithi08434663-19 2024-09-18T14:10:30.667 INFO:tasks.rados.rados.0.smithi084.stdout:2470: finishing rollback tid 1 to smithi08434663-14 2024-09-18T14:10:30.667 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 0 (ObjNum 1169 snap 0 seq_num 2495257168) dirty dne 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2465: done (5 left) 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2466: done (4 left) 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2467: done (3 left) 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2468: done (2 left) 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2470: done (1 left) 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2471: read oid 7 snap 257 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2471: expect (ObjNum 856 snap 256 seq_num 856) 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2472: write oid 17 current snap is 258 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2469: finishing write tid 2 to smithi08434663-19 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2469: finishing write tid 3 to smithi08434663-19 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2469: finishing write tid 4 to smithi08434663-19 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1221 (ObjNum 864 snap 258 seq_num 864) dirty exists 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2469: left oid 19 (ObjNum 864 snap 258 seq_num 864) 2024-09-18T14:10:30.668 INFO:tasks.rados.rados.0.smithi084.stdout:2472: seq_num 865 ranges {538055=638274,1867983=670072} 2024-09-18T14:10:30.673 INFO:tasks.rados.rados.0.smithi084.stdout:2472: writing smithi08434663-17 from 538055 to 1176329 tid 1 2024-09-18T14:10:30.681 INFO:tasks.rados.rados.0.smithi084.stdout:2472: writing smithi08434663-17 from 1867983 to 2538055 tid 2 2024-09-18T14:10:30.682 INFO:tasks.rados.rados.0.smithi084.stdout:2469: done (2 left) 2024-09-18T14:10:30.682 INFO:tasks.rados.rados.0.smithi084.stdout:2473: snap_create 2024-09-18T14:10:30.682 INFO:tasks.rados.rados.0.smithi084.stdout:2471: expect (ObjNum 856 snap 256 seq_num 856) 2024-09-18T14:10:30.777 INFO:tasks.rados.rados.0.smithi084.stdout:2472: finishing write tid 1 to smithi08434663-17 2024-09-18T14:10:30.777 INFO:tasks.rados.rados.0.smithi084.stdout:2472: finishing write tid 2 to smithi08434663-17 2024-09-18T14:10:30.777 INFO:tasks.rados.rados.0.smithi084.stdout:2472: finishing write tid 3 to smithi08434663-17 2024-09-18T14:10:30.777 INFO:tasks.rados.rados.0.smithi084.stdout:2472: finishing write tid 4 to smithi08434663-17 2024-09-18T14:10:30.777 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 904 (ObjNum 865 snap 258 seq_num 865) dirty exists 2024-09-18T14:10:30.777 INFO:tasks.rados.rados.0.smithi084.stdout:2472: left oid 17 (ObjNum 865 snap 258 seq_num 865) 2024-09-18T14:10:31.296 INFO:tasks.rados.rados.0.smithi084.stdout:2471: done (2 left) 2024-09-18T14:10:31.296 INFO:tasks.rados.rados.0.smithi084.stdout:2472: done (1 left) 2024-09-18T14:10:31.296 INFO:tasks.rados.rados.0.smithi084.stdout:2473: done (0 left) 2024-09-18T14:10:31.296 INFO:tasks.rados.rados.0.smithi084.stdout:2474: copy_from oid 27 from oid 12 current snap is 259 2024-09-18T14:10:31.296 INFO:tasks.rados.rados.0.smithi084.stdout:2475: delete oid 30 current snap is 259 2024-09-18T14:10:31.297 INFO:tasks.rados.rados.0.smithi084.stdout:2475: done (1 left) 2024-09-18T14:10:31.297 INFO:tasks.rados.rados.0.smithi084.stdout:2476: read oid 21 snap -1 2024-09-18T14:10:31.297 INFO:tasks.rados.rados.0.smithi084.stdout:2476: expect (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:10:31.297 INFO:tasks.rados.rados.0.smithi084.stdout:2477: delete oid 13 current snap is 259 2024-09-18T14:10:31.302 INFO:tasks.rados.rados.0.smithi084.stdout:2476: expect (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:10:31.470 INFO:tasks.rados.rados.0.smithi084.stdout:2474: finishing copy_from racing read to smithi08434663-27 2024-09-18T14:10:31.470 INFO:tasks.rados.rados.0.smithi084.stdout:2474: finishing copy_from to smithi08434663-27 2024-09-18T14:10:31.470 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1205 (ObjNum 697 snap 205 seq_num 697) dirty exists 2024-09-18T14:10:31.470 INFO:tasks.rados.rados.0.smithi084.stdout:2474: done (2 left) 2024-09-18T14:10:31.471 INFO:tasks.rados.rados.0.smithi084.stdout:2476: done (1 left) 2024-09-18T14:10:31.471 INFO:tasks.rados.rados.0.smithi084.stdout:2477: done (0 left) 2024-09-18T14:10:31.471 INFO:tasks.rados.rados.0.smithi084.stdout:2478: snap_remove snap 258 2024-09-18T14:10:32.304 INFO:tasks.rados.rados.0.smithi084.stdout:2478: done (0 left) 2024-09-18T14:10:32.304 INFO:tasks.rados.rados.0.smithi084.stdout:2479: read oid 38 snap -1 2024-09-18T14:10:32.304 INFO:tasks.rados.rados.0.smithi084.stdout:2479: expect (ObjNum 789 snap 233 seq_num 789) 2024-09-18T14:10:32.304 INFO:tasks.rados.rados.0.smithi084.stdout:2480: delete oid 27 current snap is 259 2024-09-18T14:10:32.312 INFO:tasks.rados.rados.0.smithi084.stdout:2480: done (1 left) 2024-09-18T14:10:32.312 INFO:tasks.rados.rados.0.smithi084.stdout:2481: snap_create 2024-09-18T14:10:32.314 INFO:tasks.rados.rados.0.smithi084.stdout:2479: expect (ObjNum 789 snap 233 seq_num 789) 2024-09-18T14:10:33.305 INFO:tasks.rados.rados.0.smithi084.stdout:2479: done (1 left) 2024-09-18T14:10:33.305 INFO:tasks.rados.rados.0.smithi084.stdout:2481: done (0 left) 2024-09-18T14:10:33.305 INFO:tasks.rados.rados.0.smithi084.stdout:2482: rollback oid 14 current snap is 260 2024-09-18T14:10:33.305 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 14 to 259 2024-09-18T14:10:33.305 INFO:tasks.rados.rados.0.smithi084.stdout:2483: rollback oid 32 current snap is 260 2024-09-18T14:10:33.306 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 32 to 259 2024-09-18T14:10:33.306 INFO:tasks.rados.rados.0.smithi084.stdout:2484: snap_remove snap 257 2024-09-18T14:10:34.315 INFO:tasks.rados.rados.0.smithi084.stdout:2484: done (2 left) 2024-09-18T14:10:34.315 INFO:tasks.rados.rados.0.smithi084.stdout:2485: write (excl) oid 24 current snap is 260 2024-09-18T14:10:34.315 INFO:tasks.rados.rados.0.smithi084.stdout:2483: finishing rollback tid 1 to smithi08434663-32 2024-09-18T14:10:34.316 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:10:34.316 INFO:tasks.rados.rados.0.smithi084.stdout:2485: seq_num 869 ranges {605283=655623,1701911=747296} 2024-09-18T14:10:34.323 INFO:tasks.rados.rados.0.smithi084.stdout:2485: writing smithi08434663-24 from 605283 to 1260906 tid 1 2024-09-18T14:10:34.331 INFO:tasks.rados.rados.0.smithi084.stdout:2485: writing smithi08434663-24 from 1701911 to 2449207 tid 2 2024-09-18T14:10:34.331 INFO:tasks.rados.rados.0.smithi084.stdout:2483: done (2 left) 2024-09-18T14:10:34.331 INFO:tasks.rados.rados.0.smithi084.stdout:2486: rollback oid 28 current snap is 260 2024-09-18T14:10:34.331 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 28 to 259 2024-09-18T14:10:34.331 INFO:tasks.rados.rados.0.smithi084.stdout:2482: finishing rollback tid 1 to smithi08434663-14 2024-09-18T14:10:34.331 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:10:34.331 INFO:tasks.rados.rados.0.smithi084.stdout:2485: finishing write tid 1 to smithi08434663-24 2024-09-18T14:10:34.331 INFO:tasks.rados.rados.0.smithi084.stdout:2482: done (2 left) 2024-09-18T14:10:34.331 INFO:tasks.rados.rados.0.smithi084.stdout:2487: read oid 40 snap 259 2024-09-18T14:10:34.331 INFO:tasks.rados.rados.0.smithi084.stdout:2487: expect (ObjNum 838 snap 252 seq_num 838) 2024-09-18T14:10:34.331 INFO:tasks.rados.rados.0.smithi084.stdout:2488: rollback oid 34 current snap is 260 2024-09-18T14:10:34.332 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 34 to 259 2024-09-18T14:10:34.332 INFO:tasks.rados.rados.0.smithi084.stdout:2489: snap_create 2024-09-18T14:10:34.335 INFO:tasks.rados.rados.0.smithi084.stdout:2485: finishing write tid 2 to smithi08434663-24 2024-09-18T14:10:34.337 INFO:tasks.rados.rados.0.smithi084.stdout:2485: finishing write tid 3 to smithi08434663-24 2024-09-18T14:10:34.337 INFO:tasks.rados.rados.0.smithi084.stdout:2485: finishing write tid 4 to smithi08434663-24 2024-09-18T14:10:34.337 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 586 (ObjNum 869 snap 260 seq_num 869) dirty exists 2024-09-18T14:10:34.337 INFO:tasks.rados.rados.0.smithi084.stdout:2485: left oid 24 (ObjNum 869 snap 260 seq_num 869) 2024-09-18T14:10:34.337 INFO:tasks.rados.rados.0.smithi084.stdout:2486: finishing rollback tid 0 to smithi08434663-28 2024-09-18T14:10:34.337 INFO:tasks.rados.rados.0.smithi084.stdout:2486: finishing rollback tid 1 to smithi08434663-28 2024-09-18T14:10:34.337 INFO:tasks.rados.rados.0.smithi084.stdout:2486: finishing rollback tid 2 to smithi08434663-28 2024-09-18T14:10:34.337 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1228 (ObjNum 765 snap 223 seq_num 765) dirty exists 2024-09-18T14:10:34.337 INFO:tasks.rados.rados.0.smithi084.stdout:2488: finishing rollback tid 0 to smithi08434663-34 2024-09-18T14:10:34.337 INFO:tasks.rados.rados.0.smithi084.stdout:2488: finishing rollback tid 1 to smithi08434663-34 2024-09-18T14:10:34.337 INFO:tasks.rados.rados.0.smithi084.stdout:2488: finishing rollback tid 2 to smithi08434663-34 2024-09-18T14:10:34.338 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1117 (ObjNum 863 snap 258 seq_num 863) dirty exists 2024-09-18T14:10:34.338 INFO:tasks.rados.rados.0.smithi084.stdout:2487: expect (ObjNum 838 snap 252 seq_num 838) 2024-09-18T14:10:35.332 INFO:tasks.rados.rados.0.smithi084.stdout:2485: done (4 left) 2024-09-18T14:10:35.332 INFO:tasks.rados.rados.0.smithi084.stdout:2486: done (3 left) 2024-09-18T14:10:35.332 INFO:tasks.rados.rados.0.smithi084.stdout:2487: done (2 left) 2024-09-18T14:10:35.333 INFO:tasks.rados.rados.0.smithi084.stdout:2488: done (1 left) 2024-09-18T14:10:35.333 INFO:tasks.rados.rados.0.smithi084.stdout:2489: done (0 left) 2024-09-18T14:10:35.333 INFO:tasks.rados.rados.0.smithi084.stdout:2490: delete oid 29 current snap is 261 2024-09-18T14:10:35.337 INFO:tasks.rados.rados.0.smithi084.stdout:2490: done (0 left) 2024-09-18T14:10:35.337 INFO:tasks.rados.rados.0.smithi084.stdout:2491: write (excl) oid 6 current snap is 261 2024-09-18T14:10:35.337 INFO:tasks.rados.rados.0.smithi084.stdout:2491: seq_num 871 ranges {638897=406121,1818875=750965} 2024-09-18T14:10:35.342 INFO:tasks.rados.rados.0.smithi084.stdout:2491: writing smithi08434663-6 from 638897 to 1045018 tid 1 2024-09-18T14:10:35.351 INFO:tasks.rados.rados.0.smithi084.stdout:2491: writing smithi08434663-6 from 1818875 to 2569840 tid 2 2024-09-18T14:10:35.351 INFO:tasks.rados.rados.0.smithi084.stdout:2492: read oid 3 snap -1 2024-09-18T14:10:35.351 INFO:tasks.rados.rados.0.smithi084.stdout:2492: expect (ObjNum 825 snap 251 seq_num 825) 2024-09-18T14:10:35.351 INFO:tasks.rados.rados.0.smithi084.stdout:2493: delete oid 41 current snap is 261 2024-09-18T14:10:35.351 INFO:tasks.rados.rados.0.smithi084.stdout:2491: finishing write tid 1 to smithi08434663-6 2024-09-18T14:10:35.355 INFO:tasks.rados.rados.0.smithi084.stdout:2493: done (2 left) 2024-09-18T14:10:35.356 INFO:tasks.rados.rados.0.smithi084.stdout:2494: read oid 31 snap 259 2024-09-18T14:10:35.356 INFO:tasks.rados.rados.0.smithi084.stdout:2494: expect deleted 2024-09-18T14:10:35.356 INFO:tasks.rados.rados.0.smithi084.stdout:2495: write (excl) oid 32 current snap is 261 2024-09-18T14:10:35.356 INFO:tasks.rados.rados.0.smithi084.stdout:2495: seq_num 873 ranges {672511=556619,1819486=754634} 2024-09-18T14:10:35.362 INFO:tasks.rados.rados.0.smithi084.stdout:2495: writing smithi08434663-32 from 672511 to 1229130 tid 1 2024-09-18T14:10:35.370 INFO:tasks.rados.rados.0.smithi084.stdout:2495: writing smithi08434663-32 from 1819486 to 2574120 tid 2 2024-09-18T14:10:35.370 INFO:tasks.rados.rados.0.smithi084.stdout:2496: rollback oid 28 current snap is 261 2024-09-18T14:10:35.370 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 28 to 260 2024-09-18T14:10:35.370 INFO:tasks.rados.rados.0.smithi084.stdout:2491: finishing write tid 2 to smithi08434663-6 2024-09-18T14:10:35.370 INFO:tasks.rados.rados.0.smithi084.stdout:2491: finishing write tid 3 to smithi08434663-6 2024-09-18T14:10:35.370 INFO:tasks.rados.rados.0.smithi084.stdout:2491: finishing write tid 4 to smithi08434663-6 2024-09-18T14:10:35.370 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 963 (ObjNum 871 snap 261 seq_num 871) dirty exists 2024-09-18T14:10:35.371 INFO:tasks.rados.rados.0.smithi084.stdout:2491: left oid 6 (ObjNum 871 snap 261 seq_num 871) 2024-09-18T14:10:35.371 INFO:tasks.rados.rados.0.smithi084.stdout:2492: expect (ObjNum 825 snap 251 seq_num 825) 2024-09-18T14:10:35.584 INFO:tasks.rados.rados.0.smithi084.stdout:2495: finishing write tid 1 to smithi08434663-32 2024-09-18T14:10:35.584 INFO:tasks.rados.rados.0.smithi084.stdout:2495: finishing write tid 2 to smithi08434663-32 2024-09-18T14:10:35.584 INFO:tasks.rados.rados.0.smithi084.stdout:2495: finishing write tid 3 to smithi08434663-32 2024-09-18T14:10:35.584 INFO:tasks.rados.rados.0.smithi084.stdout:2495: finishing write tid 4 to smithi08434663-32 2024-09-18T14:10:35.584 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 1231 (ObjNum 873 snap 261 seq_num 873) dirty exists 2024-09-18T14:10:35.584 INFO:tasks.rados.rados.0.smithi084.stdout:2495: left oid 32 (ObjNum 873 snap 261 seq_num 873) 2024-09-18T14:10:35.584 INFO:tasks.rados.rados.0.smithi084.stdout:2496: finishing rollback tid 0 to smithi08434663-28 2024-09-18T14:10:35.584 INFO:tasks.rados.rados.0.smithi084.stdout:2491: done (4 left) 2024-09-18T14:10:35.584 INFO:tasks.rados.rados.0.smithi084.stdout:2492: done (3 left) 2024-09-18T14:10:35.584 INFO:tasks.rados.rados.0.smithi084.stdout:2494: done (2 left) 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:2495: done (1 left) 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:2497: read oid 20 snap -1 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:2497: expect (ObjNum 697 snap 205 seq_num 697) 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:2498: read oid 41 snap -1 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:2498: expect deleted 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:2499: rollback oid 33 current snap is 261 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 33 to 260 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:2496: finishing rollback tid 1 to smithi08434663-28 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:2496: finishing rollback tid 2 to smithi08434663-28 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1235 (ObjNum 765 snap 223 seq_num 765) dirty exists 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:2496: done (3 left) 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:2500: write oid 16 current snap is 261 2024-09-18T14:10:35.585 INFO:tasks.rados.rados.0.smithi084.stdout:2500: seq_num 874 ranges {689318=631868,1877968=498292} 2024-09-18T14:10:35.591 INFO:tasks.rados.rados.0.smithi084.stdout:2500: writing smithi08434663-16 from 689318 to 1321186 tid 1 2024-09-18T14:10:35.597 INFO:tasks.rados.rados.0.smithi084.stdout:2500: writing smithi08434663-16 from 1877968 to 2376260 tid 2 2024-09-18T14:10:35.597 INFO:tasks.rados.rados.0.smithi084.stdout:2501: read oid 7 snap 260 2024-09-18T14:10:35.598 INFO:tasks.rados.rados.0.smithi084.stdout:2501: expect (ObjNum 856 snap 256 seq_num 856) 2024-09-18T14:10:35.598 INFO:tasks.rados.rados.0.smithi084.stdout:2502: snap_create 2024-09-18T14:10:35.598 INFO:tasks.rados.rados.0.smithi084.stdout:2499: finishing rollback tid 0 to smithi08434663-33 2024-09-18T14:10:35.598 INFO:tasks.rados.rados.0.smithi084.stdout:2499: finishing rollback tid 1 to smithi08434663-33 2024-09-18T14:10:35.598 INFO:tasks.rados.rados.0.smithi084.stdout:2499: finishing rollback tid 2 to smithi08434663-33 2024-09-18T14:10:35.598 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 1016 (ObjNum 847 snap 255 seq_num 847) dirty exists 2024-09-18T14:10:35.598 INFO:tasks.rados.rados.0.smithi084.stdout:2497: expect (ObjNum 697 snap 205 seq_num 697) 2024-09-18T14:10:35.765 INFO:tasks.rados.rados.0.smithi084.stdout:2500: finishing write tid 1 to smithi08434663-16 2024-09-18T14:10:35.765 INFO:tasks.rados.rados.0.smithi084.stdout:2500: finishing write tid 2 to smithi08434663-16 2024-09-18T14:10:35.766 INFO:tasks.rados.rados.0.smithi084.stdout:2500: finishing write tid 3 to smithi08434663-16 2024-09-18T14:10:35.766 INFO:tasks.rados.rados.0.smithi084.stdout:2500: finishing write tid 4 to smithi08434663-16 2024-09-18T14:10:35.766 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1055 (ObjNum 874 snap 261 seq_num 874) dirty exists 2024-09-18T14:10:35.766 INFO:tasks.rados.rados.0.smithi084.stdout:2500: left oid 16 (ObjNum 874 snap 261 seq_num 874) 2024-09-18T14:10:35.766 INFO:tasks.rados.rados.0.smithi084.stdout:2501: expect (ObjNum 856 snap 256 seq_num 856) 2024-09-18T14:10:36.338 INFO:tasks.rados.rados.0.smithi084.stdout:2497: done (5 left) 2024-09-18T14:10:36.338 INFO:tasks.rados.rados.0.smithi084.stdout:2498: done (4 left) 2024-09-18T14:10:36.338 INFO:tasks.rados.rados.0.smithi084.stdout:2499: done (3 left) 2024-09-18T14:10:36.338 INFO:tasks.rados.rados.0.smithi084.stdout:2500: done (2 left) 2024-09-18T14:10:36.338 INFO:tasks.rados.rados.0.smithi084.stdout:2501: done (1 left) 2024-09-18T14:10:36.338 INFO:tasks.rados.rados.0.smithi084.stdout:2502: done (0 left) 2024-09-18T14:10:36.338 INFO:tasks.rados.rados.0.smithi084.stdout:2503: snap_create 2024-09-18T14:10:37.275 INFO:tasks.rados.rados.0.smithi084.stdout:2503: done (0 left) 2024-09-18T14:10:37.275 INFO:tasks.rados.rados.0.smithi084.stdout:2504: read oid 8 snap -1 2024-09-18T14:10:37.275 INFO:tasks.rados.rados.0.smithi084.stdout:2504: expect deleted 2024-09-18T14:10:37.275 INFO:tasks.rados.rados.0.smithi084.stdout:2505: snap_remove snap 259 2024-09-18T14:10:38.276 INFO:tasks.rados.rados.0.smithi084.stdout:2505: done (1 left) 2024-09-18T14:10:38.276 INFO:tasks.rados.rados.0.smithi084.stdout:2506: snap_create 2024-09-18T14:10:39.286 INFO:tasks.rados.rados.0.smithi084.stdout:2504: done (1 left) 2024-09-18T14:10:39.286 INFO:tasks.rados.rados.0.smithi084.stdout:2506: done (0 left) 2024-09-18T14:10:39.286 INFO:tasks.rados.rados.0.smithi084.stdout:2507: write (excl) oid 24 current snap is 264 2024-09-18T14:10:39.286 INFO:tasks.rados.rados.0.smithi084.stdout:2507: seq_num 875 ranges {706125=423470,1652803=758303} 2024-09-18T14:10:39.292 INFO:tasks.rados.rados.0.smithi084.stdout:2507: writing smithi08434663-24 from 706125 to 1129595 tid 1 2024-09-18T14:10:39.300 INFO:tasks.rados.rados.0.smithi084.stdout:2507: writing smithi08434663-24 from 1652803 to 2411106 tid 2 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:2508: read oid 12 snap -1 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:2508: expect (ObjNum 697 snap 205 seq_num 697) 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:2509: rollback oid 23 current snap is 264 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 23 to 261 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:2507: finishing write tid 1 to smithi08434663-24 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:2510: read oid 17 snap -1 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:2510: expect (ObjNum 865 snap 258 seq_num 865) 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:2511: read oid 36 snap -1 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:2511: expect deleted 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:2512: rollback oid 7 current snap is 264 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 7 to 260 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:2513: read oid 3 snap -1 2024-09-18T14:10:39.301 INFO:tasks.rados.rados.0.smithi084.stdout:2513: expect (ObjNum 825 snap 251 seq_num 825) 2024-09-18T14:10:39.302 INFO:tasks.rados.rados.0.smithi084.stdout:2514: copy_from oid 2 from oid 4 current snap is 264 2024-09-18T14:10:39.302 INFO:tasks.rados.rados.0.smithi084.stdout:2515: write (excl) oid 39 current snap is 264 2024-09-18T14:10:39.302 INFO:tasks.rados.rados.0.smithi084.stdout:2515: seq_num 876 ranges {722932=498719,1994932=501961} 2024-09-18T14:10:39.306 INFO:tasks.rados.rados.0.smithi084.stdout:2515: writing smithi08434663-39 from 722932 to 1221651 tid 1 2024-09-18T14:10:39.314 INFO:tasks.rados.rados.0.smithi084.stdout:2515: writing smithi08434663-39 from 1994932 to 2496893 tid 2 2024-09-18T14:10:39.314 INFO:tasks.rados.rados.0.smithi084.stdout:2516: write (excl) oid 44 current snap is 264 2024-09-18T14:10:39.314 INFO:tasks.rados.rados.0.smithi084.stdout:2516: seq_num 877 ranges {739739=573968,2053414=686325} 2024-09-18T14:10:39.321 INFO:tasks.rados.rados.0.smithi084.stdout:2516: writing smithi08434663-44 from 739739 to 1313707 tid 1 2024-09-18T14:10:39.329 INFO:tasks.rados.rados.0.smithi084.stdout:2516: writing smithi08434663-44 from 2053414 to 2739739 tid 2 2024-09-18T14:10:39.329 INFO:tasks.rados.rados.0.smithi084.stdout:2517: write oid 34 current snap is 264 2024-09-18T14:10:39.329 INFO:tasks.rados.rados.0.smithi084.stdout:2517: seq_num 878 ranges {756546=649217,2111896=505630} 2024-09-18T14:10:39.336 INFO:tasks.rados.rados.0.smithi084.stdout:2517: writing smithi08434663-34 from 756546 to 1405763 tid 1 2024-09-18T14:10:39.343 INFO:tasks.rados.rados.0.smithi084.stdout:2517: writing smithi08434663-34 from 2111896 to 2617526 tid 2 2024-09-18T14:10:39.343 INFO:tasks.rados.rados.0.smithi084.stdout:2518: rollback oid 43 current snap is 264 2024-09-18T14:10:39.343 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 261 2024-09-18T14:10:39.343 INFO:tasks.rados.rados.0.smithi084.stdout:2508: expect (ObjNum 697 snap 205 seq_num 697) 2024-09-18T14:10:39.513 INFO:tasks.rados.rados.0.smithi084.stdout:2514: finishing copy_from to smithi08434663-2 2024-09-18T14:10:39.513 INFO:tasks.rados.rados.0.smithi084.stdout:2514: got expected ENOENT (src dne) 2024-09-18T14:10:39.513 INFO:tasks.rados.rados.0.smithi084.stdout:2514: finishing copy_from racing read to smithi08434663-2 2024-09-18T14:10:39.513 INFO:tasks.rados.rados.0.smithi084.stdout:2507: finishing write tid 2 to smithi08434663-24 2024-09-18T14:10:39.513 INFO:tasks.rados.rados.0.smithi084.stdout:2507: finishing write tid 3 to smithi08434663-24 2024-09-18T14:10:39.513 INFO:tasks.rados.rados.0.smithi084.stdout:2507: finishing write tid 4 to smithi08434663-24 2024-09-18T14:10:39.513 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 589 (ObjNum 875 snap 264 seq_num 875) dirty exists 2024-09-18T14:10:39.513 INFO:tasks.rados.rados.0.smithi084.stdout:2507: left oid 24 (ObjNum 875 snap 264 seq_num 875) 2024-09-18T14:10:39.514 INFO:tasks.rados.rados.0.smithi084.stdout:2513: expect (ObjNum 825 snap 251 seq_num 825) 2024-09-18T14:10:39.723 INFO:tasks.rados.rados.0.smithi084.stdout:2515: finishing write tid 1 to smithi08434663-39 2024-09-18T14:10:39.723 INFO:tasks.rados.rados.0.smithi084.stdout:2510: expect (ObjNum 865 snap 258 seq_num 865) 2024-09-18T14:10:39.874 INFO:tasks.rados.rados.0.smithi084.stdout:2509: finishing rollback tid 0 to smithi08434663-23 2024-09-18T14:10:39.874 INFO:tasks.rados.rados.0.smithi084.stdout:2509: finishing rollback tid 1 to smithi08434663-23 2024-09-18T14:10:39.874 INFO:tasks.rados.rados.0.smithi084.stdout:2509: finishing rollback tid 2 to smithi08434663-23 2024-09-18T14:10:39.874 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 966 (ObjNum 827 snap 251 seq_num 827) dirty exists 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:2512: finishing rollback tid 0 to smithi08434663-7 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:2512: finishing rollback tid 1 to smithi08434663-7 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:2512: finishing rollback tid 2 to smithi08434663-7 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 969 (ObjNum 856 snap 256 seq_num 856) dirty exists 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:2515: finishing write tid 2 to smithi08434663-39 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:2515: finishing write tid 3 to smithi08434663-39 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:2515: finishing write tid 4 to smithi08434663-39 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 1061 (ObjNum 876 snap 264 seq_num 876) dirty exists 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:2515: left oid 39 (ObjNum 876 snap 264 seq_num 876) 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:2516: finishing write tid 1 to smithi08434663-44 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:2516: finishing write tid 2 to smithi08434663-44 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:2507: done (11 left) 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:2508: done (10 left) 2024-09-18T14:10:39.875 INFO:tasks.rados.rados.0.smithi084.stdout:2509: done (9 left) 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:2510: done (8 left) 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:2511: done (7 left) 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:2512: done (6 left) 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:2513: done (5 left) 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:2514: done (4 left) 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:2515: done (3 left) 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:2519: read oid 50 snap -1 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:2519: expect (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:2520: rollback oid 45 current snap is 264 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 45 to 260 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:2516: finishing write tid 3 to smithi08434663-44 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:2516: finishing write tid 4 to smithi08434663-44 2024-09-18T14:10:39.876 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 954 (ObjNum 877 snap 264 seq_num 877) dirty exists 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:2516: left oid 44 (ObjNum 877 snap 264 seq_num 877) 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:2517: finishing write tid 1 to smithi08434663-34 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:2518: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:2517: finishing write tid 2 to smithi08434663-34 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:2517: finishing write tid 3 to smithi08434663-34 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:2517: finishing write tid 4 to smithi08434663-34 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1127 (ObjNum 878 snap 264 seq_num 878) dirty exists 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:2517: left oid 34 (ObjNum 878 snap 264 seq_num 878) 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:2516: done (4 left) 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:2517: done (3 left) 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:2518: done (2 left) 2024-09-18T14:10:39.877 INFO:tasks.rados.rados.0.smithi084.stdout:2521: snap_create 2024-09-18T14:10:39.878 INFO:tasks.rados.rados.0.smithi084.stdout:2519: expect (ObjNum 790 snap 234 seq_num 790) 2024-09-18T14:10:40.047 INFO:tasks.rados.rados.0.smithi084.stdout:2520: finishing rollback tid 0 to smithi08434663-45 2024-09-18T14:10:40.047 INFO:tasks.rados.rados.0.smithi084.stdout:2520: finishing rollback tid 1 to smithi08434663-45 2024-09-18T14:10:40.047 INFO:tasks.rados.rados.0.smithi084.stdout:2520: finishing rollback tid 2 to smithi08434663-45 2024-09-18T14:10:40.047 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 1131 (ObjNum 833 snap 252 seq_num 833) dirty exists 2024-09-18T14:10:40.292 INFO:tasks.rados.rados.0.smithi084.stdout:2519: done (2 left) 2024-09-18T14:10:40.293 INFO:tasks.rados.rados.0.smithi084.stdout:2520: done (1 left) 2024-09-18T14:10:40.293 INFO:tasks.rados.rados.0.smithi084.stdout:2521: done (0 left) 2024-09-18T14:10:40.293 INFO:tasks.rados.rados.0.smithi084.stdout:2522: read oid 34 snap 261 2024-09-18T14:10:40.293 INFO:tasks.rados.rados.0.smithi084.stdout:2522: expect (ObjNum 863 snap 258 seq_num 863) 2024-09-18T14:10:40.293 INFO:tasks.rados.rados.0.smithi084.stdout:2523: write oid 15 current snap is 265 2024-09-18T14:10:40.293 INFO:tasks.rados.rados.0.smithi084.stdout:2523: seq_num 879 ranges {773353=724466,2170378=602975} 2024-09-18T14:10:40.300 INFO:tasks.rados.rados.0.smithi084.stdout:2523: writing smithi08434663-15 from 773353 to 1497819 tid 1 2024-09-18T14:10:40.308 INFO:tasks.rados.rados.0.smithi084.stdout:2523: writing smithi08434663-15 from 2170378 to 2773353 tid 2 2024-09-18T14:10:40.308 INFO:tasks.rados.rados.0.smithi084.stdout:2524: read oid 49 snap -1 2024-09-18T14:10:40.308 INFO:tasks.rados.rados.0.smithi084.stdout:2524: expect (ObjNum 836 snap 252 seq_num 836) 2024-09-18T14:10:40.308 INFO:tasks.rados.rados.0.smithi084.stdout:2525: snap_create 2024-09-18T14:10:40.308 INFO:tasks.rados.rados.0.smithi084.stdout:2522: expect (ObjNum 863 snap 258 seq_num 863) 2024-09-18T14:10:40.387 INFO:tasks.rados.rados.0.smithi084.stdout:2523: finishing write tid 1 to smithi08434663-15 2024-09-18T14:10:40.387 INFO:tasks.rados.rados.0.smithi084.stdout:2523: finishing write tid 2 to smithi08434663-15 2024-09-18T14:10:40.387 INFO:tasks.rados.rados.0.smithi084.stdout:2523: finishing write tid 3 to smithi08434663-15 2024-09-18T14:10:40.387 INFO:tasks.rados.rados.0.smithi084.stdout:2523: finishing write tid 4 to smithi08434663-15 2024-09-18T14:10:40.387 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 1058 (ObjNum 879 snap 265 seq_num 879) dirty exists 2024-09-18T14:10:40.387 INFO:tasks.rados.rados.0.smithi084.stdout:2523: left oid 15 (ObjNum 879 snap 265 seq_num 879) 2024-09-18T14:10:40.387 INFO:tasks.rados.rados.0.smithi084.stdout:2524: expect (ObjNum 836 snap 252 seq_num 836) 2024-09-18T14:10:41.294 INFO:tasks.rados.rados.0.smithi084.stdout:2522: done (3 left) 2024-09-18T14:10:41.294 INFO:tasks.rados.rados.0.smithi084.stdout:2523: done (2 left) 2024-09-18T14:10:41.294 INFO:tasks.rados.rados.0.smithi084.stdout:2524: done (1 left) 2024-09-18T14:10:41.294 INFO:tasks.rados.rados.0.smithi084.stdout:2525: done (0 left) 2024-09-18T14:10:41.294 INFO:tasks.rados.rados.0.smithi084.stdout:2526: delete oid 20 current snap is 266 2024-09-18T14:10:41.299 INFO:tasks.rados.rados.0.smithi084.stdout:2526: done (0 left) 2024-09-18T14:10:41.300 INFO:tasks.rados.rados.0.smithi084.stdout:2527: delete oid 1 current snap is 266 2024-09-18T14:10:41.304 INFO:tasks.rados.rados.0.smithi084.stdout:2527: done (0 left) 2024-09-18T14:10:41.304 INFO:tasks.rados.rados.0.smithi084.stdout:2528: copy_from oid 40 from oid 33 current snap is 266 2024-09-18T14:10:41.304 INFO:tasks.rados.rados.0.smithi084.stdout:2529: write (excl) oid 41 current snap is 266 2024-09-18T14:10:41.304 INFO:tasks.rados.rados.0.smithi084.stdout:2529: seq_num 882 ranges {423774=666566,1545824=512968,2704609=119165} 2024-09-18T14:10:41.309 INFO:tasks.rados.rados.0.smithi084.stdout:2529: writing smithi08434663-41 from 423774 to 1090340 tid 1 2024-09-18T14:10:41.315 INFO:tasks.rados.rados.0.smithi084.stdout:2529: writing smithi08434663-41 from 1545824 to 2058792 tid 2 2024-09-18T14:10:41.319 INFO:tasks.rados.rados.0.smithi084.stdout:2529: writing smithi08434663-41 from 2704609 to 2823774 tid 3 2024-09-18T14:10:41.319 INFO:tasks.rados.rados.0.smithi084.stdout:2530: delete oid 3 current snap is 266 2024-09-18T14:10:41.319 INFO:tasks.rados.rados.0.smithi084.stdout:2528: finishing copy_from racing read to smithi08434663-40 2024-09-18T14:10:41.319 INFO:tasks.rados.rados.0.smithi084.stdout:2528: finishing copy_from to smithi08434663-40 2024-09-18T14:10:41.320 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 1016 (ObjNum 847 snap 255 seq_num 847) dirty exists 2024-09-18T14:10:41.320 INFO:tasks.rados.rados.0.smithi084.stdout:2529: finishing write tid 1 to smithi08434663-41 2024-09-18T14:10:41.320 INFO:tasks.rados.rados.0.smithi084.stdout:2529: finishing write tid 2 to smithi08434663-41 2024-09-18T14:10:41.321 INFO:tasks.rados.rados.0.smithi084.stdout:2529: finishing write tid 3 to smithi08434663-41 2024-09-18T14:10:41.321 INFO:tasks.rados.rados.0.smithi084.stdout:2529: finishing write tid 4 to smithi08434663-41 2024-09-18T14:10:41.322 INFO:tasks.rados.rados.0.smithi084.stdout:2529: finishing write tid 5 to smithi08434663-41 2024-09-18T14:10:41.322 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 1135 (ObjNum 882 snap 266 seq_num 882) dirty exists 2024-09-18T14:10:41.322 INFO:tasks.rados.rados.0.smithi084.stdout:2529: left oid 41 (ObjNum 882 snap 266 seq_num 882) 2024-09-18T14:10:41.323 INFO:tasks.rados.rados.0.smithi084.stdout:2528: done (2 left) 2024-09-18T14:10:41.323 INFO:tasks.rados.rados.0.smithi084.stdout:2529: done (1 left) 2024-09-18T14:10:41.323 INFO:tasks.rados.rados.0.smithi084.stdout:2530: done (0 left) 2024-09-18T14:10:41.323 INFO:tasks.rados.rados.0.smithi084.stdout:2531: snap_remove snap 260 2024-09-18T14:10:42.307 INFO:tasks.rados.rados.0.smithi084.stdout:2531: done (0 left) 2024-09-18T14:10:42.307 INFO:tasks.rados.rados.0.smithi084.stdout:2532: snap_create 2024-09-18T14:10:43.316 INFO:tasks.rados.rados.0.smithi084.stdout:2532: done (0 left) 2024-09-18T14:10:43.316 INFO:tasks.rados.rados.0.smithi084.stdout:2533: snap_create 2024-09-18T14:10:44.324 INFO:tasks.rados.rados.0.smithi084.stdout:2533: done (0 left) 2024-09-18T14:10:44.324 INFO:tasks.rados.rados.0.smithi084.stdout:2534: copy_from oid 37 from oid 20 current snap is 268 2024-09-18T14:10:44.324 INFO:tasks.rados.rados.0.smithi084.stdout:2535: read oid 22 snap 266 2024-09-18T14:10:44.324 INFO:tasks.rados.rados.0.smithi084.stdout:2535: expect (ObjNum 861 snap 258 seq_num 861) 2024-09-18T14:10:44.324 INFO:tasks.rados.rados.0.smithi084.stdout:2536: copy_from oid 25 from oid 32 current snap is 268 2024-09-18T14:10:44.324 INFO:tasks.rados.rados.0.smithi084.stdout:2537: snap_remove snap 267 2024-09-18T14:10:45.328 INFO:tasks.rados.rados.0.smithi084.stdout:2537: done (3 left) 2024-09-18T14:10:45.328 INFO:tasks.rados.rados.0.smithi084.stdout:2538: read oid 3 snap 264 2024-09-18T14:10:45.328 INFO:tasks.rados.rados.0.smithi084.stdout:2538: expect (ObjNum 825 snap 251 seq_num 825) 2024-09-18T14:10:45.328 INFO:tasks.rados.rados.0.smithi084.stdout:2539: read oid 16 snap -1 2024-09-18T14:10:45.328 INFO:tasks.rados.rados.0.smithi084.stdout:2539: expect (ObjNum 874 snap 261 seq_num 874) 2024-09-18T14:10:45.328 INFO:tasks.rados.rados.0.smithi084.stdout:2540: write oid 45 current snap is 268 2024-09-18T14:10:45.328 INFO:tasks.rados.rados.0.smithi084.stdout:2534: finishing copy_from to smithi08434663-37 2024-09-18T14:10:45.329 INFO:tasks.rados.rados.0.smithi084.stdout:2534: got expected ENOENT (src dne) 2024-09-18T14:10:45.329 INFO:tasks.rados.rados.0.smithi084.stdout:2534: finishing copy_from racing read to smithi08434663-37 2024-09-18T14:10:45.329 INFO:tasks.rados.rados.0.smithi084.stdout:2535: expect (ObjNum 861 snap 258 seq_num 861) 2024-09-18T14:10:45.406 INFO:tasks.rados.rados.0.smithi084.stdout:2536: finishing copy_from racing read to smithi08434663-25 2024-09-18T14:10:45.406 INFO:tasks.rados.rados.0.smithi084.stdout:2536: finishing copy_from to smithi08434663-25 2024-09-18T14:10:45.406 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 1231 (ObjNum 873 snap 261 seq_num 873) dirty exists 2024-09-18T14:10:45.406 INFO:tasks.rados.rados.0.smithi084.stdout:2539: expect (ObjNum 874 snap 261 seq_num 874) 2024-09-18T14:10:45.586 INFO:tasks.rados.rados.0.smithi084.stdout:2538: expect (ObjNum 825 snap 251 seq_num 825) 2024-09-18T14:10:45.796 INFO:tasks.rados.rados.0.smithi084.stdout:2540: seq_num 884 ranges {457388=417064,1546435=516637,2643102=214286} 2024-09-18T14:10:45.801 INFO:tasks.rados.rados.0.smithi084.stdout:2540: writing smithi08434663-45 from 457388 to 874452 tid 1 2024-09-18T14:10:45.807 INFO:tasks.rados.rados.0.smithi084.stdout:2540: writing smithi08434663-45 from 1546435 to 2063072 tid 2 2024-09-18T14:10:45.812 INFO:tasks.rados.rados.0.smithi084.stdout:2540: writing smithi08434663-45 from 2643102 to 2857388 tid 3 2024-09-18T14:10:45.812 INFO:tasks.rados.rados.0.smithi084.stdout:2534: done (5 left) 2024-09-18T14:10:45.812 INFO:tasks.rados.rados.0.smithi084.stdout:2535: done (4 left) 2024-09-18T14:10:45.812 INFO:tasks.rados.rados.0.smithi084.stdout:2536: done (3 left) 2024-09-18T14:10:45.812 INFO:tasks.rados.rados.0.smithi084.stdout:2538: done (2 left) 2024-09-18T14:10:45.812 INFO:tasks.rados.rados.0.smithi084.stdout:2539: done (1 left) 2024-09-18T14:10:45.812 INFO:tasks.rados.rados.0.smithi084.stdout:2541: snap_remove snap 264 2024-09-18T14:10:46.342 INFO:tasks.rados.rados.0.smithi084.stdout:2541: done (1 left) 2024-09-18T14:10:46.342 INFO:tasks.rados.rados.0.smithi084.stdout:2542: write oid 46 current snap is 268 2024-09-18T14:10:46.342 INFO:tasks.rados.rados.0.smithi084.stdout:2542: seq_num 885 ranges {474195=492313,1604917=660295,2670525=203670} 2024-09-18T14:10:46.347 INFO:tasks.rados.rados.0.smithi084.stdout:2542: writing smithi08434663-46 from 474195 to 966508 tid 1 2024-09-18T14:10:46.354 INFO:tasks.rados.rados.0.smithi084.stdout:2542: writing smithi08434663-46 from 1604917 to 2265212 tid 2 2024-09-18T14:10:46.358 INFO:tasks.rados.rados.0.smithi084.stdout:2542: writing smithi08434663-46 from 2670525 to 2874195 tid 3 2024-09-18T14:10:46.358 INFO:tasks.rados.rados.0.smithi084.stdout:2543: snap_create 2024-09-18T14:10:46.358 INFO:tasks.rados.rados.0.smithi084.stdout:2540: finishing write tid 1 to smithi08434663-45 2024-09-18T14:10:46.358 INFO:tasks.rados.rados.0.smithi084.stdout:2540: finishing write tid 2 to smithi08434663-45 2024-09-18T14:10:46.358 INFO:tasks.rados.rados.0.smithi084.stdout:2540: finishing write tid 3 to smithi08434663-45 2024-09-18T14:10:46.358 INFO:tasks.rados.rados.0.smithi084.stdout:2540: finishing write tid 4 to smithi08434663-45 2024-09-18T14:10:46.358 INFO:tasks.rados.rados.0.smithi084.stdout:2540: finishing write tid 5 to smithi08434663-45 2024-09-18T14:10:46.358 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 1146 (ObjNum 884 snap 268 seq_num 884) dirty exists 2024-09-18T14:10:46.359 INFO:tasks.rados.rados.0.smithi084.stdout:2540: left oid 45 (ObjNum 884 snap 268 seq_num 884) 2024-09-18T14:10:46.359 INFO:tasks.rados.rados.0.smithi084.stdout:2542: finishing write tid 1 to smithi08434663-46 2024-09-18T14:10:46.359 INFO:tasks.rados.rados.0.smithi084.stdout:2542: finishing write tid 2 to smithi08434663-46 2024-09-18T14:10:46.360 INFO:tasks.rados.rados.0.smithi084.stdout:2542: finishing write tid 3 to smithi08434663-46 2024-09-18T14:10:46.360 INFO:tasks.rados.rados.0.smithi084.stdout:2542: finishing write tid 4 to smithi08434663-46 2024-09-18T14:10:46.360 INFO:tasks.rados.rados.0.smithi084.stdout:2542: finishing write tid 5 to smithi08434663-46 2024-09-18T14:10:46.360 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1062 (ObjNum 885 snap 268 seq_num 885) dirty exists 2024-09-18T14:10:46.360 INFO:tasks.rados.rados.0.smithi084.stdout:2542: left oid 46 (ObjNum 885 snap 268 seq_num 885) 2024-09-18T14:10:47.276 INFO:tasks.rados.rados.0.smithi084.stdout:2540: done (2 left) 2024-09-18T14:10:47.277 INFO:tasks.rados.rados.0.smithi084.stdout:2542: done (1 left) 2024-09-18T14:10:47.277 INFO:tasks.rados.rados.0.smithi084.stdout:2543: done (0 left) 2024-09-18T14:10:47.277 INFO:tasks.rados.rados.0.smithi084.stdout:2544: snap_create 2024-09-18T14:10:48.278 INFO:tasks.rados.rados.0.smithi084.stdout:2544: done (0 left) 2024-09-18T14:10:48.278 INFO:tasks.rados.rados.0.smithi084.stdout:2545: delete oid 21 current snap is 270 2024-09-18T14:10:48.283 INFO:tasks.rados.rados.0.smithi084.stdout:2545: done (0 left) 2024-09-18T14:10:48.284 INFO:tasks.rados.rados.0.smithi084.stdout:2546: read oid 25 snap -1 2024-09-18T14:10:48.284 INFO:tasks.rados.rados.0.smithi084.stdout:2546: expect (ObjNum 873 snap 261 seq_num 873) 2024-09-18T14:10:48.284 INFO:tasks.rados.rados.0.smithi084.stdout:2547: delete oid 24 current snap is 270 2024-09-18T14:10:48.286 INFO:tasks.rados.rados.0.smithi084.stdout:2547: done (1 left) 2024-09-18T14:10:48.286 INFO:tasks.rados.rados.0.smithi084.stdout:2548: write oid 7 current snap is 270 2024-09-18T14:10:48.286 INFO:tasks.rados.rados.0.smithi084.stdout:2548: seq_num 888 ranges {524616=718060,1664010=523975,2752794=171822} 2024-09-18T14:10:48.293 INFO:tasks.rados.rados.0.smithi084.stdout:2548: writing smithi08434663-7 from 524616 to 1242676 tid 1 2024-09-18T14:10:48.298 INFO:tasks.rados.rados.0.smithi084.stdout:2548: writing smithi08434663-7 from 1664010 to 2187985 tid 2 2024-09-18T14:10:48.303 INFO:tasks.rados.rados.0.smithi084.stdout:2548: writing smithi08434663-7 from 2752794 to 2924616 tid 3 2024-09-18T14:10:48.303 INFO:tasks.rados.rados.0.smithi084.stdout:2549: copy_from oid 44 from oid 2 current snap is 270 2024-09-18T14:10:48.303 INFO:tasks.rados.rados.0.smithi084.stdout:2546: expect (ObjNum 873 snap 261 seq_num 873) 2024-09-18T14:10:48.388 INFO:tasks.rados.rados.0.smithi084.stdout:2548: finishing write tid 1 to smithi08434663-7 2024-09-18T14:10:48.388 INFO:tasks.rados.rados.0.smithi084.stdout:2548: finishing write tid 2 to smithi08434663-7 2024-09-18T14:10:48.388 INFO:tasks.rados.rados.0.smithi084.stdout:2549: finishing copy_from to smithi08434663-44 2024-09-18T14:10:48.388 INFO:tasks.rados.rados.0.smithi084.stdout:2549: got expected ENOENT (src dne) 2024-09-18T14:10:48.388 INFO:tasks.rados.rados.0.smithi084.stdout:2549: finishing copy_from racing read to smithi08434663-44 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:2548: finishing write tid 3 to smithi08434663-7 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:2546: done (2 left) 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:2549: done (1 left) 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:2550: rollback oid 13 current snap is 270 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 13 to 268 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:2548: finishing write tid 4 to smithi08434663-7 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:2548: finishing write tid 5 to smithi08434663-7 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 973 (ObjNum 888 snap 270 seq_num 888) dirty exists 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:2548: left oid 7 (ObjNum 888 snap 270 seq_num 888) 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:2548: done (1 left) 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:2551: rollback oid 12 current snap is 270 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 12 to 269 2024-09-18T14:10:48.389 INFO:tasks.rados.rados.0.smithi084.stdout:2552: delete oid 34 current snap is 270 2024-09-18T14:10:48.390 INFO:tasks.rados.rados.0.smithi084.stdout:2550: finishing rollback tid 1 to smithi08434663-13 2024-09-18T14:10:48.390 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:10:48.393 INFO:tasks.rados.rados.0.smithi084.stdout:2551: finishing rollback tid 0 to smithi08434663-12 2024-09-18T14:10:48.394 INFO:tasks.rados.rados.0.smithi084.stdout:2551: finishing rollback tid 1 to smithi08434663-12 2024-09-18T14:10:48.394 INFO:tasks.rados.rados.0.smithi084.stdout:2550: done (2 left) 2024-09-18T14:10:48.394 INFO:tasks.rados.rados.0.smithi084.stdout:2552: done (1 left) 2024-09-18T14:10:48.394 INFO:tasks.rados.rados.0.smithi084.stdout:2553: read oid 4 snap -1 2024-09-18T14:10:48.394 INFO:tasks.rados.rados.0.smithi084.stdout:2553: expect deleted 2024-09-18T14:10:48.394 INFO:tasks.rados.rados.0.smithi084.stdout:2554: rollback oid 6 current snap is 270 2024-09-18T14:10:48.394 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 262 2024-09-18T14:10:48.394 INFO:tasks.rados.rados.0.smithi084.stdout:2551: finishing rollback tid 2 to smithi08434663-12 2024-09-18T14:10:48.394 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 1243 (ObjNum 697 snap 205 seq_num 697) dirty exists 2024-09-18T14:10:48.394 INFO:tasks.rados.rados.0.smithi084.stdout:2551: done (2 left) 2024-09-18T14:10:48.394 INFO:tasks.rados.rados.0.smithi084.stdout:2555: snap_create 2024-09-18T14:10:48.399 INFO:tasks.rados.rados.0.smithi084.stdout:2554: finishing rollback tid 0 to smithi08434663-6 2024-09-18T14:10:48.399 INFO:tasks.rados.rados.0.smithi084.stdout:2554: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:10:48.399 INFO:tasks.rados.rados.0.smithi084.stdout:2554: finishing rollback tid 2 to smithi08434663-6 2024-09-18T14:10:48.399 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 976 (ObjNum 871 snap 261 seq_num 871) dirty exists 2024-09-18T14:10:49.349 INFO:tasks.rados.rados.0.smithi084.stdout:2553: done (2 left) 2024-09-18T14:10:49.349 INFO:tasks.rados.rados.0.smithi084.stdout:2554: done (1 left) 2024-09-18T14:10:49.349 INFO:tasks.rados.rados.0.smithi084.stdout:2555: done (0 left) 2024-09-18T14:10:49.349 INFO:tasks.rados.rados.0.smithi084.stdout:2556: rollback oid 34 current snap is 271 2024-09-18T14:10:49.350 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 34 to 270 2024-09-18T14:10:49.350 INFO:tasks.rados.rados.0.smithi084.stdout:2557: snap_create 2024-09-18T14:10:49.353 INFO:tasks.rados.rados.0.smithi084.stdout:2556: finishing rollback tid 1 to smithi08434663-34 2024-09-18T14:10:49.353 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1150 (ObjNum 113 snap 0 seq_num 1568230291) dirty dne 2024-09-18T14:10:50.355 INFO:tasks.rados.rados.0.smithi084.stdout:2556: done (1 left) 2024-09-18T14:10:50.356 INFO:tasks.rados.rados.0.smithi084.stdout:2557: done (0 left) 2024-09-18T14:10:50.356 INFO:tasks.rados.rados.0.smithi084.stdout:2558: copy_from oid 27 from oid 17 current snap is 272 2024-09-18T14:10:50.356 INFO:tasks.rados.rados.0.smithi084.stdout:2559: write oid 41 current snap is 272 2024-09-18T14:10:50.356 INFO:tasks.rados.rados.0.smithi084.stdout:2559: seq_num 890 ranges {558230=584911,1897327=527644,2923993=34237} 2024-09-18T14:10:50.362 INFO:tasks.rados.rados.0.smithi084.stdout:2559: writing smithi08434663-41 from 558230 to 1143141 tid 1 2024-09-18T14:10:50.369 INFO:tasks.rados.rados.0.smithi084.stdout:2559: writing smithi08434663-41 from 1897327 to 2424971 tid 2 2024-09-18T14:10:50.372 INFO:tasks.rados.rados.0.smithi084.stdout:2559: writing smithi08434663-41 from 2923993 to 2958230 tid 3 2024-09-18T14:10:50.372 INFO:tasks.rados.rados.0.smithi084.stdout:2560: write (excl) oid 4 current snap is 272 2024-09-18T14:10:50.372 INFO:tasks.rados.rados.0.smithi084.stdout:2560: seq_num 891 ranges {575037=660160,1955809=671302} 2024-09-18T14:10:50.378 INFO:tasks.rados.rados.0.smithi084.stdout:2560: writing smithi08434663-4 from 575037 to 1235197 tid 1 2024-09-18T14:10:50.386 INFO:tasks.rados.rados.0.smithi084.stdout:2560: writing smithi08434663-4 from 1955809 to 2627111 tid 2 2024-09-18T14:10:50.386 INFO:tasks.rados.rados.0.smithi084.stdout:2561: read oid 49 snap -1 2024-09-18T14:10:50.386 INFO:tasks.rados.rados.0.smithi084.stdout:2561: expect (ObjNum 836 snap 252 seq_num 836) 2024-09-18T14:10:50.386 INFO:tasks.rados.rados.0.smithi084.stdout:2562: snap_create 2024-09-18T14:10:50.386 INFO:tasks.rados.rados.0.smithi084.stdout:2558: finishing copy_from racing read to smithi08434663-27 2024-09-18T14:10:50.387 INFO:tasks.rados.rados.0.smithi084.stdout:2559: finishing write tid 1 to smithi08434663-41 2024-09-18T14:10:50.387 INFO:tasks.rados.rados.0.smithi084.stdout:2558: finishing copy_from to smithi08434663-27 2024-09-18T14:10:50.387 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 904 (ObjNum 865 snap 258 seq_num 865) dirty exists 2024-09-18T14:10:50.387 INFO:tasks.rados.rados.0.smithi084.stdout:2559: finishing write tid 2 to smithi08434663-41 2024-09-18T14:10:50.387 INFO:tasks.rados.rados.0.smithi084.stdout:2559: finishing write tid 3 to smithi08434663-41 2024-09-18T14:10:50.387 INFO:tasks.rados.rados.0.smithi084.stdout:2559: finishing write tid 4 to smithi08434663-41 2024-09-18T14:10:50.387 INFO:tasks.rados.rados.0.smithi084.stdout:2559: finishing write tid 5 to smithi08434663-41 2024-09-18T14:10:50.387 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 1156 (ObjNum 890 snap 272 seq_num 890) dirty exists 2024-09-18T14:10:50.387 INFO:tasks.rados.rados.0.smithi084.stdout:2559: left oid 41 (ObjNum 890 snap 272 seq_num 890) 2024-09-18T14:10:50.387 INFO:tasks.rados.rados.0.smithi084.stdout:2560: finishing write tid 1 to smithi08434663-4 2024-09-18T14:10:50.391 INFO:tasks.rados.rados.0.smithi084.stdout:2560: finishing write tid 2 to smithi08434663-4 2024-09-18T14:10:50.391 INFO:tasks.rados.rados.0.smithi084.stdout:2560: finishing write tid 3 to smithi08434663-4 2024-09-18T14:10:50.391 INFO:tasks.rados.rados.0.smithi084.stdout:2560: finishing write tid 4 to smithi08434663-4 2024-09-18T14:10:50.391 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1159 (ObjNum 891 snap 272 seq_num 891) dirty exists 2024-09-18T14:10:50.392 INFO:tasks.rados.rados.0.smithi084.stdout:2560: left oid 4 (ObjNum 891 snap 272 seq_num 891) 2024-09-18T14:10:50.392 INFO:tasks.rados.rados.0.smithi084.stdout:2561: expect (ObjNum 836 snap 252 seq_num 836) 2024-09-18T14:10:51.361 INFO:tasks.rados.rados.0.smithi084.stdout:2558: done (4 left) 2024-09-18T14:10:51.361 INFO:tasks.rados.rados.0.smithi084.stdout:2559: done (3 left) 2024-09-18T14:10:51.361 INFO:tasks.rados.rados.0.smithi084.stdout:2560: done (2 left) 2024-09-18T14:10:51.361 INFO:tasks.rados.rados.0.smithi084.stdout:2561: done (1 left) 2024-09-18T14:10:51.361 INFO:tasks.rados.rados.0.smithi084.stdout:2562: done (0 left) 2024-09-18T14:10:51.361 INFO:tasks.rados.rados.0.smithi084.stdout:2563: read oid 12 snap -1 2024-09-18T14:10:51.361 INFO:tasks.rados.rados.0.smithi084.stdout:2563: expect (ObjNum 697 snap 205 seq_num 697) 2024-09-18T14:10:51.362 INFO:tasks.rados.rados.0.smithi084.stdout:2564: read oid 32 snap 271 2024-09-18T14:10:51.362 INFO:tasks.rados.rados.0.smithi084.stdout:2564: expect (ObjNum 873 snap 261 seq_num 873) 2024-09-18T14:10:51.362 INFO:tasks.rados.rados.0.smithi084.stdout:2565: snap_create 2024-09-18T14:10:51.369 INFO:tasks.rados.rados.0.smithi084.stdout:2563: expect (ObjNum 697 snap 205 seq_num 697) 2024-09-18T14:10:51.536 INFO:tasks.rados.rados.0.smithi084.stdout:2564: expect (ObjNum 873 snap 261 seq_num 873) 2024-09-18T14:10:52.365 INFO:tasks.rados.rados.0.smithi084.stdout:2563: done (2 left) 2024-09-18T14:10:52.365 INFO:tasks.rados.rados.0.smithi084.stdout:2564: done (1 left) 2024-09-18T14:10:52.365 INFO:tasks.rados.rados.0.smithi084.stdout:2565: done (0 left) 2024-09-18T14:10:52.365 INFO:tasks.rados.rados.0.smithi084.stdout:2566: snap_remove snap 266 2024-09-18T14:10:53.373 INFO:tasks.rados.rados.0.smithi084.stdout:2566: done (0 left) 2024-09-18T14:10:53.373 INFO:tasks.rados.rados.0.smithi084.stdout:2567: copy_from oid 16 from oid 12 current snap is 274 2024-09-18T14:10:53.374 INFO:tasks.rados.rados.0.smithi084.stdout:2568: snap_create 2024-09-18T14:10:53.387 INFO:tasks.rados.rados.0.smithi084.stdout:2567: finishing copy_from racing read to smithi08434663-16 2024-09-18T14:10:53.390 INFO:tasks.rados.rados.0.smithi084.stdout:2567: finishing copy_from to smithi08434663-16 2024-09-18T14:10:53.390 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1243 (ObjNum 697 snap 205 seq_num 697) dirty exists 2024-09-18T14:10:54.380 INFO:tasks.rados.rados.0.smithi084.stdout:2567: done (1 left) 2024-09-18T14:10:54.380 INFO:tasks.rados.rados.0.smithi084.stdout:2568: done (0 left) 2024-09-18T14:10:54.380 INFO:tasks.rados.rados.0.smithi084.stdout:2569: snap_remove snap 271 2024-09-18T14:10:55.382 INFO:tasks.rados.rados.0.smithi084.stdout:2569: done (0 left) 2024-09-18T14:10:55.382 INFO:tasks.rados.rados.0.smithi084.stdout:2570: snap_create 2024-09-18T14:10:56.402 INFO:tasks.rados.rados.0.smithi084.stdout:2570: done (0 left) 2024-09-18T14:10:56.402 INFO:tasks.rados.rados.0.smithi084.stdout:2571: copy_from oid 19 from oid 38 current snap is 276 2024-09-18T14:10:56.402 INFO:tasks.rados.rados.0.smithi084.stdout:2572: copy_from oid 44 from oid 10 current snap is 276 2024-09-18T14:10:56.402 INFO:tasks.rados.rados.0.smithi084.stdout:2573: rollback oid 17 current snap is 276 2024-09-18T14:10:56.403 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 17 to 262 2024-09-18T14:10:56.403 INFO:tasks.rados.rados.0.smithi084.stdout:2574: snap_create 2024-09-18T14:10:56.411 INFO:tasks.rados.rados.0.smithi084.stdout:2573: finishing rollback tid 0 to smithi08434663-17 2024-09-18T14:10:56.413 INFO:tasks.rados.rados.0.smithi084.stdout:2573: finishing rollback tid 1 to smithi08434663-17 2024-09-18T14:10:56.413 INFO:tasks.rados.rados.0.smithi084.stdout:2573: finishing rollback tid 2 to smithi08434663-17 2024-09-18T14:10:56.414 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1019 (ObjNum 865 snap 258 seq_num 865) dirty exists 2024-09-18T14:10:56.416 INFO:tasks.rados.rados.0.smithi084.stdout:2572: finishing copy_from racing read to smithi08434663-44 2024-09-18T14:10:56.418 INFO:tasks.rados.rados.0.smithi084.stdout:2571: finishing copy_from racing read to smithi08434663-19 2024-09-18T14:10:56.428 INFO:tasks.rados.rados.0.smithi084.stdout:2572: finishing copy_from to smithi08434663-44 2024-09-18T14:10:56.428 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 894 (ObjNum 859 snap 257 seq_num 859) dirty exists 2024-09-18T14:10:56.428 INFO:tasks.rados.rados.0.smithi084.stdout:2571: finishing copy_from to smithi08434663-19 2024-09-18T14:10:56.428 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 639 (ObjNum 789 snap 233 seq_num 789) dirty exists 2024-09-18T14:10:57.281 INFO:tasks.rados.rados.0.smithi084.stdout:2571: done (3 left) 2024-09-18T14:10:57.282 INFO:tasks.rados.rados.0.smithi084.stdout:2572: done (2 left) 2024-09-18T14:10:57.282 INFO:tasks.rados.rados.0.smithi084.stdout:2573: done (1 left) 2024-09-18T14:10:57.282 INFO:tasks.rados.rados.0.smithi084.stdout:2574: done (0 left) 2024-09-18T14:10:57.282 INFO:tasks.rados.rados.0.smithi084.stdout:2575: read oid 27 snap -1 2024-09-18T14:10:57.282 INFO:tasks.rados.rados.0.smithi084.stdout:2575: expect (ObjNum 865 snap 258 seq_num 865) 2024-09-18T14:10:57.282 INFO:tasks.rados.rados.0.smithi084.stdout:2576: rollback oid 8 current snap is 277 2024-09-18T14:10:57.282 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 8 to 262 2024-09-18T14:10:57.282 INFO:tasks.rados.rados.0.smithi084.stdout:2577: snap_create 2024-09-18T14:10:57.284 INFO:tasks.rados.rados.0.smithi084.stdout:2576: finishing rollback tid 1 to smithi08434663-8 2024-09-18T14:10:57.285 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 0 (ObjNum 145 snap 0 seq_num 2495431008) dirty dne 2024-09-18T14:10:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:2575: expect (ObjNum 865 snap 258 seq_num 865) 2024-09-18T14:10:58.291 INFO:tasks.rados.rados.0.smithi084.stdout:2575: done (2 left) 2024-09-18T14:10:58.292 INFO:tasks.rados.rados.0.smithi084.stdout:2576: done (1 left) 2024-09-18T14:10:58.292 INFO:tasks.rados.rados.0.smithi084.stdout:2577: done (0 left) 2024-09-18T14:10:58.292 INFO:tasks.rados.rados.0.smithi084.stdout:2578: read oid 25 snap -1 2024-09-18T14:10:58.292 INFO:tasks.rados.rados.0.smithi084.stdout:2578: expect (ObjNum 873 snap 261 seq_num 873) 2024-09-18T14:10:58.292 INFO:tasks.rados.rados.0.smithi084.stdout:2579: snap_remove snap 262 2024-09-18T14:10:59.296 INFO:tasks.rados.rados.0.smithi084.stdout:2579: done (1 left) 2024-09-18T14:10:59.296 INFO:tasks.rados.rados.0.smithi084.stdout:2580: write (excl) oid 17 current snap is 278 2024-09-18T14:10:59.296 INFO:tasks.rados.rados.0.smithi084.stdout:2580: seq_num 892 ranges {591844=735409,1897938=531313,2862486=129358} 2024-09-18T14:10:59.303 INFO:tasks.rados.rados.0.smithi084.stdout:2580: writing smithi08434663-17 from 591844 to 1327253 tid 1 2024-09-18T14:10:59.309 INFO:tasks.rados.rados.0.smithi084.stdout:2580: writing smithi08434663-17 from 1897938 to 2429251 tid 2 2024-09-18T14:10:59.313 INFO:tasks.rados.rados.0.smithi084.stdout:2580: writing smithi08434663-17 from 2862486 to 2991844 tid 3 2024-09-18T14:10:59.313 INFO:tasks.rados.rados.0.smithi084.stdout:2581: read oid 36 snap -1 2024-09-18T14:10:59.313 INFO:tasks.rados.rados.0.smithi084.stdout:2581: expect deleted 2024-09-18T14:10:59.313 INFO:tasks.rados.rados.0.smithi084.stdout:2582: delete oid 7 current snap is 278 2024-09-18T14:10:59.313 INFO:tasks.rados.rados.0.smithi084.stdout:2578: expect (ObjNum 873 snap 261 seq_num 873) 2024-09-18T14:10:59.397 INFO:tasks.rados.rados.0.smithi084.stdout:2580: finishing write tid 1 to smithi08434663-17 2024-09-18T14:10:59.397 INFO:tasks.rados.rados.0.smithi084.stdout:2580: finishing write tid 2 to smithi08434663-17 2024-09-18T14:10:59.397 INFO:tasks.rados.rados.0.smithi084.stdout:2580: finishing write tid 3 to smithi08434663-17 2024-09-18T14:10:59.397 INFO:tasks.rados.rados.0.smithi084.stdout:2580: finishing write tid 4 to smithi08434663-17 2024-09-18T14:10:59.397 INFO:tasks.rados.rados.0.smithi084.stdout:2580: finishing write tid 5 to smithi08434663-17 2024-09-18T14:10:59.397 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1023 (ObjNum 892 snap 278 seq_num 892) dirty exists 2024-09-18T14:10:59.397 INFO:tasks.rados.rados.0.smithi084.stdout:2580: left oid 17 (ObjNum 892 snap 278 seq_num 892) 2024-09-18T14:10:59.398 INFO:tasks.rados.rados.0.smithi084.stdout:2578: done (3 left) 2024-09-18T14:10:59.398 INFO:tasks.rados.rados.0.smithi084.stdout:2580: done (2 left) 2024-09-18T14:10:59.398 INFO:tasks.rados.rados.0.smithi084.stdout:2581: done (1 left) 2024-09-18T14:10:59.398 INFO:tasks.rados.rados.0.smithi084.stdout:2582: done (0 left) 2024-09-18T14:10:59.398 INFO:tasks.rados.rados.0.smithi084.stdout:2583: copy_from oid 41 from oid 31 current snap is 278 2024-09-18T14:10:59.398 INFO:tasks.rados.rados.0.smithi084.stdout:2584: write (excl) oid 27 current snap is 278 2024-09-18T14:10:59.398 INFO:tasks.rados.rados.0.smithi084.stdout:2584: seq_num 894 ranges {625458=485907,1614902=534982,2917332=108126} 2024-09-18T14:10:59.402 INFO:tasks.rados.rados.0.smithi084.stdout:2584: writing smithi08434663-27 from 625458 to 1111365 tid 1 2024-09-18T14:10:59.408 INFO:tasks.rados.rados.0.smithi084.stdout:2584: writing smithi08434663-27 from 1614902 to 2149884 tid 2 2024-09-18T14:10:59.414 INFO:tasks.rados.rados.0.smithi084.stdout:2584: writing smithi08434663-27 from 2917332 to 3025458 tid 3 2024-09-18T14:10:59.414 INFO:tasks.rados.rados.0.smithi084.stdout:2583: finishing copy_from to smithi08434663-41 2024-09-18T14:10:59.414 INFO:tasks.rados.rados.0.smithi084.stdout:2583: got expected ENOENT (src dne) 2024-09-18T14:10:59.414 INFO:tasks.rados.rados.0.smithi084.stdout:2583: finishing copy_from racing read to smithi08434663-41 2024-09-18T14:10:59.414 INFO:tasks.rados.rados.0.smithi084.stdout:2584: finishing write tid 1 to smithi08434663-27 2024-09-18T14:10:59.414 INFO:tasks.rados.rados.0.smithi084.stdout:2584: finishing write tid 2 to smithi08434663-27 2024-09-18T14:10:59.414 INFO:tasks.rados.rados.0.smithi084.stdout:2585: read oid 25 snap -1 2024-09-18T14:10:59.415 INFO:tasks.rados.rados.0.smithi084.stdout:2585: expect (ObjNum 873 snap 261 seq_num 873) 2024-09-18T14:10:59.415 INFO:tasks.rados.rados.0.smithi084.stdout:2583: done (2 left) 2024-09-18T14:10:59.415 INFO:tasks.rados.rados.0.smithi084.stdout:2586: copy_from oid 48 from oid 23 current snap is 278 2024-09-18T14:10:59.415 INFO:tasks.rados.rados.0.smithi084.stdout:2587: snap_create 2024-09-18T14:10:59.416 INFO:tasks.rados.rados.0.smithi084.stdout:2584: finishing write tid 3 to smithi08434663-27 2024-09-18T14:10:59.417 INFO:tasks.rados.rados.0.smithi084.stdout:2584: finishing write tid 4 to smithi08434663-27 2024-09-18T14:10:59.417 INFO:tasks.rados.rados.0.smithi084.stdout:2584: finishing write tid 5 to smithi08434663-27 2024-09-18T14:10:59.417 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1211 (ObjNum 894 snap 278 seq_num 894) dirty exists 2024-09-18T14:10:59.417 INFO:tasks.rados.rados.0.smithi084.stdout:2584: left oid 27 (ObjNum 894 snap 278 seq_num 894) 2024-09-18T14:10:59.417 INFO:tasks.rados.rados.0.smithi084.stdout:2585: expect (ObjNum 873 snap 261 seq_num 873) 2024-09-18T14:10:59.502 INFO:tasks.rados.rados.0.smithi084.stdout:2586: finishing copy_from racing read to smithi08434663-48 2024-09-18T14:10:59.502 INFO:tasks.rados.rados.0.smithi084.stdout:2586: finishing copy_from to smithi08434663-48 2024-09-18T14:10:59.502 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 966 (ObjNum 827 snap 251 seq_num 827) dirty exists 2024-09-18T14:11:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:2584: done (3 left) 2024-09-18T14:11:00.315 INFO:tasks.rados.rados.0.smithi084.stdout:2585: done (2 left) 2024-09-18T14:11:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:2586: done (1 left) 2024-09-18T14:11:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:2587: done (0 left) 2024-09-18T14:11:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:2588: write oid 12 current snap is 279 2024-09-18T14:11:00.316 INFO:tasks.rados.rados.0.smithi084.stdout:2588: seq_num 895 ranges {642265=561156,1673384=678640,2828402=213863} 2024-09-18T14:11:00.321 INFO:tasks.rados.rados.0.smithi084.stdout:2588: writing smithi08434663-12 from 642265 to 1203421 tid 1 2024-09-18T14:11:00.327 INFO:tasks.rados.rados.0.smithi084.stdout:2588: writing smithi08434663-12 from 1673384 to 2352024 tid 2 2024-09-18T14:11:00.331 INFO:tasks.rados.rados.0.smithi084.stdout:2588: writing smithi08434663-12 from 2828402 to 3042265 tid 3 2024-09-18T14:11:00.331 INFO:tasks.rados.rados.0.smithi084.stdout:2589: snap_remove snap 268 2024-09-18T14:11:01.334 INFO:tasks.rados.rados.0.smithi084.stdout:2589: done (1 left) 2024-09-18T14:11:01.334 INFO:tasks.rados.rados.0.smithi084.stdout:2590: read oid 36 snap 269 2024-09-18T14:11:01.334 INFO:tasks.rados.rados.0.smithi084.stdout:2590: expect deleted 2024-09-18T14:11:01.334 INFO:tasks.rados.rados.0.smithi084.stdout:2591: rollback oid 28 current snap is 279 2024-09-18T14:11:01.334 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 28 to 273 2024-09-18T14:11:01.334 INFO:tasks.rados.rados.0.smithi084.stdout:2588: finishing write tid 1 to smithi08434663-12 2024-09-18T14:11:01.335 INFO:tasks.rados.rados.0.smithi084.stdout:2588: finishing write tid 2 to smithi08434663-12 2024-09-18T14:11:01.335 INFO:tasks.rados.rados.0.smithi084.stdout:2588: finishing write tid 3 to smithi08434663-12 2024-09-18T14:11:01.335 INFO:tasks.rados.rados.0.smithi084.stdout:2588: finishing write tid 4 to smithi08434663-12 2024-09-18T14:11:01.335 INFO:tasks.rados.rados.0.smithi084.stdout:2588: finishing write tid 5 to smithi08434663-12 2024-09-18T14:11:01.335 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 1256 (ObjNum 895 snap 279 seq_num 895) dirty exists 2024-09-18T14:11:01.335 INFO:tasks.rados.rados.0.smithi084.stdout:2588: left oid 12 (ObjNum 895 snap 279 seq_num 895) 2024-09-18T14:11:01.335 INFO:tasks.rados.rados.0.smithi084.stdout:2588: done (2 left) 2024-09-18T14:11:01.335 INFO:tasks.rados.rados.0.smithi084.stdout:2592: read oid 18 snap 270 2024-09-18T14:11:01.335 INFO:tasks.rados.rados.0.smithi084.stdout:2592: expect (ObjNum 860 snap 257 seq_num 860) 2024-09-18T14:11:01.335 INFO:tasks.rados.rados.0.smithi084.stdout:2593: snap_create 2024-09-18T14:11:01.348 INFO:tasks.rados.rados.0.smithi084.stdout:2592: expect (ObjNum 860 snap 257 seq_num 860) 2024-09-18T14:11:01.425 INFO:tasks.rados.rados.0.smithi084.stdout:2591: finishing rollback tid 0 to smithi08434663-28 2024-09-18T14:11:01.425 INFO:tasks.rados.rados.0.smithi084.stdout:2591: finishing rollback tid 1 to smithi08434663-28 2024-09-18T14:11:01.425 INFO:tasks.rados.rados.0.smithi084.stdout:2591: finishing rollback tid 2 to smithi08434663-28 2024-09-18T14:11:01.426 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1264 (ObjNum 765 snap 223 seq_num 765) dirty exists 2024-09-18T14:11:02.333 INFO:tasks.rados.rados.0.smithi084.stdout:2590: done (3 left) 2024-09-18T14:11:02.333 INFO:tasks.rados.rados.0.smithi084.stdout:2591: done (2 left) 2024-09-18T14:11:02.333 INFO:tasks.rados.rados.0.smithi084.stdout:2592: done (1 left) 2024-09-18T14:11:02.333 INFO:tasks.rados.rados.0.smithi084.stdout:2593: done (0 left) 2024-09-18T14:11:02.333 INFO:tasks.rados.rados.0.smithi084.stdout:2594: snap_remove snap 277 2024-09-18T14:11:03.336 INFO:tasks.rados.rados.0.smithi084.stdout:2594: done (0 left) 2024-09-18T14:11:03.336 INFO:tasks.rados.rados.0.smithi084.stdout:2595: write oid 10 current snap is 280 2024-09-18T14:11:03.337 INFO:tasks.rados.rados.0.smithi084.stdout:2595: seq_num 896 ranges {659072=636405,1731866=422298,2855825=203247} 2024-09-18T14:11:03.343 INFO:tasks.rados.rados.0.smithi084.stdout:2595: writing smithi08434663-10 from 659072 to 1295477 tid 1 2024-09-18T14:11:03.348 INFO:tasks.rados.rados.0.smithi084.stdout:2595: writing smithi08434663-10 from 1731866 to 2154164 tid 2 2024-09-18T14:11:03.354 INFO:tasks.rados.rados.0.smithi084.stdout:2595: writing smithi08434663-10 from 2855825 to 3059072 tid 3 2024-09-18T14:11:03.354 INFO:tasks.rados.rados.0.smithi084.stdout:2596: rollback oid 35 current snap is 280 2024-09-18T14:11:03.354 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 35 to 261 2024-09-18T14:11:03.354 INFO:tasks.rados.rados.0.smithi084.stdout:2595: finishing write tid 1 to smithi08434663-10 2024-09-18T14:11:03.354 INFO:tasks.rados.rados.0.smithi084.stdout:2595: finishing write tid 2 to smithi08434663-10 2024-09-18T14:11:03.354 INFO:tasks.rados.rados.0.smithi084.stdout:2597: delete oid 6 current snap is 280 2024-09-18T14:11:03.356 INFO:tasks.rados.rados.0.smithi084.stdout:2595: finishing write tid 3 to smithi08434663-10 2024-09-18T14:11:03.357 INFO:tasks.rados.rados.0.smithi084.stdout:2597: done (2 left) 2024-09-18T14:11:03.357 INFO:tasks.rados.rados.0.smithi084.stdout:2598: snap_create 2024-09-18T14:11:03.357 INFO:tasks.rados.rados.0.smithi084.stdout:2596: finishing rollback tid 0 to smithi08434663-35 2024-09-18T14:11:03.358 INFO:tasks.rados.rados.0.smithi084.stdout:2595: finishing write tid 4 to smithi08434663-10 2024-09-18T14:11:03.358 INFO:tasks.rados.rados.0.smithi084.stdout:2595: finishing write tid 5 to smithi08434663-10 2024-09-18T14:11:03.358 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 1215 (ObjNum 896 snap 280 seq_num 896) dirty exists 2024-09-18T14:11:03.359 INFO:tasks.rados.rados.0.smithi084.stdout:2595: left oid 10 (ObjNum 896 snap 280 seq_num 896) 2024-09-18T14:11:03.359 INFO:tasks.rados.rados.0.smithi084.stdout:2596: finishing rollback tid 1 to smithi08434663-35 2024-09-18T14:11:03.359 INFO:tasks.rados.rados.0.smithi084.stdout:2596: finishing rollback tid 2 to smithi08434663-35 2024-09-18T14:11:03.359 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 1246 (ObjNum 843 snap 255 seq_num 843) dirty exists 2024-09-18T14:11:04.353 INFO:tasks.rados.rados.0.smithi084.stdout:2595: done (2 left) 2024-09-18T14:11:04.354 INFO:tasks.rados.rados.0.smithi084.stdout:2596: done (1 left) 2024-09-18T14:11:04.354 INFO:tasks.rados.rados.0.smithi084.stdout:2598: done (0 left) 2024-09-18T14:11:04.354 INFO:tasks.rados.rados.0.smithi084.stdout:2599: delete oid 46 current snap is 281 2024-09-18T14:11:04.359 INFO:tasks.rados.rados.0.smithi084.stdout:2599: done (0 left) 2024-09-18T14:11:04.359 INFO:tasks.rados.rados.0.smithi084.stdout:2600: write (excl) oid 49 current snap is 281 2024-09-18T14:11:04.359 INFO:tasks.rados.rados.0.smithi084.stdout:2600: seq_num 899 ranges {709493=578505,1907312=685978} 2024-09-18T14:11:04.365 INFO:tasks.rados.rados.0.smithi084.stdout:2600: writing smithi08434663-49 from 709493 to 1287998 tid 1 2024-09-18T14:11:04.373 INFO:tasks.rados.rados.0.smithi084.stdout:2600: writing smithi08434663-49 from 1907312 to 2593290 tid 2 2024-09-18T14:11:04.373 INFO:tasks.rados.rados.0.smithi084.stdout:2601: copy_from oid 19 from oid 38 current snap is 281 2024-09-18T14:11:04.374 INFO:tasks.rados.rados.0.smithi084.stdout:2600: finishing write tid 1 to smithi08434663-49 2024-09-18T14:11:04.374 INFO:tasks.rados.rados.0.smithi084.stdout:2602: write (excl) oid 6 current snap is 281 2024-09-18T14:11:04.374 INFO:tasks.rados.rados.0.smithi084.stdout:2602: seq_num 900 ranges {726300=653754,1965794=429636,2965517=160783} 2024-09-18T14:11:04.381 INFO:tasks.rados.rados.0.smithi084.stdout:2602: writing smithi08434663-6 from 726300 to 1380054 tid 1 2024-09-18T14:11:04.387 INFO:tasks.rados.rados.0.smithi084.stdout:2602: writing smithi08434663-6 from 1965794 to 2395430 tid 2 2024-09-18T14:11:04.392 INFO:tasks.rados.rados.0.smithi084.stdout:2602: writing smithi08434663-6 from 2965517 to 3126300 tid 3 2024-09-18T14:11:04.392 INFO:tasks.rados.rados.0.smithi084.stdout:2603: read oid 26 snap -1 2024-09-18T14:11:04.393 INFO:tasks.rados.rados.0.smithi084.stdout:2603: expect (ObjNum 841 snap 255 seq_num 841) 2024-09-18T14:11:04.393 INFO:tasks.rados.rados.0.smithi084.stdout:2604: copy_from oid 18 from oid 17 current snap is 281 2024-09-18T14:11:04.393 INFO:tasks.rados.rados.0.smithi084.stdout:2600: finishing write tid 2 to smithi08434663-49 2024-09-18T14:11:04.393 INFO:tasks.rados.rados.0.smithi084.stdout:2600: finishing write tid 3 to smithi08434663-49 2024-09-18T14:11:04.393 INFO:tasks.rados.rados.0.smithi084.stdout:2600: finishing write tid 4 to smithi08434663-49 2024-09-18T14:11:04.393 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 1019 (ObjNum 899 snap 281 seq_num 899) dirty exists 2024-09-18T14:11:04.394 INFO:tasks.rados.rados.0.smithi084.stdout:2600: left oid 49 (ObjNum 899 snap 281 seq_num 899) 2024-09-18T14:11:04.394 INFO:tasks.rados.rados.0.smithi084.stdout:2601: finishing copy_from racing read to smithi08434663-19 2024-09-18T14:11:04.394 INFO:tasks.rados.rados.0.smithi084.stdout:2602: finishing write tid 1 to smithi08434663-6 2024-09-18T14:11:04.394 INFO:tasks.rados.rados.0.smithi084.stdout:2601: finishing copy_from to smithi08434663-19 2024-09-18T14:11:04.394 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 639 (ObjNum 789 snap 233 seq_num 789) dirty exists 2024-09-18T14:11:04.394 INFO:tasks.rados.rados.0.smithi084.stdout:2602: finishing write tid 2 to smithi08434663-6 2024-09-18T14:11:04.394 INFO:tasks.rados.rados.0.smithi084.stdout:2600: done (4 left) 2024-09-18T14:11:04.394 INFO:tasks.rados.rados.0.smithi084.stdout:2601: done (3 left) 2024-09-18T14:11:04.394 INFO:tasks.rados.rados.0.smithi084.stdout:2605: rollback oid 2 current snap is 281 2024-09-18T14:11:04.394 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 2 to 276 2024-09-18T14:11:04.395 INFO:tasks.rados.rados.0.smithi084.stdout:2606: read oid 33 snap -1 2024-09-18T14:11:04.395 INFO:tasks.rados.rados.0.smithi084.stdout:2606: expect (ObjNum 847 snap 255 seq_num 847) 2024-09-18T14:11:04.395 INFO:tasks.rados.rados.0.smithi084.stdout:2607: read oid 4 snap 279 2024-09-18T14:11:04.395 INFO:tasks.rados.rados.0.smithi084.stdout:2607: expect (ObjNum 891 snap 272 seq_num 891) 2024-09-18T14:11:04.395 INFO:tasks.rados.rados.0.smithi084.stdout:2608: write oid 32 current snap is 281 2024-09-18T14:11:04.395 INFO:tasks.rados.rados.0.smithi084.stdout:2608: seq_num 901 ranges {743107=729003,1907923=689647} 2024-09-18T14:11:04.400 INFO:tasks.rados.rados.0.smithi084.stdout:2608: writing smithi08434663-32 from 743107 to 1472110 tid 1 2024-09-18T14:11:04.406 INFO:tasks.rados.rados.0.smithi084.stdout:2608: writing smithi08434663-32 from 1907923 to 2597570 tid 2 2024-09-18T14:11:04.406 INFO:tasks.rados.rados.0.smithi084.stdout:2609: write (excl) oid 22 current snap is 281 2024-09-18T14:11:04.406 INFO:tasks.rados.rados.0.smithi084.stdout:2605: finishing rollback tid 1 to smithi08434663-2 2024-09-18T14:11:04.406 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 0 (ObjNum 241 snap 0 seq_num 2495160400) dirty dne 2024-09-18T14:11:04.407 INFO:tasks.rados.rados.0.smithi084.stdout:2609: seq_num 902 ranges {759914=404252,1566405=433305,2504010=655904} 2024-09-18T14:11:04.413 INFO:tasks.rados.rados.0.smithi084.stdout:2609: writing smithi08434663-22 from 759914 to 1164166 tid 1 2024-09-18T14:11:04.417 INFO:tasks.rados.rados.0.smithi084.stdout:2609: writing smithi08434663-22 from 1566405 to 1999710 tid 2 2024-09-18T14:11:04.424 INFO:tasks.rados.rados.0.smithi084.stdout:2609: writing smithi08434663-22 from 2504010 to 3159914 tid 3 2024-09-18T14:11:04.424 INFO:tasks.rados.rados.0.smithi084.stdout:2605: done (7 left) 2024-09-18T14:11:04.424 INFO:tasks.rados.rados.0.smithi084.stdout:2610: read oid 23 snap -1 2024-09-18T14:11:04.425 INFO:tasks.rados.rados.0.smithi084.stdout:2610: expect (ObjNum 827 snap 251 seq_num 827) 2024-09-18T14:11:04.425 INFO:tasks.rados.rados.0.smithi084.stdout:2611: read oid 9 snap -1 2024-09-18T14:11:04.425 INFO:tasks.rados.rados.0.smithi084.stdout:2611: expect (ObjNum 804 snap 240 seq_num 804) 2024-09-18T14:11:04.425 INFO:tasks.rados.rados.0.smithi084.stdout:2612: write oid 42 current snap is 281 2024-09-18T14:11:04.425 INFO:tasks.rados.rados.0.smithi084.stdout:2602: finishing write tid 3 to smithi08434663-6 2024-09-18T14:11:04.425 INFO:tasks.rados.rados.0.smithi084.stdout:2603: expect (ObjNum 841 snap 255 seq_num 841) 2024-09-18T14:11:04.554 INFO:tasks.rados.rados.0.smithi084.stdout:2602: finishing write tid 4 to smithi08434663-6 2024-09-18T14:11:04.555 INFO:tasks.rados.rados.0.smithi084.stdout:2602: finishing write tid 5 to smithi08434663-6 2024-09-18T14:11:04.556 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 982 (ObjNum 900 snap 281 seq_num 900) dirty exists 2024-09-18T14:11:04.556 INFO:tasks.rados.rados.0.smithi084.stdout:2602: left oid 6 (ObjNum 900 snap 281 seq_num 900) 2024-09-18T14:11:04.556 INFO:tasks.rados.rados.0.smithi084.stdout:2606: expect (ObjNum 847 snap 255 seq_num 847) 2024-09-18T14:11:04.653 INFO:tasks.rados.rados.0.smithi084.stdout:2604: finishing copy_from racing read to smithi08434663-18 2024-09-18T14:11:04.653 INFO:tasks.rados.rados.0.smithi084.stdout:2608: finishing write tid 1 to smithi08434663-32 2024-09-18T14:11:04.653 INFO:tasks.rados.rados.0.smithi084.stdout:2607: expect (ObjNum 891 snap 272 seq_num 891) 2024-09-18T14:11:04.746 INFO:tasks.rados.rados.0.smithi084.stdout:2608: finishing write tid 2 to smithi08434663-32 2024-09-18T14:11:04.746 INFO:tasks.rados.rados.0.smithi084.stdout:2608: finishing write tid 3 to smithi08434663-32 2024-09-18T14:11:04.746 INFO:tasks.rados.rados.0.smithi084.stdout:2608: finishing write tid 4 to smithi08434663-32 2024-09-18T14:11:04.747 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 1274 (ObjNum 901 snap 281 seq_num 901) dirty exists 2024-09-18T14:11:04.747 INFO:tasks.rados.rados.0.smithi084.stdout:2608: left oid 32 (ObjNum 901 snap 281 seq_num 901) 2024-09-18T14:11:04.747 INFO:tasks.rados.rados.0.smithi084.stdout:2604: finishing copy_from to smithi08434663-18 2024-09-18T14:11:04.747 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 1023 (ObjNum 892 snap 278 seq_num 892) dirty exists 2024-09-18T14:11:04.747 INFO:tasks.rados.rados.0.smithi084.stdout:2609: finishing write tid 1 to smithi08434663-22 2024-09-18T14:11:04.747 INFO:tasks.rados.rados.0.smithi084.stdout:2612: seq_num 903 ranges {776721=479501,2024887=693316} 2024-09-18T14:11:04.753 INFO:tasks.rados.rados.0.smithi084.stdout:2612: writing smithi08434663-42 from 776721 to 1256222 tid 1 2024-09-18T14:11:04.761 INFO:tasks.rados.rados.0.smithi084.stdout:2612: writing smithi08434663-42 from 2024887 to 2718203 tid 2 2024-09-18T14:11:04.761 INFO:tasks.rados.rados.0.smithi084.stdout:2602: done (9 left) 2024-09-18T14:11:04.761 INFO:tasks.rados.rados.0.smithi084.stdout:2603: done (8 left) 2024-09-18T14:11:04.761 INFO:tasks.rados.rados.0.smithi084.stdout:2604: done (7 left) 2024-09-18T14:11:04.761 INFO:tasks.rados.rados.0.smithi084.stdout:2606: done (6 left) 2024-09-18T14:11:04.761 INFO:tasks.rados.rados.0.smithi084.stdout:2607: done (5 left) 2024-09-18T14:11:04.761 INFO:tasks.rados.rados.0.smithi084.stdout:2608: done (4 left) 2024-09-18T14:11:04.761 INFO:tasks.rados.rados.0.smithi084.stdout:2613: read oid 19 snap -1 2024-09-18T14:11:04.761 INFO:tasks.rados.rados.0.smithi084.stdout:2613: expect (ObjNum 789 snap 233 seq_num 789) 2024-09-18T14:11:04.762 INFO:tasks.rados.rados.0.smithi084.stdout:2614: write (excl) oid 10 current snap is 281 2024-09-18T14:11:04.762 INFO:tasks.rados.rados.0.smithi084.stdout:2609: finishing write tid 2 to smithi08434663-22 2024-09-18T14:11:04.762 INFO:tasks.rados.rados.0.smithi084.stdout:2611: expect (ObjNum 804 snap 240 seq_num 804) 2024-09-18T14:11:04.947 INFO:tasks.rados.rados.0.smithi084.stdout:2610: expect (ObjNum 827 snap 251 seq_num 827) 2024-09-18T14:11:05.118 INFO:tasks.rados.rados.0.smithi084.stdout:2609: finishing write tid 3 to smithi08434663-22 2024-09-18T14:11:05.118 INFO:tasks.rados.rados.0.smithi084.stdout:2609: finishing write tid 4 to smithi08434663-22 2024-09-18T14:11:05.118 INFO:tasks.rados.rados.0.smithi084.stdout:2609: finishing write tid 5 to smithi08434663-22 2024-09-18T14:11:05.118 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 1027 (ObjNum 902 snap 281 seq_num 902) dirty exists 2024-09-18T14:11:05.118 INFO:tasks.rados.rados.0.smithi084.stdout:2609: left oid 22 (ObjNum 902 snap 281 seq_num 902) 2024-09-18T14:11:05.118 INFO:tasks.rados.rados.0.smithi084.stdout:2614: seq_num 904 ranges {793528=554750,2083369=436974,2958856=234672} 2024-09-18T14:11:05.125 INFO:tasks.rados.rados.0.smithi084.stdout:2614: writing smithi08434663-10 from 793528 to 1348278 tid 1 2024-09-18T14:11:05.131 INFO:tasks.rados.rados.0.smithi084.stdout:2614: writing smithi08434663-10 from 2083369 to 2520343 tid 2 2024-09-18T14:11:05.135 INFO:tasks.rados.rados.0.smithi084.stdout:2614: writing smithi08434663-10 from 2958856 to 3193528 tid 3 2024-09-18T14:11:05.135 INFO:tasks.rados.rados.0.smithi084.stdout:2609: done (5 left) 2024-09-18T14:11:05.135 INFO:tasks.rados.rados.0.smithi084.stdout:2610: done (4 left) 2024-09-18T14:11:05.135 INFO:tasks.rados.rados.0.smithi084.stdout:2611: done (3 left) 2024-09-18T14:11:05.136 INFO:tasks.rados.rados.0.smithi084.stdout:2615: rollback oid 14 current snap is 281 2024-09-18T14:11:05.136 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 14 to 275 2024-09-18T14:11:05.136 INFO:tasks.rados.rados.0.smithi084.stdout:2612: finishing write tid 1 to smithi08434663-42 2024-09-18T14:11:05.136 INFO:tasks.rados.rados.0.smithi084.stdout:2613: expect (ObjNum 789 snap 233 seq_num 789) 2024-09-18T14:11:05.236 INFO:tasks.rados.rados.0.smithi084.stdout:2612: finishing write tid 2 to smithi08434663-42 2024-09-18T14:11:05.236 INFO:tasks.rados.rados.0.smithi084.stdout:2612: finishing write tid 3 to smithi08434663-42 2024-09-18T14:11:05.236 INFO:tasks.rados.rados.0.smithi084.stdout:2612: finishing write tid 4 to smithi08434663-42 2024-09-18T14:11:05.236 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1179 (ObjNum 903 snap 281 seq_num 903) dirty exists 2024-09-18T14:11:05.236 INFO:tasks.rados.rados.0.smithi084.stdout:2612: left oid 42 (ObjNum 903 snap 281 seq_num 903) 2024-09-18T14:11:05.236 INFO:tasks.rados.rados.0.smithi084.stdout:2612: done (3 left) 2024-09-18T14:11:05.236 INFO:tasks.rados.rados.0.smithi084.stdout:2613: done (2 left) 2024-09-18T14:11:05.236 INFO:tasks.rados.rados.0.smithi084.stdout:2616: snap_remove snap 274 2024-09-18T14:11:05.358 INFO:tasks.rados.rados.0.smithi084.stdout:2616: done (2 left) 2024-09-18T14:11:05.358 INFO:tasks.rados.rados.0.smithi084.stdout:2617: snap_remove snap 261 2024-09-18T14:11:06.366 INFO:tasks.rados.rados.0.smithi084.stdout:2617: done (2 left) 2024-09-18T14:11:06.366 INFO:tasks.rados.rados.0.smithi084.stdout:2618: snap_remove snap 269 2024-09-18T14:11:07.284 INFO:tasks.rados.rados.0.smithi084.stdout:2618: done (2 left) 2024-09-18T14:11:07.284 INFO:tasks.rados.rados.0.smithi084.stdout:2619: snap_remove snap 279 2024-09-18T14:11:08.294 INFO:tasks.rados.rados.0.smithi084.stdout:2619: done (2 left) 2024-09-18T14:11:08.294 INFO:tasks.rados.rados.0.smithi084.stdout:2620: write oid 6 current snap is 281 2024-09-18T14:11:08.294 INFO:tasks.rados.rados.0.smithi084.stdout:2620: seq_num 905 ranges {410335=746352,1741851=696985,3102632=107703} 2024-09-18T14:11:08.300 INFO:tasks.rados.rados.0.smithi084.stdout:2620: writing smithi08434663-6 from 410335 to 1156687 tid 1 2024-09-18T14:11:08.307 INFO:tasks.rados.rados.0.smithi084.stdout:2620: writing smithi08434663-6 from 1741851 to 2438836 tid 2 2024-09-18T14:11:08.313 INFO:tasks.rados.rados.0.smithi084.stdout:2620: writing smithi08434663-6 from 3102632 to 3210335 tid 3 2024-09-18T14:11:08.313 INFO:tasks.rados.rados.0.smithi084.stdout:2621: read oid 1 snap -1 2024-09-18T14:11:08.313 INFO:tasks.rados.rados.0.smithi084.stdout:2621: expect deleted 2024-09-18T14:11:08.313 INFO:tasks.rados.rados.0.smithi084.stdout:2622: read oid 27 snap -1 2024-09-18T14:11:08.313 INFO:tasks.rados.rados.0.smithi084.stdout:2622: expect (ObjNum 894 snap 278 seq_num 894) 2024-09-18T14:11:08.313 INFO:tasks.rados.rados.0.smithi084.stdout:2623: read oid 11 snap 278 2024-09-18T14:11:08.313 INFO:tasks.rados.rados.0.smithi084.stdout:2623: expect (ObjNum 810 snap 244 seq_num 810) 2024-09-18T14:11:08.313 INFO:tasks.rados.rados.0.smithi084.stdout:2624: rollback oid 46 current snap is 281 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 46 to 265 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:2614: finishing write tid 1 to smithi08434663-10 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:2614: finishing write tid 2 to smithi08434663-10 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:2615: finishing rollback tid 1 to smithi08434663-14 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 0 (ObjNum 65 snap 0 seq_num 0) dirty dne 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:2614: finishing write tid 3 to smithi08434663-10 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:2614: finishing write tid 4 to smithi08434663-10 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:2614: finishing write tid 5 to smithi08434663-10 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 1219 (ObjNum 904 snap 281 seq_num 904) dirty exists 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:2614: left oid 10 (ObjNum 904 snap 281 seq_num 904) 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:2620: finishing write tid 1 to smithi08434663-6 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:2614: done (6 left) 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:2615: done (5 left) 2024-09-18T14:11:08.314 INFO:tasks.rados.rados.0.smithi084.stdout:2625: delete oid 38 current snap is 281 2024-09-18T14:11:08.315 INFO:tasks.rados.rados.0.smithi084.stdout:2620: finishing write tid 2 to smithi08434663-6 2024-09-18T14:11:08.315 INFO:tasks.rados.rados.0.smithi084.stdout:2624: finishing rollback tid 1 to smithi08434663-46 2024-09-18T14:11:08.315 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1247 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:11:08.316 INFO:tasks.rados.rados.0.smithi084.stdout:2623: expect (ObjNum 810 snap 244 seq_num 810) 2024-09-18T14:11:08.537 INFO:tasks.rados.rados.0.smithi084.stdout:2620: finishing write tid 3 to smithi08434663-6 2024-09-18T14:11:08.538 INFO:tasks.rados.rados.0.smithi084.stdout:2620: finishing write tid 4 to smithi08434663-6 2024-09-18T14:11:08.538 INFO:tasks.rados.rados.0.smithi084.stdout:2620: finishing write tid 5 to smithi08434663-6 2024-09-18T14:11:08.538 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 986 (ObjNum 905 snap 281 seq_num 905) dirty exists 2024-09-18T14:11:08.538 INFO:tasks.rados.rados.0.smithi084.stdout:2620: left oid 6 (ObjNum 905 snap 281 seq_num 905) 2024-09-18T14:11:08.538 INFO:tasks.rados.rados.0.smithi084.stdout:2622: expect (ObjNum 894 snap 278 seq_num 894) 2024-09-18T14:11:08.696 INFO:tasks.rados.rados.0.smithi084.stdout:2620: done (5 left) 2024-09-18T14:11:08.696 INFO:tasks.rados.rados.0.smithi084.stdout:2621: done (4 left) 2024-09-18T14:11:08.696 INFO:tasks.rados.rados.0.smithi084.stdout:2622: done (3 left) 2024-09-18T14:11:08.696 INFO:tasks.rados.rados.0.smithi084.stdout:2623: done (2 left) 2024-09-18T14:11:08.696 INFO:tasks.rados.rados.0.smithi084.stdout:2624: done (1 left) 2024-09-18T14:11:08.696 INFO:tasks.rados.rados.0.smithi084.stdout:2625: done (0 left) 2024-09-18T14:11:08.696 INFO:tasks.rados.rados.0.smithi084.stdout:2626: write oid 6 current snap is 281 2024-09-18T14:11:08.697 INFO:tasks.rados.rados.0.smithi084.stdout:2626: seq_num 907 ranges {443949=496850,1458815=584301,2641125=602824} 2024-09-18T14:11:08.700 INFO:tasks.rados.rados.0.smithi084.stdout:2626: writing smithi08434663-6 from 443949 to 940799 tid 1 2024-09-18T14:11:08.706 INFO:tasks.rados.rados.0.smithi084.stdout:2626: writing smithi08434663-6 from 1458815 to 2043116 tid 2 2024-09-18T14:11:08.714 INFO:tasks.rados.rados.0.smithi084.stdout:2626: writing smithi08434663-6 from 2641125 to 3243949 tid 3 2024-09-18T14:11:08.714 INFO:tasks.rados.rados.0.smithi084.stdout:2627: read oid 15 snap -1 2024-09-18T14:11:08.714 INFO:tasks.rados.rados.0.smithi084.stdout:2627: expect (ObjNum 879 snap 265 seq_num 879) 2024-09-18T14:11:08.714 INFO:tasks.rados.rados.0.smithi084.stdout:2628: snap_remove snap 265 2024-09-18T14:11:09.369 INFO:tasks.rados.rados.0.smithi084.stdout:2628: done (2 left) 2024-09-18T14:11:09.369 INFO:tasks.rados.rados.0.smithi084.stdout:2629: copy_from oid 17 from oid 34 current snap is 281 2024-09-18T14:11:09.369 INFO:tasks.rados.rados.0.smithi084.stdout:2626: finishing write tid 1 to smithi08434663-6 2024-09-18T14:11:09.369 INFO:tasks.rados.rados.0.smithi084.stdout:2626: finishing write tid 2 to smithi08434663-6 2024-09-18T14:11:09.369 INFO:tasks.rados.rados.0.smithi084.stdout:2626: finishing write tid 3 to smithi08434663-6 2024-09-18T14:11:09.369 INFO:tasks.rados.rados.0.smithi084.stdout:2626: finishing write tid 4 to smithi08434663-6 2024-09-18T14:11:09.369 INFO:tasks.rados.rados.0.smithi084.stdout:2626: finishing write tid 5 to smithi08434663-6 2024-09-18T14:11:09.369 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 990 (ObjNum 907 snap 281 seq_num 907) dirty exists 2024-09-18T14:11:09.369 INFO:tasks.rados.rados.0.smithi084.stdout:2626: left oid 6 (ObjNum 907 snap 281 seq_num 907) 2024-09-18T14:11:09.369 INFO:tasks.rados.rados.0.smithi084.stdout:2627: expect (ObjNum 879 snap 265 seq_num 879) 2024-09-18T14:11:09.486 INFO:tasks.rados.rados.0.smithi084.stdout:2629: finishing copy_from to smithi08434663-17 2024-09-18T14:11:09.486 INFO:tasks.rados.rados.0.smithi084.stdout:2629: got expected ENOENT (src dne) 2024-09-18T14:11:09.486 INFO:tasks.rados.rados.0.smithi084.stdout:2629: finishing copy_from racing read to smithi08434663-17 2024-09-18T14:11:09.486 INFO:tasks.rados.rados.0.smithi084.stdout:2626: done (2 left) 2024-09-18T14:11:09.486 INFO:tasks.rados.rados.0.smithi084.stdout:2627: done (1 left) 2024-09-18T14:11:09.487 INFO:tasks.rados.rados.0.smithi084.stdout:2629: done (0 left) 2024-09-18T14:11:09.487 INFO:tasks.rados.rados.0.smithi084.stdout:2630: snap_remove snap 270 2024-09-18T14:11:10.375 INFO:tasks.rados.rados.0.smithi084.stdout:2630: done (0 left) 2024-09-18T14:11:10.375 INFO:tasks.rados.rados.0.smithi084.stdout:2631: copy_from oid 34 from oid 49 current snap is 281 2024-09-18T14:11:10.376 INFO:tasks.rados.rados.0.smithi084.stdout:2632: write oid 44 current snap is 281 2024-09-18T14:11:10.376 INFO:tasks.rados.rados.0.smithi084.stdout:2632: seq_num 908 ranges {460756=572099,1517297=444312,2668548=472870} 2024-09-18T14:11:10.382 INFO:tasks.rados.rados.0.smithi084.stdout:2632: writing smithi08434663-44 from 460756 to 1032855 tid 1 2024-09-18T14:11:10.387 INFO:tasks.rados.rados.0.smithi084.stdout:2632: writing smithi08434663-44 from 1517297 to 1961609 tid 2 2024-09-18T14:11:10.394 INFO:tasks.rados.rados.0.smithi084.stdout:2632: writing smithi08434663-44 from 2668548 to 3141418 tid 3 2024-09-18T14:11:10.395 INFO:tasks.rados.rados.0.smithi084.stdout:2633: read oid 13 snap 278 2024-09-18T14:11:10.395 INFO:tasks.rados.rados.0.smithi084.stdout:2633: expect deleted 2024-09-18T14:11:10.395 INFO:tasks.rados.rados.0.smithi084.stdout:2634: read oid 46 snap -1 2024-09-18T14:11:10.395 INFO:tasks.rados.rados.0.smithi084.stdout:2634: expect deleted 2024-09-18T14:11:10.395 INFO:tasks.rados.rados.0.smithi084.stdout:2635: read oid 31 snap -1 2024-09-18T14:11:10.395 INFO:tasks.rados.rados.0.smithi084.stdout:2635: expect deleted 2024-09-18T14:11:10.395 INFO:tasks.rados.rados.0.smithi084.stdout:2636: read oid 47 snap -1 2024-09-18T14:11:10.395 INFO:tasks.rados.rados.0.smithi084.stdout:2636: expect deleted 2024-09-18T14:11:10.395 INFO:tasks.rados.rados.0.smithi084.stdout:2637: snap_remove snap 280 2024-09-18T14:11:11.377 INFO:tasks.rados.rados.0.smithi084.stdout:2637: done (6 left) 2024-09-18T14:11:11.378 INFO:tasks.rados.rados.0.smithi084.stdout:2638: write oid 25 current snap is 281 2024-09-18T14:11:11.378 INFO:tasks.rados.rados.0.smithi084.stdout:2638: seq_num 909 ranges {477563=647348,1859426=587970,2979618=297945} 2024-09-18T14:11:11.384 INFO:tasks.rados.rados.0.smithi084.stdout:2638: writing smithi08434663-25 from 477563 to 1124911 tid 1 2024-09-18T14:11:11.392 INFO:tasks.rados.rados.0.smithi084.stdout:2638: writing smithi08434663-25 from 1859426 to 2447396 tid 2 2024-09-18T14:11:11.398 INFO:tasks.rados.rados.0.smithi084.stdout:2638: writing smithi08434663-25 from 2979618 to 3277563 tid 3 2024-09-18T14:11:11.398 INFO:tasks.rados.rados.0.smithi084.stdout:2639: read oid 20 snap -1 2024-09-18T14:11:11.398 INFO:tasks.rados.rados.0.smithi084.stdout:2639: expect deleted 2024-09-18T14:11:11.398 INFO:tasks.rados.rados.0.smithi084.stdout:2640: read oid 22 snap -1 2024-09-18T14:11:11.398 INFO:tasks.rados.rados.0.smithi084.stdout:2640: expect (ObjNum 902 snap 281 seq_num 902) 2024-09-18T14:11:11.398 INFO:tasks.rados.rados.0.smithi084.stdout:2641: snap_remove snap 276 2024-09-18T14:11:12.393 INFO:tasks.rados.rados.0.smithi084.stdout:2631: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:11:12.394 INFO:tasks.rados.rados.0.smithi084.stdout:2631: finishing copy_from to smithi08434663-34 2024-09-18T14:11:12.394 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1019 (ObjNum 899 snap 281 seq_num 899) dirty exists 2024-09-18T14:11:12.394 INFO:tasks.rados.rados.0.smithi084.stdout:2632: finishing write tid 1 to smithi08434663-44 2024-09-18T14:11:12.394 INFO:tasks.rados.rados.0.smithi084.stdout:2632: finishing write tid 2 to smithi08434663-44 2024-09-18T14:11:12.394 INFO:tasks.rados.rados.0.smithi084.stdout:2632: finishing write tid 3 to smithi08434663-44 2024-09-18T14:11:12.394 INFO:tasks.rados.rados.0.smithi084.stdout:2632: finishing write tid 4 to smithi08434663-44 2024-09-18T14:11:12.394 INFO:tasks.rados.rados.0.smithi084.stdout:2632: finishing write tid 5 to smithi08434663-44 2024-09-18T14:11:12.394 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 958 (ObjNum 908 snap 281 seq_num 908) dirty exists 2024-09-18T14:11:12.394 INFO:tasks.rados.rados.0.smithi084.stdout:2632: left oid 44 (ObjNum 908 snap 281 seq_num 908) 2024-09-18T14:11:12.394 INFO:tasks.rados.rados.0.smithi084.stdout:2638: finishing write tid 1 to smithi08434663-25 2024-09-18T14:11:12.394 INFO:tasks.rados.rados.0.smithi084.stdout:2638: finishing write tid 2 to smithi08434663-25 2024-09-18T14:11:12.394 INFO:tasks.rados.rados.0.smithi084.stdout:2638: finishing write tid 3 to smithi08434663-25 2024-09-18T14:11:12.395 INFO:tasks.rados.rados.0.smithi084.stdout:2638: finishing write tid 4 to smithi08434663-25 2024-09-18T14:11:12.395 INFO:tasks.rados.rados.0.smithi084.stdout:2638: finishing write tid 5 to smithi08434663-25 2024-09-18T14:11:12.395 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 1236 (ObjNum 909 snap 281 seq_num 909) dirty exists 2024-09-18T14:11:12.395 INFO:tasks.rados.rados.0.smithi084.stdout:2638: left oid 25 (ObjNum 909 snap 281 seq_num 909) 2024-09-18T14:11:12.395 INFO:tasks.rados.rados.0.smithi084.stdout:2640: expect (ObjNum 902 snap 281 seq_num 902) 2024-09-18T14:11:12.525 INFO:tasks.rados.rados.0.smithi084.stdout:2631: done (9 left) 2024-09-18T14:11:12.525 INFO:tasks.rados.rados.0.smithi084.stdout:2632: done (8 left) 2024-09-18T14:11:12.525 INFO:tasks.rados.rados.0.smithi084.stdout:2633: done (7 left) 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2634: done (6 left) 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2635: done (5 left) 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2636: done (4 left) 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2638: done (3 left) 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2639: done (2 left) 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2640: done (1 left) 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2641: done (0 left) 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2642: copy_from oid 47 from oid 50 current snap is 281 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2643: copy_from oid 49 from oid 35 current snap is 281 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2644: read oid 28 snap -1 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2644: expect (ObjNum 765 snap 223 seq_num 765) 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2645: read oid 38 snap 275 2024-09-18T14:11:12.526 INFO:tasks.rados.rados.0.smithi084.stdout:2645: expect (ObjNum 789 snap 233 seq_num 789) 2024-09-18T14:11:12.527 INFO:tasks.rados.rados.0.smithi084.stdout:2646: delete oid 21 current snap is 281 2024-09-18T14:11:12.527 INFO:tasks.rados.rados.0.smithi084.stdout:2646: done (4 left) 2024-09-18T14:11:12.527 INFO:tasks.rados.rados.0.smithi084.stdout:2647: read oid 5 snap -1 2024-09-18T14:11:12.527 INFO:tasks.rados.rados.0.smithi084.stdout:2647: expect deleted 2024-09-18T14:11:12.527 INFO:tasks.rados.rados.0.smithi084.stdout:2648: read oid 26 snap -1 2024-09-18T14:11:12.527 INFO:tasks.rados.rados.0.smithi084.stdout:2648: expect (ObjNum 841 snap 255 seq_num 841) 2024-09-18T14:11:12.528 INFO:tasks.rados.rados.0.smithi084.stdout:2649: snap_create 2024-09-18T14:11:12.531 INFO:tasks.rados.rados.0.smithi084.stdout:2648: expect (ObjNum 841 snap 255 seq_num 841) 2024-09-18T14:11:12.663 INFO:tasks.rados.rados.0.smithi084.stdout:2644: expect (ObjNum 765 snap 223 seq_num 765) 2024-09-18T14:11:12.755 INFO:tasks.rados.rados.0.smithi084.stdout:2645: expect (ObjNum 789 snap 233 seq_num 789) 2024-09-18T14:11:12.855 INFO:tasks.rados.rados.0.smithi084.stdout:2643: finishing copy_from racing read to smithi08434663-49 2024-09-18T14:11:12.855 INFO:tasks.rados.rados.0.smithi084.stdout:2642: finishing copy_from racing read to smithi08434663-47 2024-09-18T14:11:12.855 INFO:tasks.rados.rados.0.smithi084.stdout:2642: finishing copy_from to smithi08434663-47 2024-09-18T14:11:12.855 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 837 (ObjNum 790 snap 234 seq_num 790) dirty exists 2024-09-18T14:11:12.855 INFO:tasks.rados.rados.0.smithi084.stdout:2643: finishing copy_from to smithi08434663-49 2024-09-18T14:11:12.855 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 1246 (ObjNum 843 snap 255 seq_num 843) dirty exists 2024-09-18T14:11:13.393 INFO:tasks.rados.rados.0.smithi084.stdout:2642: done (6 left) 2024-09-18T14:11:13.393 INFO:tasks.rados.rados.0.smithi084.stdout:2643: done (5 left) 2024-09-18T14:11:13.393 INFO:tasks.rados.rados.0.smithi084.stdout:2644: done (4 left) 2024-09-18T14:11:13.393 INFO:tasks.rados.rados.0.smithi084.stdout:2645: done (3 left) 2024-09-18T14:11:13.394 INFO:tasks.rados.rados.0.smithi084.stdout:2647: done (2 left) 2024-09-18T14:11:13.394 INFO:tasks.rados.rados.0.smithi084.stdout:2648: done (1 left) 2024-09-18T14:11:13.394 INFO:tasks.rados.rados.0.smithi084.stdout:2649: done (0 left) 2024-09-18T14:11:13.394 INFO:tasks.rados.rados.0.smithi084.stdout:2650: snap_create 2024-09-18T14:11:14.411 INFO:tasks.rados.rados.0.smithi084.stdout:2650: done (0 left) 2024-09-18T14:11:14.411 INFO:tasks.rados.rados.0.smithi084.stdout:2651: snap_remove snap 273 2024-09-18T14:11:15.415 INFO:tasks.rados.rados.0.smithi084.stdout:2651: done (0 left) 2024-09-18T14:11:15.613 INFO:tasks.rados.rados.0.smithi084.stdout:2652: snap_remove snap 272 2024-09-18T14:11:16.434 INFO:tasks.rados.rados.0.smithi084.stdout:2652: done (0 left) 2024-09-18T14:11:16.434 INFO:tasks.rados.rados.0.smithi084.stdout:2653: snap_create 2024-09-18T14:11:17.288 INFO:tasks.rados.rados.0.smithi084.stdout:2653: done (0 left) 2024-09-18T14:11:17.288 INFO:tasks.rados.rados.0.smithi084.stdout:2654: read oid 19 snap 275 2024-09-18T14:11:17.288 INFO:tasks.rados.rados.0.smithi084.stdout:2654: expect (ObjNum 864 snap 258 seq_num 864) 2024-09-18T14:11:17.288 INFO:tasks.rados.rados.0.smithi084.stdout:2655: snap_create 2024-09-18T14:11:17.300 INFO:tasks.rados.rados.0.smithi084.stdout:2654: expect (ObjNum 864 snap 258 seq_num 864) 2024-09-18T14:11:18.294 INFO:tasks.rados.rados.0.smithi084.stdout:2654: done (1 left) 2024-09-18T14:11:18.294 INFO:tasks.rados.rados.0.smithi084.stdout:2655: done (0 left) 2024-09-18T14:11:18.294 INFO:tasks.rados.rados.0.smithi084.stdout:2656: copy_from oid 41 from oid 37 current snap is 285 2024-09-18T14:11:18.294 INFO:tasks.rados.rados.0.smithi084.stdout:2657: rollback oid 24 current snap is 285 2024-09-18T14:11:18.294 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 24 to 281 2024-09-18T14:11:18.294 INFO:tasks.rados.rados.0.smithi084.stdout:2658: snap_remove snap 282 2024-09-18T14:11:19.297 INFO:tasks.rados.rados.0.smithi084.stdout:2658: done (2 left) 2024-09-18T14:11:19.297 INFO:tasks.rados.rados.0.smithi084.stdout:2659: read oid 19 snap -1 2024-09-18T14:11:19.297 INFO:tasks.rados.rados.0.smithi084.stdout:2659: expect (ObjNum 789 snap 233 seq_num 789) 2024-09-18T14:11:19.297 INFO:tasks.rados.rados.0.smithi084.stdout:2660: snap_create 2024-09-18T14:11:19.297 INFO:tasks.rados.rados.0.smithi084.stdout:2657: finishing rollback tid 1 to smithi08434663-24 2024-09-18T14:11:19.297 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 590 (ObjNum 49 snap 0 seq_num 2495093440) dirty dne 2024-09-18T14:11:19.297 INFO:tasks.rados.rados.0.smithi084.stdout:2656: finishing copy_from racing read to smithi08434663-41 2024-09-18T14:11:19.297 INFO:tasks.rados.rados.0.smithi084.stdout:2656: finishing copy_from to smithi08434663-41 2024-09-18T14:11:19.297 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 1013 (ObjNum 857 snap 257 seq_num 857) dirty exists 2024-09-18T14:11:19.309 INFO:tasks.rados.rados.0.smithi084.stdout:2659: expect (ObjNum 789 snap 233 seq_num 789) 2024-09-18T14:11:20.308 INFO:tasks.rados.rados.0.smithi084.stdout:2656: done (3 left) 2024-09-18T14:11:20.308 INFO:tasks.rados.rados.0.smithi084.stdout:2657: done (2 left) 2024-09-18T14:11:20.308 INFO:tasks.rados.rados.0.smithi084.stdout:2659: done (1 left) 2024-09-18T14:11:20.308 INFO:tasks.rados.rados.0.smithi084.stdout:2660: done (0 left) 2024-09-18T14:11:20.308 INFO:tasks.rados.rados.0.smithi084.stdout:2661: read oid 26 snap 285 2024-09-18T14:11:20.308 INFO:tasks.rados.rados.0.smithi084.stdout:2661: expect (ObjNum 841 snap 255 seq_num 841) 2024-09-18T14:11:20.308 INFO:tasks.rados.rados.0.smithi084.stdout:2662: write oid 17 current snap is 286 2024-09-18T14:11:20.308 INFO:tasks.rados.rados.0.smithi084.stdout:2662: seq_num 911 ranges {511177=797846,1976390=591639,3034464=276713} 2024-09-18T14:11:20.315 INFO:tasks.rados.rados.0.smithi084.stdout:2662: writing smithi08434663-17 from 511177 to 1309023 tid 1 2024-09-18T14:11:20.321 INFO:tasks.rados.rados.0.smithi084.stdout:2662: writing smithi08434663-17 from 1976390 to 2568029 tid 2 2024-09-18T14:11:20.326 INFO:tasks.rados.rados.0.smithi084.stdout:2662: writing smithi08434663-17 from 3034464 to 3311177 tid 3 2024-09-18T14:11:20.326 INFO:tasks.rados.rados.0.smithi084.stdout:2663: snap_remove snap 278 2024-09-18T14:11:21.324 INFO:tasks.rados.rados.0.smithi084.stdout:2663: done (2 left) 2024-09-18T14:11:21.324 INFO:tasks.rados.rados.0.smithi084.stdout:2664: rollback oid 30 current snap is 286 2024-09-18T14:11:21.324 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 30 to 285 2024-09-18T14:11:21.325 INFO:tasks.rados.rados.0.smithi084.stdout:2661: expect (ObjNum 841 snap 255 seq_num 841) 2024-09-18T14:11:21.457 INFO:tasks.rados.rados.0.smithi084.stdout:2662: finishing write tid 1 to smithi08434663-17 2024-09-18T14:11:21.457 INFO:tasks.rados.rados.0.smithi084.stdout:2662: finishing write tid 2 to smithi08434663-17 2024-09-18T14:11:21.457 INFO:tasks.rados.rados.0.smithi084.stdout:2662: finishing write tid 3 to smithi08434663-17 2024-09-18T14:11:21.457 INFO:tasks.rados.rados.0.smithi084.stdout:2662: finishing write tid 4 to smithi08434663-17 2024-09-18T14:11:21.457 INFO:tasks.rados.rados.0.smithi084.stdout:2662: finishing write tid 5 to smithi08434663-17 2024-09-18T14:11:21.457 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1027 (ObjNum 911 snap 286 seq_num 911) dirty exists 2024-09-18T14:11:21.457 INFO:tasks.rados.rados.0.smithi084.stdout:2662: left oid 17 (ObjNum 911 snap 286 seq_num 911) 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:2661: done (2 left) 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:2662: done (1 left) 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:2665: read oid 8 snap 263 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:2665: expect deleted 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:2666: read oid 26 snap 263 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:2666: expect (ObjNum 841 snap 255 seq_num 841) 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:2667: rollback oid 39 current snap is 286 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 39 to 263 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:2664: finishing rollback tid 1 to smithi08434663-30 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 0 (ObjNum 305 snap 0 seq_num 1568230329) dirty dne 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:2664: done (3 left) 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:2668: read oid 5 snap -1 2024-09-18T14:11:21.458 INFO:tasks.rados.rados.0.smithi084.stdout:2668: expect deleted 2024-09-18T14:11:21.459 INFO:tasks.rados.rados.0.smithi084.stdout:2669: copy_from oid 35 from oid 34 current snap is 286 2024-09-18T14:11:21.459 INFO:tasks.rados.rados.0.smithi084.stdout:2670: delete oid 43 current snap is 286 2024-09-18T14:11:21.459 INFO:tasks.rados.rados.0.smithi084.stdout:2665: done (5 left) 2024-09-18T14:11:21.459 INFO:tasks.rados.rados.0.smithi084.stdout:2668: done (4 left) 2024-09-18T14:11:21.459 INFO:tasks.rados.rados.0.smithi084.stdout:2670: done (3 left) 2024-09-18T14:11:21.459 INFO:tasks.rados.rados.0.smithi084.stdout:2671: copy_from oid 36 from oid 40 current snap is 286 2024-09-18T14:11:21.460 INFO:tasks.rados.rados.0.smithi084.stdout:2672: copy_from oid 50 from oid 33 current snap is 286 2024-09-18T14:11:21.460 INFO:tasks.rados.rados.0.smithi084.stdout:2673: read oid 7 snap 285 2024-09-18T14:11:21.460 INFO:tasks.rados.rados.0.smithi084.stdout:2673: expect deleted 2024-09-18T14:11:21.460 INFO:tasks.rados.rados.0.smithi084.stdout:2674: write (excl) oid 43 current snap is 286 2024-09-18T14:11:21.460 INFO:tasks.rados.rados.0.smithi084.stdout:2674: seq_num 913 ranges {544791=664697,1693354=595308,2689310=445583} 2024-09-18T14:11:21.466 INFO:tasks.rados.rados.0.smithi084.stdout:2674: writing smithi08434663-43 from 544791 to 1209488 tid 1 2024-09-18T14:11:21.472 INFO:tasks.rados.rados.0.smithi084.stdout:2674: writing smithi08434663-43 from 1693354 to 2288662 tid 2 2024-09-18T14:11:21.477 INFO:tasks.rados.rados.0.smithi084.stdout:2674: writing smithi08434663-43 from 2689310 to 3134893 tid 3 2024-09-18T14:11:21.477 INFO:tasks.rados.rados.0.smithi084.stdout:2675: snap_remove snap 275 2024-09-18T14:11:22.321 INFO:tasks.rados.rados.0.smithi084.stdout:2675: done (7 left) 2024-09-18T14:11:22.322 INFO:tasks.rados.rados.0.smithi084.stdout:2676: rollback oid 13 current snap is 286 2024-09-18T14:11:22.322 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 13 to 285 2024-09-18T14:11:22.322 INFO:tasks.rados.rados.0.smithi084.stdout:2666: expect (ObjNum 841 snap 255 seq_num 841) 2024-09-18T14:11:22.454 INFO:tasks.rados.rados.0.smithi084.stdout:2667: finishing rollback tid 0 to smithi08434663-39 2024-09-18T14:11:22.455 INFO:tasks.rados.rados.0.smithi084.stdout:2667: finishing rollback tid 1 to smithi08434663-39 2024-09-18T14:11:22.455 INFO:tasks.rados.rados.0.smithi084.stdout:2667: finishing rollback tid 2 to smithi08434663-39 2024-09-18T14:11:22.455 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 1064 (ObjNum 807 snap 244 seq_num 807) dirty exists 2024-09-18T14:11:22.455 INFO:tasks.rados.rados.0.smithi084.stdout:2671: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:11:22.455 INFO:tasks.rados.rados.0.smithi084.stdout:2669: finishing copy_from racing read to smithi08434663-35 2024-09-18T14:11:22.455 INFO:tasks.rados.rados.0.smithi084.stdout:2674: finishing write tid 1 to smithi08434663-43 2024-09-18T14:11:22.455 INFO:tasks.rados.rados.0.smithi084.stdout:2672: finishing copy_from racing read to smithi08434663-50 2024-09-18T14:11:22.455 INFO:tasks.rados.rados.0.smithi084.stdout:2671: finishing copy_from to smithi08434663-36 2024-09-18T14:11:22.455 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 1016 (ObjNum 847 snap 255 seq_num 847) dirty exists 2024-09-18T14:11:22.455 INFO:tasks.rados.rados.0.smithi084.stdout:2672: finishing copy_from to smithi08434663-50 2024-09-18T14:11:22.456 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 1016 (ObjNum 847 snap 255 seq_num 847) dirty exists 2024-09-18T14:11:22.456 INFO:tasks.rados.rados.0.smithi084.stdout:2674: finishing write tid 2 to smithi08434663-43 2024-09-18T14:11:22.456 INFO:tasks.rados.rados.0.smithi084.stdout:2669: finishing copy_from to smithi08434663-35 2024-09-18T14:11:22.456 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 1019 (ObjNum 899 snap 281 seq_num 899) dirty exists 2024-09-18T14:11:22.456 INFO:tasks.rados.rados.0.smithi084.stdout:2674: finishing write tid 3 to smithi08434663-43 2024-09-18T14:11:22.456 INFO:tasks.rados.rados.0.smithi084.stdout:2674: finishing write tid 4 to smithi08434663-43 2024-09-18T14:11:22.456 INFO:tasks.rados.rados.0.smithi084.stdout:2674: finishing write tid 5 to smithi08434663-43 2024-09-18T14:11:22.456 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 644 (ObjNum 913 snap 286 seq_num 913) dirty exists 2024-09-18T14:11:22.456 INFO:tasks.rados.rados.0.smithi084.stdout:2674: left oid 43 (ObjNum 913 snap 286 seq_num 913) 2024-09-18T14:11:22.456 INFO:tasks.rados.rados.0.smithi084.stdout:2666: done (7 left) 2024-09-18T14:11:22.456 INFO:tasks.rados.rados.0.smithi084.stdout:2667: done (6 left) 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:2669: done (5 left) 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:2671: done (4 left) 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:2672: done (3 left) 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:2673: done (2 left) 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:2674: done (1 left) 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:2677: copy_from oid 22 from oid 3 current snap is 286 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:2676: finishing rollback tid 1 to smithi08434663-13 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 0 (ObjNum 241 snap 0 seq_num 2495160400) dirty dne 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:2676: done (1 left) 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:2678: delete oid 33 current snap is 286 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:2677: finishing copy_from to smithi08434663-22 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:2677: got expected ENOENT (src dne) 2024-09-18T14:11:22.457 INFO:tasks.rados.rados.0.smithi084.stdout:2677: finishing copy_from racing read to smithi08434663-22 2024-09-18T14:11:22.458 INFO:tasks.rados.rados.0.smithi084.stdout:2677: done (1 left) 2024-09-18T14:11:22.458 INFO:tasks.rados.rados.0.smithi084.stdout:2678: done (0 left) 2024-09-18T14:11:22.458 INFO:tasks.rados.rados.0.smithi084.stdout:2679: write oid 16 current snap is 286 2024-09-18T14:11:22.458 INFO:tasks.rados.rados.0.smithi084.stdout:2679: seq_num 915 ranges {578405=415195,1410318=598977,2744156=584480} 2024-09-18T14:11:22.461 INFO:tasks.rados.rados.0.smithi084.stdout:2679: writing smithi08434663-16 from 578405 to 993600 tid 1 2024-09-18T14:11:22.467 INFO:tasks.rados.rados.0.smithi084.stdout:2679: writing smithi08434663-16 from 1410318 to 2009295 tid 2 2024-09-18T14:11:22.478 INFO:tasks.rados.rados.0.smithi084.stdout:2679: writing smithi08434663-16 from 2744156 to 3328636 tid 3 2024-09-18T14:11:22.478 INFO:tasks.rados.rados.0.smithi084.stdout:2680: read oid 27 snap -1 2024-09-18T14:11:22.478 INFO:tasks.rados.rados.0.smithi084.stdout:2680: expect (ObjNum 894 snap 278 seq_num 894) 2024-09-18T14:11:22.478 INFO:tasks.rados.rados.0.smithi084.stdout:2681: read oid 20 snap -1 2024-09-18T14:11:22.478 INFO:tasks.rados.rados.0.smithi084.stdout:2681: expect deleted 2024-09-18T14:11:22.478 INFO:tasks.rados.rados.0.smithi084.stdout:2682: delete oid 1 current snap is 286 2024-09-18T14:11:22.478 INFO:tasks.rados.rados.0.smithi084.stdout:2679: finishing write tid 1 to smithi08434663-16 2024-09-18T14:11:22.478 INFO:tasks.rados.rados.0.smithi084.stdout:2679: finishing write tid 2 to smithi08434663-16 2024-09-18T14:11:22.479 INFO:tasks.rados.rados.0.smithi084.stdout:2679: finishing write tid 3 to smithi08434663-16 2024-09-18T14:11:22.479 INFO:tasks.rados.rados.0.smithi084.stdout:2680: expect (ObjNum 894 snap 278 seq_num 894) 2024-09-18T14:11:22.637 INFO:tasks.rados.rados.0.smithi084.stdout:2679: finishing write tid 4 to smithi08434663-16 2024-09-18T14:11:22.637 INFO:tasks.rados.rados.0.smithi084.stdout:2679: finishing write tid 5 to smithi08434663-16 2024-09-18T14:11:22.637 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1251 (ObjNum 915 snap 286 seq_num 915) dirty exists 2024-09-18T14:11:22.637 INFO:tasks.rados.rados.0.smithi084.stdout:2679: left oid 16 (ObjNum 915 snap 286 seq_num 915) 2024-09-18T14:11:22.637 INFO:tasks.rados.rados.0.smithi084.stdout:2679: done (3 left) 2024-09-18T14:11:22.637 INFO:tasks.rados.rados.0.smithi084.stdout:2680: done (2 left) 2024-09-18T14:11:22.637 INFO:tasks.rados.rados.0.smithi084.stdout:2681: done (1 left) 2024-09-18T14:11:22.638 INFO:tasks.rados.rados.0.smithi084.stdout:2682: done (0 left) 2024-09-18T14:11:22.638 INFO:tasks.rados.rados.0.smithi084.stdout:2683: write (excl) oid 43 current snap is 286 2024-09-18T14:11:22.638 INFO:tasks.rados.rados.0.smithi084.stdout:2683: seq_num 917 ranges {612019=565693,1810929=602646,3082649=329370} 2024-09-18T14:11:22.642 INFO:tasks.rados.rados.0.smithi084.stdout:2683: writing smithi08434663-43 from 612019 to 1177712 tid 1 2024-09-18T14:11:22.649 INFO:tasks.rados.rados.0.smithi084.stdout:2683: writing smithi08434663-43 from 1810929 to 2413575 tid 2 2024-09-18T14:11:22.655 INFO:tasks.rados.rados.0.smithi084.stdout:2683: writing smithi08434663-43 from 3082649 to 3412019 tid 3 2024-09-18T14:11:22.655 INFO:tasks.rados.rados.0.smithi084.stdout:2684: rollback oid 28 current snap is 286 2024-09-18T14:11:22.655 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 28 to 263 2024-09-18T14:11:22.655 INFO:tasks.rados.rados.0.smithi084.stdout:2683: finishing write tid 1 to smithi08434663-43 2024-09-18T14:11:22.656 INFO:tasks.rados.rados.0.smithi084.stdout:2683: finishing write tid 2 to smithi08434663-43 2024-09-18T14:11:22.656 INFO:tasks.rados.rados.0.smithi084.stdout:2685: read oid 26 snap -1 2024-09-18T14:11:22.656 INFO:tasks.rados.rados.0.smithi084.stdout:2685: expect (ObjNum 841 snap 255 seq_num 841) 2024-09-18T14:11:22.656 INFO:tasks.rados.rados.0.smithi084.stdout:2686: write (excl) oid 23 current snap is 286 2024-09-18T14:11:22.656 INFO:tasks.rados.rados.0.smithi084.stdout:2686: seq_num 918 ranges {628826=640942,1869411=462657,2826425=534649} 2024-09-18T14:11:22.661 INFO:tasks.rados.rados.0.smithi084.stdout:2686: writing smithi08434663-23 from 628826 to 1269768 tid 1 2024-09-18T14:11:22.667 INFO:tasks.rados.rados.0.smithi084.stdout:2686: writing smithi08434663-23 from 1869411 to 2332068 tid 2 2024-09-18T14:11:22.673 INFO:tasks.rados.rados.0.smithi084.stdout:2686: writing smithi08434663-23 from 2826425 to 3361074 tid 3 2024-09-18T14:11:22.673 INFO:tasks.rados.rados.0.smithi084.stdout:2687: delete oid 41 current snap is 286 2024-09-18T14:11:22.673 INFO:tasks.rados.rados.0.smithi084.stdout:2683: finishing write tid 3 to smithi08434663-43 2024-09-18T14:11:22.673 INFO:tasks.rados.rados.0.smithi084.stdout:2683: finishing write tid 4 to smithi08434663-43 2024-09-18T14:11:22.673 INFO:tasks.rados.rados.0.smithi084.stdout:2683: finishing write tid 5 to smithi08434663-43 2024-09-18T14:11:22.674 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 648 (ObjNum 917 snap 286 seq_num 917) dirty exists 2024-09-18T14:11:22.674 INFO:tasks.rados.rados.0.smithi084.stdout:2683: left oid 43 (ObjNum 917 snap 286 seq_num 917) 2024-09-18T14:11:22.675 INFO:tasks.rados.rados.0.smithi084.stdout:2685: expect (ObjNum 841 snap 255 seq_num 841) 2024-09-18T14:11:22.802 INFO:tasks.rados.rados.0.smithi084.stdout:2684: finishing rollback tid 0 to smithi08434663-28 2024-09-18T14:11:22.802 INFO:tasks.rados.rados.0.smithi084.stdout:2684: finishing rollback tid 1 to smithi08434663-28 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:2684: finishing rollback tid 2 to smithi08434663-28 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1371 (ObjNum 765 snap 223 seq_num 765) dirty exists 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:2686: finishing write tid 1 to smithi08434663-23 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:2686: finishing write tid 2 to smithi08434663-23 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:2686: finishing write tid 3 to smithi08434663-23 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:2686: finishing write tid 4 to smithi08434663-23 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:2686: finishing write tid 5 to smithi08434663-23 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 994 (ObjNum 918 snap 286 seq_num 918) dirty exists 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:2686: left oid 23 (ObjNum 918 snap 286 seq_num 918) 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:2683: done (4 left) 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:2684: done (3 left) 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:2685: done (2 left) 2024-09-18T14:11:22.803 INFO:tasks.rados.rados.0.smithi084.stdout:2686: done (1 left) 2024-09-18T14:11:22.804 INFO:tasks.rados.rados.0.smithi084.stdout:2687: done (0 left) 2024-09-18T14:11:22.804 INFO:tasks.rados.rados.0.smithi084.stdout:2688: snap_create 2024-09-18T14:11:23.321 INFO:tasks.rados.rados.0.smithi084.stdout:2688: done (0 left) 2024-09-18T14:11:23.321 INFO:tasks.rados.rados.0.smithi084.stdout:2689: snap_create 2024-09-18T14:11:24.322 INFO:tasks.rados.rados.0.smithi084.stdout:2689: done (0 left) 2024-09-18T14:11:24.323 INFO:tasks.rados.rados.0.smithi084.stdout:2690: rollback oid 28 current snap is 288 2024-09-18T14:11:24.323 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 28 to 281 2024-09-18T14:11:24.323 INFO:tasks.rados.rados.0.smithi084.stdout:2691: read oid 14 snap -1 2024-09-18T14:11:24.323 INFO:tasks.rados.rados.0.smithi084.stdout:2691: expect deleted 2024-09-18T14:11:24.323 INFO:tasks.rados.rados.0.smithi084.stdout:2692: snap_create 2024-09-18T14:11:24.331 INFO:tasks.rados.rados.0.smithi084.stdout:2690: finishing rollback tid 0 to smithi08434663-28 2024-09-18T14:11:24.334 INFO:tasks.rados.rados.0.smithi084.stdout:2690: finishing rollback tid 1 to smithi08434663-28 2024-09-18T14:11:24.334 INFO:tasks.rados.rados.0.smithi084.stdout:2690: finishing rollback tid 2 to smithi08434663-28 2024-09-18T14:11:24.334 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1375 (ObjNum 765 snap 223 seq_num 765) dirty exists 2024-09-18T14:11:25.335 INFO:tasks.rados.rados.0.smithi084.stdout:2690: done (2 left) 2024-09-18T14:11:25.335 INFO:tasks.rados.rados.0.smithi084.stdout:2691: done (1 left) 2024-09-18T14:11:25.335 INFO:tasks.rados.rados.0.smithi084.stdout:2692: done (0 left) 2024-09-18T14:11:25.335 INFO:tasks.rados.rados.0.smithi084.stdout:2693: snap_remove snap 281 2024-09-18T14:11:26.348 INFO:tasks.rados.rados.0.smithi084.stdout:2693: done (0 left) 2024-09-18T14:11:26.349 INFO:tasks.rados.rados.0.smithi084.stdout:2694: delete oid 28 current snap is 289 2024-09-18T14:11:26.364 INFO:tasks.rados.rados.0.smithi084.stdout:2694: done (0 left) 2024-09-18T14:11:26.364 INFO:tasks.rados.rados.0.smithi084.stdout:2695: delete oid 23 current snap is 289 2024-09-18T14:11:26.366 INFO:tasks.rados.rados.0.smithi084.stdout:2695: done (0 left) 2024-09-18T14:11:26.366 INFO:tasks.rados.rados.0.smithi084.stdout:2696: write oid 47 current snap is 289 2024-09-18T14:11:26.366 INFO:tasks.rados.rados.0.smithi084.stdout:2696: seq_num 922 ranges {696054=658291,2103339=753642} 2024-09-18T14:11:26.373 INFO:tasks.rados.rados.0.smithi084.stdout:2696: writing smithi08434663-47 from 696054 to 1354345 tid 1 2024-09-18T14:11:26.381 INFO:tasks.rados.rados.0.smithi084.stdout:2696: writing smithi08434663-47 from 2103339 to 2856981 tid 2 2024-09-18T14:11:26.381 INFO:tasks.rados.rados.0.smithi084.stdout:2697: delete oid 29 current snap is 289 2024-09-18T14:11:26.381 INFO:tasks.rados.rados.0.smithi084.stdout:2696: finishing write tid 1 to smithi08434663-47 2024-09-18T14:11:26.383 INFO:tasks.rados.rados.0.smithi084.stdout:2697: done (1 left) 2024-09-18T14:11:26.383 INFO:tasks.rados.rados.0.smithi084.stdout:2698: write (excl) oid 24 current snap is 289 2024-09-18T14:11:26.383 INFO:tasks.rados.rados.0.smithi084.stdout:2698: seq_num 924 ranges {729668=408789,1820303=757311,3274610=255058} 2024-09-18T14:11:26.389 INFO:tasks.rados.rados.0.smithi084.stdout:2698: writing smithi08434663-24 from 729668 to 1138457 tid 1 2024-09-18T14:11:26.397 INFO:tasks.rados.rados.0.smithi084.stdout:2698: writing smithi08434663-24 from 1820303 to 2577614 tid 2 2024-09-18T14:11:26.403 INFO:tasks.rados.rados.0.smithi084.stdout:2698: writing smithi08434663-24 from 3274610 to 3529668 tid 3 2024-09-18T14:11:26.403 INFO:tasks.rados.rados.0.smithi084.stdout:2699: snap_create 2024-09-18T14:11:26.403 INFO:tasks.rados.rados.0.smithi084.stdout:2696: finishing write tid 2 to smithi08434663-47 2024-09-18T14:11:26.403 INFO:tasks.rados.rados.0.smithi084.stdout:2696: finishing write tid 3 to smithi08434663-47 2024-09-18T14:11:26.403 INFO:tasks.rados.rados.0.smithi084.stdout:2696: finishing write tid 4 to smithi08434663-47 2024-09-18T14:11:26.403 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 1067 (ObjNum 922 snap 289 seq_num 922) dirty exists 2024-09-18T14:11:26.403 INFO:tasks.rados.rados.0.smithi084.stdout:2696: left oid 47 (ObjNum 922 snap 289 seq_num 922) 2024-09-18T14:11:26.403 INFO:tasks.rados.rados.0.smithi084.stdout:2698: finishing write tid 1 to smithi08434663-24 2024-09-18T14:11:26.403 INFO:tasks.rados.rados.0.smithi084.stdout:2698: finishing write tid 2 to smithi08434663-24 2024-09-18T14:11:26.405 INFO:tasks.rados.rados.0.smithi084.stdout:2698: finishing write tid 3 to smithi08434663-24 2024-09-18T14:11:26.405 INFO:tasks.rados.rados.0.smithi084.stdout:2698: finishing write tid 4 to smithi08434663-24 2024-09-18T14:11:26.405 INFO:tasks.rados.rados.0.smithi084.stdout:2698: finishing write tid 5 to smithi08434663-24 2024-09-18T14:11:26.405 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 594 (ObjNum 924 snap 289 seq_num 924) dirty exists 2024-09-18T14:11:26.405 INFO:tasks.rados.rados.0.smithi084.stdout:2698: left oid 24 (ObjNum 924 snap 289 seq_num 924) 2024-09-18T14:11:27.289 INFO:tasks.rados.rados.0.smithi084.stdout:2696: done (2 left) 2024-09-18T14:11:27.289 INFO:tasks.rados.rados.0.smithi084.stdout:2698: done (1 left) 2024-09-18T14:11:27.290 INFO:tasks.rados.rados.0.smithi084.stdout:2699: done (0 left) 2024-09-18T14:11:27.290 INFO:tasks.rados.rados.0.smithi084.stdout:2700: write (excl) oid 12 current snap is 290 2024-09-18T14:11:27.290 INFO:tasks.rados.rados.0.smithi084.stdout:2700: seq_num 925 ranges {746475=484038,1762432=617322,2785680=529906} 2024-09-18T14:11:27.297 INFO:tasks.rados.rados.0.smithi084.stdout:2700: writing smithi08434663-12 from 746475 to 1230513 tid 1 2024-09-18T14:11:27.303 INFO:tasks.rados.rados.0.smithi084.stdout:2700: writing smithi08434663-12 from 1762432 to 2379754 tid 2 2024-09-18T14:11:27.308 INFO:tasks.rados.rados.0.smithi084.stdout:2700: writing smithi08434663-12 from 2785680 to 3315586 tid 3 2024-09-18T14:11:27.309 INFO:tasks.rados.rados.0.smithi084.stdout:2701: snap_create 2024-09-18T14:11:27.309 INFO:tasks.rados.rados.0.smithi084.stdout:2700: finishing write tid 1 to smithi08434663-12 2024-09-18T14:11:27.309 INFO:tasks.rados.rados.0.smithi084.stdout:2700: finishing write tid 2 to smithi08434663-12 2024-09-18T14:11:27.312 INFO:tasks.rados.rados.0.smithi084.stdout:2700: finishing write tid 3 to smithi08434663-12 2024-09-18T14:11:27.313 INFO:tasks.rados.rados.0.smithi084.stdout:2700: finishing write tid 4 to smithi08434663-12 2024-09-18T14:11:27.313 INFO:tasks.rados.rados.0.smithi084.stdout:2700: finishing write tid 5 to smithi08434663-12 2024-09-18T14:11:27.313 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 1386 (ObjNum 925 snap 290 seq_num 925) dirty exists 2024-09-18T14:11:27.313 INFO:tasks.rados.rados.0.smithi084.stdout:2700: left oid 12 (ObjNum 925 snap 290 seq_num 925) 2024-09-18T14:11:28.295 INFO:tasks.rados.rados.0.smithi084.stdout:2700: done (1 left) 2024-09-18T14:11:28.296 INFO:tasks.rados.rados.0.smithi084.stdout:2701: done (0 left) 2024-09-18T14:11:28.296 INFO:tasks.rados.rados.0.smithi084.stdout:2702: copy_from oid 25 from oid 37 current snap is 291 2024-09-18T14:11:28.296 INFO:tasks.rados.rados.0.smithi084.stdout:2703: read oid 8 snap -1 2024-09-18T14:11:28.296 INFO:tasks.rados.rados.0.smithi084.stdout:2703: expect deleted 2024-09-18T14:11:28.296 INFO:tasks.rados.rados.0.smithi084.stdout:2704: delete oid 35 current snap is 291 2024-09-18T14:11:28.300 INFO:tasks.rados.rados.0.smithi084.stdout:2703: done (2 left) 2024-09-18T14:11:28.300 INFO:tasks.rados.rados.0.smithi084.stdout:2704: done (1 left) 2024-09-18T14:11:28.300 INFO:tasks.rados.rados.0.smithi084.stdout:2705: snap_create 2024-09-18T14:11:28.307 INFO:tasks.rados.rados.0.smithi084.stdout:2702: finishing copy_from racing read to smithi08434663-25 2024-09-18T14:11:28.310 INFO:tasks.rados.rados.0.smithi084.stdout:2702: finishing copy_from to smithi08434663-25 2024-09-18T14:11:28.310 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 1013 (ObjNum 857 snap 257 seq_num 857) dirty exists 2024-09-18T14:11:29.300 INFO:tasks.rados.rados.0.smithi084.stdout:2702: done (1 left) 2024-09-18T14:11:29.300 INFO:tasks.rados.rados.0.smithi084.stdout:2705: done (0 left) 2024-09-18T14:11:29.300 INFO:tasks.rados.rados.0.smithi084.stdout:2706: read oid 40 snap -1 2024-09-18T14:11:29.300 INFO:tasks.rados.rados.0.smithi084.stdout:2706: expect (ObjNum 847 snap 255 seq_num 847) 2024-09-18T14:11:29.300 INFO:tasks.rados.rados.0.smithi084.stdout:2707: snap_create 2024-09-18T14:11:29.306 INFO:tasks.rados.rados.0.smithi084.stdout:2706: expect (ObjNum 847 snap 255 seq_num 847) 2024-09-18T14:11:30.309 INFO:tasks.rados.rados.0.smithi084.stdout:2706: done (1 left) 2024-09-18T14:11:30.309 INFO:tasks.rados.rados.0.smithi084.stdout:2707: done (0 left) 2024-09-18T14:11:30.309 INFO:tasks.rados.rados.0.smithi084.stdout:2708: rollback oid 7 current snap is 293 2024-09-18T14:11:30.309 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 7 to 291 2024-09-18T14:11:30.309 INFO:tasks.rados.rados.0.smithi084.stdout:2709: read oid 12 snap -1 2024-09-18T14:11:30.309 INFO:tasks.rados.rados.0.smithi084.stdout:2709: expect (ObjNum 925 snap 290 seq_num 925) 2024-09-18T14:11:30.309 INFO:tasks.rados.rados.0.smithi084.stdout:2710: write (excl) oid 28 current snap is 293 2024-09-18T14:11:30.310 INFO:tasks.rados.rados.0.smithi084.stdout:2710: seq_num 927 ranges {780089=634536,1879396=620991,3240526=339563} 2024-09-18T14:11:30.317 INFO:tasks.rados.rados.0.smithi084.stdout:2710: writing smithi08434663-28 from 780089 to 1414625 tid 1 2024-09-18T14:11:30.323 INFO:tasks.rados.rados.0.smithi084.stdout:2710: writing smithi08434663-28 from 1879396 to 2500387 tid 2 2024-09-18T14:11:30.329 INFO:tasks.rados.rados.0.smithi084.stdout:2710: writing smithi08434663-28 from 3240526 to 3580089 tid 3 2024-09-18T14:11:30.329 INFO:tasks.rados.rados.0.smithi084.stdout:2711: snap_remove snap 287 2024-09-18T14:11:31.313 INFO:tasks.rados.rados.0.smithi084.stdout:2711: done (3 left) 2024-09-18T14:11:31.313 INFO:tasks.rados.rados.0.smithi084.stdout:2712: read oid 22 snap -1 2024-09-18T14:11:31.313 INFO:tasks.rados.rados.0.smithi084.stdout:2712: expect (ObjNum 902 snap 281 seq_num 902) 2024-09-18T14:11:31.313 INFO:tasks.rados.rados.0.smithi084.stdout:2713: write oid 45 current snap is 293 2024-09-18T14:11:31.313 INFO:tasks.rados.rados.0.smithi084.stdout:2708: finishing rollback tid 1 to smithi08434663-7 2024-09-18T14:11:31.313 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 977 (ObjNum 97 snap 0 seq_num 0) dirty dne 2024-09-18T14:11:31.313 INFO:tasks.rados.rados.0.smithi084.stdout:2713: seq_num 928 ranges {796896=426138,1654231=764649,2984302=480075} 2024-09-18T14:11:31.320 INFO:tasks.rados.rados.0.smithi084.stdout:2713: writing smithi08434663-45 from 796896 to 1223034 tid 1 2024-09-18T14:11:31.328 INFO:tasks.rados.rados.0.smithi084.stdout:2713: writing smithi08434663-45 from 1654231 to 2418880 tid 2 2024-09-18T14:11:31.334 INFO:tasks.rados.rados.0.smithi084.stdout:2713: writing smithi08434663-45 from 2984302 to 3464377 tid 3 2024-09-18T14:11:31.334 INFO:tasks.rados.rados.0.smithi084.stdout:2708: done (4 left) 2024-09-18T14:11:31.335 INFO:tasks.rados.rados.0.smithi084.stdout:2714: rollback oid 6 current snap is 293 2024-09-18T14:11:31.335 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 291 2024-09-18T14:11:31.335 INFO:tasks.rados.rados.0.smithi084.stdout:2709: expect (ObjNum 925 snap 290 seq_num 925) 2024-09-18T14:11:31.540 INFO:tasks.rados.rados.0.smithi084.stdout:2710: finishing write tid 1 to smithi08434663-28 2024-09-18T14:11:31.541 INFO:tasks.rados.rados.0.smithi084.stdout:2710: finishing write tid 2 to smithi08434663-28 2024-09-18T14:11:31.541 INFO:tasks.rados.rados.0.smithi084.stdout:2710: finishing write tid 3 to smithi08434663-28 2024-09-18T14:11:31.541 INFO:tasks.rados.rados.0.smithi084.stdout:2710: finishing write tid 4 to smithi08434663-28 2024-09-18T14:11:31.541 INFO:tasks.rados.rados.0.smithi084.stdout:2710: finishing write tid 5 to smithi08434663-28 2024-09-18T14:11:31.541 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1390 (ObjNum 927 snap 293 seq_num 927) dirty exists 2024-09-18T14:11:31.541 INFO:tasks.rados.rados.0.smithi084.stdout:2710: left oid 28 (ObjNum 927 snap 293 seq_num 927) 2024-09-18T14:11:31.541 INFO:tasks.rados.rados.0.smithi084.stdout:2709: done (4 left) 2024-09-18T14:11:31.541 INFO:tasks.rados.rados.0.smithi084.stdout:2710: done (3 left) 2024-09-18T14:11:31.541 INFO:tasks.rados.rados.0.smithi084.stdout:2715: snap_create 2024-09-18T14:11:31.541 INFO:tasks.rados.rados.0.smithi084.stdout:2713: finishing write tid 1 to smithi08434663-45 2024-09-18T14:11:31.541 INFO:tasks.rados.rados.0.smithi084.stdout:2713: finishing write tid 2 to smithi08434663-45 2024-09-18T14:11:31.541 INFO:tasks.rados.rados.0.smithi084.stdout:2712: expect (ObjNum 902 snap 281 seq_num 902) 2024-09-18T14:11:31.667 INFO:tasks.rados.rados.0.smithi084.stdout:2713: finishing write tid 3 to smithi08434663-45 2024-09-18T14:11:31.667 INFO:tasks.rados.rados.0.smithi084.stdout:2713: finishing write tid 4 to smithi08434663-45 2024-09-18T14:11:31.667 INFO:tasks.rados.rados.0.smithi084.stdout:2713: finishing write tid 5 to smithi08434663-45 2024-09-18T14:11:31.667 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 1235 (ObjNum 928 snap 293 seq_num 928) dirty exists 2024-09-18T14:11:31.668 INFO:tasks.rados.rados.0.smithi084.stdout:2713: left oid 45 (ObjNum 928 snap 293 seq_num 928) 2024-09-18T14:11:31.668 INFO:tasks.rados.rados.0.smithi084.stdout:2714: finishing rollback tid 0 to smithi08434663-6 2024-09-18T14:11:31.668 INFO:tasks.rados.rados.0.smithi084.stdout:2714: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:11:31.668 INFO:tasks.rados.rados.0.smithi084.stdout:2714: finishing rollback tid 2 to smithi08434663-6 2024-09-18T14:11:31.668 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 998 (ObjNum 907 snap 281 seq_num 907) dirty exists 2024-09-18T14:11:32.321 INFO:tasks.rados.rados.0.smithi084.stdout:2712: done (3 left) 2024-09-18T14:11:32.321 INFO:tasks.rados.rados.0.smithi084.stdout:2713: done (2 left) 2024-09-18T14:11:32.321 INFO:tasks.rados.rados.0.smithi084.stdout:2714: done (1 left) 2024-09-18T14:11:32.321 INFO:tasks.rados.rados.0.smithi084.stdout:2715: done (0 left) 2024-09-18T14:11:32.321 INFO:tasks.rados.rados.0.smithi084.stdout:2716: snap_create 2024-09-18T14:11:33.323 INFO:tasks.rados.rados.0.smithi084.stdout:2716: done (0 left) 2024-09-18T14:11:33.323 INFO:tasks.rados.rados.0.smithi084.stdout:2717: rollback oid 12 current snap is 295 2024-09-18T14:11:33.323 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 12 to 294 2024-09-18T14:11:33.323 INFO:tasks.rados.rados.0.smithi084.stdout:2718: copy_from oid 9 from oid 38 current snap is 295 2024-09-18T14:11:33.323 INFO:tasks.rados.rados.0.smithi084.stdout:2719: snap_remove snap 286 2024-09-18T14:11:34.333 INFO:tasks.rados.rados.0.smithi084.stdout:2718: finishing copy_from to smithi08434663-9 2024-09-18T14:11:34.333 INFO:tasks.rados.rados.0.smithi084.stdout:2718: got expected ENOENT (src dne) 2024-09-18T14:11:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:2718: finishing copy_from racing read to smithi08434663-9 2024-09-18T14:11:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:2717: finishing rollback tid 0 to smithi08434663-12 2024-09-18T14:11:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:2717: finishing rollback tid 1 to smithi08434663-12 2024-09-18T14:11:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:2717: finishing rollback tid 2 to smithi08434663-12 2024-09-18T14:11:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 1398 (ObjNum 925 snap 290 seq_num 925) dirty exists 2024-09-18T14:11:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:2717: done (2 left) 2024-09-18T14:11:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:2718: done (1 left) 2024-09-18T14:11:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:2719: done (0 left) 2024-09-18T14:11:34.334 INFO:tasks.rados.rados.0.smithi084.stdout:2720: snap_create 2024-09-18T14:11:35.335 INFO:tasks.rados.rados.0.smithi084.stdout:2720: done (0 left) 2024-09-18T14:11:35.335 INFO:tasks.rados.rados.0.smithi084.stdout:2721: delete oid 4 current snap is 296 2024-09-18T14:11:35.339 INFO:tasks.rados.rados.0.smithi084.stdout:2721: done (0 left) 2024-09-18T14:11:35.339 INFO:tasks.rados.rados.0.smithi084.stdout:2722: write (excl) oid 19 current snap is 296 2024-09-18T14:11:35.339 INFO:tasks.rados.rados.0.smithi084.stdout:2722: seq_num 930 ranges {430510=576636,1654842=768318,2922795=502619} 2024-09-18T14:11:35.344 INFO:tasks.rados.rados.0.smithi084.stdout:2722: writing smithi08434663-19 from 430510 to 1007146 tid 1 2024-09-18T14:11:35.352 INFO:tasks.rados.rados.0.smithi084.stdout:2722: writing smithi08434663-19 from 1654842 to 2423160 tid 2 2024-09-18T14:11:35.358 INFO:tasks.rados.rados.0.smithi084.stdout:2722: writing smithi08434663-19 from 2922795 to 3425414 tid 3 2024-09-18T14:11:35.359 INFO:tasks.rados.rados.0.smithi084.stdout:2723: rollback oid 47 current snap is 296 2024-09-18T14:11:35.359 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 47 to 294 2024-09-18T14:11:35.359 INFO:tasks.rados.rados.0.smithi084.stdout:2722: finishing write tid 1 to smithi08434663-19 2024-09-18T14:11:35.359 INFO:tasks.rados.rados.0.smithi084.stdout:2724: write (excl) oid 48 current snap is 296 2024-09-18T14:11:35.359 INFO:tasks.rados.rados.0.smithi084.stdout:2724: seq_num 931 ranges {447317=651885,1713324=628329,3066571=580746} 2024-09-18T14:11:35.364 INFO:tasks.rados.rados.0.smithi084.stdout:2724: writing smithi08434663-48 from 447317 to 1099202 tid 1 2024-09-18T14:11:35.371 INFO:tasks.rados.rados.0.smithi084.stdout:2724: writing smithi08434663-48 from 1713324 to 2341653 tid 2 2024-09-18T14:11:35.379 INFO:tasks.rados.rados.0.smithi084.stdout:2724: writing smithi08434663-48 from 3066571 to 3647317 tid 3 2024-09-18T14:11:35.379 INFO:tasks.rados.rados.0.smithi084.stdout:2725: read oid 4 snap -1 2024-09-18T14:11:35.379 INFO:tasks.rados.rados.0.smithi084.stdout:2725: expect deleted 2024-09-18T14:11:35.379 INFO:tasks.rados.rados.0.smithi084.stdout:2726: delete oid 1 current snap is 296 2024-09-18T14:11:35.379 INFO:tasks.rados.rados.0.smithi084.stdout:2722: finishing write tid 2 to smithi08434663-19 2024-09-18T14:11:35.379 INFO:tasks.rados.rados.0.smithi084.stdout:2723: finishing rollback tid 0 to smithi08434663-47 2024-09-18T14:11:35.379 INFO:tasks.rados.rados.0.smithi084.stdout:2722: finishing write tid 3 to smithi08434663-19 2024-09-18T14:11:35.379 INFO:tasks.rados.rados.0.smithi084.stdout:2722: finishing write tid 4 to smithi08434663-19 2024-09-18T14:11:35.379 INFO:tasks.rados.rados.0.smithi084.stdout:2723: finishing rollback tid 1 to smithi08434663-47 2024-09-18T14:11:35.380 INFO:tasks.rados.rados.0.smithi084.stdout:2723: finishing rollback tid 2 to smithi08434663-47 2024-09-18T14:11:35.380 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 1070 (ObjNum 922 snap 289 seq_num 922) dirty exists 2024-09-18T14:11:35.380 INFO:tasks.rados.rados.0.smithi084.stdout:2722: finishing write tid 5 to smithi08434663-19 2024-09-18T14:11:35.380 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1407 (ObjNum 930 snap 296 seq_num 930) dirty exists 2024-09-18T14:11:35.380 INFO:tasks.rados.rados.0.smithi084.stdout:2722: left oid 19 (ObjNum 930 snap 296 seq_num 930) 2024-09-18T14:11:35.380 INFO:tasks.rados.rados.0.smithi084.stdout:2724: finishing write tid 1 to smithi08434663-48 2024-09-18T14:11:35.380 INFO:tasks.rados.rados.0.smithi084.stdout:2724: finishing write tid 2 to smithi08434663-48 2024-09-18T14:11:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:2724: finishing write tid 3 to smithi08434663-48 2024-09-18T14:11:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:2722: done (4 left) 2024-09-18T14:11:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:2723: done (3 left) 2024-09-18T14:11:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:2725: done (2 left) 2024-09-18T14:11:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:2726: done (1 left) 2024-09-18T14:11:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:2727: read oid 50 snap -1 2024-09-18T14:11:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:2727: expect (ObjNum 847 snap 255 seq_num 847) 2024-09-18T14:11:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:2728: copy_from oid 47 from oid 15 current snap is 296 2024-09-18T14:11:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:2729: read oid 29 snap 289 2024-09-18T14:11:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:2729: expect deleted 2024-09-18T14:11:35.382 INFO:tasks.rados.rados.0.smithi084.stdout:2730: snap_remove snap 285 2024-09-18T14:11:36.357 INFO:tasks.rados.rados.0.smithi084.stdout:2724: finishing write tid 4 to smithi08434663-48 2024-09-18T14:11:36.357 INFO:tasks.rados.rados.0.smithi084.stdout:2724: finishing write tid 5 to smithi08434663-48 2024-09-18T14:11:36.357 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1240 (ObjNum 931 snap 296 seq_num 931) dirty exists 2024-09-18T14:11:36.358 INFO:tasks.rados.rados.0.smithi084.stdout:2724: left oid 48 (ObjNum 931 snap 296 seq_num 931) 2024-09-18T14:11:36.358 INFO:tasks.rados.rados.0.smithi084.stdout:2727: expect (ObjNum 847 snap 255 seq_num 847) 2024-09-18T14:11:36.459 INFO:tasks.rados.rados.0.smithi084.stdout:2728: finishing copy_from racing read to smithi08434663-47 2024-09-18T14:11:36.459 INFO:tasks.rados.rados.0.smithi084.stdout:2728: finishing copy_from to smithi08434663-47 2024-09-18T14:11:36.459 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 1058 (ObjNum 879 snap 265 seq_num 879) dirty exists 2024-09-18T14:11:36.459 INFO:tasks.rados.rados.0.smithi084.stdout:2724: done (4 left) 2024-09-18T14:11:36.459 INFO:tasks.rados.rados.0.smithi084.stdout:2727: done (3 left) 2024-09-18T14:11:36.459 INFO:tasks.rados.rados.0.smithi084.stdout:2728: done (2 left) 2024-09-18T14:11:36.459 INFO:tasks.rados.rados.0.smithi084.stdout:2729: done (1 left) 2024-09-18T14:11:36.459 INFO:tasks.rados.rados.0.smithi084.stdout:2730: done (0 left) 2024-09-18T14:11:36.459 INFO:tasks.rados.rados.0.smithi084.stdout:2731: rollback oid 49 current snap is 296 2024-09-18T14:11:36.459 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 49 to 289 2024-09-18T14:11:36.460 INFO:tasks.rados.rados.0.smithi084.stdout:2732: read oid 2 snap 295 2024-09-18T14:11:36.460 INFO:tasks.rados.rados.0.smithi084.stdout:2732: expect deleted 2024-09-18T14:11:36.460 INFO:tasks.rados.rados.0.smithi084.stdout:2733: rollback oid 47 current snap is 296 2024-09-18T14:11:36.460 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 47 to 284 2024-09-18T14:11:36.460 INFO:tasks.rados.rados.0.smithi084.stdout:2734: delete oid 19 current snap is 296 2024-09-18T14:11:36.461 INFO:tasks.rados.rados.0.smithi084.stdout:2732: done (3 left) 2024-09-18T14:11:36.461 INFO:tasks.rados.rados.0.smithi084.stdout:2734: done (2 left) 2024-09-18T14:11:36.461 INFO:tasks.rados.rados.0.smithi084.stdout:2735: rollback oid 46 current snap is 296 2024-09-18T14:11:36.461 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 46 to 293 2024-09-18T14:11:36.461 INFO:tasks.rados.rados.0.smithi084.stdout:2736: snap_remove snap 288 2024-09-18T14:11:37.293 INFO:tasks.rados.rados.0.smithi084.stdout:2736: done (3 left) 2024-09-18T14:11:37.294 INFO:tasks.rados.rados.0.smithi084.stdout:2737: snap_remove snap 290 2024-09-18T14:11:38.303 INFO:tasks.rados.rados.0.smithi084.stdout:2737: done (3 left) 2024-09-18T14:11:38.303 INFO:tasks.rados.rados.0.smithi084.stdout:2738: copy_from oid 25 from oid 6 current snap is 296 2024-09-18T14:11:38.303 INFO:tasks.rados.rados.0.smithi084.stdout:2731: finishing rollback tid 0 to smithi08434663-49 2024-09-18T14:11:38.303 INFO:tasks.rados.rados.0.smithi084.stdout:2735: finishing rollback tid 1 to smithi08434663-46 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 0 (ObjNum 225 snap 0 seq_num 2495305600) dirty dne 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:2733: finishing rollback tid 0 to smithi08434663-47 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:2731: finishing rollback tid 1 to smithi08434663-49 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:2733: finishing rollback tid 1 to smithi08434663-47 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:2731: finishing rollback tid 2 to smithi08434663-49 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 1250 (ObjNum 843 snap 255 seq_num 843) dirty exists 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:2733: finishing rollback tid 2 to smithi08434663-47 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 1073 (ObjNum 790 snap 234 seq_num 790) dirty exists 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:2731: done (3 left) 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:2733: done (2 left) 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:2735: done (1 left) 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:2739: read oid 45 snap -1 2024-09-18T14:11:38.304 INFO:tasks.rados.rados.0.smithi084.stdout:2739: expect (ObjNum 928 snap 293 seq_num 928) 2024-09-18T14:11:38.305 INFO:tasks.rados.rados.0.smithi084.stdout:2740: read oid 46 snap -1 2024-09-18T14:11:38.305 INFO:tasks.rados.rados.0.smithi084.stdout:2740: expect deleted 2024-09-18T14:11:38.305 INFO:tasks.rados.rados.0.smithi084.stdout:2741: copy_from oid 32 from oid 33 current snap is 296 2024-09-18T14:11:38.305 INFO:tasks.rados.rados.0.smithi084.stdout:2742: delete oid 29 current snap is 296 2024-09-18T14:11:38.307 INFO:tasks.rados.rados.0.smithi084.stdout:2741: finishing copy_from to smithi08434663-32 2024-09-18T14:11:38.307 INFO:tasks.rados.rados.0.smithi084.stdout:2741: got expected ENOENT (src dne) 2024-09-18T14:11:38.307 INFO:tasks.rados.rados.0.smithi084.stdout:2741: finishing copy_from racing read to smithi08434663-32 2024-09-18T14:11:38.313 INFO:tasks.rados.rados.0.smithi084.stdout:2740: done (4 left) 2024-09-18T14:11:38.313 INFO:tasks.rados.rados.0.smithi084.stdout:2741: done (3 left) 2024-09-18T14:11:38.313 INFO:tasks.rados.rados.0.smithi084.stdout:2742: done (2 left) 2024-09-18T14:11:38.313 INFO:tasks.rados.rados.0.smithi084.stdout:2743: read oid 30 snap -1 2024-09-18T14:11:38.313 INFO:tasks.rados.rados.0.smithi084.stdout:2743: expect deleted 2024-09-18T14:11:38.313 INFO:tasks.rados.rados.0.smithi084.stdout:2744: copy_from oid 24 from oid 4 current snap is 296 2024-09-18T14:11:38.313 INFO:tasks.rados.rados.0.smithi084.stdout:2745: copy_from oid 9 from oid 37 current snap is 296 2024-09-18T14:11:38.313 INFO:tasks.rados.rados.0.smithi084.stdout:2746: write oid 33 current snap is 296 2024-09-18T14:11:38.313 INFO:tasks.rados.rados.0.smithi084.stdout:2746: seq_num 935 ranges {514545=552881,1830899=519314,2943557=475332} 2024-09-18T14:11:38.319 INFO:tasks.rados.rados.0.smithi084.stdout:2746: writing smithi08434663-33 from 514545 to 1067426 tid 1 2024-09-18T14:11:38.326 INFO:tasks.rados.rados.0.smithi084.stdout:2746: writing smithi08434663-33 from 1830899 to 2350213 tid 2 2024-09-18T14:11:38.333 INFO:tasks.rados.rados.0.smithi084.stdout:2746: writing smithi08434663-33 from 2943557 to 3418889 tid 3 2024-09-18T14:11:38.333 INFO:tasks.rados.rados.0.smithi084.stdout:2747: read oid 8 snap -1 2024-09-18T14:11:38.333 INFO:tasks.rados.rados.0.smithi084.stdout:2747: expect deleted 2024-09-18T14:11:38.333 INFO:tasks.rados.rados.0.smithi084.stdout:2748: rollback oid 48 current snap is 296 2024-09-18T14:11:38.333 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 48 to 289 2024-09-18T14:11:38.333 INFO:tasks.rados.rados.0.smithi084.stdout:2744: finishing copy_from to smithi08434663-24 2024-09-18T14:11:38.334 INFO:tasks.rados.rados.0.smithi084.stdout:2744: got expected ENOENT (src dne) 2024-09-18T14:11:38.334 INFO:tasks.rados.rados.0.smithi084.stdout:2744: finishing copy_from racing read to smithi08434663-24 2024-09-18T14:11:38.334 INFO:tasks.rados.rados.0.smithi084.stdout:2739: expect (ObjNum 928 snap 293 seq_num 928) 2024-09-18T14:11:38.531 INFO:tasks.rados.rados.0.smithi084.stdout:2738: finishing copy_from racing read to smithi08434663-25 2024-09-18T14:11:38.531 INFO:tasks.rados.rados.0.smithi084.stdout:2738: finishing copy_from to smithi08434663-25 2024-09-18T14:11:38.531 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 998 (ObjNum 907 snap 281 seq_num 907) dirty exists 2024-09-18T14:11:38.531 INFO:tasks.rados.rados.0.smithi084.stdout:2745: finishing copy_from racing read to smithi08434663-9 2024-09-18T14:11:38.531 INFO:tasks.rados.rados.0.smithi084.stdout:2746: finishing write tid 1 to smithi08434663-33 2024-09-18T14:11:38.531 INFO:tasks.rados.rados.0.smithi084.stdout:2738: done (7 left) 2024-09-18T14:11:38.531 INFO:tasks.rados.rados.0.smithi084.stdout:2739: done (6 left) 2024-09-18T14:11:38.532 INFO:tasks.rados.rados.0.smithi084.stdout:2743: done (5 left) 2024-09-18T14:11:38.532 INFO:tasks.rados.rados.0.smithi084.stdout:2744: done (4 left) 2024-09-18T14:11:38.532 INFO:tasks.rados.rados.0.smithi084.stdout:2749: write oid 29 current snap is 296 2024-09-18T14:11:38.532 INFO:tasks.rados.rados.0.smithi084.stdout:2745: finishing copy_from to smithi08434663-9 2024-09-18T14:11:38.532 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 1013 (ObjNum 857 snap 257 seq_num 857) dirty exists 2024-09-18T14:11:38.532 INFO:tasks.rados.rados.0.smithi084.stdout:2746: finishing write tid 2 to smithi08434663-33 2024-09-18T14:11:38.532 INFO:tasks.rados.rados.0.smithi084.stdout:2749: seq_num 936 ranges {531352=744483,2005734=779325,3487333=244019} 2024-09-18T14:11:38.537 INFO:tasks.rados.rados.0.smithi084.stdout:2749: writing smithi08434663-29 from 531352 to 1275835 tid 1 2024-09-18T14:11:38.545 INFO:tasks.rados.rados.0.smithi084.stdout:2749: writing smithi08434663-29 from 2005734 to 2785059 tid 2 2024-09-18T14:11:38.551 INFO:tasks.rados.rados.0.smithi084.stdout:2749: writing smithi08434663-29 from 3487333 to 3731352 tid 3 2024-09-18T14:11:38.551 INFO:tasks.rados.rados.0.smithi084.stdout:2745: done (4 left) 2024-09-18T14:11:38.551 INFO:tasks.rados.rados.0.smithi084.stdout:2750: read oid 25 snap -1 2024-09-18T14:11:38.551 INFO:tasks.rados.rados.0.smithi084.stdout:2750: expect (ObjNum 907 snap 281 seq_num 907) 2024-09-18T14:11:38.551 INFO:tasks.rados.rados.0.smithi084.stdout:2751: snap_create 2024-09-18T14:11:38.551 INFO:tasks.rados.rados.0.smithi084.stdout:2746: finishing write tid 3 to smithi08434663-33 2024-09-18T14:11:38.551 INFO:tasks.rados.rados.0.smithi084.stdout:2748: finishing rollback tid 0 to smithi08434663-48 2024-09-18T14:11:38.551 INFO:tasks.rados.rados.0.smithi084.stdout:2746: finishing write tid 4 to smithi08434663-33 2024-09-18T14:11:38.551 INFO:tasks.rados.rados.0.smithi084.stdout:2748: finishing rollback tid 1 to smithi08434663-48 2024-09-18T14:11:38.551 INFO:tasks.rados.rados.0.smithi084.stdout:2746: finishing write tid 5 to smithi08434663-33 2024-09-18T14:11:38.551 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 1254 (ObjNum 935 snap 296 seq_num 935) dirty exists 2024-09-18T14:11:38.551 INFO:tasks.rados.rados.0.smithi084.stdout:2746: left oid 33 (ObjNum 935 snap 296 seq_num 935) 2024-09-18T14:11:38.552 INFO:tasks.rados.rados.0.smithi084.stdout:2748: finishing rollback tid 2 to smithi08434663-48 2024-09-18T14:11:38.552 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1243 (ObjNum 827 snap 251 seq_num 827) dirty exists 2024-09-18T14:11:38.552 INFO:tasks.rados.rados.0.smithi084.stdout:2749: finishing write tid 1 to smithi08434663-29 2024-09-18T14:11:38.552 INFO:tasks.rados.rados.0.smithi084.stdout:2749: finishing write tid 2 to smithi08434663-29 2024-09-18T14:11:38.553 INFO:tasks.rados.rados.0.smithi084.stdout:2749: finishing write tid 3 to smithi08434663-29 2024-09-18T14:11:38.553 INFO:tasks.rados.rados.0.smithi084.stdout:2749: finishing write tid 4 to smithi08434663-29 2024-09-18T14:11:38.553 INFO:tasks.rados.rados.0.smithi084.stdout:2749: finishing write tid 5 to smithi08434663-29 2024-09-18T14:11:38.553 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 1031 (ObjNum 936 snap 296 seq_num 936) dirty exists 2024-09-18T14:11:38.553 INFO:tasks.rados.rados.0.smithi084.stdout:2749: left oid 29 (ObjNum 936 snap 296 seq_num 936) 2024-09-18T14:11:38.555 INFO:tasks.rados.rados.0.smithi084.stdout:2750: expect (ObjNum 907 snap 281 seq_num 907) 2024-09-18T14:11:39.357 INFO:tasks.rados.rados.0.smithi084.stdout:2746: done (5 left) 2024-09-18T14:11:39.358 INFO:tasks.rados.rados.0.smithi084.stdout:2747: done (4 left) 2024-09-18T14:11:39.358 INFO:tasks.rados.rados.0.smithi084.stdout:2748: done (3 left) 2024-09-18T14:11:39.358 INFO:tasks.rados.rados.0.smithi084.stdout:2749: done (2 left) 2024-09-18T14:11:39.358 INFO:tasks.rados.rados.0.smithi084.stdout:2750: done (1 left) 2024-09-18T14:11:39.358 INFO:tasks.rados.rados.0.smithi084.stdout:2751: done (0 left) 2024-09-18T14:11:39.358 INFO:tasks.rados.rados.0.smithi084.stdout:2752: snap_create 2024-09-18T14:11:40.363 INFO:tasks.rados.rados.0.smithi084.stdout:2752: done (0 left) 2024-09-18T14:11:40.364 INFO:tasks.rados.rados.0.smithi084.stdout:2753: snap_create 2024-09-18T14:11:41.366 INFO:tasks.rados.rados.0.smithi084.stdout:2753: done (0 left) 2024-09-18T14:11:41.366 INFO:tasks.rados.rados.0.smithi084.stdout:2754: snap_remove snap 296 2024-09-18T14:11:42.378 INFO:tasks.rados.rados.0.smithi084.stdout:2754: done (0 left) 2024-09-18T14:11:42.378 INFO:tasks.rados.rados.0.smithi084.stdout:2755: write oid 18 current snap is 299 2024-09-18T14:11:42.378 INFO:tasks.rados.rados.0.smithi084.stdout:2755: seq_num 937 ranges {548159=419732,1664216=522983,2714756=614229} 2024-09-18T14:11:42.383 INFO:tasks.rados.rados.0.smithi084.stdout:2755: writing smithi08434663-18 from 548159 to 967891 tid 1 2024-09-18T14:11:42.390 INFO:tasks.rados.rados.0.smithi084.stdout:2755: writing smithi08434663-18 from 1664216 to 2187199 tid 2 2024-09-18T14:11:42.396 INFO:tasks.rados.rados.0.smithi084.stdout:2755: writing smithi08434663-18 from 2714756 to 3328985 tid 3 2024-09-18T14:11:42.396 INFO:tasks.rados.rados.0.smithi084.stdout:2756: snap_create 2024-09-18T14:11:42.396 INFO:tasks.rados.rados.0.smithi084.stdout:2755: finishing write tid 1 to smithi08434663-18 2024-09-18T14:11:42.396 INFO:tasks.rados.rados.0.smithi084.stdout:2755: finishing write tid 2 to smithi08434663-18 2024-09-18T14:11:42.400 INFO:tasks.rados.rados.0.smithi084.stdout:2755: finishing write tid 3 to smithi08434663-18 2024-09-18T14:11:42.400 INFO:tasks.rados.rados.0.smithi084.stdout:2755: finishing write tid 4 to smithi08434663-18 2024-09-18T14:11:42.400 INFO:tasks.rados.rados.0.smithi084.stdout:2755: finishing write tid 5 to smithi08434663-18 2024-09-18T14:11:42.400 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 1031 (ObjNum 937 snap 299 seq_num 937) dirty exists 2024-09-18T14:11:42.400 INFO:tasks.rados.rados.0.smithi084.stdout:2755: left oid 18 (ObjNum 937 snap 299 seq_num 937) 2024-09-18T14:11:43.381 INFO:tasks.rados.rados.0.smithi084.stdout:2755: done (1 left) 2024-09-18T14:11:43.381 INFO:tasks.rados.rados.0.smithi084.stdout:2756: done (0 left) 2024-09-18T14:11:43.382 INFO:tasks.rados.rados.0.smithi084.stdout:2757: snap_remove snap 291 2024-09-18T14:11:44.389 INFO:tasks.rados.rados.0.smithi084.stdout:2757: done (0 left) 2024-09-18T14:11:44.389 INFO:tasks.rados.rados.0.smithi084.stdout:2758: write oid 26 current snap is 300 2024-09-18T14:11:44.390 INFO:tasks.rados.rados.0.smithi084.stdout:2758: seq_num 938 ranges {564966=494981,1606345=782994,3025826=425501} 2024-09-18T14:11:44.395 INFO:tasks.rados.rados.0.smithi084.stdout:2758: writing smithi08434663-26 from 564966 to 1059947 tid 1 2024-09-18T14:11:44.402 INFO:tasks.rados.rados.0.smithi084.stdout:2758: writing smithi08434663-26 from 1606345 to 2389339 tid 2 2024-09-18T14:11:44.409 INFO:tasks.rados.rados.0.smithi084.stdout:2758: writing smithi08434663-26 from 3025826 to 3451327 tid 3 2024-09-18T14:11:44.409 INFO:tasks.rados.rados.0.smithi084.stdout:2759: delete oid 19 current snap is 300 2024-09-18T14:11:44.409 INFO:tasks.rados.rados.0.smithi084.stdout:2758: finishing write tid 1 to smithi08434663-26 2024-09-18T14:11:44.409 INFO:tasks.rados.rados.0.smithi084.stdout:2758: finishing write tid 2 to smithi08434663-26 2024-09-18T14:11:44.411 INFO:tasks.rados.rados.0.smithi084.stdout:2759: done (1 left) 2024-09-18T14:11:44.411 INFO:tasks.rados.rados.0.smithi084.stdout:2760: snap_remove snap 298 2024-09-18T14:11:45.390 INFO:tasks.rados.rados.0.smithi084.stdout:2760: done (1 left) 2024-09-18T14:11:45.390 INFO:tasks.rados.rados.0.smithi084.stdout:2761: read oid 14 snap -1 2024-09-18T14:11:45.390 INFO:tasks.rados.rados.0.smithi084.stdout:2761: expect deleted 2024-09-18T14:11:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2762: snap_create 2024-09-18T14:11:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2758: finishing write tid 3 to smithi08434663-26 2024-09-18T14:11:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2758: finishing write tid 4 to smithi08434663-26 2024-09-18T14:11:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2758: finishing write tid 5 to smithi08434663-26 2024-09-18T14:11:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 1223 (ObjNum 938 snap 300 seq_num 938) dirty exists 2024-09-18T14:11:45.391 INFO:tasks.rados.rados.0.smithi084.stdout:2758: left oid 26 (ObjNum 938 snap 300 seq_num 938) 2024-09-18T14:11:46.402 INFO:tasks.rados.rados.0.smithi084.stdout:2758: done (2 left) 2024-09-18T14:11:46.402 INFO:tasks.rados.rados.0.smithi084.stdout:2761: done (1 left) 2024-09-18T14:11:46.403 INFO:tasks.rados.rados.0.smithi084.stdout:2762: done (0 left) 2024-09-18T14:11:46.403 INFO:tasks.rados.rados.0.smithi084.stdout:2763: read oid 25 snap -1 2024-09-18T14:11:46.403 INFO:tasks.rados.rados.0.smithi084.stdout:2763: expect (ObjNum 907 snap 281 seq_num 907) 2024-09-18T14:11:46.403 INFO:tasks.rados.rados.0.smithi084.stdout:2764: read oid 6 snap -1 2024-09-18T14:11:46.403 INFO:tasks.rados.rados.0.smithi084.stdout:2764: expect (ObjNum 907 snap 281 seq_num 907) 2024-09-18T14:11:46.404 INFO:tasks.rados.rados.0.smithi084.stdout:2765: write oid 17 current snap is 301 2024-09-18T14:11:46.404 INFO:tasks.rados.rados.0.smithi084.stdout:2765: seq_num 940 ranges {598580=645479,1723309=786663,3080672=448045} 2024-09-18T14:11:46.409 INFO:tasks.rados.rados.0.smithi084.stdout:2765: writing smithi08434663-17 from 598580 to 1244059 tid 1 2024-09-18T14:11:46.416 INFO:tasks.rados.rados.0.smithi084.stdout:2765: writing smithi08434663-17 from 1723309 to 2509972 tid 2 2024-09-18T14:11:46.422 INFO:tasks.rados.rados.0.smithi084.stdout:2765: writing smithi08434663-17 from 3080672 to 3528717 tid 3 2024-09-18T14:11:46.424 INFO:tasks.rados.rados.0.smithi084.stdout:2766: read oid 40 snap -1 2024-09-18T14:11:46.424 INFO:tasks.rados.rados.0.smithi084.stdout:2766: expect (ObjNum 847 snap 255 seq_num 847) 2024-09-18T14:11:46.424 INFO:tasks.rados.rados.0.smithi084.stdout:2767: write (excl) oid 34 current snap is 301 2024-09-18T14:11:46.424 INFO:tasks.rados.rados.0.smithi084.stdout:2767: seq_num 941 ranges {615387=720728,1781791=530321,3108095=707292} 2024-09-18T14:11:46.429 INFO:tasks.rados.rados.0.smithi084.stdout:2767: writing smithi08434663-34 from 615387 to 1336115 tid 1 2024-09-18T14:11:46.434 INFO:tasks.rados.rados.0.smithi084.stdout:2767: writing smithi08434663-34 from 1781791 to 2312112 tid 2 2024-09-18T14:11:46.442 INFO:tasks.rados.rados.0.smithi084.stdout:2767: writing smithi08434663-34 from 3108095 to 3815387 tid 3 2024-09-18T14:11:46.443 INFO:tasks.rados.rados.0.smithi084.stdout:2768: write (excl) oid 2 current snap is 301 2024-09-18T14:11:46.443 INFO:tasks.rados.rados.0.smithi084.stdout:2768: seq_num 942 ranges {632194=795977,2123920=790332,3419165=413029} 2024-09-18T14:11:46.450 INFO:tasks.rados.rados.0.smithi084.stdout:2768: writing smithi08434663-2 from 632194 to 1428171 tid 1 2024-09-18T14:11:46.458 INFO:tasks.rados.rados.0.smithi084.stdout:2768: writing smithi08434663-2 from 2123920 to 2914252 tid 2 2024-09-18T14:11:46.463 INFO:tasks.rados.rados.0.smithi084.stdout:2768: writing smithi08434663-2 from 3419165 to 3832194 tid 3 2024-09-18T14:11:46.463 INFO:tasks.rados.rados.0.smithi084.stdout:2769: snap_create 2024-09-18T14:11:46.463 INFO:tasks.rados.rados.0.smithi084.stdout:2763: expect (ObjNum 907 snap 281 seq_num 907) 2024-09-18T14:11:46.612 INFO:tasks.rados.rados.0.smithi084.stdout:2764: expect (ObjNum 907 snap 281 seq_num 907) 2024-09-18T14:11:46.759 INFO:tasks.rados.rados.0.smithi084.stdout:2765: finishing write tid 1 to smithi08434663-17 2024-09-18T14:11:46.760 INFO:tasks.rados.rados.0.smithi084.stdout:2765: finishing write tid 2 to smithi08434663-17 2024-09-18T14:11:46.760 INFO:tasks.rados.rados.0.smithi084.stdout:2765: finishing write tid 3 to smithi08434663-17 2024-09-18T14:11:46.760 INFO:tasks.rados.rados.0.smithi084.stdout:2765: finishing write tid 4 to smithi08434663-17 2024-09-18T14:11:46.760 INFO:tasks.rados.rados.0.smithi084.stdout:2765: finishing write tid 5 to smithi08434663-17 2024-09-18T14:11:46.760 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1035 (ObjNum 940 snap 301 seq_num 940) dirty exists 2024-09-18T14:11:46.760 INFO:tasks.rados.rados.0.smithi084.stdout:2765: left oid 17 (ObjNum 940 snap 301 seq_num 940) 2024-09-18T14:11:46.760 INFO:tasks.rados.rados.0.smithi084.stdout:2766: expect (ObjNum 847 snap 255 seq_num 847) 2024-09-18T14:11:46.857 INFO:tasks.rados.rados.0.smithi084.stdout:2767: finishing write tid 1 to smithi08434663-34 2024-09-18T14:11:46.857 INFO:tasks.rados.rados.0.smithi084.stdout:2767: finishing write tid 2 to smithi08434663-34 2024-09-18T14:11:46.857 INFO:tasks.rados.rados.0.smithi084.stdout:2767: finishing write tid 3 to smithi08434663-34 2024-09-18T14:11:46.857 INFO:tasks.rados.rados.0.smithi084.stdout:2767: finishing write tid 4 to smithi08434663-34 2024-09-18T14:11:46.858 INFO:tasks.rados.rados.0.smithi084.stdout:2767: finishing write tid 5 to smithi08434663-34 2024-09-18T14:11:46.858 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1262 (ObjNum 941 snap 301 seq_num 941) dirty exists 2024-09-18T14:11:46.858 INFO:tasks.rados.rados.0.smithi084.stdout:2767: left oid 34 (ObjNum 941 snap 301 seq_num 941) 2024-09-18T14:11:46.858 INFO:tasks.rados.rados.0.smithi084.stdout:2768: finishing write tid 1 to smithi08434663-2 2024-09-18T14:11:46.858 INFO:tasks.rados.rados.0.smithi084.stdout:2768: finishing write tid 2 to smithi08434663-2 2024-09-18T14:11:46.858 INFO:tasks.rados.rados.0.smithi084.stdout:2768: finishing write tid 3 to smithi08434663-2 2024-09-18T14:11:46.858 INFO:tasks.rados.rados.0.smithi084.stdout:2768: finishing write tid 4 to smithi08434663-2 2024-09-18T14:11:46.858 INFO:tasks.rados.rados.0.smithi084.stdout:2768: finishing write tid 5 to smithi08434663-2 2024-09-18T14:11:46.858 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 1435 (ObjNum 942 snap 301 seq_num 942) dirty exists 2024-09-18T14:11:46.858 INFO:tasks.rados.rados.0.smithi084.stdout:2768: left oid 2 (ObjNum 942 snap 301 seq_num 942) 2024-09-18T14:11:47.295 INFO:tasks.rados.rados.0.smithi084.stdout:2763: done (6 left) 2024-09-18T14:11:47.296 INFO:tasks.rados.rados.0.smithi084.stdout:2764: done (5 left) 2024-09-18T14:11:47.296 INFO:tasks.rados.rados.0.smithi084.stdout:2765: done (4 left) 2024-09-18T14:11:47.296 INFO:tasks.rados.rados.0.smithi084.stdout:2766: done (3 left) 2024-09-18T14:11:47.296 INFO:tasks.rados.rados.0.smithi084.stdout:2767: done (2 left) 2024-09-18T14:11:47.296 INFO:tasks.rados.rados.0.smithi084.stdout:2768: done (1 left) 2024-09-18T14:11:47.296 INFO:tasks.rados.rados.0.smithi084.stdout:2769: done (0 left) 2024-09-18T14:11:47.296 INFO:tasks.rados.rados.0.smithi084.stdout:2770: delete oid 48 current snap is 302 2024-09-18T14:11:47.310 INFO:tasks.rados.rados.0.smithi084.stdout:2770: done (0 left) 2024-09-18T14:11:47.310 INFO:tasks.rados.rados.0.smithi084.stdout:2771: rollback oid 39 current snap is 302 2024-09-18T14:11:47.310 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 39 to 300 2024-09-18T14:11:47.310 INFO:tasks.rados.rados.0.smithi084.stdout:2772: write (excl) oid 13 current snap is 302 2024-09-18T14:11:47.310 INFO:tasks.rados.rados.0.smithi084.stdout:2772: seq_num 944 ranges {665808=662828,1957237=677648,3074011=609486} 2024-09-18T14:11:47.317 INFO:tasks.rados.rados.0.smithi084.stdout:2772: writing smithi08434663-13 from 665808 to 1328636 tid 1 2024-09-18T14:11:47.325 INFO:tasks.rados.rados.0.smithi084.stdout:2772: writing smithi08434663-13 from 1957237 to 2634885 tid 2 2024-09-18T14:11:47.331 INFO:tasks.rados.rados.0.smithi084.stdout:2772: writing smithi08434663-13 from 3074011 to 3683497 tid 3 2024-09-18T14:11:47.331 INFO:tasks.rados.rados.0.smithi084.stdout:2773: delete oid 50 current snap is 302 2024-09-18T14:11:47.331 INFO:tasks.rados.rados.0.smithi084.stdout:2771: finishing rollback tid 0 to smithi08434663-39 2024-09-18T14:11:47.331 INFO:tasks.rados.rados.0.smithi084.stdout:2771: finishing rollback tid 1 to smithi08434663-39 2024-09-18T14:11:47.331 INFO:tasks.rados.rados.0.smithi084.stdout:2771: finishing rollback tid 2 to smithi08434663-39 2024-09-18T14:11:47.331 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 1076 (ObjNum 807 snap 244 seq_num 807) dirty exists 2024-09-18T14:11:47.331 INFO:tasks.rados.rados.0.smithi084.stdout:2772: finishing write tid 1 to smithi08434663-13 2024-09-18T14:11:47.331 INFO:tasks.rados.rados.0.smithi084.stdout:2772: finishing write tid 2 to smithi08434663-13 2024-09-18T14:11:47.334 INFO:tasks.rados.rados.0.smithi084.stdout:2771: done (2 left) 2024-09-18T14:11:47.334 INFO:tasks.rados.rados.0.smithi084.stdout:2773: done (1 left) 2024-09-18T14:11:47.334 INFO:tasks.rados.rados.0.smithi084.stdout:2774: read oid 11 snap -1 2024-09-18T14:11:47.334 INFO:tasks.rados.rados.0.smithi084.stdout:2774: expect (ObjNum 810 snap 244 seq_num 810) 2024-09-18T14:11:47.334 INFO:tasks.rados.rados.0.smithi084.stdout:2775: copy_from oid 4 from oid 39 current snap is 302 2024-09-18T14:11:47.334 INFO:tasks.rados.rados.0.smithi084.stdout:2776: read oid 24 snap -1 2024-09-18T14:11:47.334 INFO:tasks.rados.rados.0.smithi084.stdout:2776: expect (ObjNum 924 snap 289 seq_num 924) 2024-09-18T14:11:47.334 INFO:tasks.rados.rados.0.smithi084.stdout:2777: rollback oid 16 current snap is 302 2024-09-18T14:11:47.334 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 16 to 284 2024-09-18T14:11:47.334 INFO:tasks.rados.rados.0.smithi084.stdout:2778: write (excl) oid 35 current snap is 302 2024-09-18T14:11:47.334 INFO:tasks.rados.rados.0.smithi084.stdout:2778: seq_num 946 ranges {699422=413326,1557848=681317,3012504=748383} 2024-09-18T14:11:47.340 INFO:tasks.rados.rados.0.smithi084.stdout:2778: writing smithi08434663-35 from 699422 to 1112748 tid 1 2024-09-18T14:11:47.347 INFO:tasks.rados.rados.0.smithi084.stdout:2778: writing smithi08434663-35 from 1557848 to 2239165 tid 2 2024-09-18T14:11:47.356 INFO:tasks.rados.rados.0.smithi084.stdout:2778: writing smithi08434663-35 from 3012504 to 3760887 tid 3 2024-09-18T14:11:47.356 INFO:tasks.rados.rados.0.smithi084.stdout:2779: snap_create 2024-09-18T14:11:47.356 INFO:tasks.rados.rados.0.smithi084.stdout:2772: finishing write tid 3 to smithi08434663-13 2024-09-18T14:11:47.356 INFO:tasks.rados.rados.0.smithi084.stdout:2772: finishing write tid 4 to smithi08434663-13 2024-09-18T14:11:47.356 INFO:tasks.rados.rados.0.smithi084.stdout:2772: finishing write tid 5 to smithi08434663-13 2024-09-18T14:11:47.356 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 1039 (ObjNum 944 snap 302 seq_num 944) dirty exists 2024-09-18T14:11:47.356 INFO:tasks.rados.rados.0.smithi084.stdout:2772: left oid 13 (ObjNum 944 snap 302 seq_num 944) 2024-09-18T14:11:47.357 INFO:tasks.rados.rados.0.smithi084.stdout:2774: expect (ObjNum 810 snap 244 seq_num 810) 2024-09-18T14:11:47.577 INFO:tasks.rados.rados.0.smithi084.stdout:2777: finishing rollback tid 0 to smithi08434663-16 2024-09-18T14:11:47.577 INFO:tasks.rados.rados.0.smithi084.stdout:2777: finishing rollback tid 1 to smithi08434663-16 2024-09-18T14:11:47.578 INFO:tasks.rados.rados.0.smithi084.stdout:2777: finishing rollback tid 2 to smithi08434663-16 2024-09-18T14:11:47.578 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1255 (ObjNum 697 snap 205 seq_num 697) dirty exists 2024-09-18T14:11:47.578 INFO:tasks.rados.rados.0.smithi084.stdout:2778: finishing write tid 1 to smithi08434663-35 2024-09-18T14:11:47.578 INFO:tasks.rados.rados.0.smithi084.stdout:2775: finishing copy_from racing read to smithi08434663-4 2024-09-18T14:11:47.578 INFO:tasks.rados.rados.0.smithi084.stdout:2776: expect (ObjNum 924 snap 289 seq_num 924) 2024-09-18T14:11:47.691 INFO:tasks.rados.rados.0.smithi084.stdout:2775: finishing copy_from to smithi08434663-4 2024-09-18T14:11:47.691 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1076 (ObjNum 807 snap 244 seq_num 807) dirty exists 2024-09-18T14:11:47.692 INFO:tasks.rados.rados.0.smithi084.stdout:2778: finishing write tid 2 to smithi08434663-35 2024-09-18T14:11:47.692 INFO:tasks.rados.rados.0.smithi084.stdout:2778: finishing write tid 3 to smithi08434663-35 2024-09-18T14:11:47.692 INFO:tasks.rados.rados.0.smithi084.stdout:2778: finishing write tid 4 to smithi08434663-35 2024-09-18T14:11:47.692 INFO:tasks.rados.rados.0.smithi084.stdout:2778: finishing write tid 5 to smithi08434663-35 2024-09-18T14:11:47.692 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 1259 (ObjNum 946 snap 302 seq_num 946) dirty exists 2024-09-18T14:11:47.692 INFO:tasks.rados.rados.0.smithi084.stdout:2778: left oid 35 (ObjNum 946 snap 302 seq_num 946) 2024-09-18T14:11:48.301 INFO:tasks.rados.rados.0.smithi084.stdout:2772: done (6 left) 2024-09-18T14:11:48.302 INFO:tasks.rados.rados.0.smithi084.stdout:2774: done (5 left) 2024-09-18T14:11:48.302 INFO:tasks.rados.rados.0.smithi084.stdout:2775: done (4 left) 2024-09-18T14:11:48.302 INFO:tasks.rados.rados.0.smithi084.stdout:2776: done (3 left) 2024-09-18T14:11:48.302 INFO:tasks.rados.rados.0.smithi084.stdout:2777: done (2 left) 2024-09-18T14:11:48.302 INFO:tasks.rados.rados.0.smithi084.stdout:2778: done (1 left) 2024-09-18T14:11:48.302 INFO:tasks.rados.rados.0.smithi084.stdout:2779: done (0 left) 2024-09-18T14:11:48.302 INFO:tasks.rados.rados.0.smithi084.stdout:2780: copy_from oid 25 from oid 1 current snap is 303 2024-09-18T14:11:48.302 INFO:tasks.rados.rados.0.smithi084.stdout:2781: delete oid 3 current snap is 303 2024-09-18T14:11:48.304 INFO:tasks.rados.rados.0.smithi084.stdout:2781: done (1 left) 2024-09-18T14:11:48.305 INFO:tasks.rados.rados.0.smithi084.stdout:2782: delete oid 13 current snap is 303 2024-09-18T14:11:48.305 INFO:tasks.rados.rados.0.smithi084.stdout:2780: finishing copy_from to smithi08434663-25 2024-09-18T14:11:48.305 INFO:tasks.rados.rados.0.smithi084.stdout:2780: got expected ENOENT (src dne) 2024-09-18T14:11:48.305 INFO:tasks.rados.rados.0.smithi084.stdout:2780: finishing copy_from racing read to smithi08434663-25 2024-09-18T14:11:48.308 INFO:tasks.rados.rados.0.smithi084.stdout:2780: done (1 left) 2024-09-18T14:11:48.308 INFO:tasks.rados.rados.0.smithi084.stdout:2782: done (0 left) 2024-09-18T14:11:48.308 INFO:tasks.rados.rados.0.smithi084.stdout:2783: write (excl) oid 48 current snap is 303 2024-09-18T14:11:48.308 INFO:tasks.rados.rados.0.smithi084.stdout:2783: seq_num 949 ranges {749843=639073,2133294=544997,3211126=582199} 2024-09-18T14:11:48.315 INFO:tasks.rados.rados.0.smithi084.stdout:2783: writing smithi08434663-48 from 749843 to 1388916 tid 1 2024-09-18T14:11:48.322 INFO:tasks.rados.rados.0.smithi084.stdout:2783: writing smithi08434663-48 from 2133294 to 2678291 tid 2 2024-09-18T14:11:48.328 INFO:tasks.rados.rados.0.smithi084.stdout:2783: writing smithi08434663-48 from 3211126 to 3793325 tid 3 2024-09-18T14:11:48.329 INFO:tasks.rados.rados.0.smithi084.stdout:2784: write (excl) oid 12 current snap is 303 2024-09-18T14:11:48.329 INFO:tasks.rados.rados.0.smithi084.stdout:2784: seq_num 950 ranges {766650=714322,2075423=688655,3522196=444454} 2024-09-18T14:11:48.336 INFO:tasks.rados.rados.0.smithi084.stdout:2784: writing smithi08434663-12 from 766650 to 1480972 tid 1 2024-09-18T14:11:48.343 INFO:tasks.rados.rados.0.smithi084.stdout:2784: writing smithi08434663-12 from 2075423 to 2764078 tid 2 2024-09-18T14:11:48.350 INFO:tasks.rados.rados.0.smithi084.stdout:2784: writing smithi08434663-12 from 3522196 to 3966650 tid 3 2024-09-18T14:11:48.351 INFO:tasks.rados.rados.0.smithi084.stdout:2785: rollback oid 32 current snap is 303 2024-09-18T14:11:48.351 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 32 to 284 2024-09-18T14:11:48.351 INFO:tasks.rados.rados.0.smithi084.stdout:2783: finishing write tid 1 to smithi08434663-48 2024-09-18T14:11:48.351 INFO:tasks.rados.rados.0.smithi084.stdout:2783: finishing write tid 2 to smithi08434663-48 2024-09-18T14:11:48.351 INFO:tasks.rados.rados.0.smithi084.stdout:2783: finishing write tid 3 to smithi08434663-48 2024-09-18T14:11:48.351 INFO:tasks.rados.rados.0.smithi084.stdout:2783: finishing write tid 4 to smithi08434663-48 2024-09-18T14:11:48.351 INFO:tasks.rados.rados.0.smithi084.stdout:2783: finishing write tid 5 to smithi08434663-48 2024-09-18T14:11:48.351 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1248 (ObjNum 949 snap 303 seq_num 949) dirty exists 2024-09-18T14:11:48.351 INFO:tasks.rados.rados.0.smithi084.stdout:2783: left oid 48 (ObjNum 949 snap 303 seq_num 949) 2024-09-18T14:11:48.351 INFO:tasks.rados.rados.0.smithi084.stdout:2784: finishing write tid 1 to smithi08434663-12 2024-09-18T14:11:48.351 INFO:tasks.rados.rados.0.smithi084.stdout:2784: finishing write tid 2 to smithi08434663-12 2024-09-18T14:11:48.351 INFO:tasks.rados.rados.0.smithi084.stdout:2783: done (2 left) 2024-09-18T14:11:48.352 INFO:tasks.rados.rados.0.smithi084.stdout:2786: snap_remove snap 283 2024-09-18T14:11:49.304 INFO:tasks.rados.rados.0.smithi084.stdout:2786: done (2 left) 2024-09-18T14:11:49.305 INFO:tasks.rados.rados.0.smithi084.stdout:2787: write oid 30 current snap is 303 2024-09-18T14:11:49.305 INFO:tasks.rados.rados.0.smithi084.stdout:2787: seq_num 951 ranges {783457=505924,1850258=548666,2865972=721096} 2024-09-18T14:11:49.312 INFO:tasks.rados.rados.0.smithi084.stdout:2787: writing smithi08434663-30 from 783457 to 1289381 tid 1 2024-09-18T14:11:49.318 INFO:tasks.rados.rados.0.smithi084.stdout:2787: writing smithi08434663-30 from 1850258 to 2398924 tid 2 2024-09-18T14:11:49.324 INFO:tasks.rados.rados.0.smithi084.stdout:2787: writing smithi08434663-30 from 2865972 to 3587068 tid 3 2024-09-18T14:11:49.325 INFO:tasks.rados.rados.0.smithi084.stdout:2788: snap_remove snap 294 2024-09-18T14:11:50.315 INFO:tasks.rados.rados.0.smithi084.stdout:2788: done (3 left) 2024-09-18T14:11:50.315 INFO:tasks.rados.rados.0.smithi084.stdout:2789: write oid 3 current snap is 303 2024-09-18T14:11:50.315 INFO:tasks.rados.rados.0.smithi084.stdout:2789: seq_num 952 ranges {400264=581173,1508740=491524} 2024-09-18T14:11:50.321 INFO:tasks.rados.rados.0.smithi084.stdout:2789: writing smithi08434663-3 from 400264 to 981437 tid 1 2024-09-18T14:11:50.328 INFO:tasks.rados.rados.0.smithi084.stdout:2789: writing smithi08434663-3 from 1508740 to 2000264 tid 2 2024-09-18T14:11:50.328 INFO:tasks.rados.rados.0.smithi084.stdout:2790: rollback oid 16 current snap is 303 2024-09-18T14:11:50.328 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 16 to 299 2024-09-18T14:11:50.328 INFO:tasks.rados.rados.0.smithi084.stdout:2784: finishing write tid 3 to smithi08434663-12 2024-09-18T14:11:50.328 INFO:tasks.rados.rados.0.smithi084.stdout:2784: finishing write tid 4 to smithi08434663-12 2024-09-18T14:11:50.328 INFO:tasks.rados.rados.0.smithi084.stdout:2784: finishing write tid 5 to smithi08434663-12 2024-09-18T14:11:50.329 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 1442 (ObjNum 950 snap 303 seq_num 950) dirty exists 2024-09-18T14:11:50.329 INFO:tasks.rados.rados.0.smithi084.stdout:2784: left oid 12 (ObjNum 950 snap 303 seq_num 950) 2024-09-18T14:11:50.329 INFO:tasks.rados.rados.0.smithi084.stdout:2785: finishing rollback tid 0 to smithi08434663-32 2024-09-18T14:11:50.329 INFO:tasks.rados.rados.0.smithi084.stdout:2785: finishing rollback tid 1 to smithi08434663-32 2024-09-18T14:11:50.329 INFO:tasks.rados.rados.0.smithi084.stdout:2785: finishing rollback tid 2 to smithi08434663-32 2024-09-18T14:11:50.329 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 1446 (ObjNum 901 snap 281 seq_num 901) dirty exists 2024-09-18T14:11:50.329 INFO:tasks.rados.rados.0.smithi084.stdout:2787: finishing write tid 1 to smithi08434663-30 2024-09-18T14:11:50.329 INFO:tasks.rados.rados.0.smithi084.stdout:2787: finishing write tid 2 to smithi08434663-30 2024-09-18T14:11:50.330 INFO:tasks.rados.rados.0.smithi084.stdout:2787: finishing write tid 3 to smithi08434663-30 2024-09-18T14:11:50.330 INFO:tasks.rados.rados.0.smithi084.stdout:2787: finishing write tid 4 to smithi08434663-30 2024-09-18T14:11:50.330 INFO:tasks.rados.rados.0.smithi084.stdout:2787: finishing write tid 5 to smithi08434663-30 2024-09-18T14:11:50.330 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 860 (ObjNum 951 snap 303 seq_num 951) dirty exists 2024-09-18T14:11:50.330 INFO:tasks.rados.rados.0.smithi084.stdout:2787: left oid 30 (ObjNum 951 snap 303 seq_num 951) 2024-09-18T14:11:50.330 INFO:tasks.rados.rados.0.smithi084.stdout:2784: done (4 left) 2024-09-18T14:11:50.330 INFO:tasks.rados.rados.0.smithi084.stdout:2785: done (3 left) 2024-09-18T14:11:50.330 INFO:tasks.rados.rados.0.smithi084.stdout:2787: done (2 left) 2024-09-18T14:11:50.330 INFO:tasks.rados.rados.0.smithi084.stdout:2791: snap_create 2024-09-18T14:11:50.331 INFO:tasks.rados.rados.0.smithi084.stdout:2789: finishing write tid 1 to smithi08434663-3 2024-09-18T14:11:50.331 INFO:tasks.rados.rados.0.smithi084.stdout:2789: finishing write tid 2 to smithi08434663-3 2024-09-18T14:11:50.331 INFO:tasks.rados.rados.0.smithi084.stdout:2789: finishing write tid 3 to smithi08434663-3 2024-09-18T14:11:50.331 INFO:tasks.rados.rados.0.smithi084.stdout:2789: finishing write tid 4 to smithi08434663-3 2024-09-18T14:11:50.331 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 1034 (ObjNum 952 snap 303 seq_num 952) dirty exists 2024-09-18T14:11:50.332 INFO:tasks.rados.rados.0.smithi084.stdout:2789: left oid 3 (ObjNum 952 snap 303 seq_num 952) 2024-09-18T14:11:50.332 INFO:tasks.rados.rados.0.smithi084.stdout:2790: finishing rollback tid 0 to smithi08434663-16 2024-09-18T14:11:50.332 INFO:tasks.rados.rados.0.smithi084.stdout:2790: finishing rollback tid 1 to smithi08434663-16 2024-09-18T14:11:50.333 INFO:tasks.rados.rados.0.smithi084.stdout:2790: finishing rollback tid 2 to smithi08434663-16 2024-09-18T14:11:50.333 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1262 (ObjNum 915 snap 286 seq_num 915) dirty exists 2024-09-18T14:11:51.313 INFO:tasks.rados.rados.0.smithi084.stdout:2789: done (2 left) 2024-09-18T14:11:51.313 INFO:tasks.rados.rados.0.smithi084.stdout:2790: done (1 left) 2024-09-18T14:11:51.313 INFO:tasks.rados.rados.0.smithi084.stdout:2791: done (0 left) 2024-09-18T14:11:51.313 INFO:tasks.rados.rados.0.smithi084.stdout:2792: snap_remove snap 293 2024-09-18T14:11:52.330 INFO:tasks.rados.rados.0.smithi084.stdout:2792: done (0 left) 2024-09-18T14:11:52.330 INFO:tasks.rados.rados.0.smithi084.stdout:2793: write oid 36 current snap is 304 2024-09-18T14:11:52.331 INFO:tasks.rados.rados.0.smithi084.stdout:2793: seq_num 953 ranges {417071=656422,1567222=449849} 2024-09-18T14:11:52.334 INFO:tasks.rados.rados.0.smithi084.stdout:2793: writing smithi08434663-36 from 417071 to 1073493 tid 1 2024-09-18T14:11:52.340 INFO:tasks.rados.rados.0.smithi084.stdout:2793: writing smithi08434663-36 from 1567222 to 2017071 tid 2 2024-09-18T14:11:52.340 INFO:tasks.rados.rados.0.smithi084.stdout:2794: snap_remove snap 263 2024-09-18T14:11:53.334 INFO:tasks.rados.rados.0.smithi084.stdout:2794: done (1 left) 2024-09-18T14:11:53.334 INFO:tasks.rados.rados.0.smithi084.stdout:2795: write (excl) oid 38 current snap is 304 2024-09-18T14:11:53.335 INFO:tasks.rados.rados.0.smithi084.stdout:2793: finishing write tid 1 to smithi08434663-36 2024-09-18T14:11:53.335 INFO:tasks.rados.rados.0.smithi084.stdout:2795: seq_num 954 ranges {433878=731671,1909351=124527} 2024-09-18T14:11:53.341 INFO:tasks.rados.rados.0.smithi084.stdout:2795: writing smithi08434663-38 from 433878 to 1165549 tid 1 2024-09-18T14:11:53.347 INFO:tasks.rados.rados.0.smithi084.stdout:2795: writing smithi08434663-38 from 1909351 to 2033878 tid 2 2024-09-18T14:11:53.347 INFO:tasks.rados.rados.0.smithi084.stdout:2796: read oid 23 snap -1 2024-09-18T14:11:53.347 INFO:tasks.rados.rados.0.smithi084.stdout:2796: expect deleted 2024-09-18T14:11:53.347 INFO:tasks.rados.rados.0.smithi084.stdout:2797: copy_from oid 17 from oid 14 current snap is 304 2024-09-18T14:11:53.347 INFO:tasks.rados.rados.0.smithi084.stdout:2793: finishing write tid 2 to smithi08434663-36 2024-09-18T14:11:53.347 INFO:tasks.rados.rados.0.smithi084.stdout:2793: finishing write tid 3 to smithi08434663-36 2024-09-18T14:11:53.347 INFO:tasks.rados.rados.0.smithi084.stdout:2793: finishing write tid 4 to smithi08434663-36 2024-09-18T14:11:53.348 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 1257 (ObjNum 953 snap 304 seq_num 953) dirty exists 2024-09-18T14:11:53.348 INFO:tasks.rados.rados.0.smithi084.stdout:2793: left oid 36 (ObjNum 953 snap 304 seq_num 953) 2024-09-18T14:11:53.348 INFO:tasks.rados.rados.0.smithi084.stdout:2793: done (3 left) 2024-09-18T14:11:53.348 INFO:tasks.rados.rados.0.smithi084.stdout:2798: write oid 25 current snap is 304 2024-09-18T14:11:53.348 INFO:tasks.rados.rados.0.smithi084.stdout:2798: seq_num 955 ranges {450685=406920,1567833=482852} 2024-09-18T14:11:53.352 INFO:tasks.rados.rados.0.smithi084.stdout:2798: writing smithi08434663-25 from 450685 to 857605 tid 1 2024-09-18T14:11:53.359 INFO:tasks.rados.rados.0.smithi084.stdout:2798: writing smithi08434663-25 from 1567833 to 2050685 tid 2 2024-09-18T14:11:53.359 INFO:tasks.rados.rados.0.smithi084.stdout:2799: copy_from oid 15 from oid 3 current snap is 304 2024-09-18T14:11:53.359 INFO:tasks.rados.rados.0.smithi084.stdout:2795: finishing write tid 1 to smithi08434663-38 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:2795: finishing write tid 2 to smithi08434663-38 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:2795: finishing write tid 3 to smithi08434663-38 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:2795: finishing write tid 4 to smithi08434663-38 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 651 (ObjNum 954 snap 304 seq_num 954) dirty exists 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:2795: left oid 38 (ObjNum 954 snap 304 seq_num 954) 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:2797: finishing copy_from to smithi08434663-17 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:2797: got expected ENOENT (src dne) 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:2797: finishing copy_from racing read to smithi08434663-17 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:2798: finishing write tid 1 to smithi08434663-25 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:2795: done (4 left) 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:2796: done (3 left) 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:2797: done (2 left) 2024-09-18T14:11:53.360 INFO:tasks.rados.rados.0.smithi084.stdout:2800: snap_remove snap 284 2024-09-18T14:11:54.341 INFO:tasks.rados.rados.0.smithi084.stdout:2800: done (2 left) 2024-09-18T14:11:54.341 INFO:tasks.rados.rados.0.smithi084.stdout:2801: delete oid 45 current snap is 304 2024-09-18T14:11:54.341 INFO:tasks.rados.rados.0.smithi084.stdout:2798: finishing write tid 2 to smithi08434663-25 2024-09-18T14:11:54.341 INFO:tasks.rados.rados.0.smithi084.stdout:2798: finishing write tid 3 to smithi08434663-25 2024-09-18T14:11:54.341 INFO:tasks.rados.rados.0.smithi084.stdout:2798: finishing write tid 4 to smithi08434663-25 2024-09-18T14:11:54.341 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 1251 (ObjNum 955 snap 304 seq_num 955) dirty exists 2024-09-18T14:11:54.341 INFO:tasks.rados.rados.0.smithi084.stdout:2798: left oid 25 (ObjNum 955 snap 304 seq_num 955) 2024-09-18T14:11:54.342 INFO:tasks.rados.rados.0.smithi084.stdout:2799: finishing copy_from racing read to smithi08434663-15 2024-09-18T14:11:54.342 INFO:tasks.rados.rados.0.smithi084.stdout:2799: finishing copy_from to smithi08434663-15 2024-09-18T14:11:54.342 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 1034 (ObjNum 952 snap 303 seq_num 952) dirty exists 2024-09-18T14:11:54.351 INFO:tasks.rados.rados.0.smithi084.stdout:2798: done (2 left) 2024-09-18T14:11:54.351 INFO:tasks.rados.rados.0.smithi084.stdout:2799: done (1 left) 2024-09-18T14:11:54.352 INFO:tasks.rados.rados.0.smithi084.stdout:2801: done (0 left) 2024-09-18T14:11:54.352 INFO:tasks.rados.rados.0.smithi084.stdout:2802: write (excl) oid 9 current snap is 304 2024-09-18T14:11:54.352 INFO:tasks.rados.rados.0.smithi084.stdout:2802: seq_num 957 ranges {484299=557418,1684797=399502} 2024-09-18T14:11:54.357 INFO:tasks.rados.rados.0.smithi084.stdout:2802: writing smithi08434663-9 from 484299 to 1041717 tid 1 2024-09-18T14:11:54.363 INFO:tasks.rados.rados.0.smithi084.stdout:2802: writing smithi08434663-9 from 1684797 to 2084299 tid 2 2024-09-18T14:11:54.363 INFO:tasks.rados.rados.0.smithi084.stdout:2803: snap_create 2024-09-18T14:11:54.363 INFO:tasks.rados.rados.0.smithi084.stdout:2802: finishing write tid 1 to smithi08434663-9 2024-09-18T14:11:54.366 INFO:tasks.rados.rados.0.smithi084.stdout:2802: finishing write tid 2 to smithi08434663-9 2024-09-18T14:11:54.366 INFO:tasks.rados.rados.0.smithi084.stdout:2802: finishing write tid 3 to smithi08434663-9 2024-09-18T14:11:54.367 INFO:tasks.rados.rados.0.smithi084.stdout:2802: finishing write tid 4 to smithi08434663-9 2024-09-18T14:11:54.367 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 1265 (ObjNum 957 snap 304 seq_num 957) dirty exists 2024-09-18T14:11:54.367 INFO:tasks.rados.rados.0.smithi084.stdout:2802: left oid 9 (ObjNum 957 snap 304 seq_num 957) 2024-09-18T14:11:55.342 INFO:tasks.rados.rados.0.smithi084.stdout:2802: done (1 left) 2024-09-18T14:11:55.342 INFO:tasks.rados.rados.0.smithi084.stdout:2803: done (0 left) 2024-09-18T14:11:55.342 INFO:tasks.rados.rados.0.smithi084.stdout:2804: copy_from oid 11 from oid 35 current snap is 305 2024-09-18T14:11:55.342 INFO:tasks.rados.rados.0.smithi084.stdout:2805: snap_create 2024-09-18T14:11:55.358 INFO:tasks.rados.rados.0.smithi084.stdout:2804: finishing copy_from racing read to smithi08434663-11 2024-09-18T14:11:55.370 INFO:tasks.rados.rados.0.smithi084.stdout:2804: finishing copy_from to smithi08434663-11 2024-09-18T14:11:55.370 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1259 (ObjNum 946 snap 302 seq_num 946) dirty exists 2024-09-18T14:11:56.352 INFO:tasks.rados.rados.0.smithi084.stdout:2804: done (1 left) 2024-09-18T14:11:56.353 INFO:tasks.rados.rados.0.smithi084.stdout:2805: done (0 left) 2024-09-18T14:11:56.353 INFO:tasks.rados.rados.0.smithi084.stdout:2806: copy_from oid 44 from oid 7 current snap is 306 2024-09-18T14:11:56.353 INFO:tasks.rados.rados.0.smithi084.stdout:2807: snap_remove snap 302 2024-09-18T14:11:57.299 INFO:tasks.rados.rados.0.smithi084.stdout:2806: finishing copy_from to smithi08434663-44 2024-09-18T14:11:57.299 INFO:tasks.rados.rados.0.smithi084.stdout:2806: got expected ENOENT (src dne) 2024-09-18T14:11:57.299 INFO:tasks.rados.rados.0.smithi084.stdout:2806: finishing copy_from racing read to smithi08434663-44 2024-09-18T14:11:57.299 INFO:tasks.rados.rados.0.smithi084.stdout:2806: done (1 left) 2024-09-18T14:11:57.299 INFO:tasks.rados.rados.0.smithi084.stdout:2807: done (0 left) 2024-09-18T14:11:57.299 INFO:tasks.rados.rados.0.smithi084.stdout:2808: rollback oid 30 current snap is 306 2024-09-18T14:11:57.299 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 30 to 292 2024-09-18T14:11:57.299 INFO:tasks.rados.rados.0.smithi084.stdout:2809: write oid 48 current snap is 306 2024-09-18T14:11:57.300 INFO:tasks.rados.rados.0.smithi084.stdout:2809: seq_num 958 ranges {501106=749020,1743279=357827} 2024-09-18T14:11:57.308 INFO:tasks.rados.rados.0.smithi084.stdout:2809: writing smithi08434663-48 from 501106 to 1250126 tid 1 2024-09-18T14:11:57.315 INFO:tasks.rados.rados.0.smithi084.stdout:2809: writing smithi08434663-48 from 1743279 to 2101106 tid 2 2024-09-18T14:11:57.315 INFO:tasks.rados.rados.0.smithi084.stdout:2810: copy_from oid 1 from oid 5 current snap is 306 2024-09-18T14:11:57.315 INFO:tasks.rados.rados.0.smithi084.stdout:2808: finishing rollback tid 0 to smithi08434663-30 2024-09-18T14:11:57.315 INFO:tasks.rados.rados.0.smithi084.stdout:2808: finishing rollback tid 1 to smithi08434663-30 2024-09-18T14:11:57.315 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 862 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:11:57.315 INFO:tasks.rados.rados.0.smithi084.stdout:2808: done (2 left) 2024-09-18T14:11:57.315 INFO:tasks.rados.rados.0.smithi084.stdout:2811: read oid 9 snap -1 2024-09-18T14:11:57.315 INFO:tasks.rados.rados.0.smithi084.stdout:2811: expect (ObjNum 957 snap 304 seq_num 957) 2024-09-18T14:11:57.315 INFO:tasks.rados.rados.0.smithi084.stdout:2812: copy_from oid 38 from oid 37 current snap is 306 2024-09-18T14:11:57.315 INFO:tasks.rados.rados.0.smithi084.stdout:2813: write (excl) oid 24 current snap is 306 2024-09-18T14:11:57.315 INFO:tasks.rados.rados.0.smithi084.stdout:2813: seq_num 959 ranges {517913=424269,1401761=446989} 2024-09-18T14:11:57.320 INFO:tasks.rados.rados.0.smithi084.stdout:2813: writing smithi08434663-24 from 517913 to 942182 tid 1 2024-09-18T14:11:57.325 INFO:tasks.rados.rados.0.smithi084.stdout:2813: writing smithi08434663-24 from 1401761 to 1848750 tid 2 2024-09-18T14:11:57.325 INFO:tasks.rados.rados.0.smithi084.stdout:2814: snap_remove snap 304 2024-09-18T14:11:58.309 INFO:tasks.rados.rados.0.smithi084.stdout:2810: finishing copy_from to smithi08434663-1 2024-09-18T14:11:58.309 INFO:tasks.rados.rados.0.smithi084.stdout:2810: got expected ENOENT (src dne) 2024-09-18T14:11:58.309 INFO:tasks.rados.rados.0.smithi084.stdout:2810: finishing copy_from racing read to smithi08434663-1 2024-09-18T14:11:58.309 INFO:tasks.rados.rados.0.smithi084.stdout:2809: finishing write tid 1 to smithi08434663-48 2024-09-18T14:11:58.309 INFO:tasks.rados.rados.0.smithi084.stdout:2812: finishing copy_from racing read to smithi08434663-38 2024-09-18T14:11:58.309 INFO:tasks.rados.rados.0.smithi084.stdout:2812: finishing copy_from to smithi08434663-38 2024-09-18T14:11:58.310 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 1013 (ObjNum 857 snap 257 seq_num 857) dirty exists 2024-09-18T14:11:58.310 INFO:tasks.rados.rados.0.smithi084.stdout:2811: expect (ObjNum 957 snap 304 seq_num 957) 2024-09-18T14:11:58.395 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-18T14:11:58.396 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-18T14:11:58.396 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-18T14:11:58.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-18T14:11:58.401 INFO:tasks.ceph.mon.f:Started 2024-09-18T14:11:58.401 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-09-18T14:11:58.401 INFO:tasks.ceph.mon.a:Sent signal 19 2024-09-18T14:11:58.401 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-18T14:11:58.402 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-18T14:11:58.402 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-18T14:11:58.440 INFO:tasks.rados.rados.0.smithi084.stdout:2809: finishing write tid 2 to smithi08434663-48 2024-09-18T14:11:58.440 INFO:tasks.rados.rados.0.smithi084.stdout:2809: finishing write tid 3 to smithi08434663-48 2024-09-18T14:11:58.440 INFO:tasks.rados.rados.0.smithi084.stdout:2809: finishing write tid 4 to smithi08434663-48 2024-09-18T14:11:58.440 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1254 (ObjNum 958 snap 306 seq_num 958) dirty exists 2024-09-18T14:11:58.440 INFO:tasks.rados.rados.0.smithi084.stdout:2809: left oid 48 (ObjNum 958 snap 306 seq_num 958) 2024-09-18T14:11:58.441 INFO:tasks.rados.rados.0.smithi084.stdout:2813: finishing write tid 1 to smithi08434663-24 2024-09-18T14:11:58.441 INFO:tasks.rados.rados.0.smithi084.stdout:2813: finishing write tid 2 to smithi08434663-24 2024-09-18T14:11:58.441 INFO:tasks.rados.rados.0.smithi084.stdout:2813: finishing write tid 3 to smithi08434663-24 2024-09-18T14:11:58.441 INFO:tasks.rados.rados.0.smithi084.stdout:2809: done (5 left) 2024-09-18T14:11:58.441 INFO:tasks.rados.rados.0.smithi084.stdout:2810: done (4 left) 2024-09-18T14:11:58.441 INFO:tasks.rados.rados.0.smithi084.stdout:2811: done (3 left) 2024-09-18T14:11:58.441 INFO:tasks.rados.rados.0.smithi084.stdout:2812: done (2 left) 2024-09-18T14:11:58.441 INFO:tasks.rados.rados.0.smithi084.stdout:2814: done (1 left) 2024-09-18T14:11:58.441 INFO:tasks.rados.rados.0.smithi084.stdout:2815: snap_remove snap 289 2024-09-18T14:12:13.402 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-09-18T14:12:13.403 INFO:tasks.ceph.mon.a:Sent signal 18 2024-09-18T14:12:13.404 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-18T14:12:13.404 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-18T14:12:13.404 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:12:13.404 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:12:13.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.521+0000 7fd6614c9640 10 monclient: get_monmap_and_config 2024-09-18T14:12:13.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.521+0000 7fd6614c9640 10 monclient: build_initial_monmap 2024-09-18T14:12:13.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.521+0000 7fd6614c9640 10 monclient: monmap: 2024-09-18T14:12:13.522 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:13.522 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:13.523018+0000 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:13.523018+0000 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:13.524 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.521+0000 7fd6614c9640 1 Processor -- start 2024-09-18T14:12:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.521+0000 7fd6614c9640 1 -- start start 2024-09-18T14:12:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.521+0000 7fd6614c9640 10 monclient: init 2024-09-18T14:12:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd6614c9640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd6614c9640 10 monclient: _add_conns ranks=[2,1,7,5,6,8,3,0,4] 2024-09-18T14:12:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd6614c9640 10 monclient(hunting): picked mon.noname-b con 0x7fd65c06e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd6614c9640 10 monclient(hunting): picked mon.noname-f con 0x7fd65c0a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd6614c9640 10 monclient(hunting): picked mon.noname-i con 0x7fd65c0a1080 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd6614c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c05ed70 con 0x7fd65c0a0cb0 2024-09-18T14:12:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd6614c9640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c097620 con 0x7fd65c06e810 2024-09-18T14:12:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd6614c9640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c097340 con 0x7fd65c0a1080 2024-09-18T14:12:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd6614c9640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd6614c9640 10 monclient(hunting): authenticate will time out at 1548.861421s 2024-09-18T14:12:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65affd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fd65c06e810 0x7fd65c0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55762/0 (socket says 172.21.15.45:55762) 2024-09-18T14:12:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65affd640 1 -- 172.21.15.45:0/2630802221 learned_addr learned my addr 172.21.15.45:0/2630802221 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324955150 0 0) 0x7fd65c097620 con 0x7fd65c06e810 2024-09-18T14:12:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 10 monclient(hunting): my global_id is 145055 2024-09-18T14:12:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6480034e0 con 0x7fd65c06e810 2024-09-18T14:12:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766061934 0 0) 0x7fd65c097340 con 0x7fd65c0a1080 2024-09-18T14:12:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 10 monclient(hunting): my global_id is 184354 2024-09-18T14:12:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd65c097620 con 0x7fd65c0a1080 2024-09-18T14:12:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 415489254 0 0) 0x7fd65c05ed70 con 0x7fd65c0a0cb0 2024-09-18T14:12:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 10 monclient(hunting): my global_id is 204103 2024-09-18T14:12:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.522+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd65c097340 con 0x7fd65c0a0cb0 2024-09-18T14:12:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2245047920 0 0) 0x7fd6480034e0 con 0x7fd65c06e810 2024-09-18T14:12:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 >> v1:172.21.15.84:6792/0 conn(0x7fd65c0a1080 legacy=0x7fd65c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 >> v1:172.21.15.84:6789/0 conn(0x7fd65c0a0cb0 legacy=0x7fd65c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 1 monclient: found mon.noname-b 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c0a1660 con 0x7fd65c06e810 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd6614c9640 5 monclient: authenticate success, global_id 145055 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd64c001830 con 0x7fd65c06e810 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:12:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 10 monclient: dump: 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.536 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd6614c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd64c001a10 con 0x7fd65c06e810 2024-09-18T14:12:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 1 -- 172.21.15.45:0/2630802221 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd64c004ba0 con 0x7fd65c06e810 2024-09-18T14:12:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd65a7fc640 10 monclient: dump: 2024-09-18T14:12:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.537 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd6614c9640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd6614c9640 4 set_mon_vals no callback set 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd6614c9640 10 monclient: shutdown 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd6614c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd6614c9640 1 -- 172.21.15.45:0/2630802221 >> v1:172.21.15.45:6790/0 conn(0x7fd65c06e810 legacy=0x7fd65c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd6614c9640 1 -- 172.21.15.45:0/2630802221 shutdown_connections 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd6614c9640 1 -- 172.21.15.45:0/2630802221 >> 172.21.15.45:0/2630802221 conn(0x7fd65c0a5bb0 msgr2=0x7fd65c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd6614c9640 1 -- 172.21.15.45:0/2630802221 shutdown_connections 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.523+0000 7fd6614c9640 1 -- 172.21.15.45:0/2630802221 wait complete. 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.524+0000 7fd6614c9640 10 monclient: build_initial_monmap 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.524+0000 7fd6614c9640 10 monclient: monmap: 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:13.539 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.524+0000 7fd6614c9640 1 Processor -- start 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.524+0000 7fd6614c9640 1 -- start start 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.524+0000 7fd6614c9640 10 monclient: init 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd6614c9640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd6614c9640 10 monclient: _add_conns ranks=[8,6,0,3,2,7,1,5,4] 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd6614c9640 10 monclient(hunting): picked mon.noname-e con 0x7fd65c06e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd6614c9640 10 monclient(hunting): picked mon.noname-d con 0x7fd65c0a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd6614c9640 10 monclient(hunting): picked mon.noname-a con 0x7fd65c0a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd6614c9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c122930 con 0x7fd65c0a1080 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd6614c9640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c13ea20 con 0x7fd65c0a0cb0 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd6614c9640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c13dcc0 con 0x7fd65c06e810 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd6614c9640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd6614c9640 10 monclient(hunting): authenticate will time out at 1548.864484s 2024-09-18T14:12:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd65affd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fd65c06e810 0x7fd65c13d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:41552/0 (socket says 172.21.15.45:41552) 2024-09-18T14:12:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd65affd640 1 -- 172.21.15.45:0/1113374639 learned_addr learned my addr 172.21.15.45:0/1113374639 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314898835 0 0) 0x7fd65c13ea20 con 0x7fd65c0a0cb0 2024-09-18T14:12:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 10 monclient(hunting): my global_id is 184362 2024-09-18T14:12:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd63c0034e0 con 0x7fd65c0a0cb0 2024-09-18T14:12:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440346706 0 0) 0x7fd65c13dcc0 con 0x7fd65c06e810 2024-09-18T14:12:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 10 monclient(hunting): my global_id is 5642 2024-09-18T14:12:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd65c13ea20 con 0x7fd65c06e810 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220076869 0 0) 0x7fd65c122930 con 0x7fd65c0a1080 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 10 monclient(hunting): my global_id is 5805 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.525+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd65c13dcc0 con 0x7fd65c0a1080 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 421217114 0 0) 0x7fd65c13ea20 con 0x7fd65c06e810 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd65c122930 con 0x7fd65c06e810 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4211486926 0 0) 0x7fd63c0034e0 con 0x7fd65c0a0cb0 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd65c13ea20 con 0x7fd65c0a0cb0 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2405065437 0 0) 0x7fd65c13dcc0 con 0x7fd65c0a1080 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd63c0034e0 con 0x7fd65c0a1080 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd64c0047a0 con 0x7fd65c06e810 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient(hunting): dump: 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.544 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd6440019e0 con 0x7fd65c0a0cb0 2024-09-18T14:12:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient(hunting): dump: 2024-09-18T14:12:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd654001c30 con 0x7fd65c0a1080 2024-09-18T14:12:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient(hunting): dump: 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3987526041 0 0) 0x7fd65c122930 con 0x7fd65c06e810 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 >> v1:172.21.15.45:6792/0 conn(0x7fd65c0a0cb0 legacy=0x7fd65c146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 >> v1:172.21.15.45:6789/0 conn(0x7fd65c0a1080 legacy=0x7fd65c149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 monclient: found mon.e 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c122650 con 0x7fd65c06e810 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient: _finish_auth 0 2024-09-18T14:12:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd6614c9640 5 monclient: authenticate success, global_id 5642 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd6614c9640 10 monclient: _renew_subs 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd6614c9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd6614c9640 1 -- 172.21.15.45:0/1113374639 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c122c10 con 0x7fd65c06e810 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd6614c9640 10 monclient: _renew_subs 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd6614c9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd6614c9640 1 -- 172.21.15.45:0/1113374639 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c121920 con 0x7fd65c06e810 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd64c004e20 con 0x7fd65c06e810 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd64c0052d0 con 0x7fd65c06e810 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd659ffb640 4 set_mon_vals no callback set 2024-09-18T14:12:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.526+0000 7fd652ffd640 10 monclient: dump: 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.550 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.527+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd64c0102a0 con 0x7fd65c06e810 2024-09-18T14:12:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.527+0000 7fd6614c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.527+0000 7fd6614c9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.527+0000 7fd6614c9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:12:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.527+0000 7fd6614c9640 1 -- 172.21.15.45:0/1113374639 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd640005010 con 0x7fd65c06e810 2024-09-18T14:12:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.527+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(625..625 src has 424..625) v4 ==== 3959+0+0 (unknown 2606322421 0 0) 0x7fd64c046da0 con 0x7fd65c06e810 2024-09-18T14:12:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.528+0000 7fd653fff640 10 monclient: get_auth_request con 0x7fd63c03a970 auth_method 0 2024-09-18T14:12:13.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.532+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fd64c0155c0 con 0x7fd65c06e810 2024-09-18T14:12:13.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.532+0000 7fd652ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:13.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.532+0000 7fd652ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:13.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.756+0000 7fd6614c9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:12:13.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.756+0000 7fd6614c9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:12:13.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.756+0000 7fd6614c9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:12:13.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.756+0000 7fd6614c9640 1 -- 172.21.15.45:0/1113374639 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd640005840 con 0x7fd65c06e810 2024-09-18T14:12:13.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.757+0000 7fd652ffd640 1 -- 172.21.15.45:0/1113374639 <== mon.8 v1:172.21.15.45:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2689 (unknown 254746657 0 3223046992) 0x7fd64c014fc0 con 0x7fd65c06e810 2024-09-18T14:12:13.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.757+0000 7fd652ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:12:13.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.757+0000 7fd652ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:13.759 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":106,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:12:13.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.759+0000 7fd6614c9640 1 -- 172.21.15.45:0/1113374639 >> v1:172.21.15.84:6812/31478 conn(0x7fd63c03a970 legacy=0x7fd63c03ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.759+0000 7fd6614c9640 10 monclient: shutdown 2024-09-18T14:12:13.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.759+0000 7fd6614c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:13.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.759+0000 7fd6614c9640 1 -- 172.21.15.45:0/1113374639 >> v1:172.21.15.45:6793/0 conn(0x7fd65c06e810 legacy=0x7fd65c13d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.759+0000 7fd6614c9640 1 -- 172.21.15.45:0/1113374639 shutdown_connections 2024-09-18T14:12:13.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.759+0000 7fd6614c9640 1 -- 172.21.15.45:0/1113374639 >> 172.21.15.45:0/1113374639 conn(0x7fd65c0a5bb0 msgr2=0x7fd65c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:13.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.759+0000 7fd6614c9640 1 -- 172.21.15.45:0/1113374639 shutdown_connections 2024-09-18T14:12:13.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.759+0000 7fd6614c9640 1 -- 172.21.15.45:0/1113374639 wait complete. 2024-09-18T14:12:13.770 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:12:13.770 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:12:13.771 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:12:13.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.866+0000 7f94b540f640 10 monclient: get_monmap_and_config 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.866+0000 7f94b540f640 10 monclient: build_initial_monmap 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.866+0000 7f94b540f640 10 monclient: monmap: 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:13.868377+0000 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:13.868377+0000 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:13.868 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:13.869 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:13.869 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:13.869 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:13.869 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.866+0000 7f94b540f640 1 Processor -- start 2024-09-18T14:12:13.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94b540f640 1 -- start start 2024-09-18T14:12:13.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94b540f640 10 monclient: init 2024-09-18T14:12:13.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94b540f640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:13.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94b540f640 10 monclient: _add_conns ranks=[4,8,2,6,1,7,3,0,5] 2024-09-18T14:12:13.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94b540f640 10 monclient(hunting): picked mon.noname-c con 0x7f94b00a2630 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:13.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94b540f640 10 monclient(hunting): picked mon.noname-e con 0x7f94b009ee00 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:13.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94b540f640 10 monclient(hunting): picked mon.noname-b con 0x7f94b009f1d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:13.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94b540f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b005ed70 con 0x7f94b009f1d0 2024-09-18T14:12:13.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94b540f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b00972b0 con 0x7f94b00a2630 2024-09-18T14:12:13.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94b540f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b00a0d40 con 0x7f94b009ee00 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94b540f640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94b540f640 10 monclient(hunting): authenticate will time out at 1549.206573s 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94af7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f94b009f1d0 0x7f94b013aeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55774/0 (socket says 172.21.15.45:55774) 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94aeffd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f94b00a2630 0x7f94b00a0630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:55748/0 (socket says 172.21.15.45:55748) 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94af7fe640 1 -- 172.21.15.45:0/89977111 learned_addr learned my addr 172.21.15.45:0/89977111 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.867+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278854582 0 0) 0x7f94b005ed70 con 0x7f94b009f1d0 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient(hunting): my global_id is 145064 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94980034e0 con 0x7f94b009f1d0 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099131006 0 0) 0x7f94b00972b0 con 0x7f94b00a2630 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient(hunting): my global_id is 164542 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94b005ed70 con 0x7f94b00a2630 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284566248 0 0) 0x7f94b00a0d40 con 0x7f94b009ee00 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient(hunting): my global_id is 5651 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94b00972b0 con 0x7f94b009ee00 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 542008124 0 0) 0x7f94980034e0 con 0x7f94b009f1d0 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 >> v1:172.21.15.45:6793/0 conn(0x7f94b009ee00 legacy=0x7f94b0137780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 >> v1:172.21.15.45:6791/0 conn(0x7f94b00a2630 legacy=0x7f94b00a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient: _finish_hunting 0 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 monclient: found mon.noname-b 2024-09-18T14:12:13.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b00a2a00 con 0x7f94b009f1d0 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient: _finish_auth 0 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f94a8001bb0 con 0x7f94b009f1d0 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient: dump: 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.885 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94b540f640 5 monclient: authenticate success, global_id 145064 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94b540f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f94a8001d90 con 0x7f94b009f1d0 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:13.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 1 -- 172.21.15.45:0/89977111 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f94a8004f20 con 0x7f94b009f1d0 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94b540f640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94b540f640 4 set_mon_vals no callback set 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94adffb640 10 monclient: dump: 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.887 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94b540f640 10 monclient: shutdown 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94b540f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.868+0000 7f94b540f640 1 -- 172.21.15.45:0/89977111 >> v1:172.21.15.45:6790/0 conn(0x7f94b009f1d0 legacy=0x7f94b013aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.869+0000 7f94b540f640 1 -- 172.21.15.45:0/89977111 shutdown_connections 2024-09-18T14:12:13.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.869+0000 7f94b540f640 1 -- 172.21.15.45:0/89977111 >> 172.21.15.45:0/89977111 conn(0x7f94b0063150 msgr2=0x7f94b0063550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.869+0000 7f94b540f640 1 -- 172.21.15.45:0/89977111 shutdown_connections 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.869+0000 7f94b540f640 1 -- 172.21.15.45:0/89977111 wait complete. 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.869+0000 7f94b540f640 10 monclient: build_initial_monmap 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.869+0000 7f94b540f640 10 monclient: monmap: 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:13.889 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.869+0000 7f94b540f640 1 Processor -- start 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94b540f640 1 -- start start 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94b540f640 10 monclient: init 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94b540f640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94b540f640 10 monclient: _add_conns ranks=[0,8,4,2,6,3,1,5,7] 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94b540f640 10 monclient(hunting): picked mon.noname-a con 0x7f94b009ee00 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94b540f640 10 monclient(hunting): picked mon.noname-e con 0x7f94b009f1d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:13.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94b540f640 10 monclient(hunting): picked mon.noname-c con 0x7f94b00a2630 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94b540f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b01228e0 con 0x7f94b009ee00 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94b540f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b013dc70 con 0x7f94b00a2630 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94b540f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b0120400 con 0x7f94b009f1d0 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94b540f640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94b540f640 10 monclient(hunting): authenticate will time out at 1549.209477s 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94ae7fc640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f94b009f1d0 0x7f94b01467b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:41566/0 (socket says 172.21.15.45:41566) 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94af7fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f94b00a2630 0x7f94b0149ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:55764/0 (socket says 172.21.15.45:55764) 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94ae7fc640 1 -- 172.21.15.45:0/2511066928 learned_addr learned my addr 172.21.15.45:0/2511066928 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f94aeffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f94b009ee00 0x7f94b013d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59902/0 (socket says 172.21.15.45:59902) 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752420840 0 0) 0x7f94b01228e0 con 0x7f94b009ee00 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 10 monclient(hunting): my global_id is 5814 2024-09-18T14:12:13.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f949c0034e0 con 0x7f94b009ee00 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233533087 0 0) 0x7f94b013dc70 con 0x7f94b00a2630 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 10 monclient(hunting): my global_id is 164551 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94b01228e0 con 0x7f94b00a2630 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436096149 0 0) 0x7f94b0120400 con 0x7f94b009f1d0 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 10 monclient(hunting): my global_id is 5660 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.870+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94b013dc70 con 0x7f94b009f1d0 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 718101437 0 0) 0x7f949c0034e0 con 0x7f94b009ee00 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f94b0120400 con 0x7f94b009ee00 2024-09-18T14:12:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f94a0001930 con 0x7f94b009ee00 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient(hunting): dump: 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 281647752 0 0) 0x7f94b01228e0 con 0x7f94b00a2630 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f949c0034e0 con 0x7f94b00a2630 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4107906966 0 0) 0x7f94b013dc70 con 0x7f94b009f1d0 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f94b01228e0 con 0x7f94b009f1d0 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f94a8001b70 con 0x7f94b00a2630 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient(hunting): dump: 2024-09-18T14:12:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f94a4002950 con 0x7f94b009f1d0 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient(hunting): dump: 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 579759298 0 0) 0x7f94b0120400 con 0x7f94b009ee00 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 >> v1:172.21.15.45:6793/0 conn(0x7f94b009f1d0 legacy=0x7f94b01467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 >> v1:172.21.15.45:6791/0 conn(0x7f94b00a2630 legacy=0x7f94b0149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 monclient: found mon.a 2024-09-18T14:12:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b0122bc0 con 0x7f94b009ee00 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient: _finish_auth 0 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f94b540f640 5 monclient: authenticate success, global_id 5814 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f94b540f640 10 monclient: _renew_subs 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f94b540f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f94b540f640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b0121730 con 0x7f94b009ee00 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f94b540f640 10 monclient: _renew_subs 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f94b540f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f94b540f640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b013e9c0 con 0x7f94b009ee00 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f94a00047a0 con 0x7f94b009ee00 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f94a0004cd0 con 0x7f94b009ee00 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f94ad7fa640 4 set_mon_vals no callback set 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.871+0000 7f9496ffd640 10 monclient: dump: 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.872+0000 7f94b540f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.872+0000 7f94b540f640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9490006580 con 0x7f9490002830 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.872+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f94a000fd20 con 0x7f94b009ee00 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.872+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(625..625 src has 424..625) v4 ==== 3959+0+0 (unknown 2606322421 0 0) 0x7f94a00469f0 con 0x7f94b009ee00 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.872+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049482858 0 0) 0x7f9490006580 con 0x7f9490002830 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.872+0000 7f9496ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.872+0000 7f9496ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.872+0000 7f9496ffd640 10 monclient(hunting): my global_id is 5823 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.872+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94b0120400 con 0x7f9490002830 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.872+0000 7f94af7fe640 10 monclient: get_auth_request con 0x7f949c03a930 auth_method 0 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.872+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1059626344 0 0) 0x7f94b0120400 con 0x7f9490002830 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.873+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9490006580 con 0x7f9490002830 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.873+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f94a4001750 con 0x7f9490002830 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.873+0000 7f9496ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.873+0000 7f9496ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.873+0000 7f9496ffd640 10 monclient: dump: 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.873+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3344125369 0 0) 0x7f9490006580 con 0x7f9490002830 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.873+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f949000a0a0 con 0x7f9490002830 2024-09-18T14:12:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.873+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f94a4001a10 con 0x7f9490002830 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.873+0000 7f9496ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.873+0000 7f9496ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.873+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 >> v1:172.21.15.45:6789/0 conn(0x7f9490002830 legacy=0x7f9490004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.873+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 >> v1:172.21.15.45:6789/0 conn(0x7f9490002830 legacy=0x7f9490004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f94b540f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f94b540f640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f94900074f0 con 0x7f94900068a0 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811758988 0 0) 0x7f94900074f0 con 0x7f94900068a0 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 10 monclient(hunting): my global_id is 5832 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9490006580 con 0x7f94900068a0 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1169538012 0 0) 0x7f9490006580 con 0x7f94900068a0 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f94900074f0 con 0x7f94900068a0 2024-09-18T14:12:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f94a4001f90 con 0x7f94900068a0 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 10 monclient: dump: 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:13.905 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4233713472 0 0) 0x7f94900074f0 con 0x7f94900068a0 2024-09-18T14:12:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.879+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9490008750 con 0x7f94900068a0 2024-09-18T14:12:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 4020134509) 0x7f94a4007b00 con 0x7f94900068a0 2024-09-18T14:12:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f9496ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f9496ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 >> v1:172.21.15.45:6789/0 conn(0x7f94900068a0 legacy=0x7f9490006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f9496ffd640 1 -- 172.21.15.45:0/2511066928 >> v1:172.21.15.45:6789/0 conn(0x7f94900068a0 legacy=0x7f9490006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:13.906 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:13.906 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:13.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:13.908 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:13.909 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:13.910 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:13.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:13.913 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:13.914 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:13.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.916 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:13.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:13.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:13.920 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:13.921 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:13.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.923 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:13.924 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:12:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:13.925 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:13.925 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f94b540f640 1 -- 172.21.15.45:0/2511066928 >> v1:172.21.15.84:6812/31478 conn(0x7f949c03a930 legacy=0x7f949c03cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f94b540f640 10 monclient: shutdown 2024-09-18T14:12:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f94b540f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f94b540f640 1 -- 172.21.15.45:0/2511066928 >> v1:172.21.15.45:6789/0 conn(0x7f94b009ee00 legacy=0x7f94b013d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f94af7fe640 1 -- 172.21.15.45:0/2511066928 reap_dead start 2024-09-18T14:12:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f94af7fe640 1 -- 172.21.15.45:0/2511066928 reap_dead start 2024-09-18T14:12:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f94b540f640 1 -- 172.21.15.45:0/2511066928 shutdown_connections 2024-09-18T14:12:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f94b540f640 1 -- 172.21.15.45:0/2511066928 >> 172.21.15.45:0/2511066928 conn(0x7f94b0063150 msgr2=0x7f94b00a3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f94b540f640 1 -- 172.21.15.45:0/2511066928 shutdown_connections 2024-09-18T14:12:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:13.880+0000 7f94b540f640 1 -- 172.21.15.45:0/2511066928 wait complete. 2024-09-18T14:12:13.928 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:13.928 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:12:13.928 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:12:14.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.023+0000 7f20a4ce8640 10 monclient: get_monmap_and_config 2024-09-18T14:12:14.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.023+0000 7f20a4ce8640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.023+0000 7f20a4ce8640 10 monclient: monmap: 2024-09-18T14:12:14.025 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.025 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.025 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:14.025821+0000 2024-09-18T14:12:14.025 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:14.025821+0000 2024-09-18T14:12:14.025 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.025 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.026 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.026 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.026 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.026 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.026 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.026 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.026 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.026 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.026 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.026 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.024+0000 7f20a4ce8640 1 Processor -- start 2024-09-18T14:12:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.024+0000 7f20a4ce8640 1 -- start start 2024-09-18T14:12:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.024+0000 7f20a4ce8640 10 monclient: init 2024-09-18T14:12:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f20a4ce8640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f20a4ce8640 10 monclient: _add_conns ranks=[3,4,2,6,5,8,0,7,1] 2024-09-18T14:12:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f20a4ce8640 10 monclient(hunting): picked mon.noname-g con 0x7f20a009f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f20a4ce8640 10 monclient(hunting): picked mon.noname-c con 0x7f20a009fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f20a4ce8640 10 monclient(hunting): picked mon.noname-b con 0x7f20a009ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f20a4ce8640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a005ed70 con 0x7f20a009ff70 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f20a4ce8640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a009dfc0 con 0x7f20a009f7d0 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f20a4ce8640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a009dce0 con 0x7f20a009fba0 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f20a4ce8640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f20a4ce8640 10 monclient(hunting): authenticate will time out at 1549.364321s 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209ed9d640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f20a009ff70 0x7f20a013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55782/0 (socket says 172.21.15.45:55782) 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209ed9d640 1 -- 172.21.15.45:0/404390589 learned_addr learned my addr 172.21.15.45:0/404390589 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 909371091 0 0) 0x7f20a005ed70 con 0x7f20a009ff70 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient(hunting): my global_id is 145073 2024-09-18T14:12:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20880034e0 con 0x7f20a009ff70 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500374469 0 0) 0x7f20a009dce0 con 0x7f20a009fba0 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient(hunting): my global_id is 164560 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a005ed70 con 0x7f20a009fba0 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221368311 0 0) 0x7f20a009dfc0 con 0x7f20a009f7d0 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient(hunting): my global_id is 184314 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a009dce0 con 0x7f20a009f7d0 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3682725368 0 0) 0x7f20880034e0 con 0x7f20a009ff70 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 >> v1:172.21.15.45:6791/0 conn(0x7f20a009fba0 legacy=0x7f20a0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 >> v1:172.21.15.84:6790/0 conn(0x7f20a009f7d0 legacy=0x7f20a0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 1 monclient: found mon.noname-b 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00a1c00 con 0x7f20a009ff70 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f20a4ce8640 5 monclient: authenticate success, global_id 145073 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2098001bb0 con 0x7f20a009ff70 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f20a4ce8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.025+0000 7f209d59a640 10 monclient: dump: 2024-09-18T14:12:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.038 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f20a4ce8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2098001d10 con 0x7f20a009ff70 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f209d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f20a4ce8640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f209d59a640 1 -- 172.21.15.45:0/404390589 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2098004f20 con 0x7f20a009ff70 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f20a4ce8640 4 set_mon_vals no callback set 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f209d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f209d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f209d59a640 10 monclient: dump: 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f20a4ce8640 10 monclient: shutdown 2024-09-18T14:12:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f20a4ce8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f20a4ce8640 1 -- 172.21.15.45:0/404390589 >> v1:172.21.15.45:6790/0 conn(0x7f20a009ff70 legacy=0x7f20a013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f20a4ce8640 1 -- 172.21.15.45:0/404390589 shutdown_connections 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f20a4ce8640 1 -- 172.21.15.45:0/404390589 >> 172.21.15.45:0/404390589 conn(0x7f20a00630b0 msgr2=0x7f20a00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f20a4ce8640 1 -- 172.21.15.45:0/404390589 shutdown_connections 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.026+0000 7f20a4ce8640 1 -- 172.21.15.45:0/404390589 wait complete. 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.027+0000 7f20a4ce8640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.027+0000 7f20a4ce8640 10 monclient: monmap: 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.027+0000 7f20a4ce8640 1 Processor -- start 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.027+0000 7f20a4ce8640 1 -- start start 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.027+0000 7f20a4ce8640 10 monclient: init 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.027+0000 7f20a4ce8640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.027+0000 7f20a4ce8640 10 monclient: _add_conns ranks=[6,3,2,0,4,8,7,5,1] 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.027+0000 7f20a4ce8640 10 monclient(hunting): picked mon.noname-d con 0x7f20a009f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.027+0000 7f20a4ce8640 10 monclient(hunting): picked mon.noname-g con 0x7f20a009fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.027+0000 7f20a4ce8640 10 monclient(hunting): picked mon.noname-b con 0x7f20a009ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.027+0000 7f20a4ce8640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a009d310 con 0x7f20a009ff70 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.027+0000 7f20a4ce8640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a00a2560 con 0x7f20a009fba0 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f20a4ce8640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a0142730 con 0x7f20a009f7d0 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f20a4ce8640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f209e59c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f20a009f7d0 0x7f20a0141840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40366/0 (socket says 172.21.15.45:40366) 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f20a4ce8640 10 monclient(hunting): authenticate will time out at 1549.367073s 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f209e59c640 1 -- 172.21.15.45:0/1680993543 learned_addr learned my addr 172.21.15.45:0/1680993543 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990866565 0 0) 0x7f20a009d310 con 0x7f20a009ff70 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): my global_id is 145082 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20800034e0 con 0x7f20a009ff70 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403259620 0 0) 0x7f20a0142730 con 0x7f20a009f7d0 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): my global_id is 184371 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a009d310 con 0x7f20a009f7d0 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71123933 0 0) 0x7f20a00a2560 con 0x7f20a009fba0 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): my global_id is 184323 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a0142730 con 0x7f20a009fba0 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3965432200 0 0) 0x7f20800034e0 con 0x7f20a009ff70 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20a00a2560 con 0x7f20a009ff70 2024-09-18T14:12:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1713881124 0 0) 0x7f20a009d310 con 0x7f20a009f7d0 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20800034e0 con 0x7f20a009f7d0 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2098001af0 con 0x7f20a009ff70 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.045 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2090001a20 con 0x7f20a009f7d0 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2165515974 0 0) 0x7f20a0142730 con 0x7f20a009fba0 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20a009d310 con 0x7f20a009fba0 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2094001a50 con 0x7f20a009fba0 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1330450074 0 0) 0x7f20a00a2560 con 0x7f20a009ff70 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 >> v1:172.21.15.45:6792/0 conn(0x7f20a009f7d0 legacy=0x7f20a0141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 >> v1:172.21.15.84:6790/0 conn(0x7f20a009fba0 legacy=0x7f20a0131a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 monclient: found mon.b 2024-09-18T14:12:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a0141f50 con 0x7f20a009ff70 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f20a4ce8640 5 monclient: authenticate success, global_id 145082 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f20a4ce8640 10 monclient: _renew_subs 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f20a4ce8640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f20a4ce8640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a0120c90 con 0x7f20a009ff70 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f20a4ce8640 10 monclient: _renew_subs 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f20a4ce8640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f20a4ce8640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a01200c0 con 0x7f20a009ff70 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2098004ec0 con 0x7f20a009ff70 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f209cd99640 4 set_mon_vals no callback set 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.028+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2098005470 con 0x7f20a009ff70 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.029+0000 7f208e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.029+0000 7f208e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.029+0000 7f208e7fc640 10 monclient: dump: 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.029+0000 7f20a4ce8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.029+0000 7f20a4ce8640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2084006580 con 0x7f2084002830 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.029+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2098010440 con 0x7f20a009ff70 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.029+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(625..625 src has 424..625) v4 ==== 3959+0+0 (unknown 2606322421 0 0) 0x7f2098046e40 con 0x7f20a009ff70 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905181760 0 0) 0x7f2084006580 con 0x7f2084002830 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 10 monclient(hunting): my global_id is 145091 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a00a2560 con 0x7f2084002830 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f209dd9b640 10 monclient: get_auth_request con 0x7f208003a8b0 auth_method 0 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 259636229 0 0) 0x7f20a00a2560 con 0x7f2084002830 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2084006580 con 0x7f2084002830 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2090002620 con 0x7f2084002830 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 10 monclient: dump: 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.053 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2538320936 0 0) 0x7f2084006580 con 0x7f2084002830 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f208400a0a0 con 0x7f2084002830 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2090002940 con 0x7f2084002830 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 >> v1:172.21.15.45:6790/0 conn(0x7f2084002830 legacy=0x7f2084004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.030+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 >> v1:172.21.15.45:6790/0 conn(0x7f2084002830 legacy=0x7f2084004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f20a4ce8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f20a4ce8640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2084009e70 con 0x7f20840068f0 2024-09-18T14:12:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316697583 0 0) 0x7f2084009e70 con 0x7f20840068f0 2024-09-18T14:12:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f208e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f208e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f208e7fc640 10 monclient(hunting): my global_id is 145100 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2084006580 con 0x7f20840068f0 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1259760412 0 0) 0x7f2084006580 con 0x7f20840068f0 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2084009e70 con 0x7f20840068f0 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2090005ac0 con 0x7f20840068f0 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f208e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f208e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.036+0000 7f208e7fc640 10 monclient: dump: 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3234552796 0 0) 0x7f2084009e70 con 0x7f20840068f0 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2084008750 con 0x7f20840068f0 2024-09-18T14:12:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 52410466) 0x7f2090007d90 con 0x7f20840068f0 2024-09-18T14:12:14.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f208e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:14.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f208e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:14.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 >> v1:172.21.15.45:6790/0 conn(0x7f20840068f0 legacy=0x7f2084006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f208e7fc640 1 -- 172.21.15.45:0/1680993543 >> v1:172.21.15.45:6790/0 conn(0x7f20840068f0 legacy=0x7f2084006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:14.059 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.073 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.076 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.077 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:12:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.078 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:14.078 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f20a4ce8640 1 -- 172.21.15.45:0/1680993543 >> v1:172.21.15.84:6812/31478 conn(0x7f208003a8b0 legacy=0x7f208003cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f20a4ce8640 10 monclient: shutdown 2024-09-18T14:12:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f20a4ce8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f20a4ce8640 1 -- 172.21.15.45:0/1680993543 >> v1:172.21.15.45:6790/0 conn(0x7f20a009ff70 legacy=0x7f20a0149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f209ed9d640 1 -- 172.21.15.45:0/1680993543 reap_dead start 2024-09-18T14:12:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f209ed9d640 1 -- 172.21.15.45:0/1680993543 reap_dead start 2024-09-18T14:12:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f20a4ce8640 1 -- 172.21.15.45:0/1680993543 shutdown_connections 2024-09-18T14:12:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f20a4ce8640 1 -- 172.21.15.45:0/1680993543 >> 172.21.15.45:0/1680993543 conn(0x7f20a00630b0 msgr2=0x7f20a00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f20a4ce8640 1 -- 172.21.15.45:0/1680993543 shutdown_connections 2024-09-18T14:12:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.037+0000 7f20a4ce8640 1 -- 172.21.15.45:0/1680993543 wait complete. 2024-09-18T14:12:14.080 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:14.080 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:12:14.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:12:14.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.176+0000 7f198d99d640 10 monclient: get_monmap_and_config 2024-09-18T14:12:14.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.176+0000 7f198d99d640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.176+0000 7f198d99d640 10 monclient: monmap: 2024-09-18T14:12:14.178 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.178 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.178 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:14.178821+0000 2024-09-18T14:12:14.178 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:14.178821+0000 2024-09-18T14:12:14.178 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.178 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.179 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.179 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.179 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.179 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.179 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.179 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.179 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.179 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.179 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.179 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.177+0000 7f198d99d640 1 Processor -- start 2024-09-18T14:12:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.177+0000 7f198d99d640 1 -- start start 2024-09-18T14:12:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.177+0000 7f198d99d640 10 monclient: init 2024-09-18T14:12:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 10 monclient: _add_conns ranks=[3,6,1,5,0,7,2,4,8] 2024-09-18T14:12:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 10 monclient(hunting): picked mon.noname-g con 0x7f198809f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 10 monclient(hunting): picked mon.noname-d con 0x7f198809fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 10 monclient(hunting): picked mon.noname-f con 0x7f198809ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198805ed70 con 0x7f198809ff70 2024-09-18T14:12:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198809dfc0 con 0x7f198809f7d0 2024-09-18T14:12:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198809dce0 con 0x7f198809fba0 2024-09-18T14:12:14.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 10 monclient(hunting): authenticate will time out at 1549.517206s 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f19867fc640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f198809fba0 0x7f1988137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40372/0 (socket says 172.21.15.45:40372) 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f19867fc640 1 -- 172.21.15.45:0/1361278209 learned_addr learned my addr 172.21.15.45:0/1361278209 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 1 -- 172.21.15.45:0/1361278209 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053675874 0 0) 0x7f198809dce0 con 0x7f198809fba0 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient(hunting): my global_id is 184380 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 1 -- 172.21.15.45:0/1361278209 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19700034e0 con 0x7f198809fba0 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 1 -- 172.21.15.45:0/1361278209 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198018587 0 0) 0x7f198809dfc0 con 0x7f198809f7d0 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient(hunting): my global_id is 184332 2024-09-18T14:12:14.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 1 -- 172.21.15.45:0/1361278209 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f198809dce0 con 0x7f198809f7d0 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 1 -- 172.21.15.45:0/1361278209 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3025618415 0 0) 0x7f19700034e0 con 0x7f198809fba0 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 1 -- 172.21.15.45:0/1361278209 >> v1:172.21.15.84:6790/0 conn(0x7f198809f7d0 legacy=0x7f1988132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 1 -- 172.21.15.45:0/1361278209 >> v1:172.21.15.84:6789/0 conn(0x7f198809ff70 legacy=0x7f198813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 1 monclient: found mon.noname-d 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 1 -- 172.21.15.45:0/1361278209 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19880a1c00 con 0x7f198809fba0 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 1 -- 172.21.15.45:0/1361278209 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f197c001830 con 0x7f198809fba0 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 5 monclient: authenticate success, global_id 184380 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient: dump: 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 1 -- 172.21.15.45:0/1361278209 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f197c001990 con 0x7f198809fba0 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 1 -- 172.21.15.45:0/1361278209 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f197c004ba0 con 0x7f198809fba0 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f198d99d640 4 set_mon_vals no callback set 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.178+0000 7f1985ffb640 10 monclient: dump: 2024-09-18T14:12:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.179+0000 7f198d99d640 10 monclient: shutdown 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.179+0000 7f198d99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.179+0000 7f198d99d640 1 -- 172.21.15.45:0/1361278209 >> v1:172.21.15.45:6792/0 conn(0x7f198809fba0 legacy=0x7f1988137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.179+0000 7f198d99d640 1 -- 172.21.15.45:0/1361278209 shutdown_connections 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.179+0000 7f198d99d640 1 -- 172.21.15.45:0/1361278209 >> 172.21.15.45:0/1361278209 conn(0x7f19880630b0 msgr2=0x7f19880634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.179+0000 7f198d99d640 1 -- 172.21.15.45:0/1361278209 shutdown_connections 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.179+0000 7f198d99d640 1 -- 172.21.15.45:0/1361278209 wait complete. 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 10 monclient: monmap: 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 1 Processor -- start 2024-09-18T14:12:14.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 1 -- start start 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 10 monclient: init 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 10 monclient: _add_conns ranks=[1,8,2,5,6,4,7,0,3] 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 10 monclient(hunting): picked mon.noname-f con 0x7f198809f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 10 monclient(hunting): picked mon.noname-e con 0x7f198809fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 10 monclient(hunting): picked mon.noname-b con 0x7f198809ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19881208e0 con 0x7f198809f7d0 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198809d310 con 0x7f198809ff70 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19880a2560 con 0x7f198809fba0 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.180+0000 7f198d99d640 10 monclient(hunting): authenticate will time out at 1549.519974s 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f19867fc640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f198809fba0 0x7f1988131a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:41570/0 (socket says 172.21.15.45:41570) 2024-09-18T14:12:14.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f19867fc640 1 -- 172.21.15.45:0/375575235 learned_addr learned my addr 172.21.15.45:0/375575235 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f19877fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f198809ff70 0x7f1988149bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55822/0 (socket says 172.21.15.45:55822) 2024-09-18T14:12:14.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474092370 0 0) 0x7f19880a2560 con 0x7f198809fba0 2024-09-18T14:12:14.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): my global_id is 5669 2024-09-18T14:12:14.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19740034e0 con 0x7f198809fba0 2024-09-18T14:12:14.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538208475 0 0) 0x7f198809d310 con 0x7f198809ff70 2024-09-18T14:12:14.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): my global_id is 145109 2024-09-18T14:12:14.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19880a2560 con 0x7f198809ff70 2024-09-18T14:12:14.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820125079 0 0) 0x7f19881208e0 con 0x7f198809f7d0 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): my global_id is 204121 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f198809d310 con 0x7f198809f7d0 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3131560449 0 0) 0x7f19740034e0 con 0x7f198809fba0 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19881208e0 con 0x7f198809fba0 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 681273383 0 0) 0x7f19880a2560 con 0x7f198809ff70 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19740034e0 con 0x7f198809ff70 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1333236124 0 0) 0x7f198809d310 con 0x7f198809f7d0 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19880a2560 con 0x7f198809f7d0 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f197c001830 con 0x7f198809fba0 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:12:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): dump: 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.199 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f19800018c0 con 0x7f198809ff70 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): dump: 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.200 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.201 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.201 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.201 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.201 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.201 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.201 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.201 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.201 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.201 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.201 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1978001f70 con 0x7f198809f7d0 2024-09-18T14:12:14.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient(hunting): dump: 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.202 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4170756807 0 0) 0x7f19881208e0 con 0x7f198809fba0 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 >> v1:172.21.15.45:6790/0 conn(0x7f198809ff70 legacy=0x7f1988149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 >> v1:172.21.15.84:6789/0 conn(0x7f198809f7d0 legacy=0x7f1988141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 monclient: found mon.e 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:12:14.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1988121f30 con 0x7f198809fba0 2024-09-18T14:12:14.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f196effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f198d99d640 5 monclient: authenticate success, global_id 5669 2024-09-18T14:12:14.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f198d99d640 10 monclient: _renew_subs 2024-09-18T14:12:14.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f198d99d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:12:14.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.181+0000 7f198d99d640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1988120140 con 0x7f198809fba0 2024-09-18T14:12:14.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f198d99d640 10 monclient: _renew_subs 2024-09-18T14:12:14.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f198d99d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:12:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f198d99d640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1988120cb0 con 0x7f198809fba0 2024-09-18T14:12:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f197c0047a0 con 0x7f198809fba0 2024-09-18T14:12:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f196effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f197c001210 con 0x7f198809fba0 2024-09-18T14:12:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f196effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f19857fa640 4 set_mon_vals no callback set 2024-09-18T14:12:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f196effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f196effd640 10 monclient: dump: 2024-09-18T14:12:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.206 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.206 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.206 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.206 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.206 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.206 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.206 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.206 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.206 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.206 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.206 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f198d99d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f197c010090 con 0x7f198809fba0 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.182+0000 7f198d99d640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1968006580 con 0x7f1968002830 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(625..625 src has 424..625) v4 ==== 3959+0+0 (unknown 2606322421 0 0) 0x7f197c046c90 con 0x7f198809fba0 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146456598 0 0) 0x7f1968006580 con 0x7f1968002830 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 10 monclient(hunting): my global_id is 164569 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19881208e0 con 0x7f1968002830 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f19877fe640 10 monclient: get_auth_request con 0x7f197403a980 auth_method 0 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2198691693 0 0) 0x7f19881208e0 con 0x7f1968002830 2024-09-18T14:12:14.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1968006580 con 0x7f1968002830 2024-09-18T14:12:14.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1978002410 con 0x7f1968002830 2024-09-18T14:12:14.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:14.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 10 monclient: dump: 2024-09-18T14:12:14.208 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.208 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.208 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.208 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.208 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.208 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.208 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2873974745 0 0) 0x7f1968006580 con 0x7f1968002830 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f196800a0a0 con 0x7f1968002830 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1978004670 con 0x7f1968002830 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:14.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:14.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 >> v1:172.21.15.45:6791/0 conn(0x7f1968002830 legacy=0x7f1968004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.183+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 >> v1:172.21.15.45:6791/0 conn(0x7f1968002830 legacy=0x7f1968004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:14.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.189+0000 7f198d99d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:14.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.189+0000 7f198d99d640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1968009610 con 0x7f1968008ff0 2024-09-18T14:12:14.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819029393 0 0) 0x7f1968009610 con 0x7f1968008ff0 2024-09-18T14:12:14.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 10 monclient(hunting): my global_id is 164578 2024-09-18T14:12:14.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1968006580 con 0x7f1968008ff0 2024-09-18T14:12:14.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3988875575 0 0) 0x7f1968006580 con 0x7f1968008ff0 2024-09-18T14:12:14.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1968009610 con 0x7f1968008ff0 2024-09-18T14:12:14.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1978002e60 con 0x7f1968008ff0 2024-09-18T14:12:14.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:14.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 10 monclient: dump: 2024-09-18T14:12:14.212 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.212 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.212 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.213 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.213 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.213 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.213 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.213 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.213 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.214 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.214 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.214 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.214 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.214 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.214 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.214 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3907706295 0 0) 0x7f1968009610 con 0x7f1968008ff0 2024-09-18T14:12:14.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f19680098d0 con 0x7f1968008ff0 2024-09-18T14:12:14.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 1042630591) 0x7f1978004940 con 0x7f1968008ff0 2024-09-18T14:12:14.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:14.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:14.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 >> v1:172.21.15.45:6791/0 conn(0x7f1968008ff0 legacy=0x7f1968008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.190+0000 7f196effd640 1 -- 172.21.15.45:0/375575235 >> v1:172.21.15.45:6791/0 conn(0x7f1968008ff0 legacy=0x7f1968008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:14.215 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.226 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.228 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:14.229 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.230 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.231 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:14.232 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:14.233 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.235 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:14.235 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.191+0000 7f198d99d640 1 -- 172.21.15.45:0/375575235 >> v1:172.21.15.84:6812/31478 conn(0x7f197403a980 legacy=0x7f197403ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.191+0000 7f198d99d640 10 monclient: shutdown 2024-09-18T14:12:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.191+0000 7f198d99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.191+0000 7f198d99d640 1 -- 172.21.15.45:0/375575235 >> v1:172.21.15.45:6793/0 conn(0x7f198809fba0 legacy=0x7f1988131a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.191+0000 7f19877fe640 1 -- 172.21.15.45:0/375575235 reap_dead start 2024-09-18T14:12:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.191+0000 7f19877fe640 1 -- 172.21.15.45:0/375575235 reap_dead start 2024-09-18T14:12:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.191+0000 7f198d99d640 1 -- 172.21.15.45:0/375575235 shutdown_connections 2024-09-18T14:12:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.191+0000 7f198d99d640 1 -- 172.21.15.45:0/375575235 >> 172.21.15.45:0/375575235 conn(0x7f19880630b0 msgr2=0x7f19880a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.191+0000 7f198d99d640 1 -- 172.21.15.45:0/375575235 shutdown_connections 2024-09-18T14:12:14.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.191+0000 7f198d99d640 1 -- 172.21.15.45:0/375575235 wait complete. 2024-09-18T14:12:14.237 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:14.237 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:12:14.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:12:14.326 INFO:tasks.rados.rados.0.smithi084.stdout:2815: done (1 left) 2024-09-18T14:12:14.326 INFO:tasks.rados.rados.0.smithi084.stdout:2816: rollback oid 7 current snap is 306 2024-09-18T14:12:14.326 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 7 to 292 2024-09-18T14:12:14.327 INFO:tasks.rados.rados.0.smithi084.stdout:2813: finishing write tid 4 to smithi08434663-24 2024-09-18T14:12:14.327 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 597 (ObjNum 959 snap 306 seq_num 959) dirty exists 2024-09-18T14:12:14.327 INFO:tasks.rados.rados.0.smithi084.stdout:2813: left oid 24 (ObjNum 959 snap 306 seq_num 959) 2024-09-18T14:12:14.327 INFO:tasks.rados.rados.0.smithi084.stdout:2813: done (1 left) 2024-09-18T14:12:14.327 INFO:tasks.rados.rados.0.smithi084.stdout:2817: copy_from oid 3 from oid 38 current snap is 306 2024-09-18T14:12:14.327 INFO:tasks.rados.rados.0.smithi084.stdout:2818: copy_from oid 48 from oid 33 current snap is 306 2024-09-18T14:12:14.327 INFO:tasks.rados.rados.0.smithi084.stdout:2819: write (excl) oid 19 current snap is 306 2024-09-18T14:12:14.327 INFO:tasks.rados.rados.0.smithi084.stdout:2819: seq_num 960 ranges {534720=499518,1460243=674477} 2024-09-18T14:12:14.332 INFO:tasks.rados.rados.0.smithi084.stdout:2819: writing smithi08434663-19 from 534720 to 1034238 tid 1 2024-09-18T14:12:14.340 INFO:tasks.rados.rados.0.smithi084.stdout:2819: writing smithi08434663-19 from 1460243 to 2134720 tid 2 2024-09-18T14:12:14.340 INFO:tasks.rados.rados.0.smithi084.stdout:2820: read oid 6 snap -1 2024-09-18T14:12:14.341 INFO:tasks.rados.rados.0.smithi084.stdout:2820: expect (ObjNum 907 snap 281 seq_num 907) 2024-09-18T14:12:14.341 INFO:tasks.rados.rados.0.smithi084.stdout:2821: write (excl) oid 1 current snap is 306 2024-09-18T14:12:14.341 INFO:tasks.rados.rados.0.smithi084.stdout:2821: seq_num 961 ranges {551527=574767,1918725=232802} 2024-09-18T14:12:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.345+0000 7f922b59e640 10 monclient: get_monmap_and_config 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.345+0000 7f922b59e640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.345+0000 7f922b59e640 10 monclient: monmap: 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:14.347593+0000 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:14.347593+0000 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.348 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.348 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.348 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.348 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.348 INFO:tasks.rados.rados.0.smithi084.stdout:2821: writing smithi08434663-1 from 551527 to 1126294 tid 1 2024-09-18T14:12:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:2821: writing smithi08434663-1 from 1918725 to 2151527 tid 2 2024-09-18T14:12:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:2822: read oid 8 snap -1 2024-09-18T14:12:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:2822: expect deleted 2024-09-18T14:12:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:2823: read oid 9 snap -1 2024-09-18T14:12:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:2823: expect (ObjNum 957 snap 304 seq_num 957) 2024-09-18T14:12:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:2824: delete oid 11 current snap is 306 2024-09-18T14:12:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:2816: finishing rollback tid 1 to smithi08434663-7 2024-09-18T14:12:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:12:14.355 INFO:tasks.rados.rados.0.smithi084.stdout:2817: finishing copy_from racing read to smithi08434663-3 2024-09-18T14:12:14.356 INFO:tasks.rados.rados.0.smithi084.stdout:2819: finishing write tid 1 to smithi08434663-19 2024-09-18T14:12:14.356 INFO:tasks.rados.rados.0.smithi084.stdout:2819: finishing write tid 2 to smithi08434663-19 2024-09-18T14:12:14.356 INFO:tasks.rados.rados.0.smithi084.stdout:2819: finishing write tid 3 to smithi08434663-19 2024-09-18T14:12:14.356 INFO:tasks.rados.rados.0.smithi084.stdout:2817: finishing copy_from to smithi08434663-3 2024-09-18T14:12:14.356 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 1013 (ObjNum 857 snap 257 seq_num 857) dirty exists 2024-09-18T14:12:14.356 INFO:tasks.rados.rados.0.smithi084.stdout:2818: finishing copy_from racing read to smithi08434663-48 2024-09-18T14:12:14.356 INFO:tasks.rados.rados.0.smithi084.stdout:2819: finishing write tid 4 to smithi08434663-19 2024-09-18T14:12:14.356 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1507 (ObjNum 960 snap 306 seq_num 960) dirty exists 2024-09-18T14:12:14.356 INFO:tasks.rados.rados.0.smithi084.stdout:2819: left oid 19 (ObjNum 960 snap 306 seq_num 960) 2024-09-18T14:12:14.356 INFO:tasks.rados.rados.0.smithi084.stdout:2818: finishing copy_from to smithi08434663-48 2024-09-18T14:12:14.356 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1254 (ObjNum 935 snap 296 seq_num 935) dirty exists 2024-09-18T14:12:14.356 INFO:tasks.rados.rados.0.smithi084.stdout:2820: expect (ObjNum 907 snap 281 seq_num 907) 2024-09-18T14:12:14.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.346+0000 7f922b59e640 1 Processor -- start 2024-09-18T14:12:14.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.346+0000 7f922b59e640 1 -- start start 2024-09-18T14:12:14.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.346+0000 7f922b59e640 10 monclient: init 2024-09-18T14:12:14.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.346+0000 7f922b59e640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f922b59e640 10 monclient: _add_conns ranks=[0,6,2,7,1,8,5,4,3] 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f922b59e640 10 monclient(hunting): picked mon.noname-a con 0x7f922c09c9d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f922b59e640 10 monclient(hunting): picked mon.noname-d con 0x7f922c09d4b0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f922b59e640 10 monclient(hunting): picked mon.noname-b con 0x7f922c139140 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f922b59e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c05ed70 con 0x7f922c09c9d0 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f922b59e640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c0974a0 con 0x7f922c139140 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f922b59e640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c096ee0 con 0x7f922c09d4b0 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f922b59e640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f922b59e640 10 monclient(hunting): authenticate will time out at 1549.686137s 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f922a59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f922c09c9d0 0x7f922c09cda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59934/0 (socket says 172.21.15.45:59934) 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9223fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f922c09d4b0 0x7f922c137a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40376/0 (socket says 172.21.15.45:40376) 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f922a59c640 1 -- 172.21.15.45:0/3622589545 learned_addr learned my addr 172.21.15.45:0/3622589545 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947517367 0 0) 0x7f922c05ed70 con 0x7f922c09c9d0 2024-09-18T14:12:14.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient(hunting): my global_id is 5841 2024-09-18T14:12:14.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92180034e0 con 0x7f922c09c9d0 2024-09-18T14:12:14.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052804211 0 0) 0x7f922c096ee0 con 0x7f922c09d4b0 2024-09-18T14:12:14.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient(hunting): my global_id is 184389 2024-09-18T14:12:14.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f922c05ed70 con 0x7f922c09d4b0 2024-09-18T14:12:14.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060005539 0 0) 0x7f922c0974a0 con 0x7f922c139140 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient(hunting): my global_id is 145118 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f922c096ee0 con 0x7f922c139140 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642334363 0 0) 0x7f922c05ed70 con 0x7f922c09d4b0 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 >> v1:172.21.15.45:6790/0 conn(0x7f922c139140 legacy=0x7f922c13b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 >> v1:172.21.15.45:6789/0 conn(0x7f922c09c9d0 legacy=0x7f922c09cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 1 monclient: found mon.noname-d 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c0971c0 con 0x7f922c09d4b0 2024-09-18T14:12:14.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9214001810 con 0x7f922c09d4b0 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.347+0000 7f9229d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f9229d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f9229d9b640 10 monclient: dump: 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f922b59e640 5 monclient: authenticate success, global_id 184389 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f922b59e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9214001970 con 0x7f922c09d4b0 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f9229d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f9229d9b640 1 -- 172.21.15.45:0/3622589545 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9214004ae0 con 0x7f922c09d4b0 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f922b59e640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f922b59e640 4 set_mon_vals no callback set 2024-09-18T14:12:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f9229d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f9229d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f9229d9b640 10 monclient: dump: 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f922b59e640 10 monclient: shutdown 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f922b59e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.348+0000 7f922b59e640 1 -- 172.21.15.45:0/3622589545 >> v1:172.21.15.45:6792/0 conn(0x7f922c09d4b0 legacy=0x7f922c137a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.349+0000 7f922b59e640 1 -- 172.21.15.45:0/3622589545 shutdown_connections 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.349+0000 7f922b59e640 1 -- 172.21.15.45:0/3622589545 >> 172.21.15.45:0/3622589545 conn(0x7f922c0a2650 msgr2=0x7f922c09ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.349+0000 7f922b59e640 1 -- 172.21.15.45:0/3622589545 shutdown_connections 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.349+0000 7f922b59e640 1 -- 172.21.15.45:0/3622589545 wait complete. 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.349+0000 7f922b59e640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.349+0000 7f922b59e640 10 monclient: monmap: 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.369 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.349+0000 7f922b59e640 1 Processor -- start 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.349+0000 7f922b59e640 1 -- start start 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.349+0000 7f922b59e640 10 monclient: init 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f922b59e640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f922b59e640 10 monclient: _add_conns ranks=[5,4,1,3,0,6,8,7,2] 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f922b59e640 10 monclient(hunting): picked mon.noname-h con 0x7f922c09c9d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f922b59e640 10 monclient(hunting): picked mon.noname-c con 0x7f922c09d4b0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f922b59e640 10 monclient(hunting): picked mon.noname-f con 0x7f922c139140 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f922b59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c098340 con 0x7f922c139140 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f922b59e640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c11bc70 con 0x7f922c09d4b0 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f922b59e640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c09e570 con 0x7f922c09c9d0 2024-09-18T14:12:14.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f922b59e640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f922b59e640 10 monclient(hunting): authenticate will time out at 1549.689514s 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f9223fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f922c09d4b0 0x7f922c146850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:55788/0 (socket says 172.21.15.45:55788) 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f9223fff640 1 -- 172.21.15.45:0/2235200983 learned_addr learned my addr 172.21.15.45:0/2235200983 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929272246 0 0) 0x7f922c11bc70 con 0x7f922c09d4b0 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 10 monclient(hunting): my global_id is 164587 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f920c0034e0 con 0x7f922c09d4b0 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291924007 0 0) 0x7f922c09e570 con 0x7f922c09c9d0 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 10 monclient(hunting): my global_id is 184343 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f922c11bc70 con 0x7f922c09c9d0 2024-09-18T14:12:14.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688056863 0 0) 0x7f922c098340 con 0x7f922c139140 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 10 monclient(hunting): my global_id is 204130 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.350+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f922c09e570 con 0x7f922c139140 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 19279895 0 0) 0x7f920c0034e0 con 0x7f922c09d4b0 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f922c098340 con 0x7f922c09d4b0 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9214001410 con 0x7f922c09d4b0 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 10 monclient(hunting): dump: 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.372 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:14.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1395706692 0 0) 0x7f922c098340 con 0x7f922c09d4b0 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 >> v1:172.21.15.84:6791/0 conn(0x7f922c09c9d0 legacy=0x7f922c141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 >> v1:172.21.15.84:6789/0 conn(0x7f922c139140 legacy=0x7f922c149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 1 monclient: found mon.c 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c142410 con 0x7f922c09d4b0 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f922b59e640 5 monclient: authenticate success, global_id 164587 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f922b59e640 10 monclient: _renew_subs 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f922b59e640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f922b59e640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c11b9f0 con 0x7f922c09d4b0 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f922b59e640 10 monclient: _renew_subs 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f922b59e640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f922b59e640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c131e20 con 0x7f922c09d4b0 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9214004d60 con 0x7f922c09d4b0 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9214005210 con 0x7f922c09d4b0 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f92227fc640 10 monclient: dump: 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.351+0000 7f922959a640 4 set_mon_vals no callback set 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.352+0000 7f922b59e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.352+0000 7f922b59e640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f922c0663f0 con 0x7f922c11aa30 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.352+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f9214005370 con 0x7f922c09d4b0 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.352+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(626..626 src has 424..626) v4 ==== 3923+0+0 (unknown 1148901446 0 0) 0x7f9214046c80 con 0x7f922c09d4b0 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.352+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941717797 0 0) 0x7f922c0663f0 con 0x7f922c11aa30 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.352+0000 7f92227fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.352+0000 7f92227fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.352+0000 7f92227fc640 10 monclient(hunting): my global_id is 184398 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.352+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f922c098340 con 0x7f922c11aa30 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.352+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3169403316 0 0) 0x7f922c098340 con 0x7f922c11aa30 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.353+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f922c0663f0 con 0x7f922c11aa30 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.353+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f921c001590 con 0x7f922c11aa30 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.353+0000 7f92227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.353+0000 7f92227fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.353+0000 7f92227fc640 10 monclient: dump: 2024-09-18T14:12:14.378 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.378 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.378 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.378 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.378 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.378 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.378 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.378 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.353+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3873115810 0 0) 0x7f922c0663f0 con 0x7f922c11aa30 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.353+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f922c119f10 con 0x7f922c11aa30 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.353+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f921c0047a0 con 0x7f922c11aa30 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.353+0000 7f92227fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.353+0000 7f92227fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.353+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 >> v1:172.21.15.45:6792/0 conn(0x7f922c11aa30 legacy=0x7f922c09a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.353+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 >> v1:172.21.15.45:6792/0 conn(0x7f922c11aa30 legacy=0x7f922c09a580 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.354+0000 7f922ad9d640 10 monclient: get_auth_request con 0x7f920c03a040 auth_method 0 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f922b59e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f922b59e640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f922c04c4a0 con 0x7f922c119650 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023835457 0 0) 0x7f922c04c4a0 con 0x7f922c119650 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 10 monclient(hunting): my global_id is 184407 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f922c0663f0 con 0x7f922c119650 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4226792407 0 0) 0x7f922c0663f0 con 0x7f922c119650 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f922c04c4a0 con 0x7f922c119650 2024-09-18T14:12:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f921c004f60 con 0x7f922c119650 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 10 monclient: dump: 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2459173651 0 0) 0x7f922c04c4a0 con 0x7f922c119650 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.359+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f922c11be20 con 0x7f922c119650 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 1548528718) 0x7f921c007160 con 0x7f922c119650 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f92227fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f92227fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 >> v1:172.21.15.45:6792/0 conn(0x7f922c119650 legacy=0x7f922c09ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f92227fc640 1 -- 172.21.15.45:0/2235200983 >> v1:172.21.15.45:6792/0 conn(0x7f922c119650 legacy=0x7f922c09ac90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:14.383 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:14.383 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.384 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.385 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:14.386 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.389 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.390 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.395 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:14.397 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.398 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:14.399 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.400 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.401 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.402 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:14.402 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:14.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f922b59e640 1 -- 172.21.15.45:0/2235200983 >> v1:172.21.15.84:6812/31478 conn(0x7f920c03a040 legacy=0x7f920c03c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f922b59e640 10 monclient: shutdown 2024-09-18T14:12:14.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f922b59e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f922b59e640 1 -- 172.21.15.45:0/2235200983 >> v1:172.21.15.45:6791/0 conn(0x7f922c09d4b0 legacy=0x7f922c146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f922ad9d640 1 -- 172.21.15.45:0/2235200983 reap_dead start 2024-09-18T14:12:14.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f922ad9d640 1 -- 172.21.15.45:0/2235200983 reap_dead start 2024-09-18T14:12:14.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f922b59e640 1 -- 172.21.15.45:0/2235200983 shutdown_connections 2024-09-18T14:12:14.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f922b59e640 1 -- 172.21.15.45:0/2235200983 >> 172.21.15.45:0/2235200983 conn(0x7f922c0a2650 msgr2=0x7f922c134cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f922b59e640 1 -- 172.21.15.45:0/2235200983 shutdown_connections 2024-09-18T14:12:14.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.360+0000 7f922b59e640 1 -- 172.21.15.45:0/2235200983 wait complete. 2024-09-18T14:12:14.404 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:14.404 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:12:14.405 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:12:14.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.502+0000 7fa81db9d640 10 monclient: get_monmap_and_config 2024-09-18T14:12:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.502+0000 7fa81db9d640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.502+0000 7fa81db9d640 10 monclient: monmap: 2024-09-18T14:12:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:14.504378+0000 2024-09-18T14:12:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:14.504378+0000 2024-09-18T14:12:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.505 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.505 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.505 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.505 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.505 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.505 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 1 Processor -- start 2024-09-18T14:12:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 1 -- start start 2024-09-18T14:12:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 10 monclient: init 2024-09-18T14:12:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 10 monclient: _add_conns ranks=[4,2,1,7,8,3,5,6,0] 2024-09-18T14:12:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 10 monclient(hunting): picked mon.noname-c con 0x7fa81812d4b0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 10 monclient(hunting): picked mon.noname-b con 0x7fa81809f7a0 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 10 monclient(hunting): picked mon.noname-f con 0x7fa818136d40 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81805ed70 con 0x7fa818136d40 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81806d240 con 0x7fa81809f7a0 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8180a1650 con 0x7fa81812d4b0 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa81db9d640 10 monclient(hunting): authenticate will time out at 1549.842739s 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa8177fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fa81812d4b0 0x7fa81812d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:55792/0 (socket says 172.21.15.45:55792) 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa8177fe640 1 -- 172.21.15.45:0/3434215985 learned_addr learned my addr 172.21.15.45:0/3434215985 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.503+0000 7fa816ffd640 1 --1- 172.21.15.45:0/3434215985 >> v1:172.21.15.45:6790/0 conn(0x7fa81809f7a0 0x7fa818135620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55840/0 (socket says 172.21.15.45:55840) 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847304194 0 0) 0x7fa8180a1650 con 0x7fa81812d4b0 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient(hunting): my global_id is 164596 2024-09-18T14:12:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8000034e0 con 0x7fa81812d4b0 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057076862 0 0) 0x7fa81806d240 con 0x7fa81809f7a0 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient(hunting): my global_id is 145127 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8180a1650 con 0x7fa81809f7a0 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476283137 0 0) 0x7fa81805ed70 con 0x7fa818136d40 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient(hunting): my global_id is 204139 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa81806d240 con 0x7fa818136d40 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1155959514 0 0) 0x7fa8180a1650 con 0x7fa81809f7a0 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 >> v1:172.21.15.45:6791/0 conn(0x7fa81812d4b0 legacy=0x7fa81812d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 >> v1:172.21.15.84:6789/0 conn(0x7fa818136d40 legacy=0x7fa818139120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 monclient: found mon.noname-b 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81812df90 con 0x7fa81809f7a0 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa80c001810 con 0x7fa81809f7a0 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient: dump: 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa81db9d640 5 monclient: authenticate success, global_id 145127 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa81db9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa80c0019f0 con 0x7fa81809f7a0 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 1 -- 172.21.15.45:0/3434215985 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa80c004ae0 con 0x7fa81809f7a0 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa81db9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa81db9d640 4 set_mon_vals no callback set 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa8167fc640 10 monclient: dump: 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa81db9d640 10 monclient: shutdown 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa81db9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.504+0000 7fa81db9d640 1 -- 172.21.15.45:0/3434215985 >> v1:172.21.15.45:6790/0 conn(0x7fa81809f7a0 legacy=0x7fa818135620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.505+0000 7fa81db9d640 1 -- 172.21.15.45:0/3434215985 shutdown_connections 2024-09-18T14:12:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.505+0000 7fa81db9d640 1 -- 172.21.15.45:0/3434215985 >> 172.21.15.45:0/3434215985 conn(0x7fa8180a19a0 msgr2=0x7fa8180a1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.505+0000 7fa81db9d640 1 -- 172.21.15.45:0/3434215985 shutdown_connections 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.505+0000 7fa81db9d640 1 -- 172.21.15.45:0/3434215985 wait complete. 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.505+0000 7fa81db9d640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.505+0000 7fa81db9d640 10 monclient: monmap: 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.505+0000 7fa81db9d640 1 Processor -- start 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa81db9d640 1 -- start start 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa81db9d640 10 monclient: init 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa81db9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa81db9d640 10 monclient: _add_conns ranks=[7,0,4,6,2,5,3,8,1] 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa81db9d640 10 monclient(hunting): picked mon.noname-i con 0x7fa81809f7a0 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa81db9d640 10 monclient(hunting): picked mon.noname-a con 0x7fa81812d4b0 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa81db9d640 10 monclient(hunting): picked mon.noname-c con 0x7fa818136d40 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa81db9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa818133a00 con 0x7fa81812d4b0 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa81db9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81806b990 con 0x7fa818136d40 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa81db9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81806b000 con 0x7fa81809f7a0 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa81db9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa81db9d640 10 monclient(hunting): authenticate will time out at 1549.845494s 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa817fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fa818136d40 0x7fa81814a210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:55798/0 (socket says 172.21.15.45:55798) 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa817fff640 1 -- 172.21.15.45:0/3792444389 learned_addr learned my addr 172.21.15.45:0/3792444389 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335331839 0 0) 0x7fa81806b990 con 0x7fa818136d40 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 10 monclient(hunting): my global_id is 164605 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8040034e0 con 0x7fa818136d40 2024-09-18T14:12:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520653795 0 0) 0x7fa818133a00 con 0x7fa81812d4b0 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 10 monclient(hunting): my global_id is 5850 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa81806b990 con 0x7fa81812d4b0 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585399762 0 0) 0x7fa81806b000 con 0x7fa81809f7a0 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 10 monclient(hunting): my global_id is 184363 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.506+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa818133a00 con 0x7fa81809f7a0 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1317582033 0 0) 0x7fa8040034e0 con 0x7fa818136d40 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa81806b000 con 0x7fa818136d40 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa810002e50 con 0x7fa818136d40 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:12:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient(hunting): dump: 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1108854377 0 0) 0x7fa81806b990 con 0x7fa81812d4b0 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa8040034e0 con 0x7fa81812d4b0 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3162832797 0 0) 0x7fa818133a00 con 0x7fa81809f7a0 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa81806b990 con 0x7fa81809f7a0 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa80c001410 con 0x7fa81812d4b0 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient(hunting): dump: 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.525 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa8080015b0 con 0x7fa81809f7a0 2024-09-18T14:12:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:12:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient(hunting): dump: 2024-09-18T14:12:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.527 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1272288357 0 0) 0x7fa81806b000 con 0x7fa818136d40 2024-09-18T14:12:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 >> v1:172.21.15.84:6792/0 conn(0x7fa81809f7a0 legacy=0x7fa81806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 >> v1:172.21.15.45:6789/0 conn(0x7fa81812d4b0 legacy=0x7fa81806a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 monclient: found mon.c 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81806b550 con 0x7fa818136d40 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa81db9d640 5 monclient: authenticate success, global_id 164605 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa81db9d640 10 monclient: _renew_subs 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa81db9d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa81db9d640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81809be30 con 0x7fa818136d40 2024-09-18T14:12:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa81db9d640 10 monclient: _renew_subs 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa81db9d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa81db9d640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81809afe0 con 0x7fa818136d40 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa810003110 con 0x7fa818136d40 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa810001d00 con 0x7fa818136d40 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa815ffb640 4 set_mon_vals no callback set 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.507+0000 7fa7ff7fe640 10 monclient: dump: 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa81db9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa81db9d640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa7f8006580 con 0x7fa7f8002830 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fa810001e60 con 0x7fa818136d40 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(626..626 src has 424..626) v4 ==== 3923+0+0 (unknown 1148901446 0 0) 0x7fa810046e40 con 0x7fa818136d40 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027617275 0 0) 0x7fa7f8006580 con 0x7fa7f8002830 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 10 monclient(hunting): my global_id is 5678 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa81806b000 con 0x7fa7f8002830 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa816ffd640 10 monclient: get_auth_request con 0x7fa80403a7d0 auth_method 0 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3216396030 0 0) 0x7fa81806b000 con 0x7fa7f8002830 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa7f8006580 con 0x7fa7f8002830 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa808004d90 con 0x7fa7f8002830 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.508+0000 7fa7ff7fe640 10 monclient: dump: 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.533 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.509+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2086332062 0 0) 0x7fa7f8006580 con 0x7fa7f8002830 2024-09-18T14:12:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.509+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7f800a0a0 con 0x7fa7f8002830 2024-09-18T14:12:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.509+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa808005130 con 0x7fa7f8002830 2024-09-18T14:12:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.509+0000 7fa7ff7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.509+0000 7fa7ff7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.509+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 >> v1:172.21.15.45:6793/0 conn(0x7fa7f8002830 legacy=0x7fa7f8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.509+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 >> v1:172.21.15.45:6793/0 conn(0x7fa7f8002830 legacy=0x7fa7f8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:14.534 INFO:tasks.rados.rados.0.smithi084.stdout:2821: finishing write tid 1 to smithi08434663-1 2024-09-18T14:12:14.534 INFO:tasks.rados.rados.0.smithi084.stdout:2816: done (8 left) 2024-09-18T14:12:14.535 INFO:tasks.rados.rados.0.smithi084.stdout:2817: done (7 left) 2024-09-18T14:12:14.535 INFO:tasks.rados.rados.0.smithi084.stdout:2818: done (6 left) 2024-09-18T14:12:14.535 INFO:tasks.rados.rados.0.smithi084.stdout:2819: done (5 left) 2024-09-18T14:12:14.535 INFO:tasks.rados.rados.0.smithi084.stdout:2820: done (4 left) 2024-09-18T14:12:14.535 INFO:tasks.rados.rados.0.smithi084.stdout:2824: done (3 left) 2024-09-18T14:12:14.535 INFO:tasks.rados.rados.0.smithi084.stdout:2825: write (excl) oid 7 current snap is 306 2024-09-18T14:12:14.535 INFO:tasks.rados.rados.0.smithi084.stdout:2823: expect (ObjNum 957 snap 304 seq_num 957) 2024-09-18T14:12:14.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa81db9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:14.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa81db9d640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa7f8007540 con 0x7fa7f80068f0 2024-09-18T14:12:14.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3670068194 0 0) 0x7fa7f8007540 con 0x7fa7f80068f0 2024-09-18T14:12:14.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 10 monclient(hunting): my global_id is 5687 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7f8006580 con 0x7fa7f80068f0 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 904644810 0 0) 0x7fa7f8006580 con 0x7fa7f80068f0 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa7f8007540 con 0x7fa7f80068f0 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa808007fc0 con 0x7fa7f80068f0 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 10 monclient: dump: 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.537 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 557625085 0 0) 0x7fa7f8007540 con 0x7fa7f80068f0 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.515+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa7f80087a0 con 0x7fa7f80068f0 2024-09-18T14:12:14.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7559 (unknown 0 0 2417706182) 0x7fa808008120 con 0x7fa7f80068f0 2024-09-18T14:12:14.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa7ff7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:14.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa7ff7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:14.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 >> v1:172.21.15.45:6793/0 conn(0x7fa7f80068f0 legacy=0x7fa7f8006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa7ff7fe640 1 -- 172.21.15.45:0/3792444389 >> v1:172.21.15.45:6793/0 conn(0x7fa7f80068f0 legacy=0x7fa7f8006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:14.539 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:12:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:14.539 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:14.540 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.541 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:14.542 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.543 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.544 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.545 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.546 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:14.549 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.550 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.552 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.553 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.554 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.556 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.557 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:12:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.558 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:14.558 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:14.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa81db9d640 1 -- 172.21.15.45:0/3792444389 >> v1:172.21.15.84:6812/31478 conn(0x7fa80403a7d0 legacy=0x7fa80403cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa81db9d640 10 monclient: shutdown 2024-09-18T14:12:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa81db9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa81db9d640 1 -- 172.21.15.45:0/3792444389 >> v1:172.21.15.45:6791/0 conn(0x7fa818136d40 legacy=0x7fa81814a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa817fff640 1 -- 172.21.15.45:0/3792444389 reap_dead start 2024-09-18T14:12:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa817fff640 1 -- 172.21.15.45:0/3792444389 reap_dead start 2024-09-18T14:12:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa81db9d640 1 -- 172.21.15.45:0/3792444389 shutdown_connections 2024-09-18T14:12:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa81db9d640 1 -- 172.21.15.45:0/3792444389 >> 172.21.15.45:0/3792444389 conn(0x7fa8180a19a0 msgr2=0x7fa81809df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa81db9d640 1 -- 172.21.15.45:0/3792444389 shutdown_connections 2024-09-18T14:12:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.516+0000 7fa81db9d640 1 -- 172.21.15.45:0/3792444389 wait complete. 2024-09-18T14:12:14.560 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:14.561 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:12:14.561 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:12:14.633 INFO:tasks.rados.rados.0.smithi084.stdout:2821: finishing write tid 2 to smithi08434663-1 2024-09-18T14:12:14.633 INFO:tasks.rados.rados.0.smithi084.stdout:2821: finishing write tid 3 to smithi08434663-1 2024-09-18T14:12:14.633 INFO:tasks.rados.rados.0.smithi084.stdout:2821: finishing write tid 4 to smithi08434663-1 2024-09-18T14:12:14.634 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 1226 (ObjNum 961 snap 306 seq_num 961) dirty exists 2024-09-18T14:12:14.634 INFO:tasks.rados.rados.0.smithi084.stdout:2821: left oid 1 (ObjNum 961 snap 306 seq_num 961) 2024-09-18T14:12:14.634 INFO:tasks.rados.rados.0.smithi084.stdout:2825: seq_num 963 ranges {585141=725265,1919336=265805} 2024-09-18T14:12:14.640 INFO:tasks.rados.rados.0.smithi084.stdout:2825: writing smithi08434663-7 from 585141 to 1310406 tid 1 2024-09-18T14:12:14.645 INFO:tasks.rados.rados.0.smithi084.stdout:2825: writing smithi08434663-7 from 1919336 to 2185141 tid 2 2024-09-18T14:12:14.645 INFO:tasks.rados.rados.0.smithi084.stdout:2821: done (3 left) 2024-09-18T14:12:14.645 INFO:tasks.rados.rados.0.smithi084.stdout:2822: done (2 left) 2024-09-18T14:12:14.645 INFO:tasks.rados.rados.0.smithi084.stdout:2823: done (1 left) 2024-09-18T14:12:14.645 INFO:tasks.rados.rados.0.smithi084.stdout:2826: delete oid 47 current snap is 306 2024-09-18T14:12:14.646 INFO:tasks.rados.rados.0.smithi084.stdout:2825: finishing write tid 1 to smithi08434663-7 2024-09-18T14:12:14.647 INFO:tasks.rados.rados.0.smithi084.stdout:2826: done (1 left) 2024-09-18T14:12:14.647 INFO:tasks.rados.rados.0.smithi084.stdout:2827: write (excl) oid 17 current snap is 306 2024-09-18T14:12:14.647 INFO:tasks.rados.rados.0.smithi084.stdout:2825: finishing write tid 2 to smithi08434663-7 2024-09-18T14:12:14.647 INFO:tasks.rados.rados.0.smithi084.stdout:2827: seq_num 965 ranges {618755=475763,1636300=457996} 2024-09-18T14:12:14.653 INFO:tasks.rados.rados.0.smithi084.stdout:2827: writing smithi08434663-17 from 618755 to 1094518 tid 1 2024-09-18T14:12:14.658 INFO:tasks.rados.rados.0.smithi084.stdout:2827: writing smithi08434663-17 from 1636300 to 2094296 tid 2 2024-09-18T14:12:14.658 INFO:tasks.rados.rados.0.smithi084.stdout:2828: snap_remove snap 299 2024-09-18T14:12:14.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.659+0000 7f757084f640 10 monclient: get_monmap_and_config 2024-09-18T14:12:14.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.659+0000 7f757084f640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.659+0000 7f757084f640 10 monclient: monmap: 2024-09-18T14:12:14.661 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.661 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.661 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:14.661836+0000 2024-09-18T14:12:14.661 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:14.661836+0000 2024-09-18T14:12:14.661 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.661 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.661 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.661 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.662 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.662 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.662 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.662 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.662 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.662 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.662 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.662 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.660+0000 7f757084f640 1 Processor -- start 2024-09-18T14:12:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.660+0000 7f757084f640 1 -- start start 2024-09-18T14:12:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.660+0000 7f757084f640 10 monclient: init 2024-09-18T14:12:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f757084f640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f757084f640 10 monclient: _add_conns ranks=[3,7,0,4,5,6,2,1,8] 2024-09-18T14:12:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f757084f640 10 monclient(hunting): picked mon.noname-g con 0x7f756c0a2610 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f757084f640 10 monclient(hunting): picked mon.noname-i con 0x7f756c1330b0 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f757084f640 10 monclient(hunting): picked mon.noname-a con 0x7f756c096a20 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f757084f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756c05ed70 con 0x7f756c096a20 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f757084f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756c0a1740 con 0x7f756c0a2610 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f757084f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756c0a29e0 con 0x7f756c1330b0 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f757084f640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f757084f640 10 monclient(hunting): authenticate will time out at 1550.000698s 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f756ad9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f756c096a20 0x7f756c13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59958/0 (socket says 172.21.15.45:59958) 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f756ad9d640 1 -- 172.21.15.45:0/1645287048 learned_addr learned my addr 172.21.15.45:0/1645287048 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f756a59c640 1 --1- 172.21.15.45:0/1645287048 >> v1:172.21.15.84:6790/0 conn(0x7f756c0a2610 0x7f756c1329a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:39268/0 (socket says 172.21.15.45:39268) 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.661+0000 7f7569d9b640 1 --1- 172.21.15.45:0/1645287048 >> v1:172.21.15.84:6792/0 conn(0x7f756c1330b0 0x7f756c137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:33722/0 (socket says 172.21.15.45:33722) 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211531217 0 0) 0x7f756c0a1740 con 0x7f756c0a2610 2024-09-18T14:12:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient(hunting): my global_id is 184341 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75540034e0 con 0x7f756c0a2610 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615781096 0 0) 0x7f756c05ed70 con 0x7f756c096a20 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient(hunting): my global_id is 5859 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f756c0a1740 con 0x7f756c096a20 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86634773 0 0) 0x7f756c0a29e0 con 0x7f756c1330b0 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient(hunting): my global_id is 184372 2024-09-18T14:12:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f756c05ed70 con 0x7f756c1330b0 2024-09-18T14:12:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3420761292 0 0) 0x7f756c0a1740 con 0x7f756c096a20 2024-09-18T14:12:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 >> v1:172.21.15.84:6792/0 conn(0x7f756c1330b0 legacy=0x7f756c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 >> v1:172.21.15.84:6790/0 conn(0x7f756c0a2610 legacy=0x7f756c1329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 monclient: found mon.noname-a 2024-09-18T14:12:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:12:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c0970a0 con 0x7f756c096a20 2024-09-18T14:12:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7564001b10 con 0x7f756c096a20 2024-09-18T14:12:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f757084f640 5 monclient: authenticate success, global_id 5859 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f757084f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient: dump: 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f757084f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7564001c70 con 0x7f756c096a20 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 1 -- 172.21.15.45:0/1645287048 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7564004be0 con 0x7f756c096a20 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f757084f640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f757084f640 4 set_mon_vals no callback set 2024-09-18T14:12:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.662+0000 7f756959a640 10 monclient: dump: 2024-09-18T14:12:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.663+0000 7f757084f640 10 monclient: shutdown 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.663+0000 7f757084f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.663+0000 7f757084f640 1 -- 172.21.15.45:0/1645287048 >> v1:172.21.15.45:6789/0 conn(0x7f756c096a20 legacy=0x7f756c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.663+0000 7f757084f640 1 -- 172.21.15.45:0/1645287048 shutdown_connections 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.663+0000 7f757084f640 1 -- 172.21.15.45:0/1645287048 >> 172.21.15.45:0/1645287048 conn(0x7f756c0630f0 msgr2=0x7f756c0634f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.663+0000 7f757084f640 1 -- 172.21.15.45:0/1645287048 shutdown_connections 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.663+0000 7f757084f640 1 -- 172.21.15.45:0/1645287048 wait complete. 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.663+0000 7f757084f640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.663+0000 7f757084f640 10 monclient: monmap: 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 1 Processor -- start 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 1 -- start start 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 10 monclient: init 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 10 monclient: _add_conns ranks=[2,7,8,3,6,4,5,1,0] 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 10 monclient(hunting): picked mon.noname-b con 0x7f756c096a20 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 10 monclient(hunting): picked mon.noname-i con 0x7f756c0a2610 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 10 monclient(hunting): picked mon.noname-e con 0x7f756c1330b0 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756c0a2350 con 0x7f756c096a20 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756c11d430 con 0x7f756c0a2610 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756c11d150 con 0x7f756c1330b0 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f757084f640 10 monclient(hunting): authenticate will time out at 1550.003528s 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f756a59c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f756c096a20 0x7f756c141b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55848/0 (socket says 172.21.15.45:55848) 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f756a59c640 1 -- 172.21.15.45:0/1534766501 learned_addr learned my addr 172.21.15.45:0/1534766501 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503983299 0 0) 0x7f756c0a2350 con 0x7f756c096a20 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 10 monclient(hunting): my global_id is 145136 2024-09-18T14:12:14.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f754c0034e0 con 0x7f756c096a20 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141654605 0 0) 0x7f756c11d150 con 0x7f756c1330b0 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 10 monclient(hunting): my global_id is 5696 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f756c0a2350 con 0x7f756c1330b0 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490564004 0 0) 0x7f756c11d430 con 0x7f756c0a2610 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 10 monclient(hunting): my global_id is 184381 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.664+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f756c11d150 con 0x7f756c0a2610 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1345239650 0 0) 0x7f754c0034e0 con 0x7f756c096a20 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f756c11d430 con 0x7f756c096a20 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f755c001520 con 0x7f756c096a20 2024-09-18T14:12:14.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.682 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 549265946 0 0) 0x7f756c11d150 con 0x7f756c0a2610 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f754c0034e0 con 0x7f756c0a2610 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2296345542 0 0) 0x7f756c0a2350 con 0x7f756c1330b0 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f756c11d150 con 0x7f756c1330b0 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7560001840 con 0x7f756c0a2610 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:12:14.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7564001a50 con 0x7f756c1330b0 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.685 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2662064368 0 0) 0x7f756c11d430 con 0x7f756c096a20 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 >> v1:172.21.15.45:6793/0 conn(0x7f756c1330b0 legacy=0x7f756c14a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 >> v1:172.21.15.84:6792/0 conn(0x7f756c0a2610 legacy=0x7f756c146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 monclient: found mon.b 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c09f9d0 con 0x7f756c096a20 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f757084f640 5 monclient: authenticate success, global_id 145136 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f757084f640 10 monclient: _renew_subs 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f757084f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f757084f640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c11e400 con 0x7f756c096a20 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f757084f640 10 monclient: _renew_subs 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f757084f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f757084f640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c09d9d0 con 0x7f756c096a20 2024-09-18T14:12:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f755c004a50 con 0x7f756c096a20 2024-09-18T14:12:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f755c0017a0 con 0x7f756c096a20 2024-09-18T14:12:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f755a7fc640 10 monclient: dump: 2024-09-18T14:12:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.665+0000 7f7568d99640 4 set_mon_vals no callback set 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.666+0000 7f757084f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.666+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f755c00fdf0 con 0x7f756c096a20 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.666+0000 7f757084f640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f756c04c4a0 con 0x7f756c142210 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.666+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(626..626 src has 424..626) v4 ==== 3923+0+0 (unknown 1148901446 0 0) 0x7f755c046ba0 con 0x7f756c096a20 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.666+0000 7f756ad9d640 10 monclient: get_auth_request con 0x7f754c03aa70 auth_method 0 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.666+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948221401 0 0) 0x7f756c04c4a0 con 0x7f756c142210 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.666+0000 7f755a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.666+0000 7f755a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.666+0000 7f755a7fc640 10 monclient(hunting): my global_id is 204148 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.666+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f756c11d430 con 0x7f756c142210 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2767713017 0 0) 0x7f756c11d430 con 0x7f756c142210 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f756c04c4a0 con 0x7f756c142210 2024-09-18T14:12:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7560004cd0 con 0x7f756c142210 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 10 monclient: dump: 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2749986641 0 0) 0x7f756c04c4a0 con 0x7f756c142210 2024-09-18T14:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f756c09e3c0 con 0x7f756c142210 2024-09-18T14:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f75600050c0 con 0x7f756c142210 2024-09-18T14:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 >> v1:172.21.15.84:6789/0 conn(0x7f756c142210 legacy=0x7f756c099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.667+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 >> v1:172.21.15.84:6789/0 conn(0x7f756c142210 legacy=0x7f756c099e50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:14.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.673+0000 7f757084f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.673+0000 7f757084f640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f756c11d830 con 0x7f756c02a8d0 2024-09-18T14:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.673+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430025697 0 0) 0x7f756c11d830 con 0x7f756c02a8d0 2024-09-18T14:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.673+0000 7f755a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.673+0000 7f755a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.673+0000 7f755a7fc640 10 monclient(hunting): my global_id is 204157 2024-09-18T14:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.673+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f756c04c4a0 con 0x7f756c02a8d0 2024-09-18T14:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2265867295 0 0) 0x7f756c04c4a0 con 0x7f756c02a8d0 2024-09-18T14:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f756c11d830 con 0x7f756c02a8d0 2024-09-18T14:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7560005920 con 0x7f756c02a8d0 2024-09-18T14:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 10 monclient: dump: 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.696 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2101002901 0 0) 0x7f756c11d830 con 0x7f756c02a8d0 2024-09-18T14:12:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f756c04b720 con 0x7f756c02a8d0 2024-09-18T14:12:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 1247679270) 0x7f75600080b0 con 0x7f756c02a8d0 2024-09-18T14:12:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 >> v1:172.21.15.84:6789/0 conn(0x7f756c02a8d0 legacy=0x7f756c09a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f755a7fc640 1 -- 172.21.15.45:0/1534766501 >> v1:172.21.15.84:6789/0 conn(0x7f756c02a8d0 legacy=0x7f756c09a560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:14.696 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:14.700 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.701 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.702 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:14.703 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.704 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:14.705 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.707 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.708 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.710 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:14.711 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.712 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.713 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:14.716 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:14.717 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:14.718 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.720 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:14.720 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:14.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f757084f640 1 -- 172.21.15.45:0/1534766501 >> v1:172.21.15.84:6812/31478 conn(0x7f754c03aa70 legacy=0x7f754c03cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f757084f640 10 monclient: shutdown 2024-09-18T14:12:14.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f757084f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f757084f640 1 -- 172.21.15.45:0/1534766501 >> v1:172.21.15.45:6790/0 conn(0x7f756c096a20 legacy=0x7f756c141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f756ad9d640 1 -- 172.21.15.45:0/1534766501 reap_dead start 2024-09-18T14:12:14.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f756ad9d640 1 -- 172.21.15.45:0/1534766501 reap_dead start 2024-09-18T14:12:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f757084f640 1 -- 172.21.15.45:0/1534766501 shutdown_connections 2024-09-18T14:12:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f757084f640 1 -- 172.21.15.45:0/1534766501 >> 172.21.15.45:0/1534766501 conn(0x7f756c0630f0 msgr2=0x7f756c0a0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f757084f640 1 -- 172.21.15.45:0/1534766501 shutdown_connections 2024-09-18T14:12:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.674+0000 7f757084f640 1 -- 172.21.15.45:0/1534766501 wait complete. 2024-09-18T14:12:14.722 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:14.722 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:12:14.722 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:12:14.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.818+0000 7f10c379d640 10 monclient: get_monmap_and_config 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.818+0000 7f10c379d640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.818+0000 7f10c379d640 10 monclient: monmap: 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:14.820004+0000 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:14.820004+0000 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.820 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.821 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.818+0000 7f10c379d640 1 Processor -- start 2024-09-18T14:12:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.818+0000 7f10c379d640 1 -- start start 2024-09-18T14:12:14.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.818+0000 7f10c379d640 10 monclient: init 2024-09-18T14:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10c379d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10c379d640 10 monclient: _add_conns ranks=[7,4,3,8,5,1,0,6,2] 2024-09-18T14:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10c379d640 10 monclient(hunting): picked mon.noname-i con 0x7f10bc06e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10c379d640 10 monclient(hunting): picked mon.noname-c con 0x7f10bc0a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10c379d640 10 monclient(hunting): picked mon.noname-g con 0x7f10bc0a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10c379d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc05ed70 con 0x7f10bc0a1080 2024-09-18T14:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10c379d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc097620 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10c379d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc097340 con 0x7f10bc06e810 2024-09-18T14:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10c379d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10c379d640 10 monclient(hunting): authenticate will time out at 1550.158383s 2024-09-18T14:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10c0d38640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f10bc0a0cb0 0x7f10bc137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:55814/0 (socket says 172.21.15.45:55814) 2024-09-18T14:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10c0d38640 1 -- 172.21.15.45:0/2515814774 learned_addr learned my addr 172.21.15.45:0/2515814774 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555395047 0 0) 0x7f10bc097620 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 10 monclient(hunting): my global_id is 164614 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10a80034e0 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321502105 0 0) 0x7f10bc05ed70 con 0x7f10bc0a1080 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 10 monclient(hunting): my global_id is 184350 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10bc097620 con 0x7f10bc0a1080 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917775337 0 0) 0x7f10bc097340 con 0x7f10bc06e810 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 10 monclient(hunting): my global_id is 184390 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10bc05ed70 con 0x7f10bc06e810 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1584344369 0 0) 0x7f10a80034e0 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.819+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 >> v1:172.21.15.84:6792/0 conn(0x7f10bc06e810 legacy=0x7f10bc0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 >> v1:172.21.15.84:6790/0 conn(0x7f10bc0a1080 legacy=0x7f10bc13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 1 monclient: found mon.noname-c 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc0a1660 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10c379d640 5 monclient: authenticate success, global_id 164614 2024-09-18T14:12:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10b0001830 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10c379d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 10 monclient: dump: 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10c379d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f10b0001a10 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10c379d640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 1 -- 172.21.15.45:0/2515814774 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10b0004ba0 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10c379d640 4 set_mon_vals no callback set 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10affff640 10 monclient: dump: 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10c379d640 10 monclient: shutdown 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10c379d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10c379d640 1 -- 172.21.15.45:0/2515814774 >> v1:172.21.15.45:6791/0 conn(0x7f10bc0a0cb0 legacy=0x7f10bc137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10c379d640 1 -- 172.21.15.45:0/2515814774 shutdown_connections 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10c379d640 1 -- 172.21.15.45:0/2515814774 >> 172.21.15.45:0/2515814774 conn(0x7f10bc0a5bb0 msgr2=0x7f10bc0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10c379d640 1 -- 172.21.15.45:0/2515814774 shutdown_connections 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.820+0000 7f10c379d640 1 -- 172.21.15.45:0/2515814774 wait complete. 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.821+0000 7f10c379d640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.821+0000 7f10c379d640 10 monclient: monmap: 2024-09-18T14:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.821+0000 7f10c379d640 1 Processor -- start 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.821+0000 7f10c379d640 1 -- start start 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.821+0000 7f10c379d640 10 monclient: init 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10c379d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10c379d640 10 monclient: _add_conns ranks=[6,7,5,0,4,8,2,1,3] 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10c379d640 10 monclient(hunting): picked mon.noname-d con 0x7f10bc06e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10c379d640 10 monclient(hunting): picked mon.noname-i con 0x7f10bc0a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10c379d640 10 monclient(hunting): picked mon.noname-h con 0x7f10bc0a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10c379d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc122dd0 con 0x7f10bc0a1080 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10c379d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc13dbc0 con 0x7f10bc06e810 2024-09-18T14:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10c379d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc122810 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10c379d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10c379d640 10 monclient(hunting): authenticate will time out at 1550.161238s 2024-09-18T14:12:14.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10c1539640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f10bc06e810 0x7f10bc13d490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40406/0 (socket says 172.21.15.45:40406) 2024-09-18T14:12:14.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10c1539640 1 -- 172.21.15.45:0/671776247 learned_addr learned my addr 172.21.15.45:0/671776247 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005156985 0 0) 0x7f10bc13dbc0 con 0x7f10bc06e810 2024-09-18T14:12:14.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): my global_id is 184416 2024-09-18T14:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10a00034e0 con 0x7f10bc06e810 2024-09-18T14:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308694062 0 0) 0x7f10bc122810 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): my global_id is 184399 2024-09-18T14:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10bc13dbc0 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057506288 0 0) 0x7f10bc122dd0 con 0x7f10bc0a1080 2024-09-18T14:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): my global_id is 184352 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10bc122810 con 0x7f10bc0a1080 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3241826028 0 0) 0x7f10a00034e0 con 0x7f10bc06e810 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10bc122dd0 con 0x7f10bc06e810 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10b8001b70 con 0x7f10bc06e810 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): dump: 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 39542303 0 0) 0x7f10bc13dbc0 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10a00034e0 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10b00018b0 con 0x7f10bc0a0cb0 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): dump: 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 461277620 0 0) 0x7f10bc122810 con 0x7f10bc0a1080 2024-09-18T14:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10bc13dbc0 con 0x7f10bc0a1080 2024-09-18T14:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10b4001c40 con 0x7f10bc0a1080 2024-09-18T14:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 10 monclient(hunting): dump: 2024-09-18T14:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3861375551 0 0) 0x7f10bc122dd0 con 0x7f10bc06e810 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 >> v1:172.21.15.84:6792/0 conn(0x7f10bc0a0cb0 legacy=0x7f10bc1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.822+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 >> v1:172.21.15.84:6791/0 conn(0x7f10bc0a1080 legacy=0x7f10bc149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 1 monclient: found mon.d 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc1202d0 con 0x7f10bc06e810 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10c379d640 5 monclient: authenticate success, global_id 184416 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10c379d640 10 monclient: _renew_subs 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10c379d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10c379d640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc122530 con 0x7f10bc06e810 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10c379d640 10 monclient: _renew_subs 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10c379d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10c379d640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc121660 con 0x7f10bc06e810 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f10b8002000 con 0x7f10bc06e810 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10b80025b0 con 0x7f10bc06e810 2024-09-18T14:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10af7fe640 4 set_mon_vals no callback set 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 10 monclient: dump: 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.849 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.849 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.849 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.849 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.849 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10c379d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:14.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10c379d640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f109c006580 con 0x7f109c002830 2024-09-18T14:12:14.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.823+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f10b8002790 con 0x7f10bc06e810 2024-09-18T14:12:14.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(626..626 src has 424..626) v4 ==== 3923+0+0 (unknown 1148901446 0 0) 0x7f10b8046b20 con 0x7f10bc06e810 2024-09-18T14:12:14.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239292696 0 0) 0x7f109c006580 con 0x7f109c002830 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10c1d3a640 10 monclient: get_auth_request con 0x7f10a003aa70 auth_method 0 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 10 monclient(hunting): my global_id is 184359 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10bc122dd0 con 0x7f109c002830 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3160521127 0 0) 0x7f10bc122dd0 con 0x7f109c002830 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f109c006580 con 0x7f109c002830 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10b0001270 con 0x7f109c002830 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 10 monclient: dump: 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.850 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2657433889 0 0) 0x7f109c006580 con 0x7f109c002830 2024-09-18T14:12:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.824+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f109c00a0a0 con 0x7f109c002830 2024-09-18T14:12:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.825+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f10b00051a0 con 0x7f109c002830 2024-09-18T14:12:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.825+0000 7f10ad7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.825+0000 7f10ad7fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.825+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 >> v1:172.21.15.84:6790/0 conn(0x7f109c002830 legacy=0x7f109c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.825+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 >> v1:172.21.15.84:6790/0 conn(0x7f109c002830 legacy=0x7f109c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.830+0000 7f10c379d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.830+0000 7f10c379d640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f109c009610 con 0x7f109c008ff0 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965443571 0 0) 0x7f109c009610 con 0x7f109c008ff0 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 10 monclient(hunting): my global_id is 184368 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f109c006580 con 0x7f109c008ff0 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1207564821 0 0) 0x7f109c006580 con 0x7f109c008ff0 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f109c009610 con 0x7f109c008ff0 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f10b00059e0 con 0x7f109c008ff0 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 10 monclient: dump: 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 282278463 0 0) 0x7f109c009610 con 0x7f109c008ff0 2024-09-18T14:12:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.831+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f109c0098d0 con 0x7f109c008ff0 2024-09-18T14:12:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 3196865610) 0x7f10b0008190 con 0x7f109c008ff0 2024-09-18T14:12:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10ad7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10ad7fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 >> v1:172.21.15.84:6790/0 conn(0x7f109c008ff0 legacy=0x7f109c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10ad7fa640 1 -- 172.21.15.45:0/671776247 >> v1:172.21.15.84:6790/0 conn(0x7f109c008ff0 legacy=0x7f109c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:14.856 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:14.856 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:14.857 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:12:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.858 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.859 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:14.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:14.861 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.863 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:14.864 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.865 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.867 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:14.868 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:14.876 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:14.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10c379d640 1 -- 172.21.15.45:0/671776247 >> v1:172.21.15.84:6812/31478 conn(0x7f10a003aa70 legacy=0x7f10a003cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10c379d640 10 monclient: shutdown 2024-09-18T14:12:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10c379d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10c379d640 1 -- 172.21.15.45:0/671776247 >> v1:172.21.15.45:6792/0 conn(0x7f10bc06e810 legacy=0x7f10bc13d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10c1d3a640 1 -- 172.21.15.45:0/671776247 reap_dead start 2024-09-18T14:12:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10c1d3a640 1 -- 172.21.15.45:0/671776247 reap_dead start 2024-09-18T14:12:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10c379d640 1 -- 172.21.15.45:0/671776247 shutdown_connections 2024-09-18T14:12:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10c379d640 1 -- 172.21.15.45:0/671776247 >> 172.21.15.45:0/671776247 conn(0x7f10bc0a5bb0 msgr2=0x7f10bc0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10c379d640 1 -- 172.21.15.45:0/671776247 shutdown_connections 2024-09-18T14:12:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.832+0000 7f10c379d640 1 -- 172.21.15.45:0/671776247 wait complete. 2024-09-18T14:12:14.878 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:14.878 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:12:14.879 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:12:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.974+0000 7feb48e34640 10 monclient: get_monmap_and_config 2024-09-18T14:12:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.974+0000 7feb48e34640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.974+0000 7feb48e34640 10 monclient: monmap: 2024-09-18T14:12:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.975 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:14.976058+0000 2024-09-18T14:12:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:14.976058+0000 2024-09-18T14:12:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.976 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.977 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.977 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.977 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.977 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.974+0000 7feb48e34640 1 Processor -- start 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.974+0000 7feb48e34640 1 -- start start 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.974+0000 7feb48e34640 10 monclient: init 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb48e34640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb48e34640 10 monclient: _add_conns ranks=[8,6,4,2,1,0,7,3,5] 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb48e34640 10 monclient(hunting): picked mon.noname-e con 0x7feb4406e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb48e34640 10 monclient(hunting): picked mon.noname-d con 0x7feb440a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb48e34640 10 monclient(hunting): picked mon.noname-c con 0x7feb440a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb48e34640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4405ed70 con 0x7feb440a1080 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb48e34640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb44097620 con 0x7feb440a0cb0 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb48e34640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb44097340 con 0x7feb4406e810 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb48e34640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb48e34640 10 monclient(hunting): authenticate will time out at 1550.314579s 2024-09-18T14:12:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4259c640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7feb4406e810 0x7feb440a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:41618/0 (socket says 172.21.15.45:41618) 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4259c640 1 -- 172.21.15.45:0/1181490800 learned_addr learned my addr 172.21.15.45:0/1181490800 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb41d9b640 1 --1- 172.21.15.45:0/1181490800 >> v1:172.21.15.45:6792/0 conn(0x7feb440a0cb0 0x7feb44137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40416/0 (socket says 172.21.15.45:40416) 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586958157 0 0) 0x7feb44097620 con 0x7feb440a0cb0 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 10 monclient(hunting): my global_id is 184425 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb300034e0 con 0x7feb440a0cb0 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062454976 0 0) 0x7feb44097340 con 0x7feb4406e810 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 10 monclient(hunting): my global_id is 5705 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb44097620 con 0x7feb4406e810 2024-09-18T14:12:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441961425 0 0) 0x7feb4405ed70 con 0x7feb440a1080 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 10 monclient(hunting): my global_id is 164623 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.975+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb44097340 con 0x7feb440a1080 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1337743911 0 0) 0x7feb300034e0 con 0x7feb440a0cb0 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 >> v1:172.21.15.45:6793/0 conn(0x7feb4406e810 legacy=0x7feb440a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 >> v1:172.21.15.45:6791/0 conn(0x7feb440a1080 legacy=0x7feb4413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 10 monclient: _finish_hunting 0 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 1 monclient: found mon.noname-d 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb440a1660 con 0x7feb440a0cb0 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 10 monclient: _finish_auth 0 2024-09-18T14:12:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb2c001830 con 0x7feb440a0cb0 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 10 monclient: dump: 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb48e34640 5 monclient: authenticate success, global_id 184425 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb48e34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feb2c001990 con 0x7feb440a0cb0 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 1 -- 172.21.15.45:0/1181490800 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb2c0047a0 con 0x7feb440a0cb0 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb48e34640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb48e34640 4 set_mon_vals no callback set 2024-09-18T14:12:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb4159a640 10 monclient: dump: 2024-09-18T14:12:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb48e34640 10 monclient: shutdown 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb48e34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb48e34640 1 -- 172.21.15.45:0/1181490800 >> v1:172.21.15.45:6792/0 conn(0x7feb440a0cb0 legacy=0x7feb44137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb48e34640 1 -- 172.21.15.45:0/1181490800 shutdown_connections 2024-09-18T14:12:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb48e34640 1 -- 172.21.15.45:0/1181490800 >> 172.21.15.45:0/1181490800 conn(0x7feb440a5bb0 msgr2=0x7feb440a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.976+0000 7feb48e34640 1 -- 172.21.15.45:0/1181490800 shutdown_connections 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.977+0000 7feb48e34640 1 -- 172.21.15.45:0/1181490800 wait complete. 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.977+0000 7feb48e34640 10 monclient: build_initial_monmap 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.977+0000 7feb48e34640 10 monclient: monmap: 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.977+0000 7feb48e34640 1 Processor -- start 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.977+0000 7feb48e34640 1 -- start start 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.977+0000 7feb48e34640 10 monclient: init 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb48e34640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb48e34640 10 monclient: _add_conns ranks=[5,3,4,1,6,8,2,7,0] 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb48e34640 10 monclient(hunting): picked mon.noname-h con 0x7feb4406e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb48e34640 10 monclient(hunting): picked mon.noname-g con 0x7feb440a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb48e34640 10 monclient(hunting): picked mon.noname-c con 0x7feb440a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb48e34640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb440a2270 con 0x7feb440a0cb0 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb48e34640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb44122730 con 0x7feb440a1080 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb48e34640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4413da30 con 0x7feb4406e810 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb48e34640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb48e34640 10 monclient(hunting): authenticate will time out at 1550.317320s 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb42d9d640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7feb440a1080 0x7feb44149c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:55832/0 (socket says 172.21.15.45:55832) 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb42d9d640 1 -- 172.21.15.45:0/200267715 learned_addr learned my addr 172.21.15.45:0/200267715 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381125735 0 0) 0x7feb44122730 con 0x7feb440a1080 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): my global_id is 164632 2024-09-18T14:12:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb240034e0 con 0x7feb440a1080 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3832153822 0 0) 0x7feb240034e0 con 0x7feb440a1080 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb44122730 con 0x7feb440a1080 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb34001c30 con 0x7feb440a1080 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917143142 0 0) 0x7feb440a2270 con 0x7feb440a0cb0 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): my global_id is 184377 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb240034e0 con 0x7feb440a0cb0 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1659402330 0 0) 0x7feb4413da30 con 0x7feb4406e810 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 10 monclient(hunting): my global_id is 184361 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.978+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb440a2270 con 0x7feb4406e810 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2823993689 0 0) 0x7feb44122730 con 0x7feb440a1080 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 >> v1:172.21.15.84:6791/0 conn(0x7feb4406e810 legacy=0x7feb4413d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 >> v1:172.21.15.84:6790/0 conn(0x7feb440a0cb0 legacy=0x7feb44146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 1 monclient: found mon.c 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4413e6f0 con 0x7feb440a1080 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb48e34640 5 monclient: authenticate success, global_id 164632 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb48e34640 10 monclient: _renew_subs 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb48e34640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb48e34640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb441223c0 con 0x7feb440a1080 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb48e34640 10 monclient: _renew_subs 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb48e34640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb48e34640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4413ded0 con 0x7feb440a1080 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feb34004b70 con 0x7feb440a1080 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb34005020 con 0x7feb440a1080 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb40d99640 4 set_mon_vals no callback set 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 10 monclient: dump: 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb48e34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb48e34640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7feb28006580 con 0x7feb28002830 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.979+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7feb3400fff0 con 0x7feb440a1080 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(626..626 src has 424..626) v4 ==== 3923+0+0 (unknown 1148901446 0 0) 0x7feb34046ce0 con 0x7feb440a1080 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933000590 0 0) 0x7feb28006580 con 0x7feb28002830 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 10 monclient(hunting): my global_id is 184370 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb44122730 con 0x7feb28002830 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb41d9b640 10 monclient: get_auth_request con 0x7feb2403a040 auth_method 0 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3879237093 0 0) 0x7feb44122730 con 0x7feb28002830 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb28006580 con 0x7feb28002830 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb3c001480 con 0x7feb28002830 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 10 monclient: dump: 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2763864280 0 0) 0x7feb28006580 con 0x7feb28002830 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.980+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb2800a0a0 con 0x7feb28002830 2024-09-18T14:12:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.981+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7feb3c001720 con 0x7feb28002830 2024-09-18T14:12:15.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.981+0000 7feb3a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:15.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.981+0000 7feb3a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:15.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.981+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 >> v1:172.21.15.84:6791/0 conn(0x7feb28002830 legacy=0x7feb28004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.981+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 >> v1:172.21.15.84:6791/0 conn(0x7feb28002830 legacy=0x7feb28004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:15.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.986+0000 7feb48e34640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:15.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.986+0000 7feb48e34640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7feb28009e70 con 0x7feb280068f0 2024-09-18T14:12:15.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263200109 0 0) 0x7feb28009e70 con 0x7feb280068f0 2024-09-18T14:12:15.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 10 monclient(hunting): my global_id is 184379 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb28006580 con 0x7feb280068f0 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2118926060 0 0) 0x7feb28006580 con 0x7feb280068f0 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb28009e70 con 0x7feb280068f0 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feb3c004ef0 con 0x7feb280068f0 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 10 monclient: dump: 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2605870254 0 0) 0x7feb28009e70 con 0x7feb280068f0 2024-09-18T14:12:15.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.987+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7feb28008750 con 0x7feb280068f0 2024-09-18T14:12:15.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 3914106645) 0x7feb3c007190 con 0x7feb280068f0 2024-09-18T14:12:15.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb3a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:15.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb3a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:15.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 >> v1:172.21.15.84:6791/0 conn(0x7feb280068f0 legacy=0x7feb28006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb3a7fc640 1 -- 172.21.15.45:0/200267715 >> v1:172.21.15.84:6791/0 conn(0x7feb280068f0 legacy=0x7feb28006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:15.010 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:12:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:15.012 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:15.013 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:15.014 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:15.015 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.017 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.018 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.021 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.022 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.023 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.024 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:15.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.026 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.030 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb48e34640 1 -- 172.21.15.45:0/200267715 >> v1:172.21.15.84:6812/31478 conn(0x7feb2403a040 legacy=0x7feb2403c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb48e34640 10 monclient: shutdown 2024-09-18T14:12:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb48e34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb48e34640 1 -- 172.21.15.45:0/200267715 >> v1:172.21.15.45:6791/0 conn(0x7feb440a1080 legacy=0x7feb44149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb42d9d640 1 -- 172.21.15.45:0/200267715 reap_dead start 2024-09-18T14:12:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb42d9d640 1 -- 172.21.15.45:0/200267715 reap_dead start 2024-09-18T14:12:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb48e34640 1 -- 172.21.15.45:0/200267715 shutdown_connections 2024-09-18T14:12:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb48e34640 1 -- 172.21.15.45:0/200267715 >> 172.21.15.45:0/200267715 conn(0x7feb440a5bb0 msgr2=0x7feb4409e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb48e34640 1 -- 172.21.15.45:0/200267715 shutdown_connections 2024-09-18T14:12:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:14.988+0000 7feb48e34640 1 -- 172.21.15.45:0/200267715 wait complete. 2024-09-18T14:12:15.032 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:15.032 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:12:15.032 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:12:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.127+0000 7fd770b5d640 10 monclient: get_monmap_and_config 2024-09-18T14:12:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.127+0000 7fd770b5d640 10 monclient: build_initial_monmap 2024-09-18T14:12:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.127+0000 7fd770b5d640 10 monclient: monmap: 2024-09-18T14:12:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:15.129864+0000 2024-09-18T14:12:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:15.129864+0000 2024-09-18T14:12:15.130 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:15.130 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.130 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:15.130 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:15.130 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:15.130 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:15.130 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:15.130 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:15.130 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:15.130 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:15.130 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:15.130 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.128+0000 7fd770b5d640 1 Processor -- start 2024-09-18T14:12:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.128+0000 7fd770b5d640 1 -- start start 2024-09-18T14:12:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.128+0000 7fd770b5d640 10 monclient: init 2024-09-18T14:12:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd770b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd770b5d640 10 monclient: _add_conns ranks=[0,7,5,6,4,3,8,2,1] 2024-09-18T14:12:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd770b5d640 10 monclient(hunting): picked mon.noname-a con 0x7fd76c0a2520 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd770b5d640 10 monclient(hunting): picked mon.noname-i con 0x7fd76c096a20 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd770b5d640 10 monclient(hunting): picked mon.noname-h con 0x7fd76c132fc0 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd770b5d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76c05ed70 con 0x7fd76c0a2520 2024-09-18T14:12:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd770b5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76c097040 con 0x7fd76c132fc0 2024-09-18T14:12:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd770b5d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76c0a28f0 con 0x7fd76c096a20 2024-09-18T14:12:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd770b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd770b5d640 10 monclient(hunting): authenticate will time out at 1550.468348s 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76a59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd76c0a2520 0x7fd76c1328b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59964/0 (socket says 172.21.15.45:59964) 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76a59c640 1 -- 172.21.15.45:0/4150242694 learned_addr learned my addr 172.21.15.45:0/4150242694 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874425407 0 0) 0x7fd76c05ed70 con 0x7fd76c0a2520 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient(hunting): my global_id is 5868 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7540034e0 con 0x7fd76c0a2520 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494979030 0 0) 0x7fd76c0a28f0 con 0x7fd76c096a20 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient(hunting): my global_id is 184408 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd76c05ed70 con 0x7fd76c096a20 2024-09-18T14:12:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261277714 0 0) 0x7fd76c097040 con 0x7fd76c132fc0 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient(hunting): my global_id is 184388 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd76c0a28f0 con 0x7fd76c132fc0 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3721839363 0 0) 0x7fd7540034e0 con 0x7fd76c0a2520 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 >> v1:172.21.15.84:6792/0 conn(0x7fd76c096a20 legacy=0x7fd76c137750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 >> v1:172.21.15.84:6791/0 conn(0x7fd76c132fc0 legacy=0x7fd76c13ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient: _finish_hunting 0 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 1 monclient: found mon.noname-a 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c09f4b0 con 0x7fd76c0a2520 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient: _finish_auth 0 2024-09-18T14:12:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd770b5d640 5 monclient: authenticate success, global_id 5868 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd770b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd75c001830 con 0x7fd76c0a2520 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.129+0000 7fd76959a640 10 monclient: dump: 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd770b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd75c001a10 con 0x7fd76c0a2520 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd76959a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd770b5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd770b5d640 4 set_mon_vals no callback set 2024-09-18T14:12:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd76959a640 1 -- 172.21.15.45:0/4150242694 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd75c004ba0 con 0x7fd76c0a2520 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd76959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd76959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd76959a640 10 monclient: dump: 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd770b5d640 10 monclient: shutdown 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd770b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd770b5d640 1 -- 172.21.15.45:0/4150242694 >> v1:172.21.15.45:6789/0 conn(0x7fd76c0a2520 legacy=0x7fd76c1328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd770b5d640 1 -- 172.21.15.45:0/4150242694 shutdown_connections 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd770b5d640 1 -- 172.21.15.45:0/4150242694 >> 172.21.15.45:0/4150242694 conn(0x7fd76c063130 msgr2=0x7fd76c063500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd770b5d640 1 -- 172.21.15.45:0/4150242694 shutdown_connections 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.130+0000 7fd770b5d640 1 -- 172.21.15.45:0/4150242694 wait complete. 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 10 monclient: build_initial_monmap 2024-09-18T14:12:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 10 monclient: monmap: 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 1 Processor -- start 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 1 -- start start 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 10 monclient: init 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 10 monclient: _add_conns ranks=[5,2,4,6,1,7,3,0,8] 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 10 monclient(hunting): picked mon.noname-h con 0x7fd76c096a20 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 10 monclient(hunting): picked mon.noname-b con 0x7fd76c0a2520 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 10 monclient(hunting): picked mon.noname-c con 0x7fd76c132fc0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76c09f8e0 con 0x7fd76c0a2520 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76c0a2270 con 0x7fd76c132fc0 2024-09-18T14:12:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76c121b90 con 0x7fd76c096a20 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.131+0000 7fd770b5d640 10 monclient(hunting): authenticate will time out at 1550.470968s 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd76a59c640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7fd76c096a20 0x7fd76c1416f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:44418/0 (socket says 172.21.15.45:44418) 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd76a59c640 1 -- 172.21.15.45:0/3550591185 learned_addr learned my addr 172.21.15.45:0/3550591185 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294800605 0 0) 0x7fd76c121b90 con 0x7fd76c096a20 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): my global_id is 184397 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd74c0034e0 con 0x7fd76c096a20 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564402182 0 0) 0x7fd76c0a2270 con 0x7fd76c132fc0 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): my global_id is 164641 2024-09-18T14:12:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd76c121b90 con 0x7fd76c132fc0 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736387322 0 0) 0x7fd76c09f8e0 con 0x7fd76c0a2520 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): my global_id is 145145 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd76c0a2270 con 0x7fd76c0a2520 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1895954532 0 0) 0x7fd76c121b90 con 0x7fd76c132fc0 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd76c09f8e0 con 0x7fd76c132fc0 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd764001d00 con 0x7fd76c132fc0 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2525989453 0 0) 0x7fd74c0034e0 con 0x7fd76c096a20 2024-09-18T14:12:15.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd76c121b90 con 0x7fd76c096a20 2024-09-18T14:12:15.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4046212878 0 0) 0x7fd76c0a2270 con 0x7fd76c0a2520 2024-09-18T14:12:15.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd74c0034e0 con 0x7fd76c0a2520 2024-09-18T14:12:15.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd75c004b60 con 0x7fd76c096a20 2024-09-18T14:12:15.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:12:15.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:15.151 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.151 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.152 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.152 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.152 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.152 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.152 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.152 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.152 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.152 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.152 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.152 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd760001a60 con 0x7fd76c0a2520 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2037195055 0 0) 0x7fd76c09f8e0 con 0x7fd76c132fc0 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 >> v1:172.21.15.84:6791/0 conn(0x7fd76c096a20 legacy=0x7fd76c1416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 >> v1:172.21.15.45:6790/0 conn(0x7fd76c0a2520 legacy=0x7fd76c131760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 monclient: found mon.c 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c141e00 con 0x7fd76c132fc0 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd75a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd770b5d640 5 monclient: authenticate success, global_id 164641 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd770b5d640 10 monclient: _renew_subs 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd770b5d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd770b5d640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76c11fd80 con 0x7fd76c132fc0 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd770b5d640 10 monclient: _renew_subs 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd770b5d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.132+0000 7fd770b5d640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76c120950 con 0x7fd76c132fc0 2024-09-18T14:12:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.133+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd764002110 con 0x7fd76c132fc0 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.133+0000 7fd75a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.133+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd7640026c0 con 0x7fd76c132fc0 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.133+0000 7fd75a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.133+0000 7fd768d99640 4 set_mon_vals no callback set 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.133+0000 7fd75a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.133+0000 7fd75a7fc640 10 monclient: dump: 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.133+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd7640028a0 con 0x7fd76c132fc0 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.133+0000 7fd770b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.133+0000 7fd770b5d640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd750006580 con 0x7fd750002830 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(626..626 src has 424..626) v4 ==== 3923+0+0 (unknown 1148901446 0 0) 0x7fd764044360 con 0x7fd76c132fc0 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd769d9b640 10 monclient: get_auth_request con 0x7fd74c03aba0 auth_method 0 2024-09-18T14:12:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688572959 0 0) 0x7fd750006580 con 0x7fd750002830 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 10 monclient(hunting): my global_id is 184417 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd76c09f8e0 con 0x7fd750002830 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 26177494 0 0) 0x7fd76c09f8e0 con 0x7fd750002830 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd750006580 con 0x7fd750002830 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd75c0011d0 con 0x7fd750002830 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 10 monclient: dump: 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1502438479 0 0) 0x7fd750006580 con 0x7fd750002830 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd75000a0a0 con 0x7fd750002830 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd75c007200 con 0x7fd750002830 2024-09-18T14:12:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:15.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:15.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 >> v1:172.21.15.84:6792/0 conn(0x7fd750002830 legacy=0x7fd750004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.134+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 >> v1:172.21.15.84:6792/0 conn(0x7fd750002830 legacy=0x7fd750004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:15.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.140+0000 7fd770b5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:15.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.140+0000 7fd770b5d640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd750007540 con 0x7fd7500068f0 2024-09-18T14:12:15.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619222222 0 0) 0x7fd750007540 con 0x7fd7500068f0 2024-09-18T14:12:15.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 10 monclient(hunting): my global_id is 184426 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd750006580 con 0x7fd7500068f0 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2843659470 0 0) 0x7fd750006580 con 0x7fd7500068f0 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd750007540 con 0x7fd7500068f0 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd75c005a40 con 0x7fd7500068f0 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 10 monclient: dump: 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 802116166 0 0) 0x7fd750007540 con 0x7fd7500068f0 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd7500087a0 con 0x7fd7500068f0 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 2928627838) 0x7fd75c00a1f0 con 0x7fd7500068f0 2024-09-18T14:12:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:15.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:15.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 >> v1:172.21.15.84:6792/0 conn(0x7fd7500068f0 legacy=0x7fd750006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.141+0000 7fd75a7fc640 1 -- 172.21.15.45:0/3550591185 >> v1:172.21.15.84:6792/0 conn(0x7fd7500068f0 legacy=0x7fd750006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:15.164 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:15.164 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:15.165 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:15.166 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:15.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:15.170 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.171 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.172 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:15.174 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.175 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.177 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.178 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.179 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:15.182 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:15.183 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:15.183 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.142+0000 7fd770b5d640 1 -- 172.21.15.45:0/3550591185 >> v1:172.21.15.84:6812/31478 conn(0x7fd74c03aba0 legacy=0x7fd74c03d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.142+0000 7fd770b5d640 10 monclient: shutdown 2024-09-18T14:12:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.142+0000 7fd770b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.142+0000 7fd770b5d640 1 -- 172.21.15.45:0/3550591185 >> v1:172.21.15.45:6791/0 conn(0x7fd76c132fc0 legacy=0x7fd76c149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.142+0000 7fd76ad9d640 1 -- 172.21.15.45:0/3550591185 reap_dead start 2024-09-18T14:12:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.142+0000 7fd76ad9d640 1 -- 172.21.15.45:0/3550591185 reap_dead start 2024-09-18T14:12:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.142+0000 7fd770b5d640 1 -- 172.21.15.45:0/3550591185 shutdown_connections 2024-09-18T14:12:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.142+0000 7fd770b5d640 1 -- 172.21.15.45:0/3550591185 >> 172.21.15.45:0/3550591185 conn(0x7fd76c063130 msgr2=0x7fd76c0a3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.142+0000 7fd770b5d640 1 -- 172.21.15.45:0/3550591185 shutdown_connections 2024-09-18T14:12:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.142+0000 7fd770b5d640 1 -- 172.21.15.45:0/3550591185 wait complete. 2024-09-18T14:12:15.185 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:15.186 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-18T14:12:15.186 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-18T14:12:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.281+0000 7f04c5300640 10 monclient: get_monmap_and_config 2024-09-18T14:12:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.281+0000 7f04c5300640 10 monclient: build_initial_monmap 2024-09-18T14:12:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.281+0000 7f04c5300640 10 monclient: monmap: 2024-09-18T14:12:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:15.283275+0000 2024-09-18T14:12:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:15.283275+0000 2024-09-18T14:12:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:15.284 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.284 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:15.284 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:15.284 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:15.284 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:15.284 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:15.284 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:15.284 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:15.284 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:15.284 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:15.285 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.281+0000 7f04c5300640 1 Processor -- start 2024-09-18T14:12:15.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.281+0000 7f04c5300640 1 -- start start 2024-09-18T14:12:15.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.281+0000 7f04c5300640 10 monclient: init 2024-09-18T14:12:15.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04c5300640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04c5300640 10 monclient: _add_conns ranks=[2,6,0,8,4,1,5,3,7] 2024-09-18T14:12:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04c5300640 10 monclient(hunting): picked mon.noname-b con 0x7f04c009f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04c5300640 10 monclient(hunting): picked mon.noname-d con 0x7f04c009fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04c5300640 10 monclient(hunting): picked mon.noname-a con 0x7f04c009ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04c5300640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c005ed70 con 0x7f04c009ff70 2024-09-18T14:12:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04c5300640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c009dfc0 con 0x7f04c009f7d0 2024-09-18T14:12:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04c5300640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c009dce0 con 0x7f04c009fba0 2024-09-18T14:12:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04c5300640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04c5300640 10 monclient(hunting): authenticate will time out at 1550.621473s 2024-09-18T14:12:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04beffd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f04c009f7d0 0x7f04c0132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55858/0 (socket says 172.21.15.45:55858) 2024-09-18T14:12:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04beffd640 1 -- 172.21.15.45:0/3591883899 learned_addr learned my addr 172.21.15.45:0/3591883899 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04bf7fe640 1 --1- 172.21.15.45:0/3591883899 >> v1:172.21.15.45:6789/0 conn(0x7f04c009ff70 0x7f04c013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59980/0 (socket says 172.21.15.45:59980) 2024-09-18T14:12:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401770710 0 0) 0x7f04c005ed70 con 0x7f04c009ff70 2024-09-18T14:12:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 10 monclient(hunting): my global_id is 5877 2024-09-18T14:12:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04ac0034e0 con 0x7f04c009ff70 2024-09-18T14:12:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316907617 0 0) 0x7f04c009dce0 con 0x7f04c009fba0 2024-09-18T14:12:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 10 monclient(hunting): my global_id is 184434 2024-09-18T14:12:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04c005ed70 con 0x7f04c009fba0 2024-09-18T14:12:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084582998 0 0) 0x7f04c009dfc0 con 0x7f04c009f7d0 2024-09-18T14:12:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 10 monclient(hunting): my global_id is 145154 2024-09-18T14:12:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.282+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04c009dce0 con 0x7f04c009f7d0 2024-09-18T14:12:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1731418154 0 0) 0x7f04ac0034e0 con 0x7f04c009ff70 2024-09-18T14:12:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 >> v1:172.21.15.45:6792/0 conn(0x7f04c009fba0 legacy=0x7f04c0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 >> v1:172.21.15.45:6790/0 conn(0x7f04c009f7d0 legacy=0x7f04c0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 1 monclient: found mon.noname-a 2024-09-18T14:12:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:12:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c00a1c00 con 0x7f04c009ff70 2024-09-18T14:12:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f04b8001bb0 con 0x7f04c009ff70 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 10 monclient: dump: 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.297 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04c5300640 5 monclient: authenticate success, global_id 5877 2024-09-18T14:12:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04c5300640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f04b8001d90 con 0x7f04c009ff70 2024-09-18T14:12:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 1 -- 172.21.15.45:0/3591883899 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f04b8004e60 con 0x7f04c009ff70 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04c5300640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04c5300640 4 set_mon_vals no callback set 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04be7fc640 10 monclient: dump: 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04c5300640 10 monclient: shutdown 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04c5300640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04c5300640 1 -- 172.21.15.45:0/3591883899 >> v1:172.21.15.45:6789/0 conn(0x7f04c009ff70 legacy=0x7f04c013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04c5300640 1 -- 172.21.15.45:0/3591883899 shutdown_connections 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04c5300640 1 -- 172.21.15.45:0/3591883899 >> 172.21.15.45:0/3591883899 conn(0x7f04c00630b0 msgr2=0x7f04c00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04c5300640 1 -- 172.21.15.45:0/3591883899 shutdown_connections 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.283+0000 7f04c5300640 1 -- 172.21.15.45:0/3591883899 wait complete. 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.284+0000 7f04c5300640 10 monclient: build_initial_monmap 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.284+0000 7f04c5300640 10 monclient: monmap: 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.284+0000 7f04c5300640 1 Processor -- start 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.284+0000 7f04c5300640 1 -- start start 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.284+0000 7f04c5300640 10 monclient: init 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04c5300640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04c5300640 10 monclient: _add_conns ranks=[4,6,0,8,5,7,3,1,2] 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04c5300640 10 monclient(hunting): picked mon.noname-c con 0x7f04c009f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04c5300640 10 monclient(hunting): picked mon.noname-d con 0x7f04c009fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04c5300640 10 monclient(hunting): picked mon.noname-a con 0x7f04c009ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04c5300640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c009d310 con 0x7f04c009ff70 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04c5300640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c00a2560 con 0x7f04c009f7d0 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04c5300640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c0120420 con 0x7f04c009fba0 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04c5300640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04c5300640 10 monclient(hunting): authenticate will time out at 1550.624362s 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b67fc640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f04c009fba0 0x7f04c011d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40428/0 (socket says 172.21.15.45:40428) 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b67fc640 1 -- 172.21.15.45:0/1077628623 learned_addr learned my addr 172.21.15.45:0/1077628623 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04bf7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f04c009ff70 0x7f04c0149c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59996/0 (socket says 172.21.15.45:59996) 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365929551 0 0) 0x7f04c0120420 con 0x7f04c009fba0 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 10 monclient(hunting): my global_id is 184443 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04a00034e0 con 0x7f04c009fba0 2024-09-18T14:12:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017324375 0 0) 0x7f04c009d310 con 0x7f04c009ff70 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 10 monclient(hunting): my global_id is 5886 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04c0120420 con 0x7f04c009ff70 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186180461 0 0) 0x7f04c00a2560 con 0x7f04c009f7d0 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 10 monclient(hunting): my global_id is 164650 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04c009d310 con 0x7f04c009f7d0 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1417065527 0 0) 0x7f04a00034e0 con 0x7f04c009fba0 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.285+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f04c00a2560 con 0x7f04c009fba0 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f04a8001d90 con 0x7f04c009fba0 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:15.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient(hunting): dump: 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.304 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 77979334 0 0) 0x7f04c0120420 con 0x7f04c009ff70 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f04a00034e0 con 0x7f04c009ff70 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1639910055 0 0) 0x7f04c009d310 con 0x7f04c009f7d0 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f04c0120420 con 0x7f04c009f7d0 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f04b8001af0 con 0x7f04c009ff70 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient(hunting): dump: 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f04b0001b20 con 0x7f04c009f7d0 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient(hunting): dump: 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2816098097 0 0) 0x7f04c00a2560 con 0x7f04c009fba0 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 >> v1:172.21.15.45:6791/0 conn(0x7f04c009f7d0 legacy=0x7f04c011d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 >> v1:172.21.15.45:6789/0 conn(0x7f04c009ff70 legacy=0x7f04c0149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient: _finish_hunting 0 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 monclient: found mon.d 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c0120700 con 0x7f04c009fba0 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient: _finish_auth 0 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04c5300640 5 monclient: authenticate success, global_id 184443 2024-09-18T14:12:15.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04c5300640 10 monclient: _renew_subs 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04c5300640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04c5300640 1 -- 172.21.15.45:0/1077628623 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c011e8f0 con 0x7f04c009fba0 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04c5300640 10 monclient: _renew_subs 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04c5300640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04c5300640 1 -- 172.21.15.45:0/1077628623 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c011dfa0 con 0x7f04c009fba0 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f04a8002220 con 0x7f04c009fba0 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f04a80027d0 con 0x7f04c009fba0 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04bdffb640 4 set_mon_vals no callback set 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.286+0000 7f04b77fe640 10 monclient: dump: 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:15.312 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.312 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:15.312 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:15.312 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:15.312 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:15.312 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:15.313 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:15.313 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:15.313 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:15.313 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:15.313 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:15.313 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:15.313 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:15.313 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:15.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.287+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f04a8010120 con 0x7f04c009fba0 2024-09-18T14:12:15.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.287+0000 7f04c5300640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:15.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.287+0000 7f04c5300640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:15.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.287+0000 7f04c5300640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:15.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.287+0000 7f04c5300640 1 -- 172.21.15.45:0/1077628623 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a4005010 con 0x7f04c009fba0 2024-09-18T14:12:15.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.287+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(626..626 src has 424..626) v4 ==== 3923+0+0 (unknown 1148901446 0 0) 0x7f04a8046b90 con 0x7f04c009fba0 2024-09-18T14:12:15.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.287+0000 7f04beffd640 10 monclient: get_auth_request con 0x7f04a003aa40 auth_method 0 2024-09-18T14:12:15.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.291+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f04a80153d0 con 0x7f04c009fba0 2024-09-18T14:12:15.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.291+0000 7f04b77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:15.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.291+0000 7f04b77fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:15.503 INFO:tasks.rados.rados.0.smithi084.stdout:2828: done (2 left) 2024-09-18T14:12:15.503 INFO:tasks.rados.rados.0.smithi084.stdout:2829: snap_create 2024-09-18T14:12:15.503 INFO:tasks.rados.rados.0.smithi084.stdout:2825: finishing write tid 3 to smithi08434663-7 2024-09-18T14:12:15.503 INFO:tasks.rados.rados.0.smithi084.stdout:2825: finishing write tid 4 to smithi08434663-7 2024-09-18T14:12:15.503 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 1001 (ObjNum 963 snap 306 seq_num 963) dirty exists 2024-09-18T14:12:15.504 INFO:tasks.rados.rados.0.smithi084.stdout:2825: left oid 7 (ObjNum 963 snap 306 seq_num 963) 2024-09-18T14:12:15.504 INFO:tasks.rados.rados.0.smithi084.stdout:2827: finishing write tid 1 to smithi08434663-17 2024-09-18T14:12:15.504 INFO:tasks.rados.rados.0.smithi084.stdout:2827: finishing write tid 2 to smithi08434663-17 2024-09-18T14:12:15.504 INFO:tasks.rados.rados.0.smithi084.stdout:2827: finishing write tid 3 to smithi08434663-17 2024-09-18T14:12:15.504 INFO:tasks.rados.rados.0.smithi084.stdout:2827: finishing write tid 4 to smithi08434663-17 2024-09-18T14:12:15.504 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1043 (ObjNum 965 snap 306 seq_num 965) dirty exists 2024-09-18T14:12:15.504 INFO:tasks.rados.rados.0.smithi084.stdout:2827: left oid 17 (ObjNum 965 snap 306 seq_num 965) 2024-09-18T14:12:15.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.535+0000 7f04c5300640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-18T14:12:15.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.535+0000 7f04c5300640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-18T14:12:15.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.535+0000 7f04c5300640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:15.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.535+0000 7f04c5300640 1 -- 172.21.15.45:0/1077628623 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f04a4005b90 con 0x7f04c009fba0 2024-09-18T14:12:15.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f04b77fe640 1 -- 172.21.15.45:0/1077628623 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f04a8014dd0 con 0x7f04c009fba0 2024-09-18T14:12:15.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f04b77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-18T14:12:15.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f04b77fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:15.541 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.541 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=29,config=2,health=11,kv=6,logm=52} crc {auth=2241148382,config=3140514009,health=3210141515,kv=895653279,logm=549848104}) 2024-09-18T14:12:15.541 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=7,logm=51} crc {auth=3180952647,config=3140514009,health=627080698,kv=4171427160,logm=3784954329}) 2024-09-18T14:12:15.541 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.541 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=29,config=2,health=11,kv=6,logm=52} crc {auth=2241148382,config=3140514009,health=3210141515,kv=895653279,logm=549848104}) 2024-09-18T14:12:15.541 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=29,config=2,health=10,kv=8,logm=51} crc {auth=4243473701,config=3140514009,health=1208592964,kv=594742435,logm=719416270}) 2024-09-18T14:12:15.541 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.541 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=29,config=2,health=11,kv=6,logm=52} crc {auth=2241148382,config=3140514009,health=3210141515,kv=895653279,logm=549848104}) 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=29,config=2,health=12,kv=8,logm=49} crc {auth=4225634134,config=3140514009,health=627080698,kv=594742435,logm=1974106}) 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=29,config=2,health=11,kv=6,logm=52} crc {auth=2241148382,config=3140514009,health=3210141515,kv=895653279,logm=549848104}) 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=1650957834,config=3140514009,health=964254145,kv=594742435,logm=3004089220}) 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=29,config=2,health=11,kv=6,logm=52} crc {auth=2241148382,config=3140514009,health=3210141515,kv=895653279,logm=549848104}) 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=4138993399,config=3140514009,health=627080698,kv=1599486584,logm=3284624641}) 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=29,config=2,health=11,kv=6,logm=52} crc {auth=2241148382,config=3140514009,health=3210141515,kv=895653279,logm=549848104}) 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=25,config=2,health=11,kv=8,logm=54} crc {auth=3708524627,config=3140514009,health=214134192,kv=594742435,logm=2422932804}) 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=29,config=2,health=11,kv=6,logm=52} crc {auth=2241148382,config=3140514009,health=3210141515,kv=895653279,logm=549848104}) 2024-09-18T14:12:15.542 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=28,config=2,health=11,kv=8,logm=51} crc {auth=917482134,config=3140514009,health=214134192,kv=1783591300,logm=3439942693}) 2024-09-18T14:12:15.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=29,config=2,health=11,kv=6,logm=52} crc {auth=2241148382,config=3140514009,health=3210141515,kv=895653279,logm=549848104}) 2024-09-18T14:12:15.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.537+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=29,config=2,health=11,kv=7,logm=51} crc {auth=2766092114,config=3140514009,health=3210141515,kv=4140378858,logm=4055679883}) 2024-09-18T14:12:15.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=996998964}) 2024-09-18T14:12:15.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1508078874}) 2024-09-18T14:12:15.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=996998964}) 2024-09-18T14:12:15.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1400215685}) 2024-09-18T14:12:15.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=996998964}) 2024-09-18T14:12:15.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1646187342}) 2024-09-18T14:12:15.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=996998964}) 2024-09-18T14:12:15.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4211112445}) 2024-09-18T14:12:15.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=996998964}) 2024-09-18T14:12:15.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1948301288}) 2024-09-18T14:12:15.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=996998964}) 2024-09-18T14:12:15.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=481751577}) 2024-09-18T14:12:15.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=996998964}) 2024-09-18T14:12:15.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3199909272}) 2024-09-18T14:12:15.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=996998964}) 2024-09-18T14:12:15.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.538+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3352050919}) 2024-09-18T14:12:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f04c5300640 1 -- 172.21.15.45:0/1077628623 >> v1:172.21.15.84:6812/31478 conn(0x7f04a003aa40 legacy=0x7f04a003cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.546 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f04c5300640 10 monclient: shutdown 2024-09-18T14:12:15.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f04c5300640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:15.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f04c5300640 1 -- 172.21.15.45:0/1077628623 >> v1:172.21.15.45:6792/0 conn(0x7f04c009fba0 legacy=0x7f04c011d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:15.547 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2884435474}) 2024-09-18T14:12:15.547 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=259076166}) 2024-09-18T14:12:15.547 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.547 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2884435474}) 2024-09-18T14:12:15.547 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1527090968}) 2024-09-18T14:12:15.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f04c5300640 1 -- 172.21.15.45:0/1077628623 shutdown_connections 2024-09-18T14:12:15.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f04c5300640 1 -- 172.21.15.45:0/1077628623 >> 172.21.15.45:0/1077628623 conn(0x7f04c00630b0 msgr2=0x7f04c00a4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:15.548 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.548 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2884435474}) 2024-09-18T14:12:15.548 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2335964642}) 2024-09-18T14:12:15.548 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f04c5300640 1 -- 172.21.15.45:0/1077628623 shutdown_connections 2024-09-18T14:12:15.549 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2884435474}) 2024-09-18T14:12:15.549 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1563511725}) 2024-09-18T14:12:15.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:15.539+0000 7f04c5300640 1 -- 172.21.15.45:0/1077628623 wait complete. 2024-09-18T14:12:15.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2884435474}) 2024-09-18T14:12:15.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=767929885}) 2024-09-18T14:12:15.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2884435474}) 2024-09-18T14:12:15.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240598716}) 2024-09-18T14:12:15.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2884435474}) 2024-09-18T14:12:15.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=51440484}) 2024-09-18T14:12:15.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2884435474}) 2024-09-18T14:12:15.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=931383815}) 2024-09-18T14:12:15.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=847071673}) 2024-09-18T14:12:15.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2990873037}) 2024-09-18T14:12:15.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=847071673}) 2024-09-18T14:12:15.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4134139}) 2024-09-18T14:12:15.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=847071673}) 2024-09-18T14:12:15.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=411828472}) 2024-09-18T14:12:15.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=847071673}) 2024-09-18T14:12:15.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2083513331}) 2024-09-18T14:12:15.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=847071673}) 2024-09-18T14:12:15.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2564620117}) 2024-09-18T14:12:15.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=847071673}) 2024-09-18T14:12:15.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2472765785}) 2024-09-18T14:12:15.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=847071673}) 2024-09-18T14:12:15.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=23522702}) 2024-09-18T14:12:15.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=847071673}) 2024-09-18T14:12:15.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=741851907}) 2024-09-18T14:12:15.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2465660955}) 2024-09-18T14:12:15.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3699377976}) 2024-09-18T14:12:15.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2465660955}) 2024-09-18T14:12:15.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=790636354}) 2024-09-18T14:12:15.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2465660955}) 2024-09-18T14:12:15.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=859659776}) 2024-09-18T14:12:15.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2465660955}) 2024-09-18T14:12:15.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=460548289}) 2024-09-18T14:12:15.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2465660955}) 2024-09-18T14:12:15.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2787970858}) 2024-09-18T14:12:15.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2465660955}) 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2914199431}) 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.543+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.543+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2465660955}) 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.543+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2619120892}) 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.543+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.543+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2465660955}) 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.543+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3548284583}) 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.543+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.543+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2750428768}) 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1932167031}) 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2750428768}) 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3857124552}) 2024-09-18T14:12:15.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2750428768}) 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2923186131}) 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2750428768}) 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2507492146}) 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2750428768}) 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=162249960}) 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2750428768}) 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3872326566}) 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2750428768}) 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3005731271}) 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2750428768}) 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1107324135}) 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2678999770}) 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=788836596}) 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2678999770}) 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1077162478}) 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2678999770}) 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1761127025}) 2024-09-18T14:12:15.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2678999770}) 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3679902140}) 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2678999770}) 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1072276275}) 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2678999770}) 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2406297022}) 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2678999770}) 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4067405110}) 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2678999770}) 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4029661701}) 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=979594366}) 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1086137498}) 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=979594366}) 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=445045694}) 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=979594366}) 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3424560254}) 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=979594366}) 2024-09-18T14:12:15.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4076519133}) 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=979594366}) 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3062646511}) 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=979594366}) 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1408392254}) 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=979594366}) 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2892483859}) 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=979594366}) 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4211711190}) 2024-09-18T14:12:15.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3583428713}) 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698363202}) 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3583428713}) 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3622215070}) 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3583428713}) 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3389875102}) 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3583428713}) 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=198066118}) 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3583428713}) 2024-09-18T14:12:15.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3774989923}) 2024-09-18T14:12:15.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3583428713}) 2024-09-18T14:12:15.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2238331256}) 2024-09-18T14:12:15.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3583428713}) 2024-09-18T14:12:15.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1054446900}) 2024-09-18T14:12:15.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3583428713}) 2024-09-18T14:12:15.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1187029098}) 2024-09-18T14:12:15.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4067133856}) 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3502330929}) 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4067133856}) 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=57775003}) 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4067133856}) 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1767430842}) 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4067133856}) 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1520380199}) 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4067133856}) 2024-09-18T14:12:15.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=786000887}) 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4067133856}) 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1872866295}) 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4067133856}) 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4252270404}) 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4067133856}) 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=608536435}) 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2941868218}) 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1920552640}) 2024-09-18T14:12:15.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2941868218}) 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4039817749}) 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2941868218}) 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3196809488}) 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2941868218}) 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=882676706}) 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2941868218}) 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=420290659}) 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2941868218}) 2024-09-18T14:12:15.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1185271528}) 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2941868218}) 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3113432292}) 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2941868218}) 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1590966542}) 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940667630}) 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3644873024}) 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940667630}) 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2761023132}) 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.580 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940667630}) 2024-09-18T14:12:15.581 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2362627435}) 2024-09-18T14:12:15.581 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.581 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940667630}) 2024-09-18T14:12:15.581 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1846045735}) 2024-09-18T14:12:15.581 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.581 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940667630}) 2024-09-18T14:12:15.581 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3664246418}) 2024-09-18T14:12:15.581 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.581 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940667630}) 2024-09-18T14:12:15.581 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1105041699}) 2024-09-18T14:12:15.581 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.581 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940667630}) 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1508153346}) 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940667630}) 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1909619418}) 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2651604706}) 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1704339616}) 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2651604706}) 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2282628354}) 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2651604706}) 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2080986374}) 2024-09-18T14:12:15.582 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2651604706}) 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=875249812}) 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2651604706}) 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=818863373}) 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2651604706}) 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3289599452}) 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2651604706}) 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3082001226}) 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2651604706}) 2024-09-18T14:12:15.583 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3972503119}) 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=83369331}) 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=77303509}) 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=83369331}) 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1485122960}) 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=83369331}) 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4094948914}) 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=83369331}) 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4117880417}) 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.584 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=83369331}) 2024-09-18T14:12:15.585 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=912238938}) 2024-09-18T14:12:15.585 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.585 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=83369331}) 2024-09-18T14:12:15.585 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=592911280}) 2024-09-18T14:12:15.585 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.585 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=83369331}) 2024-09-18T14:12:15.585 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3089449242}) 2024-09-18T14:12:15.585 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.585 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=83369331}) 2024-09-18T14:12:15.585 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1099626775}) 2024-09-18T14:12:15.586 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-18T14:12:15.674 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.674 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3939162864}) 2024-09-18T14:12:15.674 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4031648157}) 2024-09-18T14:12:15.674 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.674 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3939162864}) 2024-09-18T14:12:15.674 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1730603204}) 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3939162864}) 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1517106625}) 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3939162864}) 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=504269613}) 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3939162864}) 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3777365297}) 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3939162864}) 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3299610909}) 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.675 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3939162864}) 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3081382760}) 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3939162864}) 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3756018461}) 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1576796013}) 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=396057678}) 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1576796013}) 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1271288648}) 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1576796013}) 2024-09-18T14:12:15.676 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1135543815}) 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1576796013}) 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=269186690}) 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1576796013}) 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3006818142}) 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1576796013}) 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3176352134}) 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1576796013}) 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3553465715}) 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.677 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1576796013}) 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1352790902}) 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716277203}) 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2778701461}) 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716277203}) 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=740940193}) 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716277203}) 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=769971721}) 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716277203}) 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=823796558}) 2024-09-18T14:12:15.678 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716277203}) 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2600067140}) 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716277203}) 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2804813307}) 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716277203}) 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=614421270}) 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716277203}) 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1609988329}) 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2186567058}) 2024-09-18T14:12:15.679 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3772792621}) 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2186567058}) 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=196027785}) 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2186567058}) 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1644020122}) 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2186567058}) 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1014890340}) 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2186567058}) 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2496868635}) 2024-09-18T14:12:15.680 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2186567058}) 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1484827130}) 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2186567058}) 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3023912439}) 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2186567058}) 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1389663725}) 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4237993701}) 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1084954056}) 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4237993701}) 2024-09-18T14:12:15.681 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4266972013}) 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4237993701}) 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=346776895}) 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4237993701}) 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2275913814}) 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4237993701}) 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2382423374}) 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4237993701}) 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1523704345}) 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4237993701}) 2024-09-18T14:12:15.682 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3858139659}) 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4237993701}) 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2501331976}) 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2060967327}) 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3098663080}) 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2060967327}) 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3772409569}) 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2060967327}) 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3881248688}) 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.683 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2060967327}) 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2882451628}) 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2060967327}) 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3521742493}) 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2060967327}) 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2714399535}) 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2060967327}) 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1015258830}) 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2060967327}) 2024-09-18T14:12:15.684 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1567338628}) 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=806278756}) 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1685573698}) 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=806278756}) 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=895252799}) 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=806278756}) 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2916741363}) 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=806278756}) 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1324941271}) 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.685 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=806278756}) 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2499373390}) 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=806278756}) 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=884789580}) 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=806278756}) 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2645490216}) 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=806278756}) 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=392318441}) 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2779450352}) 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3804722845}) 2024-09-18T14:12:15.686 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2779450352}) 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1780243764}) 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2779450352}) 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=663748686}) 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2779450352}) 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3135009199}) 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2779450352}) 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3799328775}) 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2779450352}) 2024-09-18T14:12:15.687 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1607318003}) 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2779450352}) 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=368554552}) 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2779450352}) 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2808461082}) 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2295873730}) 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=496584019}) 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2295873730}) 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3447231973}) 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2295873730}) 2024-09-18T14:12:15.688 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=376218013}) 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2295873730}) 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=817440997}) 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2295873730}) 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=265786274}) 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2295873730}) 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1192783270}) 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2295873730}) 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4127463588}) 2024-09-18T14:12:15.689 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2295873730}) 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2141017610}) 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1760324452}) 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4013197129}) 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1760324452}) 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2030486528}) 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1760324452}) 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3003848782}) 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1760324452}) 2024-09-18T14:12:15.690 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2702116999}) 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1760324452}) 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1252575030}) 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1760324452}) 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2508897764}) 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1760324452}) 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3034745546}) 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1760324452}) 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=146630581}) 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.691 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3632074969}) 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3812242740}) 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3632074969}) 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2757106563}) 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3632074969}) 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=291294650}) 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3632074969}) 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1370800060}) 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3632074969}) 2024-09-18T14:12:15.692 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2431469254}) 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3632074969}) 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3536576250}) 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3632074969}) 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2995064454}) 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3632074969}) 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=249084686}) 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=726191291}) 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1525487049}) 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.693 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=726191291}) 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=91134175}) 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=726191291}) 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4195757019}) 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=726191291}) 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2143254548}) 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=726191291}) 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2256075499}) 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=726191291}) 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=72445022}) 2024-09-18T14:12:15.694 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=726191291}) 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1442262449}) 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=726191291}) 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3839626352}) 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=442662389}) 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2565535077}) 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=442662389}) 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=384163821}) 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=442662389}) 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=995444018}) 2024-09-18T14:12:15.695 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.696 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=442662389}) 2024-09-18T14:12:15.696 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2832333489}) 2024-09-18T14:12:15.696 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.696 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=442662389}) 2024-09-18T14:12:15.696 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2792135614}) 2024-09-18T14:12:15.696 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.696 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=442662389}) 2024-09-18T14:12:15.696 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3709429891}) 2024-09-18T14:12:15.696 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=442662389}) 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3941340162}) 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=442662389}) 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=444064834}) 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3032864551}) 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2419628837}) 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3032864551}) 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2750942669}) 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3032864551}) 2024-09-18T14:12:15.697 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1087870693}) 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3032864551}) 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1258578526}) 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3032864551}) 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3574519682}) 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3032864551}) 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2205537561}) 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3032864551}) 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2806151407}) 2024-09-18T14:12:15.698 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3032864551}) 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=312708629}) 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=165363569}) 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=605924509}) 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=165363569}) 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4012924358}) 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=165363569}) 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1384706585}) 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=165363569}) 2024-09-18T14:12:15.699 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4266054531}) 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=165363569}) 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2231481018}) 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=165363569}) 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2355059202}) 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=165363569}) 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3854436229}) 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=165363569}) 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3420077098}) 2024-09-18T14:12:15.700 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4294582857}) 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1981328424}) 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4294582857}) 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=841325612}) 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4294582857}) 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=461485110}) 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4294582857}) 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1213548588}) 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4294582857}) 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=485745122}) 2024-09-18T14:12:15.701 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4294582857}) 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1387521587}) 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4294582857}) 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3455707343}) 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4294582857}) 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3791124289}) 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=921291366}) 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=421950859}) 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=921291366}) 2024-09-18T14:12:15.702 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1182633418}) 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=921291366}) 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=139595814}) 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=921291366}) 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3083349032}) 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=921291366}) 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1430926265}) 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=921291366}) 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1398847489}) 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.703 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=921291366}) 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2828651698}) 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=921291366}) 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3986245397}) 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2498442068}) 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=755143752}) 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2498442068}) 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3157017868}) 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2498442068}) 2024-09-18T14:12:15.704 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=311717749}) 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2498442068}) 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3491802427}) 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2498442068}) 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=354021774}) 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2498442068}) 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=403796181}) 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2498442068}) 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2283869057}) 2024-09-18T14:12:15.705 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2498442068}) 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2495111058}) 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2452230403}) 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=194264696}) 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2452230403}) 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3237637646}) 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2452230403}) 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3130208079}) 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2452230403}) 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2647813524}) 2024-09-18T14:12:15.706 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2452230403}) 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=778317770}) 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2452230403}) 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=608511298}) 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2452230403}) 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=602694405}) 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.577+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2452230403}) 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.577+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3155873585}) 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.577+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1031309845}) 2024-09-18T14:12:15.707 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4051811702}) 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1031309845}) 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1417295176}) 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1031309845}) 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3049400531}) 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1031309845}) 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=759022249}) 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1031309845}) 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4259003861}) 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.708 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1031309845}) 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2099616897}) 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1031309845}) 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4175558448}) 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1031309845}) 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=324467987}) 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2765729597}) 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=457874738}) 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2765729597}) 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=925171030}) 2024-09-18T14:12:15.709 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2765729597}) 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=363010915}) 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2765729597}) 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2388275034}) 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2765729597}) 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=610427735}) 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2765729597}) 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1801623898}) 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2765729597}) 2024-09-18T14:12:15.710 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2031867200}) 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2765729597}) 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2042122284}) 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2064616350}) 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2798535030}) 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2064616350}) 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1442530762}) 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2064616350}) 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=79665073}) 2024-09-18T14:12:15.711 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2064616350}) 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2337345396}) 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2064616350}) 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3504036300}) 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2064616350}) 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2656873257}) 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2064616350}) 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1242776874}) 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.712 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2064616350}) 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3272170486}) 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1408591930}) 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=304538250}) 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1408591930}) 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1349742322}) 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1408591930}) 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2486388918}) 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1408591930}) 2024-09-18T14:12:15.713 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1469670910}) 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1408591930}) 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1391799695}) 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1408591930}) 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=821575569}) 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1408591930}) 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2784448158}) 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1408591930}) 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2580906892}) 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.714 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1786908156}) 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2926285956}) 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1786908156}) 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4218227954}) 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1786908156}) 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=145356767}) 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1786908156}) 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1010400328}) 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1786908156}) 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=884903989}) 2024-09-18T14:12:15.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1786908156}) 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2741707626}) 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1786908156}) 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2449420611}) 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1786908156}) 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4122827821}) 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4248452989}) 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=501764615}) 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4248452989}) 2024-09-18T14:12:15.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=271663255}) 2024-09-18T14:12:15.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4248452989}) 2024-09-18T14:12:15.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1579881930}) 2024-09-18T14:12:15.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4248452989}) 2024-09-18T14:12:15.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1456077605}) 2024-09-18T14:12:15.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4248452989}) 2024-09-18T14:12:15.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2392405927}) 2024-09-18T14:12:15.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4248452989}) 2024-09-18T14:12:15.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3914898895}) 2024-09-18T14:12:15.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4248452989}) 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1001443794}) 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4248452989}) 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.584+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2925129269}) 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3073737151}) 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3279938635}) 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3073737151}) 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4056757627}) 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3073737151}) 2024-09-18T14:12:15.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2556838634}) 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3073737151}) 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1038613118}) 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3073737151}) 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1216571525}) 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3073737151}) 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2233417030}) 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3073737151}) 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2718280459}) 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3073737151}) 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=114103593}) 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=444594855}) 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1322689704}) 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=444594855}) 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1087849237}) 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=444594855}) 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3904607227}) 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=444594855}) 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=945058348}) 2024-09-18T14:12:15.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=444594855}) 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1632877570}) 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=444594855}) 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2143462856}) 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=444594855}) 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2886252455}) 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=444594855}) 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=112673432}) 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=285587354}) 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4288517772}) 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=285587354}) 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4169336412}) 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=285587354}) 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2497259529}) 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=285587354}) 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2759032816}) 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=285587354}) 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=161580875}) 2024-09-18T14:12:15.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=285587354}) 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2879445285}) 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=285587354}) 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3012819076}) 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=285587354}) 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1154557726}) 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=546636051}) 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2732551383}) 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=546636051}) 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=507521970}) 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=546636051}) 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2557924555}) 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=546636051}) 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3007206062}) 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=546636051}) 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2211888245}) 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=546636051}) 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2555527341}) 2024-09-18T14:12:15.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=546636051}) 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2581482130}) 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=546636051}) 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1769685640}) 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2595427490}) 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2784403030}) 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2595427490}) 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2793134808}) 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2595427490}) 2024-09-18T14:12:15.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=339855684}) 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2595427490}) 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3498457906}) 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2595427490}) 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=523752079}) 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2595427490}) 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4244784585}) 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2595427490}) 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2777149482}) 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2595427490}) 2024-09-18T14:12:15.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4008173068}) 2024-09-18T14:12:15.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933142946}) 2024-09-18T14:12:15.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1669296456}) 2024-09-18T14:12:15.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933142946}) 2024-09-18T14:12:15.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1941570741}) 2024-09-18T14:12:15.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933142946}) 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=528825445}) 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933142946}) 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2318128936}) 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933142946}) 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3159681168}) 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933142946}) 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4193193087}) 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933142946}) 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4006312833}) 2024-09-18T14:12:15.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933142946}) 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1621002184}) 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1789195507}) 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3569839121}) 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1789195507}) 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3207420044}) 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1789195507}) 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=356999735}) 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1789195507}) 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1805158987}) 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1789195507}) 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1302886070}) 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1789195507}) 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2656960838}) 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1789195507}) 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3185651528}) 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1789195507}) 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4177110880}) 2024-09-18T14:12:15.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3920336120}) 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2577581873}) 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3920336120}) 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2889668357}) 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3920336120}) 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2271140}) 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3920336120}) 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=503766401}) 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3920336120}) 2024-09-18T14:12:15.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=648563438}) 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3920336120}) 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1986663356}) 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3920336120}) 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1575890168}) 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3920336120}) 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4015025055}) 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4068361850}) 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4119181460}) 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4068361850}) 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=506318533}) 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4068361850}) 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4196559954}) 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4068361850}) 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4226001005}) 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4068361850}) 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3071668825}) 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4068361850}) 2024-09-18T14:12:15.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3393321849}) 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4068361850}) 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2513187862}) 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4068361850}) 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3874379291}) 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=307225505}) 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1496775047}) 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=307225505}) 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3194076868}) 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=307225505}) 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3438758290}) 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=307225505}) 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3965047128}) 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=307225505}) 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=494672346}) 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=307225505}) 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3323016737}) 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=307225505}) 2024-09-18T14:12:15.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2783242705}) 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=307225505}) 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3358470224}) 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1595940966}) 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3188074805}) 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1595940966}) 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1482133615}) 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1595940966}) 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3303140286}) 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1595940966}) 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2827788161}) 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1595940966}) 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3110584978}) 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1595940966}) 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3299258829}) 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1595940966}) 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2622146039}) 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1595940966}) 2024-09-18T14:12:15.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3909660128}) 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2914884929}) 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836315410}) 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2914884929}) 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3378018874}) 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2914884929}) 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2431958486}) 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2914884929}) 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3367976234}) 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2914884929}) 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=31980539}) 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2914884929}) 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3701440322}) 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2914884929}) 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3822076258}) 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2914884929}) 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3303569489}) 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4207919998}) 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1807453550}) 2024-09-18T14:12:15.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4207919998}) 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3186529174}) 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4207919998}) 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2277379735}) 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4207919998}) 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=928212838}) 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4207919998}) 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=736395392}) 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4207919998}) 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1237614363}) 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4207919998}) 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1487579501}) 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4207919998}) 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=953768840}) 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1385097491}) 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3086963051}) 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1385097491}) 2024-09-18T14:12:15.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=723031300}) 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1385097491}) 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4080137137}) 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1385097491}) 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3497803986}) 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1385097491}) 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=822185406}) 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1385097491}) 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4197376158}) 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1385097491}) 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2024268105}) 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1385097491}) 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2694978855}) 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=5,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=60} crc {logm=2150392205,mdsmap=1531151339,mgr=4288950570,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=754439339,mon_config_key=1321300959,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=634474383}) 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=5,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=60} crc {logm=2150392206,mdsmap=1531151339,mgr=2394081750,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=443711616,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=2152830318}) 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=5,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=60} crc {logm=2150392205,mdsmap=1531151339,mgr=4288950570,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=754439339,mon_config_key=1321300959,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=634474383}) 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=5,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=57} crc {logm=2150392205,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=797257216,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=912867238}) 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=5,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=60} crc {logm=2150392205,mdsmap=1531151339,mgr=4288950570,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=754439339,mon_config_key=1321300959,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=634474383}) 2024-09-18T14:12:15.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=5,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=58} crc {logm=2150392205,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgrstat=797257216,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=452797529}) 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=5,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=60} crc {logm=2150392205,mdsmap=1531151339,mgr=4288950570,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=754439339,mon_config_key=1321300959,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=634474383}) 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=4,mdsmap=3,mgr=7,mgr_command_descs=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=60} crc {logm=889601754,mdsmap=1531151339,mgr=4288950570,mgr_command_descs=3545737492,mgrstat=797257216,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=1160633456}) 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=5,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=60} crc {logm=2150392205,mdsmap=1531151339,mgr=4288950570,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=754439339,mon_config_key=1321300959,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=634474383}) 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=5,mdsmap=3,mgr=7,mgr_metadata=1,mgrstat=10,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=60} crc {logm=2150392205,mdsmap=1531151339,mgr=1821329879,mgr_metadata=270970526,mgrstat=797257216,mon_config_key=212926899,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=1081334690}) 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=5,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=60} crc {logm=2150392205,mdsmap=1531151339,mgr=4288950570,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=754439339,mon_config_key=1321300959,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=634474383}) 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=4,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=59} crc {logm=372393730,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=797257216,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_snap=580600538}) 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=5,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=60} crc {logm=2150392205,mdsmap=1531151339,mgr=4288950570,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=754439339,mon_config_key=1321300959,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=634474383}) 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=5,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=58} crc {logm=2150392205,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=797257216,mon_config_key=1810658514,monmap=455718034,osd_metadata=1675047786,osd_snap=4086078284}) 2024-09-18T14:12:15.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=5,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=60} crc {logm=2150392205,mdsmap=1531151339,mgr=4288950570,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=754439339,mon_config_key=1321300959,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=452261149,osd_snap=634474383}) 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=5,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=60} crc {logm=2150392205,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2454204102,mon_config_key=1810658514,monmap=3140514009,osd_metadata=1798237175,osd_pg_creating=452261149,osd_snap=1481221122}) 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=35,osdmap=65} crc {osd_snap=1013971182,osdmap=1524868173}) 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=39,osdmap=61} crc {osd_snap=2026961318,osdmap=4095718451}) 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=35,osdmap=65} crc {osd_snap=1013971182,osdmap=1524868173}) 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=34,osdmap=66} crc {osd_snap=2008103262,osdmap=1486608783}) 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=35,osdmap=65} crc {osd_snap=1013971182,osdmap=1524868173}) 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=38,osdmap=62} crc {osd_snap=967586333,osdmap=2676333472}) 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=35,osdmap=65} crc {osd_snap=1013971182,osdmap=1524868173}) 2024-09-18T14:12:15.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=39,osdmap=61} crc {osd_snap=1881898236,osdmap=143517169}) 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=35,osdmap=65} crc {osd_snap=1013971182,osdmap=1524868173}) 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=39,osdmap=61} crc {osd_snap=2304075423,osdmap=547446482}) 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=35,osdmap=65} crc {osd_snap=1013971182,osdmap=1524868173}) 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=40,osdmap=60} crc {osd_snap=1897562673,osdmap=1113031773}) 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=35,osdmap=65} crc {osd_snap=1013971182,osdmap=1524868173}) 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=34,osdmap=66} crc {osd_snap=1478209016,osdmap=3273924370}) 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=35,osdmap=65} crc {osd_snap=1013971182,osdmap=1524868173}) 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=38,osdmap=62} crc {osd_snap=3499304515,osdmap=3782425554}) 2024-09-18T14:12:15.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1852261291}) 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1944656402}) 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1852261291}) 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2247960181}) 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1852261291}) 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=735543657}) 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1852261291}) 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3045663948}) 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1852261291}) 2024-09-18T14:12:15.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3610549506}) 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1852261291}) 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=82721995}) 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1852261291}) 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1569398568}) 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1852261291}) 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=4161167037}) 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1183317490}) 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3034137387}) 2024-09-18T14:12:15.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1183317490}) 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=629600895}) 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1183317490}) 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3720981799}) 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1183317490}) 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2980509}) 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1183317490}) 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2475438860}) 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1183317490}) 2024-09-18T14:12:15.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2252771988}) 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1183317490}) 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=956560182}) 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1183317490}) 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=794840954}) 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.611+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.611+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=42} crc {osdmap=2349935990}) 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.611+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=42} crc {osdmap=3069319352}) 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.611+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.611+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=42} crc {osdmap=2349935990}) 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.611+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=41} crc {osdmap=3414048347}) 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.611+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=42} crc {osdmap=2349935990}) 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=42} crc {osdmap=1234789824}) 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=42} crc {osdmap=2349935990}) 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=40} crc {osdmap=623815602}) 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=42} crc {osdmap=2349935990}) 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=41} crc {osdmap=641447994}) 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=42} crc {osdmap=2349935990}) 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=42} crc {osdmap=79391014}) 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=42} crc {osdmap=2349935990}) 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=43} crc {osdmap=1910434577}) 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:12:15.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=42} crc {osdmap=2349935990}) 2024-09-18T14:12:15.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:12:15.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=43} crc {osdmap=3513566366}) 2024-09-18T14:12:16.565 INFO:tasks.rados.rados.0.smithi084.stdout:2825: done (2 left) 2024-09-18T14:12:16.565 INFO:tasks.rados.rados.0.smithi084.stdout:2827: done (1 left) 2024-09-18T14:12:16.565 INFO:tasks.rados.rados.0.smithi084.stdout:2829: done (0 left) 2024-09-18T14:12:16.565 INFO:tasks.rados.rados.0.smithi084.stdout:2830: copy_from oid 6 from oid 5 current snap is 307 2024-09-18T14:12:16.565 INFO:tasks.rados.rados.0.smithi084.stdout:2831: write oid 49 current snap is 307 2024-09-18T14:12:16.565 INFO:tasks.rados.rados.0.smithi084.stdout:2831: seq_num 966 ranges {635562=667365,2094782=140780} 2024-09-18T14:12:16.571 INFO:tasks.rados.rados.0.smithi084.stdout:2831: writing smithi08434663-49 from 635562 to 1302927 tid 1 2024-09-18T14:12:16.577 INFO:tasks.rados.rados.0.smithi084.stdout:2831: writing smithi08434663-49 from 2094782 to 2235562 tid 2 2024-09-18T14:12:16.577 INFO:tasks.rados.rados.0.smithi084.stdout:2832: rollback oid 42 current snap is 307 2024-09-18T14:12:16.577 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 42 to 303 2024-09-18T14:12:16.578 INFO:tasks.rados.rados.0.smithi084.stdout:2830: finishing copy_from to smithi08434663-6 2024-09-18T14:12:16.578 INFO:tasks.rados.rados.0.smithi084.stdout:2830: got expected ENOENT (src dne) 2024-09-18T14:12:16.578 INFO:tasks.rados.rados.0.smithi084.stdout:2830: finishing copy_from racing read to smithi08434663-6 2024-09-18T14:12:16.578 INFO:tasks.rados.rados.0.smithi084.stdout:2831: finishing write tid 1 to smithi08434663-49 2024-09-18T14:12:16.578 INFO:tasks.rados.rados.0.smithi084.stdout:2830: done (2 left) 2024-09-18T14:12:16.578 INFO:tasks.rados.rados.0.smithi084.stdout:2833: read oid 15 snap 295 2024-09-18T14:12:16.578 INFO:tasks.rados.rados.0.smithi084.stdout:2833: expect (ObjNum 879 snap 265 seq_num 879) 2024-09-18T14:12:16.578 INFO:tasks.rados.rados.0.smithi084.stdout:2834: copy_from oid 20 from oid 44 current snap is 307 2024-09-18T14:12:16.578 INFO:tasks.rados.rados.0.smithi084.stdout:2835: snap_create 2024-09-18T14:12:16.579 INFO:tasks.rados.rados.0.smithi084.stdout:2831: finishing write tid 2 to smithi08434663-49 2024-09-18T14:12:16.580 INFO:tasks.rados.rados.0.smithi084.stdout:2831: finishing write tid 3 to smithi08434663-49 2024-09-18T14:12:16.580 INFO:tasks.rados.rados.0.smithi084.stdout:2831: finishing write tid 4 to smithi08434663-49 2024-09-18T14:12:16.581 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 1260 (ObjNum 966 snap 307 seq_num 966) dirty exists 2024-09-18T14:12:16.581 INFO:tasks.rados.rados.0.smithi084.stdout:2831: left oid 49 (ObjNum 966 snap 307 seq_num 966) 2024-09-18T14:12:16.581 INFO:tasks.rados.rados.0.smithi084.stdout:2833: expect (ObjNum 879 snap 265 seq_num 879) 2024-09-18T14:12:16.588 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:12:16.588 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:12:16.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.685+0000 7fe183f9d640 10 monclient: get_monmap_and_config 2024-09-18T14:12:16.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.685+0000 7fe183f9d640 10 monclient: build_initial_monmap 2024-09-18T14:12:16.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.685+0000 7fe183f9d640 10 monclient: monmap: 2024-09-18T14:12:16.686 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:16.687306+0000 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:16.687306+0000 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:16.688 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:16.688 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:16.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.685+0000 7fe183f9d640 1 Processor -- start 2024-09-18T14:12:16.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe183f9d640 1 -- start start 2024-09-18T14:12:16.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe183f9d640 10 monclient: init 2024-09-18T14:12:16.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe183f9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:16.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe183f9d640 10 monclient: _add_conns ranks=[4,6,7,1,2,3,0,8,5] 2024-09-18T14:12:16.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe183f9d640 10 monclient(hunting): picked mon.noname-c con 0x7fe17c09f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:16.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe183f9d640 10 monclient(hunting): picked mon.noname-d con 0x7fe17c09fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:16.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe183f9d640 10 monclient(hunting): picked mon.noname-i con 0x7fe17c09ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:16.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe183f9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c05ed70 con 0x7fe17c09f7d0 2024-09-18T14:12:16.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe183f9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c09dfc0 con 0x7fe17c09fba0 2024-09-18T14:12:16.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe183f9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c09dce0 con 0x7fe17c09ff70 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe183f9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe183f9d640 10 monclient(hunting): authenticate will time out at 1552.025758s 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe181d39640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fe17c09f7d0 0x7fe17c132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:55858/0 (socket says 172.21.15.45:55858) 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe181d39640 1 -- 172.21.15.45:0/714217763 learned_addr learned my addr 172.21.15.45:0/714217763 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.686+0000 7fe181538640 1 --1- 172.21.15.45:0/714217763 >> v1:172.21.15.45:6792/0 conn(0x7fe17c09fba0 0x7fe17c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40444/0 (socket says 172.21.15.45:40444) 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338690807 0 0) 0x7fe17c09dfc0 con 0x7fe17c09fba0 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient(hunting): my global_id is 184452 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1680034e0 con 0x7fe17c09fba0 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136934738 0 0) 0x7fe17c05ed70 con 0x7fe17c09f7d0 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:16.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient(hunting): my global_id is 164659 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe17c09dfc0 con 0x7fe17c09f7d0 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311698050 0 0) 0x7fe17c09dce0 con 0x7fe17c09ff70 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient(hunting): my global_id is 184435 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe17c05ed70 con 0x7fe17c09ff70 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3744554930 0 0) 0x7fe1680034e0 con 0x7fe17c09fba0 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 >> v1:172.21.15.84:6792/0 conn(0x7fe17c09ff70 legacy=0x7fe17c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 >> v1:172.21.15.45:6791/0 conn(0x7fe17c09f7d0 legacy=0x7fe17c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient: _finish_hunting 0 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 monclient: found mon.noname-d 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c0a1c00 con 0x7fe17c09fba0 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient: _finish_auth 0 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe170001830 con 0x7fe17c09fba0 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient: dump: 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe183f9d640 5 monclient: authenticate success, global_id 184452 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe183f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe170001a10 con 0x7fe17c09fba0 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 1 -- 172.21.15.45:0/714217763 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe170004ae0 con 0x7fe17c09fba0 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe183f9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe183f9d640 4 set_mon_vals no callback set 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe180d37640 10 monclient: dump: 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:16.700 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe183f9d640 10 monclient: shutdown 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe183f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe183f9d640 1 -- 172.21.15.45:0/714217763 >> v1:172.21.15.45:6792/0 conn(0x7fe17c09fba0 legacy=0x7fe17c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe183f9d640 1 -- 172.21.15.45:0/714217763 shutdown_connections 2024-09-18T14:12:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.687+0000 7fe183f9d640 1 -- 172.21.15.45:0/714217763 >> 172.21.15.45:0/714217763 conn(0x7fe17c0630b0 msgr2=0x7fe17c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.688+0000 7fe183f9d640 1 -- 172.21.15.45:0/714217763 shutdown_connections 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.688+0000 7fe183f9d640 1 -- 172.21.15.45:0/714217763 wait complete. 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.688+0000 7fe183f9d640 10 monclient: build_initial_monmap 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.688+0000 7fe183f9d640 10 monclient: monmap: 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.688+0000 7fe183f9d640 1 Processor -- start 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.688+0000 7fe183f9d640 1 -- start start 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.688+0000 7fe183f9d640 10 monclient: init 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe183f9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe183f9d640 10 monclient: _add_conns ranks=[7,1,5,2,6,3,0,8,4] 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe183f9d640 10 monclient(hunting): picked mon.noname-i con 0x7fe17c09f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe183f9d640 10 monclient(hunting): picked mon.noname-f con 0x7fe17c09fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe183f9d640 10 monclient(hunting): picked mon.noname-h con 0x7fe17c09ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe183f9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c09d310 con 0x7fe17c09fba0 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe183f9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c0a2560 con 0x7fe17c09ff70 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe183f9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c11e9f0 con 0x7fe17c09f7d0 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe183f9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe183f9d640 10 monclient(hunting): authenticate will time out at 1552.028512s 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe181d39640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7fe17c09f7d0 0x7fe17c131560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:33806/0 (socket says 172.21.15.45:33806) 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe181d39640 1 -- 172.21.15.45:0/2375464160 learned_addr learned my addr 172.21.15.45:0/2375464160 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 791643194 0 0) 0x7fe17c09d310 con 0x7fe17c09fba0 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 10 monclient(hunting): my global_id is 204166 2024-09-18T14:12:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1600034e0 con 0x7fe17c09fba0 2024-09-18T14:12:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242785862 0 0) 0x7fe17c11e9f0 con 0x7fe17c09f7d0 2024-09-18T14:12:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 10 monclient(hunting): my global_id is 184444 2024-09-18T14:12:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe17c09d310 con 0x7fe17c09f7d0 2024-09-18T14:12:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059838880 0 0) 0x7fe17c0a2560 con 0x7fe17c09ff70 2024-09-18T14:12:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 10 monclient(hunting): my global_id is 184406 2024-09-18T14:12:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.689+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe17c11e9f0 con 0x7fe17c09ff70 2024-09-18T14:12:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2780212186 0 0) 0x7fe1600034e0 con 0x7fe17c09fba0 2024-09-18T14:12:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe17c0a2560 con 0x7fe17c09fba0 2024-09-18T14:12:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe170004b20 con 0x7fe17c09fba0 2024-09-18T14:12:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:12:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient(hunting): dump: 2024-09-18T14:12:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 279833994 0 0) 0x7fe17c09d310 con 0x7fe17c09f7d0 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe1600034e0 con 0x7fe17c09f7d0 2024-09-18T14:12:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3184489695 0 0) 0x7fe17c11e9f0 con 0x7fe17c09ff70 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe17c09d310 con 0x7fe17c09ff70 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe178001590 con 0x7fe17c09f7d0 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient(hunting): dump: 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe174001d00 con 0x7fe17c09ff70 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient(hunting): dump: 2024-09-18T14:12:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 123333891 0 0) 0x7fe17c0a2560 con 0x7fe17c09fba0 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 >> v1:172.21.15.84:6792/0 conn(0x7fe17c09f7d0 legacy=0x7fe17c131560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 >> v1:172.21.15.84:6791/0 conn(0x7fe17c09ff70 legacy=0x7fe17c14a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient: _finish_hunting 0 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 monclient: found mon.f 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c136ea0 con 0x7fe17c09fba0 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient: _finish_auth 0 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe183f9d640 5 monclient: authenticate success, global_id 204166 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe183f9d640 10 monclient: _renew_subs 2024-09-18T14:12:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe183f9d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe183f9d640 1 -- 172.21.15.45:0/2375464160 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c11d790 con 0x7fe17c09fba0 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe170001880 con 0x7fe17c09fba0 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe1700050e0 con 0x7fe17c09fba0 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16dffb640 10 monclient: dump: 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe16ffff640 4 set_mon_vals no callback set 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe183f9d640 10 monclient: _renew_subs 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe183f9d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:12:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.690+0000 7fe183f9d640 1 -- 172.21.15.45:0/2375464160 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c09b7c0 con 0x7fe17c09fba0 2024-09-18T14:12:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.691+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fe170010130 con 0x7fe17c09fba0 2024-09-18T14:12:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.691+0000 7fe183f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.691+0000 7fe183f9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.691+0000 7fe183f9d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:12:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.691+0000 7fe183f9d640 1 -- 172.21.15.45:0/2375464160 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c04c4a0 con 0x7fe17c09fba0 2024-09-18T14:12:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.691+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(628..628 src has 424..628) v4 ==== 3911+0+0 (unknown 334880263 0 0) 0x7fe170046be0 con 0x7fe17c09fba0 2024-09-18T14:12:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.691+0000 7fe181d39640 10 monclient: get_auth_request con 0x7fe16003ab30 auth_method 0 2024-09-18T14:12:16.715 INFO:tasks.rados.rados.0.smithi084.stdout:2832: finishing rollback tid 0 to smithi08434663-42 2024-09-18T14:12:16.715 INFO:tasks.rados.rados.0.smithi084.stdout:2832: finishing rollback tid 1 to smithi08434663-42 2024-09-18T14:12:16.715 INFO:tasks.rados.rados.0.smithi084.stdout:2832: finishing rollback tid 2 to smithi08434663-42 2024-09-18T14:12:16.715 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1311 (ObjNum 903 snap 281 seq_num 903) dirty exists 2024-09-18T14:12:16.715 INFO:tasks.rados.rados.0.smithi084.stdout:2834: finishing copy_from racing read to smithi08434663-20 2024-09-18T14:12:16.715 INFO:tasks.rados.rados.0.smithi084.stdout:2834: finishing copy_from to smithi08434663-20 2024-09-18T14:12:16.715 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 958 (ObjNum 908 snap 281 seq_num 908) dirty exists 2024-09-18T14:12:16.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.696+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.1 v1:172.21.15.84:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fe170015060 con 0x7fe17c09fba0 2024-09-18T14:12:16.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.696+0000 7fe16dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:16.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.696+0000 7fe16dffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:16.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.919+0000 7fe183f9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:12:16.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.919+0000 7fe183f9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:12:16.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.919+0000 7fe183f9d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:12:16.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.919+0000 7fe183f9d640 1 -- 172.21.15.45:0/2375464160 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe17c11eba0 con 0x7fe17c09fba0 2024-09-18T14:12:16.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.921+0000 7fe16dffb640 1 -- 172.21.15.45:0/2375464160 <== mon.1 v1:172.21.15.84:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2689 (unknown 254746657 0 597164934) 0x7fe17c11eba0 con 0x7fe17c09fba0 2024-09-18T14:12:16.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.921+0000 7fe16dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:12:16.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.921+0000 7fe16dffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:16.922 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":106,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:12:16.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.922+0000 7fe183f9d640 1 -- 172.21.15.45:0/2375464160 >> v1:172.21.15.84:6812/31478 conn(0x7fe16003ab30 legacy=0x7fe16003cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:16.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.922+0000 7fe183f9d640 10 monclient: shutdown 2024-09-18T14:12:16.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.922+0000 7fe183f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:16.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.922+0000 7fe183f9d640 1 -- 172.21.15.45:0/2375464160 >> v1:172.21.15.84:6789/0 conn(0x7fe17c09fba0 legacy=0x7fe17c146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:16.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.922+0000 7fe183f9d640 1 -- 172.21.15.45:0/2375464160 shutdown_connections 2024-09-18T14:12:16.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.922+0000 7fe183f9d640 1 -- 172.21.15.45:0/2375464160 >> 172.21.15.45:0/2375464160 conn(0x7fe17c0630b0 msgr2=0x7fe17c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:16.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.922+0000 7fe183f9d640 1 -- 172.21.15.45:0/2375464160 shutdown_connections 2024-09-18T14:12:16.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:16.922+0000 7fe183f9d640 1 -- 172.21.15.45:0/2375464160 wait complete. 2024-09-18T14:12:16.933 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:12:16.933 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-18T14:12:16.933 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:12:16.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:12:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.030+0000 7ff8752b8640 10 monclient: get_monmap_and_config 2024-09-18T14:12:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.030+0000 7ff8752b8640 10 monclient: build_initial_monmap 2024-09-18T14:12:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.030+0000 7ff8752b8640 10 monclient: monmap: 2024-09-18T14:12:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:17.032573+0000 2024-09-18T14:12:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:17.032573+0000 2024-09-18T14:12:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:17.033 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 1 Processor -- start 2024-09-18T14:12:17.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 1 -- start start 2024-09-18T14:12:17.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 10 monclient: init 2024-09-18T14:12:17.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:17.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 10 monclient: _add_conns ranks=[4,7,6,2,1,8,0,3,5] 2024-09-18T14:12:17.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 10 monclient(hunting): picked mon.noname-c con 0x7ff87009f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:17.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 10 monclient(hunting): picked mon.noname-i con 0x7ff87009fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:17.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 10 monclient(hunting): picked mon.noname-d con 0x7ff87009ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87005ed70 con 0x7ff87009f7d0 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87009dfc0 con 0x7ff87009ff70 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87009dce0 con 0x7ff87009fba0 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.031+0000 7ff8752b8640 10 monclient(hunting): authenticate will time out at 1552.370894s 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86effd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7ff87009f7d0 0x7ff870132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:55864/0 (socket says 172.21.15.45:55864) 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86effd640 1 -- 172.21.15.45:0/202021064 learned_addr learned my addr 172.21.15.45:0/202021064 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86f7fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7ff87009ff70 0x7ff87013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40450/0 (socket says 172.21.15.45:40450) 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561734766 0 0) 0x7ff87009dfc0 con 0x7ff87009ff70 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient(hunting): my global_id is 184461 2024-09-18T14:12:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff85c0034e0 con 0x7ff87009ff70 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109901787 0 0) 0x7ff87005ed70 con 0x7ff87009f7d0 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient(hunting): my global_id is 164668 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff87009dfc0 con 0x7ff87009f7d0 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696504364 0 0) 0x7ff87009dce0 con 0x7ff87009fba0 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient(hunting): my global_id is 184453 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff87005ed70 con 0x7ff87009fba0 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1152672838 0 0) 0x7ff85c0034e0 con 0x7ff87009ff70 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 >> v1:172.21.15.84:6792/0 conn(0x7ff87009fba0 legacy=0x7ff870137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 >> v1:172.21.15.45:6791/0 conn(0x7ff87009f7d0 legacy=0x7ff870132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 monclient: found mon.noname-d 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8700a1c00 con 0x7ff87009ff70 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff8752b8640 5 monclient: authenticate success, global_id 184461 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff868001bb0 con 0x7ff87009ff70 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient: dump: 2024-09-18T14:12:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff8752b8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff868001d90 con 0x7ff87009ff70 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 1 -- 172.21.15.45:0/202021064 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff868004e60 con 0x7ff87009ff70 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff8752b8640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff8752b8640 4 set_mon_vals no callback set 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.032+0000 7ff86e7fc640 10 monclient: dump: 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.047 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.033+0000 7ff8752b8640 10 monclient: shutdown 2024-09-18T14:12:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.033+0000 7ff8752b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.033+0000 7ff8752b8640 1 -- 172.21.15.45:0/202021064 >> v1:172.21.15.45:6792/0 conn(0x7ff87009ff70 legacy=0x7ff87013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.033+0000 7ff8752b8640 1 -- 172.21.15.45:0/202021064 shutdown_connections 2024-09-18T14:12:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.033+0000 7ff8752b8640 1 -- 172.21.15.45:0/202021064 >> 172.21.15.45:0/202021064 conn(0x7ff8700630b0 msgr2=0x7ff8700634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.033+0000 7ff8752b8640 1 -- 172.21.15.45:0/202021064 shutdown_connections 2024-09-18T14:12:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.033+0000 7ff8752b8640 1 -- 172.21.15.45:0/202021064 wait complete. 2024-09-18T14:12:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.033+0000 7ff8752b8640 10 monclient: build_initial_monmap 2024-09-18T14:12:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.033+0000 7ff8752b8640 10 monclient: monmap: 2024-09-18T14:12:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 1 Processor -- start 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 1 -- start start 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 10 monclient: init 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 10 monclient: _add_conns ranks=[6,5,0,7,8,4,2,3,1] 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 10 monclient(hunting): picked mon.noname-d con 0x7ff87009f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 10 monclient(hunting): picked mon.noname-h con 0x7ff87009fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 10 monclient(hunting): picked mon.noname-a con 0x7ff87009ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff870121b90 con 0x7ff87009ff70 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff870141eb0 con 0x7ff87009fba0 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87009d310 con 0x7ff87009f7d0 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff8752b8640 10 monclient(hunting): authenticate will time out at 1552.373672s 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff86effd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7ff87009f7d0 0x7ff8701417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40454/0 (socket says 172.21.15.45:40454) 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff86effd640 1 -- 172.21.15.45:0/3002822330 learned_addr learned my addr 172.21.15.45:0/3002822330 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767444051 0 0) 0x7ff870121b90 con 0x7ff87009ff70 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff866ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff866ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff866ffd640 10 monclient(hunting): my global_id is 5895 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.034+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8500034e0 con 0x7ff87009ff70 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827797957 0 0) 0x7ff87009d310 con 0x7ff87009f7d0 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): my global_id is 184470 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff870121b90 con 0x7ff87009f7d0 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696382204 0 0) 0x7ff870141eb0 con 0x7ff87009fba0 2024-09-18T14:12:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): my global_id is 184415 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff87009d310 con 0x7ff87009fba0 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1275574592 0 0) 0x7ff87009d310 con 0x7ff87009fba0 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff870141eb0 con 0x7ff87009fba0 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff858001aa0 con 0x7ff87009fba0 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): dump: 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3047095185 0 0) 0x7ff8500034e0 con 0x7ff87009ff70 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff87009d310 con 0x7ff87009ff70 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1477174732 0 0) 0x7ff870121b90 con 0x7ff87009f7d0 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8500034e0 con 0x7ff87009f7d0 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff868001af0 con 0x7ff87009ff70 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): dump: 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff860001480 con 0x7ff87009f7d0 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient(hunting): dump: 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.056 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 332567053 0 0) 0x7ff870141eb0 con 0x7ff87009fba0 2024-09-18T14:12:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 >> v1:172.21.15.45:6792/0 conn(0x7ff87009f7d0 legacy=0x7ff8701417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 >> v1:172.21.15.45:6789/0 conn(0x7ff87009ff70 legacy=0x7ff870149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:12:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 monclient: found mon.h 2024-09-18T14:12:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:12:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8700a2560 con 0x7ff87009fba0 2024-09-18T14:12:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient: _finish_auth 0 2024-09-18T14:12:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff8752b8640 5 monclient: authenticate success, global_id 184415 2024-09-18T14:12:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff8752b8640 10 monclient: _renew_subs 2024-09-18T14:12:17.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff8752b8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:12:17.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff8752b8640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.84:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8701207e0 con 0x7ff87009fba0 2024-09-18T14:12:17.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff8752b8640 10 monclient: _renew_subs 2024-09-18T14:12:17.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff8752b8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:12:17.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff8752b8640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.84:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff870120060 con 0x7ff87009fba0 2024-09-18T14:12:17.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.5 v1:172.21.15.84:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff858002cf0 con 0x7ff87009fba0 2024-09-18T14:12:17.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:17.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.5 v1:172.21.15.84:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff858002370 con 0x7ff87009fba0 2024-09-18T14:12:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff86dffb640 4 set_mon_vals no callback set 2024-09-18T14:12:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:12:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.035+0000 7ff866ffd640 10 monclient: dump: 2024-09-18T14:12:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.5 v1:172.21.15.84:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff8580024d0 con 0x7ff87009fba0 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff8752b8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff8752b8640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff854006580 con 0x7ff854002830 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.5 v1:172.21.15.84:6791/0 8 ==== osd_map(628..628 src has 424..628) v4 ==== 3911+0+0 (unknown 334880263 0 0) 0x7ff858044230 con 0x7ff87009fba0 2024-09-18T14:12:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442855418 0 0) 0x7ff854006580 con 0x7ff854002830 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 10 monclient(hunting): my global_id is 5904 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff870141eb0 con 0x7ff854002830 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff86f7fe640 10 monclient: get_auth_request con 0x7ff85003aa50 auth_method 0 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1620273803 0 0) 0x7ff870141eb0 con 0x7ff854002830 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff854006580 con 0x7ff854002830 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff860001700 con 0x7ff854002830 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.036+0000 7ff866ffd640 10 monclient: dump: 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.061 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.063 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.037+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 399120161 0 0) 0x7ff854006580 con 0x7ff854002830 2024-09-18T14:12:17.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.037+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff85400a0a0 con 0x7ff854002830 2024-09-18T14:12:17.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.037+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff860005040 con 0x7ff854002830 2024-09-18T14:12:17.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.037+0000 7ff866ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:17.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.037+0000 7ff866ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:17.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.037+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 >> v1:172.21.15.45:6789/0 conn(0x7ff854002830 legacy=0x7ff854004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.037+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 >> v1:172.21.15.45:6789/0 conn(0x7ff854002830 legacy=0x7ff854004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff8752b8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff8752b8640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff854009610 con 0x7ff854008ff0 2024-09-18T14:12:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851160294 0 0) 0x7ff854009610 con 0x7ff854008ff0 2024-09-18T14:12:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 10 monclient(hunting): my global_id is 5913 2024-09-18T14:12:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff854006580 con 0x7ff854008ff0 2024-09-18T14:12:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2920173554 0 0) 0x7ff854006580 con 0x7ff854008ff0 2024-09-18T14:12:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff854009610 con 0x7ff854008ff0 2024-09-18T14:12:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff8600057e0 con 0x7ff854008ff0 2024-09-18T14:12:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 10 monclient: dump: 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.067 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3971455724 0 0) 0x7ff854009610 con 0x7ff854008ff0 2024-09-18T14:12:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff8540098d0 con 0x7ff854008ff0 2024-09-18T14:12:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 3631053343) 0x7ff860005b40 con 0x7ff854008ff0 2024-09-18T14:12:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 >> v1:172.21.15.45:6789/0 conn(0x7ff854008ff0 legacy=0x7ff854008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.043+0000 7ff866ffd640 1 -- 172.21.15.45:0/3002822330 >> v1:172.21.15.45:6789/0 conn(0x7ff854008ff0 legacy=0x7ff854008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.068 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:17.068 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:17.068 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:17.068 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:17.068 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:17.068 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:17.068 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:17.068 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:17.068 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:17.068 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:17.068 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.069 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.070 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:17.071 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.074 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.075 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.078 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:17.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:17.078 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.078 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.078 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:17.078 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.078 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.078 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.078 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:17.079 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:17.079 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.079 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.079 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.079 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:17.079 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:17.079 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.079 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.079 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.080 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:17.080 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.080 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.080 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.080 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:17.080 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:17.080 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.080 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.080 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.080 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.081 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:17.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:17.081 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.081 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.081 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.081 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:17.081 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.081 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.081 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.081 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:17.082 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.083 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.083 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.083 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:17.083 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:17.083 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.083 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.083 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.083 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:17.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.084 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.085 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.085 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:17.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:17.085 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.085 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.085 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:17.085 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.085 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:17.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.087 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:17.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.088 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.088 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.088 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:12:17.088 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.088 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.088 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:17.088 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:17.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.044+0000 7ff8752b8640 1 -- 172.21.15.45:0/3002822330 >> v1:172.21.15.84:6812/31478 conn(0x7ff85003aa50 legacy=0x7ff85003cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.044+0000 7ff8752b8640 10 monclient: shutdown 2024-09-18T14:12:17.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.044+0000 7ff8752b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:17.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.044+0000 7ff8752b8640 1 -- 172.21.15.45:0/3002822330 >> v1:172.21.15.84:6791/0 conn(0x7ff87009fba0 legacy=0x7ff870131a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.044+0000 7ff86f7fe640 1 -- 172.21.15.45:0/3002822330 reap_dead start 2024-09-18T14:12:17.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.044+0000 7ff86f7fe640 1 -- 172.21.15.45:0/3002822330 reap_dead start 2024-09-18T14:12:17.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.044+0000 7ff8752b8640 1 -- 172.21.15.45:0/3002822330 shutdown_connections 2024-09-18T14:12:17.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.044+0000 7ff8752b8640 1 -- 172.21.15.45:0/3002822330 >> 172.21.15.45:0/3002822330 conn(0x7ff8700630b0 msgr2=0x7ff8700a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:17.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.044+0000 7ff8752b8640 1 -- 172.21.15.45:0/3002822330 shutdown_connections 2024-09-18T14:12:17.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.044+0000 7ff8752b8640 1 -- 172.21.15.45:0/3002822330 wait complete. 2024-09-18T14:12:17.090 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:17.090 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:12:17.091 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:12:17.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.186+0000 7f0cfb59d640 10 monclient: get_monmap_and_config 2024-09-18T14:12:17.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.186+0000 7f0cfb59d640 10 monclient: build_initial_monmap 2024-09-18T14:12:17.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.186+0000 7f0cfb59d640 10 monclient: monmap: 2024-09-18T14:12:17.188 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:17.188 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.188 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:17.188696+0000 2024-09-18T14:12:17.188 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:17.188696+0000 2024-09-18T14:12:17.188 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:17.188 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.188 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:17.189 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:17.189 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:17.189 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:17.189 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:17.189 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:17.189 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:17.189 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:17.189 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:17.189 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.187+0000 7f0cfb59d640 1 Processor -- start 2024-09-18T14:12:17.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.187+0000 7f0cfb59d640 1 -- start start 2024-09-18T14:12:17.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.187+0000 7f0cfb59d640 10 monclient: init 2024-09-18T14:12:17.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.187+0000 7f0cfb59d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:17.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.187+0000 7f0cfb59d640 10 monclient: _add_conns ranks=[4,1,3,8,6,0,5,2,7] 2024-09-18T14:12:17.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.187+0000 7f0cfb59d640 10 monclient(hunting): picked mon.noname-c con 0x7f0cf409f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:17.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.187+0000 7f0cfb59d640 10 monclient(hunting): picked mon.noname-f con 0x7f0cf409fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:17.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.187+0000 7f0cfb59d640 10 monclient(hunting): picked mon.noname-g con 0x7f0cf409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:17.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.187+0000 7f0cfb59d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf405ed70 con 0x7f0cf409fba0 2024-09-18T14:12:17.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cfb59d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf409dfc0 con 0x7f0cf409ff70 2024-09-18T14:12:17.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cfb59d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf409dce0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cfb59d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cfb59d640 10 monclient(hunting): authenticate will time out at 1552.527038s 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cf9339640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f0cf409f7d0 0x7f0cf4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:55878/0 (socket says 172.21.15.45:55878) 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cf9339640 1 -- 172.21.15.45:0/1646731373 learned_addr learned my addr 172.21.15.45:0/1646731373 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337017116 0 0) 0x7f0cf409dce0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient(hunting): my global_id is 164677 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ce00034e0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858083930 0 0) 0x7f0cf409dfc0 con 0x7f0cf409ff70 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient(hunting): my global_id is 184386 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf409dce0 con 0x7f0cf409ff70 2024-09-18T14:12:17.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572731799 0 0) 0x7f0cf405ed70 con 0x7f0cf409fba0 2024-09-18T14:12:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient(hunting): my global_id is 204175 2024-09-18T14:12:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf409dfc0 con 0x7f0cf409fba0 2024-09-18T14:12:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2441720875 0 0) 0x7f0ce00034e0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 >> v1:172.21.15.84:6790/0 conn(0x7f0cf409ff70 legacy=0x7f0cf413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 >> v1:172.21.15.84:6789/0 conn(0x7f0cf409fba0 legacy=0x7f0cf4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient: _finish_hunting 0 2024-09-18T14:12:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 monclient: found mon.noname-c 2024-09-18T14:12:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:12:17.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf40a1c00 con 0x7f0cf409f7d0 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient: _finish_auth 0 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0cf0001830 con 0x7f0cf409f7d0 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient: dump: 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.201 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cfb59d640 5 monclient: authenticate success, global_id 164677 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cfb59d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0cf0001990 con 0x7f0cf409f7d0 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:17.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 1 -- 172.21.15.45:0/1646731373 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0cf0004ba0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cfb59d640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cfb59d640 4 set_mon_vals no callback set 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.188+0000 7f0cebfff640 10 monclient: dump: 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.203 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.204 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.204 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.204 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.204 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.204 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.204 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.189+0000 7f0cfb59d640 10 monclient: shutdown 2024-09-18T14:12:17.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.189+0000 7f0cfb59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:17.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.189+0000 7f0cfb59d640 1 -- 172.21.15.45:0/1646731373 >> v1:172.21.15.45:6791/0 conn(0x7f0cf409f7d0 legacy=0x7f0cf4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.189+0000 7f0cfb59d640 1 -- 172.21.15.45:0/1646731373 shutdown_connections 2024-09-18T14:12:17.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.189+0000 7f0cfb59d640 1 -- 172.21.15.45:0/1646731373 >> 172.21.15.45:0/1646731373 conn(0x7f0cf40630b0 msgr2=0x7f0cf40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:17.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.189+0000 7f0cfb59d640 1 -- 172.21.15.45:0/1646731373 shutdown_connections 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.189+0000 7f0cfb59d640 1 -- 172.21.15.45:0/1646731373 wait complete. 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.189+0000 7f0cfb59d640 10 monclient: build_initial_monmap 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.189+0000 7f0cfb59d640 10 monclient: monmap: 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:17.205 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 1 Processor -- start 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 1 -- start start 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 10 monclient: init 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 10 monclient: _add_conns ranks=[2,5,3,0,8,7,1,4,6] 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 10 monclient(hunting): picked mon.noname-b con 0x7f0cf409f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 10 monclient(hunting): picked mon.noname-h con 0x7f0cf409fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:17.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 10 monclient(hunting): picked mon.noname-g con 0x7f0cf409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf409d310 con 0x7f0cf409f7d0 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf40a2560 con 0x7f0cf409ff70 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf4121a90 con 0x7f0cf409fba0 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.190+0000 7f0cfb59d640 10 monclient(hunting): authenticate will time out at 1552.529999s 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0cf9339640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f0cf409f7d0 0x7f0cf4131660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55870/0 (socket says 172.21.15.45:55870) 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0cf9339640 1 -- 172.21.15.45:0/2106021414 learned_addr learned my addr 172.21.15.45:0/2106021414 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198713450 0 0) 0x7f0cf409d310 con 0x7f0cf409f7d0 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): my global_id is 145163 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd80034e0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171493694 0 0) 0x7f0cf40a2560 con 0x7f0cf409ff70 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): my global_id is 184395 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf409d310 con 0x7f0cf409ff70 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2478608555 0 0) 0x7f0cf4121a90 con 0x7f0cf409fba0 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): my global_id is 184424 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf40a2560 con 0x7f0cf409fba0 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2795202406 0 0) 0x7f0cd80034e0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cf4121a90 con 0x7f0cf409f7d0 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0cf00047a0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): dump: 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.209 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1616372471 0 0) 0x7f0cf40a2560 con 0x7f0cf409fba0 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cd80034e0 con 0x7f0cf409fba0 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1485568438 0 0) 0x7f0cf409d310 con 0x7f0cf409ff70 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cf40a2560 con 0x7f0cf409ff70 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ce4001990 con 0x7f0cf409fba0 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:12:17.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): dump: 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0cec001d00 con 0x7f0cf409ff70 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient(hunting): dump: 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.213 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1608592017 0 0) 0x7f0cf4121a90 con 0x7f0cf409f7d0 2024-09-18T14:12:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 >> v1:172.21.15.84:6791/0 conn(0x7f0cf409fba0 legacy=0x7f0cf4146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 >> v1:172.21.15.84:6790/0 conn(0x7f0cf409ff70 legacy=0x7f0cf414a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient: _finish_hunting 0 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 monclient: found mon.b 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf411fcf0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient: _finish_auth 0 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0cfb59d640 5 monclient: authenticate success, global_id 145163 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0cfb59d640 10 monclient: _renew_subs 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0cfb59d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0cfb59d640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf41217b0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0cfb59d640 10 monclient: _renew_subs 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0cfb59d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0cfb59d640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf4131d70 con 0x7f0cf409f7d0 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0cf0004e20 con 0x7f0cf409f7d0 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0cf00052d0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ceb7fe640 4 set_mon_vals no callback set 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.191+0000 7f0ce97fa640 10 monclient: dump: 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.216 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.216 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.216 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.216 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.216 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.216 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.216 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.216 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.216 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.192+0000 7f0cfb59d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.192+0000 7f0cfb59d640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0cd4006580 con 0x7f0cd4002830 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.192+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f0cf00102a0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.192+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(628..628 src has 424..628) v4 ==== 3911+0+0 (unknown 334880263 0 0) 0x7f0cf0046de0 con 0x7f0cf409f7d0 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098684614 0 0) 0x7f0cd4006580 con 0x7f0cd4002830 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 10 monclient(hunting): my global_id is 145172 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf4121a90 con 0x7f0cd4002830 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0cf9b3a640 10 monclient: get_auth_request con 0x7f0cd803aa70 auth_method 0 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3216665143 0 0) 0x7f0cf4121a90 con 0x7f0cd4002830 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cd4006580 con 0x7f0cd4002830 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ce4002020 con 0x7f0cd4002830 2024-09-18T14:12:17.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 10 monclient: dump: 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.218 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.219 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 33185275 0 0) 0x7f0cd4006580 con 0x7f0cd4002830 2024-09-18T14:12:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cd400a0a0 con 0x7f0cd4002830 2024-09-18T14:12:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0ce40042a0 con 0x7f0cd4002830 2024-09-18T14:12:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 >> v1:172.21.15.45:6790/0 conn(0x7f0cd4002830 legacy=0x7f0cd4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.193+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 >> v1:172.21.15.45:6790/0 conn(0x7f0cd4002830 legacy=0x7f0cd4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.199+0000 7f0cfb59d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.199+0000 7f0cfb59d640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0cd40074f0 con 0x7f0cd40068a0 2024-09-18T14:12:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.199+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102233222 0 0) 0x7f0cd40074f0 con 0x7f0cd40068a0 2024-09-18T14:12:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.199+0000 7f0ce97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.199+0000 7f0ce97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.199+0000 7f0ce97fa640 10 monclient(hunting): my global_id is 145181 2024-09-18T14:12:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.199+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd4006580 con 0x7f0cd40068a0 2024-09-18T14:12:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.199+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2710498254 0 0) 0x7f0cd4006580 con 0x7f0cd40068a0 2024-09-18T14:12:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cd40074f0 con 0x7f0cd40068a0 2024-09-18T14:12:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ce4002aa0 con 0x7f0cd40068a0 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0ce97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0ce97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0ce97fa640 10 monclient: dump: 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.223 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 819350478 0 0) 0x7f0cd40074f0 con 0x7f0cd40068a0 2024-09-18T14:12:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0cd4008750 con 0x7f0cd40068a0 2024-09-18T14:12:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 1161378326) 0x7f0ce4007290 con 0x7f0cd40068a0 2024-09-18T14:12:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0ce97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0ce97fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 >> v1:172.21.15.45:6790/0 conn(0x7f0cd40068a0 legacy=0x7f0cd4006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0ce97fa640 1 -- 172.21.15.45:0/2106021414 >> v1:172.21.15.45:6790/0 conn(0x7f0cd40068a0 legacy=0x7f0cd4006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.224 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:17.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:17.224 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:17.224 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:17.224 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:17.224 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:17.224 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:17.224 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:17.224 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:17.224 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:17.224 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.225 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.226 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:17.227 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.228 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.229 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.229 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.229 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.229 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.229 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.229 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.229 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:17.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.229 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:17.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.229 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:17.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.230 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.231 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.233 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:17.233 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.233 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.233 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.233 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:17.233 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:17.233 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.233 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.233 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:17.243 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:17.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0cfb59d640 1 -- 172.21.15.45:0/2106021414 >> v1:172.21.15.84:6812/31478 conn(0x7f0cd803aa70 legacy=0x7f0cd803cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0cfb59d640 10 monclient: shutdown 2024-09-18T14:12:17.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0cfb59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:17.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0cfb59d640 1 -- 172.21.15.45:0/2106021414 >> v1:172.21.15.45:6790/0 conn(0x7f0cf409f7d0 legacy=0x7f0cf4131660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0cf9b3a640 1 -- 172.21.15.45:0/2106021414 reap_dead start 2024-09-18T14:12:17.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0cf9b3a640 1 -- 172.21.15.45:0/2106021414 reap_dead start 2024-09-18T14:12:17.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0cfb59d640 1 -- 172.21.15.45:0/2106021414 shutdown_connections 2024-09-18T14:12:17.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0cfb59d640 1 -- 172.21.15.45:0/2106021414 >> 172.21.15.45:0/2106021414 conn(0x7f0cf40630b0 msgr2=0x7f0cf40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:17.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0cfb59d640 1 -- 172.21.15.45:0/2106021414 shutdown_connections 2024-09-18T14:12:17.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.200+0000 7f0cfb59d640 1 -- 172.21.15.45:0/2106021414 wait complete. 2024-09-18T14:12:17.246 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:17.246 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:12:17.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:12:17.300 INFO:tasks.rados.rados.0.smithi084.stdout:2831: done (4 left) 2024-09-18T14:12:17.300 INFO:tasks.rados.rados.0.smithi084.stdout:2832: done (3 left) 2024-09-18T14:12:17.300 INFO:tasks.rados.rados.0.smithi084.stdout:2833: done (2 left) 2024-09-18T14:12:17.301 INFO:tasks.rados.rados.0.smithi084.stdout:2834: done (1 left) 2024-09-18T14:12:17.301 INFO:tasks.rados.rados.0.smithi084.stdout:2835: done (0 left) 2024-09-18T14:12:17.301 INFO:tasks.rados.rados.0.smithi084.stdout:2836: write oid 3 current snap is 308 2024-09-18T14:12:17.301 INFO:tasks.rados.rados.0.smithi084.stdout:2836: seq_num 967 ranges {652369=742614,2153264=99105} 2024-09-18T14:12:17.307 INFO:tasks.rados.rados.0.smithi084.stdout:2836: writing smithi08434663-3 from 652369 to 1394983 tid 1 2024-09-18T14:12:17.312 INFO:tasks.rados.rados.0.smithi084.stdout:2836: writing smithi08434663-3 from 2153264 to 2252369 tid 2 2024-09-18T14:12:17.312 INFO:tasks.rados.rados.0.smithi084.stdout:2837: write oid 17 current snap is 308 2024-09-18T14:12:17.313 INFO:tasks.rados.rados.0.smithi084.stdout:2837: seq_num 968 ranges {669176=417863,1811746=457430} 2024-09-18T14:12:17.318 INFO:tasks.rados.rados.0.smithi084.stdout:2837: writing smithi08434663-17 from 669176 to 1087039 tid 1 2024-09-18T14:12:17.325 INFO:tasks.rados.rados.0.smithi084.stdout:2837: writing smithi08434663-17 from 1811746 to 2269176 tid 2 2024-09-18T14:12:17.325 INFO:tasks.rados.rados.0.smithi084.stdout:2838: delete oid 19 current snap is 308 2024-09-18T14:12:17.325 INFO:tasks.rados.rados.0.smithi084.stdout:2836: finishing write tid 1 to smithi08434663-3 2024-09-18T14:12:17.325 INFO:tasks.rados.rados.0.smithi084.stdout:2836: finishing write tid 2 to smithi08434663-3 2024-09-18T14:12:17.325 INFO:tasks.rados.rados.0.smithi084.stdout:2836: finishing write tid 3 to smithi08434663-3 2024-09-18T14:12:17.325 INFO:tasks.rados.rados.0.smithi084.stdout:2836: finishing write tid 4 to smithi08434663-3 2024-09-18T14:12:17.325 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 1037 (ObjNum 967 snap 308 seq_num 967) dirty exists 2024-09-18T14:12:17.325 INFO:tasks.rados.rados.0.smithi084.stdout:2836: left oid 3 (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:12:17.326 INFO:tasks.rados.rados.0.smithi084.stdout:2837: finishing write tid 1 to smithi08434663-17 2024-09-18T14:12:17.329 INFO:tasks.rados.rados.0.smithi084.stdout:2837: finishing write tid 2 to smithi08434663-17 2024-09-18T14:12:17.329 INFO:tasks.rados.rados.0.smithi084.stdout:2837: finishing write tid 3 to smithi08434663-17 2024-09-18T14:12:17.329 INFO:tasks.rados.rados.0.smithi084.stdout:2837: finishing write tid 4 to smithi08434663-17 2024-09-18T14:12:17.329 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1046 (ObjNum 968 snap 308 seq_num 968) dirty exists 2024-09-18T14:12:17.329 INFO:tasks.rados.rados.0.smithi084.stdout:2837: left oid 17 (ObjNum 968 snap 308 seq_num 968) 2024-09-18T14:12:17.329 INFO:tasks.rados.rados.0.smithi084.stdout:2836: done (2 left) 2024-09-18T14:12:17.329 INFO:tasks.rados.rados.0.smithi084.stdout:2837: done (1 left) 2024-09-18T14:12:17.329 INFO:tasks.rados.rados.0.smithi084.stdout:2838: done (0 left) 2024-09-18T14:12:17.330 INFO:tasks.rados.rados.0.smithi084.stdout:2839: write oid 48 current snap is 308 2024-09-18T14:12:17.330 INFO:tasks.rados.rados.0.smithi084.stdout:2839: seq_num 970 ranges {702790=568361,1812357=490433} 2024-09-18T14:12:17.335 INFO:tasks.rados.rados.0.smithi084.stdout:2839: writing smithi08434663-48 from 702790 to 1271151 tid 1 2024-09-18T14:12:17.341 INFO:tasks.rados.rados.0.smithi084.stdout:2839: writing smithi08434663-48 from 1812357 to 2302790 tid 2 2024-09-18T14:12:17.341 INFO:tasks.rados.rados.0.smithi084.stdout:2840: write oid 28 current snap is 308 2024-09-18T14:12:17.342 INFO:tasks.rados.rados.0.smithi084.stdout:2840: seq_num 971 ranges {719597=643610,1870839=448758} 2024-09-18T14:12:17.349 INFO:tasks.rados.rados.0.smithi084.stdout:2840: writing smithi08434663-28 from 719597 to 1363207 tid 1 2024-09-18T14:12:17.362 INFO:tasks.rados.rados.0.smithi084.stdout:2840: writing smithi08434663-28 from 1870839 to 2319597 tid 2 2024-09-18T14:12:17.362 INFO:tasks.rados.rados.0.smithi084.stdout:2841: read oid 21 snap 292 2024-09-18T14:12:17.363 INFO:tasks.rados.rados.0.smithi084.stdout:2841: expect deleted 2024-09-18T14:12:17.363 INFO:tasks.rados.rados.0.smithi084.stdout:2842: snap_create 2024-09-18T14:12:17.363 INFO:tasks.rados.rados.0.smithi084.stdout:2839: finishing write tid 1 to smithi08434663-48 2024-09-18T14:12:17.363 INFO:tasks.rados.rados.0.smithi084.stdout:2839: finishing write tid 2 to smithi08434663-48 2024-09-18T14:12:17.363 INFO:tasks.rados.rados.0.smithi084.stdout:2839: finishing write tid 3 to smithi08434663-48 2024-09-18T14:12:17.363 INFO:tasks.rados.rados.0.smithi084.stdout:2839: finishing write tid 4 to smithi08434663-48 2024-09-18T14:12:17.363 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1257 (ObjNum 970 snap 308 seq_num 970) dirty exists 2024-09-18T14:12:17.363 INFO:tasks.rados.rados.0.smithi084.stdout:2839: left oid 48 (ObjNum 970 snap 308 seq_num 970) 2024-09-18T14:12:17.363 INFO:tasks.rados.rados.0.smithi084.stdout:2840: finishing write tid 1 to smithi08434663-28 2024-09-18T14:12:17.363 INFO:tasks.rados.rados.0.smithi084.stdout:2840: finishing write tid 2 to smithi08434663-28 2024-09-18T14:12:17.363 INFO:tasks.rados.rados.0.smithi084.stdout:2840: finishing write tid 3 to smithi08434663-28 2024-09-18T14:12:17.364 INFO:tasks.rados.rados.0.smithi084.stdout:2840: finishing write tid 4 to smithi08434663-28 2024-09-18T14:12:17.364 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1519 (ObjNum 971 snap 308 seq_num 971) dirty exists 2024-09-18T14:12:17.364 INFO:tasks.rados.rados.0.smithi084.stdout:2840: left oid 28 (ObjNum 971 snap 308 seq_num 971) 2024-09-18T14:12:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.350+0000 7fc7a1d1d640 10 monclient: get_monmap_and_config 2024-09-18T14:12:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.350+0000 7fc7a1d1d640 10 monclient: build_initial_monmap 2024-09-18T14:12:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.350+0000 7fc7a1d1d640 10 monclient: monmap: 2024-09-18T14:12:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:17.352901+0000 2024-09-18T14:12:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:17.352901+0000 2024-09-18T14:12:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:17.370 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.351+0000 7fc7a1d1d640 1 Processor -- start 2024-09-18T14:12:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.351+0000 7fc7a1d1d640 1 -- start start 2024-09-18T14:12:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.351+0000 7fc7a1d1d640 10 monclient: init 2024-09-18T14:12:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc7a1d1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc7a1d1d640 10 monclient: _add_conns ranks=[7,8,1,2,4,6,0,5,3] 2024-09-18T14:12:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc7a1d1d640 10 monclient(hunting): picked mon.noname-i con 0x7fc79c06e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc7a1d1d640 10 monclient(hunting): picked mon.noname-e con 0x7fc79c0a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc7a1d1d640 10 monclient(hunting): picked mon.noname-f con 0x7fc79c0a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc7a1d1d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c05ed70 con 0x7fc79c0a1080 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc7a1d1d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c097620 con 0x7fc79c06e810 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc7a1d1d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c097340 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc7a1d1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc7a1d1d640 10 monclient(hunting): authenticate will time out at 1552.691396s 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79affd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fc79c0a0cb0 0x7fc79c137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:41640/0 (socket says 172.21.15.45:41640) 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79affd640 1 -- 172.21.15.45:0/4026191128 learned_addr learned my addr 172.21.15.45:0/4026191128 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060644132 0 0) 0x7fc79c097340 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient(hunting): my global_id is 5714 2024-09-18T14:12:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7840034e0 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1357732409 0 0) 0x7fc79c097620 con 0x7fc79c06e810 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient(hunting): my global_id is 184462 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc79c097340 con 0x7fc79c06e810 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482792590 0 0) 0x7fc79c05ed70 con 0x7fc79c0a1080 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient(hunting): my global_id is 204184 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc79c097620 con 0x7fc79c0a1080 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858675278 0 0) 0x7fc7840034e0 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 >> v1:172.21.15.84:6792/0 conn(0x7fc79c06e810 legacy=0x7fc79c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 >> v1:172.21.15.84:6789/0 conn(0x7fc79c0a1080 legacy=0x7fc79c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 1 monclient: found mon.noname-e 2024-09-18T14:12:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:17.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:12:17.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c0a1660 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:17.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:17.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc790001830 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:12:17.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc79a7fc640 10 monclient: dump: 2024-09-18T14:12:17.375 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.375 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.375 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.375 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.375 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.376 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.376 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.376 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.376 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.376 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.377 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.377 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.377 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.377 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.377 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.378 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc7a1d1d640 5 monclient: authenticate success, global_id 5714 2024-09-18T14:12:17.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.352+0000 7fc7a1d1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:17.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc790001a10 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc79a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:17.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc79a7fc640 1 -- 172.21.15.45:0/4026191128 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc790004ba0 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc7a1d1d640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc7a1d1d640 4 set_mon_vals no callback set 2024-09-18T14:12:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc79a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc79a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc79a7fc640 10 monclient: dump: 2024-09-18T14:12:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.380 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.380 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.380 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.380 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.380 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.382 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.382 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.382 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.382 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc7a1d1d640 10 monclient: shutdown 2024-09-18T14:12:17.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc7a1d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/4026191128 >> v1:172.21.15.45:6793/0 conn(0x7fc79c0a0cb0 legacy=0x7fc79c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/4026191128 shutdown_connections 2024-09-18T14:12:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/4026191128 >> 172.21.15.45:0/4026191128 conn(0x7fc79c0a5bb0 msgr2=0x7fc79c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/4026191128 shutdown_connections 2024-09-18T14:12:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.353+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/4026191128 wait complete. 2024-09-18T14:12:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.354+0000 7fc7a1d1d640 10 monclient: build_initial_monmap 2024-09-18T14:12:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.354+0000 7fc7a1d1d640 10 monclient: monmap: 2024-09-18T14:12:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.354+0000 7fc7a1d1d640 1 Processor -- start 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.354+0000 7fc7a1d1d640 1 -- start start 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.354+0000 7fc7a1d1d640 10 monclient: init 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7a1d1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7a1d1d640 10 monclient: _add_conns ranks=[0,3,8,1,2,6,5,7,4] 2024-09-18T14:12:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7a1d1d640 10 monclient(hunting): picked mon.noname-a con 0x7fc79c06e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7a1d1d640 10 monclient(hunting): picked mon.noname-g con 0x7fc79c0a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7a1d1d640 10 monclient(hunting): picked mon.noname-e con 0x7fc79c0a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7a1d1d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c0a2270 con 0x7fc79c06e810 2024-09-18T14:12:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7a1d1d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c136a90 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc79b7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc79c06e810 0x7fc79c135310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60034/0 (socket says 172.21.15.45:60034) 2024-09-18T14:12:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc79b7fe640 1 -- 172.21.15.45:0/2104563231 learned_addr learned my addr 172.21.15.45:0/2104563231 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c120e80 con 0x7fc79c0a1080 2024-09-18T14:12:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7a1d1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7a1d1d640 10 monclient(hunting): authenticate will time out at 1552.694761s 2024-09-18T14:12:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464376969 0 0) 0x7fc79c120e80 con 0x7fc79c0a1080 2024-09-18T14:12:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7837fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7837fe640 10 monclient(hunting): my global_id is 5723 2024-09-18T14:12:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7880034e0 con 0x7fc79c0a1080 2024-09-18T14:12:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116969518 0 0) 0x7fc79c0a2270 con 0x7fc79c06e810 2024-09-18T14:12:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7837fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.355+0000 7fc7837fe640 10 monclient(hunting): my global_id is 5922 2024-09-18T14:12:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc79c120e80 con 0x7fc79c06e810 2024-09-18T14:12:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823457784 0 0) 0x7fc79c136a90 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): my global_id is 184404 2024-09-18T14:12:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc79c0a2270 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 322765054 0 0) 0x7fc7880034e0 con 0x7fc79c0a1080 2024-09-18T14:12:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc79c136a90 con 0x7fc79c0a1080 2024-09-18T14:12:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc794001d80 con 0x7fc79c0a1080 2024-09-18T14:12:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:12:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): dump: 2024-09-18T14:12:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.393 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2126670504 0 0) 0x7fc79c120e80 con 0x7fc79c06e810 2024-09-18T14:12:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc7880034e0 con 0x7fc79c06e810 2024-09-18T14:12:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc78c001aa0 con 0x7fc79c06e810 2024-09-18T14:12:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): dump: 2024-09-18T14:12:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.396 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4151538575 0 0) 0x7fc79c0a2270 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc79c120e80 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc7900017f0 con 0x7fc79c0a0cb0 2024-09-18T14:12:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient(hunting): dump: 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 185876332 0 0) 0x7fc79c136a90 con 0x7fc79c0a1080 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 >> v1:172.21.15.84:6790/0 conn(0x7fc79c0a0cb0 legacy=0x7fc79c1468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 >> v1:172.21.15.45:6789/0 conn(0x7fc79c06e810 legacy=0x7fc79c135310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient: _finish_hunting 0 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 monclient: found mon.e 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c121540 con 0x7fc79c0a1080 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient: _finish_auth 0 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7a1d1d640 5 monclient: authenticate success, global_id 5723 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7a1d1d640 10 monclient: _renew_subs 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7a1d1d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79c1229f0 con 0x7fc79c0a1080 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7a1d1d640 10 monclient: _renew_subs 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7a1d1d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79c120970 con 0x7fc79c0a1080 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc794002ff0 con 0x7fc79c0a1080 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc794002550 con 0x7fc79c0a1080 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc799ffb640 4 set_mon_vals no callback set 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.356+0000 7fc7837fe640 10 monclient: dump: 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.357+0000 7fc7a1d1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.357+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc77c006580 con 0x7fc77c002830 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.357+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc7940026b0 con 0x7fc79c0a1080 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(629..629 src has 424..629) v4 ==== 3911+0+0 (unknown 665274774 0 0) 0x7fc794044480 con 0x7fc79c0a1080 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909952426 0 0) 0x7fc77c006580 con 0x7fc77c002830 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 10 monclient(hunting): my global_id is 164686 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc79c136a90 con 0x7fc77c002830 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc79affd640 10 monclient: get_auth_request con 0x7fc78803aaf0 auth_method 0 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3211017331 0 0) 0x7fc79c136a90 con 0x7fc77c002830 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc77c006580 con 0x7fc77c002830 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc78c002230 con 0x7fc77c002830 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 10 monclient: dump: 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 853320889 0 0) 0x7fc77c006580 con 0x7fc77c002830 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc77c00a0a0 con 0x7fc77c002830 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc78c005780 con 0x7fc77c002830 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 >> v1:172.21.15.45:6791/0 conn(0x7fc77c002830 legacy=0x7fc77c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.358+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 >> v1:172.21.15.45:6791/0 conn(0x7fc77c002830 legacy=0x7fc77c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.364+0000 7fc7a1d1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.364+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc77c009660 con 0x7fc77c009040 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.364+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670603025 0 0) 0x7fc77c009660 con 0x7fc77c009040 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.364+0000 7fc7837fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.364+0000 7fc7837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.364+0000 7fc7837fe640 10 monclient(hunting): my global_id is 164695 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.364+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc77c006580 con 0x7fc77c009040 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 607144199 0 0) 0x7fc77c006580 con 0x7fc77c009040 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc77c009660 con 0x7fc77c009040 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc78c002cd0 con 0x7fc77c009040 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 10 monclient: dump: 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.404 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.405 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3845225626 0 0) 0x7fc77c009660 con 0x7fc77c009040 2024-09-18T14:12:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc77c009920 con 0x7fc77c009040 2024-09-18T14:12:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 1568593926) 0x7fc78c007cf0 con 0x7fc77c009040 2024-09-18T14:12:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 >> v1:172.21.15.45:6791/0 conn(0x7fc77c009040 legacy=0x7fc77c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7837fe640 1 -- 172.21.15.45:0/2104563231 >> v1:172.21.15.45:6791/0 conn(0x7fc77c009040 legacy=0x7fc77c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.407 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:17.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:17.407 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:17.407 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:17.408 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:17.408 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:17.408 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:17.408 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:17.408 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:17.409 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:17.409 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:17.409 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:17.409 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:17.409 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:17.409 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.410 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:17.412 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.415 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.418 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.420 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:17.420 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.420 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.420 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.420 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:17.420 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:17.420 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.421 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.421 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.421 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:17.421 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.422 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.424 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:17.425 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.426 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.426 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:17.426 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.426 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.426 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:17.426 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:17.426 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.426 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:17.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.428 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.428 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.428 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:12:17.428 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.428 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.429 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:17.429 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.365+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/2104563231 >> v1:172.21.15.84:6812/31478 conn(0x7fc78803aaf0 legacy=0x7fc78803cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.366+0000 7fc7a1d1d640 10 monclient: shutdown 2024-09-18T14:12:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.366+0000 7fc7a1d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.366+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/2104563231 >> v1:172.21.15.45:6793/0 conn(0x7fc79c0a1080 legacy=0x7fc79c149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.366+0000 7fc79bfff640 1 -- 172.21.15.45:0/2104563231 reap_dead start 2024-09-18T14:12:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.366+0000 7fc79bfff640 1 -- 172.21.15.45:0/2104563231 reap_dead start 2024-09-18T14:12:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.366+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/2104563231 shutdown_connections 2024-09-18T14:12:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.366+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/2104563231 >> 172.21.15.45:0/2104563231 conn(0x7fc79c0a5bb0 msgr2=0x7fc79c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.366+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/2104563231 shutdown_connections 2024-09-18T14:12:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.366+0000 7fc7a1d1d640 1 -- 172.21.15.45:0/2104563231 wait complete. 2024-09-18T14:12:17.432 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:17.432 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:12:17.432 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:12:17.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.530+0000 7fe34ef09640 10 monclient: get_monmap_and_config 2024-09-18T14:12:17.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.530+0000 7fe34ef09640 10 monclient: build_initial_monmap 2024-09-18T14:12:17.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.530+0000 7fe34ef09640 10 monclient: monmap: 2024-09-18T14:12:17.532 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:17.532 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.532 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:17.532508+0000 2024-09-18T14:12:17.532 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:17.532508+0000 2024-09-18T14:12:17.532 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:17.532 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.532 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:17.532 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:17.532 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:17.533 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:17.533 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:17.533 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:17.533 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:17.533 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:17.533 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:17.533 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 1 Processor -- start 2024-09-18T14:12:17.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 1 -- start start 2024-09-18T14:12:17.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 10 monclient: init 2024-09-18T14:12:17.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:17.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 10 monclient: _add_conns ranks=[2,6,0,7,1,8,3,5,4] 2024-09-18T14:12:17.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 10 monclient(hunting): picked mon.noname-b con 0x7fe348132470 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:17.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 10 monclient(hunting): picked mon.noname-d con 0x7fe34806e380 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:17.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 10 monclient(hunting): picked mon.noname-a con 0x7fe3481387b0 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:17.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34805ed70 con 0x7fe3481387b0 2024-09-18T14:12:17.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3480681d0 con 0x7fe348132470 2024-09-18T14:12:17.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe348132f50 con 0x7fe34806e380 2024-09-18T14:12:17.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:17.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34ef09640 10 monclient(hunting): authenticate will time out at 1552.870880s 2024-09-18T14:12:17.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34d4a6640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe3481387b0 0x7fe34813ab90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36944/0 (socket says 172.21.15.45:36944) 2024-09-18T14:12:17.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34cca5640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fe348132470 0x7fe348132840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:57316/0 (socket says 172.21.15.45:57316) 2024-09-18T14:12:17.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe34d4a6640 1 -- 172.21.15.45:0/1057122084 learned_addr learned my addr 172.21.15.45:0/1057122084 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:17.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.531+0000 7fe347fff640 1 --1- 172.21.15.45:0/1057122084 >> v1:172.21.15.45:6792/0 conn(0x7fe34806e380 0x7fe34806e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:55448/0 (socket says 172.21.15.45:55448) 2024-09-18T14:12:17.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900330970 0 0) 0x7fe34805ed70 con 0x7fe3481387b0 2024-09-18T14:12:17.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient(hunting): my global_id is 5931 2024-09-18T14:12:17.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3380034e0 con 0x7fe3481387b0 2024-09-18T14:12:17.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837514639 0 0) 0x7fe3480681d0 con 0x7fe348132470 2024-09-18T14:12:17.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient(hunting): my global_id is 145190 2024-09-18T14:12:17.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe34805ed70 con 0x7fe348132470 2024-09-18T14:12:17.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707008126 0 0) 0x7fe348132f50 con 0x7fe34806e380 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient(hunting): my global_id is 184479 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3480681d0 con 0x7fe34806e380 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 547251123 0 0) 0x7fe3380034e0 con 0x7fe3481387b0 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 >> v1:172.21.15.45:6792/0 conn(0x7fe34806e380 legacy=0x7fe34806e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 >> v1:172.21.15.45:6790/0 conn(0x7fe348132470 legacy=0x7fe348132840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient: _finish_hunting 0 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 monclient: found mon.noname-a 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3480a0220 con 0x7fe3481387b0 2024-09-18T14:12:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient: _finish_auth 0 2024-09-18T14:12:17.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:17.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe34ef09640 5 monclient: authenticate success, global_id 5931 2024-09-18T14:12:17.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe340001bb0 con 0x7fe3481387b0 2024-09-18T14:12:17.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe34ef09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:17.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:12:17.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient: dump: 2024-09-18T14:12:17.548 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.548 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.548 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.548 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.548 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.548 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.548 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.548 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.548 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe34ef09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe340001d90 con 0x7fe3481387b0 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe34ef09640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 1 -- 172.21.15.45:0/1057122084 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe340004f20 con 0x7fe3481387b0 2024-09-18T14:12:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe34ef09640 4 set_mon_vals no callback set 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.532+0000 7fe3477fe640 10 monclient: dump: 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.533+0000 7fe34ef09640 10 monclient: shutdown 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.533+0000 7fe34ef09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.533+0000 7fe34ef09640 1 -- 172.21.15.45:0/1057122084 >> v1:172.21.15.45:6789/0 conn(0x7fe3481387b0 legacy=0x7fe34813ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.533+0000 7fe34ef09640 1 -- 172.21.15.45:0/1057122084 shutdown_connections 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.533+0000 7fe34ef09640 1 -- 172.21.15.45:0/1057122084 >> 172.21.15.45:0/1057122084 conn(0x7fe3480a4530 msgr2=0x7fe3480a05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.533+0000 7fe34ef09640 1 -- 172.21.15.45:0/1057122084 shutdown_connections 2024-09-18T14:12:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.533+0000 7fe34ef09640 1 -- 172.21.15.45:0/1057122084 wait complete. 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.533+0000 7fe34ef09640 10 monclient: build_initial_monmap 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.533+0000 7fe34ef09640 10 monclient: monmap: 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 1 Processor -- start 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 1 -- start start 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 10 monclient: init 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 10 monclient: _add_conns ranks=[2,1,7,3,0,5,4,6,8] 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 10 monclient(hunting): picked mon.noname-b con 0x7fe34806e380 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 10 monclient(hunting): picked mon.noname-f con 0x7fe348132470 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 10 monclient(hunting): picked mon.noname-i con 0x7fe3481387b0 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe348068a10 con 0x7fe348132470 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe348068700 con 0x7fe34806e380 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3481349f0 con 0x7fe3481387b0 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34cca5640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fe34806e380 0x7fe34809b300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:57324/0 (socket says 172.21.15.45:57324) 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34cca5640 1 -- 172.21.15.45:0/1317457633 learned_addr learned my addr 172.21.15.45:0/1317457633 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.534+0000 7fe34ef09640 10 monclient(hunting): authenticate will time out at 1552.873886s 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492384470 0 0) 0x7fe348068700 con 0x7fe34806e380 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): my global_id is 145199 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe32c0034e0 con 0x7fe34806e380 2024-09-18T14:12:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471476869 0 0) 0x7fe348068a10 con 0x7fe348132470 2024-09-18T14:12:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): my global_id is 204193 2024-09-18T14:12:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe348068700 con 0x7fe348132470 2024-09-18T14:12:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159534254 0 0) 0x7fe3481349f0 con 0x7fe3481387b0 2024-09-18T14:12:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): my global_id is 184471 2024-09-18T14:12:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe348068a10 con 0x7fe3481387b0 2024-09-18T14:12:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4113473326 0 0) 0x7fe32c0034e0 con 0x7fe34806e380 2024-09-18T14:12:17.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe3481349f0 con 0x7fe34806e380 2024-09-18T14:12:17.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe33c0015f0 con 0x7fe34806e380 2024-09-18T14:12:17.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:12:17.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): dump: 2024-09-18T14:12:17.556 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.557 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.557 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.562 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3663201342 0 0) 0x7fe348068700 con 0x7fe348132470 2024-09-18T14:12:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe32c0034e0 con 0x7fe348132470 2024-09-18T14:12:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3555384825 0 0) 0x7fe3481349f0 con 0x7fe34806e380 2024-09-18T14:12:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 >> v1:172.21.15.84:6792/0 conn(0x7fe3481387b0 legacy=0x7fe348149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 >> v1:172.21.15.84:6789/0 conn(0x7fe348132470 legacy=0x7fe34809ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:12:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 monclient: found mon.b 2024-09-18T14:12:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34809cf40 con 0x7fe34806e380 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient: _finish_auth 0 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe34ef09640 5 monclient: authenticate success, global_id 145199 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe34ef09640 10 monclient: _renew_subs 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe34ef09640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe34ef09640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34809c120 con 0x7fe34806e380 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe34ef09640 10 monclient: _renew_subs 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe34ef09640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe34ef09640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34809c770 con 0x7fe34806e380 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe33c004a50 con 0x7fe34806e380 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe33c004f00 con 0x7fe34806e380 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe344ff9640 10 monclient: dump: 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.535+0000 7fe3467fc640 4 set_mon_vals no callback set 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.536+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fe33c00fed0 con 0x7fe34806e380 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.536+0000 7fe34ef09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.536+0000 7fe34ef09640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe34804c4a0 con 0x7fe348138090 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.536+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(629..629 src has 424..629) v4 ==== 3911+0+0 (unknown 665274774 0 0) 0x7fe33c046bf0 con 0x7fe34806e380 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.536+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1418353116 0 0) 0x7fe34804c4a0 con 0x7fe348138090 2024-09-18T14:12:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.536+0000 7fe344ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.536+0000 7fe344ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.536+0000 7fe344ff9640 10 monclient(hunting): my global_id is 184488 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.536+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3481349f0 con 0x7fe348138090 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.536+0000 7fe34d4a6640 10 monclient: get_auth_request con 0x7fe32c03a1e0 auth_method 0 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.536+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3044923715 0 0) 0x7fe3481349f0 con 0x7fe348138090 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.536+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe34804c4a0 con 0x7fe348138090 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.537+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe334004ae0 con 0x7fe348138090 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.537+0000 7fe344ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.537+0000 7fe344ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.537+0000 7fe344ff9640 10 monclient: dump: 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.567 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.537+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2843033734 0 0) 0x7fe34804c4a0 con 0x7fe348138090 2024-09-18T14:12:17.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.537+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe34809dc30 con 0x7fe348138090 2024-09-18T14:12:17.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.537+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe334006fc0 con 0x7fe348138090 2024-09-18T14:12:17.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.537+0000 7fe344ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:17.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.537+0000 7fe344ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:17.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.537+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 >> v1:172.21.15.45:6792/0 conn(0x7fe348138090 legacy=0x7fe3480a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.537+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 >> v1:172.21.15.45:6792/0 conn(0x7fe348138090 legacy=0x7fe3480a1ec0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe34ef09640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe34ef09640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe348138460 con 0x7fe34802a8d0 2024-09-18T14:12:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478976929 0 0) 0x7fe348138460 con 0x7fe34802a8d0 2024-09-18T14:12:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 10 monclient(hunting): my global_id is 184497 2024-09-18T14:12:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe34804c4a0 con 0x7fe34802a8d0 2024-09-18T14:12:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2120371593 0 0) 0x7fe34804c4a0 con 0x7fe34802a8d0 2024-09-18T14:12:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe348138460 con 0x7fe34802a8d0 2024-09-18T14:12:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe334005670 con 0x7fe34802a8d0 2024-09-18T14:12:17.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:17.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 10 monclient: dump: 2024-09-18T14:12:17.571 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.571 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.571 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.571 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.571 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.571 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3166503653 0 0) 0x7fe348138460 con 0x7fe34802a8d0 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe34804b720 con 0x7fe34802a8d0 2024-09-18T14:12:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 607300541) 0x7fe3340072d0 con 0x7fe34802a8d0 2024-09-18T14:12:17.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:17.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:17.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 >> v1:172.21.15.45:6792/0 conn(0x7fe34802a8d0 legacy=0x7fe3480a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe344ff9640 1 -- 172.21.15.45:0/1317457633 >> v1:172.21.15.45:6792/0 conn(0x7fe34802a8d0 legacy=0x7fe3480a25d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.573 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:17.574 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.576 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.577 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.577 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.577 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.577 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.577 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:17.577 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:17.577 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:17.577 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:17.577 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:17.577 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:17.577 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.578 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.579 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.579 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.579 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.579 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.579 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.579 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.579 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:17.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.579 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:17.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.580 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:17.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:17.580 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.580 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.580 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:17.580 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.580 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.580 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.581 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:17.581 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:17.581 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.581 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.581 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.581 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:17.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:17.582 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.582 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.582 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.582 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:17.582 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.582 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.582 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.582 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:17.582 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:17.582 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.583 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.585 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.586 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.589 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.592 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.593 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.594 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:17.594 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:17.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe34ef09640 1 -- 172.21.15.45:0/1317457633 >> v1:172.21.15.84:6812/31478 conn(0x7fe32c03a1e0 legacy=0x7fe32c03c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe34ef09640 10 monclient: shutdown 2024-09-18T14:12:17.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe34ef09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:17.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe34ef09640 1 -- 172.21.15.45:0/1317457633 >> v1:172.21.15.45:6790/0 conn(0x7fe34806e380 legacy=0x7fe34809b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe34d4a6640 1 -- 172.21.15.45:0/1317457633 reap_dead start 2024-09-18T14:12:17.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe34d4a6640 1 -- 172.21.15.45:0/1317457633 reap_dead start 2024-09-18T14:12:17.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe34ef09640 1 -- 172.21.15.45:0/1317457633 shutdown_connections 2024-09-18T14:12:17.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe34ef09640 1 -- 172.21.15.45:0/1317457633 >> 172.21.15.45:0/1317457633 conn(0x7fe3480a4530 msgr2=0x7fe3480670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:17.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe34ef09640 1 -- 172.21.15.45:0/1317457633 shutdown_connections 2024-09-18T14:12:17.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.544+0000 7fe34ef09640 1 -- 172.21.15.45:0/1317457633 wait complete. 2024-09-18T14:12:17.597 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:17.597 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:12:17.597 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:12:17.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 10 monclient: get_monmap_and_config 2024-09-18T14:12:17.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 10 monclient: build_initial_monmap 2024-09-18T14:12:17.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 10 monclient: monmap: 2024-09-18T14:12:17.695 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:17.695 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.695 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:17.695369+0000 2024-09-18T14:12:17.695 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:17.695369+0000 2024-09-18T14:12:17.695 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:17.695 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.695 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:17.695 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:17.696 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:17.696 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:17.696 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:17.696 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:17.696 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:17.696 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:17.696 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:17.696 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 1 Processor -- start 2024-09-18T14:12:17.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 1 -- start start 2024-09-18T14:12:17.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 10 monclient: init 2024-09-18T14:12:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 10 monclient: _add_conns ranks=[0,3,2,7,6,5,8,1,4] 2024-09-18T14:12:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 10 monclient(hunting): picked mon.noname-a con 0x7f79f009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 10 monclient(hunting): picked mon.noname-g con 0x7f79f009fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 10 monclient(hunting): picked mon.noname-b con 0x7f79f009ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f005ed70 con 0x7f79f009f7d0 2024-09-18T14:12:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f009dfc0 con 0x7f79f009ff70 2024-09-18T14:12:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f009dce0 con 0x7f79f009fba0 2024-09-18T14:12:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79f5cc9640 10 monclient(hunting): authenticate will time out at 1553.033710s 2024-09-18T14:12:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79effff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f79f009ff70 0x7f79f013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:57332/0 (socket says 172.21.15.45:57332) 2024-09-18T14:12:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79ef7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f79f009f7d0 0x7f79f0132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36960/0 (socket says 172.21.15.45:36960) 2024-09-18T14:12:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.694+0000 7f79effff640 1 -- 172.21.15.45:0/2358986663 learned_addr learned my addr 172.21.15.45:0/2358986663 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706138503 0 0) 0x7f79f005ed70 con 0x7f79f009f7d0 2024-09-18T14:12:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient(hunting): my global_id is 5940 2024-09-18T14:12:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79d80034e0 con 0x7f79f009f7d0 2024-09-18T14:12:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339099244 0 0) 0x7f79f009dfc0 con 0x7f79f009ff70 2024-09-18T14:12:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient(hunting): my global_id is 145208 2024-09-18T14:12:17.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f005ed70 con 0x7f79f009ff70 2024-09-18T14:12:17.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719235457 0 0) 0x7f79f009dce0 con 0x7f79f009fba0 2024-09-18T14:12:17.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient(hunting): my global_id is 184413 2024-09-18T14:12:17.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f009dfc0 con 0x7f79f009fba0 2024-09-18T14:12:17.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2913450657 0 0) 0x7f79d80034e0 con 0x7f79f009f7d0 2024-09-18T14:12:17.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 >> v1:172.21.15.84:6790/0 conn(0x7f79f009fba0 legacy=0x7f79f0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 >> v1:172.21.15.45:6790/0 conn(0x7f79f009ff70 legacy=0x7f79f013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 monclient: found mon.noname-a 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f00a1c00 con 0x7f79f009f7d0 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79f5cc9640 5 monclient: authenticate success, global_id 5940 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79f5cc9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f79e0001830 con 0x7f79f009f7d0 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:12:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient: dump: 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.711 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79f5cc9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f79e0001a10 con 0x7f79f009f7d0 2024-09-18T14:12:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79f5cc9640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 1 -- 172.21.15.45:0/2358986663 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f79e0004ae0 con 0x7f79f009f7d0 2024-09-18T14:12:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79f5cc9640 4 set_mon_vals no callback set 2024-09-18T14:12:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79ee7fc640 10 monclient: dump: 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79f5cc9640 10 monclient: shutdown 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79f5cc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.695+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2358986663 >> v1:172.21.15.45:6789/0 conn(0x7f79f009f7d0 legacy=0x7f79f0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.696+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2358986663 shutdown_connections 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.696+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2358986663 >> 172.21.15.45:0/2358986663 conn(0x7f79f00630b0 msgr2=0x7f79f00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.696+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2358986663 shutdown_connections 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.696+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2358986663 wait complete. 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.696+0000 7f79f5cc9640 10 monclient: build_initial_monmap 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.696+0000 7f79f5cc9640 10 monclient: monmap: 2024-09-18T14:12:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 1 Processor -- start 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 1 -- start start 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 10 monclient: init 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 10 monclient: _add_conns ranks=[2,7,8,1,5,6,4,0,3] 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 10 monclient(hunting): picked mon.noname-b con 0x7f79f009f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 10 monclient(hunting): picked mon.noname-i con 0x7f79f009fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 10 monclient(hunting): picked mon.noname-e con 0x7f79f009ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f009a840 con 0x7f79f009f7d0 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f00a2560 con 0x7f79f009fba0 2024-09-18T14:12:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f009a350 con 0x7f79f009ff70 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79ef7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f79f009f7d0 0x7f79f0098550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:57340/0 (socket says 172.21.15.45:57340) 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79f5cc9640 10 monclient(hunting): authenticate will time out at 1553.036697s 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79ef7fe640 1 -- 172.21.15.45:0/2707255358 learned_addr learned my addr 172.21.15.45:0/2707255358 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291931764 0 0) 0x7f79f009a840 con 0x7f79f009f7d0 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79d77fe640 10 monclient(hunting): my global_id is 145217 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79dc0034e0 con 0x7f79f009f7d0 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308176096 0 0) 0x7f79f009a350 con 0x7f79f009ff70 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.697+0000 7f79d77fe640 10 monclient(hunting): my global_id is 5732 2024-09-18T14:12:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f009a840 con 0x7f79f009ff70 2024-09-18T14:12:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316406987 0 0) 0x7f79f00a2560 con 0x7f79f009fba0 2024-09-18T14:12:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): my global_id is 184480 2024-09-18T14:12:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f009a350 con 0x7f79f009fba0 2024-09-18T14:12:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3490748308 0 0) 0x7f79dc0034e0 con 0x7f79f009f7d0 2024-09-18T14:12:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f79f00a2560 con 0x7f79f009f7d0 2024-09-18T14:12:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f79e0001430 con 0x7f79f009f7d0 2024-09-18T14:12:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): dump: 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.719 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 223945303 0 0) 0x7f79f009a840 con 0x7f79f009ff70 2024-09-18T14:12:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f79dc0034e0 con 0x7f79f009ff70 2024-09-18T14:12:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1120674735 0 0) 0x7f79f009a350 con 0x7f79f009fba0 2024-09-18T14:12:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f79f009a840 con 0x7f79f009fba0 2024-09-18T14:12:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f79e8001d80 con 0x7f79f009ff70 2024-09-18T14:12:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): dump: 2024-09-18T14:12:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f79e4001250 con 0x7f79f009fba0 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:12:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient(hunting): dump: 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.723 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1978536323 0 0) 0x7f79f00a2560 con 0x7f79f009f7d0 2024-09-18T14:12:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 >> v1:172.21.15.45:6793/0 conn(0x7f79f009ff70 legacy=0x7f79f0149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 >> v1:172.21.15.84:6792/0 conn(0x7f79f009fba0 legacy=0x7f79f0098c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient: _finish_hunting 0 2024-09-18T14:12:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 monclient: found mon.b 2024-09-18T14:12:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f009d310 con 0x7f79f009f7d0 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient: _finish_auth 0 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79f5cc9640 5 monclient: authenticate success, global_id 145217 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79f5cc9640 10 monclient: _renew_subs 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79f5cc9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f009bec0 con 0x7f79f009f7d0 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79f5cc9640 10 monclient: _renew_subs 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79f5cc9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f009c5c0 con 0x7f79f009f7d0 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f79e0001230 con 0x7f79f009f7d0 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f79e00050a0 con 0x7f79f009f7d0 2024-09-18T14:12:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79edffb640 4 set_mon_vals no callback set 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.698+0000 7f79d77fe640 10 monclient: dump: 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.699+0000 7f79f5cc9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.699+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f79d0006580 con 0x7f79d0002830 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.699+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f79e0010070 con 0x7f79f009f7d0 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.699+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(629..629 src has 424..629) v4 ==== 3911+0+0 (unknown 665274774 0 0) 0x7f79e0046cc0 con 0x7f79f009f7d0 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.699+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007406402 0 0) 0x7f79d0006580 con 0x7f79d0002830 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.699+0000 7f79d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.699+0000 7f79d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.699+0000 7f79d77fe640 10 monclient(hunting): my global_id is 5741 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.699+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f00a2560 con 0x7f79d0002830 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.699+0000 7f79effff640 10 monclient: get_auth_request con 0x7f79dc03aac0 auth_method 0 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 261852845 0 0) 0x7f79f00a2560 con 0x7f79d0002830 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f79d0006580 con 0x7f79d0002830 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f79e4002090 con 0x7f79d0002830 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 10 monclient: dump: 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 700910181 0 0) 0x7f79d0006580 con 0x7f79d0002830 2024-09-18T14:12:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79d000a0a0 con 0x7f79d0002830 2024-09-18T14:12:17.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f79e4004370 con 0x7f79d0002830 2024-09-18T14:12:17.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:17.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:17.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 >> v1:172.21.15.45:6793/0 conn(0x7f79d0002830 legacy=0x7f79d0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.700+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 >> v1:172.21.15.45:6793/0 conn(0x7f79d0002830 legacy=0x7f79d0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79f5cc9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f79d00074f0 con 0x7f79d00068a0 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3502890301 0 0) 0x7f79d00074f0 con 0x7f79d00068a0 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 10 monclient(hunting): my global_id is 5750 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79d0006580 con 0x7f79d00068a0 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 872892672 0 0) 0x7f79d0006580 con 0x7f79d00068a0 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f79d00074f0 con 0x7f79d00068a0 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f79e4002b90 con 0x7f79d00068a0 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 10 monclient: dump: 2024-09-18T14:12:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.733 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3907957421 0 0) 0x7f79d00074f0 con 0x7f79d00068a0 2024-09-18T14:12:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.706+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f79d0008750 con 0x7f79d00068a0 2024-09-18T14:12:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7559 (unknown 0 0 1517339226) 0x7f79e4007360 con 0x7f79d00068a0 2024-09-18T14:12:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79d77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79d77fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 >> v1:172.21.15.45:6793/0 conn(0x7f79d00068a0 legacy=0x7f79d0006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79d77fe640 1 -- 172.21.15.45:0/2707255358 >> v1:172.21.15.45:6793/0 conn(0x7f79d00068a0 legacy=0x7f79d0006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.733 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:12:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:17.754 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:17.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2707255358 >> v1:172.21.15.84:6812/31478 conn(0x7f79dc03aac0 legacy=0x7f79dc03cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79f5cc9640 10 monclient: shutdown 2024-09-18T14:12:17.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79f5cc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:17.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2707255358 >> v1:172.21.15.45:6790/0 conn(0x7f79f009f7d0 legacy=0x7f79f0098550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79effff640 1 -- 172.21.15.45:0/2707255358 reap_dead start 2024-09-18T14:12:17.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79effff640 1 -- 172.21.15.45:0/2707255358 reap_dead start 2024-09-18T14:12:17.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2707255358 shutdown_connections 2024-09-18T14:12:17.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2707255358 >> 172.21.15.45:0/2707255358 conn(0x7f79f00630b0 msgr2=0x7f79f0130ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:17.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2707255358 shutdown_connections 2024-09-18T14:12:17.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.707+0000 7f79f5cc9640 1 -- 172.21.15.45:0/2707255358 wait complete. 2024-09-18T14:12:17.756 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:17.756 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:12:17.757 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:12:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 10 monclient: get_monmap_and_config 2024-09-18T14:12:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 10 monclient: build_initial_monmap 2024-09-18T14:12:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 10 monclient: monmap: 2024-09-18T14:12:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:17.854572+0000 2024-09-18T14:12:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:17.854572+0000 2024-09-18T14:12:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.855 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:17.855 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:17.855 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:17.855 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:17.855 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:17.855 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:17.855 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:17.855 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:17.855 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:17.855 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 1 Processor -- start 2024-09-18T14:12:17.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 1 -- start start 2024-09-18T14:12:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 10 monclient: init 2024-09-18T14:12:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 10 monclient: _add_conns ranks=[8,4,1,6,2,0,7,3,5] 2024-09-18T14:12:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 10 monclient(hunting): picked mon.noname-e con 0x7f953409f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 10 monclient(hunting): picked mon.noname-c con 0x7f953409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 10 monclient(hunting): picked mon.noname-f con 0x7f953409ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953405ed70 con 0x7f953409ff70 2024-09-18T14:12:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953409dfc0 con 0x7f953409fba0 2024-09-18T14:12:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.853+0000 7f953b79d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953409dce0 con 0x7f953409f7d0 2024-09-18T14:12:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f953b79d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f953b79d640 10 monclient(hunting): authenticate will time out at 1553.193025s 2024-09-18T14:12:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f9539539640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f953409f7d0 0x7f9534132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:41896/0 (socket says 172.21.15.45:41896) 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f9539539640 1 -- 172.21.15.45:0/1972832335 learned_addr learned my addr 172.21.15.45:0/1972832335 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f9538d38640 1 --1- 172.21.15.45:0/1972832335 >> v1:172.21.15.45:6791/0 conn(0x7f953409fba0 0x7f9534137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:52086/0 (socket says 172.21.15.45:52086) 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997858479 0 0) 0x7f953409dce0 con 0x7f953409f7d0 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient(hunting): my global_id is 5759 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95200034e0 con 0x7f953409f7d0 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990129951 0 0) 0x7f953409dfc0 con 0x7f953409fba0 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient(hunting): my global_id is 164704 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f953409dce0 con 0x7f953409fba0 2024-09-18T14:12:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502564026 0 0) 0x7f953405ed70 con 0x7f953409ff70 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient(hunting): my global_id is 204202 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f953409dfc0 con 0x7f953409ff70 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2073527075 0 0) 0x7f95200034e0 con 0x7f953409f7d0 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 >> v1:172.21.15.45:6791/0 conn(0x7f953409fba0 legacy=0x7f9534137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 >> v1:172.21.15.84:6789/0 conn(0x7f953409ff70 legacy=0x7f953413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 monclient: found mon.noname-e 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95340a1c00 con 0x7f953409f7d0 2024-09-18T14:12:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient: _finish_auth 0 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9530001830 con 0x7f953409f7d0 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f953b79d640 5 monclient: authenticate success, global_id 5759 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f953b79d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient: dump: 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f953b79d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9530001a10 con 0x7f953409f7d0 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 1 -- 172.21.15.45:0/1972832335 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9530004ba0 con 0x7f953409f7d0 2024-09-18T14:12:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f952bfff640 10 monclient: dump: 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.871 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f953b79d640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.854+0000 7f953b79d640 4 set_mon_vals no callback set 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.855+0000 7f953b79d640 10 monclient: shutdown 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.855+0000 7f953b79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.855+0000 7f953b79d640 1 -- 172.21.15.45:0/1972832335 >> v1:172.21.15.45:6793/0 conn(0x7f953409f7d0 legacy=0x7f9534132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.855+0000 7f953b79d640 1 -- 172.21.15.45:0/1972832335 shutdown_connections 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.855+0000 7f953b79d640 1 -- 172.21.15.45:0/1972832335 >> 172.21.15.45:0/1972832335 conn(0x7f95340630b0 msgr2=0x7f95340634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.855+0000 7f953b79d640 1 -- 172.21.15.45:0/1972832335 shutdown_connections 2024-09-18T14:12:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.855+0000 7f953b79d640 1 -- 172.21.15.45:0/1972832335 wait complete. 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.855+0000 7f953b79d640 10 monclient: build_initial_monmap 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.855+0000 7f953b79d640 10 monclient: monmap: 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 1 Processor -- start 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 1 -- start start 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 10 monclient: init 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 10 monclient: _add_conns ranks=[2,7,5,8,4,3,1,6,0] 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 10 monclient(hunting): picked mon.noname-b con 0x7f953409f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 10 monclient(hunting): picked mon.noname-i con 0x7f953409fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 10 monclient(hunting): picked mon.noname-h con 0x7f953409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953409d310 con 0x7f953409f7d0 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95340a2560 con 0x7f953409ff70 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953411e780 con 0x7f953409fba0 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f953b79d640 10 monclient(hunting): authenticate will time out at 1553.195855s 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f9539539640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f953409f7d0 0x7f95341315d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:57354/0 (socket says 172.21.15.45:57354) 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.856+0000 7f9539539640 1 -- 172.21.15.45:0/3862777062 learned_addr learned my addr 172.21.15.45:0/3862777062 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742861891 0 0) 0x7f953409d310 con 0x7f953409f7d0 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): my global_id is 145226 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95180034e0 con 0x7f953409f7d0 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985865559 0 0) 0x7f953411e780 con 0x7f953409fba0 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): my global_id is 184489 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f953409d310 con 0x7f953409fba0 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988150038 0 0) 0x7f95340a2560 con 0x7f953409ff70 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): my global_id is 184433 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f953411e780 con 0x7f953409ff70 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1789879242 0 0) 0x7f95180034e0 con 0x7f953409f7d0 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95340a2560 con 0x7f953409f7d0 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9530004ae0 con 0x7f953409f7d0 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:12:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): dump: 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2514060345 0 0) 0x7f953409d310 con 0x7f953409fba0 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95180034e0 con 0x7f953409fba0 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9524001420 con 0x7f953409fba0 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): dump: 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 35131049 0 0) 0x7f953411e780 con 0x7f953409ff70 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f953409d310 con 0x7f953409ff70 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f952c001d00 con 0x7f953409ff70 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f9539539640 1 -- 172.21.15.45:0/3862777062 >> v1:172.21.15.45:6790/0 conn(0x7f953409f7d0 legacy=0x7f95341315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f9539539640 1 -- 172.21.15.45:0/3862777062 >> v1:172.21.15.45:6790/0 conn(0x7f953409f7d0 legacy=0x7f95341315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f9539539640 1 --1- 172.21.15.45:0/3862777062 >> v1:172.21.15.45:6790/0 conn(0x7f953409f7d0 0x7f95341315d0 :-1 s=OPENED pgs=469 cs=1 l=1).handle_message read tag failed 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f9539539640 1 --1- 172.21.15.45:0/3862777062 >> v1:172.21.15.45:6790/0 conn(0x7f953409f7d0 0x7f95341315d0 :-1 s=OPENED pgs=469 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): dump: 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.880 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.45:6790/0 2024-09-18T14:12:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2870656722 0 0) 0x7f95180034e0 con 0x7f953409fba0 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 >> v1:172.21.15.84:6791/0 conn(0x7f953409ff70 legacy=0x7f953414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 >> v1:172.21.15.45:6790/0 conn(0x7f953409f7d0 legacy=0x7f95341315d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient: _finish_hunting 0 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 monclient: found mon.i 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9534136ea0 con 0x7f953409fba0 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient: _finish_auth 0 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f953b79d640 5 monclient: authenticate success, global_id 184489 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f953b79d640 10 monclient: _renew_subs 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f953b79d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:12:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f953b79d640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953409bba0 con 0x7f953409fba0 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f953b79d640 10 monclient: _renew_subs 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f953b79d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f953b79d640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f953409b690 con 0x7f953409fba0 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f95240047a0 con 0x7f953409fba0 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9524004c50 con 0x7f953409fba0 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f952b7fe640 4 set_mon_vals no callback set 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.857+0000 7f95297fa640 10 monclient: dump: 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.858+0000 7f953b79d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:17.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.858+0000 7f953b79d640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9514006580 con 0x7f9514002830 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.858+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f952400fca0 con 0x7f953409fba0 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.858+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(629..629 src has 424..629) v4 ==== 3911+0+0 (unknown 665274774 0 0) 0x7f9524046a50 con 0x7f953409fba0 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.858+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083520857 0 0) 0x7f9514006580 con 0x7f9514002830 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.858+0000 7f95297fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.858+0000 7f95297fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.858+0000 7f95297fa640 10 monclient(hunting): my global_id is 204211 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.858+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95180034e0 con 0x7f9514002830 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f9539d3a640 10 monclient: get_auth_request con 0x7f951803a8e0 auth_method 0 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2212507069 0 0) 0x7f95180034e0 con 0x7f9514002830 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9514006580 con 0x7f9514002830 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f95300051a0 con 0x7f9514002830 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:12:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 10 monclient: dump: 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.887 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2116704401 0 0) 0x7f9514006580 con 0x7f9514002830 2024-09-18T14:12:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f951400a0a0 con 0x7f9514002830 2024-09-18T14:12:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9530005440 con 0x7f9514002830 2024-09-18T14:12:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 >> v1:172.21.15.84:6789/0 conn(0x7f9514002830 legacy=0x7f9514004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.859+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 >> v1:172.21.15.84:6789/0 conn(0x7f9514002830 legacy=0x7f9514004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f953b79d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:17.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f953b79d640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9514009660 con 0x7f9514009040 2024-09-18T14:12:17.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257807331 0 0) 0x7f9514009660 con 0x7f9514009040 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 10 monclient(hunting): my global_id is 204220 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9514006580 con 0x7f9514009040 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3692436387 0 0) 0x7f9514006580 con 0x7f9514009040 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9514009660 con 0x7f9514009040 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9530005b60 con 0x7f9514009040 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 10 monclient: dump: 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.889 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4261471163 0 0) 0x7f9514009660 con 0x7f9514009040 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9514009920 con 0x7f9514009040 2024-09-18T14:12:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 4021466093) 0x7f9530005f00 con 0x7f9514009040 2024-09-18T14:12:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 >> v1:172.21.15.84:6789/0 conn(0x7f9514009040 legacy=0x7f9514008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.866+0000 7f95297fa640 1 -- 172.21.15.45:0/3862777062 >> v1:172.21.15.84:6789/0 conn(0x7f9514009040 legacy=0x7f9514008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:17.891 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:17.910 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.867+0000 7f953b79d640 1 -- 172.21.15.45:0/3862777062 >> v1:172.21.15.84:6812/31478 conn(0x7f951803a8e0 legacy=0x7f951803cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.867+0000 7f953b79d640 10 monclient: shutdown 2024-09-18T14:12:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.867+0000 7f953b79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.867+0000 7f953b79d640 1 -- 172.21.15.45:0/3862777062 >> v1:172.21.15.84:6792/0 conn(0x7f953409fba0 legacy=0x7f9534146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.867+0000 7f9539d3a640 1 -- 172.21.15.45:0/3862777062 reap_dead start 2024-09-18T14:12:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.867+0000 7f9539d3a640 1 -- 172.21.15.45:0/3862777062 reap_dead start 2024-09-18T14:12:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.867+0000 7f953b79d640 1 -- 172.21.15.45:0/3862777062 shutdown_connections 2024-09-18T14:12:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.867+0000 7f953b79d640 1 -- 172.21.15.45:0/3862777062 >> 172.21.15.45:0/3862777062 conn(0x7f95340630b0 msgr2=0x7f95340a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.867+0000 7f953b79d640 1 -- 172.21.15.45:0/3862777062 shutdown_connections 2024-09-18T14:12:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:17.867+0000 7f953b79d640 1 -- 172.21.15.45:0/3862777062 wait complete. 2024-09-18T14:12:17.913 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:17.913 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:12:17.913 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:12:18.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.009+0000 7fa6fdf5d640 10 monclient: get_monmap_and_config 2024-09-18T14:12:18.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.009+0000 7fa6fdf5d640 10 monclient: build_initial_monmap 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.009+0000 7fa6fdf5d640 10 monclient: monmap: 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:18.011244+0000 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:18.011244+0000 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:18.011 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:18.012 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:18.012 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:18.012 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 1 Processor -- start 2024-09-18T14:12:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 1 -- start start 2024-09-18T14:12:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 10 monclient: init 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 10 monclient: _add_conns ranks=[3,5,1,6,2,8,0,7,4] 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 10 monclient(hunting): picked mon.noname-g con 0x7fa6f806e810 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 10 monclient(hunting): picked mon.noname-h con 0x7fa6f80a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 10 monclient(hunting): picked mon.noname-f con 0x7fa6f80a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f805ed70 con 0x7fa6f80a1080 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f8097620 con 0x7fa6f806e810 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f8097340 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6fdf5d640 10 monclient(hunting): authenticate will time out at 1553.349586s 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6f77fe640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7fa6f806e810 0x7fa6f80a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:44940/0 (socket says 172.21.15.45:44940) 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.010+0000 7fa6f77fe640 1 -- 172.21.15.45:0/1148996469 learned_addr learned my addr 172.21.15.45:0/1148996469 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924171864 0 0) 0x7fa6f8097340 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient(hunting): my global_id is 184442 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e40034e0 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088925972 0 0) 0x7fa6f8097620 con 0x7fa6f806e810 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient(hunting): my global_id is 184422 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f8097340 con 0x7fa6f806e810 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022378848 0 0) 0x7fa6f805ed70 con 0x7fa6f80a1080 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient(hunting): my global_id is 204229 2024-09-18T14:12:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f8097620 con 0x7fa6f80a1080 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2505296583 0 0) 0x7fa6e40034e0 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 >> v1:172.21.15.84:6790/0 conn(0x7fa6f806e810 legacy=0x7fa6f80a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 >> v1:172.21.15.84:6789/0 conn(0x7fa6f80a1080 legacy=0x7fa6f813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 monclient: found mon.noname-h 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80a1660 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient: _finish_auth 0 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa6e0001830 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:12:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient: dump: 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6fdf5d640 5 monclient: authenticate success, global_id 184442 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6fdf5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa6e0001a10 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6fdf5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6fdf5d640 4 set_mon_vals no callback set 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 1 -- 172.21.15.45:0/1148996469 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa6e0001f40 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.011+0000 7fa6f6ffd640 10 monclient: dump: 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.012+0000 7fa6fdf5d640 10 monclient: shutdown 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.012+0000 7fa6fdf5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.012+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/1148996469 >> v1:172.21.15.84:6791/0 conn(0x7fa6f80a0cb0 legacy=0x7fa6f8137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.012+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/1148996469 shutdown_connections 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.012+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/1148996469 >> 172.21.15.45:0/1148996469 conn(0x7fa6f80a5bb0 msgr2=0x7fa6f80a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.012+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/1148996469 shutdown_connections 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.012+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/1148996469 wait complete. 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.012+0000 7fa6fdf5d640 10 monclient: build_initial_monmap 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.012+0000 7fa6fdf5d640 10 monclient: monmap: 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.012+0000 7fa6fdf5d640 1 Processor -- start 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6fdf5d640 1 -- start start 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6fdf5d640 10 monclient: init 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6fdf5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6fdf5d640 10 monclient: _add_conns ranks=[1,6,4,2,8,7,5,0,3] 2024-09-18T14:12:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6fdf5d640 10 monclient(hunting): picked mon.noname-f con 0x7fa6f806e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6fdf5d640 10 monclient(hunting): picked mon.noname-d con 0x7fa6f80a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6fdf5d640 10 monclient(hunting): picked mon.noname-c con 0x7fa6f80a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6fdf5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f81227a0 con 0x7fa6f806e810 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6fdf5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f813db30 con 0x7fa6f80a1080 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6fdf5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f813e830 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6fdf5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6fdf5d640 10 monclient(hunting): authenticate will time out at 1553.352639s 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6f7fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fa6f80a1080 0x7fa6f8149da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:52088/0 (socket says 172.21.15.45:52088) 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6f7fff640 1 -- 172.21.15.45:0/819638464 learned_addr learned my addr 172.21.15.45:0/819638464 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6eeffd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fa6f80a0cb0 0x7fa6f8146670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:55478/0 (socket says 172.21.15.45:55478) 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.013+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980873691 0 0) 0x7fa6f81227a0 con 0x7fa6f806e810 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): my global_id is 204238 2024-09-18T14:12:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6d80034e0 con 0x7fa6f806e810 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689455785 0 0) 0x7fa6f813e830 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): my global_id is 184506 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f81227a0 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647237419 0 0) 0x7fa6f813db30 con 0x7fa6f80a1080 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): my global_id is 164713 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f813e830 con 0x7fa6f80a1080 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2249263 0 0) 0x7fa6d80034e0 con 0x7fa6f806e810 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6f813db30 con 0x7fa6f806e810 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa6e8001ae0 con 0x7fa6f806e810 2024-09-18T14:12:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): dump: 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1905365397 0 0) 0x7fa6f81227a0 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6d80034e0 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1639062095 0 0) 0x7fa6f813e830 con 0x7fa6f80a1080 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6f81227a0 con 0x7fa6f80a1080 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa6e0001ec0 con 0x7fa6f80a0cb0 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): dump: 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa6f0001c00 con 0x7fa6f80a1080 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient(hunting): dump: 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3775512828 0 0) 0x7fa6f813db30 con 0x7fa6f806e810 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 >> v1:172.21.15.45:6792/0 conn(0x7fa6f80a0cb0 legacy=0x7fa6f8146670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 >> v1:172.21.15.45:6791/0 conn(0x7fa6f80a1080 legacy=0x7fa6f8149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient: _finish_hunting 0 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 monclient: found mon.f 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:12:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f81224c0 con 0x7fa6f806e810 2024-09-18T14:12:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient: _finish_auth 0 2024-09-18T14:12:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6fdf5d640 5 monclient: authenticate success, global_id 204238 2024-09-18T14:12:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6fdf5d640 10 monclient: _renew_subs 2024-09-18T14:12:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6fdf5d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:12:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f8122a80 con 0x7fa6f806e810 2024-09-18T14:12:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6fdf5d640 10 monclient: _renew_subs 2024-09-18T14:12:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6fdf5d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:12:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f81215f0 con 0x7fa6f806e810 2024-09-18T14:12:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa6e8001d40 con 0x7fa6f806e810 2024-09-18T14:12:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa6e8004e50 con 0x7fa6f806e810 2024-09-18T14:12:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6f5ffb640 4 set_mon_vals no callback set 2024-09-18T14:12:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:12:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.014+0000 7fa6effff640 10 monclient: dump: 2024-09-18T14:12:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.015+0000 7fa6fdf5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.015+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa6dc006580 con 0x7fa6dc002830 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.015+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fa6e8005030 con 0x7fa6f806e810 2024-09-18T14:12:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.015+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(629..629 src has 424..629) v4 ==== 3911+0+0 (unknown 665274774 0 0) 0x7fa6e8046bf0 con 0x7fa6f806e810 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.015+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441874416 0 0) 0x7fa6dc006580 con 0x7fa6dc002830 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.015+0000 7fa6effff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 10 monclient(hunting): my global_id is 184431 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f813db30 con 0x7fa6dc002830 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6f7fff640 10 monclient: get_auth_request con 0x7fa6d803aa90 auth_method 0 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2478702121 0 0) 0x7fa6f813db30 con 0x7fa6dc002830 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6dc006580 con 0x7fa6dc002830 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa6e00027b0 con 0x7fa6dc002830 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 10 monclient: dump: 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.039 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2201072960 0 0) 0x7fa6dc006580 con 0x7fa6dc002830 2024-09-18T14:12:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6dc00a0a0 con 0x7fa6dc002830 2024-09-18T14:12:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa6e0005a20 con 0x7fa6dc002830 2024-09-18T14:12:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 >> v1:172.21.15.84:6790/0 conn(0x7fa6dc002830 legacy=0x7fa6dc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.016+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 >> v1:172.21.15.84:6790/0 conn(0x7fa6dc002830 legacy=0x7fa6dc004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6fdf5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa6dc007520 con 0x7fa6dc0068d0 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582341237 0 0) 0x7fa6dc007520 con 0x7fa6dc0068d0 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 10 monclient(hunting): my global_id is 184440 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6dc006580 con 0x7fa6dc0068d0 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3631963647 0 0) 0x7fa6dc006580 con 0x7fa6dc0068d0 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6dc007520 con 0x7fa6dc0068d0 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa6e0002f80 con 0x7fa6dc0068d0 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 10 monclient: dump: 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4279252400 0 0) 0x7fa6dc007520 con 0x7fa6dc0068d0 2024-09-18T14:12:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa6dc0087a0 con 0x7fa6dc0068d0 2024-09-18T14:12:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 1961347798) 0x7fa6e0007d90 con 0x7fa6dc0068d0 2024-09-18T14:12:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 >> v1:172.21.15.84:6790/0 conn(0x7fa6dc0068d0 legacy=0x7fa6dc006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.023+0000 7fa6effff640 1 -- 172.21.15.45:0/819638464 >> v1:172.21.15.84:6790/0 conn(0x7fa6dc0068d0 legacy=0x7fa6dc006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:18.044 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:18.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:18.044 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:18.044 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.045 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.052 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:18.053 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.054 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.054 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.054 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:18.054 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:18.054 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.054 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.054 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:18.054 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:18.055 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.056 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.057 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.058 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.059 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.061 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:18.062 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:18.062 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.062 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.062 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.062 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.062 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.062 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:18.062 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:18.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.062 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:18.064 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:18.064 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:18.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.064 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.064 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:18.064 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:18.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.024+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/819638464 >> v1:172.21.15.84:6812/31478 conn(0x7fa6d803aa90 legacy=0x7fa6d803cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.024+0000 7fa6fdf5d640 10 monclient: shutdown 2024-09-18T14:12:18.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.024+0000 7fa6fdf5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:18.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.024+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/819638464 >> v1:172.21.15.84:6789/0 conn(0x7fa6f806e810 legacy=0x7fa6f813d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.024+0000 7fa6f7fff640 1 -- 172.21.15.45:0/819638464 reap_dead start 2024-09-18T14:12:18.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.024+0000 7fa6f7fff640 1 -- 172.21.15.45:0/819638464 reap_dead start 2024-09-18T14:12:18.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.024+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/819638464 shutdown_connections 2024-09-18T14:12:18.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.024+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/819638464 >> 172.21.15.45:0/819638464 conn(0x7fa6f80a5bb0 msgr2=0x7fa6f80a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:18.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.024+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/819638464 shutdown_connections 2024-09-18T14:12:18.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.024+0000 7fa6fdf5d640 1 -- 172.21.15.45:0/819638464 wait complete. 2024-09-18T14:12:18.066 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:18.066 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:12:18.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:12:18.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.162+0000 7f193a900640 10 monclient: get_monmap_and_config 2024-09-18T14:12:18.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.162+0000 7f193a900640 10 monclient: build_initial_monmap 2024-09-18T14:12:18.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.162+0000 7f193a900640 10 monclient: monmap: 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:18.164105+0000 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:18.164105+0000 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:18.165 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:18.165 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:18.165 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.162+0000 7f193a900640 1 Processor -- start 2024-09-18T14:12:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.162+0000 7f193a900640 1 -- start start 2024-09-18T14:12:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.162+0000 7f193a900640 10 monclient: init 2024-09-18T14:12:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f193a900640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f193a900640 10 monclient: _add_conns ranks=[6,3,1,0,8,2,5,4,7] 2024-09-18T14:12:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f193a900640 10 monclient(hunting): picked mon.noname-d con 0x7f193406e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f193a900640 10 monclient(hunting): picked mon.noname-g con 0x7f19340a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f193a900640 10 monclient(hunting): picked mon.noname-f con 0x7f19340a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f193a900640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193405ed70 con 0x7f19340a1080 2024-09-18T14:12:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f193a900640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1934097620 con 0x7f19340a0cb0 2024-09-18T14:12:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f193a900640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1934097340 con 0x7f193406e810 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f193a900640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f193a900640 10 monclient(hunting): authenticate will time out at 1553.502457s 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1933fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f193406e810 0x7f19340a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:55482/0 (socket says 172.21.15.45:55482) 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1933fff640 1 -- 172.21.15.45:0/3521273679 learned_addr learned my addr 172.21.15.45:0/3521273679 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760761987 0 0) 0x7f1934097340 con 0x7f193406e810 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 10 monclient(hunting): my global_id is 184515 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f191c0034e0 con 0x7f193406e810 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754230520 0 0) 0x7f1934097620 con 0x7f19340a0cb0 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 10 monclient(hunting): my global_id is 184449 2024-09-18T14:12:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1934097340 con 0x7f19340a0cb0 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355702181 0 0) 0x7f193405ed70 con 0x7f19340a1080 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 10 monclient(hunting): my global_id is 204247 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.163+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1934097620 con 0x7f19340a1080 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3919983262 0 0) 0x7f191c0034e0 con 0x7f193406e810 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 >> v1:172.21.15.84:6790/0 conn(0x7f19340a0cb0 legacy=0x7f1934137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 >> v1:172.21.15.84:6789/0 conn(0x7f19340a1080 legacy=0x7f193413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 1 monclient: found mon.noname-d 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:12:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19340a1660 con 0x7f193406e810 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 10 monclient: _finish_auth 0 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1924001830 con 0x7f193406e810 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 10 monclient: dump: 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f193a900640 5 monclient: authenticate success, global_id 184515 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f193a900640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1924001a10 con 0x7f193406e810 2024-09-18T14:12:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 1 -- 172.21.15.45:0/3521273679 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1924004ba0 con 0x7f193406e810 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f193a900640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f193a900640 4 set_mon_vals no callback set 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f1932ffd640 10 monclient: dump: 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f193a900640 10 monclient: shutdown 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f193a900640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f193a900640 1 -- 172.21.15.45:0/3521273679 >> v1:172.21.15.45:6792/0 conn(0x7f193406e810 legacy=0x7f19340a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f193a900640 1 -- 172.21.15.45:0/3521273679 shutdown_connections 2024-09-18T14:12:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f193a900640 1 -- 172.21.15.45:0/3521273679 >> 172.21.15.45:0/3521273679 conn(0x7f19340a5bb0 msgr2=0x7f19340a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f193a900640 1 -- 172.21.15.45:0/3521273679 shutdown_connections 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.164+0000 7f193a900640 1 -- 172.21.15.45:0/3521273679 wait complete. 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.165+0000 7f193a900640 10 monclient: build_initial_monmap 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.165+0000 7f193a900640 10 monclient: monmap: 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:18.185 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:18.185 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:18.185 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:18.185 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.165+0000 7f193a900640 1 Processor -- start 2024-09-18T14:12:18.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.165+0000 7f193a900640 1 -- start start 2024-09-18T14:12:18.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.165+0000 7f193a900640 10 monclient: init 2024-09-18T14:12:18.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f193a900640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:18.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f193a900640 10 monclient: _add_conns ranks=[0,7,8,1,2,3,5,6,4] 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f193a900640 10 monclient(hunting): picked mon.noname-a con 0x7f193406e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f193a900640 10 monclient(hunting): picked mon.noname-i con 0x7f19340a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f193a900640 10 monclient(hunting): picked mon.noname-e con 0x7f19340a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f193a900640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1934122f30 con 0x7f193406e810 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f193a900640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193413dd20 con 0x7f19340a0cb0 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f193a900640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1934122970 con 0x7f19340a1080 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f193a900640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f193a900640 10 monclient(hunting): authenticate will time out at 1553.505353s 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f1933fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f193406e810 0x7f193413d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36964/0 (socket says 172.21.15.45:36964) 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f1933fff640 1 -- 172.21.15.45:0/2451367604 learned_addr learned my addr 172.21.15.45:0/2451367604 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888021581 0 0) 0x7f1934122f30 con 0x7f193406e810 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): my global_id is 5949 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19200034e0 con 0x7f193406e810 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982864113 0 0) 0x7f1934122970 con 0x7f19340a1080 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): my global_id is 5768 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1934122f30 con 0x7f19340a1080 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1902901844 0 0) 0x7f193413dd20 con 0x7f19340a0cb0 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): my global_id is 184498 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1934122970 con 0x7f19340a0cb0 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 446883411 0 0) 0x7f19200034e0 con 0x7f193406e810 2024-09-18T14:12:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f193413dd20 con 0x7f193406e810 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f19240047a0 con 0x7f193406e810 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): dump: 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.188 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 119624357 0 0) 0x7f1934122f30 con 0x7f19340a1080 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.166+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19200034e0 con 0x7f19340a1080 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f192c001d00 con 0x7f19340a1080 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 10 monclient(hunting): dump: 2024-09-18T14:12:18.189 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.190 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4243932668 0 0) 0x7f1934122970 con 0x7f19340a0cb0 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1934122f30 con 0x7f19340a0cb0 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3227149190 0 0) 0x7f193413dd20 con 0x7f193406e810 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 >> v1:172.21.15.45:6793/0 conn(0x7f19340a1080 legacy=0x7f1934149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 >> v1:172.21.15.84:6792/0 conn(0x7f19340a0cb0 legacy=0x7f1934146840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 1 monclient: found mon.a 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934122690 con 0x7f193406e810 2024-09-18T14:12:18.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 10 monclient: _finish_auth 0 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f193a900640 5 monclient: authenticate success, global_id 5949 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f193a900640 10 monclient: _renew_subs 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f193a900640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f193a900640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1934122c50 con 0x7f193406e810 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f193a900640 10 monclient: _renew_subs 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f193a900640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f193a900640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f19341217c0 con 0x7f193406e810 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1924004c30 con 0x7f193406e810 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f19240051a0 con 0x7f193406e810 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f19327fc640 4 set_mon_vals no callback set 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f191bfff640 10 monclient: dump: 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.193 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.167+0000 7f193a900640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f193a900640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1914006580 con 0x7f1914002830 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1924010170 con 0x7f193406e810 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(629..629 src has 424..629) v4 ==== 3911+0+0 (unknown 665274774 0 0) 0x7f1924046d90 con 0x7f193406e810 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672059555 0 0) 0x7f1914006580 con 0x7f1914002830 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f1938e9d640 10 monclient: get_auth_request con 0x7f192003a8b0 auth_method 0 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 10 monclient(hunting): my global_id is 184451 2024-09-18T14:12:18.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f193413dd20 con 0x7f1914002830 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2717863123 0 0) 0x7f193413dd20 con 0x7f1914002830 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1914006580 con 0x7f1914002830 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1928002230 con 0x7f1914002830 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 10 monclient: dump: 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1478556591 0 0) 0x7f1914006580 con 0x7f1914002830 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.168+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f191400a0a0 con 0x7f1914002830 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.169+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f19280025c0 con 0x7f1914002830 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.169+0000 7f191bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.169+0000 7f191bfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.169+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 >> v1:172.21.15.84:6791/0 conn(0x7f1914002830 legacy=0x7f1914004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.169+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 >> v1:172.21.15.84:6791/0 conn(0x7f1914002830 legacy=0x7f1914004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:18.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f193a900640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:18.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f193a900640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f19140074f0 con 0x7f19140068a0 2024-09-18T14:12:18.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252894688 0 0) 0x7f19140074f0 con 0x7f19140068a0 2024-09-18T14:12:18.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f191bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f191bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f191bfff640 10 monclient(hunting): my global_id is 184460 2024-09-18T14:12:18.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1914006580 con 0x7f19140068a0 2024-09-18T14:12:18.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3057254305 0 0) 0x7f1914006580 con 0x7f19140068a0 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19140074f0 con 0x7f19140068a0 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1928002e00 con 0x7f19140068a0 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f191bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f191bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.175+0000 7f191bfff640 10 monclient: dump: 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 797968637 0 0) 0x7f19140074f0 con 0x7f19140068a0 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1914008750 con 0x7f19140068a0 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 589758857) 0x7f19280031a0 con 0x7f19140068a0 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f191bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f191bfff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 >> v1:172.21.15.84:6791/0 conn(0x7f19140068a0 legacy=0x7f1914006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f191bfff640 1 -- 172.21.15.45:0/2451367604 >> v1:172.21.15.84:6791/0 conn(0x7f19140068a0 legacy=0x7f1914006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:18.201 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:18.201 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:18.201 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:18.201 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:18.201 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:18.201 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:18.201 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:18.201 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:18.201 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:18.201 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:18.201 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:18.204 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:18.204 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:18.204 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:18.204 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.204 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:18.204 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:18.204 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:18.204 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:18.204 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:18.204 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:18.205 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:18.206 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.208 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.209 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:18.210 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.211 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.212 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:18.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:18.212 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.212 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.212 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:18.212 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.212 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.212 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.214 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:18.215 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:18.219 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.220 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.220 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:12:18.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.220 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:18.220 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:18.220 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:18.220 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.220 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.220 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:18.220 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f193a900640 1 -- 172.21.15.45:0/2451367604 >> v1:172.21.15.84:6812/31478 conn(0x7f192003a8b0 legacy=0x7f192003cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f193a900640 10 monclient: shutdown 2024-09-18T14:12:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f193a900640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f193a900640 1 -- 172.21.15.45:0/2451367604 >> v1:172.21.15.45:6789/0 conn(0x7f193406e810 legacy=0x7f193413d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f1938e9d640 1 -- 172.21.15.45:0/2451367604 reap_dead start 2024-09-18T14:12:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f1938e9d640 1 -- 172.21.15.45:0/2451367604 reap_dead start 2024-09-18T14:12:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f193a900640 1 -- 172.21.15.45:0/2451367604 shutdown_connections 2024-09-18T14:12:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f193a900640 1 -- 172.21.15.45:0/2451367604 >> 172.21.15.45:0/2451367604 conn(0x7f19340a5bb0 msgr2=0x7f19340a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.176+0000 7f193a900640 1 -- 172.21.15.45:0/2451367604 shutdown_connections 2024-09-18T14:12:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.177+0000 7f193a900640 1 -- 172.21.15.45:0/2451367604 wait complete. 2024-09-18T14:12:18.223 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:18.223 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:12:18.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:12:18.301 INFO:tasks.rados.rados.0.smithi084.stdout:2839: done (3 left) 2024-09-18T14:12:18.301 INFO:tasks.rados.rados.0.smithi084.stdout:2840: done (2 left) 2024-09-18T14:12:18.301 INFO:tasks.rados.rados.0.smithi084.stdout:2841: done (1 left) 2024-09-18T14:12:18.301 INFO:tasks.rados.rados.0.smithi084.stdout:2842: done (0 left) 2024-09-18T14:12:18.302 INFO:tasks.rados.rados.0.smithi084.stdout:2843: write oid 36 current snap is 309 2024-09-18T14:12:18.302 INFO:tasks.rados.rados.0.smithi084.stdout:2843: seq_num 972 ranges {736404=718859,1929321=407083} 2024-09-18T14:12:18.309 INFO:tasks.rados.rados.0.smithi084.stdout:2843: writing smithi08434663-36 from 736404 to 1455263 tid 1 2024-09-18T14:12:18.314 INFO:tasks.rados.rados.0.smithi084.stdout:2843: writing smithi08434663-36 from 1929321 to 2336404 tid 2 2024-09-18T14:12:18.315 INFO:tasks.rados.rados.0.smithi084.stdout:2844: delete oid 33 current snap is 309 2024-09-18T14:12:18.317 INFO:tasks.rados.rados.0.smithi084.stdout:2843: finishing write tid 1 to smithi08434663-36 2024-09-18T14:12:18.320 INFO:tasks.rados.rados.0.smithi084.stdout:2843: finishing write tid 2 to smithi08434663-36 2024-09-18T14:12:18.320 INFO:tasks.rados.rados.0.smithi084.stdout:2843: finishing write tid 3 to smithi08434663-36 2024-09-18T14:12:18.321 INFO:tasks.rados.rados.0.smithi084.stdout:2843: finishing write tid 4 to smithi08434663-36 2024-09-18T14:12:18.321 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 1263 (ObjNum 972 snap 309 seq_num 972) dirty exists 2024-09-18T14:12:18.321 INFO:tasks.rados.rados.0.smithi084.stdout:2843: left oid 36 (ObjNum 972 snap 309 seq_num 972) 2024-09-18T14:12:18.321 INFO:tasks.rados.rados.0.smithi084.stdout:2843: done (1 left) 2024-09-18T14:12:18.321 INFO:tasks.rados.rados.0.smithi084.stdout:2844: done (0 left) 2024-09-18T14:12:18.321 INFO:tasks.rados.rados.0.smithi084.stdout:2845: snap_create 2024-09-18T14:12:18.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.322+0000 7fbac4c20640 10 monclient: get_monmap_and_config 2024-09-18T14:12:18.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.322+0000 7fbac4c20640 10 monclient: build_initial_monmap 2024-09-18T14:12:18.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.322+0000 7fbac4c20640 10 monclient: monmap: 2024-09-18T14:12:18.323 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:18.324553+0000 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:18.324553+0000 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:18.325 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:18.325 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 1 Processor -- start 2024-09-18T14:12:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 1 -- start start 2024-09-18T14:12:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 10 monclient: init 2024-09-18T14:12:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 10 monclient: _add_conns ranks=[6,2,1,8,5,0,7,4,3] 2024-09-18T14:12:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 10 monclient(hunting): picked mon.noname-d con 0x7fbac006e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 10 monclient(hunting): picked mon.noname-b con 0x7fbac00a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 10 monclient(hunting): picked mon.noname-f con 0x7fbac00a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac005ed70 con 0x7fbac00a1080 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac0097620 con 0x7fbac00a0cb0 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac0097340 con 0x7fbac006e810 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.323+0000 7fbac4c20640 10 monclient(hunting): authenticate will time out at 1553.662898s 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabe59c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fbac006e810 0x7fbac00a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:55488/0 (socket says 172.21.15.45:55488) 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabe59c640 1 -- 172.21.15.45:0/2354691631 learned_addr learned my addr 172.21.15.45:0/2354691631 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801236124 0 0) 0x7fbac0097620 con 0x7fbac00a0cb0 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient(hunting): my global_id is 145235 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaac0034e0 con 0x7fbac00a0cb0 2024-09-18T14:12:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480944807 0 0) 0x7fbac0097340 con 0x7fbac006e810 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient(hunting): my global_id is 184524 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac0097620 con 0x7fbac006e810 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481809548 0 0) 0x7fbac005ed70 con 0x7fbac00a1080 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient(hunting): my global_id is 204256 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac0097340 con 0x7fbac00a1080 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4236417396 0 0) 0x7fbaac0034e0 con 0x7fbac00a0cb0 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 >> v1:172.21.15.45:6792/0 conn(0x7fbac006e810 legacy=0x7fbac00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 >> v1:172.21.15.84:6789/0 conn(0x7fbac00a1080 legacy=0x7fbac013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient: _finish_hunting 0 2024-09-18T14:12:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 monclient: found mon.noname-b 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac00a1660 con 0x7fbac00a0cb0 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient: _finish_auth 0 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbac4c20640 5 monclient: authenticate success, global_id 145235 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbaa8001830 con 0x7fbac00a0cb0 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbac4c20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient: dump: 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbac4c20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbaa8001990 con 0x7fbac00a0cb0 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 1 -- 172.21.15.45:0/2354691631 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbaa8001ec0 con 0x7fbac00a0cb0 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbac4c20640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbac4c20640 4 set_mon_vals no callback set 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.324+0000 7fbabdd9b640 10 monclient: dump: 2024-09-18T14:12:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.325+0000 7fbac4c20640 10 monclient: shutdown 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.325+0000 7fbac4c20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.325+0000 7fbac4c20640 1 -- 172.21.15.45:0/2354691631 >> v1:172.21.15.45:6790/0 conn(0x7fbac00a0cb0 legacy=0x7fbac0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.325+0000 7fbac4c20640 1 -- 172.21.15.45:0/2354691631 shutdown_connections 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.325+0000 7fbac4c20640 1 -- 172.21.15.45:0/2354691631 >> 172.21.15.45:0/2354691631 conn(0x7fbac00a5bb0 msgr2=0x7fbac00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.325+0000 7fbac4c20640 1 -- 172.21.15.45:0/2354691631 shutdown_connections 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.325+0000 7fbac4c20640 1 -- 172.21.15.45:0/2354691631 wait complete. 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.325+0000 7fbac4c20640 10 monclient: build_initial_monmap 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.325+0000 7fbac4c20640 10 monclient: monmap: 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 1 Processor -- start 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 1 -- start start 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 10 monclient: init 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 10 monclient: _add_conns ranks=[4,0,8,6,3,1,7,2,5] 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 10 monclient(hunting): picked mon.noname-c con 0x7fbac006e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 10 monclient(hunting): picked mon.noname-a con 0x7fbac00a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 10 monclient(hunting): picked mon.noname-e con 0x7fbac00a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac00a2270 con 0x7fbac00a0cb0 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac0122730 con 0x7fbac006e810 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac013da30 con 0x7fbac00a1080 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbac4c20640 10 monclient(hunting): authenticate will time out at 1553.665696s 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbabe59c640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fbac006e810 0x7fbac013d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:52100/0 (socket says 172.21.15.45:52100) 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbabed9d640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fbac00a1080 0x7fbac0149c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:41926/0 (socket says 172.21.15.45:41926) 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.326+0000 7fbabe59c640 1 -- 172.21.15.45:0/1057958 learned_addr learned my addr 172.21.15.45:0/1057958 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831678299 0 0) 0x7fbac0122730 con 0x7fbac006e810 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): my global_id is 164722 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaa00034e0 con 0x7fbac006e810 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994590967 0 0) 0x7fbac013da30 con 0x7fbac00a1080 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): my global_id is 5777 2024-09-18T14:12:18.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac0122730 con 0x7fbac00a1080 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120491329 0 0) 0x7fbac00a2270 con 0x7fbac00a0cb0 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): my global_id is 5958 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac013da30 con 0x7fbac00a0cb0 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 858296986 0 0) 0x7fbaa00034e0 con 0x7fbac006e810 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbac00a2270 con 0x7fbac006e810 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbab0001650 con 0x7fbac006e810 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): dump: 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.344 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.345 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:18.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3974324192 0 0) 0x7fbac013da30 con 0x7fbac00a0cb0 2024-09-18T14:12:18.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbaa00034e0 con 0x7fbac00a0cb0 2024-09-18T14:12:18.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1161234468 0 0) 0x7fbac0122730 con 0x7fbac00a1080 2024-09-18T14:12:18.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbac013da30 con 0x7fbac00a1080 2024-09-18T14:12:18.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbaa80016f0 con 0x7fbac00a0cb0 2024-09-18T14:12:18.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:18.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): dump: 2024-09-18T14:12:18.346 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbab8001cc0 con 0x7fbac00a1080 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient(hunting): dump: 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3424886439 0 0) 0x7fbac00a2270 con 0x7fbac006e810 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 >> v1:172.21.15.45:6793/0 conn(0x7fbac00a1080 legacy=0x7fbac0149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 >> v1:172.21.15.45:6789/0 conn(0x7fbac00a0cb0 legacy=0x7fbac0146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 monclient: found mon.c 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac013e6f0 con 0x7fbac006e810 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbac4c20640 5 monclient: authenticate success, global_id 164722 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbac4c20640 10 monclient: _renew_subs 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbac4c20640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbac4c20640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac01223c0 con 0x7fbac006e810 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbac4c20640 10 monclient: _renew_subs 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbac4c20640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbac4c20640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac013ded0 con 0x7fbac006e810 2024-09-18T14:12:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbab0004a50 con 0x7fbac006e810 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbab67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.327+0000 7fbabd59a640 4 set_mon_vals no callback set 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.328+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbab0004f80 con 0x7fbac006e810 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.328+0000 7fbab67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.328+0000 7fbab67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.328+0000 7fbab67fc640 10 monclient: dump: 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.328+0000 7fbac4c20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.328+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fbab000ffd0 con 0x7fbac006e810 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.328+0000 7fbac4c20640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbac004c4a0 con 0x7fbac01214a0 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.328+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(630..630 src has 424..630) v4 ==== 3911+0+0 (unknown 2068902181 0 0) 0x7fbab0046ca0 con 0x7fbac006e810 2024-09-18T14:12:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.328+0000 7fbabed9d640 10 monclient: get_auth_request con 0x7fbaa003aae0 auth_method 0 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701463109 0 0) 0x7fbac004c4a0 con 0x7fbac01214a0 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 10 monclient(hunting): my global_id is 184507 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac00a2270 con 0x7fbac01214a0 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2883975605 0 0) 0x7fbac00a2270 con 0x7fbac01214a0 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbac004c4a0 con 0x7fbac01214a0 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbaa8002730 con 0x7fbac01214a0 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 10 monclient: dump: 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.355 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3811777029 0 0) 0x7fbac004c4a0 con 0x7fbac01214a0 2024-09-18T14:12:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbac004b720 con 0x7fbac01214a0 2024-09-18T14:12:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fbaa8002a50 con 0x7fbac01214a0 2024-09-18T14:12:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 >> v1:172.21.15.84:6792/0 conn(0x7fbac01214a0 legacy=0x7fbac005e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.329+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 >> v1:172.21.15.84:6792/0 conn(0x7fbac01214a0 legacy=0x7fbac005e310 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:18.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.335+0000 7fbac4c20640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:18.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.335+0000 7fbac4c20640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbac0062dd0 con 0x7fbac002aa40 2024-09-18T14:12:18.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238096444 0 0) 0x7fbac0062dd0 con 0x7fbac002aa40 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 10 monclient(hunting): my global_id is 184516 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac004c4a0 con 0x7fbac002aa40 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1861770636 0 0) 0x7fbac004c4a0 con 0x7fbac002aa40 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbac0062dd0 con 0x7fbac002aa40 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbaa8003250 con 0x7fbac002aa40 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 10 monclient: dump: 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2310772083 0 0) 0x7fbac0062dd0 con 0x7fbac002aa40 2024-09-18T14:12:18.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbac002ae10 con 0x7fbac002aa40 2024-09-18T14:12:18.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 191099061) 0x7fbaa8005430 con 0x7fbac002aa40 2024-09-18T14:12:18.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:18.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:18.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 >> v1:172.21.15.84:6792/0 conn(0x7fbac002aa40 legacy=0x7fbac009b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.336+0000 7fbab67fc640 1 -- 172.21.15.45:0/1057958 >> v1:172.21.15.84:6792/0 conn(0x7fbac002aa40 legacy=0x7fbac009b9d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:18.359 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:18.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:18.359 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:18.359 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:18.359 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 106, 2024-09-18T14:12:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:12:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:12:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.361 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:18.362 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:18.363 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:18.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.369 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.371 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:18.372 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.372 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.372 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:18.372 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:18.372 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.372 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.372 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.372 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.373 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:18.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:18.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:18.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:18.378 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:18.378 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.337+0000 7fbac4c20640 1 -- 172.21.15.45:0/1057958 >> v1:172.21.15.84:6812/31478 conn(0x7fbaa003aae0 legacy=0x7fbaa003cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.337+0000 7fbac4c20640 10 monclient: shutdown 2024-09-18T14:12:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.337+0000 7fbac4c20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.337+0000 7fbac4c20640 1 -- 172.21.15.45:0/1057958 >> v1:172.21.15.45:6791/0 conn(0x7fbac006e810 legacy=0x7fbac013d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.337+0000 7fbabed9d640 1 -- 172.21.15.45:0/1057958 reap_dead start 2024-09-18T14:12:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.337+0000 7fbabed9d640 1 -- 172.21.15.45:0/1057958 reap_dead start 2024-09-18T14:12:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.337+0000 7fbac4c20640 1 -- 172.21.15.45:0/1057958 shutdown_connections 2024-09-18T14:12:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.337+0000 7fbac4c20640 1 -- 172.21.15.45:0/1057958 >> 172.21.15.45:0/1057958 conn(0x7fbac00a5bb0 msgr2=0x7fbac009e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.337+0000 7fbac4c20640 1 -- 172.21.15.45:0/1057958 shutdown_connections 2024-09-18T14:12:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:18.337+0000 7fbac4c20640 1 -- 172.21.15.45:0/1057958 wait complete. 2024-09-18T14:12:18.381 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:12:18.381 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['f', 'i'] 2024-09-18T14:12:18.381 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['e'] 2024-09-18T14:12:18.381 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-18T14:12:18.381 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-18T14:12:18.381 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-18T14:12:18.382 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:12:18.394 INFO:tasks.ceph.mon.f:Stopped 2024-09-18T14:12:18.394 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-18T14:12:18.394 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-18T14:12:18.394 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-18T14:12:18.394 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:12:18.446 INFO:tasks.ceph.mon.i:Stopped 2024-09-18T14:12:18.446 INFO:tasks.mon_thrash:Sending STOP to mon e 2024-09-18T14:12:18.447 INFO:tasks.ceph.mon.e:Sent signal 19 2024-09-18T14:12:18.447 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-18T14:12:33.373 INFO:tasks.rados.rados.0.smithi084.stdout:2845: done (0 left) 2024-09-18T14:12:33.373 INFO:tasks.rados.rados.0.smithi084.stdout:2846: write (excl) oid 29 current snap is 310 2024-09-18T14:12:33.373 INFO:tasks.rados.rados.0.smithi084.stdout:2846: seq_num 974 ranges {770018=585710,2046285=323733} 2024-09-18T14:12:33.380 INFO:tasks.rados.rados.0.smithi084.stdout:2846: writing smithi08434663-29 from 770018 to 1355728 tid 1 2024-09-18T14:12:33.386 INFO:tasks.rados.rados.0.smithi084.stdout:2846: writing smithi08434663-29 from 2046285 to 2370018 tid 2 2024-09-18T14:12:33.386 INFO:tasks.rados.rados.0.smithi084.stdout:2847: rollback oid 26 current snap is 310 2024-09-18T14:12:33.386 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 26 to 305 2024-09-18T14:12:33.386 INFO:tasks.rados.rados.0.smithi084.stdout:2848: write (excl) oid 16 current snap is 310 2024-09-18T14:12:33.386 INFO:tasks.rados.rados.0.smithi084.stdout:2848: seq_num 975 ranges {786825=660959,2104767=282058} 2024-09-18T14:12:33.394 INFO:tasks.rados.rados.0.smithi084.stdout:2848: writing smithi08434663-16 from 786825 to 1447784 tid 1 2024-09-18T14:12:33.400 INFO:tasks.rados.rados.0.smithi084.stdout:2848: writing smithi08434663-16 from 2104767 to 2386825 tid 2 2024-09-18T14:12:33.400 INFO:tasks.rados.rados.0.smithi084.stdout:2849: read oid 18 snap -1 2024-09-18T14:12:33.400 INFO:tasks.rados.rados.0.smithi084.stdout:2849: expect (ObjNum 937 snap 299 seq_num 937) 2024-09-18T14:12:33.400 INFO:tasks.rados.rados.0.smithi084.stdout:2850: rollback oid 19 current snap is 310 2024-09-18T14:12:33.400 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 19 to 309 2024-09-18T14:12:33.400 INFO:tasks.rados.rados.0.smithi084.stdout:2847: finishing rollback tid 0 to smithi08434663-26 2024-09-18T14:12:33.400 INFO:tasks.rados.rados.0.smithi084.stdout:2847: finishing rollback tid 1 to smithi08434663-26 2024-09-18T14:12:33.400 INFO:tasks.rados.rados.0.smithi084.stdout:2847: finishing rollback tid 2 to smithi08434663-26 2024-09-18T14:12:33.401 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 1229 (ObjNum 938 snap 300 seq_num 938) dirty exists 2024-09-18T14:12:33.401 INFO:tasks.rados.rados.0.smithi084.stdout:2847: done (4 left) 2024-09-18T14:12:33.401 INFO:tasks.rados.rados.0.smithi084.stdout:2851: read oid 3 snap -1 2024-09-18T14:12:33.401 INFO:tasks.rados.rados.0.smithi084.stdout:2851: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:12:33.401 INFO:tasks.rados.rados.0.smithi084.stdout:2852: write oid 7 current snap is 310 2024-09-18T14:12:33.401 INFO:tasks.rados.rados.0.smithi084.stdout:2852: seq_num 976 ranges {403632=736208,1763249=619999} 2024-09-18T14:12:33.406 INFO:tasks.rados.rados.0.smithi084.stdout:2852: writing smithi08434663-7 from 403632 to 1139840 tid 1 2024-09-18T14:12:33.413 INFO:tasks.rados.rados.0.smithi084.stdout:2852: writing smithi08434663-7 from 1763249 to 2383248 tid 2 2024-09-18T14:12:33.413 INFO:tasks.rados.rados.0.smithi084.stdout:2853: snap_remove snap 307 2024-09-18T14:12:33.447 INFO:tasks.mon_thrash:Sending CONT to mon e 2024-09-18T14:12:33.448 INFO:tasks.ceph.mon.e:Sent signal 18 2024-09-18T14:12:33.448 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-18T14:12:33.448 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:12:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.547+0000 7fed48b09640 10 monclient: get_monmap_and_config 2024-09-18T14:12:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.547+0000 7fed48b09640 10 monclient: build_initial_monmap 2024-09-18T14:12:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.547+0000 7fed48b09640 10 monclient: monmap: 2024-09-18T14:12:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:33.548808+0000 2024-09-18T14:12:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:33.548808+0000 2024-09-18T14:12:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:33.549 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.548+0000 7fed48b09640 1 Processor -- start 2024-09-18T14:12:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.548+0000 7fed48b09640 1 -- start start 2024-09-18T14:12:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.548+0000 7fed48b09640 10 monclient: init 2024-09-18T14:12:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed48b09640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed48b09640 10 monclient: _add_conns ranks=[7,1,8,6,3,0,5,2,4] 2024-09-18T14:12:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed48b09640 10 monclient(hunting): picked mon.noname-i con 0x7fed4406e140 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed48b09640 10 monclient(hunting): picked mon.noname-f con 0x7fed4406e510 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed48b09640 10 monclient(hunting): picked mon.noname-e con 0x7fed4406e910 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed48b09640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4405ed70 con 0x7fed4406e510 2024-09-18T14:12:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed48b09640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4409fcd0 con 0x7fed4406e140 2024-09-18T14:12:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed48b09640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4412ff00 con 0x7fed4406e910 2024-09-18T14:12:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed48b09640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed48b09640 10 monclient(hunting): authenticate will time out at 1568.887126s 2024-09-18T14:12:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed4259c640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7fed4406e140 legacy=0x7fed4412f7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed3bfff640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7fed4406e510 legacy=0x7fed44134510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:12:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed42d9d640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fed4406e910 0x7fed44137c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:60846/0 (socket says 172.21.15.45:60846) 2024-09-18T14:12:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.549+0000 7fed42d9d640 1 -- 172.21.15.45:0/2005585845 learned_addr learned my addr 172.21.15.45:0/2005585845 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.750+0000 7fed3bfff640 1 -- 172.21.15.45:0/2005585845 >> v1:172.21.15.84:6789/0 conn(0x7fed4406e510 legacy=0x7fed44134510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:12:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:33.750+0000 7fed4259c640 1 -- 172.21.15.45:0/2005585845 >> v1:172.21.15.84:6792/0 conn(0x7fed4406e140 legacy=0x7fed4412f7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:34.151+0000 7fed4259c640 1 -- 172.21.15.45:0/2005585845 >> v1:172.21.15.84:6792/0 conn(0x7fed4406e140 legacy=0x7fed4412f7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:34.151+0000 7fed3bfff640 1 -- 172.21.15.45:0/2005585845 >> v1:172.21.15.84:6789/0 conn(0x7fed4406e510 legacy=0x7fed44134510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:12:34.421 INFO:tasks.rados.rados.0.smithi084.stdout:2853: done (6 left) 2024-09-18T14:12:34.421 INFO:tasks.rados.rados.0.smithi084.stdout:2854: snap_create 2024-09-18T14:12:34.421 INFO:tasks.rados.rados.0.smithi084.stdout:2848: finishing write tid 1 to smithi08434663-16 2024-09-18T14:12:34.421 INFO:tasks.rados.rados.0.smithi084.stdout:2848: finishing write tid 2 to smithi08434663-16 2024-09-18T14:12:34.422 INFO:tasks.rados.rados.0.smithi084.stdout:2848: finishing write tid 3 to smithi08434663-16 2024-09-18T14:12:34.422 INFO:tasks.rados.rados.0.smithi084.stdout:2848: finishing write tid 4 to smithi08434663-16 2024-09-18T14:12:34.422 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1268 (ObjNum 975 snap 310 seq_num 975) dirty exists 2024-09-18T14:12:34.422 INFO:tasks.rados.rados.0.smithi084.stdout:2848: left oid 16 (ObjNum 975 snap 310 seq_num 975) 2024-09-18T14:12:34.422 INFO:tasks.rados.rados.0.smithi084.stdout:2850: finishing rollback tid 1 to smithi08434663-19 2024-09-18T14:12:34.422 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1515 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:12:34.422 INFO:tasks.rados.rados.0.smithi084.stdout:2846: finishing write tid 1 to smithi08434663-29 2024-09-18T14:12:34.422 INFO:tasks.rados.rados.0.smithi084.stdout:2849: expect (ObjNum 937 snap 299 seq_num 937) 2024-09-18T14:12:34.697 INFO:tasks.rados.rados.0.smithi084.stdout:2852: finishing write tid 1 to smithi08434663-7 2024-09-18T14:12:34.697 INFO:tasks.rados.rados.0.smithi084.stdout:2846: finishing write tid 2 to smithi08434663-29 2024-09-18T14:12:34.697 INFO:tasks.rados.rados.0.smithi084.stdout:2846: finishing write tid 3 to smithi08434663-29 2024-09-18T14:12:34.697 INFO:tasks.rados.rados.0.smithi084.stdout:2852: finishing write tid 2 to smithi08434663-7 2024-09-18T14:12:34.697 INFO:tasks.rados.rados.0.smithi084.stdout:2852: finishing write tid 3 to smithi08434663-7 2024-09-18T14:12:34.697 INFO:tasks.rados.rados.0.smithi084.stdout:2846: finishing write tid 4 to smithi08434663-29 2024-09-18T14:12:34.697 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 1049 (ObjNum 974 snap 310 seq_num 974) dirty exists 2024-09-18T14:12:34.697 INFO:tasks.rados.rados.0.smithi084.stdout:2846: left oid 29 (ObjNum 974 snap 310 seq_num 974) 2024-09-18T14:12:34.698 INFO:tasks.rados.rados.0.smithi084.stdout:2852: finishing write tid 4 to smithi08434663-7 2024-09-18T14:12:34.698 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 1004 (ObjNum 976 snap 310 seq_num 976) dirty exists 2024-09-18T14:12:34.698 INFO:tasks.rados.rados.0.smithi084.stdout:2852: left oid 7 (ObjNum 976 snap 310 seq_num 976) 2024-09-18T14:12:34.698 INFO:tasks.rados.rados.0.smithi084.stdout:2851: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:12:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:34.952+0000 7fed3bfff640 1 -- 172.21.15.45:0/2005585845 >> v1:172.21.15.84:6789/0 conn(0x7fed4406e510 legacy=0x7fed44134510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:12:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:34.952+0000 7fed4259c640 1 -- 172.21.15.45:0/2005585845 >> v1:172.21.15.84:6792/0 conn(0x7fed4406e140 legacy=0x7fed4412f7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:35.423 INFO:tasks.rados.rados.0.smithi084.stdout:2846: done (6 left) 2024-09-18T14:12:35.423 INFO:tasks.rados.rados.0.smithi084.stdout:2848: done (5 left) 2024-09-18T14:12:35.423 INFO:tasks.rados.rados.0.smithi084.stdout:2849: done (4 left) 2024-09-18T14:12:35.423 INFO:tasks.rados.rados.0.smithi084.stdout:2850: done (3 left) 2024-09-18T14:12:35.423 INFO:tasks.rados.rados.0.smithi084.stdout:2851: done (2 left) 2024-09-18T14:12:35.423 INFO:tasks.rados.rados.0.smithi084.stdout:2852: done (1 left) 2024-09-18T14:12:35.423 INFO:tasks.rados.rados.0.smithi084.stdout:2854: done (0 left) 2024-09-18T14:12:35.423 INFO:tasks.rados.rados.0.smithi084.stdout:2855: copy_from oid 19 from oid 25 current snap is 311 2024-09-18T14:12:35.424 INFO:tasks.rados.rados.0.smithi084.stdout:2856: read oid 31 snap -1 2024-09-18T14:12:35.424 INFO:tasks.rados.rados.0.smithi084.stdout:2856: expect deleted 2024-09-18T14:12:35.424 INFO:tasks.rados.rados.0.smithi084.stdout:2857: snap_create 2024-09-18T14:12:35.476 INFO:tasks.rados.rados.0.smithi084.stdout:2855: finishing copy_from racing read to smithi08434663-19 2024-09-18T14:12:35.478 INFO:tasks.rados.rados.0.smithi084.stdout:2855: finishing copy_from to smithi08434663-19 2024-09-18T14:12:35.478 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1251 (ObjNum 955 snap 304 seq_num 955) dirty exists 2024-09-18T14:12:36.437 INFO:tasks.rados.rados.0.smithi084.stdout:2855: done (2 left) 2024-09-18T14:12:36.437 INFO:tasks.rados.rados.0.smithi084.stdout:2856: done (1 left) 2024-09-18T14:12:36.437 INFO:tasks.rados.rados.0.smithi084.stdout:2857: done (0 left) 2024-09-18T14:12:36.437 INFO:tasks.rados.rados.0.smithi084.stdout:2858: delete oid 11 current snap is 312 2024-09-18T14:12:36.439 INFO:tasks.rados.rados.0.smithi084.stdout:2858: done (0 left) 2024-09-18T14:12:36.439 INFO:tasks.rados.rados.0.smithi084.stdout:2859: snap_remove snap 295 2024-09-18T14:12:36.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 10 monclient(hunting): tick 2024-09-18T14:12:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 1 monclient(hunting): continuing hunt 2024-09-18T14:12:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:12:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 1 -- 172.21.15.45:0/2005585845 >> v1:172.21.15.45:6793/0 conn(0x7fed4406e910 legacy=0x7fed44137c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 1 -- 172.21.15.45:0/2005585845 >> v1:172.21.15.84:6792/0 conn(0x7fed4406e140 legacy=0x7fed4412f7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:12:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 1 -- 172.21.15.45:0/2005585845 >> v1:172.21.15.84:6789/0 conn(0x7fed4406e510 legacy=0x7fed44134510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:12:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 10 monclient: _add_conns ranks=[6,4,0,2,5,3] 2024-09-18T14:12:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 10 monclient(hunting): picked mon.noname-d con 0x7fed24003760 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 10 monclient(hunting): picked mon.noname-c con 0x7fed240073e0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 10 monclient(hunting): picked mon.noname-a con 0x7fed2400b140 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 1 -- 172.21.15.45:0/2005585845 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4406c370 con 0x7fed2400b140 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 1 -- 172.21.15.45:0/2005585845 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4405ed70 con 0x7fed240073e0 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 1 -- 172.21.15.45:0/2005585845 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4409fcd0 con 0x7fed24003760 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed40d99640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686662720 0 0) 0x7fed4409fcd0 con 0x7fed24003760 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed41d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed41d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed41d9b640 10 monclient(hunting): my global_id is 184533 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.549+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed300034e0 con 0x7fed24003760 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563651391 0 0) 0x7fed4406c370 con 0x7fed2400b140 2024-09-18T14:12:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient(hunting): my global_id is 5967 2024-09-18T14:12:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed4409fcd0 con 0x7fed2400b140 2024-09-18T14:12:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057991978 0 0) 0x7fed4405ed70 con 0x7fed240073e0 2024-09-18T14:12:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient(hunting): my global_id is 164731 2024-09-18T14:12:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed4406c370 con 0x7fed240073e0 2024-09-18T14:12:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2592421242 0 0) 0x7fed300034e0 con 0x7fed24003760 2024-09-18T14:12:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 >> v1:172.21.15.45:6791/0 conn(0x7fed240073e0 legacy=0x7fed24009870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 >> v1:172.21.15.45:6789/0 conn(0x7fed2400b140 legacy=0x7fed2400d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 1 monclient: found mon.noname-d 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed42d9d640 1 -- 172.21.15.45:0/2005585845 reap_dead start 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed24011fd0 con 0x7fed24003760 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient: _finish_auth 0 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fed34004160 con 0x7fed24003760 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed48b09640 5 monclient: authenticate success, global_id 184533 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed48b09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient: dump: 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed48b09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fed34004330 con 0x7fed24003760 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 1 -- 172.21.15.45:0/2005585845 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fed340048e0 con 0x7fed24003760 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed41d9b640 10 monclient: dump: 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:36.558 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed48b09640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.550+0000 7fed48b09640 4 set_mon_vals no callback set 2024-09-18T14:12:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.551+0000 7fed48b09640 10 monclient: shutdown 2024-09-18T14:12:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.551+0000 7fed48b09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.551+0000 7fed48b09640 1 -- 172.21.15.45:0/2005585845 >> v1:172.21.15.45:6792/0 conn(0x7fed24003760 legacy=0x7fed24005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.551+0000 7fed48b09640 1 -- 172.21.15.45:0/2005585845 shutdown_connections 2024-09-18T14:12:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.551+0000 7fed48b09640 1 -- 172.21.15.45:0/2005585845 >> 172.21.15.45:0/2005585845 conn(0x7fed440a19a0 msgr2=0x7fed440a1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.551+0000 7fed48b09640 1 -- 172.21.15.45:0/2005585845 shutdown_connections 2024-09-18T14:12:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.551+0000 7fed48b09640 1 -- 172.21.15.45:0/2005585845 wait complete. 2024-09-18T14:12:36.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.552+0000 7fed48b09640 10 monclient: build_initial_monmap 2024-09-18T14:12:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.552+0000 7fed48b09640 10 monclient: monmap: 2024-09-18T14:12:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.552+0000 7fed48b09640 1 Processor -- start 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.552+0000 7fed48b09640 1 -- start start 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.552+0000 7fed48b09640 10 monclient: init 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed48b09640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed48b09640 10 monclient: _add_conns ranks=[2,7,6,1,3,8,4,0,5] 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed48b09640 10 monclient(hunting): picked mon.noname-b con 0x7fed24003760 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed48b09640 10 monclient(hunting): picked mon.noname-i con 0x7fed4412e850 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed48b09640 10 monclient(hunting): picked mon.noname-d con 0x7fed441477d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed48b09640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4411c410 con 0x7fed24003760 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed48b09640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4409bd10 con 0x7fed441477d0 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed48b09640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4411d280 con 0x7fed4412e850 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed48b09640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed48b09640 10 monclient(hunting): authenticate will time out at 1571.891408s 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed4259c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fed24003760 0x7fed4412e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:59578/0 (socket says 172.21.15.45:59578) 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed4259c640 1 -- 172.21.15.45:0/192870502 learned_addr learned my addr 172.21.15.45:0/192870502 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed3bfff640 1 -- 172.21.15.45:0/192870502 >> v1:172.21.15.84:6792/0 conn(0x7fed4412e850 legacy=0x7fed44097a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936335217 0 0) 0x7fed4409bd10 con 0x7fed441477d0 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed3a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed3a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed3a7fc640 10 monclient(hunting): my global_id is 184542 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed240115a0 con 0x7fed441477d0 2024-09-18T14:12:36.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217941578 0 0) 0x7fed4411c410 con 0x7fed24003760 2024-09-18T14:12:36.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed3a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:36.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed3a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:36.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed3a7fc640 10 monclient(hunting): my global_id is 145244 2024-09-18T14:12:36.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.553+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed4409bd10 con 0x7fed24003760 2024-09-18T14:12:36.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3141433790 0 0) 0x7fed240115a0 con 0x7fed441477d0 2024-09-18T14:12:36.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed4411c410 con 0x7fed441477d0 2024-09-18T14:12:36.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fed3c001c10 con 0x7fed441477d0 2024-09-18T14:12:36.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:36.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:36.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:36.567 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:36.567 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:36.568 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2022059325 0 0) 0x7fed4409bd10 con 0x7fed24003760 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed240115a0 con 0x7fed24003760 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fed34004150 con 0x7fed24003760 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2417383910 0 0) 0x7fed4411c410 con 0x7fed441477d0 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 >> v1:172.21.15.84:6792/0 conn(0x7fed4412e850 legacy=0x7fed44097a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 >> v1:172.21.15.45:6790/0 conn(0x7fed24003760 legacy=0x7fed4412e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 monclient: found mon.d 2024-09-18T14:12:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4411d560 con 0x7fed441477d0 2024-09-18T14:12:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed48b09640 5 monclient: authenticate success, global_id 184542 2024-09-18T14:12:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed48b09640 10 monclient: _renew_subs 2024-09-18T14:12:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed48b09640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed48b09640 1 -- 172.21.15.45:0/192870502 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4412ef80 con 0x7fed441477d0 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed48b09640 10 monclient: _renew_subs 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed48b09640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed48b09640 1 -- 172.21.15.45:0/192870502 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fed44134b60 con 0x7fed441477d0 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fed3c004b20 con 0x7fed441477d0 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fed3c004fd0 con 0x7fed441477d0 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed4159a640 4 set_mon_vals no callback set 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.554+0000 7fed3a7fc640 10 monclient: dump: 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:36.573 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.555+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fed3c00ffa0 con 0x7fed441477d0 2024-09-18T14:12:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.555+0000 7fed48b09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.555+0000 7fed48b09640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.555+0000 7fed48b09640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.555+0000 7fed48b09640 1 -- 172.21.15.45:0/192870502 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed28005010 con 0x7fed441477d0 2024-09-18T14:12:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.555+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(635..635 src has 424..635) v4 ==== 3895+0+0 (unknown 1966843663 0 0) 0x7fed3c046aa0 con 0x7fed441477d0 2024-09-18T14:12:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.556+0000 7fed4259c640 10 monclient: get_auth_request con 0x7fed2403b5e0 auth_method 0 2024-09-18T14:12:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.560+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fed3c014ee0 con 0x7fed441477d0 2024-09-18T14:12:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.560+0000 7fed3a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.560+0000 7fed3a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:36.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.791+0000 7fed48b09640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:12:36.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.791+0000 7fed48b09640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:12:36.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.791+0000 7fed48b09640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:36.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.791+0000 7fed48b09640 1 -- 172.21.15.45:0/192870502 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fed28005840 con 0x7fed441477d0 2024-09-18T14:12:36.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.791+0000 7fed3a7fc640 1 -- 172.21.15.45:0/192870502 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2671 (unknown 254746657 0 2704913403) 0x7fed3c014d00 con 0x7fed441477d0 2024-09-18T14:12:36.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.791+0000 7fed3a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:12:36.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.791+0000 7fed3a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":112,"quorum":[0,2,3,4,5,6],"quorum_names":["a","b","g","c","h","d"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:12:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.793+0000 7fed48b09640 1 -- 172.21.15.45:0/192870502 >> v1:172.21.15.84:6812/31478 conn(0x7fed2403b5e0 legacy=0x7fed2403da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.793+0000 7fed48b09640 10 monclient: shutdown 2024-09-18T14:12:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.793+0000 7fed48b09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.793+0000 7fed48b09640 1 -- 172.21.15.45:0/192870502 >> v1:172.21.15.45:6792/0 conn(0x7fed441477d0 legacy=0x7fed44149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.793+0000 7fed48b09640 1 -- 172.21.15.45:0/192870502 shutdown_connections 2024-09-18T14:12:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.793+0000 7fed48b09640 1 -- 172.21.15.45:0/192870502 >> 172.21.15.45:0/192870502 conn(0x7fed440a19a0 msgr2=0x7fed4409c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.793+0000 7fed48b09640 1 -- 172.21.15.45:0/192870502 shutdown_connections 2024-09-18T14:12:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:36.794+0000 7fed48b09640 1 -- 172.21.15.45:0/192870502 wait complete. 2024-09-18T14:12:37.438 INFO:tasks.rados.rados.0.smithi084.stdout:2859: done (0 left) 2024-09-18T14:12:37.438 INFO:tasks.rados.rados.0.smithi084.stdout:2860: copy_from oid 46 from oid 17 current snap is 312 2024-09-18T14:12:37.439 INFO:tasks.rados.rados.0.smithi084.stdout:2861: copy_from oid 15 from oid 50 current snap is 312 2024-09-18T14:12:37.439 INFO:tasks.rados.rados.0.smithi084.stdout:2862: rollback oid 16 current snap is 312 2024-09-18T14:12:37.439 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 16 to 309 2024-09-18T14:12:37.439 INFO:tasks.rados.rados.0.smithi084.stdout:2863: read oid 24 snap -1 2024-09-18T14:12:37.439 INFO:tasks.rados.rados.0.smithi084.stdout:2863: expect (ObjNum 959 snap 306 seq_num 959) 2024-09-18T14:12:37.439 INFO:tasks.rados.rados.0.smithi084.stdout:2864: copy_from oid 37 from oid 42 current snap is 312 2024-09-18T14:12:37.440 INFO:tasks.rados.rados.0.smithi084.stdout:2865: rollback oid 30 current snap is 312 2024-09-18T14:12:37.440 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 30 to 303 2024-09-18T14:12:37.440 INFO:tasks.rados.rados.0.smithi084.stdout:2866: write (excl) oid 39 current snap is 312 2024-09-18T14:12:37.440 INFO:tasks.rados.rados.0.smithi084.stdout:2866: seq_num 978 ranges {437246=486706,1363860=623668} 2024-09-18T14:12:37.445 INFO:tasks.rados.rados.0.smithi084.stdout:2866: writing smithi08434663-39 from 437246 to 923952 tid 1 2024-09-18T14:12:37.451 INFO:tasks.rados.rados.0.smithi084.stdout:2866: writing smithi08434663-39 from 1363860 to 1987528 tid 2 2024-09-18T14:12:37.451 INFO:tasks.rados.rados.0.smithi084.stdout:2863: expect (ObjNum 959 snap 306 seq_num 959) 2024-09-18T14:12:37.552 INFO:tasks.rados.rados.0.smithi084.stdout:2865: finishing rollback tid 1 to smithi08434663-30 2024-09-18T14:12:37.552 INFO:tasks.rados.rados.0.smithi084.stdout:2865: finishing rollback tid 2 to smithi08434663-30 2024-09-18T14:12:37.552 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 864 (ObjNum 951 snap 303 seq_num 951) dirty exists 2024-09-18T14:12:37.552 INFO:tasks.rados.rados.0.smithi084.stdout:2861: finishing copy_from to smithi08434663-15 2024-09-18T14:12:37.552 INFO:tasks.rados.rados.0.smithi084.stdout:2861: got expected ENOENT (src dne) 2024-09-18T14:12:37.552 INFO:tasks.rados.rados.0.smithi084.stdout:2864: finishing copy_from racing read to smithi08434663-37 2024-09-18T14:12:37.552 INFO:tasks.rados.rados.0.smithi084.stdout:2866: finishing write tid 1 to smithi08434663-39 2024-09-18T14:12:37.552 INFO:tasks.rados.rados.0.smithi084.stdout:2860: finishing copy_from racing read to smithi08434663-46 2024-09-18T14:12:37.552 INFO:tasks.rados.rados.0.smithi084.stdout:2863: done (6 left) 2024-09-18T14:12:37.552 INFO:tasks.rados.rados.0.smithi084.stdout:2865: done (5 left) 2024-09-18T14:12:37.552 INFO:tasks.rados.rados.0.smithi084.stdout:2867: read oid 33 snap -1 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:2867: expect deleted 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:2868: copy_from oid 14 from oid 13 current snap is 312 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:2861: finishing copy_from racing read to smithi08434663-15 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:2862: finishing rollback tid 0 to smithi08434663-16 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:2862: finishing rollback tid 1 to smithi08434663-16 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:2862: finishing rollback tid 2 to smithi08434663-16 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1271 (ObjNum 915 snap 286 seq_num 915) dirty exists 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:2866: finishing write tid 2 to smithi08434663-39 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:2866: finishing write tid 3 to smithi08434663-39 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:2866: finishing write tid 4 to smithi08434663-39 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 1264 (ObjNum 978 snap 312 seq_num 978) dirty exists 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:2866: left oid 39 (ObjNum 978 snap 312 seq_num 978) 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:2860: finishing copy_from to smithi08434663-46 2024-09-18T14:12:37.553 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1046 (ObjNum 968 snap 308 seq_num 968) dirty exists 2024-09-18T14:12:37.554 INFO:tasks.rados.rados.0.smithi084.stdout:2864: finishing copy_from to smithi08434663-37 2024-09-18T14:12:37.554 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 1311 (ObjNum 903 snap 281 seq_num 903) dirty exists 2024-09-18T14:12:37.554 INFO:tasks.rados.rados.0.smithi084.stdout:2860: done (6 left) 2024-09-18T14:12:37.554 INFO:tasks.rados.rados.0.smithi084.stdout:2861: done (5 left) 2024-09-18T14:12:37.554 INFO:tasks.rados.rados.0.smithi084.stdout:2862: done (4 left) 2024-09-18T14:12:37.554 INFO:tasks.rados.rados.0.smithi084.stdout:2864: done (3 left) 2024-09-18T14:12:37.554 INFO:tasks.rados.rados.0.smithi084.stdout:2866: done (2 left) 2024-09-18T14:12:37.554 INFO:tasks.rados.rados.0.smithi084.stdout:2869: rollback oid 44 current snap is 312 2024-09-18T14:12:37.554 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 44 to 310 2024-09-18T14:12:37.554 INFO:tasks.rados.rados.0.smithi084.stdout:2870: write (excl) oid 41 current snap is 312 2024-09-18T14:12:37.554 INFO:tasks.rados.rados.0.smithi084.stdout:2870: seq_num 979 ranges {454053=561955,1422342=483679,2368404=85649} 2024-09-18T14:12:37.556 INFO:tasks.rados.rados.0.smithi084.stdout:2870: writing smithi08434663-41 from 454053 to 1016008 tid 1 2024-09-18T14:12:37.561 INFO:tasks.rados.rados.0.smithi084.stdout:2870: writing smithi08434663-41 from 1422342 to 1906021 tid 2 2024-09-18T14:12:37.564 INFO:tasks.rados.rados.0.smithi084.stdout:2870: writing smithi08434663-41 from 2368404 to 2454053 tid 3 2024-09-18T14:12:37.565 INFO:tasks.rados.rados.0.smithi084.stdout:2871: read oid 28 snap -1 2024-09-18T14:12:37.565 INFO:tasks.rados.rados.0.smithi084.stdout:2871: expect (ObjNum 971 snap 308 seq_num 971) 2024-09-18T14:12:37.565 INFO:tasks.rados.rados.0.smithi084.stdout:2872: snap_remove snap 300 2024-09-18T14:12:38.416 INFO:tasks.rados.rados.0.smithi084.stdout:2872: done (5 left) 2024-09-18T14:12:38.417 INFO:tasks.rados.rados.0.smithi084.stdout:2873: write oid 27 current snap is 312 2024-09-18T14:12:38.417 INFO:tasks.rados.rados.0.smithi084.stdout:2873: seq_num 980 ranges {470860=637204,1880824=590036} 2024-09-18T14:12:38.423 INFO:tasks.rados.rados.0.smithi084.stdout:2873: writing smithi08434663-27 from 470860 to 1108064 tid 1 2024-09-18T14:12:38.432 INFO:tasks.rados.rados.0.smithi084.stdout:2873: writing smithi08434663-27 from 1880824 to 2470860 tid 2 2024-09-18T14:12:38.432 INFO:tasks.rados.rados.0.smithi084.stdout:2874: write (excl) oid 31 current snap is 312 2024-09-18T14:12:38.432 INFO:tasks.rados.rados.0.smithi084.stdout:2874: seq_num 981 ranges {487667=428806,1655659=770995} 2024-09-18T14:12:38.437 INFO:tasks.rados.rados.0.smithi084.stdout:2874: writing smithi08434663-31 from 487667 to 916473 tid 1 2024-09-18T14:12:38.447 INFO:tasks.rados.rados.0.smithi084.stdout:2874: writing smithi08434663-31 from 1655659 to 2426654 tid 2 2024-09-18T14:12:38.447 INFO:tasks.rados.rados.0.smithi084.stdout:2875: read oid 8 snap 309 2024-09-18T14:12:38.447 INFO:tasks.rados.rados.0.smithi084.stdout:2875: expect deleted 2024-09-18T14:12:38.447 INFO:tasks.rados.rados.0.smithi084.stdout:2876: copy_from oid 46 from oid 32 current snap is 312 2024-09-18T14:12:38.447 INFO:tasks.rados.rados.0.smithi084.stdout:2868: finishing copy_from to smithi08434663-14 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2868: got expected ENOENT (src dne) 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2868: finishing copy_from racing read to smithi08434663-14 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2869: finishing rollback tid 0 to smithi08434663-44 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2869: finishing rollback tid 1 to smithi08434663-44 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2869: finishing rollback tid 2 to smithi08434663-44 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 961 (ObjNum 908 snap 281 seq_num 908) dirty exists 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2870: finishing write tid 1 to smithi08434663-41 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2870: finishing write tid 2 to smithi08434663-41 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2867: done (8 left) 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2868: done (7 left) 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2869: done (6 left) 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2877: read oid 49 snap 310 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2877: expect (ObjNum 966 snap 307 seq_num 966) 2024-09-18T14:12:38.448 INFO:tasks.rados.rados.0.smithi084.stdout:2878: copy_from oid 40 from oid 48 current snap is 312 2024-09-18T14:12:38.449 INFO:tasks.rados.rados.0.smithi084.stdout:2870: finishing write tid 3 to smithi08434663-41 2024-09-18T14:12:38.449 INFO:tasks.rados.rados.0.smithi084.stdout:2870: finishing write tid 4 to smithi08434663-41 2024-09-18T14:12:38.449 INFO:tasks.rados.rados.0.smithi084.stdout:2870: finishing write tid 5 to smithi08434663-41 2024-09-18T14:12:38.449 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 1323 (ObjNum 979 snap 312 seq_num 979) dirty exists 2024-09-18T14:12:38.449 INFO:tasks.rados.rados.0.smithi084.stdout:2870: left oid 41 (ObjNum 979 snap 312 seq_num 979) 2024-09-18T14:12:38.449 INFO:tasks.rados.rados.0.smithi084.stdout:2871: expect (ObjNum 971 snap 308 seq_num 971) 2024-09-18T14:12:38.543 INFO:tasks.rados.rados.0.smithi084.stdout:2873: finishing write tid 1 to smithi08434663-27 2024-09-18T14:12:38.543 INFO:tasks.rados.rados.0.smithi084.stdout:2870: done (7 left) 2024-09-18T14:12:38.543 INFO:tasks.rados.rados.0.smithi084.stdout:2871: done (6 left) 2024-09-18T14:12:38.543 INFO:tasks.rados.rados.0.smithi084.stdout:2879: copy_from oid 17 from oid 30 current snap is 312 2024-09-18T14:12:38.544 INFO:tasks.rados.rados.0.smithi084.stdout:2873: finishing write tid 2 to smithi08434663-27 2024-09-18T14:12:38.544 INFO:tasks.rados.rados.0.smithi084.stdout:2873: finishing write tid 3 to smithi08434663-27 2024-09-18T14:12:38.544 INFO:tasks.rados.rados.0.smithi084.stdout:2873: finishing write tid 4 to smithi08434663-27 2024-09-18T14:12:38.544 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1232 (ObjNum 980 snap 312 seq_num 980) dirty exists 2024-09-18T14:12:38.544 INFO:tasks.rados.rados.0.smithi084.stdout:2873: left oid 27 (ObjNum 980 snap 312 seq_num 980) 2024-09-18T14:12:38.544 INFO:tasks.rados.rados.0.smithi084.stdout:2874: finishing write tid 1 to smithi08434663-31 2024-09-18T14:12:38.544 INFO:tasks.rados.rados.0.smithi084.stdout:2877: expect (ObjNum 966 snap 307 seq_num 966) 2024-09-18T14:12:38.643 INFO:tasks.rados.rados.0.smithi084.stdout:2874: finishing write tid 2 to smithi08434663-31 2024-09-18T14:12:38.643 INFO:tasks.rados.rados.0.smithi084.stdout:2874: finishing write tid 3 to smithi08434663-31 2024-09-18T14:12:38.643 INFO:tasks.rados.rados.0.smithi084.stdout:2874: finishing write tid 4 to smithi08434663-31 2024-09-18T14:12:38.643 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 600 (ObjNum 981 snap 312 seq_num 981) dirty exists 2024-09-18T14:12:38.643 INFO:tasks.rados.rados.0.smithi084.stdout:2874: left oid 31 (ObjNum 981 snap 312 seq_num 981) 2024-09-18T14:12:38.643 INFO:tasks.rados.rados.0.smithi084.stdout:2876: finishing copy_from racing read to smithi08434663-46 2024-09-18T14:12:38.643 INFO:tasks.rados.rados.0.smithi084.stdout:2878: finishing copy_from racing read to smithi08434663-40 2024-09-18T14:12:38.643 INFO:tasks.rados.rados.0.smithi084.stdout:2878: finishing copy_from to smithi08434663-40 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 1257 (ObjNum 970 snap 308 seq_num 970) dirty exists 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:2876: finishing copy_from to smithi08434663-46 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1446 (ObjNum 901 snap 281 seq_num 901) dirty exists 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:2873: done (6 left) 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:2874: done (5 left) 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:2875: done (4 left) 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:2876: done (3 left) 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:2877: done (2 left) 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:2878: done (1 left) 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:2880: delete oid 29 current snap is 312 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:2879: finishing copy_from racing read to smithi08434663-17 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:2879: finishing copy_from to smithi08434663-17 2024-09-18T14:12:38.644 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 864 (ObjNum 951 snap 303 seq_num 951) dirty exists 2024-09-18T14:12:38.646 INFO:tasks.rados.rados.0.smithi084.stdout:2879: done (1 left) 2024-09-18T14:12:38.646 INFO:tasks.rados.rados.0.smithi084.stdout:2880: done (0 left) 2024-09-18T14:12:38.646 INFO:tasks.rados.rados.0.smithi084.stdout:2881: snap_create 2024-09-18T14:12:39.450 INFO:tasks.rados.rados.0.smithi084.stdout:2881: done (0 left) 2024-09-18T14:12:39.451 INFO:tasks.rados.rados.0.smithi084.stdout:2882: copy_from oid 19 from oid 31 current snap is 313 2024-09-18T14:12:39.451 INFO:tasks.rados.rados.0.smithi084.stdout:2883: snap_create 2024-09-18T14:12:39.540 INFO:tasks.rados.rados.0.smithi084.stdout:2882: finishing copy_from racing read to smithi08434663-19 2024-09-18T14:12:39.541 INFO:tasks.rados.rados.0.smithi084.stdout:2882: finishing copy_from to smithi08434663-19 2024-09-18T14:12:39.541 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 600 (ObjNum 981 snap 312 seq_num 981) dirty exists 2024-09-18T14:12:39.805 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:12:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.913+0000 7fc74c35d640 10 monclient: get_monmap_and_config 2024-09-18T14:12:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.913+0000 7fc74c35d640 10 monclient: build_initial_monmap 2024-09-18T14:12:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.913+0000 7fc74c35d640 10 monclient: monmap: 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:39.914367+0000 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:39.914367+0000 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:39.915 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 1 Processor -- start 2024-09-18T14:12:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 1 -- start start 2024-09-18T14:12:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 10 monclient: init 2024-09-18T14:12:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 10 monclient: _add_conns ranks=[7,6,3,2,4,1,0,5,8] 2024-09-18T14:12:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 10 monclient(hunting): picked mon.noname-i con 0x7fc7440a2520 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 10 monclient(hunting): picked mon.noname-d con 0x7fc744096a20 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 10 monclient(hunting): picked mon.noname-g con 0x7fc744132fc0 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74405ed70 con 0x7fc744132fc0 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc744097040 con 0x7fc744096a20 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7440a28f0 con 0x7fc7440a2520 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74c35d640 10 monclient(hunting): authenticate will time out at 1575.252786s 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc74a0f9640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7fc7440a2520 legacy=0x7fc7441328b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc7498f8640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fc744096a20 0x7fc744137750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:57204/0 (socket says 172.21.15.45:57204) 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.914+0000 7fc7498f8640 1 -- 172.21.15.45:0/631034838 learned_addr learned my addr 172.21.15.45:0/631034838 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 1 -- 172.21.15.45:0/631034838 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585807768 0 0) 0x7fc744097040 con 0x7fc744096a20 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient(hunting): my global_id is 184551 2024-09-18T14:12:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 1 -- 172.21.15.45:0/631034838 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7300034e0 con 0x7fc744096a20 2024-09-18T14:12:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 1 -- 172.21.15.45:0/631034838 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323062432 0 0) 0x7fc74405ed70 con 0x7fc744132fc0 2024-09-18T14:12:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient(hunting): my global_id is 184458 2024-09-18T14:12:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 1 -- 172.21.15.45:0/631034838 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc744097040 con 0x7fc744132fc0 2024-09-18T14:12:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 1 -- 172.21.15.45:0/631034838 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 898937420 0 0) 0x7fc7300034e0 con 0x7fc744096a20 2024-09-18T14:12:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 1 -- 172.21.15.45:0/631034838 >> v1:172.21.15.84:6792/0 conn(0x7fc7440a2520 legacy=0x7fc7441328b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:12:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 1 -- 172.21.15.45:0/631034838 >> v1:172.21.15.84:6790/0 conn(0x7fc744132fc0 legacy=0x7fc74413ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient: _finish_hunting 0 2024-09-18T14:12:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 1 monclient: found mon.noname-d 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 1 -- 172.21.15.45:0/631034838 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74409f4b0 con 0x7fc744096a20 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient: _finish_auth 0 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 1 -- 172.21.15.45:0/631034838 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc734001830 con 0x7fc744096a20 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient: dump: 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc74c35d640 5 monclient: authenticate success, global_id 184551 2024-09-18T14:12:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc74c35d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 1 -- 172.21.15.45:0/631034838 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc734001a10 con 0x7fc744096a20 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 1 -- 172.21.15.45:0/631034838 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc734004ba0 con 0x7fc744096a20 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc74c35d640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc74c35d640 4 set_mon_vals no callback set 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.915+0000 7fc7490f7640 10 monclient: dump: 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.916+0000 7fc74c35d640 10 monclient: shutdown 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.916+0000 7fc74c35d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.916+0000 7fc74c35d640 1 -- 172.21.15.45:0/631034838 >> v1:172.21.15.45:6792/0 conn(0x7fc744096a20 legacy=0x7fc744137750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.916+0000 7fc74c35d640 1 -- 172.21.15.45:0/631034838 shutdown_connections 2024-09-18T14:12:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.916+0000 7fc74c35d640 1 -- 172.21.15.45:0/631034838 >> 172.21.15.45:0/631034838 conn(0x7fc744063130 msgr2=0x7fc744063500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.916+0000 7fc74c35d640 1 -- 172.21.15.45:0/631034838 shutdown_connections 2024-09-18T14:12:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.916+0000 7fc74c35d640 1 -- 172.21.15.45:0/631034838 wait complete. 2024-09-18T14:12:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.916+0000 7fc74c35d640 10 monclient: build_initial_monmap 2024-09-18T14:12:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.916+0000 7fc74c35d640 10 monclient: monmap: 2024-09-18T14:12:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 1 Processor -- start 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 1 -- start start 2024-09-18T14:12:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 10 monclient: init 2024-09-18T14:12:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 10 monclient: _add_conns ranks=[8,7,3,5,1,6,0,2,4] 2024-09-18T14:12:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 10 monclient(hunting): picked mon.noname-e con 0x7fc744096a20 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 10 monclient(hunting): picked mon.noname-i con 0x7fc7440a2520 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 10 monclient(hunting): picked mon.noname-g con 0x7fc744132fc0 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7440a2270 con 0x7fc744132fc0 2024-09-18T14:12:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc744133950 con 0x7fc7440a2520 2024-09-18T14:12:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc744121a80 con 0x7fc744096a20 2024-09-18T14:12:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc7498f8640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7fc7440a2520 legacy=0x7fc7441467a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74c35d640 10 monclient(hunting): authenticate will time out at 1575.255692s 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74a0f9640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fc744096a20 0x7fc744131370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:47334/0 (socket says 172.21.15.45:47334) 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.917+0000 7fc74a0f9640 1 -- 172.21.15.45:0/1180361523 learned_addr learned my addr 172.21.15.45:0/1180361523 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784827813 0 0) 0x7fc7440a2270 con 0x7fc744132fc0 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient(hunting): my global_id is 184467 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7280034e0 con 0x7fc744132fc0 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1609311853 0 0) 0x7fc7280034e0 con 0x7fc744132fc0 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc7440a2270 con 0x7fc744132fc0 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc73c002a30 con 0x7fc744132fc0 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:12:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:39.935 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2550127801 0 0) 0x7fc7440a2270 con 0x7fc744132fc0 2024-09-18T14:12:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 >> v1:172.21.15.45:6793/0 conn(0x7fc744096a20 legacy=0x7fc744131370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 >> v1:172.21.15.84:6792/0 conn(0x7fc7440a2520 legacy=0x7fc7441467a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:12:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 1 monclient: found mon.g 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc744120560 con 0x7fc744132fc0 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc74c35d640 5 monclient: authenticate success, global_id 184467 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc74c35d640 10 monclient: _renew_subs 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc74c35d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc74c35d640 1 -- 172.21.15.45:0/1180361523 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7441214c0 con 0x7fc744132fc0 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc74c35d640 10 monclient: _renew_subs 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc74c35d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc74c35d640 1 -- 172.21.15.45:0/1180361523 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc744142290 con 0x7fc744132fc0 2024-09-18T14:12:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc73c002d30 con 0x7fc744132fc0 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc73c001900 con 0x7fc744132fc0 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc7488f6640 4 set_mon_vals no callback set 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.918+0000 7fc73a7fc640 10 monclient: dump: 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.919+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc73c001ae0 con 0x7fc744132fc0 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.919+0000 7fc74c35d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.919+0000 7fc74c35d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.919+0000 7fc74c35d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:12:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.919+0000 7fc74c35d640 1 -- 172.21.15.45:0/1180361523 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72c005010 con 0x7fc744132fc0 2024-09-18T14:12:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.919+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(638..638 src has 424..638) v4 ==== 3927+0+0 (unknown 3928081348 0 0) 0x7fc73c0442f0 con 0x7fc744132fc0 2024-09-18T14:12:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.920+0000 7fc74a0f9640 10 monclient: get_auth_request con 0x7fc728039b90 auth_method 0 2024-09-18T14:12:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.924+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 <== mon.3 v1:172.21.15.84:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fc73c012ac0 con 0x7fc744132fc0 2024-09-18T14:12:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.924+0000 7fc73a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:39.924+0000 7fc73a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.152+0000 7fc74c35d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:12:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.152+0000 7fc74c35d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:12:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.152+0000 7fc74c35d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:12:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.152+0000 7fc74c35d640 1 -- 172.21.15.45:0/1180361523 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc72c005840 con 0x7fc744132fc0 2024-09-18T14:12:40.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.153+0000 7fc73a7fc640 1 -- 172.21.15.45:0/1180361523 <== mon.3 v1:172.21.15.84:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2671 (unknown 254746657 0 2152832112) 0x7fc73c0124c0 con 0x7fc744132fc0 2024-09-18T14:12:40.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.153+0000 7fc73a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:12:40.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.153+0000 7fc73a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:40.153 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":112,"quorum":[0,2,3,4,5,6],"quorum_names":["a","b","g","c","h","d"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:12:40.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.155+0000 7fc74c35d640 1 -- 172.21.15.45:0/1180361523 >> v1:172.21.15.84:6812/31478 conn(0x7fc728039b90 legacy=0x7fc72803c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.155+0000 7fc74c35d640 10 monclient: shutdown 2024-09-18T14:12:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.155+0000 7fc74c35d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.155+0000 7fc74c35d640 1 -- 172.21.15.45:0/1180361523 >> v1:172.21.15.84:6790/0 conn(0x7fc744132fc0 legacy=0x7fc744149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.155+0000 7fc74c35d640 1 -- 172.21.15.45:0/1180361523 shutdown_connections 2024-09-18T14:12:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.155+0000 7fc74c35d640 1 -- 172.21.15.45:0/1180361523 >> 172.21.15.45:0/1180361523 conn(0x7fc744063130 msgr2=0x7fc7440a3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.155+0000 7fc74c35d640 1 -- 172.21.15.45:0/1180361523 shutdown_connections 2024-09-18T14:12:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:40.155+0000 7fc74c35d640 1 -- 172.21.15.45:0/1180361523 wait complete. 2024-09-18T14:12:43.166 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:12:43.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.274+0000 7f97000dd640 10 monclient: get_monmap_and_config 2024-09-18T14:12:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.274+0000 7f97000dd640 10 monclient: build_initial_monmap 2024-09-18T14:12:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.274+0000 7f97000dd640 10 monclient: monmap: 2024-09-18T14:12:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:43.275705+0000 2024-09-18T14:12:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:43.275705+0000 2024-09-18T14:12:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:43.276 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:43.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.275+0000 7f97000dd640 1 Processor -- start 2024-09-18T14:12:43.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.275+0000 7f97000dd640 1 -- start start 2024-09-18T14:12:43.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.275+0000 7f97000dd640 10 monclient: init 2024-09-18T14:12:43.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 10 monclient: _add_conns ranks=[5,6,1,3,2,0,4,7,8] 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 10 monclient(hunting): picked mon.noname-h con 0x7f96f809f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 10 monclient(hunting): picked mon.noname-d con 0x7f96f809fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 10 monclient(hunting): picked mon.noname-f con 0x7f96f809ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f805ed70 con 0x7f96f809ff70 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f809dfc0 con 0x7f96f809f7d0 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f809dce0 con 0x7f96f809fba0 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 10 monclient(hunting): authenticate will time out at 1578.614192s 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fd678640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f96f809fba0 0x7f96f8137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:57216/0 (socket says 172.21.15.45:57216) 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fd678640 1 -- 172.21.15.45:0/858258140 learned_addr learned my addr 172.21.15.45:0/858258140 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fe67a640 1 -- 172.21.15.45:0/858258140 >> v1:172.21.15.84:6789/0 conn(0x7f96f809ff70 legacy=0x7f96f813b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 1 -- 172.21.15.45:0/858258140 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762981060 0 0) 0x7f96f809dce0 con 0x7f96f809fba0 2024-09-18T14:12:43.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 10 monclient(hunting): my global_id is 184560 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 1 -- 172.21.15.45:0/858258140 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96e40034e0 con 0x7f96f809fba0 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 1 -- 172.21.15.45:0/858258140 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 456239051 0 0) 0x7f96e40034e0 con 0x7f96f809fba0 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 1 -- 172.21.15.45:0/858258140 >> v1:172.21.15.84:6791/0 conn(0x7f96f809f7d0 legacy=0x7f96f8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 1 -- 172.21.15.45:0/858258140 >> v1:172.21.15.84:6789/0 conn(0x7f96f809ff70 legacy=0x7f96f813b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 10 monclient: _finish_hunting 0 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 1 monclient: found mon.noname-d 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 1 -- 172.21.15.45:0/858258140 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f80a1c00 con 0x7f96f809fba0 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 10 monclient: _finish_auth 0 2024-09-18T14:12:43.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 1 -- 172.21.15.45:0/858258140 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f96e8001830 con 0x7f96f809fba0 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 10 monclient: dump: 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 5 monclient: authenticate success, global_id 184560 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 1 -- 172.21.15.45:0/858258140 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f96e8001a10 con 0x7f96f809fba0 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 1 -- 172.21.15.45:0/858258140 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f96e8004ba0 con 0x7f96f809fba0 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f97000dd640 4 set_mon_vals no callback set 2024-09-18T14:12:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.276+0000 7f96fce77640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.277+0000 7f96fce77640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.277+0000 7f96fce77640 10 monclient: dump: 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.277+0000 7f97000dd640 10 monclient: shutdown 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.277+0000 7f97000dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.277+0000 7f97000dd640 1 -- 172.21.15.45:0/858258140 >> v1:172.21.15.45:6792/0 conn(0x7f96f809fba0 legacy=0x7f96f8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.277+0000 7f97000dd640 1 -- 172.21.15.45:0/858258140 shutdown_connections 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.277+0000 7f97000dd640 1 -- 172.21.15.45:0/858258140 >> 172.21.15.45:0/858258140 conn(0x7f96f80630b0 msgr2=0x7f96f80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.277+0000 7f97000dd640 1 -- 172.21.15.45:0/858258140 shutdown_connections 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.277+0000 7f97000dd640 1 -- 172.21.15.45:0/858258140 wait complete. 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 10 monclient: build_initial_monmap 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 10 monclient: monmap: 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 1 Processor -- start 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 1 -- start start 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 10 monclient: init 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 10 monclient: _add_conns ranks=[6,7,4,5,3,2,0,1,8] 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 10 monclient(hunting): picked mon.noname-d con 0x7f96f809f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 10 monclient(hunting): picked mon.noname-i con 0x7f96f809fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 10 monclient(hunting): picked mon.noname-c con 0x7f96f809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f80a2560 con 0x7f96f809ff70 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f8121ce0 con 0x7f96f809f7d0 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f8121a00 con 0x7f96f809fba0 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.278+0000 7f97000dd640 10 monclient(hunting): authenticate will time out at 1578.616992s 2024-09-18T14:12:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96fd678640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7f96f809fba0 legacy=0x7f96f81318b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96fde79640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f96f809f7d0 0x7f96f8141730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:57230/0 (socket says 172.21.15.45:57230) 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96fde79640 1 -- 172.21.15.45:0/1290126656 learned_addr learned my addr 172.21.15.45:0/1290126656 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64869896 0 0) 0x7f96f8121ce0 con 0x7f96f809f7d0 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): my global_id is 184569 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96dc0034e0 con 0x7f96f809f7d0 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256170901 0 0) 0x7f96f80a2560 con 0x7f96f809ff70 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): my global_id is 164740 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f8121ce0 con 0x7f96f809ff70 2024-09-18T14:12:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2077612466 0 0) 0x7f96dc0034e0 con 0x7f96f809f7d0 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f96f80a2560 con 0x7f96f809f7d0 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1957190805 0 0) 0x7f96f8121ce0 con 0x7f96f809ff70 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f96dc0034e0 con 0x7f96f809ff70 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f96f4001ba0 con 0x7f96f809f7d0 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): dump: 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f96f00045a0 con 0x7f96f809ff70 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient(hunting): dump: 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3976740881 0 0) 0x7f96f80a2560 con 0x7f96f809f7d0 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 >> v1:172.21.15.84:6792/0 conn(0x7f96f809fba0 legacy=0x7f96f81318b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 >> v1:172.21.15.45:6791/0 conn(0x7f96f809ff70 legacy=0x7f96f8149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient: _finish_hunting 0 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 monclient: found mon.d 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f8141e40 con 0x7f96f809f7d0 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 10 monclient: _finish_auth 0 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f97000dd640 5 monclient: authenticate success, global_id 184569 2024-09-18T14:12:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f97000dd640 10 monclient: _renew_subs 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f97000dd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f97000dd640 1 -- 172.21.15.45:0/1290126656 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f8131fc0 con 0x7f96f809f7d0 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.279+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f96f4001d00 con 0x7f96f809f7d0 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.280+0000 7f96edffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.280+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f96f40025f0 con 0x7f96f809f7d0 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.280+0000 7f96edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.280+0000 7f96edffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.280+0000 7f96edffb640 10 monclient: dump: 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.280+0000 7f97000dd640 10 monclient: _renew_subs 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.280+0000 7f96effff640 4 set_mon_vals no callback set 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.280+0000 7f97000dd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.280+0000 7f97000dd640 1 -- 172.21.15.45:0/1290126656 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f8120cb0 con 0x7f96f809f7d0 2024-09-18T14:12:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.280+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f96f40027d0 con 0x7f96f809f7d0 2024-09-18T14:12:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.280+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(638..638 src has 424..638) v4 ==== 3927+0+0 (unknown 3928081348 0 0) 0x7f96f4046b90 con 0x7f96f809f7d0 2024-09-18T14:12:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.281+0000 7f97000dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.281+0000 7f97000dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.281+0000 7f97000dd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.281+0000 7f97000dd640 1 -- 172.21.15.45:0/1290126656 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f804c4a0 con 0x7f96f809f7d0 2024-09-18T14:12:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.281+0000 7f96fd678640 10 monclient: get_auth_request con 0x7f96dc03a320 auth_method 0 2024-09-18T14:12:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.285+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f96f4014f40 con 0x7f96f809f7d0 2024-09-18T14:12:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.285+0000 7f96edffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:43.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.285+0000 7f96edffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:43.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.511+0000 7f97000dd640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:12:43.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.512+0000 7f97000dd640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:12:43.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.512+0000 7f97000dd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:12:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.512+0000 7f97000dd640 1 -- 172.21.15.45:0/1290126656 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f96f804b720 con 0x7f96f809f7d0 2024-09-18T14:12:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.512+0000 7f96edffb640 1 -- 172.21.15.45:0/1290126656 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2672 (unknown 254746657 0 1726563270) 0x7f96f4014d60 con 0x7f96f809f7d0 2024-09-18T14:12:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.512+0000 7f96edffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:12:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.512+0000 7f96edffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:43.513 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":112,"quorum":[0,2,3,4,5,6],"quorum_names":["a","b","g","c","h","d"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:12:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.514+0000 7f97000dd640 1 -- 172.21.15.45:0/1290126656 >> v1:172.21.15.84:6812/31478 conn(0x7f96dc03a320 legacy=0x7f96dc03c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.514+0000 7f97000dd640 10 monclient: shutdown 2024-09-18T14:12:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.514+0000 7f97000dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.514+0000 7f97000dd640 1 -- 172.21.15.45:0/1290126656 >> v1:172.21.15.45:6792/0 conn(0x7f96f809f7d0 legacy=0x7f96f8141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.514+0000 7f97000dd640 1 -- 172.21.15.45:0/1290126656 shutdown_connections 2024-09-18T14:12:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.514+0000 7f97000dd640 1 -- 172.21.15.45:0/1290126656 >> 172.21.15.45:0/1290126656 conn(0x7f96f80630b0 msgr2=0x7f96f80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.514+0000 7f97000dd640 1 -- 172.21.15.45:0/1290126656 shutdown_connections 2024-09-18T14:12:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:43.514+0000 7f97000dd640 1 -- 172.21.15.45:0/1290126656 wait complete. 2024-09-18T14:12:46.526 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:12:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.634+0000 7f0304d00640 10 monclient: get_monmap_and_config 2024-09-18T14:12:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.634+0000 7f0304d00640 10 monclient: build_initial_monmap 2024-09-18T14:12:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.634+0000 7f0304d00640 10 monclient: monmap: 2024-09-18T14:12:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:46.636 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:46.635574+0000 2024-09-18T14:12:46.636 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:46.635574+0000 2024-09-18T14:12:46.636 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:46.636 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:46.636 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:46.638 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.635+0000 7f0304d00640 1 Processor -- start 2024-09-18T14:12:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.635+0000 7f0304d00640 1 -- start start 2024-09-18T14:12:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.635+0000 7f0304d00640 10 monclient: init 2024-09-18T14:12:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.636+0000 7f0304d00640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.636+0000 7f0304d00640 10 monclient: _add_conns ranks=[3,2,5,1,0,8,6,4,7] 2024-09-18T14:12:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.636+0000 7f0304d00640 10 monclient(hunting): picked mon.noname-g con 0x7f030009f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.636+0000 7f0304d00640 10 monclient(hunting): picked mon.noname-b con 0x7f030009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.636+0000 7f0304d00640 10 monclient(hunting): picked mon.noname-h con 0x7f030009ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:46.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.636+0000 7f0304d00640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030005ed70 con 0x7f030009fba0 2024-09-18T14:12:46.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.636+0000 7f0304d00640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030009dfc0 con 0x7f030009f7d0 2024-09-18T14:12:46.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.636+0000 7f0304d00640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030009dce0 con 0x7f030009ff70 2024-09-18T14:12:46.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.636+0000 7f0304d00640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:46.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.636+0000 7f0304d00640 10 monclient(hunting): authenticate will time out at 1581.974229s 2024-09-18T14:12:46.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.636+0000 7f02fdd9b640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f030009fba0 0x7f0300137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:47858/0 (socket says 172.21.15.45:47858) 2024-09-18T14:12:46.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:46.636+0000 7f02fdd9b640 1 -- 172.21.15.45:0/426788519 learned_addr learned my addr 172.21.15.45:0/426788519 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:49.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 10 monclient(hunting): tick 2024-09-18T14:12:49.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 1 monclient(hunting): continuing hunt 2024-09-18T14:12:49.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:12:49.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6791/0 conn(0x7f030009ff70 legacy=0x7f030013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:49.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6790/0 conn(0x7f030009f7d0 legacy=0x7f0300132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:49.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.45:6790/0 conn(0x7f030009fba0 legacy=0x7f0300137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:49.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 10 monclient: _add_conns ranks=[0,8,6,7,1,4] 2024-09-18T14:12:49.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 10 monclient(hunting): picked mon.noname-a con 0x7f02e00037a0 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:49.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 10 monclient(hunting): picked mon.noname-e con 0x7f02e0007380 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:49.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 10 monclient(hunting): picked mon.noname-d con 0x7f02e000b0c0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:49.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03000a1c00 con 0x7f02e00037a0 2024-09-18T14:12:49.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e000ff60 con 0x7f02e000b0c0 2024-09-18T14:12:49.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e00111c0 con 0x7f02e0007380 2024-09-18T14:12:49.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:49.636+0000 7f02effff640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 10 monclient(hunting): tick 2024-09-18T14:12:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 1 monclient(hunting): continuing hunt 2024-09-18T14:12:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:12:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.45:6793/0 conn(0x7f02e0007380 legacy=0x7f02e0009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.45:6792/0 conn(0x7f02e000b0c0 legacy=0x7f02e000d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.45:6789/0 conn(0x7f02e00037a0 legacy=0x7f02e0005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 10 monclient: _add_conns ranks=[1,4,7] 2024-09-18T14:12:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 10 monclient(hunting): picked mon.noname-f con 0x7f02e000f110 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02fed9d640 1 -- 172.21.15.45:0/426788519 reap_dead start 2024-09-18T14:12:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 10 monclient(hunting): picked mon.noname-c con 0x7f02e00100f0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 10 monclient(hunting): picked mon.noname-i con 0x7f02e0006310 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02fed9d640 1 -- 172.21.15.45:0/426788519 reap_dead start 2024-09-18T14:12:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e0012420 con 0x7f02e000f110 2024-09-18T14:12:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e0010ee0 con 0x7f02e00100f0 2024-09-18T14:12:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e0006730 con 0x7f02e0006310 2024-09-18T14:12:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02effff640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02fe59c640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6789/0 conn(0x7f02e000f110 legacy=0x7f02e000f540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:12:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.636+0000 7f02fed9d640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6792/0 conn(0x7f02e0006310 legacy=0x7f02e0005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:52.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.837+0000 7f02fe59c640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6789/0 conn(0x7f02e000f110 legacy=0x7f02e000f540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:12:52.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:52.837+0000 7f02fed9d640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6792/0 conn(0x7f02e0006310 legacy=0x7f02e0005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:53.238+0000 7f02fe59c640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6789/0 conn(0x7f02e000f110 legacy=0x7f02e000f540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:12:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:53.238+0000 7f02fed9d640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6792/0 conn(0x7f02e0006310 legacy=0x7f02e0005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:54.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:54.039+0000 7f02fe59c640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6789/0 conn(0x7f02e000f110 legacy=0x7f02e000f540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:12:54.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:54.039+0000 7f02fed9d640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6792/0 conn(0x7f02e0006310 legacy=0x7f02e0005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 10 monclient(hunting): tick 2024-09-18T14:12:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 1 monclient(hunting): continuing hunt 2024-09-18T14:12:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:12:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6792/0 conn(0x7f02e0006310 legacy=0x7f02e0005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:12:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.45:6791/0 conn(0x7f02e00100f0 legacy=0x7f02e0010590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6789/0 conn(0x7f02e000f110 legacy=0x7f02e000f540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:12:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 10 monclient: _add_conns ranks=[6,2,8,0,4,1,5,3,7] 2024-09-18T14:12:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 10 monclient(hunting): picked mon.noname-d con 0x7f02e00068c0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 10 monclient(hunting): picked mon.noname-b con 0x7f02e0006c90 addr v1:172.21.15.45:6790/0 2024-09-18T14:12:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 10 monclient(hunting): picked mon.noname-e con 0x7f02e000d4b0 addr v1:172.21.15.45:6793/0 2024-09-18T14:12:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e00098c0 con 0x7f02e0006c90 2024-09-18T14:12:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e0012420 con 0x7f02e00068c0 2024-09-18T14:12:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e0006730 con 0x7f02e000d4b0 2024-09-18T14:12:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:55.637+0000 7f02effff640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:58.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 10 monclient(hunting): tick 2024-09-18T14:12:58.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 1 monclient(hunting): continuing hunt 2024-09-18T14:12:58.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:12:58.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.45:6793/0 conn(0x7f02e000d4b0 legacy=0x7f02e001b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:58.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.45:6792/0 conn(0x7f02e00068c0 legacy=0x7f02e000dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:58.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.45:6790/0 conn(0x7f02e0006c90 legacy=0x7f02e000e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:58.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 10 monclient: _add_conns ranks=[4,0,5,7,1,3] 2024-09-18T14:12:58.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 10 monclient(hunting): picked mon.noname-c con 0x7f02e00100f0 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:58.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 10 monclient(hunting): picked mon.noname-a con 0x7f02e001ce80 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:58.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02fed9d640 1 -- 172.21.15.45:0/426788519 reap_dead start 2024-09-18T14:12:58.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 10 monclient(hunting): picked mon.noname-h con 0x7f02e0020980 addr v1:172.21.15.84:6791/0 2024-09-18T14:12:58.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e000d910 con 0x7f02e001ce80 2024-09-18T14:12:58.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e0025550 con 0x7f02e00100f0 2024-09-18T14:12:58.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e000e430 con 0x7f02e0020980 2024-09-18T14:12:58.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:58.637+0000 7f02effff640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:59.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 1 -- 172.21.15.45:0/426788519 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374878935 0 0) 0x7f02e000d910 con 0x7f02e001ce80 2024-09-18T14:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 10 monclient(hunting): my global_id is 5976 2024-09-18T14:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02e80034e0 con 0x7f02e001ce80 2024-09-18T14:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 1 -- 172.21.15.45:0/426788519 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210223170 0 0) 0x7f02e0025550 con 0x7f02e00100f0 2024-09-18T14:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 10 monclient(hunting): my global_id is 164749 2024-09-18T14:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02e000d910 con 0x7f02e00100f0 2024-09-18T14:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 1 -- 172.21.15.45:0/426788519 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 391773223 0 0) 0x7f02e000d910 con 0x7f02e00100f0 2024-09-18T14:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.84:6791/0 conn(0x7f02e0020980 legacy=0x7f02e0022d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.45:6789/0 conn(0x7f02e001ce80 legacy=0x7f02e001f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 10 monclient: _finish_hunting 0 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 1 monclient: found mon.noname-c 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 1 -- 172.21.15.45:0/426788519 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e00068c0 con 0x7f02e00100f0 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 10 monclient: _finish_auth 0 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 1 -- 172.21.15.45:0/426788519 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f02f00040b0 con 0x7f02e00100f0 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f0304d00640 5 monclient: authenticate success, global_id 164749 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f0304d00640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:12:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f02fd59a640 10 monclient: dump: 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.505+0000 7f0304d00640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.506+0000 7f02fd59a640 1 -- 172.21.15.45:0/426788519 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f02f0004210 con 0x7f02e00100f0 2024-09-18T14:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.506+0000 7f02fd59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.506+0000 7f02fd59a640 1 -- 172.21.15.45:0/426788519 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f02f0004740 con 0x7f02e00100f0 2024-09-18T14:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.506+0000 7f0304d00640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.506+0000 7f0304d00640 4 set_mon_vals no callback set 2024-09-18T14:12:59.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.506+0000 7f02fd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:59.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.506+0000 7f02fd59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:59.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.506+0000 7f02fd59a640 10 monclient: dump: 2024-09-18T14:12:59.511 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:59.512 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.512 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.512 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.512 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:59.512 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.512 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:59.513 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.507+0000 7f0304d00640 10 monclient: shutdown 2024-09-18T14:12:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.507+0000 7f0304d00640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.507+0000 7f0304d00640 1 -- 172.21.15.45:0/426788519 >> v1:172.21.15.45:6791/0 conn(0x7f02e00100f0 legacy=0x7f02e0010590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.507+0000 7f0304d00640 1 -- 172.21.15.45:0/426788519 shutdown_connections 2024-09-18T14:12:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.507+0000 7f0304d00640 1 -- 172.21.15.45:0/426788519 >> 172.21.15.45:0/426788519 conn(0x7f03000630b0 msgr2=0x7f03000634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.507+0000 7f0304d00640 1 -- 172.21.15.45:0/426788519 shutdown_connections 2024-09-18T14:12:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.507+0000 7f0304d00640 1 -- 172.21.15.45:0/426788519 wait complete. 2024-09-18T14:12:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.508+0000 7f0304d00640 10 monclient: build_initial_monmap 2024-09-18T14:12:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.508+0000 7f0304d00640 10 monclient: monmap: 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.509+0000 7f0304d00640 1 Processor -- start 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.509+0000 7f0304d00640 1 -- start start 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.509+0000 7f0304d00640 10 monclient: init 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.510+0000 7f0304d00640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.510+0000 7f0304d00640 10 monclient: _add_conns ranks=[0,6,4,1,3,5,2,8,7] 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.510+0000 7f0304d00640 10 monclient(hunting): picked mon.noname-a con 0x7f02e00100f0 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.510+0000 7f0304d00640 10 monclient(hunting): picked mon.noname-d con 0x7f02e001ce80 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.510+0000 7f0304d00640 10 monclient(hunting): picked mon.noname-c con 0x7f02e0020980 addr v1:172.21.15.45:6791/0 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.510+0000 7f02fe59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f02e00100f0 0x7f0300131690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49838/0 (socket says 172.21.15.45:49838) 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.510+0000 7f02fe59c640 1 -- 172.21.15.45:0/3929168264 learned_addr learned my addr 172.21.15.45:0/3929168264 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.510+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030009d310 con 0x7f02e00100f0 2024-09-18T14:12:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.510+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03000a2560 con 0x7f02e0020980 2024-09-18T14:12:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.510+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030011d8c0 con 0x7f02e001ce80 2024-09-18T14:12:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.510+0000 7f0304d00640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.510+0000 7f0304d00640 10 monclient(hunting): authenticate will time out at 1594.848523s 2024-09-18T14:12:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830432164 0 0) 0x7f030009d310 con 0x7f02e00100f0 2024-09-18T14:12:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 10 monclient(hunting): my global_id is 5985 2024-09-18T14:12:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02e0009d80 con 0x7f02e00100f0 2024-09-18T14:12:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771101157 0 0) 0x7f030011d8c0 con 0x7f02e001ce80 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 10 monclient(hunting): my global_id is 184578 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f030009d310 con 0x7f02e001ce80 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734303995 0 0) 0x7f03000a2560 con 0x7f02e0020980 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 10 monclient(hunting): my global_id is 164758 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.514+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f030011d8c0 con 0x7f02e0020980 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2125505192 0 0) 0x7f02e00068c0 con 0x7f02e00100f0 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f03000a2560 con 0x7f02e00100f0 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f02f00046c0 con 0x7f02e00100f0 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:12:59.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:59.524 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:59.524 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.524 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.524 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.524 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:59.524 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.524 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:59.525 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2568580895 0 0) 0x7f030009d310 con 0x7f02e001ce80 2024-09-18T14:12:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f02e00068c0 con 0x7f02e001ce80 2024-09-18T14:12:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3542725402 0 0) 0x7f030011d8c0 con 0x7f02e0020980 2024-09-18T14:12:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f030009d310 con 0x7f02e0020980 2024-09-18T14:12:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f02f4001890 con 0x7f02e001ce80 2024-09-18T14:12:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:12:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:59.527 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f02f8001c40 con 0x7f02e0020980 2024-09-18T14:12:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:12:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 786283413 0 0) 0x7f03000a2560 con 0x7f02e00100f0 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 >> v1:172.21.15.45:6792/0 conn(0x7f02e001ce80 legacy=0x7f0300146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 >> v1:172.21.15.45:6791/0 conn(0x7f02e0020980 legacy=0x7f030014a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 monclient: found mon.a 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030009b950 con 0x7f02e00100f0 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f0304d00640 5 monclient: authenticate success, global_id 5985 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f0304d00640 10 monclient: _renew_subs 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f0304d00640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0300136ea0 con 0x7f02e00100f0 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f0304d00640 10 monclient: _renew_subs 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f0304d00640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0300131f40 con 0x7f02e00100f0 2024-09-18T14:12:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f02f0004bc0 con 0x7f02e00100f0 2024-09-18T14:12:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.515+0000 7f02ee7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.516+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f02f00040b0 con 0x7f02e00100f0 2024-09-18T14:12:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.516+0000 7f02fcd99640 4 set_mon_vals no callback set 2024-09-18T14:12:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.516+0000 7f02ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.516+0000 7f02ee7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.516+0000 7f02ee7fc640 10 monclient: dump: 2024-09-18T14:12:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.532 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.532 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.532 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:59.532 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.532 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:59.532 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:59.532 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:59.532 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:59.532 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:59.532 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:59.533 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.517+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f02f000ffa0 con 0x7f02e00100f0 2024-09-18T14:12:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.517+0000 7f0304d00640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.517+0000 7f0304d00640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.517+0000 7f0304d00640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.517+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030004c4a0 con 0x7f02e00100f0 2024-09-18T14:12:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.517+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(638..638 src has 424..638) v4 ==== 3927+0+0 (unknown 3928081348 0 0) 0x7f02f00468b0 con 0x7f02e00100f0 2024-09-18T14:12:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.517+0000 7f02fdd9b640 10 monclient: get_auth_request con 0x7f02e003e7b0 auth_method 0 2024-09-18T14:12:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.521+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f02f00151a0 con 0x7f02e00100f0 2024-09-18T14:12:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.521+0000 7f02ee7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.521+0000 7f02ee7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:59.570 INFO:tasks.rados.rados.0.smithi084.stdout:2882: done (1 left) 2024-09-18T14:12:59.570 INFO:tasks.rados.rados.0.smithi084.stdout:2883: done (0 left) 2024-09-18T14:12:59.570 INFO:tasks.rados.rados.0.smithi084.stdout:2884: read oid 1 snap 292 2024-09-18T14:12:59.570 INFO:tasks.rados.rados.0.smithi084.stdout:2884: expect deleted 2024-09-18T14:12:59.570 INFO:tasks.rados.rados.0.smithi084.stdout:2885: read oid 36 snap -1 2024-09-18T14:12:59.570 INFO:tasks.rados.rados.0.smithi084.stdout:2885: expect (ObjNum 972 snap 309 seq_num 972) 2024-09-18T14:12:59.570 INFO:tasks.rados.rados.0.smithi084.stdout:2886: snap_create 2024-09-18T14:12:59.580 INFO:tasks.rados.rados.0.smithi084.stdout:2885: expect (ObjNum 972 snap 309 seq_num 972) 2024-09-18T14:12:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.754+0000 7f0304d00640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:12:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.754+0000 7f0304d00640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:12:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.754+0000 7f0304d00640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.754+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f030004b720 con 0x7f02e00100f0 2024-09-18T14:12:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.754+0000 7f02ee7fc640 1 -- 172.21.15.45:0/3929168264 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2677 (unknown 254746657 0 806183530) 0x7f02f0014ba0 con 0x7f02e00100f0 2024-09-18T14:12:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.754+0000 7f02ee7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:12:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.754+0000 7f02ee7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:59.755 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":120,"quorum":[0,2,3,4,5,6,8],"quorum_names":["a","b","g","c","h","d","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:12:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.756+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 >> v1:172.21.15.84:6812/31478 conn(0x7f02e003e7b0 legacy=0x7f02e0040c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.756+0000 7f0304d00640 10 monclient: shutdown 2024-09-18T14:12:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.756+0000 7f0304d00640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.756+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 >> v1:172.21.15.45:6789/0 conn(0x7f02e00100f0 legacy=0x7f0300131690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.756+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 shutdown_connections 2024-09-18T14:12:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.756+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 >> 172.21.15.45:0/3929168264 conn(0x7f03000630b0 msgr2=0x7f03000a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.756+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 shutdown_connections 2024-09-18T14:12:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.756+0000 7f0304d00640 1 -- 172.21.15.45:0/3929168264 wait complete. 2024-09-18T14:12:59.767 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-18T14:12:59.767 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-09-18T14:12:59.768 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.864+0000 7f55646e8640 10 monclient: get_monmap_and_config 2024-09-18T14:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.864+0000 7f55646e8640 10 monclient: build_initial_monmap 2024-09-18T14:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.864+0000 7f55646e8640 10 monclient: monmap: 2024-09-18T14:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:12:59.865661+0000 2024-09-18T14:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:12:59.865661+0000 2024-09-18T14:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:59.866 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:59.866 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:59.866 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:59.866 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:59.866 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:59.866 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:59.866 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:59.866 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.865+0000 7f55646e8640 1 Processor -- start 2024-09-18T14:12:59.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.865+0000 7f55646e8640 1 -- start start 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.865+0000 7f55646e8640 10 monclient: init 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.865+0000 7f55646e8640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 10 monclient: _add_conns ranks=[0,6,3,4,5,2,8,7,1] 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 10 monclient(hunting): picked mon.noname-a con 0x7f555c09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 10 monclient(hunting): picked mon.noname-d con 0x7f555c09fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 10 monclient(hunting): picked mon.noname-g con 0x7f555c09ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c05ed70 con 0x7f555c09f7d0 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c09dfc0 con 0x7f555c09ff70 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c09dce0 con 0x7f555c09fba0 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 10 monclient(hunting): authenticate will time out at 1595.204177s 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5562484640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f555c09f7d0 0x7f555c132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49860/0 (socket says 172.21.15.45:49860) 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5562484640 1 -- 172.21.15.45:0/615350217 learned_addr learned my addr 172.21.15.45:0/615350217 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190066151 0 0) 0x7f555c05ed70 con 0x7f555c09f7d0 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient(hunting): my global_id is 5994 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55480034e0 con 0x7f555c09f7d0 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628222858 0 0) 0x7f555c09dce0 con 0x7f555c09fba0 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient(hunting): my global_id is 184587 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f555c05ed70 con 0x7f555c09fba0 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580781629 0 0) 0x7f555c09dfc0 con 0x7f555c09ff70 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:59.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient(hunting): my global_id is 184476 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f555c09dce0 con 0x7f555c09ff70 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2120016730 0 0) 0x7f55480034e0 con 0x7f555c09f7d0 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 >> v1:172.21.15.45:6792/0 conn(0x7f555c09fba0 legacy=0x7f555c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 >> v1:172.21.15.84:6790/0 conn(0x7f555c09ff70 legacy=0x7f555c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient: _finish_hunting 0 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 monclient: found mon.noname-a 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c0a1c00 con 0x7f555c09f7d0 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient: _finish_auth 0 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5558001830 con 0x7f555c09f7d0 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 5 monclient: authenticate success, global_id 5994 2024-09-18T14:12:59.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient: dump: 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:59.877 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5558001990 con 0x7f555c09f7d0 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 1 -- 172.21.15.45:0/615350217 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5558004ba0 con 0x7f555c09f7d0 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f5561482640 10 monclient: dump: 2024-09-18T14:12:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:59.879 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 10 monclient: get_monmap_and_config success 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.866+0000 7f55646e8640 4 set_mon_vals no callback set 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.867+0000 7f55646e8640 10 monclient: shutdown 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.867+0000 7f55646e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.867+0000 7f55646e8640 1 -- 172.21.15.45:0/615350217 >> v1:172.21.15.45:6789/0 conn(0x7f555c09f7d0 legacy=0x7f555c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.867+0000 7f55646e8640 1 -- 172.21.15.45:0/615350217 shutdown_connections 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.867+0000 7f55646e8640 1 -- 172.21.15.45:0/615350217 >> 172.21.15.45:0/615350217 conn(0x7f555c0630b0 msgr2=0x7f555c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.867+0000 7f55646e8640 1 -- 172.21.15.45:0/615350217 shutdown_connections 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.867+0000 7f55646e8640 1 -- 172.21.15.45:0/615350217 wait complete. 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.867+0000 7f55646e8640 10 monclient: build_initial_monmap 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 10 monclient: monmap: 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:12:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 1 Processor -- start 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 1 -- start start 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 10 monclient: init 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 10 monclient: _reopen_session rank -1 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 10 monclient: _add_conns ranks=[1,7,0,8,5,3,4,6,2] 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 10 monclient(hunting): picked mon.noname-f con 0x7f555c09f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 10 monclient(hunting): picked mon.noname-i con 0x7f555c09fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 10 monclient(hunting): picked mon.noname-a con 0x7f555c09ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c09d310 con 0x7f555c09ff70 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c0a2560 con 0x7f555c09f7d0 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c142930 con 0x7f555c09fba0 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 10 monclient(hunting): _renew_subs 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f55646e8640 10 monclient(hunting): authenticate will time out at 1595.206883s 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f5562484640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7f555c09f7d0 legacy=0x7f555c1316b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:12:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f5561c83640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7f555c09fba0 legacy=0x7f555c146d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f5562c85640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f555c09ff70 0x7f555c14a470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49862/0 (socket says 172.21.15.45:49862) 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.868+0000 7f5562c85640 1 -- 172.21.15.45:0/767003239 learned_addr learned my addr 172.21.15.45:0/767003239 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547503789 0 0) 0x7f555c09d310 con 0x7f555c09ff70 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient(hunting): my global_id is 6003 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55400034e0 con 0x7f555c09ff70 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1270672284 0 0) 0x7f55400034e0 con 0x7f555c09ff70 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f555c09d310 con 0x7f555c09ff70 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5554002e40 con 0x7f555c09ff70 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:12:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient(hunting): dump: 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2972086379 0 0) 0x7f555c09d310 con 0x7f555c09ff70 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 >> v1:172.21.15.84:6792/0 conn(0x7f555c09fba0 legacy=0x7f555c146d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 >> v1:172.21.15.84:6789/0 conn(0x7f555c09f7d0 legacy=0x7f555c1316b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 1 monclient: found mon.a 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c11d8e0 con 0x7f555c09ff70 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:12:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f55646e8640 5 monclient: authenticate success, global_id 6003 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f55646e8640 10 monclient: _renew_subs 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f55646e8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f55646e8640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555c11e7d0 con 0x7f555c09ff70 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f55646e8640 10 monclient: _renew_subs 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f55646e8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f55646e8640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c136f20 con 0x7f555c09ff70 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5554001710 con 0x7f555c09ff70 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5554001c40 con 0x7f555c09ff70 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f554e7fc640 10 monclient: dump: 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:59.886 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.869+0000 7f5560c81640 4 set_mon_vals no callback set 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.870+0000 7f55646e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.870+0000 7f55646e8640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f555c04c4a0 con 0x7f555c09acc0 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.870+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f5554001e20 con 0x7f555c09ff70 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.870+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(639..639 src has 424..639) v4 ==== 3927+0+0 (unknown 1083233178 0 0) 0x7f5554046ea0 con 0x7f555c09ff70 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.870+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910767381 0 0) 0x7f555c04c4a0 con 0x7f555c09acc0 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.870+0000 7f554e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.870+0000 7f554e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.870+0000 7f554e7fc640 10 monclient(hunting): my global_id is 6012 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.870+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f555c09d310 con 0x7f555c09acc0 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.870+0000 7f5561c83640 10 monclient: get_auth_request con 0x7f5540039b20 auth_method 0 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 108376110 0 0) 0x7f555c09d310 con 0x7f555c09acc0 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f555c04c4a0 con 0x7f555c09acc0 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5558001250 con 0x7f555c09acc0 2024-09-18T14:12:59.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 10 monclient: dump: 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:59.890 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2874593072 0 0) 0x7f555c04c4a0 con 0x7f555c09acc0 2024-09-18T14:12:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f555c04b720 con 0x7f555c09acc0 2024-09-18T14:12:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5558001cf0 con 0x7f555c09acc0 2024-09-18T14:12:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:12:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:12:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 >> v1:172.21.15.45:6789/0 conn(0x7f555c09acc0 legacy=0x7f555c096a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.871+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 >> v1:172.21.15.45:6789/0 conn(0x7f555c09acc0 legacy=0x7f555c096a20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f55646e8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:12:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f55646e8640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f555c071550 con 0x7f555c02a8d0 2024-09-18T14:12:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260126562 0 0) 0x7f555c071550 con 0x7f555c02a8d0 2024-09-18T14:12:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:12:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:12:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 10 monclient(hunting): my global_id is 6021 2024-09-18T14:12:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f555c04c4a0 con 0x7f555c02a8d0 2024-09-18T14:12:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2725220148 0 0) 0x7f555c04c4a0 con 0x7f555c02a8d0 2024-09-18T14:12:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f555c071550 con 0x7f555c02a8d0 2024-09-18T14:12:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5558005340 con 0x7f555c02a8d0 2024-09-18T14:12:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:12:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:12:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 10 monclient: dump: 2024-09-18T14:12:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:12:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:12:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:12:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:12:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:12:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2416308911 0 0) 0x7f555c071550 con 0x7f555c02a8d0 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.877+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f555c09b090 con 0x7f555c02a8d0 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7365 (unknown 0 0 2805414431) 0x7f5558007520 con 0x7f555c02a8d0 2024-09-18T14:12:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f554e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:12:59.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f554e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:12:59.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 >> v1:172.21.15.45:6789/0 conn(0x7f555c02a8d0 legacy=0x7f555c097130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f554e7fc640 1 -- 172.21.15.45:0/767003239 >> v1:172.21.15.45:6789/0 conn(0x7f555c02a8d0 legacy=0x7f555c097130 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:12:59.895 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:12:59.895 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:59.895 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:59.895 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:12:59.895 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 120, 2024-09-18T14:12:59.895 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:12:59.895 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:59.896 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:59.897 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:59.898 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:59.898 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:59.898 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:59.898 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:59.898 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:59.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.898 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:12:59.898 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:12:59.898 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:12:59.898 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:12:59.899 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.900 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:59.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:59.903 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:12:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:59.905 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.906 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:12:59.907 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:59.908 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:12:59.910 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:59.912 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:12:59.913 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:12:59.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f55646e8640 1 -- 172.21.15.45:0/767003239 >> v1:172.21.15.84:6812/31478 conn(0x7f5540039b20 legacy=0x7f554003bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f55646e8640 10 monclient: shutdown 2024-09-18T14:12:59.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f55646e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:12:59.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f55646e8640 1 -- 172.21.15.45:0/767003239 >> v1:172.21.15.45:6789/0 conn(0x7f555c09ff70 legacy=0x7f555c14a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:12:59.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f5562c85640 1 -- 172.21.15.45:0/767003239 reap_dead start 2024-09-18T14:12:59.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f5562c85640 1 -- 172.21.15.45:0/767003239 reap_dead start 2024-09-18T14:12:59.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f55646e8640 1 -- 172.21.15.45:0/767003239 shutdown_connections 2024-09-18T14:12:59.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f55646e8640 1 -- 172.21.15.45:0/767003239 >> 172.21.15.45:0/767003239 conn(0x7f555c0630b0 msgr2=0x7f555c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:12:59.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f55646e8640 1 -- 172.21.15.45:0/767003239 shutdown_connections 2024-09-18T14:12:59.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:12:59.878+0000 7f55646e8640 1 -- 172.21.15.45:0/767003239 wait complete. 2024-09-18T14:12:59.915 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:12:59.916 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-09-18T14:12:59.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:13:00.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.014+0000 7f778b99d640 10 monclient: get_monmap_and_config 2024-09-18T14:13:00.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.014+0000 7f778b99d640 10 monclient: build_initial_monmap 2024-09-18T14:13:00.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.014+0000 7f778b99d640 10 monclient: monmap: 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:13:00.015333+0000 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:13:00.015333+0000 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:13:00.016 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:13:00.016 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:13:00.016 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.014+0000 7f778b99d640 1 Processor -- start 2024-09-18T14:13:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f778b99d640 1 -- start start 2024-09-18T14:13:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f778b99d640 10 monclient: init 2024-09-18T14:13:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f778b99d640 10 monclient: _reopen_session rank -1 2024-09-18T14:13:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f778b99d640 10 monclient: _add_conns ranks=[7,8,2,4,5,6,1,3,0] 2024-09-18T14:13:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f778b99d640 10 monclient(hunting): picked mon.noname-i con 0x7f778409f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:13:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f778b99d640 10 monclient(hunting): picked mon.noname-e con 0x7f778409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:13:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f778b99d640 10 monclient(hunting): picked mon.noname-b con 0x7f778409ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f778b99d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778405ed70 con 0x7f778409ff70 2024-09-18T14:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f778b99d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778409dfc0 con 0x7f778409f7d0 2024-09-18T14:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f778b99d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778409dce0 con 0x7f778409fba0 2024-09-18T14:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f778b99d640 10 monclient(hunting): _renew_subs 2024-09-18T14:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f778b99d640 10 monclient(hunting): authenticate will time out at 1595.353701s 2024-09-18T14:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f7789739640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7f778409f7d0 legacy=0x7f7784132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f7788f38640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f778409fba0 0x7f7784137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51346/0 (socket says 172.21.15.45:51346) 2024-09-18T14:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f7788f38640 1 -- 172.21.15.45:0/2294967090 learned_addr learned my addr 172.21.15.45:0/2294967090 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.015+0000 7f7789f3a640 1 --1- 172.21.15.45:0/2294967090 >> v1:172.21.15.45:6790/0 conn(0x7f778409ff70 0x7f778413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:50634/0 (socket says 172.21.15.45:50634) 2024-09-18T14:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 1 -- 172.21.15.45:0/2294967090 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538219220 0 0) 0x7f778405ed70 con 0x7f778409ff70 2024-09-18T14:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient(hunting): my global_id is 145253 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 1 -- 172.21.15.45:0/2294967090 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77700034e0 con 0x7f778409ff70 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 1 -- 172.21.15.45:0/2294967090 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145118743 0 0) 0x7f778409dce0 con 0x7f778409fba0 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient(hunting): my global_id is 5786 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 1 -- 172.21.15.45:0/2294967090 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f778405ed70 con 0x7f778409fba0 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 1 -- 172.21.15.45:0/2294967090 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3164380830 0 0) 0x7f77700034e0 con 0x7f778409ff70 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 1 -- 172.21.15.45:0/2294967090 >> v1:172.21.15.45:6793/0 conn(0x7f778409fba0 legacy=0x7f7784137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 1 -- 172.21.15.45:0/2294967090 >> v1:172.21.15.84:6792/0 conn(0x7f778409f7d0 legacy=0x7f7784132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 1 monclient: found mon.noname-b 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 1 -- 172.21.15.45:0/2294967090 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77840a1c00 con 0x7f778409ff70 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient: _finish_auth 0 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f778b99d640 5 monclient: authenticate success, global_id 145253 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 1 -- 172.21.15.45:0/2294967090 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f777c001bb0 con 0x7f778409ff70 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient: dump: 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f778b99d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 1 -- 172.21.15.45:0/2294967090 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f777c001d10 con 0x7f778409ff70 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:13:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 1 -- 172.21.15.45:0/2294967090 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f777c004e60 con 0x7f778409ff70 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f778b99d640 10 monclient: get_monmap_and_config success 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f778b99d640 4 set_mon_vals no callback set 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f777bfff640 10 monclient: dump: 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f778b99d640 10 monclient: shutdown 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f778b99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f778b99d640 1 -- 172.21.15.45:0/2294967090 >> v1:172.21.15.45:6790/0 conn(0x7f778409ff70 legacy=0x7f778413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f778b99d640 1 -- 172.21.15.45:0/2294967090 shutdown_connections 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.016+0000 7f778b99d640 1 -- 172.21.15.45:0/2294967090 >> 172.21.15.45:0/2294967090 conn(0x7f77840630b0 msgr2=0x7f77840634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.017+0000 7f778b99d640 1 -- 172.21.15.45:0/2294967090 shutdown_connections 2024-09-18T14:13:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.017+0000 7f778b99d640 1 -- 172.21.15.45:0/2294967090 wait complete. 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.017+0000 7f778b99d640 10 monclient: build_initial_monmap 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.017+0000 7f778b99d640 10 monclient: monmap: 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:13:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.017+0000 7f778b99d640 1 Processor -- start 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.017+0000 7f778b99d640 1 -- start start 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.017+0000 7f778b99d640 10 monclient: init 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f778b99d640 10 monclient: _reopen_session rank -1 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f778b99d640 10 monclient: _add_conns ranks=[1,4,8,6,3,5,2,7,0] 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f778b99d640 10 monclient(hunting): picked mon.noname-f con 0x7f778409f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f778b99d640 10 monclient(hunting): picked mon.noname-c con 0x7f778409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f778b99d640 10 monclient(hunting): picked mon.noname-e con 0x7f778409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:13:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f778b99d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77840a2560 con 0x7f778409f7d0 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f778b99d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7784099e50 con 0x7f778409fba0 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f778b99d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778409b980 con 0x7f778409ff70 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f778b99d640 10 monclient(hunting): _renew_subs 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f778b99d640 10 monclient(hunting): authenticate will time out at 1595.356482s 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f7788f38640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f778409fba0 0x7f7784098c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:37492/0 (socket says 172.21.15.45:37492) 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f7789f3a640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f778409ff70 0x7f7784149b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51362/0 (socket says 172.21.15.45:51362) 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f7788f38640 1 -- 172.21.15.45:0/332407443 learned_addr learned my addr 172.21.15.45:0/332407443 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f7789739640 1 -- 172.21.15.45:0/332407443 >> v1:172.21.15.84:6789/0 conn(0x7f778409f7d0 legacy=0x7f7784098510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939433561 0 0) 0x7f778409b980 con 0x7f778409ff70 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f77797fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f77797fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f77797fa640 10 monclient(hunting): my global_id is 5795 2024-09-18T14:13:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7768003510 con 0x7f778409ff70 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539763570 0 0) 0x7f7784099e50 con 0x7f778409fba0 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f77797fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f77797fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f77797fa640 10 monclient(hunting): my global_id is 164767 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.018+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f778409b980 con 0x7f778409fba0 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1927691468 0 0) 0x7f7768003510 con 0x7f778409ff70 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7784099e50 con 0x7f778409ff70 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f777c001af0 con 0x7f778409ff70 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient(hunting): dump: 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.033 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4058506378 0 0) 0x7f778409b980 con 0x7f778409fba0 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7768003510 con 0x7f778409fba0 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7774001470 con 0x7f778409fba0 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient(hunting): dump: 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.035 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4189008012 0 0) 0x7f7784099e50 con 0x7f778409ff70 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 >> v1:172.21.15.45:6791/0 conn(0x7f778409fba0 legacy=0x7f7784098c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 >> v1:172.21.15.84:6789/0 conn(0x7f778409f7d0 legacy=0x7f7784098510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient: _finish_hunting 0 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 monclient: found mon.e 2024-09-18T14:13:00.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778409bc60 con 0x7f778409ff70 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient: _finish_auth 0 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f778b99d640 5 monclient: authenticate success, global_id 5795 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f778b99d640 10 monclient: _renew_subs 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f778b99d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f778b99d640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778409b670 con 0x7f778409ff70 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f778b99d640 10 monclient: _renew_subs 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f778b99d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f778b99d640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f778409c7e0 con 0x7f778409ff70 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f777c004ed0 con 0x7f778409ff70 2024-09-18T14:13:00.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f777c005380 con 0x7f778409ff70 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f777b7fe640 4 set_mon_vals no callback set 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.019+0000 7f77797fa640 10 monclient: dump: 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.038 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f778b99d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f778b99d640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7764006580 con 0x7f7764002830 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f777c005560 con 0x7f778409ff70 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(639..639 src has 424..639) v4 ==== 3927+0+0 (unknown 1083233178 0 0) 0x7f777c046f50 con 0x7f778409ff70 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466689868 0 0) 0x7f7764006580 con 0x7f7764002830 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 10 monclient(hunting): my global_id is 145262 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7784099e50 con 0x7f7764002830 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 529009616 0 0) 0x7f7784099e50 con 0x7f7764002830 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7764006580 con 0x7f7764002830 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f7788f38640 10 monclient: get_auth_request con 0x7f776803a3a0 auth_method 0 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f77800040d0 con 0x7f7764002830 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 10 monclient: dump: 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.040 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1789152481 0 0) 0x7f7764006580 con 0x7f7764002830 2024-09-18T14:13:00.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.020+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f776400a0a0 con 0x7f7764002830 2024-09-18T14:13:00.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.021+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f77800063f0 con 0x7f7764002830 2024-09-18T14:13:00.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.021+0000 7f77797fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:13:00.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.021+0000 7f77797fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:13:00.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.021+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 >> v1:172.21.15.45:6790/0 conn(0x7f7764002830 legacy=0x7f7764004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.021+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 >> v1:172.21.15.45:6790/0 conn(0x7f7764002830 legacy=0x7f7764004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:13:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f778b99d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:13:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f778b99d640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7764009660 con 0x7f7764009040 2024-09-18T14:13:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334137318 0 0) 0x7f7764009660 con 0x7f7764009040 2024-09-18T14:13:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 10 monclient(hunting): my global_id is 145271 2024-09-18T14:13:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7764006580 con 0x7f7764009040 2024-09-18T14:13:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2237623921 0 0) 0x7f7764006580 con 0x7f7764009040 2024-09-18T14:13:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7764009660 con 0x7f7764009040 2024-09-18T14:13:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7780004b90 con 0x7f7764009040 2024-09-18T14:13:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 10 monclient: dump: 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.046 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3495638881 0 0) 0x7f7764009660 con 0x7f7764009040 2024-09-18T14:13:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7764009920 con 0x7f7764009040 2024-09-18T14:13:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 2291791500) 0x7f77800060a0 con 0x7f7764009040 2024-09-18T14:13:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:13:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.027+0000 7f77797fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:13:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.028+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 >> v1:172.21.15.45:6790/0 conn(0x7f7764009040 legacy=0x7f7764008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.028+0000 7f77797fa640 1 -- 172.21.15.45:0/332407443 >> v1:172.21.15.45:6790/0 conn(0x7f7764009040 legacy=0x7f7764008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:13:00.047 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:13:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:13:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:13:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:13:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 120, 2024-09-18T14:13:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:13:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:13:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:13:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:13:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:13:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:13:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:13:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:13:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:13:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:13:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:13:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:13:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:13:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:13:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:13:00.066 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:13:00.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.028+0000 7f778b99d640 1 -- 172.21.15.45:0/332407443 >> v1:172.21.15.84:6812/31478 conn(0x7f776803a3a0 legacy=0x7f776803c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.028+0000 7f778b99d640 10 monclient: shutdown 2024-09-18T14:13:00.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.028+0000 7f778b99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:13:00.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.028+0000 7f778b99d640 1 -- 172.21.15.45:0/332407443 >> v1:172.21.15.45:6793/0 conn(0x7f778409ff70 legacy=0x7f7784149b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.028+0000 7f7789f3a640 1 -- 172.21.15.45:0/332407443 reap_dead start 2024-09-18T14:13:00.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.028+0000 7f7789f3a640 1 -- 172.21.15.45:0/332407443 reap_dead start 2024-09-18T14:13:00.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.028+0000 7f778b99d640 1 -- 172.21.15.45:0/332407443 shutdown_connections 2024-09-18T14:13:00.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.028+0000 7f778b99d640 1 -- 172.21.15.45:0/332407443 >> 172.21.15.45:0/332407443 conn(0x7f77840630b0 msgr2=0x7f7784130cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:13:00.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.028+0000 7f778b99d640 1 -- 172.21.15.45:0/332407443 shutdown_connections 2024-09-18T14:13:00.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.028+0000 7f778b99d640 1 -- 172.21.15.45:0/332407443 wait complete. 2024-09-18T14:13:00.068 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:13:00.068 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-09-18T14:13:00.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:13:00.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.165+0000 7fb41091d640 10 monclient: get_monmap_and_config 2024-09-18T14:13:00.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.165+0000 7fb41091d640 10 monclient: build_initial_monmap 2024-09-18T14:13:00.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.165+0000 7fb41091d640 10 monclient: monmap: 2024-09-18T14:13:00.165 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:13:00.166284+0000 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:13:00.166284+0000 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:13:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:13:00.167 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.165+0000 7fb41091d640 1 Processor -- start 2024-09-18T14:13:00.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.165+0000 7fb41091d640 1 -- start start 2024-09-18T14:13:00.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.165+0000 7fb41091d640 10 monclient: init 2024-09-18T14:13:00.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb41091d640 10 monclient: _reopen_session rank -1 2024-09-18T14:13:00.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb41091d640 10 monclient: _add_conns ranks=[7,6,3,2,0,4,8,1,5] 2024-09-18T14:13:00.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb41091d640 10 monclient(hunting): picked mon.noname-i con 0x7fb40c09f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:13:00.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb41091d640 10 monclient(hunting): picked mon.noname-d con 0x7fb40c09fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:13:00.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb41091d640 10 monclient(hunting): picked mon.noname-g con 0x7fb40c09ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:13:00.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb41091d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40c05ed70 con 0x7fb40c09ff70 2024-09-18T14:13:00.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb41091d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40c09dfc0 con 0x7fb40c09fba0 2024-09-18T14:13:00.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb41091d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40c09dce0 con 0x7fb40c09f7d0 2024-09-18T14:13:00.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb41091d640 10 monclient(hunting): _renew_subs 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb41091d640 10 monclient(hunting): authenticate will time out at 1595.504480s 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb401d9b640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fb40c09fba0 0x7fb40c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:35642/0 (socket says 172.21.15.45:35642) 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb401d9b640 1 -- 172.21.15.45:0/825896801 learned_addr learned my addr 172.21.15.45:0/825896801 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb40a59c640 1 -- 172.21.15.45:0/825896801 >> v1:172.21.15.84:6792/0 conn(0x7fb40c09f7d0 legacy=0x7fb40c132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb409d9b640 1 -- 172.21.15.45:0/825896801 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741247590 0 0) 0x7fb40c09dfc0 con 0x7fb40c09fba0 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb409d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb409d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb409d9b640 10 monclient(hunting): my global_id is 184596 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb409d9b640 1 -- 172.21.15.45:0/825896801 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f80034e0 con 0x7fb40c09fba0 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb409d9b640 1 -- 172.21.15.45:0/825896801 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778464344 0 0) 0x7fb40c05ed70 con 0x7fb40c09ff70 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb409d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb409d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.166+0000 7fb409d9b640 10 monclient(hunting): my global_id is 184485 2024-09-18T14:13:00.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 1 -- 172.21.15.45:0/825896801 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb40c09dfc0 con 0x7fb40c09ff70 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 1 -- 172.21.15.45:0/825896801 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 645790874 0 0) 0x7fb3f80034e0 con 0x7fb40c09fba0 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 1 -- 172.21.15.45:0/825896801 >> v1:172.21.15.84:6792/0 conn(0x7fb40c09f7d0 legacy=0x7fb40c132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 1 -- 172.21.15.45:0/825896801 >> v1:172.21.15.84:6790/0 conn(0x7fb40c09ff70 legacy=0x7fb40c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 1 monclient: found mon.noname-d 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 1 -- 172.21.15.45:0/825896801 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c0a1c00 con 0x7fb40c09fba0 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 10 monclient: _finish_auth 0 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 1 -- 172.21.15.45:0/825896801 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb3f4001830 con 0x7fb40c09fba0 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 10 monclient: dump: 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.177 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb41091d640 5 monclient: authenticate success, global_id 184596 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb41091d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 1 -- 172.21.15.45:0/825896801 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb3f4001a10 con 0x7fb40c09fba0 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 1 -- 172.21.15.45:0/825896801 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb3f4004ba0 con 0x7fb40c09fba0 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb41091d640 10 monclient: get_monmap_and_config success 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb41091d640 4 set_mon_vals no callback set 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb409d9b640 10 monclient: dump: 2024-09-18T14:13:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.179 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb41091d640 10 monclient: shutdown 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb41091d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb41091d640 1 -- 172.21.15.45:0/825896801 >> v1:172.21.15.45:6792/0 conn(0x7fb40c09fba0 legacy=0x7fb40c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb41091d640 1 -- 172.21.15.45:0/825896801 shutdown_connections 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb41091d640 1 -- 172.21.15.45:0/825896801 >> 172.21.15.45:0/825896801 conn(0x7fb40c0630b0 msgr2=0x7fb40c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb41091d640 1 -- 172.21.15.45:0/825896801 shutdown_connections 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.167+0000 7fb41091d640 1 -- 172.21.15.45:0/825896801 wait complete. 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.168+0000 7fb41091d640 10 monclient: build_initial_monmap 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.168+0000 7fb41091d640 10 monclient: monmap: 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:13:00.180 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.168+0000 7fb41091d640 1 Processor -- start 2024-09-18T14:13:00.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.168+0000 7fb41091d640 1 -- start start 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.168+0000 7fb41091d640 10 monclient: init 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb41091d640 10 monclient: _reopen_session rank -1 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb41091d640 10 monclient: _add_conns ranks=[0,4,7,3,5,2,6,8,1] 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb41091d640 10 monclient(hunting): picked mon.noname-a con 0x7fb40c09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb41091d640 10 monclient(hunting): picked mon.noname-c con 0x7fb40c09fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb41091d640 10 monclient(hunting): picked mon.noname-i con 0x7fb40c09ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb41091d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40c0a2560 con 0x7fb40c09f7d0 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb41091d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40c09d310 con 0x7fb40c09fba0 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb41091d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40c1207a0 con 0x7fb40c09ff70 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb41091d640 10 monclient(hunting): _renew_subs 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb41091d640 10 monclient(hunting): authenticate will time out at 1595.507265s 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb40a59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb40c09f7d0 0x7fb40c1417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49882/0 (socket says 172.21.15.45:49882) 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb40a59c640 1 -- 172.21.15.45:0/4001618931 learned_addr learned my addr 172.21.15.45:0/4001618931 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:13:00.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb40ad9d640 1 -- 172.21.15.45:0/4001618931 >> v1:172.21.15.84:6792/0 conn(0x7fb40c09ff70 legacy=0x7fb40c149b10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528773465 0 0) 0x7fb40c0a2560 con 0x7fb40c09f7d0 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): my global_id is 6030 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3ec0034e0 con 0x7fb40c09f7d0 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513620423 0 0) 0x7fb40c09d310 con 0x7fb40c09fba0 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): my global_id is 164776 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb40c0a2560 con 0x7fb40c09fba0 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2419901433 0 0) 0x7fb3ec0034e0 con 0x7fb40c09f7d0 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb40c09d310 con 0x7fb40c09f7d0 2024-09-18T14:13:00.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb3fc004370 con 0x7fb40c09f7d0 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): dump: 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.184 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2850405890 0 0) 0x7fb40c0a2560 con 0x7fb40c09fba0 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb3ec0034e0 con 0x7fb40c09fba0 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb3f4004ae0 con 0x7fb40c09fba0 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 10 monclient(hunting): dump: 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.169+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 441510456 0 0) 0x7fb40c09d310 con 0x7fb40c09f7d0 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 >> v1:172.21.15.84:6792/0 conn(0x7fb40c09ff70 legacy=0x7fb40c149b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 >> v1:172.21.15.45:6791/0 conn(0x7fb40c09fba0 legacy=0x7fb40c131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 1 monclient: found mon.a 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c120bf0 con 0x7fb40c09f7d0 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 10 monclient: _finish_auth 0 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb41091d640 5 monclient: authenticate success, global_id 6030 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb41091d640 10 monclient: _renew_subs 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb41091d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb41091d640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40c120020 con 0x7fb40c09f7d0 2024-09-18T14:13:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb41091d640 10 monclient: _renew_subs 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb41091d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb41091d640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40c1322b0 con 0x7fb40c09f7d0 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb3fc004550 con 0x7fb40c09f7d0 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb3fc004a80 con 0x7fb40c09f7d0 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb40959a640 4 set_mon_vals no callback set 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 10 monclient: dump: 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.170+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fb3fc004be0 con 0x7fb40c09f7d0 2024-09-18T14:13:00.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb41091d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb41091d640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb3f0006580 con 0x7fb3f0002830 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(639..639 src has 424..639) v4 ==== 3927+0+0 (unknown 1083233178 0 0) 0x7fb3fc046b10 con 0x7fb40c09f7d0 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741945913 0 0) 0x7fb3f0006580 con 0x7fb3f0002830 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 10 monclient(hunting): my global_id is 164785 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb40c09d310 con 0x7fb3f0002830 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb40ad9d640 10 monclient: get_auth_request con 0x7fb3ec03a300 auth_method 0 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 40206666 0 0) 0x7fb40c09d310 con 0x7fb3f0002830 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb3f0006580 con 0x7fb3f0002830 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb3f4005160 con 0x7fb3f0002830 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:13:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 10 monclient: dump: 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.192 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1407182622 0 0) 0x7fb3f0006580 con 0x7fb3f0002830 2024-09-18T14:13:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.171+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3f000a0a0 con 0x7fb3f0002830 2024-09-18T14:13:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.172+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb3f4005330 con 0x7fb3f0002830 2024-09-18T14:13:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.172+0000 7fb402ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:13:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.172+0000 7fb402ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:13:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.172+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 >> v1:172.21.15.45:6791/0 conn(0x7fb3f0002830 legacy=0x7fb3f0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.172+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 >> v1:172.21.15.45:6791/0 conn(0x7fb3f0002830 legacy=0x7fb3f0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:13:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.177+0000 7fb41091d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:13:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.177+0000 7fb41091d640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb3f0009640 con 0x7fb3f0009020 2024-09-18T14:13:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665601642 0 0) 0x7fb3f0009640 con 0x7fb3f0009020 2024-09-18T14:13:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 10 monclient(hunting): my global_id is 164794 2024-09-18T14:13:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f0006580 con 0x7fb3f0009020 2024-09-18T14:13:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2126958562 0 0) 0x7fb3f0006580 con 0x7fb3f0009020 2024-09-18T14:13:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb3f0009640 con 0x7fb3f0009020 2024-09-18T14:13:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb3f4005cb0 con 0x7fb3f0009020 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 10 monclient: dump: 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.196 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2155758321 0 0) 0x7fb3f0009640 con 0x7fb3f0009020 2024-09-18T14:13:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb3f0009900 con 0x7fb3f0009020 2024-09-18T14:13:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 4147839872) 0x7fb3f4008320 con 0x7fb3f0009020 2024-09-18T14:13:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:13:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:13:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 >> v1:172.21.15.45:6791/0 conn(0x7fb3f0009020 legacy=0x7fb3f0008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.178+0000 7fb402ffd640 1 -- 172.21.15.45:0/4001618931 >> v1:172.21.15.45:6791/0 conn(0x7fb3f0009020 legacy=0x7fb3f0008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 120, 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:13:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:13:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:13:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:13:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:13:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:13:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:13:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:13:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:13:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.179+0000 7fb41091d640 1 -- 172.21.15.45:0/4001618931 >> v1:172.21.15.84:6812/31478 conn(0x7fb3ec03a300 legacy=0x7fb3ec03c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.179+0000 7fb41091d640 10 monclient: shutdown 2024-09-18T14:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.179+0000 7fb41091d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.179+0000 7fb41091d640 1 -- 172.21.15.45:0/4001618931 >> v1:172.21.15.45:6789/0 conn(0x7fb40c09f7d0 legacy=0x7fb40c1417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.179+0000 7fb40ad9d640 1 -- 172.21.15.45:0/4001618931 reap_dead start 2024-09-18T14:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.179+0000 7fb40ad9d640 1 -- 172.21.15.45:0/4001618931 reap_dead start 2024-09-18T14:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.179+0000 7fb41091d640 1 -- 172.21.15.45:0/4001618931 shutdown_connections 2024-09-18T14:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.179+0000 7fb41091d640 1 -- 172.21.15.45:0/4001618931 >> 172.21.15.45:0/4001618931 conn(0x7fb40c0630b0 msgr2=0x7fb40c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.179+0000 7fb41091d640 1 -- 172.21.15.45:0/4001618931 shutdown_connections 2024-09-18T14:13:00.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.179+0000 7fb41091d640 1 -- 172.21.15.45:0/4001618931 wait complete. 2024-09-18T14:13:00.221 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:13:00.221 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-18T14:13:00.221 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:13:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.318+0000 7ff9dd15d640 10 monclient: get_monmap_and_config 2024-09-18T14:13:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.318+0000 7ff9dd15d640 10 monclient: build_initial_monmap 2024-09-18T14:13:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.318+0000 7ff9dd15d640 10 monclient: monmap: 2024-09-18T14:13:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:13:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:13:00.319672+0000 2024-09-18T14:13:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:13:00.319672+0000 2024-09-18T14:13:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:13:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:13:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:13:00.320 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:13:00.320 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:13:00.320 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:13:00.320 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:13:00.320 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:13:00.320 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:13:00.320 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:13:00.320 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 1 Processor -- start 2024-09-18T14:13:00.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 1 -- start start 2024-09-18T14:13:00.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 10 monclient: init 2024-09-18T14:13:00.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 10 monclient: _reopen_session rank -1 2024-09-18T14:13:00.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 10 monclient: _add_conns ranks=[1,3,5,8,6,2,7,4,0] 2024-09-18T14:13:00.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 10 monclient(hunting): picked mon.noname-f con 0x7ff9d809f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:13:00.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 10 monclient(hunting): picked mon.noname-g con 0x7ff9d809fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:13:00.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 10 monclient(hunting): picked mon.noname-h con 0x7ff9d809ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:13:00.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d805ed70 con 0x7ff9d809f7d0 2024-09-18T14:13:00.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d809dfc0 con 0x7ff9d809fba0 2024-09-18T14:13:00.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d809dce0 con 0x7ff9d809ff70 2024-09-18T14:13:00.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 10 monclient(hunting): _renew_subs 2024-09-18T14:13:00.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.319+0000 7ff9dd15d640 10 monclient(hunting): authenticate will time out at 1595.657948s 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d6d9d640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7ff9d809f7d0 legacy=0x7ff9d8132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d759e640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7ff9d809ff70 0x7ff9d813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:45344/0 (socket says 172.21.15.45:45344) 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d659c640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7ff9d809fba0 0x7ff9d8137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:39276/0 (socket says 172.21.15.45:39276) 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d759e640 1 -- 172.21.15.45:0/3842362258 learned_addr learned my addr 172.21.15.45:0/3842362258 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 1 -- 172.21.15.45:0/3842362258 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670926494 0 0) 0x7ff9d809dfc0 con 0x7ff9d809fba0 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 10 monclient(hunting): my global_id is 184494 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 1 -- 172.21.15.45:0/3842362258 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9c00034e0 con 0x7ff9d809fba0 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 1 -- 172.21.15.45:0/3842362258 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236921323 0 0) 0x7ff9d809dce0 con 0x7ff9d809ff70 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 10 monclient(hunting): my global_id is 184487 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 1 -- 172.21.15.45:0/3842362258 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d809dfc0 con 0x7ff9d809ff70 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 1 -- 172.21.15.45:0/3842362258 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2104004598 0 0) 0x7ff9d809dfc0 con 0x7ff9d809ff70 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 1 -- 172.21.15.45:0/3842362258 >> v1:172.21.15.84:6790/0 conn(0x7ff9d809fba0 legacy=0x7ff9d8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 1 -- 172.21.15.45:0/3842362258 >> v1:172.21.15.84:6789/0 conn(0x7ff9d809f7d0 legacy=0x7ff9d8132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 1 monclient: found mon.noname-h 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 1 -- 172.21.15.45:0/3842362258 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d80a1c00 con 0x7ff9d809ff70 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 10 monclient: _finish_auth 0 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 1 -- 172.21.15.45:0/3842362258 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff9d0001ac0 con 0x7ff9d809ff70 2024-09-18T14:13:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9dd15d640 5 monclient: authenticate success, global_id 184487 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9dd15d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9d5d9b640 10 monclient: dump: 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.333 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.320+0000 7ff9dd15d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:13:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9d5d9b640 1 -- 172.21.15.45:0/3842362258 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff9d0001c20 con 0x7ff9d809ff70 2024-09-18T14:13:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9d5d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:13:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9dd15d640 10 monclient: get_monmap_and_config success 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9d5d9b640 1 -- 172.21.15.45:0/3842362258 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff9d0004be0 con 0x7ff9d809ff70 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9dd15d640 4 set_mon_vals no callback set 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9d5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9d5d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9d5d9b640 10 monclient: dump: 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9dd15d640 10 monclient: shutdown 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9dd15d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3842362258 >> v1:172.21.15.84:6791/0 conn(0x7ff9d809ff70 legacy=0x7ff9d813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3842362258 shutdown_connections 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3842362258 >> 172.21.15.45:0/3842362258 conn(0x7ff9d80630b0 msgr2=0x7ff9d80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:13:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3842362258 shutdown_connections 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.321+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3842362258 wait complete. 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.322+0000 7ff9dd15d640 10 monclient: build_initial_monmap 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.322+0000 7ff9dd15d640 10 monclient: monmap: 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:13:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.322+0000 7ff9dd15d640 1 Processor -- start 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.322+0000 7ff9dd15d640 1 -- start start 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.322+0000 7ff9dd15d640 10 monclient: init 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9dd15d640 10 monclient: _reopen_session rank -1 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9dd15d640 10 monclient: _add_conns ranks=[1,5,6,3,7,4,8,2,0] 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9dd15d640 10 monclient(hunting): picked mon.noname-f con 0x7ff9d809f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9dd15d640 10 monclient(hunting): picked mon.noname-h con 0x7ff9d809fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9dd15d640 10 monclient(hunting): picked mon.noname-d con 0x7ff9d809ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:13:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9dd15d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d80a2560 con 0x7ff9d809f7d0 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9d6d9d640 1 -- >> v1:172.21.15.84:6789/0 conn(0x7ff9d809f7d0 legacy=0x7ff9d8098600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6789/0 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9dd15d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d809bbb0 con 0x7ff9d809fba0 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9dd15d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d809b8d0 con 0x7ff9d809ff70 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9dd15d640 10 monclient(hunting): _renew_subs 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9dd15d640 10 monclient(hunting): authenticate will time out at 1595.661199s 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9d759e640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7ff9d809ff70 0x7ff9d8149cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:35650/0 (socket says 172.21.15.45:35650) 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9d759e640 1 -- 172.21.15.45:0/3558140172 learned_addr learned my addr 172.21.15.45:0/3558140172 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363496586 0 0) 0x7ff9d809b8d0 con 0x7ff9d809ff70 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): my global_id is 184605 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9c40034e0 con 0x7ff9d809ff70 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175876579 0 0) 0x7ff9d809bbb0 con 0x7ff9d809fba0 2024-09-18T14:13:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): my global_id is 184496 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d809b8d0 con 0x7ff9d809fba0 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2376214402 0 0) 0x7ff9c40034e0 con 0x7ff9d809ff70 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9d809bbb0 con 0x7ff9d809ff70 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff9d0001a00 con 0x7ff9d809ff70 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): dump: 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4101744934 0 0) 0x7ff9d809b8d0 con 0x7ff9d809fba0 2024-09-18T14:13:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9c40034e0 con 0x7ff9d809fba0 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff9cc0015b0 con 0x7ff9d809fba0 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient(hunting): dump: 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2580293289 0 0) 0x7ff9d809bbb0 con 0x7ff9d809ff70 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 >> v1:172.21.15.84:6791/0 conn(0x7ff9d809fba0 legacy=0x7ff9d8098d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 >> v1:172.21.15.84:6789/0 conn(0x7ff9d809f7d0 legacy=0x7ff9d8098600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient: _finish_hunting 0 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 monclient: found mon.d 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:13:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d809a970 con 0x7ff9d809ff70 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 10 monclient: _finish_auth 0 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9beffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9dd15d640 5 monclient: authenticate success, global_id 184605 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.323+0000 7ff9dd15d640 10 monclient: _renew_subs 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9dd15d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d809b4b0 con 0x7ff9d809ff70 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9dd15d640 10 monclient: _renew_subs 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9dd15d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d80995c0 con 0x7ff9d809ff70 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff9d0004dd0 con 0x7ff9d809ff70 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9beffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:13:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff9d0005300 con 0x7ff9d809ff70 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9beffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9d559a640 4 set_mon_vals no callback set 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9beffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9beffd640 10 monclient: dump: 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9dd15d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff9b8006580 con 0x7ff9b8002830 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.324+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff9d00054e0 con 0x7ff9d809ff70 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(639..639 src has 424..639) v4 ==== 3927+0+0 (unknown 1083233178 0 0) 0x7ff9d0046e10 con 0x7ff9d809ff70 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700097963 0 0) 0x7ff9b8006580 con 0x7ff9b8002830 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 10 monclient(hunting): my global_id is 184614 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d809bbb0 con 0x7ff9b8002830 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9d659c640 10 monclient: get_auth_request con 0x7ff9c403a360 auth_method 0 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 971409349 0 0) 0x7ff9d809bbb0 con 0x7ff9b8002830 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9b8006580 con 0x7ff9b8002830 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff9c8004110 con 0x7ff9b8002830 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 10 monclient: dump: 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4015806291 0 0) 0x7ff9b8006580 con 0x7ff9b8002830 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9b800a0a0 con 0x7ff9b8002830 2024-09-18T14:13:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff9c8006450 con 0x7ff9b8002830 2024-09-18T14:13:00.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:13:00.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:13:00.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 >> v1:172.21.15.45:6792/0 conn(0x7ff9b8002830 legacy=0x7ff9b8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.325+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 >> v1:172.21.15.45:6792/0 conn(0x7ff9b8002830 legacy=0x7ff9b8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:13:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.331+0000 7ff9dd15d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:13:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.331+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff9b8007540 con 0x7ff9b80068f0 2024-09-18T14:13:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.331+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813623157 0 0) 0x7ff9b8007540 con 0x7ff9b80068f0 2024-09-18T14:13:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.331+0000 7ff9beffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.331+0000 7ff9beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.331+0000 7ff9beffd640 10 monclient(hunting): my global_id is 184623 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.331+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b8006580 con 0x7ff9b80068f0 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2548440945 0 0) 0x7ff9b8006580 con 0x7ff9b80068f0 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9b8007540 con 0x7ff9b80068f0 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff9c8004c10 con 0x7ff9b80068f0 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 10 monclient: dump: 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1301740940 0 0) 0x7ff9b8007540 con 0x7ff9b80068f0 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff9b80087a0 con 0x7ff9b80068f0 2024-09-18T14:13:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7365 (unknown 0 0 2622046160) 0x7ff9c80066e0 con 0x7ff9b80068f0 2024-09-18T14:13:00.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:13:00.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:13:00.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 >> v1:172.21.15.45:6792/0 conn(0x7ff9b80068f0 legacy=0x7ff9b8006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9beffd640 1 -- 172.21.15.45:0/3558140172 >> v1:172.21.15.45:6792/0 conn(0x7ff9b80068f0 legacy=0x7ff9b8006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:13:00.352 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:13:00.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:13:00.352 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:13:00.352 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:13:00.352 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 120, 2024-09-18T14:13:00.352 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:13:00.353 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.354 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:13:00.355 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.357 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.359 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.361 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.362 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:13:00.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.366 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.367 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:13:00.368 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:13:00.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.370 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:13:00.371 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:13:00.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3558140172 >> v1:172.21.15.84:6812/31478 conn(0x7ff9c403a360 legacy=0x7ff9c403c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9dd15d640 10 monclient: shutdown 2024-09-18T14:13:00.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9dd15d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:13:00.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3558140172 >> v1:172.21.15.45:6792/0 conn(0x7ff9d809ff70 legacy=0x7ff9d8149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9d759e640 1 -- 172.21.15.45:0/3558140172 reap_dead start 2024-09-18T14:13:00.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9d759e640 1 -- 172.21.15.45:0/3558140172 reap_dead start 2024-09-18T14:13:00.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3558140172 shutdown_connections 2024-09-18T14:13:00.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.332+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3558140172 >> 172.21.15.45:0/3558140172 conn(0x7ff9d80630b0 msgr2=0x7ff9d8130c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:13:00.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.333+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3558140172 shutdown_connections 2024-09-18T14:13:00.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.333+0000 7ff9dd15d640 1 -- 172.21.15.45:0/3558140172 wait complete. 2024-09-18T14:13:00.373 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:13:00.373 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-09-18T14:13:00.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:13:00.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.471+0000 7fb74499d640 10 monclient: get_monmap_and_config 2024-09-18T14:13:00.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.471+0000 7fb74499d640 10 monclient: build_initial_monmap 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.471+0000 7fb74499d640 10 monclient: monmap: 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:13:00.472296+0000 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:13:00.472296+0000 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:13:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:13:00.473 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:13:00.473 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:13:00.473 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:13:00.473 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.471+0000 7fb74499d640 1 Processor -- start 2024-09-18T14:13:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb74499d640 1 -- start start 2024-09-18T14:13:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb74499d640 10 monclient: init 2024-09-18T14:13:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb74499d640 10 monclient: _reopen_session rank -1 2024-09-18T14:13:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb74499d640 10 monclient: _add_conns ranks=[4,7,8,1,0,5,6,3,2] 2024-09-18T14:13:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb74499d640 10 monclient(hunting): picked mon.noname-c con 0x7fb74006e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:13:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb74499d640 10 monclient(hunting): picked mon.noname-i con 0x7fb7400a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:13:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb74499d640 10 monclient(hunting): picked mon.noname-e con 0x7fb7400a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:13:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb74499d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74005ed70 con 0x7fb74006e810 2024-09-18T14:13:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb74499d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb740097620 con 0x7fb7400a0cb0 2024-09-18T14:13:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb74499d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb740097340 con 0x7fb7400a1080 2024-09-18T14:13:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb74499d640 10 monclient(hunting): _renew_subs 2024-09-18T14:13:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb74499d640 10 monclient(hunting): authenticate will time out at 1595.810836s 2024-09-18T14:13:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb73e59c640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fb74006e810 0x7fb7400a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:37530/0 (socket says 172.21.15.45:37530) 2024-09-18T14:13:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb73e59c640 1 -- 172.21.15.45:0/2972609763 learned_addr learned my addr 172.21.15.45:0/2972609763 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:13:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.472+0000 7fb73dd9b640 1 -- 172.21.15.45:0/2972609763 >> v1:172.21.15.84:6792/0 conn(0x7fb7400a0cb0 legacy=0x7fb740137810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:13:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 1 -- 172.21.15.45:0/2972609763 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557599906 0 0) 0x7fb740097340 con 0x7fb7400a1080 2024-09-18T14:13:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient(hunting): my global_id is 5804 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 1 -- 172.21.15.45:0/2972609763 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7280034e0 con 0x7fb7400a1080 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 1 -- 172.21.15.45:0/2972609763 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1357976879 0 0) 0x7fb74005ed70 con 0x7fb74006e810 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient(hunting): my global_id is 164803 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 1 -- 172.21.15.45:0/2972609763 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb740097340 con 0x7fb74006e810 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 1 -- 172.21.15.45:0/2972609763 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 642484632 0 0) 0x7fb7280034e0 con 0x7fb7400a1080 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 1 -- 172.21.15.45:0/2972609763 >> v1:172.21.15.84:6792/0 conn(0x7fb7400a0cb0 legacy=0x7fb740137810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 1 -- 172.21.15.45:0/2972609763 >> v1:172.21.15.45:6791/0 conn(0x7fb74006e810 legacy=0x7fb7400a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient: _finish_hunting 0 2024-09-18T14:13:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 1 monclient: found mon.noname-e 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 1 -- 172.21.15.45:0/2972609763 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400a1660 con 0x7fb7400a1080 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient: _finish_auth 0 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb74499d640 5 monclient: authenticate success, global_id 5804 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 1 -- 172.21.15.45:0/2972609763 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb738001bb0 con 0x7fb7400a1080 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb74499d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient: dump: 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb74499d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 1 -- 172.21.15.45:0/2972609763 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb738001d90 con 0x7fb7400a1080 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 1 -- 172.21.15.45:0/2972609763 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb7380022c0 con 0x7fb7400a1080 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb74499d640 10 monclient: get_monmap_and_config success 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb74499d640 4 set_mon_vals no callback set 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb73d59a640 10 monclient: dump: 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb74499d640 10 monclient: shutdown 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb74499d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.473+0000 7fb74499d640 1 -- 172.21.15.45:0/2972609763 >> v1:172.21.15.45:6793/0 conn(0x7fb7400a1080 legacy=0x7fb74013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.474+0000 7fb74499d640 1 -- 172.21.15.45:0/2972609763 shutdown_connections 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.474+0000 7fb74499d640 1 -- 172.21.15.45:0/2972609763 >> 172.21.15.45:0/2972609763 conn(0x7fb7400a5bb0 msgr2=0x7fb7400a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.474+0000 7fb74499d640 1 -- 172.21.15.45:0/2972609763 shutdown_connections 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.474+0000 7fb74499d640 1 -- 172.21.15.45:0/2972609763 wait complete. 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.474+0000 7fb74499d640 10 monclient: build_initial_monmap 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.474+0000 7fb74499d640 10 monclient: monmap: 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:13:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.474+0000 7fb74499d640 1 Processor -- start 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.474+0000 7fb74499d640 1 -- start start 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.474+0000 7fb74499d640 10 monclient: init 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb74499d640 10 monclient: _reopen_session rank -1 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb74499d640 10 monclient: _add_conns ranks=[6,0,2,3,4,7,1,5,8] 2024-09-18T14:13:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb74499d640 10 monclient(hunting): picked mon.noname-d con 0x7fb74006e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb74499d640 10 monclient(hunting): picked mon.noname-a con 0x7fb7400a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb74499d640 10 monclient(hunting): picked mon.noname-b con 0x7fb7400a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb74499d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb740122650 con 0x7fb7400a0cb0 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb74499d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74013e760 con 0x7fb7400a1080 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb74499d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74013d9e0 con 0x7fb74006e810 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb74499d640 10 monclient(hunting): _renew_subs 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb74499d640 10 monclient(hunting): authenticate will time out at 1595.813445s 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb73e59c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fb74006e810 0x7fb74013d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:35664/0 (socket says 172.21.15.45:35664) 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb73e59c640 1 -- 172.21.15.45:0/425383077 learned_addr learned my addr 172.21.15.45:0/425383077 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197539548 0 0) 0x7fb74013d9e0 con 0x7fb74006e810 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 10 monclient(hunting): my global_id is 184632 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7200034e0 con 0x7fb74006e810 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220399981 0 0) 0x7fb74013e760 con 0x7fb7400a1080 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 10 monclient(hunting): my global_id is 145280 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb74013d9e0 con 0x7fb7400a1080 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857956512 0 0) 0x7fb740122650 con 0x7fb7400a0cb0 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 10 monclient(hunting): my global_id is 6039 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.475+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb74013e760 con 0x7fb7400a0cb0 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3217462615 0 0) 0x7fb7200034e0 con 0x7fb74006e810 2024-09-18T14:13:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb740122650 con 0x7fb74006e810 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb730001530 con 0x7fb74006e810 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient(hunting): dump: 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3763617723 0 0) 0x7fb74013e760 con 0x7fb7400a0cb0 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7200034e0 con 0x7fb7400a0cb0 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 796369732 0 0) 0x7fb74013d9e0 con 0x7fb7400a1080 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb74013e760 con 0x7fb7400a1080 2024-09-18T14:13:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb734004170 con 0x7fb7400a0cb0 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient(hunting): dump: 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb738001c20 con 0x7fb7400a1080 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient(hunting): dump: 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 446284104 0 0) 0x7fb740122650 con 0x7fb74006e810 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 >> v1:172.21.15.45:6790/0 conn(0x7fb7400a1080 legacy=0x7fb740149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 >> v1:172.21.15.45:6789/0 conn(0x7fb7400a0cb0 legacy=0x7fb740146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:13:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 monclient: found mon.d 2024-09-18T14:13:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:13:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:13:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb740122370 con 0x7fb74006e810 2024-09-18T14:13:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:13:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:13:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb74499d640 5 monclient: authenticate success, global_id 184632 2024-09-18T14:13:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb74499d640 10 monclient: _renew_subs 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb74499d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb74499d640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb740122930 con 0x7fb74006e810 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb74499d640 10 monclient: _renew_subs 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb74499d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb74499d640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb740121640 con 0x7fb74006e810 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb7300017b0 con 0x7fb74006e810 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb730004dd0 con 0x7fb74006e810 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb72e7fc640 10 monclient: dump: 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.498 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.476+0000 7fb73cd99640 4 set_mon_vals no callback set 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fb73000fe20 con 0x7fb74006e810 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb74499d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb74499d640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb74004c4a0 con 0x7fb74011ff00 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(639..639 src has 424..639) v4 ==== 3927+0+0 (unknown 1083233178 0 0) 0x7fb730046ac0 con 0x7fb74006e810 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521041216 0 0) 0x7fb74004c4a0 con 0x7fb74011ff00 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 10 monclient(hunting): my global_id is 5813 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb740122650 con 0x7fb74011ff00 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb73ed9d640 10 monclient: get_auth_request con 0x7fb72003aaa0 auth_method 0 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 827509749 0 0) 0x7fb740122650 con 0x7fb74011ff00 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb74004c4a0 con 0x7fb74011ff00 2024-09-18T14:13:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb734004940 con 0x7fb74011ff00 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.477+0000 7fb72e7fc640 10 monclient: dump: 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.502 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.478+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3405791845 0 0) 0x7fb74004c4a0 con 0x7fb74011ff00 2024-09-18T14:13:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.478+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb740120960 con 0x7fb74011ff00 2024-09-18T14:13:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.478+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb734004e80 con 0x7fb74011ff00 2024-09-18T14:13:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.478+0000 7fb72e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:13:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.478+0000 7fb72e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:13:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.478+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 >> v1:172.21.15.45:6793/0 conn(0x7fb74011ff00 legacy=0x7fb74009b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.478+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 >> v1:172.21.15.45:6793/0 conn(0x7fb74011ff00 legacy=0x7fb74009b790 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:13:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb74499d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:13:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb74499d640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb74013dff0 con 0x7fb74013dc20 2024-09-18T14:13:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152588444 0 0) 0x7fb74013dff0 con 0x7fb74013dc20 2024-09-18T14:13:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 10 monclient(hunting): my global_id is 5822 2024-09-18T14:13:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb74004c4a0 con 0x7fb74013dc20 2024-09-18T14:13:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3168591140 0 0) 0x7fb74004c4a0 con 0x7fb74013dc20 2024-09-18T14:13:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb74013dff0 con 0x7fb74013dc20 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb734005b10 con 0x7fb74013dc20 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 10 monclient: dump: 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 29688436 0 0) 0x7fb74013dff0 con 0x7fb74013dc20 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb74004b720 con 0x7fb74013dc20 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.484+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7367 (unknown 0 0 2633488331) 0x7fb734007cd0 con 0x7fb74013dc20 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb72e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb72e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 >> v1:172.21.15.45:6793/0 conn(0x7fb74013dc20 legacy=0x7fb74009bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb72e7fc640 1 -- 172.21.15.45:0/425383077 >> v1:172.21.15.45:6793/0 conn(0x7fb74013dc20 legacy=0x7fb74009bea0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:13:00.507 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:13:00.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:13:00.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:13:00.507 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:13:00.507 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 120, 2024-09-18T14:13:00.507 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:13:00.507 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:13:00.507 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:13:00.507 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:13:00.507 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:13:00.507 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.509 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:13:00.510 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.511 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:13:00.514 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.516 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:13:00.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:13:00.518 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.518 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.518 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:13:00.518 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:13:00.521 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.522 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.525 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:13:00.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.526 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.526 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:13:00.526 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:13:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb74499d640 1 -- 172.21.15.45:0/425383077 >> v1:172.21.15.84:6812/31478 conn(0x7fb72003aaa0 legacy=0x7fb72003cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb74499d640 10 monclient: shutdown 2024-09-18T14:13:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb74499d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:13:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb74499d640 1 -- 172.21.15.45:0/425383077 >> v1:172.21.15.45:6792/0 conn(0x7fb74006e810 legacy=0x7fb74013d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb73ed9d640 1 -- 172.21.15.45:0/425383077 reap_dead start 2024-09-18T14:13:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb73ed9d640 1 -- 172.21.15.45:0/425383077 reap_dead start 2024-09-18T14:13:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb74499d640 1 -- 172.21.15.45:0/425383077 shutdown_connections 2024-09-18T14:13:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb74499d640 1 -- 172.21.15.45:0/425383077 >> 172.21.15.45:0/425383077 conn(0x7fb7400a5bb0 msgr2=0x7fb7400a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:13:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb74499d640 1 -- 172.21.15.45:0/425383077 shutdown_connections 2024-09-18T14:13:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.485+0000 7fb74499d640 1 -- 172.21.15.45:0/425383077 wait complete. 2024-09-18T14:13:00.528 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:13:00.528 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-09-18T14:13:00.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:13:00.574 INFO:tasks.rados.rados.0.smithi084.stdout:2884: done (2 left) 2024-09-18T14:13:00.575 INFO:tasks.rados.rados.0.smithi084.stdout:2885: done (1 left) 2024-09-18T14:13:00.575 INFO:tasks.rados.rados.0.smithi084.stdout:2886: done (0 left) 2024-09-18T14:13:00.575 INFO:tasks.rados.rados.0.smithi084.stdout:2887: read oid 50 snap -1 2024-09-18T14:13:00.575 INFO:tasks.rados.rados.0.smithi084.stdout:2887: expect deleted 2024-09-18T14:13:00.575 INFO:tasks.rados.rados.0.smithi084.stdout:2888: copy_from oid 12 from oid 10 current snap is 315 2024-09-18T14:13:00.575 INFO:tasks.rados.rados.0.smithi084.stdout:2889: copy_from oid 15 from oid 9 current snap is 315 2024-09-18T14:13:00.575 INFO:tasks.rados.rados.0.smithi084.stdout:2890: snap_create 2024-09-18T14:13:00.584 INFO:tasks.rados.rados.0.smithi084.stdout:2889: finishing copy_from racing read to smithi08434663-15 2024-09-18T14:13:00.584 INFO:tasks.rados.rados.0.smithi084.stdout:2889: finishing copy_from to smithi08434663-15 2024-09-18T14:13:00.584 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 1265 (ObjNum 957 snap 304 seq_num 957) dirty exists 2024-09-18T14:13:00.589 INFO:tasks.rados.rados.0.smithi084.stdout:2888: finishing copy_from racing read to smithi08434663-12 2024-09-18T14:13:00.593 INFO:tasks.rados.rados.0.smithi084.stdout:2888: finishing copy_from to smithi08434663-12 2024-09-18T14:13:00.593 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 1219 (ObjNum 904 snap 281 seq_num 904) dirty exists 2024-09-18T14:13:00.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.630+0000 7faad6914640 10 monclient: get_monmap_and_config 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.630+0000 7faad6914640 10 monclient: build_initial_monmap 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.630+0000 7faad6914640 10 monclient: monmap: 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:13:00.631441+0000 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:13:00.631441+0000 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:13:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:13:00.632 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:13:00.632 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:13:00.632 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:13:00.632 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:13:00.632 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.631+0000 7faad6914640 1 Processor -- start 2024-09-18T14:13:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.631+0000 7faad6914640 1 -- start start 2024-09-18T14:13:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.631+0000 7faad6914640 10 monclient: init 2024-09-18T14:13:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.631+0000 7faad6914640 10 monclient: _reopen_session rank -1 2024-09-18T14:13:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.631+0000 7faad6914640 10 monclient: _add_conns ranks=[0,7,3,8,4,1,5,2,6] 2024-09-18T14:13:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.631+0000 7faad6914640 10 monclient(hunting): picked mon.noname-a con 0x7faad009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:13:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.631+0000 7faad6914640 10 monclient(hunting): picked mon.noname-i con 0x7faad009fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:13:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.631+0000 7faad6914640 10 monclient(hunting): picked mon.noname-g con 0x7faad009ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:13:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.631+0000 7faad6914640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad005ed70 con 0x7faad009f7d0 2024-09-18T14:13:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.631+0000 7faad6914640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad009dfc0 con 0x7faad009ff70 2024-09-18T14:13:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.631+0000 7faad6914640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad009dce0 con 0x7faad009fba0 2024-09-18T14:13:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faad6914640 10 monclient(hunting): _renew_subs 2024-09-18T14:13:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faad6914640 10 monclient(hunting): authenticate will time out at 1595.970016s 2024-09-18T14:13:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faacffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7faad009f7d0 0x7faad0132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49910/0 (socket says 172.21.15.45:49910) 2024-09-18T14:13:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faacffff640 1 -- 172.21.15.45:0/2189878068 learned_addr learned my addr 172.21.15.45:0/2189878068 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:13:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faacf7fe640 1 -- 172.21.15.45:0/2189878068 >> v1:172.21.15.84:6792/0 conn(0x7faad009fba0 legacy=0x7faad0137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:13:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 1 -- 172.21.15.45:0/2189878068 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396182503 0 0) 0x7faad005ed70 con 0x7faad009f7d0 2024-09-18T14:13:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient(hunting): my global_id is 6048 2024-09-18T14:13:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 1 -- 172.21.15.45:0/2189878068 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faabc0034e0 con 0x7faad009f7d0 2024-09-18T14:13:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 1 -- 172.21.15.45:0/2189878068 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848631555 0 0) 0x7faad009dfc0 con 0x7faad009ff70 2024-09-18T14:13:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient(hunting): my global_id is 184503 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 1 -- 172.21.15.45:0/2189878068 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faad005ed70 con 0x7faad009ff70 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 1 -- 172.21.15.45:0/2189878068 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3311309870 0 0) 0x7faabc0034e0 con 0x7faad009f7d0 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 1 -- 172.21.15.45:0/2189878068 >> v1:172.21.15.84:6792/0 conn(0x7faad009fba0 legacy=0x7faad0137a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 1 -- 172.21.15.45:0/2189878068 >> v1:172.21.15.84:6790/0 conn(0x7faad009ff70 legacy=0x7faad013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient: _finish_hunting 0 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 1 monclient: found mon.noname-a 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 1 -- 172.21.15.45:0/2189878068 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad00a1c00 con 0x7faad009f7d0 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient: _finish_auth 0 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:13:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faad6914640 5 monclient: authenticate success, global_id 6048 2024-09-18T14:13:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faad6914640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:13:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 1 -- 172.21.15.45:0/2189878068 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faac0001830 con 0x7faad009f7d0 2024-09-18T14:13:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:13:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient: dump: 2024-09-18T14:13:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faad6914640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 1 -- 172.21.15.45:0/2189878068 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faac0001a10 con 0x7faad009f7d0 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 1 -- 172.21.15.45:0/2189878068 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faac0004ba0 con 0x7faad009f7d0 2024-09-18T14:13:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faad6914640 10 monclient: get_monmap_and_config success 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faad6914640 4 set_mon_vals no callback set 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.632+0000 7faaceffd640 10 monclient: dump: 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.633+0000 7faad6914640 10 monclient: shutdown 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.633+0000 7faad6914640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.633+0000 7faad6914640 1 -- 172.21.15.45:0/2189878068 >> v1:172.21.15.45:6789/0 conn(0x7faad009f7d0 legacy=0x7faad0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.633+0000 7faad6914640 1 -- 172.21.15.45:0/2189878068 shutdown_connections 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.633+0000 7faad6914640 1 -- 172.21.15.45:0/2189878068 >> 172.21.15.45:0/2189878068 conn(0x7faad00630b0 msgr2=0x7faad00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.633+0000 7faad6914640 1 -- 172.21.15.45:0/2189878068 shutdown_connections 2024-09-18T14:13:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.633+0000 7faad6914640 1 -- 172.21.15.45:0/2189878068 wait complete. 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.633+0000 7faad6914640 10 monclient: build_initial_monmap 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.633+0000 7faad6914640 10 monclient: monmap: 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:13:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 1 Processor -- start 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 1 -- start start 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 10 monclient: init 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 10 monclient: _reopen_session rank -1 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 10 monclient: _add_conns ranks=[2,3,4,7,0,5,1,8,6] 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 10 monclient(hunting): picked mon.noname-b con 0x7faad009f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 10 monclient(hunting): picked mon.noname-g con 0x7faad009fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:13:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 10 monclient(hunting): picked mon.noname-c con 0x7faad009ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad009bd10 con 0x7faad009f7d0 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad011d2b0 con 0x7faad009fba0 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad009d310 con 0x7faad009ff70 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 10 monclient(hunting): _renew_subs 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faad6914640 10 monclient(hunting): authenticate will time out at 1595.972811s 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faacffff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7faad009f7d0 0x7faad011c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:50670/0 (socket says 172.21.15.45:50670) 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.634+0000 7faacffff640 1 -- 172.21.15.45:0/1251819031 learned_addr learned my addr 172.21.15.45:0/1251819031 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464398991 0 0) 0x7faad009bd10 con 0x7faad009f7d0 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): my global_id is 145289 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faab00034e0 con 0x7faad009f7d0 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278262193 0 0) 0x7faad009d310 con 0x7faad009ff70 2024-09-18T14:13:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): my global_id is 164812 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faad009bd10 con 0x7faad009ff70 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201954526 0 0) 0x7faad011d2b0 con 0x7faad009fba0 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): my global_id is 184512 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faad009d310 con 0x7faad009fba0 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 885617018 0 0) 0x7faad009bd10 con 0x7faad009ff70 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faad011d2b0 con 0x7faad009ff70 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faac8001d80 con 0x7faad009ff70 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): dump: 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3817523303 0 0) 0x7faab00034e0 con 0x7faad009f7d0 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faad009bd10 con 0x7faad009f7d0 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faac0001830 con 0x7faad009f7d0 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): dump: 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1653864853 0 0) 0x7faad009d310 con 0x7faad009fba0 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faab00034e0 con 0x7faad009fba0 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faab8004230 con 0x7faad009fba0 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient(hunting): dump: 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 527132246 0 0) 0x7faad011d2b0 con 0x7faad009ff70 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 >> v1:172.21.15.84:6790/0 conn(0x7faad009fba0 legacy=0x7faad011cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 >> v1:172.21.15.45:6790/0 conn(0x7faad009f7d0 legacy=0x7faad011c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient: _finish_hunting 0 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 monclient: found mon.c 2024-09-18T14:13:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad00a2560 con 0x7faad009ff70 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient: _finish_auth 0 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faad6914640 5 monclient: authenticate success, global_id 164812 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faad6914640 10 monclient: _renew_subs 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faad6914640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faad6914640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad011e620 con 0x7faad009ff70 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faad6914640 10 monclient: _renew_subs 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faad6914640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faad6914640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faad011e2c0 con 0x7faad009ff70 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faac8002ff0 con 0x7faad009ff70 2024-09-18T14:13:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faac8002650 con 0x7faad009ff70 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faace7fc640 4 set_mon_vals no callback set 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.635+0000 7faac7fff640 10 monclient: dump: 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.636+0000 7faad6914640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.636+0000 7faad6914640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7faab4006580 con 0x7faab4002830 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.636+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7faac8002830 con 0x7faad009ff70 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(640..640 src has 424..640) v4 ==== 3927+0+0 (unknown 2531669223 0 0) 0x7faac80445f0 con 0x7faad009ff70 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211163703 0 0) 0x7faab4006580 con 0x7faab4002830 2024-09-18T14:13:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 10 monclient(hunting): my global_id is 184521 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faad011d2b0 con 0x7faab4002830 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faacf7fe640 10 monclient: get_auth_request con 0x7faab003aa90 auth_method 0 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1273854741 0 0) 0x7faad011d2b0 con 0x7faab4002830 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faab4006580 con 0x7faab4002830 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faac0001210 con 0x7faab4002830 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 10 monclient: dump: 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.661 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 417462467 0 0) 0x7faab4006580 con 0x7faab4002830 2024-09-18T14:13:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faab400a0a0 con 0x7faab4002830 2024-09-18T14:13:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7faac0005210 con 0x7faab4002830 2024-09-18T14:13:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:13:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:13:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 >> v1:172.21.15.84:6790/0 conn(0x7faab4002830 legacy=0x7faab4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.637+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 >> v1:172.21.15.84:6790/0 conn(0x7faab4002830 legacy=0x7faab4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:13:00.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.643+0000 7faad6914640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:13:00.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.643+0000 7faad6914640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7faab40074d0 con 0x7faab40068a0 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966662841 0 0) 0x7faab40074d0 con 0x7faab40068a0 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 10 monclient(hunting): my global_id is 184530 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faab4006580 con 0x7faab40068a0 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3191048199 0 0) 0x7faab4006580 con 0x7faab40068a0 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faab40074d0 con 0x7faab40068a0 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7faac00059d0 con 0x7faab40068a0 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 10 monclient: dump: 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 372362117 0 0) 0x7faab40074d0 con 0x7faab40068a0 2024-09-18T14:13:00.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7faab4008750 con 0x7faab40068a0 2024-09-18T14:13:00.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 2221408119) 0x7faac0008200 con 0x7faab40068a0 2024-09-18T14:13:00.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:13:00.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:13:00.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.644+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 >> v1:172.21.15.84:6790/0 conn(0x7faab40068a0 legacy=0x7faab4006d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.645+0000 7faac7fff640 1 -- 172.21.15.45:0/1251819031 >> v1:172.21.15.84:6790/0 conn(0x7faab40068a0 legacy=0x7faab4006d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:13:00.666 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:13:00.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:13:00.666 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:13:00.666 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 120, 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:13:00.667 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:13:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:13:00.669 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:13:00.670 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:13:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:13:00.674 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:13:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:13:00.680 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:13:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.685 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.685 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:13:00.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.685 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.685 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.685 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:13:00.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.685 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.685 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:13:00.685 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:13:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.645+0000 7faad6914640 1 -- 172.21.15.45:0/1251819031 >> v1:172.21.15.84:6812/31478 conn(0x7faab003aa90 legacy=0x7faab003cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.645+0000 7faad6914640 10 monclient: shutdown 2024-09-18T14:13:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.645+0000 7faad6914640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:13:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.645+0000 7faad6914640 1 -- 172.21.15.45:0/1251819031 >> v1:172.21.15.45:6791/0 conn(0x7faad009ff70 legacy=0x7faad0149ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.645+0000 7faad4eb1640 1 -- 172.21.15.45:0/1251819031 reap_dead start 2024-09-18T14:13:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.645+0000 7faad4eb1640 1 -- 172.21.15.45:0/1251819031 reap_dead start 2024-09-18T14:13:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.645+0000 7faad6914640 1 -- 172.21.15.45:0/1251819031 shutdown_connections 2024-09-18T14:13:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.645+0000 7faad6914640 1 -- 172.21.15.45:0/1251819031 >> 172.21.15.45:0/1251819031 conn(0x7faad00630b0 msgr2=0x7faad00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:13:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.645+0000 7faad6914640 1 -- 172.21.15.45:0/1251819031 shutdown_connections 2024-09-18T14:13:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.645+0000 7faad6914640 1 -- 172.21.15.45:0/1251819031 wait complete. 2024-09-18T14:13:00.688 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:13:00.688 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-18T14:13:00.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:13:00.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.785+0000 7f63bc689640 10 monclient: get_monmap_and_config 2024-09-18T14:13:00.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.785+0000 7f63bc689640 10 monclient: build_initial_monmap 2024-09-18T14:13:00.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.785+0000 7f63bc689640 10 monclient: monmap: 2024-09-18T14:13:00.785 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:13:00.785 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:13:00.786115+0000 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:13:00.786115+0000 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:13:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:13:00.787 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.785+0000 7f63bc689640 1 Processor -- start 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.785+0000 7f63bc689640 1 -- start start 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.785+0000 7f63bc689640 10 monclient: init 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63bc689640 10 monclient: _reopen_session rank -1 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63bc689640 10 monclient: _add_conns ranks=[8,5,0,1,2,7,3,6,4] 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63bc689640 10 monclient(hunting): picked mon.noname-e con 0x7f63b406e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63bc689640 10 monclient(hunting): picked mon.noname-h con 0x7f63b40a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63bc689640 10 monclient(hunting): picked mon.noname-a con 0x7f63b40a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63bc689640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b405ed70 con 0x7f63b40a1080 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63bc689640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b4097620 con 0x7f63b40a0cb0 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63bc689640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b4097340 con 0x7f63b406e810 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63bc689640 10 monclient(hunting): _renew_subs 2024-09-18T14:13:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63bc689640 10 monclient(hunting): authenticate will time out at 1596.124482s 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63ba425640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f63b406e810 0x7f63b40a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:51398/0 (socket says 172.21.15.45:51398) 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63bac26640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f63b40a1080 0x7f63b413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49912/0 (socket says 172.21.15.45:49912) 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63ba425640 1 -- 172.21.15.45:0/1833325754 learned_addr learned my addr 172.21.15.45:0/1833325754 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63b9423640 1 -- 172.21.15.45:0/1833325754 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458028780 0 0) 0x7f63b405ed70 con 0x7f63b40a1080 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63b9423640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63b9423640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63b9423640 10 monclient(hunting): my global_id is 6057 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63b9423640 1 -- 172.21.15.45:0/1833325754 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63a00034e0 con 0x7f63b40a1080 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63b9423640 1 -- 172.21.15.45:0/1833325754 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984672920 0 0) 0x7f63b4097340 con 0x7f63b406e810 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63b9423640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63b9423640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63b9423640 10 monclient(hunting): my global_id is 5831 2024-09-18T14:13:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.786+0000 7f63b9423640 1 -- 172.21.15.45:0/1833325754 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63b405ed70 con 0x7f63b406e810 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 1 -- 172.21.15.45:0/1833325754 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4191495730 0 0) 0x7f63b405ed70 con 0x7f63b406e810 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 1 -- 172.21.15.45:0/1833325754 >> v1:172.21.15.84:6791/0 conn(0x7f63b40a0cb0 legacy=0x7f63b4137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 1 -- 172.21.15.45:0/1833325754 >> v1:172.21.15.45:6789/0 conn(0x7f63b40a1080 legacy=0x7f63b413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 10 monclient: _finish_hunting 0 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 1 monclient: found mon.noname-e 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 1 -- 172.21.15.45:0/1833325754 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b40a1660 con 0x7f63b406e810 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 10 monclient: _finish_auth 0 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63bc689640 5 monclient: authenticate success, global_id 5831 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 1 -- 172.21.15.45:0/1833325754 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f63b0001810 con 0x7f63b406e810 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:13:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 10 monclient: dump: 2024-09-18T14:13:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63bc689640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 1 -- 172.21.15.45:0/1833325754 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f63b0001970 con 0x7f63b406e810 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 1 -- 172.21.15.45:0/1833325754 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f63b0004860 con 0x7f63b406e810 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63b9423640 10 monclient: dump: 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63bc689640 10 monclient: get_monmap_and_config success 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63bc689640 4 set_mon_vals no callback set 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63bc689640 10 monclient: shutdown 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63bc689640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63bc689640 1 -- 172.21.15.45:0/1833325754 >> v1:172.21.15.45:6793/0 conn(0x7f63b406e810 legacy=0x7f63b40a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63bc689640 1 -- 172.21.15.45:0/1833325754 shutdown_connections 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63bc689640 1 -- 172.21.15.45:0/1833325754 >> 172.21.15.45:0/1833325754 conn(0x7f63b40a5bb0 msgr2=0x7f63b40a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63bc689640 1 -- 172.21.15.45:0/1833325754 shutdown_connections 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.787+0000 7f63bc689640 1 -- 172.21.15.45:0/1833325754 wait complete. 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.788+0000 7f63bc689640 10 monclient: build_initial_monmap 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.788+0000 7f63bc689640 10 monclient: monmap: 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:13:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.788+0000 7f63bc689640 1 Processor -- start 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.788+0000 7f63bc689640 1 -- start start 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.788+0000 7f63bc689640 10 monclient: init 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63bc689640 10 monclient: _reopen_session rank -1 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63bc689640 10 monclient: _add_conns ranks=[0,7,5,2,6,8,4,3,1] 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63bc689640 10 monclient(hunting): picked mon.noname-a con 0x7f63b406e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63bc689640 10 monclient(hunting): picked mon.noname-i con 0x7f63b40a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63bc689640 10 monclient(hunting): picked mon.noname-h con 0x7f63b40a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63bc689640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b409fa10 con 0x7f63b406e810 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63bc689640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b413d9f0 con 0x7f63b40a1080 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63bc689640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b40a2270 con 0x7f63b40a0cb0 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63bc689640 10 monclient(hunting): _renew_subs 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63bc689640 10 monclient(hunting): authenticate will time out at 1596.127378s 2024-09-18T14:13:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63b9c24640 1 -- >> v1:172.21.15.84:6792/0 conn(0x7f63b40a0cb0 legacy=0x7f63b41464f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6792/0 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63ba425640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f63b406e810 0x7f63b413d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49926/0 (socket says 172.21.15.45:49926) 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63ba425640 1 -- 172.21.15.45:0/1948819458 learned_addr learned my addr 172.21.15.45:0/1948819458 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570498722 0 0) 0x7f63b409fa10 con 0x7f63b406e810 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): my global_id is 6066 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63980034e0 con 0x7f63b406e810 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150266655 0 0) 0x7f63b413d9f0 con 0x7f63b40a1080 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): my global_id is 184514 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63b409fa10 con 0x7f63b40a1080 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2299436713 0 0) 0x7f63980034e0 con 0x7f63b406e810 2024-09-18T14:13:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63b413d9f0 con 0x7f63b406e810 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f63b00017d0 con 0x7f63b406e810 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): dump: 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2421952341 0 0) 0x7f63b409fa10 con 0x7f63b40a1080 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63980034e0 con 0x7f63b40a1080 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f63ac001da0 con 0x7f63b40a1080 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:13:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.789+0000 7f63aa7fc640 10 monclient(hunting): dump: 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1984268693 0 0) 0x7f63b413d9f0 con 0x7f63b406e810 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 >> v1:172.21.15.84:6792/0 conn(0x7f63b40a0cb0 legacy=0x7f63b41464f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 >> v1:172.21.15.84:6791/0 conn(0x7f63b40a1080 legacy=0x7f63b4149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 1 monclient: found mon.a 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4122380 con 0x7f63b406e810 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 10 monclient: _finish_auth 0 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63bc689640 5 monclient: authenticate success, global_id 6066 2024-09-18T14:13:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63bc689640 10 monclient: _renew_subs 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63bc689640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63bc689640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b4122660 con 0x7f63b406e810 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63bc689640 10 monclient: _renew_subs 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63bc689640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63bc689640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b413e6d0 con 0x7f63b406e810 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f63b0004d90 con 0x7f63b406e810 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f63b0005240 con 0x7f63b406e810 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63b8c22640 4 set_mon_vals no callback set 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63aa7fc640 10 monclient: dump: 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.812 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63bc689640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.790+0000 7f63bc689640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f639c006580 con 0x7f639c002830 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f63b0010210 con 0x7f63b406e810 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(640..640 src has 424..640) v4 ==== 3927+0+0 (unknown 2531669223 0 0) 0x7f63b0046de0 con 0x7f63b406e810 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339038345 0 0) 0x7f639c006580 con 0x7f639c002830 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 10 monclient(hunting): my global_id is 184523 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63b413d9f0 con 0x7f639c002830 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63bac26640 10 monclient: get_auth_request con 0x7f639803a3b0 auth_method 0 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1504003729 0 0) 0x7f63b413d9f0 con 0x7f639c002830 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f639c006580 con 0x7f639c002830 2024-09-18T14:13:00.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f63a4001920 con 0x7f639c002830 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 10 monclient: dump: 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.815 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1521693728 0 0) 0x7f639c006580 con 0x7f639c002830 2024-09-18T14:13:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.791+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f639c00a0a0 con 0x7f639c002830 2024-09-18T14:13:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.792+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f63a4005780 con 0x7f639c002830 2024-09-18T14:13:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.792+0000 7f63aa7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:13:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.792+0000 7f63aa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:13:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.792+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 >> v1:172.21.15.84:6791/0 conn(0x7f639c002830 legacy=0x7f639c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.792+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 >> v1:172.21.15.84:6791/0 conn(0x7f639c002830 legacy=0x7f639c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:13:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.797+0000 7f63bc689640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:13:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.797+0000 7f63bc689640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f639c009660 con 0x7f639c009040 2024-09-18T14:13:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439924680 0 0) 0x7f639c009660 con 0x7f639c009040 2024-09-18T14:13:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:13:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:13:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 10 monclient(hunting): my global_id is 184532 2024-09-18T14:13:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f639c006580 con 0x7f639c009040 2024-09-18T14:13:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2057257558 0 0) 0x7f639c006580 con 0x7f639c009040 2024-09-18T14:13:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f639c009660 con 0x7f639c009040 2024-09-18T14:13:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f63a4002030 con 0x7f639c009040 2024-09-18T14:13:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 10 monclient: dump: 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:13:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:13:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:13:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:13:00.819 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:13:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3660447527 0 0) 0x7f639c009660 con 0x7f639c009040 2024-09-18T14:13:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.798+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f639c009920 con 0x7f639c009040 2024-09-18T14:13:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 1005609583) 0x7f63a40059b0 con 0x7f639c009040 2024-09-18T14:13:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63aa7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:13:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63aa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:13:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 >> v1:172.21.15.84:6791/0 conn(0x7f639c009040 legacy=0x7f639c008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63aa7fc640 1 -- 172.21.15.45:0/1948819458 >> v1:172.21.15.84:6791/0 conn(0x7f639c009040 legacy=0x7f639c008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:13:00.819 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:13:00.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:13:00.820 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:13:00.820 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:13:00.820 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 120, 2024-09-18T14:13:00.820 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:13:00.820 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:13:00.820 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:13:00.820 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:13:00.820 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:13:00.820 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:13:00.820 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:13:00.820 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:13:00.821 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.821 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:13:00.821 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:13:00.821 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:13:00.821 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:13:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:13:00.823 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:13:00.824 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.825 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:13:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:13:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:13:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:13:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:13:00.829 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.830 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.832 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:13:00.832 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.832 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.832 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.832 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:13:00.832 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:13:00.832 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.832 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.832 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.833 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:13:00.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:13:00.833 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.833 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.833 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:13:00.833 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.833 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.834 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:13:00.834 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:13:00.834 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.834 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.834 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.834 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:13:00.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:13:00.834 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.834 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.835 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:13:00.836 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:13:00.836 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.836 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.836 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:13:00.836 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.836 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.836 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:13:00.836 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:13:00.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.838 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:13:00.839 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:13:00.840 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:13:00.840 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:13:00.840 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:13:00.840 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:13:00.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:13:00.840 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:13:00.840 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:13:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63bc689640 1 -- 172.21.15.45:0/1948819458 >> v1:172.21.15.84:6812/31478 conn(0x7f639803a3b0 legacy=0x7f639803c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63bc689640 10 monclient: shutdown 2024-09-18T14:13:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63bc689640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:13:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63bc689640 1 -- 172.21.15.45:0/1948819458 >> v1:172.21.15.45:6789/0 conn(0x7f63b406e810 legacy=0x7f63b413d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:13:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63bac26640 1 -- 172.21.15.45:0/1948819458 reap_dead start 2024-09-18T14:13:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63bac26640 1 -- 172.21.15.45:0/1948819458 reap_dead start 2024-09-18T14:13:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63bc689640 1 -- 172.21.15.45:0/1948819458 shutdown_connections 2024-09-18T14:13:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63bc689640 1 -- 172.21.15.45:0/1948819458 >> 172.21.15.45:0/1948819458 conn(0x7f63b40a5bb0 msgr2=0x7f63b40a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:13:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63bc689640 1 -- 172.21.15.45:0/1948819458 shutdown_connections 2024-09-18T14:13:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:13:00.799+0000 7f63bc689640 1 -- 172.21.15.45:0/1948819458 wait complete. 2024-09-18T14:13:00.843 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:13:00.843 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-18T14:13:01.575 INFO:tasks.rados.rados.0.smithi084.stdout:2887: done (3 left) 2024-09-18T14:13:01.575 INFO:tasks.rados.rados.0.smithi084.stdout:2888: done (2 left) 2024-09-18T14:13:01.575 INFO:tasks.rados.rados.0.smithi084.stdout:2889: done (1 left) 2024-09-18T14:13:01.575 INFO:tasks.rados.rados.0.smithi084.stdout:2890: done (0 left) 2024-09-18T14:13:01.575 INFO:tasks.rados.rados.0.smithi084.stdout:2891: snap_remove snap 314 2024-09-18T14:13:02.595 INFO:tasks.rados.rados.0.smithi084.stdout:2891: done (0 left) 2024-09-18T14:13:02.595 INFO:tasks.rados.rados.0.smithi084.stdout:2892: write (excl) oid 34 current snap is 316 2024-09-18T14:13:02.595 INFO:tasks.rados.rados.0.smithi084.stdout:2892: seq_num 983 ranges {521281=579304,1656270=774664} 2024-09-18T14:13:02.601 INFO:tasks.rados.rados.0.smithi084.stdout:2892: writing smithi08434663-34 from 521281 to 1100585 tid 1 2024-09-18T14:13:02.608 INFO:tasks.rados.rados.0.smithi084.stdout:2892: writing smithi08434663-34 from 1656270 to 2430934 tid 2 2024-09-18T14:13:02.608 INFO:tasks.rados.rados.0.smithi084.stdout:2893: read oid 46 snap 301 2024-09-18T14:13:02.608 INFO:tasks.rados.rados.0.smithi084.stdout:2893: expect deleted 2024-09-18T14:13:02.608 INFO:tasks.rados.rados.0.smithi084.stdout:2894: delete oid 40 current snap is 316 2024-09-18T14:13:02.608 INFO:tasks.rados.rados.0.smithi084.stdout:2892: finishing write tid 1 to smithi08434663-34 2024-09-18T14:13:02.613 INFO:tasks.rados.rados.0.smithi084.stdout:2892: finishing write tid 2 to smithi08434663-34 2024-09-18T14:13:02.613 INFO:tasks.rados.rados.0.smithi084.stdout:2892: finishing write tid 3 to smithi08434663-34 2024-09-18T14:13:02.613 INFO:tasks.rados.rados.0.smithi084.stdout:2893: done (2 left) 2024-09-18T14:13:02.614 INFO:tasks.rados.rados.0.smithi084.stdout:2894: done (1 left) 2024-09-18T14:13:02.614 INFO:tasks.rados.rados.0.smithi084.stdout:2895: snap_remove snap 305 2024-09-18T14:13:03.458 INFO:tasks.rados.rados.0.smithi084.stdout:2895: done (1 left) 2024-09-18T14:13:03.459 INFO:tasks.rados.rados.0.smithi084.stdout:2896: write oid 42 current snap is 316 2024-09-18T14:13:03.459 INFO:tasks.rados.rados.0.smithi084.stdout:2892: finishing write tid 4 to smithi08434663-34 2024-09-18T14:13:03.459 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1333 (ObjNum 983 snap 316 seq_num 983) dirty exists 2024-09-18T14:13:03.459 INFO:tasks.rados.rados.0.smithi084.stdout:2892: left oid 34 (ObjNum 983 snap 316 seq_num 983) 2024-09-18T14:13:03.459 INFO:tasks.rados.rados.0.smithi084.stdout:2896: seq_num 985 ranges {554895=729802,1773234=778333} 2024-09-18T14:13:03.466 INFO:tasks.rados.rados.0.smithi084.stdout:2896: writing smithi08434663-42 from 554895 to 1284697 tid 1 2024-09-18T14:13:03.477 INFO:tasks.rados.rados.0.smithi084.stdout:2896: writing smithi08434663-42 from 1773234 to 2551567 tid 2 2024-09-18T14:13:03.477 INFO:tasks.rados.rados.0.smithi084.stdout:2892: done (1 left) 2024-09-18T14:13:03.477 INFO:tasks.rados.rados.0.smithi084.stdout:2897: read oid 27 snap -1 2024-09-18T14:13:03.477 INFO:tasks.rados.rados.0.smithi084.stdout:2897: expect (ObjNum 980 snap 312 seq_num 980) 2024-09-18T14:13:03.477 INFO:tasks.rados.rados.0.smithi084.stdout:2898: snap_create 2024-09-18T14:13:03.481 INFO:tasks.rados.rados.0.smithi084.stdout:2897: expect (ObjNum 980 snap 312 seq_num 980) 2024-09-18T14:13:03.622 INFO:tasks.rados.rados.0.smithi084.stdout:2896: finishing write tid 1 to smithi08434663-42 2024-09-18T14:13:03.622 INFO:tasks.rados.rados.0.smithi084.stdout:2896: finishing write tid 2 to smithi08434663-42 2024-09-18T14:13:03.622 INFO:tasks.rados.rados.0.smithi084.stdout:2896: finishing write tid 3 to smithi08434663-42 2024-09-18T14:13:03.622 INFO:tasks.rados.rados.0.smithi084.stdout:2896: finishing write tid 4 to smithi08434663-42 2024-09-18T14:13:03.622 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1341 (ObjNum 985 snap 316 seq_num 985) dirty exists 2024-09-18T14:13:03.622 INFO:tasks.rados.rados.0.smithi084.stdout:2896: left oid 42 (ObjNum 985 snap 316 seq_num 985) 2024-09-18T14:13:04.467 INFO:tasks.rados.rados.0.smithi084.stdout:2896: done (2 left) 2024-09-18T14:13:04.467 INFO:tasks.rados.rados.0.smithi084.stdout:2897: done (1 left) 2024-09-18T14:13:04.467 INFO:tasks.rados.rados.0.smithi084.stdout:2898: done (0 left) 2024-09-18T14:13:04.467 INFO:tasks.rados.rados.0.smithi084.stdout:2899: snap_remove snap 316 2024-09-18T14:13:05.469 INFO:tasks.rados.rados.0.smithi084.stdout:2899: done (0 left) 2024-09-18T14:13:05.469 INFO:tasks.rados.rados.0.smithi084.stdout:2900: copy_from oid 48 from oid 25 current snap is 317 2024-09-18T14:13:05.470 INFO:tasks.rados.rados.0.smithi084.stdout:2901: rollback oid 19 current snap is 317 2024-09-18T14:13:05.470 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 19 to 311 2024-09-18T14:13:05.470 INFO:tasks.rados.rados.0.smithi084.stdout:2902: write oid 41 current snap is 317 2024-09-18T14:13:05.470 INFO:tasks.rados.rados.0.smithi084.stdout:2902: seq_num 986 ranges {571702=405051,1431716=638344,2560365=11337} 2024-09-18T14:13:05.475 INFO:tasks.rados.rados.0.smithi084.stdout:2902: writing smithi08434663-41 from 571702 to 976753 tid 1 2024-09-18T14:13:05.481 INFO:tasks.rados.rados.0.smithi084.stdout:2902: writing smithi08434663-41 from 1431716 to 2070060 tid 2 2024-09-18T14:13:05.485 INFO:tasks.rados.rados.0.smithi084.stdout:2902: writing smithi08434663-41 from 2560365 to 2571702 tid 3 2024-09-18T14:13:05.485 INFO:tasks.rados.rados.0.smithi084.stdout:2903: snap_create 2024-09-18T14:13:05.485 INFO:tasks.rados.rados.0.smithi084.stdout:2901: finishing rollback tid 0 to smithi08434663-19 2024-09-18T14:13:05.485 INFO:tasks.rados.rados.0.smithi084.stdout:2901: finishing rollback tid 1 to smithi08434663-19 2024-09-18T14:13:05.485 INFO:tasks.rados.rados.0.smithi084.stdout:2901: finishing rollback tid 2 to smithi08434663-19 2024-09-18T14:13:05.485 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1557 (ObjNum 955 snap 304 seq_num 955) dirty exists 2024-09-18T14:13:05.485 INFO:tasks.rados.rados.0.smithi084.stdout:2900: finishing copy_from racing read to smithi08434663-48 2024-09-18T14:13:05.485 INFO:tasks.rados.rados.0.smithi084.stdout:2900: finishing copy_from to smithi08434663-48 2024-09-18T14:13:05.485 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1251 (ObjNum 955 snap 304 seq_num 955) dirty exists 2024-09-18T14:13:05.493 INFO:tasks.rados.rados.0.smithi084.stdout:2902: finishing write tid 1 to smithi08434663-41 2024-09-18T14:13:05.494 INFO:tasks.rados.rados.0.smithi084.stdout:2902: finishing write tid 2 to smithi08434663-41 2024-09-18T14:13:05.494 INFO:tasks.rados.rados.0.smithi084.stdout:2902: finishing write tid 3 to smithi08434663-41 2024-09-18T14:13:05.494 INFO:tasks.rados.rados.0.smithi084.stdout:2902: finishing write tid 4 to smithi08434663-41 2024-09-18T14:13:05.494 INFO:tasks.rados.rados.0.smithi084.stdout:2902: finishing write tid 5 to smithi08434663-41 2024-09-18T14:13:05.494 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 1346 (ObjNum 986 snap 317 seq_num 986) dirty exists 2024-09-18T14:13:05.495 INFO:tasks.rados.rados.0.smithi084.stdout:2902: left oid 41 (ObjNum 986 snap 317 seq_num 986) 2024-09-18T14:13:06.478 INFO:tasks.rados.rados.0.smithi084.stdout:2900: done (3 left) 2024-09-18T14:13:06.478 INFO:tasks.rados.rados.0.smithi084.stdout:2901: done (2 left) 2024-09-18T14:13:06.478 INFO:tasks.rados.rados.0.smithi084.stdout:2902: done (1 left) 2024-09-18T14:13:06.478 INFO:tasks.rados.rados.0.smithi084.stdout:2903: done (0 left) 2024-09-18T14:13:06.478 INFO:tasks.rados.rados.0.smithi084.stdout:2904: delete oid 27 current snap is 318 2024-09-18T14:13:06.482 INFO:tasks.rados.rados.0.smithi084.stdout:2904: done (0 left) 2024-09-18T14:13:06.483 INFO:tasks.rados.rados.0.smithi084.stdout:2905: snap_create 2024-09-18T14:13:07.480 INFO:tasks.rados.rados.0.smithi084.stdout:2905: done (0 left) 2024-09-18T14:13:07.481 INFO:tasks.rados.rados.0.smithi084.stdout:2906: delete oid 9 current snap is 319 2024-09-18T14:13:07.488 INFO:tasks.rados.rados.0.smithi084.stdout:2906: done (0 left) 2024-09-18T14:13:07.489 INFO:tasks.rados.rados.0.smithi084.stdout:2907: snap_create 2024-09-18T14:13:08.493 INFO:tasks.rados.rados.0.smithi084.stdout:2907: done (0 left) 2024-09-18T14:13:08.493 INFO:tasks.rados.rados.0.smithi084.stdout:2908: read oid 5 snap 313 2024-09-18T14:13:08.493 INFO:tasks.rados.rados.0.smithi084.stdout:2908: expect deleted 2024-09-18T14:13:08.493 INFO:tasks.rados.rados.0.smithi084.stdout:2909: snap_create 2024-09-18T14:13:09.494 INFO:tasks.rados.rados.0.smithi084.stdout:2908: done (1 left) 2024-09-18T14:13:09.494 INFO:tasks.rados.rados.0.smithi084.stdout:2909: done (0 left) 2024-09-18T14:13:09.494 INFO:tasks.rados.rados.0.smithi084.stdout:2910: delete oid 7 current snap is 321 2024-09-18T14:13:09.503 INFO:tasks.rados.rados.0.smithi084.stdout:2910: done (0 left) 2024-09-18T14:13:09.503 INFO:tasks.rados.rados.0.smithi084.stdout:2911: read oid 23 snap -1 2024-09-18T14:13:09.503 INFO:tasks.rados.rados.0.smithi084.stdout:2911: expect deleted 2024-09-18T14:13:09.503 INFO:tasks.rados.rados.0.smithi084.stdout:2912: copy_from oid 17 from oid 3 current snap is 321 2024-09-18T14:13:09.503 INFO:tasks.rados.rados.0.smithi084.stdout:2913: snap_remove snap 312 2024-09-18T14:13:10.513 INFO:tasks.rados.rados.0.smithi084.stdout:2913: done (2 left) 2024-09-18T14:13:10.513 INFO:tasks.rados.rados.0.smithi084.stdout:2914: delete oid 9 current snap is 321 2024-09-18T14:13:10.513 INFO:tasks.rados.rados.0.smithi084.stdout:2912: finishing copy_from racing read to smithi08434663-17 2024-09-18T14:13:10.513 INFO:tasks.rados.rados.0.smithi084.stdout:2912: finishing copy_from to smithi08434663-17 2024-09-18T14:13:10.514 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1037 (ObjNum 967 snap 308 seq_num 967) dirty exists 2024-09-18T14:13:10.520 INFO:tasks.rados.rados.0.smithi084.stdout:2911: done (2 left) 2024-09-18T14:13:10.520 INFO:tasks.rados.rados.0.smithi084.stdout:2912: done (1 left) 2024-09-18T14:13:10.520 INFO:tasks.rados.rados.0.smithi084.stdout:2914: done (0 left) 2024-09-18T14:13:10.520 INFO:tasks.rados.rados.0.smithi084.stdout:2915: write oid 22 current snap is 321 2024-09-18T14:13:10.520 INFO:tasks.rados.rados.0.smithi084.stdout:2915: seq_num 991 ranges {655737=497649,1607773=789340} 2024-09-18T14:13:10.526 INFO:tasks.rados.rados.0.smithi084.stdout:2915: writing smithi08434663-22 from 655737 to 1153386 tid 1 2024-09-18T14:13:10.534 INFO:tasks.rados.rados.0.smithi084.stdout:2915: writing smithi08434663-22 from 1607773 to 2397113 tid 2 2024-09-18T14:13:10.534 INFO:tasks.rados.rados.0.smithi084.stdout:2916: copy_from oid 46 from oid 30 current snap is 321 2024-09-18T14:13:10.534 INFO:tasks.rados.rados.0.smithi084.stdout:2915: finishing write tid 1 to smithi08434663-22 2024-09-18T14:13:10.534 INFO:tasks.rados.rados.0.smithi084.stdout:2917: write (excl) oid 49 current snap is 321 2024-09-18T14:13:10.534 INFO:tasks.rados.rados.0.smithi084.stdout:2917: seq_num 992 ranges {672544=572898,1666255=532998} 2024-09-18T14:13:10.541 INFO:tasks.rados.rados.0.smithi084.stdout:2917: writing smithi08434663-49 from 672544 to 1245442 tid 1 2024-09-18T14:13:10.547 INFO:tasks.rados.rados.0.smithi084.stdout:2917: writing smithi08434663-49 from 1666255 to 2199253 tid 2 2024-09-18T14:13:10.547 INFO:tasks.rados.rados.0.smithi084.stdout:2918: write oid 41 current snap is 321 2024-09-18T14:13:10.547 INFO:tasks.rados.rados.0.smithi084.stdout:2915: finishing write tid 2 to smithi08434663-22 2024-09-18T14:13:10.547 INFO:tasks.rados.rados.0.smithi084.stdout:2915: finishing write tid 3 to smithi08434663-22 2024-09-18T14:13:10.547 INFO:tasks.rados.rados.0.smithi084.stdout:2915: finishing write tid 4 to smithi08434663-22 2024-09-18T14:13:10.547 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 1040 (ObjNum 991 snap 321 seq_num 991) dirty exists 2024-09-18T14:13:10.547 INFO:tasks.rados.rados.0.smithi084.stdout:2915: left oid 22 (ObjNum 991 snap 321 seq_num 991) 2024-09-18T14:13:10.547 INFO:tasks.rados.rados.0.smithi084.stdout:2918: seq_num 993 ranges {689351=648147,2124737=564614} 2024-09-18T14:13:10.553 INFO:tasks.rados.rados.0.smithi084.stdout:2918: writing smithi08434663-41 from 689351 to 1337498 tid 1 2024-09-18T14:13:10.561 INFO:tasks.rados.rados.0.smithi084.stdout:2918: writing smithi08434663-41 from 2124737 to 2689351 tid 2 2024-09-18T14:13:10.561 INFO:tasks.rados.rados.0.smithi084.stdout:2915: done (3 left) 2024-09-18T14:13:10.562 INFO:tasks.rados.rados.0.smithi084.stdout:2919: read oid 38 snap -1 2024-09-18T14:13:10.562 INFO:tasks.rados.rados.0.smithi084.stdout:2919: expect (ObjNum 857 snap 257 seq_num 857) 2024-09-18T14:13:10.562 INFO:tasks.rados.rados.0.smithi084.stdout:2920: snap_remove snap 311 2024-09-18T14:13:11.522 INFO:tasks.rados.rados.0.smithi084.stdout:2920: done (4 left) 2024-09-18T14:13:11.522 INFO:tasks.rados.rados.0.smithi084.stdout:2921: copy_from oid 34 from oid 50 current snap is 321 2024-09-18T14:13:11.522 INFO:tasks.rados.rados.0.smithi084.stdout:2916: finishing copy_from racing read to smithi08434663-46 2024-09-18T14:13:11.522 INFO:tasks.rados.rados.0.smithi084.stdout:2916: finishing copy_from to smithi08434663-46 2024-09-18T14:13:11.522 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 864 (ObjNum 951 snap 303 seq_num 951) dirty exists 2024-09-18T14:13:11.522 INFO:tasks.rados.rados.0.smithi084.stdout:2917: finishing write tid 1 to smithi08434663-49 2024-09-18T14:13:11.523 INFO:tasks.rados.rados.0.smithi084.stdout:2917: finishing write tid 2 to smithi08434663-49 2024-09-18T14:13:11.523 INFO:tasks.rados.rados.0.smithi084.stdout:2917: finishing write tid 3 to smithi08434663-49 2024-09-18T14:13:11.523 INFO:tasks.rados.rados.0.smithi084.stdout:2917: finishing write tid 4 to smithi08434663-49 2024-09-18T14:13:11.523 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 1314 (ObjNum 992 snap 321 seq_num 992) dirty exists 2024-09-18T14:13:11.523 INFO:tasks.rados.rados.0.smithi084.stdout:2917: left oid 49 (ObjNum 992 snap 321 seq_num 992) 2024-09-18T14:13:11.523 INFO:tasks.rados.rados.0.smithi084.stdout:2918: finishing write tid 1 to smithi08434663-41 2024-09-18T14:13:11.523 INFO:tasks.rados.rados.0.smithi084.stdout:2918: finishing write tid 2 to smithi08434663-41 2024-09-18T14:13:11.523 INFO:tasks.rados.rados.0.smithi084.stdout:2918: finishing write tid 3 to smithi08434663-41 2024-09-18T14:13:11.523 INFO:tasks.rados.rados.0.smithi084.stdout:2918: finishing write tid 4 to smithi08434663-41 2024-09-18T14:13:11.523 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 1356 (ObjNum 993 snap 321 seq_num 993) dirty exists 2024-09-18T14:13:11.523 INFO:tasks.rados.rados.0.smithi084.stdout:2918: left oid 41 (ObjNum 993 snap 321 seq_num 993) 2024-09-18T14:13:11.523 INFO:tasks.rados.rados.0.smithi084.stdout:2919: expect (ObjNum 857 snap 257 seq_num 857) 2024-09-18T14:13:11.659 INFO:tasks.rados.rados.0.smithi084.stdout:2921: finishing copy_from to smithi08434663-34 2024-09-18T14:13:11.659 INFO:tasks.rados.rados.0.smithi084.stdout:2921: got expected ENOENT (src dne) 2024-09-18T14:13:11.659 INFO:tasks.rados.rados.0.smithi084.stdout:2916: done (4 left) 2024-09-18T14:13:11.659 INFO:tasks.rados.rados.0.smithi084.stdout:2917: done (3 left) 2024-09-18T14:13:11.659 INFO:tasks.rados.rados.0.smithi084.stdout:2918: done (2 left) 2024-09-18T14:13:11.659 INFO:tasks.rados.rados.0.smithi084.stdout:2919: done (1 left) 2024-09-18T14:13:11.659 INFO:tasks.rados.rados.0.smithi084.stdout:2922: rollback oid 1 current snap is 321 2024-09-18T14:13:11.659 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 1 to 319 2024-09-18T14:13:11.660 INFO:tasks.rados.rados.0.smithi084.stdout:2921: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:13:11.660 INFO:tasks.rados.rados.0.smithi084.stdout:2921: done (1 left) 2024-09-18T14:13:11.660 INFO:tasks.rados.rados.0.smithi084.stdout:2923: snap_remove snap 315 2024-09-18T14:13:12.531 INFO:tasks.rados.rados.0.smithi084.stdout:2922: finishing rollback tid 0 to smithi08434663-1 2024-09-18T14:13:12.532 INFO:tasks.rados.rados.0.smithi084.stdout:2922: finishing rollback tid 1 to smithi08434663-1 2024-09-18T14:13:12.532 INFO:tasks.rados.rados.0.smithi084.stdout:2922: finishing rollback tid 2 to smithi08434663-1 2024-09-18T14:13:12.532 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 1236 (ObjNum 961 snap 306 seq_num 961) dirty exists 2024-09-18T14:13:12.532 INFO:tasks.rados.rados.0.smithi084.stdout:2922: done (1 left) 2024-09-18T14:13:12.532 INFO:tasks.rados.rados.0.smithi084.stdout:2923: done (0 left) 2024-09-18T14:13:12.532 INFO:tasks.rados.rados.0.smithi084.stdout:2924: snap_remove snap 297 2024-09-18T14:13:13.461 INFO:tasks.rados.rados.0.smithi084.stdout:2924: done (0 left) 2024-09-18T14:13:13.461 INFO:tasks.rados.rados.0.smithi084.stdout:2925: snap_create 2024-09-18T14:13:14.470 INFO:tasks.rados.rados.0.smithi084.stdout:2925: done (0 left) 2024-09-18T14:13:14.470 INFO:tasks.rados.rados.0.smithi084.stdout:2926: rollback oid 8 current snap is 322 2024-09-18T14:13:14.471 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 8 to 310 2024-09-18T14:13:14.471 INFO:tasks.rados.rados.0.smithi084.stdout:2927: delete oid 5 current snap is 322 2024-09-18T14:13:14.472 INFO:tasks.rados.rados.0.smithi084.stdout:2926: finishing rollback tid 1 to smithi08434663-8 2024-09-18T14:13:14.472 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 0 (ObjNum 145 snap 0 seq_num 2495254712) dirty dne 2024-09-18T14:13:14.473 INFO:tasks.rados.rados.0.smithi084.stdout:2926: done (1 left) 2024-09-18T14:13:14.474 INFO:tasks.rados.rados.0.smithi084.stdout:2927: done (0 left) 2024-09-18T14:13:14.474 INFO:tasks.rados.rados.0.smithi084.stdout:2928: rollback oid 39 current snap is 322 2024-09-18T14:13:14.474 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 39 to 306 2024-09-18T14:13:14.474 INFO:tasks.rados.rados.0.smithi084.stdout:2929: write (excl) oid 17 current snap is 322 2024-09-18T14:13:14.474 INFO:tasks.rados.rados.0.smithi084.stdout:2929: seq_num 995 ranges {722965=798645,2125348=597617} 2024-09-18T14:13:14.482 INFO:tasks.rados.rados.0.smithi084.stdout:2929: writing smithi08434663-17 from 722965 to 1521610 tid 1 2024-09-18T14:13:14.489 INFO:tasks.rados.rados.0.smithi084.stdout:2929: writing smithi08434663-17 from 2125348 to 2722965 tid 2 2024-09-18T14:13:14.489 INFO:tasks.rados.rados.0.smithi084.stdout:2930: read oid 38 snap 308 2024-09-18T14:13:14.489 INFO:tasks.rados.rados.0.smithi084.stdout:2930: expect (ObjNum 857 snap 257 seq_num 857) 2024-09-18T14:13:14.489 INFO:tasks.rados.rados.0.smithi084.stdout:2931: rollback oid 4 current snap is 322 2024-09-18T14:13:14.489 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 4 to 319 2024-09-18T14:13:14.489 INFO:tasks.rados.rados.0.smithi084.stdout:2928: finishing rollback tid 0 to smithi08434663-39 2024-09-18T14:13:14.489 INFO:tasks.rados.rados.0.smithi084.stdout:2928: finishing rollback tid 1 to smithi08434663-39 2024-09-18T14:13:14.490 INFO:tasks.rados.rados.0.smithi084.stdout:2928: finishing rollback tid 2 to smithi08434663-39 2024-09-18T14:13:14.490 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 1267 (ObjNum 807 snap 244 seq_num 807) dirty exists 2024-09-18T14:13:14.490 INFO:tasks.rados.rados.0.smithi084.stdout:2929: finishing write tid 1 to smithi08434663-17 2024-09-18T14:13:14.490 INFO:tasks.rados.rados.0.smithi084.stdout:2928: done (3 left) 2024-09-18T14:13:14.490 INFO:tasks.rados.rados.0.smithi084.stdout:2932: write oid 9 current snap is 322 2024-09-18T14:13:14.490 INFO:tasks.rados.rados.0.smithi084.stdout:2932: seq_num 996 ranges {739772=590247,1900183=540336} 2024-09-18T14:13:14.496 INFO:tasks.rados.rados.0.smithi084.stdout:2932: writing smithi08434663-9 from 739772 to 1330019 tid 1 2024-09-18T14:13:14.502 INFO:tasks.rados.rados.0.smithi084.stdout:2932: writing smithi08434663-9 from 1900183 to 2440519 tid 2 2024-09-18T14:13:14.503 INFO:tasks.rados.rados.0.smithi084.stdout:2933: copy_from oid 10 from oid 22 current snap is 322 2024-09-18T14:13:14.503 INFO:tasks.rados.rados.0.smithi084.stdout:2931: finishing rollback tid 0 to smithi08434663-4 2024-09-18T14:13:14.503 INFO:tasks.rados.rados.0.smithi084.stdout:2931: finishing rollback tid 1 to smithi08434663-4 2024-09-18T14:13:14.503 INFO:tasks.rados.rados.0.smithi084.stdout:2931: finishing rollback tid 2 to smithi08434663-4 2024-09-18T14:13:14.503 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1376 (ObjNum 807 snap 244 seq_num 807) dirty exists 2024-09-18T14:13:14.503 INFO:tasks.rados.rados.0.smithi084.stdout:2930: expect (ObjNum 857 snap 257 seq_num 857) 2024-09-18T14:13:14.641 INFO:tasks.rados.rados.0.smithi084.stdout:2929: finishing write tid 2 to smithi08434663-17 2024-09-18T14:13:14.641 INFO:tasks.rados.rados.0.smithi084.stdout:2929: finishing write tid 3 to smithi08434663-17 2024-09-18T14:13:14.641 INFO:tasks.rados.rados.0.smithi084.stdout:2929: finishing write tid 4 to smithi08434663-17 2024-09-18T14:13:14.641 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1262 (ObjNum 995 snap 322 seq_num 995) dirty exists 2024-09-18T14:13:14.641 INFO:tasks.rados.rados.0.smithi084.stdout:2929: left oid 17 (ObjNum 995 snap 322 seq_num 995) 2024-09-18T14:13:14.642 INFO:tasks.rados.rados.0.smithi084.stdout:2932: finishing write tid 1 to smithi08434663-9 2024-09-18T14:13:14.642 INFO:tasks.rados.rados.0.smithi084.stdout:2929: done (4 left) 2024-09-18T14:13:14.642 INFO:tasks.rados.rados.0.smithi084.stdout:2930: done (3 left) 2024-09-18T14:13:14.642 INFO:tasks.rados.rados.0.smithi084.stdout:2931: done (2 left) 2024-09-18T14:13:14.642 INFO:tasks.rados.rados.0.smithi084.stdout:2934: write (excl) oid 17 current snap is 322 2024-09-18T14:13:14.642 INFO:tasks.rados.rados.0.smithi084.stdout:2932: finishing write tid 2 to smithi08434663-9 2024-09-18T14:13:14.642 INFO:tasks.rados.rados.0.smithi084.stdout:2932: finishing write tid 3 to smithi08434663-9 2024-09-18T14:13:14.642 INFO:tasks.rados.rados.0.smithi084.stdout:2932: finishing write tid 4 to smithi08434663-9 2024-09-18T14:13:14.642 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 1450 (ObjNum 996 snap 322 seq_num 996) dirty exists 2024-09-18T14:13:14.642 INFO:tasks.rados.rados.0.smithi084.stdout:2932: left oid 9 (ObjNum 996 snap 322 seq_num 996) 2024-09-18T14:13:14.642 INFO:tasks.rados.rados.0.smithi084.stdout:2933: finishing copy_from racing read to smithi08434663-10 2024-09-18T14:13:14.643 INFO:tasks.rados.rados.0.smithi084.stdout:2933: finishing copy_from to smithi08434663-10 2024-09-18T14:13:14.643 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 1040 (ObjNum 991 snap 321 seq_num 991) dirty exists 2024-09-18T14:13:14.643 INFO:tasks.rados.rados.0.smithi084.stdout:2934: seq_num 997 ranges {756579=665496,1958665=400347} 2024-09-18T14:13:14.648 INFO:tasks.rados.rados.0.smithi084.stdout:2934: writing smithi08434663-17 from 756579 to 1422075 tid 1 2024-09-18T14:13:14.653 INFO:tasks.rados.rados.0.smithi084.stdout:2934: writing smithi08434663-17 from 1958665 to 2359012 tid 2 2024-09-18T14:13:14.653 INFO:tasks.rados.rados.0.smithi084.stdout:2932: done (2 left) 2024-09-18T14:13:14.653 INFO:tasks.rados.rados.0.smithi084.stdout:2933: done (1 left) 2024-09-18T14:13:14.653 INFO:tasks.rados.rados.0.smithi084.stdout:2935: snap_remove snap 320 2024-09-18T14:13:15.543 INFO:tasks.rados.rados.0.smithi084.stdout:2934: finishing write tid 1 to smithi08434663-17 2024-09-18T14:13:15.543 INFO:tasks.rados.rados.0.smithi084.stdout:2934: finishing write tid 2 to smithi08434663-17 2024-09-18T14:13:15.543 INFO:tasks.rados.rados.0.smithi084.stdout:2934: finishing write tid 3 to smithi08434663-17 2024-09-18T14:13:15.544 INFO:tasks.rados.rados.0.smithi084.stdout:2934: finishing write tid 4 to smithi08434663-17 2024-09-18T14:13:15.544 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1265 (ObjNum 997 snap 322 seq_num 997) dirty exists 2024-09-18T14:13:15.544 INFO:tasks.rados.rados.0.smithi084.stdout:2934: left oid 17 (ObjNum 997 snap 322 seq_num 997) 2024-09-18T14:13:15.544 INFO:tasks.rados.rados.0.smithi084.stdout:2934: done (1 left) 2024-09-18T14:13:15.544 INFO:tasks.rados.rados.0.smithi084.stdout:2935: done (0 left) 2024-09-18T14:13:15.544 INFO:tasks.rados.rados.0.smithi084.stdout:2936: read oid 11 snap -1 2024-09-18T14:13:15.544 INFO:tasks.rados.rados.0.smithi084.stdout:2936: expect deleted 2024-09-18T14:13:15.544 INFO:tasks.rados.rados.0.smithi084.stdout:2937: write (excl) oid 25 current snap is 322 2024-09-18T14:13:15.544 INFO:tasks.rados.rados.0.smithi084.stdout:2937: seq_num 998 ranges {773386=740745,2017147=544005} 2024-09-18T14:13:15.551 INFO:tasks.rados.rados.0.smithi084.stdout:2937: writing smithi08434663-25 from 773386 to 1514131 tid 1 2024-09-18T14:13:15.557 INFO:tasks.rados.rados.0.smithi084.stdout:2937: writing smithi08434663-25 from 2017147 to 2561152 tid 2 2024-09-18T14:13:15.558 INFO:tasks.rados.rados.0.smithi084.stdout:2938: read oid 10 snap 317 2024-09-18T14:13:15.558 INFO:tasks.rados.rados.0.smithi084.stdout:2938: expect (ObjNum 904 snap 281 seq_num 904) 2024-09-18T14:13:15.558 INFO:tasks.rados.rados.0.smithi084.stdout:2939: delete oid 24 current snap is 322 2024-09-18T14:13:15.560 INFO:tasks.rados.rados.0.smithi084.stdout:2937: finishing write tid 1 to smithi08434663-25 2024-09-18T14:13:15.561 INFO:tasks.rados.rados.0.smithi084.stdout:2938: expect (ObjNum 904 snap 281 seq_num 904) 2024-09-18T14:13:15.788 INFO:tasks.rados.rados.0.smithi084.stdout:2937: finishing write tid 2 to smithi08434663-25 2024-09-18T14:13:15.788 INFO:tasks.rados.rados.0.smithi084.stdout:2937: finishing write tid 3 to smithi08434663-25 2024-09-18T14:13:15.788 INFO:tasks.rados.rados.0.smithi084.stdout:2936: done (3 left) 2024-09-18T14:13:15.788 INFO:tasks.rados.rados.0.smithi084.stdout:2938: done (2 left) 2024-09-18T14:13:15.788 INFO:tasks.rados.rados.0.smithi084.stdout:2939: done (1 left) 2024-09-18T14:13:15.788 INFO:tasks.rados.rados.0.smithi084.stdout:2940: read oid 29 snap -1 2024-09-18T14:13:15.789 INFO:tasks.rados.rados.0.smithi084.stdout:2940: expect deleted 2024-09-18T14:13:15.789 INFO:tasks.rados.rados.0.smithi084.stdout:2941: read oid 1 snap -1 2024-09-18T14:13:15.789 INFO:tasks.rados.rados.0.smithi084.stdout:2941: expect (ObjNum 961 snap 306 seq_num 961) 2024-09-18T14:13:15.789 INFO:tasks.rados.rados.0.smithi084.stdout:2942: read oid 4 snap 318 2024-09-18T14:13:15.789 INFO:tasks.rados.rados.0.smithi084.stdout:2942: expect (ObjNum 807 snap 244 seq_num 807) 2024-09-18T14:13:15.789 INFO:tasks.rados.rados.0.smithi084.stdout:2943: rollback oid 16 current snap is 322 2024-09-18T14:13:15.789 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 16 to 308 2024-09-18T14:13:15.789 INFO:tasks.rados.rados.0.smithi084.stdout:2937: finishing write tid 4 to smithi08434663-25 2024-09-18T14:13:15.789 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 1260 (ObjNum 998 snap 322 seq_num 998) dirty exists 2024-09-18T14:13:15.789 INFO:tasks.rados.rados.0.smithi084.stdout:2937: left oid 25 (ObjNum 998 snap 322 seq_num 998) 2024-09-18T14:13:15.789 INFO:tasks.rados.rados.0.smithi084.stdout:2937: done (4 left) 2024-09-18T14:13:15.789 INFO:tasks.rados.rados.0.smithi084.stdout:2944: write oid 37 current snap is 322 2024-09-18T14:13:15.790 INFO:tasks.rados.rados.0.smithi084.stdout:2944: seq_num 1000 ranges {407000=491243,1617758=547674,2595228=211772} 2024-09-18T14:13:15.792 INFO:tasks.rados.rados.0.smithi084.stdout:2944: writing smithi08434663-37 from 407000 to 898243 tid 1 2024-09-18T14:13:15.799 INFO:tasks.rados.rados.0.smithi084.stdout:2944: writing smithi08434663-37 from 1617758 to 2165432 tid 2 2024-09-18T14:13:15.803 INFO:tasks.rados.rados.0.smithi084.stdout:2944: writing smithi08434663-37 from 2595228 to 2807000 tid 3 2024-09-18T14:13:15.803 INFO:tasks.rados.rados.0.smithi084.stdout:2945: write oid 50 current snap is 322 2024-09-18T14:13:15.803 INFO:tasks.rados.rados.0.smithi084.stdout:2945: seq_num 1001 ranges {423807=566492,1676240=407685,2739004=84803} 2024-09-18T14:13:15.808 INFO:tasks.rados.rados.0.smithi084.stdout:2945: writing smithi08434663-50 from 423807 to 990299 tid 1 2024-09-18T14:13:15.814 INFO:tasks.rados.rados.0.smithi084.stdout:2945: writing smithi08434663-50 from 1676240 to 2083925 tid 2 2024-09-18T14:13:15.819 INFO:tasks.rados.rados.0.smithi084.stdout:2945: writing smithi08434663-50 from 2739004 to 2823807 tid 3 2024-09-18T14:13:15.819 INFO:tasks.rados.rados.0.smithi084.stdout:2946: read oid 28 snap -1 2024-09-18T14:13:15.819 INFO:tasks.rados.rados.0.smithi084.stdout:2946: expect (ObjNum 971 snap 308 seq_num 971) 2024-09-18T14:13:15.819 INFO:tasks.rados.rados.0.smithi084.stdout:2947: snap_create 2024-09-18T14:13:15.819 INFO:tasks.rados.rados.0.smithi084.stdout:2941: expect (ObjNum 961 snap 306 seq_num 961) 2024-09-18T14:13:15.891 INFO:tasks.rados.rados.0.smithi084.stdout:2943: finishing rollback tid 0 to smithi08434663-16 2024-09-18T14:13:15.891 INFO:tasks.rados.rados.0.smithi084.stdout:2943: finishing rollback tid 1 to smithi08434663-16 2024-09-18T14:13:15.891 INFO:tasks.rados.rados.0.smithi084.stdout:2943: finishing rollback tid 2 to smithi08434663-16 2024-09-18T14:13:15.891 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1453 (ObjNum 915 snap 286 seq_num 915) dirty exists 2024-09-18T14:13:15.891 INFO:tasks.rados.rados.0.smithi084.stdout:2942: expect (ObjNum 807 snap 244 seq_num 807) 2024-09-18T14:13:16.078 INFO:tasks.rados.rados.0.smithi084.stdout:2944: finishing write tid 1 to smithi08434663-37 2024-09-18T14:13:16.078 INFO:tasks.rados.rados.0.smithi084.stdout:2944: finishing write tid 2 to smithi08434663-37 2024-09-18T14:13:16.079 INFO:tasks.rados.rados.0.smithi084.stdout:2944: finishing write tid 3 to smithi08434663-37 2024-09-18T14:13:16.079 INFO:tasks.rados.rados.0.smithi084.stdout:2944: finishing write tid 4 to smithi08434663-37 2024-09-18T14:13:16.079 INFO:tasks.rados.rados.0.smithi084.stdout:2944: finishing write tid 5 to smithi08434663-37 2024-09-18T14:13:16.079 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 1318 (ObjNum 1000 snap 322 seq_num 1000) dirty exists 2024-09-18T14:13:16.079 INFO:tasks.rados.rados.0.smithi084.stdout:2944: left oid 37 (ObjNum 1000 snap 322 seq_num 1000) 2024-09-18T14:13:16.079 INFO:tasks.rados.rados.0.smithi084.stdout:2945: finishing write tid 1 to smithi08434663-50 2024-09-18T14:13:16.079 INFO:tasks.rados.rados.0.smithi084.stdout:2945: finishing write tid 2 to smithi08434663-50 2024-09-18T14:13:16.079 INFO:tasks.rados.rados.0.smithi084.stdout:2945: finishing write tid 3 to smithi08434663-50 2024-09-18T14:13:16.079 INFO:tasks.rados.rados.0.smithi084.stdout:2945: finishing write tid 4 to smithi08434663-50 2024-09-18T14:13:16.079 INFO:tasks.rados.rados.0.smithi084.stdout:2945: finishing write tid 5 to smithi08434663-50 2024-09-18T14:13:16.079 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 1579 (ObjNum 1001 snap 322 seq_num 1001) dirty exists 2024-09-18T14:13:16.079 INFO:tasks.rados.rados.0.smithi084.stdout:2945: left oid 50 (ObjNum 1001 snap 322 seq_num 1001) 2024-09-18T14:13:16.080 INFO:tasks.rados.rados.0.smithi084.stdout:2946: expect (ObjNum 971 snap 308 seq_num 971) 2024-09-18T14:13:16.548 INFO:tasks.rados.rados.0.smithi084.stdout:2940: done (7 left) 2024-09-18T14:13:16.548 INFO:tasks.rados.rados.0.smithi084.stdout:2941: done (6 left) 2024-09-18T14:13:16.548 INFO:tasks.rados.rados.0.smithi084.stdout:2942: done (5 left) 2024-09-18T14:13:16.548 INFO:tasks.rados.rados.0.smithi084.stdout:2943: done (4 left) 2024-09-18T14:13:16.548 INFO:tasks.rados.rados.0.smithi084.stdout:2944: done (3 left) 2024-09-18T14:13:16.548 INFO:tasks.rados.rados.0.smithi084.stdout:2945: done (2 left) 2024-09-18T14:13:16.548 INFO:tasks.rados.rados.0.smithi084.stdout:2946: done (1 left) 2024-09-18T14:13:16.548 INFO:tasks.rados.rados.0.smithi084.stdout:2947: done (0 left) 2024-09-18T14:13:16.548 INFO:tasks.rados.rados.0.smithi084.stdout:2948: delete oid 30 current snap is 323 2024-09-18T14:13:16.551 INFO:tasks.rados.rados.0.smithi084.stdout:2948: done (0 left) 2024-09-18T14:13:16.552 INFO:tasks.rados.rados.0.smithi084.stdout:2949: copy_from oid 26 from oid 29 current snap is 323 2024-09-18T14:13:16.552 INFO:tasks.rados.rados.0.smithi084.stdout:2950: delete oid 41 current snap is 323 2024-09-18T14:13:16.555 INFO:tasks.rados.rados.0.smithi084.stdout:2949: finishing copy_from to smithi08434663-26 2024-09-18T14:13:16.555 INFO:tasks.rados.rados.0.smithi084.stdout:2949: got expected ENOENT (src dne) 2024-09-18T14:13:16.555 INFO:tasks.rados.rados.0.smithi084.stdout:2949: finishing copy_from racing read to smithi08434663-26 2024-09-18T14:13:16.555 INFO:tasks.rados.rados.0.smithi084.stdout:2949: done (1 left) 2024-09-18T14:13:16.555 INFO:tasks.rados.rados.0.smithi084.stdout:2950: done (0 left) 2024-09-18T14:13:16.555 INFO:tasks.rados.rados.0.smithi084.stdout:2951: delete oid 9 current snap is 323 2024-09-18T14:13:16.557 INFO:tasks.rados.rados.0.smithi084.stdout:2951: done (0 left) 2024-09-18T14:13:16.557 INFO:tasks.rados.rados.0.smithi084.stdout:2952: copy_from oid 35 from oid 49 current snap is 323 2024-09-18T14:13:16.557 INFO:tasks.rados.rados.0.smithi084.stdout:2953: snap_create 2024-09-18T14:13:16.568 INFO:tasks.rados.rados.0.smithi084.stdout:2952: finishing copy_from racing read to smithi08434663-35 2024-09-18T14:13:16.571 INFO:tasks.rados.rados.0.smithi084.stdout:2952: finishing copy_from to smithi08434663-35 2024-09-18T14:13:16.571 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 1314 (ObjNum 992 snap 321 seq_num 992) dirty exists 2024-09-18T14:13:17.550 INFO:tasks.rados.rados.0.smithi084.stdout:2952: done (1 left) 2024-09-18T14:13:17.550 INFO:tasks.rados.rados.0.smithi084.stdout:2953: done (0 left) 2024-09-18T14:13:17.550 INFO:tasks.rados.rados.0.smithi084.stdout:2954: read oid 10 snap -1 2024-09-18T14:13:17.550 INFO:tasks.rados.rados.0.smithi084.stdout:2954: expect (ObjNum 991 snap 321 seq_num 991) 2024-09-18T14:13:17.550 INFO:tasks.rados.rados.0.smithi084.stdout:2955: read oid 40 snap -1 2024-09-18T14:13:17.550 INFO:tasks.rados.rados.0.smithi084.stdout:2955: expect deleted 2024-09-18T14:13:17.550 INFO:tasks.rados.rados.0.smithi084.stdout:2956: write oid 6 current snap is 324 2024-09-18T14:13:17.550 INFO:tasks.rados.rados.0.smithi084.stdout:2956: seq_num 1005 ranges {491035=583841,1510168=698670,2732343=158692} 2024-09-18T14:13:17.555 INFO:tasks.rados.rados.0.smithi084.stdout:2956: writing smithi08434663-6 from 491035 to 1074876 tid 1 2024-09-18T14:13:17.562 INFO:tasks.rados.rados.0.smithi084.stdout:2956: writing smithi08434663-6 from 1510168 to 2208838 tid 2 2024-09-18T14:13:17.566 INFO:tasks.rados.rados.0.smithi084.stdout:2956: writing smithi08434663-6 from 2732343 to 2891035 tid 3 2024-09-18T14:13:17.566 INFO:tasks.rados.rados.0.smithi084.stdout:2957: rollback oid 32 current snap is 324 2024-09-18T14:13:17.566 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 32 to 313 2024-09-18T14:13:17.566 INFO:tasks.rados.rados.0.smithi084.stdout:2954: expect (ObjNum 991 snap 321 seq_num 991) 2024-09-18T14:13:17.725 INFO:tasks.rados.rados.0.smithi084.stdout:2956: finishing write tid 1 to smithi08434663-6 2024-09-18T14:13:17.725 INFO:tasks.rados.rados.0.smithi084.stdout:2954: done (3 left) 2024-09-18T14:13:17.725 INFO:tasks.rados.rados.0.smithi084.stdout:2955: done (2 left) 2024-09-18T14:13:17.725 INFO:tasks.rados.rados.0.smithi084.stdout:2958: write (excl) oid 30 current snap is 324 2024-09-18T14:13:17.725 INFO:tasks.rados.rados.0.smithi084.stdout:2956: finishing write tid 2 to smithi08434663-6 2024-09-18T14:13:17.725 INFO:tasks.rados.rados.0.smithi084.stdout:2956: finishing write tid 3 to smithi08434663-6 2024-09-18T14:13:17.725 INFO:tasks.rados.rados.0.smithi084.stdout:2956: finishing write tid 4 to smithi08434663-6 2024-09-18T14:13:17.725 INFO:tasks.rados.rados.0.smithi084.stdout:2956: finishing write tid 5 to smithi08434663-6 2024-09-18T14:13:17.725 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 1009 (ObjNum 1005 snap 324 seq_num 1005) dirty exists 2024-09-18T14:13:17.725 INFO:tasks.rados.rados.0.smithi084.stdout:2956: left oid 6 (ObjNum 1005 snap 324 seq_num 1005) 2024-09-18T14:13:17.726 INFO:tasks.rados.rados.0.smithi084.stdout:2957: finishing rollback tid 0 to smithi08434663-32 2024-09-18T14:13:17.726 INFO:tasks.rados.rados.0.smithi084.stdout:2957: finishing rollback tid 1 to smithi08434663-32 2024-09-18T14:13:17.726 INFO:tasks.rados.rados.0.smithi084.stdout:2957: finishing rollback tid 2 to smithi08434663-32 2024-09-18T14:13:17.726 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 1583 (ObjNum 901 snap 281 seq_num 901) dirty exists 2024-09-18T14:13:17.726 INFO:tasks.rados.rados.0.smithi084.stdout:2958: seq_num 1006 ranges {507842=659090,1568650=558681,2759766=148076} 2024-09-18T14:13:17.730 INFO:tasks.rados.rados.0.smithi084.stdout:2958: writing smithi08434663-30 from 507842 to 1166932 tid 1 2024-09-18T14:13:17.736 INFO:tasks.rados.rados.0.smithi084.stdout:2958: writing smithi08434663-30 from 1568650 to 2127331 tid 2 2024-09-18T14:13:17.741 INFO:tasks.rados.rados.0.smithi084.stdout:2958: writing smithi08434663-30 from 2759766 to 2907842 tid 3 2024-09-18T14:13:17.741 INFO:tasks.rados.rados.0.smithi084.stdout:2956: done (2 left) 2024-09-18T14:13:17.741 INFO:tasks.rados.rados.0.smithi084.stdout:2957: done (1 left) 2024-09-18T14:13:17.741 INFO:tasks.rados.rados.0.smithi084.stdout:2959: read oid 32 snap 310 2024-09-18T14:13:17.741 INFO:tasks.rados.rados.0.smithi084.stdout:2959: expect (ObjNum 901 snap 281 seq_num 901) 2024-09-18T14:13:17.741 INFO:tasks.rados.rados.0.smithi084.stdout:2960: write (excl) oid 4 current snap is 324 2024-09-18T14:13:17.741 INFO:tasks.rados.rados.0.smithi084.stdout:2958: finishing write tid 1 to smithi08434663-30 2024-09-18T14:13:17.741 INFO:tasks.rados.rados.0.smithi084.stdout:2958: finishing write tid 2 to smithi08434663-30 2024-09-18T14:13:17.741 INFO:tasks.rados.rados.0.smithi084.stdout:2960: seq_num 1007 ranges {524649=734339,1910779=702339} 2024-09-18T14:13:17.747 INFO:tasks.rados.rados.0.smithi084.stdout:2960: writing smithi08434663-4 from 524649 to 1258988 tid 1 2024-09-18T14:13:17.755 INFO:tasks.rados.rados.0.smithi084.stdout:2960: writing smithi08434663-4 from 1910779 to 2613118 tid 2 2024-09-18T14:13:17.755 INFO:tasks.rados.rados.0.smithi084.stdout:2961: write oid 46 current snap is 324 2024-09-18T14:13:17.755 INFO:tasks.rados.rados.0.smithi084.stdout:2961: seq_num 1008 ranges {541456=409588,1569261=562350,2814612=126844} 2024-09-18T14:13:17.760 INFO:tasks.rados.rados.0.smithi084.stdout:2961: writing smithi08434663-46 from 541456 to 951044 tid 1 2024-09-18T14:13:17.766 INFO:tasks.rados.rados.0.smithi084.stdout:2961: writing smithi08434663-46 from 1569261 to 2131611 tid 2 2024-09-18T14:13:17.770 INFO:tasks.rados.rados.0.smithi084.stdout:2961: writing smithi08434663-46 from 2814612 to 2941456 tid 3 2024-09-18T14:13:17.771 INFO:tasks.rados.rados.0.smithi084.stdout:2962: write oid 38 current snap is 324 2024-09-18T14:13:17.771 INFO:tasks.rados.rados.0.smithi084.stdout:2962: seq_num 1009 ranges {558263=484837,1627743=706008} 2024-09-18T14:13:17.776 INFO:tasks.rados.rados.0.smithi084.stdout:2962: writing smithi08434663-38 from 558263 to 1043100 tid 1 2024-09-18T14:13:17.783 INFO:tasks.rados.rados.0.smithi084.stdout:2962: writing smithi08434663-38 from 1627743 to 2333751 tid 2 2024-09-18T14:13:17.783 INFO:tasks.rados.rados.0.smithi084.stdout:2963: copy_from oid 21 from oid 3 current snap is 324 2024-09-18T14:13:17.783 INFO:tasks.rados.rados.0.smithi084.stdout:2958: finishing write tid 3 to smithi08434663-30 2024-09-18T14:13:17.783 INFO:tasks.rados.rados.0.smithi084.stdout:2959: expect (ObjNum 901 snap 281 seq_num 901) 2024-09-18T14:13:17.923 INFO:tasks.rados.rados.0.smithi084.stdout:2958: finishing write tid 4 to smithi08434663-30 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:2958: finishing write tid 5 to smithi08434663-30 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 869 (ObjNum 1006 snap 324 seq_num 1006) dirty exists 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:2958: left oid 30 (ObjNum 1006 snap 324 seq_num 1006) 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:2960: finishing write tid 1 to smithi08434663-4 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:2958: done (5 left) 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:2959: done (4 left) 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:2964: read oid 2 snap -1 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:2964: expect (ObjNum 942 snap 301 seq_num 942) 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:2965: delete oid 34 current snap is 324 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:2960: finishing write tid 2 to smithi08434663-4 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:2960: finishing write tid 3 to smithi08434663-4 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:2960: finishing write tid 4 to smithi08434663-4 2024-09-18T14:13:17.924 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1386 (ObjNum 1007 snap 324 seq_num 1007) dirty exists 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:2960: left oid 4 (ObjNum 1007 snap 324 seq_num 1007) 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:2961: finishing write tid 1 to smithi08434663-46 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:2961: finishing write tid 2 to smithi08434663-46 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:2961: finishing write tid 3 to smithi08434663-46 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:2961: finishing write tid 4 to smithi08434663-46 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:2961: finishing write tid 5 to smithi08434663-46 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1458 (ObjNum 1008 snap 324 seq_num 1008) dirty exists 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:2961: left oid 46 (ObjNum 1008 snap 324 seq_num 1008) 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:2962: finishing write tid 1 to smithi08434663-38 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:2962: finishing write tid 2 to smithi08434663-38 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:2962: finishing write tid 3 to smithi08434663-38 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:2962: finishing write tid 4 to smithi08434663-38 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 1016 (ObjNum 1009 snap 324 seq_num 1009) dirty exists 2024-09-18T14:13:17.925 INFO:tasks.rados.rados.0.smithi084.stdout:2962: left oid 38 (ObjNum 1009 snap 324 seq_num 1009) 2024-09-18T14:13:17.926 INFO:tasks.rados.rados.0.smithi084.stdout:2963: finishing copy_from racing read to smithi08434663-21 2024-09-18T14:13:17.926 INFO:tasks.rados.rados.0.smithi084.stdout:2963: finishing copy_from to smithi08434663-21 2024-09-18T14:13:17.926 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 1037 (ObjNum 967 snap 308 seq_num 967) dirty exists 2024-09-18T14:13:17.926 INFO:tasks.rados.rados.0.smithi084.stdout:2960: done (5 left) 2024-09-18T14:13:17.926 INFO:tasks.rados.rados.0.smithi084.stdout:2961: done (4 left) 2024-09-18T14:13:17.926 INFO:tasks.rados.rados.0.smithi084.stdout:2962: done (3 left) 2024-09-18T14:13:17.926 INFO:tasks.rados.rados.0.smithi084.stdout:2963: done (2 left) 2024-09-18T14:13:17.927 INFO:tasks.rados.rados.0.smithi084.stdout:2965: done (1 left) 2024-09-18T14:13:17.927 INFO:tasks.rados.rados.0.smithi084.stdout:2966: rollback oid 14 current snap is 324 2024-09-18T14:13:17.927 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 14 to 292 2024-09-18T14:13:17.927 INFO:tasks.rados.rados.0.smithi084.stdout:2967: snap_create 2024-09-18T14:13:17.929 INFO:tasks.rados.rados.0.smithi084.stdout:2964: expect (ObjNum 942 snap 301 seq_num 942) 2024-09-18T14:13:18.049 INFO:tasks.rados.rados.0.smithi084.stdout:2966: finishing rollback tid 1 to smithi08434663-14 2024-09-18T14:13:18.049 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 0 (ObjNum 257 snap 0 seq_num 2494624832) dirty dne 2024-09-18T14:13:18.462 INFO:tasks.rados.rados.0.smithi084.stdout:2964: done (2 left) 2024-09-18T14:13:18.463 INFO:tasks.rados.rados.0.smithi084.stdout:2966: done (1 left) 2024-09-18T14:13:18.463 INFO:tasks.rados.rados.0.smithi084.stdout:2967: done (0 left) 2024-09-18T14:13:18.463 INFO:tasks.rados.rados.0.smithi084.stdout:2968: copy_from oid 35 from oid 21 current snap is 325 2024-09-18T14:13:18.463 INFO:tasks.rados.rados.0.smithi084.stdout:2969: write (excl) oid 41 current snap is 325 2024-09-18T14:13:18.463 INFO:tasks.rados.rados.0.smithi084.stdout:2969: seq_num 1011 ranges {591877=635335,1628354=709677} 2024-09-18T14:13:18.470 INFO:tasks.rados.rados.0.smithi084.stdout:2969: writing smithi08434663-41 from 591877 to 1227212 tid 1 2024-09-18T14:13:18.477 INFO:tasks.rados.rados.0.smithi084.stdout:2969: writing smithi08434663-41 from 1628354 to 2338031 tid 2 2024-09-18T14:13:18.477 INFO:tasks.rados.rados.0.smithi084.stdout:2970: write oid 23 current snap is 325 2024-09-18T14:13:18.477 INFO:tasks.rados.rados.0.smithi084.stdout:2970: seq_num 1012 ranges {608684=426937,1803189=569688,2924304=84380} 2024-09-18T14:13:18.483 INFO:tasks.rados.rados.0.smithi084.stdout:2970: writing smithi08434663-23 from 608684 to 1035621 tid 1 2024-09-18T14:13:18.491 INFO:tasks.rados.rados.0.smithi084.stdout:2970: writing smithi08434663-23 from 1803189 to 2372877 tid 2 2024-09-18T14:13:18.495 INFO:tasks.rados.rados.0.smithi084.stdout:2970: writing smithi08434663-23 from 2924304 to 3008684 tid 3 2024-09-18T14:13:18.495 INFO:tasks.rados.rados.0.smithi084.stdout:2971: copy_from oid 30 from oid 25 current snap is 325 2024-09-18T14:13:18.495 INFO:tasks.rados.rados.0.smithi084.stdout:2968: finishing copy_from racing read to smithi08434663-35 2024-09-18T14:13:18.495 INFO:tasks.rados.rados.0.smithi084.stdout:2968: finishing copy_from to smithi08434663-35 2024-09-18T14:13:18.495 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 1037 (ObjNum 967 snap 308 seq_num 967) dirty exists 2024-09-18T14:13:18.495 INFO:tasks.rados.rados.0.smithi084.stdout:2969: finishing write tid 1 to smithi08434663-41 2024-09-18T14:13:18.495 INFO:tasks.rados.rados.0.smithi084.stdout:2969: finishing write tid 2 to smithi08434663-41 2024-09-18T14:13:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:2969: finishing write tid 3 to smithi08434663-41 2024-09-18T14:13:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:2969: finishing write tid 4 to smithi08434663-41 2024-09-18T14:13:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 1391 (ObjNum 1011 snap 325 seq_num 1011) dirty exists 2024-09-18T14:13:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:2969: left oid 41 (ObjNum 1011 snap 325 seq_num 1011) 2024-09-18T14:13:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:2970: finishing write tid 1 to smithi08434663-23 2024-09-18T14:13:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:2968: done (3 left) 2024-09-18T14:13:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:2969: done (2 left) 2024-09-18T14:13:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:2972: copy_from oid 38 from oid 26 current snap is 325 2024-09-18T14:13:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:2973: read oid 33 snap -1 2024-09-18T14:13:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:2973: expect deleted 2024-09-18T14:13:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:2974: copy_from oid 40 from oid 28 current snap is 325 2024-09-18T14:13:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:2975: copy_from oid 18 from oid 29 current snap is 325 2024-09-18T14:13:18.497 INFO:tasks.rados.rados.0.smithi084.stdout:2976: snap_remove snap 301 2024-09-18T14:13:19.478 INFO:tasks.rados.rados.0.smithi084.stdout:2976: done (6 left) 2024-09-18T14:13:19.479 INFO:tasks.rados.rados.0.smithi084.stdout:2977: copy_from oid 27 from oid 1 current snap is 325 2024-09-18T14:13:19.479 INFO:tasks.rados.rados.0.smithi084.stdout:2975: finishing copy_from to smithi08434663-18 2024-09-18T14:13:19.479 INFO:tasks.rados.rados.0.smithi084.stdout:2975: got expected ENOENT (src dne) 2024-09-18T14:13:19.479 INFO:tasks.rados.rados.0.smithi084.stdout:2975: finishing copy_from racing read to smithi08434663-18 2024-09-18T14:13:19.479 INFO:tasks.rados.rados.0.smithi084.stdout:2970: finishing write tid 2 to smithi08434663-23 2024-09-18T14:13:19.479 INFO:tasks.rados.rados.0.smithi084.stdout:2970: finishing write tid 3 to smithi08434663-23 2024-09-18T14:13:19.479 INFO:tasks.rados.rados.0.smithi084.stdout:2970: finishing write tid 4 to smithi08434663-23 2024-09-18T14:13:19.479 INFO:tasks.rados.rados.0.smithi084.stdout:2970: finishing write tid 5 to smithi08434663-23 2024-09-18T14:13:19.479 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1013 (ObjNum 1012 snap 325 seq_num 1012) dirty exists 2024-09-18T14:13:19.479 INFO:tasks.rados.rados.0.smithi084.stdout:2970: left oid 23 (ObjNum 1012 snap 325 seq_num 1012) 2024-09-18T14:13:19.479 INFO:tasks.rados.rados.0.smithi084.stdout:2974: finishing copy_from racing read to smithi08434663-40 2024-09-18T14:13:19.479 INFO:tasks.rados.rados.0.smithi084.stdout:2971: finishing copy_from racing read to smithi08434663-30 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:2974: finishing copy_from to smithi08434663-40 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 1519 (ObjNum 971 snap 308 seq_num 971) dirty exists 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:2971: finishing copy_from to smithi08434663-30 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 1260 (ObjNum 998 snap 322 seq_num 998) dirty exists 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:2972: finishing copy_from racing read to smithi08434663-38 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:2972: finishing copy_from to smithi08434663-38 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 1229 (ObjNum 938 snap 300 seq_num 938) dirty exists 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:2970: done (6 left) 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:2971: done (5 left) 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:2972: done (4 left) 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:2973: done (3 left) 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:2974: done (2 left) 2024-09-18T14:13:19.480 INFO:tasks.rados.rados.0.smithi084.stdout:2975: done (1 left) 2024-09-18T14:13:19.481 INFO:tasks.rados.rados.0.smithi084.stdout:2978: snap_remove snap 317 2024-09-18T14:13:20.480 INFO:tasks.rados.rados.0.smithi084.stdout:2978: done (1 left) 2024-09-18T14:13:20.480 INFO:tasks.rados.rados.0.smithi084.stdout:2979: write (excl) oid 24 current snap is 325 2024-09-18T14:13:20.480 INFO:tasks.rados.rados.0.smithi084.stdout:2979: seq_num 1013 ranges {625491=502186,1861671=713346} 2024-09-18T14:13:20.486 INFO:tasks.rados.rados.0.smithi084.stdout:2979: writing smithi08434663-24 from 625491 to 1127677 tid 1 2024-09-18T14:13:20.495 INFO:tasks.rados.rados.0.smithi084.stdout:2979: writing smithi08434663-24 from 1861671 to 2575017 tid 2 2024-09-18T14:13:20.495 INFO:tasks.rados.rados.0.smithi084.stdout:2980: rollback oid 47 current snap is 325 2024-09-18T14:13:20.495 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 47 to 318 2024-09-18T14:13:20.495 INFO:tasks.rados.rados.0.smithi084.stdout:2977: finishing copy_from racing read to smithi08434663-27 2024-09-18T14:13:20.495 INFO:tasks.rados.rados.0.smithi084.stdout:2977: finishing copy_from to smithi08434663-27 2024-09-18T14:13:20.495 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1236 (ObjNum 961 snap 306 seq_num 961) dirty exists 2024-09-18T14:13:20.495 INFO:tasks.rados.rados.0.smithi084.stdout:2979: finishing write tid 1 to smithi08434663-24 2024-09-18T14:13:20.495 INFO:tasks.rados.rados.0.smithi084.stdout:2977: done (2 left) 2024-09-18T14:13:20.496 INFO:tasks.rados.rados.0.smithi084.stdout:2981: write oid 20 current snap is 325 2024-09-18T14:13:20.496 INFO:tasks.rados.rados.0.smithi084.stdout:2981: seq_num 1014 ranges {642298=577435,1920153=573357,2979150=63148} 2024-09-18T14:13:20.502 INFO:tasks.rados.rados.0.smithi084.stdout:2981: writing smithi08434663-20 from 642298 to 1219733 tid 1 2024-09-18T14:13:20.509 INFO:tasks.rados.rados.0.smithi084.stdout:2981: writing smithi08434663-20 from 1920153 to 2493510 tid 2 2024-09-18T14:13:20.513 INFO:tasks.rados.rados.0.smithi084.stdout:2981: writing smithi08434663-20 from 2979150 to 3042298 tid 3 2024-09-18T14:13:20.513 INFO:tasks.rados.rados.0.smithi084.stdout:2982: copy_from oid 38 from oid 15 current snap is 325 2024-09-18T14:13:20.513 INFO:tasks.rados.rados.0.smithi084.stdout:2980: finishing rollback tid 1 to smithi08434663-47 2024-09-18T14:13:20.513 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 1261 (ObjNum 65 snap 0 seq_num 2475021424) dirty dne 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:2979: finishing write tid 2 to smithi08434663-24 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:2979: finishing write tid 3 to smithi08434663-24 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:2979: finishing write tid 4 to smithi08434663-24 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 604 (ObjNum 1013 snap 325 seq_num 1013) dirty exists 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:2979: left oid 24 (ObjNum 1013 snap 325 seq_num 1013) 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:2981: finishing write tid 1 to smithi08434663-20 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:2979: done (3 left) 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:2980: done (2 left) 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:2983: read oid 31 snap -1 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:2983: expect (ObjNum 981 snap 312 seq_num 981) 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:2984: read oid 47 snap -1 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:2984: expect deleted 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:2985: rollback oid 46 current snap is 325 2024-09-18T14:13:20.514 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 46 to 303 2024-09-18T14:13:20.515 INFO:tasks.rados.rados.0.smithi084.stdout:2981: finishing write tid 2 to smithi08434663-20 2024-09-18T14:13:20.515 INFO:tasks.rados.rados.0.smithi084.stdout:2986: rollback oid 27 current snap is 325 2024-09-18T14:13:20.515 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 27 to 309 2024-09-18T14:13:20.515 INFO:tasks.rados.rados.0.smithi084.stdout:2987: snap_create 2024-09-18T14:13:20.515 INFO:tasks.rados.rados.0.smithi084.stdout:2981: finishing write tid 3 to smithi08434663-20 2024-09-18T14:13:20.515 INFO:tasks.rados.rados.0.smithi084.stdout:2981: finishing write tid 4 to smithi08434663-20 2024-09-18T14:13:20.516 INFO:tasks.rados.rados.0.smithi084.stdout:2981: finishing write tid 5 to smithi08434663-20 2024-09-18T14:13:20.516 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 1210 (ObjNum 1014 snap 325 seq_num 1014) dirty exists 2024-09-18T14:13:20.516 INFO:tasks.rados.rados.0.smithi084.stdout:2981: left oid 20 (ObjNum 1014 snap 325 seq_num 1014) 2024-09-18T14:13:20.516 INFO:tasks.rados.rados.0.smithi084.stdout:2986: finishing rollback tid 0 to smithi08434663-27 2024-09-18T14:13:20.516 INFO:tasks.rados.rados.0.smithi084.stdout:2983: expect (ObjNum 981 snap 312 seq_num 981) 2024-09-18T14:13:20.596 INFO:tasks.rados.rados.0.smithi084.stdout:2986: finishing rollback tid 1 to smithi08434663-27 2024-09-18T14:13:20.596 INFO:tasks.rados.rados.0.smithi084.stdout:2986: finishing rollback tid 2 to smithi08434663-27 2024-09-18T14:13:20.596 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1239 (ObjNum 894 snap 278 seq_num 894) dirty exists 2024-09-18T14:13:20.596 INFO:tasks.rados.rados.0.smithi084.stdout:2985: finishing rollback tid 0 to smithi08434663-46 2024-09-18T14:13:20.597 INFO:tasks.rados.rados.0.smithi084.stdout:2985: finishing rollback tid 1 to smithi08434663-46 2024-09-18T14:13:20.597 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1460 (ObjNum 129 snap 0 seq_num 2494608984) dirty dne 2024-09-18T14:13:20.597 INFO:tasks.rados.rados.0.smithi084.stdout:2982: finishing copy_from racing read to smithi08434663-38 2024-09-18T14:13:20.597 INFO:tasks.rados.rados.0.smithi084.stdout:2982: finishing copy_from to smithi08434663-38 2024-09-18T14:13:20.597 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 1265 (ObjNum 957 snap 304 seq_num 957) dirty exists 2024-09-18T14:13:21.481 INFO:tasks.rados.rados.0.smithi084.stdout:2981: done (6 left) 2024-09-18T14:13:21.482 INFO:tasks.rados.rados.0.smithi084.stdout:2982: done (5 left) 2024-09-18T14:13:21.482 INFO:tasks.rados.rados.0.smithi084.stdout:2983: done (4 left) 2024-09-18T14:13:21.482 INFO:tasks.rados.rados.0.smithi084.stdout:2984: done (3 left) 2024-09-18T14:13:21.482 INFO:tasks.rados.rados.0.smithi084.stdout:2985: done (2 left) 2024-09-18T14:13:21.482 INFO:tasks.rados.rados.0.smithi084.stdout:2986: done (1 left) 2024-09-18T14:13:21.482 INFO:tasks.rados.rados.0.smithi084.stdout:2987: done (0 left) 2024-09-18T14:13:21.482 INFO:tasks.rados.rados.0.smithi084.stdout:2988: write (excl) oid 16 current snap is 326 2024-09-18T14:13:21.482 INFO:tasks.rados.rados.0.smithi084.stdout:2988: seq_num 1015 ranges {659105=652684,1862282=717015} 2024-09-18T14:13:21.488 INFO:tasks.rados.rados.0.smithi084.stdout:2988: writing smithi08434663-16 from 659105 to 1311789 tid 1 2024-09-18T14:13:21.495 INFO:tasks.rados.rados.0.smithi084.stdout:2988: writing smithi08434663-16 from 1862282 to 2579297 tid 2 2024-09-18T14:13:21.495 INFO:tasks.rados.rados.0.smithi084.stdout:2989: delete oid 15 current snap is 326 2024-09-18T14:13:21.495 INFO:tasks.rados.rados.0.smithi084.stdout:2988: finishing write tid 1 to smithi08434663-16 2024-09-18T14:13:21.501 INFO:tasks.rados.rados.0.smithi084.stdout:2988: finishing write tid 2 to smithi08434663-16 2024-09-18T14:13:21.504 INFO:tasks.rados.rados.0.smithi084.stdout:2988: finishing write tid 3 to smithi08434663-16 2024-09-18T14:13:21.504 INFO:tasks.rados.rados.0.smithi084.stdout:2988: finishing write tid 4 to smithi08434663-16 2024-09-18T14:13:21.504 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1463 (ObjNum 1015 snap 326 seq_num 1015) dirty exists 2024-09-18T14:13:21.505 INFO:tasks.rados.rados.0.smithi084.stdout:2988: left oid 16 (ObjNum 1015 snap 326 seq_num 1015) 2024-09-18T14:13:21.505 INFO:tasks.rados.rados.0.smithi084.stdout:2988: done (1 left) 2024-09-18T14:13:21.505 INFO:tasks.rados.rados.0.smithi084.stdout:2989: done (0 left) 2024-09-18T14:13:21.505 INFO:tasks.rados.rados.0.smithi084.stdout:2990: rollback oid 14 current snap is 326 2024-09-18T14:13:21.505 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 14 to 325 2024-09-18T14:13:21.505 INFO:tasks.rados.rados.0.smithi084.stdout:2991: write (excl) oid 29 current snap is 326 2024-09-18T14:13:21.505 INFO:tasks.rados.rados.0.smithi084.stdout:2991: seq_num 1017 ranges {692719=403182,1579246=720684,2828713=264006} 2024-09-18T14:13:21.510 INFO:tasks.rados.rados.0.smithi084.stdout:2991: writing smithi08434663-29 from 692719 to 1095901 tid 1 2024-09-18T14:13:21.517 INFO:tasks.rados.rados.0.smithi084.stdout:2991: writing smithi08434663-29 from 1579246 to 2299930 tid 2 2024-09-18T14:13:21.521 INFO:tasks.rados.rados.0.smithi084.stdout:2991: writing smithi08434663-29 from 2828713 to 3092719 tid 3 2024-09-18T14:13:21.521 INFO:tasks.rados.rados.0.smithi084.stdout:2992: write (excl) oid 21 current snap is 326 2024-09-18T14:13:21.521 INFO:tasks.rados.rados.0.smithi084.stdout:2992: seq_num 1018 ranges {709526=478431,1637728=464342,2856136=253390} 2024-09-18T14:13:21.527 INFO:tasks.rados.rados.0.smithi084.stdout:2992: writing smithi08434663-21 from 709526 to 1187957 tid 1 2024-09-18T14:13:21.533 INFO:tasks.rados.rados.0.smithi084.stdout:2992: writing smithi08434663-21 from 1637728 to 2102070 tid 2 2024-09-18T14:13:21.539 INFO:tasks.rados.rados.0.smithi084.stdout:2992: writing smithi08434663-21 from 2856136 to 3109526 tid 3 2024-09-18T14:13:21.539 INFO:tasks.rados.rados.0.smithi084.stdout:2993: read oid 19 snap -1 2024-09-18T14:13:21.539 INFO:tasks.rados.rados.0.smithi084.stdout:2993: expect (ObjNum 955 snap 304 seq_num 955) 2024-09-18T14:13:21.539 INFO:tasks.rados.rados.0.smithi084.stdout:2994: write oid 32 current snap is 326 2024-09-18T14:13:21.539 INFO:tasks.rados.rados.0.smithi084.stdout:2990: finishing rollback tid 1 to smithi08434663-14 2024-09-18T14:13:21.539 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-18T14:13:21.539 INFO:tasks.rados.rados.0.smithi084.stdout:2991: finishing write tid 1 to smithi08434663-29 2024-09-18T14:13:21.539 INFO:tasks.rados.rados.0.smithi084.stdout:2991: finishing write tid 2 to smithi08434663-29 2024-09-18T14:13:21.540 INFO:tasks.rados.rados.0.smithi084.stdout:2991: finishing write tid 3 to smithi08434663-29 2024-09-18T14:13:21.540 INFO:tasks.rados.rados.0.smithi084.stdout:2991: finishing write tid 4 to smithi08434663-29 2024-09-18T14:13:21.540 INFO:tasks.rados.rados.0.smithi084.stdout:2994: seq_num 1019 ranges {726333=670033,2096210=724353} 2024-09-18T14:13:21.546 INFO:tasks.rados.rados.0.smithi084.stdout:2994: writing smithi08434663-32 from 726333 to 1396366 tid 1 2024-09-18T14:13:21.555 INFO:tasks.rados.rados.0.smithi084.stdout:2994: writing smithi08434663-32 from 2096210 to 2820563 tid 2 2024-09-18T14:13:21.555 INFO:tasks.rados.rados.0.smithi084.stdout:2990: done (4 left) 2024-09-18T14:13:21.555 INFO:tasks.rados.rados.0.smithi084.stdout:2995: read oid 23 snap -1 2024-09-18T14:13:21.555 INFO:tasks.rados.rados.0.smithi084.stdout:2995: expect (ObjNum 1012 snap 325 seq_num 1012) 2024-09-18T14:13:21.555 INFO:tasks.rados.rados.0.smithi084.stdout:2996: copy_from oid 44 from oid 22 current snap is 326 2024-09-18T14:13:21.555 INFO:tasks.rados.rados.0.smithi084.stdout:2991: finishing write tid 5 to smithi08434663-29 2024-09-18T14:13:21.555 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 1523 (ObjNum 1017 snap 326 seq_num 1017) dirty exists 2024-09-18T14:13:21.555 INFO:tasks.rados.rados.0.smithi084.stdout:2991: left oid 29 (ObjNum 1017 snap 326 seq_num 1017) 2024-09-18T14:13:21.555 INFO:tasks.rados.rados.0.smithi084.stdout:2992: finishing write tid 1 to smithi08434663-21 2024-09-18T14:13:21.556 INFO:tasks.rados.rados.0.smithi084.stdout:2992: finishing write tid 2 to smithi08434663-21 2024-09-18T14:13:21.556 INFO:tasks.rados.rados.0.smithi084.stdout:2992: finishing write tid 3 to smithi08434663-21 2024-09-18T14:13:21.556 INFO:tasks.rados.rados.0.smithi084.stdout:2992: finishing write tid 4 to smithi08434663-21 2024-09-18T14:13:21.556 INFO:tasks.rados.rados.0.smithi084.stdout:2992: finishing write tid 5 to smithi08434663-21 2024-09-18T14:13:21.556 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 1264 (ObjNum 1018 snap 326 seq_num 1018) dirty exists 2024-09-18T14:13:21.556 INFO:tasks.rados.rados.0.smithi084.stdout:2992: left oid 21 (ObjNum 1018 snap 326 seq_num 1018) 2024-09-18T14:13:21.556 INFO:tasks.rados.rados.0.smithi084.stdout:2993: expect (ObjNum 955 snap 304 seq_num 955) 2024-09-18T14:13:21.677 INFO:tasks.rados.rados.0.smithi084.stdout:2994: finishing write tid 1 to smithi08434663-32 2024-09-18T14:13:21.677 INFO:tasks.rados.rados.0.smithi084.stdout:2994: finishing write tid 2 to smithi08434663-32 2024-09-18T14:13:21.678 INFO:tasks.rados.rados.0.smithi084.stdout:2994: finishing write tid 3 to smithi08434663-32 2024-09-18T14:13:21.678 INFO:tasks.rados.rados.0.smithi084.stdout:2994: finishing write tid 4 to smithi08434663-32 2024-09-18T14:13:21.678 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 1599 (ObjNum 1019 snap 326 seq_num 1019) dirty exists 2024-09-18T14:13:21.678 INFO:tasks.rados.rados.0.smithi084.stdout:2994: left oid 32 (ObjNum 1019 snap 326 seq_num 1019) 2024-09-18T14:13:21.678 INFO:tasks.rados.rados.0.smithi084.stdout:2995: expect (ObjNum 1012 snap 325 seq_num 1012) 2024-09-18T14:13:21.776 INFO:tasks.rados.rados.0.smithi084.stdout:2996: finishing copy_from racing read to smithi08434663-44 2024-09-18T14:13:21.776 INFO:tasks.rados.rados.0.smithi084.stdout:2996: finishing copy_from to smithi08434663-44 2024-09-18T14:13:21.776 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 1040 (ObjNum 991 snap 321 seq_num 991) dirty exists 2024-09-18T14:13:21.776 INFO:tasks.rados.rados.0.smithi084.stdout:2991: done (5 left) 2024-09-18T14:13:21.776 INFO:tasks.rados.rados.0.smithi084.stdout:2992: done (4 left) 2024-09-18T14:13:21.776 INFO:tasks.rados.rados.0.smithi084.stdout:2993: done (3 left) 2024-09-18T14:13:21.776 INFO:tasks.rados.rados.0.smithi084.stdout:2994: done (2 left) 2024-09-18T14:13:21.776 INFO:tasks.rados.rados.0.smithi084.stdout:2995: done (1 left) 2024-09-18T14:13:21.777 INFO:tasks.rados.rados.0.smithi084.stdout:2996: done (0 left) 2024-09-18T14:13:21.777 INFO:tasks.rados.rados.0.smithi084.stdout:2997: write (excl) oid 49 current snap is 326 2024-09-18T14:13:21.777 INFO:tasks.rados.rados.0.smithi084.stdout:2997: seq_num 1020 ranges {743140=745282,2154692=468011} 2024-09-18T14:13:21.783 INFO:tasks.rados.rados.0.smithi084.stdout:2997: writing smithi08434663-49 from 743140 to 1488422 tid 1 2024-09-18T14:13:21.790 INFO:tasks.rados.rados.0.smithi084.stdout:2997: writing smithi08434663-49 from 2154692 to 2622703 tid 2 2024-09-18T14:13:21.790 INFO:tasks.rados.rados.0.smithi084.stdout:2998: write (excl) oid 30 current snap is 326 2024-09-18T14:13:21.790 INFO:tasks.rados.rados.0.smithi084.stdout:2998: seq_num 1021 ranges {759947=420531,1813174=728022} 2024-09-18T14:13:21.796 INFO:tasks.rados.rados.0.smithi084.stdout:2998: writing smithi08434663-30 from 759947 to 1180478 tid 1 2024-09-18T14:13:21.804 INFO:tasks.rados.rados.0.smithi084.stdout:2998: writing smithi08434663-30 from 1813174 to 2541196 tid 2 2024-09-18T14:13:21.804 INFO:tasks.rados.rados.0.smithi084.stdout:2999: delete oid 13 current snap is 326 2024-09-18T14:13:21.804 INFO:tasks.rados.rados.0.smithi084.stdout:2997: finishing write tid 1 to smithi08434663-49 2024-09-18T14:13:21.804 INFO:tasks.rados.rados.0.smithi084.stdout:2997: finishing write tid 2 to smithi08434663-49 2024-09-18T14:13:21.805 INFO:tasks.rados.rados.0.smithi084.stdout:2997: finishing write tid 3 to smithi08434663-49 2024-09-18T14:13:21.805 INFO:tasks.rados.rados.0.smithi084.stdout:2997: finishing write tid 4 to smithi08434663-49 2024-09-18T14:13:21.805 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 1321 (ObjNum 1020 snap 326 seq_num 1020) dirty exists 2024-09-18T14:13:21.805 INFO:tasks.rados.rados.0.smithi084.stdout:2997: left oid 49 (ObjNum 1020 snap 326 seq_num 1020) 2024-09-18T14:13:21.805 INFO:tasks.rados.rados.0.smithi084.stdout:2998: finishing write tid 1 to smithi08434663-30 2024-09-18T14:13:21.806 INFO:tasks.rados.rados.0.smithi084.stdout:2997: done (2 left) 2024-09-18T14:13:21.806 INFO:tasks.rados.rados.0.smithi084.stdout:2999: done (1 left) 2024-09-18T14:13:21.806 INFO:tasks.rados.rados.0.smithi084.stdout:3000: read oid 20 snap -1 2024-09-18T14:13:21.806 INFO:tasks.rados.rados.0.smithi084.stdout:3000: expect (ObjNum 1014 snap 325 seq_num 1014) 2024-09-18T14:13:21.806 INFO:tasks.rados.rados.0.smithi084.stdout:3001: read oid 48 snap -1 2024-09-18T14:13:21.807 INFO:tasks.rados.rados.0.smithi084.stdout:3001: expect (ObjNum 955 snap 304 seq_num 955) 2024-09-18T14:13:21.807 INFO:tasks.rados.rados.0.smithi084.stdout:3002: snap_create 2024-09-18T14:13:21.809 INFO:tasks.rados.rados.0.smithi084.stdout:2998: finishing write tid 2 to smithi08434663-30 2024-09-18T14:13:21.809 INFO:tasks.rados.rados.0.smithi084.stdout:2998: finishing write tid 3 to smithi08434663-30 2024-09-18T14:13:21.809 INFO:tasks.rados.rados.0.smithi084.stdout:2998: finishing write tid 4 to smithi08434663-30 2024-09-18T14:13:21.809 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 1263 (ObjNum 1021 snap 326 seq_num 1021) dirty exists 2024-09-18T14:13:21.809 INFO:tasks.rados.rados.0.smithi084.stdout:2998: left oid 30 (ObjNum 1021 snap 326 seq_num 1021) 2024-09-18T14:13:21.811 INFO:tasks.rados.rados.0.smithi084.stdout:3001: expect (ObjNum 955 snap 304 seq_num 955) 2024-09-18T14:13:21.933 INFO:tasks.rados.rados.0.smithi084.stdout:3000: expect (ObjNum 1014 snap 325 seq_num 1014) 2024-09-18T14:13:22.493 INFO:tasks.rados.rados.0.smithi084.stdout:2998: done (3 left) 2024-09-18T14:13:22.494 INFO:tasks.rados.rados.0.smithi084.stdout:3000: done (2 left) 2024-09-18T14:13:22.494 INFO:tasks.rados.rados.0.smithi084.stdout:3001: done (1 left) 2024-09-18T14:13:22.494 INFO:tasks.rados.rados.0.smithi084.stdout:3002: done (0 left) 2024-09-18T14:13:22.494 INFO:tasks.rados.rados.0.smithi084.stdout:3003: copy_from oid 24 from oid 16 current snap is 327 2024-09-18T14:13:22.494 INFO:tasks.rados.rados.0.smithi084.stdout:3004: delete oid 15 current snap is 327 2024-09-18T14:13:22.499 INFO:tasks.rados.rados.0.smithi084.stdout:3004: done (1 left) 2024-09-18T14:13:22.499 INFO:tasks.rados.rados.0.smithi084.stdout:3005: delete oid 2 current snap is 327 2024-09-18T14:13:22.503 INFO:tasks.rados.rados.0.smithi084.stdout:3005: done (1 left) 2024-09-18T14:13:22.503 INFO:tasks.rados.rados.0.smithi084.stdout:3006: snap_create 2024-09-18T14:13:22.512 INFO:tasks.rados.rados.0.smithi084.stdout:3003: finishing copy_from racing read to smithi08434663-24 2024-09-18T14:13:22.512 INFO:tasks.rados.rados.0.smithi084.stdout:3003: finishing copy_from to smithi08434663-24 2024-09-18T14:13:22.512 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 1463 (ObjNum 1015 snap 326 seq_num 1015) dirty exists 2024-09-18T14:13:23.465 INFO:tasks.rados.rados.0.smithi084.stdout:3003: done (1 left) 2024-09-18T14:13:23.465 INFO:tasks.rados.rados.0.smithi084.stdout:3006: done (0 left) 2024-09-18T14:13:23.465 INFO:tasks.rados.rados.0.smithi084.stdout:3007: rollback oid 27 current snap is 328 2024-09-18T14:13:23.466 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 27 to 318 2024-09-18T14:13:23.466 INFO:tasks.rados.rados.0.smithi084.stdout:3008: read oid 5 snap -1 2024-09-18T14:13:23.466 INFO:tasks.rados.rados.0.smithi084.stdout:3008: expect deleted 2024-09-18T14:13:23.466 INFO:tasks.rados.rados.0.smithi084.stdout:3009: write oid 23 current snap is 328 2024-09-18T14:13:23.466 INFO:tasks.rados.rados.0.smithi084.stdout:3009: seq_num 1025 ranges {427175=721527,1930749=735360} 2024-09-18T14:13:23.471 INFO:tasks.rados.rados.0.smithi084.stdout:3009: writing smithi08434663-23 from 427175 to 1148702 tid 1 2024-09-18T14:13:23.480 INFO:tasks.rados.rados.0.smithi084.stdout:3009: writing smithi08434663-23 from 1930749 to 2666109 tid 2 2024-09-18T14:13:23.480 INFO:tasks.rados.rados.0.smithi084.stdout:3010: read oid 16 snap -1 2024-09-18T14:13:23.481 INFO:tasks.rados.rados.0.smithi084.stdout:3010: expect (ObjNum 1015 snap 326 seq_num 1015) 2024-09-18T14:13:23.481 INFO:tasks.rados.rados.0.smithi084.stdout:3011: copy_from oid 8 from oid 31 current snap is 328 2024-09-18T14:13:23.481 INFO:tasks.rados.rados.0.smithi084.stdout:3007: finishing rollback tid 0 to smithi08434663-27 2024-09-18T14:13:23.481 INFO:tasks.rados.rados.0.smithi084.stdout:3007: finishing rollback tid 1 to smithi08434663-27 2024-09-18T14:13:23.481 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1241 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:13:23.481 INFO:tasks.rados.rados.0.smithi084.stdout:3007: done (4 left) 2024-09-18T14:13:23.481 INFO:tasks.rados.rados.0.smithi084.stdout:3008: done (3 left) 2024-09-18T14:13:23.481 INFO:tasks.rados.rados.0.smithi084.stdout:3012: copy_from oid 20 from oid 41 current snap is 328 2024-09-18T14:13:23.481 INFO:tasks.rados.rados.0.smithi084.stdout:3013: snap_create 2024-09-18T14:13:23.484 INFO:tasks.rados.rados.0.smithi084.stdout:3010: expect (ObjNum 1015 snap 326 seq_num 1015) 2024-09-18T14:13:23.649 INFO:tasks.rados.rados.0.smithi084.stdout:3009: finishing write tid 1 to smithi08434663-23 2024-09-18T14:13:23.724 INFO:tasks.rados.rados.0.smithi084.stdout:3009: finishing write tid 2 to smithi08434663-23 2024-09-18T14:13:23.725 INFO:tasks.rados.rados.0.smithi084.stdout:3009: finishing write tid 3 to smithi08434663-23 2024-09-18T14:13:23.725 INFO:tasks.rados.rados.0.smithi084.stdout:3009: finishing write tid 4 to smithi08434663-23 2024-09-18T14:13:23.725 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1016 (ObjNum 1025 snap 328 seq_num 1025) dirty exists 2024-09-18T14:13:23.725 INFO:tasks.rados.rados.0.smithi084.stdout:3009: left oid 23 (ObjNum 1025 snap 328 seq_num 1025) 2024-09-18T14:13:23.725 INFO:tasks.rados.rados.0.smithi084.stdout:3012: finishing copy_from racing read to smithi08434663-20 2024-09-18T14:13:23.725 INFO:tasks.rados.rados.0.smithi084.stdout:3011: finishing copy_from racing read to smithi08434663-8 2024-09-18T14:13:23.725 INFO:tasks.rados.rados.0.smithi084.stdout:3011: finishing copy_from to smithi08434663-8 2024-09-18T14:13:23.725 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 600 (ObjNum 981 snap 312 seq_num 981) dirty exists 2024-09-18T14:13:23.725 INFO:tasks.rados.rados.0.smithi084.stdout:3012: finishing copy_from to smithi08434663-20 2024-09-18T14:13:23.725 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 1391 (ObjNum 1011 snap 325 seq_num 1011) dirty exists 2024-09-18T14:13:24.474 INFO:tasks.rados.rados.0.smithi084.stdout:3009: done (4 left) 2024-09-18T14:13:24.474 INFO:tasks.rados.rados.0.smithi084.stdout:3010: done (3 left) 2024-09-18T14:13:24.474 INFO:tasks.rados.rados.0.smithi084.stdout:3011: done (2 left) 2024-09-18T14:13:24.474 INFO:tasks.rados.rados.0.smithi084.stdout:3012: done (1 left) 2024-09-18T14:13:24.474 INFO:tasks.rados.rados.0.smithi084.stdout:3013: done (0 left) 2024-09-18T14:13:24.474 INFO:tasks.rados.rados.0.smithi084.stdout:3014: snap_create 2024-09-18T14:13:25.476 INFO:tasks.rados.rados.0.smithi084.stdout:3014: done (0 left) 2024-09-18T14:13:25.476 INFO:tasks.rados.rados.0.smithi084.stdout:3015: snap_remove snap 306 2024-09-18T14:13:26.482 INFO:tasks.rados.rados.0.smithi084.stdout:3015: done (0 left) 2024-09-18T14:13:26.482 INFO:tasks.rados.rados.0.smithi084.stdout:3016: write (excl) oid 25 current snap is 330 2024-09-18T14:13:26.482 INFO:tasks.rados.rados.0.smithi084.stdout:3016: seq_num 1026 ranges {443982=796776,1989231=479018,2959167=284815} 2024-09-18T14:13:26.489 INFO:tasks.rados.rados.0.smithi084.stdout:3016: writing smithi08434663-25 from 443982 to 1240758 tid 1 2024-09-18T14:13:26.496 INFO:tasks.rados.rados.0.smithi084.stdout:3016: writing smithi08434663-25 from 1989231 to 2468249 tid 2 2024-09-18T14:13:26.501 INFO:tasks.rados.rados.0.smithi084.stdout:3016: writing smithi08434663-25 from 2959167 to 3243982 tid 3 2024-09-18T14:13:26.501 INFO:tasks.rados.rados.0.smithi084.stdout:3017: write (excl) oid 24 current snap is 330 2024-09-18T14:13:26.501 INFO:tasks.rados.rados.0.smithi084.stdout:3016: finishing write tid 1 to smithi08434663-25 2024-09-18T14:13:26.501 INFO:tasks.rados.rados.0.smithi084.stdout:3017: seq_num 1027 ranges {460789=588378,1764066=739029,3219296=41493} 2024-09-18T14:13:26.506 INFO:tasks.rados.rados.0.smithi084.stdout:3017: writing smithi08434663-24 from 460789 to 1049167 tid 1 2024-09-18T14:13:26.514 INFO:tasks.rados.rados.0.smithi084.stdout:3017: writing smithi08434663-24 from 1764066 to 2503095 tid 2 2024-09-18T14:13:26.519 INFO:tasks.rados.rados.0.smithi084.stdout:3017: writing smithi08434663-24 from 3219296 to 3260789 tid 3 2024-09-18T14:13:26.519 INFO:tasks.rados.rados.0.smithi084.stdout:3018: write (excl) oid 26 current snap is 330 2024-09-18T14:13:26.519 INFO:tasks.rados.rados.0.smithi084.stdout:3018: seq_num 1028 ranges {477596=663627,1706195=482687,2614013=567041} 2024-09-18T14:13:26.525 INFO:tasks.rados.rados.0.smithi084.stdout:3018: writing smithi08434663-26 from 477596 to 1141223 tid 1 2024-09-18T14:13:26.530 INFO:tasks.rados.rados.0.smithi084.stdout:3018: writing smithi08434663-26 from 1706195 to 2188882 tid 2 2024-09-18T14:13:26.536 INFO:tasks.rados.rados.0.smithi084.stdout:3018: writing smithi08434663-26 from 2614013 to 3181054 tid 3 2024-09-18T14:13:26.537 INFO:tasks.rados.rados.0.smithi084.stdout:3019: write oid 42 current snap is 330 2024-09-18T14:13:26.537 INFO:tasks.rados.rados.0.smithi084.stdout:3016: finishing write tid 2 to smithi08434663-25 2024-09-18T14:13:26.537 INFO:tasks.rados.rados.0.smithi084.stdout:3019: seq_num 1029 ranges {494403=738876,1764677=626345,3041436=252967} 2024-09-18T14:13:26.543 INFO:tasks.rados.rados.0.smithi084.stdout:3019: writing smithi08434663-42 from 494403 to 1233279 tid 1 2024-09-18T14:13:26.549 INFO:tasks.rados.rados.0.smithi084.stdout:3019: writing smithi08434663-42 from 1764677 to 2391022 tid 2 2024-09-18T14:13:26.555 INFO:tasks.rados.rados.0.smithi084.stdout:3019: writing smithi08434663-42 from 3041436 to 3294403 tid 3 2024-09-18T14:13:26.555 INFO:tasks.rados.rados.0.smithi084.stdout:3020: read oid 21 snap -1 2024-09-18T14:13:26.555 INFO:tasks.rados.rados.0.smithi084.stdout:3020: expect (ObjNum 1018 snap 326 seq_num 1018) 2024-09-18T14:13:26.555 INFO:tasks.rados.rados.0.smithi084.stdout:3021: read oid 27 snap -1 2024-09-18T14:13:26.555 INFO:tasks.rados.rados.0.smithi084.stdout:3021: expect deleted 2024-09-18T14:13:26.555 INFO:tasks.rados.rados.0.smithi084.stdout:3022: copy_from oid 36 from oid 2 current snap is 330 2024-09-18T14:13:26.555 INFO:tasks.rados.rados.0.smithi084.stdout:3016: finishing write tid 3 to smithi08434663-25 2024-09-18T14:13:26.555 INFO:tasks.rados.rados.0.smithi084.stdout:3016: finishing write tid 4 to smithi08434663-25 2024-09-18T14:13:26.555 INFO:tasks.rados.rados.0.smithi084.stdout:3016: finishing write tid 5 to smithi08434663-25 2024-09-18T14:13:26.555 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 1268 (ObjNum 1026 snap 330 seq_num 1026) dirty exists 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:3016: left oid 25 (ObjNum 1026 snap 330 seq_num 1026) 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:3017: finishing write tid 1 to smithi08434663-24 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:3017: finishing write tid 2 to smithi08434663-24 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:3017: finishing write tid 3 to smithi08434663-24 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:3017: finishing write tid 4 to smithi08434663-24 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:3017: finishing write tid 5 to smithi08434663-24 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 1467 (ObjNum 1027 snap 330 seq_num 1027) dirty exists 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:3017: left oid 24 (ObjNum 1027 snap 330 seq_num 1027) 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:3018: finishing write tid 1 to smithi08434663-26 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:3018: finishing write tid 2 to smithi08434663-26 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:3018: finishing write tid 3 to smithi08434663-26 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:3018: finishing write tid 4 to smithi08434663-26 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:3018: finishing write tid 5 to smithi08434663-26 2024-09-18T14:13:26.556 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 1245 (ObjNum 1028 snap 330 seq_num 1028) dirty exists 2024-09-18T14:13:26.557 INFO:tasks.rados.rados.0.smithi084.stdout:3018: left oid 26 (ObjNum 1028 snap 330 seq_num 1028) 2024-09-18T14:13:26.557 INFO:tasks.rados.rados.0.smithi084.stdout:3019: finishing write tid 1 to smithi08434663-42 2024-09-18T14:13:26.557 INFO:tasks.rados.rados.0.smithi084.stdout:3019: finishing write tid 2 to smithi08434663-42 2024-09-18T14:13:26.557 INFO:tasks.rados.rados.0.smithi084.stdout:3016: done (6 left) 2024-09-18T14:13:26.557 INFO:tasks.rados.rados.0.smithi084.stdout:3017: done (5 left) 2024-09-18T14:13:26.557 INFO:tasks.rados.rados.0.smithi084.stdout:3018: done (4 left) 2024-09-18T14:13:26.557 INFO:tasks.rados.rados.0.smithi084.stdout:3023: rollback oid 23 current snap is 330 2024-09-18T14:13:26.557 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 23 to 310 2024-09-18T14:13:26.557 INFO:tasks.rados.rados.0.smithi084.stdout:3024: copy_from oid 22 from oid 33 current snap is 330 2024-09-18T14:13:26.557 INFO:tasks.rados.rados.0.smithi084.stdout:3025: snap_remove snap 309 2024-09-18T14:13:27.487 INFO:tasks.rados.rados.0.smithi084.stdout:3025: done (6 left) 2024-09-18T14:13:27.487 INFO:tasks.rados.rados.0.smithi084.stdout:3026: write oid 26 current snap is 330 2024-09-18T14:13:27.487 INFO:tasks.rados.rados.0.smithi084.stdout:3026: seq_num 1030 ranges {511210=414125,1423159=486356,2668859=589585} 2024-09-18T14:13:27.492 INFO:tasks.rados.rados.0.smithi084.stdout:3026: writing smithi08434663-26 from 511210 to 925335 tid 1 2024-09-18T14:13:27.497 INFO:tasks.rados.rados.0.smithi084.stdout:3026: writing smithi08434663-26 from 1423159 to 1909515 tid 2 2024-09-18T14:13:27.506 INFO:tasks.rados.rados.0.smithi084.stdout:3026: writing smithi08434663-26 from 2668859 to 3258444 tid 3 2024-09-18T14:13:27.506 INFO:tasks.rados.rados.0.smithi084.stdout:3027: rollback oid 13 current snap is 330 2024-09-18T14:13:27.506 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 13 to 313 2024-09-18T14:13:27.506 INFO:tasks.rados.rados.0.smithi084.stdout:3024: finishing copy_from to smithi08434663-22 2024-09-18T14:13:27.506 INFO:tasks.rados.rados.0.smithi084.stdout:3024: got expected ENOENT (src dne) 2024-09-18T14:13:27.506 INFO:tasks.rados.rados.0.smithi084.stdout:3022: finishing copy_from to smithi08434663-36 2024-09-18T14:13:27.506 INFO:tasks.rados.rados.0.smithi084.stdout:3022: got expected ENOENT (src dne) 2024-09-18T14:13:27.506 INFO:tasks.rados.rados.0.smithi084.stdout:3024: finishing copy_from racing read to smithi08434663-22 2024-09-18T14:13:27.507 INFO:tasks.rados.rados.0.smithi084.stdout:3022: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:13:27.507 INFO:tasks.rados.rados.0.smithi084.stdout:3019: finishing write tid 3 to smithi08434663-42 2024-09-18T14:13:27.507 INFO:tasks.rados.rados.0.smithi084.stdout:3019: finishing write tid 4 to smithi08434663-42 2024-09-18T14:13:27.507 INFO:tasks.rados.rados.0.smithi084.stdout:3021: done (7 left) 2024-09-18T14:13:27.507 INFO:tasks.rados.rados.0.smithi084.stdout:3022: done (6 left) 2024-09-18T14:13:27.507 INFO:tasks.rados.rados.0.smithi084.stdout:3024: done (5 left) 2024-09-18T14:13:27.507 INFO:tasks.rados.rados.0.smithi084.stdout:3028: snap_remove snap 327 2024-09-18T14:13:28.506 INFO:tasks.rados.rados.0.smithi084.stdout:3028: done (5 left) 2024-09-18T14:13:28.507 INFO:tasks.rados.rados.0.smithi084.stdout:3029: write oid 17 current snap is 330 2024-09-18T14:13:28.507 INFO:tasks.rados.rados.0.smithi084.stdout:3029: seq_num 1031 ranges {528017=489374,1481641=630014,2696282=400857} 2024-09-18T14:13:28.511 INFO:tasks.rados.rados.0.smithi084.stdout:3029: writing smithi08434663-17 from 528017 to 1017391 tid 1 2024-09-18T14:13:28.517 INFO:tasks.rados.rados.0.smithi084.stdout:3029: writing smithi08434663-17 from 1481641 to 2111655 tid 2 2024-09-18T14:13:28.523 INFO:tasks.rados.rados.0.smithi084.stdout:3029: writing smithi08434663-17 from 2696282 to 3097139 tid 3 2024-09-18T14:13:28.523 INFO:tasks.rados.rados.0.smithi084.stdout:3030: snap_create 2024-09-18T14:13:28.523 INFO:tasks.rados.rados.0.smithi084.stdout:3019: finishing write tid 5 to smithi08434663-42 2024-09-18T14:13:28.523 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1414 (ObjNum 1029 snap 330 seq_num 1029) dirty exists 2024-09-18T14:13:28.524 INFO:tasks.rados.rados.0.smithi084.stdout:3019: left oid 42 (ObjNum 1029 snap 330 seq_num 1029) 2024-09-18T14:13:28.524 INFO:tasks.rados.rados.0.smithi084.stdout:3023: finishing rollback tid 0 to smithi08434663-23 2024-09-18T14:13:28.524 INFO:tasks.rados.rados.0.smithi084.stdout:3023: finishing rollback tid 1 to smithi08434663-23 2024-09-18T14:13:28.524 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1018 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-18T14:13:28.524 INFO:tasks.rados.rados.0.smithi084.stdout:3020: expect (ObjNum 1018 snap 326 seq_num 1018) 2024-09-18T14:13:28.730 INFO:tasks.rados.rados.0.smithi084.stdout:3027: finishing rollback tid 1 to smithi08434663-13 2024-09-18T14:13:28.731 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 0 (ObjNum 129 snap 0 seq_num 2494604616) dirty dne 2024-09-18T14:13:28.731 INFO:tasks.rados.rados.0.smithi084.stdout:3026: finishing write tid 1 to smithi08434663-26 2024-09-18T14:13:28.731 INFO:tasks.rados.rados.0.smithi084.stdout:3026: finishing write tid 2 to smithi08434663-26 2024-09-18T14:13:28.731 INFO:tasks.rados.rados.0.smithi084.stdout:3026: finishing write tid 3 to smithi08434663-26 2024-09-18T14:13:28.731 INFO:tasks.rados.rados.0.smithi084.stdout:3026: finishing write tid 4 to smithi08434663-26 2024-09-18T14:13:28.731 INFO:tasks.rados.rados.0.smithi084.stdout:3026: finishing write tid 5 to smithi08434663-26 2024-09-18T14:13:28.731 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 1249 (ObjNum 1030 snap 330 seq_num 1030) dirty exists 2024-09-18T14:13:28.731 INFO:tasks.rados.rados.0.smithi084.stdout:3026: left oid 26 (ObjNum 1030 snap 330 seq_num 1030) 2024-09-18T14:13:28.731 INFO:tasks.rados.rados.0.smithi084.stdout:3029: finishing write tid 1 to smithi08434663-17 2024-09-18T14:13:28.731 INFO:tasks.rados.rados.0.smithi084.stdout:3029: finishing write tid 2 to smithi08434663-17 2024-09-18T14:13:28.731 INFO:tasks.rados.rados.0.smithi084.stdout:3029: finishing write tid 3 to smithi08434663-17 2024-09-18T14:13:28.731 INFO:tasks.rados.rados.0.smithi084.stdout:3029: finishing write tid 4 to smithi08434663-17 2024-09-18T14:13:28.732 INFO:tasks.rados.rados.0.smithi084.stdout:3029: finishing write tid 5 to smithi08434663-17 2024-09-18T14:13:28.732 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1527 (ObjNum 1031 snap 330 seq_num 1031) dirty exists 2024-09-18T14:13:28.732 INFO:tasks.rados.rados.0.smithi084.stdout:3029: left oid 17 (ObjNum 1031 snap 330 seq_num 1031) 2024-09-18T14:13:29.512 INFO:tasks.rados.rados.0.smithi084.stdout:3019: done (6 left) 2024-09-18T14:13:29.513 INFO:tasks.rados.rados.0.smithi084.stdout:3020: done (5 left) 2024-09-18T14:13:29.513 INFO:tasks.rados.rados.0.smithi084.stdout:3023: done (4 left) 2024-09-18T14:13:29.513 INFO:tasks.rados.rados.0.smithi084.stdout:3026: done (3 left) 2024-09-18T14:13:29.513 INFO:tasks.rados.rados.0.smithi084.stdout:3027: done (2 left) 2024-09-18T14:13:29.513 INFO:tasks.rados.rados.0.smithi084.stdout:3029: done (1 left) 2024-09-18T14:13:29.513 INFO:tasks.rados.rados.0.smithi084.stdout:3030: done (0 left) 2024-09-18T14:13:29.513 INFO:tasks.rados.rados.0.smithi084.stdout:3031: write oid 11 current snap is 331 2024-09-18T14:13:29.513 INFO:tasks.rados.rados.0.smithi084.stdout:3031: seq_num 1032 ranges {544824=564623,1823770=490025,3007352=337472} 2024-09-18T14:13:29.518 INFO:tasks.rados.rados.0.smithi084.stdout:3031: writing smithi08434663-11 from 544824 to 1109447 tid 1 2024-09-18T14:13:29.525 INFO:tasks.rados.rados.0.smithi084.stdout:3031: writing smithi08434663-11 from 1823770 to 2313795 tid 2 2024-09-18T14:13:29.532 INFO:tasks.rados.rados.0.smithi084.stdout:3031: writing smithi08434663-11 from 3007352 to 3344824 tid 3 2024-09-18T14:13:29.532 INFO:tasks.rados.rados.0.smithi084.stdout:3032: rollback oid 25 current snap is 331 2024-09-18T14:13:29.532 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 25 to 329 2024-09-18T14:13:29.532 INFO:tasks.rados.rados.0.smithi084.stdout:3031: finishing write tid 1 to smithi08434663-11 2024-09-18T14:13:29.532 INFO:tasks.rados.rados.0.smithi084.stdout:3031: finishing write tid 2 to smithi08434663-11 2024-09-18T14:13:29.532 INFO:tasks.rados.rados.0.smithi084.stdout:3033: write oid 32 current snap is 331 2024-09-18T14:13:29.532 INFO:tasks.rados.rados.0.smithi084.stdout:3033: seq_num 1033 ranges {561631=639872,1882252=633683,3034775=326856} 2024-09-18T14:13:29.538 INFO:tasks.rados.rados.0.smithi084.stdout:3033: writing smithi08434663-32 from 561631 to 1201503 tid 1 2024-09-18T14:13:29.545 INFO:tasks.rados.rados.0.smithi084.stdout:3033: writing smithi08434663-32 from 1882252 to 2515935 tid 2 2024-09-18T14:13:29.550 INFO:tasks.rados.rados.0.smithi084.stdout:3033: writing smithi08434663-32 from 3034775 to 3361631 tid 3 2024-09-18T14:13:29.550 INFO:tasks.rados.rados.0.smithi084.stdout:3034: snap_remove snap 292 2024-09-18T14:13:30.521 INFO:tasks.rados.rados.0.smithi084.stdout:3034: done (3 left) 2024-09-18T14:13:30.521 INFO:tasks.rados.rados.0.smithi084.stdout:3035: write oid 13 current snap is 331 2024-09-18T14:13:30.521 INFO:tasks.rados.rados.0.smithi084.stdout:3035: seq_num 1034 ranges {578438=431474,1657087=493694,2778551=599887} 2024-09-18T14:13:30.526 INFO:tasks.rados.rados.0.smithi084.stdout:3035: writing smithi08434663-13 from 578438 to 1009912 tid 1 2024-09-18T14:13:30.533 INFO:tasks.rados.rados.0.smithi084.stdout:3035: writing smithi08434663-13 from 1657087 to 2150781 tid 2 2024-09-18T14:13:30.540 INFO:tasks.rados.rados.0.smithi084.stdout:3035: writing smithi08434663-13 from 2778551 to 3378438 tid 3 2024-09-18T14:13:30.540 INFO:tasks.rados.rados.0.smithi084.stdout:3036: write oid 4 current snap is 331 2024-09-18T14:13:30.540 INFO:tasks.rados.rados.0.smithi084.stdout:3031: finishing write tid 3 to smithi08434663-11 2024-09-18T14:13:30.540 INFO:tasks.rados.rados.0.smithi084.stdout:3036: seq_num 1035 ranges {595245=506723,1715569=637352,2805974=562298} 2024-09-18T14:13:30.546 INFO:tasks.rados.rados.0.smithi084.stdout:3036: writing smithi08434663-4 from 595245 to 1101968 tid 1 2024-09-18T14:13:30.553 INFO:tasks.rados.rados.0.smithi084.stdout:3036: writing smithi08434663-4 from 1715569 to 2352921 tid 2 2024-09-18T14:13:30.558 INFO:tasks.rados.rados.0.smithi084.stdout:3036: writing smithi08434663-4 from 2805974 to 3368272 tid 3 2024-09-18T14:13:30.559 INFO:tasks.rados.rados.0.smithi084.stdout:3037: read oid 34 snap -1 2024-09-18T14:13:30.559 INFO:tasks.rados.rados.0.smithi084.stdout:3037: expect deleted 2024-09-18T14:13:30.559 INFO:tasks.rados.rados.0.smithi084.stdout:3038: read oid 20 snap -1 2024-09-18T14:13:30.559 INFO:tasks.rados.rados.0.smithi084.stdout:3038: expect (ObjNum 1011 snap 325 seq_num 1011) 2024-09-18T14:13:30.559 INFO:tasks.rados.rados.0.smithi084.stdout:3039: snap_remove snap 326 2024-09-18T14:13:31.522 INFO:tasks.rados.rados.0.smithi084.stdout:3039: done (7 left) 2024-09-18T14:13:31.522 INFO:tasks.rados.rados.0.smithi084.stdout:3040: copy_from oid 45 from oid 40 current snap is 331 2024-09-18T14:13:31.522 INFO:tasks.rados.rados.0.smithi084.stdout:3031: finishing write tid 4 to smithi08434663-11 2024-09-18T14:13:31.522 INFO:tasks.rados.rados.0.smithi084.stdout:3031: finishing write tid 5 to smithi08434663-11 2024-09-18T14:13:31.523 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1271 (ObjNum 1032 snap 331 seq_num 1032) dirty exists 2024-09-18T14:13:31.523 INFO:tasks.rados.rados.0.smithi084.stdout:3031: left oid 11 (ObjNum 1032 snap 331 seq_num 1032) 2024-09-18T14:13:31.523 INFO:tasks.rados.rados.0.smithi084.stdout:3032: finishing rollback tid 0 to smithi08434663-25 2024-09-18T14:13:31.523 INFO:tasks.rados.rados.0.smithi084.stdout:3032: finishing rollback tid 1 to smithi08434663-25 2024-09-18T14:13:31.523 INFO:tasks.rados.rados.0.smithi084.stdout:3032: finishing rollback tid 2 to smithi08434663-25 2024-09-18T14:13:31.523 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 1271 (ObjNum 998 snap 322 seq_num 998) dirty exists 2024-09-18T14:13:31.523 INFO:tasks.rados.rados.0.smithi084.stdout:3033: finishing write tid 1 to smithi08434663-32 2024-09-18T14:13:31.523 INFO:tasks.rados.rados.0.smithi084.stdout:3033: finishing write tid 2 to smithi08434663-32 2024-09-18T14:13:31.523 INFO:tasks.rados.rados.0.smithi084.stdout:3033: finishing write tid 3 to smithi08434663-32 2024-09-18T14:13:31.523 INFO:tasks.rados.rados.0.smithi084.stdout:3033: finishing write tid 4 to smithi08434663-32 2024-09-18T14:13:31.524 INFO:tasks.rados.rados.0.smithi084.stdout:3033: finishing write tid 5 to smithi08434663-32 2024-09-18T14:13:31.524 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 1622 (ObjNum 1033 snap 331 seq_num 1033) dirty exists 2024-09-18T14:13:31.524 INFO:tasks.rados.rados.0.smithi084.stdout:3033: left oid 32 (ObjNum 1033 snap 331 seq_num 1033) 2024-09-18T14:13:31.524 INFO:tasks.rados.rados.0.smithi084.stdout:3035: finishing write tid 1 to smithi08434663-13 2024-09-18T14:13:31.524 INFO:tasks.rados.rados.0.smithi084.stdout:3035: finishing write tid 2 to smithi08434663-13 2024-09-18T14:13:31.524 INFO:tasks.rados.rados.0.smithi084.stdout:3035: finishing write tid 3 to smithi08434663-13 2024-09-18T14:13:31.524 INFO:tasks.rados.rados.0.smithi084.stdout:3035: finishing write tid 4 to smithi08434663-13 2024-09-18T14:13:31.524 INFO:tasks.rados.rados.0.smithi084.stdout:3035: finishing write tid 5 to smithi08434663-13 2024-09-18T14:13:31.524 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 1531 (ObjNum 1034 snap 331 seq_num 1034) dirty exists 2024-09-18T14:13:31.524 INFO:tasks.rados.rados.0.smithi084.stdout:3035: left oid 13 (ObjNum 1034 snap 331 seq_num 1034) 2024-09-18T14:13:31.524 INFO:tasks.rados.rados.0.smithi084.stdout:3036: finishing write tid 1 to smithi08434663-4 2024-09-18T14:13:31.524 INFO:tasks.rados.rados.0.smithi084.stdout:3036: finishing write tid 2 to smithi08434663-4 2024-09-18T14:13:31.525 INFO:tasks.rados.rados.0.smithi084.stdout:3036: finishing write tid 3 to smithi08434663-4 2024-09-18T14:13:31.525 INFO:tasks.rados.rados.0.smithi084.stdout:3036: finishing write tid 4 to smithi08434663-4 2024-09-18T14:13:31.525 INFO:tasks.rados.rados.0.smithi084.stdout:3036: finishing write tid 5 to smithi08434663-4 2024-09-18T14:13:31.525 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1435 (ObjNum 1035 snap 331 seq_num 1035) dirty exists 2024-09-18T14:13:31.525 INFO:tasks.rados.rados.0.smithi084.stdout:3036: left oid 4 (ObjNum 1035 snap 331 seq_num 1035) 2024-09-18T14:13:31.525 INFO:tasks.rados.rados.0.smithi084.stdout:3038: expect (ObjNum 1011 snap 325 seq_num 1011) 2024-09-18T14:13:31.621 INFO:tasks.rados.rados.0.smithi084.stdout:3040: finishing copy_from racing read to smithi08434663-45 2024-09-18T14:13:31.621 INFO:tasks.rados.rados.0.smithi084.stdout:3040: finishing copy_from to smithi08434663-45 2024-09-18T14:13:31.621 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 1519 (ObjNum 971 snap 308 seq_num 971) dirty exists 2024-09-18T14:13:31.621 INFO:tasks.rados.rados.0.smithi084.stdout:3031: done (7 left) 2024-09-18T14:13:31.621 INFO:tasks.rados.rados.0.smithi084.stdout:3032: done (6 left) 2024-09-18T14:13:31.621 INFO:tasks.rados.rados.0.smithi084.stdout:3033: done (5 left) 2024-09-18T14:13:31.621 INFO:tasks.rados.rados.0.smithi084.stdout:3035: done (4 left) 2024-09-18T14:13:31.622 INFO:tasks.rados.rados.0.smithi084.stdout:3036: done (3 left) 2024-09-18T14:13:31.622 INFO:tasks.rados.rados.0.smithi084.stdout:3037: done (2 left) 2024-09-18T14:13:31.622 INFO:tasks.rados.rados.0.smithi084.stdout:3038: done (1 left) 2024-09-18T14:13:31.622 INFO:tasks.rados.rados.0.smithi084.stdout:3040: done (0 left) 2024-09-18T14:13:31.622 INFO:tasks.rados.rados.0.smithi084.stdout:3041: snap_remove snap 303 2024-09-18T14:13:32.540 INFO:tasks.rados.rados.0.smithi084.stdout:3041: done (0 left) 2024-09-18T14:13:32.540 INFO:tasks.rados.rados.0.smithi084.stdout:3042: snap_remove snap 325 2024-09-18T14:13:33.472 INFO:tasks.rados.rados.0.smithi084.stdout:3042: done (0 left) 2024-09-18T14:13:33.472 INFO:tasks.rados.rados.0.smithi084.stdout:3043: read oid 4 snap 330 2024-09-18T14:13:33.472 INFO:tasks.rados.rados.0.smithi084.stdout:3043: expect (ObjNum 1007 snap 324 seq_num 1007) 2024-09-18T14:13:33.472 INFO:tasks.rados.rados.0.smithi084.stdout:3044: read oid 20 snap 328 2024-09-18T14:13:33.472 INFO:tasks.rados.rados.0.smithi084.stdout:3044: expect (ObjNum 1011 snap 325 seq_num 1011) 2024-09-18T14:13:33.472 INFO:tasks.rados.rados.0.smithi084.stdout:3045: snap_create 2024-09-18T14:13:33.483 INFO:tasks.rados.rados.0.smithi084.stdout:3044: expect (ObjNum 1011 snap 325 seq_num 1011) 2024-09-18T14:13:33.582 INFO:tasks.rados.rados.0.smithi084.stdout:3043: expect (ObjNum 1007 snap 324 seq_num 1007) 2024-09-18T14:13:34.478 INFO:tasks.rados.rados.0.smithi084.stdout:3043: done (2 left) 2024-09-18T14:13:34.478 INFO:tasks.rados.rados.0.smithi084.stdout:3044: done (1 left) 2024-09-18T14:13:34.478 INFO:tasks.rados.rados.0.smithi084.stdout:3045: done (0 left) 2024-09-18T14:13:34.478 INFO:tasks.rados.rados.0.smithi084.stdout:3046: read oid 34 snap 324 2024-09-18T14:13:34.478 INFO:tasks.rados.rados.0.smithi084.stdout:3046: expect deleted 2024-09-18T14:13:34.479 INFO:tasks.rados.rados.0.smithi084.stdout:3047: rollback oid 7 current snap is 332 2024-09-18T14:13:34.479 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 7 to 319 2024-09-18T14:13:34.479 INFO:tasks.rados.rados.0.smithi084.stdout:3048: read oid 20 snap 324 2024-09-18T14:13:34.479 INFO:tasks.rados.rados.0.smithi084.stdout:3048: expect (ObjNum 908 snap 281 seq_num 908) 2024-09-18T14:13:34.479 INFO:tasks.rados.rados.0.smithi084.stdout:3049: write oid 42 current snap is 332 2024-09-18T14:13:34.479 INFO:tasks.rados.rados.0.smithi084.stdout:3049: seq_num 1036 ranges {612052=581972,1657698=497363,2717044=695008} 2024-09-18T14:13:34.484 INFO:tasks.rados.rados.0.smithi084.stdout:3049: writing smithi08434663-42 from 612052 to 1194024 tid 1 2024-09-18T14:13:34.489 INFO:tasks.rados.rados.0.smithi084.stdout:3049: writing smithi08434663-42 from 1657698 to 2155061 tid 2 2024-09-18T14:13:34.497 INFO:tasks.rados.rados.0.smithi084.stdout:3049: writing smithi08434663-42 from 2717044 to 3412052 tid 3 2024-09-18T14:13:34.497 INFO:tasks.rados.rados.0.smithi084.stdout:3050: snap_create 2024-09-18T14:13:34.497 INFO:tasks.rados.rados.0.smithi084.stdout:3047: finishing rollback tid 1 to smithi08434663-7 2024-09-18T14:13:34.497 INFO:tasks.rados.rados.0.smithi084.stdout:3047: finishing rollback tid 2 to smithi08434663-7 2024-09-18T14:13:34.497 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 1020 (ObjNum 976 snap 310 seq_num 976) dirty exists 2024-09-18T14:13:34.497 INFO:tasks.rados.rados.0.smithi084.stdout:3048: expect (ObjNum 908 snap 281 seq_num 908) 2024-09-18T14:13:34.676 INFO:tasks.rados.rados.0.smithi084.stdout:3049: finishing write tid 1 to smithi08434663-42 2024-09-18T14:13:34.676 INFO:tasks.rados.rados.0.smithi084.stdout:3049: finishing write tid 2 to smithi08434663-42 2024-09-18T14:13:34.676 INFO:tasks.rados.rados.0.smithi084.stdout:3049: finishing write tid 3 to smithi08434663-42 2024-09-18T14:13:34.676 INFO:tasks.rados.rados.0.smithi084.stdout:3049: finishing write tid 4 to smithi08434663-42 2024-09-18T14:13:34.676 INFO:tasks.rados.rados.0.smithi084.stdout:3049: finishing write tid 5 to smithi08434663-42 2024-09-18T14:13:34.676 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1523 (ObjNum 1036 snap 332 seq_num 1036) dirty exists 2024-09-18T14:13:34.676 INFO:tasks.rados.rados.0.smithi084.stdout:3049: left oid 42 (ObjNum 1036 snap 332 seq_num 1036) 2024-09-18T14:13:35.479 INFO:tasks.rados.rados.0.smithi084.stdout:3046: done (4 left) 2024-09-18T14:13:35.479 INFO:tasks.rados.rados.0.smithi084.stdout:3047: done (3 left) 2024-09-18T14:13:35.479 INFO:tasks.rados.rados.0.smithi084.stdout:3048: done (2 left) 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:3049: done (1 left) 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:3050: done (0 left) 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:3051: read oid 21 snap -1 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:3051: expect (ObjNum 1018 snap 326 seq_num 1018) 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:3052: rollback oid 13 current snap is 333 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 13 to 310 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:3053: rollback oid 6 current snap is 333 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 323 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:3054: rollback oid 34 current snap is 333 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 34 to 323 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:3055: rollback oid 47 current snap is 333 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 47 to 308 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:3056: write (excl) oid 33 current snap is 333 2024-09-18T14:13:35.480 INFO:tasks.rados.rados.0.smithi084.stdout:3056: seq_num 1037 ranges {628859=657221,1716180=641021,3144467=284392} 2024-09-18T14:13:35.486 INFO:tasks.rados.rados.0.smithi084.stdout:3056: writing smithi08434663-33 from 628859 to 1286080 tid 1 2024-09-18T14:13:35.491 INFO:tasks.rados.rados.0.smithi084.stdout:3056: writing smithi08434663-33 from 1716180 to 2357201 tid 2 2024-09-18T14:13:35.498 INFO:tasks.rados.rados.0.smithi084.stdout:3056: writing smithi08434663-33 from 3144467 to 3428859 tid 3 2024-09-18T14:13:35.498 INFO:tasks.rados.rados.0.smithi084.stdout:3057: delete oid 11 current snap is 333 2024-09-18T14:13:35.498 INFO:tasks.rados.rados.0.smithi084.stdout:3055: finishing rollback tid 1 to smithi08434663-47 2024-09-18T14:13:35.498 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 0 (ObjNum 65 snap 0 seq_num 2475021424) dirty dne 2024-09-18T14:13:35.498 INFO:tasks.rados.rados.0.smithi084.stdout:3051: expect (ObjNum 1018 snap 326 seq_num 1018) 2024-09-18T14:13:35.707 INFO:tasks.rados.rados.0.smithi084.stdout:3054: finishing rollback tid 1 to smithi08434663-34 2024-09-18T14:13:35.708 INFO:tasks.rados.rados.0.smithi084.stdout:3052: finishing rollback tid 0 to smithi08434663-13 2024-09-18T14:13:35.708 INFO:tasks.rados.rados.0.smithi084.stdout:3052: finishing rollback tid 1 to smithi08434663-13 2024-09-18T14:13:35.708 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 1533 (ObjNum 81 snap 0 seq_num 2494624160) dirty dne 2024-09-18T14:13:35.708 INFO:tasks.rados.rados.0.smithi084.stdout:3053: finishing rollback tid 0 to smithi08434663-6 2024-09-18T14:13:35.708 INFO:tasks.rados.rados.0.smithi084.stdout:3053: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:13:35.708 INFO:tasks.rados.rados.0.smithi084.stdout:3053: finishing rollback tid 2 to smithi08434663-6 2024-09-18T14:13:35.708 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 1023 (ObjNum 907 snap 281 seq_num 907) dirty exists 2024-09-18T14:13:35.708 INFO:tasks.rados.rados.0.smithi084.stdout:3054: finishing rollback tid 2 to smithi08434663-34 2024-09-18T14:13:35.709 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1525 (ObjNum 983 snap 316 seq_num 983) dirty exists 2024-09-18T14:13:35.709 INFO:tasks.rados.rados.0.smithi084.stdout:3056: finishing write tid 1 to smithi08434663-33 2024-09-18T14:13:35.709 INFO:tasks.rados.rados.0.smithi084.stdout:3051: done (6 left) 2024-09-18T14:13:35.709 INFO:tasks.rados.rados.0.smithi084.stdout:3052: done (5 left) 2024-09-18T14:13:35.709 INFO:tasks.rados.rados.0.smithi084.stdout:3053: done (4 left) 2024-09-18T14:13:35.709 INFO:tasks.rados.rados.0.smithi084.stdout:3054: done (3 left) 2024-09-18T14:13:35.709 INFO:tasks.rados.rados.0.smithi084.stdout:3055: done (2 left) 2024-09-18T14:13:35.709 INFO:tasks.rados.rados.0.smithi084.stdout:3057: done (1 left) 2024-09-18T14:13:35.710 INFO:tasks.rados.rados.0.smithi084.stdout:3058: snap_remove snap 319 2024-09-18T14:13:36.486 INFO:tasks.rados.rados.0.smithi084.stdout:3058: done (1 left) 2024-09-18T14:13:36.486 INFO:tasks.rados.rados.0.smithi084.stdout:3059: write oid 40 current snap is 333 2024-09-18T14:13:36.486 INFO:tasks.rados.rados.0.smithi084.stdout:3056: finishing write tid 2 to smithi08434663-33 2024-09-18T14:13:36.486 INFO:tasks.rados.rados.0.smithi084.stdout:3059: seq_num 1039 ranges {662473=407719,1833144=644690,3199313=263160} 2024-09-18T14:13:36.492 INFO:tasks.rados.rados.0.smithi084.stdout:3059: writing smithi08434663-40 from 662473 to 1070192 tid 1 2024-09-18T14:13:36.499 INFO:tasks.rados.rados.0.smithi084.stdout:3059: writing smithi08434663-40 from 1833144 to 2477834 tid 2 2024-09-18T14:13:36.505 INFO:tasks.rados.rados.0.smithi084.stdout:3059: writing smithi08434663-40 from 3199313 to 3462473 tid 3 2024-09-18T14:13:36.505 INFO:tasks.rados.rados.0.smithi084.stdout:3060: snap_remove snap 323 2024-09-18T14:13:36.505 INFO:tasks.rados.rados.0.smithi084.stdout:3056: finishing write tid 3 to smithi08434663-33 2024-09-18T14:13:36.505 INFO:tasks.rados.rados.0.smithi084.stdout:3056: finishing write tid 4 to smithi08434663-33 2024-09-18T14:13:37.487 INFO:tasks.rados.rados.0.smithi084.stdout:3060: done (2 left) 2024-09-18T14:13:37.487 INFO:tasks.rados.rados.0.smithi084.stdout:3061: copy_from oid 48 from oid 42 current snap is 333 2024-09-18T14:13:37.487 INFO:tasks.rados.rados.0.smithi084.stdout:3056: finishing write tid 5 to smithi08434663-33 2024-09-18T14:13:37.487 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 1325 (ObjNum 1037 snap 333 seq_num 1037) dirty exists 2024-09-18T14:13:37.487 INFO:tasks.rados.rados.0.smithi084.stdout:3056: left oid 33 (ObjNum 1037 snap 333 seq_num 1037) 2024-09-18T14:13:37.487 INFO:tasks.rados.rados.0.smithi084.stdout:3059: finishing write tid 1 to smithi08434663-40 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:3059: finishing write tid 2 to smithi08434663-40 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:3059: finishing write tid 3 to smithi08434663-40 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:3059: finishing write tid 4 to smithi08434663-40 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:3059: finishing write tid 5 to smithi08434663-40 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 1537 (ObjNum 1039 snap 333 seq_num 1039) dirty exists 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:3059: left oid 40 (ObjNum 1039 snap 333 seq_num 1039) 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:3056: done (2 left) 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:3059: done (1 left) 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:3062: copy_from oid 21 from oid 30 current snap is 333 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:3063: read oid 45 snap 329 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:3063: expect deleted 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:3064: write oid 36 current snap is 333 2024-09-18T14:13:37.488 INFO:tasks.rados.rados.0.smithi084.stdout:3064: seq_num 1040 ranges {679280=482968,1775273=788348,3110383=368897} 2024-09-18T14:13:37.493 INFO:tasks.rados.rados.0.smithi084.stdout:3064: writing smithi08434663-36 from 679280 to 1162248 tid 1 2024-09-18T14:13:37.502 INFO:tasks.rados.rados.0.smithi084.stdout:3064: writing smithi08434663-36 from 1775273 to 2563621 tid 2 2024-09-18T14:13:37.508 INFO:tasks.rados.rados.0.smithi084.stdout:3064: writing smithi08434663-36 from 3110383 to 3479280 tid 3 2024-09-18T14:13:37.508 INFO:tasks.rados.rados.0.smithi084.stdout:3065: write oid 8 current snap is 333 2024-09-18T14:13:37.508 INFO:tasks.rados.rados.0.smithi084.stdout:3065: seq_num 1041 ranges {696087=558217,1833755=648359,3137806=358281} 2024-09-18T14:13:37.515 INFO:tasks.rados.rados.0.smithi084.stdout:3065: writing smithi08434663-8 from 696087 to 1254304 tid 1 2024-09-18T14:13:37.522 INFO:tasks.rados.rados.0.smithi084.stdout:3065: writing smithi08434663-8 from 1833755 to 2482114 tid 2 2024-09-18T14:13:37.528 INFO:tasks.rados.rados.0.smithi084.stdout:3065: writing smithi08434663-8 from 3137806 to 3496087 tid 3 2024-09-18T14:13:37.528 INFO:tasks.rados.rados.0.smithi084.stdout:3066: delete oid 12 current snap is 333 2024-09-18T14:13:37.528 INFO:tasks.rados.rados.0.smithi084.stdout:3064: finishing write tid 1 to smithi08434663-36 2024-09-18T14:13:37.528 INFO:tasks.rados.rados.0.smithi084.stdout:3064: finishing write tid 2 to smithi08434663-36 2024-09-18T14:13:37.528 INFO:tasks.rados.rados.0.smithi084.stdout:3064: finishing write tid 3 to smithi08434663-36 2024-09-18T14:13:37.528 INFO:tasks.rados.rados.0.smithi084.stdout:3064: finishing write tid 4 to smithi08434663-36 2024-09-18T14:13:37.528 INFO:tasks.rados.rados.0.smithi084.stdout:3064: finishing write tid 5 to smithi08434663-36 2024-09-18T14:13:37.529 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 1329 (ObjNum 1040 snap 333 seq_num 1040) dirty exists 2024-09-18T14:13:37.529 INFO:tasks.rados.rados.0.smithi084.stdout:3064: left oid 36 (ObjNum 1040 snap 333 seq_num 1040) 2024-09-18T14:13:37.529 INFO:tasks.rados.rados.0.smithi084.stdout:3065: finishing write tid 1 to smithi08434663-8 2024-09-18T14:13:37.529 INFO:tasks.rados.rados.0.smithi084.stdout:3061: finishing copy_from racing read to smithi08434663-48 2024-09-18T14:13:37.529 INFO:tasks.rados.rados.0.smithi084.stdout:3062: finishing copy_from racing read to smithi08434663-21 2024-09-18T14:13:37.529 INFO:tasks.rados.rados.0.smithi084.stdout:3062: finishing copy_from to smithi08434663-21 2024-09-18T14:13:37.529 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 1263 (ObjNum 1021 snap 326 seq_num 1021) dirty exists 2024-09-18T14:13:37.529 INFO:tasks.rados.rados.0.smithi084.stdout:3061: finishing copy_from to smithi08434663-48 2024-09-18T14:13:37.529 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1523 (ObjNum 1036 snap 332 seq_num 1036) dirty exists 2024-09-18T14:13:37.529 INFO:tasks.rados.rados.0.smithi084.stdout:3065: finishing write tid 2 to smithi08434663-8 2024-09-18T14:13:37.530 INFO:tasks.rados.rados.0.smithi084.stdout:3065: finishing write tid 3 to smithi08434663-8 2024-09-18T14:13:37.530 INFO:tasks.rados.rados.0.smithi084.stdout:3065: finishing write tid 4 to smithi08434663-8 2024-09-18T14:13:37.530 INFO:tasks.rados.rados.0.smithi084.stdout:3065: finishing write tid 5 to smithi08434663-8 2024-09-18T14:13:37.531 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 1395 (ObjNum 1041 snap 333 seq_num 1041) dirty exists 2024-09-18T14:13:37.531 INFO:tasks.rados.rados.0.smithi084.stdout:3065: left oid 8 (ObjNum 1041 snap 333 seq_num 1041) 2024-09-18T14:13:37.531 INFO:tasks.rados.rados.0.smithi084.stdout:3061: done (5 left) 2024-09-18T14:13:37.531 INFO:tasks.rados.rados.0.smithi084.stdout:3062: done (4 left) 2024-09-18T14:13:37.532 INFO:tasks.rados.rados.0.smithi084.stdout:3063: done (3 left) 2024-09-18T14:13:37.532 INFO:tasks.rados.rados.0.smithi084.stdout:3064: done (2 left) 2024-09-18T14:13:37.532 INFO:tasks.rados.rados.0.smithi084.stdout:3065: done (1 left) 2024-09-18T14:13:37.532 INFO:tasks.rados.rados.0.smithi084.stdout:3066: done (0 left) 2024-09-18T14:13:37.532 INFO:tasks.rados.rados.0.smithi084.stdout:3067: read oid 9 snap -1 2024-09-18T14:13:37.532 INFO:tasks.rados.rados.0.smithi084.stdout:3067: expect deleted 2024-09-18T14:13:37.532 INFO:tasks.rados.rados.0.smithi084.stdout:3068: snap_create 2024-09-18T14:13:38.471 INFO:tasks.rados.rados.0.smithi084.stdout:3067: done (1 left) 2024-09-18T14:13:38.471 INFO:tasks.rados.rados.0.smithi084.stdout:3068: done (0 left) 2024-09-18T14:13:38.471 INFO:tasks.rados.rados.0.smithi084.stdout:3069: write oid 4 current snap is 334 2024-09-18T14:13:38.471 INFO:tasks.rados.rados.0.smithi084.stdout:3069: seq_num 1043 ranges {729701=425068,1667072=652028,2909005=485180} 2024-09-18T14:13:38.477 INFO:tasks.rados.rados.0.smithi084.stdout:3069: writing smithi08434663-4 from 729701 to 1154769 tid 1 2024-09-18T14:13:38.483 INFO:tasks.rados.rados.0.smithi084.stdout:3069: writing smithi08434663-4 from 1667072 to 2319100 tid 2 2024-09-18T14:13:38.489 INFO:tasks.rados.rados.0.smithi084.stdout:3069: writing smithi08434663-4 from 2909005 to 3394185 tid 3 2024-09-18T14:13:38.490 INFO:tasks.rados.rados.0.smithi084.stdout:3070: snap_remove snap 322 2024-09-18T14:13:39.478 INFO:tasks.rados.rados.0.smithi084.stdout:3070: done (1 left) 2024-09-18T14:13:39.478 INFO:tasks.rados.rados.0.smithi084.stdout:3071: read oid 40 snap -1 2024-09-18T14:13:39.478 INFO:tasks.rados.rados.0.smithi084.stdout:3071: expect (ObjNum 1039 snap 333 seq_num 1039) 2024-09-18T14:13:39.478 INFO:tasks.rados.rados.0.smithi084.stdout:3072: write (excl) oid 39 current snap is 334 2024-09-18T14:13:39.479 INFO:tasks.rados.rados.0.smithi084.stdout:3072: seq_num 1044 ranges {746508=500317,2009201=795686,3220075=326433} 2024-09-18T14:13:39.485 INFO:tasks.rados.rados.0.smithi084.stdout:3072: writing smithi08434663-39 from 746508 to 1246825 tid 1 2024-09-18T14:13:39.495 INFO:tasks.rados.rados.0.smithi084.stdout:3072: writing smithi08434663-39 from 2009201 to 2804887 tid 2 2024-09-18T14:13:39.499 INFO:tasks.rados.rados.0.smithi084.stdout:3072: writing smithi08434663-39 from 3220075 to 3546508 tid 3 2024-09-18T14:13:39.500 INFO:tasks.rados.rados.0.smithi084.stdout:3073: write (excl) oid 45 current snap is 334 2024-09-18T14:13:39.500 INFO:tasks.rados.rados.0.smithi084.stdout:3069: finishing write tid 1 to smithi08434663-4 2024-09-18T14:13:39.500 INFO:tasks.rados.rados.0.smithi084.stdout:3073: seq_num 1045 ranges {763315=575566,2067683=655697,3247498=315817} 2024-09-18T14:13:39.506 INFO:tasks.rados.rados.0.smithi084.stdout:3073: writing smithi08434663-45 from 763315 to 1338881 tid 1 2024-09-18T14:13:39.514 INFO:tasks.rados.rados.0.smithi084.stdout:3073: writing smithi08434663-45 from 2067683 to 2723380 tid 2 2024-09-18T14:13:39.520 INFO:tasks.rados.rados.0.smithi084.stdout:3073: writing smithi08434663-45 from 3247498 to 3563315 tid 3 2024-09-18T14:13:39.520 INFO:tasks.rados.rados.0.smithi084.stdout:3074: snap_create 2024-09-18T14:13:39.521 INFO:tasks.rados.rados.0.smithi084.stdout:3069: finishing write tid 2 to smithi08434663-4 2024-09-18T14:13:39.521 INFO:tasks.rados.rados.0.smithi084.stdout:3069: finishing write tid 3 to smithi08434663-4 2024-09-18T14:13:39.521 INFO:tasks.rados.rados.0.smithi084.stdout:3069: finishing write tid 4 to smithi08434663-4 2024-09-18T14:13:39.521 INFO:tasks.rados.rados.0.smithi084.stdout:3069: finishing write tid 5 to smithi08434663-4 2024-09-18T14:13:39.521 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1529 (ObjNum 1043 snap 334 seq_num 1043) dirty exists 2024-09-18T14:13:39.521 INFO:tasks.rados.rados.0.smithi084.stdout:3069: left oid 4 (ObjNum 1043 snap 334 seq_num 1043) 2024-09-18T14:13:39.521 INFO:tasks.rados.rados.0.smithi084.stdout:3072: finishing write tid 1 to smithi08434663-39 2024-09-18T14:13:39.521 INFO:tasks.rados.rados.0.smithi084.stdout:3071: expect (ObjNum 1039 snap 333 seq_num 1039) 2024-09-18T14:13:39.689 INFO:tasks.rados.rados.0.smithi084.stdout:3072: finishing write tid 2 to smithi08434663-39 2024-09-18T14:13:39.689 INFO:tasks.rados.rados.0.smithi084.stdout:3072: finishing write tid 3 to smithi08434663-39 2024-09-18T14:13:39.689 INFO:tasks.rados.rados.0.smithi084.stdout:3072: finishing write tid 4 to smithi08434663-39 2024-09-18T14:13:39.689 INFO:tasks.rados.rados.0.smithi084.stdout:3072: finishing write tid 5 to smithi08434663-39 2024-09-18T14:13:39.689 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 1276 (ObjNum 1044 snap 334 seq_num 1044) dirty exists 2024-09-18T14:13:39.689 INFO:tasks.rados.rados.0.smithi084.stdout:3072: left oid 39 (ObjNum 1044 snap 334 seq_num 1044) 2024-09-18T14:13:39.689 INFO:tasks.rados.rados.0.smithi084.stdout:3073: finishing write tid 1 to smithi08434663-45 2024-09-18T14:13:39.689 INFO:tasks.rados.rados.0.smithi084.stdout:3073: finishing write tid 2 to smithi08434663-45 2024-09-18T14:13:39.689 INFO:tasks.rados.rados.0.smithi084.stdout:3073: finishing write tid 3 to smithi08434663-45 2024-09-18T14:13:39.689 INFO:tasks.rados.rados.0.smithi084.stdout:3073: finishing write tid 4 to smithi08434663-45 2024-09-18T14:13:39.690 INFO:tasks.rados.rados.0.smithi084.stdout:3073: finishing write tid 5 to smithi08434663-45 2024-09-18T14:13:39.690 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 1533 (ObjNum 1045 snap 334 seq_num 1045) dirty exists 2024-09-18T14:13:39.690 INFO:tasks.rados.rados.0.smithi084.stdout:3073: left oid 45 (ObjNum 1045 snap 334 seq_num 1045) 2024-09-18T14:13:40.483 INFO:tasks.rados.rados.0.smithi084.stdout:3069: done (4 left) 2024-09-18T14:13:40.483 INFO:tasks.rados.rados.0.smithi084.stdout:3071: done (3 left) 2024-09-18T14:13:40.483 INFO:tasks.rados.rados.0.smithi084.stdout:3072: done (2 left) 2024-09-18T14:13:40.483 INFO:tasks.rados.rados.0.smithi084.stdout:3073: done (1 left) 2024-09-18T14:13:40.483 INFO:tasks.rados.rados.0.smithi084.stdout:3074: done (0 left) 2024-09-18T14:13:40.484 INFO:tasks.rados.rados.0.smithi084.stdout:3075: read oid 35 snap -1 2024-09-18T14:13:40.484 INFO:tasks.rados.rados.0.smithi084.stdout:3075: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:13:40.484 INFO:tasks.rados.rados.0.smithi084.stdout:3076: read oid 45 snap 330 2024-09-18T14:13:40.484 INFO:tasks.rados.rados.0.smithi084.stdout:3076: expect deleted 2024-09-18T14:13:40.484 INFO:tasks.rados.rados.0.smithi084.stdout:3077: delete oid 15 current snap is 335 2024-09-18T14:13:40.488 INFO:tasks.rados.rados.0.smithi084.stdout:3075: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:13:40.638 INFO:tasks.rados.rados.0.smithi084.stdout:3075: done (2 left) 2024-09-18T14:13:40.639 INFO:tasks.rados.rados.0.smithi084.stdout:3076: done (1 left) 2024-09-18T14:13:40.639 INFO:tasks.rados.rados.0.smithi084.stdout:3077: done (0 left) 2024-09-18T14:13:40.639 INFO:tasks.rados.rados.0.smithi084.stdout:3078: read oid 3 snap -1 2024-09-18T14:13:40.639 INFO:tasks.rados.rados.0.smithi084.stdout:3078: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:13:40.639 INFO:tasks.rados.rados.0.smithi084.stdout:3079: rollback oid 31 current snap is 335 2024-09-18T14:13:40.639 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 31 to 310 2024-09-18T14:13:40.639 INFO:tasks.rados.rados.0.smithi084.stdout:3080: write oid 9 current snap is 335 2024-09-18T14:13:40.640 INFO:tasks.rados.rados.0.smithi084.stdout:3080: seq_num 1047 ranges {796929=726064,2184647=659366,3302344=294585} 2024-09-18T14:13:40.646 INFO:tasks.rados.rados.0.smithi084.stdout:3080: writing smithi08434663-9 from 796929 to 1522993 tid 1 2024-09-18T14:13:40.653 INFO:tasks.rados.rados.0.smithi084.stdout:3080: writing smithi08434663-9 from 2184647 to 2844013 tid 2 2024-09-18T14:13:40.658 INFO:tasks.rados.rados.0.smithi084.stdout:3080: writing smithi08434663-9 from 3302344 to 3596929 tid 3 2024-09-18T14:13:40.658 INFO:tasks.rados.rados.0.smithi084.stdout:3081: copy_from oid 26 from oid 8 current snap is 335 2024-09-18T14:13:40.658 INFO:tasks.rados.rados.0.smithi084.stdout:3079: finishing rollback tid 0 to smithi08434663-31 2024-09-18T14:13:40.658 INFO:tasks.rados.rados.0.smithi084.stdout:3079: finishing rollback tid 1 to smithi08434663-31 2024-09-18T14:13:40.658 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 1469 (ObjNum 129 snap 0 seq_num 2495143104) dirty dne 2024-09-18T14:13:40.658 INFO:tasks.rados.rados.0.smithi084.stdout:3078: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:13:40.809 INFO:tasks.rados.rados.0.smithi084.stdout:3080: finishing write tid 1 to smithi08434663-9 2024-09-18T14:13:40.809 INFO:tasks.rados.rados.0.smithi084.stdout:3080: finishing write tid 2 to smithi08434663-9 2024-09-18T14:13:40.809 INFO:tasks.rados.rados.0.smithi084.stdout:3080: finishing write tid 3 to smithi08434663-9 2024-09-18T14:13:40.809 INFO:tasks.rados.rados.0.smithi084.stdout:3080: finishing write tid 4 to smithi08434663-9 2024-09-18T14:13:40.809 INFO:tasks.rados.rados.0.smithi084.stdout:3080: finishing write tid 5 to smithi08434663-9 2024-09-18T14:13:40.809 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 1468 (ObjNum 1047 snap 335 seq_num 1047) dirty exists 2024-09-18T14:13:40.809 INFO:tasks.rados.rados.0.smithi084.stdout:3080: left oid 9 (ObjNum 1047 snap 335 seq_num 1047) 2024-09-18T14:13:40.809 INFO:tasks.rados.rados.0.smithi084.stdout:3081: finishing copy_from racing read to smithi08434663-26 2024-09-18T14:13:40.809 INFO:tasks.rados.rados.0.smithi084.stdout:3081: finishing copy_from to smithi08434663-26 2024-09-18T14:13:40.809 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 1395 (ObjNum 1041 snap 333 seq_num 1041) dirty exists 2024-09-18T14:13:40.810 INFO:tasks.rados.rados.0.smithi084.stdout:3078: done (3 left) 2024-09-18T14:13:40.810 INFO:tasks.rados.rados.0.smithi084.stdout:3079: done (2 left) 2024-09-18T14:13:40.810 INFO:tasks.rados.rados.0.smithi084.stdout:3080: done (1 left) 2024-09-18T14:13:40.810 INFO:tasks.rados.rados.0.smithi084.stdout:3081: done (0 left) 2024-09-18T14:13:40.810 INFO:tasks.rados.rados.0.smithi084.stdout:3082: snap_remove snap 324 2024-09-18T14:13:41.492 INFO:tasks.rados.rados.0.smithi084.stdout:3082: done (0 left) 2024-09-18T14:13:41.492 INFO:tasks.rados.rados.0.smithi084.stdout:3083: read oid 21 snap -1 2024-09-18T14:13:41.492 INFO:tasks.rados.rados.0.smithi084.stdout:3083: expect (ObjNum 1021 snap 326 seq_num 1021) 2024-09-18T14:13:41.492 INFO:tasks.rados.rados.0.smithi084.stdout:3084: delete oid 27 current snap is 335 2024-09-18T14:13:41.501 INFO:tasks.rados.rados.0.smithi084.stdout:3084: done (1 left) 2024-09-18T14:13:41.501 INFO:tasks.rados.rados.0.smithi084.stdout:3085: write (excl) oid 27 current snap is 335 2024-09-18T14:13:41.502 INFO:tasks.rados.rados.0.smithi084.stdout:3085: seq_num 1049 ranges {430543=476562,1385258=663035,2840837=669165} 2024-09-18T14:13:41.506 INFO:tasks.rados.rados.0.smithi084.stdout:3085: writing smithi08434663-27 from 430543 to 907105 tid 1 2024-09-18T14:13:41.513 INFO:tasks.rados.rados.0.smithi084.stdout:3085: writing smithi08434663-27 from 1385258 to 2048293 tid 2 2024-09-18T14:13:41.522 INFO:tasks.rados.rados.0.smithi084.stdout:3085: writing smithi08434663-27 from 2840837 to 3510002 tid 3 2024-09-18T14:13:41.522 INFO:tasks.rados.rados.0.smithi084.stdout:3086: read oid 34 snap -1 2024-09-18T14:13:41.522 INFO:tasks.rados.rados.0.smithi084.stdout:3086: expect (ObjNum 983 snap 316 seq_num 983) 2024-09-18T14:13:41.522 INFO:tasks.rados.rados.0.smithi084.stdout:3087: read oid 28 snap -1 2024-09-18T14:13:41.522 INFO:tasks.rados.rados.0.smithi084.stdout:3087: expect (ObjNum 971 snap 308 seq_num 971) 2024-09-18T14:13:41.522 INFO:tasks.rados.rados.0.smithi084.stdout:3088: snap_create 2024-09-18T14:13:41.522 INFO:tasks.rados.rados.0.smithi084.stdout:3083: expect (ObjNum 1021 snap 326 seq_num 1021) 2024-09-18T14:13:41.729 INFO:tasks.rados.rados.0.smithi084.stdout:3085: finishing write tid 1 to smithi08434663-27 2024-09-18T14:13:41.729 INFO:tasks.rados.rados.0.smithi084.stdout:3085: finishing write tid 2 to smithi08434663-27 2024-09-18T14:13:41.729 INFO:tasks.rados.rados.0.smithi084.stdout:3087: expect (ObjNum 971 snap 308 seq_num 971) 2024-09-18T14:13:41.823 INFO:tasks.rados.rados.0.smithi084.stdout:3085: finishing write tid 3 to smithi08434663-27 2024-09-18T14:13:41.823 INFO:tasks.rados.rados.0.smithi084.stdout:3085: finishing write tid 4 to smithi08434663-27 2024-09-18T14:13:41.823 INFO:tasks.rados.rados.0.smithi084.stdout:3085: finishing write tid 5 to smithi08434663-27 2024-09-18T14:13:41.823 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1399 (ObjNum 1049 snap 335 seq_num 1049) dirty exists 2024-09-18T14:13:41.823 INFO:tasks.rados.rados.0.smithi084.stdout:3085: left oid 27 (ObjNum 1049 snap 335 seq_num 1049) 2024-09-18T14:13:41.823 INFO:tasks.rados.rados.0.smithi084.stdout:3086: expect (ObjNum 983 snap 316 seq_num 983) 2024-09-18T14:13:42.502 INFO:tasks.rados.rados.0.smithi084.stdout:3083: done (4 left) 2024-09-18T14:13:42.503 INFO:tasks.rados.rados.0.smithi084.stdout:3085: done (3 left) 2024-09-18T14:13:42.503 INFO:tasks.rados.rados.0.smithi084.stdout:3086: done (2 left) 2024-09-18T14:13:42.503 INFO:tasks.rados.rados.0.smithi084.stdout:3087: done (1 left) 2024-09-18T14:13:42.503 INFO:tasks.rados.rados.0.smithi084.stdout:3088: done (0 left) 2024-09-18T14:13:42.503 INFO:tasks.rados.rados.0.smithi084.stdout:3089: write oid 20 current snap is 336 2024-09-18T14:13:42.503 INFO:tasks.rados.rados.0.smithi084.stdout:3089: seq_num 1050 ranges {447350=668164,1560093=406693,2584613=480437} 2024-09-18T14:13:42.508 INFO:tasks.rados.rados.0.smithi084.stdout:3089: writing smithi08434663-20 from 447350 to 1115514 tid 1 2024-09-18T14:13:42.513 INFO:tasks.rados.rados.0.smithi084.stdout:3089: writing smithi08434663-20 from 1560093 to 1966786 tid 2 2024-09-18T14:13:42.519 INFO:tasks.rados.rados.0.smithi084.stdout:3089: writing smithi08434663-20 from 2584613 to 3065050 tid 3 2024-09-18T14:13:42.519 INFO:tasks.rados.rados.0.smithi084.stdout:3090: write (excl) oid 23 current snap is 336 2024-09-18T14:13:42.519 INFO:tasks.rados.rados.0.smithi084.stdout:3089: finishing write tid 1 to smithi08434663-20 2024-09-18T14:13:42.519 INFO:tasks.rados.rados.0.smithi084.stdout:3089: finishing write tid 2 to smithi08434663-20 2024-09-18T14:13:42.519 INFO:tasks.rados.rados.0.smithi084.stdout:3090: seq_num 1051 ranges {464157=743413,1618575=666704,3012036=408062} 2024-09-18T14:13:42.525 INFO:tasks.rados.rados.0.smithi084.stdout:3090: writing smithi08434663-23 from 464157 to 1207570 tid 1 2024-09-18T14:13:42.531 INFO:tasks.rados.rados.0.smithi084.stdout:3090: writing smithi08434663-23 from 1618575 to 2285279 tid 2 2024-09-18T14:13:42.537 INFO:tasks.rados.rados.0.smithi084.stdout:3090: writing smithi08434663-23 from 3012036 to 3420098 tid 3 2024-09-18T14:13:42.537 INFO:tasks.rados.rados.0.smithi084.stdout:3091: read oid 1 snap -1 2024-09-18T14:13:42.537 INFO:tasks.rados.rados.0.smithi084.stdout:3091: expect (ObjNum 961 snap 306 seq_num 961) 2024-09-18T14:13:42.537 INFO:tasks.rados.rados.0.smithi084.stdout:3092: write (excl) oid 50 current snap is 336 2024-09-18T14:13:42.538 INFO:tasks.rados.rados.0.smithi084.stdout:3092: seq_num 1052 ranges {480964=418662,1560704=410362,2523106=619334} 2024-09-18T14:13:42.542 INFO:tasks.rados.rados.0.smithi084.stdout:3092: writing smithi08434663-50 from 480964 to 899626 tid 1 2024-09-18T14:13:42.548 INFO:tasks.rados.rados.0.smithi084.stdout:3092: writing smithi08434663-50 from 1560704 to 1971066 tid 2 2024-09-18T14:13:42.555 INFO:tasks.rados.rados.0.smithi084.stdout:3092: writing smithi08434663-50 from 2523106 to 3142440 tid 3 2024-09-18T14:13:42.555 INFO:tasks.rados.rados.0.smithi084.stdout:3093: delete oid 7 current snap is 336 2024-09-18T14:13:42.555 INFO:tasks.rados.rados.0.smithi084.stdout:3089: finishing write tid 3 to smithi08434663-20 2024-09-18T14:13:42.555 INFO:tasks.rados.rados.0.smithi084.stdout:3089: finishing write tid 4 to smithi08434663-20 2024-09-18T14:13:42.555 INFO:tasks.rados.rados.0.smithi084.stdout:3089: finishing write tid 5 to smithi08434663-20 2024-09-18T14:13:42.555 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 1399 (ObjNum 1050 snap 336 seq_num 1050) dirty exists 2024-09-18T14:13:42.555 INFO:tasks.rados.rados.0.smithi084.stdout:3089: left oid 20 (ObjNum 1050 snap 336 seq_num 1050) 2024-09-18T14:13:42.555 INFO:tasks.rados.rados.0.smithi084.stdout:3090: finishing write tid 1 to smithi08434663-23 2024-09-18T14:13:42.555 INFO:tasks.rados.rados.0.smithi084.stdout:3090: finishing write tid 2 to smithi08434663-23 2024-09-18T14:13:42.556 INFO:tasks.rados.rados.0.smithi084.stdout:3090: finishing write tid 3 to smithi08434663-23 2024-09-18T14:13:42.556 INFO:tasks.rados.rados.0.smithi084.stdout:3091: expect (ObjNum 961 snap 306 seq_num 961) 2024-09-18T14:13:42.626 INFO:tasks.rados.rados.0.smithi084.stdout:3090: finishing write tid 4 to smithi08434663-23 2024-09-18T14:13:42.626 INFO:tasks.rados.rados.0.smithi084.stdout:3090: finishing write tid 5 to smithi08434663-23 2024-09-18T14:13:42.626 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1027 (ObjNum 1051 snap 336 seq_num 1051) dirty exists 2024-09-18T14:13:42.626 INFO:tasks.rados.rados.0.smithi084.stdout:3090: left oid 23 (ObjNum 1051 snap 336 seq_num 1051) 2024-09-18T14:13:42.627 INFO:tasks.rados.rados.0.smithi084.stdout:3092: finishing write tid 1 to smithi08434663-50 2024-09-18T14:13:42.627 INFO:tasks.rados.rados.0.smithi084.stdout:3092: finishing write tid 2 to smithi08434663-50 2024-09-18T14:13:42.627 INFO:tasks.rados.rados.0.smithi084.stdout:3092: finishing write tid 3 to smithi08434663-50 2024-09-18T14:13:42.627 INFO:tasks.rados.rados.0.smithi084.stdout:3092: finishing write tid 4 to smithi08434663-50 2024-09-18T14:13:42.627 INFO:tasks.rados.rados.0.smithi084.stdout:3092: finishing write tid 5 to smithi08434663-50 2024-09-18T14:13:42.627 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 1673 (ObjNum 1052 snap 336 seq_num 1052) dirty exists 2024-09-18T14:13:42.627 INFO:tasks.rados.rados.0.smithi084.stdout:3092: left oid 50 (ObjNum 1052 snap 336 seq_num 1052) 2024-09-18T14:13:42.627 INFO:tasks.rados.rados.0.smithi084.stdout:3089: done (4 left) 2024-09-18T14:13:42.627 INFO:tasks.rados.rados.0.smithi084.stdout:3090: done (3 left) 2024-09-18T14:13:42.627 INFO:tasks.rados.rados.0.smithi084.stdout:3091: done (2 left) 2024-09-18T14:13:42.627 INFO:tasks.rados.rados.0.smithi084.stdout:3092: done (1 left) 2024-09-18T14:13:42.627 INFO:tasks.rados.rados.0.smithi084.stdout:3093: done (0 left) 2024-09-18T14:13:42.628 INFO:tasks.rados.rados.0.smithi084.stdout:3094: write (excl) oid 42 current snap is 336 2024-09-18T14:13:42.628 INFO:tasks.rados.rados.0.smithi084.stdout:3094: seq_num 1054 ranges {514578=569160,1677668=414031,2577952=641878} 2024-09-18T14:13:42.631 INFO:tasks.rados.rados.0.smithi084.stdout:3094: writing smithi08434663-42 from 514578 to 1083738 tid 1 2024-09-18T14:13:42.637 INFO:tasks.rados.rados.0.smithi084.stdout:3094: writing smithi08434663-42 from 1677668 to 2091699 tid 2 2024-09-18T14:13:42.643 INFO:tasks.rados.rados.0.smithi084.stdout:3094: writing smithi08434663-42 from 2577952 to 3219830 tid 3 2024-09-18T14:13:42.643 INFO:tasks.rados.rados.0.smithi084.stdout:3095: snap_create 2024-09-18T14:13:42.643 INFO:tasks.rados.rados.0.smithi084.stdout:3094: finishing write tid 1 to smithi08434663-42 2024-09-18T14:13:42.643 INFO:tasks.rados.rados.0.smithi084.stdout:3094: finishing write tid 2 to smithi08434663-42 2024-09-18T14:13:42.647 INFO:tasks.rados.rados.0.smithi084.stdout:3094: finishing write tid 3 to smithi08434663-42 2024-09-18T14:13:42.647 INFO:tasks.rados.rados.0.smithi084.stdout:3094: finishing write tid 4 to smithi08434663-42 2024-09-18T14:13:42.647 INFO:tasks.rados.rados.0.smithi084.stdout:3094: finishing write tid 5 to smithi08434663-42 2024-09-18T14:13:42.647 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1537 (ObjNum 1054 snap 336 seq_num 1054) dirty exists 2024-09-18T14:13:42.647 INFO:tasks.rados.rados.0.smithi084.stdout:3094: left oid 42 (ObjNum 1054 snap 336 seq_num 1054) 2024-09-18T14:13:43.471 INFO:tasks.rados.rados.0.smithi084.stdout:3094: done (1 left) 2024-09-18T14:13:43.472 INFO:tasks.rados.rados.0.smithi084.stdout:3095: done (0 left) 2024-09-18T14:13:43.472 INFO:tasks.rados.rados.0.smithi084.stdout:3096: write (excl) oid 45 current snap is 337 2024-09-18T14:13:43.472 INFO:tasks.rados.rados.0.smithi084.stdout:3096: seq_num 1055 ranges {531385=644409,1736150=557689,3005375=569503} 2024-09-18T14:13:43.478 INFO:tasks.rados.rados.0.smithi084.stdout:3096: writing smithi08434663-45 from 531385 to 1175794 tid 1 2024-09-18T14:13:43.484 INFO:tasks.rados.rados.0.smithi084.stdout:3096: writing smithi08434663-45 from 1736150 to 2293839 tid 2 2024-09-18T14:13:43.491 INFO:tasks.rados.rados.0.smithi084.stdout:3096: writing smithi08434663-45 from 3005375 to 3574878 tid 3 2024-09-18T14:13:43.491 INFO:tasks.rados.rados.0.smithi084.stdout:3097: write oid 16 current snap is 337 2024-09-18T14:13:43.492 INFO:tasks.rados.rados.0.smithi084.stdout:3096: finishing write tid 1 to smithi08434663-45 2024-09-18T14:13:43.492 INFO:tasks.rados.rados.0.smithi084.stdout:3097: seq_num 1056 ranges {548192=719658,1678279=417700,2516445=780775} 2024-09-18T14:13:43.498 INFO:tasks.rados.rados.0.smithi084.stdout:3097: writing smithi08434663-16 from 548192 to 1267850 tid 1 2024-09-18T14:13:43.503 INFO:tasks.rados.rados.0.smithi084.stdout:3097: writing smithi08434663-16 from 1678279 to 2095979 tid 2 2024-09-18T14:13:43.510 INFO:tasks.rados.rados.0.smithi084.stdout:3097: writing smithi08434663-16 from 2516445 to 3297220 tid 3 2024-09-18T14:13:43.510 INFO:tasks.rados.rados.0.smithi084.stdout:3098: snap_remove snap 328 2024-09-18T14:13:44.507 INFO:tasks.rados.rados.0.smithi084.stdout:3098: done (2 left) 2024-09-18T14:13:44.507 INFO:tasks.rados.rados.0.smithi084.stdout:3099: write oid 28 current snap is 337 2024-09-18T14:13:44.507 INFO:tasks.rados.rados.0.smithi084.stdout:3096: finishing write tid 2 to smithi08434663-45 2024-09-18T14:13:44.507 INFO:tasks.rados.rados.0.smithi084.stdout:3099: seq_num 1057 ranges {564999=511260,1853114=561358,3060221=592047} 2024-09-18T14:13:44.512 INFO:tasks.rados.rados.0.smithi084.stdout:3099: writing smithi08434663-28 from 564999 to 1076259 tid 1 2024-09-18T14:13:44.520 INFO:tasks.rados.rados.0.smithi084.stdout:3099: writing smithi08434663-28 from 1853114 to 2414472 tid 2 2024-09-18T14:13:44.527 INFO:tasks.rados.rados.0.smithi084.stdout:3099: writing smithi08434663-28 from 3060221 to 3652268 tid 3 2024-09-18T14:13:44.527 INFO:tasks.rados.rados.0.smithi084.stdout:3100: read oid 23 snap -1 2024-09-18T14:13:44.527 INFO:tasks.rados.rados.0.smithi084.stdout:3100: expect (ObjNum 1051 snap 336 seq_num 1051) 2024-09-18T14:13:44.527 INFO:tasks.rados.rados.0.smithi084.stdout:3101: snap_create 2024-09-18T14:13:44.528 INFO:tasks.rados.rados.0.smithi084.stdout:3096: finishing write tid 3 to smithi08434663-45 2024-09-18T14:13:44.528 INFO:tasks.rados.rados.0.smithi084.stdout:3096: finishing write tid 4 to smithi08434663-45 2024-09-18T14:13:44.528 INFO:tasks.rados.rados.0.smithi084.stdout:3096: finishing write tid 5 to smithi08434663-45 2024-09-18T14:13:44.528 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 1541 (ObjNum 1055 snap 337 seq_num 1055) dirty exists 2024-09-18T14:13:44.528 INFO:tasks.rados.rados.0.smithi084.stdout:3096: left oid 45 (ObjNum 1055 snap 337 seq_num 1055) 2024-09-18T14:13:44.528 INFO:tasks.rados.rados.0.smithi084.stdout:3097: finishing write tid 1 to smithi08434663-16 2024-09-18T14:13:44.528 INFO:tasks.rados.rados.0.smithi084.stdout:3097: finishing write tid 2 to smithi08434663-16 2024-09-18T14:13:44.528 INFO:tasks.rados.rados.0.smithi084.stdout:3097: finishing write tid 3 to smithi08434663-16 2024-09-18T14:13:44.528 INFO:tasks.rados.rados.0.smithi084.stdout:3097: finishing write tid 4 to smithi08434663-16 2024-09-18T14:13:44.528 INFO:tasks.rados.rados.0.smithi084.stdout:3097: finishing write tid 5 to smithi08434663-16 2024-09-18T14:13:44.528 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1472 (ObjNum 1056 snap 337 seq_num 1056) dirty exists 2024-09-18T14:13:44.528 INFO:tasks.rados.rados.0.smithi084.stdout:3097: left oid 16 (ObjNum 1056 snap 337 seq_num 1056) 2024-09-18T14:13:44.529 INFO:tasks.rados.rados.0.smithi084.stdout:3099: finishing write tid 1 to smithi08434663-28 2024-09-18T14:13:44.529 INFO:tasks.rados.rados.0.smithi084.stdout:3099: finishing write tid 2 to smithi08434663-28 2024-09-18T14:13:44.531 INFO:tasks.rados.rados.0.smithi084.stdout:3099: finishing write tid 3 to smithi08434663-28 2024-09-18T14:13:44.531 INFO:tasks.rados.rados.0.smithi084.stdout:3099: finishing write tid 4 to smithi08434663-28 2024-09-18T14:13:44.531 INFO:tasks.rados.rados.0.smithi084.stdout:3099: finishing write tid 5 to smithi08434663-28 2024-09-18T14:13:44.531 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1684 (ObjNum 1057 snap 337 seq_num 1057) dirty exists 2024-09-18T14:13:44.532 INFO:tasks.rados.rados.0.smithi084.stdout:3099: left oid 28 (ObjNum 1057 snap 337 seq_num 1057) 2024-09-18T14:13:44.534 INFO:tasks.rados.rados.0.smithi084.stdout:3100: expect (ObjNum 1051 snap 336 seq_num 1051) 2024-09-18T14:13:45.512 INFO:tasks.rados.rados.0.smithi084.stdout:3096: done (4 left) 2024-09-18T14:13:45.512 INFO:tasks.rados.rados.0.smithi084.stdout:3097: done (3 left) 2024-09-18T14:13:45.512 INFO:tasks.rados.rados.0.smithi084.stdout:3099: done (2 left) 2024-09-18T14:13:45.512 INFO:tasks.rados.rados.0.smithi084.stdout:3100: done (1 left) 2024-09-18T14:13:45.512 INFO:tasks.rados.rados.0.smithi084.stdout:3101: done (0 left) 2024-09-18T14:13:45.512 INFO:tasks.rados.rados.0.smithi084.stdout:3102: rollback oid 26 current snap is 338 2024-09-18T14:13:45.512 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 26 to 333 2024-09-18T14:13:45.512 INFO:tasks.rados.rados.0.smithi084.stdout:3103: rollback oid 2 current snap is 338 2024-09-18T14:13:45.512 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 2 to 333 2024-09-18T14:13:45.512 INFO:tasks.rados.rados.0.smithi084.stdout:3104: write oid 46 current snap is 338 2024-09-18T14:13:45.512 INFO:tasks.rados.rados.0.smithi084.stdout:3104: seq_num 1058 ranges {581806=586509,1911596=421369,3087644=403319} 2024-09-18T14:13:45.518 INFO:tasks.rados.rados.0.smithi084.stdout:3104: writing smithi08434663-46 from 581806 to 1168315 tid 1 2024-09-18T14:13:45.524 INFO:tasks.rados.rados.0.smithi084.stdout:3104: writing smithi08434663-46 from 1911596 to 2332965 tid 2 2024-09-18T14:13:45.531 INFO:tasks.rados.rados.0.smithi084.stdout:3104: writing smithi08434663-46 from 3087644 to 3490963 tid 3 2024-09-18T14:13:45.531 INFO:tasks.rados.rados.0.smithi084.stdout:3105: snap_create 2024-09-18T14:13:45.531 INFO:tasks.rados.rados.0.smithi084.stdout:3103: finishing rollback tid 1 to smithi08434663-2 2024-09-18T14:13:45.531 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 1601 (ObjNum 49 snap 0 seq_num 2475021424) dirty dne 2024-09-18T14:13:45.531 INFO:tasks.rados.rados.0.smithi084.stdout:3102: finishing rollback tid 0 to smithi08434663-26 2024-09-18T14:13:45.531 INFO:tasks.rados.rados.0.smithi084.stdout:3102: finishing rollback tid 1 to smithi08434663-26 2024-09-18T14:13:45.531 INFO:tasks.rados.rados.0.smithi084.stdout:3102: finishing rollback tid 2 to smithi08434663-26 2024-09-18T14:13:45.531 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 1402 (ObjNum 1030 snap 330 seq_num 1030) dirty exists 2024-09-18T14:13:45.531 INFO:tasks.rados.rados.0.smithi084.stdout:3104: finishing write tid 1 to smithi08434663-46 2024-09-18T14:13:45.532 INFO:tasks.rados.rados.0.smithi084.stdout:3104: finishing write tid 2 to smithi08434663-46 2024-09-18T14:13:45.536 INFO:tasks.rados.rados.0.smithi084.stdout:3104: finishing write tid 3 to smithi08434663-46 2024-09-18T14:13:45.536 INFO:tasks.rados.rados.0.smithi084.stdout:3104: finishing write tid 4 to smithi08434663-46 2024-09-18T14:13:45.536 INFO:tasks.rados.rados.0.smithi084.stdout:3104: finishing write tid 5 to smithi08434663-46 2024-09-18T14:13:45.536 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1476 (ObjNum 1058 snap 338 seq_num 1058) dirty exists 2024-09-18T14:13:45.536 INFO:tasks.rados.rados.0.smithi084.stdout:3104: left oid 46 (ObjNum 1058 snap 338 seq_num 1058) 2024-09-18T14:13:46.517 INFO:tasks.rados.rados.0.smithi084.stdout:3102: done (3 left) 2024-09-18T14:13:46.517 INFO:tasks.rados.rados.0.smithi084.stdout:3103: done (2 left) 2024-09-18T14:13:46.518 INFO:tasks.rados.rados.0.smithi084.stdout:3104: done (1 left) 2024-09-18T14:13:46.518 INFO:tasks.rados.rados.0.smithi084.stdout:3105: done (0 left) 2024-09-18T14:13:46.518 INFO:tasks.rados.rados.0.smithi084.stdout:3106: read oid 7 snap 335 2024-09-18T14:13:46.518 INFO:tasks.rados.rados.0.smithi084.stdout:3106: expect (ObjNum 976 snap 310 seq_num 976) 2024-09-18T14:13:46.518 INFO:tasks.rados.rados.0.smithi084.stdout:3107: delete oid 24 current snap is 339 2024-09-18T14:13:46.521 INFO:tasks.rados.rados.0.smithi084.stdout:3107: done (1 left) 2024-09-18T14:13:46.521 INFO:tasks.rados.rados.0.smithi084.stdout:3108: read oid 15 snap 308 2024-09-18T14:13:46.521 INFO:tasks.rados.rados.0.smithi084.stdout:3108: expect (ObjNum 952 snap 303 seq_num 952) 2024-09-18T14:13:46.521 INFO:tasks.rados.rados.0.smithi084.stdout:3109: write (excl) oid 34 current snap is 339 2024-09-18T14:13:46.521 INFO:tasks.rados.rados.0.smithi084.stdout:3109: seq_num 1060 ranges {615420=737007,1912207=425038,3026137=542216} 2024-09-18T14:13:46.528 INFO:tasks.rados.rados.0.smithi084.stdout:3109: writing smithi08434663-34 from 615420 to 1352427 tid 1 2024-09-18T14:13:46.534 INFO:tasks.rados.rados.0.smithi084.stdout:3109: writing smithi08434663-34 from 1912207 to 2337245 tid 2 2024-09-18T14:13:46.541 INFO:tasks.rados.rados.0.smithi084.stdout:3109: writing smithi08434663-34 from 3026137 to 3568353 tid 3 2024-09-18T14:13:46.541 INFO:tasks.rados.rados.0.smithi084.stdout:3110: read oid 39 snap -1 2024-09-18T14:13:46.541 INFO:tasks.rados.rados.0.smithi084.stdout:3110: expect (ObjNum 1044 snap 334 seq_num 1044) 2024-09-18T14:13:46.541 INFO:tasks.rados.rados.0.smithi084.stdout:3111: write (excl) oid 6 current snap is 339 2024-09-18T14:13:46.541 INFO:tasks.rados.rados.0.smithi084.stdout:3111: seq_num 1061 ranges {632227=412256,1570689=568696,2653560=753488} 2024-09-18T14:13:46.546 INFO:tasks.rados.rados.0.smithi084.stdout:3111: writing smithi08434663-6 from 632227 to 1044483 tid 1 2024-09-18T14:13:46.552 INFO:tasks.rados.rados.0.smithi084.stdout:3111: writing smithi08434663-6 from 1570689 to 2139385 tid 2 2024-09-18T14:13:46.559 INFO:tasks.rados.rados.0.smithi084.stdout:3111: writing smithi08434663-6 from 2653560 to 3407048 tid 3 2024-09-18T14:13:46.559 INFO:tasks.rados.rados.0.smithi084.stdout:3112: read oid 32 snap -1 2024-09-18T14:13:46.559 INFO:tasks.rados.rados.0.smithi084.stdout:3112: expect (ObjNum 1033 snap 331 seq_num 1033) 2024-09-18T14:13:46.559 INFO:tasks.rados.rados.0.smithi084.stdout:3113: write (excl) oid 40 current snap is 339 2024-09-18T14:13:46.559 INFO:tasks.rados.rados.0.smithi084.stdout:3113: seq_num 1062 ranges {649034=487505,1629171=428707,2680983=564760,3798768=50266} 2024-09-18T14:13:46.565 INFO:tasks.rados.rados.0.smithi084.stdout:3113: writing smithi08434663-40 from 649034 to 1136539 tid 1 2024-09-18T14:13:46.570 INFO:tasks.rados.rados.0.smithi084.stdout:3113: writing smithi08434663-40 from 1629171 to 2057878 tid 2 2024-09-18T14:13:46.576 INFO:tasks.rados.rados.0.smithi084.stdout:3113: writing smithi08434663-40 from 2680983 to 3245743 tid 3 2024-09-18T14:13:46.580 INFO:tasks.rados.rados.0.smithi084.stdout:3113: writing smithi08434663-40 from 3798768 to 3849034 tid 4 2024-09-18T14:13:46.580 INFO:tasks.rados.rados.0.smithi084.stdout:3114: write (excl) oid 16 current snap is 339 2024-09-18T14:13:46.580 INFO:tasks.rados.rados.0.smithi084.stdout:3106: expect (ObjNum 976 snap 310 seq_num 976) 2024-09-18T14:13:46.688 INFO:tasks.rados.rados.0.smithi084.stdout:3108: expect (ObjNum 952 snap 303 seq_num 952) 2024-09-18T14:13:46.749 INFO:tasks.rados.rados.0.smithi084.stdout:3109: finishing write tid 1 to smithi08434663-34 2024-09-18T14:13:46.749 INFO:tasks.rados.rados.0.smithi084.stdout:3109: finishing write tid 2 to smithi08434663-34 2024-09-18T14:13:46.749 INFO:tasks.rados.rados.0.smithi084.stdout:3109: finishing write tid 3 to smithi08434663-34 2024-09-18T14:13:46.749 INFO:tasks.rados.rados.0.smithi084.stdout:3109: finishing write tid 4 to smithi08434663-34 2024-09-18T14:13:46.749 INFO:tasks.rados.rados.0.smithi084.stdout:3109: finishing write tid 5 to smithi08434663-34 2024-09-18T14:13:46.750 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1545 (ObjNum 1060 snap 339 seq_num 1060) dirty exists 2024-09-18T14:13:46.750 INFO:tasks.rados.rados.0.smithi084.stdout:3109: left oid 34 (ObjNum 1060 snap 339 seq_num 1060) 2024-09-18T14:13:46.750 INFO:tasks.rados.rados.0.smithi084.stdout:3110: expect (ObjNum 1044 snap 334 seq_num 1044) 2024-09-18T14:13:46.954 INFO:tasks.rados.rados.0.smithi084.stdout:3111: finishing write tid 1 to smithi08434663-6 2024-09-18T14:13:46.954 INFO:tasks.rados.rados.0.smithi084.stdout:3111: finishing write tid 2 to smithi08434663-6 2024-09-18T14:13:46.954 INFO:tasks.rados.rados.0.smithi084.stdout:3111: finishing write tid 3 to smithi08434663-6 2024-09-18T14:13:46.954 INFO:tasks.rados.rados.0.smithi084.stdout:3111: finishing write tid 4 to smithi08434663-6 2024-09-18T14:13:46.954 INFO:tasks.rados.rados.0.smithi084.stdout:3111: finishing write tid 5 to smithi08434663-6 2024-09-18T14:13:46.955 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 1032 (ObjNum 1061 snap 339 seq_num 1061) dirty exists 2024-09-18T14:13:46.955 INFO:tasks.rados.rados.0.smithi084.stdout:3111: left oid 6 (ObjNum 1061 snap 339 seq_num 1061) 2024-09-18T14:13:46.955 INFO:tasks.rados.rados.0.smithi084.stdout:3114: seq_num 1063 ranges {665841=562754,1687653=572365,2708406=776032} 2024-09-18T14:13:46.961 INFO:tasks.rados.rados.0.smithi084.stdout:3114: writing smithi08434663-16 from 665841 to 1228595 tid 1 2024-09-18T14:13:46.967 INFO:tasks.rados.rados.0.smithi084.stdout:3114: writing smithi08434663-16 from 1687653 to 2260018 tid 2 2024-09-18T14:13:46.974 INFO:tasks.rados.rados.0.smithi084.stdout:3114: writing smithi08434663-16 from 2708406 to 3484438 tid 3 2024-09-18T14:13:46.974 INFO:tasks.rados.rados.0.smithi084.stdout:3106: done (7 left) 2024-09-18T14:13:46.974 INFO:tasks.rados.rados.0.smithi084.stdout:3108: done (6 left) 2024-09-18T14:13:46.974 INFO:tasks.rados.rados.0.smithi084.stdout:3109: done (5 left) 2024-09-18T14:13:46.974 INFO:tasks.rados.rados.0.smithi084.stdout:3110: done (4 left) 2024-09-18T14:13:46.974 INFO:tasks.rados.rados.0.smithi084.stdout:3111: done (3 left) 2024-09-18T14:13:46.975 INFO:tasks.rados.rados.0.smithi084.stdout:3115: write (excl) oid 38 current snap is 339 2024-09-18T14:13:46.975 INFO:tasks.rados.rados.0.smithi084.stdout:3115: seq_num 1064 ranges {682648=638003,2029782=432376,3019476=587304} 2024-09-18T14:13:46.981 INFO:tasks.rados.rados.0.smithi084.stdout:3115: writing smithi08434663-38 from 682648 to 1320651 tid 1 2024-09-18T14:13:46.987 INFO:tasks.rados.rados.0.smithi084.stdout:3115: writing smithi08434663-38 from 2029782 to 2462158 tid 2 2024-09-18T14:13:46.993 INFO:tasks.rados.rados.0.smithi084.stdout:3115: writing smithi08434663-38 from 3019476 to 3606780 tid 3 2024-09-18T14:13:46.993 INFO:tasks.rados.rados.0.smithi084.stdout:3116: write (excl) oid 20 current snap is 339 2024-09-18T14:13:46.993 INFO:tasks.rados.rados.0.smithi084.stdout:3116: seq_num 1065 ranges {699455=429605,1804617=576034,3163252=514929} 2024-09-18T14:13:46.999 INFO:tasks.rados.rados.0.smithi084.stdout:3116: writing smithi08434663-20 from 699455 to 1129060 tid 1 2024-09-18T14:13:47.005 INFO:tasks.rados.rados.0.smithi084.stdout:3116: writing smithi08434663-20 from 1804617 to 2380651 tid 2 2024-09-18T14:13:47.012 INFO:tasks.rados.rados.0.smithi084.stdout:3116: writing smithi08434663-20 from 3163252 to 3678181 tid 3 2024-09-18T14:13:47.012 INFO:tasks.rados.rados.0.smithi084.stdout:3117: snap_remove snap 337 2024-09-18T14:13:47.527 INFO:tasks.rados.rados.0.smithi084.stdout:3117: done (5 left) 2024-09-18T14:13:47.527 INFO:tasks.rados.rados.0.smithi084.stdout:3118: delete oid 12 current snap is 339 2024-09-18T14:13:47.528 INFO:tasks.rados.rados.0.smithi084.stdout:3112: expect (ObjNum 1033 snap 331 seq_num 1033) 2024-09-18T14:13:47.737 INFO:tasks.rados.rados.0.smithi084.stdout:3113: finishing write tid 1 to smithi08434663-40 2024-09-18T14:13:47.738 INFO:tasks.rados.rados.0.smithi084.stdout:3113: finishing write tid 2 to smithi08434663-40 2024-09-18T14:13:47.738 INFO:tasks.rados.rados.0.smithi084.stdout:3113: finishing write tid 3 to smithi08434663-40 2024-09-18T14:13:47.738 INFO:tasks.rados.rados.0.smithi084.stdout:3113: finishing write tid 4 to smithi08434663-40 2024-09-18T14:13:47.738 INFO:tasks.rados.rados.0.smithi084.stdout:3113: finishing write tid 5 to smithi08434663-40 2024-09-18T14:13:47.738 INFO:tasks.rados.rados.0.smithi084.stdout:3113: finishing write tid 6 to smithi08434663-40 2024-09-18T14:13:47.738 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 1542 (ObjNum 1062 snap 339 seq_num 1062) dirty exists 2024-09-18T14:13:47.738 INFO:tasks.rados.rados.0.smithi084.stdout:3113: left oid 40 (ObjNum 1062 snap 339 seq_num 1062) 2024-09-18T14:13:47.738 INFO:tasks.rados.rados.0.smithi084.stdout:3112: done (5 left) 2024-09-18T14:13:47.738 INFO:tasks.rados.rados.0.smithi084.stdout:3113: done (4 left) 2024-09-18T14:13:47.738 INFO:tasks.rados.rados.0.smithi084.stdout:3118: done (3 left) 2024-09-18T14:13:47.738 INFO:tasks.rados.rados.0.smithi084.stdout:3119: rollback oid 47 current snap is 339 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 47 to 332 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:3114: finishing write tid 1 to smithi08434663-16 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:3114: finishing write tid 2 to smithi08434663-16 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:3114: finishing write tid 3 to smithi08434663-16 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:3114: finishing write tid 4 to smithi08434663-16 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:3114: finishing write tid 5 to smithi08434663-16 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1480 (ObjNum 1063 snap 339 seq_num 1063) dirty exists 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:3114: left oid 16 (ObjNum 1063 snap 339 seq_num 1063) 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:3115: finishing write tid 1 to smithi08434663-38 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:3115: finishing write tid 2 to smithi08434663-38 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:3115: finishing write tid 3 to smithi08434663-38 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:3115: finishing write tid 4 to smithi08434663-38 2024-09-18T14:13:47.739 INFO:tasks.rados.rados.0.smithi084.stdout:3115: finishing write tid 5 to smithi08434663-38 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 1269 (ObjNum 1064 snap 339 seq_num 1064) dirty exists 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:3115: left oid 38 (ObjNum 1064 snap 339 seq_num 1064) 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:3116: finishing write tid 1 to smithi08434663-20 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:3116: finishing write tid 2 to smithi08434663-20 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:3116: finishing write tid 3 to smithi08434663-20 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:3116: finishing write tid 4 to smithi08434663-20 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:3116: finishing write tid 5 to smithi08434663-20 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 1403 (ObjNum 1065 snap 339 seq_num 1065) dirty exists 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:3116: left oid 20 (ObjNum 1065 snap 339 seq_num 1065) 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:3114: done (3 left) 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:3115: done (2 left) 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:3116: done (1 left) 2024-09-18T14:13:47.740 INFO:tasks.rados.rados.0.smithi084.stdout:3120: read oid 1 snap -1 2024-09-18T14:13:47.741 INFO:tasks.rados.rados.0.smithi084.stdout:3120: expect (ObjNum 961 snap 306 seq_num 961) 2024-09-18T14:13:47.741 INFO:tasks.rados.rados.0.smithi084.stdout:3121: snap_create 2024-09-18T14:13:47.741 INFO:tasks.rados.rados.0.smithi084.stdout:3120: expect (ObjNum 961 snap 306 seq_num 961) 2024-09-18T14:13:47.808 INFO:tasks.rados.rados.0.smithi084.stdout:3119: finishing rollback tid 1 to smithi08434663-47 2024-09-18T14:13:47.809 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 0 (ObjNum 337 snap 0 seq_num 2494578064) dirty dne 2024-09-18T14:13:48.473 INFO:tasks.rados.rados.0.smithi084.stdout:3119: done (2 left) 2024-09-18T14:13:48.473 INFO:tasks.rados.rados.0.smithi084.stdout:3120: done (1 left) 2024-09-18T14:13:48.473 INFO:tasks.rados.rados.0.smithi084.stdout:3121: done (0 left) 2024-09-18T14:13:48.473 INFO:tasks.rados.rados.0.smithi084.stdout:3122: write oid 27 current snap is 340 2024-09-18T14:13:48.473 INFO:tasks.rados.rados.0.smithi084.stdout:3122: seq_num 1067 ranges {733069=580103,1921581=579703,3218098=537473} 2024-09-18T14:13:48.479 INFO:tasks.rados.rados.0.smithi084.stdout:3122: writing smithi08434663-27 from 733069 to 1313172 tid 1 2024-09-18T14:13:48.487 INFO:tasks.rados.rados.0.smithi084.stdout:3122: writing smithi08434663-27 from 1921581 to 2501284 tid 2 2024-09-18T14:13:48.494 INFO:tasks.rados.rados.0.smithi084.stdout:3122: writing smithi08434663-27 from 3218098 to 3755571 tid 3 2024-09-18T14:13:48.495 INFO:tasks.rados.rados.0.smithi084.stdout:3123: read oid 29 snap 336 2024-09-18T14:13:48.495 INFO:tasks.rados.rados.0.smithi084.stdout:3123: expect (ObjNum 1017 snap 326 seq_num 1017) 2024-09-18T14:13:48.495 INFO:tasks.rados.rados.0.smithi084.stdout:3124: rollback oid 42 current snap is 340 2024-09-18T14:13:48.495 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 42 to 339 2024-09-18T14:13:48.495 INFO:tasks.rados.rados.0.smithi084.stdout:3122: finishing write tid 1 to smithi08434663-27 2024-09-18T14:13:48.495 INFO:tasks.rados.rados.0.smithi084.stdout:3122: finishing write tid 2 to smithi08434663-27 2024-09-18T14:13:48.495 INFO:tasks.rados.rados.0.smithi084.stdout:3125: copy_from oid 41 from oid 1 current snap is 340 2024-09-18T14:13:48.495 INFO:tasks.rados.rados.0.smithi084.stdout:3126: read oid 33 snap -1 2024-09-18T14:13:48.495 INFO:tasks.rados.rados.0.smithi084.stdout:3126: expect (ObjNum 1037 snap 333 seq_num 1037) 2024-09-18T14:13:48.495 INFO:tasks.rados.rados.0.smithi084.stdout:3127: write (excl) oid 19 current snap is 340 2024-09-18T14:13:48.495 INFO:tasks.rados.rados.0.smithi084.stdout:3127: seq_num 1068 ranges {749876=655352,1863710=723361,3012815=748745} 2024-09-18T14:13:48.503 INFO:tasks.rados.rados.0.smithi084.stdout:3127: writing smithi08434663-19 from 749876 to 1405228 tid 1 2024-09-18T14:13:48.511 INFO:tasks.rados.rados.0.smithi084.stdout:3127: writing smithi08434663-19 from 1863710 to 2587071 tid 2 2024-09-18T14:13:48.518 INFO:tasks.rados.rados.0.smithi084.stdout:3127: writing smithi08434663-19 from 3012815 to 3761560 tid 3 2024-09-18T14:13:48.518 INFO:tasks.rados.rados.0.smithi084.stdout:3128: read oid 48 snap -1 2024-09-18T14:13:48.518 INFO:tasks.rados.rados.0.smithi084.stdout:3128: expect (ObjNum 1036 snap 332 seq_num 1036) 2024-09-18T14:13:48.518 INFO:tasks.rados.rados.0.smithi084.stdout:3129: write (excl) oid 11 current snap is 340 2024-09-18T14:13:48.518 INFO:tasks.rados.rados.0.smithi084.stdout:3122: finishing write tid 3 to smithi08434663-27 2024-09-18T14:13:48.518 INFO:tasks.rados.rados.0.smithi084.stdout:3129: seq_num 1069 ranges {766683=730601,1922192=583372,3156591=676370} 2024-09-18T14:13:48.526 INFO:tasks.rados.rados.0.smithi084.stdout:3129: writing smithi08434663-11 from 766683 to 1497284 tid 1 2024-09-18T14:13:48.532 INFO:tasks.rados.rados.0.smithi084.stdout:3129: writing smithi08434663-11 from 1922192 to 2505564 tid 2 2024-09-18T14:13:48.540 INFO:tasks.rados.rados.0.smithi084.stdout:3129: writing smithi08434663-11 from 3156591 to 3832961 tid 3 2024-09-18T14:13:48.541 INFO:tasks.rados.rados.0.smithi084.stdout:3130: rollback oid 34 current snap is 340 2024-09-18T14:13:48.541 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 34 to 310 2024-09-18T14:13:48.541 INFO:tasks.rados.rados.0.smithi084.stdout:3122: finishing write tid 4 to smithi08434663-27 2024-09-18T14:13:48.541 INFO:tasks.rados.rados.0.smithi084.stdout:3122: finishing write tid 5 to smithi08434663-27 2024-09-18T14:13:48.541 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1406 (ObjNum 1067 snap 340 seq_num 1067) dirty exists 2024-09-18T14:13:48.541 INFO:tasks.rados.rados.0.smithi084.stdout:3122: left oid 27 (ObjNum 1067 snap 340 seq_num 1067) 2024-09-18T14:13:48.541 INFO:tasks.rados.rados.0.smithi084.stdout:3124: finishing rollback tid 0 to smithi08434663-42 2024-09-18T14:13:48.541 INFO:tasks.rados.rados.0.smithi084.stdout:3124: finishing rollback tid 1 to smithi08434663-42 2024-09-18T14:13:48.541 INFO:tasks.rados.rados.0.smithi084.stdout:3124: finishing rollback tid 2 to smithi08434663-42 2024-09-18T14:13:48.541 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1548 (ObjNum 1054 snap 336 seq_num 1054) dirty exists 2024-09-18T14:13:48.541 INFO:tasks.rados.rados.0.smithi084.stdout:3127: finishing write tid 1 to smithi08434663-19 2024-09-18T14:13:48.541 INFO:tasks.rados.rados.0.smithi084.stdout:3123: expect (ObjNum 1017 snap 326 seq_num 1017) 2024-09-18T14:13:48.639 INFO:tasks.rados.rados.0.smithi084.stdout:3125: finishing copy_from racing read to smithi08434663-41 2024-09-18T14:13:48.640 INFO:tasks.rados.rados.0.smithi084.stdout:3125: finishing copy_from to smithi08434663-41 2024-09-18T14:13:48.640 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 1236 (ObjNum 961 snap 306 seq_num 961) dirty exists 2024-09-18T14:13:48.640 INFO:tasks.rados.rados.0.smithi084.stdout:3126: expect (ObjNum 1037 snap 333 seq_num 1037) 2024-09-18T14:13:48.747 INFO:tasks.rados.rados.0.smithi084.stdout:3127: finishing write tid 2 to smithi08434663-19 2024-09-18T14:13:48.748 INFO:tasks.rados.rados.0.smithi084.stdout:3128: expect (ObjNum 1036 snap 332 seq_num 1036) 2024-09-18T14:13:48.986 INFO:tasks.rados.rados.0.smithi084.stdout:3127: finishing write tid 3 to smithi08434663-19 2024-09-18T14:13:48.986 INFO:tasks.rados.rados.0.smithi084.stdout:3127: finishing write tid 4 to smithi08434663-19 2024-09-18T14:13:48.986 INFO:tasks.rados.rados.0.smithi084.stdout:3127: finishing write tid 5 to smithi08434663-19 2024-09-18T14:13:48.987 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1691 (ObjNum 1068 snap 340 seq_num 1068) dirty exists 2024-09-18T14:13:48.987 INFO:tasks.rados.rados.0.smithi084.stdout:3127: left oid 19 (ObjNum 1068 snap 340 seq_num 1068) 2024-09-18T14:13:48.987 INFO:tasks.rados.rados.0.smithi084.stdout:3129: finishing write tid 1 to smithi08434663-11 2024-09-18T14:13:48.987 INFO:tasks.rados.rados.0.smithi084.stdout:3122: done (8 left) 2024-09-18T14:13:48.987 INFO:tasks.rados.rados.0.smithi084.stdout:3123: done (7 left) 2024-09-18T14:13:48.987 INFO:tasks.rados.rados.0.smithi084.stdout:3124: done (6 left) 2024-09-18T14:13:48.987 INFO:tasks.rados.rados.0.smithi084.stdout:3125: done (5 left) 2024-09-18T14:13:48.987 INFO:tasks.rados.rados.0.smithi084.stdout:3126: done (4 left) 2024-09-18T14:13:48.987 INFO:tasks.rados.rados.0.smithi084.stdout:3127: done (3 left) 2024-09-18T14:13:48.987 INFO:tasks.rados.rados.0.smithi084.stdout:3128: done (2 left) 2024-09-18T14:13:48.987 INFO:tasks.rados.rados.0.smithi084.stdout:3131: read oid 50 snap 339 2024-09-18T14:13:48.987 INFO:tasks.rados.rados.0.smithi084.stdout:3131: expect (ObjNum 1052 snap 336 seq_num 1052) 2024-09-18T14:13:48.988 INFO:tasks.rados.rados.0.smithi084.stdout:3132: delete oid 15 current snap is 340 2024-09-18T14:13:48.988 INFO:tasks.rados.rados.0.smithi084.stdout:3129: finishing write tid 2 to smithi08434663-11 2024-09-18T14:13:48.988 INFO:tasks.rados.rados.0.smithi084.stdout:3129: finishing write tid 3 to smithi08434663-11 2024-09-18T14:13:48.988 INFO:tasks.rados.rados.0.smithi084.stdout:3129: finishing write tid 4 to smithi08434663-11 2024-09-18T14:13:48.988 INFO:tasks.rados.rados.0.smithi084.stdout:3129: finishing write tid 5 to smithi08434663-11 2024-09-18T14:13:48.988 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1280 (ObjNum 1069 snap 340 seq_num 1069) dirty exists 2024-09-18T14:13:48.988 INFO:tasks.rados.rados.0.smithi084.stdout:3129: left oid 11 (ObjNum 1069 snap 340 seq_num 1069) 2024-09-18T14:13:48.988 INFO:tasks.rados.rados.0.smithi084.stdout:3130: finishing rollback tid 0 to smithi08434663-34 2024-09-18T14:13:48.988 INFO:tasks.rados.rados.0.smithi084.stdout:3130: finishing rollback tid 1 to smithi08434663-34 2024-09-18T14:13:48.988 INFO:tasks.rados.rados.0.smithi084.stdout:3130: finishing rollback tid 2 to smithi08434663-34 2024-09-18T14:13:48.988 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1551 (ObjNum 941 snap 301 seq_num 941) dirty exists 2024-09-18T14:13:48.989 INFO:tasks.rados.rados.0.smithi084.stdout:3129: done (3 left) 2024-09-18T14:13:48.989 INFO:tasks.rados.rados.0.smithi084.stdout:3130: done (2 left) 2024-09-18T14:13:48.989 INFO:tasks.rados.rados.0.smithi084.stdout:3132: done (1 left) 2024-09-18T14:13:48.989 INFO:tasks.rados.rados.0.smithi084.stdout:3133: copy_from oid 27 from oid 22 current snap is 340 2024-09-18T14:13:48.989 INFO:tasks.rados.rados.0.smithi084.stdout:3134: copy_from oid 24 from oid 48 current snap is 340 2024-09-18T14:13:48.989 INFO:tasks.rados.rados.0.smithi084.stdout:3135: copy_from oid 14 from oid 8 current snap is 340 2024-09-18T14:13:48.990 INFO:tasks.rados.rados.0.smithi084.stdout:3136: delete oid 5 current snap is 340 2024-09-18T14:13:48.991 INFO:tasks.rados.rados.0.smithi084.stdout:3131: expect (ObjNum 1052 snap 336 seq_num 1052) 2024-09-18T14:13:49.160 INFO:tasks.rados.rados.0.smithi084.stdout:3133: finishing copy_from racing read to smithi08434663-27 2024-09-18T14:13:49.160 INFO:tasks.rados.rados.0.smithi084.stdout:3134: finishing copy_from racing read to smithi08434663-24 2024-09-18T14:13:49.160 INFO:tasks.rados.rados.0.smithi084.stdout:3133: finishing copy_from to smithi08434663-27 2024-09-18T14:13:49.160 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1040 (ObjNum 991 snap 321 seq_num 991) dirty exists 2024-09-18T14:13:49.160 INFO:tasks.rados.rados.0.smithi084.stdout:3134: finishing copy_from to smithi08434663-24 2024-09-18T14:13:49.160 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 1523 (ObjNum 1036 snap 332 seq_num 1036) dirty exists 2024-09-18T14:13:49.161 INFO:tasks.rados.rados.0.smithi084.stdout:3135: finishing copy_from racing read to smithi08434663-14 2024-09-18T14:13:49.161 INFO:tasks.rados.rados.0.smithi084.stdout:3135: finishing copy_from to smithi08434663-14 2024-09-18T14:13:49.161 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 1395 (ObjNum 1041 snap 333 seq_num 1041) dirty exists 2024-09-18T14:13:49.161 INFO:tasks.rados.rados.0.smithi084.stdout:3131: done (4 left) 2024-09-18T14:13:49.161 INFO:tasks.rados.rados.0.smithi084.stdout:3133: done (3 left) 2024-09-18T14:13:49.161 INFO:tasks.rados.rados.0.smithi084.stdout:3134: done (2 left) 2024-09-18T14:13:49.161 INFO:tasks.rados.rados.0.smithi084.stdout:3135: done (1 left) 2024-09-18T14:13:49.161 INFO:tasks.rados.rados.0.smithi084.stdout:3136: done (0 left) 2024-09-18T14:13:49.161 INFO:tasks.rados.rados.0.smithi084.stdout:3137: snap_create 2024-09-18T14:13:49.532 INFO:tasks.rados.rados.0.smithi084.stdout:3137: done (0 left) 2024-09-18T14:13:49.532 INFO:tasks.rados.rados.0.smithi084.stdout:3138: rollback oid 50 current snap is 341 2024-09-18T14:13:49.532 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 50 to 332 2024-09-18T14:13:49.532 INFO:tasks.rados.rados.0.smithi084.stdout:3139: write (excl) oid 44 current snap is 341 2024-09-18T14:13:49.532 INFO:tasks.rados.rados.0.smithi084.stdout:3139: seq_num 1072 ranges {417104=672701,1813991=203113} 2024-09-18T14:13:49.537 INFO:tasks.rados.rados.0.smithi084.stdout:3139: writing smithi08434663-44 from 417104 to 1089805 tid 1 2024-09-18T14:13:49.544 INFO:tasks.rados.rados.0.smithi084.stdout:3139: writing smithi08434663-44 from 1813991 to 2017104 tid 2 2024-09-18T14:13:49.544 INFO:tasks.rados.rados.0.smithi084.stdout:3140: read oid 39 snap -1 2024-09-18T14:13:49.544 INFO:tasks.rados.rados.0.smithi084.stdout:3140: expect (ObjNum 1044 snap 334 seq_num 1044) 2024-09-18T14:13:49.544 INFO:tasks.rados.rados.0.smithi084.stdout:3141: rollback oid 42 current snap is 341 2024-09-18T14:13:49.544 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 42 to 329 2024-09-18T14:13:49.544 INFO:tasks.rados.rados.0.smithi084.stdout:3138: finishing rollback tid 0 to smithi08434663-50 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:3138: finishing rollback tid 1 to smithi08434663-50 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:3138: finishing rollback tid 2 to smithi08434663-50 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 1696 (ObjNum 1001 snap 322 seq_num 1001) dirty exists 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:3138: done (3 left) 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:3142: read oid 22 snap 329 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:3142: expect (ObjNum 991 snap 321 seq_num 991) 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:3143: read oid 36 snap -1 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:3143: expect (ObjNum 1040 snap 333 seq_num 1040) 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:3144: read oid 25 snap 334 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:3144: expect (ObjNum 998 snap 322 seq_num 998) 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:3145: read oid 31 snap -1 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:3145: expect deleted 2024-09-18T14:13:49.545 INFO:tasks.rados.rados.0.smithi084.stdout:3146: read oid 40 snap 329 2024-09-18T14:13:49.546 INFO:tasks.rados.rados.0.smithi084.stdout:3146: expect (ObjNum 971 snap 308 seq_num 971) 2024-09-18T14:13:49.546 INFO:tasks.rados.rados.0.smithi084.stdout:3147: rollback oid 8 current snap is 341 2024-09-18T14:13:49.546 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 8 to 330 2024-09-18T14:13:49.546 INFO:tasks.rados.rados.0.smithi084.stdout:3148: write (excl) oid 11 current snap is 341 2024-09-18T14:13:49.546 INFO:tasks.rados.rados.0.smithi084.stdout:3148: seq_num 1073 ranges {433911=747950,1756120=277791} 2024-09-18T14:13:49.551 INFO:tasks.rados.rados.0.smithi084.stdout:3148: writing smithi08434663-11 from 433911 to 1181861 tid 1 2024-09-18T14:13:49.556 INFO:tasks.rados.rados.0.smithi084.stdout:3148: writing smithi08434663-11 from 1756120 to 2033911 tid 2 2024-09-18T14:13:49.556 INFO:tasks.rados.rados.0.smithi084.stdout:3149: read oid 50 snap -1 2024-09-18T14:13:49.557 INFO:tasks.rados.rados.0.smithi084.stdout:3149: expect (ObjNum 1001 snap 322 seq_num 1001) 2024-09-18T14:13:49.557 INFO:tasks.rados.rados.0.smithi084.stdout:3150: write (excl) oid 9 current snap is 341 2024-09-18T14:13:49.557 INFO:tasks.rados.rados.0.smithi084.stdout:3150: seq_num 1074 ranges {450718=423199,1414602=636116} 2024-09-18T14:13:49.561 INFO:tasks.rados.rados.0.smithi084.stdout:3150: writing smithi08434663-9 from 450718 to 873917 tid 1 2024-09-18T14:13:49.569 INFO:tasks.rados.rados.0.smithi084.stdout:3150: writing smithi08434663-9 from 1414602 to 2050718 tid 2 2024-09-18T14:13:49.569 INFO:tasks.rados.rados.0.smithi084.stdout:3151: delete oid 45 current snap is 341 2024-09-18T14:13:49.569 INFO:tasks.rados.rados.0.smithi084.stdout:3139: finishing write tid 1 to smithi08434663-44 2024-09-18T14:13:49.569 INFO:tasks.rados.rados.0.smithi084.stdout:3139: finishing write tid 2 to smithi08434663-44 2024-09-18T14:13:49.569 INFO:tasks.rados.rados.0.smithi084.stdout:3139: finishing write tid 3 to smithi08434663-44 2024-09-18T14:13:49.569 INFO:tasks.rados.rados.0.smithi084.stdout:3139: finishing write tid 4 to smithi08434663-44 2024-09-18T14:13:49.569 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 1043 (ObjNum 1072 snap 341 seq_num 1072) dirty exists 2024-09-18T14:13:49.569 INFO:tasks.rados.rados.0.smithi084.stdout:3139: left oid 44 (ObjNum 1072 snap 341 seq_num 1072) 2024-09-18T14:13:49.569 INFO:tasks.rados.rados.0.smithi084.stdout:3147: finishing rollback tid 0 to smithi08434663-8 2024-09-18T14:13:49.569 INFO:tasks.rados.rados.0.smithi084.stdout:3147: finishing rollback tid 1 to smithi08434663-8 2024-09-18T14:13:49.570 INFO:tasks.rados.rados.0.smithi084.stdout:3147: finishing rollback tid 2 to smithi08434663-8 2024-09-18T14:13:49.570 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 1406 (ObjNum 981 snap 312 seq_num 981) dirty exists 2024-09-18T14:13:49.570 INFO:tasks.rados.rados.0.smithi084.stdout:3140: expect (ObjNum 1044 snap 334 seq_num 1044) 2024-09-18T14:13:49.774 INFO:tasks.rados.rados.0.smithi084.stdout:3141: finishing rollback tid 0 to smithi08434663-42 2024-09-18T14:13:49.775 INFO:tasks.rados.rados.0.smithi084.stdout:3141: finishing rollback tid 1 to smithi08434663-42 2024-09-18T14:13:49.775 INFO:tasks.rados.rados.0.smithi084.stdout:3141: finishing rollback tid 2 to smithi08434663-42 2024-09-18T14:13:49.775 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1554 (ObjNum 985 snap 316 seq_num 985) dirty exists 2024-09-18T14:13:49.775 INFO:tasks.rados.rados.0.smithi084.stdout:3144: expect (ObjNum 998 snap 322 seq_num 998) 2024-09-18T14:13:49.921 INFO:tasks.rados.rados.0.smithi084.stdout:3149: expect (ObjNum 1001 snap 322 seq_num 1001) 2024-09-18T14:13:50.013 INFO:tasks.rados.rados.0.smithi084.stdout:3148: finishing write tid 1 to smithi08434663-11 2024-09-18T14:13:50.014 INFO:tasks.rados.rados.0.smithi084.stdout:3148: finishing write tid 2 to smithi08434663-11 2024-09-18T14:13:50.014 INFO:tasks.rados.rados.0.smithi084.stdout:3146: expect (ObjNum 971 snap 308 seq_num 971) 2024-09-18T14:13:50.108 INFO:tasks.rados.rados.0.smithi084.stdout:3148: finishing write tid 3 to smithi08434663-11 2024-09-18T14:13:50.108 INFO:tasks.rados.rados.0.smithi084.stdout:3148: finishing write tid 4 to smithi08434663-11 2024-09-18T14:13:50.108 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1283 (ObjNum 1073 snap 341 seq_num 1073) dirty exists 2024-09-18T14:13:50.108 INFO:tasks.rados.rados.0.smithi084.stdout:3148: left oid 11 (ObjNum 1073 snap 341 seq_num 1073) 2024-09-18T14:13:50.108 INFO:tasks.rados.rados.0.smithi084.stdout:3142: expect (ObjNum 991 snap 321 seq_num 991) 2024-09-18T14:13:50.265 INFO:tasks.rados.rados.0.smithi084.stdout:3143: expect (ObjNum 1040 snap 333 seq_num 1040) 2024-09-18T14:13:50.433 INFO:tasks.rados.rados.0.smithi084.stdout:3150: finishing write tid 1 to smithi08434663-9 2024-09-18T14:13:50.433 INFO:tasks.rados.rados.0.smithi084.stdout:3150: finishing write tid 2 to smithi08434663-9 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3150: finishing write tid 3 to smithi08434663-9 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3150: finishing write tid 4 to smithi08434663-9 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 1483 (ObjNum 1074 snap 341 seq_num 1074) dirty exists 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3150: left oid 9 (ObjNum 1074 snap 341 seq_num 1074) 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3139: done (12 left) 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3140: done (11 left) 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3141: done (10 left) 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3142: done (9 left) 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3143: done (8 left) 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3144: done (7 left) 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3145: done (6 left) 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3146: done (5 left) 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3147: done (4 left) 2024-09-18T14:13:50.434 INFO:tasks.rados.rados.0.smithi084.stdout:3148: done (3 left) 2024-09-18T14:13:50.435 INFO:tasks.rados.rados.0.smithi084.stdout:3149: done (2 left) 2024-09-18T14:13:50.435 INFO:tasks.rados.rados.0.smithi084.stdout:3150: done (1 left) 2024-09-18T14:13:50.435 INFO:tasks.rados.rados.0.smithi084.stdout:3151: done (0 left) 2024-09-18T14:13:50.435 INFO:tasks.rados.rados.0.smithi084.stdout:3152: write oid 47 current snap is 341 2024-09-18T14:13:50.435 INFO:tasks.rados.rados.0.smithi084.stdout:3152: seq_num 1076 ranges {484332=573697,1531566=552766} 2024-09-18T14:13:50.438 INFO:tasks.rados.rados.0.smithi084.stdout:3152: writing smithi08434663-47 from 484332 to 1058029 tid 1 2024-09-18T14:13:50.445 INFO:tasks.rados.rados.0.smithi084.stdout:3152: writing smithi08434663-47 from 1531566 to 2084332 tid 2 2024-09-18T14:13:50.445 INFO:tasks.rados.rados.0.smithi084.stdout:3153: rollback oid 32 current snap is 341 2024-09-18T14:13:50.445 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 32 to 321 2024-09-18T14:13:50.445 INFO:tasks.rados.rados.0.smithi084.stdout:3152: finishing write tid 1 to smithi08434663-47 2024-09-18T14:13:50.445 INFO:tasks.rados.rados.0.smithi084.stdout:3154: read oid 30 snap -1 2024-09-18T14:13:50.445 INFO:tasks.rados.rados.0.smithi084.stdout:3154: expect (ObjNum 1021 snap 326 seq_num 1021) 2024-09-18T14:13:50.445 INFO:tasks.rados.rados.0.smithi084.stdout:3155: read oid 35 snap -1 2024-09-18T14:13:50.445 INFO:tasks.rados.rados.0.smithi084.stdout:3155: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:13:50.446 INFO:tasks.rados.rados.0.smithi084.stdout:3156: delete oid 40 current snap is 341 2024-09-18T14:13:50.448 INFO:tasks.rados.rados.0.smithi084.stdout:3155: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:13:50.601 INFO:tasks.rados.rados.0.smithi084.stdout:3154: expect (ObjNum 1021 snap 326 seq_num 1021) 2024-09-18T14:13:50.807 INFO:tasks.rados.rados.0.smithi084.stdout:3152: finishing write tid 2 to smithi08434663-47 2024-09-18T14:13:50.807 INFO:tasks.rados.rados.0.smithi084.stdout:3153: finishing rollback tid 0 to smithi08434663-32 2024-09-18T14:13:50.807 INFO:tasks.rados.rados.0.smithi084.stdout:3152: finishing write tid 3 to smithi08434663-47 2024-09-18T14:13:50.807 INFO:tasks.rados.rados.0.smithi084.stdout:3152: finishing write tid 4 to smithi08434663-47 2024-09-18T14:13:50.807 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 1286 (ObjNum 1076 snap 341 seq_num 1076) dirty exists 2024-09-18T14:13:50.807 INFO:tasks.rados.rados.0.smithi084.stdout:3152: left oid 47 (ObjNum 1076 snap 341 seq_num 1076) 2024-09-18T14:13:50.807 INFO:tasks.rados.rados.0.smithi084.stdout:3153: finishing rollback tid 1 to smithi08434663-32 2024-09-18T14:13:50.807 INFO:tasks.rados.rados.0.smithi084.stdout:3153: finishing rollback tid 2 to smithi08434663-32 2024-09-18T14:13:50.807 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 1700 (ObjNum 901 snap 281 seq_num 901) dirty exists 2024-09-18T14:13:50.807 INFO:tasks.rados.rados.0.smithi084.stdout:3152: done (4 left) 2024-09-18T14:13:50.808 INFO:tasks.rados.rados.0.smithi084.stdout:3153: done (3 left) 2024-09-18T14:13:50.808 INFO:tasks.rados.rados.0.smithi084.stdout:3154: done (2 left) 2024-09-18T14:13:50.808 INFO:tasks.rados.rados.0.smithi084.stdout:3155: done (1 left) 2024-09-18T14:13:50.808 INFO:tasks.rados.rados.0.smithi084.stdout:3156: done (0 left) 2024-09-18T14:13:50.808 INFO:tasks.rados.rados.0.smithi084.stdout:3157: delete oid 31 current snap is 341 2024-09-18T14:13:50.809 INFO:tasks.rados.rados.0.smithi084.stdout:3157: done (0 left) 2024-09-18T14:13:50.809 INFO:tasks.rados.rados.0.smithi084.stdout:3158: snap_remove snap 336 2024-09-18T14:13:51.543 INFO:tasks.rados.rados.0.smithi084.stdout:3158: done (0 left) 2024-09-18T14:13:51.543 INFO:tasks.rados.rados.0.smithi084.stdout:3159: copy_from oid 28 from oid 9 current snap is 341 2024-09-18T14:13:51.543 INFO:tasks.rados.rados.0.smithi084.stdout:3160: copy_from oid 15 from oid 11 current snap is 341 2024-09-18T14:13:51.543 INFO:tasks.rados.rados.0.smithi084.stdout:3161: read oid 18 snap -1 2024-09-18T14:13:51.543 INFO:tasks.rados.rados.0.smithi084.stdout:3161: expect (ObjNum 937 snap 299 seq_num 937) 2024-09-18T14:13:51.543 INFO:tasks.rados.rados.0.smithi084.stdout:3162: snap_remove snap 318 2024-09-18T14:13:52.548 INFO:tasks.rados.rados.0.smithi084.stdout:3162: done (3 left) 2024-09-18T14:13:52.549 INFO:tasks.rados.rados.0.smithi084.stdout:3163: write oid 35 current snap is 341 2024-09-18T14:13:52.549 INFO:tasks.rados.rados.0.smithi084.stdout:3163: seq_num 1079 ranges {534753=799444,1990659=144094} 2024-09-18T14:13:52.556 INFO:tasks.rados.rados.0.smithi084.stdout:3163: writing smithi08434663-35 from 534753 to 1334197 tid 1 2024-09-18T14:13:52.562 INFO:tasks.rados.rados.0.smithi084.stdout:3163: writing smithi08434663-35 from 1990659 to 2134753 tid 2 2024-09-18T14:13:52.562 INFO:tasks.rados.rados.0.smithi084.stdout:3164: delete oid 4 current snap is 341 2024-09-18T14:13:52.562 INFO:tasks.rados.rados.0.smithi084.stdout:3160: finishing copy_from racing read to smithi08434663-15 2024-09-18T14:13:52.562 INFO:tasks.rados.rados.0.smithi084.stdout:3159: finishing copy_from racing read to smithi08434663-28 2024-09-18T14:13:52.562 INFO:tasks.rados.rados.0.smithi084.stdout:3161: expect (ObjNum 937 snap 299 seq_num 937) 2024-09-18T14:13:52.838 INFO:tasks.rados.rados.0.smithi084.stdout:3160: finishing copy_from to smithi08434663-15 2024-09-18T14:13:52.838 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 1283 (ObjNum 1073 snap 341 seq_num 1073) dirty exists 2024-09-18T14:13:52.838 INFO:tasks.rados.rados.0.smithi084.stdout:3159: finishing copy_from to smithi08434663-28 2024-09-18T14:13:52.838 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1483 (ObjNum 1074 snap 341 seq_num 1074) dirty exists 2024-09-18T14:13:52.838 INFO:tasks.rados.rados.0.smithi084.stdout:3163: finishing write tid 1 to smithi08434663-35 2024-09-18T14:13:52.838 INFO:tasks.rados.rados.0.smithi084.stdout:3159: done (4 left) 2024-09-18T14:13:52.838 INFO:tasks.rados.rados.0.smithi084.stdout:3160: done (3 left) 2024-09-18T14:13:52.838 INFO:tasks.rados.rados.0.smithi084.stdout:3161: done (2 left) 2024-09-18T14:13:52.838 INFO:tasks.rados.rados.0.smithi084.stdout:3164: done (1 left) 2024-09-18T14:13:52.838 INFO:tasks.rados.rados.0.smithi084.stdout:3165: read oid 49 snap -1 2024-09-18T14:13:52.839 INFO:tasks.rados.rados.0.smithi084.stdout:3165: expect (ObjNum 1020 snap 326 seq_num 1020) 2024-09-18T14:13:52.839 INFO:tasks.rados.rados.0.smithi084.stdout:3166: read oid 27 snap -1 2024-09-18T14:13:52.839 INFO:tasks.rados.rados.0.smithi084.stdout:3166: expect (ObjNum 991 snap 321 seq_num 991) 2024-09-18T14:13:52.839 INFO:tasks.rados.rados.0.smithi084.stdout:3167: delete oid 24 current snap is 341 2024-09-18T14:13:52.839 INFO:tasks.rados.rados.0.smithi084.stdout:3163: finishing write tid 2 to smithi08434663-35 2024-09-18T14:13:52.839 INFO:tasks.rados.rados.0.smithi084.stdout:3163: finishing write tid 3 to smithi08434663-35 2024-09-18T14:13:52.839 INFO:tasks.rados.rados.0.smithi084.stdout:3163: finishing write tid 4 to smithi08434663-35 2024-09-18T14:13:52.839 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 1486 (ObjNum 1079 snap 341 seq_num 1079) dirty exists 2024-09-18T14:13:52.839 INFO:tasks.rados.rados.0.smithi084.stdout:3163: left oid 35 (ObjNum 1079 snap 341 seq_num 1079) 2024-09-18T14:13:52.840 INFO:tasks.rados.rados.0.smithi084.stdout:3163: done (3 left) 2024-09-18T14:13:52.841 INFO:tasks.rados.rados.0.smithi084.stdout:3167: done (2 left) 2024-09-18T14:13:52.841 INFO:tasks.rados.rados.0.smithi084.stdout:3168: read oid 20 snap -1 2024-09-18T14:13:52.841 INFO:tasks.rados.rados.0.smithi084.stdout:3168: expect (ObjNum 1065 snap 339 seq_num 1065) 2024-09-18T14:13:52.841 INFO:tasks.rados.rados.0.smithi084.stdout:3169: copy_from oid 25 from oid 17 current snap is 341 2024-09-18T14:13:52.841 INFO:tasks.rados.rados.0.smithi084.stdout:3170: snap_remove snap 313 2024-09-18T14:13:53.475 INFO:tasks.rados.rados.0.smithi084.stdout:3170: done (4 left) 2024-09-18T14:13:53.475 INFO:tasks.rados.rados.0.smithi084.stdout:3171: write (excl) oid 33 current snap is 341 2024-09-18T14:13:53.475 INFO:tasks.rados.rados.0.smithi084.stdout:3171: seq_num 1082 ranges {585174=741544,1766105=419069} 2024-09-18T14:13:53.482 INFO:tasks.rados.rados.0.smithi084.stdout:3171: writing smithi08434663-33 from 585174 to 1326718 tid 1 2024-09-18T14:13:53.489 INFO:tasks.rados.rados.0.smithi084.stdout:3171: writing smithi08434663-33 from 1766105 to 2185174 tid 2 2024-09-18T14:13:53.489 INFO:tasks.rados.rados.0.smithi084.stdout:3172: copy_from oid 35 from oid 16 current snap is 341 2024-09-18T14:13:53.489 INFO:tasks.rados.rados.0.smithi084.stdout:3173: write (excl) oid 50 current snap is 341 2024-09-18T14:13:53.489 INFO:tasks.rados.rados.0.smithi084.stdout:3173: seq_num 1083 ranges {601981=416793,1424587=492702} 2024-09-18T14:13:53.494 INFO:tasks.rados.rados.0.smithi084.stdout:3173: writing smithi08434663-50 from 601981 to 1018774 tid 1 2024-09-18T14:13:53.500 INFO:tasks.rados.rados.0.smithi084.stdout:3173: writing smithi08434663-50 from 1424587 to 1917289 tid 2 2024-09-18T14:13:53.500 INFO:tasks.rados.rados.0.smithi084.stdout:3174: delete oid 46 current snap is 341 2024-09-18T14:13:53.500 INFO:tasks.rados.rados.0.smithi084.stdout:3166: expect (ObjNum 991 snap 321 seq_num 991) 2024-09-18T14:13:53.660 INFO:tasks.rados.rados.0.smithi084.stdout:3169: finishing copy_from racing read to smithi08434663-25 2024-09-18T14:13:53.661 INFO:tasks.rados.rados.0.smithi084.stdout:3168: expect (ObjNum 1065 snap 339 seq_num 1065) 2024-09-18T14:13:53.877 INFO:tasks.rados.rados.0.smithi084.stdout:3165: expect (ObjNum 1020 snap 326 seq_num 1020) 2024-09-18T14:13:54.080 INFO:tasks.rados.rados.0.smithi084.stdout:3169: finishing copy_from to smithi08434663-25 2024-09-18T14:13:54.080 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 1527 (ObjNum 1031 snap 330 seq_num 1031) dirty exists 2024-09-18T14:13:54.080 INFO:tasks.rados.rados.0.smithi084.stdout:3171: finishing write tid 1 to smithi08434663-33 2024-09-18T14:13:54.081 INFO:tasks.rados.rados.0.smithi084.stdout:3171: finishing write tid 2 to smithi08434663-33 2024-09-18T14:13:54.081 INFO:tasks.rados.rados.0.smithi084.stdout:3165: done (7 left) 2024-09-18T14:13:54.081 INFO:tasks.rados.rados.0.smithi084.stdout:3166: done (6 left) 2024-09-18T14:13:54.081 INFO:tasks.rados.rados.0.smithi084.stdout:3168: done (5 left) 2024-09-18T14:13:54.081 INFO:tasks.rados.rados.0.smithi084.stdout:3169: done (4 left) 2024-09-18T14:13:54.081 INFO:tasks.rados.rados.0.smithi084.stdout:3174: done (3 left) 2024-09-18T14:13:54.081 INFO:tasks.rados.rados.0.smithi084.stdout:3175: read oid 42 snap -1 2024-09-18T14:13:54.081 INFO:tasks.rados.rados.0.smithi084.stdout:3175: expect (ObjNum 985 snap 316 seq_num 985) 2024-09-18T14:13:54.082 INFO:tasks.rados.rados.0.smithi084.stdout:3176: rollback oid 15 current snap is 341 2024-09-18T14:13:54.082 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 15 to 339 2024-09-18T14:13:54.082 INFO:tasks.rados.rados.0.smithi084.stdout:3171: finishing write tid 3 to smithi08434663-33 2024-09-18T14:13:54.082 INFO:tasks.rados.rados.0.smithi084.stdout:3171: finishing write tid 4 to smithi08434663-33 2024-09-18T14:13:54.082 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 1332 (ObjNum 1082 snap 341 seq_num 1082) dirty exists 2024-09-18T14:13:54.082 INFO:tasks.rados.rados.0.smithi084.stdout:3171: left oid 33 (ObjNum 1082 snap 341 seq_num 1082) 2024-09-18T14:13:54.082 INFO:tasks.rados.rados.0.smithi084.stdout:3173: finishing write tid 1 to smithi08434663-50 2024-09-18T14:13:54.082 INFO:tasks.rados.rados.0.smithi084.stdout:3173: finishing write tid 2 to smithi08434663-50 2024-09-18T14:13:54.083 INFO:tasks.rados.rados.0.smithi084.stdout:3173: finishing write tid 3 to smithi08434663-50 2024-09-18T14:13:54.083 INFO:tasks.rados.rados.0.smithi084.stdout:3173: finishing write tid 4 to smithi08434663-50 2024-09-18T14:13:54.083 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 1733 (ObjNum 1083 snap 341 seq_num 1083) dirty exists 2024-09-18T14:13:54.083 INFO:tasks.rados.rados.0.smithi084.stdout:3173: left oid 50 (ObjNum 1083 snap 341 seq_num 1083) 2024-09-18T14:13:54.083 INFO:tasks.rados.rados.0.smithi084.stdout:3172: finishing copy_from racing read to smithi08434663-35 2024-09-18T14:13:54.083 INFO:tasks.rados.rados.0.smithi084.stdout:3172: finishing copy_from to smithi08434663-35 2024-09-18T14:13:54.083 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 1480 (ObjNum 1063 snap 339 seq_num 1063) dirty exists 2024-09-18T14:13:54.084 INFO:tasks.rados.rados.0.smithi084.stdout:3171: done (4 left) 2024-09-18T14:13:54.084 INFO:tasks.rados.rados.0.smithi084.stdout:3172: done (3 left) 2024-09-18T14:13:54.084 INFO:tasks.rados.rados.0.smithi084.stdout:3173: done (2 left) 2024-09-18T14:13:54.084 INFO:tasks.rados.rados.0.smithi084.stdout:3177: snap_create 2024-09-18T14:13:54.084 INFO:tasks.rados.rados.0.smithi084.stdout:3176: finishing rollback tid 0 to smithi08434663-15 2024-09-18T14:13:54.084 INFO:tasks.rados.rados.0.smithi084.stdout:3176: finishing rollback tid 1 to smithi08434663-15 2024-09-18T14:13:54.084 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 1489 (ObjNum 129 snap 0 seq_num 3385454817) dirty dne 2024-09-18T14:13:54.087 INFO:tasks.rados.rados.0.smithi084.stdout:3175: expect (ObjNum 985 snap 316 seq_num 985) 2024-09-18T14:13:54.553 INFO:tasks.rados.rados.0.smithi084.stdout:3175: done (2 left) 2024-09-18T14:13:54.553 INFO:tasks.rados.rados.0.smithi084.stdout:3176: done (1 left) 2024-09-18T14:13:54.553 INFO:tasks.rados.rados.0.smithi084.stdout:3177: done (0 left) 2024-09-18T14:13:54.554 INFO:tasks.rados.rados.0.smithi084.stdout:3178: read oid 25 snap 338 2024-09-18T14:13:54.554 INFO:tasks.rados.rados.0.smithi084.stdout:3178: expect (ObjNum 998 snap 322 seq_num 998) 2024-09-18T14:13:54.554 INFO:tasks.rados.rados.0.smithi084.stdout:3179: delete oid 17 current snap is 342 2024-09-18T14:13:54.556 INFO:tasks.rados.rados.0.smithi084.stdout:3178: expect (ObjNum 998 snap 322 seq_num 998) 2024-09-18T14:13:54.705 INFO:tasks.rados.rados.0.smithi084.stdout:3178: done (1 left) 2024-09-18T14:13:54.705 INFO:tasks.rados.rados.0.smithi084.stdout:3179: done (0 left) 2024-09-18T14:13:54.705 INFO:tasks.rados.rados.0.smithi084.stdout:3180: snap_remove snap 310 2024-09-18T14:13:55.557 INFO:tasks.rados.rados.0.smithi084.stdout:3180: done (0 left) 2024-09-18T14:13:55.557 INFO:tasks.rados.rados.0.smithi084.stdout:3181: copy_from oid 26 from oid 28 current snap is 342 2024-09-18T14:13:55.557 INFO:tasks.rados.rados.0.smithi084.stdout:3182: delete oid 15 current snap is 342 2024-09-18T14:13:55.565 INFO:tasks.rados.rados.0.smithi084.stdout:3182: done (1 left) 2024-09-18T14:13:55.565 INFO:tasks.rados.rados.0.smithi084.stdout:3183: delete oid 34 current snap is 342 2024-09-18T14:13:55.571 INFO:tasks.rados.rados.0.smithi084.stdout:3183: done (1 left) 2024-09-18T14:13:55.571 INFO:tasks.rados.rados.0.smithi084.stdout:3184: write (excl) oid 48 current snap is 342 2024-09-18T14:13:55.572 INFO:tasks.rados.rados.0.smithi084.stdout:3184: seq_num 1088 ranges {686016=509391,1716997=569019} 2024-09-18T14:13:55.578 INFO:tasks.rados.rados.0.smithi084.stdout:3184: writing smithi08434663-48 from 686016 to 1195407 tid 1 2024-09-18T14:13:55.584 INFO:tasks.rados.rados.0.smithi084.stdout:3184: writing smithi08434663-48 from 1716997 to 2286016 tid 2 2024-09-18T14:13:55.585 INFO:tasks.rados.rados.0.smithi084.stdout:3185: delete oid 29 current snap is 342 2024-09-18T14:13:55.585 INFO:tasks.rados.rados.0.smithi084.stdout:3181: finishing copy_from racing read to smithi08434663-26 2024-09-18T14:13:55.585 INFO:tasks.rados.rados.0.smithi084.stdout:3181: finishing copy_from to smithi08434663-26 2024-09-18T14:13:55.585 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 1483 (ObjNum 1074 snap 341 seq_num 1074) dirty exists 2024-09-18T14:13:55.585 INFO:tasks.rados.rados.0.smithi084.stdout:3184: finishing write tid 1 to smithi08434663-48 2024-09-18T14:13:55.587 INFO:tasks.rados.rados.0.smithi084.stdout:3181: done (2 left) 2024-09-18T14:13:55.587 INFO:tasks.rados.rados.0.smithi084.stdout:3185: done (1 left) 2024-09-18T14:13:55.587 INFO:tasks.rados.rados.0.smithi084.stdout:3186: read oid 7 snap -1 2024-09-18T14:13:55.587 INFO:tasks.rados.rados.0.smithi084.stdout:3186: expect deleted 2024-09-18T14:13:55.588 INFO:tasks.rados.rados.0.smithi084.stdout:3187: snap_create 2024-09-18T14:13:55.588 INFO:tasks.rados.rados.0.smithi084.stdout:3184: finishing write tid 2 to smithi08434663-48 2024-09-18T14:13:55.588 INFO:tasks.rados.rados.0.smithi084.stdout:3184: finishing write tid 3 to smithi08434663-48 2024-09-18T14:13:55.588 INFO:tasks.rados.rados.0.smithi084.stdout:3184: finishing write tid 4 to smithi08434663-48 2024-09-18T14:13:55.588 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1530 (ObjNum 1088 snap 342 seq_num 1088) dirty exists 2024-09-18T14:13:55.588 INFO:tasks.rados.rados.0.smithi084.stdout:3184: left oid 48 (ObjNum 1088 snap 342 seq_num 1088) 2024-09-18T14:13:56.567 INFO:tasks.rados.rados.0.smithi084.stdout:3184: done (2 left) 2024-09-18T14:13:56.568 INFO:tasks.rados.rados.0.smithi084.stdout:3186: done (1 left) 2024-09-18T14:13:56.568 INFO:tasks.rados.rados.0.smithi084.stdout:3187: done (0 left) 2024-09-18T14:13:56.568 INFO:tasks.rados.rados.0.smithi084.stdout:3188: snap_create 2024-09-18T14:13:57.569 INFO:tasks.rados.rados.0.smithi084.stdout:3188: done (0 left) 2024-09-18T14:13:57.570 INFO:tasks.rados.rados.0.smithi084.stdout:3189: rollback oid 8 current snap is 344 2024-09-18T14:13:57.570 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 8 to 339 2024-09-18T14:13:57.570 INFO:tasks.rados.rados.0.smithi084.stdout:3190: read oid 14 snap -1 2024-09-18T14:13:57.570 INFO:tasks.rados.rados.0.smithi084.stdout:3190: expect (ObjNum 1041 snap 333 seq_num 1041) 2024-09-18T14:13:57.570 INFO:tasks.rados.rados.0.smithi084.stdout:3191: snap_create 2024-09-18T14:13:57.575 INFO:tasks.rados.rados.0.smithi084.stdout:3189: finishing rollback tid 0 to smithi08434663-8 2024-09-18T14:13:57.577 INFO:tasks.rados.rados.0.smithi084.stdout:3190: expect (ObjNum 1041 snap 333 seq_num 1041) 2024-09-18T14:13:57.724 INFO:tasks.rados.rados.0.smithi084.stdout:3189: finishing rollback tid 1 to smithi08434663-8 2024-09-18T14:13:57.724 INFO:tasks.rados.rados.0.smithi084.stdout:3189: finishing rollback tid 2 to smithi08434663-8 2024-09-18T14:13:57.724 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 1409 (ObjNum 1041 snap 333 seq_num 1041) dirty exists 2024-09-18T14:13:58.478 INFO:tasks.rados.rados.0.smithi084.stdout:3189: done (2 left) 2024-09-18T14:13:58.478 INFO:tasks.rados.rados.0.smithi084.stdout:3190: done (1 left) 2024-09-18T14:13:58.478 INFO:tasks.rados.rados.0.smithi084.stdout:3191: done (0 left) 2024-09-18T14:13:58.478 INFO:tasks.rados.rados.0.smithi084.stdout:3192: copy_from oid 50 from oid 35 current snap is 345 2024-09-18T14:13:58.478 INFO:tasks.rados.rados.0.smithi084.stdout:3193: snap_create 2024-09-18T14:13:58.494 INFO:tasks.rados.rados.0.smithi084.stdout:3192: finishing copy_from racing read to smithi08434663-50 2024-09-18T14:13:58.499 INFO:tasks.rados.rados.0.smithi084.stdout:3192: finishing copy_from to smithi08434663-50 2024-09-18T14:13:58.499 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 1480 (ObjNum 1063 snap 339 seq_num 1063) dirty exists 2024-09-18T14:13:59.485 INFO:tasks.rados.rados.0.smithi084.stdout:3192: done (1 left) 2024-09-18T14:13:59.485 INFO:tasks.rados.rados.0.smithi084.stdout:3193: done (0 left) 2024-09-18T14:13:59.485 INFO:tasks.rados.rados.0.smithi084.stdout:3194: snap_remove snap 339 2024-09-18T14:14:00.494 INFO:tasks.rados.rados.0.smithi084.stdout:3194: done (0 left) 2024-09-18T14:14:00.494 INFO:tasks.rados.rados.0.smithi084.stdout:3195: write (excl) oid 31 current snap is 346 2024-09-18T14:14:00.494 INFO:tasks.rados.rados.0.smithi084.stdout:3195: seq_num 1090 ranges {719630=659889,2117608=202022} 2024-09-18T14:14:00.501 INFO:tasks.rados.rados.0.smithi084.stdout:3195: writing smithi08434663-31 from 719630 to 1379519 tid 1 2024-09-18T14:14:00.507 INFO:tasks.rados.rados.0.smithi084.stdout:3195: writing smithi08434663-31 from 2117608 to 2319630 tid 2 2024-09-18T14:14:00.507 INFO:tasks.rados.rados.0.smithi084.stdout:3196: snap_create 2024-09-18T14:14:00.507 INFO:tasks.rados.rados.0.smithi084.stdout:3195: finishing write tid 1 to smithi08434663-31 2024-09-18T14:14:00.509 INFO:tasks.rados.rados.0.smithi084.stdout:3195: finishing write tid 2 to smithi08434663-31 2024-09-18T14:14:00.509 INFO:tasks.rados.rados.0.smithi084.stdout:3195: finishing write tid 3 to smithi08434663-31 2024-09-18T14:14:00.509 INFO:tasks.rados.rados.0.smithi084.stdout:3195: finishing write tid 4 to smithi08434663-31 2024-09-18T14:14:00.510 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 1527 (ObjNum 1090 snap 346 seq_num 1090) dirty exists 2024-09-18T14:14:00.510 INFO:tasks.rados.rados.0.smithi084.stdout:3195: left oid 31 (ObjNum 1090 snap 346 seq_num 1090) 2024-09-18T14:14:01.499 INFO:tasks.rados.rados.0.smithi084.stdout:3195: done (1 left) 2024-09-18T14:14:01.499 INFO:tasks.rados.rados.0.smithi084.stdout:3196: done (0 left) 2024-09-18T14:14:01.499 INFO:tasks.rados.rados.0.smithi084.stdout:3197: delete oid 24 current snap is 347 2024-09-18T14:14:01.501 INFO:tasks.rados.rados.0.smithi084.stdout:3197: done (0 left) 2024-09-18T14:14:01.501 INFO:tasks.rados.rados.0.smithi084.stdout:3198: copy_from oid 25 from oid 24 current snap is 347 2024-09-18T14:14:01.501 INFO:tasks.rados.rados.0.smithi084.stdout:3199: rollback oid 21 current snap is 347 2024-09-18T14:14:01.501 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 21 to 340 2024-09-18T14:14:01.501 INFO:tasks.rados.rados.0.smithi084.stdout:3200: snap_create 2024-09-18T14:14:01.505 INFO:tasks.rados.rados.0.smithi084.stdout:3198: finishing copy_from to smithi08434663-25 2024-09-18T14:14:01.505 INFO:tasks.rados.rados.0.smithi084.stdout:3198: got expected ENOENT (src dne) 2024-09-18T14:14:01.505 INFO:tasks.rados.rados.0.smithi084.stdout:3198: finishing copy_from racing read to smithi08434663-25 2024-09-18T14:14:01.506 INFO:tasks.rados.rados.0.smithi084.stdout:3199: finishing rollback tid 0 to smithi08434663-21 2024-09-18T14:14:01.507 INFO:tasks.rados.rados.0.smithi084.stdout:3199: finishing rollback tid 1 to smithi08434663-21 2024-09-18T14:14:01.509 INFO:tasks.rados.rados.0.smithi084.stdout:3199: finishing rollback tid 2 to smithi08434663-21 2024-09-18T14:14:01.510 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 1533 (ObjNum 1021 snap 326 seq_num 1021) dirty exists 2024-09-18T14:14:02.512 INFO:tasks.rados.rados.0.smithi084.stdout:3198: done (2 left) 2024-09-18T14:14:02.512 INFO:tasks.rados.rados.0.smithi084.stdout:3199: done (1 left) 2024-09-18T14:14:02.512 INFO:tasks.rados.rados.0.smithi084.stdout:3200: done (0 left) 2024-09-18T14:14:02.512 INFO:tasks.rados.rados.0.smithi084.stdout:3201: delete oid 50 current snap is 348 2024-09-18T14:14:02.522 INFO:tasks.rados.rados.0.smithi084.stdout:3201: done (0 left) 2024-09-18T14:14:02.522 INFO:tasks.rados.rados.0.smithi084.stdout:3202: read oid 18 snap 347 2024-09-18T14:14:02.523 INFO:tasks.rados.rados.0.smithi084.stdout:3202: expect (ObjNum 937 snap 299 seq_num 937) 2024-09-18T14:14:02.523 INFO:tasks.rados.rados.0.smithi084.stdout:3203: write oid 4 current snap is 348 2024-09-18T14:14:02.523 INFO:tasks.rados.rados.0.smithi084.stdout:3203: seq_num 1093 ranges {770051=485636,1776701=511047} 2024-09-18T14:14:02.529 INFO:tasks.rados.rados.0.smithi084.stdout:3203: writing smithi08434663-4 from 770051 to 1255687 tid 1 2024-09-18T14:14:02.535 INFO:tasks.rados.rados.0.smithi084.stdout:3203: writing smithi08434663-4 from 1776701 to 2287748 tid 2 2024-09-18T14:14:02.535 INFO:tasks.rados.rados.0.smithi084.stdout:3204: copy_from oid 14 from oid 49 current snap is 348 2024-09-18T14:14:02.535 INFO:tasks.rados.rados.0.smithi084.stdout:3202: expect (ObjNum 937 snap 299 seq_num 937) 2024-09-18T14:14:02.811 INFO:tasks.rados.rados.0.smithi084.stdout:3203: finishing write tid 1 to smithi08434663-4 2024-09-18T14:14:02.811 INFO:tasks.rados.rados.0.smithi084.stdout:3203: finishing write tid 2 to smithi08434663-4 2024-09-18T14:14:02.811 INFO:tasks.rados.rados.0.smithi084.stdout:3203: finishing write tid 3 to smithi08434663-4 2024-09-18T14:14:02.811 INFO:tasks.rados.rados.0.smithi084.stdout:3203: finishing write tid 4 to smithi08434663-4 2024-09-18T14:14:02.811 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1579 (ObjNum 1093 snap 348 seq_num 1093) dirty exists 2024-09-18T14:14:02.811 INFO:tasks.rados.rados.0.smithi084.stdout:3203: left oid 4 (ObjNum 1093 snap 348 seq_num 1093) 2024-09-18T14:14:02.812 INFO:tasks.rados.rados.0.smithi084.stdout:3204: finishing copy_from racing read to smithi08434663-14 2024-09-18T14:14:02.812 INFO:tasks.rados.rados.0.smithi084.stdout:3202: done (2 left) 2024-09-18T14:14:02.812 INFO:tasks.rados.rados.0.smithi084.stdout:3203: done (1 left) 2024-09-18T14:14:02.812 INFO:tasks.rados.rados.0.smithi084.stdout:3205: read oid 16 snap -1 2024-09-18T14:14:02.812 INFO:tasks.rados.rados.0.smithi084.stdout:3205: expect (ObjNum 1063 snap 339 seq_num 1063) 2024-09-18T14:14:02.812 INFO:tasks.rados.rados.0.smithi084.stdout:3206: snap_create 2024-09-18T14:14:02.812 INFO:tasks.rados.rados.0.smithi084.stdout:3204: finishing copy_from to smithi08434663-14 2024-09-18T14:14:02.812 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 1321 (ObjNum 1020 snap 326 seq_num 1020) dirty exists 2024-09-18T14:14:02.816 INFO:tasks.rados.rados.0.smithi084.stdout:3205: expect (ObjNum 1063 snap 339 seq_num 1063) 2024-09-18T14:14:03.477 INFO:tasks.rados.rados.0.smithi084.stdout:3204: done (2 left) 2024-09-18T14:14:03.478 INFO:tasks.rados.rados.0.smithi084.stdout:3205: done (1 left) 2024-09-18T14:14:03.478 INFO:tasks.rados.rados.0.smithi084.stdout:3206: done (0 left) 2024-09-18T14:14:03.478 INFO:tasks.rados.rados.0.smithi084.stdout:3207: snap_create 2024-09-18T14:14:04.479 INFO:tasks.rados.rados.0.smithi084.stdout:3207: done (0 left) 2024-09-18T14:14:04.479 INFO:tasks.rados.rados.0.smithi084.stdout:3208: read oid 31 snap -1 2024-09-18T14:14:04.479 INFO:tasks.rados.rados.0.smithi084.stdout:3208: expect (ObjNum 1090 snap 346 seq_num 1090) 2024-09-18T14:14:04.479 INFO:tasks.rados.rados.0.smithi084.stdout:3209: write (excl) oid 19 current snap is 350 2024-09-18T14:14:04.480 INFO:tasks.rados.rados.0.smithi084.stdout:3209: seq_num 1094 ranges {786858=560885,1835183=551675} 2024-09-18T14:14:04.486 INFO:tasks.rados.rados.0.smithi084.stdout:3209: writing smithi08434663-19 from 786858 to 1347743 tid 1 2024-09-18T14:14:04.493 INFO:tasks.rados.rados.0.smithi084.stdout:3209: writing smithi08434663-19 from 1835183 to 2386858 tid 2 2024-09-18T14:14:04.493 INFO:tasks.rados.rados.0.smithi084.stdout:3210: read oid 27 snap 338 2024-09-18T14:14:04.493 INFO:tasks.rados.rados.0.smithi084.stdout:3210: expect (ObjNum 1049 snap 335 seq_num 1049) 2024-09-18T14:14:04.493 INFO:tasks.rados.rados.0.smithi084.stdout:3211: delete oid 35 current snap is 350 2024-09-18T14:14:04.493 INFO:tasks.rados.rados.0.smithi084.stdout:3208: expect (ObjNum 1090 snap 346 seq_num 1090) 2024-09-18T14:14:04.573 INFO:tasks.rados.rados.0.smithi084.stdout:3209: finishing write tid 1 to smithi08434663-19 2024-09-18T14:14:04.573 INFO:tasks.rados.rados.0.smithi084.stdout:3209: finishing write tid 2 to smithi08434663-19 2024-09-18T14:14:04.573 INFO:tasks.rados.rados.0.smithi084.stdout:3209: finishing write tid 3 to smithi08434663-19 2024-09-18T14:14:04.573 INFO:tasks.rados.rados.0.smithi084.stdout:3209: finishing write tid 4 to smithi08434663-19 2024-09-18T14:14:04.573 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1759 (ObjNum 1094 snap 350 seq_num 1094) dirty exists 2024-09-18T14:14:04.574 INFO:tasks.rados.rados.0.smithi084.stdout:3209: left oid 19 (ObjNum 1094 snap 350 seq_num 1094) 2024-09-18T14:14:04.574 INFO:tasks.rados.rados.0.smithi084.stdout:3210: expect (ObjNum 1049 snap 335 seq_num 1049) 2024-09-18T14:14:04.686 INFO:tasks.rados.rados.0.smithi084.stdout:3208: done (3 left) 2024-09-18T14:14:04.686 INFO:tasks.rados.rados.0.smithi084.stdout:3209: done (2 left) 2024-09-18T14:14:04.686 INFO:tasks.rados.rados.0.smithi084.stdout:3210: done (1 left) 2024-09-18T14:14:04.686 INFO:tasks.rados.rados.0.smithi084.stdout:3211: done (0 left) 2024-09-18T14:14:04.686 INFO:tasks.rados.rados.0.smithi084.stdout:3212: rollback oid 29 current snap is 350 2024-09-18T14:14:04.686 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 29 to 343 2024-09-18T14:14:04.686 INFO:tasks.rados.rados.0.smithi084.stdout:3213: write (excl) oid 32 current snap is 350 2024-09-18T14:14:04.686 INFO:tasks.rados.rados.0.smithi084.stdout:3213: seq_num 1096 ranges {420472=427736,1268500=658374,2347953=72519} 2024-09-18T14:14:04.690 INFO:tasks.rados.rados.0.smithi084.stdout:3213: writing smithi08434663-32 from 420472 to 848208 tid 1 2024-09-18T14:14:04.696 INFO:tasks.rados.rados.0.smithi084.stdout:3213: writing smithi08434663-32 from 1268500 to 1926874 tid 2 2024-09-18T14:14:04.699 INFO:tasks.rados.rados.0.smithi084.stdout:3213: writing smithi08434663-32 from 2347953 to 2420472 tid 3 2024-09-18T14:14:04.699 INFO:tasks.rados.rados.0.smithi084.stdout:3214: write oid 43 current snap is 350 2024-09-18T14:14:04.699 INFO:tasks.rados.rados.0.smithi084.stdout:3214: seq_num 1097 ranges {437279=502985,1610629=518385} 2024-09-18T14:14:04.704 INFO:tasks.rados.rados.0.smithi084.stdout:3214: writing smithi08434663-43 from 437279 to 940264 tid 1 2024-09-18T14:14:04.711 INFO:tasks.rados.rados.0.smithi084.stdout:3214: writing smithi08434663-43 from 1610629 to 2129014 tid 2 2024-09-18T14:14:04.711 INFO:tasks.rados.rados.0.smithi084.stdout:3215: read oid 19 snap -1 2024-09-18T14:14:04.711 INFO:tasks.rados.rados.0.smithi084.stdout:3215: expect (ObjNum 1094 snap 350 seq_num 1094) 2024-09-18T14:14:04.712 INFO:tasks.rados.rados.0.smithi084.stdout:3216: write oid 13 current snap is 350 2024-09-18T14:14:04.712 INFO:tasks.rados.rados.0.smithi084.stdout:3212: finishing rollback tid 1 to smithi08434663-29 2024-09-18T14:14:04.712 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 1545 (ObjNum 0 snap 0 seq_num 304) dirty dne 2024-09-18T14:14:04.712 INFO:tasks.rados.rados.0.smithi084.stdout:3213: finishing write tid 1 to smithi08434663-32 2024-09-18T14:14:04.712 INFO:tasks.rados.rados.0.smithi084.stdout:3213: finishing write tid 2 to smithi08434663-32 2024-09-18T14:14:04.712 INFO:tasks.rados.rados.0.smithi084.stdout:3213: finishing write tid 3 to smithi08434663-32 2024-09-18T14:14:04.712 INFO:tasks.rados.rados.0.smithi084.stdout:3213: finishing write tid 4 to smithi08434663-32 2024-09-18T14:14:04.712 INFO:tasks.rados.rados.0.smithi084.stdout:3213: finishing write tid 5 to smithi08434663-32 2024-09-18T14:14:04.712 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 1764 (ObjNum 1096 snap 350 seq_num 1096) dirty exists 2024-09-18T14:14:04.712 INFO:tasks.rados.rados.0.smithi084.stdout:3213: left oid 32 (ObjNum 1096 snap 350 seq_num 1096) 2024-09-18T14:14:04.712 INFO:tasks.rados.rados.0.smithi084.stdout:3214: finishing write tid 1 to smithi08434663-43 2024-09-18T14:14:04.712 INFO:tasks.rados.rados.0.smithi084.stdout:3216: seq_num 1098 ranges {454086=578234,1669111=662043} 2024-09-18T14:14:04.717 INFO:tasks.rados.rados.0.smithi084.stdout:3216: writing smithi08434663-13 from 454086 to 1032320 tid 1 2024-09-18T14:14:04.724 INFO:tasks.rados.rados.0.smithi084.stdout:3216: writing smithi08434663-13 from 1669111 to 2331154 tid 2 2024-09-18T14:14:04.724 INFO:tasks.rados.rados.0.smithi084.stdout:3212: done (4 left) 2024-09-18T14:14:04.724 INFO:tasks.rados.rados.0.smithi084.stdout:3213: done (3 left) 2024-09-18T14:14:04.724 INFO:tasks.rados.rados.0.smithi084.stdout:3217: write (excl) oid 14 current snap is 350 2024-09-18T14:14:04.724 INFO:tasks.rados.rados.0.smithi084.stdout:3217: seq_num 1099 ranges {470893=653483,1727593=522054} 2024-09-18T14:14:04.730 INFO:tasks.rados.rados.0.smithi084.stdout:3217: writing smithi08434663-14 from 470893 to 1124376 tid 1 2024-09-18T14:14:04.737 INFO:tasks.rados.rados.0.smithi084.stdout:3217: writing smithi08434663-14 from 1727593 to 2249647 tid 2 2024-09-18T14:14:04.737 INFO:tasks.rados.rados.0.smithi084.stdout:3218: read oid 42 snap -1 2024-09-18T14:14:04.737 INFO:tasks.rados.rados.0.smithi084.stdout:3218: expect (ObjNum 985 snap 316 seq_num 985) 2024-09-18T14:14:04.737 INFO:tasks.rados.rados.0.smithi084.stdout:3219: read oid 39 snap 338 2024-09-18T14:14:04.737 INFO:tasks.rados.rados.0.smithi084.stdout:3219: expect (ObjNum 1044 snap 334 seq_num 1044) 2024-09-18T14:14:04.737 INFO:tasks.rados.rados.0.smithi084.stdout:3220: delete oid 45 current snap is 350 2024-09-18T14:14:04.737 INFO:tasks.rados.rados.0.smithi084.stdout:3214: finishing write tid 2 to smithi08434663-43 2024-09-18T14:14:04.737 INFO:tasks.rados.rados.0.smithi084.stdout:3214: finishing write tid 3 to smithi08434663-43 2024-09-18T14:14:04.737 INFO:tasks.rados.rados.0.smithi084.stdout:3214: finishing write tid 4 to smithi08434663-43 2024-09-18T14:14:04.737 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 1272 (ObjNum 1097 snap 350 seq_num 1097) dirty exists 2024-09-18T14:14:04.737 INFO:tasks.rados.rados.0.smithi084.stdout:3214: left oid 43 (ObjNum 1097 snap 350 seq_num 1097) 2024-09-18T14:14:04.737 INFO:tasks.rados.rados.0.smithi084.stdout:3215: expect (ObjNum 1094 snap 350 seq_num 1094) 2024-09-18T14:14:04.885 INFO:tasks.rados.rados.0.smithi084.stdout:3216: finishing write tid 1 to smithi08434663-13 2024-09-18T14:14:04.885 INFO:tasks.rados.rados.0.smithi084.stdout:3216: finishing write tid 2 to smithi08434663-13 2024-09-18T14:14:04.886 INFO:tasks.rados.rados.0.smithi084.stdout:3216: finishing write tid 3 to smithi08434663-13 2024-09-18T14:14:04.886 INFO:tasks.rados.rados.0.smithi084.stdout:3216: finishing write tid 4 to smithi08434663-13 2024-09-18T14:14:04.886 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 1548 (ObjNum 1098 snap 350 seq_num 1098) dirty exists 2024-09-18T14:14:04.886 INFO:tasks.rados.rados.0.smithi084.stdout:3216: left oid 13 (ObjNum 1098 snap 350 seq_num 1098) 2024-09-18T14:14:04.886 INFO:tasks.rados.rados.0.smithi084.stdout:3217: finishing write tid 1 to smithi08434663-14 2024-09-18T14:14:04.886 INFO:tasks.rados.rados.0.smithi084.stdout:3214: done (6 left) 2024-09-18T14:14:04.886 INFO:tasks.rados.rados.0.smithi084.stdout:3215: done (5 left) 2024-09-18T14:14:04.886 INFO:tasks.rados.rados.0.smithi084.stdout:3216: done (4 left) 2024-09-18T14:14:04.887 INFO:tasks.rados.rados.0.smithi084.stdout:3220: done (3 left) 2024-09-18T14:14:04.887 INFO:tasks.rados.rados.0.smithi084.stdout:3221: rollback oid 38 current snap is 350 2024-09-18T14:14:04.887 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 38 to 335 2024-09-18T14:14:04.887 INFO:tasks.rados.rados.0.smithi084.stdout:3217: finishing write tid 2 to smithi08434663-14 2024-09-18T14:14:04.887 INFO:tasks.rados.rados.0.smithi084.stdout:3217: finishing write tid 3 to smithi08434663-14 2024-09-18T14:14:04.887 INFO:tasks.rados.rados.0.smithi084.stdout:3217: finishing write tid 4 to smithi08434663-14 2024-09-18T14:14:04.887 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 1768 (ObjNum 1099 snap 350 seq_num 1099) dirty exists 2024-09-18T14:14:04.887 INFO:tasks.rados.rados.0.smithi084.stdout:3217: left oid 14 (ObjNum 1099 snap 350 seq_num 1099) 2024-09-18T14:14:04.887 INFO:tasks.rados.rados.0.smithi084.stdout:3219: expect (ObjNum 1044 snap 334 seq_num 1044) 2024-09-18T14:14:05.091 INFO:tasks.rados.rados.0.smithi084.stdout:3218: expect (ObjNum 985 snap 316 seq_num 985) 2024-09-18T14:14:05.231 INFO:tasks.rados.rados.0.smithi084.stdout:3221: finishing rollback tid 0 to smithi08434663-38 2024-09-18T14:14:05.231 INFO:tasks.rados.rados.0.smithi084.stdout:3221: finishing rollback tid 1 to smithi08434663-38 2024-09-18T14:14:05.231 INFO:tasks.rados.rados.0.smithi084.stdout:3221: finishing rollback tid 2 to smithi08434663-38 2024-09-18T14:14:05.231 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 1275 (ObjNum 957 snap 304 seq_num 957) dirty exists 2024-09-18T14:14:05.231 INFO:tasks.rados.rados.0.smithi084.stdout:3217: done (3 left) 2024-09-18T14:14:05.231 INFO:tasks.rados.rados.0.smithi084.stdout:3218: done (2 left) 2024-09-18T14:14:05.231 INFO:tasks.rados.rados.0.smithi084.stdout:3219: done (1 left) 2024-09-18T14:14:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:3221: done (0 left) 2024-09-18T14:14:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:3222: write (excl) oid 12 current snap is 350 2024-09-18T14:14:05.232 INFO:tasks.rados.rados.0.smithi084.stdout:3222: seq_num 1101 ranges {504507=403981,1328204=409370,2252362=252145} 2024-09-18T14:14:05.235 INFO:tasks.rados.rados.0.smithi084.stdout:3222: writing smithi08434663-12 from 504507 to 908488 tid 1 2024-09-18T14:14:05.240 INFO:tasks.rados.rados.0.smithi084.stdout:3222: writing smithi08434663-12 from 1328204 to 1737574 tid 2 2024-09-18T14:14:05.244 INFO:tasks.rados.rados.0.smithi084.stdout:3222: writing smithi08434663-12 from 2252362 to 2504507 tid 3 2024-09-18T14:14:05.245 INFO:tasks.rados.rados.0.smithi084.stdout:3223: snap_create 2024-09-18T14:14:05.245 INFO:tasks.rados.rados.0.smithi084.stdout:3222: finishing write tid 1 to smithi08434663-12 2024-09-18T14:14:05.245 INFO:tasks.rados.rados.0.smithi084.stdout:3222: finishing write tid 2 to smithi08434663-12 2024-09-18T14:14:05.249 INFO:tasks.rados.rados.0.smithi084.stdout:3222: finishing write tid 3 to smithi08434663-12 2024-09-18T14:14:05.249 INFO:tasks.rados.rados.0.smithi084.stdout:3222: finishing write tid 4 to smithi08434663-12 2024-09-18T14:14:05.249 INFO:tasks.rados.rados.0.smithi084.stdout:3222: finishing write tid 5 to smithi08434663-12 2024-09-18T14:14:05.249 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 1772 (ObjNum 1101 snap 350 seq_num 1101) dirty exists 2024-09-18T14:14:05.249 INFO:tasks.rados.rados.0.smithi084.stdout:3222: left oid 12 (ObjNum 1101 snap 350 seq_num 1101) 2024-09-18T14:14:05.513 INFO:tasks.rados.rados.0.smithi084.stdout:3222: done (1 left) 2024-09-18T14:14:05.514 INFO:tasks.rados.rados.0.smithi084.stdout:3223: done (0 left) 2024-09-18T14:14:05.514 INFO:tasks.rados.rados.0.smithi084.stdout:3224: read oid 29 snap 329 2024-09-18T14:14:05.514 INFO:tasks.rados.rados.0.smithi084.stdout:3224: expect (ObjNum 1017 snap 326 seq_num 1017) 2024-09-18T14:14:05.514 INFO:tasks.rados.rados.0.smithi084.stdout:3225: snap_create 2024-09-18T14:14:05.524 INFO:tasks.rados.rados.0.smithi084.stdout:3224: expect (ObjNum 1017 snap 326 seq_num 1017) 2024-09-18T14:14:06.519 INFO:tasks.rados.rados.0.smithi084.stdout:3224: done (1 left) 2024-09-18T14:14:06.520 INFO:tasks.rados.rados.0.smithi084.stdout:3225: done (0 left) 2024-09-18T14:14:06.520 INFO:tasks.rados.rados.0.smithi084.stdout:3226: copy_from oid 16 from oid 44 current snap is 352 2024-09-18T14:14:06.520 INFO:tasks.rados.rados.0.smithi084.stdout:3227: read oid 3 snap -1 2024-09-18T14:14:06.520 INFO:tasks.rados.rados.0.smithi084.stdout:3227: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:14:06.520 INFO:tasks.rados.rados.0.smithi084.stdout:3228: write (excl) oid 25 current snap is 352 2024-09-18T14:14:06.520 INFO:tasks.rados.rados.0.smithi084.stdout:3228: seq_num 1102 ranges {521314=479230,1786686=669381} 2024-09-18T14:14:06.525 INFO:tasks.rados.rados.0.smithi084.stdout:3228: writing smithi08434663-25 from 521314 to 1000544 tid 1 2024-09-18T14:14:06.534 INFO:tasks.rados.rados.0.smithi084.stdout:3228: writing smithi08434663-25 from 1786686 to 2456067 tid 2 2024-09-18T14:14:06.534 INFO:tasks.rados.rados.0.smithi084.stdout:3229: write oid 31 current snap is 352 2024-09-18T14:14:06.534 INFO:tasks.rados.rados.0.smithi084.stdout:3229: seq_num 1103 ranges {538121=670832,1961521=413039} 2024-09-18T14:14:06.540 INFO:tasks.rados.rados.0.smithi084.stdout:3229: writing smithi08434663-31 from 538121 to 1208953 tid 1 2024-09-18T14:14:06.547 INFO:tasks.rados.rados.0.smithi084.stdout:3229: writing smithi08434663-31 from 1961521 to 2374560 tid 2 2024-09-18T14:14:06.547 INFO:tasks.rados.rados.0.smithi084.stdout:3230: snap_create 2024-09-18T14:14:06.547 INFO:tasks.rados.rados.0.smithi084.stdout:3227: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:14:06.696 INFO:tasks.rados.rados.0.smithi084.stdout:3226: finishing copy_from racing read to smithi08434663-16 2024-09-18T14:14:06.696 INFO:tasks.rados.rados.0.smithi084.stdout:3228: finishing write tid 1 to smithi08434663-25 2024-09-18T14:14:06.696 INFO:tasks.rados.rados.0.smithi084.stdout:3226: finishing copy_from to smithi08434663-16 2024-09-18T14:14:06.696 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1043 (ObjNum 1072 snap 341 seq_num 1072) dirty exists 2024-09-18T14:14:06.696 INFO:tasks.rados.rados.0.smithi084.stdout:3228: finishing write tid 2 to smithi08434663-25 2024-09-18T14:14:06.697 INFO:tasks.rados.rados.0.smithi084.stdout:3228: finishing write tid 3 to smithi08434663-25 2024-09-18T14:14:06.697 INFO:tasks.rados.rados.0.smithi084.stdout:3228: finishing write tid 4 to smithi08434663-25 2024-09-18T14:14:06.697 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 1536 (ObjNum 1102 snap 352 seq_num 1102) dirty exists 2024-09-18T14:14:06.697 INFO:tasks.rados.rados.0.smithi084.stdout:3228: left oid 25 (ObjNum 1102 snap 352 seq_num 1102) 2024-09-18T14:14:06.697 INFO:tasks.rados.rados.0.smithi084.stdout:3229: finishing write tid 1 to smithi08434663-31 2024-09-18T14:14:06.697 INFO:tasks.rados.rados.0.smithi084.stdout:3229: finishing write tid 2 to smithi08434663-31 2024-09-18T14:14:06.697 INFO:tasks.rados.rados.0.smithi084.stdout:3229: finishing write tid 3 to smithi08434663-31 2024-09-18T14:14:06.697 INFO:tasks.rados.rados.0.smithi084.stdout:3229: finishing write tid 4 to smithi08434663-31 2024-09-18T14:14:06.697 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 1530 (ObjNum 1103 snap 352 seq_num 1103) dirty exists 2024-09-18T14:14:06.697 INFO:tasks.rados.rados.0.smithi084.stdout:3229: left oid 31 (ObjNum 1103 snap 352 seq_num 1103) 2024-09-18T14:14:07.520 INFO:tasks.rados.rados.0.smithi084.stdout:3226: done (4 left) 2024-09-18T14:14:07.520 INFO:tasks.rados.rados.0.smithi084.stdout:3227: done (3 left) 2024-09-18T14:14:07.520 INFO:tasks.rados.rados.0.smithi084.stdout:3228: done (2 left) 2024-09-18T14:14:07.520 INFO:tasks.rados.rados.0.smithi084.stdout:3229: done (1 left) 2024-09-18T14:14:07.520 INFO:tasks.rados.rados.0.smithi084.stdout:3230: done (0 left) 2024-09-18T14:14:07.520 INFO:tasks.rados.rados.0.smithi084.stdout:3231: delete oid 9 current snap is 353 2024-09-18T14:14:07.525 INFO:tasks.rados.rados.0.smithi084.stdout:3231: done (0 left) 2024-09-18T14:14:07.525 INFO:tasks.rados.rados.0.smithi084.stdout:3232: delete oid 1 current snap is 353 2024-09-18T14:14:07.531 INFO:tasks.rados.rados.0.smithi084.stdout:3232: done (0 left) 2024-09-18T14:14:07.532 INFO:tasks.rados.rados.0.smithi084.stdout:3233: copy_from oid 29 from oid 2 current snap is 353 2024-09-18T14:14:07.532 INFO:tasks.rados.rados.0.smithi084.stdout:3234: write oid 7 current snap is 353 2024-09-18T14:14:07.532 INFO:tasks.rados.rados.0.smithi084.stdout:3234: seq_num 1106 ranges {588542=496579,1620614=676719} 2024-09-18T14:14:07.538 INFO:tasks.rados.rados.0.smithi084.stdout:3234: writing smithi08434663-7 from 588542 to 1085121 tid 1 2024-09-18T14:14:07.545 INFO:tasks.rados.rados.0.smithi084.stdout:3234: writing smithi08434663-7 from 1620614 to 2297333 tid 2 2024-09-18T14:14:07.545 INFO:tasks.rados.rados.0.smithi084.stdout:3235: read oid 26 snap -1 2024-09-18T14:14:07.545 INFO:tasks.rados.rados.0.smithi084.stdout:3235: expect (ObjNum 1074 snap 341 seq_num 1074) 2024-09-18T14:14:07.545 INFO:tasks.rados.rados.0.smithi084.stdout:3236: read oid 35 snap -1 2024-09-18T14:14:07.545 INFO:tasks.rados.rados.0.smithi084.stdout:3236: expect deleted 2024-09-18T14:14:07.545 INFO:tasks.rados.rados.0.smithi084.stdout:3237: snap_create 2024-09-18T14:14:07.545 INFO:tasks.rados.rados.0.smithi084.stdout:3233: finishing copy_from to smithi08434663-29 2024-09-18T14:14:07.545 INFO:tasks.rados.rados.0.smithi084.stdout:3233: got expected ENOENT (src dne) 2024-09-18T14:14:07.546 INFO:tasks.rados.rados.0.smithi084.stdout:3233: finishing copy_from racing read to smithi08434663-29 2024-09-18T14:14:07.546 INFO:tasks.rados.rados.0.smithi084.stdout:3234: finishing write tid 1 to smithi08434663-7 2024-09-18T14:14:07.548 INFO:tasks.rados.rados.0.smithi084.stdout:3235: expect (ObjNum 1074 snap 341 seq_num 1074) 2024-09-18T14:14:07.627 INFO:tasks.rados.rados.0.smithi084.stdout:3234: finishing write tid 2 to smithi08434663-7 2024-09-18T14:14:07.628 INFO:tasks.rados.rados.0.smithi084.stdout:3234: finishing write tid 3 to smithi08434663-7 2024-09-18T14:14:07.628 INFO:tasks.rados.rados.0.smithi084.stdout:3234: finishing write tid 4 to smithi08434663-7 2024-09-18T14:14:07.628 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 1035 (ObjNum 1106 snap 353 seq_num 1106) dirty exists 2024-09-18T14:14:07.628 INFO:tasks.rados.rados.0.smithi084.stdout:3234: left oid 7 (ObjNum 1106 snap 353 seq_num 1106) 2024-09-18T14:14:08.479 INFO:tasks.rados.rados.0.smithi084.stdout:3233: done (4 left) 2024-09-18T14:14:08.479 INFO:tasks.rados.rados.0.smithi084.stdout:3234: done (3 left) 2024-09-18T14:14:08.479 INFO:tasks.rados.rados.0.smithi084.stdout:3235: done (2 left) 2024-09-18T14:14:08.479 INFO:tasks.rados.rados.0.smithi084.stdout:3236: done (1 left) 2024-09-18T14:14:08.479 INFO:tasks.rados.rados.0.smithi084.stdout:3237: done (0 left) 2024-09-18T14:14:08.479 INFO:tasks.rados.rados.0.smithi084.stdout:3238: read oid 4 snap -1 2024-09-18T14:14:08.479 INFO:tasks.rados.rados.0.smithi084.stdout:3238: expect (ObjNum 1093 snap 348 seq_num 1093) 2024-09-18T14:14:08.479 INFO:tasks.rados.rados.0.smithi084.stdout:3239: snap_create 2024-09-18T14:14:08.500 INFO:tasks.rados.rados.0.smithi084.stdout:3238: expect (ObjNum 1093 snap 348 seq_num 1093) 2024-09-18T14:14:09.486 INFO:tasks.rados.rados.0.smithi084.stdout:3238: done (1 left) 2024-09-18T14:14:09.486 INFO:tasks.rados.rados.0.smithi084.stdout:3239: done (0 left) 2024-09-18T14:14:09.486 INFO:tasks.rados.rados.0.smithi084.stdout:3240: write (excl) oid 46 current snap is 355 2024-09-18T14:14:09.486 INFO:tasks.rados.rados.0.smithi084.stdout:3240: seq_num 1107 ranges {605349=571828,1679096=420377} 2024-09-18T14:14:09.492 INFO:tasks.rados.rados.0.smithi084.stdout:3240: writing smithi08434663-46 from 605349 to 1177177 tid 1 2024-09-18T14:14:09.497 INFO:tasks.rados.rados.0.smithi084.stdout:3240: writing smithi08434663-46 from 1679096 to 2099473 tid 2 2024-09-18T14:14:09.497 INFO:tasks.rados.rados.0.smithi084.stdout:3241: snap_remove snap 350 2024-09-18T14:14:10.492 INFO:tasks.rados.rados.0.smithi084.stdout:3241: done (1 left) 2024-09-18T14:14:10.492 INFO:tasks.rados.rados.0.smithi084.stdout:3242: read oid 48 snap 330 2024-09-18T14:14:10.492 INFO:tasks.rados.rados.0.smithi084.stdout:3242: expect (ObjNum 955 snap 304 seq_num 955) 2024-09-18T14:14:10.493 INFO:tasks.rados.rados.0.smithi084.stdout:3243: delete oid 6 current snap is 355 2024-09-18T14:14:10.493 INFO:tasks.rados.rados.0.smithi084.stdout:3240: finishing write tid 1 to smithi08434663-46 2024-09-18T14:14:10.493 INFO:tasks.rados.rados.0.smithi084.stdout:3240: finishing write tid 2 to smithi08434663-46 2024-09-18T14:14:10.493 INFO:tasks.rados.rados.0.smithi084.stdout:3240: finishing write tid 3 to smithi08434663-46 2024-09-18T14:14:10.493 INFO:tasks.rados.rados.0.smithi084.stdout:3240: finishing write tid 4 to smithi08434663-46 2024-09-18T14:14:10.493 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1494 (ObjNum 1107 snap 355 seq_num 1107) dirty exists 2024-09-18T14:14:10.493 INFO:tasks.rados.rados.0.smithi084.stdout:3240: left oid 46 (ObjNum 1107 snap 355 seq_num 1107) 2024-09-18T14:14:10.497 INFO:tasks.rados.rados.0.smithi084.stdout:3240: done (2 left) 2024-09-18T14:14:10.497 INFO:tasks.rados.rados.0.smithi084.stdout:3243: done (1 left) 2024-09-18T14:14:10.497 INFO:tasks.rados.rados.0.smithi084.stdout:3244: copy_from oid 23 from oid 49 current snap is 355 2024-09-18T14:14:10.497 INFO:tasks.rados.rados.0.smithi084.stdout:3245: delete oid 6 current snap is 355 2024-09-18T14:14:10.499 INFO:tasks.rados.rados.0.smithi084.stdout:3242: expect (ObjNum 955 snap 304 seq_num 955) 2024-09-18T14:14:10.621 INFO:tasks.rados.rados.0.smithi084.stdout:3244: finishing copy_from racing read to smithi08434663-23 2024-09-18T14:14:10.621 INFO:tasks.rados.rados.0.smithi084.stdout:3244: finishing copy_from to smithi08434663-23 2024-09-18T14:14:10.621 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1321 (ObjNum 1020 snap 326 seq_num 1020) dirty exists 2024-09-18T14:14:10.622 INFO:tasks.rados.rados.0.smithi084.stdout:3242: done (2 left) 2024-09-18T14:14:10.622 INFO:tasks.rados.rados.0.smithi084.stdout:3244: done (1 left) 2024-09-18T14:14:10.622 INFO:tasks.rados.rados.0.smithi084.stdout:3245: done (0 left) 2024-09-18T14:14:10.622 INFO:tasks.rados.rados.0.smithi084.stdout:3246: delete oid 47 current snap is 355 2024-09-18T14:14:10.629 INFO:tasks.rados.rados.0.smithi084.stdout:3246: done (0 left) 2024-09-18T14:14:10.629 INFO:tasks.rados.rados.0.smithi084.stdout:3247: write oid 46 current snap is 355 2024-09-18T14:14:10.629 INFO:tasks.rados.rados.0.smithi084.stdout:3247: seq_num 1111 ranges {672577=589177,1913024=427715} 2024-09-18T14:14:10.635 INFO:tasks.rados.rados.0.smithi084.stdout:3247: writing smithi08434663-46 from 672577 to 1261754 tid 1 2024-09-18T14:14:10.641 INFO:tasks.rados.rados.0.smithi084.stdout:3247: writing smithi08434663-46 from 1913024 to 2340739 tid 2 2024-09-18T14:14:10.641 INFO:tasks.rados.rados.0.smithi084.stdout:3248: delete oid 37 current snap is 355 2024-09-18T14:14:10.641 INFO:tasks.rados.rados.0.smithi084.stdout:3247: finishing write tid 1 to smithi08434663-46 2024-09-18T14:14:10.644 INFO:tasks.rados.rados.0.smithi084.stdout:3248: done (1 left) 2024-09-18T14:14:10.645 INFO:tasks.rados.rados.0.smithi084.stdout:3249: snap_create 2024-09-18T14:14:10.646 INFO:tasks.rados.rados.0.smithi084.stdout:3247: finishing write tid 2 to smithi08434663-46 2024-09-18T14:14:10.646 INFO:tasks.rados.rados.0.smithi084.stdout:3247: finishing write tid 3 to smithi08434663-46 2024-09-18T14:14:10.646 INFO:tasks.rados.rados.0.smithi084.stdout:3247: finishing write tid 4 to smithi08434663-46 2024-09-18T14:14:10.646 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1497 (ObjNum 1111 snap 355 seq_num 1111) dirty exists 2024-09-18T14:14:10.646 INFO:tasks.rados.rados.0.smithi084.stdout:3247: left oid 46 (ObjNum 1111 snap 355 seq_num 1111) 2024-09-18T14:14:11.493 INFO:tasks.rados.rados.0.smithi084.stdout:3247: done (1 left) 2024-09-18T14:14:11.493 INFO:tasks.rados.rados.0.smithi084.stdout:3249: done (0 left) 2024-09-18T14:14:11.493 INFO:tasks.rados.rados.0.smithi084.stdout:3250: read oid 1 snap -1 2024-09-18T14:14:11.493 INFO:tasks.rados.rados.0.smithi084.stdout:3250: expect deleted 2024-09-18T14:14:11.493 INFO:tasks.rados.rados.0.smithi084.stdout:3251: rollback oid 42 current snap is 356 2024-09-18T14:14:11.493 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 42 to 340 2024-09-18T14:14:11.493 INFO:tasks.rados.rados.0.smithi084.stdout:3252: read oid 28 snap -1 2024-09-18T14:14:11.493 INFO:tasks.rados.rados.0.smithi084.stdout:3252: expect (ObjNum 1074 snap 341 seq_num 1074) 2024-09-18T14:14:11.493 INFO:tasks.rados.rados.0.smithi084.stdout:3253: write (excl) oid 19 current snap is 356 2024-09-18T14:14:11.494 INFO:tasks.rados.rados.0.smithi084.stdout:3253: seq_num 1113 ranges {706191=739675,1913635=431384} 2024-09-18T14:14:11.500 INFO:tasks.rados.rados.0.smithi084.stdout:3253: writing smithi08434663-19 from 706191 to 1445866 tid 1 2024-09-18T14:14:11.506 INFO:tasks.rados.rados.0.smithi084.stdout:3253: writing smithi08434663-19 from 1913635 to 2345019 tid 2 2024-09-18T14:14:11.506 INFO:tasks.rados.rados.0.smithi084.stdout:3254: delete oid 35 current snap is 356 2024-09-18T14:14:11.506 INFO:tasks.rados.rados.0.smithi084.stdout:3252: expect (ObjNum 1074 snap 341 seq_num 1074) 2024-09-18T14:14:11.587 INFO:tasks.rados.rados.0.smithi084.stdout:3251: finishing rollback tid 0 to smithi08434663-42 2024-09-18T14:14:11.587 INFO:tasks.rados.rados.0.smithi084.stdout:3251: finishing rollback tid 1 to smithi08434663-42 2024-09-18T14:14:11.587 INFO:tasks.rados.rados.0.smithi084.stdout:3251: finishing rollback tid 2 to smithi08434663-42 2024-09-18T14:14:11.587 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1584 (ObjNum 1054 snap 336 seq_num 1054) dirty exists 2024-09-18T14:14:11.587 INFO:tasks.rados.rados.0.smithi084.stdout:3253: finishing write tid 1 to smithi08434663-19 2024-09-18T14:14:11.587 INFO:tasks.rados.rados.0.smithi084.stdout:3253: finishing write tid 2 to smithi08434663-19 2024-09-18T14:14:11.587 INFO:tasks.rados.rados.0.smithi084.stdout:3253: finishing write tid 3 to smithi08434663-19 2024-09-18T14:14:11.587 INFO:tasks.rados.rados.0.smithi084.stdout:3250: done (4 left) 2024-09-18T14:14:11.587 INFO:tasks.rados.rados.0.smithi084.stdout:3251: done (3 left) 2024-09-18T14:14:11.587 INFO:tasks.rados.rados.0.smithi084.stdout:3252: done (2 left) 2024-09-18T14:14:11.588 INFO:tasks.rados.rados.0.smithi084.stdout:3254: done (1 left) 2024-09-18T14:14:11.588 INFO:tasks.rados.rados.0.smithi084.stdout:3255: copy_from oid 4 from oid 44 current snap is 356 2024-09-18T14:14:11.588 INFO:tasks.rados.rados.0.smithi084.stdout:3253: finishing write tid 4 to smithi08434663-19 2024-09-18T14:14:11.588 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1776 (ObjNum 1113 snap 356 seq_num 1113) dirty exists 2024-09-18T14:14:11.588 INFO:tasks.rados.rados.0.smithi084.stdout:3253: left oid 19 (ObjNum 1113 snap 356 seq_num 1113) 2024-09-18T14:14:11.588 INFO:tasks.rados.rados.0.smithi084.stdout:3253: done (1 left) 2024-09-18T14:14:11.588 INFO:tasks.rados.rados.0.smithi084.stdout:3256: write (excl) oid 19 current snap is 356 2024-09-18T14:14:11.588 INFO:tasks.rados.rados.0.smithi084.stdout:3256: seq_num 1115 ranges {739805=490173,1630599=435053,2636284=103521} 2024-09-18T14:14:11.593 INFO:tasks.rados.rados.0.smithi084.stdout:3256: writing smithi08434663-19 from 739805 to 1229978 tid 1 2024-09-18T14:14:11.598 INFO:tasks.rados.rados.0.smithi084.stdout:3256: writing smithi08434663-19 from 1630599 to 2065652 tid 2 2024-09-18T14:14:11.602 INFO:tasks.rados.rados.0.smithi084.stdout:3256: writing smithi08434663-19 from 2636284 to 2739805 tid 3 2024-09-18T14:14:11.602 INFO:tasks.rados.rados.0.smithi084.stdout:3257: delete oid 14 current snap is 356 2024-09-18T14:14:11.602 INFO:tasks.rados.rados.0.smithi084.stdout:3255: finishing copy_from racing read to smithi08434663-4 2024-09-18T14:14:11.602 INFO:tasks.rados.rados.0.smithi084.stdout:3255: finishing copy_from to smithi08434663-4 2024-09-18T14:14:11.602 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1043 (ObjNum 1072 snap 341 seq_num 1072) dirty exists 2024-09-18T14:14:11.602 INFO:tasks.rados.rados.0.smithi084.stdout:3256: finishing write tid 1 to smithi08434663-19 2024-09-18T14:14:11.602 INFO:tasks.rados.rados.0.smithi084.stdout:3256: finishing write tid 2 to smithi08434663-19 2024-09-18T14:14:11.606 INFO:tasks.rados.rados.0.smithi084.stdout:3256: finishing write tid 3 to smithi08434663-19 2024-09-18T14:14:11.606 INFO:tasks.rados.rados.0.smithi084.stdout:3256: finishing write tid 4 to smithi08434663-19 2024-09-18T14:14:11.606 INFO:tasks.rados.rados.0.smithi084.stdout:3256: finishing write tid 5 to smithi08434663-19 2024-09-18T14:14:11.606 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1780 (ObjNum 1115 snap 356 seq_num 1115) dirty exists 2024-09-18T14:14:11.606 INFO:tasks.rados.rados.0.smithi084.stdout:3256: left oid 19 (ObjNum 1115 snap 356 seq_num 1115) 2024-09-18T14:14:11.606 INFO:tasks.rados.rados.0.smithi084.stdout:3255: done (2 left) 2024-09-18T14:14:11.606 INFO:tasks.rados.rados.0.smithi084.stdout:3256: done (1 left) 2024-09-18T14:14:11.606 INFO:tasks.rados.rados.0.smithi084.stdout:3257: done (0 left) 2024-09-18T14:14:11.606 INFO:tasks.rados.rados.0.smithi084.stdout:3258: read oid 8 snap -1 2024-09-18T14:14:11.607 INFO:tasks.rados.rados.0.smithi084.stdout:3258: expect (ObjNum 1041 snap 333 seq_num 1041) 2024-09-18T14:14:11.607 INFO:tasks.rados.rados.0.smithi084.stdout:3259: snap_remove snap 334 2024-09-18T14:14:12.512 INFO:tasks.rados.rados.0.smithi084.stdout:3258: expect (ObjNum 1041 snap 333 seq_num 1041) 2024-09-18T14:14:12.658 INFO:tasks.rados.rados.0.smithi084.stdout:3258: done (1 left) 2024-09-18T14:14:12.658 INFO:tasks.rados.rados.0.smithi084.stdout:3259: done (0 left) 2024-09-18T14:14:12.658 INFO:tasks.rados.rados.0.smithi084.stdout:3260: snap_remove snap 341 2024-09-18T14:14:13.481 INFO:tasks.rados.rados.0.smithi084.stdout:3260: done (0 left) 2024-09-18T14:14:13.481 INFO:tasks.rados.rados.0.smithi084.stdout:3261: copy_from oid 36 from oid 44 current snap is 356 2024-09-18T14:14:13.481 INFO:tasks.rados.rados.0.smithi084.stdout:3262: copy_from oid 32 from oid 30 current snap is 356 2024-09-18T14:14:13.481 INFO:tasks.rados.rados.0.smithi084.stdout:3263: copy_from oid 1 from oid 40 current snap is 356 2024-09-18T14:14:13.481 INFO:tasks.rados.rados.0.smithi084.stdout:3264: copy_from oid 19 from oid 10 current snap is 356 2024-09-18T14:14:13.481 INFO:tasks.rados.rados.0.smithi084.stdout:3265: snap_create 2024-09-18T14:14:13.494 INFO:tasks.rados.rados.0.smithi084.stdout:3263: finishing copy_from to smithi08434663-1 2024-09-18T14:14:13.494 INFO:tasks.rados.rados.0.smithi084.stdout:3263: got expected ENOENT (src dne) 2024-09-18T14:14:13.494 INFO:tasks.rados.rados.0.smithi084.stdout:3263: finishing copy_from racing read to smithi08434663-1 2024-09-18T14:14:13.495 INFO:tasks.rados.rados.0.smithi084.stdout:3261: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:14:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3264: finishing copy_from racing read to smithi08434663-19 2024-09-18T14:14:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3262: finishing copy_from racing read to smithi08434663-32 2024-09-18T14:14:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3261: finishing copy_from to smithi08434663-36 2024-09-18T14:14:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 1043 (ObjNum 1072 snap 341 seq_num 1072) dirty exists 2024-09-18T14:14:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3262: finishing copy_from to smithi08434663-32 2024-09-18T14:14:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 1263 (ObjNum 1021 snap 326 seq_num 1021) dirty exists 2024-09-18T14:14:13.511 INFO:tasks.rados.rados.0.smithi084.stdout:3264: finishing copy_from to smithi08434663-19 2024-09-18T14:14:13.511 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1040 (ObjNum 991 snap 321 seq_num 991) dirty exists 2024-09-18T14:14:14.481 INFO:tasks.rados.rados.0.smithi084.stdout:3261: done (4 left) 2024-09-18T14:14:14.481 INFO:tasks.rados.rados.0.smithi084.stdout:3262: done (3 left) 2024-09-18T14:14:14.481 INFO:tasks.rados.rados.0.smithi084.stdout:3263: done (2 left) 2024-09-18T14:14:14.481 INFO:tasks.rados.rados.0.smithi084.stdout:3264: done (1 left) 2024-09-18T14:14:14.481 INFO:tasks.rados.rados.0.smithi084.stdout:3265: done (0 left) 2024-09-18T14:14:14.481 INFO:tasks.rados.rados.0.smithi084.stdout:3266: snap_create 2024-09-18T14:14:15.483 INFO:tasks.rados.rados.0.smithi084.stdout:3266: done (0 left) 2024-09-18T14:14:15.483 INFO:tasks.rados.rados.0.smithi084.stdout:3267: write (excl) oid 23 current snap is 358 2024-09-18T14:14:15.483 INFO:tasks.rados.rados.0.smithi084.stdout:3267: seq_num 1117 ranges {773419=640671,2147563=438722} 2024-09-18T14:14:15.490 INFO:tasks.rados.rados.0.smithi084.stdout:3267: writing smithi08434663-23 from 773419 to 1414090 tid 1 2024-09-18T14:14:15.497 INFO:tasks.rados.rados.0.smithi084.stdout:3267: writing smithi08434663-23 from 2147563 to 2586285 tid 2 2024-09-18T14:14:15.497 INFO:tasks.rados.rados.0.smithi084.stdout:3268: snap_create 2024-09-18T14:14:15.497 INFO:tasks.rados.rados.0.smithi084.stdout:3267: finishing write tid 1 to smithi08434663-23 2024-09-18T14:14:15.500 INFO:tasks.rados.rados.0.smithi084.stdout:3267: finishing write tid 2 to smithi08434663-23 2024-09-18T14:14:15.500 INFO:tasks.rados.rados.0.smithi084.stdout:3267: finishing write tid 3 to smithi08434663-23 2024-09-18T14:14:15.501 INFO:tasks.rados.rados.0.smithi084.stdout:3267: finishing write tid 4 to smithi08434663-23 2024-09-18T14:14:15.501 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1324 (ObjNum 1117 snap 358 seq_num 1117) dirty exists 2024-09-18T14:14:15.501 INFO:tasks.rados.rados.0.smithi084.stdout:3267: left oid 23 (ObjNum 1117 snap 358 seq_num 1117) 2024-09-18T14:14:16.488 INFO:tasks.rados.rados.0.smithi084.stdout:3267: done (1 left) 2024-09-18T14:14:16.489 INFO:tasks.rados.rados.0.smithi084.stdout:3268: done (0 left) 2024-09-18T14:14:16.489 INFO:tasks.rados.rados.0.smithi084.stdout:3269: snap_create 2024-09-18T14:14:17.494 INFO:tasks.rados.rados.0.smithi084.stdout:3269: done (0 left) 2024-09-18T14:14:17.494 INFO:tasks.rados.rados.0.smithi084.stdout:3270: rollback oid 2 current snap is 360 2024-09-18T14:14:17.494 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 2 to 330 2024-09-18T14:14:17.494 INFO:tasks.rados.rados.0.smithi084.stdout:3271: snap_remove snap 353 2024-09-18T14:14:18.482 INFO:tasks.rados.rados.0.smithi084.stdout:3270: finishing rollback tid 1 to smithi08434663-2 2024-09-18T14:14:18.482 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 1601 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-18T14:14:18.483 INFO:tasks.rados.rados.0.smithi084.stdout:3270: done (1 left) 2024-09-18T14:14:18.483 INFO:tasks.rados.rados.0.smithi084.stdout:3271: done (0 left) 2024-09-18T14:14:18.483 INFO:tasks.rados.rados.0.smithi084.stdout:3272: write (excl) oid 13 current snap is 360 2024-09-18T14:14:18.483 INFO:tasks.rados.rados.0.smithi084.stdout:3272: seq_num 1118 ranges {790226=432273,1806045=582380} 2024-09-18T14:14:18.489 INFO:tasks.rados.rados.0.smithi084.stdout:3272: writing smithi08434663-13 from 790226 to 1222499 tid 1 2024-09-18T14:14:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:3272: writing smithi08434663-13 from 1806045 to 2388425 tid 2 2024-09-18T14:14:18.496 INFO:tasks.rados.rados.0.smithi084.stdout:3273: snap_remove snap 346 2024-09-18T14:14:19.517 INFO:tasks.rados.rados.0.smithi084.stdout:3273: done (1 left) 2024-09-18T14:14:19.517 INFO:tasks.rados.rados.0.smithi084.stdout:3274: write (excl) oid 19 current snap is 360 2024-09-18T14:14:19.517 INFO:tasks.rados.rados.0.smithi084.stdout:3272: finishing write tid 1 to smithi08434663-13 2024-09-18T14:14:19.517 INFO:tasks.rados.rados.0.smithi084.stdout:3274: seq_num 1119 ranges {407033=507522,1464527=442391,2345976=461057} 2024-09-18T14:14:19.522 INFO:tasks.rados.rados.0.smithi084.stdout:3274: writing smithi08434663-19 from 407033 to 914555 tid 1 2024-09-18T14:14:19.527 INFO:tasks.rados.rados.0.smithi084.stdout:3274: writing smithi08434663-19 from 1464527 to 1906918 tid 2 2024-09-18T14:14:19.533 INFO:tasks.rados.rados.0.smithi084.stdout:3274: writing smithi08434663-19 from 2345976 to 2807033 tid 3 2024-09-18T14:14:19.533 INFO:tasks.rados.rados.0.smithi084.stdout:3275: read oid 35 snap 356 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:3275: expect deleted 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:3276: read oid 24 snap -1 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:3276: expect deleted 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:3277: copy_from oid 10 from oid 46 current snap is 360 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:3272: finishing write tid 2 to smithi08434663-13 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:3272: finishing write tid 3 to smithi08434663-13 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:3272: finishing write tid 4 to smithi08434663-13 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 1551 (ObjNum 1118 snap 360 seq_num 1118) dirty exists 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:3272: left oid 13 (ObjNum 1118 snap 360 seq_num 1118) 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:3274: finishing write tid 1 to smithi08434663-19 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:3274: finishing write tid 2 to smithi08434663-19 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:3272: done (4 left) 2024-09-18T14:14:19.534 INFO:tasks.rados.rados.0.smithi084.stdout:3278: write (excl) oid 33 current snap is 360 2024-09-18T14:14:19.535 INFO:tasks.rados.rados.0.smithi084.stdout:3278: seq_num 1120 ranges {423840=582771,1523009=586049,2657046=166794} 2024-09-18T14:14:19.539 INFO:tasks.rados.rados.0.smithi084.stdout:3278: writing smithi08434663-33 from 423840 to 1006611 tid 1 2024-09-18T14:14:19.545 INFO:tasks.rados.rados.0.smithi084.stdout:3278: writing smithi08434663-33 from 1523009 to 2109058 tid 2 2024-09-18T14:14:19.549 INFO:tasks.rados.rados.0.smithi084.stdout:3278: writing smithi08434663-33 from 2657046 to 2823840 tid 3 2024-09-18T14:14:19.550 INFO:tasks.rados.rados.0.smithi084.stdout:3279: snap_remove snap 331 2024-09-18T14:14:20.523 INFO:tasks.rados.rados.0.smithi084.stdout:3274: finishing write tid 3 to smithi08434663-19 2024-09-18T14:14:20.523 INFO:tasks.rados.rados.0.smithi084.stdout:3274: finishing write tid 4 to smithi08434663-19 2024-09-18T14:14:20.523 INFO:tasks.rados.rados.0.smithi084.stdout:3274: finishing write tid 5 to smithi08434663-19 2024-09-18T14:14:20.523 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1821 (ObjNum 1119 snap 360 seq_num 1119) dirty exists 2024-09-18T14:14:20.523 INFO:tasks.rados.rados.0.smithi084.stdout:3274: left oid 19 (ObjNum 1119 snap 360 seq_num 1119) 2024-09-18T14:14:20.523 INFO:tasks.rados.rados.0.smithi084.stdout:3277: finishing copy_from racing read to smithi08434663-10 2024-09-18T14:14:20.524 INFO:tasks.rados.rados.0.smithi084.stdout:3278: finishing write tid 1 to smithi08434663-33 2024-09-18T14:14:20.524 INFO:tasks.rados.rados.0.smithi084.stdout:3277: finishing copy_from to smithi08434663-10 2024-09-18T14:14:20.524 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 1497 (ObjNum 1111 snap 355 seq_num 1111) dirty exists 2024-09-18T14:14:20.524 INFO:tasks.rados.rados.0.smithi084.stdout:3278: finishing write tid 2 to smithi08434663-33 2024-09-18T14:14:20.524 INFO:tasks.rados.rados.0.smithi084.stdout:3278: finishing write tid 3 to smithi08434663-33 2024-09-18T14:14:20.524 INFO:tasks.rados.rados.0.smithi084.stdout:3278: finishing write tid 4 to smithi08434663-33 2024-09-18T14:14:20.524 INFO:tasks.rados.rados.0.smithi084.stdout:3278: finishing write tid 5 to smithi08434663-33 2024-09-18T14:14:20.524 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 1337 (ObjNum 1120 snap 360 seq_num 1120) dirty exists 2024-09-18T14:14:20.524 INFO:tasks.rados.rados.0.smithi084.stdout:3278: left oid 33 (ObjNum 1120 snap 360 seq_num 1120) 2024-09-18T14:14:20.524 INFO:tasks.rados.rados.0.smithi084.stdout:3274: done (5 left) 2024-09-18T14:14:20.524 INFO:tasks.rados.rados.0.smithi084.stdout:3275: done (4 left) 2024-09-18T14:14:20.524 INFO:tasks.rados.rados.0.smithi084.stdout:3276: done (3 left) 2024-09-18T14:14:20.525 INFO:tasks.rados.rados.0.smithi084.stdout:3277: done (2 left) 2024-09-18T14:14:20.525 INFO:tasks.rados.rados.0.smithi084.stdout:3278: done (1 left) 2024-09-18T14:14:20.525 INFO:tasks.rados.rados.0.smithi084.stdout:3279: done (0 left) 2024-09-18T14:14:20.525 INFO:tasks.rados.rados.0.smithi084.stdout:3280: write (excl) oid 32 current snap is 360 2024-09-18T14:14:20.525 INFO:tasks.rados.rados.0.smithi084.stdout:3280: seq_num 1121 ranges {440647=658020,1581491=446060,2800822=39825} 2024-09-18T14:14:20.529 INFO:tasks.rados.rados.0.smithi084.stdout:3280: writing smithi08434663-32 from 440647 to 1098667 tid 1 2024-09-18T14:14:20.535 INFO:tasks.rados.rados.0.smithi084.stdout:3280: writing smithi08434663-32 from 1581491 to 2027551 tid 2 2024-09-18T14:14:20.539 INFO:tasks.rados.rados.0.smithi084.stdout:3280: writing smithi08434663-32 from 2800822 to 2840647 tid 3 2024-09-18T14:14:20.539 INFO:tasks.rados.rados.0.smithi084.stdout:3281: snap_remove snap 333 2024-09-18T14:14:21.527 INFO:tasks.rados.rados.0.smithi084.stdout:3281: done (1 left) 2024-09-18T14:14:21.528 INFO:tasks.rados.rados.0.smithi084.stdout:3282: rollback oid 29 current snap is 360 2024-09-18T14:14:21.528 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 29 to 359 2024-09-18T14:14:21.528 INFO:tasks.rados.rados.0.smithi084.stdout:3280: finishing write tid 1 to smithi08434663-32 2024-09-18T14:14:21.528 INFO:tasks.rados.rados.0.smithi084.stdout:3280: finishing write tid 2 to smithi08434663-32 2024-09-18T14:14:21.528 INFO:tasks.rados.rados.0.smithi084.stdout:3280: finishing write tid 3 to smithi08434663-32 2024-09-18T14:14:21.528 INFO:tasks.rados.rados.0.smithi084.stdout:3280: finishing write tid 4 to smithi08434663-32 2024-09-18T14:14:21.528 INFO:tasks.rados.rados.0.smithi084.stdout:3280: finishing write tid 5 to smithi08434663-32 2024-09-18T14:14:21.528 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 1836 (ObjNum 1121 snap 360 seq_num 1121) dirty exists 2024-09-18T14:14:21.528 INFO:tasks.rados.rados.0.smithi084.stdout:3280: left oid 32 (ObjNum 1121 snap 360 seq_num 1121) 2024-09-18T14:14:21.529 INFO:tasks.rados.rados.0.smithi084.stdout:3280: done (1 left) 2024-09-18T14:14:21.529 INFO:tasks.rados.rados.0.smithi084.stdout:3283: read oid 45 snap -1 2024-09-18T14:14:21.529 INFO:tasks.rados.rados.0.smithi084.stdout:3283: expect deleted 2024-09-18T14:14:21.529 INFO:tasks.rados.rados.0.smithi084.stdout:3284: copy_from oid 48 from oid 35 current snap is 360 2024-09-18T14:14:21.529 INFO:tasks.rados.rados.0.smithi084.stdout:3285: read oid 21 snap -1 2024-09-18T14:14:21.529 INFO:tasks.rados.rados.0.smithi084.stdout:3285: expect (ObjNum 1021 snap 326 seq_num 1021) 2024-09-18T14:14:21.529 INFO:tasks.rados.rados.0.smithi084.stdout:3286: rollback oid 6 current snap is 360 2024-09-18T14:14:21.529 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 349 2024-09-18T14:14:21.529 INFO:tasks.rados.rados.0.smithi084.stdout:3287: write (excl) oid 11 current snap is 360 2024-09-18T14:14:21.529 INFO:tasks.rados.rados.0.smithi084.stdout:3287: seq_num 1122 ranges {457454=733269,1923620=589718} 2024-09-18T14:14:21.534 INFO:tasks.rados.rados.0.smithi084.stdout:3287: writing smithi08434663-11 from 457454 to 1190723 tid 1 2024-09-18T14:14:21.543 INFO:tasks.rados.rados.0.smithi084.stdout:3287: writing smithi08434663-11 from 1923620 to 2513338 tid 2 2024-09-18T14:14:21.543 INFO:tasks.rados.rados.0.smithi084.stdout:3288: write oid 4 current snap is 360 2024-09-18T14:14:21.543 INFO:tasks.rados.rados.0.smithi084.stdout:3288: seq_num 1123 ranges {474261=408518,1582102=449729,2739315=134946} 2024-09-18T14:14:21.548 INFO:tasks.rados.rados.0.smithi084.stdout:3288: writing smithi08434663-4 from 474261 to 882779 tid 1 2024-09-18T14:14:21.554 INFO:tasks.rados.rados.0.smithi084.stdout:3288: writing smithi08434663-4 from 1582102 to 2031831 tid 2 2024-09-18T14:14:21.559 INFO:tasks.rados.rados.0.smithi084.stdout:3288: writing smithi08434663-4 from 2739315 to 2874261 tid 3 2024-09-18T14:14:21.560 INFO:tasks.rados.rados.0.smithi084.stdout:3289: read oid 43 snap -1 2024-09-18T14:14:21.560 INFO:tasks.rados.rados.0.smithi084.stdout:3289: expect (ObjNum 1097 snap 350 seq_num 1097) 2024-09-18T14:14:21.560 INFO:tasks.rados.rados.0.smithi084.stdout:3290: read oid 17 snap -1 2024-09-18T14:14:21.560 INFO:tasks.rados.rados.0.smithi084.stdout:3290: expect deleted 2024-09-18T14:14:21.560 INFO:tasks.rados.rados.0.smithi084.stdout:3286: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:14:21.561 INFO:tasks.rados.rados.0.smithi084.stdout:3286: finishing rollback tid 2 to smithi08434663-6 2024-09-18T14:14:21.561 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 1326 (ObjNum 1061 snap 339 seq_num 1061) dirty exists 2024-09-18T14:14:21.561 INFO:tasks.rados.rados.0.smithi084.stdout:3282: finishing rollback tid 1 to smithi08434663-29 2024-09-18T14:14:21.561 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 1545 (ObjNum 129 snap 0 seq_num 2495519384) dirty dne 2024-09-18T14:14:21.561 INFO:tasks.rados.rados.0.smithi084.stdout:3285: expect (ObjNum 1021 snap 326 seq_num 1021) 2024-09-18T14:14:21.765 INFO:tasks.rados.rados.0.smithi084.stdout:3284: finishing copy_from to smithi08434663-48 2024-09-18T14:14:21.766 INFO:tasks.rados.rados.0.smithi084.stdout:3284: got expected ENOENT (src dne) 2024-09-18T14:14:21.766 INFO:tasks.rados.rados.0.smithi084.stdout:3284: finishing copy_from racing read to smithi08434663-48 2024-09-18T14:14:21.766 INFO:tasks.rados.rados.0.smithi084.stdout:3291: read oid 40 snap -1 2024-09-18T14:14:21.766 INFO:tasks.rados.rados.0.smithi084.stdout:3291: expect deleted 2024-09-18T14:14:21.766 INFO:tasks.rados.rados.0.smithi084.stdout:3282: done (9 left) 2024-09-18T14:14:21.766 INFO:tasks.rados.rados.0.smithi084.stdout:3283: done (8 left) 2024-09-18T14:14:21.766 INFO:tasks.rados.rados.0.smithi084.stdout:3284: done (7 left) 2024-09-18T14:14:21.766 INFO:tasks.rados.rados.0.smithi084.stdout:3285: done (6 left) 2024-09-18T14:14:21.766 INFO:tasks.rados.rados.0.smithi084.stdout:3286: done (5 left) 2024-09-18T14:14:21.766 INFO:tasks.rados.rados.0.smithi084.stdout:3292: copy_from oid 27 from oid 20 current snap is 360 2024-09-18T14:14:21.766 INFO:tasks.rados.rados.0.smithi084.stdout:3287: finishing write tid 1 to smithi08434663-11 2024-09-18T14:14:21.766 INFO:tasks.rados.rados.0.smithi084.stdout:3288: finishing write tid 1 to smithi08434663-4 2024-09-18T14:14:21.767 INFO:tasks.rados.rados.0.smithi084.stdout:3287: finishing write tid 2 to smithi08434663-11 2024-09-18T14:14:21.767 INFO:tasks.rados.rados.0.smithi084.stdout:3287: finishing write tid 3 to smithi08434663-11 2024-09-18T14:14:21.767 INFO:tasks.rados.rados.0.smithi084.stdout:3287: finishing write tid 4 to smithi08434663-11 2024-09-18T14:14:21.767 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1290 (ObjNum 1122 snap 360 seq_num 1122) dirty exists 2024-09-18T14:14:21.767 INFO:tasks.rados.rados.0.smithi084.stdout:3287: left oid 11 (ObjNum 1122 snap 360 seq_num 1122) 2024-09-18T14:14:21.767 INFO:tasks.rados.rados.0.smithi084.stdout:3288: finishing write tid 2 to smithi08434663-4 2024-09-18T14:14:21.767 INFO:tasks.rados.rados.0.smithi084.stdout:3288: finishing write tid 3 to smithi08434663-4 2024-09-18T14:14:21.767 INFO:tasks.rados.rados.0.smithi084.stdout:3288: finishing write tid 4 to smithi08434663-4 2024-09-18T14:14:21.767 INFO:tasks.rados.rados.0.smithi084.stdout:3288: finishing write tid 5 to smithi08434663-4 2024-09-18T14:14:21.767 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1629 (ObjNum 1123 snap 360 seq_num 1123) dirty exists 2024-09-18T14:14:21.767 INFO:tasks.rados.rados.0.smithi084.stdout:3288: left oid 4 (ObjNum 1123 snap 360 seq_num 1123) 2024-09-18T14:14:21.767 INFO:tasks.rados.rados.0.smithi084.stdout:3289: expect (ObjNum 1097 snap 350 seq_num 1097) 2024-09-18T14:14:21.897 INFO:tasks.rados.rados.0.smithi084.stdout:3292: finishing copy_from racing read to smithi08434663-27 2024-09-18T14:14:21.897 INFO:tasks.rados.rados.0.smithi084.stdout:3292: finishing copy_from to smithi08434663-27 2024-09-18T14:14:21.898 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1403 (ObjNum 1065 snap 339 seq_num 1065) dirty exists 2024-09-18T14:14:21.898 INFO:tasks.rados.rados.0.smithi084.stdout:3287: done (5 left) 2024-09-18T14:14:21.898 INFO:tasks.rados.rados.0.smithi084.stdout:3288: done (4 left) 2024-09-18T14:14:21.898 INFO:tasks.rados.rados.0.smithi084.stdout:3289: done (3 left) 2024-09-18T14:14:21.898 INFO:tasks.rados.rados.0.smithi084.stdout:3290: done (2 left) 2024-09-18T14:14:21.898 INFO:tasks.rados.rados.0.smithi084.stdout:3291: done (1 left) 2024-09-18T14:14:21.898 INFO:tasks.rados.rados.0.smithi084.stdout:3292: done (0 left) 2024-09-18T14:14:21.898 INFO:tasks.rados.rados.0.smithi084.stdout:3293: delete oid 21 current snap is 360 2024-09-18T14:14:21.901 INFO:tasks.rados.rados.0.smithi084.stdout:3293: done (0 left) 2024-09-18T14:14:21.901 INFO:tasks.rados.rados.0.smithi084.stdout:3294: read oid 29 snap -1 2024-09-18T14:14:21.901 INFO:tasks.rados.rados.0.smithi084.stdout:3294: expect deleted 2024-09-18T14:14:21.901 INFO:tasks.rados.rados.0.smithi084.stdout:3295: read oid 19 snap -1 2024-09-18T14:14:21.901 INFO:tasks.rados.rados.0.smithi084.stdout:3295: expect (ObjNum 1119 snap 360 seq_num 1119) 2024-09-18T14:14:21.902 INFO:tasks.rados.rados.0.smithi084.stdout:3296: copy_from oid 34 from oid 30 current snap is 360 2024-09-18T14:14:21.902 INFO:tasks.rados.rados.0.smithi084.stdout:3297: read oid 38 snap -1 2024-09-18T14:14:21.902 INFO:tasks.rados.rados.0.smithi084.stdout:3297: expect (ObjNum 957 snap 304 seq_num 957) 2024-09-18T14:14:21.902 INFO:tasks.rados.rados.0.smithi084.stdout:3298: write oid 43 current snap is 360 2024-09-18T14:14:21.902 INFO:tasks.rados.rados.0.smithi084.stdout:3298: seq_num 1125 ranges {507875=559016,1699066=737045} 2024-09-18T14:14:21.906 INFO:tasks.rados.rados.0.smithi084.stdout:3298: writing smithi08434663-43 from 507875 to 1066891 tid 1 2024-09-18T14:14:21.914 INFO:tasks.rados.rados.0.smithi084.stdout:3298: writing smithi08434663-43 from 1699066 to 2436111 tid 2 2024-09-18T14:14:21.914 INFO:tasks.rados.rados.0.smithi084.stdout:3299: snap_create 2024-09-18T14:14:21.914 INFO:tasks.rados.rados.0.smithi084.stdout:3295: expect (ObjNum 1119 snap 360 seq_num 1119) 2024-09-18T14:14:22.067 INFO:tasks.rados.rados.0.smithi084.stdout:3297: expect (ObjNum 957 snap 304 seq_num 957) 2024-09-18T14:14:22.192 INFO:tasks.rados.rados.0.smithi084.stdout:3296: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:14:22.192 INFO:tasks.rados.rados.0.smithi084.stdout:3296: finishing copy_from to smithi08434663-34 2024-09-18T14:14:22.192 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1263 (ObjNum 1021 snap 326 seq_num 1021) dirty exists 2024-09-18T14:14:22.192 INFO:tasks.rados.rados.0.smithi084.stdout:3298: finishing write tid 1 to smithi08434663-43 2024-09-18T14:14:22.193 INFO:tasks.rados.rados.0.smithi084.stdout:3298: finishing write tid 2 to smithi08434663-43 2024-09-18T14:14:22.193 INFO:tasks.rados.rados.0.smithi084.stdout:3298: finishing write tid 3 to smithi08434663-43 2024-09-18T14:14:22.193 INFO:tasks.rados.rados.0.smithi084.stdout:3298: finishing write tid 4 to smithi08434663-43 2024-09-18T14:14:22.193 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 1278 (ObjNum 1125 snap 360 seq_num 1125) dirty exists 2024-09-18T14:14:22.193 INFO:tasks.rados.rados.0.smithi084.stdout:3298: left oid 43 (ObjNum 1125 snap 360 seq_num 1125) 2024-09-18T14:14:22.539 INFO:tasks.rados.rados.0.smithi084.stdout:3294: done (5 left) 2024-09-18T14:14:22.539 INFO:tasks.rados.rados.0.smithi084.stdout:3295: done (4 left) 2024-09-18T14:14:22.539 INFO:tasks.rados.rados.0.smithi084.stdout:3296: done (3 left) 2024-09-18T14:14:22.539 INFO:tasks.rados.rados.0.smithi084.stdout:3297: done (2 left) 2024-09-18T14:14:22.539 INFO:tasks.rados.rados.0.smithi084.stdout:3298: done (1 left) 2024-09-18T14:14:22.539 INFO:tasks.rados.rados.0.smithi084.stdout:3299: done (0 left) 2024-09-18T14:14:22.539 INFO:tasks.rados.rados.0.smithi084.stdout:3300: copy_from oid 26 from oid 36 current snap is 361 2024-09-18T14:14:22.539 INFO:tasks.rados.rados.0.smithi084.stdout:3301: write (excl) oid 40 current snap is 361 2024-09-18T14:14:22.539 INFO:tasks.rados.rados.0.smithi084.stdout:3301: seq_num 1126 ranges {524682=750618,1757548=597056} 2024-09-18T14:14:22.545 INFO:tasks.rados.rados.0.smithi084.stdout:3301: writing smithi08434663-40 from 524682 to 1275300 tid 1 2024-09-18T14:14:22.552 INFO:tasks.rados.rados.0.smithi084.stdout:3301: writing smithi08434663-40 from 1757548 to 2354604 tid 2 2024-09-18T14:14:22.552 INFO:tasks.rados.rados.0.smithi084.stdout:3302: snap_remove snap 349 2024-09-18T14:14:23.487 INFO:tasks.rados.rados.0.smithi084.stdout:3302: done (2 left) 2024-09-18T14:14:23.487 INFO:tasks.rados.rados.0.smithi084.stdout:3303: write oid 7 current snap is 361 2024-09-18T14:14:23.487 INFO:tasks.rados.rados.0.smithi084.stdout:3303: seq_num 1127 ranges {541489=425867,1416030=740714,2732654=208835} 2024-09-18T14:14:23.492 INFO:tasks.rados.rados.0.smithi084.stdout:3303: writing smithi08434663-7 from 541489 to 967356 tid 1 2024-09-18T14:14:23.499 INFO:tasks.rados.rados.0.smithi084.stdout:3303: writing smithi08434663-7 from 1416030 to 2156744 tid 2 2024-09-18T14:14:23.504 INFO:tasks.rados.rados.0.smithi084.stdout:3303: writing smithi08434663-7 from 2732654 to 2941489 tid 3 2024-09-18T14:14:23.504 INFO:tasks.rados.rados.0.smithi084.stdout:3304: snap_create 2024-09-18T14:14:23.504 INFO:tasks.rados.rados.0.smithi084.stdout:3301: finishing write tid 1 to smithi08434663-40 2024-09-18T14:14:23.504 INFO:tasks.rados.rados.0.smithi084.stdout:3301: finishing write tid 2 to smithi08434663-40 2024-09-18T14:14:23.504 INFO:tasks.rados.rados.0.smithi084.stdout:3301: finishing write tid 3 to smithi08434663-40 2024-09-18T14:14:23.504 INFO:tasks.rados.rados.0.smithi084.stdout:3301: finishing write tid 4 to smithi08434663-40 2024-09-18T14:14:23.504 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 1554 (ObjNum 1126 snap 361 seq_num 1126) dirty exists 2024-09-18T14:14:23.504 INFO:tasks.rados.rados.0.smithi084.stdout:3301: left oid 40 (ObjNum 1126 snap 361 seq_num 1126) 2024-09-18T14:14:23.504 INFO:tasks.rados.rados.0.smithi084.stdout:3300: finishing copy_from racing read to smithi08434663-26 2024-09-18T14:14:23.504 INFO:tasks.rados.rados.0.smithi084.stdout:3300: finishing copy_from to smithi08434663-26 2024-09-18T14:14:23.505 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 1043 (ObjNum 1072 snap 341 seq_num 1072) dirty exists 2024-09-18T14:14:23.505 INFO:tasks.rados.rados.0.smithi084.stdout:3303: finishing write tid 1 to smithi08434663-7 2024-09-18T14:14:23.505 INFO:tasks.rados.rados.0.smithi084.stdout:3303: finishing write tid 2 to smithi08434663-7 2024-09-18T14:14:23.506 INFO:tasks.rados.rados.0.smithi084.stdout:3303: finishing write tid 3 to smithi08434663-7 2024-09-18T14:14:23.506 INFO:tasks.rados.rados.0.smithi084.stdout:3303: finishing write tid 4 to smithi08434663-7 2024-09-18T14:14:23.506 INFO:tasks.rados.rados.0.smithi084.stdout:3303: finishing write tid 5 to smithi08434663-7 2024-09-18T14:14:23.506 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 1330 (ObjNum 1127 snap 361 seq_num 1127) dirty exists 2024-09-18T14:14:23.506 INFO:tasks.rados.rados.0.smithi084.stdout:3303: left oid 7 (ObjNum 1127 snap 361 seq_num 1127) 2024-09-18T14:14:24.485 INFO:tasks.rados.rados.0.smithi084.stdout:3300: done (3 left) 2024-09-18T14:14:24.485 INFO:tasks.rados.rados.0.smithi084.stdout:3301: done (2 left) 2024-09-18T14:14:24.485 INFO:tasks.rados.rados.0.smithi084.stdout:3303: done (1 left) 2024-09-18T14:14:24.485 INFO:tasks.rados.rados.0.smithi084.stdout:3304: done (0 left) 2024-09-18T14:14:24.485 INFO:tasks.rados.rados.0.smithi084.stdout:3305: rollback oid 16 current snap is 362 2024-09-18T14:14:24.485 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 16 to 351 2024-09-18T14:14:24.486 INFO:tasks.rados.rados.0.smithi084.stdout:3306: rollback oid 15 current snap is 362 2024-09-18T14:14:24.486 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 15 to 332 2024-09-18T14:14:24.486 INFO:tasks.rados.rados.0.smithi084.stdout:3307: snap_remove snap 359 2024-09-18T14:14:25.487 INFO:tasks.rados.rados.0.smithi084.stdout:3307: done (2 left) 2024-09-18T14:14:25.487 INFO:tasks.rados.rados.0.smithi084.stdout:3308: delete oid 35 current snap is 362 2024-09-18T14:14:25.487 INFO:tasks.rados.rados.0.smithi084.stdout:3305: finishing rollback tid 0 to smithi08434663-16 2024-09-18T14:14:25.487 INFO:tasks.rados.rados.0.smithi084.stdout:3305: finishing rollback tid 1 to smithi08434663-16 2024-09-18T14:14:25.487 INFO:tasks.rados.rados.0.smithi084.stdout:3305: finishing rollback tid 2 to smithi08434663-16 2024-09-18T14:14:25.487 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1500 (ObjNum 1063 snap 339 seq_num 1063) dirty exists 2024-09-18T14:14:25.487 INFO:tasks.rados.rados.0.smithi084.stdout:3306: finishing rollback tid 1 to smithi08434663-15 2024-09-18T14:14:25.488 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 1489 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:14:25.490 INFO:tasks.rados.rados.0.smithi084.stdout:3305: done (2 left) 2024-09-18T14:14:25.490 INFO:tasks.rados.rados.0.smithi084.stdout:3306: done (1 left) 2024-09-18T14:14:25.490 INFO:tasks.rados.rados.0.smithi084.stdout:3308: done (0 left) 2024-09-18T14:14:25.490 INFO:tasks.rados.rados.0.smithi084.stdout:3309: delete oid 6 current snap is 362 2024-09-18T14:14:25.495 INFO:tasks.rados.rados.0.smithi084.stdout:3309: done (0 left) 2024-09-18T14:14:25.495 INFO:tasks.rados.rados.0.smithi084.stdout:3310: write (excl) oid 32 current snap is 362 2024-09-18T14:14:25.495 INFO:tasks.rados.rados.0.smithi084.stdout:3310: seq_num 1130 ranges {591910=651614,1875123=604394} 2024-09-18T14:14:25.501 INFO:tasks.rados.rados.0.smithi084.stdout:3310: writing smithi08434663-32 from 591910 to 1243524 tid 1 2024-09-18T14:14:25.509 INFO:tasks.rados.rados.0.smithi084.stdout:3310: writing smithi08434663-32 from 1875123 to 2479517 tid 2 2024-09-18T14:14:25.509 INFO:tasks.rados.rados.0.smithi084.stdout:3311: snap_create 2024-09-18T14:14:25.509 INFO:tasks.rados.rados.0.smithi084.stdout:3310: finishing write tid 1 to smithi08434663-32 2024-09-18T14:14:25.512 INFO:tasks.rados.rados.0.smithi084.stdout:3310: finishing write tid 2 to smithi08434663-32 2024-09-18T14:14:25.513 INFO:tasks.rados.rados.0.smithi084.stdout:3310: finishing write tid 3 to smithi08434663-32 2024-09-18T14:14:25.513 INFO:tasks.rados.rados.0.smithi084.stdout:3310: finishing write tid 4 to smithi08434663-32 2024-09-18T14:14:25.513 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 32 v 1858 (ObjNum 1130 snap 362 seq_num 1130) dirty exists 2024-09-18T14:14:25.513 INFO:tasks.rados.rados.0.smithi084.stdout:3310: left oid 32 (ObjNum 1130 snap 362 seq_num 1130) 2024-09-18T14:14:26.492 INFO:tasks.rados.rados.0.smithi084.stdout:3310: done (1 left) 2024-09-18T14:14:26.492 INFO:tasks.rados.rados.0.smithi084.stdout:3311: done (0 left) 2024-09-18T14:14:26.492 INFO:tasks.rados.rados.0.smithi084.stdout:3312: snap_create 2024-09-18T14:14:27.495 INFO:tasks.rados.rados.0.smithi084.stdout:3312: done (0 left) 2024-09-18T14:14:27.495 INFO:tasks.rados.rados.0.smithi084.stdout:3313: read oid 43 snap -1 2024-09-18T14:14:27.495 INFO:tasks.rados.rados.0.smithi084.stdout:3313: expect (ObjNum 1125 snap 360 seq_num 1125) 2024-09-18T14:14:27.495 INFO:tasks.rados.rados.0.smithi084.stdout:3314: read oid 40 snap -1 2024-09-18T14:14:27.496 INFO:tasks.rados.rados.0.smithi084.stdout:3314: expect (ObjNum 1126 snap 361 seq_num 1126) 2024-09-18T14:14:27.496 INFO:tasks.rados.rados.0.smithi084.stdout:3315: copy_from oid 9 from oid 19 current snap is 364 2024-09-18T14:14:27.496 INFO:tasks.rados.rados.0.smithi084.stdout:3316: copy_from oid 28 from oid 26 current snap is 364 2024-09-18T14:14:27.496 INFO:tasks.rados.rados.0.smithi084.stdout:3317: read oid 34 snap 342 2024-09-18T14:14:27.496 INFO:tasks.rados.rados.0.smithi084.stdout:3317: expect deleted 2024-09-18T14:14:27.496 INFO:tasks.rados.rados.0.smithi084.stdout:3318: copy_from oid 5 from oid 14 current snap is 364 2024-09-18T14:14:27.496 INFO:tasks.rados.rados.0.smithi084.stdout:3319: write (excl) oid 18 current snap is 364 2024-09-18T14:14:27.496 INFO:tasks.rados.rados.0.smithi084.stdout:3319: seq_num 1131 ranges {608717=726863,1933605=748052} 2024-09-18T14:14:27.502 INFO:tasks.rados.rados.0.smithi084.stdout:3319: writing smithi08434663-18 from 608717 to 1335580 tid 1 2024-09-18T14:14:27.510 INFO:tasks.rados.rados.0.smithi084.stdout:3319: writing smithi08434663-18 from 1933605 to 2681657 tid 2 2024-09-18T14:14:27.510 INFO:tasks.rados.rados.0.smithi084.stdout:3320: delete oid 21 current snap is 364 2024-09-18T14:14:27.511 INFO:tasks.rados.rados.0.smithi084.stdout:3318: finishing copy_from to smithi08434663-5 2024-09-18T14:14:27.511 INFO:tasks.rados.rados.0.smithi084.stdout:3318: got expected ENOENT (src dne) 2024-09-18T14:14:27.511 INFO:tasks.rados.rados.0.smithi084.stdout:3318: finishing copy_from racing read to smithi08434663-5 2024-09-18T14:14:27.511 INFO:tasks.rados.rados.0.smithi084.stdout:3315: finishing copy_from racing read to smithi08434663-9 2024-09-18T14:14:27.512 INFO:tasks.rados.rados.0.smithi084.stdout:3315: finishing copy_from to smithi08434663-9 2024-09-18T14:14:27.512 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 1821 (ObjNum 1119 snap 360 seq_num 1119) dirty exists 2024-09-18T14:14:27.513 INFO:tasks.rados.rados.0.smithi084.stdout:3315: done (7 left) 2024-09-18T14:14:27.513 INFO:tasks.rados.rados.0.smithi084.stdout:3317: done (6 left) 2024-09-18T14:14:27.513 INFO:tasks.rados.rados.0.smithi084.stdout:3318: done (5 left) 2024-09-18T14:14:27.513 INFO:tasks.rados.rados.0.smithi084.stdout:3320: done (4 left) 2024-09-18T14:14:27.513 INFO:tasks.rados.rados.0.smithi084.stdout:3321: delete oid 44 current snap is 364 2024-09-18T14:14:27.516 INFO:tasks.rados.rados.0.smithi084.stdout:3316: finishing copy_from racing read to smithi08434663-28 2024-09-18T14:14:27.518 INFO:tasks.rados.rados.0.smithi084.stdout:3313: expect (ObjNum 1125 snap 360 seq_num 1125) 2024-09-18T14:14:27.684 INFO:tasks.rados.rados.0.smithi084.stdout:3314: expect (ObjNum 1126 snap 361 seq_num 1126) 2024-09-18T14:14:27.776 INFO:tasks.rados.rados.0.smithi084.stdout:3319: finishing write tid 1 to smithi08434663-18 2024-09-18T14:14:27.776 INFO:tasks.rados.rados.0.smithi084.stdout:3319: finishing write tid 2 to smithi08434663-18 2024-09-18T14:14:27.776 INFO:tasks.rados.rados.0.smithi084.stdout:3319: finishing write tid 3 to smithi08434663-18 2024-09-18T14:14:27.776 INFO:tasks.rados.rados.0.smithi084.stdout:3319: finishing write tid 4 to smithi08434663-18 2024-09-18T14:14:27.776 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 1043 (ObjNum 1131 snap 364 seq_num 1131) dirty exists 2024-09-18T14:14:27.776 INFO:tasks.rados.rados.0.smithi084.stdout:3319: left oid 18 (ObjNum 1131 snap 364 seq_num 1131) 2024-09-18T14:14:27.776 INFO:tasks.rados.rados.0.smithi084.stdout:3316: finishing copy_from to smithi08434663-28 2024-09-18T14:14:27.776 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 1043 (ObjNum 1072 snap 341 seq_num 1072) dirty exists 2024-09-18T14:14:27.777 INFO:tasks.rados.rados.0.smithi084.stdout:3313: done (4 left) 2024-09-18T14:14:27.777 INFO:tasks.rados.rados.0.smithi084.stdout:3314: done (3 left) 2024-09-18T14:14:27.777 INFO:tasks.rados.rados.0.smithi084.stdout:3316: done (2 left) 2024-09-18T14:14:27.777 INFO:tasks.rados.rados.0.smithi084.stdout:3319: done (1 left) 2024-09-18T14:14:27.777 INFO:tasks.rados.rados.0.smithi084.stdout:3321: done (0 left) 2024-09-18T14:14:27.777 INFO:tasks.rados.rados.0.smithi084.stdout:3322: write oid 22 current snap is 364 2024-09-18T14:14:27.777 INFO:tasks.rados.rados.0.smithi084.stdout:3322: seq_num 1134 ranges {659138=668963,2109051=611732} 2024-09-18T14:14:27.783 INFO:tasks.rados.rados.0.smithi084.stdout:3322: writing smithi08434663-22 from 659138 to 1328101 tid 1 2024-09-18T14:14:27.790 INFO:tasks.rados.rados.0.smithi084.stdout:3322: writing smithi08434663-22 from 2109051 to 2720783 tid 2 2024-09-18T14:14:27.790 INFO:tasks.rados.rados.0.smithi084.stdout:3323: rollback oid 30 current snap is 364 2024-09-18T14:14:27.791 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 30 to 342 2024-09-18T14:14:27.791 INFO:tasks.rados.rados.0.smithi084.stdout:3322: finishing write tid 1 to smithi08434663-22 2024-09-18T14:14:27.791 INFO:tasks.rados.rados.0.smithi084.stdout:3324: rollback oid 16 current snap is 364 2024-09-18T14:14:27.791 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 16 to 351 2024-09-18T14:14:27.791 INFO:tasks.rados.rados.0.smithi084.stdout:3325: snap_remove snap 321 2024-09-18T14:14:28.489 INFO:tasks.rados.rados.0.smithi084.stdout:3325: done (3 left) 2024-09-18T14:14:28.489 INFO:tasks.rados.rados.0.smithi084.stdout:3326: rollback oid 48 current snap is 364 2024-09-18T14:14:28.489 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 48 to 342 2024-09-18T14:14:28.489 INFO:tasks.rados.rados.0.smithi084.stdout:3322: finishing write tid 2 to smithi08434663-22 2024-09-18T14:14:28.489 INFO:tasks.rados.rados.0.smithi084.stdout:3322: finishing write tid 3 to smithi08434663-22 2024-09-18T14:14:28.489 INFO:tasks.rados.rados.0.smithi084.stdout:3322: finishing write tid 4 to smithi08434663-22 2024-09-18T14:14:28.489 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 1046 (ObjNum 1134 snap 364 seq_num 1134) dirty exists 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:3322: left oid 22 (ObjNum 1134 snap 364 seq_num 1134) 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:3323: finishing rollback tid 0 to smithi08434663-30 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:3324: finishing rollback tid 0 to smithi08434663-16 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:3323: finishing rollback tid 1 to smithi08434663-30 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:3324: finishing rollback tid 1 to smithi08434663-16 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:3323: finishing rollback tid 2 to smithi08434663-30 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 1266 (ObjNum 1021 snap 326 seq_num 1021) dirty exists 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:3324: finishing rollback tid 2 to smithi08434663-16 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 1824 (ObjNum 1063 snap 339 seq_num 1063) dirty exists 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:3322: done (3 left) 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:3323: done (2 left) 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:3324: done (1 left) 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:3327: read oid 34 snap -1 2024-09-18T14:14:28.490 INFO:tasks.rados.rados.0.smithi084.stdout:3327: expect (ObjNum 1021 snap 326 seq_num 1021) 2024-09-18T14:14:28.491 INFO:tasks.rados.rados.0.smithi084.stdout:3328: copy_from oid 43 from oid 7 current snap is 364 2024-09-18T14:14:28.491 INFO:tasks.rados.rados.0.smithi084.stdout:3329: delete oid 42 current snap is 364 2024-09-18T14:14:28.501 INFO:tasks.rados.rados.0.smithi084.stdout:3326: finishing rollback tid 0 to smithi08434663-48 2024-09-18T14:14:28.503 INFO:tasks.rados.rados.0.smithi084.stdout:3326: finishing rollback tid 1 to smithi08434663-48 2024-09-18T14:14:28.503 INFO:tasks.rados.rados.0.smithi084.stdout:3326: finishing rollback tid 2 to smithi08434663-48 2024-09-18T14:14:28.503 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1540 (ObjNum 1088 snap 342 seq_num 1088) dirty exists 2024-09-18T14:14:28.521 INFO:tasks.rados.rados.0.smithi084.stdout:3328: finishing copy_from racing read to smithi08434663-43 2024-09-18T14:14:28.528 INFO:tasks.rados.rados.0.smithi084.stdout:3328: finishing copy_from to smithi08434663-43 2024-09-18T14:14:28.528 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 1330 (ObjNum 1127 snap 361 seq_num 1127) dirty exists 2024-09-18T14:14:28.530 INFO:tasks.rados.rados.0.smithi084.stdout:3327: expect (ObjNum 1021 snap 326 seq_num 1021) 2024-09-18T14:14:28.735 INFO:tasks.rados.rados.0.smithi084.stdout:3326: done (3 left) 2024-09-18T14:14:28.736 INFO:tasks.rados.rados.0.smithi084.stdout:3327: done (2 left) 2024-09-18T14:14:28.736 INFO:tasks.rados.rados.0.smithi084.stdout:3328: done (1 left) 2024-09-18T14:14:28.736 INFO:tasks.rados.rados.0.smithi084.stdout:3329: done (0 left) 2024-09-18T14:14:28.736 INFO:tasks.rados.rados.0.smithi084.stdout:3330: delete oid 9 current snap is 364 2024-09-18T14:14:28.738 INFO:tasks.rados.rados.0.smithi084.stdout:3330: done (0 left) 2024-09-18T14:14:28.738 INFO:tasks.rados.rados.0.smithi084.stdout:3331: write oid 29 current snap is 364 2024-09-18T14:14:28.738 INFO:tasks.rados.rados.0.smithi084.stdout:3331: seq_num 1137 ranges {709559=494710,1884497=759059} 2024-09-18T14:14:28.744 INFO:tasks.rados.rados.0.smithi084.stdout:3331: writing smithi08434663-29 from 709559 to 1204269 tid 1 2024-09-18T14:14:28.752 INFO:tasks.rados.rados.0.smithi084.stdout:3331: writing smithi08434663-29 from 1884497 to 2643556 tid 2 2024-09-18T14:14:28.752 INFO:tasks.rados.rados.0.smithi084.stdout:3332: rollback oid 25 current snap is 364 2024-09-18T14:14:28.752 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 25 to 358 2024-09-18T14:14:28.752 INFO:tasks.rados.rados.0.smithi084.stdout:3331: finishing write tid 1 to smithi08434663-29 2024-09-18T14:14:28.752 INFO:tasks.rados.rados.0.smithi084.stdout:3333: snap_remove snap 344 2024-09-18T14:14:29.530 INFO:tasks.rados.rados.0.smithi084.stdout:3333: done (2 left) 2024-09-18T14:14:29.531 INFO:tasks.rados.rados.0.smithi084.stdout:3334: write (excl) oid 41 current snap is 364 2024-09-18T14:14:29.531 INFO:tasks.rados.rados.0.smithi084.stdout:3331: finishing write tid 2 to smithi08434663-29 2024-09-18T14:14:29.531 INFO:tasks.rados.rados.0.smithi084.stdout:3334: seq_num 1138 ranges {726366=569959,1826626=502717,2801601=324765} 2024-09-18T14:14:29.538 INFO:tasks.rados.rados.0.smithi084.stdout:3334: writing smithi08434663-41 from 726366 to 1296325 tid 1 2024-09-18T14:14:29.543 INFO:tasks.rados.rados.0.smithi084.stdout:3334: writing smithi08434663-41 from 1826626 to 2329343 tid 2 2024-09-18T14:14:29.548 INFO:tasks.rados.rados.0.smithi084.stdout:3334: writing smithi08434663-41 from 2801601 to 3126366 tid 3 2024-09-18T14:14:29.548 INFO:tasks.rados.rados.0.smithi084.stdout:3335: snap_remove snap 348 2024-09-18T14:14:30.537 INFO:tasks.rados.rados.0.smithi084.stdout:3335: done (3 left) 2024-09-18T14:14:30.537 INFO:tasks.rados.rados.0.smithi084.stdout:3336: snap_create 2024-09-18T14:14:30.538 INFO:tasks.rados.rados.0.smithi084.stdout:3332: finishing rollback tid 0 to smithi08434663-25 2024-09-18T14:14:30.538 INFO:tasks.rados.rados.0.smithi084.stdout:3331: finishing write tid 3 to smithi08434663-29 2024-09-18T14:14:30.538 INFO:tasks.rados.rados.0.smithi084.stdout:3332: finishing rollback tid 1 to smithi08434663-25 2024-09-18T14:14:30.538 INFO:tasks.rados.rados.0.smithi084.stdout:3332: finishing rollback tid 2 to smithi08434663-25 2024-09-18T14:14:30.538 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 1543 (ObjNum 1102 snap 352 seq_num 1102) dirty exists 2024-09-18T14:14:30.538 INFO:tasks.rados.rados.0.smithi084.stdout:3331: finishing write tid 4 to smithi08434663-29 2024-09-18T14:14:30.538 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 1557 (ObjNum 1137 snap 364 seq_num 1137) dirty exists 2024-09-18T14:14:30.538 INFO:tasks.rados.rados.0.smithi084.stdout:3331: left oid 29 (ObjNum 1137 snap 364 seq_num 1137) 2024-09-18T14:14:30.539 INFO:tasks.rados.rados.0.smithi084.stdout:3334: finishing write tid 1 to smithi08434663-41 2024-09-18T14:14:30.539 INFO:tasks.rados.rados.0.smithi084.stdout:3334: finishing write tid 2 to smithi08434663-41 2024-09-18T14:14:30.539 INFO:tasks.rados.rados.0.smithi084.stdout:3334: finishing write tid 3 to smithi08434663-41 2024-09-18T14:14:30.539 INFO:tasks.rados.rados.0.smithi084.stdout:3334: finishing write tid 4 to smithi08434663-41 2024-09-18T14:14:30.539 INFO:tasks.rados.rados.0.smithi084.stdout:3334: finishing write tid 5 to smithi08434663-41 2024-09-18T14:14:30.539 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 1653 (ObjNum 1138 snap 364 seq_num 1138) dirty exists 2024-09-18T14:14:30.539 INFO:tasks.rados.rados.0.smithi084.stdout:3334: left oid 41 (ObjNum 1138 snap 364 seq_num 1138) 2024-09-18T14:14:30.844 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-18T14:14:30.844 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-18T14:14:30.844 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-18T14:14:30.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-18T14:14:30.849 INFO:tasks.ceph.mon.f:Started 2024-09-18T14:14:30.849 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-18T14:14:30.849 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-18T14:14:30.849 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-18T14:14:30.849 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-18T14:14:30.851 INFO:tasks.ceph.mon.i:Started 2024-09-18T14:14:30.851 INFO:tasks.mon_thrash:Sending STOP to mon e 2024-09-18T14:14:30.852 INFO:tasks.ceph.mon.e:Sent signal 19 2024-09-18T14:14:30.852 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-18T14:14:45.604 INFO:tasks.rados.rados.0.smithi084.stdout:3331: done (3 left) 2024-09-18T14:14:45.605 INFO:tasks.rados.rados.0.smithi084.stdout:3332: done (2 left) 2024-09-18T14:14:45.605 INFO:tasks.rados.rados.0.smithi084.stdout:3334: done (1 left) 2024-09-18T14:14:45.605 INFO:tasks.rados.rados.0.smithi084.stdout:3336: done (0 left) 2024-09-18T14:14:45.605 INFO:tasks.rados.rados.0.smithi084.stdout:3337: delete oid 22 current snap is 365 2024-09-18T14:14:45.609 INFO:tasks.rados.rados.0.smithi084.stdout:3337: done (0 left) 2024-09-18T14:14:45.609 INFO:tasks.rados.rados.0.smithi084.stdout:3338: copy_from oid 40 from oid 36 current snap is 365 2024-09-18T14:14:45.609 INFO:tasks.rados.rados.0.smithi084.stdout:3339: read oid 9 snap 351 2024-09-18T14:14:45.609 INFO:tasks.rados.rados.0.smithi084.stdout:3339: expect (ObjNum 1074 snap 341 seq_num 1074) 2024-09-18T14:14:45.609 INFO:tasks.rados.rados.0.smithi084.stdout:3340: snap_create 2024-09-18T14:14:45.617 INFO:tasks.rados.rados.0.smithi084.stdout:3339: expect (ObjNum 1074 snap 341 seq_num 1074) 2024-09-18T14:14:45.702 INFO:tasks.rados.rados.0.smithi084.stdout:3338: finishing copy_from racing read to smithi08434663-40 2024-09-18T14:14:45.702 INFO:tasks.rados.rados.0.smithi084.stdout:3338: finishing copy_from to smithi08434663-40 2024-09-18T14:14:45.702 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 1043 (ObjNum 1072 snap 341 seq_num 1072) dirty exists 2024-09-18T14:14:45.852 INFO:tasks.mon_thrash:Sending CONT to mon e 2024-09-18T14:14:45.853 INFO:tasks.ceph.mon.e:Sent signal 18 2024-09-18T14:14:45.853 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:14:45.853 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:14:45.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.955+0000 7fa2ec19d640 10 monclient: get_monmap_and_config 2024-09-18T14:14:45.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.955+0000 7fa2ec19d640 10 monclient: build_initial_monmap 2024-09-18T14:14:45.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.955+0000 7fa2ec19d640 10 monclient: monmap: 2024-09-18T14:14:45.956 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:14:45.956 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:45.956 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:14:45.956850+0000 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:14:45.956850+0000 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:14:45.957 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:45.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.956+0000 7fa2ec19d640 1 Processor -- start 2024-09-18T14:14:45.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.956+0000 7fa2ec19d640 1 -- start start 2024-09-18T14:14:45.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.956+0000 7fa2ec19d640 10 monclient: init 2024-09-18T14:14:45.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2ec19d640 10 monclient: _reopen_session rank -1 2024-09-18T14:14:45.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2ec19d640 10 monclient: _add_conns ranks=[4,5,1,0,2,6,3,8,7] 2024-09-18T14:14:45.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2ec19d640 10 monclient(hunting): picked mon.noname-c con 0x7fa2e40a41c0 addr v1:172.21.15.45:6791/0 2024-09-18T14:14:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2ec19d640 10 monclient(hunting): picked mon.noname-h con 0x7fa2e406e480 addr v1:172.21.15.84:6791/0 2024-09-18T14:14:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2ec19d640 10 monclient(hunting): picked mon.noname-f con 0x7fa2e406e850 addr v1:172.21.15.84:6789/0 2024-09-18T14:14:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2ec19d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e405ed70 con 0x7fa2e406e850 2024-09-18T14:14:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2ec19d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e409ffd0 con 0x7fa2e40a41c0 2024-09-18T14:14:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2ec19d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e4133340 con 0x7fa2e406e480 2024-09-18T14:14:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2ec19d640 10 monclient(hunting): _renew_subs 2024-09-18T14:14:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2ec19d640 10 monclient(hunting): authenticate will time out at 1701.295220s 2024-09-18T14:14:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e9f39640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fa2e40a41c0 0x7fa2e4132470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:35560/0 (socket says 172.21.15.45:35560) 2024-09-18T14:14:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e9f39640 1 -- 172.21.15.45:0/2089295168 learned_addr learned my addr 172.21.15.45:0/2089295168 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:14:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800106201 0 0) 0x7fa2e409ffd0 con 0x7fa2e40a41c0 2024-09-18T14:14:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient(hunting): my global_id is 164821 2024-09-18T14:14:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2d00034e0 con 0x7fa2e40a41c0 2024-09-18T14:14:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700146362 0 0) 0x7fa2e4133340 con 0x7fa2e406e480 2024-09-18T14:14:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient(hunting): my global_id is 184541 2024-09-18T14:14:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e409ffd0 con 0x7fa2e406e480 2024-09-18T14:14:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902645748 0 0) 0x7fa2e405ed70 con 0x7fa2e406e850 2024-09-18T14:14:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient(hunting): my global_id is 214102 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e4133340 con 0x7fa2e406e850 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1824002521 0 0) 0x7fa2e409ffd0 con 0x7fa2e406e480 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 >> v1:172.21.15.45:6791/0 conn(0x7fa2e40a41c0 legacy=0x7fa2e4132470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 >> v1:172.21.15.84:6789/0 conn(0x7fa2e406e850 legacy=0x7fa2e413aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient: _finish_hunting 0 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 1 monclient: found mon.noname-h 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e4069030 con 0x7fa2e406e480 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient: _finish_auth 0 2024-09-18T14:14:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2d4001810 con 0x7fa2e406e480 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2e8f37640 10 monclient: dump: 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2ec19d640 5 monclient: authenticate success, global_id 184541 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.957+0000 7fa2ec19d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa2d40019f0 con 0x7fa2e406e480 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2e8f37640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2ec19d640 10 monclient: get_monmap_and_config success 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2ec19d640 4 set_mon_vals no callback set 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2e8f37640 1 -- 172.21.15.45:0/2089295168 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2d4001fa0 con 0x7fa2e406e480 2024-09-18T14:14:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2e8f37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2e8f37640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2e8f37640 10 monclient: dump: 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2ec19d640 10 monclient: shutdown 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2ec19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2ec19d640 1 -- 172.21.15.45:0/2089295168 >> v1:172.21.15.84:6791/0 conn(0x7fa2e406e480 legacy=0x7fa2e41377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2ec19d640 1 -- 172.21.15.45:0/2089295168 shutdown_connections 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2ec19d640 1 -- 172.21.15.45:0/2089295168 >> 172.21.15.45:0/2089295168 conn(0x7fa2e40636e0 msgr2=0x7fa2e40a3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2ec19d640 1 -- 172.21.15.45:0/2089295168 shutdown_connections 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.958+0000 7fa2ec19d640 1 -- 172.21.15.45:0/2089295168 wait complete. 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 10 monclient: build_initial_monmap 2024-09-18T14:14:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 10 monclient: monmap: 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:14:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 1 Processor -- start 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 1 -- start start 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 10 monclient: init 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 10 monclient: _reopen_session rank -1 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 10 monclient: _add_conns ranks=[1,3,5,6,2,0,7,4,8] 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 10 monclient(hunting): picked mon.noname-f con 0x7fa2e406e480 addr v1:172.21.15.84:6789/0 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 10 monclient(hunting): picked mon.noname-g con 0x7fa2e406e850 addr v1:172.21.15.84:6790/0 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 10 monclient(hunting): picked mon.noname-h con 0x7fa2e40a41c0 addr v1:172.21.15.84:6791/0 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e40686b0 con 0x7fa2e406e480 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e40683f0 con 0x7fa2e406e850 2024-09-18T14:14:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e41208e0 con 0x7fa2e40a41c0 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 10 monclient(hunting): _renew_subs 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2ec19d640 10 monclient(hunting): authenticate will time out at 1701.297897s 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2e9f39640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa2e406e480 0x7fa2e4141490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:59836/0 (socket says 172.21.15.45:59836) 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.959+0000 7fa2e9f39640 1 -- 172.21.15.45:0/4292157466 learned_addr learned my addr 172.21.15.45:0/4292157466 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100028217 0 0) 0x7fa2e40686b0 con 0x7fa2e406e480 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): my global_id is 214111 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c80034e0 con 0x7fa2e406e480 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564139838 0 0) 0x7fa2e41208e0 con 0x7fa2e40a41c0 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:45.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): my global_id is 184550 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e40686b0 con 0x7fa2e40a41c0 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235971122 0 0) 0x7fa2e40683f0 con 0x7fa2e406e850 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): my global_id is 184539 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e41208e0 con 0x7fa2e406e850 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 996884992 0 0) 0x7fa2c80034e0 con 0x7fa2e406e480 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2e40683f0 con 0x7fa2e406e480 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2e0001a60 con 0x7fa2e406e480 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): dump: 2024-09-18T14:14:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:45.977 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1816647953 0 0) 0x7fa2e40686b0 con 0x7fa2e40a41c0 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2c80034e0 con 0x7fa2e40a41c0 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2dc001a00 con 0x7fa2e40a41c0 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): dump: 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4233363983 0 0) 0x7fa2e41208e0 con 0x7fa2e406e850 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2e40686b0 con 0x7fa2e406e850 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2d4001ea0 con 0x7fa2e406e850 2024-09-18T14:14:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient(hunting): dump: 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3419956673 0 0) 0x7fa2e40683f0 con 0x7fa2e406e480 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 >> v1:172.21.15.84:6791/0 conn(0x7fa2e40a41c0 legacy=0x7fa2e4149800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 >> v1:172.21.15.84:6790/0 conn(0x7fa2e406e850 legacy=0x7fa2e4130420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 monclient: found mon.f 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e4142300 con 0x7fa2e406e480 2024-09-18T14:14:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient: _finish_auth 0 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2ec19d640 5 monclient: authenticate success, global_id 214111 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2ec19d640 10 monclient: _renew_subs 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2ec19d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2ec19d640 1 -- 172.21.15.45:0/4292157466 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e4120570 con 0x7fa2e406e480 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa2e0001d40 con 0x7fa2e406e480 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2e0004d90 con 0x7fa2e406e480 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2d9ffb640 10 monclient: dump: 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2dbfff640 4 set_mon_vals no callback set 2024-09-18T14:14:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2ec19d640 10 monclient: _renew_subs 2024-09-18T14:14:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2ec19d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:14:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.960+0000 7fa2ec19d640 1 -- 172.21.15.45:0/4292157466 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e4130b30 con 0x7fa2e406e480 2024-09-18T14:14:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.961+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fa2e000fde0 con 0x7fa2e406e480 2024-09-18T14:14:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.961+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(730..730 src has 529..730) v4 ==== 3943+0+0 (unknown 2288278748 0 0) 0x7fa2e0046b60 con 0x7fa2e406e480 2024-09-18T14:14:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.961+0000 7fa2ec19d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:14:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.961+0000 7fa2ec19d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:14:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.961+0000 7fa2ec19d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:14:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.961+0000 7fa2ec19d640 1 -- 172.21.15.45:0/4292157466 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e404c4a0 con 0x7fa2e406e480 2024-09-18T14:14:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.961+0000 7fa2e9738640 10 monclient: get_auth_request con 0x7fa2c803a950 auth_method 0 2024-09-18T14:14:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.966+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.1 v1:172.21.15.84:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fa2e00152c0 con 0x7fa2e406e480 2024-09-18T14:14:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.966+0000 7fa2d9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:14:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:45.966+0000 7fa2d9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:14:46.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.198+0000 7fa2ec19d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:14:46.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.198+0000 7fa2ec19d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:14:46.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.198+0000 7fa2ec19d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:14:46.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.198+0000 7fa2ec19d640 1 -- 172.21.15.45:0/4292157466 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa2e404b720 con 0x7fa2e406e480 2024-09-18T14:14:46.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.199+0000 7fa2d9ffb640 1 -- 172.21.15.45:0/4292157466 <== mon.1 v1:172.21.15.84:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2683 (unknown 254746657 0 1031934170) 0x7fa2e0014cc0 con 0x7fa2e406e480 2024-09-18T14:14:46.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.199+0000 7fa2d9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:14:46.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.199+0000 7fa2d9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:14:46.200 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":122,"quorum":[0,1,2,3,4,5,6,7],"quorum_names":["a","f","b","g","c","h","d","i"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:14:46.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.201+0000 7fa2ec19d640 1 -- 172.21.15.45:0/4292157466 >> v1:172.21.15.84:6812/31478 conn(0x7fa2c803a950 legacy=0x7fa2c803ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:46.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.201+0000 7fa2ec19d640 10 monclient: shutdown 2024-09-18T14:14:46.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.201+0000 7fa2ec19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:14:46.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.201+0000 7fa2ec19d640 1 -- 172.21.15.45:0/4292157466 >> v1:172.21.15.84:6789/0 conn(0x7fa2e406e480 legacy=0x7fa2e4141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:46.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.201+0000 7fa2ec19d640 1 -- 172.21.15.45:0/4292157466 shutdown_connections 2024-09-18T14:14:46.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.201+0000 7fa2ec19d640 1 -- 172.21.15.45:0/4292157466 >> 172.21.15.45:0/4292157466 conn(0x7fa2e40636e0 msgr2=0x7fa2e40a2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:14:46.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.201+0000 7fa2ec19d640 1 -- 172.21.15.45:0/4292157466 shutdown_connections 2024-09-18T14:14:46.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:46.201+0000 7fa2ec19d640 1 -- 172.21.15.45:0/4292157466 wait complete. 2024-09-18T14:14:46.610 INFO:tasks.rados.rados.0.smithi084.stdout:3338: done (2 left) 2024-09-18T14:14:46.610 INFO:tasks.rados.rados.0.smithi084.stdout:3339: done (1 left) 2024-09-18T14:14:46.610 INFO:tasks.rados.rados.0.smithi084.stdout:3340: done (0 left) 2024-09-18T14:14:46.610 INFO:tasks.rados.rados.0.smithi084.stdout:3341: delete oid 49 current snap is 366 2024-09-18T14:14:46.613 INFO:tasks.rados.rados.0.smithi084.stdout:3341: done (0 left) 2024-09-18T14:14:46.614 INFO:tasks.rados.rados.0.smithi084.stdout:3342: snap_remove snap 355 2024-09-18T14:14:47.613 INFO:tasks.rados.rados.0.smithi084.stdout:3342: done (0 left) 2024-09-18T14:14:47.613 INFO:tasks.rados.rados.0.smithi084.stdout:3343: delete oid 11 current snap is 366 2024-09-18T14:14:47.630 INFO:tasks.rados.rados.0.smithi084.stdout:3343: done (0 left) 2024-09-18T14:14:47.631 INFO:tasks.rados.rados.0.smithi084.stdout:3344: read oid 10 snap -1 2024-09-18T14:14:47.631 INFO:tasks.rados.rados.0.smithi084.stdout:3344: expect (ObjNum 1111 snap 355 seq_num 1111) 2024-09-18T14:14:47.631 INFO:tasks.rados.rados.0.smithi084.stdout:3345: rollback oid 49 current snap is 366 2024-09-18T14:14:47.631 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 49 to 329 2024-09-18T14:14:47.631 INFO:tasks.rados.rados.0.smithi084.stdout:3346: read oid 11 snap -1 2024-09-18T14:14:47.631 INFO:tasks.rados.rados.0.smithi084.stdout:3346: expect deleted 2024-09-18T14:14:47.631 INFO:tasks.rados.rados.0.smithi084.stdout:3347: snap_create 2024-09-18T14:14:47.634 INFO:tasks.rados.rados.0.smithi084.stdout:3345: finishing rollback tid 1 to smithi08434663-49 2024-09-18T14:14:47.634 INFO:tasks.rados.rados.0.smithi084.stdout:3345: finishing rollback tid 2 to smithi08434663-49 2024-09-18T14:14:47.634 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 1340 (ObjNum 1020 snap 326 seq_num 1020) dirty exists 2024-09-18T14:14:47.637 INFO:tasks.rados.rados.0.smithi084.stdout:3344: expect (ObjNum 1111 snap 355 seq_num 1111) 2024-09-18T14:14:48.426 INFO:tasks.rados.rados.0.smithi084.stdout:3344: done (3 left) 2024-09-18T14:14:48.426 INFO:tasks.rados.rados.0.smithi084.stdout:3345: done (2 left) 2024-09-18T14:14:48.426 INFO:tasks.rados.rados.0.smithi084.stdout:3346: done (1 left) 2024-09-18T14:14:48.426 INFO:tasks.rados.rados.0.smithi084.stdout:3347: done (0 left) 2024-09-18T14:14:48.426 INFO:tasks.rados.rados.0.smithi084.stdout:3348: snap_create 2024-09-18T14:14:49.213 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:14:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.320+0000 7f32a9aa8640 10 monclient: get_monmap_and_config 2024-09-18T14:14:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.320+0000 7f32a9aa8640 10 monclient: build_initial_monmap 2024-09-18T14:14:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.320+0000 7f32a9aa8640 10 monclient: monmap: 2024-09-18T14:14:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:14:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:14:49.321790+0000 2024-09-18T14:14:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:14:49.321790+0000 2024-09-18T14:14:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:14:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:49.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.321+0000 7f32a9aa8640 1 Processor -- start 2024-09-18T14:14:49.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.321+0000 7f32a9aa8640 1 -- start start 2024-09-18T14:14:49.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.321+0000 7f32a9aa8640 10 monclient: init 2024-09-18T14:14:49.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a9aa8640 10 monclient: _reopen_session rank -1 2024-09-18T14:14:49.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a9aa8640 10 monclient: _add_conns ranks=[8,0,4,7,5,6,2,1,3] 2024-09-18T14:14:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a9aa8640 10 monclient(hunting): picked mon.noname-e con 0x7f32a40a2520 addr v1:172.21.15.45:6793/0 2024-09-18T14:14:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a9aa8640 10 monclient(hunting): picked mon.noname-a con 0x7f32a4096a20 addr v1:172.21.15.45:6789/0 2024-09-18T14:14:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a9aa8640 10 monclient(hunting): picked mon.noname-c con 0x7f32a4132fc0 addr v1:172.21.15.45:6791/0 2024-09-18T14:14:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a9aa8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a405ed70 con 0x7f32a4096a20 2024-09-18T14:14:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a9aa8640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a4097040 con 0x7f32a4132fc0 2024-09-18T14:14:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a9aa8640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a40a28f0 con 0x7f32a40a2520 2024-09-18T14:14:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a9aa8640 10 monclient(hunting): _renew_subs 2024-09-18T14:14:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a9aa8640 10 monclient(hunting): authenticate will time out at 1704.660248s 2024-09-18T14:14:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a2ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f32a4096a20 0x7f32a4137750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60130/0 (socket says 172.21.15.45:60130) 2024-09-18T14:14:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a37fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f32a40a2520 0x7f32a41328b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:49438/0 (socket says 172.21.15.45:49438) 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a3fff640 1 --1- 172.21.15.45:0/1572238325 >> v1:172.21.15.45:6791/0 conn(0x7f32a4132fc0 0x7f32a413ae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36134/0 (socket says 172.21.15.45:36134) 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a2ffd640 1 -- 172.21.15.45:0/1572238325 learned_addr learned my addr 172.21.15.45:0/1572238325 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 1 -- 172.21.15.45:0/1572238325 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173580075 0 0) 0x7f32a405ed70 con 0x7f32a4096a20 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 10 monclient(hunting): my global_id is 6075 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 1 -- 172.21.15.45:0/1572238325 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f328c0034e0 con 0x7f32a4096a20 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 1 -- 172.21.15.45:0/1572238325 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917562198 0 0) 0x7f32a4097040 con 0x7f32a4132fc0 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 10 monclient(hunting): my global_id is 164830 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 1 -- 172.21.15.45:0/1572238325 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a405ed70 con 0x7f32a4132fc0 2024-09-18T14:14:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 1 -- 172.21.15.45:0/1572238325 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4253372180 0 0) 0x7f328c0034e0 con 0x7f32a4096a20 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 1 -- 172.21.15.45:0/1572238325 >> v1:172.21.15.45:6793/0 conn(0x7f32a40a2520 legacy=0x7f32a41328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.322+0000 7f32a27fc640 1 -- 172.21.15.45:0/1572238325 >> v1:172.21.15.45:6791/0 conn(0x7f32a4132fc0 legacy=0x7f32a413ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 10 monclient: _finish_hunting 0 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 1 monclient: found mon.noname-a 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 1 -- 172.21.15.45:0/1572238325 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a409f4b0 con 0x7f32a4096a20 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 10 monclient: _finish_auth 0 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 1 -- 172.21.15.45:0/1572238325 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3298001830 con 0x7f32a4096a20 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:14:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 10 monclient: dump: 2024-09-18T14:14:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:49.335 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a9aa8640 5 monclient: authenticate success, global_id 6075 2024-09-18T14:14:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a9aa8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:14:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 1 -- 172.21.15.45:0/1572238325 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3298001a10 con 0x7f32a4096a20 2024-09-18T14:14:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:14:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 1 -- 172.21.15.45:0/1572238325 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3298004ba0 con 0x7f32a4096a20 2024-09-18T14:14:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a9aa8640 10 monclient: get_monmap_and_config success 2024-09-18T14:14:49.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a9aa8640 4 set_mon_vals no callback set 2024-09-18T14:14:49.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:49.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:14:49.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a27fc640 10 monclient: dump: 2024-09-18T14:14:49.336 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:49.336 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:49.336 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:49.336 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:49.336 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:49.336 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:49.337 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:49.337 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:49.337 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:49.337 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:49.337 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:49.337 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:49.337 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:49.337 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:49.337 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:49.337 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:49.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a9aa8640 10 monclient: shutdown 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a9aa8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a9aa8640 1 -- 172.21.15.45:0/1572238325 >> v1:172.21.15.45:6789/0 conn(0x7f32a4096a20 legacy=0x7f32a4137750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a9aa8640 1 -- 172.21.15.45:0/1572238325 shutdown_connections 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a9aa8640 1 -- 172.21.15.45:0/1572238325 >> 172.21.15.45:0/1572238325 conn(0x7f32a4063130 msgr2=0x7f32a4063500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a9aa8640 1 -- 172.21.15.45:0/1572238325 shutdown_connections 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.323+0000 7f32a9aa8640 1 -- 172.21.15.45:0/1572238325 wait complete. 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.324+0000 7f32a9aa8640 10 monclient: build_initial_monmap 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.324+0000 7f32a9aa8640 10 monclient: monmap: 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:14:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.324+0000 7f32a9aa8640 1 Processor -- start 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.324+0000 7f32a9aa8640 1 -- start start 2024-09-18T14:14:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.324+0000 7f32a9aa8640 10 monclient: init 2024-09-18T14:14:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f32a9aa8640 10 monclient: _reopen_session rank -1 2024-09-18T14:14:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f32a9aa8640 10 monclient: _add_conns ranks=[6,0,7,3,1,2,4,5,8] 2024-09-18T14:14:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f32a9aa8640 10 monclient(hunting): picked mon.noname-d con 0x7f32a4096a20 addr v1:172.21.15.45:6792/0 2024-09-18T14:14:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f32a9aa8640 10 monclient(hunting): picked mon.noname-a con 0x7f32a40a2520 addr v1:172.21.15.45:6789/0 2024-09-18T14:14:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f32a9aa8640 10 monclient(hunting): picked mon.noname-i con 0x7f32a4132fc0 addr v1:172.21.15.84:6792/0 2024-09-18T14:14:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f32a9aa8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a409f8e0 con 0x7f32a40a2520 2024-09-18T14:14:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f32a9aa8640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a40a2270 con 0x7f32a4096a20 2024-09-18T14:14:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f32a9aa8640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a41205a0 con 0x7f32a4132fc0 2024-09-18T14:14:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f32a9aa8640 10 monclient(hunting): _renew_subs 2024-09-18T14:14:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f32a9aa8640 10 monclient(hunting): authenticate will time out at 1704.663517s 2024-09-18T14:14:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f32a2ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f32a40a2520 0x7f32a41467a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60140/0 (socket says 172.21.15.45:60140) 2024-09-18T14:14:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f32a2ffd640 1 -- 172.21.15.45:0/3426036219 learned_addr learned my addr 172.21.15.45:0/3426036219 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:14:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361976476 0 0) 0x7f32a40a2270 con 0x7f32a4096a20 2024-09-18T14:14:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f328b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f328b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f328b7fe640 10 monclient(hunting): my global_id is 184641 2024-09-18T14:14:49.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32900034e0 con 0x7f32a4096a20 2024-09-18T14:14:49.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815206093 0 0) 0x7f32a409f8e0 con 0x7f32a40a2520 2024-09-18T14:14:49.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f328b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:49.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f328b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:49.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f328b7fe640 10 monclient(hunting): my global_id is 6084 2024-09-18T14:14:49.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.325+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a40a2270 con 0x7f32a40a2520 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4219638246 0 0) 0x7f32900034e0 con 0x7f32a4096a20 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f32a409f8e0 con 0x7f32a4096a20 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f32940019f0 con 0x7f32a4096a20 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient(hunting): dump: 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 280424017 0 0) 0x7f32a40a2270 con 0x7f32a40a2520 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f32900034e0 con 0x7f32a40a2520 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3298001770 con 0x7f32a40a2520 2024-09-18T14:14:49.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient(hunting): dump: 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187818523 0 0) 0x7f32a41205a0 con 0x7f32a4132fc0 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient(hunting): my global_id is 214108 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a40a2270 con 0x7f32a4132fc0 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1854232260 0 0) 0x7f32a409f8e0 con 0x7f32a4096a20 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 >> v1:172.21.15.84:6792/0 conn(0x7f32a4132fc0 legacy=0x7f32a4149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 >> v1:172.21.15.45:6789/0 conn(0x7f32a40a2520 legacy=0x7f32a41467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 monclient: found mon.d 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a41214e0 con 0x7f32a4096a20 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient: _finish_auth 0 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f32a9aa8640 5 monclient: authenticate success, global_id 184641 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f32a9aa8640 10 monclient: _renew_subs 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f32a9aa8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f32a9aa8640 1 -- 172.21.15.45:0/3426036219 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a4142300 con 0x7f32a4096a20 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f32a9aa8640 10 monclient: _renew_subs 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f32a9aa8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f32a9aa8640 1 -- 172.21.15.45:0/3426036219 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a41200a0 con 0x7f32a4096a20 2024-09-18T14:14:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3294001cf0 con 0x7f32a4096a20 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3294004e90 con 0x7f32a4096a20 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f328b7fe640 10 monclient: dump: 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.326+0000 7f32a1ffb640 4 set_mon_vals no callback set 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.327+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f329400fe60 con 0x7f32a4096a20 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.327+0000 7f32a9aa8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.327+0000 7f32a9aa8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.327+0000 7f32a9aa8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:14:49.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.327+0000 7f32a9aa8640 1 -- 172.21.15.45:0/3426036219 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3284005010 con 0x7f32a4096a20 2024-09-18T14:14:49.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.327+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(733..733 src has 529..733) v4 ==== 3971+0+0 (unknown 1141189619 0 0) 0x7f3294046ac0 con 0x7f32a4096a20 2024-09-18T14:14:49.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.327+0000 7f32a2ffd640 10 monclient: get_auth_request con 0x7f329003a4c0 auth_method 0 2024-09-18T14:14:49.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.331+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f3294014e60 con 0x7f32a4096a20 2024-09-18T14:14:49.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.331+0000 7f328b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:14:49.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.331+0000 7f328b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:14:49.434 INFO:tasks.rados.rados.0.smithi084.stdout:3348: done (0 left) 2024-09-18T14:14:49.435 INFO:tasks.rados.rados.0.smithi084.stdout:3349: copy_from oid 36 from oid 41 current snap is 368 2024-09-18T14:14:49.435 INFO:tasks.rados.rados.0.smithi084.stdout:3350: read oid 38 snap 356 2024-09-18T14:14:49.435 INFO:tasks.rados.rados.0.smithi084.stdout:3350: expect (ObjNum 957 snap 304 seq_num 957) 2024-09-18T14:14:49.435 INFO:tasks.rados.rados.0.smithi084.stdout:3351: snap_remove snap 338 2024-09-18T14:14:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.563+0000 7f32a9aa8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:14:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.563+0000 7f32a9aa8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:14:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.563+0000 7f32a9aa8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:14:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.563+0000 7f32a9aa8640 1 -- 172.21.15.45:0/3426036219 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3284005840 con 0x7f32a4096a20 2024-09-18T14:14:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.563+0000 7f328b7fe640 1 -- 172.21.15.45:0/3426036219 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2683 (unknown 254746657 0 1114339555) 0x7f3294014c80 con 0x7f32a4096a20 2024-09-18T14:14:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.563+0000 7f328b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:14:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.563+0000 7f328b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:14:49.564 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":122,"quorum":[0,1,2,3,4,5,6,7],"quorum_names":["a","f","b","g","c","h","d","i"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:14:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.565+0000 7f32a9aa8640 1 -- 172.21.15.45:0/3426036219 >> v1:172.21.15.84:6812/31478 conn(0x7f329003a4c0 legacy=0x7f329003c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.565+0000 7f32a9aa8640 10 monclient: shutdown 2024-09-18T14:14:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.565+0000 7f32a9aa8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:14:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.565+0000 7f32a9aa8640 1 -- 172.21.15.45:0/3426036219 >> v1:172.21.15.45:6792/0 conn(0x7f32a4096a20 legacy=0x7f32a4131390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.565+0000 7f32a9aa8640 1 -- 172.21.15.45:0/3426036219 shutdown_connections 2024-09-18T14:14:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.565+0000 7f32a9aa8640 1 -- 172.21.15.45:0/3426036219 >> 172.21.15.45:0/3426036219 conn(0x7f32a4063130 msgr2=0x7f32a40a3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:14:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.565+0000 7f32a9aa8640 1 -- 172.21.15.45:0/3426036219 shutdown_connections 2024-09-18T14:14:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:49.565+0000 7f32a9aa8640 1 -- 172.21.15.45:0/3426036219 wait complete. 2024-09-18T14:14:50.444 INFO:tasks.rados.rados.0.smithi084.stdout:3351: done (2 left) 2024-09-18T14:14:50.444 INFO:tasks.rados.rados.0.smithi084.stdout:3352: snap_remove snap 332 2024-09-18T14:14:50.444 INFO:tasks.rados.rados.0.smithi084.stdout:3349: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:14:50.444 INFO:tasks.rados.rados.0.smithi084.stdout:3350: expect (ObjNum 957 snap 304 seq_num 957) 2024-09-18T14:14:50.574 INFO:tasks.rados.rados.0.smithi084.stdout:3349: finishing copy_from to smithi08434663-36 2024-09-18T14:14:50.574 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 1653 (ObjNum 1138 snap 364 seq_num 1138) dirty exists 2024-09-18T14:14:51.443 INFO:tasks.rados.rados.0.smithi084.stdout:3349: done (2 left) 2024-09-18T14:14:51.443 INFO:tasks.rados.rados.0.smithi084.stdout:3350: done (1 left) 2024-09-18T14:14:51.444 INFO:tasks.rados.rados.0.smithi084.stdout:3352: done (0 left) 2024-09-18T14:14:51.444 INFO:tasks.rados.rados.0.smithi084.stdout:3353: snap_remove snap 329 2024-09-18T14:14:52.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:14:52.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.684+0000 7f2f58280640 10 monclient: get_monmap_and_config 2024-09-18T14:14:52.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.684+0000 7f2f58280640 10 monclient: build_initial_monmap 2024-09-18T14:14:52.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.684+0000 7f2f58280640 10 monclient: monmap: 2024-09-18T14:14:52.685 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:14:52.685 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:52.685 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:14:52.685575+0000 2024-09-18T14:14:52.685 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:14:52.685575+0000 2024-09-18T14:14:52.685 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:14:52.685 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:52.686 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:14:52.686 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:14:52.686 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:14:52.686 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:14:52.686 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:14:52.686 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:14:52.686 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:14:52.686 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:14:52.686 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:14:52.686 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.685+0000 7f2f58280640 1 Processor -- start 2024-09-18T14:14:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.685+0000 7f2f58280640 1 -- start start 2024-09-18T14:14:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.685+0000 7f2f58280640 10 monclient: init 2024-09-18T14:14:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.685+0000 7f2f58280640 10 monclient: _reopen_session rank -1 2024-09-18T14:14:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.685+0000 7f2f58280640 10 monclient: _add_conns ranks=[1,8,7,2,6,5,4,3,0] 2024-09-18T14:14:52.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.685+0000 7f2f58280640 10 monclient(hunting): picked mon.noname-f con 0x7f2f500a05a0 addr v1:172.21.15.84:6789/0 2024-09-18T14:14:52.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.685+0000 7f2f58280640 10 monclient(hunting): picked mon.noname-e con 0x7f2f500981e0 addr v1:172.21.15.45:6793/0 2024-09-18T14:14:52.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.685+0000 7f2f58280640 10 monclient(hunting): picked mon.noname-i con 0x7f2f50096a20 addr v1:172.21.15.84:6792/0 2024-09-18T14:14:52.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.685+0000 7f2f58280640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5005ed70 con 0x7f2f500a05a0 2024-09-18T14:14:52.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.685+0000 7f2f58280640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f500a1a40 con 0x7f2f50096a20 2024-09-18T14:14:52.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.685+0000 7f2f58280640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f50097520 con 0x7f2f500981e0 2024-09-18T14:14:52.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f58280640 10 monclient(hunting): _renew_subs 2024-09-18T14:14:52.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f58280640 10 monclient(hunting): authenticate will time out at 1708.024010s 2024-09-18T14:14:52.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5581b640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f2f500981e0 0x7f2f50137930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:49450/0 (socket says 172.21.15.45:49450) 2024-09-18T14:14:52.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5581b640 1 -- 172.21.15.45:0/140938977 learned_addr learned my addr 172.21.15.45:0/140938977 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:14:52.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 1 -- 172.21.15.45:0/140938977 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271298153 0 0) 0x7f2f500a1a40 con 0x7f2f50096a20 2024-09-18T14:14:52.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:52.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:52.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 10 monclient(hunting): my global_id is 214117 2024-09-18T14:14:52.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 1 -- 172.21.15.45:0/140938977 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f3c0034e0 con 0x7f2f50096a20 2024-09-18T14:14:52.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 1 -- 172.21.15.45:0/140938977 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720517904 0 0) 0x7f2f5005ed70 con 0x7f2f500a05a0 2024-09-18T14:14:52.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:52.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:52.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 10 monclient(hunting): my global_id is 214120 2024-09-18T14:14:52.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 1 -- 172.21.15.45:0/140938977 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f500a1a40 con 0x7f2f500a05a0 2024-09-18T14:14:52.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 1 -- 172.21.15.45:0/140938977 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1883184897 0 0) 0x7f2f3c0034e0 con 0x7f2f50096a20 2024-09-18T14:14:52.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 1 -- 172.21.15.45:0/140938977 >> v1:172.21.15.45:6793/0 conn(0x7f2f500981e0 legacy=0x7f2f50137930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 1 -- 172.21.15.45:0/140938977 >> v1:172.21.15.84:6789/0 conn(0x7f2f500a05a0 legacy=0x7f2f500a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 10 monclient: _finish_hunting 0 2024-09-18T14:14:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 1 monclient: found mon.noname-i 2024-09-18T14:14:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:14:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T14:14:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 1 -- 172.21.15.45:0/140938977 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f50097240 con 0x7f2f50096a20 2024-09-18T14:14:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 10 monclient: _finish_auth 0 2024-09-18T14:14:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:14:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 1 -- 172.21.15.45:0/140938977 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f48001bb0 con 0x7f2f50096a20 2024-09-18T14:14:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f58280640 5 monclient: authenticate success, global_id 214117 2024-09-18T14:14:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f58280640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:14:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:14:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f5501a640 10 monclient: dump: 2024-09-18T14:14:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:52.699 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.686+0000 7f2f58280640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:14:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f5501a640 1 -- 172.21.15.45:0/140938977 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f48001d90 con 0x7f2f50096a20 2024-09-18T14:14:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f5501a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:14:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f5501a640 1 -- 172.21.15.45:0/140938977 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f480025c0 con 0x7f2f50096a20 2024-09-18T14:14:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f58280640 10 monclient: get_monmap_and_config success 2024-09-18T14:14:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f58280640 4 set_mon_vals no callback set 2024-09-18T14:14:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f5501a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f5501a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:14:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f5501a640 10 monclient: dump: 2024-09-18T14:14:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:52.701 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f58280640 10 monclient: shutdown 2024-09-18T14:14:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f58280640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:14:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f58280640 1 -- 172.21.15.45:0/140938977 >> v1:172.21.15.84:6792/0 conn(0x7f2f50096a20 legacy=0x7f2f5013b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f58280640 1 -- 172.21.15.45:0/140938977 shutdown_connections 2024-09-18T14:14:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f58280640 1 -- 172.21.15.45:0/140938977 >> 172.21.15.45:0/140938977 conn(0x7f2f50063110 msgr2=0x7f2f50063510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:14:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f58280640 1 -- 172.21.15.45:0/140938977 shutdown_connections 2024-09-18T14:14:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.687+0000 7f2f58280640 1 -- 172.21.15.45:0/140938977 wait complete. 2024-09-18T14:14:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 10 monclient: build_initial_monmap 2024-09-18T14:14:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 10 monclient: monmap: 2024-09-18T14:14:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:14:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:14:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:14:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:14:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:14:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:14:52.703 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:14:52.703 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:14:52.703 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:14:52.703 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:14:52.703 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:14:52.703 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:14:52.703 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:14:52.703 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:14:52.703 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:14:52.703 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:52.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 1 Processor -- start 2024-09-18T14:14:52.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 1 -- start start 2024-09-18T14:14:52.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 10 monclient: init 2024-09-18T14:14:52.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 10 monclient: _reopen_session rank -1 2024-09-18T14:14:52.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 10 monclient: _add_conns ranks=[3,1,8,2,6,5,0,4,7] 2024-09-18T14:14:52.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 10 monclient(hunting): picked mon.noname-g con 0x7f2f50096a20 addr v1:172.21.15.84:6790/0 2024-09-18T14:14:52.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 10 monclient(hunting): picked mon.noname-f con 0x7f2f500981e0 addr v1:172.21.15.84:6789/0 2024-09-18T14:14:52.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 10 monclient(hunting): picked mon.noname-e con 0x7f2f500a05a0 addr v1:172.21.15.45:6793/0 2024-09-18T14:14:52.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f500a2160 con 0x7f2f500981e0 2024-09-18T14:14:52.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5011ddb0 con 0x7f2f50096a20 2024-09-18T14:14:52.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5009f8a0 con 0x7f2f500a05a0 2024-09-18T14:14:52.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.688+0000 7f2f58280640 10 monclient(hunting): _renew_subs 2024-09-18T14:14:52.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f58280640 10 monclient(hunting): authenticate will time out at 1708.027033s 2024-09-18T14:14:52.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f5681d640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f2f500a05a0 0x7f2f5014a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:49452/0 (socket says 172.21.15.45:49452) 2024-09-18T14:14:52.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f5681d640 1 -- 172.21.15.45:0/698953275 learned_addr learned my addr 172.21.15.45:0/698953275 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:14:52.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530440046 0 0) 0x7f2f500a2160 con 0x7f2f500981e0 2024-09-18T14:14:52.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:52.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:52.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): my global_id is 214129 2024-09-18T14:14:52.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f340034e0 con 0x7f2f500981e0 2024-09-18T14:14:52.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138761927 0 0) 0x7f2f5011ddb0 con 0x7f2f50096a20 2024-09-18T14:14:52.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): my global_id is 184548 2024-09-18T14:14:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f500a2160 con 0x7f2f50096a20 2024-09-18T14:14:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4082706439 0 0) 0x7f2f340034e0 con 0x7f2f500981e0 2024-09-18T14:14:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f5011ddb0 con 0x7f2f500981e0 2024-09-18T14:14:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f400019d0 con 0x7f2f500981e0 2024-09-18T14:14:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:14:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): dump: 2024-09-18T14:14:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:52.708 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:14:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3554062911 0 0) 0x7f2f500a2160 con 0x7f2f50096a20 2024-09-18T14:14:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f340034e0 con 0x7f2f50096a20 2024-09-18T14:14:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f4c001980 con 0x7f2f50096a20 2024-09-18T14:14:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:14:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient(hunting): dump: 2024-09-18T14:14:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:52.709 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:52.709 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:52.709 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:52.709 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:52.709 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:52.709 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:52.709 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:52.709 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:52.709 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:52.709 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:52.710 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 592183668 0 0) 0x7f2f5011ddb0 con 0x7f2f500981e0 2024-09-18T14:14:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 >> v1:172.21.15.45:6793/0 conn(0x7f2f500a05a0 legacy=0x7f2f5014a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 >> v1:172.21.15.84:6790/0 conn(0x7f2f50096a20 legacy=0x7f2f50132280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient: _finish_hunting 0 2024-09-18T14:14:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 monclient: found mon.f 2024-09-18T14:14:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:14:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:14:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5011f0b0 con 0x7f2f500981e0 2024-09-18T14:14:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 10 monclient: _finish_auth 0 2024-09-18T14:14:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:14:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f58280640 5 monclient: authenticate success, global_id 214129 2024-09-18T14:14:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f58280640 10 monclient: _renew_subs 2024-09-18T14:14:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f58280640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:14:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.689+0000 7f2f58280640 1 -- 172.21.15.45:0/698953275 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5011d870 con 0x7f2f500981e0 2024-09-18T14:14:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f58280640 10 monclient: _renew_subs 2024-09-18T14:14:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f58280640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:14:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f58280640 1 -- 172.21.15.45:0/698953275 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f50132990 con 0x7f2f500981e0 2024-09-18T14:14:52.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f400048a0 con 0x7f2f500981e0 2024-09-18T14:14:52.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f467fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:14:52.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f40004d50 con 0x7f2f500981e0 2024-09-18T14:14:52.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:52.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f467fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:14:52.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f54819640 4 set_mon_vals no callback set 2024-09-18T14:14:52.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f467fc640 10 monclient: dump: 2024-09-18T14:14:52.712 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:52.712 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:52.712 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:52.712 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:52.712 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:52.713 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:52.713 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:52.713 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:52.713 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:52.713 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:52.713 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:52.713 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:52.713 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:52.713 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:52.713 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:52.713 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:52.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f58280640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:14:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f58280640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:14:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f58280640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:14:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2f4000fd20 con 0x7f2f500981e0 2024-09-18T14:14:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.690+0000 7f2f58280640 1 -- 172.21.15.45:0/698953275 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f38005010 con 0x7f2f500981e0 2024-09-18T14:14:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.691+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(736..736 src has 529..736) v4 ==== 3955+0+0 (unknown 4091660173 0 0) 0x7f2f40046ba0 con 0x7f2f500981e0 2024-09-18T14:14:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.691+0000 7f2f5601c640 10 monclient: get_auth_request con 0x7f2f3403a410 auth_method 0 2024-09-18T14:14:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.695+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.1 v1:172.21.15.84:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f2f40011310 con 0x7f2f500981e0 2024-09-18T14:14:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.695+0000 7f2f467fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:14:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.695+0000 7f2f467fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:14:52.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.922+0000 7f2f58280640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:14:52.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.922+0000 7f2f58280640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:14:52.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.922+0000 7f2f58280640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:14:52.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.922+0000 7f2f58280640 1 -- 172.21.15.45:0/698953275 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2f38005840 con 0x7f2f500981e0 2024-09-18T14:14:52.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.923+0000 7f2f467fc640 1 -- 172.21.15.45:0/698953275 <== mon.1 v1:172.21.15.84:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2683 (unknown 254746657 0 2716381237) 0x7f2f40014c90 con 0x7f2f500981e0 2024-09-18T14:14:52.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.923+0000 7f2f467fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:14:52.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.923+0000 7f2f467fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:14:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":122,"quorum":[0,1,2,3,4,5,6,7],"quorum_names":["a","f","b","g","c","h","d","i"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:14:52.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.925+0000 7f2f58280640 1 -- 172.21.15.45:0/698953275 >> v1:172.21.15.84:6812/31478 conn(0x7f2f3403a410 legacy=0x7f2f3403c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:52.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.925+0000 7f2f58280640 10 monclient: shutdown 2024-09-18T14:14:52.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.925+0000 7f2f58280640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:14:52.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.925+0000 7f2f58280640 1 -- 172.21.15.45:0/698953275 >> v1:172.21.15.84:6789/0 conn(0x7f2f500981e0 legacy=0x7f2f50146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:52.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.925+0000 7f2f58280640 1 -- 172.21.15.45:0/698953275 shutdown_connections 2024-09-18T14:14:52.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.925+0000 7f2f58280640 1 -- 172.21.15.45:0/698953275 >> 172.21.15.45:0/698953275 conn(0x7f2f50063110 msgr2=0x7f2f500a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:14:52.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.925+0000 7f2f58280640 1 -- 172.21.15.45:0/698953275 shutdown_connections 2024-09-18T14:14:52.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:52.925+0000 7f2f58280640 1 -- 172.21.15.45:0/698953275 wait complete. 2024-09-18T14:14:55.938 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:14:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.047+0000 7f261819d640 10 monclient: get_monmap_and_config 2024-09-18T14:14:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.047+0000 7f261819d640 10 monclient: build_initial_monmap 2024-09-18T14:14:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.047+0000 7f261819d640 10 monclient: monmap: 2024-09-18T14:14:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:14:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:14:56.048370+0000 2024-09-18T14:14:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:14:56.048370+0000 2024-09-18T14:14:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:14:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:14:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:14:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:14:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:14:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:14:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:14:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:14:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:14:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:14:56.049 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 1 Processor -- start 2024-09-18T14:14:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 1 -- start start 2024-09-18T14:14:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 10 monclient: init 2024-09-18T14:14:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 10 monclient: _reopen_session rank -1 2024-09-18T14:14:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 10 monclient: _add_conns ranks=[3,4,5,0,1,2,6,8,7] 2024-09-18T14:14:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 10 monclient(hunting): picked mon.noname-g con 0x7f261006e810 addr v1:172.21.15.84:6790/0 2024-09-18T14:14:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 10 monclient(hunting): picked mon.noname-c con 0x7f26100a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:14:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 10 monclient(hunting): picked mon.noname-h con 0x7f26100a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:14:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261005ed70 con 0x7f261006e810 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2610097620 con 0x7f26100a0cb0 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2610097340 con 0x7f26100a1080 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 10 monclient(hunting): _renew_subs 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f261819d640 10 monclient(hunting): authenticate will time out at 1711.386816s 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f2615738640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f26100a0cb0 0x7f2610137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36144/0 (socket says 172.21.15.45:36144) 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.048+0000 7f2615738640 1 -- 172.21.15.45:0/1095195053 learned_addr learned my addr 172.21.15.45:0/1095195053 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 1 -- 172.21.15.45:0/1095195053 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620812860 0 0) 0x7f2610097620 con 0x7f26100a0cb0 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient(hunting): my global_id is 164839 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 1 -- 172.21.15.45:0/1095195053 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25fc0034e0 con 0x7f26100a0cb0 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 1 -- 172.21.15.45:0/1095195053 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1283756365 0 0) 0x7f25fc0034e0 con 0x7f26100a0cb0 2024-09-18T14:14:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 1 -- 172.21.15.45:0/1095195053 >> v1:172.21.15.84:6791/0 conn(0x7f26100a1080 legacy=0x7f261013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 1 -- 172.21.15.45:0/1095195053 >> v1:172.21.15.84:6790/0 conn(0x7f261006e810 legacy=0x7f26100a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient: _finish_hunting 0 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 1 monclient: found mon.noname-c 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 1 -- 172.21.15.45:0/1095195053 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26100a1660 con 0x7f26100a0cb0 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient: _finish_auth 0 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 1 -- 172.21.15.45:0/1095195053 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2600001830 con 0x7f26100a0cb0 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f261819d640 5 monclient: authenticate success, global_id 164839 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f261819d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient: dump: 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:56.061 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f261819d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:14:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 1 -- 172.21.15.45:0/1095195053 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2600001990 con 0x7f26100a0cb0 2024-09-18T14:14:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:14:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f261819d640 10 monclient: get_monmap_and_config success 2024-09-18T14:14:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 1 -- 172.21.15.45:0/1095195053 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2600004860 con 0x7f26100a0cb0 2024-09-18T14:14:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f261819d640 4 set_mon_vals no callback set 2024-09-18T14:14:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:14:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.049+0000 7f2614f37640 10 monclient: dump: 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.050+0000 7f261819d640 10 monclient: shutdown 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.050+0000 7f261819d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.050+0000 7f261819d640 1 -- 172.21.15.45:0/1095195053 >> v1:172.21.15.45:6791/0 conn(0x7f26100a0cb0 legacy=0x7f2610137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.050+0000 7f261819d640 1 -- 172.21.15.45:0/1095195053 shutdown_connections 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.050+0000 7f261819d640 1 -- 172.21.15.45:0/1095195053 >> 172.21.15.45:0/1095195053 conn(0x7f26100a5bb0 msgr2=0x7f26100a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.050+0000 7f261819d640 1 -- 172.21.15.45:0/1095195053 shutdown_connections 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.050+0000 7f261819d640 1 -- 172.21.15.45:0/1095195053 wait complete. 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.050+0000 7f261819d640 10 monclient: build_initial_monmap 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.050+0000 7f261819d640 10 monclient: monmap: 2024-09-18T14:14:56.063 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:14:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 1 Processor -- start 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 1 -- start start 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 10 monclient: init 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 10 monclient: _reopen_session rank -1 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 10 monclient: _add_conns ranks=[3,1,7,0,5,8,2,6,4] 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 10 monclient(hunting): picked mon.noname-g con 0x7f261006e810 addr v1:172.21.15.84:6790/0 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 10 monclient(hunting): picked mon.noname-f con 0x7f26100a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 10 monclient(hunting): picked mon.noname-i con 0x7f26100a1080 addr v1:172.21.15.84:6792/0 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261009fa10 con 0x7f26100a0cb0 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26100a2270 con 0x7f261006e810 2024-09-18T14:14:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2610120790 con 0x7f26100a1080 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 10 monclient(hunting): _renew_subs 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261819d640 10 monclient(hunting): authenticate will time out at 1711.389698s 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261673a640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f26100a1080 0x7f261011dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:39160/0 (socket says 172.21.15.45:39160) 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.051+0000 7f261673a640 1 -- 172.21.15.45:0/3338030675 learned_addr learned my addr 172.21.15.45:0/3338030675 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83058356 0 0) 0x7f2610120790 con 0x7f26100a1080 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): my global_id is 214126 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25f40034e0 con 0x7f26100a1080 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801833382 0 0) 0x7f261009fa10 con 0x7f26100a0cb0 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): my global_id is 214138 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2610120790 con 0x7f26100a0cb0 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284488054 0 0) 0x7f26100a2270 con 0x7f261006e810 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): my global_id is 184566 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f261009fa10 con 0x7f261006e810 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 613084532 0 0) 0x7f25f40034e0 con 0x7f26100a1080 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f26100a2270 con 0x7f26100a1080 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 766037370 0 0) 0x7f261009fa10 con 0x7f261006e810 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f25f40034e0 con 0x7f261006e810 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3147510858 0 0) 0x7f2610120790 con 0x7f26100a0cb0 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f261009fa10 con 0x7f26100a0cb0 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2608001e00 con 0x7f26100a1080 2024-09-18T14:14:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:14:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): dump: 2024-09-18T14:14:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:56.069 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:56.069 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:56.069 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:56.069 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:56.069 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:56.069 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:56.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:14:56.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f260c001070 con 0x7f261006e810 2024-09-18T14:14:56.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): dump: 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f26000016f0 con 0x7f26100a0cb0 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient(hunting): dump: 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3392039388 0 0) 0x7f26100a2270 con 0x7f26100a1080 2024-09-18T14:14:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 >> v1:172.21.15.84:6790/0 conn(0x7f261006e810 legacy=0x7f261011d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 >> v1:172.21.15.84:6789/0 conn(0x7f26100a0cb0 legacy=0x7f261011d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 monclient: found mon.i 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261011ec60 con 0x7f26100a1080 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 10 monclient: _finish_auth 0 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f2605ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f261819d640 5 monclient: authenticate success, global_id 214126 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f261819d640 10 monclient: _renew_subs 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f261819d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.052+0000 7f261819d640 1 -- 172.21.15.45:0/3338030675 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261011f830 con 0x7f26100a1080 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.053+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2608002100 con 0x7f26100a1080 2024-09-18T14:14:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.053+0000 7f2605ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.053+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f26080028f0 con 0x7f26100a1080 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.053+0000 7f2605ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.053+0000 7f2605ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.053+0000 7f2605ffb640 10 monclient: dump: 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:14:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.053+0000 7f261819d640 10 monclient: _renew_subs 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.053+0000 7f261819d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.053+0000 7f2607fff640 4 set_mon_vals no callback set 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.053+0000 7f261819d640 1 -- 172.21.15.45:0/3338030675 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f261014a9a0 con 0x7f26100a1080 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.053+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2608002ad0 con 0x7f26100a1080 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.054+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(736..736 src has 529..736) v4 ==== 3955+0+0 (unknown 4091660173 0 0) 0x7f2608046e50 con 0x7f26100a1080 2024-09-18T14:14:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.054+0000 7f2615f39640 10 monclient: get_auth_request con 0x7f25f403aa00 auth_method 0 2024-09-18T14:14:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.054+0000 7f261819d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:14:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.054+0000 7f261819d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:14:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.054+0000 7f261819d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:14:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.054+0000 7f261819d640 1 -- 172.21.15.45:0/3338030675 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261004c4a0 con 0x7f26100a1080 2024-09-18T14:14:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.059+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.7 v1:172.21.15.84:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f2608015240 con 0x7f26100a1080 2024-09-18T14:14:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.059+0000 7f2605ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:14:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.059+0000 7f2605ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:14:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.289+0000 7f261819d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:14:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.289+0000 7f261819d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:14:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.289+0000 7f261819d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:14:56.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.289+0000 7f261819d640 1 -- 172.21.15.45:0/3338030675 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f261004b720 con 0x7f26100a1080 2024-09-18T14:14:56.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.290+0000 7f2605ffb640 1 -- 172.21.15.45:0/3338030675 <== mon.7 v1:172.21.15.84:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2684 (unknown 254746657 0 1093824563) 0x7f2608015060 con 0x7f26100a1080 2024-09-18T14:14:56.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.290+0000 7f2605ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:14:56.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.290+0000 7f2605ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:14:56.290 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":122,"quorum":[0,1,2,3,4,5,6,7],"quorum_names":["a","f","b","g","c","h","d","i"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:14:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.291+0000 7f261819d640 1 -- 172.21.15.45:0/3338030675 >> v1:172.21.15.84:6812/31478 conn(0x7f25f403aa00 legacy=0x7f25f403ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.291+0000 7f261819d640 10 monclient: shutdown 2024-09-18T14:14:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.291+0000 7f261819d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:14:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.291+0000 7f261819d640 1 -- 172.21.15.45:0/3338030675 >> v1:172.21.15.84:6792/0 conn(0x7f26100a1080 legacy=0x7f261011dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:14:56.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.292+0000 7f261819d640 1 -- 172.21.15.45:0/3338030675 shutdown_connections 2024-09-18T14:14:56.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.292+0000 7f261819d640 1 -- 172.21.15.45:0/3338030675 >> 172.21.15.45:0/3338030675 conn(0x7f26100a5bb0 msgr2=0x7f2610132d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:14:56.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.292+0000 7f261819d640 1 -- 172.21.15.45:0/3338030675 shutdown_connections 2024-09-18T14:14:56.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:56.292+0000 7f261819d640 1 -- 172.21.15.45:0/3338030675 wait complete. 2024-09-18T14:14:59.303 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:14:59.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.411+0000 7f0aea75d640 10 monclient: get_monmap_and_config 2024-09-18T14:14:59.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.411+0000 7f0aea75d640 10 monclient: build_initial_monmap 2024-09-18T14:14:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.411+0000 7f0aea75d640 10 monclient: monmap: 2024-09-18T14:14:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:14:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:14:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:14:59.412137+0000 2024-09-18T14:14:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:14:59.412137+0000 2024-09-18T14:14:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:14:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:14:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:14:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:14:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:14:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:14:59.413 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:14:59.413 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:14:59.413 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:14:59.413 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:14:59.413 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:14:59.413 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:14:59.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.411+0000 7f0aea75d640 1 Processor -- start 2024-09-18T14:14:59.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0aea75d640 1 -- start start 2024-09-18T14:14:59.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0aea75d640 10 monclient: init 2024-09-18T14:14:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0aea75d640 10 monclient: _reopen_session rank -1 2024-09-18T14:14:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0aea75d640 10 monclient: _add_conns ranks=[3,6,4,0,5,7,8,1,2] 2024-09-18T14:14:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0aea75d640 10 monclient(hunting): picked mon.noname-g con 0x7f0ae409f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:14:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0aea75d640 10 monclient(hunting): picked mon.noname-d con 0x7f0ae409fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:14:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0aea75d640 10 monclient(hunting): picked mon.noname-c con 0x7f0ae409ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:14:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0aea75d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae405ed70 con 0x7f0ae409f7d0 2024-09-18T14:14:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0aea75d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae409dfc0 con 0x7f0ae409ff70 2024-09-18T14:14:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0aea75d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae409dce0 con 0x7f0ae409fba0 2024-09-18T14:14:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0aea75d640 10 monclient(hunting): _renew_subs 2024-09-18T14:14:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0aea75d640 10 monclient(hunting): authenticate will time out at 1714.750559s 2024-09-18T14:14:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0ae8cfa640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f0ae409ff70 0x7f0ae413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39544/0 (socket says 172.21.15.45:39544) 2024-09-18T14:14:59.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:14:59.412+0000 7f0ae8cfa640 1 -- 172.21.15.45:0/4186225023 learned_addr learned my addr 172.21.15.45:0/4186225023 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213814841 0 0) 0x7f0ae405ed70 con 0x7f0ae409f7d0 2024-09-18T14:15:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient(hunting): my global_id is 184575 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0acc0034e0 con 0x7f0ae409f7d0 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569093551 0 0) 0x7f0ae409dce0 con 0x7f0ae409fba0 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient(hunting): my global_id is 184650 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae405ed70 con 0x7f0ae409fba0 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114085332 0 0) 0x7f0ae409dfc0 con 0x7f0ae409ff70 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient(hunting): my global_id is 164848 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae409dce0 con 0x7f0ae409ff70 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3840898314 0 0) 0x7f0acc0034e0 con 0x7f0ae409f7d0 2024-09-18T14:15:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 >> v1:172.21.15.45:6792/0 conn(0x7f0ae409fba0 legacy=0x7f0ae4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 >> v1:172.21.15.45:6791/0 conn(0x7f0ae409ff70 legacy=0x7f0ae413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 monclient: found mon.noname-g 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae40a1c00 con 0x7f0ae409f7d0 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient: _finish_auth 0 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ad4001830 con 0x7f0ae409f7d0 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient: dump: 2024-09-18T14:15:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0aea75d640 5 monclient: authenticate success, global_id 184575 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0aea75d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0ad4001a10 con 0x7f0ae409f7d0 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 1 -- 172.21.15.45:0/4186225023 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ad4002240 con 0x7f0ae409f7d0 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0aea75d640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0aea75d640 4 set_mon_vals no callback set 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.914+0000 7f0ae2ffd640 10 monclient: dump: 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.915+0000 7f0aea75d640 10 monclient: shutdown 2024-09-18T14:15:01.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.915+0000 7f0aea75d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:01.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.915+0000 7f0aea75d640 1 -- 172.21.15.45:0/4186225023 >> v1:172.21.15.84:6790/0 conn(0x7f0ae409f7d0 legacy=0x7f0ae4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:01.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.916+0000 7f0aea75d640 1 -- 172.21.15.45:0/4186225023 shutdown_connections 2024-09-18T14:15:01.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.916+0000 7f0aea75d640 1 -- 172.21.15.45:0/4186225023 >> 172.21.15.45:0/4186225023 conn(0x7f0ae40630b0 msgr2=0x7f0ae40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:01.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.916+0000 7f0aea75d640 1 -- 172.21.15.45:0/4186225023 shutdown_connections 2024-09-18T14:15:01.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.916+0000 7f0aea75d640 1 -- 172.21.15.45:0/4186225023 wait complete. 2024-09-18T14:15:01.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.916+0000 7f0aea75d640 10 monclient: build_initial_monmap 2024-09-18T14:15:01.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.916+0000 7f0aea75d640 10 monclient: monmap: 2024-09-18T14:15:01.924 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:01.924 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:01.924 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.917+0000 7f0aea75d640 1 Processor -- start 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.918+0000 7f0aea75d640 1 -- start start 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.918+0000 7f0aea75d640 10 monclient: init 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.918+0000 7f0aea75d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.918+0000 7f0aea75d640 10 monclient: _add_conns ranks=[1,7,3,5,4,8,6,0,2] 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.918+0000 7f0aea75d640 10 monclient(hunting): picked mon.noname-f con 0x7f0ae409f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.918+0000 7f0aea75d640 10 monclient(hunting): picked mon.noname-i con 0x7f0ae409fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.918+0000 7f0aea75d640 10 monclient(hunting): picked mon.noname-g con 0x7f0ae409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.918+0000 7f0aea75d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae409d310 con 0x7f0ae409f7d0 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.918+0000 7f0aea75d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae40a2560 con 0x7f0ae409ff70 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.918+0000 7f0aea75d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae409b9a0 con 0x7f0ae409fba0 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.918+0000 7f0aea75d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.918+0000 7f0aea75d640 10 monclient(hunting): authenticate will time out at 1717.256997s 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.919+0000 7f0ae8cfa640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7f0ae409ff70 0x7f0ae414a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:36016/0 (socket says 172.21.15.45:36016) 2024-09-18T14:15:01.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.919+0000 7f0ae37fe640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f0ae409fba0 0x7f0ae4146cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:34716/0 (socket says 172.21.15.45:34716) 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.919+0000 7f0ae8cfa640 1 -- 172.21.15.45:0/38674931 learned_addr learned my addr 172.21.15.45:0/38674931 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.919+0000 7f0ae3fff640 1 --1- 172.21.15.45:0/38674931 >> v1:172.21.15.84:6789/0 conn(0x7f0ae409f7d0 0x7f0ae4131700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:35212/0 (socket says 172.21.15.45:35212) 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423479154 0 0) 0x7f0ae40a2560 con 0x7f0ae409ff70 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 10 monclient(hunting): my global_id is 184584 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ad00034e0 con 0x7f0ae409ff70 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421128166 0 0) 0x7f0ae409b9a0 con 0x7f0ae409fba0 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 10 monclient(hunting): my global_id is 214135 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae40a2560 con 0x7f0ae409fba0 2024-09-18T14:15:01.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756918017 0 0) 0x7f0ae409d310 con 0x7f0ae409f7d0 2024-09-18T14:15:01.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:01.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:01.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 10 monclient(hunting): my global_id is 214147 2024-09-18T14:15:01.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.923+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae409b9a0 con 0x7f0ae409f7d0 2024-09-18T14:15:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1741612277 0 0) 0x7f0ad00034e0 con 0x7f0ae409ff70 2024-09-18T14:15:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0ae409d310 con 0x7f0ae409ff70 2024-09-18T14:15:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0adc001cd0 con 0x7f0ae409ff70 2024-09-18T14:15:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:15:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 10 monclient(hunting): dump: 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:01.931 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1603151227 0 0) 0x7f0ae409b9a0 con 0x7f0ae409f7d0 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0ad00034e0 con 0x7f0ae409f7d0 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ad4001ec0 con 0x7f0ae409f7d0 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 10 monclient(hunting): dump: 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2960240055 0 0) 0x7f0ae409d310 con 0x7f0ae409ff70 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 >> v1:172.21.15.84:6792/0 conn(0x7f0ae409fba0 legacy=0x7f0ae4146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 >> v1:172.21.15.84:6789/0 conn(0x7f0ae409f7d0 legacy=0x7f0ae4131700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 10 monclient: _finish_hunting 0 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 1 monclient: found mon.g 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae411e820 con 0x7f0ae409ff70 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 10 monclient: _finish_auth 0 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0acbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0aea75d640 5 monclient: authenticate success, global_id 184584 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0aea75d640 10 monclient: _renew_subs 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0aea75d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0aea75d640 1 -- 172.21.15.45:0/38674931 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae411d8c0 con 0x7f0ae409ff70 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0aea75d640 10 monclient: _renew_subs 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0aea75d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:15:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.927+0000 7f0aea75d640 1 -- 172.21.15.45:0/38674931 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae4131fb0 con 0x7f0ae409ff70 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.928+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0adc004ba0 con 0x7f0ae409ff70 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.928+0000 7f0acbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.928+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0adc005150 con 0x7f0ae409ff70 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.928+0000 7f0acbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.928+0000 7f0acbfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.928+0000 7f0ae27fc640 4 set_mon_vals no callback set 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.928+0000 7f0acbfff640 10 monclient: dump: 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.928+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f0adc005330 con 0x7f0ae409ff70 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.929+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(736..736 src has 529..736) v4 ==== 3955+0+0 (unknown 4091660173 0 0) 0x7f0adc046f10 con 0x7f0ae409ff70 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.929+0000 7f0aea75d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.929+0000 7f0ae3fff640 10 monclient: get_auth_request con 0x7f0ad003a5c0 auth_method 0 2024-09-18T14:15:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.929+0000 7f0aea75d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.929+0000 7f0aea75d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:15:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.929+0000 7f0aea75d640 1 -- 172.21.15.45:0/38674931 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac4005010 con 0x7f0ae409ff70 2024-09-18T14:15:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.934+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.3 v1:172.21.15.84:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f0adc015240 con 0x7f0ae409ff70 2024-09-18T14:15:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.934+0000 7f0acbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:01.934+0000 7f0acbfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:01.985 INFO:tasks.rados.rados.0.smithi084.stdout:3353: done (0 left) 2024-09-18T14:15:01.985 INFO:tasks.rados.rados.0.smithi084.stdout:3354: write oid 13 current snap is 368 2024-09-18T14:15:01.985 INFO:tasks.rados.rados.0.smithi084.stdout:3354: seq_num 1142 ranges {793594=587308,2060554=510055} 2024-09-18T14:15:01.993 INFO:tasks.rados.rados.0.smithi084.stdout:3354: writing smithi08434663-13 from 793594 to 1380902 tid 1 2024-09-18T14:15:02.000 INFO:tasks.rados.rados.0.smithi084.stdout:3354: writing smithi08434663-13 from 2060554 to 2570609 tid 2 2024-09-18T14:15:02.000 INFO:tasks.rados.rados.0.smithi084.stdout:3355: rollback oid 50 current snap is 368 2024-09-18T14:15:02.000 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 50 to 354 2024-09-18T14:15:02.000 INFO:tasks.rados.rados.0.smithi084.stdout:3356: copy_from oid 27 from oid 43 current snap is 368 2024-09-18T14:15:02.000 INFO:tasks.rados.rados.0.smithi084.stdout:3357: snap_remove snap 351 2024-09-18T14:15:02.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.184+0000 7f0aea75d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:15:02.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.184+0000 7f0aea75d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:15:02.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.184+0000 7f0aea75d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:15:02.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.184+0000 7f0aea75d640 1 -- 172.21.15.45:0/38674931 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0ac4005840 con 0x7f0ae409ff70 2024-09-18T14:15:02.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.185+0000 7f0acbfff640 1 -- 172.21.15.45:0/38674931 <== mon.3 v1:172.21.15.84:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2689 (unknown 254746657 0 984439998) 0x7f0adc015060 con 0x7f0ae409ff70 2024-09-18T14:15:02.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.185+0000 7f0acbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:15:02.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.185+0000 7f0acbfff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:02.186 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":130,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:15:02.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.187+0000 7f0aea75d640 1 -- 172.21.15.45:0/38674931 >> v1:172.21.15.84:6812/31478 conn(0x7f0ad003a5c0 legacy=0x7f0ad003ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.187+0000 7f0aea75d640 10 monclient: shutdown 2024-09-18T14:15:02.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.187+0000 7f0aea75d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:02.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.187+0000 7f0aea75d640 1 -- 172.21.15.45:0/38674931 >> v1:172.21.15.84:6790/0 conn(0x7f0ae409ff70 legacy=0x7f0ae414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.187+0000 7f0aea75d640 1 -- 172.21.15.45:0/38674931 shutdown_connections 2024-09-18T14:15:02.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.187+0000 7f0aea75d640 1 -- 172.21.15.45:0/38674931 >> 172.21.15.45:0/38674931 conn(0x7f0ae40630b0 msgr2=0x7f0ae40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:02.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.187+0000 7f0aea75d640 1 -- 172.21.15.45:0/38674931 shutdown_connections 2024-09-18T14:15:02.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.187+0000 7f0aea75d640 1 -- 172.21.15.45:0/38674931 wait complete. 2024-09-18T14:15:02.198 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:15:02.198 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:15:02.198 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:15:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.294+0000 7fe8fd060640 10 monclient: get_monmap_and_config 2024-09-18T14:15:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.294+0000 7fe8fd060640 10 monclient: build_initial_monmap 2024-09-18T14:15:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.294+0000 7fe8fd060640 10 monclient: monmap: 2024-09-18T14:15:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:02.295858+0000 2024-09-18T14:15:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:02.295858+0000 2024-09-18T14:15:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:02.297 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.295+0000 7fe8fd060640 1 Processor -- start 2024-09-18T14:15:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.295+0000 7fe8fd060640 1 -- start start 2024-09-18T14:15:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.295+0000 7fe8fd060640 10 monclient: init 2024-09-18T14:15:02.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.295+0000 7fe8fd060640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:02.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.295+0000 7fe8fd060640 10 monclient: _add_conns ranks=[0,1,4,2,8,5,7,3,6] 2024-09-18T14:15:02.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8fd060640 10 monclient(hunting): picked mon.noname-a con 0x7fe8f809f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:02.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8fd060640 10 monclient(hunting): picked mon.noname-f con 0x7fe8f809fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:02.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8fd060640 10 monclient(hunting): picked mon.noname-c con 0x7fe8f809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:02.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8fd060640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f805ed70 con 0x7fe8f809f7d0 2024-09-18T14:15:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8fd060640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f809dfc0 con 0x7fe8f809fba0 2024-09-18T14:15:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8fd060640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f809dce0 con 0x7fe8f809ff70 2024-09-18T14:15:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8fd060640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8fd060640 10 monclient(hunting): authenticate will time out at 1717.634090s 2024-09-18T14:15:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f759e640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fe8f809ff70 0x7fe8f813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39560/0 (socket says 172.21.15.45:39560) 2024-09-18T14:15:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f759e640 1 -- 172.21.15.45:0/478705084 learned_addr learned my addr 172.21.15.45:0/478705084 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f6d9d640 1 --1- 172.21.15.45:0/478705084 >> v1:172.21.15.45:6789/0 conn(0x7fe8f809f7d0 0x7fe8f8132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56256/0 (socket says 172.21.15.45:56256) 2024-09-18T14:15:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 1 -- 172.21.15.45:0/478705084 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146705801 0 0) 0x7fe8f805ed70 con 0x7fe8f809f7d0 2024-09-18T14:15:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient(hunting): my global_id is 6093 2024-09-18T14:15:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 1 -- 172.21.15.45:0/478705084 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e00034e0 con 0x7fe8f809f7d0 2024-09-18T14:15:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 1 -- 172.21.15.45:0/478705084 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432350799 0 0) 0x7fe8f809dce0 con 0x7fe8f809ff70 2024-09-18T14:15:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient(hunting): my global_id is 164857 2024-09-18T14:15:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 1 -- 172.21.15.45:0/478705084 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f805ed70 con 0x7fe8f809ff70 2024-09-18T14:15:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 1 -- 172.21.15.45:0/478705084 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4011658990 0 0) 0x7fe8f805ed70 con 0x7fe8f809ff70 2024-09-18T14:15:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 1 -- 172.21.15.45:0/478705084 >> v1:172.21.15.84:6789/0 conn(0x7fe8f809fba0 legacy=0x7fe8f8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 1 -- 172.21.15.45:0/478705084 >> v1:172.21.15.45:6789/0 conn(0x7fe8f809f7d0 legacy=0x7fe8f8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:15:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 1 monclient: found mon.noname-c 2024-09-18T14:15:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:15:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 1 -- 172.21.15.45:0/478705084 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f80a1c00 con 0x7fe8f809ff70 2024-09-18T14:15:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient: _finish_auth 0 2024-09-18T14:15:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 1 -- 172.21.15.45:0/478705084 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe8f0001b90 con 0x7fe8f809ff70 2024-09-18T14:15:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:15:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient: dump: 2024-09-18T14:15:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.318 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8fd060640 5 monclient: authenticate success, global_id 164857 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8fd060640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 1 -- 172.21.15.45:0/478705084 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe8f0001d70 con 0x7fe8f809ff70 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 1 -- 172.21.15.45:0/478705084 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe8f0004f20 con 0x7fe8f809ff70 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8fd060640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8fd060640 4 set_mon_vals no callback set 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.296+0000 7fe8f5d9b640 10 monclient: dump: 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.297+0000 7fe8fd060640 10 monclient: shutdown 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.297+0000 7fe8fd060640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.297+0000 7fe8fd060640 1 -- 172.21.15.45:0/478705084 >> v1:172.21.15.45:6791/0 conn(0x7fe8f809ff70 legacy=0x7fe8f813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.297+0000 7fe8fd060640 1 -- 172.21.15.45:0/478705084 shutdown_connections 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.297+0000 7fe8fd060640 1 -- 172.21.15.45:0/478705084 >> 172.21.15.45:0/478705084 conn(0x7fe8f80630b0 msgr2=0x7fe8f80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.297+0000 7fe8fd060640 1 -- 172.21.15.45:0/478705084 shutdown_connections 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.297+0000 7fe8fd060640 1 -- 172.21.15.45:0/478705084 wait complete. 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.298+0000 7fe8fd060640 10 monclient: build_initial_monmap 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.298+0000 7fe8fd060640 10 monclient: monmap: 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.298+0000 7fe8fd060640 1 Processor -- start 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.298+0000 7fe8fd060640 1 -- start start 2024-09-18T14:15:02.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.298+0000 7fe8fd060640 10 monclient: init 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.298+0000 7fe8fd060640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.298+0000 7fe8fd060640 10 monclient: _add_conns ranks=[0,1,7,5,6,8,2,4,3] 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.298+0000 7fe8fd060640 10 monclient(hunting): picked mon.noname-a con 0x7fe8f809f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.298+0000 7fe8fd060640 10 monclient(hunting): picked mon.noname-f con 0x7fe8f809fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.298+0000 7fe8fd060640 10 monclient(hunting): picked mon.noname-i con 0x7fe8f809ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8fd060640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f809beb0 con 0x7fe8f809f7d0 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8fd060640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f80a2560 con 0x7fe8f809fba0 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8fd060640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f809d310 con 0x7fe8f809ff70 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8fd060640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8fd060640 10 monclient(hunting): authenticate will time out at 1717.637056s 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8f6d9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe8f809f7d0 0x7fe8f81316b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56260/0 (socket says 172.21.15.45:56260) 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8f6d9d640 1 -- 172.21.15.45:0/3647691056 learned_addr learned my addr 172.21.15.45:0/3647691056 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:02.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910569939 0 0) 0x7fe8f809beb0 con 0x7fe8f809f7d0 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): my global_id is 6102 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e40034e0 con 0x7fe8f809f7d0 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780902836 0 0) 0x7fe8f80a2560 con 0x7fe8f809fba0 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): my global_id is 214165 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f809beb0 con 0x7fe8f809fba0 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100516008 0 0) 0x7fe8f809d310 con 0x7fe8f809ff70 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): my global_id is 214144 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f80a2560 con 0x7fe8f809ff70 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2539566140 0 0) 0x7fe8e40034e0 con 0x7fe8f809f7d0 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe8f809d310 con 0x7fe8f809f7d0 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe8e80015b0 con 0x7fe8f809f7d0 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): dump: 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.325 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.326 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.326 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.326 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.326 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.326 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.326 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.326 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.326 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.326 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:02.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3056971796 0 0) 0x7fe8f809beb0 con 0x7fe8f809fba0 2024-09-18T14:15:02.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe8e40034e0 con 0x7fe8f809fba0 2024-09-18T14:15:02.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe8ec001a60 con 0x7fe8f809fba0 2024-09-18T14:15:02.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:02.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): dump: 2024-09-18T14:15:02.327 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.327 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.327 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.327 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.327 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.327 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.327 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.327 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 865831412 0 0) 0x7fe8f80a2560 con 0x7fe8f809ff70 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe8f809beb0 con 0x7fe8f809ff70 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe8f0001ad0 con 0x7fe8f809ff70 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient(hunting): dump: 2024-09-18T14:15:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 934118154 0 0) 0x7fe8f809d310 con 0x7fe8f809f7d0 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 >> v1:172.21.15.84:6792/0 conn(0x7fe8f809ff70 legacy=0x7fe8f814a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 >> v1:172.21.15.84:6789/0 conn(0x7fe8f809fba0 legacy=0x7fe8f8146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 monclient: found mon.a 2024-09-18T14:15:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f811d870 con 0x7fe8f809f7d0 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient: _finish_auth 0 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8fd060640 5 monclient: authenticate success, global_id 6102 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8fd060640 10 monclient: _renew_subs 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8fd060640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8fd060640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f8142830 con 0x7fe8f809f7d0 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8fd060640 10 monclient: _renew_subs 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8fd060640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8fd060640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f8136ea0 con 0x7fe8f809f7d0 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe8e8001830 con 0x7fe8f809f7d0 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe8e8004dd0 con 0x7fe8f809f7d0 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.299+0000 7fe8f559a640 4 set_mon_vals no callback set 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.300+0000 7fe8deffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.300+0000 7fe8deffd640 10 monclient: dump: 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.300+0000 7fe8fd060640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.300+0000 7fe8fd060640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe8d8006580 con 0x7fe8d8002830 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.300+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fe8e800fda0 con 0x7fe8f809f7d0 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(737..737 src has 529..737) v4 ==== 3971+0+0 (unknown 1184779458 0 0) 0x7fe8e8046ac0 con 0x7fe8f809f7d0 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730003579 0 0) 0x7fe8d8006580 con 0x7fe8d8002830 2024-09-18T14:15:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 10 monclient(hunting): my global_id is 6111 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f809d310 con 0x7fe8d8002830 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8f759e640 10 monclient: get_auth_request con 0x7fe8e403a990 auth_method 0 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4166919083 0 0) 0x7fe8f809d310 con 0x7fe8d8002830 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe8d8006580 con 0x7fe8d8002830 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe8ec0021b0 con 0x7fe8d8002830 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 10 monclient: dump: 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3009494226 0 0) 0x7fe8d8006580 con 0x7fe8d8002830 2024-09-18T14:15:02.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8d800a0a0 con 0x7fe8d8002830 2024-09-18T14:15:02.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe8ec0043b0 con 0x7fe8d8002830 2024-09-18T14:15:02.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:02.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:02.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 >> v1:172.21.15.45:6789/0 conn(0x7fe8d8002830 legacy=0x7fe8d8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.301+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 >> v1:172.21.15.45:6789/0 conn(0x7fe8d8002830 legacy=0x7fe8d8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:02.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.307+0000 7fe8fd060640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:02.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.307+0000 7fe8fd060640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe8d8009660 con 0x7fe8d8009040 2024-09-18T14:15:02.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.307+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040241871 0 0) 0x7fe8d8009660 con 0x7fe8d8009040 2024-09-18T14:15:02.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.307+0000 7fe8deffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.307+0000 7fe8deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.307+0000 7fe8deffd640 10 monclient(hunting): my global_id is 6120 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.307+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8d8006580 con 0x7fe8d8009040 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 701263093 0 0) 0x7fe8d8006580 con 0x7fe8d8009040 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe8d8009660 con 0x7fe8d8009040 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fe8ec002b80 con 0x7fe8d8009040 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 10 monclient: dump: 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3367988690 0 0) 0x7fe8d8009660 con 0x7fe8d8009040 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe8d8009920 con 0x7fe8d8009040 2024-09-18T14:15:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 213706369) 0x7fe8ec003f70 con 0x7fe8d8009040 2024-09-18T14:15:02.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:02.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:02.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 >> v1:172.21.15.45:6789/0 conn(0x7fe8d8009040 legacy=0x7fe8d8008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8deffd640 1 -- 172.21.15.45:0/3647691056 >> v1:172.21.15.45:6789/0 conn(0x7fe8d8009040 legacy=0x7fe8d8008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:02.339 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:02.339 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:02.339 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:02.340 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:02.340 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:02.340 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:02.340 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:02.340 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:02.340 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:02.340 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:02.340 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:02.340 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:02.340 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:02.340 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:02.340 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.341 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.342 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.342 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.342 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:02.342 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:02.342 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.342 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.342 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.342 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.342 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.342 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.342 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.342 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:02.343 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.344 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.346 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:02.347 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.348 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.348 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.348 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:02.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:02.348 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.348 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.348 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:02.348 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:02.349 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.351 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.352 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:02.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.354 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.355 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.355 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:02.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:02.355 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.355 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.355 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:02.355 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.356 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.357 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.358 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:15:02.359 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.359 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.359 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:02.359 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:02.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8fd060640 1 -- 172.21.15.45:0/3647691056 >> v1:172.21.15.84:6812/31478 conn(0x7fe8e403a990 legacy=0x7fe8e403ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8fd060640 10 monclient: shutdown 2024-09-18T14:15:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8fd060640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8fd060640 1 -- 172.21.15.45:0/3647691056 >> v1:172.21.15.45:6789/0 conn(0x7fe8f809f7d0 legacy=0x7fe8f81316b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.308+0000 7fe8f759e640 1 -- 172.21.15.45:0/3647691056 reap_dead start 2024-09-18T14:15:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.309+0000 7fe8f759e640 1 -- 172.21.15.45:0/3647691056 reap_dead start 2024-09-18T14:15:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.309+0000 7fe8fd060640 1 -- 172.21.15.45:0/3647691056 shutdown_connections 2024-09-18T14:15:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.309+0000 7fe8fd060640 1 -- 172.21.15.45:0/3647691056 >> 172.21.15.45:0/3647691056 conn(0x7fe8f80630b0 msgr2=0x7fe8f80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.309+0000 7fe8fd060640 1 -- 172.21.15.45:0/3647691056 shutdown_connections 2024-09-18T14:15:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.309+0000 7fe8fd060640 1 -- 172.21.15.45:0/3647691056 wait complete. 2024-09-18T14:15:02.361 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:02.362 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:15:02.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:15:02.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.458+0000 7f5e94b54640 10 monclient: get_monmap_and_config 2024-09-18T14:15:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.458+0000 7f5e94b54640 10 monclient: build_initial_monmap 2024-09-18T14:15:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.458+0000 7f5e94b54640 10 monclient: monmap: 2024-09-18T14:15:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:02.459435+0000 2024-09-18T14:15:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:02.459435+0000 2024-09-18T14:15:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:02.460 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 1 Processor -- start 2024-09-18T14:15:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 1 -- start start 2024-09-18T14:15:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 10 monclient: init 2024-09-18T14:15:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 10 monclient: _add_conns ranks=[3,1,2,0,6,4,7,8,5] 2024-09-18T14:15:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 10 monclient(hunting): picked mon.noname-g con 0x7f5e9009f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 10 monclient(hunting): picked mon.noname-f con 0x7f5e9009fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 10 monclient(hunting): picked mon.noname-b con 0x7f5e9009ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9005ed70 con 0x7f5e9009fba0 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9009dfc0 con 0x7f5e9009ff70 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9009dce0 con 0x7f5e9009f7d0 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e94b54640 10 monclient(hunting): authenticate will time out at 1717.797781s 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e8ed9d640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f5e9009ff70 0x7f5e9013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33076/0 (socket says 172.21.15.45:33076) 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.459+0000 7f5e8ed9d640 1 -- 172.21.15.45:0/3857879368 learned_addr learned my addr 172.21.15.45:0/3857879368 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 1 -- 172.21.15.45:0/3857879368 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512992466 0 0) 0x7f5e9009dfc0 con 0x7f5e9009ff70 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient(hunting): my global_id is 145298 2024-09-18T14:15:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 1 -- 172.21.15.45:0/3857879368 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e7c0034e0 con 0x7f5e9009ff70 2024-09-18T14:15:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 1 -- 172.21.15.45:0/3857879368 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2235164552 0 0) 0x7f5e7c0034e0 con 0x7f5e9009ff70 2024-09-18T14:15:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 1 -- 172.21.15.45:0/3857879368 >> v1:172.21.15.84:6790/0 conn(0x7f5e9009f7d0 legacy=0x7f5e90132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 1 -- 172.21.15.45:0/3857879368 >> v1:172.21.15.84:6789/0 conn(0x7f5e9009fba0 legacy=0x7f5e90137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient: _finish_hunting 0 2024-09-18T14:15:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 1 monclient: found mon.noname-b 2024-09-18T14:15:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:15:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 1 -- 172.21.15.45:0/3857879368 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e900a1c00 con 0x7f5e9009ff70 2024-09-18T14:15:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient: _finish_auth 0 2024-09-18T14:15:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 1 -- 172.21.15.45:0/3857879368 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5e88001bb0 con 0x7f5e9009ff70 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e94b54640 5 monclient: authenticate success, global_id 145298 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e94b54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient: dump: 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e94b54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 1 -- 172.21.15.45:0/3857879368 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5e88001d90 con 0x7f5e9009ff70 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 1 -- 172.21.15.45:0/3857879368 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5e88004f20 con 0x7f5e9009ff70 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e94b54640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e94b54640 4 set_mon_vals no callback set 2024-09-18T14:15:02.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e8d59a640 10 monclient: dump: 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.474 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e94b54640 10 monclient: shutdown 2024-09-18T14:15:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e94b54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.460+0000 7f5e94b54640 1 -- 172.21.15.45:0/3857879368 >> v1:172.21.15.45:6790/0 conn(0x7f5e9009ff70 legacy=0x7f5e9013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.461+0000 7f5e94b54640 1 -- 172.21.15.45:0/3857879368 shutdown_connections 2024-09-18T14:15:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.461+0000 7f5e94b54640 1 -- 172.21.15.45:0/3857879368 >> 172.21.15.45:0/3857879368 conn(0x7f5e900630b0 msgr2=0x7f5e900634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.461+0000 7f5e94b54640 1 -- 172.21.15.45:0/3857879368 shutdown_connections 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.461+0000 7f5e94b54640 1 -- 172.21.15.45:0/3857879368 wait complete. 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.461+0000 7f5e94b54640 10 monclient: build_initial_monmap 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.461+0000 7f5e94b54640 10 monclient: monmap: 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.461+0000 7f5e94b54640 1 Processor -- start 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.461+0000 7f5e94b54640 1 -- start start 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.461+0000 7f5e94b54640 10 monclient: init 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e94b54640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e94b54640 10 monclient: _add_conns ranks=[7,0,4,3,2,6,8,1,5] 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e94b54640 10 monclient(hunting): picked mon.noname-i con 0x7f5e9009f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e94b54640 10 monclient(hunting): picked mon.noname-a con 0x7f5e9009fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e94b54640 10 monclient(hunting): picked mon.noname-c con 0x7f5e9009ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e94b54640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e901428c0 con 0x7f5e9009fba0 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e94b54640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9009bda0 con 0x7f5e9009ff70 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e94b54640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e900a2560 con 0x7f5e9009f7d0 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e94b54640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e94b54640 10 monclient(hunting): authenticate will time out at 1717.800465s 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e8ed9d640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f5e9009ff70 0x7f5e9014a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39576/0 (socket says 172.21.15.45:39576) 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e8dd9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5e9009fba0 0x7f5e90146cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56294/0 (socket says 172.21.15.45:56294) 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e8ed9d640 1 -- 172.21.15.45:0/3069601716 learned_addr learned my addr 172.21.15.45:0/3069601716 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456847243 0 0) 0x7f5e901428c0 con 0x7f5e9009fba0 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 10 monclient(hunting): my global_id is 6129 2024-09-18T14:15:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e700034e0 con 0x7f5e9009fba0 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191559831 0 0) 0x7f5e9009bda0 con 0x7f5e9009ff70 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 10 monclient(hunting): my global_id is 164866 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e901428c0 con 0x7f5e9009ff70 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772724172 0 0) 0x7f5e900a2560 con 0x7f5e9009f7d0 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 10 monclient(hunting): my global_id is 214153 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.462+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e9009bda0 con 0x7f5e9009f7d0 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4266810843 0 0) 0x7f5e700034e0 con 0x7f5e9009fba0 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5e900a2560 con 0x7f5e9009fba0 2024-09-18T14:15:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5e78001b20 con 0x7f5e9009fba0 2024-09-18T14:15:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:15:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient(hunting): dump: 2024-09-18T14:15:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2382987820 0 0) 0x7f5e901428c0 con 0x7f5e9009ff70 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5e700034e0 con 0x7f5e9009ff70 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2233739452 0 0) 0x7f5e9009bda0 con 0x7f5e9009f7d0 2024-09-18T14:15:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5e901428c0 con 0x7f5e9009f7d0 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5e88001af0 con 0x7f5e9009ff70 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient(hunting): dump: 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5e80001920 con 0x7f5e9009f7d0 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient(hunting): dump: 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1441931145 0 0) 0x7f5e900a2560 con 0x7f5e9009fba0 2024-09-18T14:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 >> v1:172.21.15.84:6792/0 conn(0x7f5e9009f7d0 legacy=0x7f5e901316b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 >> v1:172.21.15.45:6791/0 conn(0x7f5e9009ff70 legacy=0x7f5e9014a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient: _finish_hunting 0 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 monclient: found mon.a 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9009d310 con 0x7f5e9009fba0 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient: _finish_auth 0 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e94b54640 5 monclient: authenticate success, global_id 6129 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e94b54640 10 monclient: _renew_subs 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e94b54640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e94b54640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9011d870 con 0x7f5e9009fba0 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e94b54640 10 monclient: _renew_subs 2024-09-18T14:15:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e94b54640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e94b54640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e90136ea0 con 0x7f5e9009fba0 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5e78001d60 con 0x7f5e9009fba0 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5e780025f0 con 0x7f5e9009fba0 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e8cd99640 4 set_mon_vals no callback set 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.463+0000 7f5e867fc640 10 monclient: dump: 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e94b54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e94b54640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5e74006580 con 0x7f5e74002830 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f5e78002750 con 0x7f5e9009fba0 2024-09-18T14:15:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(737..737 src has 529..737) v4 ==== 3971+0+0 (unknown 1184779458 0 0) 0x7f5e78046ba0 con 0x7f5e9009fba0 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614887191 0 0) 0x7f5e74006580 con 0x7f5e74002830 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 10 monclient(hunting): my global_id is 145307 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e900a2560 con 0x7f5e74002830 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e8ed9d640 10 monclient: get_auth_request con 0x7f5e7003aa50 auth_method 0 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4179083705 0 0) 0x7f5e900a2560 con 0x7f5e74002830 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5e74006580 con 0x7f5e74002830 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5e800020f0 con 0x7f5e74002830 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.464+0000 7f5e867fc640 10 monclient: dump: 2024-09-18T14:15:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.489 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.465+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4085797879 0 0) 0x7f5e74006580 con 0x7f5e74002830 2024-09-18T14:15:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.465+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e7400a0a0 con 0x7f5e74002830 2024-09-18T14:15:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.465+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5e800023f0 con 0x7f5e74002830 2024-09-18T14:15:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.465+0000 7f5e867fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.465+0000 7f5e867fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.465+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 >> v1:172.21.15.45:6790/0 conn(0x7f5e74002830 legacy=0x7f5e74004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.465+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 >> v1:172.21.15.45:6790/0 conn(0x7f5e74002830 legacy=0x7f5e74004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e94b54640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e94b54640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5e74009660 con 0x7f5e74009040 2024-09-18T14:15:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341713739 0 0) 0x7f5e74009660 con 0x7f5e74009040 2024-09-18T14:15:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 10 monclient(hunting): my global_id is 145316 2024-09-18T14:15:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e74006580 con 0x7f5e74009040 2024-09-18T14:15:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 673860433 0 0) 0x7f5e74006580 con 0x7f5e74009040 2024-09-18T14:15:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5e74009660 con 0x7f5e74009040 2024-09-18T14:15:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f5e80002cb0 con 0x7f5e74009040 2024-09-18T14:15:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 10 monclient: dump: 2024-09-18T14:15:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3511043584 0 0) 0x7f5e74009660 con 0x7f5e74009040 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5e74009920 con 0x7f5e74009040 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 2066565213) 0x7f5e800030d0 con 0x7f5e74009040 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 >> v1:172.21.15.45:6790/0 conn(0x7f5e74009040 legacy=0x7f5e74008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.471+0000 7f5e867fc640 1 -- 172.21.15.45:0/3069601716 >> v1:172.21.15.45:6790/0 conn(0x7f5e74009040 legacy=0x7f5e74008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:02.494 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:02.494 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:02.494 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:02.494 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:02.494 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:02.494 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:02.494 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:02.494 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:02.494 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:02.494 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:02.495 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:02.495 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:02.495 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:02.495 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:02.495 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:02.495 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.495 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:15:02.495 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.496 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.497 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:02.498 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.499 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:02.500 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.501 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.503 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.504 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:02.506 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.507 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.510 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:02.511 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.512 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:02.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.514 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:15:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.515 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:02.516 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.472+0000 7f5e94b54640 1 -- 172.21.15.45:0/3069601716 >> v1:172.21.15.84:6812/31478 conn(0x7f5e7003aa50 legacy=0x7f5e7003cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.472+0000 7f5e94b54640 10 monclient: shutdown 2024-09-18T14:15:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.472+0000 7f5e94b54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.472+0000 7f5e94b54640 1 -- 172.21.15.45:0/3069601716 >> v1:172.21.15.45:6789/0 conn(0x7f5e9009fba0 legacy=0x7f5e90146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.472+0000 7f5e8ed9d640 1 -- 172.21.15.45:0/3069601716 reap_dead start 2024-09-18T14:15:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.472+0000 7f5e8ed9d640 1 -- 172.21.15.45:0/3069601716 reap_dead start 2024-09-18T14:15:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.472+0000 7f5e94b54640 1 -- 172.21.15.45:0/3069601716 shutdown_connections 2024-09-18T14:15:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.472+0000 7f5e94b54640 1 -- 172.21.15.45:0/3069601716 >> 172.21.15.45:0/3069601716 conn(0x7f5e900630b0 msgr2=0x7f5e900a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.472+0000 7f5e94b54640 1 -- 172.21.15.45:0/3069601716 shutdown_connections 2024-09-18T14:15:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.472+0000 7f5e94b54640 1 -- 172.21.15.45:0/3069601716 wait complete. 2024-09-18T14:15:02.518 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:02.518 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:15:02.518 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:15:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.615+0000 7f61494ef640 10 monclient: get_monmap_and_config 2024-09-18T14:15:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.615+0000 7f61494ef640 10 monclient: build_initial_monmap 2024-09-18T14:15:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.615+0000 7f61494ef640 10 monclient: monmap: 2024-09-18T14:15:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:02.616288+0000 2024-09-18T14:15:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:02.616288+0000 2024-09-18T14:15:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:02.617 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.615+0000 7f61494ef640 1 Processor -- start 2024-09-18T14:15:02.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.615+0000 7f61494ef640 1 -- start start 2024-09-18T14:15:02.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.615+0000 7f61494ef640 10 monclient: init 2024-09-18T14:15:02.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f61494ef640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:02.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f61494ef640 10 monclient: _add_conns ranks=[0,8,7,2,6,3,5,4,1] 2024-09-18T14:15:02.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f61494ef640 10 monclient(hunting): picked mon.noname-a con 0x7f614409f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:02.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f61494ef640 10 monclient(hunting): picked mon.noname-e con 0x7f614409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:02.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f61494ef640 10 monclient(hunting): picked mon.noname-i con 0x7f614409ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f61494ef640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614405ed70 con 0x7f614409f7d0 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f61494ef640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614409dfc0 con 0x7f614409ff70 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f61494ef640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614409dce0 con 0x7f614409fba0 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f61494ef640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f61494ef640 10 monclient(hunting): authenticate will time out at 1717.954482s 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f61427fc640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f614409fba0 0x7f6144137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34126/0 (socket says 172.21.15.45:34126) 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f6142ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f614409f7d0 0x7f6144132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56300/0 (socket says 172.21.15.45:56300) 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f61427fc640 1 -- 172.21.15.45:0/681176156 learned_addr learned my addr 172.21.15.45:0/681176156 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924432409 0 0) 0x7f614405ed70 con 0x7f614409f7d0 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f6141ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f6141ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f6141ffb640 10 monclient(hunting): my global_id is 6138 2024-09-18T14:15:02.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61300034e0 con 0x7f614409f7d0 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944390304 0 0) 0x7f614409dce0 con 0x7f614409fba0 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f6141ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f6141ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f6141ffb640 10 monclient(hunting): my global_id is 5840 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.616+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f614405ed70 con 0x7f614409fba0 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865915902 0 0) 0x7f614409dfc0 con 0x7f614409ff70 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient(hunting): my global_id is 214162 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f614409dce0 con 0x7f614409ff70 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1733354401 0 0) 0x7f614405ed70 con 0x7f614409fba0 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 >> v1:172.21.15.84:6792/0 conn(0x7f614409ff70 legacy=0x7f614413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 >> v1:172.21.15.45:6789/0 conn(0x7f614409f7d0 legacy=0x7f6144132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:15:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 1 monclient: found mon.noname-e 2024-09-18T14:15:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:15:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61440a1c00 con 0x7f614409fba0 2024-09-18T14:15:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient: _finish_auth 0 2024-09-18T14:15:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f61494ef640 5 monclient: authenticate success, global_id 5840 2024-09-18T14:15:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f61494ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f612c001810 con 0x7f614409fba0 2024-09-18T14:15:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient: dump: 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.630 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.631 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f61494ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f612c001970 con 0x7f614409fba0 2024-09-18T14:15:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 1 -- 172.21.15.45:0/681176156 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f612c004ba0 con 0x7f614409fba0 2024-09-18T14:15:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f6141ffb640 10 monclient: dump: 2024-09-18T14:15:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f61494ef640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f61494ef640 4 set_mon_vals no callback set 2024-09-18T14:15:02.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f61494ef640 10 monclient: shutdown 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f61494ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f61494ef640 1 -- 172.21.15.45:0/681176156 >> v1:172.21.15.45:6793/0 conn(0x7f614409fba0 legacy=0x7f6144137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f61494ef640 1 -- 172.21.15.45:0/681176156 shutdown_connections 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.617+0000 7f61494ef640 1 -- 172.21.15.45:0/681176156 >> 172.21.15.45:0/681176156 conn(0x7f61440630b0 msgr2=0x7f61440634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.618+0000 7f61494ef640 1 -- 172.21.15.45:0/681176156 shutdown_connections 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.618+0000 7f61494ef640 1 -- 172.21.15.45:0/681176156 wait complete. 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.618+0000 7f61494ef640 10 monclient: build_initial_monmap 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.618+0000 7f61494ef640 10 monclient: monmap: 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:02.634 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 1 Processor -- start 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 1 -- start start 2024-09-18T14:15:02.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 10 monclient: init 2024-09-18T14:15:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 10 monclient: _add_conns ranks=[3,5,7,0,4,6,2,1,8] 2024-09-18T14:15:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 10 monclient(hunting): picked mon.noname-g con 0x7f614409f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 10 monclient(hunting): picked mon.noname-h con 0x7f614409fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 10 monclient(hunting): picked mon.noname-i con 0x7f614409ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614409d310 con 0x7f614409f7d0 2024-09-18T14:15:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440a2560 con 0x7f614409fba0 2024-09-18T14:15:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614409be50 con 0x7f614409ff70 2024-09-18T14:15:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:02.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f61494ef640 10 monclient(hunting): authenticate will time out at 1717.957647s 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f6142ffd640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7f614409f7d0 0x7f6144131720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:36028/0 (socket says 172.21.15.45:36028) 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.619+0000 7f6142ffd640 1 -- 172.21.15.45:0/1056292203 learned_addr learned my addr 172.21.15.45:0/1056292203 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526473353 0 0) 0x7f614409d310 con 0x7f614409f7d0 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): my global_id is 184602 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61240034e0 con 0x7f614409f7d0 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714505058 0 0) 0x7f614409be50 con 0x7f614409ff70 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): my global_id is 214171 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f614409d310 con 0x7f614409ff70 2024-09-18T14:15:02.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725135014 0 0) 0x7f61440a2560 con 0x7f614409fba0 2024-09-18T14:15:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): my global_id is 184568 2024-09-18T14:15:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f614409be50 con 0x7f614409fba0 2024-09-18T14:15:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4099260075 0 0) 0x7f61240034e0 con 0x7f614409f7d0 2024-09-18T14:15:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f61440a2560 con 0x7f614409f7d0 2024-09-18T14:15:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6134001a10 con 0x7f614409f7d0 2024-09-18T14:15:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:15:02.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): dump: 2024-09-18T14:15:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.639 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2262264986 0 0) 0x7f614409d310 con 0x7f614409ff70 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f61240034e0 con 0x7f614409ff70 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 164541030 0 0) 0x7f614409be50 con 0x7f614409fba0 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f614409d310 con 0x7f614409fba0 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f613c0018c0 con 0x7f614409ff70 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): dump: 2024-09-18T14:15:02.640 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.641 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f612c001750 con 0x7f614409fba0 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient(hunting): dump: 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.642 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1271032567 0 0) 0x7f61440a2560 con 0x7f614409f7d0 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 >> v1:172.21.15.84:6792/0 conn(0x7f614409ff70 legacy=0x7f614414a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 >> v1:172.21.15.84:6791/0 conn(0x7f614409fba0 legacy=0x7f6144146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 monclient: found mon.g 2024-09-18T14:15:02.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614409b940 con 0x7f614409f7d0 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient: _finish_auth 0 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f61494ef640 5 monclient: authenticate success, global_id 184602 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f61494ef640 10 monclient: _renew_subs 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f61494ef640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f61494ef640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6144136ea0 con 0x7f614409f7d0 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f61494ef640 10 monclient: _renew_subs 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f61494ef640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f61494ef640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f614411d8e0 con 0x7f614409f7d0 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6134004dd0 con 0x7f614409f7d0 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:02.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6134005280 con 0x7f614409f7d0 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f61417fa640 4 set_mon_vals no callback set 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.620+0000 7f613affd640 10 monclient: dump: 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.645 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.621+0000 7f61494ef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.621+0000 7f61494ef640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6128006580 con 0x7f6128002830 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.621+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f61340102d0 con 0x7f614409f7d0 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.621+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(737..737 src has 529..737) v4 ==== 3971+0+0 (unknown 1184779458 0 0) 0x7f6134046f80 con 0x7f614409f7d0 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.621+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066407365 0 0) 0x7f6128006580 con 0x7f6128002830 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.621+0000 7f613affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.621+0000 7f613affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 10 monclient(hunting): my global_id is 164875 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61440a2560 con 0x7f6128002830 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f61437fe640 10 monclient: get_auth_request con 0x7f612403ab70 auth_method 0 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4148444641 0 0) 0x7f61440a2560 con 0x7f6128002830 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6128006580 con 0x7f6128002830 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f612c004e60 con 0x7f6128002830 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 10 monclient: dump: 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1321459779 0 0) 0x7f6128006580 con 0x7f6128002830 2024-09-18T14:15:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f612800a0a0 con 0x7f6128002830 2024-09-18T14:15:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f612c006fc0 con 0x7f6128002830 2024-09-18T14:15:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 >> v1:172.21.15.45:6791/0 conn(0x7f6128002830 legacy=0x7f6128004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.622+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 >> v1:172.21.15.45:6791/0 conn(0x7f6128002830 legacy=0x7f6128004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:02.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f61494ef640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:02.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f61494ef640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6128009660 con 0x7f6128009040 2024-09-18T14:15:02.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967120340 0 0) 0x7f6128009660 con 0x7f6128009040 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f613affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f613affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f613affd640 10 monclient(hunting): my global_id is 164884 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6128006580 con 0x7f6128009040 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 83507222 0 0) 0x7f6128006580 con 0x7f6128009040 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6128009660 con 0x7f6128009040 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f612c0056b0 con 0x7f6128009040 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f613affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f613affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.628+0000 7f613affd640 10 monclient: dump: 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.651 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 362009763 0 0) 0x7f6128009660 con 0x7f6128009040 2024-09-18T14:15:02.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6128009920 con 0x7f6128009040 2024-09-18T14:15:02.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 1566892992) 0x7f612c006ac0 con 0x7f6128009040 2024-09-18T14:15:02.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f613affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:02.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f613affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:02.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 >> v1:172.21.15.45:6791/0 conn(0x7f6128009040 legacy=0x7f6128008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f613affd640 1 -- 172.21.15.45:0/1056292203 >> v1:172.21.15.45:6791/0 conn(0x7f6128009040 legacy=0x7f6128008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:02.653 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:15:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.673 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:02.673 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:02.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f61494ef640 1 -- 172.21.15.45:0/1056292203 >> v1:172.21.15.84:6812/31478 conn(0x7f612403ab70 legacy=0x7f612403d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f61494ef640 10 monclient: shutdown 2024-09-18T14:15:02.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f61494ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:02.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f61494ef640 1 -- 172.21.15.45:0/1056292203 >> v1:172.21.15.84:6790/0 conn(0x7f614409f7d0 legacy=0x7f6144131720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f61437fe640 1 -- 172.21.15.45:0/1056292203 reap_dead start 2024-09-18T14:15:02.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f61437fe640 1 -- 172.21.15.45:0/1056292203 reap_dead start 2024-09-18T14:15:02.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f61494ef640 1 -- 172.21.15.45:0/1056292203 shutdown_connections 2024-09-18T14:15:02.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f61494ef640 1 -- 172.21.15.45:0/1056292203 >> 172.21.15.45:0/1056292203 conn(0x7f61440630b0 msgr2=0x7f61440a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:02.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f61494ef640 1 -- 172.21.15.45:0/1056292203 shutdown_connections 2024-09-18T14:15:02.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.629+0000 7f61494ef640 1 -- 172.21.15.45:0/1056292203 wait complete. 2024-09-18T14:15:02.675 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:02.675 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:15:02.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:15:02.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.771+0000 7ff4e039d640 10 monclient: get_monmap_and_config 2024-09-18T14:15:02.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.771+0000 7ff4e039d640 10 monclient: build_initial_monmap 2024-09-18T14:15:02.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.771+0000 7ff4e039d640 10 monclient: monmap: 2024-09-18T14:15:02.772 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:02.772 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:02.772700+0000 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:02.772700+0000 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:02.773 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.772+0000 7ff4e039d640 1 Processor -- start 2024-09-18T14:15:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.772+0000 7ff4e039d640 1 -- start start 2024-09-18T14:15:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.772+0000 7ff4e039d640 10 monclient: init 2024-09-18T14:15:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.772+0000 7ff4e039d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.772+0000 7ff4e039d640 10 monclient: _add_conns ranks=[5,8,6,3,4,2,0,1,7] 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4e039d640 10 monclient(hunting): picked mon.noname-h con 0x7ff4d809f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4e039d640 10 monclient(hunting): picked mon.noname-e con 0x7ff4d809fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4e039d640 10 monclient(hunting): picked mon.noname-d con 0x7ff4d809ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4e039d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d805ed70 con 0x7ff4d809f7d0 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4e039d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d809dfc0 con 0x7ff4d809ff70 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4e039d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d809dce0 con 0x7ff4d809fba0 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4e039d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4e039d640 10 monclient(hunting): authenticate will time out at 1718.111078s 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4de93a640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7ff4d809ff70 0x7ff4d813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43320/0 (socket says 172.21.15.45:43320) 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd938640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7ff4d809fba0 0x7ff4d8137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34130/0 (socket says 172.21.15.45:34130) 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4de93a640 1 -- 172.21.15.45:0/2775221246 learned_addr learned my addr 172.21.15.45:0/2775221246 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663593549 0 0) 0x7ff4d809dfc0 con 0x7ff4d809ff70 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient(hunting): my global_id is 184659 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4c40034e0 con 0x7ff4d809ff70 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474175683 0 0) 0x7ff4d809dce0 con 0x7ff4d809fba0 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient(hunting): my global_id is 5849 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4d809dfc0 con 0x7ff4d809fba0 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335851910 0 0) 0x7ff4d805ed70 con 0x7ff4d809f7d0 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient(hunting): my global_id is 184577 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4d809dce0 con 0x7ff4d809f7d0 2024-09-18T14:15:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2947359077 0 0) 0x7ff4c40034e0 con 0x7ff4d809ff70 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 >> v1:172.21.15.45:6793/0 conn(0x7ff4d809fba0 legacy=0x7ff4d8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 >> v1:172.21.15.84:6791/0 conn(0x7ff4d809f7d0 legacy=0x7ff4d8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient: _finish_hunting 0 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 monclient: found mon.noname-d 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d80a1c00 con 0x7ff4d809ff70 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient: _finish_auth 0 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4e039d640 5 monclient: authenticate success, global_id 184659 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff4d0001bb0 con 0x7ff4d809ff70 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient: dump: 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.786 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4e039d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff4d0001d90 con 0x7ff4d809ff70 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4e039d640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 1 -- 172.21.15.45:0/2775221246 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff4d0004f20 con 0x7ff4d809ff70 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4e039d640 4 set_mon_vals no callback set 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.773+0000 7ff4dd137640 10 monclient: dump: 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.787 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.788 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.774+0000 7ff4e039d640 10 monclient: shutdown 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.774+0000 7ff4e039d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.774+0000 7ff4e039d640 1 -- 172.21.15.45:0/2775221246 >> v1:172.21.15.45:6792/0 conn(0x7ff4d809ff70 legacy=0x7ff4d813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.774+0000 7ff4e039d640 1 -- 172.21.15.45:0/2775221246 shutdown_connections 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.774+0000 7ff4e039d640 1 -- 172.21.15.45:0/2775221246 >> 172.21.15.45:0/2775221246 conn(0x7ff4d80630b0 msgr2=0x7ff4d80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.774+0000 7ff4e039d640 1 -- 172.21.15.45:0/2775221246 shutdown_connections 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.774+0000 7ff4e039d640 1 -- 172.21.15.45:0/2775221246 wait complete. 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 10 monclient: build_initial_monmap 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 10 monclient: monmap: 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:02.789 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 1 Processor -- start 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 1 -- start start 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 10 monclient: init 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 10 monclient: _add_conns ranks=[7,6,3,4,0,2,5,1,8] 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 10 monclient(hunting): picked mon.noname-i con 0x7ff4d809f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 10 monclient(hunting): picked mon.noname-d con 0x7ff4d809fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 10 monclient(hunting): picked mon.noname-g con 0x7ff4d809ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d809d310 con 0x7ff4d809ff70 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d80a2560 con 0x7ff4d809fba0 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d8121a80 con 0x7ff4d809f7d0 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4e039d640 10 monclient(hunting): authenticate will time out at 1718.113959s 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.775+0000 7ff4dd938640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7ff4d809fba0 0x7ff4d8146ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43326/0 (socket says 172.21.15.45:43326) 2024-09-18T14:15:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4dd938640 1 -- 172.21.15.45:0/408619637 learned_addr learned my addr 172.21.15.45:0/408619637 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144109085 0 0) 0x7ff4d80a2560 con 0x7ff4d809fba0 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): my global_id is 184668 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4bc0034e0 con 0x7ff4d809fba0 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389671338 0 0) 0x7ff4d8121a80 con 0x7ff4d809f7d0 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): my global_id is 214180 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4d80a2560 con 0x7ff4d809f7d0 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498291687 0 0) 0x7ff4d809d310 con 0x7ff4d809ff70 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): my global_id is 184611 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4d8121a80 con 0x7ff4d809ff70 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3367463275 0 0) 0x7ff4bc0034e0 con 0x7ff4d809fba0 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff4d809d310 con 0x7ff4d809fba0 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff4cc001650 con 0x7ff4d809fba0 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): dump: 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2447061178 0 0) 0x7ff4d80a2560 con 0x7ff4d809f7d0 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff4bc0034e0 con 0x7ff4d809f7d0 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff4d4001570 con 0x7ff4d809f7d0 2024-09-18T14:15:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient(hunting): dump: 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.795 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.796 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 554665842 0 0) 0x7ff4d809d310 con 0x7ff4d809fba0 2024-09-18T14:15:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 >> v1:172.21.15.84:6792/0 conn(0x7ff4d809f7d0 legacy=0x7ff4d8131650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 >> v1:172.21.15.84:6790/0 conn(0x7ff4d809ff70 legacy=0x7ff4d814a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:15:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 monclient: found mon.d 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d811fce0 con 0x7ff4d809fba0 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient: _finish_auth 0 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4e039d640 5 monclient: authenticate success, global_id 184668 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4e039d640 10 monclient: _renew_subs 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4e039d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4e039d640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d81217a0 con 0x7ff4d809fba0 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4e039d640 10 monclient: _renew_subs 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4e039d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4e039d640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d8131d60 con 0x7ff4d809fba0 2024-09-18T14:15:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff4cc004a50 con 0x7ff4d809fba0 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff4cc004f00 con 0x7ff4d809fba0 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4dc936640 4 set_mon_vals no callback set 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.776+0000 7ff4ca7fc640 10 monclient: dump: 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.777+0000 7ff4e039d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.777+0000 7ff4e039d640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff4c0006580 con 0x7ff4c0002830 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.777+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff4cc00ff50 con 0x7ff4d809fba0 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(737..737 src has 529..737) v4 ==== 3971+0+0 (unknown 1184779458 0 0) 0x7ff4cc046bc0 con 0x7ff4d809fba0 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903730409 0 0) 0x7ff4c0006580 con 0x7ff4c0002830 2024-09-18T14:15:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 10 monclient(hunting): my global_id is 184677 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4d809d310 con 0x7ff4c0002830 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4de93a640 10 monclient: get_auth_request con 0x7ff4bc03a450 auth_method 0 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3490363001 0 0) 0x7ff4d809d310 con 0x7ff4c0002830 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff4c0006580 con 0x7ff4c0002830 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff4d4002180 con 0x7ff4c0002830 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 10 monclient: dump: 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 674179537 0 0) 0x7ff4c0006580 con 0x7ff4c0002830 2024-09-18T14:15:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4c000a0a0 con 0x7ff4c0002830 2024-09-18T14:15:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff4d40024a0 con 0x7ff4c0002830 2024-09-18T14:15:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 >> v1:172.21.15.45:6792/0 conn(0x7ff4c0002830 legacy=0x7ff4c0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.778+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 >> v1:172.21.15.45:6792/0 conn(0x7ff4c0002830 legacy=0x7ff4c0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:02.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.784+0000 7ff4e039d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:02.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.784+0000 7ff4e039d640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff4c0007520 con 0x7ff4c00068d0 2024-09-18T14:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.784+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043371900 0 0) 0x7ff4c0007520 con 0x7ff4c00068d0 2024-09-18T14:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.784+0000 7ff4ca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.784+0000 7ff4ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.784+0000 7ff4ca7fc640 10 monclient(hunting): my global_id is 184686 2024-09-18T14:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.784+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4c0006580 con 0x7ff4c00068d0 2024-09-18T14:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3522079845 0 0) 0x7ff4c0006580 con 0x7ff4c00068d0 2024-09-18T14:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff4c0007520 con 0x7ff4c00068d0 2024-09-18T14:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff4d4002ce0 con 0x7ff4c00068d0 2024-09-18T14:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 10 monclient: dump: 2024-09-18T14:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.806 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 425108790 0 0) 0x7ff4c0007520 con 0x7ff4c00068d0 2024-09-18T14:15:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff4c0008750 con 0x7ff4c00068d0 2024-09-18T14:15:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 1147780798) 0x7ff4d4003080 con 0x7ff4c00068d0 2024-09-18T14:15:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 >> v1:172.21.15.45:6792/0 conn(0x7ff4c00068d0 legacy=0x7ff4c0006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4ca7fc640 1 -- 172.21.15.45:0/408619637 >> v1:172.21.15.45:6792/0 conn(0x7ff4c00068d0 legacy=0x7ff4c0006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:02.806 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:02.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.808 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:02.810 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.811 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.812 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:02.815 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.817 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.819 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.822 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.824 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.825 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.826 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:15:02.826 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.826 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.826 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:02.826 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:02.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4e039d640 1 -- 172.21.15.45:0/408619637 >> v1:172.21.15.84:6812/31478 conn(0x7ff4bc03a450 legacy=0x7ff4bc03c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4e039d640 10 monclient: shutdown 2024-09-18T14:15:02.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4e039d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:02.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4e039d640 1 -- 172.21.15.45:0/408619637 >> v1:172.21.15.45:6792/0 conn(0x7ff4d809fba0 legacy=0x7ff4d8146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4de93a640 1 -- 172.21.15.45:0/408619637 reap_dead start 2024-09-18T14:15:02.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4de93a640 1 -- 172.21.15.45:0/408619637 reap_dead start 2024-09-18T14:15:02.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4e039d640 1 -- 172.21.15.45:0/408619637 shutdown_connections 2024-09-18T14:15:02.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4e039d640 1 -- 172.21.15.45:0/408619637 >> 172.21.15.45:0/408619637 conn(0x7ff4d80630b0 msgr2=0x7ff4d80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:02.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4e039d640 1 -- 172.21.15.45:0/408619637 shutdown_connections 2024-09-18T14:15:02.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.785+0000 7ff4e039d640 1 -- 172.21.15.45:0/408619637 wait complete. 2024-09-18T14:15:02.828 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:02.828 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:15:02.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:15:02.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.925+0000 7f337d2d4640 10 monclient: get_monmap_and_config 2024-09-18T14:15:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.925+0000 7f337d2d4640 10 monclient: build_initial_monmap 2024-09-18T14:15:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.925+0000 7f337d2d4640 10 monclient: monmap: 2024-09-18T14:15:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:02.926344+0000 2024-09-18T14:15:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:02.926344+0000 2024-09-18T14:15:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:02.927 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:02.927 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:02.927 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:02.927 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:02.927 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:02.927 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.925+0000 7f337d2d4640 1 Processor -- start 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f337d2d4640 1 -- start start 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f337d2d4640 10 monclient: init 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f337d2d4640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f337d2d4640 10 monclient: _add_conns ranks=[0,1,3,4,5,8,2,6,7] 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f337d2d4640 10 monclient(hunting): picked mon.noname-a con 0x7f3378098370 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f337d2d4640 10 monclient(hunting): picked mon.noname-f con 0x7f3378098740 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f337d2d4640 10 monclient(hunting): picked mon.noname-g con 0x7f3378098b10 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f337d2d4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337805ed70 con 0x7f3378098370 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f337d2d4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33780a3000 con 0x7f3378098740 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f337d2d4640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3378096930 con 0x7f3378098b10 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f337d2d4640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f337d2d4640 10 monclient(hunting): authenticate will time out at 1718.264509s 2024-09-18T14:15:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f3376ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3378098370 0x7f3378132a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56310/0 (socket says 172.21.15.45:56310) 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f3376ffd640 1 -- 172.21.15.45:0/1947696383 learned_addr learned my addr 172.21.15.45:0/1947696383 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f3375ffb640 1 -- 172.21.15.45:0/1947696383 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270175089 0 0) 0x7f337805ed70 con 0x7f3378098370 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f3375ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f3375ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f3375ffb640 10 monclient(hunting): my global_id is 6147 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.926+0000 7f3375ffb640 1 -- 172.21.15.45:0/1947696383 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33600034e0 con 0x7f3378098370 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 1 -- 172.21.15.45:0/1947696383 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2885322669 0 0) 0x7f33600034e0 con 0x7f3378098370 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 1 -- 172.21.15.45:0/1947696383 >> v1:172.21.15.84:6790/0 conn(0x7f3378098b10 legacy=0x7f337813ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 1 -- 172.21.15.45:0/1947696383 >> v1:172.21.15.84:6789/0 conn(0x7f3378098740 legacy=0x7f3378137700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 1 monclient: found mon.noname-a 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:15:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 1 -- 172.21.15.45:0/1947696383 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3378133180 con 0x7f3378098370 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 10 monclient: _finish_auth 0 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f337d2d4640 5 monclient: authenticate success, global_id 6147 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 1 -- 172.21.15.45:0/1947696383 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3368001830 con 0x7f3378098370 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f337d2d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 10 monclient: dump: 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f337d2d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 1 -- 172.21.15.45:0/1947696383 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3368001a10 con 0x7f3378098370 2024-09-18T14:15:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 1 -- 172.21.15.45:0/1947696383 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3368004ba0 con 0x7f3378098370 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f337d2d4640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f337d2d4640 4 set_mon_vals no callback set 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f3375ffb640 10 monclient: dump: 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f337d2d4640 10 monclient: shutdown 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f337d2d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f337d2d4640 1 -- 172.21.15.45:0/1947696383 >> v1:172.21.15.45:6789/0 conn(0x7f3378098370 legacy=0x7f3378132a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f337d2d4640 1 -- 172.21.15.45:0/1947696383 shutdown_connections 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f337d2d4640 1 -- 172.21.15.45:0/1947696383 >> 172.21.15.45:0/1947696383 conn(0x7f3378067b20 msgr2=0x7f33780a2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f337d2d4640 1 -- 172.21.15.45:0/1947696383 shutdown_connections 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.927+0000 7f337d2d4640 1 -- 172.21.15.45:0/1947696383 wait complete. 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.928+0000 7f337d2d4640 10 monclient: build_initial_monmap 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.928+0000 7f337d2d4640 10 monclient: monmap: 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.928+0000 7f337d2d4640 1 Processor -- start 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.928+0000 7f337d2d4640 1 -- start start 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.928+0000 7f337d2d4640 10 monclient: init 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f337d2d4640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f337d2d4640 10 monclient: _add_conns ranks=[8,3,7,2,5,0,4,1,6] 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f337d2d4640 10 monclient(hunting): picked mon.noname-e con 0x7f3378098370 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f337d2d4640 10 monclient(hunting): picked mon.noname-g con 0x7f3378098740 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f337d2d4640 10 monclient(hunting): picked mon.noname-i con 0x7f3378098b10 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f337d2d4640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337812cb90 con 0x7f3378098740 2024-09-18T14:15:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f337d2d4640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337809e300 con 0x7f3378098b10 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f337d2d4640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337812c8b0 con 0x7f3378098370 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f337d2d4640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f337d2d4640 10 monclient(hunting): authenticate will time out at 1718.267303s 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f3376ffd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f3378098370 0x7f33780952e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34134/0 (socket says 172.21.15.45:34134) 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f3376ffd640 1 -- 172.21.15.45:0/1606317496 learned_addr learned my addr 172.21.15.45:0/1606317496 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284195208 0 0) 0x7f337812c8b0 con 0x7f3378098370 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): my global_id is 5858 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33640034e0 con 0x7f3378098370 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526157913 0 0) 0x7f337812cb90 con 0x7f3378098740 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): my global_id is 184629 2024-09-18T14:15:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f337812c8b0 con 0x7f3378098740 2024-09-18T14:15:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046209877 0 0) 0x7f337809e300 con 0x7f3378098b10 2024-09-18T14:15:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): my global_id is 214189 2024-09-18T14:15:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f337812cb90 con 0x7f3378098b10 2024-09-18T14:15:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2881805270 0 0) 0x7f33640034e0 con 0x7f3378098370 2024-09-18T14:15:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f337809e300 con 0x7f3378098370 2024-09-18T14:15:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3368001870 con 0x7f3378098370 2024-09-18T14:15:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:15:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): dump: 2024-09-18T14:15:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.947 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 425099431 0 0) 0x7f337812c8b0 con 0x7f3378098740 2024-09-18T14:15:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.929+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f33640034e0 con 0x7f3378098740 2024-09-18T14:15:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f336c001a60 con 0x7f3378098740 2024-09-18T14:15:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 10 monclient(hunting): dump: 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1451692302 0 0) 0x7f337812cb90 con 0x7f3378098b10 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f337812c8b0 con 0x7f3378098b10 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1813220356 0 0) 0x7f337809e300 con 0x7f3378098370 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 >> v1:172.21.15.84:6792/0 conn(0x7f3378098b10 legacy=0x7f3378149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 >> v1:172.21.15.84:6790/0 conn(0x7f3378098740 legacy=0x7f33780959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 1 monclient: found mon.e 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337812d860 con 0x7f3378098370 2024-09-18T14:15:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 10 monclient: _finish_auth 0 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f337d2d4640 5 monclient: authenticate success, global_id 5858 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f337d2d4640 10 monclient: _renew_subs 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f337d2d4640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f337d2d4640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337812af30 con 0x7f3378098370 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f337d2d4640 10 monclient: _renew_subs 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f337d2d4640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f337d2d4640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f337812a1a0 con 0x7f3378098370 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3368004b90 con 0x7f3378098370 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f33680051a0 con 0x7f3378098370 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f33757fa640 4 set_mon_vals no callback set 2024-09-18T14:15:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f335effd640 10 monclient: dump: 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.951 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f337d2d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.930+0000 7f337d2d4640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3358006580 con 0x7f3358002830 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f3368010170 con 0x7f3378098370 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(737..737 src has 529..737) v4 ==== 3971+0+0 (unknown 1184779458 0 0) 0x7f3368046bf0 con 0x7f3378098370 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762756417 0 0) 0x7f3358006580 con 0x7f3358002830 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 10 monclient(hunting): my global_id is 5867 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f337809e300 con 0x7f3358002830 2024-09-18T14:15:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f33777fe640 10 monclient: get_auth_request con 0x7f336403a990 auth_method 0 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 523986693 0 0) 0x7f337809e300 con 0x7f3358002830 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3358006580 con 0x7f3358002830 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f336c002290 con 0x7f3358002830 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 10 monclient: dump: 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.955 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1365504366 0 0) 0x7f3358006580 con 0x7f3358002830 2024-09-18T14:15:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.931+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f335800a0a0 con 0x7f3358002830 2024-09-18T14:15:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.932+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f336c0026f0 con 0x7f3358002830 2024-09-18T14:15:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.932+0000 7f335effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.932+0000 7f335effd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.932+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 >> v1:172.21.15.45:6793/0 conn(0x7f3358002830 legacy=0x7f3358004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.932+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 >> v1:172.21.15.45:6793/0 conn(0x7f3358002830 legacy=0x7f3358004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.937+0000 7f337d2d4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.937+0000 7f337d2d4640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f33580074f0 con 0x7f33580068a0 2024-09-18T14:15:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881847563 0 0) 0x7f33580074f0 con 0x7f33580068a0 2024-09-18T14:15:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 10 monclient(hunting): my global_id is 5876 2024-09-18T14:15:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3358006580 con 0x7f33580068a0 2024-09-18T14:15:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4165278030 0 0) 0x7f3358006580 con 0x7f33580068a0 2024-09-18T14:15:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f33580074f0 con 0x7f33580068a0 2024-09-18T14:15:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f336c002d50 con 0x7f33580068a0 2024-09-18T14:15:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 10 monclient: dump: 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:02.959 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 251371824 0 0) 0x7f33580074f0 con 0x7f33580068a0 2024-09-18T14:15:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3358008750 con 0x7f33580068a0 2024-09-18T14:15:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7389 (unknown 0 0 3841120298) 0x7f336c003480 con 0x7f33580068a0 2024-09-18T14:15:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 >> v1:172.21.15.45:6793/0 conn(0x7f33580068a0 legacy=0x7f3358006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.938+0000 7f335effd640 1 -- 172.21.15.45:0/1606317496 >> v1:172.21.15.45:6793/0 conn(0x7f33580068a0 legacy=0x7f3358006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:02.959 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:15:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:02.960 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.962 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:02.963 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:02.964 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:02.968 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.969 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.971 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:02.972 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.973 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.975 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:02.976 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:02.976 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:02.976 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:02.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.976 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:02.976 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:02.976 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:02.976 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.976 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.977 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:02.978 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:02.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.939+0000 7f337d2d4640 1 -- 172.21.15.45:0/1606317496 >> v1:172.21.15.84:6812/31478 conn(0x7f336403a990 legacy=0x7f336403ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.939+0000 7f337d2d4640 10 monclient: shutdown 2024-09-18T14:15:02.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.939+0000 7f337d2d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:02.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.939+0000 7f337d2d4640 1 -- 172.21.15.45:0/1606317496 >> v1:172.21.15.45:6793/0 conn(0x7f3378098370 legacy=0x7f33780952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:02.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.939+0000 7f33777fe640 1 -- 172.21.15.45:0/1606317496 reap_dead start 2024-09-18T14:15:02.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.939+0000 7f33777fe640 1 -- 172.21.15.45:0/1606317496 reap_dead start 2024-09-18T14:15:02.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.939+0000 7f337d2d4640 1 -- 172.21.15.45:0/1606317496 shutdown_connections 2024-09-18T14:15:02.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.939+0000 7f337d2d4640 1 -- 172.21.15.45:0/1606317496 >> 172.21.15.45:0/1606317496 conn(0x7f3378067b20 msgr2=0x7f337809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:02.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.939+0000 7f337d2d4640 1 -- 172.21.15.45:0/1606317496 shutdown_connections 2024-09-18T14:15:02.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:02.939+0000 7f337d2d4640 1 -- 172.21.15.45:0/1606317496 wait complete. 2024-09-18T14:15:02.981 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:02.981 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:15:02.981 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:15:03.001 INFO:tasks.rados.rados.0.smithi084.stdout:3357: done (3 left) 2024-09-18T14:15:03.001 INFO:tasks.rados.rados.0.smithi084.stdout:3358: read oid 49 snap -1 2024-09-18T14:15:03.001 INFO:tasks.rados.rados.0.smithi084.stdout:3358: expect (ObjNum 1020 snap 326 seq_num 1020) 2024-09-18T14:15:03.001 INFO:tasks.rados.rados.0.smithi084.stdout:3359: read oid 32 snap -1 2024-09-18T14:15:03.001 INFO:tasks.rados.rados.0.smithi084.stdout:3359: expect (ObjNum 1130 snap 362 seq_num 1130) 2024-09-18T14:15:03.001 INFO:tasks.rados.rados.0.smithi084.stdout:3360: read oid 41 snap 360 2024-09-18T14:15:03.001 INFO:tasks.rados.rados.0.smithi084.stdout:3360: expect (ObjNum 961 snap 306 seq_num 961) 2024-09-18T14:15:03.001 INFO:tasks.rados.rados.0.smithi084.stdout:3361: copy_from oid 3 from oid 22 current snap is 368 2024-09-18T14:15:03.001 INFO:tasks.rados.rados.0.smithi084.stdout:3354: finishing write tid 1 to smithi08434663-13 2024-09-18T14:15:03.001 INFO:tasks.rados.rados.0.smithi084.stdout:3354: finishing write tid 2 to smithi08434663-13 2024-09-18T14:15:03.001 INFO:tasks.rados.rados.0.smithi084.stdout:3355: finishing rollback tid 1 to smithi08434663-50 2024-09-18T14:15:03.002 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 1753 (ObjNum 129 snap 0 seq_num 2495383712) dirty dne 2024-09-18T14:15:03.002 INFO:tasks.rados.rados.0.smithi084.stdout:3354: finishing write tid 3 to smithi08434663-13 2024-09-18T14:15:03.002 INFO:tasks.rados.rados.0.smithi084.stdout:3354: finishing write tid 4 to smithi08434663-13 2024-09-18T14:15:03.002 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 1560 (ObjNum 1142 snap 368 seq_num 1142) dirty exists 2024-09-18T14:15:03.002 INFO:tasks.rados.rados.0.smithi084.stdout:3354: left oid 13 (ObjNum 1142 snap 368 seq_num 1142) 2024-09-18T14:15:03.002 INFO:tasks.rados.rados.0.smithi084.stdout:3356: finishing copy_from racing read to smithi08434663-27 2024-09-18T14:15:03.002 INFO:tasks.rados.rados.0.smithi084.stdout:3356: finishing copy_from to smithi08434663-27 2024-09-18T14:15:03.002 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1330 (ObjNum 1127 snap 361 seq_num 1127) dirty exists 2024-09-18T14:15:03.002 INFO:tasks.rados.rados.0.smithi084.stdout:3354: done (6 left) 2024-09-18T14:15:03.002 INFO:tasks.rados.rados.0.smithi084.stdout:3355: done (5 left) 2024-09-18T14:15:03.002 INFO:tasks.rados.rados.0.smithi084.stdout:3356: done (4 left) 2024-09-18T14:15:03.002 INFO:tasks.rados.rados.0.smithi084.stdout:3362: rollback oid 11 current snap is 368 2024-09-18T14:15:03.003 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 11 to 308 2024-09-18T14:15:03.003 INFO:tasks.rados.rados.0.smithi084.stdout:3363: read oid 15 snap -1 2024-09-18T14:15:03.003 INFO:tasks.rados.rados.0.smithi084.stdout:3363: expect deleted 2024-09-18T14:15:03.003 INFO:tasks.rados.rados.0.smithi084.stdout:3364: delete oid 48 current snap is 368 2024-09-18T14:15:03.010 INFO:tasks.rados.rados.0.smithi084.stdout:3361: finishing copy_from to smithi08434663-3 2024-09-18T14:15:03.010 INFO:tasks.rados.rados.0.smithi084.stdout:3361: got expected ENOENT (src dne) 2024-09-18T14:15:03.011 INFO:tasks.rados.rados.0.smithi084.stdout:3361: finishing copy_from racing read to smithi08434663-3 2024-09-18T14:15:03.014 INFO:tasks.rados.rados.0.smithi084.stdout:3361: done (6 left) 2024-09-18T14:15:03.014 INFO:tasks.rados.rados.0.smithi084.stdout:3364: done (5 left) 2024-09-18T14:15:03.014 INFO:tasks.rados.rados.0.smithi084.stdout:3365: snap_create 2024-09-18T14:15:03.015 INFO:tasks.rados.rados.0.smithi084.stdout:3360: expect (ObjNum 961 snap 306 seq_num 961) 2024-09-18T14:15:03.089 INFO:tasks.rados.rados.0.smithi084.stdout:3359: expect (ObjNum 1130 snap 362 seq_num 1130) 2024-09-18T14:15:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.094+0000 7ff7aa4ef640 10 monclient: get_monmap_and_config 2024-09-18T14:15:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.094+0000 7ff7aa4ef640 10 monclient: build_initial_monmap 2024-09-18T14:15:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.094+0000 7ff7aa4ef640 10 monclient: monmap: 2024-09-18T14:15:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:03.095915+0000 2024-09-18T14:15:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:03.095915+0000 2024-09-18T14:15:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:03.096 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.095+0000 7ff7aa4ef640 1 Processor -- start 2024-09-18T14:15:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.095+0000 7ff7aa4ef640 1 -- start start 2024-09-18T14:15:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.095+0000 7ff7aa4ef640 10 monclient: init 2024-09-18T14:15:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7aa4ef640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7aa4ef640 10 monclient: _add_conns ranks=[6,3,4,7,8,1,5,0,2] 2024-09-18T14:15:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7aa4ef640 10 monclient(hunting): picked mon.noname-d con 0x7ff7a409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7aa4ef640 10 monclient(hunting): picked mon.noname-g con 0x7ff7a409fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7aa4ef640 10 monclient(hunting): picked mon.noname-c con 0x7ff7a409ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7aa4ef640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a405ed70 con 0x7ff7a409fba0 2024-09-18T14:15:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7aa4ef640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a409dfc0 con 0x7ff7a409ff70 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7aa4ef640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a409dce0 con 0x7ff7a409f7d0 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7aa4ef640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7aa4ef640 10 monclient(hunting): authenticate will time out at 1718.434264s 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a3fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7ff7a409f7d0 0x7ff7a4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43348/0 (socket says 172.21.15.45:43348) 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a3fff640 1 -- 172.21.15.45:0/2960029523 learned_addr learned my addr 172.21.15.45:0/2960029523 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a8a8c640 1 --1- 172.21.15.45:0/2960029523 >> v1:172.21.15.45:6791/0 conn(0x7ff7a409ff70 0x7ff7a413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39594/0 (socket says 172.21.15.45:39594) 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936195428 0 0) 0x7ff7a409dce0 con 0x7ff7a409f7d0 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient(hunting): my global_id is 184695 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7900034e0 con 0x7ff7a409f7d0 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701026813 0 0) 0x7ff7a409dfc0 con 0x7ff7a409ff70 2024-09-18T14:15:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient(hunting): my global_id is 164893 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a409dce0 con 0x7ff7a409ff70 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352868080 0 0) 0x7ff7a405ed70 con 0x7ff7a409fba0 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient(hunting): my global_id is 184638 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a409dfc0 con 0x7ff7a409fba0 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1225758048 0 0) 0x7ff7900034e0 con 0x7ff7a409f7d0 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 >> v1:172.21.15.45:6791/0 conn(0x7ff7a409ff70 legacy=0x7ff7a413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 >> v1:172.21.15.84:6790/0 conn(0x7ff7a409fba0 legacy=0x7ff7a4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 1 monclient: found mon.noname-d 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a40a1c00 con 0x7ff7a409f7d0 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient: _finish_auth 0 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7aa4ef640 5 monclient: authenticate success, global_id 184695 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff794001830 con 0x7ff7a409f7d0 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.096+0000 7ff7a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7a2ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7a2ffd640 10 monclient: dump: 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7aa4ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff794001990 con 0x7ff7a409f7d0 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7a2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7aa4ef640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7aa4ef640 4 set_mon_vals no callback set 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7a2ffd640 1 -- 172.21.15.45:0/2960029523 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff794004ba0 con 0x7ff7a409f7d0 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7a2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7a2ffd640 10 monclient: dump: 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7aa4ef640 10 monclient: shutdown 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7aa4ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2960029523 >> v1:172.21.15.45:6792/0 conn(0x7ff7a409f7d0 legacy=0x7ff7a4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2960029523 shutdown_connections 2024-09-18T14:15:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2960029523 >> 172.21.15.45:0/2960029523 conn(0x7ff7a40630b0 msgr2=0x7ff7a40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2960029523 shutdown_connections 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.097+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2960029523 wait complete. 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.098+0000 7ff7aa4ef640 10 monclient: build_initial_monmap 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.098+0000 7ff7aa4ef640 10 monclient: monmap: 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:03.113 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.098+0000 7ff7aa4ef640 1 Processor -- start 2024-09-18T14:15:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.098+0000 7ff7aa4ef640 1 -- start start 2024-09-18T14:15:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.098+0000 7ff7aa4ef640 10 monclient: init 2024-09-18T14:15:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7aa4ef640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7aa4ef640 10 monclient: _add_conns ranks=[1,7,8,4,3,5,0,6,2] 2024-09-18T14:15:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7aa4ef640 10 monclient(hunting): picked mon.noname-f con 0x7ff7a409f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7aa4ef640 10 monclient(hunting): picked mon.noname-i con 0x7ff7a409fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7aa4ef640 10 monclient(hunting): picked mon.noname-e con 0x7ff7a409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7aa4ef640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a40a2560 con 0x7ff7a409f7d0 2024-09-18T14:15:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7aa4ef640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a41425d0 con 0x7ff7a409fba0 2024-09-18T14:15:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7aa4ef640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a411fda0 con 0x7ff7a409ff70 2024-09-18T14:15:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7aa4ef640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7aa4ef640 10 monclient(hunting): authenticate will time out at 1718.437315s 2024-09-18T14:15:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7a8a8c640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7ff7a409ff70 0x7ff7a414a170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34170/0 (socket says 172.21.15.45:34170) 2024-09-18T14:15:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7a8a8c640 1 -- 172.21.15.45:0/2360397396 learned_addr learned my addr 172.21.15.45:0/2360397396 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff7a37fe640 1 --1- 172.21.15.45:0/2360397396 >> v1:172.21.15.84:6792/0 conn(0x7ff7a409fba0 0x7ff7a4146a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:34786/0 (socket says 172.21.15.45:34786) 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494918384 0 0) 0x7ff7a411fda0 con 0x7ff7a409ff70 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): my global_id is 5885 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7840034e0 con 0x7ff7a409ff70 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617832168 0 0) 0x7ff7a40a2560 con 0x7ff7a409f7d0 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): my global_id is 214192 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a411fda0 con 0x7ff7a409f7d0 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105157854 0 0) 0x7ff7a41425d0 con 0x7ff7a409fba0 2024-09-18T14:15:03.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): my global_id is 214198 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a40a2560 con 0x7ff7a409fba0 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1470225757 0 0) 0x7ff7840034e0 con 0x7ff7a409ff70 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7a41425d0 con 0x7ff7a409ff70 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff79c001e00 con 0x7ff7a409ff70 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): dump: 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3470392741 0 0) 0x7ff7a411fda0 con 0x7ff7a409f7d0 2024-09-18T14:15:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7840034e0 con 0x7ff7a409f7d0 2024-09-18T14:15:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff794001770 con 0x7ff7a409f7d0 2024-09-18T14:15:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 10 monclient(hunting): dump: 2024-09-18T14:15:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.099+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4256678875 0 0) 0x7ff7a40a2560 con 0x7ff7a409fba0 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7a411fda0 con 0x7ff7a409fba0 2024-09-18T14:15:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff78c001c50 con 0x7ff7a409fba0 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 10 monclient(hunting): dump: 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 941566804 0 0) 0x7ff7a41425d0 con 0x7ff7a409ff70 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 >> v1:172.21.15.84:6792/0 conn(0x7ff7a409fba0 legacy=0x7ff7a4146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 >> v1:172.21.15.84:6789/0 conn(0x7ff7a409f7d0 legacy=0x7ff7a4131780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 1 monclient: found mon.e 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a41218d0 con 0x7ff7a409ff70 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 10 monclient: _finish_auth 0 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff7aa4ef640 5 monclient: authenticate success, global_id 5885 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff7aa4ef640 10 monclient: _renew_subs 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff7aa4ef640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a4121bb0 con 0x7ff7a409ff70 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff7aa4ef640 10 monclient: _renew_subs 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff7aa4ef640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a4131e90 con 0x7ff7a409ff70 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff79c0020e0 con 0x7ff7a409ff70 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff79c005190 con 0x7ff7a409ff70 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff7a27fc640 4 set_mon_vals no callback set 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff79bfff640 10 monclient: dump: 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff7aa4ef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.100+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff788006580 con 0x7ff788002830 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff79c0101e0 con 0x7ff7a409ff70 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(738..738 src has 529..738) v4 ==== 3987+0+0 (unknown 2673133721 0 0) 0x7ff79c046f40 con 0x7ff7a409ff70 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff7a37fe640 10 monclient: get_auth_request con 0x7ff78403a9b0 auth_method 0 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973665772 0 0) 0x7ff788006580 con 0x7ff788002830 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 10 monclient(hunting): my global_id is 214201 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a41425d0 con 0x7ff788002830 2024-09-18T14:15:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1594323356 0 0) 0x7ff7a41425d0 con 0x7ff788002830 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff788006580 con 0x7ff788002830 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff7940051a0 con 0x7ff788002830 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 10 monclient: dump: 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3626153885 0 0) 0x7ff788006580 con 0x7ff788002830 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.101+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff78800a0a0 con 0x7ff788002830 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.102+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff794005440 con 0x7ff788002830 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.102+0000 7ff79bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.102+0000 7ff79bfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.102+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 >> v1:172.21.15.84:6789/0 conn(0x7ff788002830 legacy=0x7ff788004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.102+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 >> v1:172.21.15.84:6789/0 conn(0x7ff788002830 legacy=0x7ff788004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff7aa4ef640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff788009660 con 0x7ff788009040 2024-09-18T14:15:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375018752 0 0) 0x7ff788009660 con 0x7ff788009040 2024-09-18T14:15:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 10 monclient(hunting): my global_id is 214210 2024-09-18T14:15:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff788006580 con 0x7ff788009040 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 652618706 0 0) 0x7ff788006580 con 0x7ff788009040 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff788009660 con 0x7ff788009040 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff794005c20 con 0x7ff788009040 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 10 monclient: dump: 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1909456516 0 0) 0x7ff788009660 con 0x7ff788009040 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.108+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff788009920 con 0x7ff788009040 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 2280621893) 0x7ff794007cf0 con 0x7ff788009040 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff79bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff79bfff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 >> v1:172.21.15.84:6789/0 conn(0x7ff788009040 legacy=0x7ff788008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff79bfff640 1 -- 172.21.15.45:0/2360397396 >> v1:172.21.15.84:6789/0 conn(0x7ff788009040 legacy=0x7ff788008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:03.131 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:15:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:03.151 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:03.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2360397396 >> v1:172.21.15.84:6812/31478 conn(0x7ff78403a9b0 legacy=0x7ff78403ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff7aa4ef640 10 monclient: shutdown 2024-09-18T14:15:03.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff7aa4ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:03.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2360397396 >> v1:172.21.15.45:6793/0 conn(0x7ff7a409ff70 legacy=0x7ff7a414a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff7a8a8c640 1 -- 172.21.15.45:0/2360397396 reap_dead start 2024-09-18T14:15:03.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff7a8a8c640 1 -- 172.21.15.45:0/2360397396 reap_dead start 2024-09-18T14:15:03.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2360397396 shutdown_connections 2024-09-18T14:15:03.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2360397396 >> 172.21.15.45:0/2360397396 conn(0x7ff7a40630b0 msgr2=0x7ff7a40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:03.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2360397396 shutdown_connections 2024-09-18T14:15:03.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.109+0000 7ff7aa4ef640 1 -- 172.21.15.45:0/2360397396 wait complete. 2024-09-18T14:15:03.154 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:03.154 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:15:03.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:15:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.250+0000 7f2fe4d9d640 10 monclient: get_monmap_and_config 2024-09-18T14:15:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.250+0000 7f2fe4d9d640 10 monclient: build_initial_monmap 2024-09-18T14:15:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.250+0000 7f2fe4d9d640 10 monclient: monmap: 2024-09-18T14:15:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:03.251759+0000 2024-09-18T14:15:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:03.251759+0000 2024-09-18T14:15:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:03.252 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.251+0000 7f2fe4d9d640 1 Processor -- start 2024-09-18T14:15:03.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.251+0000 7f2fe4d9d640 1 -- start start 2024-09-18T14:15:03.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.251+0000 7f2fe4d9d640 10 monclient: init 2024-09-18T14:15:03.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.251+0000 7f2fe4d9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:03.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 10 monclient: _add_conns ranks=[6,1,0,8,5,4,2,3,7] 2024-09-18T14:15:03.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 10 monclient(hunting): picked mon.noname-d con 0x7f2fe009f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 10 monclient(hunting): picked mon.noname-f con 0x7f2fe009fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 10 monclient(hunting): picked mon.noname-a con 0x7f2fe009ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe005ed70 con 0x7f2fe009ff70 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe009dfc0 con 0x7f2fe009fba0 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe009dce0 con 0x7f2fe009f7d0 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 10 monclient(hunting): authenticate will time out at 1718.590100s 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fded9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2fe009ff70 0x7f2fe013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56324/0 (socket says 172.21.15.45:56324) 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fded9d640 1 -- 172.21.15.45:0/2125782465 learned_addr learned my addr 172.21.15.45:0/2125782465 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fde59c640 1 --1- 172.21.15.45:0/2125782465 >> v1:172.21.15.45:6792/0 conn(0x7f2fe009f7d0 0x7f2fe0132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43350/0 (socket says 172.21.15.45:43350) 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281060387 0 0) 0x7f2fe005ed70 con 0x7f2fe009ff70 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient(hunting): my global_id is 6156 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fcc0034e0 con 0x7f2fe009ff70 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064070794 0 0) 0x7f2fe009dfc0 con 0x7f2fe009fba0 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient(hunting): my global_id is 214219 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe005ed70 con 0x7f2fe009fba0 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844791541 0 0) 0x7f2fe009dce0 con 0x7f2fe009f7d0 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient(hunting): my global_id is 184704 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe009dfc0 con 0x7f2fe009f7d0 2024-09-18T14:15:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2068611913 0 0) 0x7f2fe005ed70 con 0x7f2fe009fba0 2024-09-18T14:15:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 >> v1:172.21.15.45:6792/0 conn(0x7f2fe009f7d0 legacy=0x7f2fe0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 >> v1:172.21.15.45:6789/0 conn(0x7f2fe009ff70 legacy=0x7f2fe013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient: _finish_hunting 0 2024-09-18T14:15:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 monclient: found mon.noname-f 2024-09-18T14:15:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T14:15:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00a1c00 con 0x7f2fe009fba0 2024-09-18T14:15:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient: _finish_auth 0 2024-09-18T14:15:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2fc8001810 con 0x7f2fe009fba0 2024-09-18T14:15:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 5 monclient: authenticate success, global_id 214219 2024-09-18T14:15:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:15:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient: dump: 2024-09-18T14:15:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2fc8001970 con 0x7f2fe009fba0 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fe4d9d640 4 set_mon_vals no callback set 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 1 -- 172.21.15.45:0/2125782465 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2fc8001f20 con 0x7f2fe009fba0 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.252+0000 7f2fdd59a640 10 monclient: dump: 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.253+0000 7f2fe4d9d640 10 monclient: shutdown 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.253+0000 7f2fe4d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.253+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2125782465 >> v1:172.21.15.84:6789/0 conn(0x7f2fe009fba0 legacy=0x7f2fe0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.253+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2125782465 shutdown_connections 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.253+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2125782465 >> 172.21.15.45:0/2125782465 conn(0x7f2fe00630b0 msgr2=0x7f2fe00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.253+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2125782465 shutdown_connections 2024-09-18T14:15:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.253+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2125782465 wait complete. 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.253+0000 7f2fe4d9d640 10 monclient: build_initial_monmap 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 10 monclient: monmap: 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 1 Processor -- start 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 1 -- start start 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 10 monclient: init 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 10 monclient: _add_conns ranks=[1,8,0,2,5,3,7,6,4] 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 10 monclient(hunting): picked mon.noname-f con 0x7f2fe009f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 10 monclient(hunting): picked mon.noname-e con 0x7f2fe009fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 10 monclient(hunting): picked mon.noname-a con 0x7f2fe009ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe011ecd0 con 0x7f2fe009ff70 2024-09-18T14:15:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe0136ea0 con 0x7f2fe009f7d0 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe011e390 con 0x7f2fe009fba0 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.254+0000 7f2fe4d9d640 10 monclient(hunting): authenticate will time out at 1718.592935s 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fded9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2fe009ff70 0x7f2fe014a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56340/0 (socket says 172.21.15.45:56340) 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fded9d640 1 -- 172.21.15.45:0/2300423999 learned_addr learned my addr 172.21.15.45:0/2300423999 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247886501 0 0) 0x7f2fe011ecd0 con 0x7f2fe009ff70 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): my global_id is 6165 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fc00034e0 con 0x7f2fe009ff70 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919075682 0 0) 0x7f2fe011e390 con 0x7f2fe009fba0 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): my global_id is 5894 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe011ecd0 con 0x7f2fe009fba0 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334688707 0 0) 0x7f2fe0136ea0 con 0x7f2fe009f7d0 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): my global_id is 214228 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe011e390 con 0x7f2fe009f7d0 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3958518987 0 0) 0x7f2fc00034e0 con 0x7f2fe009ff70 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2fe0136ea0 con 0x7f2fe009ff70 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2fd0001a60 con 0x7f2fe009ff70 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:15:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): dump: 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2576564105 0 0) 0x7f2fe011e390 con 0x7f2fe009f7d0 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2fc00034e0 con 0x7f2fe009f7d0 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 637755253 0 0) 0x7f2fe011ecd0 con 0x7f2fe009fba0 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2fe011e390 con 0x7f2fe009fba0 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2fd8001d20 con 0x7f2fe009f7d0 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): dump: 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.275 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2fc8001e20 con 0x7f2fe009fba0 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient(hunting): dump: 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3066443813 0 0) 0x7f2fe0136ea0 con 0x7f2fe009ff70 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 >> v1:172.21.15.45:6793/0 conn(0x7f2fe009fba0 legacy=0x7f2fe0146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 >> v1:172.21.15.84:6789/0 conn(0x7f2fe009f7d0 legacy=0x7f2fe0131560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient: _finish_hunting 0 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 monclient: found mon.a 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe011d720 con 0x7f2fe009ff70 2024-09-18T14:15:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient: _finish_auth 0 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fe4d9d640 5 monclient: authenticate success, global_id 6165 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fe4d9d640 10 monclient: _renew_subs 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fe4d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe011e680 con 0x7f2fe009ff70 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fe4d9d640 10 monclient: _renew_subs 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fe4d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe009b620 con 0x7f2fe009ff70 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2fd0001c60 con 0x7f2fe009ff70 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2fd0004d90 con 0x7f2fe009ff70 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fdcd99640 4 set_mon_vals no callback set 2024-09-18T14:15:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.255+0000 7f2fd67fc640 10 monclient: dump: 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.256+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2fd000fde0 con 0x7f2fe009ff70 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.256+0000 7f2fe4d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.256+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2fc4006580 con 0x7f2fc4002830 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.256+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(738..738 src has 529..738) v4 ==== 3987+0+0 (unknown 2673133721 0 0) 0x7f2fd0046a70 con 0x7f2fe009ff70 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.256+0000 7f2fddd9b640 10 monclient: get_auth_request con 0x7f2fc003ab00 auth_method 0 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072379249 0 0) 0x7f2fc4006580 con 0x7f2fc4002830 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 10 monclient(hunting): my global_id is 184647 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe0136ea0 con 0x7f2fc4002830 2024-09-18T14:15:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1897024465 0 0) 0x7f2fe0136ea0 con 0x7f2fc4002830 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2fc4006580 con 0x7f2fc4002830 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2fd8004e60 con 0x7f2fc4002830 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 10 monclient: dump: 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2184926884 0 0) 0x7f2fc4006580 con 0x7f2fc4002830 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2fc400a0a0 con 0x7f2fc4002830 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2fd8005b50 con 0x7f2fc4002830 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 >> v1:172.21.15.84:6790/0 conn(0x7f2fc4002830 legacy=0x7f2fc4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.257+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 >> v1:172.21.15.84:6790/0 conn(0x7f2fc4002830 legacy=0x7f2fc4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.263+0000 7f2fe4d9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.263+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2fc4009610 con 0x7f2fc4008ff0 2024-09-18T14:15:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.263+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015359147 0 0) 0x7f2fc4009610 con 0x7f2fc4008ff0 2024-09-18T14:15:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.263+0000 7f2fd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.263+0000 7f2fd67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.263+0000 7f2fd67fc640 10 monclient(hunting): my global_id is 184656 2024-09-18T14:15:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.263+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fc4006580 con 0x7f2fc4008ff0 2024-09-18T14:15:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3718113351 0 0) 0x7f2fc4006580 con 0x7f2fc4008ff0 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2fc4009610 con 0x7f2fc4008ff0 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2fd8005870 con 0x7f2fc4008ff0 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 10 monclient: dump: 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2741508899 0 0) 0x7f2fc4009610 con 0x7f2fc4008ff0 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2fc40098d0 con 0x7f2fc4008ff0 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 2954748125) 0x7f2fd8007d90 con 0x7f2fc4008ff0 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 >> v1:172.21.15.84:6790/0 conn(0x7f2fc4008ff0 legacy=0x7f2fc4008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.264+0000 7f2fd67fc640 1 -- 172.21.15.45:0/2300423999 >> v1:172.21.15.84:6790/0 conn(0x7f2fc4008ff0 legacy=0x7f2fc4008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:03.287 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:03.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:03.287 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:03.287 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:03.287 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:03.287 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:03.287 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:03.287 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:03.287 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:03.287 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:03.288 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:03.289 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:03.290 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:03.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:03.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:03.291 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:03.291 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:03.291 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:03.291 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:03.291 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:03.291 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:03.291 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:03.291 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:03.291 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:03.291 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:03.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.293 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.294 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:03.295 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:03.295 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.295 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.295 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.295 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:03.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:03.295 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.295 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.296 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:03.297 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:03.297 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:03.297 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.297 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.298 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:03.298 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:03.298 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.298 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.298 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.298 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:03.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.299 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.300 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:03.301 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:03.301 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.301 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.301 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.301 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:03.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:03.301 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.301 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:03.302 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.303 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.304 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:03.305 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:03.306 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:15:03.307 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.307 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.307 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:03.307 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.265+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2300423999 >> v1:172.21.15.84:6812/31478 conn(0x7f2fc003ab00 legacy=0x7f2fc003cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.265+0000 7f2fe4d9d640 10 monclient: shutdown 2024-09-18T14:15:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.265+0000 7f2fe4d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.265+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2300423999 >> v1:172.21.15.45:6789/0 conn(0x7f2fe009ff70 legacy=0x7f2fe014a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.265+0000 7f2fded9d640 1 -- 172.21.15.45:0/2300423999 reap_dead start 2024-09-18T14:15:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.265+0000 7f2fded9d640 1 -- 172.21.15.45:0/2300423999 reap_dead start 2024-09-18T14:15:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.265+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2300423999 shutdown_connections 2024-09-18T14:15:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.265+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2300423999 >> 172.21.15.45:0/2300423999 conn(0x7f2fe00630b0 msgr2=0x7f2fe00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.265+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2300423999 shutdown_connections 2024-09-18T14:15:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.265+0000 7f2fe4d9d640 1 -- 172.21.15.45:0/2300423999 wait complete. 2024-09-18T14:15:03.309 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:03.309 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:15:03.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:15:03.374 INFO:tasks.rados.rados.0.smithi084.stdout:3362: finishing rollback tid 1 to smithi08434663-11 2024-09-18T14:15:03.374 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1291 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-18T14:15:03.374 INFO:tasks.rados.rados.0.smithi084.stdout:3358: expect (ObjNum 1020 snap 326 seq_num 1020) 2024-09-18T14:15:03.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.406+0000 7f3189f5d640 10 monclient: get_monmap_and_config 2024-09-18T14:15:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.406+0000 7f3189f5d640 10 monclient: build_initial_monmap 2024-09-18T14:15:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.406+0000 7f3189f5d640 10 monclient: monmap: 2024-09-18T14:15:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:03.407487+0000 2024-09-18T14:15:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:03.407487+0000 2024-09-18T14:15:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:03.408 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 1 Processor -- start 2024-09-18T14:15:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 1 -- start start 2024-09-18T14:15:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 10 monclient: init 2024-09-18T14:15:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 10 monclient: _add_conns ranks=[6,3,5,4,0,1,7,2,8] 2024-09-18T14:15:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 10 monclient(hunting): picked mon.noname-d con 0x7f318409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 10 monclient(hunting): picked mon.noname-g con 0x7f318409fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 10 monclient(hunting): picked mon.noname-h con 0x7f318409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f318405ed70 con 0x7f318409fba0 2024-09-18T14:15:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f318409dfc0 con 0x7f318409ff70 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f318409dce0 con 0x7f318409f7d0 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f3189f5d640 10 monclient(hunting): authenticate will time out at 1718.745870s 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f31837fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f318409f7d0 0x7f3184132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43358/0 (socket says 172.21.15.45:43358) 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.407+0000 7f31837fe640 1 -- 172.21.15.45:0/1546749085 learned_addr learned my addr 172.21.15.45:0/1546749085 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947331238 0 0) 0x7f318409dce0 con 0x7f318409f7d0 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient(hunting): my global_id is 184713 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f316c0034e0 con 0x7f318409f7d0 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823447470 0 0) 0x7f318405ed70 con 0x7f318409fba0 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient(hunting): my global_id is 184665 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f318409dce0 con 0x7f318409fba0 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230332962 0 0) 0x7f318409dfc0 con 0x7f318409ff70 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient(hunting): my global_id is 184586 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f318405ed70 con 0x7f318409ff70 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3606987498 0 0) 0x7f318409dce0 con 0x7f318409fba0 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 >> v1:172.21.15.45:6792/0 conn(0x7f318409f7d0 legacy=0x7f3184132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 >> v1:172.21.15.84:6791/0 conn(0x7f318409ff70 legacy=0x7f318413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient: _finish_hunting 0 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 monclient: found mon.noname-g 2024-09-18T14:15:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T14:15:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31840a1c00 con 0x7f318409fba0 2024-09-18T14:15:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient: _finish_auth 0 2024-09-18T14:15:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3174001810 con 0x7f318409fba0 2024-09-18T14:15:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f3189f5d640 5 monclient: authenticate success, global_id 184665 2024-09-18T14:15:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f3189f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:15:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient: dump: 2024-09-18T14:15:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.421 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f3189f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f31740019f0 con 0x7f318409fba0 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f3189f5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 1 -- 172.21.15.45:0/1546749085 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3174002240 con 0x7f318409fba0 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f3189f5d640 4 set_mon_vals no callback set 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.408+0000 7f31827fc640 10 monclient: dump: 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.422 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.409+0000 7f3189f5d640 10 monclient: shutdown 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.409+0000 7f3189f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:03.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.409+0000 7f3189f5d640 1 -- 172.21.15.45:0/1546749085 >> v1:172.21.15.84:6790/0 conn(0x7f318409fba0 legacy=0x7f3184137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.409+0000 7f3189f5d640 1 -- 172.21.15.45:0/1546749085 shutdown_connections 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.409+0000 7f3189f5d640 1 -- 172.21.15.45:0/1546749085 >> 172.21.15.45:0/1546749085 conn(0x7f31840630b0 msgr2=0x7f31840634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.409+0000 7f3189f5d640 1 -- 172.21.15.45:0/1546749085 shutdown_connections 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.409+0000 7f3189f5d640 1 -- 172.21.15.45:0/1546749085 wait complete. 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.409+0000 7f3189f5d640 10 monclient: build_initial_monmap 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.409+0000 7f3189f5d640 10 monclient: monmap: 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:03.424 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 1 Processor -- start 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 1 -- start start 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 10 monclient: init 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:03.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 10 monclient: _add_conns ranks=[6,4,8,0,3,2,7,5,1] 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 10 monclient(hunting): picked mon.noname-d con 0x7f318409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 10 monclient(hunting): picked mon.noname-c con 0x7f318409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 10 monclient(hunting): picked mon.noname-e con 0x7f318409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3184142610 con 0x7f318409fba0 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f318409d310 con 0x7f318409f7d0 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31840a2560 con 0x7f318409ff70 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f3189f5d640 10 monclient(hunting): authenticate will time out at 1718.748657s 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f31837fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f318409f7d0 0x7f31841417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43364/0 (socket says 172.21.15.45:43364) 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f31837fe640 1 -- 172.21.15.45:0/4183880598 learned_addr learned my addr 172.21.15.45:0/4183880598 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802925193 0 0) 0x7f318409d310 con 0x7f318409f7d0 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f316b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f316b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f316b7fe640 10 monclient(hunting): my global_id is 184722 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31700034e0 con 0x7f318409f7d0 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.410+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249049628 0 0) 0x7f31840a2560 con 0x7f318409ff70 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): my global_id is 5903 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f318409d310 con 0x7f318409ff70 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213419746 0 0) 0x7f3184142610 con 0x7f318409fba0 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): my global_id is 164902 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31840a2560 con 0x7f318409fba0 2024-09-18T14:15:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4294609031 0 0) 0x7f31700034e0 con 0x7f318409f7d0 2024-09-18T14:15:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3184142610 con 0x7f318409f7d0 2024-09-18T14:15:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3178001b00 con 0x7f318409f7d0 2024-09-18T14:15:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): dump: 2024-09-18T14:15:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1279981293 0 0) 0x7f318409d310 con 0x7f318409ff70 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f31700034e0 con 0x7f318409ff70 2024-09-18T14:15:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f317c001d00 con 0x7f318409ff70 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): dump: 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2336921676 0 0) 0x7f31840a2560 con 0x7f318409fba0 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f318409d310 con 0x7f318409fba0 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3174001ea0 con 0x7f318409fba0 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient(hunting): dump: 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.432 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2663238586 0 0) 0x7f3184142610 con 0x7f318409f7d0 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 >> v1:172.21.15.45:6793/0 conn(0x7f318409ff70 legacy=0x7f3184149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 >> v1:172.21.15.45:6791/0 conn(0x7f318409fba0 legacy=0x7f31841319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 monclient: found mon.d 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184120770 con 0x7f318409f7d0 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient: _finish_auth 0 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f3189f5d640 5 monclient: authenticate success, global_id 184722 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f3189f5d640 10 monclient: _renew_subs 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f3189f5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f3189f5d640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3184120000 con 0x7f318409f7d0 2024-09-18T14:15:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f3189f5d640 10 monclient: _renew_subs 2024-09-18T14:15:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f3189f5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f3189f5d640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f31841320f0 con 0x7f318409f7d0 2024-09-18T14:15:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3178001d00 con 0x7f318409f7d0 2024-09-18T14:15:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3178004e90 con 0x7f318409f7d0 2024-09-18T14:15:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f3181ffb640 4 set_mon_vals no callback set 2024-09-18T14:15:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.411+0000 7f316b7fe640 10 monclient: dump: 2024-09-18T14:15:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.412+0000 7f3189f5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.412+0000 7f3189f5d640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3164006580 con 0x7f3164002830 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.412+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f3178004ff0 con 0x7f318409f7d0 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.412+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(738..738 src has 529..738) v4 ==== 3987+0+0 (unknown 2673133721 0 0) 0x7f3178046c40 con 0x7f318409f7d0 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529489853 0 0) 0x7f3164006580 con 0x7f3164002830 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 10 monclient(hunting): my global_id is 184595 2024-09-18T14:15:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3184142610 con 0x7f3164002830 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f3183fff640 10 monclient: get_auth_request con 0x7f317003aae0 auth_method 0 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2453587251 0 0) 0x7f3184142610 con 0x7f3164002830 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3164006580 con 0x7f3164002830 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3174001210 con 0x7f3164002830 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 10 monclient: dump: 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1457570183 0 0) 0x7f3164006580 con 0x7f3164002830 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f316400a0a0 con 0x7f3164002830 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3174002680 con 0x7f3164002830 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:03.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 >> v1:172.21.15.84:6791/0 conn(0x7f3164002830 legacy=0x7f3164004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.413+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 >> v1:172.21.15.84:6791/0 conn(0x7f3164002830 legacy=0x7f3164004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:03.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.419+0000 7f3189f5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:03.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.419+0000 7f3189f5d640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3164007510 con 0x7f31640068c0 2024-09-18T14:15:03.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.419+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565942817 0 0) 0x7f3164007510 con 0x7f31640068c0 2024-09-18T14:15:03.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 10 monclient(hunting): my global_id is 184604 2024-09-18T14:15:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3164006580 con 0x7f31640068c0 2024-09-18T14:15:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1071369813 0 0) 0x7f3164006580 con 0x7f31640068c0 2024-09-18T14:15:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3164007510 con 0x7f31640068c0 2024-09-18T14:15:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3174002fc0 con 0x7f31640068c0 2024-09-18T14:15:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:15:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 10 monclient: dump: 2024-09-18T14:15:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.442 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2386148162 0 0) 0x7f3164007510 con 0x7f31640068c0 2024-09-18T14:15:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3164008750 con 0x7f31640068c0 2024-09-18T14:15:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 1504165222) 0x7f31740051e0 con 0x7f31640068c0 2024-09-18T14:15:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 >> v1:172.21.15.84:6791/0 conn(0x7f31640068c0 legacy=0x7f3164006d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.420+0000 7f316b7fe640 1 -- 172.21.15.45:0/4183880598 >> v1:172.21.15.84:6791/0 conn(0x7f31640068c0 legacy=0x7f3164006d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:03.443 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:15:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:03.463 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.421+0000 7f3189f5d640 1 -- 172.21.15.45:0/4183880598 >> v1:172.21.15.84:6812/31478 conn(0x7f317003aae0 legacy=0x7f317003cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.421+0000 7f3189f5d640 10 monclient: shutdown 2024-09-18T14:15:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.421+0000 7f3189f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.421+0000 7f3189f5d640 1 -- 172.21.15.45:0/4183880598 >> v1:172.21.15.45:6792/0 conn(0x7f318409f7d0 legacy=0x7f31841417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.421+0000 7f3183fff640 1 -- 172.21.15.45:0/4183880598 reap_dead start 2024-09-18T14:15:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.421+0000 7f3183fff640 1 -- 172.21.15.45:0/4183880598 reap_dead start 2024-09-18T14:15:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.421+0000 7f3189f5d640 1 -- 172.21.15.45:0/4183880598 shutdown_connections 2024-09-18T14:15:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.421+0000 7f3189f5d640 1 -- 172.21.15.45:0/4183880598 >> 172.21.15.45:0/4183880598 conn(0x7f31840630b0 msgr2=0x7f31840a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.421+0000 7f3189f5d640 1 -- 172.21.15.45:0/4183880598 shutdown_connections 2024-09-18T14:15:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.421+0000 7f3189f5d640 1 -- 172.21.15.45:0/4183880598 wait complete. 2024-09-18T14:15:03.466 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:03.466 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:15:03.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:15:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.562+0000 7fc352d00640 10 monclient: get_monmap_and_config 2024-09-18T14:15:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.562+0000 7fc352d00640 10 monclient: build_initial_monmap 2024-09-18T14:15:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.562+0000 7fc352d00640 10 monclient: monmap: 2024-09-18T14:15:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:03.563628+0000 2024-09-18T14:15:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:03.563628+0000 2024-09-18T14:15:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:03.565 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 1 Processor -- start 2024-09-18T14:15:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 1 -- start start 2024-09-18T14:15:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 10 monclient: init 2024-09-18T14:15:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 10 monclient: _add_conns ranks=[0,7,4,8,6,1,5,3,2] 2024-09-18T14:15:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 10 monclient(hunting): picked mon.noname-a con 0x7fc34c09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 10 monclient(hunting): picked mon.noname-i con 0x7fc34c09fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 10 monclient(hunting): picked mon.noname-c con 0x7fc34c09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34c05ed70 con 0x7fc34c09f7d0 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34c09dfc0 con 0x7fc34c09ff70 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34c09dce0 con 0x7fc34c09fba0 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.563+0000 7fc352d00640 10 monclient(hunting): authenticate will time out at 1718.901967s 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc350a9c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc34c09f7d0 0x7fc34c132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56352/0 (socket says 172.21.15.45:56352) 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc350a9c640 1 -- 172.21.15.45:0/40430292 learned_addr learned my addr 172.21.15.45:0/40430292 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108477651 0 0) 0x7fc34c09dfc0 con 0x7fc34c09ff70 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient(hunting): my global_id is 164911 2024-09-18T14:15:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc33c0034e0 con 0x7fc34c09ff70 2024-09-18T14:15:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025206852 0 0) 0x7fc34c05ed70 con 0x7fc34c09f7d0 2024-09-18T14:15:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient(hunting): my global_id is 6174 2024-09-18T14:15:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc34c09dfc0 con 0x7fc34c09f7d0 2024-09-18T14:15:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120685973 0 0) 0x7fc34c09dce0 con 0x7fc34c09fba0 2024-09-18T14:15:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient(hunting): my global_id is 214207 2024-09-18T14:15:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc34c05ed70 con 0x7fc34c09fba0 2024-09-18T14:15:03.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 285693757 0 0) 0x7fc34c09dfc0 con 0x7fc34c09f7d0 2024-09-18T14:15:03.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 >> v1:172.21.15.84:6792/0 conn(0x7fc34c09fba0 legacy=0x7fc34c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 >> v1:172.21.15.45:6791/0 conn(0x7fc34c09ff70 legacy=0x7fc34c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:15:03.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 monclient: found mon.noname-a 2024-09-18T14:15:03.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:03.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:15:03.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c0a1c00 con 0x7fc34c09f7d0 2024-09-18T14:15:03.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient: _finish_auth 0 2024-09-18T14:15:03.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:03.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc3400016c0 con 0x7fc34c09f7d0 2024-09-18T14:15:03.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc352d00640 5 monclient: authenticate success, global_id 6174 2024-09-18T14:15:03.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:15:03.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient: dump: 2024-09-18T14:15:03.577 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.577 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.577 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.578 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc352d00640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc340001820 con 0x7fc34c09f7d0 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 1 -- 172.21.15.45:0/40430292 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc340004860 con 0x7fc34c09f7d0 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc34b7fe640 10 monclient: dump: 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc352d00640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.564+0000 7fc352d00640 4 set_mon_vals no callback set 2024-09-18T14:15:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.565+0000 7fc352d00640 10 monclient: shutdown 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.565+0000 7fc352d00640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.565+0000 7fc352d00640 1 -- 172.21.15.45:0/40430292 >> v1:172.21.15.45:6789/0 conn(0x7fc34c09f7d0 legacy=0x7fc34c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.565+0000 7fc352d00640 1 -- 172.21.15.45:0/40430292 shutdown_connections 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.565+0000 7fc352d00640 1 -- 172.21.15.45:0/40430292 >> 172.21.15.45:0/40430292 conn(0x7fc34c0630b0 msgr2=0x7fc34c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.565+0000 7fc352d00640 1 -- 172.21.15.45:0/40430292 shutdown_connections 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.565+0000 7fc352d00640 1 -- 172.21.15.45:0/40430292 wait complete. 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.565+0000 7fc352d00640 10 monclient: build_initial_monmap 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.565+0000 7fc352d00640 10 monclient: monmap: 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 1 Processor -- start 2024-09-18T14:15:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 1 -- start start 2024-09-18T14:15:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 10 monclient: init 2024-09-18T14:15:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 10 monclient: _add_conns ranks=[2,1,8,0,5,3,6,7,4] 2024-09-18T14:15:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 10 monclient(hunting): picked mon.noname-b con 0x7fc34c09f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 10 monclient(hunting): picked mon.noname-f con 0x7fc34c09fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 10 monclient(hunting): picked mon.noname-e con 0x7fc34c09ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34c09d310 con 0x7fc34c09fba0 2024-09-18T14:15:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34c141eb0 con 0x7fc34c09f7d0 2024-09-18T14:15:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34c142640 con 0x7fc34c09ff70 2024-09-18T14:15:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc352d00640 10 monclient(hunting): authenticate will time out at 1718.904780s 2024-09-18T14:15:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc350a9c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fc34c09f7d0 0x7fc34c1417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33098/0 (socket says 172.21.15.45:33098) 2024-09-18T14:15:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.566+0000 7fc350a9c640 1 -- 172.21.15.45:0/2174652274 learned_addr learned my addr 172.21.15.45:0/2174652274 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265993638 0 0) 0x7fc34c141eb0 con 0x7fc34c09f7d0 2024-09-18T14:15:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): my global_id is 145325 2024-09-18T14:15:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc330003510 con 0x7fc34c09f7d0 2024-09-18T14:15:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268446524 0 0) 0x7fc34c142640 con 0x7fc34c09ff70 2024-09-18T14:15:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): my global_id is 5912 2024-09-18T14:15:03.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc34c141eb0 con 0x7fc34c09ff70 2024-09-18T14:15:03.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030779677 0 0) 0x7fc34c09d310 con 0x7fc34c09fba0 2024-09-18T14:15:03.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): my global_id is 214237 2024-09-18T14:15:03.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc34c142640 con 0x7fc34c09fba0 2024-09-18T14:15:03.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2554820490 0 0) 0x7fc330003510 con 0x7fc34c09f7d0 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc34c09d310 con 0x7fc34c09f7d0 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3404740601 0 0) 0x7fc34c141eb0 con 0x7fc34c09ff70 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc330003510 con 0x7fc34c09ff70 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc340001cd0 con 0x7fc34c09f7d0 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): dump: 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.587 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3996724461 0 0) 0x7fc34c142640 con 0x7fc34c09fba0 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc34c141eb0 con 0x7fc34c09fba0 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc344001ca0 con 0x7fc34c09ff70 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): dump: 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.589 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc338001a20 con 0x7fc34c09fba0 2024-09-18T14:15:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient(hunting): dump: 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1228882793 0 0) 0x7fc34c09d310 con 0x7fc34c09f7d0 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 >> v1:172.21.15.45:6793/0 conn(0x7fc34c09ff70 legacy=0x7fc34c149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 >> v1:172.21.15.84:6789/0 conn(0x7fc34c09fba0 legacy=0x7fc34c1319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 monclient: found mon.b 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c0a2560 con 0x7fc34c09f7d0 2024-09-18T14:15:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient: _finish_auth 0 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc352d00640 5 monclient: authenticate success, global_id 145325 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc352d00640 10 monclient: _renew_subs 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc352d00640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc352d00640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34c120050 con 0x7fc34c09f7d0 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc352d00640 10 monclient: _renew_subs 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc352d00640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc352d00640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34c120bb0 con 0x7fc34c09f7d0 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc340004d90 con 0x7fc34c09f7d0 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc3400010e0 con 0x7fc34c09f7d0 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc34affd640 4 set_mon_vals no callback set 2024-09-18T14:15:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.567+0000 7fc348ff9640 10 monclient: dump: 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.568+0000 7fc352d00640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.568+0000 7fc352d00640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc32c006580 con 0x7fc32c002830 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.568+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc3400100b0 con 0x7fc34c09f7d0 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.568+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(738..738 src has 529..738) v4 ==== 3987+0+0 (unknown 2673133721 0 0) 0x7fc340046cf0 con 0x7fc34c09f7d0 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.568+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852604725 0 0) 0x7fc32c006580 con 0x7fc32c002830 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.568+0000 7fc348ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.568+0000 7fc348ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.568+0000 7fc348ff9640 10 monclient(hunting): my global_id is 214216 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.568+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc34c09d310 con 0x7fc32c002830 2024-09-18T14:15:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.568+0000 7fc35129d640 10 monclient: get_auth_request con 0x7fc33003aa20 auth_method 0 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2956960828 0 0) 0x7fc34c09d310 con 0x7fc32c002830 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc32c006580 con 0x7fc32c002830 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc3380020f0 con 0x7fc32c002830 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 10 monclient: dump: 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 428302958 0 0) 0x7fc32c006580 con 0x7fc32c002830 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc32c00a0a0 con 0x7fc32c002830 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc338004390 con 0x7fc32c002830 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 >> v1:172.21.15.84:6792/0 conn(0x7fc32c002830 legacy=0x7fc32c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.569+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 >> v1:172.21.15.84:6792/0 conn(0x7fc32c002830 legacy=0x7fc32c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:03.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.575+0000 7fc352d00640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:03.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.575+0000 7fc352d00640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc32c0074f0 con 0x7fc32c0068a0 2024-09-18T14:15:03.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.575+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166711695 0 0) 0x7fc32c0074f0 con 0x7fc32c0068a0 2024-09-18T14:15:03.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.575+0000 7fc348ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.575+0000 7fc348ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.575+0000 7fc348ff9640 10 monclient(hunting): my global_id is 214225 2024-09-18T14:15:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.575+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc32c006580 con 0x7fc32c0068a0 2024-09-18T14:15:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3703642588 0 0) 0x7fc32c006580 con 0x7fc32c0068a0 2024-09-18T14:15:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc32c0074f0 con 0x7fc32c0068a0 2024-09-18T14:15:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc338002b70 con 0x7fc32c0068a0 2024-09-18T14:15:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 10 monclient: dump: 2024-09-18T14:15:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.601 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3274834875 0 0) 0x7fc32c0074f0 con 0x7fc32c0068a0 2024-09-18T14:15:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc32c008750 con 0x7fc32c0068a0 2024-09-18T14:15:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 1665057821) 0x7fc338003fc0 con 0x7fc32c0068a0 2024-09-18T14:15:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 >> v1:172.21.15.84:6792/0 conn(0x7fc32c0068a0 legacy=0x7fc32c006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc348ff9640 1 -- 172.21.15.45:0/2174652274 >> v1:172.21.15.84:6792/0 conn(0x7fc32c0068a0 legacy=0x7fc32c006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:03.602 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:03.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:03.602 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:03.602 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:03.602 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:03.602 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:03.602 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:03.602 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:03.626 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:03.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc352d00640 1 -- 172.21.15.45:0/2174652274 >> v1:172.21.15.84:6812/31478 conn(0x7fc33003aa20 legacy=0x7fc33003ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc352d00640 10 monclient: shutdown 2024-09-18T14:15:03.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc352d00640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:03.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc352d00640 1 -- 172.21.15.45:0/2174652274 >> v1:172.21.15.45:6790/0 conn(0x7fc34c09f7d0 legacy=0x7fc34c1417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc35129d640 1 -- 172.21.15.45:0/2174652274 reap_dead start 2024-09-18T14:15:03.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc35129d640 1 -- 172.21.15.45:0/2174652274 reap_dead start 2024-09-18T14:15:03.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc352d00640 1 -- 172.21.15.45:0/2174652274 shutdown_connections 2024-09-18T14:15:03.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.576+0000 7fc352d00640 1 -- 172.21.15.45:0/2174652274 >> 172.21.15.45:0/2174652274 conn(0x7fc34c0630b0 msgr2=0x7fc34c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:03.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.577+0000 7fc352d00640 1 -- 172.21.15.45:0/2174652274 shutdown_connections 2024-09-18T14:15:03.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.577+0000 7fc352d00640 1 -- 172.21.15.45:0/2174652274 wait complete. 2024-09-18T14:15:03.628 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:03.628 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-18T14:15:03.628 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-18T14:15:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.724+0000 7f488279d640 10 monclient: get_monmap_and_config 2024-09-18T14:15:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.724+0000 7f488279d640 10 monclient: build_initial_monmap 2024-09-18T14:15:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.724+0000 7f488279d640 10 monclient: monmap: 2024-09-18T14:15:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:03.725074+0000 2024-09-18T14:15:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:03.725074+0000 2024-09-18T14:15:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:03.726 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.724+0000 7f488279d640 1 Processor -- start 2024-09-18T14:15:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.724+0000 7f488279d640 1 -- start start 2024-09-18T14:15:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.724+0000 7f488279d640 10 monclient: init 2024-09-18T14:15:03.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f488279d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:03.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f488279d640 10 monclient: _add_conns ranks=[3,5,0,2,1,8,4,7,6] 2024-09-18T14:15:03.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f488279d640 10 monclient(hunting): picked mon.noname-g con 0x7f487c06e810 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:03.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f488279d640 10 monclient(hunting): picked mon.noname-h con 0x7f487c0a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:03.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f488279d640 10 monclient(hunting): picked mon.noname-a con 0x7f487c0a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:03.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f488279d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c05ed70 con 0x7f487c0a1080 2024-09-18T14:15:03.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f488279d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c097620 con 0x7f487c06e810 2024-09-18T14:15:03.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f488279d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c097340 con 0x7f487c0a0cb0 2024-09-18T14:15:03.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f488279d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:03.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f488279d640 10 monclient(hunting): authenticate will time out at 1719.063421s 2024-09-18T14:15:03.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f4880d3a640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f487c0a1080 0x7f487c13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56368/0 (socket says 172.21.15.45:56368) 2024-09-18T14:15:03.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f4880d3a640 1 -- 172.21.15.45:0/2491161427 learned_addr learned my addr 172.21.15.45:0/2491161427 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:03.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325882312 0 0) 0x7f487c05ed70 con 0x7f487c0a1080 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 10 monclient(hunting): my global_id is 6183 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48640034e0 con 0x7f487c0a1080 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595335752 0 0) 0x7f487c097340 con 0x7f487c0a0cb0 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 10 monclient(hunting): my global_id is 184613 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f487c05ed70 con 0x7f487c0a0cb0 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036151506 0 0) 0x7f487c097620 con 0x7f487c06e810 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 10 monclient(hunting): my global_id is 184674 2024-09-18T14:15:03.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f487c097340 con 0x7f487c06e810 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3754169704 0 0) 0x7f48640034e0 con 0x7f487c0a1080 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 >> v1:172.21.15.84:6791/0 conn(0x7f487c0a0cb0 legacy=0x7f487c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 >> v1:172.21.15.84:6790/0 conn(0x7f487c06e810 legacy=0x7f487c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 1 monclient: found mon.noname-a 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.725+0000 7f487affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c0a1660 con 0x7f487c0a1080 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 10 monclient: _finish_auth 0 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4874001bb0 con 0x7f487c0a1080 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f488279d640 5 monclient: authenticate success, global_id 6183 2024-09-18T14:15:03.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f488279d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 10 monclient: dump: 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.737 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f488279d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4874001d90 con 0x7f487c0a1080 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 1 -- 172.21.15.45:0/2491161427 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4874004f20 con 0x7f487c0a1080 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f488279d640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f488279d640 4 set_mon_vals no callback set 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f487affd640 10 monclient: dump: 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.739 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f488279d640 10 monclient: shutdown 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f488279d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f488279d640 1 -- 172.21.15.45:0/2491161427 >> v1:172.21.15.45:6789/0 conn(0x7f487c0a1080 legacy=0x7f487c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f488279d640 1 -- 172.21.15.45:0/2491161427 shutdown_connections 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f488279d640 1 -- 172.21.15.45:0/2491161427 >> 172.21.15.45:0/2491161427 conn(0x7f487c0a5bb0 msgr2=0x7f487c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f488279d640 1 -- 172.21.15.45:0/2491161427 shutdown_connections 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.726+0000 7f488279d640 1 -- 172.21.15.45:0/2491161427 wait complete. 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.727+0000 7f488279d640 10 monclient: build_initial_monmap 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.727+0000 7f488279d640 10 monclient: monmap: 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:03.740 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:03.741 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.727+0000 7f488279d640 1 Processor -- start 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.727+0000 7f488279d640 1 -- start start 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.727+0000 7f488279d640 10 monclient: init 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 10 monclient: _add_conns ranks=[4,0,7,2,8,3,5,6,1] 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 10 monclient(hunting): picked mon.noname-c con 0x7f487c06e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 10 monclient(hunting): picked mon.noname-a con 0x7f487c0a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 10 monclient(hunting): picked mon.noname-i con 0x7f487c0a1080 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c122f00 con 0x7f487c0a0cb0 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c13dcf0 con 0x7f487c06e810 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c122940 con 0x7f487c0a1080 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 10 monclient(hunting): authenticate will time out at 1719.066135s 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f487b7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f487c0a0cb0 0x7f487c146810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56382/0 (socket says 172.21.15.45:56382) 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f487b7fe640 1 -- 172.21.15.45:0/856709602 learned_addr learned my addr 172.21.15.45:0/856709602 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034175869 0 0) 0x7f487c122f00 con 0x7f487c0a0cb0 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): my global_id is 6192 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48680034e0 con 0x7f487c0a0cb0 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306652682 0 0) 0x7f487c13dcf0 con 0x7f487c06e810 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): my global_id is 164920 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f487c122f00 con 0x7f487c06e810 2024-09-18T14:15:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047066668 0 0) 0x7f487c122940 con 0x7f487c0a1080 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): my global_id is 214234 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f487c13dcf0 con 0x7f487c0a1080 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 193292731 0 0) 0x7f48680034e0 con 0x7f487c0a0cb0 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f487c122940 con 0x7f487c0a0cb0 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 329839550 0 0) 0x7f487c13dcf0 con 0x7f487c0a1080 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f48680034e0 con 0x7f487c0a1080 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3983469359 0 0) 0x7f487c122f00 con 0x7f487c06e810 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f487c13dcf0 con 0x7f487c06e810 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4870002c10 con 0x7f487c0a0cb0 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:15:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): dump: 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4874001b70 con 0x7f487c0a1080 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): dump: 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f486c001950 con 0x7f487c06e810 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient(hunting): dump: 2024-09-18T14:15:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4196818137 0 0) 0x7f48680034e0 con 0x7f487c0a1080 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 >> v1:172.21.15.45:6791/0 conn(0x7f487c06e810 legacy=0x7f487c13d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 >> v1:172.21.15.45:6789/0 conn(0x7f487c0a0cb0 legacy=0x7f487c146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient: _finish_hunting 0 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 monclient: found mon.i 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c1204e0 con 0x7f487c0a1080 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 10 monclient: _finish_auth 0 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f4863fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 5 monclient: authenticate success, global_id 214234 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 10 monclient: _renew_subs 2024-09-18T14:15:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.728+0000 7f488279d640 1 -- 172.21.15.45:0/856709602 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c122660 con 0x7f487c0a1080 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f488279d640 10 monclient: _renew_subs 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f488279d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f488279d640 1 -- 172.21.15.45:0/856709602 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c121790 con 0x7f487c0a1080 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f48740013a0 con 0x7f487c0a1080 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f4863fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4874005110 con 0x7f487c0a1080 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f4863fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f4863fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f487a7fc640 4 set_mon_vals no callback set 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f4863fff640 10 monclient: dump: 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:03.751 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:03.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f48740100e0 con 0x7f487c0a1080 2024-09-18T14:15:03.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f488279d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:03.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f488279d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:03.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f488279d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:15:03.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.729+0000 7f488279d640 1 -- 172.21.15.45:0/856709602 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485c005010 con 0x7f487c0a1080 2024-09-18T14:15:03.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.730+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(738..738 src has 529..738) v4 ==== 3987+0+0 (unknown 2673133721 0 0) 0x7f4874046d90 con 0x7f487c0a1080 2024-09-18T14:15:03.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.730+0000 7f487bfff640 10 monclient: get_auth_request con 0x7f486803a9e0 auth_method 0 2024-09-18T14:15:03.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.734+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.7 v1:172.21.15.84:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f4874015590 con 0x7f487c0a1080 2024-09-18T14:15:03.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.734+0000 7f4863fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:03.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.734+0000 7f4863fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:03.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.960+0000 7f488279d640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-18T14:15:03.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.960+0000 7f488279d640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-18T14:15:03.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.960+0000 7f488279d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:15:03.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.960+0000 7f488279d640 1 -- 172.21.15.45:0/856709602 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f485c005b90 con 0x7f487c0a1080 2024-09-18T14:15:03.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f4863fff640 1 -- 172.21.15.45:0/856709602 <== mon.7 v1:172.21.15.84:6792/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f4874014f90 con 0x7f487c0a1080 2024-09-18T14:15:03.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f4863fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-18T14:15:03.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f4863fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:03.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=30,config=2,health=12,kv=8,logm=48} crc {auth=673933447,config=3140514009,health=1677507855,kv=594742435,logm=866880196}) 2024-09-18T14:15:03.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=30,config=2,health=12,kv=8,logm=48} crc {auth=3039355834,config=3140514009,health=1677507855,kv=594742435,logm=4259844870}) 2024-09-18T14:15:03.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=30,config=2,health=12,kv=8,logm=48} crc {auth=673933447,config=3140514009,health=1677507855,kv=594742435,logm=866880196}) 2024-09-18T14:15:03.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=32,config=2,health=12,kv=8,logm=46} crc {auth=1646499946,config=3140514009,health=3258679350,kv=594742435,logm=528416704}) 2024-09-18T14:15:03.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=30,config=2,health=12,kv=8,logm=48} crc {auth=673933447,config=3140514009,health=1677507855,kv=594742435,logm=866880196}) 2024-09-18T14:15:03.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=32,config=2,health=12,kv=7,logm=47} crc {auth=1646499946,config=3140514009,health=2898176798,kv=4171427160,logm=834536649}) 2024-09-18T14:15:03.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=30,config=2,health=12,kv=8,logm=48} crc {auth=673933447,config=3140514009,health=1677507855,kv=594742435,logm=866880196}) 2024-09-18T14:15:03.965 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=30,config=2,health=11,kv=7,logm=50} crc {auth=497108476,config=3140514009,health=3513810965,kv=594742435,logm=2183649132}) 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=30,config=2,health=12,kv=8,logm=48} crc {auth=673933447,config=3140514009,health=1677507855,kv=594742435,logm=866880196}) 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=29,config=1,health=12,kv=8,logm=50} crc {auth=1983712971,config=1228700967,health=2898176798,kv=594742435,logm=1675829085}) 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=30,config=2,health=12,kv=8,logm=48} crc {auth=673933447,config=3140514009,health=1677507855,kv=594742435,logm=866880196}) 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=30,config=2,health=12,kv=8,logm=48} crc {auth=1882010054,config=3140514009,health=1677507855,kv=594742435,logm=1202684340}) 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=30,config=2,health=12,kv=8,logm=48} crc {auth=673933447,config=3140514009,health=1677507855,kv=594742435,logm=866880196}) 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=31,config=2,health=12,kv=8,logm=47} crc {auth=891405215,config=3140514009,health=2898176798,kv=594742435,logm=816915636}) 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=30,config=2,health=12,kv=8,logm=48} crc {auth=673933447,config=3140514009,health=1677507855,kv=594742435,logm=866880196}) 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.962+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=32,config=2,health=12,kv=8,logm=46} crc {auth=1646499946,config=3140514009,health=3580165959,kv=594742435,logm=67115857}) 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.966 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1422599595}) 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=953203532}) 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1422599595}) 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4231775643}) 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1422599595}) 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=222630410}) 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1422599595}) 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3474567485}) 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1422599595}) 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=210708659}) 2024-09-18T14:15:03.967 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1422599595}) 2024-09-18T14:15:03.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=320589034}) 2024-09-18T14:15:03.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1422599595}) 2024-09-18T14:15:03.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1869822449}) 2024-09-18T14:15:03.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1422599595}) 2024-09-18T14:15:03.968 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.963+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3609851681}) 2024-09-18T14:15:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f488279d640 1 -- 172.21.15.45:0/856709602 >> v1:172.21.15.84:6812/31478 conn(0x7f486803a9e0 legacy=0x7f486803ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.969 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f488279d640 10 monclient: shutdown 2024-09-18T14:15:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f488279d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f488279d640 1 -- 172.21.15.45:0/856709602 >> v1:172.21.15.84:6792/0 conn(0x7f487c0a1080 legacy=0x7f487c149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:03.970 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078989227}) 2024-09-18T14:15:03.970 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1019804297}) 2024-09-18T14:15:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f488279d640 1 -- 172.21.15.45:0/856709602 shutdown_connections 2024-09-18T14:15:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f488279d640 1 -- 172.21.15.45:0/856709602 >> 172.21.15.45:0/856709602 conn(0x7f487c0a5bb0 msgr2=0x7f487c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:03.971 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.971 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078989227}) 2024-09-18T14:15:03.971 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3710184850}) 2024-09-18T14:15:03.971 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.971 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078989227}) 2024-09-18T14:15:03.971 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3398917340}) 2024-09-18T14:15:03.971 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f488279d640 1 -- 172.21.15.45:0/856709602 shutdown_connections 2024-09-18T14:15:03.972 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078989227}) 2024-09-18T14:15:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f488279d640 1 -- 172.21.15.45:0/856709602 wait complete. 2024-09-18T14:15:03.993 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=121312484}) 2024-09-18T14:15:03.993 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.993 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078989227}) 2024-09-18T14:15:03.993 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1065983198}) 2024-09-18T14:15:03.993 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078989227}) 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3226295404}) 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078989227}) 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4232485063}) 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078989227}) 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.965+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4241168162}) 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.966+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.966+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1496574921}) 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.966+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=676871123}) 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.966+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.994 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1496574921}) 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4293647542}) 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1496574921}) 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1137436368}) 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1496574921}) 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=575484955}) 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1496574921}) 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=628101029}) 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1496574921}) 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=980928102}) 2024-09-18T14:15:03.995 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1496574921}) 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4175915885}) 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1496574921}) 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.967+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2900308040}) 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2430694655}) 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3485326748}) 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2430694655}) 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2869184082}) 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.996 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2430694655}) 2024-09-18T14:15:03.997 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3687405317}) 2024-09-18T14:15:03.997 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.997 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2430694655}) 2024-09-18T14:15:03.997 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=271554621}) 2024-09-18T14:15:03.997 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.997 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2430694655}) 2024-09-18T14:15:03.997 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1454292950}) 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2430694655}) 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3109675763}) 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2430694655}) 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3917109242}) 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2430694655}) 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.968+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2973495045}) 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.969+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.969+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1179280268}) 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.969+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1252680609}) 2024-09-18T14:15:03.998 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.969+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.969+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1179280268}) 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.969+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649602243}) 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.969+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.969+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1179280268}) 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=971464993}) 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1179280268}) 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2787658726}) 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1179280268}) 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1683370187}) 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:03.999 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1179280268}) 2024-09-18T14:15:04.000 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2358079286}) 2024-09-18T14:15:04.000 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.000 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1179280268}) 2024-09-18T14:15:04.000 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=167986558}) 2024-09-18T14:15:04.000 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.000 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1179280268}) 2024-09-18T14:15:04.000 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.970+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=855397796}) 2024-09-18T14:15:04.000 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.000 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877912190}) 2024-09-18T14:15:04.000 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2346051871}) 2024-09-18T14:15:04.001 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.001 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877912190}) 2024-09-18T14:15:04.001 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4187974271}) 2024-09-18T14:15:04.001 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.001 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877912190}) 2024-09-18T14:15:04.001 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3516017702}) 2024-09-18T14:15:04.001 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.001 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877912190}) 2024-09-18T14:15:04.001 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=425194809}) 2024-09-18T14:15:04.001 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.001 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877912190}) 2024-09-18T14:15:04.001 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1876596398}) 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877912190}) 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1455104995}) 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877912190}) 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2780894351}) 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877912190}) 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.971+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=887927986}) 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.972+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.972+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597938158}) 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.972+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2137128527}) 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.972+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.002 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.972+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597938158}) 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.972+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1198464036}) 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.972+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.972+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597938158}) 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.972+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=295990691}) 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.972+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.972+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597938158}) 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3731428283}) 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597938158}) 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=487238479}) 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597938158}) 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2256021681}) 2024-09-18T14:15:04.003 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597938158}) 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=291046972}) 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597938158}) 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.973+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2084159457}) 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3819227039}) 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4200532313}) 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3819227039}) 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=77430072}) 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.004 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3819227039}) 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3369248419}) 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3819227039}) 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=299340697}) 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3819227039}) 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2130632337}) 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3819227039}) 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1698975703}) 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3819227039}) 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3056469458}) 2024-09-18T14:15:04.005 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3819227039}) 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.974+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4042600992}) 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.975+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.975+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924015850}) 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.975+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3498061511}) 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.975+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.975+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924015850}) 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.975+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=790917345}) 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.975+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.975+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924015850}) 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.975+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1488082067}) 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.975+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.975+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924015850}) 2024-09-18T14:15:04.006 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.975+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3648348394}) 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.976+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.976+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924015850}) 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.976+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3623787059}) 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.976+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.976+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924015850}) 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.976+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=874335640}) 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.976+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.976+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924015850}) 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.976+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3739326290}) 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.976+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.976+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924015850}) 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.976+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3378507917}) 2024-09-18T14:15:04.007 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=160314242}) 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1896040330}) 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=160314242}) 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=578638588}) 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=160314242}) 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1102017588}) 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=160314242}) 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3667256030}) 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.008 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=160314242}) 2024-09-18T14:15:04.009 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=356834857}) 2024-09-18T14:15:04.009 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.009 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=160314242}) 2024-09-18T14:15:04.009 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1216174733}) 2024-09-18T14:15:04.009 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.009 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=160314242}) 2024-09-18T14:15:04.009 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4134157808}) 2024-09-18T14:15:04.009 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.009 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=160314242}) 2024-09-18T14:15:04.009 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.977+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=110644722}) 2024-09-18T14:15:04.010 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.010 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2005133212}) 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2935650407}) 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2005133212}) 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1686034281}) 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2005133212}) 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1981873066}) 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2005133212}) 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3135938462}) 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2005133212}) 2024-09-18T14:15:04.011 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1674530069}) 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2005133212}) 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3574278329}) 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2005133212}) 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=273021737}) 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2005133212}) 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.978+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2416975832}) 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=812581155}) 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2040603693}) 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.012 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=812581155}) 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3950505951}) 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=812581155}) 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=9850257}) 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=812581155}) 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1799729046}) 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=812581155}) 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=419694440}) 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=812581155}) 2024-09-18T14:15:04.013 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1264863340}) 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=812581155}) 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1431206767}) 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=812581155}) 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.979+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1875083404}) 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=93655465}) 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1702404216}) 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=93655465}) 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=332626694}) 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.014 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=93655465}) 2024-09-18T14:15:04.015 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2049387323}) 2024-09-18T14:15:04.015 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.015 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=93655465}) 2024-09-18T14:15:04.015 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3134833562}) 2024-09-18T14:15:04.015 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.015 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=93655465}) 2024-09-18T14:15:04.015 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3194389098}) 2024-09-18T14:15:04.015 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.015 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=93655465}) 2024-09-18T14:15:04.015 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=409631561}) 2024-09-18T14:15:04.016 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-18T14:15:04.018 INFO:tasks.rados.rados.0.smithi084.stdout:3358: done (5 left) 2024-09-18T14:15:04.018 INFO:tasks.rados.rados.0.smithi084.stdout:3359: done (4 left) 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3360: done (3 left) 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3362: done (2 left) 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3363: done (1 left) 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3365: done (0 left) 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3366: delete oid 47 current snap is 369 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3366: done (0 left) 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3367: write oid 44 current snap is 369 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3367: seq_num 1145 ranges {444015=413055,1436000=773735,2709915=534100} 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3367: writing smithi08434663-44 from 444015 to 857070 tid 1 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3367: writing smithi08434663-44 from 1436000 to 2209735 tid 2 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3367: writing smithi08434663-44 from 2709915 to 3244015 tid 3 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3368: delete oid 32 current snap is 369 2024-09-18T14:15:04.019 INFO:tasks.rados.rados.0.smithi084.stdout:3367: finishing write tid 1 to smithi08434663-44 2024-09-18T14:15:04.020 INFO:tasks.rados.rados.0.smithi084.stdout:3367: finishing write tid 2 to smithi08434663-44 2024-09-18T14:15:04.024 INFO:tasks.rados.rados.0.smithi084.stdout:3367: finishing write tid 3 to smithi08434663-44 2024-09-18T14:15:04.024 INFO:tasks.rados.rados.0.smithi084.stdout:3367: finishing write tid 4 to smithi08434663-44 2024-09-18T14:15:04.024 INFO:tasks.rados.rados.0.smithi084.stdout:3367: finishing write tid 5 to smithi08434663-44 2024-09-18T14:15:04.024 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 1048 (ObjNum 1145 snap 369 seq_num 1145) dirty exists 2024-09-18T14:15:04.025 INFO:tasks.rados.rados.0.smithi084.stdout:3367: left oid 44 (ObjNum 1145 snap 369 seq_num 1145) 2024-09-18T14:15:04.025 INFO:tasks.rados.rados.0.smithi084.stdout:3367: done (1 left) 2024-09-18T14:15:04.025 INFO:tasks.rados.rados.0.smithi084.stdout:3368: done (0 left) 2024-09-18T14:15:04.025 INFO:tasks.rados.rados.0.smithi084.stdout:3369: read oid 8 snap -1 2024-09-18T14:15:04.025 INFO:tasks.rados.rados.0.smithi084.stdout:3369: expect (ObjNum 1041 snap 333 seq_num 1041) 2024-09-18T14:15:04.025 INFO:tasks.rados.rados.0.smithi084.stdout:3370: snap_create 2024-09-18T14:15:04.036 INFO:tasks.rados.rados.0.smithi084.stdout:3369: expect (ObjNum 1041 snap 333 seq_num 1041) 2024-09-18T14:15:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=93655465}) 2024-09-18T14:15:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2431363258}) 2024-09-18T14:15:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=93655465}) 2024-09-18T14:15:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.981+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1177889870}) 2024-09-18T14:15:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.123 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=424618804}) 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3603926717}) 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=424618804}) 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1708815250}) 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=424618804}) 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4119735599}) 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=424618804}) 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=661104103}) 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=424618804}) 2024-09-18T14:15:04.124 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1009036334}) 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=424618804}) 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3303054809}) 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=424618804}) 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4010000983}) 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=424618804}) 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.982+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2696144383}) 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2096014555}) 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4254565752}) 2024-09-18T14:15:04.125 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2096014555}) 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2443705426}) 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2096014555}) 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=916745533}) 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2096014555}) 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=716750090}) 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2096014555}) 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3159567445}) 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.126 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2096014555}) 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1149649491}) 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2096014555}) 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4252761035}) 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2096014555}) 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.983+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2341517572}) 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2965044629}) 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=847735183}) 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2965044629}) 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2190378915}) 2024-09-18T14:15:04.127 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2965044629}) 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1069647458}) 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2965044629}) 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=911571221}) 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2965044629}) 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2270376292}) 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2965044629}) 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3275171405}) 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2965044629}) 2024-09-18T14:15:04.128 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1144761123}) 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2965044629}) 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.985+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2009555546}) 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=899514162}) 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=533867735}) 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=899514162}) 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3935101045}) 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=899514162}) 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=769632557}) 2024-09-18T14:15:04.129 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=899514162}) 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1353357513}) 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=899514162}) 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4229128283}) 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=899514162}) 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2384196637}) 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=899514162}) 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3929773176}) 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.130 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=899514162}) 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.986+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3273694424}) 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=98115278}) 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863132317}) 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=98115278}) 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=446115100}) 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=98115278}) 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=927884966}) 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=98115278}) 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1074230791}) 2024-09-18T14:15:04.131 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=98115278}) 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2840326067}) 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=98115278}) 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3629437765}) 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=98115278}) 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=89870175}) 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=98115278}) 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.998+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1939775786}) 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2190722713}) 2024-09-18T14:15:04.132 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=615938499}) 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2190722713}) 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2246994781}) 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2190722713}) 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1262756476}) 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2190722713}) 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2896458508}) 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:03.999+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2190722713}) 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.000+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1697393687}) 2024-09-18T14:15:04.133 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.000+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.000+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2190722713}) 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.000+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=687773776}) 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.000+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.000+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2190722713}) 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.000+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=469862396}) 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.000+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.000+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2190722713}) 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.000+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=567655602}) 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1208259998}) 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=587362984}) 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1208259998}) 2024-09-18T14:15:04.134 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=575722842}) 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1208259998}) 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1495558596}) 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1208259998}) 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3196528709}) 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1208259998}) 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=691783643}) 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1208259998}) 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=22706905}) 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.135 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1208259998}) 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4064030632}) 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1208259998}) 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.001+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3895074859}) 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1693890392}) 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1278300510}) 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1693890392}) 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=246864475}) 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1693890392}) 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1151380974}) 2024-09-18T14:15:04.136 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1693890392}) 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3958577515}) 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1693890392}) 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3967339814}) 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1693890392}) 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1108021003}) 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1693890392}) 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1469626716}) 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1693890392}) 2024-09-18T14:15:04.137 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.002+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1283129328}) 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=741192306}) 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3480725291}) 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=741192306}) 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=225336094}) 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=741192306}) 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=728933015}) 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=741192306}) 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3392671479}) 2024-09-18T14:15:04.138 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=741192306}) 2024-09-18T14:15:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3687202322}) 2024-09-18T14:15:04.139 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=741192306}) 2024-09-18T14:15:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2826744337}) 2024-09-18T14:15:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=741192306}) 2024-09-18T14:15:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2055702697}) 2024-09-18T14:15:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.140 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=741192306}) 2024-09-18T14:15:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.003+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2041720007}) 2024-09-18T14:15:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.004+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.004+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1119714479}) 2024-09-18T14:15:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.004+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2819564153}) 2024-09-18T14:15:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.004+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.004+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1119714479}) 2024-09-18T14:15:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.004+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1204997253}) 2024-09-18T14:15:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.004+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.004+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1119714479}) 2024-09-18T14:15:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.004+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575545903}) 2024-09-18T14:15:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.004+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.141 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1119714479}) 2024-09-18T14:15:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3667664491}) 2024-09-18T14:15:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1119714479}) 2024-09-18T14:15:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1036119943}) 2024-09-18T14:15:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1119714479}) 2024-09-18T14:15:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3500072632}) 2024-09-18T14:15:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1119714479}) 2024-09-18T14:15:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3849165970}) 2024-09-18T14:15:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.142 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1119714479}) 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.005+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1548602561}) 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2926384249}) 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1116526169}) 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2926384249}) 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=58049806}) 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2926384249}) 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1856684357}) 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2926384249}) 2024-09-18T14:15:04.143 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=242894324}) 2024-09-18T14:15:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2926384249}) 2024-09-18T14:15:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3505879518}) 2024-09-18T14:15:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2926384249}) 2024-09-18T14:15:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4005751223}) 2024-09-18T14:15:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2926384249}) 2024-09-18T14:15:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1405596285}) 2024-09-18T14:15:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2926384249}) 2024-09-18T14:15:04.144 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.006+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4153886260}) 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2180140029}) 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3430809408}) 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2180140029}) 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2274314148}) 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2180140029}) 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2382114822}) 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2180140029}) 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1655666300}) 2024-09-18T14:15:04.145 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2180140029}) 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3682701391}) 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2180140029}) 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1647719144}) 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2180140029}) 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2286819796}) 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2180140029}) 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.007+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3896288964}) 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399750800}) 2024-09-18T14:15:04.146 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4062383778}) 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399750800}) 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3696316848}) 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399750800}) 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3821514707}) 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399750800}) 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=428882807}) 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399750800}) 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2008335904}) 2024-09-18T14:15:04.147 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399750800}) 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2191713427}) 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399750800}) 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=622309745}) 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399750800}) 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.008+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2440610553}) 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=278489399}) 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2540872162}) 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.148 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=278489399}) 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3457758225}) 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=278489399}) 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1652170026}) 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=278489399}) 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=546010375}) 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=278489399}) 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=476670487}) 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=278489399}) 2024-09-18T14:15:04.149 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1773171617}) 2024-09-18T14:15:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=278489399}) 2024-09-18T14:15:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1513702897}) 2024-09-18T14:15:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=278489399}) 2024-09-18T14:15:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.009+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3130881684}) 2024-09-18T14:15:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.010+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.010+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=779656559}) 2024-09-18T14:15:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.010+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1812057121}) 2024-09-18T14:15:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.010+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.010+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=779656559}) 2024-09-18T14:15:04.150 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.010+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=991768832}) 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=779656559}) 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1180975104}) 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=779656559}) 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3087527406}) 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=779656559}) 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=903574244}) 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=779656559}) 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1889366139}) 2024-09-18T14:15:04.151 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=779656559}) 2024-09-18T14:15:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4064261640}) 2024-09-18T14:15:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=779656559}) 2024-09-18T14:15:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.011+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4003985419}) 2024-09-18T14:15:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=147251065}) 2024-09-18T14:15:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2136378170}) 2024-09-18T14:15:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=147251065}) 2024-09-18T14:15:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1953800287}) 2024-09-18T14:15:04.152 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=147251065}) 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1120354897}) 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=147251065}) 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3786596248}) 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=147251065}) 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2699627387}) 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=147251065}) 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2965644751}) 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=147251065}) 2024-09-18T14:15:04.153 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2596737339}) 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=147251065}) 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.012+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=222724969}) 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448474340}) 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977864291}) 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448474340}) 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4210516177}) 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448474340}) 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=342060051}) 2024-09-18T14:15:04.154 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448474340}) 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4217154945}) 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448474340}) 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1584239062}) 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448474340}) 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1166993475}) 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448474340}) 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2175645126}) 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.155 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448474340}) 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.013+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3572544724}) 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3243704914}) 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2202765257}) 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3243704914}) 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3057055687}) 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3243704914}) 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2884849817}) 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3243704914}) 2024-09-18T14:15:04.156 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1706657415}) 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3243704914}) 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4063208468}) 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3243704914}) 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1892845887}) 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3243704914}) 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3614332195}) 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3243704914}) 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.014+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3733123932}) 2024-09-18T14:15:04.157 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.015+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.015+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906249565}) 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.015+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2964738622}) 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.015+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.015+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906249565}) 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.015+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=654501743}) 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.015+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.015+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906249565}) 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2521737271}) 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906249565}) 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3999735025}) 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.158 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906249565}) 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=71800536}) 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906249565}) 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3092059561}) 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906249565}) 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2990037956}) 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906249565}) 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.016+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3076432088}) 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=15588707}) 2024-09-18T14:15:04.159 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1414864608}) 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=15588707}) 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1189832000}) 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=15588707}) 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2211732159}) 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=15588707}) 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2563815358}) 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=15588707}) 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4039897309}) 2024-09-18T14:15:04.160 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=15588707}) 2024-09-18T14:15:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.017+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=120913747}) 2024-09-18T14:15:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.018+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.018+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=15588707}) 2024-09-18T14:15:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.018+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3166271870}) 2024-09-18T14:15:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.018+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.018+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=15588707}) 2024-09-18T14:15:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.018+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=487232854}) 2024-09-18T14:15:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1147439807}) 2024-09-18T14:15:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139159163}) 2024-09-18T14:15:04.161 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1147439807}) 2024-09-18T14:15:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=965560834}) 2024-09-18T14:15:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1147439807}) 2024-09-18T14:15:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3657819410}) 2024-09-18T14:15:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1147439807}) 2024-09-18T14:15:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4209163455}) 2024-09-18T14:15:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1147439807}) 2024-09-18T14:15:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=231166615}) 2024-09-18T14:15:04.162 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1147439807}) 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2778015072}) 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1147439807}) 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1147284445}) 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1147439807}) 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.019+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1482059777}) 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1670535601}) 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=766088463}) 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.163 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1670535601}) 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=995489530}) 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1670535601}) 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=380026745}) 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1670535601}) 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3004451935}) 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1670535601}) 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1379490382}) 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1670535601}) 2024-09-18T14:15:04.164 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1146440664}) 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1670535601}) 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1526346973}) 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1670535601}) 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.020+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3341381588}) 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2620325089}) 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2297087897}) 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2620325089}) 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=377986227}) 2024-09-18T14:15:04.165 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2620325089}) 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1412281962}) 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2620325089}) 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=608634485}) 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2620325089}) 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=758145977}) 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2620325089}) 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3559139227}) 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.021+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2620325089}) 2024-09-18T14:15:04.166 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.022+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2868335882}) 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.022+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.022+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2620325089}) 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.022+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3978788730}) 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3627997101}) 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279676915}) 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3627997101}) 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2299914630}) 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3627997101}) 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1554801724}) 2024-09-18T14:15:04.167 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3627997101}) 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1750900658}) 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3627997101}) 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=69586182}) 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3627997101}) 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2844544091}) 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3627997101}) 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2928566881}) 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.168 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3627997101}) 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.023+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2409840842}) 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.024+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.024+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=410925778}) 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.024+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=686748490}) 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.024+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.024+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=410925778}) 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043216759}) 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=410925778}) 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1873173551}) 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=410925778}) 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2784125649}) 2024-09-18T14:15:04.169 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=410925778}) 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1334038712}) 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=410925778}) 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4028480953}) 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=410925778}) 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=303557335}) 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=410925778}) 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.025+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2882860353}) 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.026+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.026+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=175017615}) 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.026+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3087454560}) 2024-09-18T14:15:04.170 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.026+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.026+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=175017615}) 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.026+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=259780874}) 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.026+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.026+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=175017615}) 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2465303966}) 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=175017615}) 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4270638988}) 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=175017615}) 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3239175179}) 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.171 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=175017615}) 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1970343035}) 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=175017615}) 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2153151679}) 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=175017615}) 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.027+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3607555159}) 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3277599155}) 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3887575964}) 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3277599155}) 2024-09-18T14:15:04.172 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1359599508}) 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3277599155}) 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3438810564}) 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3277599155}) 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1815929810}) 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3277599155}) 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4003928174}) 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3277599155}) 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1574653297}) 2024-09-18T14:15:04.173 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3277599155}) 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3826104716}) 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3277599155}) 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.028+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1425560957}) 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=213562163}) 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3048998434}) 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=213562163}) 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=696240793}) 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=213562163}) 2024-09-18T14:15:04.174 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=561507833}) 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=213562163}) 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=956327193}) 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.029+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=213562163}) 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1462029453}) 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=213562163}) 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1087149560}) 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=213562163}) 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3957847188}) 2024-09-18T14:15:04.175 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=213562163}) 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.030+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3300493718}) 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=275860118}) 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2037117423}) 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=275860118}) 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1770558786}) 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=275860118}) 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3212275317}) 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.176 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=275860118}) 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2889124532}) 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=275860118}) 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3618521573}) 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=275860118}) 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2309314682}) 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=275860118}) 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=44923314}) 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=275860118}) 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.031+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=453132330}) 2024-09-18T14:15:04.177 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1290313249}) 2024-09-18T14:15:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2457028547}) 2024-09-18T14:15:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1290313249}) 2024-09-18T14:15:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2282891611}) 2024-09-18T14:15:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1290313249}) 2024-09-18T14:15:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1290373656}) 2024-09-18T14:15:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1290313249}) 2024-09-18T14:15:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=663838615}) 2024-09-18T14:15:04.178 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1290313249}) 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2032674140}) 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1290313249}) 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=603338856}) 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1290313249}) 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4095586150}) 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1290313249}) 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.033+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3746407999}) 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3863647611}) 2024-09-18T14:15:04.179 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100137316}) 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3863647611}) 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=786352559}) 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3863647611}) 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1028975258}) 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3863647611}) 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3349321153}) 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3863647611}) 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3754317764}) 2024-09-18T14:15:04.180 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3863647611}) 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3101643098}) 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3863647611}) 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3471189307}) 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3863647611}) 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.034+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=529285089}) 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4093341599}) 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1589912952}) 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4093341599}) 2024-09-18T14:15:04.181 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=776581817}) 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4093341599}) 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2293409483}) 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4093341599}) 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2631587357}) 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4093341599}) 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=359670338}) 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4093341599}) 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1209218854}) 2024-09-18T14:15:04.182 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4093341599}) 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4037589032}) 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4093341599}) 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.036+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2721909156}) 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1748098612}) 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2596706265}) 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1748098612}) 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=228369644}) 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.183 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1748098612}) 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1998245963}) 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1748098612}) 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=608317630}) 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1748098612}) 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1942041463}) 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1748098612}) 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1016933355}) 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.037+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1748098612}) 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=25031888}) 2024-09-18T14:15:04.184 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1748098612}) 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.038+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2849962256}) 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=531683976}) 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1032916917}) 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=531683976}) 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4042976147}) 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=531683976}) 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1544248644}) 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.185 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=531683976}) 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=8851834}) 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=531683976}) 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3148662487}) 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=531683976}) 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1944337394}) 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=531683976}) 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2638900849}) 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=531683976}) 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.039+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4028955854}) 2024-09-18T14:15:04.186 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2815884664}) 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308061235}) 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.040+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2815884664}) 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=187638389}) 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2815884664}) 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2593269224}) 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2815884664}) 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3294239147}) 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.187 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2815884664}) 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1426767526}) 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2815884664}) 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=206462500}) 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2815884664}) 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3254686629}) 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2815884664}) 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.041+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=956325819}) 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=463285336}) 2024-09-18T14:15:04.188 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3148785953}) 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=463285336}) 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=422266505}) 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=463285336}) 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=132689765}) 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=463285336}) 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4261682651}) 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=463285336}) 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2966796625}) 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.189 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=463285336}) 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1337990601}) 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=463285336}) 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1738426783}) 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=463285336}) 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.042+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3405769612}) 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4189014025}) 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=444265996}) 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4189014025}) 2024-09-18T14:15:04.190 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=861392242}) 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4189014025}) 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4137113115}) 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4189014025}) 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1950148129}) 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4189014025}) 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2659879949}) 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4189014025}) 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=403598401}) 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.191 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4189014025}) 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1791732352}) 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4189014025}) 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.043+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1945746392}) 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009143603}) 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4073436421}) 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009143603}) 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1853766988}) 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009143603}) 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=617669306}) 2024-09-18T14:15:04.192 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.044+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009143603}) 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3129565544}) 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009143603}) 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=321255827}) 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009143603}) 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2220677058}) 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009143603}) 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2184866175}) 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.193 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009143603}) 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.045+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1310624911}) 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1744085337}) 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=595559698}) 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1744085337}) 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2335006355}) 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1744085337}) 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3227650952}) 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1744085337}) 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1742302885}) 2024-09-18T14:15:04.194 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1744085337}) 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2443016789}) 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1744085337}) 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=614044836}) 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1744085337}) 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1897002254}) 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1744085337}) 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.046+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=337750434}) 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60990575}) 2024-09-18T14:15:04.195 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1335279867}) 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60990575}) 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3115375509}) 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60990575}) 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2114778253}) 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60990575}) 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3904893354}) 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60990575}) 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2228584450}) 2024-09-18T14:15:04.196 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60990575}) 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3776941333}) 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60990575}) 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=788205059}) 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60990575}) 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.047+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=934482493}) 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1734759149}) 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2338186917}) 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.197 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1734759149}) 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2187718889}) 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.048+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1734759149}) 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1461813480}) 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1734759149}) 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2731310556}) 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1734759149}) 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4174404221}) 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1734759149}) 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3349855903}) 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.198 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1734759149}) 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1234481352}) 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1734759149}) 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.049+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3505037878}) 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1828645666}) 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1384924943}) 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1828645666}) 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3069011069}) 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1828645666}) 2024-09-18T14:15:04.199 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3834032644}) 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1828645666}) 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2526812926}) 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1828645666}) 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2329942322}) 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1828645666}) 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1149844055}) 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1828645666}) 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3350386600}) 2024-09-18T14:15:04.200 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1828645666}) 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.050+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3897262224}) 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=589742154}) 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1693036301}) 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=589742154}) 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3323782490}) 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=589742154}) 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3080982922}) 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=589742154}) 2024-09-18T14:15:04.201 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=417061502}) 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=589742154}) 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4212936816}) 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=589742154}) 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3097254504}) 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=589742154}) 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3962891517}) 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=589742154}) 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.052+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2318563781}) 2024-09-18T14:15:04.202 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1775887545}) 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3924188166}) 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1775887545}) 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=332263629}) 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1775887545}) 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4060045642}) 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1775887545}) 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=756596232}) 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.203 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1775887545}) 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2744680559}) 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1775887545}) 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2334036663}) 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1775887545}) 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3344665105}) 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1775887545}) 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.053+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1368226210}) 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452205855}) 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2001638347}) 2024-09-18T14:15:04.204 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452205855}) 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1036219654}) 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.054+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452205855}) 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4103830520}) 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452205855}) 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3915639779}) 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452205855}) 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=269016749}) 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.205 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452205855}) 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=966287201}) 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452205855}) 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2937438175}) 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3452205855}) 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.055+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3513470305}) 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3586652717}) 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=602046969}) 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3586652717}) 2024-09-18T14:15:04.206 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3989928828}) 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3586652717}) 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3473124166}) 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3586652717}) 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4035304743}) 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3586652717}) 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.056+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2729973193}) 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3586652717}) 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2632942480}) 2024-09-18T14:15:04.207 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3586652717}) 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2733627290}) 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3586652717}) 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.057+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=696631820}) 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=166324908}) 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2062996087}) 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=166324908}) 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1400544269}) 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.208 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=166324908}) 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1288755295}) 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=166324908}) 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2254572279}) 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=166324908}) 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=775042349}) 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=166324908}) 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2294074766}) 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=166324908}) 2024-09-18T14:15:04.209 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=454394369}) 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=166324908}) 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.058+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2003543888}) 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=54420677}) 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=883791392}) 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=54420677}) 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3072589674}) 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=54420677}) 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=675598666}) 2024-09-18T14:15:04.210 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=54420677}) 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=88317800}) 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=54420677}) 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=954056739}) 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=54420677}) 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=614443871}) 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=54420677}) 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2745207381}) 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.211 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=54420677}) 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.059+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=305674959}) 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2995058954}) 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461614180}) 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2995058954}) 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=274557910}) 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2995058954}) 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=535601971}) 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2995058954}) 2024-09-18T14:15:04.212 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1078457053}) 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2995058954}) 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3602942069}) 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2995058954}) 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2783864278}) 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2995058954}) 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2484743123}) 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2995058954}) 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.060+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1861659330}) 2024-09-18T14:15:04.213 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2645377350}) 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2784946543}) 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2645377350}) 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2561793759}) 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.061+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2645377350}) 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4180937738}) 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2645377350}) 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2239612868}) 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.214 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2645377350}) 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1152267079}) 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2645377350}) 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1521672144}) 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2645377350}) 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2643079997}) 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2645377350}) 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.062+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3375890254}) 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=115970829}) 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2250218418}) 2024-09-18T14:15:04.215 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=115970829}) 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3944339370}) 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=115970829}) 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3107118814}) 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=115970829}) 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4251661514}) 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=115970829}) 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2884168614}) 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=115970829}) 2024-09-18T14:15:04.216 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=917808467}) 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=115970829}) 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2386303709}) 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=115970829}) 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.063+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=609085135}) 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1067490773}) 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1366975933}) 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1067490773}) 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1456703784}) 2024-09-18T14:15:04.217 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1067490773}) 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1129788526}) 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1067490773}) 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=474361671}) 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1067490773}) 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2414485097}) 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1067490773}) 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.064+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1102606628}) 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1067490773}) 2024-09-18T14:15:04.218 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=588821923}) 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1067490773}) 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.065+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4142695021}) 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280188285}) 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=933057856}) 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280188285}) 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=386478277}) 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280188285}) 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2120226283}) 2024-09-18T14:15:04.219 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280188285}) 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1201789147}) 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280188285}) 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3040254858}) 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280188285}) 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=709111851}) 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280188285}) 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2570519340}) 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280188285}) 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.066+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4199842464}) 2024-09-18T14:15:04.220 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.221 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2472971630}) 2024-09-18T14:15:04.221 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1171032622}) 2024-09-18T14:15:04.221 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.221 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2472971630}) 2024-09-18T14:15:04.221 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=498526615}) 2024-09-18T14:15:04.221 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.221 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.067+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2472971630}) 2024-09-18T14:15:04.221 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2427076523}) 2024-09-18T14:15:04.221 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.221 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2472971630}) 2024-09-18T14:15:04.221 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2283419061}) 2024-09-18T14:15:04.221 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.222 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2472971630}) 2024-09-18T14:15:04.222 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1916906363}) 2024-09-18T14:15:04.222 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.222 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2472971630}) 2024-09-18T14:15:04.222 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3126644881}) 2024-09-18T14:15:04.222 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.222 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2472971630}) 2024-09-18T14:15:04.222 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2130339185}) 2024-09-18T14:15:04.222 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.222 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2472971630}) 2024-09-18T14:15:04.222 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.068+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3283863300}) 2024-09-18T14:15:04.222 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4041717744}) 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=587251963}) 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4041717744}) 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1800002481}) 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4041717744}) 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2479737371}) 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4041717744}) 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3326197236}) 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.223 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4041717744}) 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3226774236}) 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4041717744}) 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1809008564}) 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4041717744}) 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2457503272}) 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4041717744}) 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.069+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2177432449}) 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447492587}) 2024-09-18T14:15:04.224 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3240453771}) 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447492587}) 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1694638536}) 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447492587}) 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3523714078}) 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447492587}) 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=30131021}) 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447492587}) 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=648420756}) 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.225 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447492587}) 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3816512587}) 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447492587}) 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2465575596}) 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447492587}) 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.070+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3578513513}) 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=20,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=37} crc {logm=4068349011,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_snap=3515092273}) 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=18,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=15,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=41} crc {logm=3273766296,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgrstat=2018627809,mon_config_key=1810658514,monmap=455718034,osd_metadata=2643303027,osd_pg_creating=3954490799,osd_snap=3776687753}) 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=20,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=37} crc {logm=4068349011,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_snap=3515092273}) 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=18,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39} crc {logm=4271493827,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=3155120151,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3954490799,osd_snap=2578436637}) 2024-09-18T14:15:04.226 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=20,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=37} crc {logm=4068349011,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_snap=3515092273}) 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=19,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=40} crc {logm=2573366489,mdsmap=1531151339,mgr=1902834700,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=1810658514,monmap=455718034,osd_metadata=1798237175,osd_pg_creating=3954490799,osd_snap=524414743}) 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=20,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=37} crc {logm=4068349011,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_snap=3515092273}) 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=19,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=16,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=39} crc {logm=3545540432,mdsmap=1531151339,mgr=1359932698,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=1810658514,monmap=455718034,osd_metadata=3702629295,osd_pg_creating=3954490799,osd_snap=2982230065}) 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=20,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=37} crc {logm=4068349011,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_snap=3515092273}) 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=18,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=4,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=40} crc {logm=1366750698,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=212926899,monmap=3140514009,osd_metadata=2800182873,osd_pg_creating=3954490799,osd_snap=4003268153}) 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=20,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=37} crc {logm=4068349011,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_snap=3515092273}) 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=19,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=38} crc {logm=1505080036,mdsmap=1531151339,mgr=1821329879,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3954490799,osd_snap=2947981745}) 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.227 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=20,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=37} crc {logm=4068349011,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_snap=3515092273}) 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=19,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39} crc {logm=1278166376,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2528919814,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=3954490799,osd_snap=3035367440}) 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=20,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=37} crc {logm=4068349011,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_snap=3515092273}) 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.072+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=19,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=39} crc {logm=1246903345,mdsmap=1531151339,mgr=4288950570,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2014201061,mon_config_key=1810658514,monmap=455718034,osd_metadata=3328057993,osd_pg_creating=3954490799,osd_snap=587837163}) 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=86,osdmap=14} crc {osd_snap=3103536784,osdmap=2173501685}) 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=84,osdmap=16} crc {osd_snap=2565247174,osdmap=2348273045}) 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=86,osdmap=14} crc {osd_snap=3103536784,osdmap=2173501685}) 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=88,osdmap=12} crc {osd_snap=3846416268,osdmap=487792571}) 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=86,osdmap=14} crc {osd_snap=3103536784,osdmap=2173501685}) 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=89,osdmap=11} crc {osd_snap=703105829,osdmap=777567490}) 2024-09-18T14:15:04.228 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=86,osdmap=14} crc {osd_snap=3103536784,osdmap=2173501685}) 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=88,osdmap=12} crc {osd_snap=3258018925,osdmap=487792571}) 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=86,osdmap=14} crc {osd_snap=3103536784,osdmap=2173501685}) 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=88,osdmap=12} crc {osd_snap=1153795947,osdmap=487792571}) 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=86,osdmap=14} crc {osd_snap=3103536784,osdmap=2173501685}) 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=84,osdmap=16} crc {osd_snap=560793503,osdmap=4081727682}) 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=86,osdmap=14} crc {osd_snap=3103536784,osdmap=2173501685}) 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=85,osdmap=15} crc {osd_snap=1822884894,osdmap=2003016707}) 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=86,osdmap=14} crc {osd_snap=3103536784,osdmap=2173501685}) 2024-09-18T14:15:04.229 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.073+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=87,osdmap=13} crc {osd_snap=962479940,osdmap=3382885920}) 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2213846221}) 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=41883534}) 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2213846221}) 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.074+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1281270975}) 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2213846221}) 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2641188190}) 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2213846221}) 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1305967222}) 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2213846221}) 2024-09-18T14:15:04.230 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3477701634}) 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2213846221}) 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1519834614}) 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2213846221}) 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=351920378}) 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2213846221}) 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.075+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3875136561}) 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3793400948}) 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3943646765}) 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.231 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3793400948}) 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=909813549}) 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3793400948}) 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2879976653}) 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3793400948}) 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2474417302}) 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3793400948}) 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1116095776}) 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3793400948}) 2024-09-18T14:15:04.232 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=48104699}) 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3793400948}) 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3029711579}) 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3793400948}) 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.076+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2534880123}) 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1936830449}) 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2790091390}) 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1936830449}) 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1651030933}) 2024-09-18T14:15:04.233 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1936830449}) 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1234317175}) 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1936830449}) 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2539018574}) 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1936830449}) 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2590830547}) 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1936830449}) 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=795207919}) 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1936830449}) 2024-09-18T14:15:04.234 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1626334037}) 2024-09-18T14:15:04.235 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.078+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.235 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1936830449}) 2024-09-18T14:15:04.235 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2581975811}) 2024-09-18T14:15:04.235 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:15:04.235 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=8} crc {osdmap=2938573767}) 2024-09-18T14:15:04.235 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:15:04.079+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=6} crc {osdmap=2311047909}) 2024-09-18T14:15:05.017 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:15:05.017 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:15:05.073 INFO:tasks.rados.rados.0.smithi084.stdout:3369: done (1 left) 2024-09-18T14:15:05.073 INFO:tasks.rados.rados.0.smithi084.stdout:3370: done (0 left) 2024-09-18T14:15:05.073 INFO:tasks.rados.rados.0.smithi084.stdout:3371: snap_remove snap 362 2024-09-18T14:15:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.134+0000 7f47d989f640 10 monclient: get_monmap_and_config 2024-09-18T14:15:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.134+0000 7f47d989f640 10 monclient: build_initial_monmap 2024-09-18T14:15:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.134+0000 7f47d989f640 10 monclient: monmap: 2024-09-18T14:15:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:05.135701+0000 2024-09-18T14:15:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:05.135701+0000 2024-09-18T14:15:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:05.136 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.135+0000 7f47d989f640 1 Processor -- start 2024-09-18T14:15:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.135+0000 7f47d989f640 1 -- start start 2024-09-18T14:15:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.135+0000 7f47d989f640 10 monclient: init 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.135+0000 7f47d989f640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.135+0000 7f47d989f640 10 monclient: _add_conns ranks=[6,5,7,1,0,8,2,3,4] 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.135+0000 7f47d989f640 10 monclient(hunting): picked mon.noname-d con 0x7f47d409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d989f640 10 monclient(hunting): picked mon.noname-h con 0x7f47d409fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d989f640 10 monclient(hunting): picked mon.noname-i con 0x7f47d409ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d989f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d405ed70 con 0x7f47d409fba0 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d989f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d409dfc0 con 0x7f47d409f7d0 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d989f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d409dce0 con 0x7f47d409ff70 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d989f640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d989f640 10 monclient(hunting): authenticate will time out at 1720.474059s 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d2ffd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f47d409f7d0 0x7f47d4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43384/0 (socket says 172.21.15.45:43384) 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d2ffd640 1 -- 172.21.15.45:0/732978460 learned_addr learned my addr 172.21.15.45:0/732978460 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898197705 0 0) 0x7f47d409dfc0 con 0x7f47d409f7d0 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient(hunting): my global_id is 184731 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c00034e0 con 0x7f47d409f7d0 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618761133 0 0) 0x7f47d409dce0 con 0x7f47d409ff70 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient(hunting): my global_id is 214243 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47d409dfc0 con 0x7f47d409ff70 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034029483 0 0) 0x7f47d405ed70 con 0x7f47d409fba0 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient(hunting): my global_id is 184622 2024-09-18T14:15:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47d409dce0 con 0x7f47d409fba0 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2526916657 0 0) 0x7f47c00034e0 con 0x7f47d409f7d0 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 >> v1:172.21.15.84:6792/0 conn(0x7f47d409ff70 legacy=0x7f47d413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 >> v1:172.21.15.84:6791/0 conn(0x7f47d409fba0 legacy=0x7f47d4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient: _finish_hunting 0 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 monclient: found mon.noname-d 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40a1c00 con 0x7f47d409f7d0 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient: _finish_auth 0 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f47c4001830 con 0x7f47d409f7d0 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient: dump: 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d989f640 5 monclient: authenticate success, global_id 184731 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d989f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f47c4001a10 con 0x7f47d409f7d0 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 1 -- 172.21.15.45:0/732978460 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f47c4004ba0 con 0x7f47d409f7d0 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d989f640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d989f640 4 set_mon_vals no callback set 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.136+0000 7f47d27fc640 10 monclient: dump: 2024-09-18T14:15:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.137+0000 7f47d989f640 10 monclient: shutdown 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.137+0000 7f47d989f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.137+0000 7f47d989f640 1 -- 172.21.15.45:0/732978460 >> v1:172.21.15.45:6792/0 conn(0x7f47d409f7d0 legacy=0x7f47d4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.137+0000 7f47d989f640 1 -- 172.21.15.45:0/732978460 shutdown_connections 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.137+0000 7f47d989f640 1 -- 172.21.15.45:0/732978460 >> 172.21.15.45:0/732978460 conn(0x7f47d40630b0 msgr2=0x7f47d40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.137+0000 7f47d989f640 1 -- 172.21.15.45:0/732978460 shutdown_connections 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.137+0000 7f47d989f640 1 -- 172.21.15.45:0/732978460 wait complete. 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.137+0000 7f47d989f640 10 monclient: build_initial_monmap 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.137+0000 7f47d989f640 10 monclient: monmap: 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:05.150 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 1 Processor -- start 2024-09-18T14:15:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 1 -- start start 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 10 monclient: init 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 10 monclient: _add_conns ranks=[4,2,6,3,0,5,8,1,7] 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 10 monclient(hunting): picked mon.noname-c con 0x7f47d409f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 10 monclient(hunting): picked mon.noname-b con 0x7f47d409fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 10 monclient(hunting): picked mon.noname-d con 0x7f47d409ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d409d310 con 0x7f47d409fba0 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d40a2560 con 0x7f47d409f7d0 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d4121fc0 con 0x7f47d409ff70 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d989f640 10 monclient(hunting): authenticate will time out at 1720.476745s 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d2ffd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f47d409f7d0 0x7f47d4141940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39654/0 (socket says 172.21.15.45:39654) 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d2ffd640 1 -- 172.21.15.45:0/1925621417 learned_addr learned my addr 172.21.15.45:0/1925621417 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:05.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.138+0000 7f47d37fe640 1 --1- 172.21.15.45:0/1925621417 >> v1:172.21.15.45:6792/0 conn(0x7f47d409ff70 0x7f47d4149c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43386/0 (socket says 172.21.15.45:43386) 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649529341 0 0) 0x7f47d4121fc0 con 0x7f47d409ff70 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): my global_id is 184740 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47b40034e0 con 0x7f47d409ff70 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465594979 0 0) 0x7f47d40a2560 con 0x7f47d409f7d0 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): my global_id is 164929 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47d4121fc0 con 0x7f47d409f7d0 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529861861 0 0) 0x7f47d409d310 con 0x7f47d409fba0 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): my global_id is 145334 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47d40a2560 con 0x7f47d409fba0 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4186321090 0 0) 0x7f47d40a2560 con 0x7f47d409fba0 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47d409d310 con 0x7f47d409fba0 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4021077291 0 0) 0x7f47b40034e0 con 0x7f47d409ff70 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47d40a2560 con 0x7f47d409ff70 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2945011383 0 0) 0x7f47d4121fc0 con 0x7f47d409f7d0 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47b40034e0 con 0x7f47d409f7d0 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f47bc001a60 con 0x7f47d409fba0 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): dump: 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.154 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.155 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.155 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.155 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.155 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.155 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.155 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.155 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.155 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.155 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.155 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f47cc002db0 con 0x7f47d409ff70 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): dump: 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f47c4004b20 con 0x7f47d409f7d0 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient(hunting): dump: 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1873218563 0 0) 0x7f47d409d310 con 0x7f47d409fba0 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 >> v1:172.21.15.45:6792/0 conn(0x7f47d409ff70 legacy=0x7f47d4149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 >> v1:172.21.15.45:6791/0 conn(0x7f47d409f7d0 legacy=0x7f47d4141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 monclient: found mon.b 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d4120190 con 0x7f47d409fba0 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient: _finish_auth 0 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47d989f640 5 monclient: authenticate success, global_id 145334 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47d989f640 10 monclient: _renew_subs 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47d989f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47d989f640 1 -- 172.21.15.45:0/1925621417 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d4121c50 con 0x7f47d409fba0 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47d989f640 10 monclient: _renew_subs 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47d989f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47d989f640 1 -- 172.21.15.45:0/1925621417 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d4142050 con 0x7f47d409fba0 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f47bc001c40 con 0x7f47d409fba0 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f47bc001000 con 0x7f47d409fba0 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47d1ffb640 4 set_mon_vals no callback set 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.139+0000 7f47caffd640 10 monclient: dump: 2024-09-18T14:15:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.162 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.140+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f47bc002230 con 0x7f47d409fba0 2024-09-18T14:15:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.140+0000 7f47d989f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.140+0000 7f47d989f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.140+0000 7f47d989f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.140+0000 7f47d989f640 1 -- 172.21.15.45:0/1925621417 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b8005010 con 0x7f47d409fba0 2024-09-18T14:15:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.140+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(740..740 src has 529..740) v4 ==== 3959+0+0 (unknown 1715459358 0 0) 0x7f47bc043fc0 con 0x7f47d409fba0 2024-09-18T14:15:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.141+0000 7f47d2ffd640 10 monclient: get_auth_request con 0x7f47b403a940 auth_method 0 2024-09-18T14:15:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.145+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f47bc0028c0 con 0x7f47d409fba0 2024-09-18T14:15:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.145+0000 7f47caffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.145+0000 7f47caffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:05.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.370+0000 7f47d989f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:15:05.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.370+0000 7f47d989f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:15:05.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.370+0000 7f47d989f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.370+0000 7f47d989f640 1 -- 172.21.15.45:0/1925621417 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f47b8005840 con 0x7f47d409fba0 2024-09-18T14:15:05.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.370+0000 7f47caffd640 1 -- 172.21.15.45:0/1925621417 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2689 (unknown 254746657 0 3643708520) 0x7f47bc012240 con 0x7f47d409fba0 2024-09-18T14:15:05.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.370+0000 7f47caffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:15:05.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.370+0000 7f47caffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:05.371 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":130,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:15:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.372+0000 7f47d989f640 1 -- 172.21.15.45:0/1925621417 >> v1:172.21.15.84:6812/31478 conn(0x7f47b403a940 legacy=0x7f47b403cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.372+0000 7f47d989f640 10 monclient: shutdown 2024-09-18T14:15:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.372+0000 7f47d989f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.372+0000 7f47d989f640 1 -- 172.21.15.45:0/1925621417 >> v1:172.21.15.45:6790/0 conn(0x7f47d409fba0 legacy=0x7f47d4131b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.372+0000 7f47d989f640 1 -- 172.21.15.45:0/1925621417 shutdown_connections 2024-09-18T14:15:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.372+0000 7f47d989f640 1 -- 172.21.15.45:0/1925621417 >> 172.21.15.45:0/1925621417 conn(0x7f47d40630b0 msgr2=0x7f47d40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.372+0000 7f47d989f640 1 -- 172.21.15.45:0/1925621417 shutdown_connections 2024-09-18T14:15:05.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.372+0000 7f47d989f640 1 -- 172.21.15.45:0/1925621417 wait complete. 2024-09-18T14:15:05.383 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:15:05.383 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-18T14:15:05.383 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:15:05.383 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:15:05.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.480+0000 7f8b0586f640 10 monclient: get_monmap_and_config 2024-09-18T14:15:05.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.480+0000 7f8b0586f640 10 monclient: build_initial_monmap 2024-09-18T14:15:05.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.480+0000 7f8b0586f640 10 monclient: monmap: 2024-09-18T14:15:05.481 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:05.481 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.481 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:05.481671+0000 2024-09-18T14:15:05.481 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:05.481671+0000 2024-09-18T14:15:05.481 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:05.481 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.481 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:05.482 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:05.482 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:05.482 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:05.482 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:05.482 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:05.482 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:05.482 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:05.482 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:05.482 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.481+0000 7f8b0586f640 1 Processor -- start 2024-09-18T14:15:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.481+0000 7f8b0586f640 1 -- start start 2024-09-18T14:15:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.481+0000 7f8b0586f640 10 monclient: init 2024-09-18T14:15:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.481+0000 7f8b0586f640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.481+0000 7f8b0586f640 10 monclient: _add_conns ranks=[0,1,2,4,5,6,3,7,8] 2024-09-18T14:15:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.481+0000 7f8b0586f640 10 monclient(hunting): picked mon.noname-a con 0x7f8b0009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8b0586f640 10 monclient(hunting): picked mon.noname-f con 0x7f8b0009fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8b0586f640 10 monclient(hunting): picked mon.noname-b con 0x7f8b0009ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8b0586f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0005ed70 con 0x7f8b0009f7d0 2024-09-18T14:15:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8b0586f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0009dfc0 con 0x7f8b0009fba0 2024-09-18T14:15:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8b0586f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0009dce0 con 0x7f8b0009ff70 2024-09-18T14:15:05.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8b0586f640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8b0586f640 10 monclient(hunting): authenticate will time out at 1720.820055s 2024-09-18T14:15:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8aff7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f8b0009ff70 0x7f8b0013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33112/0 (socket says 172.21.15.45:33112) 2024-09-18T14:15:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8aff7fe640 1 -- 172.21.15.45:0/1523407865 learned_addr learned my addr 172.21.15.45:0/1523407865 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afeffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8b0009f7d0 0x7f8b00132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56396/0 (socket says 172.21.15.45:56396) 2024-09-18T14:15:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216280348 0 0) 0x7f8b0005ed70 con 0x7f8b0009f7d0 2024-09-18T14:15:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient(hunting): my global_id is 6201 2024-09-18T14:15:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aec0034e0 con 0x7f8b0009f7d0 2024-09-18T14:15:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598767862 0 0) 0x7f8b0009dce0 con 0x7f8b0009ff70 2024-09-18T14:15:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient(hunting): my global_id is 145343 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b0005ed70 con 0x7f8b0009ff70 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097866049 0 0) 0x7f8b0009dfc0 con 0x7f8b0009fba0 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient(hunting): my global_id is 214246 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b0009dce0 con 0x7f8b0009fba0 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2824947053 0 0) 0x7f8b0005ed70 con 0x7f8b0009ff70 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 >> v1:172.21.15.84:6789/0 conn(0x7f8b0009fba0 legacy=0x7f8b00137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 >> v1:172.21.15.45:6789/0 conn(0x7f8b0009f7d0 legacy=0x7f8b00132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 monclient: found mon.noname-b 2024-09-18T14:15:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b000a1c00 con 0x7f8b0009ff70 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient: _finish_auth 0 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8af8001a40 con 0x7f8b0009ff70 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8b0586f640 5 monclient: authenticate success, global_id 145343 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8b0586f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient: dump: 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.494 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8b0586f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8af8001c20 con 0x7f8b0009ff70 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8b0586f640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 1 -- 172.21.15.45:0/1523407865 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8af8004be0 con 0x7f8b0009ff70 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8b0586f640 4 set_mon_vals no callback set 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.482+0000 7f8afe7fc640 10 monclient: dump: 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.496 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.483+0000 7f8b0586f640 10 monclient: shutdown 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.483+0000 7f8b0586f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.483+0000 7f8b0586f640 1 -- 172.21.15.45:0/1523407865 >> v1:172.21.15.45:6790/0 conn(0x7f8b0009ff70 legacy=0x7f8b0013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.483+0000 7f8b0586f640 1 -- 172.21.15.45:0/1523407865 shutdown_connections 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.483+0000 7f8b0586f640 1 -- 172.21.15.45:0/1523407865 >> 172.21.15.45:0/1523407865 conn(0x7f8b000630b0 msgr2=0x7f8b000634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.483+0000 7f8b0586f640 1 -- 172.21.15.45:0/1523407865 shutdown_connections 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.483+0000 7f8b0586f640 1 -- 172.21.15.45:0/1523407865 wait complete. 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.483+0000 7f8b0586f640 10 monclient: build_initial_monmap 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.483+0000 7f8b0586f640 10 monclient: monmap: 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 1 Processor -- start 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 1 -- start start 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 10 monclient: init 2024-09-18T14:15:05.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 10 monclient: _add_conns ranks=[2,1,3,5,8,0,6,7,4] 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 10 monclient(hunting): picked mon.noname-b con 0x7f8b0009f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 10 monclient(hunting): picked mon.noname-f con 0x7f8b0009fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 10 monclient(hunting): picked mon.noname-g con 0x7f8b0009ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b000a2560 con 0x7f8b0009fba0 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b00136ea0 con 0x7f8b0009f7d0 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0011ec90 con 0x7f8b0009ff70 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8b0586f640 10 monclient(hunting): authenticate will time out at 1720.822761s 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8afeffd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f8b0009f7d0 0x7f8b001315b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33116/0 (socket says 172.21.15.45:33116) 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.484+0000 7f8afeffd640 1 -- 172.21.15.45:0/812898648 learned_addr learned my addr 172.21.15.45:0/812898648 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228961421 0 0) 0x7f8b00136ea0 con 0x7f8b0009f7d0 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): my global_id is 145352 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae00034e0 con 0x7f8b0009f7d0 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74590059 0 0) 0x7f8b0011ec90 con 0x7f8b0009ff70 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): my global_id is 184683 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b00136ea0 con 0x7f8b0009ff70 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819619759 0 0) 0x7f8b000a2560 con 0x7f8b0009fba0 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): my global_id is 214255 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b0011ec90 con 0x7f8b0009fba0 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3161405259 0 0) 0x7f8ae00034e0 con 0x7f8b0009f7d0 2024-09-18T14:15:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b000a2560 con 0x7f8b0009f7d0 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8af0001670 con 0x7f8b0009f7d0 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): dump: 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2254793802 0 0) 0x7f8b00136ea0 con 0x7f8b0009ff70 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ae00034e0 con 0x7f8b0009ff70 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8af8001a00 con 0x7f8b0009ff70 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:15:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): dump: 2024-09-18T14:15:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.504 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.505 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3235149981 0 0) 0x7f8b0011ec90 con 0x7f8b0009fba0 2024-09-18T14:15:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b00136ea0 con 0x7f8b0009fba0 2024-09-18T14:15:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8ae8001aa0 con 0x7f8b0009fba0 2024-09-18T14:15:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient(hunting): dump: 2024-09-18T14:15:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.506 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 243407517 0 0) 0x7f8b000a2560 con 0x7f8b0009f7d0 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 >> v1:172.21.15.84:6790/0 conn(0x7f8b0009ff70 legacy=0x7f8b0014a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 >> v1:172.21.15.84:6789/0 conn(0x7f8b0009fba0 legacy=0x7f8b00146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 monclient: found mon.b 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0011d7e0 con 0x7f8b0009f7d0 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient: _finish_auth 0 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8b0586f640 5 monclient: authenticate success, global_id 145352 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8b0586f640 10 monclient: _renew_subs 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8b0586f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8b0586f640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0011e6d0 con 0x7f8b0009f7d0 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8b0586f640 10 monclient: _renew_subs 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8b0586f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8b0586f640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0009b7b0 con 0x7f8b0009f7d0 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8af0004a50 con 0x7f8b0009f7d0 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8af0004f00 con 0x7f8b0009f7d0 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8afdffb640 4 set_mon_vals no callback set 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.485+0000 7f8af6ffd640 10 monclient: dump: 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.486+0000 7f8b0586f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.486+0000 7f8b0586f640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8ae4006580 con 0x7f8ae4002830 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.486+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f8af000fed0 con 0x7f8b0009f7d0 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.486+0000 7f8af7fff640 1 -- 172.21.15.45:0/812898648 >> v1:172.21.15.45:6789/0 conn(0x7f8ae4002830 legacy=0x7f8ae4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.486+0000 7f8af7fff640 1 -- 172.21.15.45:0/812898648 >> v1:172.21.15.45:6789/0 conn(0x7f8ae4002830 legacy=0x7f8ae4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.486+0000 7f8af7fff640 1 --1- 172.21.15.45:0/812898648 >> v1:172.21.15.45:6789/0 conn(0x7f8ae4002830 0x7f8ae4004cc0 :-1 s=OPENED pgs=617 cs=1 l=1).handle_message read tag failed 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.486+0000 7f8af7fff640 1 --1- 172.21.15.45:0/812898648 >> v1:172.21.15.45:6789/0 conn(0x7f8ae4002830 0x7f8ae4004cc0 :-1 s=OPENED pgs=617 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:15:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.486+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 >> v1:172.21.15.45:6789/0 conn(0x7f8ae4002830 legacy=0x7f8ae4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.486+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 >> v1:172.21.15.45:6789/0 conn(0x7f8ae4002830 legacy=0x7f8ae4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.486+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8b000a2560 con 0x7f8ae003e670 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.486+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(740..740 src has 529..740) v4 ==== 3959+0+0 (unknown 1715459358 0 0) 0x7f8af0046a90 con 0x7f8b0009f7d0 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.486+0000 7f8af7fff640 10 monclient: get_auth_request con 0x7f8ae003a9b0 auth_method 0 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27226689 0 0) 0x7f8b000a2560 con 0x7f8ae003e670 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 10 monclient(hunting): my global_id is 6210 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b0011ec90 con 0x7f8ae003e670 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 259772997 0 0) 0x7f8b0011ec90 con 0x7f8ae003e670 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b000a2560 con 0x7f8ae003e670 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8af8001440 con 0x7f8ae003e670 2024-09-18T14:15:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 10 monclient: dump: 2024-09-18T14:15:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.512 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 689013771 0 0) 0x7f8b000a2560 con 0x7f8ae003e670 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ae0043220 con 0x7f8ae003e670 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8af8007150 con 0x7f8ae003e670 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 >> v1:172.21.15.45:6789/0 conn(0x7f8ae003e670 legacy=0x7f8ae0040a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.487+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 >> v1:172.21.15.45:6789/0 conn(0x7f8ae003e670 legacy=0x7f8ae0040a50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.493+0000 7f8b0586f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.493+0000 7f8b0586f640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8ae40074f0 con 0x7f8ae40068a0 2024-09-18T14:15:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.493+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246100289 0 0) 0x7f8ae40074f0 con 0x7f8ae40068a0 2024-09-18T14:15:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.493+0000 7f8af6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.493+0000 7f8af6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.493+0000 7f8af6ffd640 10 monclient(hunting): my global_id is 6219 2024-09-18T14:15:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.493+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b000a2560 con 0x7f8ae40068a0 2024-09-18T14:15:05.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.493+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3714074933 0 0) 0x7f8b000a2560 con 0x7f8ae40068a0 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ae40074f0 con 0x7f8ae40068a0 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8af8005970 con 0x7f8ae40068a0 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8af6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8af6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8af6ffd640 10 monclient: dump: 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.516 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 228566460 0 0) 0x7f8ae40074f0 con 0x7f8ae40068a0 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8ae4008750 con 0x7f8ae40068a0 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 2805904244) 0x7f8af800ccf0 con 0x7f8ae40068a0 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8af6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8af6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 >> v1:172.21.15.45:6789/0 conn(0x7f8ae40068a0 legacy=0x7f8ae4006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8af6ffd640 1 -- 172.21.15.45:0/812898648 >> v1:172.21.15.45:6789/0 conn(0x7f8ae40068a0 legacy=0x7f8ae4006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:05.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8aff7fe640 1 -- 172.21.15.45:0/812898648 reap_dead start 2024-09-18T14:15:05.518 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:05.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:05.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:05.518 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:05.518 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:05.518 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:05.518 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:05.518 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:05.518 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:05.519 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:05.520 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:05.521 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:05.522 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.523 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:05.525 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.527 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:05.529 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.530 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:05.536 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:05.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8b0586f640 1 -- 172.21.15.45:0/812898648 >> v1:172.21.15.84:6812/31478 conn(0x7f8ae003a9b0 legacy=0x7f8ae003ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8b0586f640 10 monclient: shutdown 2024-09-18T14:15:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8b0586f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8b0586f640 1 -- 172.21.15.45:0/812898648 >> v1:172.21.15.45:6790/0 conn(0x7f8b0009f7d0 legacy=0x7f8b001315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8b0586f640 1 -- 172.21.15.45:0/812898648 shutdown_connections 2024-09-18T14:15:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8b0586f640 1 -- 172.21.15.45:0/812898648 >> 172.21.15.45:0/812898648 conn(0x7f8b000630b0 msgr2=0x7f8b000a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8b0586f640 1 -- 172.21.15.45:0/812898648 shutdown_connections 2024-09-18T14:15:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.494+0000 7f8b0586f640 1 -- 172.21.15.45:0/812898648 wait complete. 2024-09-18T14:15:05.538 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:05.538 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:15:05.539 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:15:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.634+0000 7f0ac6c57640 10 monclient: get_monmap_and_config 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.634+0000 7f0ac6c57640 10 monclient: build_initial_monmap 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.634+0000 7f0ac6c57640 10 monclient: monmap: 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:05.635804+0000 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:05.635804+0000 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:05.637 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:05.637 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:05.637 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:05.637 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:05.637 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.635+0000 7f0ac6c57640 1 Processor -- start 2024-09-18T14:15:05.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.635+0000 7f0ac6c57640 1 -- start start 2024-09-18T14:15:05.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.635+0000 7f0ac6c57640 10 monclient: init 2024-09-18T14:15:05.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:05.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 10 monclient: _add_conns ranks=[0,6,7,2,5,8,4,3,1] 2024-09-18T14:15:05.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 10 monclient(hunting): picked mon.noname-a con 0x7f0ac009f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:05.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 10 monclient(hunting): picked mon.noname-d con 0x7f0ac009fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:05.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 10 monclient(hunting): picked mon.noname-i con 0x7f0ac009ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:05.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac005ed70 con 0x7f0ac009f7d0 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac009dfc0 con 0x7f0ac009fba0 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac009dce0 con 0x7f0ac009ff70 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 10 monclient(hunting): authenticate will time out at 1720.974209s 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac49f3640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0ac009f7d0 0x7f0ac0132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56432/0 (socket says 172.21.15.45:56432) 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac49f3640 1 -- 172.21.15.45:0/708851739 learned_addr learned my addr 172.21.15.45:0/708851739 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abffff640 1 --1- 172.21.15.45:0/708851739 >> v1:172.21.15.45:6792/0 conn(0x7f0ac009fba0 0x7f0ac0137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43390/0 (socket says 172.21.15.45:43390) 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 1 -- 172.21.15.45:0/708851739 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243063423 0 0) 0x7f0ac009dfc0 con 0x7f0ac009fba0 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 10 monclient(hunting): my global_id is 184749 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 1 -- 172.21.15.45:0/708851739 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab00034e0 con 0x7f0ac009fba0 2024-09-18T14:15:05.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 1 -- 172.21.15.45:0/708851739 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364048674 0 0) 0x7f0ac005ed70 con 0x7f0ac009f7d0 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 10 monclient(hunting): my global_id is 6228 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 1 -- 172.21.15.45:0/708851739 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ac009dfc0 con 0x7f0ac009f7d0 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 1 -- 172.21.15.45:0/708851739 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2738016522 0 0) 0x7f0ab00034e0 con 0x7f0ac009fba0 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 1 -- 172.21.15.45:0/708851739 >> v1:172.21.15.84:6792/0 conn(0x7f0ac009ff70 legacy=0x7f0ac013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 1 -- 172.21.15.45:0/708851739 >> v1:172.21.15.45:6789/0 conn(0x7f0ac009f7d0 legacy=0x7f0ac0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 1 monclient: found mon.noname-d 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 1 -- 172.21.15.45:0/708851739 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00a1c00 con 0x7f0ac009fba0 2024-09-18T14:15:05.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 10 monclient: _finish_auth 0 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 1 -- 172.21.15.45:0/708851739 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0aac001830 con 0x7f0ac009fba0 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 5 monclient: authenticate success, global_id 184749 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 10 monclient: dump: 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0ac6c57640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.636+0000 7f0abf7fe640 1 -- 172.21.15.45:0/708851739 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0aac001990 con 0x7f0ac009fba0 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0abf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0ac6c57640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0abf7fe640 1 -- 172.21.15.45:0/708851739 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0aac004ae0 con 0x7f0ac009fba0 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0ac6c57640 4 set_mon_vals no callback set 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0abf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0abf7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0abf7fe640 10 monclient: dump: 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.649 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0ac6c57640 10 monclient: shutdown 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0ac6c57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0ac6c57640 1 -- 172.21.15.45:0/708851739 >> v1:172.21.15.45:6792/0 conn(0x7f0ac009fba0 legacy=0x7f0ac0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0ac6c57640 1 -- 172.21.15.45:0/708851739 shutdown_connections 2024-09-18T14:15:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0ac6c57640 1 -- 172.21.15.45:0/708851739 >> 172.21.15.45:0/708851739 conn(0x7f0ac00630b0 msgr2=0x7f0ac00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0ac6c57640 1 -- 172.21.15.45:0/708851739 shutdown_connections 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.637+0000 7f0ac6c57640 1 -- 172.21.15.45:0/708851739 wait complete. 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.638+0000 7f0ac6c57640 10 monclient: build_initial_monmap 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.638+0000 7f0ac6c57640 10 monclient: monmap: 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:05.652 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:05.652 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:05.652 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:05.652 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:05.652 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:05.652 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:05.652 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.638+0000 7f0ac6c57640 1 Processor -- start 2024-09-18T14:15:05.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.638+0000 7f0ac6c57640 1 -- start start 2024-09-18T14:15:05.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.638+0000 7f0ac6c57640 10 monclient: init 2024-09-18T14:15:05.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.638+0000 7f0ac6c57640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:05.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.638+0000 7f0ac6c57640 10 monclient: _add_conns ranks=[2,5,7,1,0,8,4,3,6] 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 10 monclient(hunting): picked mon.noname-b con 0x7f0ac009f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 10 monclient(hunting): picked mon.noname-h con 0x7f0ac009fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 10 monclient(hunting): picked mon.noname-i con 0x7f0ac009ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac009d310 con 0x7f0ac009f7d0 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac00a2560 con 0x7f0ac009fba0 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac0141e40 con 0x7f0ac009ff70 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 10 monclient(hunting): authenticate will time out at 1720.977067s 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac49f3640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f0ac009f7d0 0x7f0ac0141730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33130/0 (socket says 172.21.15.45:33130) 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac49f3640 1 -- 172.21.15.45:0/2318318937 learned_addr learned my addr 172.21.15.45:0/2318318937 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288114345 0 0) 0x7f0ac009d310 con 0x7f0ac009f7d0 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): my global_id is 145361 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa40034e0 con 0x7f0ac009f7d0 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2402859782 0 0) 0x7f0aa40034e0 con 0x7f0ac009f7d0 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0ac009d310 con 0x7f0ac009f7d0 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ab40015d0 con 0x7f0ac009f7d0 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): dump: 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.655 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.655 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.655 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.655 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.655 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.655 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.655 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.655 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.655 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.655 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:05.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741955658 0 0) 0x7f0ac0141e40 con 0x7f0ac009ff70 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): my global_id is 214261 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa40034e0 con 0x7f0ac009ff70 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738972255 0 0) 0x7f0ac00a2560 con 0x7f0ac009fba0 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient(hunting): my global_id is 184631 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ac0141e40 con 0x7f0ac009fba0 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3893688303 0 0) 0x7f0ac009d310 con 0x7f0ac009f7d0 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 >> v1:172.21.15.84:6792/0 conn(0x7f0ac009ff70 legacy=0x7f0ac0149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 >> v1:172.21.15.84:6791/0 conn(0x7f0ac009fba0 legacy=0x7f0ac0131910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient: _finish_hunting 0 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 monclient: found mon.b 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac01425f0 con 0x7f0ac009f7d0 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 10 monclient: _finish_auth 0 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 5 monclient: authenticate success, global_id 145361 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 10 monclient: _renew_subs 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac0120b00 con 0x7f0ac009f7d0 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 10 monclient: _renew_subs 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0ac6c57640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac011ff30 con 0x7f0ac009f7d0 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.639+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0ab4004a50 con 0x7f0ac009f7d0 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.640+0000 7f0abcff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.640+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ab4004f00 con 0x7f0ac009f7d0 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.640+0000 7f0abcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.640+0000 7f0abe7fc640 4 set_mon_vals no callback set 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.640+0000 7f0abcff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.640+0000 7f0abcff9640 10 monclient: dump: 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.640+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f0ab400fed0 con 0x7f0ac009f7d0 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.640+0000 7f0ac6c57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.640+0000 7f0ac6c57640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0aa0006580 con 0x7f0aa0002830 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.640+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(740..740 src has 529..740) v4 ==== 3959+0+0 (unknown 1715459358 0 0) 0x7f0ab4046be0 con 0x7f0ac009f7d0 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7099405 0 0) 0x7f0aa0006580 con 0x7f0aa0002830 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 10 monclient(hunting): my global_id is 145370 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ac009d310 con 0x7f0aa0002830 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0ac51f4640 10 monclient: get_auth_request con 0x7f0aa4039f40 auth_method 0 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 718222091 0 0) 0x7f0ac009d310 con 0x7f0aa0002830 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0aa0006580 con 0x7f0aa0002830 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0aac004ae0 con 0x7f0aa0002830 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 10 monclient: dump: 2024-09-18T14:15:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.662 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1252889770 0 0) 0x7f0aa0006580 con 0x7f0aa0002830 2024-09-18T14:15:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0aa000a0a0 con 0x7f0aa0002830 2024-09-18T14:15:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0aac001570 con 0x7f0aa0002830 2024-09-18T14:15:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 >> v1:172.21.15.45:6790/0 conn(0x7f0aa0002830 legacy=0x7f0aa0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.641+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 >> v1:172.21.15.45:6790/0 conn(0x7f0aa0002830 legacy=0x7f0aa0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:05.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.647+0000 7f0ac6c57640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:05.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.647+0000 7f0ac6c57640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0aa0009640 con 0x7f0aa0009020 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.647+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205951137 0 0) 0x7f0aa0009640 con 0x7f0aa0009020 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.647+0000 7f0abcff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.647+0000 7f0abcff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.647+0000 7f0abcff9640 10 monclient(hunting): my global_id is 145379 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.647+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa0006580 con 0x7f0aa0009020 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3970682011 0 0) 0x7f0aa0006580 con 0x7f0aa0009020 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0aa0009640 con 0x7f0aa0009020 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0aac0053c0 con 0x7f0aa0009020 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 10 monclient: dump: 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.665 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3615598392 0 0) 0x7f0aa0009640 con 0x7f0aa0009020 2024-09-18T14:15:05.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0aa0009900 con 0x7f0aa0009020 2024-09-18T14:15:05.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 4124180581) 0x7f0aac007640 con 0x7f0aa0009020 2024-09-18T14:15:05.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:05.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:05.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 >> v1:172.21.15.45:6790/0 conn(0x7f0aa0009020 legacy=0x7f0aa0008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0abcff9640 1 -- 172.21.15.45:0/2318318937 >> v1:172.21.15.45:6790/0 conn(0x7f0aa0009020 legacy=0x7f0aa0008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:05.666 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:05.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:05.667 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:05.669 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:05.670 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:05.671 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:05.671 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:05.671 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:05.671 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:05.671 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:05.671 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:05.671 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:05.671 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:05.671 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:05.671 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:05.671 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:05.671 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:05.674 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:05.675 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:05.676 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.677 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:05.678 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:05.679 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:05.680 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.681 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.684 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:05.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.686 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.686 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:05.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.686 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:05.686 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:05.686 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:15:05.686 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.686 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.686 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:05.686 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0ac6c57640 1 -- 172.21.15.45:0/2318318937 >> v1:172.21.15.84:6812/31478 conn(0x7f0aa4039f40 legacy=0x7f0aa403c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0ac6c57640 10 monclient: shutdown 2024-09-18T14:15:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0ac6c57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0ac6c57640 1 -- 172.21.15.45:0/2318318937 >> v1:172.21.15.45:6790/0 conn(0x7f0ac009f7d0 legacy=0x7f0ac0141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.648+0000 7f0ac51f4640 1 -- 172.21.15.45:0/2318318937 reap_dead start 2024-09-18T14:15:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.649+0000 7f0ac51f4640 1 -- 172.21.15.45:0/2318318937 reap_dead start 2024-09-18T14:15:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.649+0000 7f0ac6c57640 1 -- 172.21.15.45:0/2318318937 shutdown_connections 2024-09-18T14:15:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.649+0000 7f0ac6c57640 1 -- 172.21.15.45:0/2318318937 >> 172.21.15.45:0/2318318937 conn(0x7f0ac00630b0 msgr2=0x7f0ac00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.649+0000 7f0ac6c57640 1 -- 172.21.15.45:0/2318318937 shutdown_connections 2024-09-18T14:15:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.649+0000 7f0ac6c57640 1 -- 172.21.15.45:0/2318318937 wait complete. 2024-09-18T14:15:05.689 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:05.689 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:15:05.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:15:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.787+0000 7f7949354640 10 monclient: get_monmap_and_config 2024-09-18T14:15:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.787+0000 7f7949354640 10 monclient: build_initial_monmap 2024-09-18T14:15:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.787+0000 7f7949354640 10 monclient: monmap: 2024-09-18T14:15:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:05.788255+0000 2024-09-18T14:15:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:05.788255+0000 2024-09-18T14:15:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:05.789 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.787+0000 7f7949354640 1 Processor -- start 2024-09-18T14:15:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.787+0000 7f7949354640 1 -- start start 2024-09-18T14:15:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.787+0000 7f7949354640 10 monclient: init 2024-09-18T14:15:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7949354640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7949354640 10 monclient: _add_conns ranks=[2,4,7,0,8,3,1,5,6] 2024-09-18T14:15:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7949354640 10 monclient(hunting): picked mon.noname-b con 0x7f794409f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7949354640 10 monclient(hunting): picked mon.noname-c con 0x7f794409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7949354640 10 monclient(hunting): picked mon.noname-i con 0x7f794409ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7949354640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794405ed70 con 0x7f794409f7d0 2024-09-18T14:15:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7949354640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794409dfc0 con 0x7f794409fba0 2024-09-18T14:15:05.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7949354640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794409dce0 con 0x7f794409ff70 2024-09-18T14:15:05.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7949354640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:05.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7949354640 10 monclient(hunting): authenticate will time out at 1721.126498s 2024-09-18T14:15:05.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f79427fc640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f794409fba0 0x7f7944137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39662/0 (socket says 172.21.15.45:39662) 2024-09-18T14:15:05.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f79427fc640 1 -- 172.21.15.45:0/1974662332 learned_addr learned my addr 172.21.15.45:0/1974662332 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:05.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316308998 0 0) 0x7f794409dfc0 con 0x7f794409fba0 2024-09-18T14:15:05.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7941ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7941ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7941ffb640 10 monclient(hunting): my global_id is 164938 2024-09-18T14:15:05.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f792c0034e0 con 0x7f794409fba0 2024-09-18T14:15:05.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620119025 0 0) 0x7f794405ed70 con 0x7f794409f7d0 2024-09-18T14:15:05.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7941ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7941ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7941ffb640 10 monclient(hunting): my global_id is 145388 2024-09-18T14:15:05.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.788+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f794409dfc0 con 0x7f794409f7d0 2024-09-18T14:15:05.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879694818 0 0) 0x7f794409dce0 con 0x7f794409ff70 2024-09-18T14:15:05.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient(hunting): my global_id is 214270 2024-09-18T14:15:05.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f794405ed70 con 0x7f794409ff70 2024-09-18T14:15:05.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3331116007 0 0) 0x7f794409dfc0 con 0x7f794409f7d0 2024-09-18T14:15:05.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 >> v1:172.21.15.84:6792/0 conn(0x7f794409ff70 legacy=0x7f794413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 >> v1:172.21.15.45:6791/0 conn(0x7f794409fba0 legacy=0x7f7944137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:15:05.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 1 monclient: found mon.noname-b 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79440a1c00 con 0x7f794409f7d0 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient: _finish_auth 0 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7934001790 con 0x7f794409f7d0 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7949354640 5 monclient: authenticate success, global_id 145388 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient: dump: 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.802 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.803 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7949354640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7934001970 con 0x7f794409f7d0 2024-09-18T14:15:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7949354640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 1 -- 172.21.15.45:0/1974662332 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7934001f20 con 0x7f794409f7d0 2024-09-18T14:15:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7949354640 4 set_mon_vals no callback set 2024-09-18T14:15:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7941ffb640 10 monclient: dump: 2024-09-18T14:15:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.805 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.805 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.805 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.805 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.805 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.805 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.805 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.805 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.805 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.805 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.805 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.806 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7949354640 10 monclient: shutdown 2024-09-18T14:15:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7949354640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7949354640 1 -- 172.21.15.45:0/1974662332 >> v1:172.21.15.45:6790/0 conn(0x7f794409f7d0 legacy=0x7f7944132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7949354640 1 -- 172.21.15.45:0/1974662332 shutdown_connections 2024-09-18T14:15:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7949354640 1 -- 172.21.15.45:0/1974662332 >> 172.21.15.45:0/1974662332 conn(0x7f79440630b0 msgr2=0x7f79440634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7949354640 1 -- 172.21.15.45:0/1974662332 shutdown_connections 2024-09-18T14:15:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.789+0000 7f7949354640 1 -- 172.21.15.45:0/1974662332 wait complete. 2024-09-18T14:15:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.790+0000 7f7949354640 10 monclient: build_initial_monmap 2024-09-18T14:15:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.790+0000 7f7949354640 10 monclient: monmap: 2024-09-18T14:15:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:05.808 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.790+0000 7f7949354640 1 Processor -- start 2024-09-18T14:15:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.790+0000 7f7949354640 1 -- start start 2024-09-18T14:15:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.790+0000 7f7949354640 10 monclient: init 2024-09-18T14:15:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f7949354640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f7949354640 10 monclient: _add_conns ranks=[1,0,3,8,5,6,4,2,7] 2024-09-18T14:15:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f7949354640 10 monclient(hunting): picked mon.noname-f con 0x7f794409f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f7949354640 10 monclient(hunting): picked mon.noname-a con 0x7f794409fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f7949354640 10 monclient(hunting): picked mon.noname-g con 0x7f794409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f7949354640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7944142660 con 0x7f794409fba0 2024-09-18T14:15:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f7949354640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794409d310 con 0x7f794409f7d0 2024-09-18T14:15:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f7949354640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79440a2560 con 0x7f794409ff70 2024-09-18T14:15:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f7949354640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f7949354640 10 monclient(hunting): authenticate will time out at 1721.129428s 2024-09-18T14:15:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f79427fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f794409fba0 0x7f7944131a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56448/0 (socket says 172.21.15.45:56448) 2024-09-18T14:15:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f79427fc640 1 -- 172.21.15.45:0/1711131666 learned_addr learned my addr 172.21.15.45:0/1711131666 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759914911 0 0) 0x7f7944142660 con 0x7f794409fba0 2024-09-18T14:15:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 10 monclient(hunting): my global_id is 6237 2024-09-18T14:15:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79300034e0 con 0x7f794409fba0 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011685313 0 0) 0x7f79440a2560 con 0x7f794409ff70 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 10 monclient(hunting): my global_id is 184692 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7944142660 con 0x7f794409ff70 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55103164 0 0) 0x7f794409d310 con 0x7f794409f7d0 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 10 monclient(hunting): my global_id is 214264 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79440a2560 con 0x7f794409f7d0 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2976381517 0 0) 0x7f79300034e0 con 0x7f794409fba0 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f794409d310 con 0x7f794409fba0 2024-09-18T14:15:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7938001670 con 0x7f794409fba0 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.791+0000 7f792affd640 10 monclient(hunting): dump: 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 458948255 0 0) 0x7f794409d310 con 0x7f794409fba0 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 >> v1:172.21.15.84:6790/0 conn(0x7f794409ff70 legacy=0x7f7944149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 >> v1:172.21.15.84:6789/0 conn(0x7f794409f7d0 legacy=0x7f79441417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 1 monclient: found mon.a 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79441207d0 con 0x7f794409fba0 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 10 monclient: _finish_auth 0 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f7949354640 5 monclient: authenticate success, global_id 6237 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f7949354640 10 monclient: _renew_subs 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f7949354640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f7949354640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7944121b90 con 0x7f794409fba0 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f7949354640 10 monclient: _renew_subs 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f7949354640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f7949354640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7944120c30 con 0x7f794409fba0 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7938004a50 con 0x7f794409fba0 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7938004f00 con 0x7f794409fba0 2024-09-18T14:15:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f79417fa640 4 set_mon_vals no callback set 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.792+0000 7f792affd640 10 monclient: dump: 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f7949354640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f7949354640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7924006580 con 0x7f7924002830 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f793800fed0 con 0x7f794409fba0 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(740..740 src has 529..740) v4 ==== 3959+0+0 (unknown 1715459358 0 0) 0x7f7938046ae0 con 0x7f794409fba0 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959081972 0 0) 0x7f7924006580 con 0x7f7924002830 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 10 monclient(hunting): my global_id is 164947 2024-09-18T14:15:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f794409d310 con 0x7f7924002830 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f79437fe640 10 monclient: get_auth_request con 0x7f7930039f50 auth_method 0 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1785300307 0 0) 0x7f794409d310 con 0x7f7924002830 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7924006580 con 0x7f7924002830 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f79340017c0 con 0x7f7924002830 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 10 monclient: dump: 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2838229727 0 0) 0x7f7924006580 con 0x7f7924002830 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.793+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f792400a0a0 con 0x7f7924002830 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.794+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7934004680 con 0x7f7924002830 2024-09-18T14:15:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.794+0000 7f792affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:05.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.794+0000 7f792affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:05.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.794+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 >> v1:172.21.15.45:6791/0 conn(0x7f7924002830 legacy=0x7f7924004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.794+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 >> v1:172.21.15.45:6791/0 conn(0x7f7924002830 legacy=0x7f7924004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.799+0000 7f7949354640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.799+0000 7f7949354640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7924007540 con 0x7f79240068f0 2024-09-18T14:15:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098709934 0 0) 0x7f7924007540 con 0x7f79240068f0 2024-09-18T14:15:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 10 monclient(hunting): my global_id is 164956 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7924006580 con 0x7f79240068f0 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1603371195 0 0) 0x7f7924006580 con 0x7f79240068f0 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7924007540 con 0x7f79240068f0 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7934002ec0 con 0x7f79240068f0 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 10 monclient: dump: 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 16991044 0 0) 0x7f7924007540 con 0x7f79240068f0 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f79240087a0 con 0x7f79240068f0 2024-09-18T14:15:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 457335220) 0x7f7934004370 con 0x7f79240068f0 2024-09-18T14:15:05.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:05.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:05.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 >> v1:172.21.15.45:6791/0 conn(0x7f79240068f0 legacy=0x7f7924006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.800+0000 7f792affd640 1 -- 172.21.15.45:0/1711131666 >> v1:172.21.15.45:6791/0 conn(0x7f79240068f0 legacy=0x7f7924006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:05.822 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:05.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:05.822 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:05.822 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:05.822 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:05.822 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:05.822 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:05.823 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:05.824 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:05.825 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:05.826 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.829 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:05.829 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:05.829 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.829 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.829 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.829 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:05.831 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.832 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.832 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.832 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:05.832 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:05.832 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.832 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.832 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.832 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:05.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.834 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.836 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:05.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:05.837 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:05.837 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.837 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.837 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:05.837 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:05.839 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:05.840 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:05.841 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:05.842 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.801+0000 7f7949354640 1 -- 172.21.15.45:0/1711131666 >> v1:172.21.15.84:6812/31478 conn(0x7f7930039f50 legacy=0x7f793003c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.801+0000 7f7949354640 10 monclient: shutdown 2024-09-18T14:15:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.801+0000 7f7949354640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.801+0000 7f7949354640 1 -- 172.21.15.45:0/1711131666 >> v1:172.21.15.45:6789/0 conn(0x7f794409fba0 legacy=0x7f7944131a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.801+0000 7f79437fe640 1 -- 172.21.15.45:0/1711131666 reap_dead start 2024-09-18T14:15:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.801+0000 7f79437fe640 1 -- 172.21.15.45:0/1711131666 reap_dead start 2024-09-18T14:15:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.801+0000 7f7949354640 1 -- 172.21.15.45:0/1711131666 shutdown_connections 2024-09-18T14:15:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.801+0000 7f7949354640 1 -- 172.21.15.45:0/1711131666 >> 172.21.15.45:0/1711131666 conn(0x7f79440630b0 msgr2=0x7f79440a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.801+0000 7f7949354640 1 -- 172.21.15.45:0/1711131666 shutdown_connections 2024-09-18T14:15:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.801+0000 7f7949354640 1 -- 172.21.15.45:0/1711131666 wait complete. 2024-09-18T14:15:05.845 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:05.845 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:15:05.845 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:15:05.862 INFO:tasks.rados.rados.0.smithi084.stdout:3371: done (0 left) 2024-09-18T14:15:05.862 INFO:tasks.rados.rados.0.smithi084.stdout:3372: read oid 3 snap 354 2024-09-18T14:15:05.863 INFO:tasks.rados.rados.0.smithi084.stdout:3372: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:15:05.863 INFO:tasks.rados.rados.0.smithi084.stdout:3373: write (excl) oid 45 current snap is 370 2024-09-18T14:15:05.863 INFO:tasks.rados.rados.0.smithi084.stdout:3373: seq_num 1147 ranges {477629=563553,1836611=777404,3048408=229221} 2024-09-18T14:15:05.868 INFO:tasks.rados.rados.0.smithi084.stdout:3373: writing smithi08434663-45 from 477629 to 1041182 tid 1 2024-09-18T14:15:05.878 INFO:tasks.rados.rados.0.smithi084.stdout:3373: writing smithi08434663-45 from 1836611 to 2614015 tid 2 2024-09-18T14:15:05.884 INFO:tasks.rados.rados.0.smithi084.stdout:3373: writing smithi08434663-45 from 3048408 to 3277629 tid 3 2024-09-18T14:15:05.885 INFO:tasks.rados.rados.0.smithi084.stdout:3374: snap_create 2024-09-18T14:15:05.887 INFO:tasks.rados.rados.0.smithi084.stdout:3372: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:15:05.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.960+0000 7fb356caf640 10 monclient: get_monmap_and_config 2024-09-18T14:15:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.960+0000 7fb356caf640 10 monclient: build_initial_monmap 2024-09-18T14:15:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.960+0000 7fb356caf640 10 monclient: monmap: 2024-09-18T14:15:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:05.961568+0000 2024-09-18T14:15:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:05.961568+0000 2024-09-18T14:15:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.962 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:05.962 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:05.962 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:05.962 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:05.962 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:05.962 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:05.962 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:05.962 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:05.962 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:05.962 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.961+0000 7fb356caf640 1 Processor -- start 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.961+0000 7fb356caf640 1 -- start start 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.961+0000 7fb356caf640 10 monclient: init 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.961+0000 7fb356caf640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.961+0000 7fb356caf640 10 monclient: _add_conns ranks=[3,7,4,5,6,1,2,0,8] 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.961+0000 7fb356caf640 10 monclient(hunting): picked mon.noname-g con 0x7fb35009f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb356caf640 10 monclient(hunting): picked mon.noname-i con 0x7fb35009fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb356caf640 10 monclient(hunting): picked mon.noname-c con 0x7fb35009ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb356caf640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35005ed70 con 0x7fb35009f7d0 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb356caf640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35009dfc0 con 0x7fb35009ff70 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb356caf640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35009dce0 con 0x7fb35009fba0 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb356caf640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb356caf640 10 monclient(hunting): authenticate will time out at 1721.300110s 2024-09-18T14:15:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb354a4b640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7fb35009f7d0 0x7fb350132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:36136/0 (socket says 172.21.15.45:36136) 2024-09-18T14:15:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34ffff640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7fb35009fba0 0x7fb350137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:34880/0 (socket says 172.21.15.45:34880) 2024-09-18T14:15:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb354a4b640 1 -- 172.21.15.45:0/1071434814 learned_addr learned my addr 172.21.15.45:0/1071434814 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb35524c640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fb35009ff70 0x7fb35013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39678/0 (socket says 172.21.15.45:39678) 2024-09-18T14:15:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701180139 0 0) 0x7fb35009dfc0 con 0x7fb35009ff70 2024-09-18T14:15:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient(hunting): my global_id is 164965 2024-09-18T14:15:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3400034e0 con 0x7fb35009ff70 2024-09-18T14:15:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571719119 0 0) 0x7fb35005ed70 con 0x7fb35009f7d0 2024-09-18T14:15:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient(hunting): my global_id is 184701 2024-09-18T14:15:05.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb35009dfc0 con 0x7fb35009f7d0 2024-09-18T14:15:05.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490493494 0 0) 0x7fb35009dce0 con 0x7fb35009fba0 2024-09-18T14:15:05.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient(hunting): my global_id is 214279 2024-09-18T14:15:05.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb35005ed70 con 0x7fb35009fba0 2024-09-18T14:15:05.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34ffff640 1 -- 172.21.15.45:0/1071434814 >> v1:172.21.15.84:6792/0 conn(0x7fb35009fba0 legacy=0x7fb350137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-18T14:15:05.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34ffff640 1 -- 172.21.15.45:0/1071434814 >> v1:172.21.15.84:6792/0 conn(0x7fb35009fba0 legacy=0x7fb350137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:15:05.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34ffff640 1 --1- 172.21.15.45:0/1071434814 >> v1:172.21.15.84:6792/0 conn(0x7fb35009fba0 0x7fb350137a60 :-1 s=OPENED pgs=443 cs=1 l=1).handle_message read tag failed 2024-09-18T14:15:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34ffff640 1 --1- 172.21.15.45:0/1071434814 >> v1:172.21.15.84:6792/0 conn(0x7fb35009fba0 0x7fb350137a60 :-1 s=OPENED pgs=443 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:15:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.84:6792/0 2024-09-18T14:15:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3175941014 0 0) 0x7fb3400034e0 con 0x7fb35009ff70 2024-09-18T14:15:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 >> v1:172.21.15.84:6792/0 conn(0x7fb35009fba0 legacy=0x7fb350137a60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 >> v1:172.21.15.84:6790/0 conn(0x7fb35009f7d0 legacy=0x7fb350132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:15:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 monclient: found mon.noname-c 2024-09-18T14:15:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:15:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3500a1c00 con 0x7fb35009ff70 2024-09-18T14:15:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient: _finish_auth 0 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb356caf640 5 monclient: authenticate success, global_id 164965 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb348001bb0 con 0x7fb35009ff70 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient: dump: 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.977 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb356caf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb348001d90 con 0x7fb35009ff70 2024-09-18T14:15:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb356caf640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 1 -- 172.21.15.45:0/1071434814 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb348004e60 con 0x7fb35009ff70 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb356caf640 4 set_mon_vals no callback set 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.962+0000 7fb34f7fe640 10 monclient: dump: 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.963+0000 7fb356caf640 10 monclient: shutdown 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.963+0000 7fb356caf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.963+0000 7fb356caf640 1 -- 172.21.15.45:0/1071434814 >> v1:172.21.15.45:6791/0 conn(0x7fb35009ff70 legacy=0x7fb35013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.963+0000 7fb356caf640 1 -- 172.21.15.45:0/1071434814 shutdown_connections 2024-09-18T14:15:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.963+0000 7fb356caf640 1 -- 172.21.15.45:0/1071434814 >> 172.21.15.45:0/1071434814 conn(0x7fb3500630b0 msgr2=0x7fb3500634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.963+0000 7fb356caf640 1 -- 172.21.15.45:0/1071434814 shutdown_connections 2024-09-18T14:15:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.963+0000 7fb356caf640 1 -- 172.21.15.45:0/1071434814 wait complete. 2024-09-18T14:15:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.963+0000 7fb356caf640 10 monclient: build_initial_monmap 2024-09-18T14:15:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.963+0000 7fb356caf640 10 monclient: monmap: 2024-09-18T14:15:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 1 Processor -- start 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 1 -- start start 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 10 monclient: init 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 10 monclient: _add_conns ranks=[7,0,6,4,8,2,1,3,5] 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 10 monclient(hunting): picked mon.noname-i con 0x7fb35009f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 10 monclient(hunting): picked mon.noname-a con 0x7fb35009fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 10 monclient(hunting): picked mon.noname-d con 0x7fb35009ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3500a2560 con 0x7fb35009fba0 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35009b870 con 0x7fb35009ff70 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35011e790 con 0x7fb35009f7d0 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb356caf640 10 monclient(hunting): authenticate will time out at 1721.302855s 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb34ffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb35009fba0 0x7fb350146cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56450/0 (socket says 172.21.15.45:56450) 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.964+0000 7fb34ffff640 1 -- 172.21.15.45:0/188635342 learned_addr learned my addr 172.21.15.45:0/188635342 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524348108 0 0) 0x7fb3500a2560 con 0x7fb35009fba0 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): my global_id is 6246 2024-09-18T14:15:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3340034e0 con 0x7fb35009fba0 2024-09-18T14:15:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776849784 0 0) 0x7fb35011e790 con 0x7fb35009f7d0 2024-09-18T14:15:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): my global_id is 214288 2024-09-18T14:15:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3500a2560 con 0x7fb35009f7d0 2024-09-18T14:15:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472351460 0 0) 0x7fb35009b870 con 0x7fb35009ff70 2024-09-18T14:15:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): my global_id is 184758 2024-09-18T14:15:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb35011e790 con 0x7fb35009ff70 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 703358773 0 0) 0x7fb3340034e0 con 0x7fb35009fba0 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb35009b870 con 0x7fb35009fba0 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1945779355 0 0) 0x7fb3500a2560 con 0x7fb35009f7d0 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb3340034e0 con 0x7fb35009f7d0 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb33c001bb0 con 0x7fb35009fba0 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): dump: 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3668142230 0 0) 0x7fb35011e790 con 0x7fb35009ff70 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb3500a2560 con 0x7fb35009ff70 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb344001a20 con 0x7fb35009f7d0 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): dump: 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb348001af0 con 0x7fb35009ff70 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient(hunting): dump: 2024-09-18T14:15:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 535213417 0 0) 0x7fb35009b870 con 0x7fb35009fba0 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 >> v1:172.21.15.84:6792/0 conn(0x7fb35009f7d0 legacy=0x7fb350131670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 >> v1:172.21.15.45:6792/0 conn(0x7fb35009ff70 legacy=0x7fb35014a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient: _finish_hunting 0 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 monclient: found mon.a 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb350136ea0 con 0x7fb35009fba0 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient: _finish_auth 0 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb356caf640 5 monclient: authenticate success, global_id 6246 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb356caf640 10 monclient: _renew_subs 2024-09-18T14:15:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb356caf640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb356caf640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35011d4a0 con 0x7fb35009fba0 2024-09-18T14:15:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb356caf640 10 monclient: _renew_subs 2024-09-18T14:15:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb356caf640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb356caf640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb350131f20 con 0x7fb35009fba0 2024-09-18T14:15:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb33c001d10 con 0x7fb35009fba0 2024-09-18T14:15:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb33c004e50 con 0x7fb35009fba0 2024-09-18T14:15:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34effd640 4 set_mon_vals no callback set 2024-09-18T14:15:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.965+0000 7fb34cff9640 10 monclient: dump: 2024-09-18T14:15:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.966+0000 7fb356caf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.966+0000 7fb356caf640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb330006580 con 0x7fb330002830 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.966+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fb33c004fb0 con 0x7fb35009fba0 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.966+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(741..741 src has 529..741) v4 ==== 4015+0+0 (unknown 1961392220 0 0) 0x7fb33c046bd0 con 0x7fb35009fba0 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853745573 0 0) 0x7fb330006580 con 0x7fb330002830 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 10 monclient(hunting): my global_id is 184767 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb35009b870 con 0x7fb330002830 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb35524c640 10 monclient: get_auth_request con 0x7fb33403aa00 auth_method 0 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 45170158 0 0) 0x7fb35009b870 con 0x7fb330002830 2024-09-18T14:15:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb330006580 con 0x7fb330002830 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb344004dd0 con 0x7fb330002830 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 10 monclient: dump: 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2087675761 0 0) 0x7fb330006580 con 0x7fb330002830 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb33000a0a0 con 0x7fb330002830 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb344005170 con 0x7fb330002830 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 >> v1:172.21.15.45:6792/0 conn(0x7fb330002830 legacy=0x7fb330004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.967+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 >> v1:172.21.15.45:6792/0 conn(0x7fb330002830 legacy=0x7fb330004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb356caf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb356caf640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb330009640 con 0x7fb330009020 2024-09-18T14:15:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785052219 0 0) 0x7fb330009640 con 0x7fb330009020 2024-09-18T14:15:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb34cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb34cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb34cff9640 10 monclient(hunting): my global_id is 184776 2024-09-18T14:15:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb330006580 con 0x7fb330009020 2024-09-18T14:15:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1181914467 0 0) 0x7fb330006580 con 0x7fb330009020 2024-09-18T14:15:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb330009640 con 0x7fb330009020 2024-09-18T14:15:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb344005bd0 con 0x7fb330009020 2024-09-18T14:15:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb34cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb34cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.973+0000 7fb34cff9640 10 monclient: dump: 2024-09-18T14:15:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:05.997 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:05.998 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1210435330 0 0) 0x7fb330009640 con 0x7fb330009020 2024-09-18T14:15:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb330009900 con 0x7fb330009020 2024-09-18T14:15:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 331498372) 0x7fb344007d90 con 0x7fb330009020 2024-09-18T14:15:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb34cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb34cff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 >> v1:172.21.15.45:6792/0 conn(0x7fb330009020 legacy=0x7fb330008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb34cff9640 1 -- 172.21.15.45:0/188635342 >> v1:172.21.15.45:6792/0 conn(0x7fb330009020 legacy=0x7fb330008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:05.999 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:05.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:05.999 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:05.999 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:06.000 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:06.000 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:06.000 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:06.000 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:06.000 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:06.000 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:06.000 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:06.000 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:06.000 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:06.000 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:06.000 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:15:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.002 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:06.003 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:06.004 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:06.004 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:06.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:06.004 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:06.004 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:06.004 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:06.004 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.004 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.004 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.004 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.004 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.007 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:06.008 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.010 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:06.011 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.012 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.013 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:06.014 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:06.015 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.016 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:06.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.019 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:06.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.022 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:06.022 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:06.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb356caf640 1 -- 172.21.15.45:0/188635342 >> v1:172.21.15.84:6812/31478 conn(0x7fb33403aa00 legacy=0x7fb33403ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb356caf640 10 monclient: shutdown 2024-09-18T14:15:06.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb356caf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:06.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb356caf640 1 -- 172.21.15.45:0/188635342 >> v1:172.21.15.45:6789/0 conn(0x7fb35009fba0 legacy=0x7fb350146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb35524c640 1 -- 172.21.15.45:0/188635342 reap_dead start 2024-09-18T14:15:06.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb35524c640 1 -- 172.21.15.45:0/188635342 reap_dead start 2024-09-18T14:15:06.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb356caf640 1 -- 172.21.15.45:0/188635342 shutdown_connections 2024-09-18T14:15:06.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb356caf640 1 -- 172.21.15.45:0/188635342 >> 172.21.15.45:0/188635342 conn(0x7fb3500630b0 msgr2=0x7fb3500a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:06.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb356caf640 1 -- 172.21.15.45:0/188635342 shutdown_connections 2024-09-18T14:15:06.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:05.974+0000 7fb356caf640 1 -- 172.21.15.45:0/188635342 wait complete. 2024-09-18T14:15:06.024 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:06.024 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:15:06.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:15:06.043 INFO:tasks.rados.rados.0.smithi084.stdout:3373: finishing write tid 1 to smithi08434663-45 2024-09-18T14:15:06.043 INFO:tasks.rados.rados.0.smithi084.stdout:3373: finishing write tid 2 to smithi08434663-45 2024-09-18T14:15:06.043 INFO:tasks.rados.rados.0.smithi084.stdout:3373: finishing write tid 3 to smithi08434663-45 2024-09-18T14:15:06.043 INFO:tasks.rados.rados.0.smithi084.stdout:3373: finishing write tid 4 to smithi08434663-45 2024-09-18T14:15:06.043 INFO:tasks.rados.rados.0.smithi084.stdout:3373: finishing write tid 5 to smithi08434663-45 2024-09-18T14:15:06.043 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 1703 (ObjNum 1147 snap 370 seq_num 1147) dirty exists 2024-09-18T14:15:06.043 INFO:tasks.rados.rados.0.smithi084.stdout:3373: left oid 45 (ObjNum 1147 snap 370 seq_num 1147) 2024-09-18T14:15:06.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.121+0000 7f692219d640 10 monclient: get_monmap_and_config 2024-09-18T14:15:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.121+0000 7f692219d640 10 monclient: build_initial_monmap 2024-09-18T14:15:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.121+0000 7f692219d640 10 monclient: monmap: 2024-09-18T14:15:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:06.122522+0000 2024-09-18T14:15:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:06.122522+0000 2024-09-18T14:15:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:06.123 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 1 Processor -- start 2024-09-18T14:15:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 1 -- start start 2024-09-18T14:15:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 10 monclient: init 2024-09-18T14:15:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 10 monclient: _add_conns ranks=[5,2,8,7,6,0,4,3,1] 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 10 monclient(hunting): picked mon.noname-h con 0x7f691c09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 10 monclient(hunting): picked mon.noname-b con 0x7f691c09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 10 monclient(hunting): picked mon.noname-e con 0x7f691c09ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c05ed70 con 0x7f691c09fba0 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c09dfc0 con 0x7f691c09f7d0 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c09dce0 con 0x7f691c09ff70 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f692219d640 10 monclient(hunting): authenticate will time out at 1721.460843s 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f691affd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f691c09fba0 0x7f691c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33170/0 (socket says 172.21.15.45:33170) 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f691affd640 1 -- 172.21.15.45:0/1926336674 learned_addr learned my addr 172.21.15.45:0/1926336674 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.122+0000 7f691bfff640 1 --1- 172.21.15.45:0/1926336674 >> v1:172.21.15.45:6793/0 conn(0x7f691c09ff70 0x7f691c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34226/0 (socket says 172.21.15.45:34226) 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836183411 0 0) 0x7f691c09dce0 con 0x7f691c09ff70 2024-09-18T14:15:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient(hunting): my global_id is 5921 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69040034e0 con 0x7f691c09ff70 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814968443 0 0) 0x7f691c05ed70 con 0x7f691c09fba0 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient(hunting): my global_id is 145397 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f691c09dce0 con 0x7f691c09fba0 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4287978323 0 0) 0x7f691c09dfc0 con 0x7f691c09f7d0 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient(hunting): my global_id is 184640 2024-09-18T14:15:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f691c05ed70 con 0x7f691c09f7d0 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1050732570 0 0) 0x7f69040034e0 con 0x7f691c09ff70 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 >> v1:172.21.15.84:6791/0 conn(0x7f691c09f7d0 legacy=0x7f691c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 >> v1:172.21.15.45:6790/0 conn(0x7f691c09fba0 legacy=0x7f691c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 monclient: found mon.noname-e 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c0a1c00 con 0x7f691c09ff70 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6914001ae0 con 0x7f691c09ff70 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:15:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient: dump: 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f692219d640 5 monclient: authenticate success, global_id 5921 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f692219d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6914001c40 con 0x7f691c09ff70 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 1 -- 172.21.15.45:0/1926336674 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6914004b20 con 0x7f691c09ff70 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f691a7fc640 10 monclient: dump: 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f692219d640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.123+0000 7f692219d640 4 set_mon_vals no callback set 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.124+0000 7f692219d640 10 monclient: shutdown 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.124+0000 7f692219d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.124+0000 7f692219d640 1 -- 172.21.15.45:0/1926336674 >> v1:172.21.15.45:6793/0 conn(0x7f691c09ff70 legacy=0x7f691c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.124+0000 7f692219d640 1 -- 172.21.15.45:0/1926336674 shutdown_connections 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.124+0000 7f692219d640 1 -- 172.21.15.45:0/1926336674 >> 172.21.15.45:0/1926336674 conn(0x7f691c0630b0 msgr2=0x7f691c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.124+0000 7f692219d640 1 -- 172.21.15.45:0/1926336674 shutdown_connections 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.124+0000 7f692219d640 1 -- 172.21.15.45:0/1926336674 wait complete. 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.124+0000 7f692219d640 10 monclient: build_initial_monmap 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.124+0000 7f692219d640 10 monclient: monmap: 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 1 Processor -- start 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 1 -- start start 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 10 monclient: init 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 10 monclient: _add_conns ranks=[5,4,7,3,8,6,2,1,0] 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 10 monclient(hunting): picked mon.noname-h con 0x7f691c09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 10 monclient(hunting): picked mon.noname-c con 0x7f691c09fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 10 monclient(hunting): picked mon.noname-i con 0x7f691c09ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c09d310 con 0x7f691c09fba0 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c0a2560 con 0x7f691c09f7d0 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c09be50 con 0x7f691c09ff70 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f692219d640 10 monclient(hunting): authenticate will time out at 1721.463822s 2024-09-18T14:15:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f691affd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f691c09fba0 0x7f691c09ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39684/0 (socket says 172.21.15.45:39684) 2024-09-18T14:15:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.125+0000 7f691affd640 1 -- 172.21.15.45:0/1985652355 learned_addr learned my addr 172.21.15.45:0/1985652355 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828114304 0 0) 0x7f691c09d310 con 0x7f691c09fba0 2024-09-18T14:15:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): my global_id is 164974 2024-09-18T14:15:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69080034e0 con 0x7f691c09fba0 2024-09-18T14:15:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199744999 0 0) 0x7f691c0a2560 con 0x7f691c09f7d0 2024-09-18T14:15:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): my global_id is 184649 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f691c09d310 con 0x7f691c09f7d0 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239402813 0 0) 0x7f691c09be50 con 0x7f691c09ff70 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): my global_id is 214297 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f691c0a2560 con 0x7f691c09ff70 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1136925930 0 0) 0x7f691c09d310 con 0x7f691c09f7d0 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f691c09be50 con 0x7f691c09f7d0 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1913615274 0 0) 0x7f69080034e0 con 0x7f691c09fba0 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f691c09d310 con 0x7f691c09fba0 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f690c001660 con 0x7f691c09f7d0 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): dump: 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.144 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6910001610 con 0x7f691c09fba0 2024-09-18T14:15:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): dump: 2024-09-18T14:15:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3594857692 0 0) 0x7f691c0a2560 con 0x7f691c09ff70 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f69080034e0 con 0x7f691c09ff70 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6914001a20 con 0x7f691c09ff70 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient(hunting): dump: 2024-09-18T14:15:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3772987770 0 0) 0x7f691c09be50 con 0x7f691c09f7d0 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 >> v1:172.21.15.84:6792/0 conn(0x7f691c09ff70 legacy=0x7f691c149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 >> v1:172.21.15.45:6791/0 conn(0x7f691c09fba0 legacy=0x7f691c09ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient: _finish_hunting 0 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 monclient: found mon.h 2024-09-18T14:15:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c11e590 con 0x7f691c09f7d0 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient: _finish_auth 0 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f692219d640 5 monclient: authenticate success, global_id 184649 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f692219d640 10 monclient: _renew_subs 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f692219d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f692219d640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.84:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691c11d370 con 0x7f691c09f7d0 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f692219d640 10 monclient: _renew_subs 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f692219d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f692219d640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.84:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f691c11cf40 con 0x7f691c09f7d0 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.5 v1:172.21.15.84:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f690c002070 con 0x7f691c09f7d0 2024-09-18T14:15:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.5 v1:172.21.15.84:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f690c002560 con 0x7f691c09f7d0 2024-09-18T14:15:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:15:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f6919ffb640 4 set_mon_vals no callback set 2024-09-18T14:15:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.126+0000 7f69037fe640 10 monclient: dump: 2024-09-18T14:15:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.151 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.127+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.5 v1:172.21.15.84:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f690c00d530 con 0x7f691c09f7d0 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.127+0000 7f692219d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.127+0000 7f692219d640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f68fc006580 con 0x7f68fc002830 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.127+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.5 v1:172.21.15.84:6791/0 8 ==== osd_map(741..741 src has 529..741) v4 ==== 4015+0+0 (unknown 1961392220 0 0) 0x7f690c044190 con 0x7f691c09f7d0 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.127+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252996001 0 0) 0x7f68fc006580 con 0x7f68fc002830 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.127+0000 7f69037fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.127+0000 7f69037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.127+0000 7f69037fe640 10 monclient(hunting): my global_id is 5930 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.127+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f691c09be50 con 0x7f68fc002830 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.127+0000 7f691bfff640 10 monclient: get_auth_request con 0x7f690803aa60 auth_method 0 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2522962995 0 0) 0x7f691c09be50 con 0x7f68fc002830 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68fc006580 con 0x7f68fc002830 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6910004be0 con 0x7f68fc002830 2024-09-18T14:15:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 10 monclient: dump: 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.154 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3543123524 0 0) 0x7f68fc006580 con 0x7f68fc002830 2024-09-18T14:15:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68fc00a0a0 con 0x7f68fc002830 2024-09-18T14:15:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6910005130 con 0x7f68fc002830 2024-09-18T14:15:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 >> v1:172.21.15.45:6793/0 conn(0x7f68fc002830 legacy=0x7f68fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.128+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 >> v1:172.21.15.45:6793/0 conn(0x7f68fc002830 legacy=0x7f68fc004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f692219d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f692219d640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f68fc0074f0 con 0x7f68fc0068a0 2024-09-18T14:15:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774785407 0 0) 0x7f68fc0074f0 con 0x7f68fc0068a0 2024-09-18T14:15:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 10 monclient(hunting): my global_id is 5939 2024-09-18T14:15:06.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68fc006580 con 0x7f68fc0068a0 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 439414657 0 0) 0x7f68fc006580 con 0x7f68fc0068a0 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68fc0074f0 con 0x7f68fc0068a0 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6910005990 con 0x7f68fc0068a0 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 10 monclient: dump: 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3484630385 0 0) 0x7f68fc0074f0 con 0x7f68fc0068a0 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.134+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f68fc008750 con 0x7f68fc0068a0 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7389 (unknown 0 0 496521412) 0x7f6910005d30 con 0x7f68fc0068a0 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f69037fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f69037fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 >> v1:172.21.15.45:6793/0 conn(0x7f68fc0068a0 legacy=0x7f68fc006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f69037fe640 1 -- 172.21.15.45:0/1985652355 >> v1:172.21.15.45:6793/0 conn(0x7f68fc0068a0 legacy=0x7f68fc006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:06.160 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:06.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:06.160 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:06.160 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:15:06.160 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:06.160 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:06.160 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:06.160 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.161 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.162 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:06.163 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:06.164 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:06.164 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:06.164 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:06.164 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:06.164 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:06.164 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:06.164 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:06.164 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:06.164 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:06.164 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:06.164 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:06.164 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.165 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:06.166 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.167 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.168 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.168 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:06.168 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:06.168 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.168 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.168 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.168 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:06.168 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:06.173 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:06.177 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.179 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:06.179 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f692219d640 1 -- 172.21.15.45:0/1985652355 >> v1:172.21.15.84:6812/31478 conn(0x7f690803aa60 legacy=0x7f690803cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f692219d640 10 monclient: shutdown 2024-09-18T14:15:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f692219d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f692219d640 1 -- 172.21.15.45:0/1985652355 >> v1:172.21.15.84:6791/0 conn(0x7f691c09f7d0 legacy=0x7f691c09a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f691bfff640 1 -- 172.21.15.45:0/1985652355 reap_dead start 2024-09-18T14:15:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f691bfff640 1 -- 172.21.15.45:0/1985652355 reap_dead start 2024-09-18T14:15:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f692219d640 1 -- 172.21.15.45:0/1985652355 shutdown_connections 2024-09-18T14:15:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f692219d640 1 -- 172.21.15.45:0/1985652355 >> 172.21.15.45:0/1985652355 conn(0x7f691c0630b0 msgr2=0x7f691c133880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f692219d640 1 -- 172.21.15.45:0/1985652355 shutdown_connections 2024-09-18T14:15:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.135+0000 7f692219d640 1 -- 172.21.15.45:0/1985652355 wait complete. 2024-09-18T14:15:06.181 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:06.181 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:15:06.182 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:15:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.278+0000 7ff1d9020640 10 monclient: get_monmap_and_config 2024-09-18T14:15:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.278+0000 7ff1d9020640 10 monclient: build_initial_monmap 2024-09-18T14:15:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.278+0000 7ff1d9020640 10 monclient: monmap: 2024-09-18T14:15:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:06.279581+0000 2024-09-18T14:15:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:06.279581+0000 2024-09-18T14:15:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:06.280 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 1 Processor -- start 2024-09-18T14:15:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 1 -- start start 2024-09-18T14:15:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 10 monclient: init 2024-09-18T14:15:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 10 monclient: _add_conns ranks=[1,3,2,7,5,8,6,4,0] 2024-09-18T14:15:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 10 monclient(hunting): picked mon.noname-f con 0x7ff1d409f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 10 monclient(hunting): picked mon.noname-g con 0x7ff1d409fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 10 monclient(hunting): picked mon.noname-b con 0x7ff1d409ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d405ed70 con 0x7ff1d409f7d0 2024-09-18T14:15:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d409dfc0 con 0x7ff1d409ff70 2024-09-18T14:15:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d409dce0 con 0x7ff1d409fba0 2024-09-18T14:15:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d9020640 10 monclient(hunting): authenticate will time out at 1721.617767s 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d359e640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7ff1d409ff70 0x7ff1d413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33186/0 (socket says 172.21.15.45:33186) 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.279+0000 7ff1d359e640 1 -- 172.21.15.45:0/649422238 learned_addr learned my addr 172.21.15.45:0/649422238 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 1 -- 172.21.15.45:0/649422238 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696990109 0 0) 0x7ff1d409dfc0 con 0x7ff1d409ff70 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient(hunting): my global_id is 145406 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d2d9d640 1 -- 172.21.15.45:0/649422238 >> v1:172.21.15.84:6789/0 conn(0x7ff1d409f7d0 legacy=0x7ff1d4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d2d9d640 1 -- 172.21.15.45:0/649422238 >> v1:172.21.15.84:6789/0 conn(0x7ff1d409f7d0 legacy=0x7ff1d4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d2d9d640 1 --1- 172.21.15.45:0/649422238 >> v1:172.21.15.84:6789/0 conn(0x7ff1d409f7d0 0x7ff1d4132b30 :-1 s=OPENED pgs=435 cs=1 l=1).handle_message read tag failed 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d2d9d640 1 --1- 172.21.15.45:0/649422238 >> v1:172.21.15.84:6789/0 conn(0x7ff1d409f7d0 0x7ff1d4132b30 :-1 s=OPENED pgs=435 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 1 -- 172.21.15.45:0/649422238 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c00034e0 con 0x7ff1d409ff70 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.84:6789/0 2024-09-18T14:15:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 1 -- 172.21.15.45:0/649422238 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193295480 0 0) 0x7ff1d409dce0 con 0x7ff1d409fba0 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient(hunting): my global_id is 184710 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 1 -- 172.21.15.45:0/649422238 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1d409dfc0 con 0x7ff1d409fba0 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 1 -- 172.21.15.45:0/649422238 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2677153075 0 0) 0x7ff1c00034e0 con 0x7ff1d409ff70 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 1 -- 172.21.15.45:0/649422238 >> v1:172.21.15.84:6790/0 conn(0x7ff1d409fba0 legacy=0x7ff1d4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 1 -- 172.21.15.45:0/649422238 >> v1:172.21.15.84:6789/0 conn(0x7ff1d409f7d0 legacy=0x7ff1d4132b30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 1 monclient: found mon.noname-b 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 1 -- 172.21.15.45:0/649422238 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d40a1c00 con 0x7ff1d409ff70 2024-09-18T14:15:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient: _finish_auth 0 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 1 -- 172.21.15.45:0/649422238 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1cc001bb0 con 0x7ff1d409ff70 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient: dump: 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d9020640 5 monclient: authenticate success, global_id 145406 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d9020640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 1 -- 172.21.15.45:0/649422238 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff1cc001d90 con 0x7ff1d409ff70 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 1 -- 172.21.15.45:0/649422238 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1cc004f20 con 0x7ff1d409ff70 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d1d9b640 10 monclient: dump: 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d9020640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d9020640 4 set_mon_vals no callback set 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d9020640 10 monclient: shutdown 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d9020640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.280+0000 7ff1d9020640 1 -- 172.21.15.45:0/649422238 >> v1:172.21.15.45:6790/0 conn(0x7ff1d409ff70 legacy=0x7ff1d413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.281+0000 7ff1d9020640 1 -- 172.21.15.45:0/649422238 shutdown_connections 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.281+0000 7ff1d9020640 1 -- 172.21.15.45:0/649422238 >> 172.21.15.45:0/649422238 conn(0x7ff1d40630b0 msgr2=0x7ff1d40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.281+0000 7ff1d9020640 1 -- 172.21.15.45:0/649422238 shutdown_connections 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.281+0000 7ff1d9020640 1 -- 172.21.15.45:0/649422238 wait complete. 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.281+0000 7ff1d9020640 10 monclient: build_initial_monmap 2024-09-18T14:15:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.281+0000 7ff1d9020640 10 monclient: monmap: 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 1 Processor -- start 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 1 -- start start 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 10 monclient: init 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 10 monclient: _add_conns ranks=[0,4,8,7,5,2,1,6,3] 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 10 monclient(hunting): picked mon.noname-a con 0x7ff1d409f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 10 monclient(hunting): picked mon.noname-c con 0x7ff1d409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 10 monclient(hunting): picked mon.noname-e con 0x7ff1d409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d4122320 con 0x7ff1d409f7d0 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d41201a0 con 0x7ff1d409fba0 2024-09-18T14:15:06.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d4121cd0 con 0x7ff1d409ff70 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d9020640 10 monclient(hunting): authenticate will time out at 1721.620560s 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d2d9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff1d409f7d0 0x7ff1d4141940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56464/0 (socket says 172.21.15.45:56464) 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1d2d9d640 1 -- 172.21.15.45:0/3409675308 learned_addr learned my addr 172.21.15.45:0/3409675308 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138806677 0 0) 0x7ff1d41201a0 con 0x7ff1d409fba0 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 10 monclient(hunting): my global_id is 164983 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b40034e0 con 0x7ff1d409fba0 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847573737 0 0) 0x7ff1d4122320 con 0x7ff1d409f7d0 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 10 monclient(hunting): my global_id is 6255 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1d41201a0 con 0x7ff1d409f7d0 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37401339 0 0) 0x7ff1d4121cd0 con 0x7ff1d409ff70 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 10 monclient(hunting): my global_id is 5948 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.282+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1d4122320 con 0x7ff1d409ff70 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 243818255 0 0) 0x7ff1b40034e0 con 0x7ff1d409fba0 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1d4121cd0 con 0x7ff1d409fba0 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1bc001ae0 con 0x7ff1d409fba0 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient(hunting): dump: 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.299 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3598002646 0 0) 0x7ff1d41201a0 con 0x7ff1d409f7d0 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1b40034e0 con 0x7ff1d409f7d0 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3343817565 0 0) 0x7ff1d4122320 con 0x7ff1d409ff70 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1d41201a0 con 0x7ff1d409ff70 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1c4001b70 con 0x7ff1d409f7d0 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient(hunting): dump: 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1cc001af0 con 0x7ff1d409ff70 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient(hunting): dump: 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 746440102 0 0) 0x7ff1b40034e0 con 0x7ff1d409f7d0 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 >> v1:172.21.15.45:6793/0 conn(0x7ff1d409ff70 legacy=0x7ff1d4149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 >> v1:172.21.15.45:6791/0 conn(0x7ff1d409fba0 legacy=0x7ff1d4131b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 monclient: found mon.a 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d4121fb0 con 0x7ff1d409f7d0 2024-09-18T14:15:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient: _finish_auth 0 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1d9020640 5 monclient: authenticate success, global_id 6255 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1d9020640 10 monclient: _renew_subs 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1d9020640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1d9020640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d4120d70 con 0x7ff1d409f7d0 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1d9020640 10 monclient: _renew_subs 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1d9020640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1d9020640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d4132290 con 0x7ff1d409f7d0 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff1c40020c0 con 0x7ff1d409f7d0 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1c4002630 con 0x7ff1d409f7d0 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1d159a640 4 set_mon_vals no callback set 2024-09-18T14:15:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.283+0000 7ff1caffd640 10 monclient: dump: 2024-09-18T14:15:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.284+0000 7ff1d9020640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.284+0000 7ff1d9020640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff1b8006580 con 0x7ff1b8002830 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.284+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff1c4002810 con 0x7ff1d409f7d0 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.284+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(741..741 src has 529..741) v4 ==== 4015+0+0 (unknown 1961392220 0 0) 0x7ff1c4046c30 con 0x7ff1d409f7d0 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.284+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7652301 0 0) 0x7ff1b8006580 con 0x7ff1b8002830 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.284+0000 7ff1caffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.284+0000 7ff1caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.284+0000 7ff1caffd640 10 monclient(hunting): my global_id is 214273 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.284+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b40034e0 con 0x7ff1b8002830 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.284+0000 7ff1d359e640 10 monclient: get_auth_request con 0x7ff1b403a990 auth_method 0 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2605783496 0 0) 0x7ff1b40034e0 con 0x7ff1b8002830 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1b8006580 con 0x7ff1b8002830 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1bc002620 con 0x7ff1b8002830 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 10 monclient: dump: 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3356170754 0 0) 0x7ff1b8006580 con 0x7ff1b8002830 2024-09-18T14:15:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1b800a0a0 con 0x7ff1b8002830 2024-09-18T14:15:06.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff1bc002800 con 0x7ff1b8002830 2024-09-18T14:15:06.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:06.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:06.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 >> v1:172.21.15.84:6789/0 conn(0x7ff1b8002830 legacy=0x7ff1b8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.285+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 >> v1:172.21.15.84:6789/0 conn(0x7ff1b8002830 legacy=0x7ff1b8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.291+0000 7ff1d9020640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.291+0000 7ff1d9020640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff1b8007540 con 0x7ff1b80068f0 2024-09-18T14:15:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979846266 0 0) 0x7ff1b8007540 con 0x7ff1b80068f0 2024-09-18T14:15:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 10 monclient(hunting): my global_id is 214282 2024-09-18T14:15:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b8006580 con 0x7ff1b80068f0 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1664787946 0 0) 0x7ff1b8006580 con 0x7ff1b80068f0 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1b8007540 con 0x7ff1b80068f0 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1bc0030c0 con 0x7ff1b80068f0 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 10 monclient: dump: 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2508655819 0 0) 0x7ff1b8007540 con 0x7ff1b80068f0 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff1b80087a0 con 0x7ff1b80068f0 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 570632078) 0x7ff1bc003420 con 0x7ff1b80068f0 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:06.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 >> v1:172.21.15.84:6789/0 conn(0x7ff1b80068f0 legacy=0x7ff1b8006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.292+0000 7ff1caffd640 1 -- 172.21.15.45:0/3409675308 >> v1:172.21.15.84:6789/0 conn(0x7ff1b80068f0 legacy=0x7ff1b8006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:06.314 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:06.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:06.314 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:06.314 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:06.314 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:06.314 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:06.314 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:06.314 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:06.315 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.316 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.318 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.319 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.319 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.319 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.319 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.319 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.319 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.319 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.319 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.319 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:06.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.319 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.321 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:06.322 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.323 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.323 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:06.324 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.324 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.324 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:06.324 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:06.324 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.324 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.324 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.324 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:06.325 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.327 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:06.329 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.332 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.333 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.334 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.334 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.334 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:06.334 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.293+0000 7ff1d9020640 1 -- 172.21.15.45:0/3409675308 >> v1:172.21.15.84:6812/31478 conn(0x7ff1b403a990 legacy=0x7ff1b403ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.293+0000 7ff1d9020640 10 monclient: shutdown 2024-09-18T14:15:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.293+0000 7ff1d9020640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.293+0000 7ff1d9020640 1 -- 172.21.15.45:0/3409675308 >> v1:172.21.15.45:6789/0 conn(0x7ff1d409f7d0 legacy=0x7ff1d4141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.293+0000 7ff1d359e640 1 -- 172.21.15.45:0/3409675308 reap_dead start 2024-09-18T14:15:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.293+0000 7ff1d359e640 1 -- 172.21.15.45:0/3409675308 reap_dead start 2024-09-18T14:15:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.293+0000 7ff1d9020640 1 -- 172.21.15.45:0/3409675308 shutdown_connections 2024-09-18T14:15:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.293+0000 7ff1d9020640 1 -- 172.21.15.45:0/3409675308 >> 172.21.15.45:0/3409675308 conn(0x7ff1d40630b0 msgr2=0x7ff1d40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.293+0000 7ff1d9020640 1 -- 172.21.15.45:0/3409675308 shutdown_connections 2024-09-18T14:15:06.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.293+0000 7ff1d9020640 1 -- 172.21.15.45:0/3409675308 wait complete. 2024-09-18T14:15:06.336 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:06.336 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:15:06.336 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:15:06.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.433+0000 7f07ea44f640 10 monclient: get_monmap_and_config 2024-09-18T14:15:06.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.433+0000 7f07ea44f640 10 monclient: build_initial_monmap 2024-09-18T14:15:06.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.434+0000 7f07ea44f640 10 monclient: monmap: 2024-09-18T14:15:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:06.434956+0000 2024-09-18T14:15:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:06.434956+0000 2024-09-18T14:15:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:06.436 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:06.436 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:06.436 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:06.436 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.434+0000 7f07ea44f640 1 Processor -- start 2024-09-18T14:15:06.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.434+0000 7f07ea44f640 1 -- start start 2024-09-18T14:15:06.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.434+0000 7f07ea44f640 10 monclient: init 2024-09-18T14:15:06.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07ea44f640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:06.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07ea44f640 10 monclient: _add_conns ranks=[5,6,4,3,7,0,2,1,8] 2024-09-18T14:15:06.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07ea44f640 10 monclient(hunting): picked mon.noname-h con 0x7f07e409f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:06.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07ea44f640 10 monclient(hunting): picked mon.noname-d con 0x7f07e409fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:06.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07ea44f640 10 monclient(hunting): picked mon.noname-c con 0x7f07e409ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:06.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07ea44f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e405ed70 con 0x7f07e409ff70 2024-09-18T14:15:06.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07ea44f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e409dfc0 con 0x7f07e409f7d0 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07ea44f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e409dce0 con 0x7f07e409fba0 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07ea44f640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07ea44f640 10 monclient(hunting): authenticate will time out at 1721.773425s 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e89ec640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f07e409ff70 0x7f07e413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39696/0 (socket says 172.21.15.45:39696) 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e89ec640 1 -- 172.21.15.45:0/3730622649 learned_addr learned my addr 172.21.15.45:0/3730622649 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e37fe640 1 --1- 172.21.15.45:0/3730622649 >> v1:172.21.15.45:6792/0 conn(0x7f07e409fba0 0x7f07e4137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43424/0 (socket says 172.21.15.45:43424) 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096828629 0 0) 0x7f07e405ed70 con 0x7f07e409ff70 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 10 monclient(hunting): my global_id is 164992 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07cc0034e0 con 0x7f07e409ff70 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153119183 0 0) 0x7f07e409dce0 con 0x7f07e409fba0 2024-09-18T14:15:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 10 monclient(hunting): my global_id is 184785 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07e405ed70 con 0x7f07e409fba0 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025188514 0 0) 0x7f07e409dfc0 con 0x7f07e409f7d0 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 10 monclient(hunting): my global_id is 184658 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.435+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07e409dce0 con 0x7f07e409f7d0 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1626202327 0 0) 0x7f07e405ed70 con 0x7f07e409fba0 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 >> v1:172.21.15.84:6791/0 conn(0x7f07e409f7d0 legacy=0x7f07e4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 >> v1:172.21.15.45:6791/0 conn(0x7f07e409ff70 legacy=0x7f07e413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 1 monclient: found mon.noname-d 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e40a1c00 con 0x7f07e409fba0 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 10 monclient: _finish_auth 0 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f07d8001810 con 0x7f07e409fba0 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07ea44f640 5 monclient: authenticate success, global_id 184785 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07ea44f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 10 monclient: dump: 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07ea44f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:06.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f07d8001970 con 0x7f07e409fba0 2024-09-18T14:15:06.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:06.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07ea44f640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:06.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 1 -- 172.21.15.45:0/3730622649 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f07d8004ae0 con 0x7f07e409fba0 2024-09-18T14:15:06.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07ea44f640 4 set_mon_vals no callback set 2024-09-18T14:15:06.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:06.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07e2ffd640 10 monclient: dump: 2024-09-18T14:15:06.450 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.450 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.450 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.451 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07ea44f640 10 monclient: shutdown 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07ea44f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07ea44f640 1 -- 172.21.15.45:0/3730622649 >> v1:172.21.15.45:6792/0 conn(0x7f07e409fba0 legacy=0x7f07e4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07ea44f640 1 -- 172.21.15.45:0/3730622649 shutdown_connections 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07ea44f640 1 -- 172.21.15.45:0/3730622649 >> 172.21.15.45:0/3730622649 conn(0x7f07e40630b0 msgr2=0x7f07e40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07ea44f640 1 -- 172.21.15.45:0/3730622649 shutdown_connections 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.436+0000 7f07ea44f640 1 -- 172.21.15.45:0/3730622649 wait complete. 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.437+0000 7f07ea44f640 10 monclient: build_initial_monmap 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.437+0000 7f07ea44f640 10 monclient: monmap: 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:06.454 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.437+0000 7f07ea44f640 1 Processor -- start 2024-09-18T14:15:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.437+0000 7f07ea44f640 1 -- start start 2024-09-18T14:15:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.437+0000 7f07ea44f640 10 monclient: init 2024-09-18T14:15:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07ea44f640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07ea44f640 10 monclient: _add_conns ranks=[3,4,8,1,0,5,6,7,2] 2024-09-18T14:15:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07ea44f640 10 monclient(hunting): picked mon.noname-g con 0x7f07e409f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07ea44f640 10 monclient(hunting): picked mon.noname-c con 0x7f07e409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07ea44f640 10 monclient(hunting): picked mon.noname-e con 0x7f07e409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07ea44f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e411c490 con 0x7f07e409f7d0 2024-09-18T14:15:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07ea44f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e40a2560 con 0x7f07e409fba0 2024-09-18T14:15:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07ea44f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e409d310 con 0x7f07e409ff70 2024-09-18T14:15:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07ea44f640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07ea44f640 10 monclient(hunting): authenticate will time out at 1721.776493s 2024-09-18T14:15:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07e37fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f07e409fba0 0x7f07e4142290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39706/0 (socket says 172.21.15.45:39706) 2024-09-18T14:15:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07e37fe640 1 -- 172.21.15.45:0/374784664 learned_addr learned my addr 172.21.15.45:0/374784664 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534391596 0 0) 0x7f07e40a2560 con 0x7f07e409fba0 2024-09-18T14:15:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 10 monclient(hunting): my global_id is 165001 2024-09-18T14:15:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07d00034e0 con 0x7f07e409fba0 2024-09-18T14:15:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227294926 0 0) 0x7f07e409d310 con 0x7f07e409ff70 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 10 monclient(hunting): my global_id is 5957 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07e40a2560 con 0x7f07e409ff70 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918658791 0 0) 0x7f07e411c490 con 0x7f07e409f7d0 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 10 monclient(hunting): my global_id is 184719 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.438+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07e409d310 con 0x7f07e409f7d0 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3379102738 0 0) 0x7f07d00034e0 con 0x7f07e409fba0 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f07e411c490 con 0x7f07e409fba0 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f07d8001410 con 0x7f07e409fba0 2024-09-18T14:15:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient(hunting): dump: 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1718101356 0 0) 0x7f07e40a2560 con 0x7f07e409ff70 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f07d00034e0 con 0x7f07e409ff70 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f07dc001de0 con 0x7f07e409ff70 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient(hunting): dump: 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.460 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1047593106 0 0) 0x7f07e409d310 con 0x7f07e409f7d0 2024-09-18T14:15:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f07e40a2560 con 0x7f07e409f7d0 2024-09-18T14:15:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f07d4001990 con 0x7f07e409f7d0 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient(hunting): dump: 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1150457672 0 0) 0x7f07e411c490 con 0x7f07e409fba0 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 >> v1:172.21.15.45:6793/0 conn(0x7f07e409ff70 legacy=0x7f07e4149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 >> v1:172.21.15.84:6790/0 conn(0x7f07e409f7d0 legacy=0x7f07e4141b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient: _finish_hunting 0 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 monclient: found mon.c 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e411d8e0 con 0x7f07e409fba0 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient: _finish_auth 0 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07ea44f640 5 monclient: authenticate success, global_id 165001 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07ea44f640 10 monclient: _renew_subs 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07ea44f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07ea44f640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e41429a0 con 0x7f07e409fba0 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07ea44f640 10 monclient: _renew_subs 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07ea44f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07ea44f640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e412f3f0 con 0x7f07e409fba0 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f07d8004d60 con 0x7f07e409fba0 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f07d8005210 con 0x7f07e409fba0 2024-09-18T14:15:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07cbfff640 10 monclient: dump: 2024-09-18T14:15:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.439+0000 7f07e27fc640 4 set_mon_vals no callback set 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.440+0000 7f07ea44f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.440+0000 7f07ea44f640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f07c4006580 con 0x7f07c4002830 2024-09-18T14:15:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.440+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f07d80101e0 con 0x7f07e409fba0 2024-09-18T14:15:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(741..741 src has 529..741) v4 ==== 4015+0+0 (unknown 1961392220 0 0) 0x7f07d8046cb0 con 0x7f07e409fba0 2024-09-18T14:15:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806924978 0 0) 0x7f07c4006580 con 0x7f07c4002830 2024-09-18T14:15:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 10 monclient(hunting): my global_id is 184728 2024-09-18T14:15:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07e411c490 con 0x7f07c4002830 2024-09-18T14:15:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07e89ec640 10 monclient: get_auth_request con 0x7f07d003ac20 auth_method 0 2024-09-18T14:15:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 630586398 0 0) 0x7f07e411c490 con 0x7f07c4002830 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f07c4006580 con 0x7f07c4002830 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f07d4001990 con 0x7f07c4002830 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 10 monclient: dump: 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.468 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1129685349 0 0) 0x7f07c4006580 con 0x7f07c4002830 2024-09-18T14:15:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07c400a0a0 con 0x7f07c4002830 2024-09-18T14:15:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f07d4003490 con 0x7f07c4002830 2024-09-18T14:15:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 >> v1:172.21.15.84:6790/0 conn(0x7f07c4002830 legacy=0x7f07c4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.441+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 >> v1:172.21.15.84:6790/0 conn(0x7f07c4002830 legacy=0x7f07c4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.447+0000 7f07ea44f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.447+0000 7f07ea44f640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f07c4009660 con 0x7f07c4009040 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951956605 0 0) 0x7f07c4009660 con 0x7f07c4009040 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 10 monclient(hunting): my global_id is 184737 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07c4006580 con 0x7f07c4009040 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3481071464 0 0) 0x7f07c4006580 con 0x7f07c4009040 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f07c4009660 con 0x7f07c4009040 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f07d4002610 con 0x7f07c4009040 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 10 monclient: dump: 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.473 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3344716069 0 0) 0x7f07c4009660 con 0x7f07c4009040 2024-09-18T14:15:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.448+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f07c4009920 con 0x7f07c4009040 2024-09-18T14:15:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 2054579777) 0x7f07d4002a40 con 0x7f07c4009040 2024-09-18T14:15:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07cbfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07cbfff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 >> v1:172.21.15.84:6790/0 conn(0x7f07c4009040 legacy=0x7f07c4008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07cbfff640 1 -- 172.21.15.45:0/374784664 >> v1:172.21.15.84:6790/0 conn(0x7f07c4009040 legacy=0x7f07c4008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:06.473 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:15:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.476 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.476 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.476 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.476 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.476 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.476 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.476 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.488 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:06.488 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.488 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.488 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.488 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:06.488 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:06.488 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.492 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:06.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.494 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.495 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.495 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:15:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:15:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.499 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.499 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:06.499 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07ea44f640 1 -- 172.21.15.45:0/374784664 >> v1:172.21.15.84:6812/31478 conn(0x7f07d003ac20 legacy=0x7f07d003d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07ea44f640 10 monclient: shutdown 2024-09-18T14:15:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07ea44f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07ea44f640 1 -- 172.21.15.45:0/374784664 >> v1:172.21.15.45:6791/0 conn(0x7f07e409fba0 legacy=0x7f07e4142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07e89ec640 1 -- 172.21.15.45:0/374784664 reap_dead start 2024-09-18T14:15:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07e89ec640 1 -- 172.21.15.45:0/374784664 reap_dead start 2024-09-18T14:15:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07ea44f640 1 -- 172.21.15.45:0/374784664 shutdown_connections 2024-09-18T14:15:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07ea44f640 1 -- 172.21.15.45:0/374784664 >> 172.21.15.45:0/374784664 conn(0x7f07e40630b0 msgr2=0x7f07e4131ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07ea44f640 1 -- 172.21.15.45:0/374784664 shutdown_connections 2024-09-18T14:15:06.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.449+0000 7f07ea44f640 1 -- 172.21.15.45:0/374784664 wait complete. 2024-09-18T14:15:06.501 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:06.501 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:15:06.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:15:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.598+0000 7f8fd659d640 10 monclient: get_monmap_and_config 2024-09-18T14:15:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.598+0000 7f8fd659d640 10 monclient: build_initial_monmap 2024-09-18T14:15:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.598+0000 7f8fd659d640 10 monclient: monmap: 2024-09-18T14:15:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:06.599870+0000 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:06.599870+0000 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:06.601 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.599+0000 7f8fd659d640 1 Processor -- start 2024-09-18T14:15:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.599+0000 7f8fd659d640 1 -- start start 2024-09-18T14:15:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.599+0000 7f8fd659d640 10 monclient: init 2024-09-18T14:15:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fd659d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fd659d640 10 monclient: _add_conns ranks=[4,5,8,1,6,3,7,0,2] 2024-09-18T14:15:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fd659d640 10 monclient(hunting): picked mon.noname-c con 0x7f8fd009f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fd659d640 10 monclient(hunting): picked mon.noname-h con 0x7f8fd009fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fd659d640 10 monclient(hunting): picked mon.noname-e con 0x7f8fd009ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fd659d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd005ed70 con 0x7f8fd009f7d0 2024-09-18T14:15:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fd659d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd009dfc0 con 0x7f8fd009fba0 2024-09-18T14:15:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fd659d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd009dce0 con 0x7f8fd009ff70 2024-09-18T14:15:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fd659d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fd659d640 10 monclient(hunting): authenticate will time out at 1721.938372s 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fcffff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f8fd009f7d0 0x7f8fd0132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39710/0 (socket says 172.21.15.45:39710) 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fcffff640 1 -- 172.21.15.45:0/2476551690 learned_addr learned my addr 172.21.15.45:0/2476551690 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 1 -- 172.21.15.45:0/2476551690 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470049741 0 0) 0x7f8fd009dce0 con 0x7f8fd009ff70 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 10 monclient(hunting): my global_id is 5966 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 1 -- 172.21.15.45:0/2476551690 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb80034e0 con 0x7f8fd009ff70 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 1 -- 172.21.15.45:0/2476551690 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445738517 0 0) 0x7f8fd005ed70 con 0x7f8fd009f7d0 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 10 monclient(hunting): my global_id is 165010 2024-09-18T14:15:06.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 1 -- 172.21.15.45:0/2476551690 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fd009dce0 con 0x7f8fd009f7d0 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 1 -- 172.21.15.45:0/2476551690 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 907196898 0 0) 0x7f8fb80034e0 con 0x7f8fd009ff70 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 1 -- 172.21.15.45:0/2476551690 >> v1:172.21.15.84:6791/0 conn(0x7f8fd009fba0 legacy=0x7f8fd0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 1 -- 172.21.15.45:0/2476551690 >> v1:172.21.15.45:6791/0 conn(0x7f8fd009f7d0 legacy=0x7f8fd0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 1 monclient: found mon.noname-e 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 1 -- 172.21.15.45:0/2476551690 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd00a1c00 con 0x7f8fd009ff70 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 10 monclient: _finish_auth 0 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 1 -- 172.21.15.45:0/2476551690 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8fc8001ae0 con 0x7f8fd009ff70 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:15:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fceffd640 10 monclient: dump: 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fd659d640 5 monclient: authenticate success, global_id 5966 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.600+0000 7f8fd659d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fceffd640 1 -- 172.21.15.45:0/2476551690 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8fc8001c40 con 0x7f8fd009ff70 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fceffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fceffd640 1 -- 172.21.15.45:0/2476551690 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8fc80021f0 con 0x7f8fd009ff70 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fceffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fceffd640 10 monclient: dump: 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fd659d640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fd659d640 4 set_mon_vals no callback set 2024-09-18T14:15:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fd659d640 10 monclient: shutdown 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fd659d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fd659d640 1 -- 172.21.15.45:0/2476551690 >> v1:172.21.15.45:6793/0 conn(0x7f8fd009ff70 legacy=0x7f8fd013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fd659d640 1 -- 172.21.15.45:0/2476551690 shutdown_connections 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fd659d640 1 -- 172.21.15.45:0/2476551690 >> 172.21.15.45:0/2476551690 conn(0x7f8fd00630b0 msgr2=0x7f8fd00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fd659d640 1 -- 172.21.15.45:0/2476551690 shutdown_connections 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.601+0000 7f8fd659d640 1 -- 172.21.15.45:0/2476551690 wait complete. 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.602+0000 7f8fd659d640 10 monclient: build_initial_monmap 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.602+0000 7f8fd659d640 10 monclient: monmap: 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.602+0000 7f8fd659d640 1 Processor -- start 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.602+0000 7f8fd659d640 1 -- start start 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.602+0000 7f8fd659d640 10 monclient: init 2024-09-18T14:15:06.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fd659d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fd659d640 10 monclient: _add_conns ranks=[6,0,2,7,5,1,8,4,3] 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fd659d640 10 monclient(hunting): picked mon.noname-d con 0x7f8fd009f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fd659d640 10 monclient(hunting): picked mon.noname-a con 0x7f8fd009fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fd659d640 10 monclient(hunting): picked mon.noname-b con 0x7f8fd009ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fd659d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd00a2560 con 0x7f8fd009fba0 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fd659d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd0120ac0 con 0x7f8fd009ff70 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fd659d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd009d310 con 0x7f8fd009f7d0 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fd659d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fd659d640 10 monclient(hunting): authenticate will time out at 1721.941278s 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fcffff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f8fd009f7d0 0x7f8fd0141a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43426/0 (socket says 172.21.15.45:43426) 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fcffff640 1 -- 172.21.15.45:0/309008739 learned_addr learned my addr 172.21.15.45:0/309008739 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194247932 0 0) 0x7f8fd009d310 con 0x7f8fd009f7d0 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): my global_id is 184794 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fbc0034e0 con 0x7f8fd009f7d0 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234021699 0 0) 0x7f8fd00a2560 con 0x7f8fd009fba0 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): my global_id is 6264 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fd009d310 con 0x7f8fd009fba0 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546905896 0 0) 0x7f8fd0120ac0 con 0x7f8fd009ff70 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): my global_id is 145415 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fd00a2560 con 0x7f8fd009ff70 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 559799758 0 0) 0x7f8fbc0034e0 con 0x7f8fd009f7d0 2024-09-18T14:15:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8fd0120ac0 con 0x7f8fd009f7d0 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8fc00015d0 con 0x7f8fd009f7d0 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): dump: 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 593397090 0 0) 0x7f8fd009d310 con 0x7f8fd009fba0 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8fbc0034e0 con 0x7f8fd009fba0 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 731875181 0 0) 0x7f8fd00a2560 con 0x7f8fd009ff70 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8fd009d310 con 0x7f8fd009ff70 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8fc4001a60 con 0x7f8fd009fba0 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): dump: 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8fc80012e0 con 0x7f8fd009ff70 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.603+0000 7f8fb7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 10 monclient(hunting): dump: 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3481853204 0 0) 0x7f8fd0120ac0 con 0x7f8fd009f7d0 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 >> v1:172.21.15.45:6790/0 conn(0x7f8fd009ff70 legacy=0x7f8fd0149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 >> v1:172.21.15.45:6789/0 conn(0x7f8fd009fba0 legacy=0x7f8fd0131ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 10 monclient: _finish_hunting 0 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 1 monclient: found mon.d 2024-09-18T14:15:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd0121e30 con 0x7f8fd009f7d0 2024-09-18T14:15:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 10 monclient: _finish_auth 0 2024-09-18T14:15:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fd659d640 5 monclient: authenticate success, global_id 184794 2024-09-18T14:15:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fd659d640 10 monclient: _renew_subs 2024-09-18T14:15:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fd659d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fd659d640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd01323f0 con 0x7f8fd009f7d0 2024-09-18T14:15:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fd659d640 10 monclient: _renew_subs 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fd659d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fd659d640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd01427a0 con 0x7f8fd009f7d0 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8fc0004a50 con 0x7f8fd009f7d0 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8fc0004f00 con 0x7f8fd009f7d0 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fce7fc640 4 set_mon_vals no callback set 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 10 monclient: dump: 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.604+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f8fc000fed0 con 0x7f8fd009f7d0 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fd659d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fd659d640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8fb0006580 con 0x7f8fb0002830 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(741..741 src has 529..741) v4 ==== 4015+0+0 (unknown 1961392220 0 0) 0x7f8fc0046cd0 con 0x7f8fd009f7d0 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fcf7fe640 10 monclient: get_auth_request con 0x7f8fbc03a9a0 auth_method 0 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210996999 0 0) 0x7f8fb0006580 con 0x7f8fb0002830 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 10 monclient(hunting): my global_id is 184676 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fd0120ac0 con 0x7f8fb0002830 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 494247756 0 0) 0x7f8fd0120ac0 con 0x7f8fb0002830 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8fb0006580 con 0x7f8fb0002830 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8fc8002720 con 0x7f8fb0002830 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:15:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 10 monclient: dump: 2024-09-18T14:15:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.630 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.605+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3634427739 0 0) 0x7f8fb0006580 con 0x7f8fb0002830 2024-09-18T14:15:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.606+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fb000a0a0 con 0x7f8fb0002830 2024-09-18T14:15:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.606+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8fc8002a80 con 0x7f8fb0002830 2024-09-18T14:15:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.606+0000 7f8fb7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.606+0000 7f8fb7fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.606+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 >> v1:172.21.15.84:6791/0 conn(0x7f8fb0002830 legacy=0x7f8fb0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.606+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 >> v1:172.21.15.84:6791/0 conn(0x7f8fb0002830 legacy=0x7f8fb0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fd659d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fd659d640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8fb0009660 con 0x7f8fb0009040 2024-09-18T14:15:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451135614 0 0) 0x7f8fb0009660 con 0x7f8fb0009040 2024-09-18T14:15:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 10 monclient(hunting): my global_id is 184685 2024-09-18T14:15:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb0006580 con 0x7f8fb0009040 2024-09-18T14:15:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4075514011 0 0) 0x7f8fb0006580 con 0x7f8fb0009040 2024-09-18T14:15:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8fb0009660 con 0x7f8fb0009040 2024-09-18T14:15:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8fc8003240 con 0x7f8fb0009040 2024-09-18T14:15:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 10 monclient: dump: 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.634 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3958543209 0 0) 0x7f8fb0009660 con 0x7f8fb0009040 2024-09-18T14:15:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.612+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8fb0009920 con 0x7f8fb0009040 2024-09-18T14:15:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 2479635450) 0x7f8fc8003550 con 0x7f8fb0009040 2024-09-18T14:15:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fb7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fb7fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 >> v1:172.21.15.84:6791/0 conn(0x7f8fb0009040 legacy=0x7f8fb0008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fb7fff640 1 -- 172.21.15.45:0/309008739 >> v1:172.21.15.84:6791/0 conn(0x7f8fb0009040 legacy=0x7f8fb0008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:06.634 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:06.635 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:06.636 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:06.636 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.636 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:15:06.636 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:06.636 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:06.636 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:06.636 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.636 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.636 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.636 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.636 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.636 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.637 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:06.638 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.639 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.640 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:06.641 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:06.643 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.644 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.646 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.647 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.649 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.652 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:06.653 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fd659d640 1 -- 172.21.15.45:0/309008739 >> v1:172.21.15.84:6812/31478 conn(0x7f8fbc03a9a0 legacy=0x7f8fbc03ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fd659d640 10 monclient: shutdown 2024-09-18T14:15:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fd659d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fd659d640 1 -- 172.21.15.45:0/309008739 >> v1:172.21.15.45:6792/0 conn(0x7f8fd009f7d0 legacy=0x7f8fd0141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fd4b3a640 1 -- 172.21.15.45:0/309008739 reap_dead start 2024-09-18T14:15:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fd4b3a640 1 -- 172.21.15.45:0/309008739 reap_dead start 2024-09-18T14:15:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fd659d640 1 -- 172.21.15.45:0/309008739 shutdown_connections 2024-09-18T14:15:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fd659d640 1 -- 172.21.15.45:0/309008739 >> 172.21.15.45:0/309008739 conn(0x7f8fd00630b0 msgr2=0x7f8fd00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fd659d640 1 -- 172.21.15.45:0/309008739 shutdown_connections 2024-09-18T14:15:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.613+0000 7f8fd659d640 1 -- 172.21.15.45:0/309008739 wait complete. 2024-09-18T14:15:06.656 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:06.656 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:15:06.656 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:15:06.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.752+0000 7efc1ead4640 10 monclient: get_monmap_and_config 2024-09-18T14:15:06.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.752+0000 7efc1ead4640 10 monclient: build_initial_monmap 2024-09-18T14:15:06.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.752+0000 7efc1ead4640 10 monclient: monmap: 2024-09-18T14:15:06.753 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:06.753 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.753 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:06.753192+0000 2024-09-18T14:15:06.753 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:06.753192+0000 2024-09-18T14:15:06.753 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:06.753 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.753 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:06.753 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:06.753 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:06.753 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:06.754 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:06.754 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:06.754 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:06.754 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:06.754 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:06.754 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 1 Processor -- start 2024-09-18T14:15:06.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 1 -- start start 2024-09-18T14:15:06.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 10 monclient: init 2024-09-18T14:15:06.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:06.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 10 monclient: _add_conns ranks=[4,7,5,1,0,8,2,3,6] 2024-09-18T14:15:06.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 10 monclient(hunting): picked mon.noname-c con 0x7efc1806e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:06.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 10 monclient(hunting): picked mon.noname-i con 0x7efc180a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:06.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 10 monclient(hunting): picked mon.noname-h con 0x7efc180a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:06.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc1805ed70 con 0x7efc1806e810 2024-09-18T14:15:06.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc18097620 con 0x7efc180a1080 2024-09-18T14:15:06.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc18097340 con 0x7efc180a0cb0 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1ead4640 10 monclient(hunting): authenticate will time out at 1722.091695s 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1c870640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7efc1806e810 0x7efc180a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39720/0 (socket says 172.21.15.45:39720) 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.753+0000 7efc1c870640 1 -- 172.21.15.45:0/1020364628 learned_addr learned my addr 172.21.15.45:0/1020364628 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025632404 0 0) 0x7efc1805ed70 con 0x7efc1806e810 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient(hunting): my global_id is 165019 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc080034e0 con 0x7efc1806e810 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628376019 0 0) 0x7efc18097620 con 0x7efc180a1080 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient(hunting): my global_id is 184694 2024-09-18T14:15:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc1805ed70 con 0x7efc180a1080 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497053153 0 0) 0x7efc18097340 con 0x7efc180a0cb0 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient(hunting): my global_id is 214306 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc18097620 con 0x7efc180a0cb0 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1017069500 0 0) 0x7efc080034e0 con 0x7efc1806e810 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 >> v1:172.21.15.84:6792/0 conn(0x7efc180a0cb0 legacy=0x7efc18137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 >> v1:172.21.15.84:6791/0 conn(0x7efc180a1080 legacy=0x7efc1813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient: _finish_hunting 0 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 monclient: found mon.noname-c 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc180a1660 con 0x7efc1806e810 2024-09-18T14:15:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient: _finish_auth 0 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efc0c001830 con 0x7efc1806e810 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc1ead4640 5 monclient: authenticate success, global_id 165019 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc1ead4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient: dump: 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc1ead4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efc0c001a10 con 0x7efc1806e810 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 1 -- 172.21.15.45:0/1020364628 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efc0c004ba0 con 0x7efc1806e810 2024-09-18T14:15:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc1ead4640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc1ead4640 4 set_mon_vals no callback set 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc177fe640 10 monclient: dump: 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc1ead4640 10 monclient: shutdown 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc1ead4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc1ead4640 1 -- 172.21.15.45:0/1020364628 >> v1:172.21.15.45:6791/0 conn(0x7efc1806e810 legacy=0x7efc180a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.754+0000 7efc1ead4640 1 -- 172.21.15.45:0/1020364628 shutdown_connections 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.755+0000 7efc1ead4640 1 -- 172.21.15.45:0/1020364628 >> 172.21.15.45:0/1020364628 conn(0x7efc180a5bb0 msgr2=0x7efc180a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.755+0000 7efc1ead4640 1 -- 172.21.15.45:0/1020364628 shutdown_connections 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.755+0000 7efc1ead4640 1 -- 172.21.15.45:0/1020364628 wait complete. 2024-09-18T14:15:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.755+0000 7efc1ead4640 10 monclient: build_initial_monmap 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.755+0000 7efc1ead4640 10 monclient: monmap: 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.755+0000 7efc1ead4640 1 Processor -- start 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1ead4640 1 -- start start 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1ead4640 10 monclient: init 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1ead4640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1ead4640 10 monclient: _add_conns ranks=[6,0,8,5,2,7,4,3,1] 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1ead4640 10 monclient(hunting): picked mon.noname-d con 0x7efc1806e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1ead4640 10 monclient(hunting): picked mon.noname-a con 0x7efc180a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1ead4640 10 monclient(hunting): picked mon.noname-e con 0x7efc180a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1ead4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc18122d60 con 0x7efc180a0cb0 2024-09-18T14:15:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1ead4640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc1813db50 con 0x7efc1806e810 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1ead4640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc181227a0 con 0x7efc180a1080 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1ead4640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1ead4640 10 monclient(hunting): authenticate will time out at 1722.094485s 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1c870640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7efc1806e810 0x7efc1813d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43436/0 (socket says 172.21.15.45:43436) 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc1c870640 1 -- 172.21.15.45:0/2067432523 learned_addr learned my addr 172.21.15.45:0/2067432523 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471172240 0 0) 0x7efc181227a0 con 0x7efc180a1080 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 10 monclient(hunting): my global_id is 5975 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbfc0034e0 con 0x7efc180a1080 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204829831 0 0) 0x7efc1813db50 con 0x7efc1806e810 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 10 monclient(hunting): my global_id is 184803 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc181227a0 con 0x7efc1806e810 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778858452 0 0) 0x7efc18122d60 con 0x7efc180a0cb0 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 10 monclient(hunting): my global_id is 6273 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.756+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc1813db50 con 0x7efc180a0cb0 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 992532395 0 0) 0x7efc181227a0 con 0x7efc1806e810 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efc18122d60 con 0x7efc1806e810 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1406824578 0 0) 0x7efbfc0034e0 con 0x7efc180a1080 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efc181227a0 con 0x7efc180a1080 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efc0c001870 con 0x7efc1806e810 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient(hunting): dump: 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.776 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1653495906 0 0) 0x7efc1813db50 con 0x7efc180a0cb0 2024-09-18T14:15:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efbfc0034e0 con 0x7efc180a0cb0 2024-09-18T14:15:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efc10001c80 con 0x7efc180a1080 2024-09-18T14:15:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient(hunting): dump: 2024-09-18T14:15:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efc04001a20 con 0x7efc180a0cb0 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient(hunting): dump: 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3436820909 0 0) 0x7efc18122d60 con 0x7efc1806e810 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 >> v1:172.21.15.45:6793/0 conn(0x7efc180a1080 legacy=0x7efc18149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 >> v1:172.21.15.45:6789/0 conn(0x7efc180a0cb0 legacy=0x7efc18146670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 monclient: found mon.d 2024-09-18T14:15:06.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc18120260 con 0x7efc1806e810 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient: _finish_auth 0 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc1ead4640 5 monclient: authenticate success, global_id 184803 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc1ead4640 10 monclient: _renew_subs 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc1ead4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc1ead4640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc181224c0 con 0x7efc1806e810 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc1ead4640 10 monclient: _renew_subs 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc1ead4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc1ead4640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efc181215f0 con 0x7efc1806e810 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efc0c004bd0 con 0x7efc1806e810 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efc0c0047a0 con 0x7efc1806e810 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc16ffd640 4 set_mon_vals no callback set 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.757+0000 7efc14ff9640 10 monclient: dump: 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.758+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7efc0c00ffd0 con 0x7efc1806e810 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.758+0000 7efc1ead4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.758+0000 7efc1ead4640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7efbf8006580 con 0x7efbf8002830 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.758+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(741..741 src has 529..741) v4 ==== 4015+0+0 (unknown 1961392220 0 0) 0x7efc0c046cf0 con 0x7efc1806e810 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.758+0000 7efc1d071640 10 monclient: get_auth_request con 0x7efbfc03a980 auth_method 0 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.758+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052043532 0 0) 0x7efbf8006580 con 0x7efbf8002830 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.758+0000 7efc14ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.758+0000 7efc14ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.758+0000 7efc14ff9640 10 monclient(hunting): my global_id is 214315 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.758+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc18122d60 con 0x7efbf8002830 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.758+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3706417310 0 0) 0x7efc18122d60 con 0x7efbf8002830 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.758+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efbf8006580 con 0x7efbf8002830 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.759+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efc040024e0 con 0x7efbf8002830 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.759+0000 7efc14ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.759+0000 7efc14ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.759+0000 7efc14ff9640 10 monclient: dump: 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.759+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 405233965 0 0) 0x7efbf8006580 con 0x7efbf8002830 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.759+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efbf800a0a0 con 0x7efbf8002830 2024-09-18T14:15:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.759+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7efc04002900 con 0x7efbf8002830 2024-09-18T14:15:06.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.759+0000 7efc14ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:06.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.759+0000 7efc14ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:06.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.759+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 >> v1:172.21.15.84:6792/0 conn(0x7efbf8002830 legacy=0x7efbf8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.759+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 >> v1:172.21.15.84:6792/0 conn(0x7efbf8002830 legacy=0x7efbf8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc1ead4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc1ead4640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7efbf80074f0 con 0x7efbf80068a0 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924136955 0 0) 0x7efbf80074f0 con 0x7efbf80068a0 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc14ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc14ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc14ff9640 10 monclient(hunting): my global_id is 214324 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbf8006580 con 0x7efbf80068a0 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2039879459 0 0) 0x7efbf8006580 con 0x7efbf80068a0 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efbf80074f0 con 0x7efbf80068a0 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efc04003340 con 0x7efbf80068a0 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc14ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc14ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.765+0000 7efc14ff9640 10 monclient: dump: 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:06.788 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 382566159 0 0) 0x7efbf80074f0 con 0x7efbf80068a0 2024-09-18T14:15:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efbf8008750 con 0x7efbf80068a0 2024-09-18T14:15:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 3335677142) 0x7efc04005520 con 0x7efbf80068a0 2024-09-18T14:15:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc14ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc14ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 >> v1:172.21.15.84:6792/0 conn(0x7efbf80068a0 legacy=0x7efbf8006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc14ff9640 1 -- 172.21.15.45:0/2067432523 >> v1:172.21.15.84:6792/0 conn(0x7efbf80068a0 legacy=0x7efbf8006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:06.789 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:06.789 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:06.789 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:06.789 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 130, 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.790 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.791 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:06.792 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:06.793 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.794 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:06.795 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:06.796 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.796 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.796 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.796 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:06.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:06.796 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.796 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.797 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.800 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:06.801 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:06.803 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.804 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:06.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:06.806 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:06.806 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:06.806 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.806 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.806 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.808 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.809 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:06.809 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:06.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:06.809 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:06.809 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:06.809 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:06.809 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:06.809 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:06.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:06.809 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:06.809 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:06.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc1ead4640 1 -- 172.21.15.45:0/2067432523 >> v1:172.21.15.84:6812/31478 conn(0x7efbfc03a980 legacy=0x7efbfc03ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc1ead4640 10 monclient: shutdown 2024-09-18T14:15:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc1ead4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc1ead4640 1 -- 172.21.15.45:0/2067432523 >> v1:172.21.15.45:6792/0 conn(0x7efc1806e810 legacy=0x7efc1813d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc1d071640 1 -- 172.21.15.45:0/2067432523 reap_dead start 2024-09-18T14:15:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc1d071640 1 -- 172.21.15.45:0/2067432523 reap_dead start 2024-09-18T14:15:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc1ead4640 1 -- 172.21.15.45:0/2067432523 shutdown_connections 2024-09-18T14:15:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc1ead4640 1 -- 172.21.15.45:0/2067432523 >> 172.21.15.45:0/2067432523 conn(0x7efc180a5bb0 msgr2=0x7efc180a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc1ead4640 1 -- 172.21.15.45:0/2067432523 shutdown_connections 2024-09-18T14:15:06.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:06.766+0000 7efc1ead4640 1 -- 172.21.15.45:0/2067432523 wait complete. 2024-09-18T14:15:06.811 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:15:06.812 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['g'] 2024-09-18T14:15:06.812 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a', 'b'] 2024-09-18T14:15:06.812 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-18T14:15:06.812 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-18T14:15:06.812 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-18T14:15:06.812 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:15:06.826 INFO:tasks.ceph.mon.g:Stopped 2024-09-18T14:15:06.826 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-09-18T14:15:06.826 INFO:tasks.ceph.mon.a:Sent signal 19 2024-09-18T14:15:06.826 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-18T14:15:06.826 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-18T14:15:06.827 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-18T14:15:21.827 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-09-18T14:15:21.828 INFO:tasks.ceph.mon.a:Sent signal 18 2024-09-18T14:15:21.828 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-18T14:15:21.828 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-18T14:15:21.828 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-09-18T14:15:21.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:15:21.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.937+0000 7efd11246640 10 monclient: get_monmap_and_config 2024-09-18T14:15:21.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.937+0000 7efd11246640 10 monclient: build_initial_monmap 2024-09-18T14:15:21.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.937+0000 7efd11246640 10 monclient: monmap: 2024-09-18T14:15:21.938 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:21.938 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:21.938 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:21.938841+0000 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:21.938841+0000 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:21.939 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.938+0000 7efd11246640 1 Processor -- start 2024-09-18T14:15:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.938+0000 7efd11246640 1 -- start start 2024-09-18T14:15:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.938+0000 7efd11246640 10 monclient: init 2024-09-18T14:15:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd11246640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd11246640 10 monclient: _add_conns ranks=[3,2,4,1,7,0,5,8,6] 2024-09-18T14:15:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd11246640 10 monclient(hunting): picked mon.noname-g con 0x7efd0c09f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd11246640 10 monclient(hunting): picked mon.noname-b con 0x7efd0c09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd11246640 10 monclient(hunting): picked mon.noname-c con 0x7efd0c09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd11246640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c05ed70 con 0x7efd0c09fba0 2024-09-18T14:15:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd11246640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c09dfc0 con 0x7efd0c09f7d0 2024-09-18T14:15:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd11246640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c09dce0 con 0x7efd0c09ff70 2024-09-18T14:15:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd11246640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd11246640 10 monclient(hunting): authenticate will time out at 1737.277258s 2024-09-18T14:15:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd0ad9d640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7efd0c09f7d0 legacy=0x7efd0c132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:15:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd0b59e640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7efd0c09ff70 0x7efd0c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:48558/0 (socket says 172.21.15.45:48558) 2024-09-18T14:15:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd0b59e640 1 -- 172.21.15.45:0/1703302704 learned_addr learned my addr 172.21.15.45:0/1703302704 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:21.939+0000 7efd0a59c640 1 --1- 172.21.15.45:0/1703302704 >> v1:172.21.15.45:6790/0 conn(0x7efd0c09fba0 0x7efd0c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:44184/0 (socket says 172.21.15.45:44184) 2024-09-18T14:15:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:22.140+0000 7efd0ad9d640 1 -- 172.21.15.45:0/1703302704 >> v1:172.21.15.84:6790/0 conn(0x7efd0c09f7d0 legacy=0x7efd0c132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:15:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:22.541+0000 7efd0ad9d640 1 -- 172.21.15.45:0/1703302704 >> v1:172.21.15.84:6790/0 conn(0x7efd0c09f7d0 legacy=0x7efd0c132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:15:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:23.342+0000 7efd0ad9d640 1 -- 172.21.15.45:0/1703302704 >> v1:172.21.15.84:6790/0 conn(0x7efd0c09f7d0 legacy=0x7efd0c132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:15:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 10 monclient(hunting): tick 2024-09-18T14:15:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 1 monclient(hunting): continuing hunt 2024-09-18T14:15:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:15:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 1 -- 172.21.15.45:0/1703302704 >> v1:172.21.15.45:6791/0 conn(0x7efd0c09ff70 legacy=0x7efd0c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 1 -- 172.21.15.45:0/1703302704 >> v1:172.21.15.84:6790/0 conn(0x7efd0c09f7d0 legacy=0x7efd0c132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:15:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 1 -- 172.21.15.45:0/1703302704 >> v1:172.21.15.45:6790/0 conn(0x7efd0c09fba0 legacy=0x7efd0c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 10 monclient: _add_conns ranks=[5,7,8,0,6,1] 2024-09-18T14:15:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 10 monclient(hunting): picked mon.noname-h con 0x7efcf8003780 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 10 monclient(hunting): picked mon.noname-i con 0x7efcf8007360 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 10 monclient(hunting): picked mon.noname-e con 0x7efcf800b0a0 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 1 -- 172.21.15.45:0/1703302704 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c0a1c00 con 0x7efcf8003780 2024-09-18T14:15:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 1 -- 172.21.15.45:0/1703302704 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c09dfc0 con 0x7efcf8007360 2024-09-18T14:15:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 1 -- 172.21.15.45:0/1703302704 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf8010f90 con 0x7efcf800b0a0 2024-09-18T14:15:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:24.940+0000 7efd08d99640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.040+0000 7efd09d9b640 1 -- 172.21.15.45:0/1703302704 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444396963 0 0) 0x7efcf8010f90 con 0x7efcf800b0a0 2024-09-18T14:15:26.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.040+0000 7efd09d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:26.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.040+0000 7efd09d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:26.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.040+0000 7efd09d9b640 10 monclient(hunting): my global_id is 5984 2024-09-18T14:15:26.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.040+0000 7efd09d9b640 1 -- 172.21.15.45:0/1703302704 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf40034e0 con 0x7efcf800b0a0 2024-09-18T14:15:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 1 -- 172.21.15.45:0/1703302704 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2604566897 0 0) 0x7efcf40034e0 con 0x7efcf800b0a0 2024-09-18T14:15:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 1 -- 172.21.15.45:0/1703302704 >> v1:172.21.15.84:6792/0 conn(0x7efcf8007360 legacy=0x7efcf80097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 1 -- 172.21.15.45:0/1703302704 >> v1:172.21.15.84:6791/0 conn(0x7efcf8003780 legacy=0x7efcf8005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 1 monclient: found mon.noname-e 2024-09-18T14:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd0b59e640 1 -- 172.21.15.45:0/1703302704 reap_dead start 2024-09-18T14:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 1 -- 172.21.15.45:0/1703302704 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80121f0 con 0x7efcf800b0a0 2024-09-18T14:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 10 monclient: _finish_auth 0 2024-09-18T14:15:26.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:26.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 1 -- 172.21.15.45:0/1703302704 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efd04001a20 con 0x7efcf800b0a0 2024-09-18T14:15:26.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd11246640 5 monclient: authenticate success, global_id 5984 2024-09-18T14:15:26.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd11246640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:26.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:26.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:15:26.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 10 monclient: dump: 2024-09-18T14:15:26.051 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:26.051 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:26.053 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd11246640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 1 -- 172.21.15.45:0/1703302704 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efd04001b80 con 0x7efcf800b0a0 2024-09-18T14:15:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 1 -- 172.21.15.45:0/1703302704 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efd040020b0 con 0x7efcf800b0a0 2024-09-18T14:15:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd11246640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd11246640 4 set_mon_vals no callback set 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.041+0000 7efd09d9b640 10 monclient: dump: 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.042+0000 7efd11246640 10 monclient: shutdown 2024-09-18T14:15:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.042+0000 7efd11246640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.042+0000 7efd11246640 1 -- 172.21.15.45:0/1703302704 >> v1:172.21.15.45:6793/0 conn(0x7efcf800b0a0 legacy=0x7efcf800d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.042+0000 7efd11246640 1 -- 172.21.15.45:0/1703302704 shutdown_connections 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.042+0000 7efd11246640 1 -- 172.21.15.45:0/1703302704 >> 172.21.15.45:0/1703302704 conn(0x7efd0c0630b0 msgr2=0x7efd0c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.042+0000 7efd11246640 1 -- 172.21.15.45:0/1703302704 shutdown_connections 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.042+0000 7efd11246640 1 -- 172.21.15.45:0/1703302704 wait complete. 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.043+0000 7efd11246640 10 monclient: build_initial_monmap 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.043+0000 7efd11246640 10 monclient: monmap: 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.043+0000 7efd11246640 1 Processor -- start 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.043+0000 7efd11246640 1 -- start start 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.043+0000 7efd11246640 10 monclient: init 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd11246640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd11246640 10 monclient: _add_conns ranks=[7,0,8,2,6,3,1,5,4] 2024-09-18T14:15:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd11246640 10 monclient(hunting): picked mon.noname-i con 0x7efcf800b0a0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd11246640 10 monclient(hunting): picked mon.noname-a con 0x7efd0c142290 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd11246640 10 monclient(hunting): picked mon.noname-e con 0x7efd0c09b3a0 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd11246640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c09d310 con 0x7efd0c142290 2024-09-18T14:15:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd11246640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c11f0c0 con 0x7efcf800b0a0 2024-09-18T14:15:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd11246640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c142660 con 0x7efd0c09b3a0 2024-09-18T14:15:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd11246640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd11246640 10 monclient(hunting): authenticate will time out at 1741.383545s 2024-09-18T14:15:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd0a59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efd0c142290 0x7efd0c1470f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51332/0 (socket says 172.21.15.45:51332) 2024-09-18T14:15:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd0ad9d640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7efcf800b0a0 0x7efd0c131970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:55990/0 (socket says 172.21.15.45:55990) 2024-09-18T14:15:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.044+0000 7efd0a59c640 1 -- 172.21.15.45:0/1630123537 learned_addr learned my addr 172.21.15.45:0/1630123537 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:26.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.051+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726406110 0 0) 0x7efcf80121f0 con 0x7efd0c09b3a0 2024-09-18T14:15:26.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.051+0000 7efcf2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:26.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.051+0000 7efcf2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:26.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.051+0000 7efcf2ffd640 10 monclient(hunting): my global_id is 5993 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.051+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf800fec0 con 0x7efd0c09b3a0 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1887091285 0 0) 0x7efcf800fec0 con 0x7efd0c09b3a0 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efcf80121f0 con 0x7efd0c09b3a0 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efd04002030 con 0x7efd0c09b3a0 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 10 monclient(hunting): dump: 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:26.062 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1764838141 0 0) 0x7efcf80121f0 con 0x7efd0c09b3a0 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 >> v1:172.21.15.84:6792/0 conn(0x7efcf800b0a0 legacy=0x7efd0c131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 >> v1:172.21.15.45:6789/0 conn(0x7efd0c142290 legacy=0x7efd0c1470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:26.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 1 monclient: found mon.e 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c0a2560 con 0x7efd0c09b3a0 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 10 monclient: _finish_auth 0 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efcf2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efd11246640 5 monclient: authenticate success, global_id 5993 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efd11246640 10 monclient: _renew_subs 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efd11246640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efd11246640 1 -- 172.21.15.45:0/1630123537 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0c11d4e0 con 0x7efd0c09b3a0 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efd11246640 10 monclient: _renew_subs 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efd11246640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.052+0000 7efd11246640 1 -- 172.21.15.45:0/1630123537 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0c132080 con 0x7efd0c09b3a0 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.053+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efd04004ed0 con 0x7efd0c09b3a0 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.053+0000 7efcf2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.053+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efd04005400 con 0x7efd0c09b3a0 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.053+0000 7efcf2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.053+0000 7efd0959a640 4 set_mon_vals no callback set 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.053+0000 7efcf2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.053+0000 7efcf2ffd640 10 monclient: dump: 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:26.065 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.053+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7efd04005560 con 0x7efd0c09b3a0 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.054+0000 7efd11246640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.054+0000 7efd11246640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.054+0000 7efd11246640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.054+0000 7efd11246640 1 -- 172.21.15.45:0/1630123537 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcec005010 con 0x7efd0c09b3a0 2024-09-18T14:15:26.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.054+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(742..742 src has 529..742) v4 ==== 3927+0+0 (unknown 2850312513 0 0) 0x7efd04046ba0 con 0x7efd0c09b3a0 2024-09-18T14:15:26.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.054+0000 7efd0ad9d640 10 monclient: get_auth_request con 0x7efcf803aea0 auth_method 0 2024-09-18T14:15:26.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.058+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7efd040154b0 con 0x7efd0c09b3a0 2024-09-18T14:15:26.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.058+0000 7efcf2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:26.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.058+0000 7efcf2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:26.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.287+0000 7efd11246640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:15:26.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.287+0000 7efd11246640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:15:26.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.287+0000 7efd11246640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:26.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.287+0000 7efd11246640 1 -- 172.21.15.45:0/1630123537 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7efcec005840 con 0x7efd0c09b3a0 2024-09-18T14:15:26.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.288+0000 7efcf2ffd640 1 -- 172.21.15.45:0/1630123537 <== mon.8 v1:172.21.15.45:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2677 (unknown 254746657 0 1445176951) 0x7efd04014eb0 con 0x7efd0c09b3a0 2024-09-18T14:15:26.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.288+0000 7efcf2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:15:26.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.288+0000 7efcf2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:26.290 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":136,"quorum":[1,2,4,5,6,7,8],"quorum_names":["f","b","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:15:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.289+0000 7efd11246640 1 -- 172.21.15.45:0/1630123537 >> v1:172.21.15.84:6812/31478 conn(0x7efcf803aea0 legacy=0x7efcf803d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.289+0000 7efd11246640 10 monclient: shutdown 2024-09-18T14:15:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.289+0000 7efd11246640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.289+0000 7efd11246640 1 -- 172.21.15.45:0/1630123537 >> v1:172.21.15.45:6793/0 conn(0x7efd0c09b3a0 legacy=0x7efd0c14a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.289+0000 7efd11246640 1 -- 172.21.15.45:0/1630123537 shutdown_connections 2024-09-18T14:15:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.289+0000 7efd11246640 1 -- 172.21.15.45:0/1630123537 >> 172.21.15.45:0/1630123537 conn(0x7efd0c0630b0 msgr2=0x7efd0c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.290+0000 7efd11246640 1 -- 172.21.15.45:0/1630123537 shutdown_connections 2024-09-18T14:15:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:26.290+0000 7efd11246640 1 -- 172.21.15.45:0/1630123537 wait complete. 2024-09-18T14:15:29.302 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:15:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.409+0000 7fee5a89a640 10 monclient: get_monmap_and_config 2024-09-18T14:15:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.409+0000 7fee5a89a640 10 monclient: build_initial_monmap 2024-09-18T14:15:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.409+0000 7fee5a89a640 10 monclient: monmap: 2024-09-18T14:15:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:29.411488+0000 2024-09-18T14:15:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:29.411488+0000 2024-09-18T14:15:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:29.412 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:29.412 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:29.412 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:29.412 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:29.412 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:29.412 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:29.412 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:29.412 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:29.412 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:29.412 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:29.412 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 1 Processor -- start 2024-09-18T14:15:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 1 -- start start 2024-09-18T14:15:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 10 monclient: init 2024-09-18T14:15:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 10 monclient: _add_conns ranks=[6,8,5,1,0,2,7,3,4] 2024-09-18T14:15:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 10 monclient(hunting): picked mon.noname-d con 0x7fee5409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 10 monclient(hunting): picked mon.noname-e con 0x7fee5409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 10 monclient(hunting): picked mon.noname-h con 0x7fee5409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5405ed70 con 0x7fee5409ff70 2024-09-18T14:15:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5409dfc0 con 0x7fee5409f7d0 2024-09-18T14:15:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5409dce0 con 0x7fee5409fba0 2024-09-18T14:15:29.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:29.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.410+0000 7fee5a89a640 10 monclient(hunting): authenticate will time out at 1744.749949s 2024-09-18T14:15:29.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.411+0000 7fee53fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fee5409f7d0 0x7fee54132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:52120/0 (socket says 172.21.15.45:52120) 2024-09-18T14:15:29.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:29.411+0000 7fee53fff640 1 -- 172.21.15.45:0/574384471 learned_addr learned my addr 172.21.15.45:0/574384471 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.839+0000 7fee52ffd640 1 -- 172.21.15.45:0/574384471 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075299599 0 0) 0x7fee5405ed70 con 0x7fee5409ff70 2024-09-18T14:15:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.839+0000 7fee52ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.839+0000 7fee52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.839+0000 7fee52ffd640 10 monclient(hunting): my global_id is 184703 2024-09-18T14:15:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.839+0000 7fee52ffd640 1 -- 172.21.15.45:0/574384471 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee3c0034e0 con 0x7fee5409ff70 2024-09-18T14:15:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.839+0000 7fee52ffd640 1 -- 172.21.15.45:0/574384471 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118870712 0 0) 0x7fee5409dfc0 con 0x7fee5409f7d0 2024-09-18T14:15:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.839+0000 7fee52ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.839+0000 7fee52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.839+0000 7fee52ffd640 10 monclient(hunting): my global_id is 184812 2024-09-18T14:15:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.839+0000 7fee52ffd640 1 -- 172.21.15.45:0/574384471 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee5405ed70 con 0x7fee5409f7d0 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 1 -- 172.21.15.45:0/574384471 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2550296643 0 0) 0x7fee5405ed70 con 0x7fee5409f7d0 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 1 -- 172.21.15.45:0/574384471 >> v1:172.21.15.45:6793/0 conn(0x7fee5409fba0 legacy=0x7fee54137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 1 -- 172.21.15.45:0/574384471 >> v1:172.21.15.84:6791/0 conn(0x7fee5409ff70 legacy=0x7fee5413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 1 monclient: found mon.noname-d 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 1 -- 172.21.15.45:0/574384471 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee540a1c00 con 0x7fee5409f7d0 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 10 monclient: _finish_auth 0 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 1 -- 172.21.15.45:0/574384471 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fee44001740 con 0x7fee5409f7d0 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee5a89a640 5 monclient: authenticate success, global_id 184812 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 10 monclient: dump: 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:31.843 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee5a89a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 1 -- 172.21.15.45:0/574384471 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fee44001920 con 0x7fee5409f7d0 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 1 -- 172.21.15.45:0/574384471 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fee44004860 con 0x7fee5409f7d0 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee52ffd640 10 monclient: dump: 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:31.844 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:31.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee5a89a640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:31.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.840+0000 7fee5a89a640 4 set_mon_vals no callback set 2024-09-18T14:15:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.841+0000 7fee5a89a640 10 monclient: shutdown 2024-09-18T14:15:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.841+0000 7fee5a89a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.841+0000 7fee5a89a640 1 -- 172.21.15.45:0/574384471 >> v1:172.21.15.45:6792/0 conn(0x7fee5409f7d0 legacy=0x7fee54132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.841+0000 7fee5a89a640 1 -- 172.21.15.45:0/574384471 shutdown_connections 2024-09-18T14:15:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.841+0000 7fee5a89a640 1 -- 172.21.15.45:0/574384471 >> 172.21.15.45:0/574384471 conn(0x7fee540630b0 msgr2=0x7fee540634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.841+0000 7fee5a89a640 1 -- 172.21.15.45:0/574384471 shutdown_connections 2024-09-18T14:15:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.842+0000 7fee5a89a640 1 -- 172.21.15.45:0/574384471 wait complete. 2024-09-18T14:15:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.842+0000 7fee5a89a640 10 monclient: build_initial_monmap 2024-09-18T14:15:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.842+0000 7fee5a89a640 10 monclient: monmap: 2024-09-18T14:15:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:31.851 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.843+0000 7fee5a89a640 1 Processor -- start 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.843+0000 7fee5a89a640 1 -- start start 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.843+0000 7fee5a89a640 10 monclient: init 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee5a89a640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee5a89a640 10 monclient: _add_conns ranks=[8,6,4,1,5,7,2,0,3] 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee5a89a640 10 monclient(hunting): picked mon.noname-e con 0x7fee5409f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee5a89a640 10 monclient(hunting): picked mon.noname-d con 0x7fee5409fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee5a89a640 10 monclient(hunting): picked mon.noname-c con 0x7fee5409ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee5a89a640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee540a2560 con 0x7fee5409ff70 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee5a89a640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee54125770 con 0x7fee5409fba0 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee5a89a640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee54125a50 con 0x7fee5409f7d0 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee5a89a640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee5a89a640 10 monclient(hunting): authenticate will time out at 1747.182787s 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee537fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fee5409fba0 0x7fee5409a5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:52136/0 (socket says 172.21.15.45:52136) 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee537fe640 1 -- 172.21.15.45:0/3260252554 learned_addr learned my addr 172.21.15.45:0/3260252554 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.844+0000 7fee58e37640 1 --1- 172.21.15.45:0/3260252554 >> v1:172.21.15.45:6791/0 conn(0x7fee5409ff70 0x7fee5409ad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:40294/0 (socket says 172.21.15.45:40294) 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.845+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904440185 0 0) 0x7fee54125a50 con 0x7fee5409f7d0 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.845+0000 7fee3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.845+0000 7fee3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.845+0000 7fee3bfff640 10 monclient(hunting): my global_id is 6002 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.845+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee40003510 con 0x7fee5409f7d0 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.845+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 32298245 0 0) 0x7fee40003510 con 0x7fee5409f7d0 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.845+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee54125a50 con 0x7fee5409f7d0 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.845+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fee44001680 con 0x7fee5409f7d0 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.845+0000 7fee3bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.845+0000 7fee3bfff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.845+0000 7fee3bfff640 10 monclient(hunting): dump: 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.845+0000 7fee3bfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2289930362 0 0) 0x7fee54125a50 con 0x7fee5409f7d0 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 >> v1:172.21.15.45:6792/0 conn(0x7fee5409fba0 legacy=0x7fee5409a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 >> v1:172.21.15.45:6791/0 conn(0x7fee5409ff70 legacy=0x7fee5409ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 1 monclient: found mon.e 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54130bd0 con 0x7fee5409f7d0 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 10 monclient: _finish_auth 0 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee5a89a640 5 monclient: authenticate success, global_id 6002 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee5a89a640 10 monclient: _renew_subs 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee5a89a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee5a89a640 1 -- 172.21.15.45:0/3260252554 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee541317c0 con 0x7fee5409f7d0 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fee44004800 con 0x7fee5409f7d0 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fee440050d0 con 0x7fee5409f7d0 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee5a89a640 10 monclient: _renew_subs 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee5a89a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee5a89a640 1 -- 172.21.15.45:0/3260252554 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5414f670 con 0x7fee5409f7d0 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee3bfff640 10 monclient: dump: 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:31.857 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.846+0000 7fee527fc640 4 set_mon_vals no callback set 2024-09-18T14:15:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.847+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fee440100a0 con 0x7fee5409f7d0 2024-09-18T14:15:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.847+0000 7fee5a89a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.847+0000 7fee5a89a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.847+0000 7fee5a89a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.847+0000 7fee5a89a640 1 -- 172.21.15.45:0/3260252554 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5404c4a0 con 0x7fee5409f7d0 2024-09-18T14:15:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.847+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(742..742 src has 529..742) v4 ==== 3927+0+0 (unknown 2850312513 0 0) 0x7fee44046b80 con 0x7fee5409f7d0 2024-09-18T14:15:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.848+0000 7fee537fe640 10 monclient: get_auth_request con 0x7fee40039e40 auth_method 0 2024-09-18T14:15:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.852+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fee44015470 con 0x7fee5409f7d0 2024-09-18T14:15:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.852+0000 7fee3bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:31.852+0000 7fee3bfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:31.907 INFO:tasks.rados.rados.0.smithi084.stdout:3372: done (2 left) 2024-09-18T14:15:31.907 INFO:tasks.rados.rados.0.smithi084.stdout:3373: done (1 left) 2024-09-18T14:15:31.907 INFO:tasks.rados.rados.0.smithi084.stdout:3374: done (0 left) 2024-09-18T14:15:31.907 INFO:tasks.rados.rados.0.smithi084.stdout:3375: write oid 29 current snap is 371 2024-09-18T14:15:31.907 INFO:tasks.rados.rados.0.smithi084.stdout:3375: seq_num 1148 ranges {494436=755155,2011446=521062,3075831=218605} 2024-09-18T14:15:31.914 INFO:tasks.rados.rados.0.smithi084.stdout:3375: writing smithi08434663-29 from 494436 to 1249591 tid 1 2024-09-18T14:15:31.921 INFO:tasks.rados.rados.0.smithi084.stdout:3375: writing smithi08434663-29 from 2011446 to 2532508 tid 2 2024-09-18T14:15:31.925 INFO:tasks.rados.rados.0.smithi084.stdout:3375: writing smithi08434663-29 from 3075831 to 3294436 tid 3 2024-09-18T14:15:31.925 INFO:tasks.rados.rados.0.smithi084.stdout:3376: snap_remove snap 308 2024-09-18T14:15:32.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.101+0000 7fee5a89a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:15:32.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.101+0000 7fee5a89a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:15:32.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.101+0000 7fee5a89a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:32.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.101+0000 7fee5a89a640 1 -- 172.21.15.45:0/3260252554 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fee54126130 con 0x7fee5409f7d0 2024-09-18T14:15:32.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.102+0000 7fee3bfff640 1 -- 172.21.15.45:0/3260252554 <== mon.8 v1:172.21.15.45:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2683 (unknown 254746657 0 1463260607) 0x7fee44014e70 con 0x7fee5409f7d0 2024-09-18T14:15:32.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.102+0000 7fee3bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:15:32.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.102+0000 7fee3bfff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:32.103 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":138,"quorum":[0,1,2,4,5,6,7,8],"quorum_names":["a","f","b","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:15:32.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.104+0000 7fee5a89a640 1 -- 172.21.15.45:0/3260252554 >> v1:172.21.15.84:6812/31478 conn(0x7fee40039e40 legacy=0x7fee4003c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.104+0000 7fee5a89a640 10 monclient: shutdown 2024-09-18T14:15:32.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.104+0000 7fee5a89a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.104+0000 7fee5a89a640 1 -- 172.21.15.45:0/3260252554 >> v1:172.21.15.45:6793/0 conn(0x7fee5409f7d0 legacy=0x7fee54099ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.104+0000 7fee5a89a640 1 -- 172.21.15.45:0/3260252554 shutdown_connections 2024-09-18T14:15:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.104+0000 7fee5a89a640 1 -- 172.21.15.45:0/3260252554 >> 172.21.15.45:0/3260252554 conn(0x7fee540630b0 msgr2=0x7fee5406dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.104+0000 7fee5a89a640 1 -- 172.21.15.45:0/3260252554 shutdown_connections 2024-09-18T14:15:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.104+0000 7fee5a89a640 1 -- 172.21.15.45:0/3260252554 wait complete. 2024-09-18T14:15:32.115 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-09-18T14:15:32.115 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-09-18T14:15:32.115 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:15:32.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.210+0000 7f01f8f1d640 10 monclient: get_monmap_and_config 2024-09-18T14:15:32.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.210+0000 7f01f8f1d640 10 monclient: build_initial_monmap 2024-09-18T14:15:32.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.210+0000 7f01f8f1d640 10 monclient: monmap: 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:32.211804+0000 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:32.211804+0000 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:32.213 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.211+0000 7f01f8f1d640 1 Processor -- start 2024-09-18T14:15:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.211+0000 7f01f8f1d640 1 -- start start 2024-09-18T14:15:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.211+0000 7f01f8f1d640 10 monclient: init 2024-09-18T14:15:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f8f1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f8f1d640 10 monclient: _add_conns ranks=[2,6,0,7,1,5,3,8,4] 2024-09-18T14:15:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f8f1d640 10 monclient(hunting): picked mon.noname-b con 0x7f01f409f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f8f1d640 10 monclient(hunting): picked mon.noname-d con 0x7f01f409fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f8f1d640 10 monclient(hunting): picked mon.noname-a con 0x7f01f409ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f8f1d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f405ed70 con 0x7f01f409ff70 2024-09-18T14:15:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f8f1d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f409dfc0 con 0x7f01f409f7d0 2024-09-18T14:15:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f8f1d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f409dce0 con 0x7f01f409fba0 2024-09-18T14:15:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f8f1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f8f1d640 10 monclient(hunting): authenticate will time out at 1747.550163s 2024-09-18T14:15:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f259c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f01f409f7d0 0x7f01f4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:57360/0 (socket says 172.21.15.45:57360) 2024-09-18T14:15:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f1d9b640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f01f409fba0 0x7f01f4137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:52142/0 (socket says 172.21.15.45:52142) 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f259c640 1 -- 172.21.15.45:0/3656883447 learned_addr learned my addr 172.21.15.45:0/3656883447 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f2d9d640 1 --1- 172.21.15.45:0/3656883447 >> v1:172.21.15.45:6789/0 conn(0x7f01f409ff70 0x7f01f413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45250/0 (socket says 172.21.15.45:45250) 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083294032 0 0) 0x7f01f409dce0 con 0x7f01f409fba0 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient(hunting): my global_id is 184821 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01dc0034e0 con 0x7f01f409fba0 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840236948 0 0) 0x7f01f409dfc0 con 0x7f01f409f7d0 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient(hunting): my global_id is 145424 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f409dce0 con 0x7f01f409f7d0 2024-09-18T14:15:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152915653 0 0) 0x7f01f405ed70 con 0x7f01f409ff70 2024-09-18T14:15:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient(hunting): my global_id is 6282 2024-09-18T14:15:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f409dfc0 con 0x7f01f409ff70 2024-09-18T14:15:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3939210417 0 0) 0x7f01f409dce0 con 0x7f01f409f7d0 2024-09-18T14:15:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 >> v1:172.21.15.45:6792/0 conn(0x7f01f409fba0 legacy=0x7f01f4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 >> v1:172.21.15.45:6789/0 conn(0x7f01f409ff70 legacy=0x7f01f413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient: _finish_hunting 0 2024-09-18T14:15:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 1 monclient: found mon.noname-b 2024-09-18T14:15:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:15:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40a1c00 con 0x7f01f409f7d0 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient: _finish_auth 0 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f8f1d640 5 monclient: authenticate success, global_id 145424 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f01e4001b90 con 0x7f01f409f7d0 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f159a640 10 monclient: dump: 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.212+0000 7f01f8f1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f01e4001d70 con 0x7f01f409f7d0 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f159a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f159a640 1 -- 172.21.15.45:0/3656883447 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f01e4004f20 con 0x7f01f409f7d0 2024-09-18T14:15:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f8f1d640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f8f1d640 4 set_mon_vals no callback set 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f159a640 10 monclient: dump: 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f8f1d640 10 monclient: shutdown 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f8f1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f8f1d640 1 -- 172.21.15.45:0/3656883447 >> v1:172.21.15.45:6790/0 conn(0x7f01f409f7d0 legacy=0x7f01f4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f8f1d640 1 -- 172.21.15.45:0/3656883447 shutdown_connections 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f8f1d640 1 -- 172.21.15.45:0/3656883447 >> 172.21.15.45:0/3656883447 conn(0x7f01f40630b0 msgr2=0x7f01f40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f8f1d640 1 -- 172.21.15.45:0/3656883447 shutdown_connections 2024-09-18T14:15:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.213+0000 7f01f8f1d640 1 -- 172.21.15.45:0/3656883447 wait complete. 2024-09-18T14:15:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 10 monclient: build_initial_monmap 2024-09-18T14:15:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 10 monclient: monmap: 2024-09-18T14:15:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 1 Processor -- start 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 1 -- start start 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 10 monclient: init 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 10 monclient: _add_conns ranks=[3,7,1,0,8,4,2,5,6] 2024-09-18T14:15:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 10 monclient(hunting): picked mon.noname-g con 0x7f01f409f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 10 monclient(hunting): picked mon.noname-i con 0x7f01f409fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 10 monclient(hunting): picked mon.noname-f con 0x7f01f409ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f409d310 con 0x7f01f409ff70 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f40a2560 con 0x7f01f409f7d0 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f4120330 con 0x7f01f409fba0 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.214+0000 7f01f8f1d640 10 monclient(hunting): authenticate will time out at 1747.552969s 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01f259c640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7f01f409f7d0 legacy=0x7f01f411d010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01f1d9b640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f01f409fba0 0x7f01f411d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:55716/0 (socket says 172.21.15.45:55716) 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01f1d9b640 1 -- 172.21.15.45:0/4007287990 learned_addr learned my addr 172.21.15.45:0/4007287990 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034345084 0 0) 0x7f01f409d310 con 0x7f01f409ff70 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): my global_id is 214291 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d40034e0 con 0x7f01f409ff70 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575988219 0 0) 0x7f01f4120330 con 0x7f01f409fba0 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): my global_id is 214333 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f409d310 con 0x7f01f409fba0 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 895960950 0 0) 0x7f01d40034e0 con 0x7f01f409ff70 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f01f4120330 con 0x7f01f409ff70 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f01ec001920 con 0x7f01f409ff70 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): dump: 2024-09-18T14:15:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3469715199 0 0) 0x7f01f409d310 con 0x7f01f409fba0 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f01d40034e0 con 0x7f01f409fba0 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f01e8001590 con 0x7f01f409fba0 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient(hunting): dump: 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1652346297 0 0) 0x7f01f4120330 con 0x7f01f409ff70 2024-09-18T14:15:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 >> v1:172.21.15.84:6792/0 conn(0x7f01f409fba0 legacy=0x7f01f411d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 >> v1:172.21.15.84:6790/0 conn(0x7f01f409f7d0 legacy=0x7f01f411d010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient: _finish_hunting 0 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 monclient: found mon.f 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f411e800 con 0x7f01f409ff70 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 10 monclient: _finish_auth 0 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01e27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01f8f1d640 5 monclient: authenticate success, global_id 214291 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01f8f1d640 10 monclient: _renew_subs 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01f8f1d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.215+0000 7f01f8f1d640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f4132370 con 0x7f01f409ff70 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01f8f1d640 10 monclient: _renew_subs 2024-09-18T14:15:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01f8f1d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01f8f1d640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f411de30 con 0x7f01f409ff70 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f01ec0047a0 con 0x7f01f409ff70 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01e27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f01ec004c90 con 0x7f01f409ff70 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01f0d99640 4 set_mon_vals no callback set 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01e27fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01e27fc640 10 monclient: dump: 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01f8f1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:32.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01f8f1d640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f01d8006580 con 0x7f01d8002830 2024-09-18T14:15:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.216+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f01ec00fc60 con 0x7f01f409ff70 2024-09-18T14:15:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(743..743 src has 529..743) v4 ==== 3927+0+0 (unknown 55918879 0 0) 0x7f01ec046a40 con 0x7f01f409ff70 2024-09-18T14:15:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511109398 0 0) 0x7f01d8006580 con 0x7f01d8002830 2024-09-18T14:15:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 10 monclient(hunting): my global_id is 6291 2024-09-18T14:15:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f4120330 con 0x7f01d8002830 2024-09-18T14:15:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01f1d9b640 10 monclient: get_auth_request con 0x7f01d403a3b0 auth_method 0 2024-09-18T14:15:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3490038022 0 0) 0x7f01f4120330 con 0x7f01d8002830 2024-09-18T14:15:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f01d8006580 con 0x7f01d8002830 2024-09-18T14:15:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f01e4004f20 con 0x7f01d8002830 2024-09-18T14:15:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 10 monclient: dump: 2024-09-18T14:15:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.240 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4020499230 0 0) 0x7f01d8006580 con 0x7f01d8002830 2024-09-18T14:15:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f01d800a0a0 con 0x7f01d8002830 2024-09-18T14:15:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f01e4001850 con 0x7f01d8002830 2024-09-18T14:15:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 >> v1:172.21.15.45:6789/0 conn(0x7f01d8002830 legacy=0x7f01d8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.217+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 >> v1:172.21.15.45:6789/0 conn(0x7f01d8002830 legacy=0x7f01d8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.223+0000 7f01f8f1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.223+0000 7f01f8f1d640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f01d8007540 con 0x7f01d80068f0 2024-09-18T14:15:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.223+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112041933 0 0) 0x7f01d8007540 con 0x7f01d80068f0 2024-09-18T14:15:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.223+0000 7f01e27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.223+0000 7f01e27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.223+0000 7f01e27fc640 10 monclient(hunting): my global_id is 6300 2024-09-18T14:15:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.223+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d8006580 con 0x7f01d80068f0 2024-09-18T14:15:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3577586013 0 0) 0x7f01d8006580 con 0x7f01d80068f0 2024-09-18T14:15:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f01d8007540 con 0x7f01d80068f0 2024-09-18T14:15:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f01e40055a0 con 0x7f01d80068f0 2024-09-18T14:15:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 10 monclient: dump: 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.247 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1569882485 0 0) 0x7f01d8007540 con 0x7f01d80068f0 2024-09-18T14:15:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f01d80087a0 con 0x7f01d80068f0 2024-09-18T14:15:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7376 (unknown 0 0 300621193) 0x7f01e40079c0 con 0x7f01d80068f0 2024-09-18T14:15:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 >> v1:172.21.15.45:6789/0 conn(0x7f01d80068f0 legacy=0x7f01d8006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01e27fc640 1 -- 172.21.15.45:0/4007287990 >> v1:172.21.15.45:6789/0 conn(0x7f01d80068f0 legacy=0x7f01d8006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:32.248 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 138, 2024-09-18T14:15:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01f8f1d640 1 -- 172.21.15.45:0/4007287990 >> v1:172.21.15.84:6812/31478 conn(0x7f01d403a3b0 legacy=0x7f01d403c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01f8f1d640 10 monclient: shutdown 2024-09-18T14:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01f8f1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01f8f1d640 1 -- 172.21.15.45:0/4007287990 >> v1:172.21.15.84:6789/0 conn(0x7f01f409ff70 legacy=0x7f01f4149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01f2d9d640 1 -- 172.21.15.45:0/4007287990 reap_dead start 2024-09-18T14:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01f2d9d640 1 -- 172.21.15.45:0/4007287990 reap_dead start 2024-09-18T14:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01f8f1d640 1 -- 172.21.15.45:0/4007287990 shutdown_connections 2024-09-18T14:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.224+0000 7f01f8f1d640 1 -- 172.21.15.45:0/4007287990 >> 172.21.15.45:0/4007287990 conn(0x7f01f40630b0 msgr2=0x7f01f40a4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.225+0000 7f01f8f1d640 1 -- 172.21.15.45:0/4007287990 shutdown_connections 2024-09-18T14:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.225+0000 7f01f8f1d640 1 -- 172.21.15.45:0/4007287990 wait complete. 2024-09-18T14:15:32.269 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:15:32.269 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-09-18T14:15:32.269 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:15:32.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.366+0000 7f0bc4e46640 10 monclient: get_monmap_and_config 2024-09-18T14:15:32.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.366+0000 7f0bc4e46640 10 monclient: build_initial_monmap 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.366+0000 7f0bc4e46640 10 monclient: monmap: 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:32.367232+0000 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:32.367232+0000 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:32.368 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:32.368 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:32.368 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:32.368 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.366+0000 7f0bc4e46640 1 Processor -- start 2024-09-18T14:15:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bc4e46640 1 -- start start 2024-09-18T14:15:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bc4e46640 10 monclient: init 2024-09-18T14:15:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bc4e46640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bc4e46640 10 monclient: _add_conns ranks=[3,4,5,0,7,8,2,6,1] 2024-09-18T14:15:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bc4e46640 10 monclient(hunting): picked mon.noname-g con 0x7f0bc006e810 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bc4e46640 10 monclient(hunting): picked mon.noname-c con 0x7f0bc00a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bc4e46640 10 monclient(hunting): picked mon.noname-h con 0x7f0bc00a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bc4e46640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc005ed70 con 0x7f0bc006e810 2024-09-18T14:15:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bc4e46640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc0097620 con 0x7f0bc00a0cb0 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bc4e46640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc0097340 con 0x7f0bc00a1080 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bc4e46640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bc4e46640 10 monclient(hunting): authenticate will time out at 1747.705602s 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bb7fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f0bc00a0cb0 0x7f0bc0137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:40306/0 (socket says 172.21.15.45:40306) 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bb7fff640 1 -- 172.21.15.45:0/670928197 learned_addr learned my addr 172.21.15.45:0/670928197 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bbe59c640 1 -- 172.21.15.45:0/670928197 >> v1:172.21.15.84:6790/0 conn(0x7f0bc006e810 legacy=0x7f0bc00a05a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bbdd9b640 1 -- 172.21.15.45:0/670928197 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331273219 0 0) 0x7f0bc0097620 con 0x7f0bc00a0cb0 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bbdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bbdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bbdd9b640 10 monclient(hunting): my global_id is 165028 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.367+0000 7f0bbdd9b640 1 -- 172.21.15.45:0/670928197 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bac0034e0 con 0x7f0bc00a0cb0 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 1 -- 172.21.15.45:0/670928197 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541030392 0 0) 0x7f0bc0097340 con 0x7f0bc00a1080 2024-09-18T14:15:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient(hunting): my global_id is 184712 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 1 -- 172.21.15.45:0/670928197 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bc0097620 con 0x7f0bc00a1080 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 1 -- 172.21.15.45:0/670928197 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2125547028 0 0) 0x7f0bac0034e0 con 0x7f0bc00a0cb0 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 1 -- 172.21.15.45:0/670928197 >> v1:172.21.15.84:6791/0 conn(0x7f0bc00a1080 legacy=0x7f0bc013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 1 -- 172.21.15.45:0/670928197 >> v1:172.21.15.84:6790/0 conn(0x7f0bc006e810 legacy=0x7f0bc00a05a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient: _finish_hunting 0 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 1 monclient: found mon.noname-c 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 1 -- 172.21.15.45:0/670928197 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc00a1660 con 0x7f0bc00a0cb0 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient: _finish_auth 0 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 1 -- 172.21.15.45:0/670928197 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ba8001830 con 0x7f0bc00a0cb0 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bc4e46640 5 monclient: authenticate success, global_id 165028 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bc4e46640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient: dump: 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bc4e46640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 1 -- 172.21.15.45:0/670928197 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0ba8001a10 con 0x7f0bc00a0cb0 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bc4e46640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 1 -- 172.21.15.45:0/670928197 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ba8004ba0 con 0x7f0bc00a0cb0 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bc4e46640 4 set_mon_vals no callback set 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bbdd9b640 10 monclient: dump: 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bc4e46640 10 monclient: shutdown 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bc4e46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bc4e46640 1 -- 172.21.15.45:0/670928197 >> v1:172.21.15.45:6791/0 conn(0x7f0bc00a0cb0 legacy=0x7f0bc0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bc4e46640 1 -- 172.21.15.45:0/670928197 shutdown_connections 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bc4e46640 1 -- 172.21.15.45:0/670928197 >> 172.21.15.45:0/670928197 conn(0x7f0bc00a5bb0 msgr2=0x7f0bc00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bc4e46640 1 -- 172.21.15.45:0/670928197 shutdown_connections 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.368+0000 7f0bc4e46640 1 -- 172.21.15.45:0/670928197 wait complete. 2024-09-18T14:15:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.369+0000 7f0bc4e46640 10 monclient: build_initial_monmap 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.369+0000 7f0bc4e46640 10 monclient: monmap: 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.369+0000 7f0bc4e46640 1 Processor -- start 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.369+0000 7f0bc4e46640 1 -- start start 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.369+0000 7f0bc4e46640 10 monclient: init 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bc4e46640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bc4e46640 10 monclient: _add_conns ranks=[7,3,0,1,5,6,2,4,8] 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bc4e46640 10 monclient(hunting): picked mon.noname-i con 0x7f0bc006e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bc4e46640 10 monclient(hunting): picked mon.noname-g con 0x7f0bc00a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bc4e46640 10 monclient(hunting): picked mon.noname-a con 0x7f0bc00a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bc4e46640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc0122da0 con 0x7f0bc00a1080 2024-09-18T14:15:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bc4e46640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc013db90 con 0x7f0bc00a0cb0 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bc4e46640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc01227e0 con 0x7f0bc006e810 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bc4e46640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bc4e46640 10 monclient(hunting): authenticate will time out at 1747.708186s 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb7fff640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7f0bc00a0cb0 legacy=0x7f0bc01466b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bbed9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0bc00a1080 0x7f0bc0149de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45266/0 (socket says 172.21.15.45:45266) 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bbed9d640 1 -- 172.21.15.45:0/1376641264 learned_addr learned my addr 172.21.15.45:0/1376641264 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033784852 0 0) 0x7f0bc0122da0 con 0x7f0bc00a1080 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): my global_id is 6309 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ba00034e0 con 0x7f0bc00a1080 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941950099 0 0) 0x7f0bc01227e0 con 0x7f0bc006e810 2024-09-18T14:15:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): my global_id is 214342 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bc0122da0 con 0x7f0bc006e810 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 901202523 0 0) 0x7f0bc0122da0 con 0x7f0bc006e810 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0bc01227e0 con 0x7f0bc006e810 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3721088857 0 0) 0x7f0ba00034e0 con 0x7f0bc00a1080 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0bc0122da0 con 0x7f0bc00a1080 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0bb00042a0 con 0x7f0bc006e810 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): dump: 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.387 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0bb8001e50 con 0x7f0bc00a1080 2024-09-18T14:15:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient(hunting): dump: 2024-09-18T14:15:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1134115276 0 0) 0x7f0bc01227e0 con 0x7f0bc006e810 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 >> v1:172.21.15.84:6790/0 conn(0x7f0bc00a0cb0 legacy=0x7f0bc01466b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 >> v1:172.21.15.45:6789/0 conn(0x7f0bc00a1080 legacy=0x7f0bc0149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 10 monclient: _finish_hunting 0 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.370+0000 7f0bb67fc640 1 monclient: found mon.i 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bb67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bb67fc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc0122500 con 0x7f0bc006e810 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bb67fc640 10 monclient: _finish_auth 0 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bb67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bc4e46640 5 monclient: authenticate success, global_id 214342 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bc4e46640 10 monclient: _renew_subs 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bc4e46640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bc4e46640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc0122ac0 con 0x7f0bc006e810 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bc4e46640 10 monclient: _renew_subs 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bc4e46640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:15:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bc4e46640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc0121630 con 0x7f0bc006e810 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0bb00047a0 con 0x7f0bc006e810 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bb67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0bb0004c90 con 0x7f0bc006e810 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bb67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bb67fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bbd59a640 4 set_mon_vals no callback set 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bb67fc640 10 monclient: dump: 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bc4e46640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bc4e46640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0ba4006580 con 0x7f0ba4002830 2024-09-18T14:15:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.371+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f0bb0004e70 con 0x7f0bc006e810 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(743..743 src has 529..743) v4 ==== 3927+0+0 (unknown 55918879 0 0) 0x7f0bb0046a00 con 0x7f0bc006e810 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573782058 0 0) 0x7f0ba4006580 con 0x7f0ba4002830 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 10 monclient(hunting): my global_id is 145433 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bc01227e0 con 0x7f0ba4002830 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bbed9d640 10 monclient: get_auth_request con 0x7f0ba003a3b0 auth_method 0 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3209851216 0 0) 0x7f0bc01227e0 con 0x7f0ba4002830 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0ba4006580 con 0x7f0ba4002830 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ba8004ba0 con 0x7f0ba4002830 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 10 monclient: dump: 2024-09-18T14:15:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.395 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1723879315 0 0) 0x7f0ba4006580 con 0x7f0ba4002830 2024-09-18T14:15:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ba400a0a0 con 0x7f0ba4002830 2024-09-18T14:15:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0ba80014f0 con 0x7f0ba4002830 2024-09-18T14:15:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 >> v1:172.21.15.45:6790/0 conn(0x7f0ba4002830 legacy=0x7f0ba4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.372+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 >> v1:172.21.15.45:6790/0 conn(0x7f0ba4002830 legacy=0x7f0ba4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.378+0000 7f0bc4e46640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.378+0000 7f0bc4e46640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0ba4009660 con 0x7f0ba4009040 2024-09-18T14:15:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.378+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341828529 0 0) 0x7f0ba4009660 con 0x7f0ba4009040 2024-09-18T14:15:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 10 monclient(hunting): my global_id is 145442 2024-09-18T14:15:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ba4006580 con 0x7f0ba4009040 2024-09-18T14:15:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 359635044 0 0) 0x7f0ba4006580 con 0x7f0ba4009040 2024-09-18T14:15:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0ba4009660 con 0x7f0ba4009040 2024-09-18T14:15:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ba8005340 con 0x7f0ba4009040 2024-09-18T14:15:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 10 monclient: dump: 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.398 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.399 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 972426521 0 0) 0x7f0ba4009660 con 0x7f0ba4009040 2024-09-18T14:15:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0ba4009920 con 0x7f0ba4009040 2024-09-18T14:15:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7546 (unknown 0 0 297572936) 0x7f0ba80074e0 con 0x7f0ba4009040 2024-09-18T14:15:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 >> v1:172.21.15.45:6790/0 conn(0x7f0ba4009040 legacy=0x7f0ba4008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bb67fc640 1 -- 172.21.15.45:0/1376641264 >> v1:172.21.15.45:6790/0 conn(0x7f0ba4009040 legacy=0x7f0ba4008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:32.400 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 138, 2024-09-18T14:15:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:15:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:32.419 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.379+0000 7f0bc4e46640 1 -- 172.21.15.45:0/1376641264 >> v1:172.21.15.84:6812/31478 conn(0x7f0ba003a3b0 legacy=0x7f0ba003c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.380+0000 7f0bc4e46640 10 monclient: shutdown 2024-09-18T14:15:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.380+0000 7f0bc4e46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.380+0000 7f0bc4e46640 1 -- 172.21.15.45:0/1376641264 >> v1:172.21.15.84:6792/0 conn(0x7f0bc006e810 legacy=0x7f0bc013d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.380+0000 7f0bbed9d640 1 -- 172.21.15.45:0/1376641264 reap_dead start 2024-09-18T14:15:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.380+0000 7f0bbed9d640 1 -- 172.21.15.45:0/1376641264 reap_dead start 2024-09-18T14:15:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.380+0000 7f0bc4e46640 1 -- 172.21.15.45:0/1376641264 shutdown_connections 2024-09-18T14:15:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.380+0000 7f0bc4e46640 1 -- 172.21.15.45:0/1376641264 >> 172.21.15.45:0/1376641264 conn(0x7f0bc00a5bb0 msgr2=0x7f0bc00a2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.380+0000 7f0bc4e46640 1 -- 172.21.15.45:0/1376641264 shutdown_connections 2024-09-18T14:15:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.380+0000 7f0bc4e46640 1 -- 172.21.15.45:0/1376641264 wait complete. 2024-09-18T14:15:32.422 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:15:32.422 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-09-18T14:15:32.422 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:15:32.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.519+0000 7ff91279d640 10 monclient: get_monmap_and_config 2024-09-18T14:15:32.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.519+0000 7ff91279d640 10 monclient: build_initial_monmap 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.519+0000 7ff91279d640 10 monclient: monmap: 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:32.520212+0000 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:32.520212+0000 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:32.521 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:32.521 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:32.521 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:32.521 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.519+0000 7ff91279d640 1 Processor -- start 2024-09-18T14:15:32.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff91279d640 1 -- start start 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff91279d640 10 monclient: init 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff91279d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff91279d640 10 monclient: _add_conns ranks=[1,0,4,2,7,3,8,6,5] 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff91279d640 10 monclient(hunting): picked mon.noname-f con 0x7ff90c09f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff91279d640 10 monclient(hunting): picked mon.noname-a con 0x7ff90c09fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff91279d640 10 monclient(hunting): picked mon.noname-c con 0x7ff90c09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff91279d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c05ed70 con 0x7ff90c09fba0 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff91279d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c09dfc0 con 0x7ff90c09f7d0 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff91279d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c09dce0 con 0x7ff90c09ff70 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff91279d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff91279d640 10 monclient(hunting): authenticate will time out at 1747.858576s 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff90b7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff90c09fba0 0x7ff90c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45276/0 (socket says 172.21.15.45:45276) 2024-09-18T14:15:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff910d3a640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7ff90c09ff70 0x7ff90c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:40322/0 (socket says 172.21.15.45:40322) 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff90b7fe640 1 -- 172.21.15.45:0/1858038552 learned_addr learned my addr 172.21.15.45:0/1858038552 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270437393 0 0) 0x7ff90c09dce0 con 0x7ff90c09ff70 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff90affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff90affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff90affd640 10 monclient(hunting): my global_id is 165037 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.520+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f40034e0 con 0x7ff90c09ff70 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224574354 0 0) 0x7ff90c05ed70 con 0x7ff90c09fba0 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient(hunting): my global_id is 6318 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff90c09dce0 con 0x7ff90c09fba0 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391282900 0 0) 0x7ff90c09dfc0 con 0x7ff90c09f7d0 2024-09-18T14:15:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient(hunting): my global_id is 214300 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff90c05ed70 con 0x7ff90c09f7d0 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3610231249 0 0) 0x7ff8f40034e0 con 0x7ff90c09ff70 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 >> v1:172.21.15.84:6789/0 conn(0x7ff90c09f7d0 legacy=0x7ff90c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 >> v1:172.21.15.45:6789/0 conn(0x7ff90c09fba0 legacy=0x7ff90c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 1 monclient: found mon.noname-c 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c0a1c00 con 0x7ff90c09ff70 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient: _finish_auth 0 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff904001bb0 con 0x7ff90c09ff70 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff91279d640 5 monclient: authenticate success, global_id 165037 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff91279d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient: dump: 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.532 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff91279d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff904001d10 con 0x7ff90c09ff70 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff91279d640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 1 -- 172.21.15.45:0/1858038552 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff904004e60 con 0x7ff90c09ff70 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff91279d640 4 set_mon_vals no callback set 2024-09-18T14:15:32.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff90affd640 10 monclient: dump: 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.534 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff91279d640 10 monclient: shutdown 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff91279d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff91279d640 1 -- 172.21.15.45:0/1858038552 >> v1:172.21.15.45:6791/0 conn(0x7ff90c09ff70 legacy=0x7ff90c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff91279d640 1 -- 172.21.15.45:0/1858038552 shutdown_connections 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff91279d640 1 -- 172.21.15.45:0/1858038552 >> 172.21.15.45:0/1858038552 conn(0x7ff90c0630b0 msgr2=0x7ff90c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.521+0000 7ff91279d640 1 -- 172.21.15.45:0/1858038552 shutdown_connections 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.522+0000 7ff91279d640 1 -- 172.21.15.45:0/1858038552 wait complete. 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.522+0000 7ff91279d640 10 monclient: build_initial_monmap 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.522+0000 7ff91279d640 10 monclient: monmap: 2024-09-18T14:15:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:32.536 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.522+0000 7ff91279d640 1 Processor -- start 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.522+0000 7ff91279d640 1 -- start start 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.522+0000 7ff91279d640 10 monclient: init 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff91279d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff91279d640 10 monclient: _add_conns ranks=[4,8,2,5,1,6,3,0,7] 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff91279d640 10 monclient(hunting): picked mon.noname-c con 0x7ff90c09f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff91279d640 10 monclient(hunting): picked mon.noname-e con 0x7ff90c09fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff91279d640 10 monclient(hunting): picked mon.noname-b con 0x7ff90c09ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff91279d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c1208b0 con 0x7ff90c09ff70 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff91279d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c121ee0 con 0x7ff90c09f7d0 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff91279d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c09d310 con 0x7ff90c09fba0 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff91279d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:32.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff91279d640 10 monclient(hunting): authenticate will time out at 1747.861498s 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff90bfff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7ff90c09f7d0 0x7ff90c1418b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:40334/0 (socket says 172.21.15.45:40334) 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff90bfff640 1 -- 172.21.15.45:0/860439074 learned_addr learned my addr 172.21.15.45:0/860439074 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543813060 0 0) 0x7ff90c121ee0 con 0x7ff90c09f7d0 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 10 monclient(hunting): my global_id is 165046 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f80034e0 con 0x7ff90c09f7d0 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062520784 0 0) 0x7ff90c09d310 con 0x7ff90c09fba0 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 10 monclient(hunting): my global_id is 6011 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff90c121ee0 con 0x7ff90c09fba0 2024-09-18T14:15:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382512960 0 0) 0x7ff90c1208b0 con 0x7ff90c09ff70 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 10 monclient(hunting): my global_id is 145451 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.523+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff90c09d310 con 0x7ff90c09ff70 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2142630525 0 0) 0x7ff90c09d310 con 0x7ff90c09ff70 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff90c1208b0 con 0x7ff90c09ff70 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff904001af0 con 0x7ff90c09ff70 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient(hunting): dump: 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1943166699 0 0) 0x7ff8f80034e0 con 0x7ff90c09f7d0 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff90c09d310 con 0x7ff90c09f7d0 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 514482855 0 0) 0x7ff90c121ee0 con 0x7ff90c09fba0 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8f80034e0 con 0x7ff90c09fba0 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff8fc002b70 con 0x7ff90c09f7d0 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient(hunting): dump: 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff900001480 con 0x7ff90c09fba0 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient(hunting): dump: 2024-09-18T14:15:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 906695580 0 0) 0x7ff90c1208b0 con 0x7ff90c09ff70 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 >> v1:172.21.15.45:6793/0 conn(0x7ff90c09fba0 legacy=0x7ff90c131ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 >> v1:172.21.15.45:6791/0 conn(0x7ff90c09f7d0 legacy=0x7ff90c1418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient: _finish_hunting 0 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 monclient: found mon.b 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c142750 con 0x7ff90c09ff70 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient: _finish_auth 0 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff91279d640 5 monclient: authenticate success, global_id 145451 2024-09-18T14:15:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff91279d640 10 monclient: _renew_subs 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff91279d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff91279d640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90c121c00 con 0x7ff90c09ff70 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff91279d640 10 monclient: _renew_subs 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff91279d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff91279d640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90c1200d0 con 0x7ff90c09ff70 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff904001580 con 0x7ff90c09ff70 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff904005110 con 0x7ff90c09ff70 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff90a7fc640 4 set_mon_vals no callback set 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.524+0000 7ff8f3fff640 10 monclient: dump: 2024-09-18T14:15:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff91279d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff91279d640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff8ec006580 con 0x7ff8ec002830 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff9040100e0 con 0x7ff90c09ff70 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(743..743 src has 529..743) v4 ==== 3927+0+0 (unknown 55918879 0 0) 0x7ff904046cb0 con 0x7ff90c09ff70 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737077875 0 0) 0x7ff8ec006580 con 0x7ff8ec002830 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 10 monclient(hunting): my global_id is 165055 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff90c1208b0 con 0x7ff8ec002830 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 603067949 0 0) 0x7ff90c1208b0 con 0x7ff8ec002830 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8ec006580 con 0x7ff8ec002830 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff90b7fe640 10 monclient: get_auth_request con 0x7ff8f803a950 auth_method 0 2024-09-18T14:15:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff8fc001cd0 con 0x7ff8ec002830 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.525+0000 7ff8f3fff640 10 monclient: dump: 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.549 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.526+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3815184663 0 0) 0x7ff8ec006580 con 0x7ff8ec002830 2024-09-18T14:15:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.526+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8ec00a0a0 con 0x7ff8ec002830 2024-09-18T14:15:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.526+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff8fc001f20 con 0x7ff8ec002830 2024-09-18T14:15:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.526+0000 7ff8f3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.526+0000 7ff8f3fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.526+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 >> v1:172.21.15.45:6791/0 conn(0x7ff8ec002830 legacy=0x7ff8ec004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.526+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 >> v1:172.21.15.45:6791/0 conn(0x7ff8ec002830 legacy=0x7ff8ec004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff91279d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff91279d640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff8ec0095f0 con 0x7ff8ec008ff0 2024-09-18T14:15:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243839460 0 0) 0x7ff8ec0095f0 con 0x7ff8ec008ff0 2024-09-18T14:15:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 10 monclient(hunting): my global_id is 165064 2024-09-18T14:15:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8ec006580 con 0x7ff8ec008ff0 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4043068874 0 0) 0x7ff8ec006580 con 0x7ff8ec008ff0 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8ec0095f0 con 0x7ff8ec008ff0 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff8fc005b00 con 0x7ff8ec008ff0 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 10 monclient: dump: 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1591067251 0 0) 0x7ff8ec0095f0 con 0x7ff8ec008ff0 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.532+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff8ec0098b0 con 0x7ff8ec008ff0 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7546 (unknown 0 0 2090697755) 0x7ff8fc005f00 con 0x7ff8ec008ff0 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff8f3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff8f3fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 >> v1:172.21.15.45:6791/0 conn(0x7ff8ec008ff0 legacy=0x7ff8ec008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff8f3fff640 1 -- 172.21.15.45:0/860439074 >> v1:172.21.15.45:6791/0 conn(0x7ff8ec008ff0 legacy=0x7ff8ec008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:32.554 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 138, 2024-09-18T14:15:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:15:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:32.573 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff91279d640 1 -- 172.21.15.45:0/860439074 >> v1:172.21.15.84:6812/31478 conn(0x7ff8f803a950 legacy=0x7ff8f803ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff91279d640 10 monclient: shutdown 2024-09-18T14:15:32.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff91279d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff91279d640 1 -- 172.21.15.45:0/860439074 >> v1:172.21.15.45:6790/0 conn(0x7ff90c09ff70 legacy=0x7ff90c149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff910d3a640 1 -- 172.21.15.45:0/860439074 reap_dead start 2024-09-18T14:15:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff910d3a640 1 -- 172.21.15.45:0/860439074 reap_dead start 2024-09-18T14:15:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff91279d640 1 -- 172.21.15.45:0/860439074 shutdown_connections 2024-09-18T14:15:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff91279d640 1 -- 172.21.15.45:0/860439074 >> 172.21.15.45:0/860439074 conn(0x7ff90c0630b0 msgr2=0x7ff90c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff91279d640 1 -- 172.21.15.45:0/860439074 shutdown_connections 2024-09-18T14:15:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.533+0000 7ff91279d640 1 -- 172.21.15.45:0/860439074 wait complete. 2024-09-18T14:15:32.575 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:15:32.575 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-09-18T14:15:32.576 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:15:32.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.672+0000 7feaf2a99640 10 monclient: get_monmap_and_config 2024-09-18T14:15:32.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.672+0000 7feaf2a99640 10 monclient: build_initial_monmap 2024-09-18T14:15:32.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.672+0000 7feaf2a99640 10 monclient: monmap: 2024-09-18T14:15:32.673 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:32.673 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.673 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:32.673790+0000 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:32.673790+0000 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:32.674 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.673+0000 7feaf2a99640 1 Processor -- start 2024-09-18T14:15:32.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.673+0000 7feaf2a99640 1 -- start start 2024-09-18T14:15:32.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.673+0000 7feaf2a99640 10 monclient: init 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 10 monclient: _add_conns ranks=[3,4,1,8,0,5,6,2,7] 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 10 monclient(hunting): picked mon.noname-g con 0x7feaec09f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 10 monclient(hunting): picked mon.noname-c con 0x7feaec09fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 10 monclient(hunting): picked mon.noname-f con 0x7feaec09ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec05ed70 con 0x7feaec09ff70 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec09dfc0 con 0x7feaec09f7d0 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec09dce0 con 0x7feaec09fba0 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 10 monclient(hunting): authenticate will time out at 1748.012168s 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf0835640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7feaec09f7d0 legacy=0x7feaec132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaebfff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7feaec09fba0 0x7feaec137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:40368/0 (socket says 172.21.15.45:40368) 2024-09-18T14:15:32.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaebfff640 1 -- 172.21.15.45:0/3136474211 learned_addr learned my addr 172.21.15.45:0/3136474211 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 1 -- 172.21.15.45:0/3136474211 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357920803 0 0) 0x7feaec09dce0 con 0x7feaec09fba0 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient(hunting): my global_id is 165073 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 1 -- 172.21.15.45:0/3136474211 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead40034e0 con 0x7feaec09fba0 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 1 -- 172.21.15.45:0/3136474211 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762052737 0 0) 0x7feaec05ed70 con 0x7feaec09ff70 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient(hunting): my global_id is 214309 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 1 -- 172.21.15.45:0/3136474211 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaec09dce0 con 0x7feaec09ff70 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 1 -- 172.21.15.45:0/3136474211 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 556513471 0 0) 0x7fead40034e0 con 0x7feaec09fba0 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 1 -- 172.21.15.45:0/3136474211 >> v1:172.21.15.84:6790/0 conn(0x7feaec09f7d0 legacy=0x7feaec132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:15:32.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 1 -- 172.21.15.45:0/3136474211 >> v1:172.21.15.84:6789/0 conn(0x7feaec09ff70 legacy=0x7feaec13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 1 monclient: found mon.noname-c 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 1 -- 172.21.15.45:0/3136474211 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec0a1c00 con 0x7feaec09fba0 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient: _finish_auth 0 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 5 monclient: authenticate success, global_id 165073 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 1 -- 172.21.15.45:0/3136474211 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feae0001830 con 0x7feaec09fba0 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient: dump: 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.685 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 1 -- 172.21.15.45:0/3136474211 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feae0001a10 con 0x7feaec09fba0 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 1 -- 172.21.15.45:0/3136474211 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feae0004ba0 con 0x7feaec09fba0 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaf2a99640 4 set_mon_vals no callback set 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.674+0000 7feaeb7fe640 10 monclient: dump: 2024-09-18T14:15:32.687 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.688 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.689 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.675+0000 7feaf2a99640 10 monclient: shutdown 2024-09-18T14:15:32.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.675+0000 7feaf2a99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:32.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.675+0000 7feaf2a99640 1 -- 172.21.15.45:0/3136474211 >> v1:172.21.15.45:6791/0 conn(0x7feaec09fba0 legacy=0x7feaec137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.675+0000 7feaf2a99640 1 -- 172.21.15.45:0/3136474211 shutdown_connections 2024-09-18T14:15:32.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.675+0000 7feaf2a99640 1 -- 172.21.15.45:0/3136474211 >> 172.21.15.45:0/3136474211 conn(0x7feaec0630b0 msgr2=0x7feaec0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:32.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.675+0000 7feaf2a99640 1 -- 172.21.15.45:0/3136474211 shutdown_connections 2024-09-18T14:15:32.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.675+0000 7feaf2a99640 1 -- 172.21.15.45:0/3136474211 wait complete. 2024-09-18T14:15:32.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.675+0000 7feaf2a99640 10 monclient: build_initial_monmap 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.675+0000 7feaf2a99640 10 monclient: monmap: 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 1 Processor -- start 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 1 -- start start 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 10 monclient: init 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 10 monclient: _add_conns ranks=[7,0,1,8,2,4,3,6,5] 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 10 monclient(hunting): picked mon.noname-i con 0x7feaec09f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 10 monclient(hunting): picked mon.noname-a con 0x7feaec09fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 10 monclient(hunting): picked mon.noname-f con 0x7feaec09ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec0a2560 con 0x7feaec09fba0 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec06bec0 con 0x7feaec09ff70 2024-09-18T14:15:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec06d980 con 0x7feaec09f7d0 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaf2a99640 10 monclient(hunting): authenticate will time out at 1748.014887s 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaebfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7feaec09fba0 0x7feaec131c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45280/0 (socket says 172.21.15.45:45280) 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.676+0000 7feaebfff640 1 -- 172.21.15.45:0/978706091 learned_addr learned my addr 172.21.15.45:0/978706091 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275428237 0 0) 0x7feaec0a2560 con 0x7feaec09fba0 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): my global_id is 6327 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead80034e0 con 0x7feaec09fba0 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232108692 0 0) 0x7feaec06d980 con 0x7feaec09f7d0 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): my global_id is 214351 2024-09-18T14:15:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaec0a2560 con 0x7feaec09f7d0 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853145983 0 0) 0x7feaec06bec0 con 0x7feaec09ff70 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): my global_id is 214318 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaec06d980 con 0x7feaec09ff70 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3331848320 0 0) 0x7fead80034e0 con 0x7feaec09fba0 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feaec06bec0 con 0x7feaec09fba0 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feae0004c20 con 0x7feaec09fba0 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): dump: 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2307821605 0 0) 0x7feaec06d980 con 0x7feaec09ff70 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fead80034e0 con 0x7feaec09ff70 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3934300755 0 0) 0x7feaec0a2560 con 0x7feaec09f7d0 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feaec06d980 con 0x7feaec09f7d0 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feae4001d00 con 0x7feaec09ff70 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): dump: 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.695 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feadc004270 con 0x7feaec09f7d0 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient(hunting): dump: 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.697 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3512122713 0 0) 0x7feaec06bec0 con 0x7feaec09fba0 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 >> v1:172.21.15.84:6792/0 conn(0x7feaec09f7d0 legacy=0x7feaec145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 >> v1:172.21.15.84:6789/0 conn(0x7feaec09ff70 legacy=0x7feaec14dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 monclient: found mon.a 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec06dc60 con 0x7feaec09fba0 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient: _finish_auth 0 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feaf2a99640 5 monclient: authenticate success, global_id 6327 2024-09-18T14:15:32.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feaf2a99640 10 monclient: _renew_subs 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feaf2a99640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feaf2a99640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec06df40 con 0x7feaec09fba0 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feaf2a99640 10 monclient: _renew_subs 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feaf2a99640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feaf2a99640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec146220 con 0x7feaec09fba0 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feae0001210 con 0x7feaec09fba0 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feae0005160 con 0x7feaec09fba0 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feaeaffd640 4 set_mon_vals no callback set 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.677+0000 7feae8ff9640 10 monclient: dump: 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.678+0000 7feaf2a99640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.678+0000 7feaf2a99640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7feacc006580 con 0x7feacc002830 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.678+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7feae0010130 con 0x7feaec09fba0 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.678+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(743..743 src has 529..743) v4 ==== 3927+0+0 (unknown 55918879 0 0) 0x7feae0046d00 con 0x7feaec09fba0 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126850948 0 0) 0x7feacc006580 con 0x7feacc002830 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 10 monclient(hunting): my global_id is 184830 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaec06bec0 con 0x7feacc002830 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feaf1036640 10 monclient: get_auth_request con 0x7fead803aa70 auth_method 0 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3742582148 0 0) 0x7feaec06bec0 con 0x7feacc002830 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feacc006580 con 0x7feacc002830 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feadc004940 con 0x7feacc002830 2024-09-18T14:15:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 10 monclient: dump: 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.703 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.703 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.703 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.703 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1724037318 0 0) 0x7feacc006580 con 0x7feacc002830 2024-09-18T14:15:32.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feacc00a0a0 con 0x7feacc002830 2024-09-18T14:15:32.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7feadc004e70 con 0x7feacc002830 2024-09-18T14:15:32.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:32.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:32.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 >> v1:172.21.15.45:6792/0 conn(0x7feacc002830 legacy=0x7feacc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.679+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 >> v1:172.21.15.45:6792/0 conn(0x7feacc002830 legacy=0x7feacc004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:32.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feaf2a99640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:32.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feaf2a99640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7feacc009e70 con 0x7feacc0068f0 2024-09-18T14:15:32.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746224865 0 0) 0x7feacc009e70 con 0x7feacc0068f0 2024-09-18T14:15:32.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feae8ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feae8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feae8ff9640 10 monclient(hunting): my global_id is 184839 2024-09-18T14:15:32.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feacc006580 con 0x7feacc0068f0 2024-09-18T14:15:32.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3534949591 0 0) 0x7feacc006580 con 0x7feacc0068f0 2024-09-18T14:15:32.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feacc009e70 con 0x7feacc0068f0 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feadc005690 con 0x7feacc0068f0 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feae8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feae8ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.685+0000 7feae8ff9640 10 monclient: dump: 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.706 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1970915712 0 0) 0x7feacc009e70 con 0x7feacc0068f0 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7feacc008750 con 0x7feacc0068f0 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7546 (unknown 0 0 631660047) 0x7feadc0057f0 con 0x7feacc0068f0 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feae8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feae8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 >> v1:172.21.15.45:6792/0 conn(0x7feacc0068f0 legacy=0x7feacc006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feae8ff9640 1 -- 172.21.15.45:0/978706091 >> v1:172.21.15.45:6792/0 conn(0x7feacc0068f0 legacy=0x7feacc006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:32.708 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 138, 2024-09-18T14:15:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feaf2a99640 1 -- 172.21.15.45:0/978706091 >> v1:172.21.15.84:6812/31478 conn(0x7fead803aa70 legacy=0x7fead803cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feaf2a99640 10 monclient: shutdown 2024-09-18T14:15:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feaf2a99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feaf2a99640 1 -- 172.21.15.45:0/978706091 >> v1:172.21.15.45:6789/0 conn(0x7feaec09fba0 legacy=0x7feaec131c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feaf1036640 1 -- 172.21.15.45:0/978706091 reap_dead start 2024-09-18T14:15:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feaf1036640 1 -- 172.21.15.45:0/978706091 reap_dead start 2024-09-18T14:15:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feaf2a99640 1 -- 172.21.15.45:0/978706091 shutdown_connections 2024-09-18T14:15:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feaf2a99640 1 -- 172.21.15.45:0/978706091 >> 172.21.15.45:0/978706091 conn(0x7feaec0630b0 msgr2=0x7feaec0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feaf2a99640 1 -- 172.21.15.45:0/978706091 shutdown_connections 2024-09-18T14:15:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.686+0000 7feaf2a99640 1 -- 172.21.15.45:0/978706091 wait complete. 2024-09-18T14:15:32.729 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:15:32.730 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-09-18T14:15:32.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:15:32.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.825+0000 7ff7ee39d640 10 monclient: get_monmap_and_config 2024-09-18T14:15:32.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.825+0000 7ff7ee39d640 10 monclient: build_initial_monmap 2024-09-18T14:15:32.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.825+0000 7ff7ee39d640 10 monclient: monmap: 2024-09-18T14:15:32.826 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:32.826 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.826 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:32.826762+0000 2024-09-18T14:15:32.826 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:32.826762+0000 2024-09-18T14:15:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:32.827 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.826+0000 7ff7ee39d640 1 Processor -- start 2024-09-18T14:15:32.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.826+0000 7ff7ee39d640 1 -- start start 2024-09-18T14:15:32.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.826+0000 7ff7ee39d640 10 monclient: init 2024-09-18T14:15:32.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7ee39d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:32.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7ee39d640 10 monclient: _add_conns ranks=[6,2,7,1,0,3,8,5,4] 2024-09-18T14:15:32.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7ee39d640 10 monclient(hunting): picked mon.noname-d con 0x7ff7e809f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:32.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7ee39d640 10 monclient(hunting): picked mon.noname-b con 0x7ff7e809fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:32.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7ee39d640 10 monclient(hunting): picked mon.noname-i con 0x7ff7e809ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:32.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7ee39d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e805ed70 con 0x7ff7e809fba0 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7ee39d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e809dfc0 con 0x7ff7e809f7d0 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7ee39d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e809dce0 con 0x7ff7e809ff70 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7ee39d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7ee39d640 10 monclient(hunting): authenticate will time out at 1748.165302s 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e7fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7ff7e809f7d0 0x7ff7e8132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:52158/0 (socket says 172.21.15.45:52158) 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e7fff640 1 -- 172.21.15.45:0/2746866606 learned_addr learned my addr 172.21.15.45:0/2746866606 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375268956 0 0) 0x7ff7e805ed70 con 0x7ff7e809fba0 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient(hunting): my global_id is 145460 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d40034e0 con 0x7ff7e809fba0 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620759090 0 0) 0x7ff7e809dfc0 con 0x7ff7e809f7d0 2024-09-18T14:15:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient(hunting): my global_id is 184848 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7e805ed70 con 0x7ff7e809f7d0 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696813744 0 0) 0x7ff7e809dce0 con 0x7ff7e809ff70 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient(hunting): my global_id is 214360 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7e809dfc0 con 0x7ff7e809ff70 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 29904971 0 0) 0x7ff7e805ed70 con 0x7ff7e809f7d0 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 >> v1:172.21.15.84:6792/0 conn(0x7ff7e809ff70 legacy=0x7ff7e813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 >> v1:172.21.15.45:6790/0 conn(0x7ff7e809fba0 legacy=0x7ff7e8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 1 monclient: found mon.noname-d 2024-09-18T14:15:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e80a1c00 con 0x7ff7e809f7d0 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient: _finish_auth 0 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7ee39d640 5 monclient: authenticate success, global_id 184848 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7ee39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff7d8001790 con 0x7ff7e809f7d0 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.827+0000 7ff7e6ffd640 10 monclient: dump: 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7ee39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff7d80018f0 con 0x7ff7e809f7d0 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7e6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7e6ffd640 1 -- 172.21.15.45:0/2746866606 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff7d8004860 con 0x7ff7e809f7d0 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7ee39d640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7ee39d640 4 set_mon_vals no callback set 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7e6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7e6ffd640 10 monclient: dump: 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7ee39d640 10 monclient: shutdown 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7ee39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7ee39d640 1 -- 172.21.15.45:0/2746866606 >> v1:172.21.15.45:6792/0 conn(0x7ff7e809f7d0 legacy=0x7ff7e8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7ee39d640 1 -- 172.21.15.45:0/2746866606 shutdown_connections 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7ee39d640 1 -- 172.21.15.45:0/2746866606 >> 172.21.15.45:0/2746866606 conn(0x7ff7e80630b0 msgr2=0x7ff7e80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7ee39d640 1 -- 172.21.15.45:0/2746866606 shutdown_connections 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.828+0000 7ff7ee39d640 1 -- 172.21.15.45:0/2746866606 wait complete. 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.829+0000 7ff7ee39d640 10 monclient: build_initial_monmap 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.829+0000 7ff7ee39d640 10 monclient: monmap: 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.829+0000 7ff7ee39d640 1 Processor -- start 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.829+0000 7ff7ee39d640 1 -- start start 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.829+0000 7ff7ee39d640 10 monclient: init 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7ee39d640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7ee39d640 10 monclient: _add_conns ranks=[7,3,1,0,4,2,8,6,5] 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7ee39d640 10 monclient(hunting): picked mon.noname-i con 0x7ff7e809f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7ee39d640 10 monclient(hunting): picked mon.noname-g con 0x7ff7e809fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7ee39d640 10 monclient(hunting): picked mon.noname-f con 0x7ff7e809ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7ee39d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e809d310 con 0x7ff7e809ff70 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7ee39d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e80a2560 con 0x7ff7e809fba0 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7ee39d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e8122120 con 0x7ff7e809f7d0 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7e77fe640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7ff7e809fba0 legacy=0x7ff7e8131a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7ee39d640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7ee39d640 10 monclient(hunting): authenticate will time out at 1748.168291s 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7e7fff640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7ff7e809f7d0 0x7ff7e8141840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:55752/0 (socket says 172.21.15.45:55752) 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7e7fff640 1 -- 172.21.15.45:0/1331568243 learned_addr learned my addr 172.21.15.45:0/1331568243 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067044400 0 0) 0x7ff7e8122120 con 0x7ff7e809f7d0 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 10 monclient(hunting): my global_id is 214369 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7c80034e0 con 0x7ff7e809f7d0 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064480614 0 0) 0x7ff7e809d310 con 0x7ff7e809ff70 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 10 monclient(hunting): my global_id is 214327 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7e8122120 con 0x7ff7e809ff70 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2297564756 0 0) 0x7ff7c80034e0 con 0x7ff7e809f7d0 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7e809d310 con 0x7ff7e809f7d0 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff7d80016d0 con 0x7ff7e809f7d0 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 10 monclient(hunting): dump: 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.847 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3595285836 0 0) 0x7ff7e8122120 con 0x7ff7e809ff70 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.830+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7c80034e0 con 0x7ff7e809ff70 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff7e0001d00 con 0x7ff7e809ff70 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 10 monclient(hunting): dump: 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.848 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2183927708 0 0) 0x7ff7e809d310 con 0x7ff7e809f7d0 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 >> v1:172.21.15.84:6790/0 conn(0x7ff7e809fba0 legacy=0x7ff7e8131a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 >> v1:172.21.15.84:6789/0 conn(0x7ff7e809ff70 legacy=0x7ff7e8149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 10 monclient: _finish_hunting 0 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 1 monclient: found mon.i 2024-09-18T14:15:32.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e81426b0 con 0x7ff7e809f7d0 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 10 monclient: _finish_auth 0 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7ee39d640 5 monclient: authenticate success, global_id 214369 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7ee39d640 10 monclient: _renew_subs 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7ee39d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7ee39d640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e8120c70 con 0x7ff7e809f7d0 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7ee39d640 10 monclient: _renew_subs 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7ee39d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7ee39d640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e8132120 con 0x7ff7e809f7d0 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff7d8004880 con 0x7ff7e809f7d0 2024-09-18T14:15:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff7d8005110 con 0x7ff7e809f7d0 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7dffff640 10 monclient: dump: 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.851 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.831+0000 7ff7e67fc640 4 set_mon_vals no callback set 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7ee39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7ee39d640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff7e804c4a0 con 0x7ff7e8099a40 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff7d80052f0 con 0x7ff7e809f7d0 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(743..743 src has 529..743) v4 ==== 3927+0+0 (unknown 55918879 0 0) 0x7ff7d8046c90 con 0x7ff7e809f7d0 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638853413 0 0) 0x7ff7e804c4a0 con 0x7ff7e8099a40 2024-09-18T14:15:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 10 monclient(hunting): my global_id is 6020 2024-09-18T14:15:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7e809d310 con 0x7ff7e8099a40 2024-09-18T14:15:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7ec93a640 10 monclient: get_auth_request con 0x7ff7c803a400 auth_method 0 2024-09-18T14:15:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3877720393 0 0) 0x7ff7e809d310 con 0x7ff7e8099a40 2024-09-18T14:15:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7e804c4a0 con 0x7ff7e8099a40 2024-09-18T14:15:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff7d0001860 con 0x7ff7e8099a40 2024-09-18T14:15:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.832+0000 7ff7dffff640 10 monclient: dump: 2024-09-18T14:15:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.857 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.833+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2083916903 0 0) 0x7ff7e804c4a0 con 0x7ff7e8099a40 2024-09-18T14:15:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.833+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7e81200d0 con 0x7ff7e8099a40 2024-09-18T14:15:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.833+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff7d0004570 con 0x7ff7e8099a40 2024-09-18T14:15:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.833+0000 7ff7dffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.833+0000 7ff7dffff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.833+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 >> v1:172.21.15.45:6793/0 conn(0x7ff7e8099a40 legacy=0x7ff7e8099e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.833+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 >> v1:172.21.15.45:6793/0 conn(0x7ff7e8099a40 legacy=0x7ff7e8099e10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.839+0000 7ff7ee39d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.839+0000 7ff7ee39d640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff7e81222b0 con 0x7ff7e809a520 2024-09-18T14:15:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425220464 0 0) 0x7ff7e81222b0 con 0x7ff7e809a520 2024-09-18T14:15:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 10 monclient(hunting): my global_id is 6029 2024-09-18T14:15:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7e804c4a0 con 0x7ff7e809a520 2024-09-18T14:15:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1780025034 0 0) 0x7ff7e804c4a0 con 0x7ff7e809a520 2024-09-18T14:15:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7e81222b0 con 0x7ff7e809a520 2024-09-18T14:15:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff7d0004df0 con 0x7ff7e809a520 2024-09-18T14:15:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 10 monclient: dump: 2024-09-18T14:15:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:32.861 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 239178282 0 0) 0x7ff7e81222b0 con 0x7ff7e809a520 2024-09-18T14:15:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff7e804b720 con 0x7ff7e809a520 2024-09-18T14:15:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7378 (unknown 0 0 2960648006) 0x7ff7d0007030 con 0x7ff7e809a520 2024-09-18T14:15:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 >> v1:172.21.15.45:6793/0 conn(0x7ff7e809a520 legacy=0x7ff7e814f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.840+0000 7ff7dffff640 1 -- 172.21.15.45:0/1331568243 >> v1:172.21.15.45:6793/0 conn(0x7ff7e809a520 legacy=0x7ff7e814f2f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:32.862 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:15:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 138, 2024-09-18T14:15:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:32.881 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.841+0000 7ff7ee39d640 1 -- 172.21.15.45:0/1331568243 >> v1:172.21.15.84:6812/31478 conn(0x7ff7c803a400 legacy=0x7ff7c803c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.841+0000 7ff7ee39d640 10 monclient: shutdown 2024-09-18T14:15:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.841+0000 7ff7ee39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.841+0000 7ff7ee39d640 1 -- 172.21.15.45:0/1331568243 >> v1:172.21.15.84:6792/0 conn(0x7ff7e809f7d0 legacy=0x7ff7e8141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.841+0000 7ff7ec93a640 1 -- 172.21.15.45:0/1331568243 reap_dead start 2024-09-18T14:15:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.841+0000 7ff7ec93a640 1 -- 172.21.15.45:0/1331568243 reap_dead start 2024-09-18T14:15:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.841+0000 7ff7ee39d640 1 -- 172.21.15.45:0/1331568243 shutdown_connections 2024-09-18T14:15:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.841+0000 7ff7ee39d640 1 -- 172.21.15.45:0/1331568243 >> 172.21.15.45:0/1331568243 conn(0x7ff7e80630b0 msgr2=0x7ff7e80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.841+0000 7ff7ee39d640 1 -- 172.21.15.45:0/1331568243 shutdown_connections 2024-09-18T14:15:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.841+0000 7ff7ee39d640 1 -- 172.21.15.45:0/1331568243 wait complete. 2024-09-18T14:15:32.883 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:15:32.883 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-09-18T14:15:32.883 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:15:32.917 INFO:tasks.rados.rados.0.smithi084.stdout:3376: done (1 left) 2024-09-18T14:15:32.917 INFO:tasks.rados.rados.0.smithi084.stdout:3377: delete oid 15 current snap is 371 2024-09-18T14:15:32.917 INFO:tasks.rados.rados.0.smithi084.stdout:3375: finishing write tid 1 to smithi08434663-29 2024-09-18T14:15:32.917 INFO:tasks.rados.rados.0.smithi084.stdout:3375: finishing write tid 2 to smithi08434663-29 2024-09-18T14:15:32.917 INFO:tasks.rados.rados.0.smithi084.stdout:3375: finishing write tid 3 to smithi08434663-29 2024-09-18T14:15:32.918 INFO:tasks.rados.rados.0.smithi084.stdout:3375: finishing write tid 4 to smithi08434663-29 2024-09-18T14:15:32.918 INFO:tasks.rados.rados.0.smithi084.stdout:3375: finishing write tid 5 to smithi08434663-29 2024-09-18T14:15:32.918 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 1564 (ObjNum 1148 snap 371 seq_num 1148) dirty exists 2024-09-18T14:15:32.918 INFO:tasks.rados.rados.0.smithi084.stdout:3375: left oid 29 (ObjNum 1148 snap 371 seq_num 1148) 2024-09-18T14:15:32.927 INFO:tasks.rados.rados.0.smithi084.stdout:3375: done (1 left) 2024-09-18T14:15:32.927 INFO:tasks.rados.rados.0.smithi084.stdout:3377: done (0 left) 2024-09-18T14:15:32.927 INFO:tasks.rados.rados.0.smithi084.stdout:3378: read oid 25 snap -1 2024-09-18T14:15:32.927 INFO:tasks.rados.rados.0.smithi084.stdout:3378: expect (ObjNum 1102 snap 352 seq_num 1102) 2024-09-18T14:15:32.927 INFO:tasks.rados.rados.0.smithi084.stdout:3379: delete oid 15 current snap is 371 2024-09-18T14:15:32.930 INFO:tasks.rados.rados.0.smithi084.stdout:3379: done (1 left) 2024-09-18T14:15:32.930 INFO:tasks.rados.rados.0.smithi084.stdout:3380: write oid 37 current snap is 371 2024-09-18T14:15:32.930 INFO:tasks.rados.rados.0.smithi084.stdout:3380: seq_num 1151 ranges {544857=580902,1670539=668389,3041747=303110} 2024-09-18T14:15:32.936 INFO:tasks.rados.rados.0.smithi084.stdout:3380: writing smithi08434663-37 from 544857 to 1125759 tid 1 2024-09-18T14:15:32.943 INFO:tasks.rados.rados.0.smithi084.stdout:3380: writing smithi08434663-37 from 1670539 to 2338928 tid 2 2024-09-18T14:15:32.949 INFO:tasks.rados.rados.0.smithi084.stdout:3380: writing smithi08434663-37 from 3041747 to 3344857 tid 3 2024-09-18T14:15:32.949 INFO:tasks.rados.rados.0.smithi084.stdout:3381: read oid 1 snap -1 2024-09-18T14:15:32.949 INFO:tasks.rados.rados.0.smithi084.stdout:3381: expect deleted 2024-09-18T14:15:32.949 INFO:tasks.rados.rados.0.smithi084.stdout:3382: snap_remove snap 352 2024-09-18T14:15:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.992+0000 7fa31c2c9640 10 monclient: get_monmap_and_config 2024-09-18T14:15:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.992+0000 7fa31c2c9640 10 monclient: build_initial_monmap 2024-09-18T14:15:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.992+0000 7fa31c2c9640 10 monclient: monmap: 2024-09-18T14:15:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:32.993113+0000 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:32.993113+0000 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:32.994 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.992+0000 7fa31c2c9640 1 Processor -- start 2024-09-18T14:15:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.992+0000 7fa31c2c9640 1 -- start start 2024-09-18T14:15:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.992+0000 7fa31c2c9640 10 monclient: init 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa31c2c9640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa31c2c9640 10 monclient: _add_conns ranks=[7,4,6,0,5,1,3,2,8] 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa31c2c9640 10 monclient(hunting): picked mon.noname-i con 0x7fa31406e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa31c2c9640 10 monclient(hunting): picked mon.noname-c con 0x7fa3140a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa31c2c9640 10 monclient(hunting): picked mon.noname-d con 0x7fa3140a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa31c2c9640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31405ed70 con 0x7fa3140a0cb0 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa31c2c9640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa314097620 con 0x7fa3140a1080 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa31c2c9640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa314097340 con 0x7fa31406e810 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa31c2c9640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa31c2c9640 10 monclient(hunting): authenticate will time out at 1748.331487s 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319864640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fa3140a0cb0 0x7fa314137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:40384/0 (socket says 172.21.15.45:40384) 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319864640 1 -- 172.21.15.45:0/421236454 learned_addr learned my addr 172.21.15.45:0/421236454 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa31a866640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fa3140a1080 0x7fa31413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:52170/0 (socket says 172.21.15.45:52170) 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491438747 0 0) 0x7fa314097620 con 0x7fa3140a1080 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 10 monclient(hunting): my global_id is 184857 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3000034e0 con 0x7fa3140a1080 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475052093 0 0) 0x7fa31405ed70 con 0x7fa3140a0cb0 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 10 monclient(hunting): my global_id is 165082 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa314097620 con 0x7fa3140a0cb0 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537021102 0 0) 0x7fa314097340 con 0x7fa31406e810 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 10 monclient(hunting): my global_id is 214378 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.993+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa31405ed70 con 0x7fa31406e810 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2248464993 0 0) 0x7fa314097620 con 0x7fa3140a0cb0 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 >> v1:172.21.15.84:6792/0 conn(0x7fa31406e810 legacy=0x7fa3140a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 >> v1:172.21.15.45:6792/0 conn(0x7fa3140a1080 legacy=0x7fa31413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 10 monclient: _finish_hunting 0 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 1 monclient: found mon.noname-c 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3140a1660 con 0x7fa3140a0cb0 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 10 monclient: _finish_auth 0 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa308001810 con 0x7fa3140a0cb0 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa31c2c9640 5 monclient: authenticate success, global_id 165082 2024-09-18T14:15:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa31c2c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 10 monclient: dump: 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa31c2c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa308001970 con 0x7fa3140a0cb0 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 1 -- 172.21.15.45:0/421236454 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa308004860 con 0x7fa3140a0cb0 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa31c2c9640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa31c2c9640 4 set_mon_vals no callback set 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa319063640 10 monclient: dump: 2024-09-18T14:15:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa31c2c9640 10 monclient: shutdown 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa31c2c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa31c2c9640 1 -- 172.21.15.45:0/421236454 >> v1:172.21.15.45:6791/0 conn(0x7fa3140a0cb0 legacy=0x7fa314137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa31c2c9640 1 -- 172.21.15.45:0/421236454 shutdown_connections 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa31c2c9640 1 -- 172.21.15.45:0/421236454 >> 172.21.15.45:0/421236454 conn(0x7fa3140a5bb0 msgr2=0x7fa3140a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.994+0000 7fa31c2c9640 1 -- 172.21.15.45:0/421236454 shutdown_connections 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.995+0000 7fa31c2c9640 1 -- 172.21.15.45:0/421236454 wait complete. 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.995+0000 7fa31c2c9640 10 monclient: build_initial_monmap 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.995+0000 7fa31c2c9640 10 monclient: monmap: 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.995+0000 7fa31c2c9640 1 Processor -- start 2024-09-18T14:15:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.995+0000 7fa31c2c9640 1 -- start start 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.995+0000 7fa31c2c9640 10 monclient: init 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa31c2c9640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa31c2c9640 10 monclient: _add_conns ranks=[7,4,6,2,5,0,8,3,1] 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa31c2c9640 10 monclient(hunting): picked mon.noname-i con 0x7fa31406e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa31c2c9640 10 monclient(hunting): picked mon.noname-c con 0x7fa3140a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa31c2c9640 10 monclient(hunting): picked mon.noname-d con 0x7fa3140a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa31c2c9640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa314122660 con 0x7fa3140a0cb0 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa31c2c9640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31413e750 con 0x7fa3140a1080 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa31c2c9640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31413d9f0 con 0x7fa31406e810 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa31c2c9640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa31c2c9640 10 monclient(hunting): authenticate will time out at 1748.334394s 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa31a866640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fa3140a1080 0x7fa314149c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:52186/0 (socket says 172.21.15.45:52186) 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa31a866640 1 -- 172.21.15.45:0/2219966915 learned_addr learned my addr 172.21.15.45:0/2219966915 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa319864640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fa3140a0cb0 0x7fa314146530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:40386/0 (socket says 172.21.15.45:40386) 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711374194 0 0) 0x7fa31413e750 con 0x7fa3140a1080 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): my global_id is 184866 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2f80034e0 con 0x7fa3140a1080 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994227569 0 0) 0x7fa314122660 con 0x7fa3140a0cb0 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): my global_id is 165091 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa31413e750 con 0x7fa3140a0cb0 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018286382 0 0) 0x7fa31413d9f0 con 0x7fa31406e810 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): my global_id is 214387 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa314122660 con 0x7fa31406e810 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3918943568 0 0) 0x7fa2f80034e0 con 0x7fa3140a1080 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa31413d9f0 con 0x7fa3140a1080 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa30c001d30 con 0x7fa3140a1080 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): dump: 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 713865022 0 0) 0x7fa31413e750 con 0x7fa3140a0cb0 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2f80034e0 con 0x7fa3140a0cb0 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa3080017d0 con 0x7fa3140a0cb0 2024-09-18T14:15:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.996+0000 7fa3067fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 10 monclient(hunting): dump: 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1695506442 0 0) 0x7fa314122660 con 0x7fa31406e810 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa31413e750 con 0x7fa31406e810 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa310001a10 con 0x7fa31406e810 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 10 monclient(hunting): dump: 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1866227464 0 0) 0x7fa31413d9f0 con 0x7fa3140a1080 2024-09-18T14:15:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 >> v1:172.21.15.84:6792/0 conn(0x7fa31406e810 legacy=0x7fa31413d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 >> v1:172.21.15.45:6791/0 conn(0x7fa3140a0cb0 legacy=0x7fa314146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 10 monclient: _finish_hunting 0 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 1 monclient: found mon.d 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa314122380 con 0x7fa3140a1080 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 10 monclient: _finish_auth 0 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa31c2c9640 5 monclient: authenticate success, global_id 184866 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa31c2c9640 10 monclient: _renew_subs 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa31c2c9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa31c2c9640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa314122940 con 0x7fa3140a1080 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa31c2c9640 10 monclient: _renew_subs 2024-09-18T14:15:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa31c2c9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa31c2c9640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa314121650 con 0x7fa3140a1080 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa30c001ff0 con 0x7fa3140a1080 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa30c005110 con 0x7fa3140a1080 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa318862640 4 set_mon_vals no callback set 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 10 monclient: dump: 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.997+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fa30c0100e0 con 0x7fa3140a1080 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa31c2c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa31c2c9640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa2fc006580 con 0x7fa2fc002830 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(744..744 src has 529..744) v4 ==== 3971+0+0 (unknown 3516258429 0 0) 0x7fa30c046d90 con 0x7fa3140a1080 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa319864640 10 monclient: get_auth_request con 0x7fa2f803aac0 auth_method 0 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974322119 0 0) 0x7fa2fc006580 con 0x7fa2fc002830 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 10 monclient(hunting): my global_id is 214336 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa31413d9f0 con 0x7fa2fc002830 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 518002643 0 0) 0x7fa31413d9f0 con 0x7fa2fc002830 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2fc006580 con 0x7fa2fc002830 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa3100021a0 con 0x7fa2fc002830 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 10 monclient: dump: 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.022 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2595670134 0 0) 0x7fa2fc006580 con 0x7fa2fc002830 2024-09-18T14:15:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.998+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2fc00a0a0 con 0x7fa2fc002830 2024-09-18T14:15:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.999+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa3100044a0 con 0x7fa2fc002830 2024-09-18T14:15:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.999+0000 7fa3067fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.999+0000 7fa3067fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.999+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 >> v1:172.21.15.84:6789/0 conn(0x7fa2fc002830 legacy=0x7fa2fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:32.999+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 >> v1:172.21.15.84:6789/0 conn(0x7fa2fc002830 legacy=0x7fa2fc004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa31c2c9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa31c2c9640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa2fc007540 con 0x7fa2fc0068f0 2024-09-18T14:15:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819816831 0 0) 0x7fa2fc007540 con 0x7fa2fc0068f0 2024-09-18T14:15:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 10 monclient(hunting): my global_id is 214345 2024-09-18T14:15:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2fc006580 con 0x7fa2fc0068f0 2024-09-18T14:15:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 813081441 0 0) 0x7fa2fc006580 con 0x7fa2fc0068f0 2024-09-18T14:15:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2fc007540 con 0x7fa2fc0068f0 2024-09-18T14:15:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa310002b70 con 0x7fa2fc0068f0 2024-09-18T14:15:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:15:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 10 monclient: dump: 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.026 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2806480894 0 0) 0x7fa2fc007540 con 0x7fa2fc0068f0 2024-09-18T14:15:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.005+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa2fc0087a0 con 0x7fa2fc0068f0 2024-09-18T14:15:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7546 (unknown 0 0 95213485) 0x7fa310003e40 con 0x7fa2fc0068f0 2024-09-18T14:15:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa3067fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa3067fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 >> v1:172.21.15.84:6789/0 conn(0x7fa2fc0068f0 legacy=0x7fa2fc006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa3067fc640 1 -- 172.21.15.45:0/2219966915 >> v1:172.21.15.84:6789/0 conn(0x7fa2fc0068f0 legacy=0x7fa2fc006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:33.026 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 138, 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.046 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:33.046 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa31c2c9640 1 -- 172.21.15.45:0/2219966915 >> v1:172.21.15.84:6812/31478 conn(0x7fa2f803aac0 legacy=0x7fa2f803cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa31c2c9640 10 monclient: shutdown 2024-09-18T14:15:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa31c2c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa31c2c9640 1 -- 172.21.15.45:0/2219966915 >> v1:172.21.15.45:6792/0 conn(0x7fa3140a1080 legacy=0x7fa314149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa31a866640 1 -- 172.21.15.45:0/2219966915 reap_dead start 2024-09-18T14:15:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa31a866640 1 -- 172.21.15.45:0/2219966915 reap_dead start 2024-09-18T14:15:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa31c2c9640 1 -- 172.21.15.45:0/2219966915 shutdown_connections 2024-09-18T14:15:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa31c2c9640 1 -- 172.21.15.45:0/2219966915 >> 172.21.15.45:0/2219966915 conn(0x7fa3140a5bb0 msgr2=0x7fa3140a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa31c2c9640 1 -- 172.21.15.45:0/2219966915 shutdown_connections 2024-09-18T14:15:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.006+0000 7fa31c2c9640 1 -- 172.21.15.45:0/2219966915 wait complete. 2024-09-18T14:15:33.048 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:15:33.048 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-09-18T14:15:33.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:15:33.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.145+0000 7fce8b057640 10 monclient: get_monmap_and_config 2024-09-18T14:15:33.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.145+0000 7fce8b057640 10 monclient: build_initial_monmap 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.145+0000 7fce8b057640 10 monclient: monmap: 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:33.146653+0000 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:33.146653+0000 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:33.148 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 1 Processor -- start 2024-09-18T14:15:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 1 -- start start 2024-09-18T14:15:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 10 monclient: init 2024-09-18T14:15:33.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 10 monclient: _add_conns ranks=[6,0,5,2,8,1,4,7,3] 2024-09-18T14:15:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 10 monclient(hunting): picked mon.noname-d con 0x7fce8409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 10 monclient(hunting): picked mon.noname-a con 0x7fce8409fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:15:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 10 monclient(hunting): picked mon.noname-h con 0x7fce8409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8405ed70 con 0x7fce8409fba0 2024-09-18T14:15:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8409dfc0 con 0x7fce8409ff70 2024-09-18T14:15:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8409dce0 con 0x7fce8409f7d0 2024-09-18T14:15:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.146+0000 7fce8b057640 10 monclient(hunting): authenticate will time out at 1748.484997s 2024-09-18T14:15:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce88df3640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fce8409f7d0 0x7fce84132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:52194/0 (socket says 172.21.15.45:52194) 2024-09-18T14:15:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce88df3640 1 -- 172.21.15.45:0/2933383720 learned_addr learned my addr 172.21.15.45:0/2933383720 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce83fff640 1 --1- 172.21.15.45:0/2933383720 >> v1:172.21.15.45:6789/0 conn(0x7fce8409fba0 0x7fce84137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45290/0 (socket says 172.21.15.45:45290) 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613780412 0 0) 0x7fce8409dce0 con 0x7fce8409f7d0 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient(hunting): my global_id is 184875 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce740034e0 con 0x7fce8409f7d0 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268387193 0 0) 0x7fce8405ed70 con 0x7fce8409fba0 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient(hunting): my global_id is 6336 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8409dce0 con 0x7fce8409fba0 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186976729 0 0) 0x7fce8409dfc0 con 0x7fce8409ff70 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient(hunting): my global_id is 184721 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8405ed70 con 0x7fce8409ff70 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4212961433 0 0) 0x7fce740034e0 con 0x7fce8409f7d0 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 >> v1:172.21.15.84:6791/0 conn(0x7fce8409ff70 legacy=0x7fce8413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 >> v1:172.21.15.45:6789/0 conn(0x7fce8409fba0 legacy=0x7fce84137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient: _finish_hunting 0 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 monclient: found mon.noname-d 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce840a1c00 con 0x7fce8409f7d0 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient: _finish_auth 0 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce8b057640 5 monclient: authenticate success, global_id 184875 2024-09-18T14:15:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce8b057640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce78001830 con 0x7fce8409f7d0 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient: dump: 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce8b057640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce78001a10 con 0x7fce8409f7d0 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 1 -- 172.21.15.45:0/2933383720 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce78004ba0 con 0x7fce8409f7d0 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce8b057640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce8b057640 4 set_mon_vals no callback set 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.147+0000 7fce837fe640 10 monclient: dump: 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.148+0000 7fce8b057640 10 monclient: shutdown 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.148+0000 7fce8b057640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.148+0000 7fce8b057640 1 -- 172.21.15.45:0/2933383720 >> v1:172.21.15.45:6792/0 conn(0x7fce8409f7d0 legacy=0x7fce84132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.148+0000 7fce8b057640 1 -- 172.21.15.45:0/2933383720 shutdown_connections 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.148+0000 7fce8b057640 1 -- 172.21.15.45:0/2933383720 >> 172.21.15.45:0/2933383720 conn(0x7fce840630b0 msgr2=0x7fce840634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.148+0000 7fce8b057640 1 -- 172.21.15.45:0/2933383720 shutdown_connections 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.148+0000 7fce8b057640 1 -- 172.21.15.45:0/2933383720 wait complete. 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.148+0000 7fce8b057640 10 monclient: build_initial_monmap 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.148+0000 7fce8b057640 10 monclient: monmap: 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 1 Processor -- start 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 1 -- start start 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 10 monclient: init 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 10 monclient: _add_conns ranks=[5,3,8,4,1,0,7,6,2] 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 10 monclient(hunting): picked mon.noname-h con 0x7fce8409f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 10 monclient(hunting): picked mon.noname-g con 0x7fce8409fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 10 monclient(hunting): picked mon.noname-e con 0x7fce8409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8409d310 con 0x7fce8409fba0 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce840a2560 con 0x7fce8409f7d0 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce84122480 con 0x7fce8409ff70 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce8b057640 10 monclient(hunting): authenticate will time out at 1748.487830s 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce83fff640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7fce8409fba0 legacy=0x7fce84131a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:15:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce895f4640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fce8409ff70 0x7fce84149b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:48270/0 (socket says 172.21.15.45:48270) 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.149+0000 7fce895f4640 1 -- 172.21.15.45:0/679059920 learned_addr learned my addr 172.21.15.45:0/679059920 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833274470 0 0) 0x7fce8405ed70 con 0x7fce8409ff70 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): my global_id is 6038 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce680034e0 con 0x7fce8409ff70 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326163676 0 0) 0x7fce840a2560 con 0x7fce8409f7d0 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): my global_id is 184730 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8405ed70 con 0x7fce8409f7d0 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 303400073 0 0) 0x7fce680034e0 con 0x7fce8409ff70 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce840a2560 con 0x7fce8409ff70 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce7c002e50 con 0x7fce8409ff70 2024-09-18T14:15:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): dump: 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1743339129 0 0) 0x7fce8405ed70 con 0x7fce8409f7d0 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce680034e0 con 0x7fce8409f7d0 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce78004b20 con 0x7fce8409f7d0 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient(hunting): dump: 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4016431868 0 0) 0x7fce840a2560 con 0x7fce8409ff70 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 >> v1:172.21.15.84:6791/0 conn(0x7fce8409f7d0 legacy=0x7fce84141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 >> v1:172.21.15.84:6790/0 conn(0x7fce8409fba0 legacy=0x7fce84131a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 monclient: found mon.e 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce84121e70 con 0x7fce8409ff70 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient: _finish_auth 0 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce8b057640 5 monclient: authenticate success, global_id 6038 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce8b057640 10 monclient: _renew_subs 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce8b057640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce8b057640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce84121b70 con 0x7fce8409ff70 2024-09-18T14:15:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce8b057640 10 monclient: _renew_subs 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce8b057640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce8b057640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fce84120c10 con 0x7fce8409ff70 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce7c003110 con 0x7fce8409ff70 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce7c001d00 con 0x7fce8409ff70 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce82ffd640 4 set_mon_vals no callback set 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.150+0000 7fce80ff9640 10 monclient: dump: 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.151+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fce7c001e60 con 0x7fce8409ff70 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.151+0000 7fce8b057640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.151+0000 7fce8b057640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fce64006580 con 0x7fce64002830 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.151+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(744..744 src has 529..744) v4 ==== 3971+0+0 (unknown 3516258429 0 0) 0x7fce7c046c60 con 0x7fce8409ff70 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.151+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152266877 0 0) 0x7fce64006580 con 0x7fce64002830 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.151+0000 7fce80ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.151+0000 7fce80ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.151+0000 7fce80ff9640 10 monclient(hunting): my global_id is 184739 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.151+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce840a2560 con 0x7fce64002830 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.151+0000 7fce83fff640 10 monclient: get_auth_request con 0x7fce6803a2d0 auth_method 0 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1687764578 0 0) 0x7fce840a2560 con 0x7fce64002830 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce64006580 con 0x7fce64002830 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce78001720 con 0x7fce64002830 2024-09-18T14:15:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 10 monclient: dump: 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.174 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.175 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.175 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.175 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.175 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.175 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 185539833 0 0) 0x7fce64006580 con 0x7fce64002830 2024-09-18T14:15:33.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce6400a0a0 con 0x7fce64002830 2024-09-18T14:15:33.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fce78005160 con 0x7fce64002830 2024-09-18T14:15:33.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:33.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:33.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 >> v1:172.21.15.84:6791/0 conn(0x7fce64002830 legacy=0x7fce64004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.152+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 >> v1:172.21.15.84:6791/0 conn(0x7fce64002830 legacy=0x7fce64004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.158+0000 7fce8b057640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.158+0000 7fce8b057640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fce64009660 con 0x7fce64009040 2024-09-18T14:15:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.158+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874912469 0 0) 0x7fce64009660 con 0x7fce64009040 2024-09-18T14:15:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.158+0000 7fce80ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.158+0000 7fce80ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.158+0000 7fce80ff9640 10 monclient(hunting): my global_id is 184748 2024-09-18T14:15:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.158+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce64006580 con 0x7fce64009040 2024-09-18T14:15:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.158+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 639735090 0 0) 0x7fce64006580 con 0x7fce64009040 2024-09-18T14:15:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce64009660 con 0x7fce64009040 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fce78005920 con 0x7fce64009040 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce80ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce80ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce80ff9640 10 monclient: dump: 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3446938675 0 0) 0x7fce64009660 con 0x7fce64009040 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fce64009920 con 0x7fce64009040 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7546 (unknown 0 0 2477642558) 0x7fce78008150 con 0x7fce64009040 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce80ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce80ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 >> v1:172.21.15.84:6791/0 conn(0x7fce64009040 legacy=0x7fce64008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce80ff9640 1 -- 172.21.15.45:0/679059920 >> v1:172.21.15.84:6791/0 conn(0x7fce64009040 legacy=0x7fce64008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:33.180 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 138, 2024-09-18T14:15:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:15:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:33.199 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce8b057640 1 -- 172.21.15.45:0/679059920 >> v1:172.21.15.84:6812/31478 conn(0x7fce6803a2d0 legacy=0x7fce6803c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce8b057640 10 monclient: shutdown 2024-09-18T14:15:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce8b057640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce8b057640 1 -- 172.21.15.45:0/679059920 >> v1:172.21.15.45:6793/0 conn(0x7fce8409ff70 legacy=0x7fce84149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce895f4640 1 -- 172.21.15.45:0/679059920 reap_dead start 2024-09-18T14:15:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce895f4640 1 -- 172.21.15.45:0/679059920 reap_dead start 2024-09-18T14:15:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce8b057640 1 -- 172.21.15.45:0/679059920 shutdown_connections 2024-09-18T14:15:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.159+0000 7fce8b057640 1 -- 172.21.15.45:0/679059920 >> 172.21.15.45:0/679059920 conn(0x7fce840630b0 msgr2=0x7fce840a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.160+0000 7fce8b057640 1 -- 172.21.15.45:0/679059920 shutdown_connections 2024-09-18T14:15:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.160+0000 7fce8b057640 1 -- 172.21.15.45:0/679059920 wait complete. 2024-09-18T14:15:33.201 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:15:33.201 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-09-18T14:15:33.202 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:15:33.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.298+0000 7f4a37057640 10 monclient: get_monmap_and_config 2024-09-18T14:15:33.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.298+0000 7f4a37057640 10 monclient: build_initial_monmap 2024-09-18T14:15:33.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.298+0000 7f4a37057640 10 monclient: monmap: 2024-09-18T14:15:33.299 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:15:33.299230+0000 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:15:33.299230+0000 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:33.301 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 1 Processor -- start 2024-09-18T14:15:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 1 -- start start 2024-09-18T14:15:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 10 monclient: init 2024-09-18T14:15:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 10 monclient: _add_conns ranks=[7,3,6,1,0,5,2,4,8] 2024-09-18T14:15:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 10 monclient(hunting): picked mon.noname-i con 0x7f4a30136da0 addr v1:172.21.15.84:6792/0 2024-09-18T14:15:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 10 monclient(hunting): picked mon.noname-g con 0x7f4a300a15b0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 10 monclient(hunting): picked mon.noname-d con 0x7f4a3013cd90 addr v1:172.21.15.45:6792/0 2024-09-18T14:15:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3005ed70 con 0x7f4a300a15b0 2024-09-18T14:15:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a300a1980 con 0x7f4a3013cd90 2024-09-18T14:15:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3006bb70 con 0x7f4a30136da0 2024-09-18T14:15:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a37057640 10 monclient(hunting): authenticate will time out at 1748.637600s 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a355f4640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f4a3013cd90 0x7f4a3013f170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:52206/0 (socket says 172.21.15.45:52206) 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a2ffff640 1 -- >> v1:172.21.15.84:6790/0 conn(0x7f4a300a15b0 legacy=0x7f4a30137170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a355f4640 1 -- 172.21.15.45:0/4269612543 learned_addr learned my addr 172.21.15.45:0/4269612543 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a2f7fe640 1 -- 172.21.15.45:0/4269612543 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658072457 0 0) 0x7f4a300a1980 con 0x7f4a3013cd90 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a2f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a2f7fe640 10 monclient(hunting): my global_id is 184884 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.299+0000 7f4a2f7fe640 1 -- 172.21.15.45:0/4269612543 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a200034e0 con 0x7f4a3013cd90 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 1 -- 172.21.15.45:0/4269612543 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304972937 0 0) 0x7f4a3006bb70 con 0x7f4a30136da0 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient(hunting): my global_id is 214396 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 1 -- 172.21.15.45:0/4269612543 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a300a1980 con 0x7f4a30136da0 2024-09-18T14:15:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 1 -- 172.21.15.45:0/4269612543 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1281577546 0 0) 0x7f4a200034e0 con 0x7f4a3013cd90 2024-09-18T14:15:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 1 -- 172.21.15.45:0/4269612543 >> v1:172.21.15.84:6792/0 conn(0x7f4a30136da0 legacy=0x7f4a3006ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 1 -- 172.21.15.45:0/4269612543 >> v1:172.21.15.84:6790/0 conn(0x7f4a300a15b0 legacy=0x7f4a30137170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:15:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:15:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 1 monclient: found mon.noname-d 2024-09-18T14:15:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:15:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 1 -- 172.21.15.45:0/4269612543 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3006be50 con 0x7f4a3013cd90 2024-09-18T14:15:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient: _finish_auth 0 2024-09-18T14:15:33.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 1 -- 172.21.15.45:0/4269612543 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a28001bb0 con 0x7f4a3013cd90 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient: dump: 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a37057640 5 monclient: authenticate success, global_id 184884 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a37057640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 1 -- 172.21.15.45:0/4269612543 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4a28001d90 con 0x7f4a3013cd90 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 1 -- 172.21.15.45:0/4269612543 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a28004f20 con 0x7f4a3013cd90 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a2f7fe640 10 monclient: dump: 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a37057640 10 monclient: get_monmap_and_config success 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a37057640 4 set_mon_vals no callback set 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a37057640 10 monclient: shutdown 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a37057640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a37057640 1 -- 172.21.15.45:0/4269612543 >> v1:172.21.15.45:6792/0 conn(0x7f4a3013cd90 legacy=0x7f4a3013f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a37057640 1 -- 172.21.15.45:0/4269612543 shutdown_connections 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a37057640 1 -- 172.21.15.45:0/4269612543 >> 172.21.15.45:0/4269612543 conn(0x7f4a300a5bb0 msgr2=0x7f4a300a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a37057640 1 -- 172.21.15.45:0/4269612543 shutdown_connections 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.300+0000 7f4a37057640 1 -- 172.21.15.45:0/4269612543 wait complete. 2024-09-18T14:15:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.301+0000 7f4a37057640 10 monclient: build_initial_monmap 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.301+0000 7f4a37057640 10 monclient: monmap: 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:15:33.315 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.301+0000 7f4a37057640 1 Processor -- start 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.301+0000 7f4a37057640 1 -- start start 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.301+0000 7f4a37057640 10 monclient: init 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a37057640 10 monclient: _reopen_session rank -1 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a37057640 10 monclient: _add_conns ranks=[2,3,4,7,8,6,5,0,1] 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a37057640 10 monclient(hunting): picked mon.noname-b con 0x7f4a300a15b0 addr v1:172.21.15.45:6790/0 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a37057640 10 monclient(hunting): picked mon.noname-g con 0x7f4a30136da0 addr v1:172.21.15.84:6790/0 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a37057640 10 monclient(hunting): picked mon.noname-c con 0x7f4a3013cd90 addr v1:172.21.15.45:6791/0 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a37057640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3009fc60 con 0x7f4a300a15b0 2024-09-18T14:15:33.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a37057640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3013b050 con 0x7f4a30136da0 2024-09-18T14:15:33.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a37057640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a30138f60 con 0x7f4a3013cd90 2024-09-18T14:15:33.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a37057640 10 monclient(hunting): _renew_subs 2024-09-18T14:15:33.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a37057640 10 monclient(hunting): authenticate will time out at 1748.640337s 2024-09-18T14:15:33.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a34df3640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f4a300a15b0 0x7f4a30138140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:57410/0 (socket says 172.21.15.45:57410) 2024-09-18T14:15:33.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a34df3640 1 -- 172.21.15.45:0/3208376986 learned_addr learned my addr 172.21.15.45:0/3208376986 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:15:33.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2ffff640 1 -- 172.21.15.45:0/3208376986 >> v1:172.21.15.84:6790/0 conn(0x7f4a30136da0 legacy=0x7f4a30138850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.84:6790/0 2024-09-18T14:15:33.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276588036 0 0) 0x7f4a30138f60 con 0x7f4a3013cd90 2024-09-18T14:15:33.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 10 monclient(hunting): my global_id is 165100 2024-09-18T14:15:33.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a140034e0 con 0x7f4a3013cd90 2024-09-18T14:15:33.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712763596 0 0) 0x7f4a3009fc60 con 0x7f4a300a15b0 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 10 monclient(hunting): my global_id is 145469 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a30138f60 con 0x7f4a300a15b0 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 442285065 0 0) 0x7f4a140034e0 con 0x7f4a3013cd90 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a3009fc60 con 0x7f4a3013cd90 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a28001b30 con 0x7f4a3013cd90 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.302+0000 7f4a2cff9640 10 monclient(hunting): dump: 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.319 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2429587538 0 0) 0x7f4a30138f60 con 0x7f4a300a15b0 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a140034e0 con 0x7f4a300a15b0 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a24001a60 con 0x7f4a300a15b0 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 10 monclient(hunting): dump: 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1963420253 0 0) 0x7f4a3009fc60 con 0x7f4a3013cd90 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 >> v1:172.21.15.84:6790/0 conn(0x7f4a30136da0 legacy=0x7f4a30138850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 >> v1:172.21.15.45:6790/0 conn(0x7f4a300a15b0 legacy=0x7f4a30138140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 10 monclient: _finish_hunting 0 2024-09-18T14:15:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 1 monclient: found mon.c 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30139240 con 0x7f4a3013cd90 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 10 monclient: _finish_auth 0 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a37057640 5 monclient: authenticate success, global_id 165100 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a37057640 10 monclient: _renew_subs 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a37057640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a37057640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3013ad70 con 0x7f4a3013cd90 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a37057640 10 monclient: _renew_subs 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a37057640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a37057640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3013bbd0 con 0x7f4a3013cd90 2024-09-18T14:15:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4a28004ed0 con 0x7f4a3013cd90 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a28005400 con 0x7f4a3013cd90 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2effd640 4 set_mon_vals no callback set 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 10 monclient: dump: 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.324 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a37057640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.303+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4a28005560 con 0x7f4a3013cd90 2024-09-18T14:15:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a37057640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4a10006580 con 0x7f4a10002830 2024-09-18T14:15:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(744..744 src has 529..744) v4 ==== 3971+0+0 (unknown 3516258429 0 0) 0x7f4a28046f70 con 0x7f4a3013cd90 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2ffff640 10 monclient: get_auth_request con 0x7f4a1403a3a0 auth_method 0 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800181191 0 0) 0x7f4a10006580 con 0x7f4a10002830 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 10 monclient(hunting): my global_id is 214405 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a3009fc60 con 0x7f4a10002830 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 620898967 0 0) 0x7f4a3009fc60 con 0x7f4a10002830 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a10006580 con 0x7f4a10002830 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a24001f40 con 0x7f4a10002830 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 10 monclient: dump: 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.327 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.327 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.304+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 214455675 0 0) 0x7f4a10006580 con 0x7f4a10002830 2024-09-18T14:15:33.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.305+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a1000a0a0 con 0x7f4a10002830 2024-09-18T14:15:33.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.305+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f4a24002480 con 0x7f4a10002830 2024-09-18T14:15:33.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.305+0000 7f4a2cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:15:33.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.305+0000 7f4a2cff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:15:33.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.305+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 >> v1:172.21.15.84:6792/0 conn(0x7f4a10002830 legacy=0x7f4a10004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.305+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 >> v1:172.21.15.84:6792/0 conn(0x7f4a10002830 legacy=0x7f4a10004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a37057640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:15:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a37057640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4a10009610 con 0x7f4a10008ff0 2024-09-18T14:15:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460673248 0 0) 0x7f4a10009610 con 0x7f4a10008ff0 2024-09-18T14:15:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a2cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:15:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a2cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:15:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a2cff9640 10 monclient(hunting): my global_id is 214414 2024-09-18T14:15:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a10006580 con 0x7f4a10008ff0 2024-09-18T14:15:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 615880171 0 0) 0x7f4a10006580 con 0x7f4a10008ff0 2024-09-18T14:15:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a10009610 con 0x7f4a10008ff0 2024-09-18T14:15:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4a24002a20 con 0x7f4a10008ff0 2024-09-18T14:15:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a2cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:15:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a2cff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.311+0000 7f4a2cff9640 10 monclient: dump: 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:15:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:15:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:15:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:15:33.331 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:15:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1702276155 0 0) 0x7f4a10009610 con 0x7f4a10008ff0 2024-09-18T14:15:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4a100098d0 con 0x7f4a10008ff0 2024-09-18T14:15:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7546 (unknown 0 0 4247892541) 0x7f4a24007d90 con 0x7f4a10008ff0 2024-09-18T14:15:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a2cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:15:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a2cff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:15:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 >> v1:172.21.15.84:6792/0 conn(0x7f4a10008ff0 legacy=0x7f4a10008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a2cff9640 1 -- 172.21.15.45:0/3208376986 >> v1:172.21.15.84:6792/0 conn(0x7f4a10008ff0 legacy=0x7f4a10008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:15:33.331 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 138, 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:15:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:15:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:15:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:15:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:15:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:15:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:15:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:15:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:15:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:15:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:15:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.343 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:15:33.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:15:33.343 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.343 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.343 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:15:33.343 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.343 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.343 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.343 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.344 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:15:33.345 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:15:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:15:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:15:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:15:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:15:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:15:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:15:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:15:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:15:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:15:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:15:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:15:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:15:33.351 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:15:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a37057640 1 -- 172.21.15.45:0/3208376986 >> v1:172.21.15.84:6812/31478 conn(0x7f4a1403a3a0 legacy=0x7f4a1403c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a37057640 10 monclient: shutdown 2024-09-18T14:15:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a37057640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:15:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a37057640 1 -- 172.21.15.45:0/3208376986 >> v1:172.21.15.45:6791/0 conn(0x7f4a3013cd90 legacy=0x7f4a3014e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:15:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a355f4640 1 -- 172.21.15.45:0/3208376986 reap_dead start 2024-09-18T14:15:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a355f4640 1 -- 172.21.15.45:0/3208376986 reap_dead start 2024-09-18T14:15:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a37057640 1 -- 172.21.15.45:0/3208376986 shutdown_connections 2024-09-18T14:15:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a37057640 1 -- 172.21.15.45:0/3208376986 >> 172.21.15.45:0/3208376986 conn(0x7f4a300a5bb0 msgr2=0x7f4a30132d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:15:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a37057640 1 -- 172.21.15.45:0/3208376986 shutdown_connections 2024-09-18T14:15:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:15:33.312+0000 7f4a37057640 1 -- 172.21.15.45:0/3208376986 wait complete. 2024-09-18T14:15:33.354 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-18T14:15:33.354 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-18T14:15:33.920 INFO:tasks.rados.rados.0.smithi084.stdout:3382: done (3 left) 2024-09-18T14:15:33.921 INFO:tasks.rados.rados.0.smithi084.stdout:3383: snap_create 2024-09-18T14:15:33.921 INFO:tasks.rados.rados.0.smithi084.stdout:3378: expect (ObjNum 1102 snap 352 seq_num 1102) 2024-09-18T14:15:34.091 INFO:tasks.rados.rados.0.smithi084.stdout:3380: finishing write tid 1 to smithi08434663-37 2024-09-18T14:15:34.091 INFO:tasks.rados.rados.0.smithi084.stdout:3380: finishing write tid 2 to smithi08434663-37 2024-09-18T14:15:34.091 INFO:tasks.rados.rados.0.smithi084.stdout:3380: finishing write tid 3 to smithi08434663-37 2024-09-18T14:15:34.091 INFO:tasks.rados.rados.0.smithi084.stdout:3380: finishing write tid 4 to smithi08434663-37 2024-09-18T14:15:34.091 INFO:tasks.rados.rados.0.smithi084.stdout:3380: finishing write tid 5 to smithi08434663-37 2024-09-18T14:15:34.091 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 1657 (ObjNum 1151 snap 371 seq_num 1151) dirty exists 2024-09-18T14:15:34.091 INFO:tasks.rados.rados.0.smithi084.stdout:3380: left oid 37 (ObjNum 1151 snap 371 seq_num 1151) 2024-09-18T14:15:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:3378: done (3 left) 2024-09-18T14:15:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:3380: done (2 left) 2024-09-18T14:15:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:3381: done (1 left) 2024-09-18T14:15:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:3383: done (0 left) 2024-09-18T14:15:34.925 INFO:tasks.rados.rados.0.smithi084.stdout:3384: snap_create 2024-09-18T14:15:35.865 INFO:tasks.rados.rados.0.smithi084.stdout:3384: done (0 left) 2024-09-18T14:15:35.865 INFO:tasks.rados.rados.0.smithi084.stdout:3385: write (excl) oid 25 current snap is 373 2024-09-18T14:15:35.865 INFO:tasks.rados.rados.0.smithi084.stdout:3385: seq_num 1152 ranges {561664=656151,1729021=528400,2669170=692494} 2024-09-18T14:15:35.872 INFO:tasks.rados.rados.0.smithi084.stdout:3385: writing smithi08434663-25 from 561664 to 1217815 tid 1 2024-09-18T14:15:35.878 INFO:tasks.rados.rados.0.smithi084.stdout:3385: writing smithi08434663-25 from 1729021 to 2257421 tid 2 2024-09-18T14:15:35.885 INFO:tasks.rados.rados.0.smithi084.stdout:3385: writing smithi08434663-25 from 2669170 to 3361664 tid 3 2024-09-18T14:15:35.885 INFO:tasks.rados.rados.0.smithi084.stdout:3386: read oid 42 snap -1 2024-09-18T14:15:35.885 INFO:tasks.rados.rados.0.smithi084.stdout:3386: expect deleted 2024-09-18T14:15:35.885 INFO:tasks.rados.rados.0.smithi084.stdout:3387: read oid 47 snap -1 2024-09-18T14:15:35.885 INFO:tasks.rados.rados.0.smithi084.stdout:3387: expect deleted 2024-09-18T14:15:35.886 INFO:tasks.rados.rados.0.smithi084.stdout:3388: snap_create 2024-09-18T14:15:35.890 INFO:tasks.rados.rados.0.smithi084.stdout:3385: finishing write tid 1 to smithi08434663-25 2024-09-18T14:15:35.896 INFO:tasks.rados.rados.0.smithi084.stdout:3385: finishing write tid 2 to smithi08434663-25 2024-09-18T14:15:35.896 INFO:tasks.rados.rados.0.smithi084.stdout:3385: finishing write tid 3 to smithi08434663-25 2024-09-18T14:15:35.896 INFO:tasks.rados.rados.0.smithi084.stdout:3385: finishing write tid 4 to smithi08434663-25 2024-09-18T14:15:35.897 INFO:tasks.rados.rados.0.smithi084.stdout:3385: finishing write tid 5 to smithi08434663-25 2024-09-18T14:15:35.897 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 1548 (ObjNum 1152 snap 373 seq_num 1152) dirty exists 2024-09-18T14:15:35.897 INFO:tasks.rados.rados.0.smithi084.stdout:3385: left oid 25 (ObjNum 1152 snap 373 seq_num 1152) 2024-09-18T14:15:36.879 INFO:tasks.rados.rados.0.smithi084.stdout:3385: done (3 left) 2024-09-18T14:15:36.879 INFO:tasks.rados.rados.0.smithi084.stdout:3386: done (2 left) 2024-09-18T14:15:36.879 INFO:tasks.rados.rados.0.smithi084.stdout:3387: done (1 left) 2024-09-18T14:15:36.879 INFO:tasks.rados.rados.0.smithi084.stdout:3388: done (0 left) 2024-09-18T14:15:36.879 INFO:tasks.rados.rados.0.smithi084.stdout:3389: read oid 11 snap -1 2024-09-18T14:15:36.880 INFO:tasks.rados.rados.0.smithi084.stdout:3389: expect deleted 2024-09-18T14:15:36.880 INFO:tasks.rados.rados.0.smithi084.stdout:3390: snap_remove snap 370 2024-09-18T14:15:37.882 INFO:tasks.rados.rados.0.smithi084.stdout:3390: done (1 left) 2024-09-18T14:15:37.882 INFO:tasks.rados.rados.0.smithi084.stdout:3391: write oid 30 current snap is 374 2024-09-18T14:15:37.882 INFO:tasks.rados.rados.0.smithi084.stdout:3391: seq_num 1153 ranges {578471=731400,1787503=672058,3096593=281878} 2024-09-18T14:15:37.889 INFO:tasks.rados.rados.0.smithi084.stdout:3391: writing smithi08434663-30 from 578471 to 1309871 tid 1 2024-09-18T14:15:37.895 INFO:tasks.rados.rados.0.smithi084.stdout:3391: writing smithi08434663-30 from 1787503 to 2459561 tid 2 2024-09-18T14:15:37.900 INFO:tasks.rados.rados.0.smithi084.stdout:3391: writing smithi08434663-30 from 3096593 to 3378471 tid 3 2024-09-18T14:15:37.900 INFO:tasks.rados.rados.0.smithi084.stdout:3392: write (excl) oid 31 current snap is 374 2024-09-18T14:15:37.901 INFO:tasks.rados.rados.0.smithi084.stdout:3392: seq_num 1154 ranges {595278=406649,1729632=532069,3007663=387615} 2024-09-18T14:15:37.906 INFO:tasks.rados.rados.0.smithi084.stdout:3392: writing smithi08434663-31 from 595278 to 1001927 tid 1 2024-09-18T14:15:37.912 INFO:tasks.rados.rados.0.smithi084.stdout:3392: writing smithi08434663-31 from 1729632 to 2261701 tid 2 2024-09-18T14:15:37.919 INFO:tasks.rados.rados.0.smithi084.stdout:3392: writing smithi08434663-31 from 3007663 to 3395278 tid 3 2024-09-18T14:15:37.919 INFO:tasks.rados.rados.0.smithi084.stdout:3393: read oid 5 snap -1 2024-09-18T14:15:37.919 INFO:tasks.rados.rados.0.smithi084.stdout:3393: expect deleted 2024-09-18T14:15:37.919 INFO:tasks.rados.rados.0.smithi084.stdout:3394: write oid 44 current snap is 374 2024-09-18T14:15:37.919 INFO:tasks.rados.rados.0.smithi084.stdout:3391: finishing write tid 1 to smithi08434663-30 2024-09-18T14:15:37.920 INFO:tasks.rados.rados.0.smithi084.stdout:3391: finishing write tid 2 to smithi08434663-30 2024-09-18T14:15:37.920 INFO:tasks.rados.rados.0.smithi084.stdout:3394: seq_num 1155 ranges {612085=481898,1788114=675727,3035086=376999} 2024-09-18T14:15:37.924 INFO:tasks.rados.rados.0.smithi084.stdout:3394: writing smithi08434663-44 from 612085 to 1093983 tid 1 2024-09-18T14:15:37.932 INFO:tasks.rados.rados.0.smithi084.stdout:3394: writing smithi08434663-44 from 1788114 to 2463841 tid 2 2024-09-18T14:15:37.938 INFO:tasks.rados.rados.0.smithi084.stdout:3394: writing smithi08434663-44 from 3035086 to 3412085 tid 3 2024-09-18T14:15:37.938 INFO:tasks.rados.rados.0.smithi084.stdout:3389: done (4 left) 2024-09-18T14:15:37.939 INFO:tasks.rados.rados.0.smithi084.stdout:3395: write oid 19 current snap is 374 2024-09-18T14:15:37.939 INFO:tasks.rados.rados.0.smithi084.stdout:3391: finishing write tid 3 to smithi08434663-30 2024-09-18T14:15:37.939 INFO:tasks.rados.rados.0.smithi084.stdout:3391: finishing write tid 4 to smithi08434663-30 2024-09-18T14:15:37.939 INFO:tasks.rados.rados.0.smithi084.stdout:3391: finishing write tid 5 to smithi08434663-30 2024-09-18T14:15:37.939 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 1270 (ObjNum 1153 snap 374 seq_num 1153) dirty exists 2024-09-18T14:15:37.939 INFO:tasks.rados.rados.0.smithi084.stdout:3391: left oid 30 (ObjNum 1153 snap 374 seq_num 1153) 2024-09-18T14:15:37.939 INFO:tasks.rados.rados.0.smithi084.stdout:3392: finishing write tid 1 to smithi08434663-31 2024-09-18T14:15:37.939 INFO:tasks.rados.rados.0.smithi084.stdout:3392: finishing write tid 2 to smithi08434663-31 2024-09-18T14:15:37.939 INFO:tasks.rados.rados.0.smithi084.stdout:3395: seq_num 1156 ranges {628892=673500,1962949=535738,3178862=250030} 2024-09-18T14:15:37.945 INFO:tasks.rados.rados.0.smithi084.stdout:3395: writing smithi08434663-19 from 628892 to 1302392 tid 1 2024-09-18T14:15:37.951 INFO:tasks.rados.rados.0.smithi084.stdout:3395: writing smithi08434663-19 from 1962949 to 2498687 tid 2 2024-09-18T14:15:37.958 INFO:tasks.rados.rados.0.smithi084.stdout:3395: writing smithi08434663-19 from 3178862 to 3428892 tid 3 2024-09-18T14:15:37.958 INFO:tasks.rados.rados.0.smithi084.stdout:3391: done (4 left) 2024-09-18T14:15:37.958 INFO:tasks.rados.rados.0.smithi084.stdout:3396: read oid 12 snap 367 2024-09-18T14:15:37.958 INFO:tasks.rados.rados.0.smithi084.stdout:3396: expect (ObjNum 1101 snap 350 seq_num 1101) 2024-09-18T14:15:37.958 INFO:tasks.rados.rados.0.smithi084.stdout:3397: copy_from oid 35 from oid 7 current snap is 374 2024-09-18T14:15:37.959 INFO:tasks.rados.rados.0.smithi084.stdout:3392: finishing write tid 3 to smithi08434663-31 2024-09-18T14:15:37.959 INFO:tasks.rados.rados.0.smithi084.stdout:3392: finishing write tid 4 to smithi08434663-31 2024-09-18T14:15:37.959 INFO:tasks.rados.rados.0.smithi084.stdout:3392: finishing write tid 5 to smithi08434663-31 2024-09-18T14:15:37.959 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 1534 (ObjNum 1154 snap 374 seq_num 1154) dirty exists 2024-09-18T14:15:37.959 INFO:tasks.rados.rados.0.smithi084.stdout:3392: left oid 31 (ObjNum 1154 snap 374 seq_num 1154) 2024-09-18T14:15:37.959 INFO:tasks.rados.rados.0.smithi084.stdout:3394: finishing write tid 1 to smithi08434663-44 2024-09-18T14:15:37.959 INFO:tasks.rados.rados.0.smithi084.stdout:3394: finishing write tid 2 to smithi08434663-44 2024-09-18T14:15:37.959 INFO:tasks.rados.rados.0.smithi084.stdout:3394: finishing write tid 3 to smithi08434663-44 2024-09-18T14:15:37.959 INFO:tasks.rados.rados.0.smithi084.stdout:3394: finishing write tid 4 to smithi08434663-44 2024-09-18T14:15:37.960 INFO:tasks.rados.rados.0.smithi084.stdout:3394: finishing write tid 5 to smithi08434663-44 2024-09-18T14:15:37.960 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 1052 (ObjNum 1155 snap 374 seq_num 1155) dirty exists 2024-09-18T14:15:37.960 INFO:tasks.rados.rados.0.smithi084.stdout:3394: left oid 44 (ObjNum 1155 snap 374 seq_num 1155) 2024-09-18T14:15:37.960 INFO:tasks.rados.rados.0.smithi084.stdout:3395: finishing write tid 1 to smithi08434663-19 2024-09-18T14:15:37.960 INFO:tasks.rados.rados.0.smithi084.stdout:3395: finishing write tid 2 to smithi08434663-19 2024-09-18T14:15:37.960 INFO:tasks.rados.rados.0.smithi084.stdout:3392: done (5 left) 2024-09-18T14:15:37.960 INFO:tasks.rados.rados.0.smithi084.stdout:3393: done (4 left) 2024-09-18T14:15:37.960 INFO:tasks.rados.rados.0.smithi084.stdout:3394: done (3 left) 2024-09-18T14:15:37.960 INFO:tasks.rados.rados.0.smithi084.stdout:3398: snap_remove snap 354 2024-09-18T14:15:38.888 INFO:tasks.rados.rados.0.smithi084.stdout:3398: done (3 left) 2024-09-18T14:15:38.888 INFO:tasks.rados.rados.0.smithi084.stdout:3399: copy_from oid 3 from oid 5 current snap is 374 2024-09-18T14:15:38.888 INFO:tasks.rados.rados.0.smithi084.stdout:3395: finishing write tid 3 to smithi08434663-19 2024-09-18T14:15:38.888 INFO:tasks.rados.rados.0.smithi084.stdout:3395: finishing write tid 4 to smithi08434663-19 2024-09-18T14:15:38.888 INFO:tasks.rados.rados.0.smithi084.stdout:3395: finishing write tid 5 to smithi08434663-19 2024-09-18T14:15:38.888 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 1960 (ObjNum 1156 snap 374 seq_num 1156) dirty exists 2024-09-18T14:15:38.888 INFO:tasks.rados.rados.0.smithi084.stdout:3395: left oid 19 (ObjNum 1156 snap 374 seq_num 1156) 2024-09-18T14:15:38.889 INFO:tasks.rados.rados.0.smithi084.stdout:3396: expect (ObjNum 1101 snap 350 seq_num 1101) 2024-09-18T14:15:38.973 INFO:tasks.rados.rados.0.smithi084.stdout:3397: finishing copy_from racing read to smithi08434663-35 2024-09-18T14:15:38.973 INFO:tasks.rados.rados.0.smithi084.stdout:3397: finishing copy_from to smithi08434663-35 2024-09-18T14:15:38.973 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 1330 (ObjNum 1127 snap 361 seq_num 1127) dirty exists 2024-09-18T14:15:38.973 INFO:tasks.rados.rados.0.smithi084.stdout:3395: done (3 left) 2024-09-18T14:15:38.973 INFO:tasks.rados.rados.0.smithi084.stdout:3396: done (2 left) 2024-09-18T14:15:38.973 INFO:tasks.rados.rados.0.smithi084.stdout:3397: done (1 left) 2024-09-18T14:15:38.973 INFO:tasks.rados.rados.0.smithi084.stdout:3400: snap_remove snap 342 2024-09-18T14:15:39.890 INFO:tasks.rados.rados.0.smithi084.stdout:3400: done (1 left) 2024-09-18T14:15:39.890 INFO:tasks.rados.rados.0.smithi084.stdout:3401: write oid 1 current snap is 374 2024-09-18T14:15:39.890 INFO:tasks.rados.rados.0.smithi084.stdout:3401: seq_num 1157 ranges {645699=748749,2021431=679396,3206285=239414} 2024-09-18T14:15:39.898 INFO:tasks.rados.rados.0.smithi084.stdout:3401: writing smithi08434663-1 from 645699 to 1394448 tid 1 2024-09-18T14:15:39.905 INFO:tasks.rados.rados.0.smithi084.stdout:3401: writing smithi08434663-1 from 2021431 to 2700827 tid 2 2024-09-18T14:15:39.910 INFO:tasks.rados.rados.0.smithi084.stdout:3401: writing smithi08434663-1 from 3206285 to 3445699 tid 3 2024-09-18T14:15:39.910 INFO:tasks.rados.rados.0.smithi084.stdout:3402: copy_from oid 42 from oid 31 current snap is 374 2024-09-18T14:15:39.910 INFO:tasks.rados.rados.0.smithi084.stdout:3399: finishing copy_from to smithi08434663-3 2024-09-18T14:15:39.911 INFO:tasks.rados.rados.0.smithi084.stdout:3399: got expected ENOENT (src dne) 2024-09-18T14:15:39.911 INFO:tasks.rados.rados.0.smithi084.stdout:3399: finishing copy_from racing read to smithi08434663-3 2024-09-18T14:15:39.911 INFO:tasks.rados.rados.0.smithi084.stdout:3401: finishing write tid 1 to smithi08434663-1 2024-09-18T14:15:39.911 INFO:tasks.rados.rados.0.smithi084.stdout:3399: done (2 left) 2024-09-18T14:15:39.911 INFO:tasks.rados.rados.0.smithi084.stdout:3403: snap_create 2024-09-18T14:15:39.913 INFO:tasks.rados.rados.0.smithi084.stdout:3401: finishing write tid 2 to smithi08434663-1 2024-09-18T14:15:39.913 INFO:tasks.rados.rados.0.smithi084.stdout:3401: finishing write tid 3 to smithi08434663-1 2024-09-18T14:15:39.913 INFO:tasks.rados.rados.0.smithi084.stdout:3401: finishing write tid 4 to smithi08434663-1 2024-09-18T14:15:39.913 INFO:tasks.rados.rados.0.smithi084.stdout:3401: finishing write tid 5 to smithi08434663-1 2024-09-18T14:15:39.914 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 1501 (ObjNum 1157 snap 374 seq_num 1157) dirty exists 2024-09-18T14:15:39.914 INFO:tasks.rados.rados.0.smithi084.stdout:3401: left oid 1 (ObjNum 1157 snap 374 seq_num 1157) 2024-09-18T14:15:39.921 INFO:tasks.rados.rados.0.smithi084.stdout:3402: finishing copy_from racing read to smithi08434663-42 2024-09-18T14:15:39.923 INFO:tasks.rados.rados.0.smithi084.stdout:3402: finishing copy_from to smithi08434663-42 2024-09-18T14:15:39.923 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1534 (ObjNum 1154 snap 374 seq_num 1154) dirty exists 2024-09-18T14:15:40.868 INFO:tasks.rados.rados.0.smithi084.stdout:3401: done (2 left) 2024-09-18T14:15:40.868 INFO:tasks.rados.rados.0.smithi084.stdout:3402: done (1 left) 2024-09-18T14:15:40.868 INFO:tasks.rados.rados.0.smithi084.stdout:3403: done (0 left) 2024-09-18T14:15:40.868 INFO:tasks.rados.rados.0.smithi084.stdout:3404: read oid 36 snap -1 2024-09-18T14:15:40.868 INFO:tasks.rados.rados.0.smithi084.stdout:3404: expect (ObjNum 1138 snap 364 seq_num 1138) 2024-09-18T14:15:40.868 INFO:tasks.rados.rados.0.smithi084.stdout:3405: read oid 10 snap -1 2024-09-18T14:15:40.869 INFO:tasks.rados.rados.0.smithi084.stdout:3405: expect (ObjNum 1111 snap 355 seq_num 1111) 2024-09-18T14:15:40.869 INFO:tasks.rados.rados.0.smithi084.stdout:3406: copy_from oid 6 from oid 42 current snap is 375 2024-09-18T14:15:40.869 INFO:tasks.rados.rados.0.smithi084.stdout:3407: read oid 14 snap -1 2024-09-18T14:15:40.869 INFO:tasks.rados.rados.0.smithi084.stdout:3407: expect deleted 2024-09-18T14:15:40.869 INFO:tasks.rados.rados.0.smithi084.stdout:3408: snap_remove snap 374 2024-09-18T14:15:41.876 INFO:tasks.rados.rados.0.smithi084.stdout:3408: done (4 left) 2024-09-18T14:15:41.876 INFO:tasks.rados.rados.0.smithi084.stdout:3409: snap_remove snap 343 2024-09-18T14:15:42.885 INFO:tasks.rados.rados.0.smithi084.stdout:3409: done (4 left) 2024-09-18T14:15:42.885 INFO:tasks.rados.rados.0.smithi084.stdout:3410: copy_from oid 47 from oid 7 current snap is 375 2024-09-18T14:15:42.885 INFO:tasks.rados.rados.0.smithi084.stdout:3405: expect (ObjNum 1111 snap 355 seq_num 1111) 2024-09-18T14:15:43.014 INFO:tasks.rados.rados.0.smithi084.stdout:3404: expect (ObjNum 1138 snap 364 seq_num 1138) 2024-09-18T14:15:43.147 INFO:tasks.rados.rados.0.smithi084.stdout:3406: finishing copy_from racing read to smithi08434663-6 2024-09-18T14:15:43.147 INFO:tasks.rados.rados.0.smithi084.stdout:3406: finishing copy_from to smithi08434663-6 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 1534 (ObjNum 1154 snap 374 seq_num 1154) dirty exists 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:3410: finishing copy_from racing read to smithi08434663-47 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:3410: finishing copy_from to smithi08434663-47 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 1330 (ObjNum 1127 snap 361 seq_num 1127) dirty exists 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:3404: done (4 left) 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:3405: done (3 left) 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:3406: done (2 left) 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:3407: done (1 left) 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:3410: done (0 left) 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:3411: rollback oid 8 current snap is 375 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 8 to 345 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:3412: read oid 38 snap -1 2024-09-18T14:15:43.148 INFO:tasks.rados.rados.0.smithi084.stdout:3412: expect (ObjNum 957 snap 304 seq_num 957) 2024-09-18T14:15:43.149 INFO:tasks.rados.rados.0.smithi084.stdout:3413: delete oid 16 current snap is 375 2024-09-18T14:15:43.152 INFO:tasks.rados.rados.0.smithi084.stdout:3413: done (2 left) 2024-09-18T14:15:43.152 INFO:tasks.rados.rados.0.smithi084.stdout:3414: read oid 50 snap 367 2024-09-18T14:15:43.152 INFO:tasks.rados.rados.0.smithi084.stdout:3414: expect deleted 2024-09-18T14:15:43.152 INFO:tasks.rados.rados.0.smithi084.stdout:3415: copy_from oid 22 from oid 46 current snap is 375 2024-09-18T14:15:43.152 INFO:tasks.rados.rados.0.smithi084.stdout:3416: rollback oid 18 current snap is 375 2024-09-18T14:15:43.153 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 18 to 364 2024-09-18T14:15:43.153 INFO:tasks.rados.rados.0.smithi084.stdout:3417: read oid 39 snap -1 2024-09-18T14:15:43.153 INFO:tasks.rados.rados.0.smithi084.stdout:3417: expect (ObjNum 1044 snap 334 seq_num 1044) 2024-09-18T14:15:43.153 INFO:tasks.rados.rados.0.smithi084.stdout:3418: snap_create 2024-09-18T14:15:43.153 INFO:tasks.rados.rados.0.smithi084.stdout:3411: finishing rollback tid 0 to smithi08434663-8 2024-09-18T14:15:43.153 INFO:tasks.rados.rados.0.smithi084.stdout:3411: finishing rollback tid 1 to smithi08434663-8 2024-09-18T14:15:43.153 INFO:tasks.rados.rados.0.smithi084.stdout:3411: finishing rollback tid 2 to smithi08434663-8 2024-09-18T14:15:43.153 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 1412 (ObjNum 1041 snap 333 seq_num 1041) dirty exists 2024-09-18T14:15:43.154 INFO:tasks.rados.rados.0.smithi084.stdout:3412: expect (ObjNum 957 snap 304 seq_num 957) 2024-09-18T14:15:43.281 INFO:tasks.rados.rados.0.smithi084.stdout:3417: expect (ObjNum 1044 snap 334 seq_num 1044) 2024-09-18T14:15:43.485 INFO:tasks.rados.rados.0.smithi084.stdout:3415: finishing copy_from racing read to smithi08434663-22 2024-09-18T14:15:43.485 INFO:tasks.rados.rados.0.smithi084.stdout:3416: finishing rollback tid 0 to smithi08434663-18 2024-09-18T14:15:43.485 INFO:tasks.rados.rados.0.smithi084.stdout:3416: finishing rollback tid 1 to smithi08434663-18 2024-09-18T14:15:43.486 INFO:tasks.rados.rados.0.smithi084.stdout:3416: finishing rollback tid 2 to smithi08434663-18 2024-09-18T14:15:43.486 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 1050 (ObjNum 1131 snap 364 seq_num 1131) dirty exists 2024-09-18T14:15:43.486 INFO:tasks.rados.rados.0.smithi084.stdout:3415: finishing copy_from to smithi08434663-22 2024-09-18T14:15:43.486 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 1497 (ObjNum 1111 snap 355 seq_num 1111) dirty exists 2024-09-18T14:15:43.887 INFO:tasks.rados.rados.0.smithi084.stdout:3411: done (6 left) 2024-09-18T14:15:43.888 INFO:tasks.rados.rados.0.smithi084.stdout:3412: done (5 left) 2024-09-18T14:15:43.888 INFO:tasks.rados.rados.0.smithi084.stdout:3414: done (4 left) 2024-09-18T14:15:43.888 INFO:tasks.rados.rados.0.smithi084.stdout:3415: done (3 left) 2024-09-18T14:15:43.888 INFO:tasks.rados.rados.0.smithi084.stdout:3416: done (2 left) 2024-09-18T14:15:43.888 INFO:tasks.rados.rados.0.smithi084.stdout:3417: done (1 left) 2024-09-18T14:15:43.888 INFO:tasks.rados.rados.0.smithi084.stdout:3418: done (0 left) 2024-09-18T14:15:43.888 INFO:tasks.rados.rados.0.smithi084.stdout:3419: read oid 7 snap -1 2024-09-18T14:15:43.888 INFO:tasks.rados.rados.0.smithi084.stdout:3419: expect (ObjNum 1127 snap 361 seq_num 1127) 2024-09-18T14:15:43.888 INFO:tasks.rados.rados.0.smithi084.stdout:3420: copy_from oid 35 from oid 45 current snap is 376 2024-09-18T14:15:43.888 INFO:tasks.rados.rados.0.smithi084.stdout:3421: snap_remove snap 372 2024-09-18T14:15:44.896 INFO:tasks.rados.rados.0.smithi084.stdout:3421: done (2 left) 2024-09-18T14:15:44.896 INFO:tasks.rados.rados.0.smithi084.stdout:3422: read oid 23 snap -1 2024-09-18T14:15:44.896 INFO:tasks.rados.rados.0.smithi084.stdout:3422: expect (ObjNum 1117 snap 358 seq_num 1117) 2024-09-18T14:15:44.896 INFO:tasks.rados.rados.0.smithi084.stdout:3423: write oid 48 current snap is 376 2024-09-18T14:15:44.896 INFO:tasks.rados.rados.0.smithi084.stdout:3423: seq_num 1159 ranges {679313=499247,1622042=683065,2744778=701557} 2024-09-18T14:15:44.902 INFO:tasks.rados.rados.0.smithi084.stdout:3423: writing smithi08434663-48 from 679313 to 1178560 tid 1 2024-09-18T14:15:44.909 INFO:tasks.rados.rados.0.smithi084.stdout:3423: writing smithi08434663-48 from 1622042 to 2305107 tid 2 2024-09-18T14:15:44.915 INFO:tasks.rados.rados.0.smithi084.stdout:3423: writing smithi08434663-48 from 2744778 to 3446335 tid 3 2024-09-18T14:15:44.915 INFO:tasks.rados.rados.0.smithi084.stdout:3424: copy_from oid 26 from oid 37 current snap is 376 2024-09-18T14:15:44.915 INFO:tasks.rados.rados.0.smithi084.stdout:3419: expect (ObjNum 1127 snap 361 seq_num 1127) 2024-09-18T14:15:45.051 INFO:tasks.rados.rados.0.smithi084.stdout:3420: finishing copy_from racing read to smithi08434663-35 2024-09-18T14:15:45.051 INFO:tasks.rados.rados.0.smithi084.stdout:3420: finishing copy_from to smithi08434663-35 2024-09-18T14:15:45.052 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 1703 (ObjNum 1147 snap 370 seq_num 1147) dirty exists 2024-09-18T14:15:45.052 INFO:tasks.rados.rados.0.smithi084.stdout:3423: finishing write tid 1 to smithi08434663-48 2024-09-18T14:15:45.052 INFO:tasks.rados.rados.0.smithi084.stdout:3419: done (4 left) 2024-09-18T14:15:45.052 INFO:tasks.rados.rados.0.smithi084.stdout:3420: done (3 left) 2024-09-18T14:15:45.052 INFO:tasks.rados.rados.0.smithi084.stdout:3425: snap_create 2024-09-18T14:15:45.052 INFO:tasks.rados.rados.0.smithi084.stdout:3422: expect (ObjNum 1117 snap 358 seq_num 1117) 2024-09-18T14:15:45.280 INFO:tasks.rados.rados.0.smithi084.stdout:3423: finishing write tid 2 to smithi08434663-48 2024-09-18T14:15:45.280 INFO:tasks.rados.rados.0.smithi084.stdout:3423: finishing write tid 3 to smithi08434663-48 2024-09-18T14:15:45.281 INFO:tasks.rados.rados.0.smithi084.stdout:3423: finishing write tid 4 to smithi08434663-48 2024-09-18T14:15:45.281 INFO:tasks.rados.rados.0.smithi084.stdout:3423: finishing write tid 5 to smithi08434663-48 2024-09-18T14:15:45.281 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1552 (ObjNum 1159 snap 376 seq_num 1159) dirty exists 2024-09-18T14:15:45.281 INFO:tasks.rados.rados.0.smithi084.stdout:3423: left oid 48 (ObjNum 1159 snap 376 seq_num 1159) 2024-09-18T14:15:45.281 INFO:tasks.rados.rados.0.smithi084.stdout:3424: finishing copy_from racing read to smithi08434663-26 2024-09-18T14:15:45.281 INFO:tasks.rados.rados.0.smithi084.stdout:3424: finishing copy_from to smithi08434663-26 2024-09-18T14:15:45.281 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 1657 (ObjNum 1151 snap 371 seq_num 1151) dirty exists 2024-09-18T14:15:45.868 INFO:tasks.rados.rados.0.smithi084.stdout:3422: done (3 left) 2024-09-18T14:15:45.868 INFO:tasks.rados.rados.0.smithi084.stdout:3423: done (2 left) 2024-09-18T14:15:45.868 INFO:tasks.rados.rados.0.smithi084.stdout:3424: done (1 left) 2024-09-18T14:15:45.868 INFO:tasks.rados.rados.0.smithi084.stdout:3425: done (0 left) 2024-09-18T14:15:45.868 INFO:tasks.rados.rados.0.smithi084.stdout:3426: copy_from oid 4 from oid 23 current snap is 377 2024-09-18T14:15:45.868 INFO:tasks.rados.rados.0.smithi084.stdout:3427: snap_remove snap 375 2024-09-18T14:15:46.878 INFO:tasks.rados.rados.0.smithi084.stdout:3427: done (1 left) 2024-09-18T14:15:46.878 INFO:tasks.rados.rados.0.smithi084.stdout:3428: read oid 32 snap -1 2024-09-18T14:15:46.878 INFO:tasks.rados.rados.0.smithi084.stdout:3428: expect deleted 2024-09-18T14:15:46.878 INFO:tasks.rados.rados.0.smithi084.stdout:3429: write (excl) oid 30 current snap is 377 2024-09-18T14:15:46.878 INFO:tasks.rados.rados.0.smithi084.stdout:3426: finishing copy_from racing read to smithi08434663-4 2024-09-18T14:15:46.878 INFO:tasks.rados.rados.0.smithi084.stdout:3426: finishing copy_from to smithi08434663-4 2024-09-18T14:15:46.878 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1324 (ObjNum 1117 snap 358 seq_num 1117) dirty exists 2024-09-18T14:15:46.878 INFO:tasks.rados.rados.0.smithi084.stdout:3429: seq_num 1160 ranges {696120=574496,1680524=543076,2888554=512829} 2024-09-18T14:15:46.884 INFO:tasks.rados.rados.0.smithi084.stdout:3429: writing smithi08434663-30 from 696120 to 1270616 tid 1 2024-09-18T14:15:46.889 INFO:tasks.rados.rados.0.smithi084.stdout:3429: writing smithi08434663-30 from 1680524 to 2223600 tid 2 2024-09-18T14:15:46.896 INFO:tasks.rados.rados.0.smithi084.stdout:3429: writing smithi08434663-30 from 2888554 to 3401383 tid 3 2024-09-18T14:15:46.896 INFO:tasks.rados.rados.0.smithi084.stdout:3426: done (2 left) 2024-09-18T14:15:46.897 INFO:tasks.rados.rados.0.smithi084.stdout:3430: rollback oid 24 current snap is 377 2024-09-18T14:15:46.897 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 24 to 345 2024-09-18T14:15:46.897 INFO:tasks.rados.rados.0.smithi084.stdout:3429: finishing write tid 1 to smithi08434663-30 2024-09-18T14:15:46.897 INFO:tasks.rados.rados.0.smithi084.stdout:3429: finishing write tid 2 to smithi08434663-30 2024-09-18T14:15:46.897 INFO:tasks.rados.rados.0.smithi084.stdout:3428: done (2 left) 2024-09-18T14:15:46.897 INFO:tasks.rados.rados.0.smithi084.stdout:3431: snap_create 2024-09-18T14:15:46.899 INFO:tasks.rados.rados.0.smithi084.stdout:3430: finishing rollback tid 1 to smithi08434663-24 2024-09-18T14:15:46.899 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 1524 (ObjNum 129 snap 0 seq_num 2495336352) dirty dne 2024-09-18T14:15:46.900 INFO:tasks.rados.rados.0.smithi084.stdout:3429: finishing write tid 3 to smithi08434663-30 2024-09-18T14:15:46.901 INFO:tasks.rados.rados.0.smithi084.stdout:3429: finishing write tid 4 to smithi08434663-30 2024-09-18T14:15:46.901 INFO:tasks.rados.rados.0.smithi084.stdout:3429: finishing write tid 5 to smithi08434663-30 2024-09-18T14:15:46.901 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 1274 (ObjNum 1160 snap 377 seq_num 1160) dirty exists 2024-09-18T14:15:46.901 INFO:tasks.rados.rados.0.smithi084.stdout:3429: left oid 30 (ObjNum 1160 snap 377 seq_num 1160) 2024-09-18T14:15:47.903 INFO:tasks.rados.rados.0.smithi084.stdout:3429: done (2 left) 2024-09-18T14:15:47.903 INFO:tasks.rados.rados.0.smithi084.stdout:3430: done (1 left) 2024-09-18T14:15:47.903 INFO:tasks.rados.rados.0.smithi084.stdout:3431: done (0 left) 2024-09-18T14:15:47.903 INFO:tasks.rados.rados.0.smithi084.stdout:3432: delete oid 46 current snap is 378 2024-09-18T14:15:47.908 INFO:tasks.rados.rados.0.smithi084.stdout:3432: done (0 left) 2024-09-18T14:15:47.908 INFO:tasks.rados.rados.0.smithi084.stdout:3433: snap_create 2024-09-18T14:15:48.911 INFO:tasks.rados.rados.0.smithi084.stdout:3433: done (0 left) 2024-09-18T14:15:48.911 INFO:tasks.rados.rados.0.smithi084.stdout:3434: write (excl) oid 30 current snap is 379 2024-09-18T14:15:48.912 INFO:tasks.rados.rados.0.smithi084.stdout:3434: seq_num 1162 ranges {729734=724994,2197488=430392,3227047=302687} 2024-09-18T14:15:48.918 INFO:tasks.rados.rados.0.smithi084.stdout:3434: writing smithi08434663-30 from 729734 to 1454728 tid 1 2024-09-18T14:15:48.925 INFO:tasks.rados.rados.0.smithi084.stdout:3434: writing smithi08434663-30 from 2197488 to 2627880 tid 2 2024-09-18T14:15:48.930 INFO:tasks.rados.rados.0.smithi084.stdout:3434: writing smithi08434663-30 from 3227047 to 3529734 tid 3 2024-09-18T14:15:48.931 INFO:tasks.rados.rados.0.smithi084.stdout:3435: write (excl) oid 34 current snap is 379 2024-09-18T14:15:48.931 INFO:tasks.rados.rados.0.smithi084.stdout:3435: seq_num 1163 ranges {746541=400243,1739617=690403,3138117=408424} 2024-09-18T14:15:48.936 INFO:tasks.rados.rados.0.smithi084.stdout:3435: writing smithi08434663-34 from 746541 to 1146784 tid 1 2024-09-18T14:15:48.943 INFO:tasks.rados.rados.0.smithi084.stdout:3435: writing smithi08434663-34 from 1739617 to 2430020 tid 2 2024-09-18T14:15:48.949 INFO:tasks.rados.rados.0.smithi084.stdout:3435: writing smithi08434663-34 from 3138117 to 3546541 tid 3 2024-09-18T14:15:48.949 INFO:tasks.rados.rados.0.smithi084.stdout:3436: snap_remove snap 367 2024-09-18T14:15:49.917 INFO:tasks.rados.rados.0.smithi084.stdout:3436: done (2 left) 2024-09-18T14:15:49.918 INFO:tasks.rados.rados.0.smithi084.stdout:3437: delete oid 5 current snap is 379 2024-09-18T14:15:49.918 INFO:tasks.rados.rados.0.smithi084.stdout:3434: finishing write tid 1 to smithi08434663-30 2024-09-18T14:15:49.918 INFO:tasks.rados.rados.0.smithi084.stdout:3434: finishing write tid 2 to smithi08434663-30 2024-09-18T14:15:49.918 INFO:tasks.rados.rados.0.smithi084.stdout:3434: finishing write tid 3 to smithi08434663-30 2024-09-18T14:15:49.918 INFO:tasks.rados.rados.0.smithi084.stdout:3434: finishing write tid 4 to smithi08434663-30 2024-09-18T14:15:49.918 INFO:tasks.rados.rados.0.smithi084.stdout:3434: finishing write tid 5 to smithi08434663-30 2024-09-18T14:15:49.918 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 1278 (ObjNum 1162 snap 379 seq_num 1162) dirty exists 2024-09-18T14:15:49.918 INFO:tasks.rados.rados.0.smithi084.stdout:3434: left oid 30 (ObjNum 1162 snap 379 seq_num 1162) 2024-09-18T14:15:49.918 INFO:tasks.rados.rados.0.smithi084.stdout:3435: finishing write tid 1 to smithi08434663-34 2024-09-18T14:15:49.918 INFO:tasks.rados.rados.0.smithi084.stdout:3435: finishing write tid 2 to smithi08434663-34 2024-09-18T14:15:49.918 INFO:tasks.rados.rados.0.smithi084.stdout:3435: finishing write tid 3 to smithi08434663-34 2024-09-18T14:15:49.918 INFO:tasks.rados.rados.0.smithi084.stdout:3435: finishing write tid 4 to smithi08434663-34 2024-09-18T14:15:49.919 INFO:tasks.rados.rados.0.smithi084.stdout:3435: finishing write tid 5 to smithi08434663-34 2024-09-18T14:15:49.919 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1739 (ObjNum 1163 snap 379 seq_num 1163) dirty exists 2024-09-18T14:15:49.919 INFO:tasks.rados.rados.0.smithi084.stdout:3435: left oid 34 (ObjNum 1163 snap 379 seq_num 1163) 2024-09-18T14:15:49.929 INFO:tasks.rados.rados.0.smithi084.stdout:3434: done (2 left) 2024-09-18T14:15:49.929 INFO:tasks.rados.rados.0.smithi084.stdout:3435: done (1 left) 2024-09-18T14:15:49.929 INFO:tasks.rados.rados.0.smithi084.stdout:3437: done (0 left) 2024-09-18T14:15:49.929 INFO:tasks.rados.rados.0.smithi084.stdout:3438: read oid 30 snap -1 2024-09-18T14:15:49.929 INFO:tasks.rados.rados.0.smithi084.stdout:3438: expect (ObjNum 1162 snap 379 seq_num 1162) 2024-09-18T14:15:49.929 INFO:tasks.rados.rados.0.smithi084.stdout:3439: delete oid 11 current snap is 379 2024-09-18T14:15:49.932 INFO:tasks.rados.rados.0.smithi084.stdout:3439: done (1 left) 2024-09-18T14:15:49.932 INFO:tasks.rados.rados.0.smithi084.stdout:3440: read oid 41 snap -1 2024-09-18T14:15:49.932 INFO:tasks.rados.rados.0.smithi084.stdout:3440: expect (ObjNum 1138 snap 364 seq_num 1138) 2024-09-18T14:15:49.932 INFO:tasks.rados.rados.0.smithi084.stdout:3441: snap_create 2024-09-18T14:15:49.938 INFO:tasks.rados.rados.0.smithi084.stdout:3438: expect (ObjNum 1162 snap 379 seq_num 1162) 2024-09-18T14:15:50.213 INFO:tasks.rados.rados.0.smithi084.stdout:3440: expect (ObjNum 1138 snap 364 seq_num 1138) 2024-09-18T14:15:50.870 INFO:tasks.rados.rados.0.smithi084.stdout:3438: done (2 left) 2024-09-18T14:15:50.871 INFO:tasks.rados.rados.0.smithi084.stdout:3440: done (1 left) 2024-09-18T14:15:50.871 INFO:tasks.rados.rados.0.smithi084.stdout:3441: done (0 left) 2024-09-18T14:15:50.871 INFO:tasks.rados.rados.0.smithi084.stdout:3442: snap_remove snap 376 2024-09-18T14:15:51.881 INFO:tasks.rados.rados.0.smithi084.stdout:3442: done (0 left) 2024-09-18T14:15:51.881 INFO:tasks.rados.rados.0.smithi084.stdout:3443: read oid 14 snap -1 2024-09-18T14:15:51.881 INFO:tasks.rados.rados.0.smithi084.stdout:3443: expect deleted 2024-09-18T14:15:51.881 INFO:tasks.rados.rados.0.smithi084.stdout:3444: read oid 1 snap 371 2024-09-18T14:15:51.881 INFO:tasks.rados.rados.0.smithi084.stdout:3444: expect deleted 2024-09-18T14:15:51.881 INFO:tasks.rados.rados.0.smithi084.stdout:3445: write (excl) oid 18 current snap is 380 2024-09-18T14:15:51.882 INFO:tasks.rados.rados.0.smithi084.stdout:3445: seq_num 1166 ranges {796962=742343,2031416=437730,2936739=413167} 2024-09-18T14:15:51.889 INFO:tasks.rados.rados.0.smithi084.stdout:3445: writing smithi08434663-18 from 796962 to 1539305 tid 1 2024-09-18T14:15:51.895 INFO:tasks.rados.rados.0.smithi084.stdout:3445: writing smithi08434663-18 from 2031416 to 2469146 tid 2 2024-09-18T14:15:51.900 INFO:tasks.rados.rados.0.smithi084.stdout:3445: writing smithi08434663-18 from 2936739 to 3349906 tid 3 2024-09-18T14:15:51.900 INFO:tasks.rados.rados.0.smithi084.stdout:3446: snap_remove snap 357 2024-09-18T14:15:52.887 INFO:tasks.rados.rados.0.smithi084.stdout:3446: done (3 left) 2024-09-18T14:15:52.887 INFO:tasks.rados.rados.0.smithi084.stdout:3447: rollback oid 11 current snap is 380 2024-09-18T14:15:52.887 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 11 to 368 2024-09-18T14:15:52.887 INFO:tasks.rados.rados.0.smithi084.stdout:3445: finishing write tid 1 to smithi08434663-18 2024-09-18T14:15:52.887 INFO:tasks.rados.rados.0.smithi084.stdout:3445: finishing write tid 2 to smithi08434663-18 2024-09-18T14:15:52.887 INFO:tasks.rados.rados.0.smithi084.stdout:3445: finishing write tid 3 to smithi08434663-18 2024-09-18T14:15:52.888 INFO:tasks.rados.rados.0.smithi084.stdout:3445: finishing write tid 4 to smithi08434663-18 2024-09-18T14:15:52.888 INFO:tasks.rados.rados.0.smithi084.stdout:3445: finishing write tid 5 to smithi08434663-18 2024-09-18T14:15:52.888 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 1501 (ObjNum 1166 snap 380 seq_num 1166) dirty exists 2024-09-18T14:15:52.888 INFO:tasks.rados.rados.0.smithi084.stdout:3445: left oid 18 (ObjNum 1166 snap 380 seq_num 1166) 2024-09-18T14:15:52.888 INFO:tasks.rados.rados.0.smithi084.stdout:3443: done (3 left) 2024-09-18T14:15:52.888 INFO:tasks.rados.rados.0.smithi084.stdout:3444: done (2 left) 2024-09-18T14:15:52.888 INFO:tasks.rados.rados.0.smithi084.stdout:3445: done (1 left) 2024-09-18T14:15:52.888 INFO:tasks.rados.rados.0.smithi084.stdout:3448: write (excl) oid 10 current snap is 380 2024-09-18T14:15:52.888 INFO:tasks.rados.rados.0.smithi084.stdout:3448: seq_num 1167 ranges {413769=417592,1573545=697741,2847809=624439} 2024-09-18T14:15:52.892 INFO:tasks.rados.rados.0.smithi084.stdout:3448: writing smithi08434663-10 from 413769 to 831361 tid 1 2024-09-18T14:15:52.900 INFO:tasks.rados.rados.0.smithi084.stdout:3448: writing smithi08434663-10 from 1573545 to 2271286 tid 2 2024-09-18T14:15:52.908 INFO:tasks.rados.rados.0.smithi084.stdout:3448: writing smithi08434663-10 from 2847809 to 3472248 tid 3 2024-09-18T14:15:52.908 INFO:tasks.rados.rados.0.smithi084.stdout:3449: write oid 15 current snap is 380 2024-09-18T14:15:52.908 INFO:tasks.rados.rados.0.smithi084.stdout:3447: finishing rollback tid 1 to smithi08434663-11 2024-09-18T14:15:52.908 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1291 (ObjNum 33 snap 0 seq_num 5) dirty dne 2024-09-18T14:15:52.908 INFO:tasks.rados.rados.0.smithi084.stdout:3449: seq_num 1168 ranges {430576=492841,1632027=441399,2475232=435711,3365397=265179} 2024-09-18T14:15:52.913 INFO:tasks.rados.rados.0.smithi084.stdout:3449: writing smithi08434663-15 from 430576 to 923417 tid 1 2024-09-18T14:15:52.919 INFO:tasks.rados.rados.0.smithi084.stdout:3449: writing smithi08434663-15 from 1632027 to 2073426 tid 2 2024-09-18T14:15:52.924 INFO:tasks.rados.rados.0.smithi084.stdout:3449: writing smithi08434663-15 from 2475232 to 2910943 tid 3 2024-09-18T14:15:52.928 INFO:tasks.rados.rados.0.smithi084.stdout:3449: writing smithi08434663-15 from 3365397 to 3630576 tid 4 2024-09-18T14:15:52.928 INFO:tasks.rados.rados.0.smithi084.stdout:3447: done (2 left) 2024-09-18T14:15:52.929 INFO:tasks.rados.rados.0.smithi084.stdout:3450: write (excl) oid 46 current snap is 380 2024-09-18T14:15:52.929 INFO:tasks.rados.rados.0.smithi084.stdout:3448: finishing write tid 1 to smithi08434663-10 2024-09-18T14:15:52.929 INFO:tasks.rados.rados.0.smithi084.stdout:3448: finishing write tid 2 to smithi08434663-10 2024-09-18T14:15:52.929 INFO:tasks.rados.rados.0.smithi084.stdout:3450: seq_num 1169 ranges {447383=568090,1690509=701410,2902655=646983} 2024-09-18T14:15:52.933 INFO:tasks.rados.rados.0.smithi084.stdout:3450: writing smithi08434663-46 from 447383 to 1015473 tid 1 2024-09-18T14:15:52.941 INFO:tasks.rados.rados.0.smithi084.stdout:3450: writing smithi08434663-46 from 1690509 to 2391919 tid 2 2024-09-18T14:15:52.948 INFO:tasks.rados.rados.0.smithi084.stdout:3450: writing smithi08434663-46 from 2902655 to 3549638 tid 3 2024-09-18T14:15:52.948 INFO:tasks.rados.rados.0.smithi084.stdout:3451: rollback oid 4 current snap is 380 2024-09-18T14:15:52.948 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 4 to 340 2024-09-18T14:15:52.948 INFO:tasks.rados.rados.0.smithi084.stdout:3448: finishing write tid 3 to smithi08434663-10 2024-09-18T14:15:52.948 INFO:tasks.rados.rados.0.smithi084.stdout:3448: finishing write tid 4 to smithi08434663-10 2024-09-18T14:15:52.948 INFO:tasks.rados.rados.0.smithi084.stdout:3448: finishing write tid 5 to smithi08434663-10 2024-09-18T14:15:52.948 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 1661 (ObjNum 1167 snap 380 seq_num 1167) dirty exists 2024-09-18T14:15:52.949 INFO:tasks.rados.rados.0.smithi084.stdout:3448: left oid 10 (ObjNum 1167 snap 380 seq_num 1167) 2024-09-18T14:15:52.949 INFO:tasks.rados.rados.0.smithi084.stdout:3449: finishing write tid 1 to smithi08434663-15 2024-09-18T14:15:52.949 INFO:tasks.rados.rados.0.smithi084.stdout:3449: finishing write tid 2 to smithi08434663-15 2024-09-18T14:15:52.949 INFO:tasks.rados.rados.0.smithi084.stdout:3449: finishing write tid 3 to smithi08434663-15 2024-09-18T14:15:52.949 INFO:tasks.rados.rados.0.smithi084.stdout:3449: finishing write tid 4 to smithi08434663-15 2024-09-18T14:15:52.949 INFO:tasks.rados.rados.0.smithi084.stdout:3449: finishing write tid 5 to smithi08434663-15 2024-09-18T14:15:52.949 INFO:tasks.rados.rados.0.smithi084.stdout:3449: finishing write tid 6 to smithi08434663-15 2024-09-18T14:15:52.949 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 1832 (ObjNum 1168 snap 380 seq_num 1168) dirty exists 2024-09-18T14:15:52.949 INFO:tasks.rados.rados.0.smithi084.stdout:3449: left oid 15 (ObjNum 1168 snap 380 seq_num 1168) 2024-09-18T14:15:52.950 INFO:tasks.rados.rados.0.smithi084.stdout:3450: finishing write tid 1 to smithi08434663-46 2024-09-18T14:15:52.950 INFO:tasks.rados.rados.0.smithi084.stdout:3450: finishing write tid 2 to smithi08434663-46 2024-09-18T14:15:52.950 INFO:tasks.rados.rados.0.smithi084.stdout:3448: done (3 left) 2024-09-18T14:15:52.954 INFO:tasks.rados.rados.0.smithi084.stdout:3449: done (2 left) 2024-09-18T14:15:52.954 INFO:tasks.rados.rados.0.smithi084.stdout:3452: snap_create 2024-09-18T14:15:52.956 INFO:tasks.rados.rados.0.smithi084.stdout:3450: finishing write tid 3 to smithi08434663-46 2024-09-18T14:15:52.956 INFO:tasks.rados.rados.0.smithi084.stdout:3450: finishing write tid 4 to smithi08434663-46 2024-09-18T14:15:52.957 INFO:tasks.rados.rados.0.smithi084.stdout:3450: finishing write tid 5 to smithi08434663-46 2024-09-18T14:15:52.957 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1836 (ObjNum 1169 snap 380 seq_num 1169) dirty exists 2024-09-18T14:15:52.957 INFO:tasks.rados.rados.0.smithi084.stdout:3450: left oid 46 (ObjNum 1169 snap 380 seq_num 1169) 2024-09-18T14:15:52.957 INFO:tasks.rados.rados.0.smithi084.stdout:3451: finishing rollback tid 0 to smithi08434663-4 2024-09-18T14:15:52.957 INFO:tasks.rados.rados.0.smithi084.stdout:3451: finishing rollback tid 1 to smithi08434663-4 2024-09-18T14:15:52.957 INFO:tasks.rados.rados.0.smithi084.stdout:3451: finishing rollback tid 2 to smithi08434663-4 2024-09-18T14:15:52.957 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1757 (ObjNum 1043 snap 334 seq_num 1043) dirty exists 2024-09-18T14:15:53.887 INFO:tasks.rados.rados.0.smithi084.stdout:3450: done (2 left) 2024-09-18T14:15:53.888 INFO:tasks.rados.rados.0.smithi084.stdout:3451: done (1 left) 2024-09-18T14:15:53.888 INFO:tasks.rados.rados.0.smithi084.stdout:3452: done (0 left) 2024-09-18T14:15:53.888 INFO:tasks.rados.rados.0.smithi084.stdout:3453: snap_create 2024-09-18T14:15:54.903 INFO:tasks.rados.rados.0.smithi084.stdout:3453: done (0 left) 2024-09-18T14:15:54.903 INFO:tasks.rados.rados.0.smithi084.stdout:3454: read oid 49 snap -1 2024-09-18T14:15:54.903 INFO:tasks.rados.rados.0.smithi084.stdout:3454: expect (ObjNum 1020 snap 326 seq_num 1020) 2024-09-18T14:15:54.904 INFO:tasks.rados.rados.0.smithi084.stdout:3455: read oid 48 snap 363 2024-09-18T14:15:54.904 INFO:tasks.rados.rados.0.smithi084.stdout:3455: expect (ObjNum 1088 snap 342 seq_num 1088) 2024-09-18T14:15:54.904 INFO:tasks.rados.rados.0.smithi084.stdout:3456: delete oid 26 current snap is 382 2024-09-18T14:15:54.907 INFO:tasks.rados.rados.0.smithi084.stdout:3456: done (2 left) 2024-09-18T14:15:54.907 INFO:tasks.rados.rados.0.smithi084.stdout:3457: rollback oid 2 current snap is 382 2024-09-18T14:15:54.908 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 2 to 363 2024-09-18T14:15:54.908 INFO:tasks.rados.rados.0.smithi084.stdout:3458: write (excl) oid 20 current snap is 382 2024-09-18T14:15:54.908 INFO:tasks.rados.rados.0.smithi084.stdout:3458: seq_num 1171 ranges {480997=434941,1407473=705079,2557501=785880} 2024-09-18T14:15:54.912 INFO:tasks.rados.rados.0.smithi084.stdout:3458: writing smithi08434663-20 from 480997 to 915938 tid 1 2024-09-18T14:15:54.918 INFO:tasks.rados.rados.0.smithi084.stdout:3458: writing smithi08434663-20 from 1407473 to 2112552 tid 2 2024-09-18T14:15:54.925 INFO:tasks.rados.rados.0.smithi084.stdout:3458: writing smithi08434663-20 from 2557501 to 3343381 tid 3 2024-09-18T14:15:54.925 INFO:tasks.rados.rados.0.smithi084.stdout:3459: copy_from oid 43 from oid 3 current snap is 382 2024-09-18T14:15:54.925 INFO:tasks.rados.rados.0.smithi084.stdout:3457: finishing rollback tid 1 to smithi08434663-2 2024-09-18T14:15:54.926 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 0 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-09-18T14:15:54.926 INFO:tasks.rados.rados.0.smithi084.stdout:3455: expect (ObjNum 1088 snap 342 seq_num 1088) 2024-09-18T14:15:55.084 INFO:tasks.rados.rados.0.smithi084.stdout:3454: expect (ObjNum 1020 snap 326 seq_num 1020) 2024-09-18T14:15:55.285 INFO:tasks.rados.rados.0.smithi084.stdout:3458: finishing write tid 1 to smithi08434663-20 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:3458: finishing write tid 2 to smithi08434663-20 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:3454: done (4 left) 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:3455: done (3 left) 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:3457: done (2 left) 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:3460: delete oid 26 current snap is 382 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:3458: finishing write tid 3 to smithi08434663-20 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:3458: finishing write tid 4 to smithi08434663-20 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:3458: finishing write tid 5 to smithi08434663-20 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 1416 (ObjNum 1171 snap 382 seq_num 1171) dirty exists 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:3458: left oid 20 (ObjNum 1171 snap 382 seq_num 1171) 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:3459: finishing copy_from racing read to smithi08434663-43 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:3459: finishing copy_from to smithi08434663-43 2024-09-18T14:15:55.286 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 1037 (ObjNum 967 snap 308 seq_num 967) dirty exists 2024-09-18T14:15:55.287 INFO:tasks.rados.rados.0.smithi084.stdout:3458: done (2 left) 2024-09-18T14:15:55.287 INFO:tasks.rados.rados.0.smithi084.stdout:3459: done (1 left) 2024-09-18T14:15:55.287 INFO:tasks.rados.rados.0.smithi084.stdout:3460: done (0 left) 2024-09-18T14:15:55.287 INFO:tasks.rados.rados.0.smithi084.stdout:3461: read oid 6 snap -1 2024-09-18T14:15:55.287 INFO:tasks.rados.rados.0.smithi084.stdout:3461: expect (ObjNum 1154 snap 374 seq_num 1154) 2024-09-18T14:15:55.288 INFO:tasks.rados.rados.0.smithi084.stdout:3462: read oid 25 snap -1 2024-09-18T14:15:55.288 INFO:tasks.rados.rados.0.smithi084.stdout:3462: expect (ObjNum 1152 snap 373 seq_num 1152) 2024-09-18T14:15:55.288 INFO:tasks.rados.rados.0.smithi084.stdout:3463: snap_remove snap 365 2024-09-18T14:15:55.877 INFO:tasks.rados.rados.0.smithi084.stdout:3462: expect (ObjNum 1152 snap 373 seq_num 1152) 2024-09-18T14:15:56.116 INFO:tasks.rados.rados.0.smithi084.stdout:3461: expect (ObjNum 1154 snap 374 seq_num 1154) 2024-09-18T14:15:56.296 INFO:tasks.rados.rados.0.smithi084.stdout:3461: done (2 left) 2024-09-18T14:15:56.296 INFO:tasks.rados.rados.0.smithi084.stdout:3462: done (1 left) 2024-09-18T14:15:56.296 INFO:tasks.rados.rados.0.smithi084.stdout:3463: done (0 left) 2024-09-18T14:15:56.296 INFO:tasks.rados.rados.0.smithi084.stdout:3464: rollback oid 50 current snap is 382 2024-09-18T14:15:56.296 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 50 to 368 2024-09-18T14:15:56.296 INFO:tasks.rados.rados.0.smithi084.stdout:3465: read oid 47 snap -1 2024-09-18T14:15:56.297 INFO:tasks.rados.rados.0.smithi084.stdout:3465: expect (ObjNum 1127 snap 361 seq_num 1127) 2024-09-18T14:15:56.297 INFO:tasks.rados.rados.0.smithi084.stdout:3466: rollback oid 7 current snap is 382 2024-09-18T14:15:56.297 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 7 to 345 2024-09-18T14:15:56.297 INFO:tasks.rados.rados.0.smithi084.stdout:3467: copy_from oid 38 from oid 20 current snap is 382 2024-09-18T14:15:56.297 INFO:tasks.rados.rados.0.smithi084.stdout:3468: read oid 3 snap 330 2024-09-18T14:15:56.297 INFO:tasks.rados.rados.0.smithi084.stdout:3468: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:15:56.297 INFO:tasks.rados.rados.0.smithi084.stdout:3469: snap_create 2024-09-18T14:15:56.301 INFO:tasks.rados.rados.0.smithi084.stdout:3465: expect (ObjNum 1127 snap 361 seq_num 1127) 2024-09-18T14:15:56.432 INFO:tasks.rados.rados.0.smithi084.stdout:3464: finishing rollback tid 1 to smithi08434663-50 2024-09-18T14:15:56.432 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 1753 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-18T14:15:56.432 INFO:tasks.rados.rados.0.smithi084.stdout:3468: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:15:56.580 INFO:tasks.rados.rados.0.smithi084.stdout:3466: finishing rollback tid 0 to smithi08434663-7 2024-09-18T14:15:56.581 INFO:tasks.rados.rados.0.smithi084.stdout:3466: finishing rollback tid 1 to smithi08434663-7 2024-09-18T14:15:56.581 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 1536 (ObjNum 49 snap 0 seq_num 2475021424) dirty dne 2024-09-18T14:15:56.581 INFO:tasks.rados.rados.0.smithi084.stdout:3467: finishing copy_from racing read to smithi08434663-38 2024-09-18T14:15:56.581 INFO:tasks.rados.rados.0.smithi084.stdout:3467: finishing copy_from to smithi08434663-38 2024-09-18T14:15:56.581 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 1416 (ObjNum 1171 snap 382 seq_num 1171) dirty exists 2024-09-18T14:15:56.917 INFO:tasks.rados.rados.0.smithi084.stdout:3464: done (5 left) 2024-09-18T14:15:56.917 INFO:tasks.rados.rados.0.smithi084.stdout:3465: done (4 left) 2024-09-18T14:15:56.917 INFO:tasks.rados.rados.0.smithi084.stdout:3466: done (3 left) 2024-09-18T14:15:56.917 INFO:tasks.rados.rados.0.smithi084.stdout:3467: done (2 left) 2024-09-18T14:15:56.917 INFO:tasks.rados.rados.0.smithi084.stdout:3468: done (1 left) 2024-09-18T14:15:56.917 INFO:tasks.rados.rados.0.smithi084.stdout:3469: done (0 left) 2024-09-18T14:15:56.917 INFO:tasks.rados.rados.0.smithi084.stdout:3470: delete oid 16 current snap is 383 2024-09-18T14:15:56.919 INFO:tasks.rados.rados.0.smithi084.stdout:3470: done (0 left) 2024-09-18T14:15:56.919 INFO:tasks.rados.rados.0.smithi084.stdout:3471: snap_remove snap 356 2024-09-18T14:15:57.924 INFO:tasks.rados.rados.0.smithi084.stdout:3471: done (0 left) 2024-09-18T14:15:57.924 INFO:tasks.rados.rados.0.smithi084.stdout:3472: snap_create 2024-09-18T14:15:58.931 INFO:tasks.rados.rados.0.smithi084.stdout:3472: done (0 left) 2024-09-18T14:15:58.931 INFO:tasks.rados.rados.0.smithi084.stdout:3473: snap_remove snap 377 2024-09-18T14:15:59.935 INFO:tasks.rados.rados.0.smithi084.stdout:3473: done (0 left) 2024-09-18T14:15:59.935 INFO:tasks.rados.rados.0.smithi084.stdout:3474: rollback oid 35 current snap is 384 2024-09-18T14:15:59.935 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 35 to 369 2024-09-18T14:15:59.935 INFO:tasks.rados.rados.0.smithi084.stdout:3475: delete oid 6 current snap is 384 2024-09-18T14:15:59.941 INFO:tasks.rados.rados.0.smithi084.stdout:3474: finishing rollback tid 0 to smithi08434663-35 2024-09-18T14:15:59.942 INFO:tasks.rados.rados.0.smithi084.stdout:3475: done (1 left) 2024-09-18T14:15:59.942 INFO:tasks.rados.rados.0.smithi084.stdout:3476: snap_create 2024-09-18T14:15:59.943 INFO:tasks.rados.rados.0.smithi084.stdout:3474: finishing rollback tid 1 to smithi08434663-35 2024-09-18T14:15:59.943 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 1838 (ObjNum 49 snap 0 seq_num 2475025320) dirty dne 2024-09-18T14:16:00.875 INFO:tasks.rados.rados.0.smithi084.stdout:3474: done (1 left) 2024-09-18T14:16:00.875 INFO:tasks.rados.rados.0.smithi084.stdout:3476: done (0 left) 2024-09-18T14:16:00.875 INFO:tasks.rados.rados.0.smithi084.stdout:3477: rollback oid 15 current snap is 385 2024-09-18T14:16:00.875 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 15 to 383 2024-09-18T14:16:00.875 INFO:tasks.rados.rados.0.smithi084.stdout:3478: copy_from oid 44 from oid 12 current snap is 385 2024-09-18T14:16:00.876 INFO:tasks.rados.rados.0.smithi084.stdout:3479: write (excl) oid 27 current snap is 385 2024-09-18T14:16:00.876 INFO:tasks.rados.rados.0.smithi084.stdout:3479: seq_num 1175 ranges {548225=735937,1925048=596064,2950840=547321} 2024-09-18T14:16:00.882 INFO:tasks.rados.rados.0.smithi084.stdout:3479: writing smithi08434663-27 from 548225 to 1284162 tid 1 2024-09-18T14:16:00.890 INFO:tasks.rados.rados.0.smithi084.stdout:3479: writing smithi08434663-27 from 1925048 to 2521112 tid 2 2024-09-18T14:16:00.896 INFO:tasks.rados.rados.0.smithi084.stdout:3479: writing smithi08434663-27 from 2950840 to 3498161 tid 3 2024-09-18T14:16:00.896 INFO:tasks.rados.rados.0.smithi084.stdout:3480: snap_remove snap 330 2024-09-18T14:16:01.885 INFO:tasks.rados.rados.0.smithi084.stdout:3480: done (3 left) 2024-09-18T14:16:01.885 INFO:tasks.rados.rados.0.smithi084.stdout:3481: snap_create 2024-09-18T14:16:01.885 INFO:tasks.rados.rados.0.smithi084.stdout:3477: finishing rollback tid 0 to smithi08434663-15 2024-09-18T14:16:01.885 INFO:tasks.rados.rados.0.smithi084.stdout:3477: finishing rollback tid 1 to smithi08434663-15 2024-09-18T14:16:01.886 INFO:tasks.rados.rados.0.smithi084.stdout:3477: finishing rollback tid 2 to smithi08434663-15 2024-09-18T14:16:01.886 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 1841 (ObjNum 1168 snap 380 seq_num 1168) dirty exists 2024-09-18T14:16:01.886 INFO:tasks.rados.rados.0.smithi084.stdout:3478: finishing copy_from racing read to smithi08434663-44 2024-09-18T14:16:01.886 INFO:tasks.rados.rados.0.smithi084.stdout:3479: finishing write tid 1 to smithi08434663-27 2024-09-18T14:16:01.886 INFO:tasks.rados.rados.0.smithi084.stdout:3478: finishing copy_from to smithi08434663-44 2024-09-18T14:16:01.886 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 1772 (ObjNum 1101 snap 350 seq_num 1101) dirty exists 2024-09-18T14:16:01.886 INFO:tasks.rados.rados.0.smithi084.stdout:3479: finishing write tid 2 to smithi08434663-27 2024-09-18T14:16:01.886 INFO:tasks.rados.rados.0.smithi084.stdout:3479: finishing write tid 3 to smithi08434663-27 2024-09-18T14:16:01.886 INFO:tasks.rados.rados.0.smithi084.stdout:3479: finishing write tid 4 to smithi08434663-27 2024-09-18T14:16:01.886 INFO:tasks.rados.rados.0.smithi084.stdout:3479: finishing write tid 5 to smithi08434663-27 2024-09-18T14:16:01.886 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1666 (ObjNum 1175 snap 385 seq_num 1175) dirty exists 2024-09-18T14:16:01.887 INFO:tasks.rados.rados.0.smithi084.stdout:3479: left oid 27 (ObjNum 1175 snap 385 seq_num 1175) 2024-09-18T14:16:02.887 INFO:tasks.rados.rados.0.smithi084.stdout:3477: done (3 left) 2024-09-18T14:16:02.887 INFO:tasks.rados.rados.0.smithi084.stdout:3478: done (2 left) 2024-09-18T14:16:02.887 INFO:tasks.rados.rados.0.smithi084.stdout:3479: done (1 left) 2024-09-18T14:16:02.887 INFO:tasks.rados.rados.0.smithi084.stdout:3481: done (0 left) 2024-09-18T14:16:02.887 INFO:tasks.rados.rados.0.smithi084.stdout:3482: snap_create 2024-09-18T14:16:03.889 INFO:tasks.rados.rados.0.smithi084.stdout:3482: done (0 left) 2024-09-18T14:16:03.889 INFO:tasks.rados.rados.0.smithi084.stdout:3483: copy_from oid 3 from oid 1 current snap is 387 2024-09-18T14:16:03.889 INFO:tasks.rados.rados.0.smithi084.stdout:3484: read oid 42 snap -1 2024-09-18T14:16:03.889 INFO:tasks.rados.rados.0.smithi084.stdout:3484: expect (ObjNum 1154 snap 374 seq_num 1154) 2024-09-18T14:16:03.889 INFO:tasks.rados.rados.0.smithi084.stdout:3485: read oid 28 snap -1 2024-09-18T14:16:03.889 INFO:tasks.rados.rados.0.smithi084.stdout:3485: expect (ObjNum 1072 snap 341 seq_num 1072) 2024-09-18T14:16:03.889 INFO:tasks.rados.rados.0.smithi084.stdout:3486: read oid 44 snap -1 2024-09-18T14:16:03.889 INFO:tasks.rados.rados.0.smithi084.stdout:3486: expect (ObjNum 1101 snap 350 seq_num 1101) 2024-09-18T14:16:03.889 INFO:tasks.rados.rados.0.smithi084.stdout:3487: write oid 14 current snap is 387 2024-09-18T14:16:03.889 INFO:tasks.rados.rados.0.smithi084.stdout:3487: seq_num 1176 ranges {565032=411186,1583530=456075,2578263=758593} 2024-09-18T14:16:03.894 INFO:tasks.rados.rados.0.smithi084.stdout:3487: writing smithi08434663-14 from 565032 to 976218 tid 1 2024-09-18T14:16:03.901 INFO:tasks.rados.rados.0.smithi084.stdout:3487: writing smithi08434663-14 from 1583530 to 2039605 tid 2 2024-09-18T14:16:03.908 INFO:tasks.rados.rados.0.smithi084.stdout:3487: writing smithi08434663-14 from 2578263 to 3336856 tid 3 2024-09-18T14:16:03.909 INFO:tasks.rados.rados.0.smithi084.stdout:3488: rollback oid 37 current snap is 387 2024-09-18T14:16:03.909 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 37 to 369 2024-09-18T14:16:03.909 INFO:tasks.rados.rados.0.smithi084.stdout:3485: expect (ObjNum 1072 snap 341 seq_num 1072) 2024-09-18T14:16:04.043 INFO:tasks.rados.rados.0.smithi084.stdout:3486: expect (ObjNum 1101 snap 350 seq_num 1101) 2024-09-18T14:16:04.123 INFO:tasks.rados.rados.0.smithi084.stdout:3484: expect (ObjNum 1154 snap 374 seq_num 1154) 2024-09-18T14:16:04.303 INFO:tasks.rados.rados.0.smithi084.stdout:3487: finishing write tid 1 to smithi08434663-14 2024-09-18T14:16:04.303 INFO:tasks.rados.rados.0.smithi084.stdout:3487: finishing write tid 2 to smithi08434663-14 2024-09-18T14:16:04.303 INFO:tasks.rados.rados.0.smithi084.stdout:3483: finishing copy_from racing read to smithi08434663-3 2024-09-18T14:16:04.303 INFO:tasks.rados.rados.0.smithi084.stdout:3483: finishing copy_from to smithi08434663-3 2024-09-18T14:16:04.303 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 1501 (ObjNum 1157 snap 374 seq_num 1157) dirty exists 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:3487: finishing write tid 3 to smithi08434663-14 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:3487: finishing write tid 4 to smithi08434663-14 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:3487: finishing write tid 5 to smithi08434663-14 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 2025 (ObjNum 1176 snap 387 seq_num 1176) dirty exists 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:3487: left oid 14 (ObjNum 1176 snap 387 seq_num 1176) 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:3488: finishing rollback tid 0 to smithi08434663-37 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:3488: finishing rollback tid 1 to smithi08434663-37 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 1659 (ObjNum 129 snap 0 seq_num 2494687040) dirty dne 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:3483: done (5 left) 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:3484: done (4 left) 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:3485: done (3 left) 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:3486: done (2 left) 2024-09-18T14:16:04.304 INFO:tasks.rados.rados.0.smithi084.stdout:3487: done (1 left) 2024-09-18T14:16:04.305 INFO:tasks.rados.rados.0.smithi084.stdout:3488: done (0 left) 2024-09-18T14:16:04.305 INFO:tasks.rados.rados.0.smithi084.stdout:3489: read oid 9 snap -1 2024-09-18T14:16:04.305 INFO:tasks.rados.rados.0.smithi084.stdout:3489: expect deleted 2024-09-18T14:16:04.305 INFO:tasks.rados.rados.0.smithi084.stdout:3490: write oid 8 current snap is 387 2024-09-18T14:16:04.305 INFO:tasks.rados.rados.0.smithi084.stdout:3490: seq_num 1177 ranges {581839=486435,1642012=599733,3005686=569865} 2024-09-18T14:16:04.308 INFO:tasks.rados.rados.0.smithi084.stdout:3490: writing smithi08434663-8 from 581839 to 1068274 tid 1 2024-09-18T14:16:04.314 INFO:tasks.rados.rados.0.smithi084.stdout:3490: writing smithi08434663-8 from 1642012 to 2241745 tid 2 2024-09-18T14:16:04.321 INFO:tasks.rados.rados.0.smithi084.stdout:3490: writing smithi08434663-8 from 3005686 to 3575551 tid 3 2024-09-18T14:16:04.322 INFO:tasks.rados.rados.0.smithi084.stdout:3491: write oid 48 current snap is 387 2024-09-18T14:16:04.322 INFO:tasks.rados.rados.0.smithi084.stdout:3491: seq_num 1178 ranges {598646=561684,1700494=459744,2633109=781137} 2024-09-18T14:16:04.327 INFO:tasks.rados.rados.0.smithi084.stdout:3491: writing smithi08434663-48 from 598646 to 1160330 tid 1 2024-09-18T14:16:04.333 INFO:tasks.rados.rados.0.smithi084.stdout:3491: writing smithi08434663-48 from 1700494 to 2160238 tid 2 2024-09-18T14:16:04.340 INFO:tasks.rados.rados.0.smithi084.stdout:3491: writing smithi08434663-48 from 2633109 to 3414246 tid 3 2024-09-18T14:16:04.340 INFO:tasks.rados.rados.0.smithi084.stdout:3492: snap_create 2024-09-18T14:16:04.341 INFO:tasks.rados.rados.0.smithi084.stdout:3490: finishing write tid 1 to smithi08434663-8 2024-09-18T14:16:04.341 INFO:tasks.rados.rados.0.smithi084.stdout:3490: finishing write tid 2 to smithi08434663-8 2024-09-18T14:16:04.341 INFO:tasks.rados.rados.0.smithi084.stdout:3490: finishing write tid 3 to smithi08434663-8 2024-09-18T14:16:04.341 INFO:tasks.rados.rados.0.smithi084.stdout:3490: finishing write tid 4 to smithi08434663-8 2024-09-18T14:16:04.341 INFO:tasks.rados.rados.0.smithi084.stdout:3490: finishing write tid 5 to smithi08434663-8 2024-09-18T14:16:04.341 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 1420 (ObjNum 1177 snap 387 seq_num 1177) dirty exists 2024-09-18T14:16:04.341 INFO:tasks.rados.rados.0.smithi084.stdout:3490: left oid 8 (ObjNum 1177 snap 387 seq_num 1177) 2024-09-18T14:16:04.341 INFO:tasks.rados.rados.0.smithi084.stdout:3491: finishing write tid 1 to smithi08434663-48 2024-09-18T14:16:04.341 INFO:tasks.rados.rados.0.smithi084.stdout:3491: finishing write tid 2 to smithi08434663-48 2024-09-18T14:16:04.344 INFO:tasks.rados.rados.0.smithi084.stdout:3491: finishing write tid 3 to smithi08434663-48 2024-09-18T14:16:04.344 INFO:tasks.rados.rados.0.smithi084.stdout:3491: finishing write tid 4 to smithi08434663-48 2024-09-18T14:16:04.344 INFO:tasks.rados.rados.0.smithi084.stdout:3491: finishing write tid 5 to smithi08434663-48 2024-09-18T14:16:04.344 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1556 (ObjNum 1178 snap 387 seq_num 1178) dirty exists 2024-09-18T14:16:04.344 INFO:tasks.rados.rados.0.smithi084.stdout:3491: left oid 48 (ObjNum 1178 snap 387 seq_num 1178) 2024-09-18T14:16:04.900 INFO:tasks.rados.rados.0.smithi084.stdout:3489: done (3 left) 2024-09-18T14:16:04.900 INFO:tasks.rados.rados.0.smithi084.stdout:3490: done (2 left) 2024-09-18T14:16:04.900 INFO:tasks.rados.rados.0.smithi084.stdout:3491: done (1 left) 2024-09-18T14:16:04.900 INFO:tasks.rados.rados.0.smithi084.stdout:3492: done (0 left) 2024-09-18T14:16:04.900 INFO:tasks.rados.rados.0.smithi084.stdout:3493: copy_from oid 33 from oid 9 current snap is 388 2024-09-18T14:16:04.901 INFO:tasks.rados.rados.0.smithi084.stdout:3494: read oid 16 snap -1 2024-09-18T14:16:04.901 INFO:tasks.rados.rados.0.smithi084.stdout:3494: expect deleted 2024-09-18T14:16:04.901 INFO:tasks.rados.rados.0.smithi084.stdout:3495: delete oid 47 current snap is 388 2024-09-18T14:16:04.903 INFO:tasks.rados.rados.0.smithi084.stdout:3493: finishing copy_from to smithi08434663-33 2024-09-18T14:16:04.903 INFO:tasks.rados.rados.0.smithi084.stdout:3493: got expected ENOENT (src dne) 2024-09-18T14:16:04.903 INFO:tasks.rados.rados.0.smithi084.stdout:3493: finishing copy_from racing read to smithi08434663-33 2024-09-18T14:16:04.904 INFO:tasks.rados.rados.0.smithi084.stdout:3493: done (2 left) 2024-09-18T14:16:04.904 INFO:tasks.rados.rados.0.smithi084.stdout:3494: done (1 left) 2024-09-18T14:16:04.904 INFO:tasks.rados.rados.0.smithi084.stdout:3495: done (0 left) 2024-09-18T14:16:04.904 INFO:tasks.rados.rados.0.smithi084.stdout:3496: delete oid 11 current snap is 388 2024-09-18T14:16:04.907 INFO:tasks.rados.rados.0.smithi084.stdout:3496: done (0 left) 2024-09-18T14:16:04.907 INFO:tasks.rados.rados.0.smithi084.stdout:3497: rollback oid 3 current snap is 388 2024-09-18T14:16:04.907 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 3 to 380 2024-09-18T14:16:04.907 INFO:tasks.rados.rados.0.smithi084.stdout:3498: read oid 32 snap 378 2024-09-18T14:16:04.907 INFO:tasks.rados.rados.0.smithi084.stdout:3498: expect deleted 2024-09-18T14:16:04.907 INFO:tasks.rados.rados.0.smithi084.stdout:3499: write (excl) oid 28 current snap is 388 2024-09-18T14:16:04.907 INFO:tasks.rados.rados.0.smithi084.stdout:3499: seq_num 1181 ranges {649067=503784,1875940=607071,3115378=731306} 2024-09-18T14:16:04.913 INFO:tasks.rados.rados.0.smithi084.stdout:3499: writing smithi08434663-28 from 649067 to 1152851 tid 1 2024-09-18T14:16:04.920 INFO:tasks.rados.rados.0.smithi084.stdout:3499: writing smithi08434663-28 from 1875940 to 2483011 tid 2 2024-09-18T14:16:04.928 INFO:tasks.rados.rados.0.smithi084.stdout:3499: writing smithi08434663-28 from 3115378 to 3846684 tid 3 2024-09-18T14:16:04.928 INFO:tasks.rados.rados.0.smithi084.stdout:3500: rollback oid 9 current snap is 388 2024-09-18T14:16:04.928 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 9 to 361 2024-09-18T14:16:04.928 INFO:tasks.rados.rados.0.smithi084.stdout:3497: finishing rollback tid 0 to smithi08434663-3 2024-09-18T14:16:04.928 INFO:tasks.rados.rados.0.smithi084.stdout:3497: finishing rollback tid 1 to smithi08434663-3 2024-09-18T14:16:04.929 INFO:tasks.rados.rados.0.smithi084.stdout:3497: finishing rollback tid 2 to smithi08434663-3 2024-09-18T14:16:04.929 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 1504 (ObjNum 967 snap 308 seq_num 967) dirty exists 2024-09-18T14:16:04.929 INFO:tasks.rados.rados.0.smithi084.stdout:3499: finishing write tid 1 to smithi08434663-28 2024-09-18T14:16:04.929 INFO:tasks.rados.rados.0.smithi084.stdout:3499: finishing write tid 2 to smithi08434663-28 2024-09-18T14:16:04.929 INFO:tasks.rados.rados.0.smithi084.stdout:3497: done (3 left) 2024-09-18T14:16:04.929 INFO:tasks.rados.rados.0.smithi084.stdout:3498: done (2 left) 2024-09-18T14:16:04.929 INFO:tasks.rados.rados.0.smithi084.stdout:3501: copy_from oid 45 from oid 24 current snap is 388 2024-09-18T14:16:04.929 INFO:tasks.rados.rados.0.smithi084.stdout:3502: snap_create 2024-09-18T14:16:04.930 INFO:tasks.rados.rados.0.smithi084.stdout:3501: finishing copy_from to smithi08434663-45 2024-09-18T14:16:04.930 INFO:tasks.rados.rados.0.smithi084.stdout:3501: got expected ENOENT (src dne) 2024-09-18T14:16:04.930 INFO:tasks.rados.rados.0.smithi084.stdout:3501: finishing copy_from racing read to smithi08434663-45 2024-09-18T14:16:04.930 INFO:tasks.rados.rados.0.smithi084.stdout:3500: finishing rollback tid 1 to smithi08434663-9 2024-09-18T14:16:04.930 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 1825 (ObjNum 129 snap 0 seq_num 2494643584) dirty dne 2024-09-18T14:16:04.931 INFO:tasks.rados.rados.0.smithi084.stdout:3499: finishing write tid 3 to smithi08434663-28 2024-09-18T14:16:04.931 INFO:tasks.rados.rados.0.smithi084.stdout:3499: finishing write tid 4 to smithi08434663-28 2024-09-18T14:16:04.932 INFO:tasks.rados.rados.0.smithi084.stdout:3499: finishing write tid 5 to smithi08434663-28 2024-09-18T14:16:04.932 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 2030 (ObjNum 1181 snap 388 seq_num 1181) dirty exists 2024-09-18T14:16:04.932 INFO:tasks.rados.rados.0.smithi084.stdout:3499: left oid 28 (ObjNum 1181 snap 388 seq_num 1181) 2024-09-18T14:16:05.877 INFO:tasks.rados.rados.0.smithi084.stdout:3499: done (3 left) 2024-09-18T14:16:05.877 INFO:tasks.rados.rados.0.smithi084.stdout:3500: done (2 left) 2024-09-18T14:16:05.877 INFO:tasks.rados.rados.0.smithi084.stdout:3501: done (1 left) 2024-09-18T14:16:05.877 INFO:tasks.rados.rados.0.smithi084.stdout:3502: done (0 left) 2024-09-18T14:16:05.877 INFO:tasks.rados.rados.0.smithi084.stdout:3503: copy_from oid 18 from oid 24 current snap is 389 2024-09-18T14:16:05.877 INFO:tasks.rados.rados.0.smithi084.stdout:3504: snap_create 2024-09-18T14:16:05.878 INFO:tasks.rados.rados.0.smithi084.stdout:3503: finishing copy_from to smithi08434663-18 2024-09-18T14:16:05.878 INFO:tasks.rados.rados.0.smithi084.stdout:3503: got expected ENOENT (src dne) 2024-09-18T14:16:05.878 INFO:tasks.rados.rados.0.smithi084.stdout:3503: finishing copy_from racing read to smithi08434663-18 2024-09-18T14:16:06.885 INFO:tasks.rados.rados.0.smithi084.stdout:3503: done (1 left) 2024-09-18T14:16:06.885 INFO:tasks.rados.rados.0.smithi084.stdout:3504: done (0 left) 2024-09-18T14:16:06.885 INFO:tasks.rados.rados.0.smithi084.stdout:3505: delete oid 10 current snap is 390 2024-09-18T14:16:06.888 INFO:tasks.rados.rados.0.smithi084.stdout:3505: done (0 left) 2024-09-18T14:16:06.888 INFO:tasks.rados.rados.0.smithi084.stdout:3506: snap_create 2024-09-18T14:16:07.888 INFO:tasks.rados.rados.0.smithi084.stdout:3506: done (0 left) 2024-09-18T14:16:07.888 INFO:tasks.rados.rados.0.smithi084.stdout:3507: copy_from oid 36 from oid 3 current snap is 391 2024-09-18T14:16:07.888 INFO:tasks.rados.rados.0.smithi084.stdout:3508: write (excl) oid 31 current snap is 391 2024-09-18T14:16:07.888 INFO:tasks.rados.rados.0.smithi084.stdout:3508: seq_num 1183 ranges {682681=654282,1876551=610740,3053871=470203} 2024-09-18T14:16:07.895 INFO:tasks.rados.rados.0.smithi084.stdout:3508: writing smithi08434663-31 from 682681 to 1336963 tid 1 2024-09-18T14:16:07.901 INFO:tasks.rados.rados.0.smithi084.stdout:3508: writing smithi08434663-31 from 1876551 to 2487291 tid 2 2024-09-18T14:16:07.908 INFO:tasks.rados.rados.0.smithi084.stdout:3508: writing smithi08434663-31 from 3053871 to 3524074 tid 3 2024-09-18T14:16:07.908 INFO:tasks.rados.rados.0.smithi084.stdout:3509: write oid 28 current snap is 391 2024-09-18T14:16:07.908 INFO:tasks.rados.rados.0.smithi084.stdout:3507: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:16:07.908 INFO:tasks.rados.rados.0.smithi084.stdout:3507: finishing copy_from to smithi08434663-36 2024-09-18T14:16:07.908 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 1504 (ObjNum 967 snap 308 seq_num 967) dirty exists 2024-09-18T14:16:07.908 INFO:tasks.rados.rados.0.smithi084.stdout:3508: finishing write tid 1 to smithi08434663-31 2024-09-18T14:16:07.908 INFO:tasks.rados.rados.0.smithi084.stdout:3509: seq_num 1184 ranges {699488=729531,1935033=470751,3081294=681475} 2024-09-18T14:16:07.915 INFO:tasks.rados.rados.0.smithi084.stdout:3509: writing smithi08434663-28 from 699488 to 1429019 tid 1 2024-09-18T14:16:07.920 INFO:tasks.rados.rados.0.smithi084.stdout:3509: writing smithi08434663-28 from 1935033 to 2405784 tid 2 2024-09-18T14:16:07.928 INFO:tasks.rados.rados.0.smithi084.stdout:3509: writing smithi08434663-28 from 3081294 to 3762769 tid 3 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:3507: done (2 left) 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:3510: rollback oid 24 current snap is 391 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 24 to 347 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:3508: finishing write tid 2 to smithi08434663-31 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:3508: finishing write tid 3 to smithi08434663-31 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:3508: finishing write tid 4 to smithi08434663-31 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:3508: finishing write tid 5 to smithi08434663-31 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 1538 (ObjNum 1183 snap 391 seq_num 1183) dirty exists 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:3508: left oid 31 (ObjNum 1183 snap 391 seq_num 1183) 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:3509: finishing write tid 1 to smithi08434663-28 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:3509: finishing write tid 2 to smithi08434663-28 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:3508: done (2 left) 2024-09-18T14:16:07.929 INFO:tasks.rados.rados.0.smithi084.stdout:3511: read oid 32 snap -1 2024-09-18T14:16:07.930 INFO:tasks.rados.rados.0.smithi084.stdout:3511: expect deleted 2024-09-18T14:16:07.930 INFO:tasks.rados.rados.0.smithi084.stdout:3512: write oid 46 current snap is 391 2024-09-18T14:16:07.930 INFO:tasks.rados.rados.0.smithi084.stdout:3512: seq_num 1185 ranges {716295=404780,1593515=614409,2708717=492747,3640519=275776} 2024-09-18T14:16:07.934 INFO:tasks.rados.rados.0.smithi084.stdout:3512: writing smithi08434663-46 from 716295 to 1121075 tid 1 2024-09-18T14:16:07.940 INFO:tasks.rados.rados.0.smithi084.stdout:3512: writing smithi08434663-46 from 1593515 to 2207924 tid 2 2024-09-18T14:16:07.945 INFO:tasks.rados.rados.0.smithi084.stdout:3512: writing smithi08434663-46 from 2708717 to 3201464 tid 3 2024-09-18T14:16:07.950 INFO:tasks.rados.rados.0.smithi084.stdout:3512: writing smithi08434663-46 from 3640519 to 3916295 tid 4 2024-09-18T14:16:07.950 INFO:tasks.rados.rados.0.smithi084.stdout:3513: write oid 25 current snap is 391 2024-09-18T14:16:07.950 INFO:tasks.rados.rados.0.smithi084.stdout:3510: finishing rollback tid 1 to smithi08434663-24 2024-09-18T14:16:07.950 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 1524 (ObjNum 33 snap 0 seq_num 2475026512) dirty dne 2024-09-18T14:16:07.950 INFO:tasks.rados.rados.0.smithi084.stdout:3513: seq_num 1186 ranges {733102=596382,1768350=758067,3136140=704019} 2024-09-18T14:16:07.957 INFO:tasks.rados.rados.0.smithi084.stdout:3513: writing smithi08434663-25 from 733102 to 1329484 tid 1 2024-09-18T14:16:07.963 INFO:tasks.rados.rados.0.smithi084.stdout:3513: writing smithi08434663-25 from 1768350 to 2526417 tid 2 2024-09-18T14:16:07.971 INFO:tasks.rados.rados.0.smithi084.stdout:3513: writing smithi08434663-25 from 3136140 to 3840159 tid 3 2024-09-18T14:16:07.971 INFO:tasks.rados.rados.0.smithi084.stdout:3510: done (4 left) 2024-09-18T14:16:07.971 INFO:tasks.rados.rados.0.smithi084.stdout:3514: read oid 30 snap -1 2024-09-18T14:16:07.971 INFO:tasks.rados.rados.0.smithi084.stdout:3514: expect (ObjNum 1162 snap 379 seq_num 1162) 2024-09-18T14:16:07.971 INFO:tasks.rados.rados.0.smithi084.stdout:3515: snap_remove snap 382 2024-09-18T14:16:08.894 INFO:tasks.rados.rados.0.smithi084.stdout:3509: finishing write tid 3 to smithi08434663-28 2024-09-18T14:16:08.895 INFO:tasks.rados.rados.0.smithi084.stdout:3509: finishing write tid 4 to smithi08434663-28 2024-09-18T14:16:08.895 INFO:tasks.rados.rados.0.smithi084.stdout:3509: finishing write tid 5 to smithi08434663-28 2024-09-18T14:16:08.895 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 2035 (ObjNum 1184 snap 391 seq_num 1184) dirty exists 2024-09-18T14:16:08.895 INFO:tasks.rados.rados.0.smithi084.stdout:3509: left oid 28 (ObjNum 1184 snap 391 seq_num 1184) 2024-09-18T14:16:08.895 INFO:tasks.rados.rados.0.smithi084.stdout:3512: finishing write tid 1 to smithi08434663-46 2024-09-18T14:16:08.895 INFO:tasks.rados.rados.0.smithi084.stdout:3512: finishing write tid 2 to smithi08434663-46 2024-09-18T14:16:08.895 INFO:tasks.rados.rados.0.smithi084.stdout:3512: finishing write tid 3 to smithi08434663-46 2024-09-18T14:16:08.895 INFO:tasks.rados.rados.0.smithi084.stdout:3512: finishing write tid 4 to smithi08434663-46 2024-09-18T14:16:08.895 INFO:tasks.rados.rados.0.smithi084.stdout:3512: finishing write tid 5 to smithi08434663-46 2024-09-18T14:16:08.895 INFO:tasks.rados.rados.0.smithi084.stdout:3512: finishing write tid 6 to smithi08434663-46 2024-09-18T14:16:08.895 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1846 (ObjNum 1185 snap 391 seq_num 1185) dirty exists 2024-09-18T14:16:08.896 INFO:tasks.rados.rados.0.smithi084.stdout:3512: left oid 46 (ObjNum 1185 snap 391 seq_num 1185) 2024-09-18T14:16:08.896 INFO:tasks.rados.rados.0.smithi084.stdout:3513: finishing write tid 1 to smithi08434663-25 2024-09-18T14:16:08.896 INFO:tasks.rados.rados.0.smithi084.stdout:3513: finishing write tid 2 to smithi08434663-25 2024-09-18T14:16:08.896 INFO:tasks.rados.rados.0.smithi084.stdout:3513: finishing write tid 3 to smithi08434663-25 2024-09-18T14:16:08.896 INFO:tasks.rados.rados.0.smithi084.stdout:3513: finishing write tid 4 to smithi08434663-25 2024-09-18T14:16:08.896 INFO:tasks.rados.rados.0.smithi084.stdout:3513: finishing write tid 5 to smithi08434663-25 2024-09-18T14:16:08.896 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 1560 (ObjNum 1186 snap 391 seq_num 1186) dirty exists 2024-09-18T14:16:08.896 INFO:tasks.rados.rados.0.smithi084.stdout:3513: left oid 25 (ObjNum 1186 snap 391 seq_num 1186) 2024-09-18T14:16:08.896 INFO:tasks.rados.rados.0.smithi084.stdout:3514: expect (ObjNum 1162 snap 379 seq_num 1162) 2024-09-18T14:16:09.170 INFO:tasks.rados.rados.0.smithi084.stdout:3509: done (5 left) 2024-09-18T14:16:09.171 INFO:tasks.rados.rados.0.smithi084.stdout:3511: done (4 left) 2024-09-18T14:16:09.171 INFO:tasks.rados.rados.0.smithi084.stdout:3512: done (3 left) 2024-09-18T14:16:09.171 INFO:tasks.rados.rados.0.smithi084.stdout:3513: done (2 left) 2024-09-18T14:16:09.171 INFO:tasks.rados.rados.0.smithi084.stdout:3514: done (1 left) 2024-09-18T14:16:09.171 INFO:tasks.rados.rados.0.smithi084.stdout:3515: done (0 left) 2024-09-18T14:16:09.171 INFO:tasks.rados.rados.0.smithi084.stdout:3516: read oid 8 snap -1 2024-09-18T14:16:09.171 INFO:tasks.rados.rados.0.smithi084.stdout:3516: expect (ObjNum 1177 snap 387 seq_num 1177) 2024-09-18T14:16:09.171 INFO:tasks.rados.rados.0.smithi084.stdout:3517: delete oid 50 current snap is 391 2024-09-18T14:16:09.174 INFO:tasks.rados.rados.0.smithi084.stdout:3517: done (1 left) 2024-09-18T14:16:09.174 INFO:tasks.rados.rados.0.smithi084.stdout:3518: read oid 46 snap 364 2024-09-18T14:16:09.174 INFO:tasks.rados.rados.0.smithi084.stdout:3518: expect (ObjNum 1111 snap 355 seq_num 1111) 2024-09-18T14:16:09.174 INFO:tasks.rados.rados.0.smithi084.stdout:3519: snap_remove snap 358 2024-09-18T14:16:09.898 INFO:tasks.rados.rados.0.smithi084.stdout:3518: expect (ObjNum 1111 snap 355 seq_num 1111) 2024-09-18T14:16:10.026 INFO:tasks.rados.rados.0.smithi084.stdout:3516: expect (ObjNum 1177 snap 387 seq_num 1177) 2024-09-18T14:16:10.235 INFO:tasks.rados.rados.0.smithi084.stdout:3516: done (2 left) 2024-09-18T14:16:10.235 INFO:tasks.rados.rados.0.smithi084.stdout:3518: done (1 left) 2024-09-18T14:16:10.235 INFO:tasks.rados.rados.0.smithi084.stdout:3519: done (0 left) 2024-09-18T14:16:10.235 INFO:tasks.rados.rados.0.smithi084.stdout:3520: rollback oid 29 current snap is 391 2024-09-18T14:16:10.235 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 29 to 371 2024-09-18T14:16:10.236 INFO:tasks.rados.rados.0.smithi084.stdout:3521: read oid 1 snap 385 2024-09-18T14:16:10.236 INFO:tasks.rados.rados.0.smithi084.stdout:3521: expect (ObjNum 1157 snap 374 seq_num 1157) 2024-09-18T14:16:10.236 INFO:tasks.rados.rados.0.smithi084.stdout:3522: write oid 3 current snap is 391 2024-09-18T14:16:10.236 INFO:tasks.rados.rados.0.smithi084.stdout:3522: seq_num 1188 ranges {766716=746880,2168961=761736,3474633=442916} 2024-09-18T14:16:10.243 INFO:tasks.rados.rados.0.smithi084.stdout:3522: writing smithi08434663-3 from 766716 to 1513596 tid 1 2024-09-18T14:16:10.251 INFO:tasks.rados.rados.0.smithi084.stdout:3522: writing smithi08434663-3 from 2168961 to 2930697 tid 2 2024-09-18T14:16:10.257 INFO:tasks.rados.rados.0.smithi084.stdout:3522: writing smithi08434663-3 from 3474633 to 3917549 tid 3 2024-09-18T14:16:10.257 INFO:tasks.rados.rados.0.smithi084.stdout:3523: snap_remove snap 364 2024-09-18T14:16:10.876 INFO:tasks.rados.rados.0.smithi084.stdout:3523: done (3 left) 2024-09-18T14:16:10.876 INFO:tasks.rados.rados.0.smithi084.stdout:3524: rollback oid 13 current snap is 391 2024-09-18T14:16:10.876 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 13 to 379 2024-09-18T14:16:10.877 INFO:tasks.rados.rados.0.smithi084.stdout:3521: expect (ObjNum 1157 snap 374 seq_num 1157) 2024-09-18T14:16:10.994 INFO:tasks.rados.rados.0.smithi084.stdout:3520: finishing rollback tid 0 to smithi08434663-29 2024-09-18T14:16:10.994 INFO:tasks.rados.rados.0.smithi084.stdout:3520: finishing rollback tid 1 to smithi08434663-29 2024-09-18T14:16:10.994 INFO:tasks.rados.rados.0.smithi084.stdout:3520: finishing rollback tid 2 to smithi08434663-29 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 1567 (ObjNum 1148 snap 371 seq_num 1148) dirty exists 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:3522: finishing write tid 1 to smithi08434663-3 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:3522: finishing write tid 2 to smithi08434663-3 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:3522: finishing write tid 3 to smithi08434663-3 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:3522: finishing write tid 4 to smithi08434663-3 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:3520: done (3 left) 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:3521: done (2 left) 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:3525: rollback oid 11 current snap is 391 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 11 to 388 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:3522: finishing write tid 5 to smithi08434663-3 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 1508 (ObjNum 1188 snap 391 seq_num 1188) dirty exists 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:3522: left oid 3 (ObjNum 1188 snap 391 seq_num 1188) 2024-09-18T14:16:10.995 INFO:tasks.rados.rados.0.smithi084.stdout:3524: finishing rollback tid 0 to smithi08434663-13 2024-09-18T14:16:10.996 INFO:tasks.rados.rados.0.smithi084.stdout:3524: finishing rollback tid 1 to smithi08434663-13 2024-09-18T14:16:10.996 INFO:tasks.rados.rados.0.smithi084.stdout:3524: finishing rollback tid 2 to smithi08434663-13 2024-09-18T14:16:10.996 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 1570 (ObjNum 1142 snap 368 seq_num 1142) dirty exists 2024-09-18T14:16:10.996 INFO:tasks.rados.rados.0.smithi084.stdout:3522: done (2 left) 2024-09-18T14:16:10.996 INFO:tasks.rados.rados.0.smithi084.stdout:3524: done (1 left) 2024-09-18T14:16:10.996 INFO:tasks.rados.rados.0.smithi084.stdout:3526: snap_create 2024-09-18T14:16:10.996 INFO:tasks.rados.rados.0.smithi084.stdout:3525: finishing rollback tid 1 to smithi08434663-11 2024-09-18T14:16:10.996 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1291 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-18T14:16:11.925 INFO:tasks.rados.rados.0.smithi084.stdout:3525: done (1 left) 2024-09-18T14:16:11.925 INFO:tasks.rados.rados.0.smithi084.stdout:3526: done (0 left) 2024-09-18T14:16:11.925 INFO:tasks.rados.rados.0.smithi084.stdout:3527: read oid 2 snap -1 2024-09-18T14:16:11.925 INFO:tasks.rados.rados.0.smithi084.stdout:3527: expect deleted 2024-09-18T14:16:11.925 INFO:tasks.rados.rados.0.smithi084.stdout:3528: read oid 22 snap -1 2024-09-18T14:16:11.925 INFO:tasks.rados.rados.0.smithi084.stdout:3528: expect (ObjNum 1111 snap 355 seq_num 1111) 2024-09-18T14:16:11.925 INFO:tasks.rados.rados.0.smithi084.stdout:3529: write oid 46 current snap is 392 2024-09-18T14:16:11.925 INFO:tasks.rados.rados.0.smithi084.stdout:3529: seq_num 1189 ranges {783523=422129,1827443=621747,3218409=654188} 2024-09-18T14:16:11.931 INFO:tasks.rados.rados.0.smithi084.stdout:3529: writing smithi08434663-46 from 783523 to 1205652 tid 1 2024-09-18T14:16:11.938 INFO:tasks.rados.rados.0.smithi084.stdout:3529: writing smithi08434663-46 from 1827443 to 2449190 tid 2 2024-09-18T14:16:11.946 INFO:tasks.rados.rados.0.smithi084.stdout:3529: writing smithi08434663-46 from 3218409 to 3872597 tid 3 2024-09-18T14:16:11.946 INFO:tasks.rados.rados.0.smithi084.stdout:3530: snap_remove snap 389 2024-09-18T14:16:12.932 INFO:tasks.rados.rados.0.smithi084.stdout:3530: done (3 left) 2024-09-18T14:16:12.932 INFO:tasks.rados.rados.0.smithi084.stdout:3531: snap_create 2024-09-18T14:16:12.932 INFO:tasks.rados.rados.0.smithi084.stdout:3528: expect (ObjNum 1111 snap 355 seq_num 1111) 2024-09-18T14:16:13.059 INFO:tasks.rados.rados.0.smithi084.stdout:3529: finishing write tid 1 to smithi08434663-46 2024-09-18T14:16:13.059 INFO:tasks.rados.rados.0.smithi084.stdout:3529: finishing write tid 2 to smithi08434663-46 2024-09-18T14:16:13.059 INFO:tasks.rados.rados.0.smithi084.stdout:3529: finishing write tid 3 to smithi08434663-46 2024-09-18T14:16:13.059 INFO:tasks.rados.rados.0.smithi084.stdout:3529: finishing write tid 4 to smithi08434663-46 2024-09-18T14:16:13.059 INFO:tasks.rados.rados.0.smithi084.stdout:3529: finishing write tid 5 to smithi08434663-46 2024-09-18T14:16:13.059 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1850 (ObjNum 1189 snap 392 seq_num 1189) dirty exists 2024-09-18T14:16:13.059 INFO:tasks.rados.rados.0.smithi084.stdout:3529: left oid 46 (ObjNum 1189 snap 392 seq_num 1189) 2024-09-18T14:16:13.931 INFO:tasks.rados.rados.0.smithi084.stdout:3527: done (3 left) 2024-09-18T14:16:13.932 INFO:tasks.rados.rados.0.smithi084.stdout:3528: done (2 left) 2024-09-18T14:16:13.932 INFO:tasks.rados.rados.0.smithi084.stdout:3529: done (1 left) 2024-09-18T14:16:13.932 INFO:tasks.rados.rados.0.smithi084.stdout:3531: done (0 left) 2024-09-18T14:16:13.932 INFO:tasks.rados.rados.0.smithi084.stdout:3532: read oid 30 snap 383 2024-09-18T14:16:13.932 INFO:tasks.rados.rados.0.smithi084.stdout:3532: expect (ObjNum 1162 snap 379 seq_num 1162) 2024-09-18T14:16:13.932 INFO:tasks.rados.rados.0.smithi084.stdout:3533: copy_from oid 15 from oid 14 current snap is 393 2024-09-18T14:16:13.932 INFO:tasks.rados.rados.0.smithi084.stdout:3534: delete oid 39 current snap is 393 2024-09-18T14:16:13.937 INFO:tasks.rados.rados.0.smithi084.stdout:3534: done (2 left) 2024-09-18T14:16:13.937 INFO:tasks.rados.rados.0.smithi084.stdout:3535: copy_from oid 10 from oid 44 current snap is 393 2024-09-18T14:16:13.937 INFO:tasks.rados.rados.0.smithi084.stdout:3536: delete oid 25 current snap is 393 2024-09-18T14:16:13.939 INFO:tasks.rados.rados.0.smithi084.stdout:3532: expect (ObjNum 1162 snap 379 seq_num 1162) 2024-09-18T14:16:14.215 INFO:tasks.rados.rados.0.smithi084.stdout:3535: finishing copy_from racing read to smithi08434663-10 2024-09-18T14:16:14.216 INFO:tasks.rados.rados.0.smithi084.stdout:3535: finishing copy_from to smithi08434663-10 2024-09-18T14:16:14.216 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 1772 (ObjNum 1101 snap 350 seq_num 1101) dirty exists 2024-09-18T14:16:14.216 INFO:tasks.rados.rados.0.smithi084.stdout:3533: finishing copy_from racing read to smithi08434663-15 2024-09-18T14:16:14.216 INFO:tasks.rados.rados.0.smithi084.stdout:3533: finishing copy_from to smithi08434663-15 2024-09-18T14:16:14.216 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 2025 (ObjNum 1176 snap 387 seq_num 1176) dirty exists 2024-09-18T14:16:14.216 INFO:tasks.rados.rados.0.smithi084.stdout:3532: done (3 left) 2024-09-18T14:16:14.216 INFO:tasks.rados.rados.0.smithi084.stdout:3533: done (2 left) 2024-09-18T14:16:14.216 INFO:tasks.rados.rados.0.smithi084.stdout:3535: done (1 left) 2024-09-18T14:16:14.216 INFO:tasks.rados.rados.0.smithi084.stdout:3536: done (0 left) 2024-09-18T14:16:14.216 INFO:tasks.rados.rados.0.smithi084.stdout:3537: read oid 21 snap -1 2024-09-18T14:16:14.217 INFO:tasks.rados.rados.0.smithi084.stdout:3537: expect deleted 2024-09-18T14:16:14.217 INFO:tasks.rados.rados.0.smithi084.stdout:3538: delete oid 27 current snap is 393 2024-09-18T14:16:14.219 INFO:tasks.rados.rados.0.smithi084.stdout:3537: done (1 left) 2024-09-18T14:16:14.219 INFO:tasks.rados.rados.0.smithi084.stdout:3538: done (0 left) 2024-09-18T14:16:14.219 INFO:tasks.rados.rados.0.smithi084.stdout:3539: rollback oid 45 current snap is 393 2024-09-18T14:16:14.219 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 45 to 345 2024-09-18T14:16:14.219 INFO:tasks.rados.rados.0.smithi084.stdout:3540: write oid 23 current snap is 393 2024-09-18T14:16:14.219 INFO:tasks.rados.rados.0.smithi084.stdout:3540: seq_num 1193 ranges {450751=723125,1945018=105733} 2024-09-18T14:16:14.225 INFO:tasks.rados.rados.0.smithi084.stdout:3540: writing smithi08434663-23 from 450751 to 1173876 tid 1 2024-09-18T14:16:14.230 INFO:tasks.rados.rados.0.smithi084.stdout:3540: writing smithi08434663-23 from 1945018 to 2050751 tid 2 2024-09-18T14:16:14.230 INFO:tasks.rados.rados.0.smithi084.stdout:3541: read oid 43 snap -1 2024-09-18T14:16:14.230 INFO:tasks.rados.rados.0.smithi084.stdout:3541: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:16:14.231 INFO:tasks.rados.rados.0.smithi084.stdout:3542: rollback oid 10 current snap is 393 2024-09-18T14:16:14.231 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 10 to 385 2024-09-18T14:16:14.231 INFO:tasks.rados.rados.0.smithi084.stdout:3539: finishing rollback tid 0 to smithi08434663-45 2024-09-18T14:16:14.231 INFO:tasks.rados.rados.0.smithi084.stdout:3539: finishing rollback tid 1 to smithi08434663-45 2024-09-18T14:16:14.231 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 1790 (ObjNum 0 snap 0 seq_num 304) dirty dne 2024-09-18T14:16:14.231 INFO:tasks.rados.rados.0.smithi084.stdout:3539: done (3 left) 2024-09-18T14:16:14.231 INFO:tasks.rados.rados.0.smithi084.stdout:3543: rollback oid 13 current snap is 393 2024-09-18T14:16:14.231 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 13 to 368 2024-09-18T14:16:14.231 INFO:tasks.rados.rados.0.smithi084.stdout:3544: snap_remove snap 347 2024-09-18T14:16:14.944 INFO:tasks.rados.rados.0.smithi084.stdout:3544: done (4 left) 2024-09-18T14:16:14.944 INFO:tasks.rados.rados.0.smithi084.stdout:3545: rollback oid 17 current snap is 393 2024-09-18T14:16:14.944 INFO:tasks.rados.rados.0.smithi084.stdout:3542: finishing rollback tid 0 to smithi08434663-10 2024-09-18T14:16:14.944 INFO:tasks.rados.rados.0.smithi084.stdout:3542: finishing rollback tid 1 to smithi08434663-10 2024-09-18T14:16:14.944 INFO:tasks.rados.rados.0.smithi084.stdout:3542: finishing rollback tid 2 to smithi08434663-10 2024-09-18T14:16:14.945 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 1776 (ObjNum 1167 snap 380 seq_num 1167) dirty exists 2024-09-18T14:16:14.945 INFO:tasks.rados.rados.0.smithi084.stdout:3540: finishing write tid 1 to smithi08434663-23 2024-09-18T14:16:14.945 INFO:tasks.rados.rados.0.smithi084.stdout:3540: finishing write tid 2 to smithi08434663-23 2024-09-18T14:16:14.945 INFO:tasks.rados.rados.0.smithi084.stdout:3541: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:16:15.099 INFO:tasks.rados.rados.0.smithi084.stdout:3540: finishing write tid 3 to smithi08434663-23 2024-09-18T14:16:15.099 INFO:tasks.rados.rados.0.smithi084.stdout:3540: finishing write tid 4 to smithi08434663-23 2024-09-18T14:16:15.099 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1540 (ObjNum 1193 snap 393 seq_num 1193) dirty exists 2024-09-18T14:16:15.099 INFO:tasks.rados.rados.0.smithi084.stdout:3540: left oid 23 (ObjNum 1193 snap 393 seq_num 1193) 2024-09-18T14:16:15.099 INFO:tasks.rados.rados.0.smithi084.stdout:3543: finishing rollback tid 0 to smithi08434663-13 2024-09-18T14:16:15.099 INFO:tasks.rados.rados.0.smithi084.stdout:3543: finishing rollback tid 1 to smithi08434663-13 2024-09-18T14:16:15.099 INFO:tasks.rados.rados.0.smithi084.stdout:3543: finishing rollback tid 2 to smithi08434663-13 2024-09-18T14:16:15.100 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 1573 (ObjNum 1142 snap 368 seq_num 1142) dirty exists 2024-09-18T14:16:15.100 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 17 to 384 2024-09-18T14:16:15.100 INFO:tasks.rados.rados.0.smithi084.stdout:3540: done (4 left) 2024-09-18T14:16:15.100 INFO:tasks.rados.rados.0.smithi084.stdout:3541: done (3 left) 2024-09-18T14:16:15.100 INFO:tasks.rados.rados.0.smithi084.stdout:3542: done (2 left) 2024-09-18T14:16:15.100 INFO:tasks.rados.rados.0.smithi084.stdout:3543: done (1 left) 2024-09-18T14:16:15.100 INFO:tasks.rados.rados.0.smithi084.stdout:3546: write oid 7 current snap is 393 2024-09-18T14:16:15.100 INFO:tasks.rados.rados.0.smithi084.stdout:3546: seq_num 1194 ranges {467558=514727,1719853=347705} 2024-09-18T14:16:15.103 INFO:tasks.rados.rados.0.smithi084.stdout:3546: writing smithi08434663-7 from 467558 to 982285 tid 1 2024-09-18T14:16:15.109 INFO:tasks.rados.rados.0.smithi084.stdout:3546: writing smithi08434663-7 from 1719853 to 2067558 tid 2 2024-09-18T14:16:15.109 INFO:tasks.rados.rados.0.smithi084.stdout:3547: snap_remove snap 390 2024-09-18T14:16:15.878 INFO:tasks.rados.rados.0.smithi084.stdout:3547: done (2 left) 2024-09-18T14:16:15.879 INFO:tasks.rados.rados.0.smithi084.stdout:3548: snap_remove snap 378 2024-09-18T14:16:16.895 INFO:tasks.rados.rados.0.smithi084.stdout:3545: finishing rollback tid 1 to smithi08434663-17 2024-09-18T14:16:16.895 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1544 (ObjNum 129 snap 0 seq_num 2495043664) dirty dne 2024-09-18T14:16:16.895 INFO:tasks.rados.rados.0.smithi084.stdout:3545: done (2 left) 2024-09-18T14:16:16.895 INFO:tasks.rados.rados.0.smithi084.stdout:3548: done (1 left) 2024-09-18T14:16:16.895 INFO:tasks.rados.rados.0.smithi084.stdout:3549: delete oid 28 current snap is 393 2024-09-18T14:16:16.895 INFO:tasks.rados.rados.0.smithi084.stdout:3546: finishing write tid 1 to smithi08434663-7 2024-09-18T14:16:16.896 INFO:tasks.rados.rados.0.smithi084.stdout:3546: finishing write tid 2 to smithi08434663-7 2024-09-18T14:16:16.896 INFO:tasks.rados.rados.0.smithi084.stdout:3546: finishing write tid 3 to smithi08434663-7 2024-09-18T14:16:16.896 INFO:tasks.rados.rados.0.smithi084.stdout:3546: finishing write tid 4 to smithi08434663-7 2024-09-18T14:16:16.896 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 1543 (ObjNum 1194 snap 393 seq_num 1194) dirty exists 2024-09-18T14:16:16.896 INFO:tasks.rados.rados.0.smithi084.stdout:3546: left oid 7 (ObjNum 1194 snap 393 seq_num 1194) 2024-09-18T14:16:16.908 INFO:tasks.rados.rados.0.smithi084.stdout:3546: done (1 left) 2024-09-18T14:16:16.908 INFO:tasks.rados.rados.0.smithi084.stdout:3549: done (0 left) 2024-09-18T14:16:16.908 INFO:tasks.rados.rados.0.smithi084.stdout:3550: read oid 1 snap -1 2024-09-18T14:16:16.908 INFO:tasks.rados.rados.0.smithi084.stdout:3550: expect (ObjNum 1157 snap 374 seq_num 1157) 2024-09-18T14:16:16.908 INFO:tasks.rados.rados.0.smithi084.stdout:3551: delete oid 46 current snap is 393 2024-09-18T14:16:16.914 INFO:tasks.rados.rados.0.smithi084.stdout:3551: done (1 left) 2024-09-18T14:16:16.914 INFO:tasks.rados.rados.0.smithi084.stdout:3552: rollback oid 16 current snap is 393 2024-09-18T14:16:16.914 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 16 to 335 2024-09-18T14:16:16.914 INFO:tasks.rados.rados.0.smithi084.stdout:3553: snap_remove snap 383 2024-09-18T14:16:16.914 INFO:tasks.rados.rados.0.smithi084.stdout:3550: expect (ObjNum 1157 snap 374 seq_num 1157) 2024-09-18T14:16:17.027 INFO:tasks.rados.rados.0.smithi084.stdout:3552: finishing rollback tid 1 to smithi08434663-16 2024-09-18T14:16:17.027 INFO:tasks.rados.rados.0.smithi084.stdout:3552: finishing rollback tid 2 to smithi08434663-16 2024-09-18T14:16:17.027 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 2028 (ObjNum 1015 snap 326 seq_num 1015) dirty exists 2024-09-18T14:16:17.952 INFO:tasks.rados.rados.0.smithi084.stdout:3550: done (2 left) 2024-09-18T14:16:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3552: done (1 left) 2024-09-18T14:16:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3553: done (0 left) 2024-09-18T14:16:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3554: copy_from oid 49 from oid 34 current snap is 393 2024-09-18T14:16:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3555: read oid 21 snap 361 2024-09-18T14:16:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3555: expect deleted 2024-09-18T14:16:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3556: write (excl) oid 24 current snap is 393 2024-09-18T14:16:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3556: seq_num 1197 ranges {517979=740474,1778946=339033} 2024-09-18T14:16:17.960 INFO:tasks.rados.rados.0.smithi084.stdout:3556: writing smithi08434663-24 from 517979 to 1258453 tid 1 2024-09-18T14:16:17.965 INFO:tasks.rados.rados.0.smithi084.stdout:3556: writing smithi08434663-24 from 1778946 to 2117979 tid 2 2024-09-18T14:16:17.965 INFO:tasks.rados.rados.0.smithi084.stdout:3557: snap_create 2024-09-18T14:16:17.968 INFO:tasks.rados.rados.0.smithi084.stdout:3556: finishing write tid 1 to smithi08434663-24 2024-09-18T14:16:17.969 INFO:tasks.rados.rados.0.smithi084.stdout:3556: finishing write tid 2 to smithi08434663-24 2024-09-18T14:16:17.969 INFO:tasks.rados.rados.0.smithi084.stdout:3556: finishing write tid 3 to smithi08434663-24 2024-09-18T14:16:17.969 INFO:tasks.rados.rados.0.smithi084.stdout:3556: finishing write tid 4 to smithi08434663-24 2024-09-18T14:16:17.969 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 1541 (ObjNum 1197 snap 393 seq_num 1197) dirty exists 2024-09-18T14:16:17.970 INFO:tasks.rados.rados.0.smithi084.stdout:3556: left oid 24 (ObjNum 1197 snap 393 seq_num 1197) 2024-09-18T14:16:17.970 INFO:tasks.rados.rados.0.smithi084.stdout:3554: finishing copy_from racing read to smithi08434663-49 2024-09-18T14:16:17.973 INFO:tasks.rados.rados.0.smithi084.stdout:3554: finishing copy_from to smithi08434663-49 2024-09-18T14:16:17.973 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 1739 (ObjNum 1163 snap 379 seq_num 1163) dirty exists 2024-09-18T14:16:18.956 INFO:tasks.rados.rados.0.smithi084.stdout:3554: done (3 left) 2024-09-18T14:16:18.956 INFO:tasks.rados.rados.0.smithi084.stdout:3555: done (2 left) 2024-09-18T14:16:18.956 INFO:tasks.rados.rados.0.smithi084.stdout:3556: done (1 left) 2024-09-18T14:16:18.956 INFO:tasks.rados.rados.0.smithi084.stdout:3557: done (0 left) 2024-09-18T14:16:18.956 INFO:tasks.rados.rados.0.smithi084.stdout:3558: rollback oid 34 current snap is 394 2024-09-18T14:16:18.957 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 34 to 340 2024-09-18T14:16:18.957 INFO:tasks.rados.rados.0.smithi084.stdout:3559: write (excl) oid 13 current snap is 394 2024-09-18T14:16:18.957 INFO:tasks.rados.rados.0.smithi084.stdout:3559: seq_num 1198 ranges {534786=415723,1437428=697358} 2024-09-18T14:16:18.961 INFO:tasks.rados.rados.0.smithi084.stdout:3559: writing smithi08434663-13 from 534786 to 950509 tid 1 2024-09-18T14:16:18.967 INFO:tasks.rados.rados.0.smithi084.stdout:3559: writing smithi08434663-13 from 1437428 to 2134786 tid 2 2024-09-18T14:16:18.968 INFO:tasks.rados.rados.0.smithi084.stdout:3560: write (excl) oid 42 current snap is 394 2024-09-18T14:16:18.968 INFO:tasks.rados.rados.0.smithi084.stdout:3560: seq_num 1199 ranges {551593=490972,1779557=372036} 2024-09-18T14:16:18.973 INFO:tasks.rados.rados.0.smithi084.stdout:3560: writing smithi08434663-42 from 551593 to 1042565 tid 1 2024-09-18T14:16:18.979 INFO:tasks.rados.rados.0.smithi084.stdout:3560: writing smithi08434663-42 from 1779557 to 2151593 tid 2 2024-09-18T14:16:18.979 INFO:tasks.rados.rados.0.smithi084.stdout:3561: write oid 39 current snap is 394 2024-09-18T14:16:18.979 INFO:tasks.rados.rados.0.smithi084.stdout:3561: seq_num 1200 ranges {568400=566221,1838039=330361} 2024-09-18T14:16:18.985 INFO:tasks.rados.rados.0.smithi084.stdout:3561: writing smithi08434663-39 from 568400 to 1134621 tid 1 2024-09-18T14:16:18.992 INFO:tasks.rados.rados.0.smithi084.stdout:3561: writing smithi08434663-39 from 1838039 to 2168400 tid 2 2024-09-18T14:16:18.992 INFO:tasks.rados.rados.0.smithi084.stdout:3562: delete oid 32 current snap is 394 2024-09-18T14:16:18.992 INFO:tasks.rados.rados.0.smithi084.stdout:3558: finishing rollback tid 0 to smithi08434663-34 2024-09-18T14:16:18.992 INFO:tasks.rados.rados.0.smithi084.stdout:3558: finishing rollback tid 1 to smithi08434663-34 2024-09-18T14:16:18.992 INFO:tasks.rados.rados.0.smithi084.stdout:3558: finishing rollback tid 2 to smithi08434663-34 2024-09-18T14:16:18.992 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1808 (ObjNum 941 snap 301 seq_num 941) dirty exists 2024-09-18T14:16:18.992 INFO:tasks.rados.rados.0.smithi084.stdout:3559: finishing write tid 1 to smithi08434663-13 2024-09-18T14:16:18.992 INFO:tasks.rados.rados.0.smithi084.stdout:3559: finishing write tid 2 to smithi08434663-13 2024-09-18T14:16:18.992 INFO:tasks.rados.rados.0.smithi084.stdout:3559: finishing write tid 3 to smithi08434663-13 2024-09-18T14:16:18.992 INFO:tasks.rados.rados.0.smithi084.stdout:3559: finishing write tid 4 to smithi08434663-13 2024-09-18T14:16:18.993 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 1576 (ObjNum 1198 snap 394 seq_num 1198) dirty exists 2024-09-18T14:16:18.993 INFO:tasks.rados.rados.0.smithi084.stdout:3559: left oid 13 (ObjNum 1198 snap 394 seq_num 1198) 2024-09-18T14:16:18.993 INFO:tasks.rados.rados.0.smithi084.stdout:3560: finishing write tid 1 to smithi08434663-42 2024-09-18T14:16:18.993 INFO:tasks.rados.rados.0.smithi084.stdout:3560: finishing write tid 2 to smithi08434663-42 2024-09-18T14:16:18.993 INFO:tasks.rados.rados.0.smithi084.stdout:3560: finishing write tid 3 to smithi08434663-42 2024-09-18T14:16:18.993 INFO:tasks.rados.rados.0.smithi084.stdout:3560: finishing write tid 4 to smithi08434663-42 2024-09-18T14:16:18.993 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1812 (ObjNum 1199 snap 394 seq_num 1199) dirty exists 2024-09-18T14:16:18.993 INFO:tasks.rados.rados.0.smithi084.stdout:3560: left oid 42 (ObjNum 1199 snap 394 seq_num 1199) 2024-09-18T14:16:18.996 INFO:tasks.rados.rados.0.smithi084.stdout:3558: done (4 left) 2024-09-18T14:16:18.996 INFO:tasks.rados.rados.0.smithi084.stdout:3559: done (3 left) 2024-09-18T14:16:18.996 INFO:tasks.rados.rados.0.smithi084.stdout:3560: done (2 left) 2024-09-18T14:16:18.997 INFO:tasks.rados.rados.0.smithi084.stdout:3562: done (1 left) 2024-09-18T14:16:18.997 INFO:tasks.rados.rados.0.smithi084.stdout:3563: write (excl) oid 44 current snap is 394 2024-09-18T14:16:18.997 INFO:tasks.rados.rados.0.smithi084.stdout:3563: seq_num 1202 ranges {602014=433072,1671356=530658} 2024-09-18T14:16:19.001 INFO:tasks.rados.rados.0.smithi084.stdout:3563: writing smithi08434663-44 from 602014 to 1035086 tid 1 2024-09-18T14:16:19.008 INFO:tasks.rados.rados.0.smithi084.stdout:3563: writing smithi08434663-44 from 1671356 to 2202014 tid 2 2024-09-18T14:16:19.008 INFO:tasks.rados.rados.0.smithi084.stdout:3564: copy_from oid 22 from oid 29 current snap is 394 2024-09-18T14:16:19.009 INFO:tasks.rados.rados.0.smithi084.stdout:3561: finishing write tid 1 to smithi08434663-39 2024-09-18T14:16:19.009 INFO:tasks.rados.rados.0.smithi084.stdout:3561: finishing write tid 2 to smithi08434663-39 2024-09-18T14:16:19.009 INFO:tasks.rados.rados.0.smithi084.stdout:3561: finishing write tid 3 to smithi08434663-39 2024-09-18T14:16:19.009 INFO:tasks.rados.rados.0.smithi084.stdout:3561: finishing write tid 4 to smithi08434663-39 2024-09-18T14:16:19.009 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 1335 (ObjNum 1200 snap 394 seq_num 1200) dirty exists 2024-09-18T14:16:19.009 INFO:tasks.rados.rados.0.smithi084.stdout:3561: left oid 39 (ObjNum 1200 snap 394 seq_num 1200) 2024-09-18T14:16:19.009 INFO:tasks.rados.rados.0.smithi084.stdout:3563: finishing write tid 1 to smithi08434663-44 2024-09-18T14:16:19.009 INFO:tasks.rados.rados.0.smithi084.stdout:3561: done (2 left) 2024-09-18T14:16:19.009 INFO:tasks.rados.rados.0.smithi084.stdout:3565: copy_from oid 39 from oid 27 current snap is 394 2024-09-18T14:16:19.010 INFO:tasks.rados.rados.0.smithi084.stdout:3566: snap_remove snap 345 2024-09-18T14:16:19.964 INFO:tasks.rados.rados.0.smithi084.stdout:3566: done (3 left) 2024-09-18T14:16:19.964 INFO:tasks.rados.rados.0.smithi084.stdout:3567: read oid 1 snap -1 2024-09-18T14:16:19.964 INFO:tasks.rados.rados.0.smithi084.stdout:3567: expect (ObjNum 1157 snap 374 seq_num 1157) 2024-09-18T14:16:19.964 INFO:tasks.rados.rados.0.smithi084.stdout:3568: snap_remove snap 380 2024-09-18T14:16:20.880 INFO:tasks.rados.rados.0.smithi084.stdout:3565: finishing copy_from to smithi08434663-39 2024-09-18T14:16:20.880 INFO:tasks.rados.rados.0.smithi084.stdout:3565: got expected ENOENT (src dne) 2024-09-18T14:16:20.880 INFO:tasks.rados.rados.0.smithi084.stdout:3565: finishing copy_from racing read to smithi08434663-39 2024-09-18T14:16:20.880 INFO:tasks.rados.rados.0.smithi084.stdout:3563: finishing write tid 2 to smithi08434663-44 2024-09-18T14:16:20.881 INFO:tasks.rados.rados.0.smithi084.stdout:3563: finishing write tid 3 to smithi08434663-44 2024-09-18T14:16:20.881 INFO:tasks.rados.rados.0.smithi084.stdout:3563: finishing write tid 4 to smithi08434663-44 2024-09-18T14:16:20.881 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 1775 (ObjNum 1202 snap 394 seq_num 1202) dirty exists 2024-09-18T14:16:20.881 INFO:tasks.rados.rados.0.smithi084.stdout:3563: left oid 44 (ObjNum 1202 snap 394 seq_num 1202) 2024-09-18T14:16:20.881 INFO:tasks.rados.rados.0.smithi084.stdout:3564: finishing copy_from racing read to smithi08434663-22 2024-09-18T14:16:20.881 INFO:tasks.rados.rados.0.smithi084.stdout:3564: finishing copy_from to smithi08434663-22 2024-09-18T14:16:20.881 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 1567 (ObjNum 1148 snap 371 seq_num 1148) dirty exists 2024-09-18T14:16:20.881 INFO:tasks.rados.rados.0.smithi084.stdout:3567: expect (ObjNum 1157 snap 374 seq_num 1157) 2024-09-18T14:16:20.998 INFO:tasks.rados.rados.0.smithi084.stdout:3563: done (4 left) 2024-09-18T14:16:20.999 INFO:tasks.rados.rados.0.smithi084.stdout:3564: done (3 left) 2024-09-18T14:16:20.999 INFO:tasks.rados.rados.0.smithi084.stdout:3565: done (2 left) 2024-09-18T14:16:20.999 INFO:tasks.rados.rados.0.smithi084.stdout:3567: done (1 left) 2024-09-18T14:16:20.999 INFO:tasks.rados.rados.0.smithi084.stdout:3568: done (0 left) 2024-09-18T14:16:20.999 INFO:tasks.rados.rados.0.smithi084.stdout:3569: snap_create 2024-09-18T14:16:21.971 INFO:tasks.rados.rados.0.smithi084.stdout:3569: done (0 left) 2024-09-18T14:16:21.971 INFO:tasks.rados.rados.0.smithi084.stdout:3570: snap_remove snap 379 2024-09-18T14:16:22.977 INFO:tasks.rados.rados.0.smithi084.stdout:3570: done (0 left) 2024-09-18T14:16:22.977 INFO:tasks.rados.rados.0.smithi084.stdout:3571: copy_from oid 42 from oid 44 current snap is 395 2024-09-18T14:16:22.977 INFO:tasks.rados.rados.0.smithi084.stdout:3572: snap_remove snap 340 2024-09-18T14:16:23.981 INFO:tasks.rados.rados.0.smithi084.stdout:3572: done (1 left) 2024-09-18T14:16:23.981 INFO:tasks.rados.rados.0.smithi084.stdout:3573: snap_create 2024-09-18T14:16:23.981 INFO:tasks.rados.rados.0.smithi084.stdout:3571: finishing copy_from racing read to smithi08434663-42 2024-09-18T14:16:23.981 INFO:tasks.rados.rados.0.smithi084.stdout:3571: finishing copy_from to smithi08434663-42 2024-09-18T14:16:23.981 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1775 (ObjNum 1202 snap 394 seq_num 1202) dirty exists 2024-09-18T14:16:24.990 INFO:tasks.rados.rados.0.smithi084.stdout:3571: done (1 left) 2024-09-18T14:16:24.990 INFO:tasks.rados.rados.0.smithi084.stdout:3573: done (0 left) 2024-09-18T14:16:24.991 INFO:tasks.rados.rados.0.smithi084.stdout:3574: snap_remove snap 381 2024-09-18T14:16:25.881 INFO:tasks.rados.rados.0.smithi084.stdout:3574: done (0 left) 2024-09-18T14:16:25.882 INFO:tasks.rados.rados.0.smithi084.stdout:3575: read oid 2 snap -1 2024-09-18T14:16:25.882 INFO:tasks.rados.rados.0.smithi084.stdout:3575: expect deleted 2024-09-18T14:16:25.882 INFO:tasks.rados.rados.0.smithi084.stdout:3576: read oid 24 snap -1 2024-09-18T14:16:25.882 INFO:tasks.rados.rados.0.smithi084.stdout:3576: expect (ObjNum 1197 snap 393 seq_num 1197) 2024-09-18T14:16:25.882 INFO:tasks.rados.rados.0.smithi084.stdout:3577: write (excl) oid 36 current snap is 396 2024-09-18T14:16:25.882 INFO:tasks.rados.rados.0.smithi084.stdout:3577: seq_num 1203 ranges {618821=508321,1613485=531077} 2024-09-18T14:16:25.888 INFO:tasks.rados.rados.0.smithi084.stdout:3577: writing smithi08434663-36 from 618821 to 1127142 tid 1 2024-09-18T14:16:25.895 INFO:tasks.rados.rados.0.smithi084.stdout:3577: writing smithi08434663-36 from 1613485 to 2144562 tid 2 2024-09-18T14:16:25.895 INFO:tasks.rados.rados.0.smithi084.stdout:3578: copy_from oid 7 from oid 21 current snap is 396 2024-09-18T14:16:25.896 INFO:tasks.rados.rados.0.smithi084.stdout:3576: expect (ObjNum 1197 snap 393 seq_num 1197) 2024-09-18T14:16:25.968 INFO:tasks.rados.rados.0.smithi084.stdout:3578: finishing copy_from to smithi08434663-7 2024-09-18T14:16:25.968 INFO:tasks.rados.rados.0.smithi084.stdout:3578: got expected ENOENT (src dne) 2024-09-18T14:16:25.968 INFO:tasks.rados.rados.0.smithi084.stdout:3578: finishing copy_from racing read to smithi08434663-7 2024-09-18T14:16:25.968 INFO:tasks.rados.rados.0.smithi084.stdout:3577: finishing write tid 1 to smithi08434663-36 2024-09-18T14:16:25.968 INFO:tasks.rados.rados.0.smithi084.stdout:3577: finishing write tid 2 to smithi08434663-36 2024-09-18T14:16:25.968 INFO:tasks.rados.rados.0.smithi084.stdout:3577: finishing write tid 3 to smithi08434663-36 2024-09-18T14:16:25.968 INFO:tasks.rados.rados.0.smithi084.stdout:3577: finishing write tid 4 to smithi08434663-36 2024-09-18T14:16:25.968 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 1742 (ObjNum 1203 snap 396 seq_num 1203) dirty exists 2024-09-18T14:16:25.968 INFO:tasks.rados.rados.0.smithi084.stdout:3577: left oid 36 (ObjNum 1203 snap 396 seq_num 1203) 2024-09-18T14:16:25.968 INFO:tasks.rados.rados.0.smithi084.stdout:3575: done (3 left) 2024-09-18T14:16:25.969 INFO:tasks.rados.rados.0.smithi084.stdout:3576: done (2 left) 2024-09-18T14:16:25.969 INFO:tasks.rados.rados.0.smithi084.stdout:3577: done (1 left) 2024-09-18T14:16:25.969 INFO:tasks.rados.rados.0.smithi084.stdout:3578: done (0 left) 2024-09-18T14:16:25.969 INFO:tasks.rados.rados.0.smithi084.stdout:3579: rollback oid 42 current snap is 396 2024-09-18T14:16:25.969 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 42 to 393 2024-09-18T14:16:25.969 INFO:tasks.rados.rados.0.smithi084.stdout:3580: copy_from oid 40 from oid 35 current snap is 396 2024-09-18T14:16:25.969 INFO:tasks.rados.rados.0.smithi084.stdout:3581: copy_from oid 14 from oid 13 current snap is 396 2024-09-18T14:16:25.969 INFO:tasks.rados.rados.0.smithi084.stdout:3582: rollback oid 16 current snap is 396 2024-09-18T14:16:25.969 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 16 to 395 2024-09-18T14:16:25.969 INFO:tasks.rados.rados.0.smithi084.stdout:3583: snap_remove snap 388 2024-09-18T14:16:26.890 INFO:tasks.rados.rados.0.smithi084.stdout:3580: finishing copy_from to smithi08434663-40 2024-09-18T14:16:26.891 INFO:tasks.rados.rados.0.smithi084.stdout:3580: got expected ENOENT (src dne) 2024-09-18T14:16:26.891 INFO:tasks.rados.rados.0.smithi084.stdout:3580: finishing copy_from racing read to smithi08434663-40 2024-09-18T14:16:26.891 INFO:tasks.rados.rados.0.smithi084.stdout:3579: finishing rollback tid 0 to smithi08434663-42 2024-09-18T14:16:26.891 INFO:tasks.rados.rados.0.smithi084.stdout:3582: finishing rollback tid 0 to smithi08434663-16 2024-09-18T14:16:26.891 INFO:tasks.rados.rados.0.smithi084.stdout:3579: finishing rollback tid 1 to smithi08434663-42 2024-09-18T14:16:26.891 INFO:tasks.rados.rados.0.smithi084.stdout:3582: finishing rollback tid 1 to smithi08434663-16 2024-09-18T14:16:26.891 INFO:tasks.rados.rados.0.smithi084.stdout:3579: finishing rollback tid 2 to smithi08434663-42 2024-09-18T14:16:26.891 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1852 (ObjNum 1154 snap 374 seq_num 1154) dirty exists 2024-09-18T14:16:26.891 INFO:tasks.rados.rados.0.smithi084.stdout:3582: finishing rollback tid 2 to smithi08434663-16 2024-09-18T14:16:26.891 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 2031 (ObjNum 1015 snap 326 seq_num 1015) dirty exists 2024-09-18T14:16:26.891 INFO:tasks.rados.rados.0.smithi084.stdout:3581: finishing copy_from racing read to smithi08434663-14 2024-09-18T14:16:26.892 INFO:tasks.rados.rados.0.smithi084.stdout:3581: finishing copy_from to smithi08434663-14 2024-09-18T14:16:26.892 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 1576 (ObjNum 1198 snap 394 seq_num 1198) dirty exists 2024-09-18T14:16:26.892 INFO:tasks.rados.rados.0.smithi084.stdout:3579: done (4 left) 2024-09-18T14:16:26.892 INFO:tasks.rados.rados.0.smithi084.stdout:3580: done (3 left) 2024-09-18T14:16:26.892 INFO:tasks.rados.rados.0.smithi084.stdout:3581: done (2 left) 2024-09-18T14:16:26.892 INFO:tasks.rados.rados.0.smithi084.stdout:3582: done (1 left) 2024-09-18T14:16:26.892 INFO:tasks.rados.rados.0.smithi084.stdout:3583: done (0 left) 2024-09-18T14:16:26.892 INFO:tasks.rados.rados.0.smithi084.stdout:3584: write oid 1 current snap is 396 2024-09-18T14:16:26.892 INFO:tasks.rados.rados.0.smithi084.stdout:3584: seq_num 1204 ranges {635628=583570,1671967=563661} 2024-09-18T14:16:26.897 INFO:tasks.rados.rados.0.smithi084.stdout:3584: writing smithi08434663-1 from 635628 to 1219198 tid 1 2024-09-18T14:16:26.903 INFO:tasks.rados.rados.0.smithi084.stdout:3584: writing smithi08434663-1 from 1671967 to 2235628 tid 2 2024-09-18T14:16:26.903 INFO:tasks.rados.rados.0.smithi084.stdout:3585: read oid 4 snap -1 2024-09-18T14:16:26.903 INFO:tasks.rados.rados.0.smithi084.stdout:3585: expect (ObjNum 1043 snap 334 seq_num 1043) 2024-09-18T14:16:26.904 INFO:tasks.rados.rados.0.smithi084.stdout:3586: read oid 33 snap -1 2024-09-18T14:16:26.904 INFO:tasks.rados.rados.0.smithi084.stdout:3586: expect (ObjNum 1120 snap 360 seq_num 1120) 2024-09-18T14:16:26.904 INFO:tasks.rados.rados.0.smithi084.stdout:3587: snap_remove snap 361 2024-09-18T14:16:27.894 INFO:tasks.rados.rados.0.smithi084.stdout:3584: finishing write tid 1 to smithi08434663-1 2024-09-18T14:16:27.894 INFO:tasks.rados.rados.0.smithi084.stdout:3584: finishing write tid 2 to smithi08434663-1 2024-09-18T14:16:27.894 INFO:tasks.rados.rados.0.smithi084.stdout:3584: finishing write tid 3 to smithi08434663-1 2024-09-18T14:16:27.894 INFO:tasks.rados.rados.0.smithi084.stdout:3584: finishing write tid 4 to smithi08434663-1 2024-09-18T14:16:27.894 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 1779 (ObjNum 1204 snap 396 seq_num 1204) dirty exists 2024-09-18T14:16:27.894 INFO:tasks.rados.rados.0.smithi084.stdout:3584: left oid 1 (ObjNum 1204 snap 396 seq_num 1204) 2024-09-18T14:16:27.894 INFO:tasks.rados.rados.0.smithi084.stdout:3586: expect (ObjNum 1120 snap 360 seq_num 1120) 2024-09-18T14:16:28.017 INFO:tasks.rados.rados.0.smithi084.stdout:3585: expect (ObjNum 1043 snap 334 seq_num 1043) 2024-09-18T14:16:28.333 INFO:tasks.rados.rados.0.smithi084.stdout:3584: done (3 left) 2024-09-18T14:16:28.333 INFO:tasks.rados.rados.0.smithi084.stdout:3585: done (2 left) 2024-09-18T14:16:28.333 INFO:tasks.rados.rados.0.smithi084.stdout:3586: done (1 left) 2024-09-18T14:16:28.334 INFO:tasks.rados.rados.0.smithi084.stdout:3587: done (0 left) 2024-09-18T14:16:28.334 INFO:tasks.rados.rados.0.smithi084.stdout:3588: delete oid 10 current snap is 396 2024-09-18T14:16:28.338 INFO:tasks.rados.rados.0.smithi084.stdout:3588: done (0 left) 2024-09-18T14:16:28.338 INFO:tasks.rados.rados.0.smithi084.stdout:3589: rollback oid 3 current snap is 396 2024-09-18T14:16:28.338 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 3 to 366 2024-09-18T14:16:28.339 INFO:tasks.rados.rados.0.smithi084.stdout:3590: write oid 8 current snap is 396 2024-09-18T14:16:28.339 INFO:tasks.rados.rados.0.smithi084.stdout:3590: seq_num 1206 ranges {669242=734068,2188931=80311} 2024-09-18T14:16:28.346 INFO:tasks.rados.rados.0.smithi084.stdout:3590: writing smithi08434663-8 from 669242 to 1403310 tid 1 2024-09-18T14:16:28.351 INFO:tasks.rados.rados.0.smithi084.stdout:3590: writing smithi08434663-8 from 2188931 to 2269242 tid 2 2024-09-18T14:16:28.351 INFO:tasks.rados.rados.0.smithi084.stdout:3591: write oid 34 current snap is 396 2024-09-18T14:16:28.351 INFO:tasks.rados.rados.0.smithi084.stdout:3591: seq_num 1207 ranges {686049=409317,1847413=438636} 2024-09-18T14:16:28.356 INFO:tasks.rados.rados.0.smithi084.stdout:3591: writing smithi08434663-34 from 686049 to 1095366 tid 1 2024-09-18T14:16:28.363 INFO:tasks.rados.rados.0.smithi084.stdout:3591: writing smithi08434663-34 from 1847413 to 2286049 tid 2 2024-09-18T14:16:28.363 INFO:tasks.rados.rados.0.smithi084.stdout:3592: delete oid 28 current snap is 396 2024-09-18T14:16:28.363 INFO:tasks.rados.rados.0.smithi084.stdout:3589: finishing rollback tid 0 to smithi08434663-3 2024-09-18T14:16:28.364 INFO:tasks.rados.rados.0.smithi084.stdout:3589: finishing rollback tid 1 to smithi08434663-3 2024-09-18T14:16:28.364 INFO:tasks.rados.rados.0.smithi084.stdout:3589: finishing rollback tid 2 to smithi08434663-3 2024-09-18T14:16:28.364 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 1570 (ObjNum 967 snap 308 seq_num 967) dirty exists 2024-09-18T14:16:28.364 INFO:tasks.rados.rados.0.smithi084.stdout:3590: finishing write tid 1 to smithi08434663-8 2024-09-18T14:16:28.364 INFO:tasks.rados.rados.0.smithi084.stdout:3590: finishing write tid 2 to smithi08434663-8 2024-09-18T14:16:28.364 INFO:tasks.rados.rados.0.smithi084.stdout:3590: finishing write tid 3 to smithi08434663-8 2024-09-18T14:16:28.364 INFO:tasks.rados.rados.0.smithi084.stdout:3590: finishing write tid 4 to smithi08434663-8 2024-09-18T14:16:28.364 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 1423 (ObjNum 1206 snap 396 seq_num 1206) dirty exists 2024-09-18T14:16:28.364 INFO:tasks.rados.rados.0.smithi084.stdout:3590: left oid 8 (ObjNum 1206 snap 396 seq_num 1206) 2024-09-18T14:16:28.364 INFO:tasks.rados.rados.0.smithi084.stdout:3591: finishing write tid 1 to smithi08434663-34 2024-09-18T14:16:28.367 INFO:tasks.rados.rados.0.smithi084.stdout:3589: done (3 left) 2024-09-18T14:16:28.367 INFO:tasks.rados.rados.0.smithi084.stdout:3590: done (2 left) 2024-09-18T14:16:28.367 INFO:tasks.rados.rados.0.smithi084.stdout:3592: done (1 left) 2024-09-18T14:16:28.367 INFO:tasks.rados.rados.0.smithi084.stdout:3593: read oid 33 snap 386 2024-09-18T14:16:28.367 INFO:tasks.rados.rados.0.smithi084.stdout:3593: expect (ObjNum 1120 snap 360 seq_num 1120) 2024-09-18T14:16:28.367 INFO:tasks.rados.rados.0.smithi084.stdout:3594: rollback oid 6 current snap is 396 2024-09-18T14:16:28.367 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 391 2024-09-18T14:16:28.367 INFO:tasks.rados.rados.0.smithi084.stdout:3595: write (excl) oid 5 current snap is 396 2024-09-18T14:16:28.368 INFO:tasks.rados.rados.0.smithi084.stdout:3595: seq_num 1209 ranges {719663=676168,1964377=355286} 2024-09-18T14:16:28.374 INFO:tasks.rados.rados.0.smithi084.stdout:3595: writing smithi08434663-5 from 719663 to 1395831 tid 1 2024-09-18T14:16:28.380 INFO:tasks.rados.rados.0.smithi084.stdout:3595: writing smithi08434663-5 from 1964377 to 2319663 tid 2 2024-09-18T14:16:28.380 INFO:tasks.rados.rados.0.smithi084.stdout:3596: read oid 26 snap -1 2024-09-18T14:16:28.380 INFO:tasks.rados.rados.0.smithi084.stdout:3596: expect deleted 2024-09-18T14:16:28.380 INFO:tasks.rados.rados.0.smithi084.stdout:3597: delete oid 2 current snap is 396 2024-09-18T14:16:28.380 INFO:tasks.rados.rados.0.smithi084.stdout:3591: finishing write tid 2 to smithi08434663-34 2024-09-18T14:16:28.380 INFO:tasks.rados.rados.0.smithi084.stdout:3591: finishing write tid 3 to smithi08434663-34 2024-09-18T14:16:28.381 INFO:tasks.rados.rados.0.smithi084.stdout:3594: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:16:28.381 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 1537 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-09-18T14:16:28.381 INFO:tasks.rados.rados.0.smithi084.stdout:3591: finishing write tid 4 to smithi08434663-34 2024-09-18T14:16:28.381 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 1870 (ObjNum 1207 snap 396 seq_num 1207) dirty exists 2024-09-18T14:16:28.381 INFO:tasks.rados.rados.0.smithi084.stdout:3591: left oid 34 (ObjNum 1207 snap 396 seq_num 1207) 2024-09-18T14:16:28.381 INFO:tasks.rados.rados.0.smithi084.stdout:3593: expect (ObjNum 1120 snap 360 seq_num 1120) 2024-09-18T14:16:28.499 INFO:tasks.rados.rados.0.smithi084.stdout:3595: finishing write tid 1 to smithi08434663-5 2024-09-18T14:16:28.499 INFO:tasks.rados.rados.0.smithi084.stdout:3595: finishing write tid 2 to smithi08434663-5 2024-09-18T14:16:28.499 INFO:tasks.rados.rados.0.smithi084.stdout:3595: finishing write tid 3 to smithi08434663-5 2024-09-18T14:16:28.499 INFO:tasks.rados.rados.0.smithi084.stdout:3595: finishing write tid 4 to smithi08434663-5 2024-09-18T14:16:28.499 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 1778 (ObjNum 1209 snap 396 seq_num 1209) dirty exists 2024-09-18T14:16:28.499 INFO:tasks.rados.rados.0.smithi084.stdout:3595: left oid 5 (ObjNum 1209 snap 396 seq_num 1209) 2024-09-18T14:16:28.499 INFO:tasks.rados.rados.0.smithi084.stdout:3591: done (5 left) 2024-09-18T14:16:28.499 INFO:tasks.rados.rados.0.smithi084.stdout:3593: done (4 left) 2024-09-18T14:16:28.499 INFO:tasks.rados.rados.0.smithi084.stdout:3594: done (3 left) 2024-09-18T14:16:28.499 INFO:tasks.rados.rados.0.smithi084.stdout:3595: done (2 left) 2024-09-18T14:16:28.499 INFO:tasks.rados.rados.0.smithi084.stdout:3596: done (1 left) 2024-09-18T14:16:28.500 INFO:tasks.rados.rados.0.smithi084.stdout:3597: done (0 left) 2024-09-18T14:16:28.500 INFO:tasks.rados.rados.0.smithi084.stdout:3598: snap_create 2024-09-18T14:16:28.900 INFO:tasks.rados.rados.0.smithi084.stdout:3598: done (0 left) 2024-09-18T14:16:28.900 INFO:tasks.rados.rados.0.smithi084.stdout:3599: write oid 41 current snap is 397 2024-09-18T14:16:28.901 INFO:tasks.rados.rados.0.smithi084.stdout:3599: seq_num 1211 ranges {753277=426666,1681341=545753} 2024-09-18T14:16:28.906 INFO:tasks.rados.rados.0.smithi084.stdout:3599: writing smithi08434663-41 from 753277 to 1179943 tid 1 2024-09-18T14:16:28.912 INFO:tasks.rados.rados.0.smithi084.stdout:3599: writing smithi08434663-41 from 1681341 to 2227094 tid 2 2024-09-18T14:16:28.912 INFO:tasks.rados.rados.0.smithi084.stdout:3600: snap_remove snap 393 2024-09-18T14:16:29.901 INFO:tasks.rados.rados.0.smithi084.stdout:3600: done (1 left) 2024-09-18T14:16:29.901 INFO:tasks.rados.rados.0.smithi084.stdout:3601: write (excl) oid 9 current snap is 397 2024-09-18T14:16:29.901 INFO:tasks.rados.rados.0.smithi084.stdout:3601: seq_num 1212 ranges {770084=501915,2023470=346614} 2024-09-18T14:16:29.908 INFO:tasks.rados.rados.0.smithi084.stdout:3601: writing smithi08434663-9 from 770084 to 1271999 tid 1 2024-09-18T14:16:29.915 INFO:tasks.rados.rados.0.smithi084.stdout:3601: writing smithi08434663-9 from 2023470 to 2370084 tid 2 2024-09-18T14:16:29.915 INFO:tasks.rados.rados.0.smithi084.stdout:3602: delete oid 14 current snap is 397 2024-09-18T14:16:29.915 INFO:tasks.rados.rados.0.smithi084.stdout:3599: finishing write tid 1 to smithi08434663-41 2024-09-18T14:16:29.915 INFO:tasks.rados.rados.0.smithi084.stdout:3599: finishing write tid 2 to smithi08434663-41 2024-09-18T14:16:29.915 INFO:tasks.rados.rados.0.smithi084.stdout:3599: finishing write tid 3 to smithi08434663-41 2024-09-18T14:16:29.915 INFO:tasks.rados.rados.0.smithi084.stdout:3599: finishing write tid 4 to smithi08434663-41 2024-09-18T14:16:29.915 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 1874 (ObjNum 1211 snap 397 seq_num 1211) dirty exists 2024-09-18T14:16:29.915 INFO:tasks.rados.rados.0.smithi084.stdout:3599: left oid 41 (ObjNum 1211 snap 397 seq_num 1211) 2024-09-18T14:16:29.919 INFO:tasks.rados.rados.0.smithi084.stdout:3601: finishing write tid 1 to smithi08434663-9 2024-09-18T14:16:29.919 INFO:tasks.rados.rados.0.smithi084.stdout:3601: finishing write tid 2 to smithi08434663-9 2024-09-18T14:16:29.919 INFO:tasks.rados.rados.0.smithi084.stdout:3601: finishing write tid 3 to smithi08434663-9 2024-09-18T14:16:29.919 INFO:tasks.rados.rados.0.smithi084.stdout:3601: finishing write tid 4 to smithi08434663-9 2024-09-18T14:16:29.919 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 2034 (ObjNum 1212 snap 397 seq_num 1212) dirty exists 2024-09-18T14:16:29.919 INFO:tasks.rados.rados.0.smithi084.stdout:3601: left oid 9 (ObjNum 1212 snap 397 seq_num 1212) 2024-09-18T14:16:29.919 INFO:tasks.rados.rados.0.smithi084.stdout:3599: done (2 left) 2024-09-18T14:16:29.919 INFO:tasks.rados.rados.0.smithi084.stdout:3601: done (1 left) 2024-09-18T14:16:29.919 INFO:tasks.rados.rados.0.smithi084.stdout:3602: done (0 left) 2024-09-18T14:16:29.920 INFO:tasks.rados.rados.0.smithi084.stdout:3603: copy_from oid 34 from oid 27 current snap is 397 2024-09-18T14:16:29.920 INFO:tasks.rados.rados.0.smithi084.stdout:3604: write (excl) oid 1 current snap is 397 2024-09-18T14:16:29.920 INFO:tasks.rados.rados.0.smithi084.stdout:3604: seq_num 1214 ranges {403698=652413,1740434=663264} 2024-09-18T14:16:29.924 INFO:tasks.rados.rados.0.smithi084.stdout:3604: writing smithi08434663-1 from 403698 to 1056111 tid 1 2024-09-18T14:16:29.932 INFO:tasks.rados.rados.0.smithi084.stdout:3604: writing smithi08434663-1 from 1740434 to 2403698 tid 2 2024-09-18T14:16:29.932 INFO:tasks.rados.rados.0.smithi084.stdout:3605: delete oid 43 current snap is 397 2024-09-18T14:16:29.932 INFO:tasks.rados.rados.0.smithi084.stdout:3603: finishing copy_from to smithi08434663-34 2024-09-18T14:16:29.932 INFO:tasks.rados.rados.0.smithi084.stdout:3603: got expected ENOENT (src dne) 2024-09-18T14:16:29.932 INFO:tasks.rados.rados.0.smithi084.stdout:3603: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:16:29.932 INFO:tasks.rados.rados.0.smithi084.stdout:3604: finishing write tid 1 to smithi08434663-1 2024-09-18T14:16:29.935 INFO:tasks.rados.rados.0.smithi084.stdout:3603: done (2 left) 2024-09-18T14:16:29.935 INFO:tasks.rados.rados.0.smithi084.stdout:3605: done (1 left) 2024-09-18T14:16:29.935 INFO:tasks.rados.rados.0.smithi084.stdout:3606: rollback oid 8 current snap is 397 2024-09-18T14:16:29.935 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 8 to 360 2024-09-18T14:16:29.935 INFO:tasks.rados.rados.0.smithi084.stdout:3607: read oid 18 snap -1 2024-09-18T14:16:29.935 INFO:tasks.rados.rados.0.smithi084.stdout:3607: expect (ObjNum 1166 snap 380 seq_num 1166) 2024-09-18T14:16:29.936 INFO:tasks.rados.rados.0.smithi084.stdout:3608: snap_create 2024-09-18T14:16:29.936 INFO:tasks.rados.rados.0.smithi084.stdout:3604: finishing write tid 2 to smithi08434663-1 2024-09-18T14:16:29.936 INFO:tasks.rados.rados.0.smithi084.stdout:3604: finishing write tid 3 to smithi08434663-1 2024-09-18T14:16:29.937 INFO:tasks.rados.rados.0.smithi084.stdout:3604: finishing write tid 4 to smithi08434663-1 2024-09-18T14:16:29.937 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 1783 (ObjNum 1214 snap 397 seq_num 1214) dirty exists 2024-09-18T14:16:29.937 INFO:tasks.rados.rados.0.smithi084.stdout:3604: left oid 1 (ObjNum 1214 snap 397 seq_num 1214) 2024-09-18T14:16:29.940 INFO:tasks.rados.rados.0.smithi084.stdout:3606: finishing rollback tid 0 to smithi08434663-8 2024-09-18T14:16:29.940 INFO:tasks.rados.rados.0.smithi084.stdout:3606: finishing rollback tid 1 to smithi08434663-8 2024-09-18T14:16:29.940 INFO:tasks.rados.rados.0.smithi084.stdout:3606: finishing rollback tid 2 to smithi08434663-8 2024-09-18T14:16:29.941 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 1426 (ObjNum 1041 snap 333 seq_num 1041) dirty exists 2024-09-18T14:16:29.951 INFO:tasks.rados.rados.0.smithi084.stdout:3607: expect (ObjNum 1166 snap 380 seq_num 1166) 2024-09-18T14:16:30.882 INFO:tasks.rados.rados.0.smithi084.stdout:3604: done (3 left) 2024-09-18T14:16:30.882 INFO:tasks.rados.rados.0.smithi084.stdout:3606: done (2 left) 2024-09-18T14:16:30.882 INFO:tasks.rados.rados.0.smithi084.stdout:3607: done (1 left) 2024-09-18T14:16:30.882 INFO:tasks.rados.rados.0.smithi084.stdout:3608: done (0 left) 2024-09-18T14:16:30.882 INFO:tasks.rados.rados.0.smithi084.stdout:3609: read oid 41 snap -1 2024-09-18T14:16:30.882 INFO:tasks.rados.rados.0.smithi084.stdout:3609: expect (ObjNum 1211 snap 397 seq_num 1211) 2024-09-18T14:16:30.882 INFO:tasks.rados.rados.0.smithi084.stdout:3610: snap_create 2024-09-18T14:16:30.897 INFO:tasks.rados.rados.0.smithi084.stdout:3609: expect (ObjNum 1211 snap 397 seq_num 1211) 2024-09-18T14:16:31.884 INFO:tasks.rados.rados.0.smithi084.stdout:3609: done (1 left) 2024-09-18T14:16:31.884 INFO:tasks.rados.rados.0.smithi084.stdout:3610: done (0 left) 2024-09-18T14:16:31.884 INFO:tasks.rados.rados.0.smithi084.stdout:3611: write oid 22 current snap is 399 2024-09-18T14:16:31.885 INFO:tasks.rados.rados.0.smithi084.stdout:3611: seq_num 1216 ranges {437312=402911,1341045=696749} 2024-09-18T14:16:31.888 INFO:tasks.rados.rados.0.smithi084.stdout:3611: writing smithi08434663-22 from 437312 to 840223 tid 1 2024-09-18T14:16:31.896 INFO:tasks.rados.rados.0.smithi084.stdout:3611: writing smithi08434663-22 from 1341045 to 2037794 tid 2 2024-09-18T14:16:31.896 INFO:tasks.rados.rados.0.smithi084.stdout:3612: snap_create 2024-09-18T14:16:31.897 INFO:tasks.rados.rados.0.smithi084.stdout:3611: finishing write tid 1 to smithi08434663-22 2024-09-18T14:16:31.901 INFO:tasks.rados.rados.0.smithi084.stdout:3611: finishing write tid 2 to smithi08434663-22 2024-09-18T14:16:31.901 INFO:tasks.rados.rados.0.smithi084.stdout:3611: finishing write tid 3 to smithi08434663-22 2024-09-18T14:16:31.902 INFO:tasks.rados.rados.0.smithi084.stdout:3611: finishing write tid 4 to smithi08434663-22 2024-09-18T14:16:31.902 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 1573 (ObjNum 1216 snap 399 seq_num 1216) dirty exists 2024-09-18T14:16:31.902 INFO:tasks.rados.rados.0.smithi084.stdout:3611: left oid 22 (ObjNum 1216 snap 399 seq_num 1216) 2024-09-18T14:16:32.888 INFO:tasks.rados.rados.0.smithi084.stdout:3611: done (1 left) 2024-09-18T14:16:32.888 INFO:tasks.rados.rados.0.smithi084.stdout:3612: done (0 left) 2024-09-18T14:16:32.889 INFO:tasks.rados.rados.0.smithi084.stdout:3613: delete oid 11 current snap is 400 2024-09-18T14:16:32.893 INFO:tasks.rados.rados.0.smithi084.stdout:3613: done (0 left) 2024-09-18T14:16:32.893 INFO:tasks.rados.rados.0.smithi084.stdout:3614: rollback oid 20 current snap is 400 2024-09-18T14:16:32.893 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 20 to 373 2024-09-18T14:16:32.893 INFO:tasks.rados.rados.0.smithi084.stdout:3615: snap_remove snap 394 2024-09-18T14:16:33.892 INFO:tasks.rados.rados.0.smithi084.stdout:3614: finishing rollback tid 0 to smithi08434663-20 2024-09-18T14:16:33.892 INFO:tasks.rados.rados.0.smithi084.stdout:3614: finishing rollback tid 1 to smithi08434663-20 2024-09-18T14:16:33.892 INFO:tasks.rados.rados.0.smithi084.stdout:3614: finishing rollback tid 2 to smithi08434663-20 2024-09-18T14:16:33.892 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 1429 (ObjNum 1065 snap 339 seq_num 1065) dirty exists 2024-09-18T14:16:33.892 INFO:tasks.rados.rados.0.smithi084.stdout:3614: done (1 left) 2024-09-18T14:16:33.892 INFO:tasks.rados.rados.0.smithi084.stdout:3615: done (0 left) 2024-09-18T14:16:33.892 INFO:tasks.rados.rados.0.smithi084.stdout:3616: rollback oid 22 current snap is 400 2024-09-18T14:16:33.892 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 22 to 391 2024-09-18T14:16:33.893 INFO:tasks.rados.rados.0.smithi084.stdout:3617: snap_remove snap 384 2024-09-18T14:16:34.907 INFO:tasks.rados.rados.0.smithi084.stdout:3617: done (1 left) 2024-09-18T14:16:34.907 INFO:tasks.rados.rados.0.smithi084.stdout:3618: copy_from oid 48 from oid 38 current snap is 400 2024-09-18T14:16:34.907 INFO:tasks.rados.rados.0.smithi084.stdout:3616: finishing rollback tid 0 to smithi08434663-22 2024-09-18T14:16:34.908 INFO:tasks.rados.rados.0.smithi084.stdout:3616: finishing rollback tid 1 to smithi08434663-22 2024-09-18T14:16:34.908 INFO:tasks.rados.rados.0.smithi084.stdout:3616: finishing rollback tid 2 to smithi08434663-22 2024-09-18T14:16:34.908 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 1576 (ObjNum 1111 snap 355 seq_num 1111) dirty exists 2024-09-18T14:16:34.908 INFO:tasks.rados.rados.0.smithi084.stdout:3616: done (1 left) 2024-09-18T14:16:34.908 INFO:tasks.rados.rados.0.smithi084.stdout:3619: write oid 44 current snap is 400 2024-09-18T14:16:34.908 INFO:tasks.rados.rados.0.smithi084.stdout:3619: seq_num 1218 ranges {470926=669762,1574362=700418} 2024-09-18T14:16:34.913 INFO:tasks.rados.rados.0.smithi084.stdout:3619: writing smithi08434663-44 from 470926 to 1140688 tid 1 2024-09-18T14:16:34.920 INFO:tasks.rados.rados.0.smithi084.stdout:3619: writing smithi08434663-44 from 1574362 to 2274780 tid 2 2024-09-18T14:16:34.920 INFO:tasks.rados.rados.0.smithi084.stdout:3620: delete oid 23 current snap is 400 2024-09-18T14:16:34.923 INFO:tasks.rados.rados.0.smithi084.stdout:3620: done (2 left) 2024-09-18T14:16:34.923 INFO:tasks.rados.rados.0.smithi084.stdout:3621: write (excl) oid 21 current snap is 400 2024-09-18T14:16:34.923 INFO:tasks.rados.rados.0.smithi084.stdout:3621: seq_num 1220 ranges {504540=420260,1574973=704087} 2024-09-18T14:16:34.928 INFO:tasks.rados.rados.0.smithi084.stdout:3621: writing smithi08434663-21 from 504540 to 924800 tid 1 2024-09-18T14:16:34.936 INFO:tasks.rados.rados.0.smithi084.stdout:3621: writing smithi08434663-21 from 1574973 to 2279060 tid 2 2024-09-18T14:16:34.936 INFO:tasks.rados.rados.0.smithi084.stdout:3622: rollback oid 17 current snap is 400 2024-09-18T14:16:34.936 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 17 to 386 2024-09-18T14:16:34.936 INFO:tasks.rados.rados.0.smithi084.stdout:3619: finishing write tid 1 to smithi08434663-44 2024-09-18T14:16:34.936 INFO:tasks.rados.rados.0.smithi084.stdout:3619: finishing write tid 2 to smithi08434663-44 2024-09-18T14:16:34.936 INFO:tasks.rados.rados.0.smithi084.stdout:3619: finishing write tid 3 to smithi08434663-44 2024-09-18T14:16:34.936 INFO:tasks.rados.rados.0.smithi084.stdout:3619: finishing write tid 4 to smithi08434663-44 2024-09-18T14:16:34.936 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 1781 (ObjNum 1218 snap 400 seq_num 1218) dirty exists 2024-09-18T14:16:34.936 INFO:tasks.rados.rados.0.smithi084.stdout:3619: left oid 44 (ObjNum 1218 snap 400 seq_num 1218) 2024-09-18T14:16:34.937 INFO:tasks.rados.rados.0.smithi084.stdout:3621: finishing write tid 1 to smithi08434663-21 2024-09-18T14:16:34.937 INFO:tasks.rados.rados.0.smithi084.stdout:3619: done (3 left) 2024-09-18T14:16:34.937 INFO:tasks.rados.rados.0.smithi084.stdout:3623: snap_create 2024-09-18T14:16:34.938 INFO:tasks.rados.rados.0.smithi084.stdout:3622: finishing rollback tid 1 to smithi08434663-17 2024-09-18T14:16:34.938 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1544 (ObjNum 145 snap 0 seq_num 2494680928) dirty dne 2024-09-18T14:16:34.945 INFO:tasks.rados.rados.0.smithi084.stdout:3618: finishing copy_from racing read to smithi08434663-48 2024-09-18T14:16:34.945 INFO:tasks.rados.rados.0.smithi084.stdout:3621: finishing write tid 2 to smithi08434663-21 2024-09-18T14:16:34.945 INFO:tasks.rados.rados.0.smithi084.stdout:3621: finishing write tid 3 to smithi08434663-21 2024-09-18T14:16:34.945 INFO:tasks.rados.rados.0.smithi084.stdout:3621: finishing write tid 4 to smithi08434663-21 2024-09-18T14:16:34.945 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 1564 (ObjNum 1220 snap 400 seq_num 1220) dirty exists 2024-09-18T14:16:34.946 INFO:tasks.rados.rados.0.smithi084.stdout:3621: left oid 21 (ObjNum 1220 snap 400 seq_num 1220) 2024-09-18T14:16:34.946 INFO:tasks.rados.rados.0.smithi084.stdout:3618: finishing copy_from to smithi08434663-48 2024-09-18T14:16:34.947 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 1416 (ObjNum 1171 snap 382 seq_num 1171) dirty exists 2024-09-18T14:16:35.885 INFO:tasks.rados.rados.0.smithi084.stdout:3618: done (3 left) 2024-09-18T14:16:35.885 INFO:tasks.rados.rados.0.smithi084.stdout:3621: done (2 left) 2024-09-18T14:16:35.885 INFO:tasks.rados.rados.0.smithi084.stdout:3622: done (1 left) 2024-09-18T14:16:35.885 INFO:tasks.rados.rados.0.smithi084.stdout:3623: done (0 left) 2024-09-18T14:16:35.885 INFO:tasks.rados.rados.0.smithi084.stdout:3624: write oid 23 current snap is 401 2024-09-18T14:16:35.885 INFO:tasks.rados.rados.0.smithi084.stdout:3624: seq_num 1221 ranges {521347=495509,1633455=564098} 2024-09-18T14:16:35.890 INFO:tasks.rados.rados.0.smithi084.stdout:3624: writing smithi08434663-23 from 521347 to 1016856 tid 1 2024-09-18T14:16:35.898 INFO:tasks.rados.rados.0.smithi084.stdout:3624: writing smithi08434663-23 from 1633455 to 2197553 tid 2 2024-09-18T14:16:35.899 INFO:tasks.rados.rados.0.smithi084.stdout:3625: delete oid 21 current snap is 401 2024-09-18T14:16:35.899 INFO:tasks.rados.rados.0.smithi084.stdout:3624: finishing write tid 1 to smithi08434663-23 2024-09-18T14:16:35.908 INFO:tasks.rados.rados.0.smithi084.stdout:3624: finishing write tid 2 to smithi08434663-23 2024-09-18T14:16:35.908 INFO:tasks.rados.rados.0.smithi084.stdout:3624: finishing write tid 3 to smithi08434663-23 2024-09-18T14:16:35.909 INFO:tasks.rados.rados.0.smithi084.stdout:3624: finishing write tid 4 to smithi08434663-23 2024-09-18T14:16:35.909 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1547 (ObjNum 1221 snap 401 seq_num 1221) dirty exists 2024-09-18T14:16:35.909 INFO:tasks.rados.rados.0.smithi084.stdout:3624: left oid 23 (ObjNum 1221 snap 401 seq_num 1221) 2024-09-18T14:16:35.915 INFO:tasks.rados.rados.0.smithi084.stdout:3624: done (1 left) 2024-09-18T14:16:35.915 INFO:tasks.rados.rados.0.smithi084.stdout:3625: done (0 left) 2024-09-18T14:16:35.915 INFO:tasks.rados.rados.0.smithi084.stdout:3626: rollback oid 22 current snap is 401 2024-09-18T14:16:35.915 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 22 to 371 2024-09-18T14:16:35.915 INFO:tasks.rados.rados.0.smithi084.stdout:3627: snap_remove snap 395 2024-09-18T14:16:36.921 INFO:tasks.rados.rados.0.smithi084.stdout:3627: done (1 left) 2024-09-18T14:16:36.922 INFO:tasks.rados.rados.0.smithi084.stdout:3628: delete oid 5 current snap is 401 2024-09-18T14:16:36.922 INFO:tasks.rados.rados.0.smithi084.stdout:3626: finishing rollback tid 0 to smithi08434663-22 2024-09-18T14:16:36.922 INFO:tasks.rados.rados.0.smithi084.stdout:3626: finishing rollback tid 1 to smithi08434663-22 2024-09-18T14:16:36.922 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 1578 (ObjNum 129 snap 0 seq_num 2494617304) dirty dne 2024-09-18T14:16:36.926 INFO:tasks.rados.rados.0.smithi084.stdout:3626: done (1 left) 2024-09-18T14:16:36.926 INFO:tasks.rados.rados.0.smithi084.stdout:3628: done (0 left) 2024-09-18T14:16:36.926 INFO:tasks.rados.rados.0.smithi084.stdout:3629: write (excl) oid 16 current snap is 401 2024-09-18T14:16:36.926 INFO:tasks.rados.rados.0.smithi084.stdout:3629: seq_num 1224 ranges {571768=437609,1808901=711425} 2024-09-18T14:16:36.931 INFO:tasks.rados.rados.0.smithi084.stdout:3629: writing smithi08434663-16 from 571768 to 1009377 tid 1 2024-09-18T14:16:36.940 INFO:tasks.rados.rados.0.smithi084.stdout:3629: writing smithi08434663-16 from 1808901 to 2520326 tid 2 2024-09-18T14:16:36.940 INFO:tasks.rados.rados.0.smithi084.stdout:3630: read oid 10 snap -1 2024-09-18T14:16:36.940 INFO:tasks.rados.rados.0.smithi084.stdout:3630: expect deleted 2024-09-18T14:16:36.940 INFO:tasks.rados.rados.0.smithi084.stdout:3631: write (excl) oid 17 current snap is 401 2024-09-18T14:16:36.940 INFO:tasks.rados.rados.0.smithi084.stdout:3629: finishing write tid 1 to smithi08434663-16 2024-09-18T14:16:36.941 INFO:tasks.rados.rados.0.smithi084.stdout:3631: seq_num 1225 ranges {588575=512858,1867383=455083} 2024-09-18T14:16:36.946 INFO:tasks.rados.rados.0.smithi084.stdout:3631: writing smithi08434663-17 from 588575 to 1101433 tid 1 2024-09-18T14:16:36.954 INFO:tasks.rados.rados.0.smithi084.stdout:3631: writing smithi08434663-17 from 1867383 to 2322466 tid 2 2024-09-18T14:16:36.954 INFO:tasks.rados.rados.0.smithi084.stdout:3632: copy_from oid 31 from oid 25 current snap is 401 2024-09-18T14:16:36.954 INFO:tasks.rados.rados.0.smithi084.stdout:3629: finishing write tid 2 to smithi08434663-16 2024-09-18T14:16:36.954 INFO:tasks.rados.rados.0.smithi084.stdout:3629: finishing write tid 3 to smithi08434663-16 2024-09-18T14:16:36.954 INFO:tasks.rados.rados.0.smithi084.stdout:3629: finishing write tid 4 to smithi08434663-16 2024-09-18T14:16:36.954 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 2037 (ObjNum 1224 snap 401 seq_num 1224) dirty exists 2024-09-18T14:16:36.954 INFO:tasks.rados.rados.0.smithi084.stdout:3629: left oid 16 (ObjNum 1224 snap 401 seq_num 1224) 2024-09-18T14:16:36.954 INFO:tasks.rados.rados.0.smithi084.stdout:3631: finishing write tid 1 to smithi08434663-17 2024-09-18T14:16:36.954 INFO:tasks.rados.rados.0.smithi084.stdout:3629: done (3 left) 2024-09-18T14:16:36.954 INFO:tasks.rados.rados.0.smithi084.stdout:3630: done (2 left) 2024-09-18T14:16:36.954 INFO:tasks.rados.rados.0.smithi084.stdout:3633: rollback oid 28 current snap is 401 2024-09-18T14:16:36.955 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 28 to 369 2024-09-18T14:16:36.955 INFO:tasks.rados.rados.0.smithi084.stdout:3634: copy_from oid 30 from oid 26 current snap is 401 2024-09-18T14:16:36.955 INFO:tasks.rados.rados.0.smithi084.stdout:3635: read oid 19 snap -1 2024-09-18T14:16:36.955 INFO:tasks.rados.rados.0.smithi084.stdout:3635: expect (ObjNum 1156 snap 374 seq_num 1156) 2024-09-18T14:16:36.955 INFO:tasks.rados.rados.0.smithi084.stdout:3636: read oid 11 snap -1 2024-09-18T14:16:36.955 INFO:tasks.rados.rados.0.smithi084.stdout:3636: expect deleted 2024-09-18T14:16:36.955 INFO:tasks.rados.rados.0.smithi084.stdout:3637: read oid 14 snap -1 2024-09-18T14:16:36.955 INFO:tasks.rados.rados.0.smithi084.stdout:3637: expect deleted 2024-09-18T14:16:36.955 INFO:tasks.rados.rados.0.smithi084.stdout:3638: delete oid 27 current snap is 401 2024-09-18T14:16:36.956 INFO:tasks.rados.rados.0.smithi084.stdout:3632: finishing copy_from to smithi08434663-31 2024-09-18T14:16:36.956 INFO:tasks.rados.rados.0.smithi084.stdout:3632: got expected ENOENT (src dne) 2024-09-18T14:16:36.956 INFO:tasks.rados.rados.0.smithi084.stdout:3634: finishing copy_from to smithi08434663-30 2024-09-18T14:16:36.956 INFO:tasks.rados.rados.0.smithi084.stdout:3634: got expected ENOENT (src dne) 2024-09-18T14:16:36.956 INFO:tasks.rados.rados.0.smithi084.stdout:3632: finishing copy_from racing read to smithi08434663-31 2024-09-18T14:16:36.956 INFO:tasks.rados.rados.0.smithi084.stdout:3634: finishing copy_from racing read to smithi08434663-30 2024-09-18T14:16:36.957 INFO:tasks.rados.rados.0.smithi084.stdout:3632: done (7 left) 2024-09-18T14:16:36.957 INFO:tasks.rados.rados.0.smithi084.stdout:3634: done (6 left) 2024-09-18T14:16:36.957 INFO:tasks.rados.rados.0.smithi084.stdout:3636: done (5 left) 2024-09-18T14:16:36.957 INFO:tasks.rados.rados.0.smithi084.stdout:3638: done (4 left) 2024-09-18T14:16:36.957 INFO:tasks.rados.rados.0.smithi084.stdout:3639: write (excl) oid 41 current snap is 401 2024-09-18T14:16:36.957 INFO:tasks.rados.rados.0.smithi084.stdout:3639: seq_num 1227 ranges {622189=663356,1984347=458752} 2024-09-18T14:16:36.963 INFO:tasks.rados.rados.0.smithi084.stdout:3639: writing smithi08434663-41 from 622189 to 1285545 tid 1 2024-09-18T14:16:36.970 INFO:tasks.rados.rados.0.smithi084.stdout:3639: writing smithi08434663-41 from 1984347 to 2443099 tid 2 2024-09-18T14:16:36.970 INFO:tasks.rados.rados.0.smithi084.stdout:3640: snap_remove snap 366 2024-09-18T14:16:37.925 INFO:tasks.rados.rados.0.smithi084.stdout:3640: done (5 left) 2024-09-18T14:16:37.925 INFO:tasks.rados.rados.0.smithi084.stdout:3641: copy_from oid 46 from oid 29 current snap is 401 2024-09-18T14:16:37.925 INFO:tasks.rados.rados.0.smithi084.stdout:3631: finishing write tid 2 to smithi08434663-17 2024-09-18T14:16:37.925 INFO:tasks.rados.rados.0.smithi084.stdout:3631: finishing write tid 3 to smithi08434663-17 2024-09-18T14:16:37.925 INFO:tasks.rados.rados.0.smithi084.stdout:3631: finishing write tid 4 to smithi08434663-17 2024-09-18T14:16:37.925 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1579 (ObjNum 1225 snap 401 seq_num 1225) dirty exists 2024-09-18T14:16:37.925 INFO:tasks.rados.rados.0.smithi084.stdout:3631: left oid 17 (ObjNum 1225 snap 401 seq_num 1225) 2024-09-18T14:16:37.925 INFO:tasks.rados.rados.0.smithi084.stdout:3633: finishing rollback tid 1 to smithi08434663-28 2024-09-18T14:16:37.925 INFO:tasks.rados.rados.0.smithi084.stdout:3633: finishing rollback tid 2 to smithi08434663-28 2024-09-18T14:16:37.925 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 2123 (ObjNum 1072 snap 341 seq_num 1072) dirty exists 2024-09-18T14:16:37.925 INFO:tasks.rados.rados.0.smithi084.stdout:3635: expect (ObjNum 1156 snap 374 seq_num 1156) 2024-09-18T14:16:38.101 INFO:tasks.rados.rados.0.smithi084.stdout:3639: finishing write tid 1 to smithi08434663-41 2024-09-18T14:16:38.101 INFO:tasks.rados.rados.0.smithi084.stdout:3639: finishing write tid 2 to smithi08434663-41 2024-09-18T14:16:38.101 INFO:tasks.rados.rados.0.smithi084.stdout:3639: finishing write tid 3 to smithi08434663-41 2024-09-18T14:16:38.101 INFO:tasks.rados.rados.0.smithi084.stdout:3639: finishing write tid 4 to smithi08434663-41 2024-09-18T14:16:38.101 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 41 v 1904 (ObjNum 1227 snap 401 seq_num 1227) dirty exists 2024-09-18T14:16:38.102 INFO:tasks.rados.rados.0.smithi084.stdout:3639: left oid 41 (ObjNum 1227 snap 401 seq_num 1227) 2024-09-18T14:16:38.102 INFO:tasks.rados.rados.0.smithi084.stdout:3641: finishing copy_from racing read to smithi08434663-46 2024-09-18T14:16:38.102 INFO:tasks.rados.rados.0.smithi084.stdout:3641: finishing copy_from to smithi08434663-46 2024-09-18T14:16:38.102 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 1567 (ObjNum 1148 snap 371 seq_num 1148) dirty exists 2024-09-18T14:16:38.102 INFO:tasks.rados.rados.0.smithi084.stdout:3631: done (5 left) 2024-09-18T14:16:38.102 INFO:tasks.rados.rados.0.smithi084.stdout:3633: done (4 left) 2024-09-18T14:16:38.102 INFO:tasks.rados.rados.0.smithi084.stdout:3635: done (3 left) 2024-09-18T14:16:38.102 INFO:tasks.rados.rados.0.smithi084.stdout:3637: done (2 left) 2024-09-18T14:16:38.102 INFO:tasks.rados.rados.0.smithi084.stdout:3639: done (1 left) 2024-09-18T14:16:38.102 INFO:tasks.rados.rados.0.smithi084.stdout:3641: done (0 left) 2024-09-18T14:16:38.102 INFO:tasks.rados.rados.0.smithi084.stdout:3642: snap_create 2024-09-18T14:16:38.934 INFO:tasks.rados.rados.0.smithi084.stdout:3642: done (0 left) 2024-09-18T14:16:38.934 INFO:tasks.rados.rados.0.smithi084.stdout:3643: snap_remove snap 400 2024-09-18T14:16:39.937 INFO:tasks.rados.rados.0.smithi084.stdout:3643: done (0 left) 2024-09-18T14:16:39.937 INFO:tasks.rados.rados.0.smithi084.stdout:3644: delete oid 29 current snap is 402 2024-09-18T14:16:39.942 INFO:tasks.rados.rados.0.smithi084.stdout:3644: done (0 left) 2024-09-18T14:16:39.943 INFO:tasks.rados.rados.0.smithi084.stdout:3645: write oid 31 current snap is 402 2024-09-18T14:16:39.943 INFO:tasks.rados.rados.0.smithi084.stdout:3645: seq_num 1229 ranges {655803=413854,1584958=462421} 2024-09-18T14:16:39.949 INFO:tasks.rados.rados.0.smithi084.stdout:3645: writing smithi08434663-31 from 655803 to 1069657 tid 1 2024-09-18T14:16:39.955 INFO:tasks.rados.rados.0.smithi084.stdout:3645: writing smithi08434663-31 from 1584958 to 2047379 tid 2 2024-09-18T14:16:39.955 INFO:tasks.rados.rados.0.smithi084.stdout:3646: read oid 29 snap -1 2024-09-18T14:16:39.955 INFO:tasks.rados.rados.0.smithi084.stdout:3646: expect deleted 2024-09-18T14:16:39.955 INFO:tasks.rados.rados.0.smithi084.stdout:3647: snap_create 2024-09-18T14:16:39.955 INFO:tasks.rados.rados.0.smithi084.stdout:3645: finishing write tid 1 to smithi08434663-31 2024-09-18T14:16:39.958 INFO:tasks.rados.rados.0.smithi084.stdout:3645: finishing write tid 2 to smithi08434663-31 2024-09-18T14:16:39.958 INFO:tasks.rados.rados.0.smithi084.stdout:3645: finishing write tid 3 to smithi08434663-31 2024-09-18T14:16:39.959 INFO:tasks.rados.rados.0.smithi084.stdout:3645: finishing write tid 4 to smithi08434663-31 2024-09-18T14:16:39.959 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 1544 (ObjNum 1229 snap 402 seq_num 1229) dirty exists 2024-09-18T14:16:39.959 INFO:tasks.rados.rados.0.smithi084.stdout:3645: left oid 31 (ObjNum 1229 snap 402 seq_num 1229) 2024-09-18T14:16:40.885 INFO:tasks.rados.rados.0.smithi084.stdout:3645: done (2 left) 2024-09-18T14:16:40.885 INFO:tasks.rados.rados.0.smithi084.stdout:3646: done (1 left) 2024-09-18T14:16:40.885 INFO:tasks.rados.rados.0.smithi084.stdout:3647: done (0 left) 2024-09-18T14:16:40.885 INFO:tasks.rados.rados.0.smithi084.stdout:3648: snap_create 2024-09-18T14:16:41.892 INFO:tasks.rados.rados.0.smithi084.stdout:3648: done (0 left) 2024-09-18T14:16:41.892 INFO:tasks.rados.rados.0.smithi084.stdout:3649: read oid 28 snap 403 2024-09-18T14:16:41.892 INFO:tasks.rados.rados.0.smithi084.stdout:3649: expect (ObjNum 1072 snap 341 seq_num 1072) 2024-09-18T14:16:41.892 INFO:tasks.rados.rados.0.smithi084.stdout:3650: snap_create 2024-09-18T14:16:41.896 INFO:tasks.rados.rados.0.smithi084.stdout:3649: expect (ObjNum 1072 snap 341 seq_num 1072) 2024-09-18T14:16:42.897 INFO:tasks.rados.rados.0.smithi084.stdout:3649: done (1 left) 2024-09-18T14:16:42.897 INFO:tasks.rados.rados.0.smithi084.stdout:3650: done (0 left) 2024-09-18T14:16:42.897 INFO:tasks.rados.rados.0.smithi084.stdout:3651: read oid 25 snap -1 2024-09-18T14:16:42.897 INFO:tasks.rados.rados.0.smithi084.stdout:3651: expect deleted 2024-09-18T14:16:42.897 INFO:tasks.rados.rados.0.smithi084.stdout:3652: read oid 49 snap -1 2024-09-18T14:16:42.897 INFO:tasks.rados.rados.0.smithi084.stdout:3652: expect (ObjNum 1163 snap 379 seq_num 1163) 2024-09-18T14:16:42.897 INFO:tasks.rados.rados.0.smithi084.stdout:3653: write oid 36 current snap is 405 2024-09-18T14:16:42.898 INFO:tasks.rados.rados.0.smithi084.stdout:3653: seq_num 1230 ranges {672610=489103,1643440=722432} 2024-09-18T14:16:42.903 INFO:tasks.rados.rados.0.smithi084.stdout:3653: writing smithi08434663-36 from 672610 to 1161713 tid 1 2024-09-18T14:16:42.910 INFO:tasks.rados.rados.0.smithi084.stdout:3653: writing smithi08434663-36 from 1643440 to 2365872 tid 2 2024-09-18T14:16:42.910 INFO:tasks.rados.rados.0.smithi084.stdout:3654: snap_create 2024-09-18T14:16:42.911 INFO:tasks.rados.rados.0.smithi084.stdout:3652: expect (ObjNum 1163 snap 379 seq_num 1163) 2024-09-18T14:16:43.167 INFO:tasks.rados.rados.0.smithi084.stdout:3653: finishing write tid 1 to smithi08434663-36 2024-09-18T14:16:43.167 INFO:tasks.rados.rados.0.smithi084.stdout:3653: finishing write tid 2 to smithi08434663-36 2024-09-18T14:16:43.168 INFO:tasks.rados.rados.0.smithi084.stdout:3653: finishing write tid 3 to smithi08434663-36 2024-09-18T14:16:43.168 INFO:tasks.rados.rados.0.smithi084.stdout:3653: finishing write tid 4 to smithi08434663-36 2024-09-18T14:16:43.168 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 1745 (ObjNum 1230 snap 405 seq_num 1230) dirty exists 2024-09-18T14:16:43.168 INFO:tasks.rados.rados.0.smithi084.stdout:3653: left oid 36 (ObjNum 1230 snap 405 seq_num 1230) 2024-09-18T14:16:43.905 INFO:tasks.rados.rados.0.smithi084.stdout:3651: done (3 left) 2024-09-18T14:16:43.905 INFO:tasks.rados.rados.0.smithi084.stdout:3652: done (2 left) 2024-09-18T14:16:43.905 INFO:tasks.rados.rados.0.smithi084.stdout:3653: done (1 left) 2024-09-18T14:16:43.905 INFO:tasks.rados.rados.0.smithi084.stdout:3654: done (0 left) 2024-09-18T14:16:43.905 INFO:tasks.rados.rados.0.smithi084.stdout:3655: snap_remove snap 371 2024-09-18T14:16:44.916 INFO:tasks.rados.rados.0.smithi084.stdout:3655: done (0 left) 2024-09-18T14:16:44.916 INFO:tasks.rados.rados.0.smithi084.stdout:3656: write oid 22 current snap is 406 2024-09-18T14:16:44.916 INFO:tasks.rados.rados.0.smithi084.stdout:3656: seq_num 1231 ranges {689417=564352,1701922=466090} 2024-09-18T14:16:44.923 INFO:tasks.rados.rados.0.smithi084.stdout:3656: writing smithi08434663-22 from 689417 to 1253769 tid 1 2024-09-18T14:16:44.928 INFO:tasks.rados.rados.0.smithi084.stdout:3656: writing smithi08434663-22 from 1701922 to 2168012 tid 2 2024-09-18T14:16:44.928 INFO:tasks.rados.rados.0.smithi084.stdout:3657: write (excl) oid 15 current snap is 406 2024-09-18T14:16:44.929 INFO:tasks.rados.rados.0.smithi084.stdout:3657: seq_num 1232 ranges {706224=755954,2160404=545820} 2024-09-18T14:16:44.936 INFO:tasks.rados.rados.0.smithi084.stdout:3657: writing smithi08434663-15 from 706224 to 1462178 tid 1 2024-09-18T14:16:44.943 INFO:tasks.rados.rados.0.smithi084.stdout:3657: writing smithi08434663-15 from 2160404 to 2706224 tid 2 2024-09-18T14:16:44.944 INFO:tasks.rados.rados.0.smithi084.stdout:3658: snap_create 2024-09-18T14:16:44.944 INFO:tasks.rados.rados.0.smithi084.stdout:3656: finishing write tid 1 to smithi08434663-22 2024-09-18T14:16:44.944 INFO:tasks.rados.rados.0.smithi084.stdout:3656: finishing write tid 2 to smithi08434663-22 2024-09-18T14:16:44.944 INFO:tasks.rados.rados.0.smithi084.stdout:3656: finishing write tid 3 to smithi08434663-22 2024-09-18T14:16:44.944 INFO:tasks.rados.rados.0.smithi084.stdout:3656: finishing write tid 4 to smithi08434663-22 2024-09-18T14:16:44.944 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 1581 (ObjNum 1231 snap 406 seq_num 1231) dirty exists 2024-09-18T14:16:44.944 INFO:tasks.rados.rados.0.smithi084.stdout:3656: left oid 22 (ObjNum 1231 snap 406 seq_num 1231) 2024-09-18T14:16:44.944 INFO:tasks.rados.rados.0.smithi084.stdout:3657: finishing write tid 1 to smithi08434663-15 2024-09-18T14:16:44.946 INFO:tasks.rados.rados.0.smithi084.stdout:3657: finishing write tid 2 to smithi08434663-15 2024-09-18T14:16:44.947 INFO:tasks.rados.rados.0.smithi084.stdout:3657: finishing write tid 3 to smithi08434663-15 2024-09-18T14:16:44.947 INFO:tasks.rados.rados.0.smithi084.stdout:3657: finishing write tid 4 to smithi08434663-15 2024-09-18T14:16:44.948 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 2040 (ObjNum 1232 snap 406 seq_num 1232) dirty exists 2024-09-18T14:16:44.948 INFO:tasks.rados.rados.0.smithi084.stdout:3657: left oid 15 (ObjNum 1232 snap 406 seq_num 1232) 2024-09-18T14:16:45.887 INFO:tasks.rados.rados.0.smithi084.stdout:3656: done (2 left) 2024-09-18T14:16:45.887 INFO:tasks.rados.rados.0.smithi084.stdout:3657: done (1 left) 2024-09-18T14:16:45.887 INFO:tasks.rados.rados.0.smithi084.stdout:3658: done (0 left) 2024-09-18T14:16:45.887 INFO:tasks.rados.rados.0.smithi084.stdout:3659: snap_remove snap 391 2024-09-18T14:16:46.896 INFO:tasks.rados.rados.0.smithi084.stdout:3659: done (0 left) 2024-09-18T14:16:46.896 INFO:tasks.rados.rados.0.smithi084.stdout:3660: read oid 32 snap 385 2024-09-18T14:16:46.896 INFO:tasks.rados.rados.0.smithi084.stdout:3660: expect deleted 2024-09-18T14:16:46.896 INFO:tasks.rados.rados.0.smithi084.stdout:3661: copy_from oid 44 from oid 20 current snap is 407 2024-09-18T14:16:46.896 INFO:tasks.rados.rados.0.smithi084.stdout:3662: copy_from oid 5 from oid 15 current snap is 407 2024-09-18T14:16:46.896 INFO:tasks.rados.rados.0.smithi084.stdout:3663: copy_from oid 29 from oid 37 current snap is 407 2024-09-18T14:16:46.896 INFO:tasks.rados.rados.0.smithi084.stdout:3664: copy_from oid 43 from oid 16 current snap is 407 2024-09-18T14:16:46.896 INFO:tasks.rados.rados.0.smithi084.stdout:3665: copy_from oid 36 from oid 45 current snap is 407 2024-09-18T14:16:46.896 INFO:tasks.rados.rados.0.smithi084.stdout:3666: snap_create 2024-09-18T14:16:46.897 INFO:tasks.rados.rados.0.smithi084.stdout:3663: finishing copy_from to smithi08434663-29 2024-09-18T14:16:46.897 INFO:tasks.rados.rados.0.smithi084.stdout:3663: got expected ENOENT (src dne) 2024-09-18T14:16:46.898 INFO:tasks.rados.rados.0.smithi084.stdout:3663: finishing copy_from racing read to smithi08434663-29 2024-09-18T14:16:46.899 INFO:tasks.rados.rados.0.smithi084.stdout:3665: finishing copy_from to smithi08434663-36 2024-09-18T14:16:46.899 INFO:tasks.rados.rados.0.smithi084.stdout:3665: got expected ENOENT (src dne) 2024-09-18T14:16:46.899 INFO:tasks.rados.rados.0.smithi084.stdout:3665: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:16:46.911 INFO:tasks.rados.rados.0.smithi084.stdout:3664: finishing copy_from racing read to smithi08434663-43 2024-09-18T14:16:46.912 INFO:tasks.rados.rados.0.smithi084.stdout:3664: finishing copy_from to smithi08434663-43 2024-09-18T14:16:46.912 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 2037 (ObjNum 1224 snap 401 seq_num 1224) dirty exists 2024-09-18T14:16:46.921 INFO:tasks.rados.rados.0.smithi084.stdout:3661: finishing copy_from racing read to smithi08434663-44 2024-09-18T14:16:46.922 INFO:tasks.rados.rados.0.smithi084.stdout:3662: finishing copy_from racing read to smithi08434663-5 2024-09-18T14:16:46.922 INFO:tasks.rados.rados.0.smithi084.stdout:3662: finishing copy_from to smithi08434663-5 2024-09-18T14:16:46.922 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 2040 (ObjNum 1232 snap 406 seq_num 1232) dirty exists 2024-09-18T14:16:46.928 INFO:tasks.rados.rados.0.smithi084.stdout:3661: finishing copy_from to smithi08434663-44 2024-09-18T14:16:46.928 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 1429 (ObjNum 1065 snap 339 seq_num 1065) dirty exists 2024-09-18T14:16:47.896 INFO:tasks.rados.rados.0.smithi084.stdout:3660: done (6 left) 2024-09-18T14:16:47.896 INFO:tasks.rados.rados.0.smithi084.stdout:3661: done (5 left) 2024-09-18T14:16:47.896 INFO:tasks.rados.rados.0.smithi084.stdout:3662: done (4 left) 2024-09-18T14:16:47.896 INFO:tasks.rados.rados.0.smithi084.stdout:3663: done (3 left) 2024-09-18T14:16:47.896 INFO:tasks.rados.rados.0.smithi084.stdout:3664: done (2 left) 2024-09-18T14:16:47.896 INFO:tasks.rados.rados.0.smithi084.stdout:3665: done (1 left) 2024-09-18T14:16:47.896 INFO:tasks.rados.rados.0.smithi084.stdout:3666: done (0 left) 2024-09-18T14:16:47.896 INFO:tasks.rados.rados.0.smithi084.stdout:3667: delete oid 21 current snap is 408 2024-09-18T14:16:47.898 INFO:tasks.rados.rados.0.smithi084.stdout:3667: done (0 left) 2024-09-18T14:16:47.898 INFO:tasks.rados.rados.0.smithi084.stdout:3668: rollback oid 42 current snap is 408 2024-09-18T14:16:47.898 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 42 to 369 2024-09-18T14:16:47.898 INFO:tasks.rados.rados.0.smithi084.stdout:3669: read oid 46 snap -1 2024-09-18T14:16:47.899 INFO:tasks.rados.rados.0.smithi084.stdout:3669: expect (ObjNum 1148 snap 371 seq_num 1148) 2024-09-18T14:16:47.899 INFO:tasks.rados.rados.0.smithi084.stdout:3670: copy_from oid 12 from oid 35 current snap is 408 2024-09-18T14:16:47.899 INFO:tasks.rados.rados.0.smithi084.stdout:3671: snap_remove snap 405 2024-09-18T14:16:48.898 INFO:tasks.rados.rados.0.smithi084.stdout:3671: done (3 left) 2024-09-18T14:16:48.898 INFO:tasks.rados.rados.0.smithi084.stdout:3672: snap_remove snap 385 2024-09-18T14:16:49.900 INFO:tasks.rados.rados.0.smithi084.stdout:3672: done (3 left) 2024-09-18T14:16:49.900 INFO:tasks.rados.rados.0.smithi084.stdout:3673: read oid 27 snap -1 2024-09-18T14:16:49.900 INFO:tasks.rados.rados.0.smithi084.stdout:3673: expect deleted 2024-09-18T14:16:49.900 INFO:tasks.rados.rados.0.smithi084.stdout:3674: write (excl) oid 4 current snap is 408 2024-09-18T14:16:49.900 INFO:tasks.rados.rados.0.smithi084.stdout:3674: seq_num 1234 ranges {739838=506452,1877368=613417} 2024-09-18T14:16:49.907 INFO:tasks.rados.rados.0.smithi084.stdout:3674: writing smithi08434663-4 from 739838 to 1246290 tid 1 2024-09-18T14:16:49.914 INFO:tasks.rados.rados.0.smithi084.stdout:3674: writing smithi08434663-4 from 1877368 to 2490785 tid 2 2024-09-18T14:16:49.915 INFO:tasks.rados.rados.0.smithi084.stdout:3675: read oid 3 snap -1 2024-09-18T14:16:49.915 INFO:tasks.rados.rados.0.smithi084.stdout:3675: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:16:49.915 INFO:tasks.rados.rados.0.smithi084.stdout:3676: read oid 41 snap -1 2024-09-18T14:16:49.915 INFO:tasks.rados.rados.0.smithi084.stdout:3676: expect (ObjNum 1227 snap 401 seq_num 1227) 2024-09-18T14:16:49.915 INFO:tasks.rados.rados.0.smithi084.stdout:3677: snap_create 2024-09-18T14:16:49.915 INFO:tasks.rados.rados.0.smithi084.stdout:3668: finishing rollback tid 0 to smithi08434663-42 2024-09-18T14:16:49.915 INFO:tasks.rados.rados.0.smithi084.stdout:3668: finishing rollback tid 1 to smithi08434663-42 2024-09-18T14:16:49.915 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 1931 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-09-18T14:16:49.915 INFO:tasks.rados.rados.0.smithi084.stdout:3670: finishing copy_from to smithi08434663-12 2024-09-18T14:16:49.915 INFO:tasks.rados.rados.0.smithi084.stdout:3670: got expected ENOENT (src dne) 2024-09-18T14:16:49.915 INFO:tasks.rados.rados.0.smithi084.stdout:3670: finishing copy_from racing read to smithi08434663-12 2024-09-18T14:16:49.915 INFO:tasks.rados.rados.0.smithi084.stdout:3669: expect (ObjNum 1148 snap 371 seq_num 1148) 2024-09-18T14:16:50.062 INFO:tasks.rados.rados.0.smithi084.stdout:3674: finishing write tid 1 to smithi08434663-4 2024-09-18T14:16:50.062 INFO:tasks.rados.rados.0.smithi084.stdout:3674: finishing write tid 2 to smithi08434663-4 2024-09-18T14:16:50.062 INFO:tasks.rados.rados.0.smithi084.stdout:3675: expect (ObjNum 967 snap 308 seq_num 967) 2024-09-18T14:16:50.212 INFO:tasks.rados.rados.0.smithi084.stdout:3674: finishing write tid 3 to smithi08434663-4 2024-09-18T14:16:50.212 INFO:tasks.rados.rados.0.smithi084.stdout:3674: finishing write tid 4 to smithi08434663-4 2024-09-18T14:16:50.212 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1945 (ObjNum 1234 snap 408 seq_num 1234) dirty exists 2024-09-18T14:16:50.212 INFO:tasks.rados.rados.0.smithi084.stdout:3674: left oid 4 (ObjNum 1234 snap 408 seq_num 1234) 2024-09-18T14:16:50.212 INFO:tasks.rados.rados.0.smithi084.stdout:3676: expect (ObjNum 1227 snap 401 seq_num 1227) 2024-09-18T14:16:50.888 INFO:tasks.rados.rados.0.smithi084.stdout:3668: done (7 left) 2024-09-18T14:16:50.888 INFO:tasks.rados.rados.0.smithi084.stdout:3669: done (6 left) 2024-09-18T14:16:50.888 INFO:tasks.rados.rados.0.smithi084.stdout:3670: done (5 left) 2024-09-18T14:16:50.888 INFO:tasks.rados.rados.0.smithi084.stdout:3673: done (4 left) 2024-09-18T14:16:50.888 INFO:tasks.rados.rados.0.smithi084.stdout:3674: done (3 left) 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:3675: done (2 left) 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:3676: done (1 left) 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:3677: done (0 left) 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:3678: read oid 38 snap -1 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:3678: expect (ObjNum 1171 snap 382 seq_num 1171) 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:3679: rollback oid 44 current snap is 409 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 44 to 373 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:3680: read oid 36 snap -1 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:3680: expect (ObjNum 1230 snap 405 seq_num 1230) 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:3681: rollback oid 19 current snap is 409 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 19 to 406 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:3682: read oid 45 snap 360 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:3682: expect deleted 2024-09-18T14:16:50.889 INFO:tasks.rados.rados.0.smithi084.stdout:3683: delete oid 25 current snap is 409 2024-09-18T14:16:50.893 INFO:tasks.rados.rados.0.smithi084.stdout:3679: finishing rollback tid 0 to smithi08434663-44 2024-09-18T14:16:50.898 INFO:tasks.rados.rados.0.smithi084.stdout:3679: finishing rollback tid 1 to smithi08434663-44 2024-09-18T14:16:50.898 INFO:tasks.rados.rados.0.smithi084.stdout:3679: finishing rollback tid 2 to smithi08434663-44 2024-09-18T14:16:50.898 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 2043 (ObjNum 1145 snap 369 seq_num 1145) dirty exists 2024-09-18T14:16:50.898 INFO:tasks.rados.rados.0.smithi084.stdout:3679: done (5 left) 2024-09-18T14:16:50.898 INFO:tasks.rados.rados.0.smithi084.stdout:3682: done (4 left) 2024-09-18T14:16:50.899 INFO:tasks.rados.rados.0.smithi084.stdout:3683: done (3 left) 2024-09-18T14:16:50.899 INFO:tasks.rados.rados.0.smithi084.stdout:3684: delete oid 46 current snap is 409 2024-09-18T14:16:50.903 INFO:tasks.rados.rados.0.smithi084.stdout:3681: finishing rollback tid 0 to smithi08434663-19 2024-09-18T14:16:50.903 INFO:tasks.rados.rados.0.smithi084.stdout:3681: finishing rollback tid 1 to smithi08434663-19 2024-09-18T14:16:50.903 INFO:tasks.rados.rados.0.smithi084.stdout:3681: finishing rollback tid 2 to smithi08434663-19 2024-09-18T14:16:50.903 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 2143 (ObjNum 1156 snap 374 seq_num 1156) dirty exists 2024-09-18T14:16:50.904 INFO:tasks.rados.rados.0.smithi084.stdout:3680: expect (ObjNum 1230 snap 405 seq_num 1230) 2024-09-18T14:16:51.103 INFO:tasks.rados.rados.0.smithi084.stdout:3678: expect (ObjNum 1171 snap 382 seq_num 1171) 2024-09-18T14:16:51.337 INFO:tasks.rados.rados.0.smithi084.stdout:3678: done (3 left) 2024-09-18T14:16:51.338 INFO:tasks.rados.rados.0.smithi084.stdout:3680: done (2 left) 2024-09-18T14:16:51.338 INFO:tasks.rados.rados.0.smithi084.stdout:3681: done (1 left) 2024-09-18T14:16:51.338 INFO:tasks.rados.rados.0.smithi084.stdout:3684: done (0 left) 2024-09-18T14:16:51.338 INFO:tasks.rados.rados.0.smithi084.stdout:3685: write oid 35 current snap is 409 2024-09-18T14:16:51.338 INFO:tasks.rados.rados.0.smithi084.stdout:3685: seq_num 1237 ranges {790259=732199,1936461=477097} 2024-09-18T14:16:51.345 INFO:tasks.rados.rados.0.smithi084.stdout:3685: writing smithi08434663-35 from 790259 to 1522458 tid 1 2024-09-18T14:16:51.350 INFO:tasks.rados.rados.0.smithi084.stdout:3685: writing smithi08434663-35 from 1936461 to 2413558 tid 2 2024-09-18T14:16:51.350 INFO:tasks.rados.rados.0.smithi084.stdout:3686: snap_remove snap 369 2024-09-18T14:16:51.928 INFO:tasks.rados.rados.0.smithi084.stdout:3686: done (1 left) 2024-09-18T14:16:51.928 INFO:tasks.rados.rados.0.smithi084.stdout:3687: rollback oid 46 current snap is 409 2024-09-18T14:16:51.928 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 46 to 397 2024-09-18T14:16:51.928 INFO:tasks.rados.rados.0.smithi084.stdout:3685: finishing write tid 1 to smithi08434663-35 2024-09-18T14:16:51.928 INFO:tasks.rados.rados.0.smithi084.stdout:3685: finishing write tid 2 to smithi08434663-35 2024-09-18T14:16:51.928 INFO:tasks.rados.rados.0.smithi084.stdout:3685: finishing write tid 3 to smithi08434663-35 2024-09-18T14:16:51.928 INFO:tasks.rados.rados.0.smithi084.stdout:3685: finishing write tid 4 to smithi08434663-35 2024-09-18T14:16:51.929 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 2044 (ObjNum 1237 snap 409 seq_num 1237) dirty exists 2024-09-18T14:16:51.929 INFO:tasks.rados.rados.0.smithi084.stdout:3685: left oid 35 (ObjNum 1237 snap 409 seq_num 1237) 2024-09-18T14:16:51.929 INFO:tasks.rados.rados.0.smithi084.stdout:3685: done (1 left) 2024-09-18T14:16:51.929 INFO:tasks.rados.rados.0.smithi084.stdout:3688: copy_from oid 17 from oid 34 current snap is 409 2024-09-18T14:16:51.929 INFO:tasks.rados.rados.0.smithi084.stdout:3689: snap_remove snap 399 2024-09-18T14:16:52.933 INFO:tasks.rados.rados.0.smithi084.stdout:3689: done (2 left) 2024-09-18T14:16:52.933 INFO:tasks.rados.rados.0.smithi084.stdout:3690: copy_from oid 36 from oid 42 current snap is 409 2024-09-18T14:16:52.933 INFO:tasks.rados.rados.0.smithi084.stdout:3687: finishing rollback tid 1 to smithi08434663-46 2024-09-18T14:16:52.933 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 2041 (ObjNum 65 snap 0 seq_num 1953066355) dirty dne 2024-09-18T14:16:52.933 INFO:tasks.rados.rados.0.smithi084.stdout:3688: finishing copy_from racing read to smithi08434663-17 2024-09-18T14:16:52.934 INFO:tasks.rados.rados.0.smithi084.stdout:3688: finishing copy_from to smithi08434663-17 2024-09-18T14:16:52.934 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1870 (ObjNum 1207 snap 396 seq_num 1207) dirty exists 2024-09-18T14:16:52.934 INFO:tasks.rados.rados.0.smithi084.stdout:3687: done (2 left) 2024-09-18T14:16:52.934 INFO:tasks.rados.rados.0.smithi084.stdout:3688: done (1 left) 2024-09-18T14:16:52.934 INFO:tasks.rados.rados.0.smithi084.stdout:3691: write oid 35 current snap is 409 2024-09-18T14:16:52.934 INFO:tasks.rados.rados.0.smithi084.stdout:3691: seq_num 1238 ranges {407066=407448,1594943=620755} 2024-09-18T14:16:52.938 INFO:tasks.rados.rados.0.smithi084.stdout:3691: writing smithi08434663-35 from 407066 to 814514 tid 1 2024-09-18T14:16:52.946 INFO:tasks.rados.rados.0.smithi084.stdout:3691: writing smithi08434663-35 from 1594943 to 2215698 tid 2 2024-09-18T14:16:52.946 INFO:tasks.rados.rados.0.smithi084.stdout:3692: read oid 34 snap -1 2024-09-18T14:16:52.946 INFO:tasks.rados.rados.0.smithi084.stdout:3692: expect (ObjNum 1207 snap 396 seq_num 1207) 2024-09-18T14:16:52.946 INFO:tasks.rados.rados.0.smithi084.stdout:3693: write (excl) oid 14 current snap is 409 2024-09-18T14:16:52.946 INFO:tasks.rados.rados.0.smithi084.stdout:3693: seq_num 1239 ranges {423873=482697,1653425=480766,2691441=132432} 2024-09-18T14:16:52.951 INFO:tasks.rados.rados.0.smithi084.stdout:3693: writing smithi08434663-14 from 423873 to 906570 tid 1 2024-09-18T14:16:52.957 INFO:tasks.rados.rados.0.smithi084.stdout:3693: writing smithi08434663-14 from 1653425 to 2134191 tid 2 2024-09-18T14:16:52.962 INFO:tasks.rados.rados.0.smithi084.stdout:3693: writing smithi08434663-14 from 2691441 to 2823873 tid 3 2024-09-18T14:16:52.962 INFO:tasks.rados.rados.0.smithi084.stdout:3694: delete oid 12 current snap is 409 2024-09-18T14:16:52.962 INFO:tasks.rados.rados.0.smithi084.stdout:3690: finishing copy_from to smithi08434663-36 2024-09-18T14:16:52.962 INFO:tasks.rados.rados.0.smithi084.stdout:3690: got expected ENOENT (src dne) 2024-09-18T14:16:52.962 INFO:tasks.rados.rados.0.smithi084.stdout:3690: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:16:52.962 INFO:tasks.rados.rados.0.smithi084.stdout:3691: finishing write tid 1 to smithi08434663-35 2024-09-18T14:16:52.962 INFO:tasks.rados.rados.0.smithi084.stdout:3691: finishing write tid 2 to smithi08434663-35 2024-09-18T14:16:52.962 INFO:tasks.rados.rados.0.smithi084.stdout:3692: expect (ObjNum 1207 snap 396 seq_num 1207) 2024-09-18T14:16:53.134 INFO:tasks.rados.rados.0.smithi084.stdout:3691: finishing write tid 3 to smithi08434663-35 2024-09-18T14:16:53.135 INFO:tasks.rados.rados.0.smithi084.stdout:3691: finishing write tid 4 to smithi08434663-35 2024-09-18T14:16:53.135 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 2047 (ObjNum 1238 snap 409 seq_num 1238) dirty exists 2024-09-18T14:16:53.135 INFO:tasks.rados.rados.0.smithi084.stdout:3691: left oid 35 (ObjNum 1238 snap 409 seq_num 1238) 2024-09-18T14:16:53.135 INFO:tasks.rados.rados.0.smithi084.stdout:3693: finishing write tid 1 to smithi08434663-14 2024-09-18T14:16:53.135 INFO:tasks.rados.rados.0.smithi084.stdout:3693: finishing write tid 2 to smithi08434663-14 2024-09-18T14:16:53.135 INFO:tasks.rados.rados.0.smithi084.stdout:3693: finishing write tid 3 to smithi08434663-14 2024-09-18T14:16:53.135 INFO:tasks.rados.rados.0.smithi084.stdout:3693: finishing write tid 4 to smithi08434663-14 2024-09-18T14:16:53.135 INFO:tasks.rados.rados.0.smithi084.stdout:3693: finishing write tid 5 to smithi08434663-14 2024-09-18T14:16:53.135 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 2153 (ObjNum 1239 snap 409 seq_num 1239) dirty exists 2024-09-18T14:16:53.135 INFO:tasks.rados.rados.0.smithi084.stdout:3693: left oid 14 (ObjNum 1239 snap 409 seq_num 1239) 2024-09-18T14:16:53.135 INFO:tasks.rados.rados.0.smithi084.stdout:3690: done (4 left) 2024-09-18T14:16:53.135 INFO:tasks.rados.rados.0.smithi084.stdout:3691: done (3 left) 2024-09-18T14:16:53.136 INFO:tasks.rados.rados.0.smithi084.stdout:3692: done (2 left) 2024-09-18T14:16:53.136 INFO:tasks.rados.rados.0.smithi084.stdout:3693: done (1 left) 2024-09-18T14:16:53.136 INFO:tasks.rados.rados.0.smithi084.stdout:3694: done (0 left) 2024-09-18T14:16:53.136 INFO:tasks.rados.rados.0.smithi084.stdout:3695: copy_from oid 18 from oid 2 current snap is 409 2024-09-18T14:16:53.136 INFO:tasks.rados.rados.0.smithi084.stdout:3696: write (excl) oid 11 current snap is 409 2024-09-18T14:16:53.136 INFO:tasks.rados.rados.0.smithi084.stdout:3696: seq_num 1241 ranges {457487=749548,1770389=484435,2746287=111200} 2024-09-18T14:16:53.140 INFO:tasks.rados.rados.0.smithi084.stdout:3696: writing smithi08434663-11 from 457487 to 1207035 tid 1 2024-09-18T14:16:53.146 INFO:tasks.rados.rados.0.smithi084.stdout:3696: writing smithi08434663-11 from 1770389 to 2254824 tid 2 2024-09-18T14:16:53.150 INFO:tasks.rados.rados.0.smithi084.stdout:3696: writing smithi08434663-11 from 2746287 to 2857487 tid 3 2024-09-18T14:16:53.150 INFO:tasks.rados.rados.0.smithi084.stdout:3697: write (excl) oid 17 current snap is 409 2024-09-18T14:16:53.150 INFO:tasks.rados.rados.0.smithi084.stdout:3695: finishing copy_from to smithi08434663-18 2024-09-18T14:16:53.150 INFO:tasks.rados.rados.0.smithi084.stdout:3695: got expected ENOENT (src dne) 2024-09-18T14:16:53.150 INFO:tasks.rados.rados.0.smithi084.stdout:3695: finishing copy_from racing read to smithi08434663-18 2024-09-18T14:16:53.150 INFO:tasks.rados.rados.0.smithi084.stdout:3696: finishing write tid 1 to smithi08434663-11 2024-09-18T14:16:53.150 INFO:tasks.rados.rados.0.smithi084.stdout:3697: seq_num 1242 ranges {474294=424797,1428871=628093,2657357=216937} 2024-09-18T14:16:53.154 INFO:tasks.rados.rados.0.smithi084.stdout:3697: writing smithi08434663-17 from 474294 to 899091 tid 1 2024-09-18T14:16:53.161 INFO:tasks.rados.rados.0.smithi084.stdout:3697: writing smithi08434663-17 from 1428871 to 2056964 tid 2 2024-09-18T14:16:53.166 INFO:tasks.rados.rados.0.smithi084.stdout:3697: writing smithi08434663-17 from 2657357 to 2874294 tid 3 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:3695: done (2 left) 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:3698: rollback oid 40 current snap is 409 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 40 to 397 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:3696: finishing write tid 2 to smithi08434663-11 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:3696: finishing write tid 3 to smithi08434663-11 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:3696: finishing write tid 4 to smithi08434663-11 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:3696: finishing write tid 5 to smithi08434663-11 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 1339 (ObjNum 1241 snap 409 seq_num 1241) dirty exists 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:3696: left oid 11 (ObjNum 1241 snap 409 seq_num 1241) 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:3697: finishing write tid 1 to smithi08434663-17 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:3697: finishing write tid 2 to smithi08434663-17 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:3696: done (2 left) 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:3699: read oid 47 snap 387 2024-09-18T14:16:53.167 INFO:tasks.rados.rados.0.smithi084.stdout:3699: expect (ObjNum 1127 snap 361 seq_num 1127) 2024-09-18T14:16:53.168 INFO:tasks.rados.rados.0.smithi084.stdout:3700: copy_from oid 15 from oid 43 current snap is 409 2024-09-18T14:16:53.168 INFO:tasks.rados.rados.0.smithi084.stdout:3701: snap_remove snap 403 2024-09-18T14:16:53.938 INFO:tasks.rados.rados.0.smithi084.stdout:3701: done (4 left) 2024-09-18T14:16:53.939 INFO:tasks.rados.rados.0.smithi084.stdout:3702: delete oid 24 current snap is 409 2024-09-18T14:16:53.939 INFO:tasks.rados.rados.0.smithi084.stdout:3699: expect (ObjNum 1127 snap 361 seq_num 1127) 2024-09-18T14:16:54.073 INFO:tasks.rados.rados.0.smithi084.stdout:3697: finishing write tid 3 to smithi08434663-17 2024-09-18T14:16:54.073 INFO:tasks.rados.rados.0.smithi084.stdout:3697: finishing write tid 4 to smithi08434663-17 2024-09-18T14:16:54.073 INFO:tasks.rados.rados.0.smithi084.stdout:3697: finishing write tid 5 to smithi08434663-17 2024-09-18T14:16:54.073 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1874 (ObjNum 1242 snap 409 seq_num 1242) dirty exists 2024-09-18T14:16:54.073 INFO:tasks.rados.rados.0.smithi084.stdout:3697: left oid 17 (ObjNum 1242 snap 409 seq_num 1242) 2024-09-18T14:16:54.073 INFO:tasks.rados.rados.0.smithi084.stdout:3698: finishing rollback tid 0 to smithi08434663-40 2024-09-18T14:16:54.073 INFO:tasks.rados.rados.0.smithi084.stdout:3698: finishing rollback tid 1 to smithi08434663-40 2024-09-18T14:16:54.073 INFO:tasks.rados.rados.0.smithi084.stdout:3698: finishing rollback tid 2 to smithi08434663-40 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 40 v 1877 (ObjNum 1072 snap 341 seq_num 1072) dirty exists 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:3700: finishing copy_from racing read to smithi08434663-15 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:3700: finishing copy_from to smithi08434663-15 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 2037 (ObjNum 1224 snap 401 seq_num 1224) dirty exists 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:3697: done (4 left) 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:3698: done (3 left) 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:3699: done (2 left) 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:3700: done (1 left) 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:3702: done (0 left) 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:3703: read oid 1 snap 387 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:3703: expect (ObjNum 1157 snap 374 seq_num 1157) 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:3704: write oid 4 current snap is 409 2024-09-18T14:16:54.074 INFO:tasks.rados.rados.0.smithi084.stdout:3704: seq_num 1244 ranges {507908=575295,1829482=631762} 2024-09-18T14:16:54.078 INFO:tasks.rados.rados.0.smithi084.stdout:3704: writing smithi08434663-4 from 507908 to 1083203 tid 1 2024-09-18T14:16:54.085 INFO:tasks.rados.rados.0.smithi084.stdout:3704: writing smithi08434663-4 from 1829482 to 2461244 tid 2 2024-09-18T14:16:54.086 INFO:tasks.rados.rados.0.smithi084.stdout:3705: snap_create 2024-09-18T14:16:54.086 INFO:tasks.rados.rados.0.smithi084.stdout:3703: expect (ObjNum 1157 snap 374 seq_num 1157) 2024-09-18T14:16:54.194 INFO:tasks.rados.rados.0.smithi084.stdout:3704: finishing write tid 1 to smithi08434663-4 2024-09-18T14:16:54.194 INFO:tasks.rados.rados.0.smithi084.stdout:3704: finishing write tid 2 to smithi08434663-4 2024-09-18T14:16:54.194 INFO:tasks.rados.rados.0.smithi084.stdout:3704: finishing write tid 3 to smithi08434663-4 2024-09-18T14:16:54.194 INFO:tasks.rados.rados.0.smithi084.stdout:3704: finishing write tid 4 to smithi08434663-4 2024-09-18T14:16:54.194 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 1965 (ObjNum 1244 snap 409 seq_num 1244) dirty exists 2024-09-18T14:16:54.194 INFO:tasks.rados.rados.0.smithi084.stdout:3704: left oid 4 (ObjNum 1244 snap 409 seq_num 1244) 2024-09-18T14:16:54.943 INFO:tasks.rados.rados.0.smithi084.stdout:3703: done (2 left) 2024-09-18T14:16:54.944 INFO:tasks.rados.rados.0.smithi084.stdout:3704: done (1 left) 2024-09-18T14:16:54.944 INFO:tasks.rados.rados.0.smithi084.stdout:3705: done (0 left) 2024-09-18T14:16:54.944 INFO:tasks.rados.rados.0.smithi084.stdout:3706: delete oid 9 current snap is 410 2024-09-18T14:16:54.946 INFO:tasks.rados.rados.0.smithi084.stdout:3706: done (0 left) 2024-09-18T14:16:54.946 INFO:tasks.rados.rados.0.smithi084.stdout:3707: snap_remove snap 409 2024-09-18T14:16:55.890 INFO:tasks.rados.rados.0.smithi084.stdout:3707: done (0 left) 2024-09-18T14:16:55.891 INFO:tasks.rados.rados.0.smithi084.stdout:3708: read oid 31 snap -1 2024-09-18T14:16:55.891 INFO:tasks.rados.rados.0.smithi084.stdout:3708: expect (ObjNum 1229 snap 402 seq_num 1229) 2024-09-18T14:16:55.891 INFO:tasks.rados.rados.0.smithi084.stdout:3709: rollback oid 7 current snap is 410 2024-09-18T14:16:55.891 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 7 to 387 2024-09-18T14:16:55.891 INFO:tasks.rados.rados.0.smithi084.stdout:3710: rollback oid 21 current snap is 410 2024-09-18T14:16:55.891 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 21 to 398 2024-09-18T14:16:55.891 INFO:tasks.rados.rados.0.smithi084.stdout:3711: copy_from oid 23 from oid 11 current snap is 410 2024-09-18T14:16:55.891 INFO:tasks.rados.rados.0.smithi084.stdout:3712: copy_from oid 15 from oid 33 current snap is 410 2024-09-18T14:16:55.891 INFO:tasks.rados.rados.0.smithi084.stdout:3713: read oid 14 snap -1 2024-09-18T14:16:55.891 INFO:tasks.rados.rados.0.smithi084.stdout:3713: expect (ObjNum 1239 snap 409 seq_num 1239) 2024-09-18T14:16:55.891 INFO:tasks.rados.rados.0.smithi084.stdout:3714: rollback oid 43 current snap is 410 2024-09-18T14:16:55.891 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 401 2024-09-18T14:16:55.892 INFO:tasks.rados.rados.0.smithi084.stdout:3715: read oid 12 snap 360 2024-09-18T14:16:55.892 INFO:tasks.rados.rados.0.smithi084.stdout:3715: expect (ObjNum 1101 snap 350 seq_num 1101) 2024-09-18T14:16:55.892 INFO:tasks.rados.rados.0.smithi084.stdout:3716: snap_remove snap 406 2024-09-18T14:16:56.905 INFO:tasks.rados.rados.0.smithi084.stdout:3716: done (8 left) 2024-09-18T14:16:56.905 INFO:tasks.rados.rados.0.smithi084.stdout:3717: read oid 39 snap -1 2024-09-18T14:16:56.905 INFO:tasks.rados.rados.0.smithi084.stdout:3717: expect (ObjNum 1200 snap 394 seq_num 1200) 2024-09-18T14:16:56.905 INFO:tasks.rados.rados.0.smithi084.stdout:3718: copy_from oid 24 from oid 28 current snap is 410 2024-09-18T14:16:56.905 INFO:tasks.rados.rados.0.smithi084.stdout:3714: finishing rollback tid 0 to smithi08434663-43 2024-09-18T14:16:56.905 INFO:tasks.rados.rados.0.smithi084.stdout:3709: finishing rollback tid 0 to smithi08434663-7 2024-09-18T14:16:56.905 INFO:tasks.rados.rados.0.smithi084.stdout:3709: finishing rollback tid 1 to smithi08434663-7 2024-09-18T14:16:56.905 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 1549 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:16:56.905 INFO:tasks.rados.rados.0.smithi084.stdout:3714: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:16:56.905 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 2039 (ObjNum 33 snap 0 seq_num 5) dirty dne 2024-09-18T14:16:56.906 INFO:tasks.rados.rados.0.smithi084.stdout:3713: expect (ObjNum 1239 snap 409 seq_num 1239) 2024-09-18T14:16:57.002 INFO:tasks.rados.rados.0.smithi084.stdout:3710: finishing rollback tid 1 to smithi08434663-21 2024-09-18T14:16:57.002 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 1565 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-18T14:16:57.002 INFO:tasks.rados.rados.0.smithi084.stdout:3715: expect (ObjNum 1101 snap 350 seq_num 1101) 2024-09-18T14:16:57.084 INFO:tasks.rados.rados.0.smithi084.stdout:3708: expect (ObjNum 1229 snap 402 seq_num 1229) 2024-09-18T14:16:57.288 INFO:tasks.rados.rados.0.smithi084.stdout:3712: finishing copy_from racing read to smithi08434663-15 2024-09-18T14:16:57.288 INFO:tasks.rados.rados.0.smithi084.stdout:3711: finishing copy_from racing read to smithi08434663-23 2024-09-18T14:16:57.288 INFO:tasks.rados.rados.0.smithi084.stdout:3711: finishing copy_from to smithi08434663-23 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1339 (ObjNum 1241 snap 409 seq_num 1241) dirty exists 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:3712: finishing copy_from to smithi08434663-15 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 1337 (ObjNum 1120 snap 360 seq_num 1120) dirty exists 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:3708: done (9 left) 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:3709: done (8 left) 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:3710: done (7 left) 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:3711: done (6 left) 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:3712: done (5 left) 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:3713: done (4 left) 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:3714: done (3 left) 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:3715: done (2 left) 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:3719: snap_create 2024-09-18T14:16:57.289 INFO:tasks.rados.rados.0.smithi084.stdout:3717: expect (ObjNum 1200 snap 394 seq_num 1200) 2024-09-18T14:16:57.356 INFO:tasks.rados.rados.0.smithi084.stdout:3718: finishing copy_from racing read to smithi08434663-24 2024-09-18T14:16:57.356 INFO:tasks.rados.rados.0.smithi084.stdout:3718: finishing copy_from to smithi08434663-24 2024-09-18T14:16:57.356 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 2123 (ObjNum 1072 snap 341 seq_num 1072) dirty exists 2024-09-18T14:16:57.951 INFO:tasks.rados.rados.0.smithi084.stdout:3717: done (2 left) 2024-09-18T14:16:57.952 INFO:tasks.rados.rados.0.smithi084.stdout:3718: done (1 left) 2024-09-18T14:16:57.952 INFO:tasks.rados.rados.0.smithi084.stdout:3719: done (0 left) 2024-09-18T14:16:57.952 INFO:tasks.rados.rados.0.smithi084.stdout:3720: write (excl) oid 30 current snap is 411 2024-09-18T14:16:57.952 INFO:tasks.rados.rados.0.smithi084.stdout:3720: seq_num 1246 ranges {541522=725793,1946446=635431} 2024-09-18T14:16:57.958 INFO:tasks.rados.rados.0.smithi084.stdout:3720: writing smithi08434663-30 from 541522 to 1267315 tid 1 2024-09-18T14:16:57.965 INFO:tasks.rados.rados.0.smithi084.stdout:3720: writing smithi08434663-30 from 1946446 to 2581877 tid 2 2024-09-18T14:16:57.966 INFO:tasks.rados.rados.0.smithi084.stdout:3721: read oid 12 snap -1 2024-09-18T14:16:57.966 INFO:tasks.rados.rados.0.smithi084.stdout:3721: expect deleted 2024-09-18T14:16:57.966 INFO:tasks.rados.rados.0.smithi084.stdout:3722: read oid 48 snap -1 2024-09-18T14:16:57.966 INFO:tasks.rados.rados.0.smithi084.stdout:3722: expect (ObjNum 1171 snap 382 seq_num 1171) 2024-09-18T14:16:57.966 INFO:tasks.rados.rados.0.smithi084.stdout:3723: write oid 21 current snap is 411 2024-09-18T14:16:57.966 INFO:tasks.rados.rados.0.smithi084.stdout:3720: finishing write tid 1 to smithi08434663-30 2024-09-18T14:16:57.966 INFO:tasks.rados.rados.0.smithi084.stdout:3723: seq_num 1247 ranges {558329=517395,1721281=779089} 2024-09-18T14:16:57.971 INFO:tasks.rados.rados.0.smithi084.stdout:3723: writing smithi08434663-21 from 558329 to 1075724 tid 1 2024-09-18T14:16:57.980 INFO:tasks.rados.rados.0.smithi084.stdout:3723: writing smithi08434663-21 from 1721281 to 2500370 tid 2 2024-09-18T14:16:57.980 INFO:tasks.rados.rados.0.smithi084.stdout:3724: delete oid 50 current snap is 411 2024-09-18T14:16:57.980 INFO:tasks.rados.rados.0.smithi084.stdout:3720: finishing write tid 2 to smithi08434663-30 2024-09-18T14:16:57.980 INFO:tasks.rados.rados.0.smithi084.stdout:3720: finishing write tid 3 to smithi08434663-30 2024-09-18T14:16:57.980 INFO:tasks.rados.rados.0.smithi084.stdout:3720: finishing write tid 4 to smithi08434663-30 2024-09-18T14:16:57.980 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 1281 (ObjNum 1246 snap 411 seq_num 1246) dirty exists 2024-09-18T14:16:57.980 INFO:tasks.rados.rados.0.smithi084.stdout:3720: left oid 30 (ObjNum 1246 snap 411 seq_num 1246) 2024-09-18T14:16:57.980 INFO:tasks.rados.rados.0.smithi084.stdout:3722: expect (ObjNum 1171 snap 382 seq_num 1171) 2024-09-18T14:16:58.216 INFO:tasks.rados.rados.0.smithi084.stdout:3723: finishing write tid 1 to smithi08434663-21 2024-09-18T14:16:58.216 INFO:tasks.rados.rados.0.smithi084.stdout:3720: done (4 left) 2024-09-18T14:16:58.216 INFO:tasks.rados.rados.0.smithi084.stdout:3721: done (3 left) 2024-09-18T14:16:58.216 INFO:tasks.rados.rados.0.smithi084.stdout:3722: done (2 left) 2024-09-18T14:16:58.216 INFO:tasks.rados.rados.0.smithi084.stdout:3724: done (1 left) 2024-09-18T14:16:58.216 INFO:tasks.rados.rados.0.smithi084.stdout:3725: write oid 31 current snap is 411 2024-09-18T14:16:58.217 INFO:tasks.rados.rados.0.smithi084.stdout:3723: finishing write tid 2 to smithi08434663-21 2024-09-18T14:16:58.217 INFO:tasks.rados.rados.0.smithi084.stdout:3723: finishing write tid 3 to smithi08434663-21 2024-09-18T14:16:58.217 INFO:tasks.rados.rados.0.smithi084.stdout:3723: finishing write tid 4 to smithi08434663-21 2024-09-18T14:16:58.217 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 1568 (ObjNum 1247 snap 411 seq_num 1247) dirty exists 2024-09-18T14:16:58.217 INFO:tasks.rados.rados.0.smithi084.stdout:3723: left oid 21 (ObjNum 1247 snap 411 seq_num 1247) 2024-09-18T14:16:58.217 INFO:tasks.rados.rados.0.smithi084.stdout:3725: seq_num 1249 ranges {591943=667893,1721892=782758} 2024-09-18T14:16:58.222 INFO:tasks.rados.rados.0.smithi084.stdout:3725: writing smithi08434663-31 from 591943 to 1259836 tid 1 2024-09-18T14:16:58.229 INFO:tasks.rados.rados.0.smithi084.stdout:3725: writing smithi08434663-31 from 1721892 to 2504650 tid 2 2024-09-18T14:16:58.229 INFO:tasks.rados.rados.0.smithi084.stdout:3723: done (1 left) 2024-09-18T14:16:58.229 INFO:tasks.rados.rados.0.smithi084.stdout:3726: snap_remove snap 386 2024-09-18T14:16:58.958 INFO:tasks.rados.rados.0.smithi084.stdout:3726: done (1 left) 2024-09-18T14:16:58.958 INFO:tasks.rados.rados.0.smithi084.stdout:3727: read oid 1 snap -1 2024-09-18T14:16:58.958 INFO:tasks.rados.rados.0.smithi084.stdout:3727: expect (ObjNum 1214 snap 397 seq_num 1214) 2024-09-18T14:16:58.958 INFO:tasks.rados.rados.0.smithi084.stdout:3728: snap_remove snap 368 2024-09-18T14:16:59.960 INFO:tasks.rados.rados.0.smithi084.stdout:3728: done (2 left) 2024-09-18T14:16:59.960 INFO:tasks.rados.rados.0.smithi084.stdout:3729: delete oid 15 current snap is 411 2024-09-18T14:16:59.960 INFO:tasks.rados.rados.0.smithi084.stdout:3725: finishing write tid 1 to smithi08434663-31 2024-09-18T14:16:59.961 INFO:tasks.rados.rados.0.smithi084.stdout:3725: finishing write tid 2 to smithi08434663-31 2024-09-18T14:16:59.961 INFO:tasks.rados.rados.0.smithi084.stdout:3725: finishing write tid 3 to smithi08434663-31 2024-09-18T14:16:59.961 INFO:tasks.rados.rados.0.smithi084.stdout:3725: finishing write tid 4 to smithi08434663-31 2024-09-18T14:16:59.961 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 2126 (ObjNum 1249 snap 411 seq_num 1249) dirty exists 2024-09-18T14:16:59.961 INFO:tasks.rados.rados.0.smithi084.stdout:3725: left oid 31 (ObjNum 1249 snap 411 seq_num 1249) 2024-09-18T14:16:59.961 INFO:tasks.rados.rados.0.smithi084.stdout:3727: expect (ObjNum 1214 snap 397 seq_num 1214) 2024-09-18T14:17:00.083 INFO:tasks.rados.rados.0.smithi084.stdout:3725: done (2 left) 2024-09-18T14:17:00.083 INFO:tasks.rados.rados.0.smithi084.stdout:3727: done (1 left) 2024-09-18T14:17:00.083 INFO:tasks.rados.rados.0.smithi084.stdout:3729: done (0 left) 2024-09-18T14:17:00.083 INFO:tasks.rados.rados.0.smithi084.stdout:3730: read oid 26 snap -1 2024-09-18T14:17:00.083 INFO:tasks.rados.rados.0.smithi084.stdout:3730: expect deleted 2024-09-18T14:17:00.083 INFO:tasks.rados.rados.0.smithi084.stdout:3731: write oid 9 current snap is 411 2024-09-18T14:17:00.083 INFO:tasks.rados.rados.0.smithi084.stdout:3731: seq_num 1251 ranges {625557=418391,1838856=786427} 2024-09-18T14:17:00.088 INFO:tasks.rados.rados.0.smithi084.stdout:3731: writing smithi08434663-9 from 625557 to 1043948 tid 1 2024-09-18T14:17:00.096 INFO:tasks.rados.rados.0.smithi084.stdout:3731: writing smithi08434663-9 from 1838856 to 2625283 tid 2 2024-09-18T14:17:00.097 INFO:tasks.rados.rados.0.smithi084.stdout:3732: snap_create 2024-09-18T14:17:00.097 INFO:tasks.rados.rados.0.smithi084.stdout:3731: finishing write tid 1 to smithi08434663-9 2024-09-18T14:17:00.100 INFO:tasks.rados.rados.0.smithi084.stdout:3731: finishing write tid 2 to smithi08434663-9 2024-09-18T14:17:00.100 INFO:tasks.rados.rados.0.smithi084.stdout:3731: finishing write tid 3 to smithi08434663-9 2024-09-18T14:17:00.100 INFO:tasks.rados.rados.0.smithi084.stdout:3731: finishing write tid 4 to smithi08434663-9 2024-09-18T14:17:00.100 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 9 v 2052 (ObjNum 1251 snap 411 seq_num 1251) dirty exists 2024-09-18T14:17:00.100 INFO:tasks.rados.rados.0.smithi084.stdout:3731: left oid 9 (ObjNum 1251 snap 411 seq_num 1251) 2024-09-18T14:17:00.892 INFO:tasks.rados.rados.0.smithi084.stdout:3730: done (2 left) 2024-09-18T14:17:00.892 INFO:tasks.rados.rados.0.smithi084.stdout:3731: done (1 left) 2024-09-18T14:17:00.892 INFO:tasks.rados.rados.0.smithi084.stdout:3732: done (0 left) 2024-09-18T14:17:00.892 INFO:tasks.rados.rados.0.smithi084.stdout:3733: read oid 43 snap -1 2024-09-18T14:17:00.892 INFO:tasks.rados.rados.0.smithi084.stdout:3733: expect deleted 2024-09-18T14:17:00.892 INFO:tasks.rados.rados.0.smithi084.stdout:3734: read oid 14 snap -1 2024-09-18T14:17:00.892 INFO:tasks.rados.rados.0.smithi084.stdout:3734: expect (ObjNum 1239 snap 409 seq_num 1239) 2024-09-18T14:17:00.892 INFO:tasks.rados.rados.0.smithi084.stdout:3735: snap_create 2024-09-18T14:17:00.900 INFO:tasks.rados.rados.0.smithi084.stdout:3734: expect (ObjNum 1239 snap 409 seq_num 1239) 2024-09-18T14:17:01.899 INFO:tasks.rados.rados.0.smithi084.stdout:3733: done (2 left) 2024-09-18T14:17:01.899 INFO:tasks.rados.rados.0.smithi084.stdout:3734: done (1 left) 2024-09-18T14:17:01.899 INFO:tasks.rados.rados.0.smithi084.stdout:3735: done (0 left) 2024-09-18T14:17:01.899 INFO:tasks.rados.rados.0.smithi084.stdout:3736: read oid 17 snap -1 2024-09-18T14:17:01.899 INFO:tasks.rados.rados.0.smithi084.stdout:3736: expect (ObjNum 1242 snap 409 seq_num 1242) 2024-09-18T14:17:01.899 INFO:tasks.rados.rados.0.smithi084.stdout:3737: read oid 15 snap -1 2024-09-18T14:17:01.899 INFO:tasks.rados.rados.0.smithi084.stdout:3737: expect deleted 2024-09-18T14:17:01.899 INFO:tasks.rados.rados.0.smithi084.stdout:3738: read oid 47 snap -1 2024-09-18T14:17:01.899 INFO:tasks.rados.rados.0.smithi084.stdout:3738: expect deleted 2024-09-18T14:17:01.899 INFO:tasks.rados.rados.0.smithi084.stdout:3739: copy_from oid 11 from oid 9 current snap is 413 2024-09-18T14:17:01.900 INFO:tasks.rados.rados.0.smithi084.stdout:3740: rollback oid 20 current snap is 413 2024-09-18T14:17:01.900 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 20 to 397 2024-09-18T14:17:01.900 INFO:tasks.rados.rados.0.smithi084.stdout:3741: snap_remove snap 401 2024-09-18T14:17:02.905 INFO:tasks.rados.rados.0.smithi084.stdout:3741: done (5 left) 2024-09-18T14:17:02.905 INFO:tasks.rados.rados.0.smithi084.stdout:3742: copy_from oid 40 from oid 27 current snap is 413 2024-09-18T14:17:02.905 INFO:tasks.rados.rados.0.smithi084.stdout:3740: finishing rollback tid 0 to smithi08434663-20 2024-09-18T14:17:02.905 INFO:tasks.rados.rados.0.smithi084.stdout:3740: finishing rollback tid 1 to smithi08434663-20 2024-09-18T14:17:02.905 INFO:tasks.rados.rados.0.smithi084.stdout:3740: finishing rollback tid 2 to smithi08434663-20 2024-09-18T14:17:02.905 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 20 v 1432 (ObjNum 1171 snap 382 seq_num 1171) dirty exists 2024-09-18T14:17:02.905 INFO:tasks.rados.rados.0.smithi084.stdout:3739: finishing copy_from racing read to smithi08434663-11 2024-09-18T14:17:02.905 INFO:tasks.rados.rados.0.smithi084.stdout:3739: finishing copy_from to smithi08434663-11 2024-09-18T14:17:02.905 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 2052 (ObjNum 1251 snap 411 seq_num 1251) dirty exists 2024-09-18T14:17:02.906 INFO:tasks.rados.rados.0.smithi084.stdout:3736: expect (ObjNum 1242 snap 409 seq_num 1242) 2024-09-18T14:17:03.078 INFO:tasks.rados.rados.0.smithi084.stdout:3742: finishing copy_from to smithi08434663-40 2024-09-18T14:17:03.078 INFO:tasks.rados.rados.0.smithi084.stdout:3742: got expected ENOENT (src dne) 2024-09-18T14:17:03.078 INFO:tasks.rados.rados.0.smithi084.stdout:3742: finishing copy_from racing read to smithi08434663-40 2024-09-18T14:17:03.078 INFO:tasks.rados.rados.0.smithi084.stdout:3736: done (5 left) 2024-09-18T14:17:03.078 INFO:tasks.rados.rados.0.smithi084.stdout:3737: done (4 left) 2024-09-18T14:17:03.078 INFO:tasks.rados.rados.0.smithi084.stdout:3738: done (3 left) 2024-09-18T14:17:03.078 INFO:tasks.rados.rados.0.smithi084.stdout:3739: done (2 left) 2024-09-18T14:17:03.078 INFO:tasks.rados.rados.0.smithi084.stdout:3740: done (1 left) 2024-09-18T14:17:03.078 INFO:tasks.rados.rados.0.smithi084.stdout:3742: done (0 left) 2024-09-18T14:17:03.078 INFO:tasks.rados.rados.0.smithi084.stdout:3743: write oid 22 current snap is 413 2024-09-18T14:17:03.078 INFO:tasks.rados.rados.0.smithi084.stdout:3743: seq_num 1252 ranges {642364=493640,1897338=646438} 2024-09-18T14:17:03.083 INFO:tasks.rados.rados.0.smithi084.stdout:3743: writing smithi08434663-22 from 642364 to 1136004 tid 1 2024-09-18T14:17:03.091 INFO:tasks.rados.rados.0.smithi084.stdout:3743: writing smithi08434663-22 from 1897338 to 2543776 tid 2 2024-09-18T14:17:03.091 INFO:tasks.rados.rados.0.smithi084.stdout:3744: snap_remove snap 392 2024-09-18T14:17:03.354 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-18T14:17:03.355 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-18T14:17:03.355 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-18T14:17:03.355 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-18T14:17:03.358 INFO:tasks.ceph.mon.g:Started 2024-09-18T14:17:03.359 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-09-18T14:17:03.359 INFO:tasks.ceph.mon.a:Sent signal 19 2024-09-18T14:17:03.359 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-18T14:17:03.359 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-18T14:17:03.359 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-18T14:17:08.551 INFO:tasks.rados.rados.0.smithi084.stdout:3743: finishing write tid 1 to smithi08434663-22 2024-09-18T14:17:08.551 INFO:tasks.rados.rados.0.smithi084.stdout:3743: finishing write tid 2 to smithi08434663-22 2024-09-18T14:17:08.551 INFO:tasks.rados.rados.0.smithi084.stdout:3743: finishing write tid 3 to smithi08434663-22 2024-09-18T14:17:08.551 INFO:tasks.rados.rados.0.smithi084.stdout:3743: finishing write tid 4 to smithi08434663-22 2024-09-18T14:17:08.551 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 1584 (ObjNum 1252 snap 413 seq_num 1252) dirty exists 2024-09-18T14:17:08.551 INFO:tasks.rados.rados.0.smithi084.stdout:3743: left oid 22 (ObjNum 1252 snap 413 seq_num 1252) 2024-09-18T14:17:08.551 INFO:tasks.rados.rados.0.smithi084.stdout:3743: done (1 left) 2024-09-18T14:17:08.551 INFO:tasks.rados.rados.0.smithi084.stdout:3744: done (0 left) 2024-09-18T14:17:08.551 INFO:tasks.rados.rados.0.smithi084.stdout:3745: snap_create 2024-09-18T14:17:09.560 INFO:tasks.rados.rados.0.smithi084.stdout:3745: done (0 left) 2024-09-18T14:17:09.561 INFO:tasks.rados.rados.0.smithi084.stdout:3746: read oid 28 snap -1 2024-09-18T14:17:09.561 INFO:tasks.rados.rados.0.smithi084.stdout:3746: expect (ObjNum 1072 snap 341 seq_num 1072) 2024-09-18T14:17:09.564 INFO:tasks.rados.rados.0.smithi084.stdout:3747: rollback oid 6 current snap is 414 2024-09-18T14:17:09.565 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 6 to 410 2024-09-18T14:17:09.565 INFO:tasks.rados.rados.0.smithi084.stdout:3748: write (excl) oid 8 current snap is 414 2024-09-18T14:17:09.565 INFO:tasks.rados.rados.0.smithi084.stdout:3748: seq_num 1253 ranges {659171=568889,1839467=790096} 2024-09-18T14:17:09.568 INFO:tasks.rados.rados.0.smithi084.stdout:3748: writing smithi08434663-8 from 659171 to 1228060 tid 1 2024-09-18T14:17:09.576 INFO:tasks.rados.rados.0.smithi084.stdout:3748: writing smithi08434663-8 from 1839467 to 2629563 tid 2 2024-09-18T14:17:09.576 INFO:tasks.rados.rados.0.smithi084.stdout:3749: snap_create 2024-09-18T14:17:09.576 INFO:tasks.rados.rados.0.smithi084.stdout:3747: finishing rollback tid 1 to smithi08434663-6 2024-09-18T14:17:09.576 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 1537 (ObjNum 129 snap 0 seq_num 2494751760) dirty dne 2024-09-18T14:17:09.576 INFO:tasks.rados.rados.0.smithi084.stdout:3748: finishing write tid 1 to smithi08434663-8 2024-09-18T14:17:09.580 INFO:tasks.rados.rados.0.smithi084.stdout:3748: finishing write tid 2 to smithi08434663-8 2024-09-18T14:17:09.580 INFO:tasks.rados.rados.0.smithi084.stdout:3748: finishing write tid 3 to smithi08434663-8 2024-09-18T14:17:09.581 INFO:tasks.rados.rados.0.smithi084.stdout:3748: finishing write tid 4 to smithi08434663-8 2024-09-18T14:17:09.581 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 1435 (ObjNum 1253 snap 414 seq_num 1253) dirty exists 2024-09-18T14:17:09.581 INFO:tasks.rados.rados.0.smithi084.stdout:3748: left oid 8 (ObjNum 1253 snap 414 seq_num 1253) 2024-09-18T14:17:09.581 INFO:tasks.rados.rados.0.smithi084.stdout:3746: expect (ObjNum 1072 snap 341 seq_num 1072) 2024-09-18T14:17:10.567 INFO:tasks.rados.rados.0.smithi084.stdout:3746: done (3 left) 2024-09-18T14:17:10.567 INFO:tasks.rados.rados.0.smithi084.stdout:3747: done (2 left) 2024-09-18T14:17:10.567 INFO:tasks.rados.rados.0.smithi084.stdout:3748: done (1 left) 2024-09-18T14:17:10.567 INFO:tasks.rados.rados.0.smithi084.stdout:3749: done (0 left) 2024-09-18T14:17:10.567 INFO:tasks.rados.rados.0.smithi084.stdout:3750: rollback oid 44 current snap is 415 2024-09-18T14:17:10.567 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 44 to 335 2024-09-18T14:17:10.567 INFO:tasks.rados.rados.0.smithi084.stdout:3751: snap_create 2024-09-18T14:17:10.571 INFO:tasks.rados.rados.0.smithi084.stdout:3750: finishing rollback tid 0 to smithi08434663-44 2024-09-18T14:17:10.573 INFO:tasks.rados.rados.0.smithi084.stdout:3750: finishing rollback tid 1 to smithi08434663-44 2024-09-18T14:17:10.575 INFO:tasks.rados.rados.0.smithi084.stdout:3750: finishing rollback tid 2 to smithi08434663-44 2024-09-18T14:17:10.576 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 2046 (ObjNum 991 snap 321 seq_num 991) dirty exists 2024-09-18T14:17:10.893 INFO:tasks.rados.rados.0.smithi084.stdout:3750: done (1 left) 2024-09-18T14:17:10.893 INFO:tasks.rados.rados.0.smithi084.stdout:3751: done (0 left) 2024-09-18T14:17:10.893 INFO:tasks.rados.rados.0.smithi084.stdout:3752: read oid 26 snap -1 2024-09-18T14:17:10.893 INFO:tasks.rados.rados.0.smithi084.stdout:3752: expect deleted 2024-09-18T14:17:10.894 INFO:tasks.rados.rados.0.smithi084.stdout:3753: snap_create 2024-09-18T14:17:11.899 INFO:tasks.rados.rados.0.smithi084.stdout:3752: done (1 left) 2024-09-18T14:17:11.899 INFO:tasks.rados.rados.0.smithi084.stdout:3753: done (0 left) 2024-09-18T14:17:11.899 INFO:tasks.rados.rados.0.smithi084.stdout:3754: rollback oid 37 current snap is 417 2024-09-18T14:17:11.899 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 37 to 404 2024-09-18T14:17:11.899 INFO:tasks.rados.rados.0.smithi084.stdout:3755: snap_create 2024-09-18T14:17:12.700 INFO:tasks.rados.rados.0.smithi084.stdout:3754: finishing rollback tid 1 to smithi08434663-37 2024-09-18T14:17:12.700 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 1659 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-18T14:17:12.905 INFO:tasks.rados.rados.0.smithi084.stdout:3754: done (1 left) 2024-09-18T14:17:12.906 INFO:tasks.rados.rados.0.smithi084.stdout:3755: done (0 left) 2024-09-18T14:17:12.906 INFO:tasks.rados.rados.0.smithi084.stdout:3756: snap_remove snap 416 2024-09-18T14:17:13.908 INFO:tasks.rados.rados.0.smithi084.stdout:3756: done (0 left) 2024-09-18T14:17:13.908 INFO:tasks.rados.rados.0.smithi084.stdout:3757: rollback oid 42 current snap is 418 2024-09-18T14:17:13.908 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 42 to 363 2024-09-18T14:17:13.908 INFO:tasks.rados.rados.0.smithi084.stdout:3758: read oid 48 snap -1 2024-09-18T14:17:13.908 INFO:tasks.rados.rados.0.smithi084.stdout:3758: expect (ObjNum 1171 snap 382 seq_num 1171) 2024-09-18T14:17:13.908 INFO:tasks.rados.rados.0.smithi084.stdout:3759: snap_create 2024-09-18T14:17:13.913 INFO:tasks.rados.rados.0.smithi084.stdout:3758: expect (ObjNum 1171 snap 382 seq_num 1171) 2024-09-18T14:17:14.151 INFO:tasks.rados.rados.0.smithi084.stdout:3757: finishing rollback tid 1 to smithi08434663-42 2024-09-18T14:17:14.151 INFO:tasks.rados.rados.0.smithi084.stdout:3757: finishing rollback tid 2 to smithi08434663-42 2024-09-18T14:17:14.151 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 2001 (ObjNum 1054 snap 336 seq_num 1054) dirty exists 2024-09-18T14:17:14.919 INFO:tasks.rados.rados.0.smithi084.stdout:3757: done (2 left) 2024-09-18T14:17:14.919 INFO:tasks.rados.rados.0.smithi084.stdout:3758: done (1 left) 2024-09-18T14:17:14.919 INFO:tasks.rados.rados.0.smithi084.stdout:3759: done (0 left) 2024-09-18T14:17:14.919 INFO:tasks.rados.rados.0.smithi084.stdout:3760: read oid 50 snap -1 2024-09-18T14:17:14.919 INFO:tasks.rados.rados.0.smithi084.stdout:3760: expect deleted 2024-09-18T14:17:14.919 INFO:tasks.rados.rados.0.smithi084.stdout:3761: snap_create 2024-09-18T14:17:15.893 INFO:tasks.rados.rados.0.smithi084.stdout:3760: done (1 left) 2024-09-18T14:17:15.894 INFO:tasks.rados.rados.0.smithi084.stdout:3761: done (0 left) 2024-09-18T14:17:15.894 INFO:tasks.rados.rados.0.smithi084.stdout:3762: read oid 2 snap 335 2024-09-18T14:17:15.894 INFO:tasks.rados.rados.0.smithi084.stdout:3762: expect deleted 2024-09-18T14:17:15.894 INFO:tasks.rados.rados.0.smithi084.stdout:3763: snap_remove snap 373 2024-09-18T14:17:16.904 INFO:tasks.rados.rados.0.smithi084.stdout:3763: done (1 left) 2024-09-18T14:17:16.904 INFO:tasks.rados.rados.0.smithi084.stdout:3764: write oid 23 current snap is 420 2024-09-18T14:17:16.904 INFO:tasks.rados.rados.0.smithi084.stdout:3764: seq_num 1254 ranges {675978=644138,1897949=650107} 2024-09-18T14:17:16.910 INFO:tasks.rados.rados.0.smithi084.stdout:3764: writing smithi08434663-23 from 675978 to 1320116 tid 1 2024-09-18T14:17:16.918 INFO:tasks.rados.rados.0.smithi084.stdout:3764: writing smithi08434663-23 from 1897949 to 2548056 tid 2 2024-09-18T14:17:16.918 INFO:tasks.rados.rados.0.smithi084.stdout:3765: rollback oid 19 current snap is 420 2024-09-18T14:17:16.918 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 19 to 412 2024-09-18T14:17:16.918 INFO:tasks.rados.rados.0.smithi084.stdout:3762: done (2 left) 2024-09-18T14:17:16.918 INFO:tasks.rados.rados.0.smithi084.stdout:3766: delete oid 21 current snap is 420 2024-09-18T14:17:16.922 INFO:tasks.rados.rados.0.smithi084.stdout:3764: finishing write tid 1 to smithi08434663-23 2024-09-18T14:17:16.923 INFO:tasks.rados.rados.0.smithi084.stdout:3764: finishing write tid 2 to smithi08434663-23 2024-09-18T14:17:16.923 INFO:tasks.rados.rados.0.smithi084.stdout:3764: finishing write tid 3 to smithi08434663-23 2024-09-18T14:17:16.923 INFO:tasks.rados.rados.0.smithi084.stdout:3764: finishing write tid 4 to smithi08434663-23 2024-09-18T14:17:16.923 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1552 (ObjNum 1254 snap 420 seq_num 1254) dirty exists 2024-09-18T14:17:16.923 INFO:tasks.rados.rados.0.smithi084.stdout:3764: left oid 23 (ObjNum 1254 snap 420 seq_num 1254) 2024-09-18T14:17:16.925 INFO:tasks.rados.rados.0.smithi084.stdout:3764: done (2 left) 2024-09-18T14:17:16.925 INFO:tasks.rados.rados.0.smithi084.stdout:3766: done (1 left) 2024-09-18T14:17:16.925 INFO:tasks.rados.rados.0.smithi084.stdout:3767: write (excl) oid 12 current snap is 420 2024-09-18T14:17:16.925 INFO:tasks.rados.rados.0.smithi084.stdout:3767: seq_num 1256 ranges {709592=510989,1731266=653776,3041279=68313} 2024-09-18T14:17:16.931 INFO:tasks.rados.rados.0.smithi084.stdout:3767: writing smithi08434663-12 from 709592 to 1220581 tid 1 2024-09-18T14:17:16.938 INFO:tasks.rados.rados.0.smithi084.stdout:3767: writing smithi08434663-12 from 1731266 to 2385042 tid 2 2024-09-18T14:17:16.942 INFO:tasks.rados.rados.0.smithi084.stdout:3767: writing smithi08434663-12 from 3041279 to 3109592 tid 3 2024-09-18T14:17:16.942 INFO:tasks.rados.rados.0.smithi084.stdout:3768: write oid 27 current snap is 420 2024-09-18T14:17:16.942 INFO:tasks.rados.rados.0.smithi084.stdout:3765: finishing rollback tid 0 to smithi08434663-19 2024-09-18T14:17:16.943 INFO:tasks.rados.rados.0.smithi084.stdout:3768: seq_num 1257 ranges {726399=586238,2073395=797434} 2024-09-18T14:17:16.950 INFO:tasks.rados.rados.0.smithi084.stdout:3768: writing smithi08434663-27 from 726399 to 1312637 tid 1 2024-09-18T14:17:16.961 INFO:tasks.rados.rados.0.smithi084.stdout:3768: writing smithi08434663-27 from 2073395 to 2870829 tid 2 2024-09-18T14:17:16.962 INFO:tasks.rados.rados.0.smithi084.stdout:3769: snap_create 2024-09-18T14:17:16.962 INFO:tasks.rados.rados.0.smithi084.stdout:3765: finishing rollback tid 1 to smithi08434663-19 2024-09-18T14:17:16.962 INFO:tasks.rados.rados.0.smithi084.stdout:3765: finishing rollback tid 2 to smithi08434663-19 2024-09-18T14:17:16.962 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 2200 (ObjNum 1156 snap 374 seq_num 1156) dirty exists 2024-09-18T14:17:16.962 INFO:tasks.rados.rados.0.smithi084.stdout:3767: finishing write tid 1 to smithi08434663-12 2024-09-18T14:17:16.962 INFO:tasks.rados.rados.0.smithi084.stdout:3767: finishing write tid 2 to smithi08434663-12 2024-09-18T14:17:16.962 INFO:tasks.rados.rados.0.smithi084.stdout:3767: finishing write tid 3 to smithi08434663-12 2024-09-18T14:17:16.962 INFO:tasks.rados.rados.0.smithi084.stdout:3767: finishing write tid 4 to smithi08434663-12 2024-09-18T14:17:16.962 INFO:tasks.rados.rados.0.smithi084.stdout:3767: finishing write tid 5 to smithi08434663-12 2024-09-18T14:17:16.962 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 2204 (ObjNum 1256 snap 420 seq_num 1256) dirty exists 2024-09-18T14:17:16.962 INFO:tasks.rados.rados.0.smithi084.stdout:3767: left oid 12 (ObjNum 1256 snap 420 seq_num 1256) 2024-09-18T14:17:16.963 INFO:tasks.rados.rados.0.smithi084.stdout:3768: finishing write tid 1 to smithi08434663-27 2024-09-18T14:17:16.977 INFO:tasks.rados.rados.0.smithi084.stdout:3768: finishing write tid 2 to smithi08434663-27 2024-09-18T14:17:16.977 INFO:tasks.rados.rados.0.smithi084.stdout:3768: finishing write tid 3 to smithi08434663-27 2024-09-18T14:17:16.977 INFO:tasks.rados.rados.0.smithi084.stdout:3768: finishing write tid 4 to smithi08434663-27 2024-09-18T14:17:16.977 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 27 v 1786 (ObjNum 1257 snap 420 seq_num 1257) dirty exists 2024-09-18T14:17:16.977 INFO:tasks.rados.rados.0.smithi084.stdout:3768: left oid 27 (ObjNum 1257 snap 420 seq_num 1257) 2024-09-18T14:17:17.952 INFO:tasks.rados.rados.0.smithi084.stdout:3765: done (3 left) 2024-09-18T14:17:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3767: done (2 left) 2024-09-18T14:17:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3768: done (1 left) 2024-09-18T14:17:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3769: done (0 left) 2024-09-18T14:17:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3770: read oid 49 snap -1 2024-09-18T14:17:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3770: expect (ObjNum 1163 snap 379 seq_num 1163) 2024-09-18T14:17:17.953 INFO:tasks.rados.rados.0.smithi084.stdout:3771: delete oid 25 current snap is 421 2024-09-18T14:17:17.957 INFO:tasks.rados.rados.0.smithi084.stdout:3771: done (1 left) 2024-09-18T14:17:17.957 INFO:tasks.rados.rados.0.smithi084.stdout:3772: write (excl) oid 30 current snap is 421 2024-09-18T14:17:17.957 INFO:tasks.rados.rados.0.smithi084.stdout:3772: seq_num 1259 ranges {760013=736736,2190359=401103} 2024-09-18T14:17:17.966 INFO:tasks.rados.rados.0.smithi084.stdout:3772: writing smithi08434663-30 from 760013 to 1496749 tid 1 2024-09-18T14:17:17.972 INFO:tasks.rados.rados.0.smithi084.stdout:3772: writing smithi08434663-30 from 2190359 to 2591462 tid 2 2024-09-18T14:17:17.972 INFO:tasks.rados.rados.0.smithi084.stdout:3773: write (excl) oid 10 current snap is 421 2024-09-18T14:17:17.972 INFO:tasks.rados.rados.0.smithi084.stdout:3773: seq_num 1260 ranges {776820=411985,1848841=544761,2918265=258555} 2024-09-18T14:17:17.978 INFO:tasks.rados.rados.0.smithi084.stdout:3773: writing smithi08434663-10 from 776820 to 1188805 tid 1 2024-09-18T14:17:17.985 INFO:tasks.rados.rados.0.smithi084.stdout:3773: writing smithi08434663-10 from 1848841 to 2393602 tid 2 2024-09-18T14:17:17.993 INFO:tasks.rados.rados.0.smithi084.stdout:3773: writing smithi08434663-10 from 2918265 to 3176820 tid 3 2024-09-18T14:17:17.993 INFO:tasks.rados.rados.0.smithi084.stdout:3774: snap_remove snap 408 2024-09-18T14:17:18.360 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-09-18T14:17:18.361 INFO:tasks.ceph.mon.a:Sent signal 18 2024-09-18T14:17:18.361 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-18T14:17:18.361 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-18T14:17:18.361 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:17:18.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:17:18.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.480+0000 7f1ebca34640 10 monclient: get_monmap_and_config 2024-09-18T14:17:18.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.480+0000 7f1ebca34640 10 monclient: build_initial_monmap 2024-09-18T14:17:18.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.480+0000 7f1ebca34640 10 monclient: monmap: 2024-09-18T14:17:18.481 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:18.481 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.481 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:18.481393+0000 2024-09-18T14:17:18.481 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:18.481393+0000 2024-09-18T14:17:18.481 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:18.481 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.481 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:18.481 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:18.482 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:18.482 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:18.482 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:18.482 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:18.482 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:18.482 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:18.482 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:18.482 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.481+0000 7f1ebca34640 1 Processor -- start 2024-09-18T14:17:18.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.481+0000 7f1ebca34640 1 -- start start 2024-09-18T14:17:18.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.481+0000 7f1ebca34640 10 monclient: init 2024-09-18T14:17:18.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1ebca34640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:18.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1ebca34640 10 monclient: _add_conns ranks=[6,0,5,3,8,1,2,7,4] 2024-09-18T14:17:18.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1ebca34640 10 monclient(hunting): picked mon.noname-d con 0x7f1eb806e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:18.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1ebca34640 10 monclient(hunting): picked mon.noname-a con 0x7f1eb80a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:18.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1ebca34640 10 monclient(hunting): picked mon.noname-h con 0x7f1eb80a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:18.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1ebca34640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb805ed70 con 0x7f1eb80a0cb0 2024-09-18T14:17:18.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1ebca34640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb8097620 con 0x7f1eb80a1080 2024-09-18T14:17:18.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1ebca34640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb8097340 con 0x7f1eb806e810 2024-09-18T14:17:18.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1ebca34640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:18.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1ebca34640 10 monclient(hunting): authenticate will time out at 1853.820221s 2024-09-18T14:17:18.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1eb5d9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1eb80a0cb0 0x7f1eb8137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45818/0 (socket says 172.21.15.45:45818) 2024-09-18T14:17:18.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1eb5d9b640 1 -- 172.21.15.45:0/469998094 learned_addr learned my addr 172.21.15.45:0/469998094 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:18.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.482+0000 7f1eb659c640 1 --1- 172.21.15.45:0/469998094 >> v1:172.21.15.45:6792/0 conn(0x7f1eb806e810 0x7f1eb80a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43034/0 (socket says 172.21.15.45:43034) 2024-09-18T14:17:18.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 1 -- 172.21.15.45:0/469998094 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469461754 0 0) 0x7f1eb805ed70 con 0x7f1eb80a0cb0 2024-09-18T14:17:18.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient(hunting): my global_id is 6345 2024-09-18T14:17:18.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 1 -- 172.21.15.45:0/469998094 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea40034e0 con 0x7f1eb80a0cb0 2024-09-18T14:17:18.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 1 -- 172.21.15.45:0/469998094 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664423435 0 0) 0x7f1eb8097340 con 0x7f1eb806e810 2024-09-18T14:17:18.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient(hunting): my global_id is 184893 2024-09-18T14:17:18.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 1 -- 172.21.15.45:0/469998094 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eb805ed70 con 0x7f1eb806e810 2024-09-18T14:17:18.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 1 -- 172.21.15.45:0/469998094 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3946646331 0 0) 0x7f1ea40034e0 con 0x7f1eb80a0cb0 2024-09-18T14:17:18.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 1 -- 172.21.15.45:0/469998094 >> v1:172.21.15.45:6792/0 conn(0x7f1eb806e810 legacy=0x7f1eb80a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 1 -- 172.21.15.45:0/469998094 >> v1:172.21.15.84:6791/0 conn(0x7f1eb80a1080 legacy=0x7f1eb813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient: _finish_hunting 0 2024-09-18T14:17:18.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 1 monclient: found mon.noname-a 2024-09-18T14:17:18.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 1 -- 172.21.15.45:0/469998094 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb80a1660 con 0x7f1eb80a0cb0 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient: _finish_auth 0 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 1 -- 172.21.15.45:0/469998094 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ea0001760 con 0x7f1eb80a0cb0 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1ebca34640 5 monclient: authenticate success, global_id 6345 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1ebca34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient: dump: 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.491 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1ebca34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 1 -- 172.21.15.45:0/469998094 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1ea00018c0 con 0x7f1eb80a0cb0 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1ebca34640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 1 -- 172.21.15.45:0/469998094 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ea0004860 con 0x7f1eb80a0cb0 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1ebca34640 4 set_mon_vals no callback set 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.486+0000 7f1eb559a640 10 monclient: dump: 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.492 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.487+0000 7f1ebca34640 10 monclient: shutdown 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.487+0000 7f1ebca34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:18.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.487+0000 7f1ebca34640 1 -- 172.21.15.45:0/469998094 >> v1:172.21.15.45:6789/0 conn(0x7f1eb80a0cb0 legacy=0x7f1eb8137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.487+0000 7f1ebca34640 1 -- 172.21.15.45:0/469998094 shutdown_connections 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.487+0000 7f1ebca34640 1 -- 172.21.15.45:0/469998094 >> 172.21.15.45:0/469998094 conn(0x7f1eb80a5bb0 msgr2=0x7f1eb80a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.487+0000 7f1ebca34640 1 -- 172.21.15.45:0/469998094 shutdown_connections 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.487+0000 7f1ebca34640 1 -- 172.21.15.45:0/469998094 wait complete. 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.487+0000 7f1ebca34640 10 monclient: build_initial_monmap 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.487+0000 7f1ebca34640 10 monclient: monmap: 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:18.494 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:18.495 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:18.495 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:18.495 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:18.495 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:18.495 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:18.495 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:18.495 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:18.495 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 1 Processor -- start 2024-09-18T14:17:18.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 1 -- start start 2024-09-18T14:17:18.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 10 monclient: init 2024-09-18T14:17:18.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:18.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 10 monclient: _add_conns ranks=[2,5,1,3,8,7,0,4,6] 2024-09-18T14:17:18.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 10 monclient(hunting): picked mon.noname-b con 0x7f1eb806e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:18.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 10 monclient(hunting): picked mon.noname-h con 0x7f1eb80a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:18.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 10 monclient(hunting): picked mon.noname-f con 0x7f1eb80a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:18.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb8122c00 con 0x7f1eb80a1080 2024-09-18T14:17:18.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb813d9f0 con 0x7f1eb806e810 2024-09-18T14:17:18.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb8122640 con 0x7f1eb80a0cb0 2024-09-18T14:17:18.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:18.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1ebca34640 10 monclient(hunting): authenticate will time out at 1853.826952s 2024-09-18T14:17:18.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1eb659c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f1eb806e810 0x7f1eb813d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60776/0 (socket says 172.21.15.45:60776) 2024-09-18T14:17:18.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.488+0000 7f1eb659c640 1 -- 172.21.15.45:0/1485775846 learned_addr learned my addr 172.21.15.45:0/1485775846 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:18.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968669906 0 0) 0x7f1eb813d9f0 con 0x7f1eb806e810 2024-09-18T14:17:18.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): my global_id is 145478 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e980034e0 con 0x7f1eb806e810 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819982965 0 0) 0x7f1eb8122640 con 0x7f1eb80a0cb0 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): my global_id is 184766 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eb813d9f0 con 0x7f1eb80a0cb0 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065013342 0 0) 0x7f1eb8122c00 con 0x7f1eb80a1080 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): my global_id is 214354 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eb8122640 con 0x7f1eb80a1080 2024-09-18T14:17:18.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 643263866 0 0) 0x7f1e980034e0 con 0x7f1eb806e810 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1eb8122c00 con 0x7f1eb806e810 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ea8001690 con 0x7f1eb806e810 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): dump: 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3271109847 0 0) 0x7f1eb8122640 con 0x7f1eb80a1080 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e980034e0 con 0x7f1eb80a1080 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1eb0001b20 con 0x7f1eb80a1080 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:18.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient(hunting): dump: 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4180158676 0 0) 0x7f1eb8122c00 con 0x7f1eb806e810 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 >> v1:172.21.15.84:6791/0 conn(0x7f1eb80a0cb0 legacy=0x7f1eb8146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 >> v1:172.21.15.84:6789/0 conn(0x7f1eb80a1080 legacy=0x7f1eb8149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 monclient: found mon.b 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb8120150 con 0x7f1eb806e810 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient: _finish_auth 0 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1ebca34640 5 monclient: authenticate success, global_id 145478 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1ebca34640 10 monclient: _renew_subs 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1ebca34640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1ebca34640 1 -- 172.21.15.45:0/1485775846 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb8122360 con 0x7f1eb806e810 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1ebca34640 10 monclient: _renew_subs 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1ebca34640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1ebca34640 1 -- 172.21.15.45:0/1485775846 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb8121490 con 0x7f1eb806e810 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1ea8004a50 con 0x7f1eb806e810 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ea8004f00 con 0x7f1eb806e810 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eae7fc640 10 monclient: dump: 2024-09-18T14:17:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.507 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.489+0000 7f1eb4d99640 4 set_mon_vals no callback set 2024-09-18T14:17:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.490+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1ea800ff50 con 0x7f1eb806e810 2024-09-18T14:17:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.490+0000 7f1ebca34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.490+0000 7f1ebca34640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.490+0000 7f1ebca34640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.490+0000 7f1ebca34640 1 -- 172.21.15.45:0/1485775846 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb804c4a0 con 0x7f1eb806e810 2024-09-18T14:17:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.490+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(845..845 src has 529..845) v4 ==== 3911+0+0 (unknown 449813312 0 0) 0x7f1ea8046ba0 con 0x7f1eb806e810 2024-09-18T14:17:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.491+0000 7f1eb5d9b640 10 monclient: get_auth_request con 0x7f1e9803a530 auth_method 0 2024-09-18T14:17:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.495+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f1ea80153e0 con 0x7f1eb806e810 2024-09-18T14:17:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.495+0000 7f1eae7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.495+0000 7f1eae7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:18.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.719+0000 7f1ebca34640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:17:18.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.719+0000 7f1ebca34640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:17:18.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.719+0000 7f1ebca34640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:18.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.719+0000 7f1ebca34640 1 -- 172.21.15.45:0/1485775846 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1eb804b720 con 0x7f1eb806e810 2024-09-18T14:17:18.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.720+0000 7f1eae7fc640 1 -- 172.21.15.45:0/1485775846 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2689 (unknown 254746657 0 3866858907) 0x7f1ea8014de0 con 0x7f1eb806e810 2024-09-18T14:17:18.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.720+0000 7f1eae7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:17:18.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.720+0000 7f1eae7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:18.721 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":146,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:17:18.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.721+0000 7f1ebca34640 1 -- 172.21.15.45:0/1485775846 >> v1:172.21.15.84:6812/31478 conn(0x7f1e9803a530 legacy=0x7f1e9803c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.721+0000 7f1ebca34640 10 monclient: shutdown 2024-09-18T14:17:18.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.721+0000 7f1ebca34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:18.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.721+0000 7f1ebca34640 1 -- 172.21.15.45:0/1485775846 >> v1:172.21.15.45:6790/0 conn(0x7f1eb806e810 legacy=0x7f1eb813d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.721+0000 7f1ebca34640 1 -- 172.21.15.45:0/1485775846 shutdown_connections 2024-09-18T14:17:18.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.721+0000 7f1ebca34640 1 -- 172.21.15.45:0/1485775846 >> 172.21.15.45:0/1485775846 conn(0x7f1eb80a5bb0 msgr2=0x7f1eb80a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.721+0000 7f1ebca34640 1 -- 172.21.15.45:0/1485775846 shutdown_connections 2024-09-18T14:17:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.721+0000 7f1ebca34640 1 -- 172.21.15.45:0/1485775846 wait complete. 2024-09-18T14:17:18.733 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:17:18.733 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:17:18.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:17:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.829+0000 7fd62346f640 10 monclient: get_monmap_and_config 2024-09-18T14:17:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.829+0000 7fd62346f640 10 monclient: build_initial_monmap 2024-09-18T14:17:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.829+0000 7fd62346f640 10 monclient: monmap: 2024-09-18T14:17:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:18.830553+0000 2024-09-18T14:17:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:18.830553+0000 2024-09-18T14:17:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:18.831 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.831 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:18.831 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:18.831 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:18.831 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:18.831 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:18.831 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:18.831 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:18.831 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:18.831 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:18.831 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.830+0000 7fd62346f640 1 Processor -- start 2024-09-18T14:17:18.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.830+0000 7fd62346f640 1 -- start start 2024-09-18T14:17:18.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.830+0000 7fd62346f640 10 monclient: init 2024-09-18T14:17:18.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.830+0000 7fd62346f640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:18.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.830+0000 7fd62346f640 10 monclient: _add_conns ranks=[8,4,0,1,5,3,6,7,2] 2024-09-18T14:17:18.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.830+0000 7fd62346f640 10 monclient(hunting): picked mon.noname-e con 0x7fd61c09f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:18.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.830+0000 7fd62346f640 10 monclient(hunting): picked mon.noname-c con 0x7fd61c09fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:18.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.830+0000 7fd62346f640 10 monclient(hunting): picked mon.noname-a con 0x7fd61c09ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:18.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.830+0000 7fd62346f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c05ed70 con 0x7fd61c09ff70 2024-09-18T14:17:18.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.830+0000 7fd62346f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c09dfc0 con 0x7fd61c09fba0 2024-09-18T14:17:18.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd62346f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c09dce0 con 0x7fd61c09f7d0 2024-09-18T14:17:18.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd62346f640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd62346f640 10 monclient(hunting): authenticate will time out at 1854.169039s 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd620a0a640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fd61c09fba0 0x7fd61c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:40928/0 (socket says 172.21.15.45:40928) 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd62120b640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fd61c09f7d0 0x7fd61c132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:32982/0 (socket says 172.21.15.45:32982) 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd620a0a640 1 -- 172.21.15.45:0/195423543 learned_addr learned my addr 172.21.15.45:0/195423543 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd621a0c640 1 --1- 172.21.15.45:0/195423543 >> v1:172.21.15.45:6789/0 conn(0x7fd61c09ff70 0x7fd61c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45834/0 (socket says 172.21.15.45:45834) 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679904865 0 0) 0x7fd61c09dce0 con 0x7fd61c09f7d0 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient(hunting): my global_id is 6047 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6080034e0 con 0x7fd61c09f7d0 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120885373 0 0) 0x7fd61c09dfc0 con 0x7fd61c09fba0 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient(hunting): my global_id is 165109 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd61c09dce0 con 0x7fd61c09fba0 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850127538 0 0) 0x7fd61c05ed70 con 0x7fd61c09ff70 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient(hunting): my global_id is 6354 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd61c09dfc0 con 0x7fd61c09ff70 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 384606412 0 0) 0x7fd6080034e0 con 0x7fd61c09f7d0 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 >> v1:172.21.15.45:6791/0 conn(0x7fd61c09fba0 legacy=0x7fd61c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 >> v1:172.21.15.45:6789/0 conn(0x7fd61c09ff70 legacy=0x7fd61c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient: _finish_hunting 0 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 monclient: found mon.noname-e 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:18.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c0a1c00 con 0x7fd61c09f7d0 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient: _finish_auth 0 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd62346f640 5 monclient: authenticate success, global_id 6047 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd618001760 con 0x7fd61c09f7d0 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd62346f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient: dump: 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.844 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd62346f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:18.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd6180018c0 con 0x7fd61c09f7d0 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 1 -- 172.21.15.45:0/195423543 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd6180047a0 con 0x7fd61c09f7d0 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd62346f640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd62346f640 4 set_mon_vals no callback set 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.831+0000 7fd613fff640 10 monclient: dump: 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.846 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.832+0000 7fd62346f640 10 monclient: shutdown 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.832+0000 7fd62346f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.832+0000 7fd62346f640 1 -- 172.21.15.45:0/195423543 >> v1:172.21.15.45:6793/0 conn(0x7fd61c09f7d0 legacy=0x7fd61c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.832+0000 7fd62346f640 1 -- 172.21.15.45:0/195423543 shutdown_connections 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.832+0000 7fd62346f640 1 -- 172.21.15.45:0/195423543 >> 172.21.15.45:0/195423543 conn(0x7fd61c0630b0 msgr2=0x7fd61c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:18.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.832+0000 7fd62346f640 1 -- 172.21.15.45:0/195423543 shutdown_connections 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.832+0000 7fd62346f640 1 -- 172.21.15.45:0/195423543 wait complete. 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.832+0000 7fd62346f640 10 monclient: build_initial_monmap 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.832+0000 7fd62346f640 10 monclient: monmap: 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:18.848 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 1 Processor -- start 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 1 -- start start 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 10 monclient: init 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 10 monclient: _add_conns ranks=[5,0,8,3,6,2,4,7,1] 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 10 monclient(hunting): picked mon.noname-h con 0x7fd61c09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 10 monclient(hunting): picked mon.noname-a con 0x7fd61c09fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:18.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 10 monclient(hunting): picked mon.noname-e con 0x7fd61c09ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:18.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c09d310 con 0x7fd61c09fba0 2024-09-18T14:17:18.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c0a2560 con 0x7fd61c09f7d0 2024-09-18T14:17:18.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c142a30 con 0x7fd61c09ff70 2024-09-18T14:17:18.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:18.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.833+0000 7fd62346f640 10 monclient(hunting): authenticate will time out at 1854.171954s 2024-09-18T14:17:18.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd620a0a640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd61c09fba0 0x7fd61c142320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45836/0 (socket says 172.21.15.45:45836) 2024-09-18T14:17:18.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd620a0a640 1 -- 172.21.15.45:0/999808421 learned_addr learned my addr 172.21.15.45:0/999808421 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:18.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd621a0c640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fd61c09ff70 0x7fd61c149e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:32990/0 (socket says 172.21.15.45:32990) 2024-09-18T14:17:18.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129821406 0 0) 0x7fd61c142a30 con 0x7fd61c09ff70 2024-09-18T14:17:18.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): my global_id is 6056 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6000034e0 con 0x7fd61c09ff70 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036276638 0 0) 0x7fd61c09d310 con 0x7fd61c09fba0 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): my global_id is 6363 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd61c142a30 con 0x7fd61c09fba0 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278115100 0 0) 0x7fd61c0a2560 con 0x7fd61c09f7d0 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): my global_id is 184775 2024-09-18T14:17:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd61c09d310 con 0x7fd61c09f7d0 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3045195280 0 0) 0x7fd61c142a30 con 0x7fd61c09fba0 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd61c0a2560 con 0x7fd61c09fba0 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd60c001570 con 0x7fd61c09fba0 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): dump: 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.854 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1584286258 0 0) 0x7fd6000034e0 con 0x7fd61c09ff70 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd61c142a30 con 0x7fd61c09ff70 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd614001bb0 con 0x7fd61c09ff70 2024-09-18T14:17:18.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): dump: 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.856 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3295452619 0 0) 0x7fd61c09d310 con 0x7fd61c09f7d0 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6000034e0 con 0x7fd61c09f7d0 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd618004820 con 0x7fd61c09f7d0 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient(hunting): dump: 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.857 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.858 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3962727165 0 0) 0x7fd61c0a2560 con 0x7fd61c09fba0 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 >> v1:172.21.15.45:6793/0 conn(0x7fd61c09ff70 legacy=0x7fd61c149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 >> v1:172.21.15.84:6791/0 conn(0x7fd61c09f7d0 legacy=0x7fd61c141c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient: _finish_hunting 0 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 monclient: found mon.a 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c09ab50 con 0x7fd61c09fba0 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient: _finish_auth 0 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd62346f640 5 monclient: authenticate success, global_id 6363 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd62346f640 10 monclient: _renew_subs 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd62346f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:18.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd62346f640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61c09be80 con 0x7fd61c09fba0 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd62346f640 10 monclient: _renew_subs 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd62346f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd62346f640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c099e00 con 0x7fd61c09fba0 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd60c001810 con 0x7fd61c09fba0 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd60c004d90 con 0x7fd61c09fba0 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.834+0000 7fd6137fe640 4 set_mon_vals no callback set 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.835+0000 7fd6117fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.835+0000 7fd6117fa640 10 monclient: dump: 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.860 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.835+0000 7fd62346f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.835+0000 7fd62346f640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd5fc006580 con 0x7fd5fc002830 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.835+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd60c00fde0 con 0x7fd61c09fba0 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(845..845 src has 529..845) v4 ==== 3911+0+0 (unknown 449813312 0 0) 0x7fd60c046a90 con 0x7fd61c09fba0 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717592846 0 0) 0x7fd5fc006580 con 0x7fd5fc002830 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 10 monclient(hunting): my global_id is 6372 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd61c0a2560 con 0x7fd5fc002830 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd621a0c640 10 monclient: get_auth_request con 0x7fd60003ab10 auth_method 0 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3940602432 0 0) 0x7fd61c0a2560 con 0x7fd5fc002830 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd5fc006580 con 0x7fd5fc002830 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd618001180 con 0x7fd5fc002830 2024-09-18T14:17:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 10 monclient: dump: 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.863 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.864 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.864 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.864 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1584343643 0 0) 0x7fd5fc006580 con 0x7fd5fc002830 2024-09-18T14:17:18.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5fc00a0a0 con 0x7fd5fc002830 2024-09-18T14:17:18.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd6180050d0 con 0x7fd5fc002830 2024-09-18T14:17:18.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:18.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:18.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 >> v1:172.21.15.45:6789/0 conn(0x7fd5fc002830 legacy=0x7fd5fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.836+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 >> v1:172.21.15.45:6789/0 conn(0x7fd5fc002830 legacy=0x7fd5fc004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.842+0000 7fd62346f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.842+0000 7fd62346f640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd5fc009660 con 0x7fd5fc009040 2024-09-18T14:17:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.842+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214478718 0 0) 0x7fd5fc009660 con 0x7fd5fc009040 2024-09-18T14:17:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.842+0000 7fd6117fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.842+0000 7fd6117fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.842+0000 7fd6117fa640 10 monclient(hunting): my global_id is 6381 2024-09-18T14:17:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.842+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5fc006580 con 0x7fd5fc009040 2024-09-18T14:17:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2843073153 0 0) 0x7fd5fc006580 con 0x7fd5fc009040 2024-09-18T14:17:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd5fc009660 con 0x7fd5fc009040 2024-09-18T14:17:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd618005870 con 0x7fd5fc009040 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 10 monclient: dump: 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:18.868 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3986229652 0 0) 0x7fd5fc009660 con 0x7fd5fc009040 2024-09-18T14:17:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd5fc009920 con 0x7fd5fc009040 2024-09-18T14:17:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 3261506020) 0x7fd618006c60 con 0x7fd5fc009040 2024-09-18T14:17:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 >> v1:172.21.15.45:6789/0 conn(0x7fd5fc009040 legacy=0x7fd5fc008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd6117fa640 1 -- 172.21.15.45:0/999808421 >> v1:172.21.15.45:6789/0 conn(0x7fd5fc009040 legacy=0x7fd5fc008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:18.872 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:18.873 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:18.876 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:18.877 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.878 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.879 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:18.880 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:18.881 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:18.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.883 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:18.884 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:18.885 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:18.886 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:18.887 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd62346f640 1 -- 172.21.15.45:0/999808421 >> v1:172.21.15.84:6812/31478 conn(0x7fd60003ab10 legacy=0x7fd60003cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd62346f640 10 monclient: shutdown 2024-09-18T14:17:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd62346f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd62346f640 1 -- 172.21.15.45:0/999808421 >> v1:172.21.15.45:6789/0 conn(0x7fd61c09fba0 legacy=0x7fd61c142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd621a0c640 1 -- 172.21.15.45:0/999808421 reap_dead start 2024-09-18T14:17:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.843+0000 7fd621a0c640 1 -- 172.21.15.45:0/999808421 reap_dead start 2024-09-18T14:17:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.844+0000 7fd62346f640 1 -- 172.21.15.45:0/999808421 shutdown_connections 2024-09-18T14:17:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.844+0000 7fd62346f640 1 -- 172.21.15.45:0/999808421 >> 172.21.15.45:0/999808421 conn(0x7fd61c0630b0 msgr2=0x7fd61c130ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.844+0000 7fd62346f640 1 -- 172.21.15.45:0/999808421 shutdown_connections 2024-09-18T14:17:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.844+0000 7fd62346f640 1 -- 172.21.15.45:0/999808421 wait complete. 2024-09-18T14:17:18.890 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:18.890 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:17:18.890 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:17:18.954 INFO:tasks.rados.rados.0.smithi084.stdout:3770: expect (ObjNum 1163 snap 379 seq_num 1163) 2024-09-18T14:17:18.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.994+0000 7ff628509640 10 monclient: get_monmap_and_config 2024-09-18T14:17:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.994+0000 7ff628509640 10 monclient: build_initial_monmap 2024-09-18T14:17:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.994+0000 7ff628509640 10 monclient: monmap: 2024-09-18T14:17:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:18.995195+0000 2024-09-18T14:17:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:18.995195+0000 2024-09-18T14:17:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:18.996 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.994+0000 7ff628509640 1 Processor -- start 2024-09-18T14:17:19.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.994+0000 7ff628509640 1 -- start start 2024-09-18T14:17:19.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.994+0000 7ff628509640 10 monclient: init 2024-09-18T14:17:19.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff628509640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff628509640 10 monclient: _add_conns ranks=[2,4,5,8,7,0,6,3,1] 2024-09-18T14:17:19.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff628509640 10 monclient(hunting): picked mon.noname-b con 0x7ff62006e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:19.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff628509640 10 monclient(hunting): picked mon.noname-c con 0x7ff6200a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:19.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff628509640 10 monclient(hunting): picked mon.noname-h con 0x7ff6200a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:19.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff628509640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62005ed70 con 0x7ff62006e810 2024-09-18T14:17:19.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff628509640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff620097620 con 0x7ff6200a0cb0 2024-09-18T14:17:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff628509640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff620097340 con 0x7ff6200a1080 2024-09-18T14:17:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff628509640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff628509640 10 monclient(hunting): authenticate will time out at 1854.333506s 2024-09-18T14:17:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6262a5640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7ff62006e810 0x7ff6200a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60784/0 (socket says 172.21.15.45:60784) 2024-09-18T14:17:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff625aa4640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7ff6200a0cb0 0x7ff620137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:40940/0 (socket says 172.21.15.45:40940) 2024-09-18T14:17:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6262a5640 1 -- 172.21.15.45:0/87468986 learned_addr learned my addr 172.21.15.45:0/87468986 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177665114 0 0) 0x7ff62005ed70 con 0x7ff62006e810 2024-09-18T14:17:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 10 monclient(hunting): my global_id is 145487 2024-09-18T14:17:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff60c0034e0 con 0x7ff62006e810 2024-09-18T14:17:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665933552 0 0) 0x7ff620097620 con 0x7ff6200a0cb0 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 10 monclient(hunting): my global_id is 165118 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff62005ed70 con 0x7ff6200a0cb0 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565586585 0 0) 0x7ff620097340 con 0x7ff6200a1080 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 10 monclient(hunting): my global_id is 184784 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.995+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff620097620 con 0x7ff6200a1080 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1921558422 0 0) 0x7ff62005ed70 con 0x7ff6200a0cb0 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 >> v1:172.21.15.84:6791/0 conn(0x7ff6200a1080 legacy=0x7ff62013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 >> v1:172.21.15.45:6790/0 conn(0x7ff62006e810 legacy=0x7ff6200a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 1 monclient: found mon.noname-c 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6200a1660 con 0x7ff6200a0cb0 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff628509640 5 monclient: authenticate success, global_id 165118 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff610001790 con 0x7ff6200a0cb0 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff628509640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 10 monclient: dump: 2024-09-18T14:17:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff628509640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff610001970 con 0x7ff6200a0cb0 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff628509640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 1 -- 172.21.15.45:0/87468986 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff6100047a0 con 0x7ff6200a0cb0 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff628509640 4 set_mon_vals no callback set 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff6252a3640 10 monclient: dump: 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.010 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.011 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.011 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.011 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff628509640 10 monclient: shutdown 2024-09-18T14:17:19.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff628509640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff628509640 1 -- 172.21.15.45:0/87468986 >> v1:172.21.15.45:6791/0 conn(0x7ff6200a0cb0 legacy=0x7ff620137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff628509640 1 -- 172.21.15.45:0/87468986 shutdown_connections 2024-09-18T14:17:19.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff628509640 1 -- 172.21.15.45:0/87468986 >> 172.21.15.45:0/87468986 conn(0x7ff6200a5bb0 msgr2=0x7ff6200a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff628509640 1 -- 172.21.15.45:0/87468986 shutdown_connections 2024-09-18T14:17:19.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.996+0000 7ff628509640 1 -- 172.21.15.45:0/87468986 wait complete. 2024-09-18T14:17:19.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.997+0000 7ff628509640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.997+0000 7ff628509640 10 monclient: monmap: 2024-09-18T14:17:19.012 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.012 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:19.012 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:19.012 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:19.012 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.997+0000 7ff628509640 1 Processor -- start 2024-09-18T14:17:19.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.997+0000 7ff628509640 1 -- start start 2024-09-18T14:17:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.997+0000 7ff628509640 10 monclient: init 2024-09-18T14:17:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff628509640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff628509640 10 monclient: _add_conns ranks=[3,1,6,4,7,8,5,0,2] 2024-09-18T14:17:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff628509640 10 monclient(hunting): picked mon.noname-g con 0x7ff62006e810 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff628509640 10 monclient(hunting): picked mon.noname-f con 0x7ff6200a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff628509640 10 monclient(hunting): picked mon.noname-d con 0x7ff6200a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff628509640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff620122c80 con 0x7ff6200a0cb0 2024-09-18T14:17:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff628509640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62013da70 con 0x7ff62006e810 2024-09-18T14:17:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff628509640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6201226c0 con 0x7ff6200a1080 2024-09-18T14:17:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff628509640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff628509640 10 monclient(hunting): authenticate will time out at 1854.336233s 2024-09-18T14:17:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff626aa6640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7ff6200a1080 0x7ff620149cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43040/0 (socket says 172.21.15.45:43040) 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff626aa6640 1 -- 172.21.15.45:0/2226820952 learned_addr learned my addr 172.21.15.45:0/2226820952 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030947701 0 0) 0x7ff6201226c0 con 0x7ff6200a1080 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): my global_id is 184902 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6040034e0 con 0x7ff6200a1080 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436437340 0 0) 0x7ff620122c80 con 0x7ff6200a0cb0 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): my global_id is 214363 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6201226c0 con 0x7ff6200a0cb0 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033158439 0 0) 0x7ff62013da70 con 0x7ff62006e810 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): my global_id is 224103 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff620122c80 con 0x7ff62006e810 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2805810771 0 0) 0x7ff6040034e0 con 0x7ff6200a1080 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff62013da70 con 0x7ff6200a1080 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff618001e20 con 0x7ff6200a1080 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): dump: 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.016 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:19.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4276241688 0 0) 0x7ff6201226c0 con 0x7ff6200a0cb0 2024-09-18T14:17:19.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff6040034e0 con 0x7ff6200a0cb0 2024-09-18T14:17:19.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff610001750 con 0x7ff6200a0cb0 2024-09-18T14:17:19.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:19.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): dump: 2024-09-18T14:17:19.021 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.021 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.021 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.021 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.021 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.021 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.022 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.022 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.022 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.022 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.022 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.022 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.023 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1197698255 0 0) 0x7ff620122c80 con 0x7ff62006e810 2024-09-18T14:17:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff6201226c0 con 0x7ff62006e810 2024-09-18T14:17:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff61c0014e0 con 0x7ff62006e810 2024-09-18T14:17:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.998+0000 7ff6167fc640 10 monclient(hunting): dump: 2024-09-18T14:17:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.025 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.025 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.025 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.025 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.025 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.025 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.026 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.026 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.026 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.026 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.026 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.027 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.027 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.027 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.027 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 464628854 0 0) 0x7ff62013da70 con 0x7ff6200a1080 2024-09-18T14:17:19.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 >> v1:172.21.15.84:6790/0 conn(0x7ff62006e810 legacy=0x7ff62013d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 >> v1:172.21.15.84:6789/0 conn(0x7ff6200a0cb0 legacy=0x7ff620146590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 1 monclient: found mon.d 2024-09-18T14:17:19.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:19.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff620120210 con 0x7ff6200a1080 2024-09-18T14:17:19.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff628509640 5 monclient: authenticate success, global_id 184902 2024-09-18T14:17:19.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff628509640 10 monclient: _renew_subs 2024-09-18T14:17:19.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff628509640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:19.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff628509640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6201223e0 con 0x7ff6200a1080 2024-09-18T14:17:19.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff628509640 10 monclient: _renew_subs 2024-09-18T14:17:19.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff628509640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:19.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff628509640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff620121510 con 0x7ff6200a1080 2024-09-18T14:17:19.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff618002330 con 0x7ff6200a1080 2024-09-18T14:17:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff618005130 con 0x7ff6200a1080 2024-09-18T14:17:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff624aa2640 4 set_mon_vals no callback set 2024-09-18T14:17:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff6167fc640 10 monclient: dump: 2024-09-18T14:17:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.032 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff628509640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:18.999+0000 7ff628509640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff608006580 con 0x7ff608002830 2024-09-18T14:17:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff618010180 con 0x7ff6200a1080 2024-09-18T14:17:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(846..846 src has 529..846) v4 ==== 3955+0+0 (unknown 1173980225 0 0) 0x7ff618046e20 con 0x7ff6200a1080 2024-09-18T14:17:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525602733 0 0) 0x7ff608006580 con 0x7ff608002830 2024-09-18T14:17:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 10 monclient(hunting): my global_id is 145496 2024-09-18T14:17:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff62013da70 con 0x7ff608002830 2024-09-18T14:17:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff625aa4640 10 monclient: get_auth_request con 0x7ff60403a9a0 auth_method 0 2024-09-18T14:17:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1803097393 0 0) 0x7ff62013da70 con 0x7ff608002830 2024-09-18T14:17:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff608006580 con 0x7ff608002830 2024-09-18T14:17:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff61c004ae0 con 0x7ff608002830 2024-09-18T14:17:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 10 monclient: dump: 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.035 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 731524758 0 0) 0x7ff608006580 con 0x7ff608002830 2024-09-18T14:17:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.000+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff60800a0a0 con 0x7ff608002830 2024-09-18T14:17:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.001+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff61c006fc0 con 0x7ff608002830 2024-09-18T14:17:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.001+0000 7ff6167fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.001+0000 7ff6167fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.001+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 >> v1:172.21.15.45:6790/0 conn(0x7ff608002830 legacy=0x7ff608004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.001+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 >> v1:172.21.15.45:6790/0 conn(0x7ff608002830 legacy=0x7ff608004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff628509640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff628509640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff608009660 con 0x7ff608009040 2024-09-18T14:17:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315264948 0 0) 0x7ff608009660 con 0x7ff608009040 2024-09-18T14:17:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 10 monclient(hunting): my global_id is 145505 2024-09-18T14:17:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff608006580 con 0x7ff608009040 2024-09-18T14:17:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2111892553 0 0) 0x7ff608006580 con 0x7ff608009040 2024-09-18T14:17:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff608009660 con 0x7ff608009040 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff61c0055b0 con 0x7ff608009040 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 10 monclient: dump: 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.040 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2442639161 0 0) 0x7ff608009660 con 0x7ff608009040 2024-09-18T14:17:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff608009920 con 0x7ff608009040 2024-09-18T14:17:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 2919651708) 0x7ff61c0069c0 con 0x7ff608009040 2024-09-18T14:17:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 >> v1:172.21.15.45:6790/0 conn(0x7ff608009040 legacy=0x7ff608008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.007+0000 7ff6167fc640 1 -- 172.21.15.45:0/2226820952 >> v1:172.21.15.45:6790/0 conn(0x7ff608009040 legacy=0x7ff608008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.042 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:19.042 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:19.042 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:19.042 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:19.043 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:19.043 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:19.043 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:19.043 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:19.043 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:19.043 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:19.043 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:19.043 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:19.043 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:19.044 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:19.044 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:19.044 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.044 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:17:19.044 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:19.044 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:19.044 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:19.044 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.044 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.044 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.045 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.045 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.045 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.045 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.045 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.045 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.045 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:19.047 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.048 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.054 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:19.054 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.054 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.054 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.054 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:19.054 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:19.054 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.054 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.054 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:19.055 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:19.056 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.056 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.056 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.056 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:19.056 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:19.056 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.057 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.057 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.057 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:19.057 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.057 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.057 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.057 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:19.057 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:17:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:19.070 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:19.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.008+0000 7ff628509640 1 -- 172.21.15.45:0/2226820952 >> v1:172.21.15.84:6812/31478 conn(0x7ff60403a9a0 legacy=0x7ff60403ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.008+0000 7ff628509640 10 monclient: shutdown 2024-09-18T14:17:19.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.008+0000 7ff628509640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.008+0000 7ff628509640 1 -- 172.21.15.45:0/2226820952 >> v1:172.21.15.45:6792/0 conn(0x7ff6200a1080 legacy=0x7ff620149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.008+0000 7ff626aa6640 1 -- 172.21.15.45:0/2226820952 reap_dead start 2024-09-18T14:17:19.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.008+0000 7ff626aa6640 1 -- 172.21.15.45:0/2226820952 reap_dead start 2024-09-18T14:17:19.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.008+0000 7ff628509640 1 -- 172.21.15.45:0/2226820952 shutdown_connections 2024-09-18T14:17:19.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.008+0000 7ff628509640 1 -- 172.21.15.45:0/2226820952 >> 172.21.15.45:0/2226820952 conn(0x7ff6200a5bb0 msgr2=0x7ff6200a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.008+0000 7ff628509640 1 -- 172.21.15.45:0/2226820952 shutdown_connections 2024-09-18T14:17:19.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.008+0000 7ff628509640 1 -- 172.21.15.45:0/2226820952 wait complete. 2024-09-18T14:17:19.074 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:19.074 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:17:19.075 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:17:19.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.171+0000 7f0b45634640 10 monclient: get_monmap_and_config 2024-09-18T14:17:19.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.171+0000 7f0b45634640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.171+0000 7f0b45634640 10 monclient: monmap: 2024-09-18T14:17:19.173 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.173 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.173 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:19.172717+0000 2024-09-18T14:17:19.173 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:19.172717+0000 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 1 Processor -- start 2024-09-18T14:17:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 1 -- start start 2024-09-18T14:17:19.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 10 monclient: init 2024-09-18T14:17:19.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 10 monclient: _add_conns ranks=[6,2,1,8,0,3,4,5,7] 2024-09-18T14:17:19.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 10 monclient(hunting): picked mon.noname-d con 0x7f0b4009f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 10 monclient(hunting): picked mon.noname-b con 0x7f0b4009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 10 monclient(hunting): picked mon.noname-f con 0x7f0b4009ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4005ed70 con 0x7f0b4009ff70 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4009dfc0 con 0x7f0b4009fba0 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4009dce0 con 0x7f0b4009f7d0 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.172+0000 7f0b45634640 10 monclient(hunting): authenticate will time out at 1854.510969s 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3e7fc640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f0b4009fba0 0x7f0b40137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60830/0 (socket says 172.21.15.45:60830) 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3effd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f0b4009f7d0 0x7f0b40132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43050/0 (socket says 172.21.15.45:43050) 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3e7fc640 1 -- 172.21.15.45:0/2797908355 learned_addr learned my addr 172.21.15.45:0/2797908355 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904848335 0 0) 0x7f0b4009dfc0 con 0x7f0b4009fba0 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient(hunting): my global_id is 145514 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b280034e0 con 0x7f0b4009fba0 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378492234 0 0) 0x7f0b4009dce0 con 0x7f0b4009f7d0 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient(hunting): my global_id is 184911 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b4009dfc0 con 0x7f0b4009f7d0 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198885754 0 0) 0x7f0b4005ed70 con 0x7f0b4009ff70 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient(hunting): my global_id is 214372 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b4009dce0 con 0x7f0b4009ff70 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3effd640 1 -- 172.21.15.45:0/2797908355 >> v1:172.21.15.45:6792/0 conn(0x7f0b4009f7d0 legacy=0x7f0b40132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-18T14:17:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3effd640 1 -- 172.21.15.45:0/2797908355 >> v1:172.21.15.45:6792/0 conn(0x7f0b4009f7d0 legacy=0x7f0b40132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3effd640 1 --1- 172.21.15.45:0/2797908355 >> v1:172.21.15.45:6792/0 conn(0x7f0b4009f7d0 0x7f0b40132b30 :-1 s=OPENED pgs=537 cs=1 l=1).handle_message read tag failed 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3effd640 1 --1- 172.21.15.45:0/2797908355 >> v1:172.21.15.45:6792/0 conn(0x7f0b4009f7d0 0x7f0b40132b30 :-1 s=OPENED pgs=537 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.45:6792/0 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2707471281 0 0) 0x7f0b280034e0 con 0x7f0b4009fba0 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 >> v1:172.21.15.45:6792/0 conn(0x7f0b4009f7d0 legacy=0x7f0b40132b30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 >> v1:172.21.15.84:6789/0 conn(0x7f0b4009ff70 legacy=0x7f0b4013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 monclient: found mon.noname-b 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400a1c00 con 0x7f0b4009fba0 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0b34001830 con 0x7f0b4009fba0 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b45634640 5 monclient: authenticate success, global_id 145514 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b45634640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient: dump: 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b45634640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0b34001990 con 0x7f0b4009fba0 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b45634640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 1 -- 172.21.15.45:0/2797908355 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0b34004ba0 con 0x7f0b4009fba0 2024-09-18T14:17:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b45634640 4 set_mon_vals no callback set 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.173+0000 7f0b3dffb640 10 monclient: dump: 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.193 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.174+0000 7f0b45634640 10 monclient: shutdown 2024-09-18T14:17:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.174+0000 7f0b45634640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.174+0000 7f0b45634640 1 -- 172.21.15.45:0/2797908355 >> v1:172.21.15.45:6790/0 conn(0x7f0b4009fba0 legacy=0x7f0b40137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.174+0000 7f0b45634640 1 -- 172.21.15.45:0/2797908355 shutdown_connections 2024-09-18T14:17:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.174+0000 7f0b45634640 1 -- 172.21.15.45:0/2797908355 >> 172.21.15.45:0/2797908355 conn(0x7f0b400630b0 msgr2=0x7f0b400634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.174+0000 7f0b45634640 1 -- 172.21.15.45:0/2797908355 shutdown_connections 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.174+0000 7f0b45634640 1 -- 172.21.15.45:0/2797908355 wait complete. 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.174+0000 7f0b45634640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.174+0000 7f0b45634640 10 monclient: monmap: 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 1 Processor -- start 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 1 -- start start 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 10 monclient: init 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 10 monclient: _add_conns ranks=[5,0,1,2,8,4,7,3,6] 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 10 monclient(hunting): picked mon.noname-h con 0x7f0b4009f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 10 monclient(hunting): picked mon.noname-a con 0x7f0b4009fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 10 monclient(hunting): picked mon.noname-f con 0x7f0b4009ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4009d310 con 0x7f0b4009fba0 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b400a2560 con 0x7f0b4009ff70 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b40125a20 con 0x7f0b4009f7d0 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b45634640 10 monclient(hunting): authenticate will time out at 1854.513914s 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.175+0000 7f0b3e7fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0b4009fba0 0x7f0b4009a650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45868/0 (socket says 172.21.15.45:45868) 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b3e7fc640 1 -- 172.21.15.45:0/488499124 learned_addr learned my addr 172.21.15.45:0/488499124 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b3effd640 1 -- 172.21.15.45:0/488499124 >> v1:172.21.15.84:6791/0 conn(0x7f0b4009f7d0 legacy=0x7f0b40099f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b3effd640 1 -- 172.21.15.45:0/488499124 >> v1:172.21.15.84:6791/0 conn(0x7f0b4009f7d0 legacy=0x7f0b40099f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b3effd640 1 --1- 172.21.15.45:0/488499124 >> v1:172.21.15.84:6791/0 conn(0x7f0b4009f7d0 0x7f0b40099f40 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=1).handle_connect_reply_1 read connect reply failed 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b3effd640 1 --1- 172.21.15.45:0/488499124 >> v1:172.21.15.84:6791/0 conn(0x7f0b4009f7d0 0x7f0b40099f40 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=1).fault on lossy channel, failing 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.84:6791/0 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549736076 0 0) 0x7f0b4009d310 con 0x7f0b4009fba0 2024-09-18T14:17:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): my global_id is 6390 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b2c0034e0 con 0x7f0b4009fba0 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631428057 0 0) 0x7f0b400a2560 con 0x7f0b4009ff70 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): my global_id is 214381 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b4009d310 con 0x7f0b4009ff70 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 375151594 0 0) 0x7f0b4009d310 con 0x7f0b4009ff70 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0b400a2560 con 0x7f0b4009ff70 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0b380018c0 con 0x7f0b4009ff70 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:17:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): dump: 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 32189870 0 0) 0x7f0b2c0034e0 con 0x7f0b4009fba0 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0b4009d310 con 0x7f0b4009fba0 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0b340047a0 con 0x7f0b4009fba0 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient(hunting): dump: 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1378653583 0 0) 0x7f0b400a2560 con 0x7f0b4009ff70 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 >> v1:172.21.15.84:6791/0 conn(0x7f0b4009f7d0 legacy=0x7f0b40099f40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 >> v1:172.21.15.45:6789/0 conn(0x7f0b4009fba0 legacy=0x7f0b4009a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 monclient: found mon.f 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4009ad80 con 0x7f0b4009ff70 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b45634640 5 monclient: authenticate success, global_id 214381 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b45634640 10 monclient: _renew_subs 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b45634640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b45634640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4009ba00 con 0x7f0b4009ff70 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b45634640 10 monclient: _renew_subs 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b45634640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b45634640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b40130b20 con 0x7f0b4009ff70 2024-09-18T14:17:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0b38001e50 con 0x7f0b4009ff70 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0b38002380 con 0x7f0b4009ff70 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b3d7fa640 4 set_mon_vals no callback set 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.176+0000 7f0b26ffd640 10 monclient: dump: 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.177+0000 7f0b45634640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.177+0000 7f0b45634640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0b20006580 con 0x7f0b20002830 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.177+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f0b38002560 con 0x7f0b4009ff70 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.177+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(846..846 src has 529..846) v4 ==== 3955+0+0 (unknown 1173980225 0 0) 0x7f0b38044240 con 0x7f0b4009ff70 2024-09-18T14:17:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b3e7fc640 10 monclient: get_auth_request con 0x7f0b2c03a3c0 auth_method 0 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275707093 0 0) 0x7f0b20006580 con 0x7f0b20002830 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 10 monclient(hunting): my global_id is 165127 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b400a2560 con 0x7f0b20002830 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 113464973 0 0) 0x7f0b400a2560 con 0x7f0b20002830 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0b20006580 con 0x7f0b20002830 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0b30001d20 con 0x7f0b20002830 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 10 monclient: dump: 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3213272624 0 0) 0x7f0b20006580 con 0x7f0b20002830 2024-09-18T14:17:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b2000a0a0 con 0x7f0b20002830 2024-09-18T14:17:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0b30005b50 con 0x7f0b20002830 2024-09-18T14:17:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 >> v1:172.21.15.45:6791/0 conn(0x7f0b20002830 legacy=0x7f0b20004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.178+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 >> v1:172.21.15.45:6791/0 conn(0x7f0b20002830 legacy=0x7f0b20004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.184+0000 7f0b45634640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.184+0000 7f0b45634640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0b20009660 con 0x7f0b20009040 2024-09-18T14:17:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.184+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845522836 0 0) 0x7f0b20009660 con 0x7f0b20009040 2024-09-18T14:17:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.184+0000 7f0b26ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.184+0000 7f0b26ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.184+0000 7f0b26ffd640 10 monclient(hunting): my global_id is 165136 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.184+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b20006580 con 0x7f0b20009040 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.184+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1131132976 0 0) 0x7f0b20006580 con 0x7f0b20009040 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.184+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0b20009660 con 0x7f0b20009040 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.184+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0b30005090 con 0x7f0b20009040 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.184+0000 7f0b26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b26ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b26ffd640 10 monclient: dump: 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2575115092 0 0) 0x7f0b20009660 con 0x7f0b20009040 2024-09-18T14:17:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0b20009920 con 0x7f0b20009040 2024-09-18T14:17:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 3907957310) 0x7f0b300053b0 con 0x7f0b20009040 2024-09-18T14:17:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b26ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b26ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 >> v1:172.21.15.45:6791/0 conn(0x7f0b20009040 legacy=0x7f0b20008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b26ffd640 1 -- 172.21.15.45:0/488499124 >> v1:172.21.15.45:6791/0 conn(0x7f0b20009040 legacy=0x7f0b20008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.209 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:19.230 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:19.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b45634640 1 -- 172.21.15.45:0/488499124 >> v1:172.21.15.84:6812/31478 conn(0x7f0b2c03a3c0 legacy=0x7f0b2c03c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b45634640 10 monclient: shutdown 2024-09-18T14:17:19.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b45634640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b45634640 1 -- 172.21.15.45:0/488499124 >> v1:172.21.15.84:6789/0 conn(0x7f0b4009ff70 legacy=0x7f0b4014e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b3f7fe640 1 -- 172.21.15.45:0/488499124 reap_dead start 2024-09-18T14:17:19.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b3f7fe640 1 -- 172.21.15.45:0/488499124 reap_dead start 2024-09-18T14:17:19.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b45634640 1 -- 172.21.15.45:0/488499124 shutdown_connections 2024-09-18T14:17:19.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b45634640 1 -- 172.21.15.45:0/488499124 >> 172.21.15.45:0/488499124 conn(0x7f0b400630b0 msgr2=0x7f0b4006dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b45634640 1 -- 172.21.15.45:0/488499124 shutdown_connections 2024-09-18T14:17:19.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.185+0000 7f0b45634640 1 -- 172.21.15.45:0/488499124 wait complete. 2024-09-18T14:17:19.233 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:19.233 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:17:19.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:17:19.234 INFO:tasks.rados.rados.0.smithi084.stdout:3772: finishing write tid 1 to smithi08434663-30 2024-09-18T14:17:19.234 INFO:tasks.rados.rados.0.smithi084.stdout:3772: finishing write tid 2 to smithi08434663-30 2024-09-18T14:17:19.234 INFO:tasks.rados.rados.0.smithi084.stdout:3772: finishing write tid 3 to smithi08434663-30 2024-09-18T14:17:19.234 INFO:tasks.rados.rados.0.smithi084.stdout:3772: finishing write tid 4 to smithi08434663-30 2024-09-18T14:17:19.234 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 1284 (ObjNum 1259 snap 421 seq_num 1259) dirty exists 2024-09-18T14:17:19.234 INFO:tasks.rados.rados.0.smithi084.stdout:3772: left oid 30 (ObjNum 1259 snap 421 seq_num 1259) 2024-09-18T14:17:19.234 INFO:tasks.rados.rados.0.smithi084.stdout:3773: finishing write tid 1 to smithi08434663-10 2024-09-18T14:17:19.234 INFO:tasks.rados.rados.0.smithi084.stdout:3770: done (3 left) 2024-09-18T14:17:19.234 INFO:tasks.rados.rados.0.smithi084.stdout:3772: done (2 left) 2024-09-18T14:17:19.234 INFO:tasks.rados.rados.0.smithi084.stdout:3774: done (1 left) 2024-09-18T14:17:19.234 INFO:tasks.rados.rados.0.smithi084.stdout:3775: delete oid 29 current snap is 421 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3773: finishing write tid 2 to smithi08434663-10 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3773: finishing write tid 3 to smithi08434663-10 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3773: finishing write tid 4 to smithi08434663-10 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3773: finishing write tid 5 to smithi08434663-10 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 10 v 1790 (ObjNum 1260 snap 421 seq_num 1260) dirty exists 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3773: left oid 10 (ObjNum 1260 snap 421 seq_num 1260) 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3773: done (1 left) 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3775: done (0 left) 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3776: read oid 1 snap -1 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3776: expect (ObjNum 1214 snap 397 seq_num 1214) 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3777: write oid 17 current snap is 421 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3777: seq_num 1262 ranges {410434=678836,1565805=548430,2573111=637323} 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3777: writing smithi08434663-17 from 410434 to 1089270 tid 1 2024-09-18T14:17:19.235 INFO:tasks.rados.rados.0.smithi084.stdout:3777: writing smithi08434663-17 from 1565805 to 2114235 tid 2 2024-09-18T14:17:19.236 INFO:tasks.rados.rados.0.smithi084.stdout:3777: writing smithi08434663-17 from 2573111 to 3210434 tid 3 2024-09-18T14:17:19.237 INFO:tasks.rados.rados.0.smithi084.stdout:3778: read oid 5 snap -1 2024-09-18T14:17:19.237 INFO:tasks.rados.rados.0.smithi084.stdout:3778: expect (ObjNum 1232 snap 406 seq_num 1232) 2024-09-18T14:17:19.237 INFO:tasks.rados.rados.0.smithi084.stdout:3779: delete oid 30 current snap is 421 2024-09-18T14:17:19.237 INFO:tasks.rados.rados.0.smithi084.stdout:3776: expect (ObjNum 1214 snap 397 seq_num 1214) 2024-09-18T14:17:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.332+0000 7f901bb54640 10 monclient: get_monmap_and_config 2024-09-18T14:17:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.332+0000 7f901bb54640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.332+0000 7f901bb54640 10 monclient: monmap: 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:19.333099+0000 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:19.333099+0000 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.334 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.332+0000 7f901bb54640 1 Processor -- start 2024-09-18T14:17:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.332+0000 7f901bb54640 1 -- start start 2024-09-18T14:17:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.332+0000 7f901bb54640 10 monclient: init 2024-09-18T14:17:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f901bb54640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f901bb54640 10 monclient: _add_conns ranks=[5,6,0,3,2,4,7,8,1] 2024-09-18T14:17:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f901bb54640 10 monclient(hunting): picked mon.noname-h con 0x7f901406e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f901bb54640 10 monclient(hunting): picked mon.noname-d con 0x7f90140a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f901bb54640 10 monclient(hunting): picked mon.noname-a con 0x7f90140a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f901bb54640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901405ed70 con 0x7f90140a1080 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f901bb54640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9014097620 con 0x7f901406e810 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f901bb54640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9014097340 con 0x7f90140a0cb0 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f901bb54640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f901bb54640 10 monclient(hunting): authenticate will time out at 1854.671511s 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90190ef640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f90140a0cb0 0x7f9014137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43064/0 (socket says 172.21.15.45:43064) 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90190ef640 1 -- 172.21.15.45:0/3396045318 learned_addr learned my addr 172.21.15.45:0/3396045318 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f901a0f1640 1 --1- 172.21.15.45:0/3396045318 >> v1:172.21.15.45:6789/0 conn(0x7f90140a1080 0x7f901413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45872/0 (socket says 172.21.15.45:45872) 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356467217 0 0) 0x7f901405ed70 con 0x7f90140a1080 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 10 monclient(hunting): my global_id is 6399 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90000034e0 con 0x7f90140a1080 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110034759 0 0) 0x7f9014097340 con 0x7f90140a0cb0 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 10 monclient(hunting): my global_id is 184920 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f901405ed70 con 0x7f90140a0cb0 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825365903 0 0) 0x7f9014097620 con 0x7f901406e810 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 10 monclient(hunting): my global_id is 184793 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.333+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9014097340 con 0x7f901406e810 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90198f0640 1 -- 172.21.15.45:0/3396045318 >> v1:172.21.15.84:6791/0 conn(0x7f901406e810 legacy=0x7f90140a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-18T14:17:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90198f0640 1 -- 172.21.15.45:0/3396045318 >> v1:172.21.15.84:6791/0 conn(0x7f901406e810 legacy=0x7f90140a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90198f0640 1 --1- 172.21.15.45:0/3396045318 >> v1:172.21.15.84:6791/0 conn(0x7f901406e810 0x7f90140a05a0 :-1 s=OPENED pgs=524 cs=1 l=1).handle_message read tag failed 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90198f0640 1 --1- 172.21.15.45:0/3396045318 >> v1:172.21.15.84:6791/0 conn(0x7f901406e810 0x7f90140a05a0 :-1 s=OPENED pgs=524 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.84:6791/0 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2999595731 0 0) 0x7f901405ed70 con 0x7f90140a0cb0 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 >> v1:172.21.15.84:6791/0 conn(0x7f901406e810 legacy=0x7f90140a05a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 >> v1:172.21.15.45:6789/0 conn(0x7f90140a1080 legacy=0x7f901413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 1 monclient: found mon.noname-d 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90140a1660 con 0x7f90140a0cb0 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9004001810 con 0x7f90140a0cb0 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f901bb54640 5 monclient: authenticate success, global_id 184920 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f901bb54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 10 monclient: dump: 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f901bb54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9004001970 con 0x7f90140a0cb0 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f901bb54640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 1 -- 172.21.15.45:0/3396045318 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9004004ba0 con 0x7f90140a0cb0 2024-09-18T14:17:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f901bb54640 4 set_mon_vals no callback set 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f90188ee640 10 monclient: dump: 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f901bb54640 10 monclient: shutdown 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f901bb54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f901bb54640 1 -- 172.21.15.45:0/3396045318 >> v1:172.21.15.45:6792/0 conn(0x7f90140a0cb0 legacy=0x7f9014137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f901bb54640 1 -- 172.21.15.45:0/3396045318 shutdown_connections 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f901bb54640 1 -- 172.21.15.45:0/3396045318 >> 172.21.15.45:0/3396045318 conn(0x7f90140a5bb0 msgr2=0x7f90140a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f901bb54640 1 -- 172.21.15.45:0/3396045318 shutdown_connections 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.334+0000 7f901bb54640 1 -- 172.21.15.45:0/3396045318 wait complete. 2024-09-18T14:17:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.335+0000 7f901bb54640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.335+0000 7f901bb54640 10 monclient: monmap: 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.335+0000 7f901bb54640 1 Processor -- start 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.335+0000 7f901bb54640 1 -- start start 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.335+0000 7f901bb54640 10 monclient: init 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f901bb54640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f901bb54640 10 monclient: _add_conns ranks=[0,6,8,2,1,7,5,3,4] 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f901bb54640 10 monclient(hunting): picked mon.noname-a con 0x7f901406e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f901bb54640 10 monclient(hunting): picked mon.noname-d con 0x7f90140a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f901bb54640 10 monclient(hunting): picked mon.noname-e con 0x7f90140a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f901bb54640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901409fa10 con 0x7f901406e810 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f901bb54640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901413dce0 con 0x7f90140a0cb0 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f901bb54640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90140a2270 con 0x7f90140a1080 2024-09-18T14:17:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f901bb54640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f901bb54640 10 monclient(hunting): authenticate will time out at 1854.674382s 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f90198f0640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f901406e810 0x7f901413d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45874/0 (socket says 172.21.15.45:45874) 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f90198f0640 1 -- 172.21.15.45:0/777913586 learned_addr learned my addr 172.21.15.45:0/777913586 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268896893 0 0) 0x7f901409fa10 con 0x7f901406e810 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): my global_id is 6408 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ff80034e0 con 0x7f901406e810 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310267317 0 0) 0x7f901413dce0 con 0x7f90140a0cb0 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): my global_id is 184929 2024-09-18T14:17:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f901409fa10 con 0x7f90140a0cb0 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869779152 0 0) 0x7f90140a2270 con 0x7f90140a1080 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): my global_id is 6065 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f901413dce0 con 0x7f90140a1080 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3888684351 0 0) 0x7f8ff80034e0 con 0x7f901406e810 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f90140a2270 con 0x7f901406e810 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9010001cd0 con 0x7f901406e810 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): dump: 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.354 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.336+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 652483002 0 0) 0x7f901409fa10 con 0x7f90140a0cb0 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ff80034e0 con 0x7f90140a0cb0 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1067287921 0 0) 0x7f901413dce0 con 0x7f90140a1080 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f901409fa10 con 0x7f90140a1080 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9004004ae0 con 0x7f90140a0cb0 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient(hunting): dump: 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f900c001d60 con 0x7f90140a1080 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient(hunting): dump: 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1224407751 0 0) 0x7f90140a2270 con 0x7f901406e810 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 >> v1:172.21.15.45:6793/0 conn(0x7f90140a1080 legacy=0x7f9014149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 >> v1:172.21.15.45:6792/0 conn(0x7f90140a0cb0 legacy=0x7f90141467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 1 monclient: found mon.a 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9014122670 con 0x7f901406e810 2024-09-18T14:17:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f901bb54640 5 monclient: authenticate success, global_id 6408 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f901bb54640 10 monclient: _renew_subs 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f901bb54640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f901bb54640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9014122950 con 0x7f901406e810 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f90100022b0 con 0x7f901406e810 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f90100027a0 con 0x7f901406e810 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f9009ffb640 10 monclient: dump: 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f901bb54640 10 monclient: _renew_subs 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f900bfff640 4 set_mon_vals no callback set 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f901bb54640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.337+0000 7f901bb54640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f901413eb80 con 0x7f901406e810 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f9010002900 con 0x7f901406e810 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f901bb54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f901bb54640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f901404c4a0 con 0x7f9014120310 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(846..846 src has 529..846) v4 ==== 3955+0+0 (unknown 1173980225 0 0) 0x7f9010046c20 con 0x7f901406e810 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135557857 0 0) 0x7f901404c4a0 con 0x7f9014120310 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 10 monclient(hunting): my global_id is 184938 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90140a2270 con 0x7f9014120310 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f901a0f1640 10 monclient: get_auth_request con 0x7f8ff803aae0 auth_method 0 2024-09-18T14:17:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2347102922 0 0) 0x7f90140a2270 con 0x7f9014120310 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f901404c4a0 con 0x7f9014120310 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9004001800 con 0x7f9014120310 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 10 monclient: dump: 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4280018954 0 0) 0x7f901404c4a0 con 0x7f9014120310 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.338+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f901404b720 con 0x7f9014120310 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.339+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9004005190 con 0x7f9014120310 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.339+0000 7f9009ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.339+0000 7f9009ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.339+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 >> v1:172.21.15.45:6792/0 conn(0x7f9014120310 legacy=0x7f901409bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.339+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 >> v1:172.21.15.45:6792/0 conn(0x7f9014120310 legacy=0x7f901409bb80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.364 INFO:tasks.rados.rados.0.smithi084.stdout:3777: finishing write tid 1 to smithi08434663-17 2024-09-18T14:17:19.364 INFO:tasks.rados.rados.0.smithi084.stdout:3777: finishing write tid 2 to smithi08434663-17 2024-09-18T14:17:19.364 INFO:tasks.rados.rados.0.smithi084.stdout:3777: finishing write tid 3 to smithi08434663-17 2024-09-18T14:17:19.365 INFO:tasks.rados.rados.0.smithi084.stdout:3777: finishing write tid 4 to smithi08434663-17 2024-09-18T14:17:19.365 INFO:tasks.rados.rados.0.smithi084.stdout:3777: finishing write tid 5 to smithi08434663-17 2024-09-18T14:17:19.365 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1881 (ObjNum 1262 snap 421 seq_num 1262) dirty exists 2024-09-18T14:17:19.365 INFO:tasks.rados.rados.0.smithi084.stdout:3777: left oid 17 (ObjNum 1262 snap 421 seq_num 1262) 2024-09-18T14:17:19.365 INFO:tasks.rados.rados.0.smithi084.stdout:3778: expect (ObjNum 1232 snap 406 seq_num 1232) 2024-09-18T14:17:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.344+0000 7f901bb54640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.344+0000 7f901bb54640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9014062dd0 con 0x7f9014121750 2024-09-18T14:17:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481683788 0 0) 0x7f9014062dd0 con 0x7f9014121750 2024-09-18T14:17:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 10 monclient(hunting): my global_id is 184947 2024-09-18T14:17:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f901404c4a0 con 0x7f9014121750 2024-09-18T14:17:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 5155934 0 0) 0x7f901404c4a0 con 0x7f9014121750 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9014062dd0 con 0x7f9014121750 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9004005990 con 0x7f9014121750 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 10 monclient: dump: 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4178639588 0 0) 0x7f9014062dd0 con 0x7f9014121750 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f90140670e0 con 0x7f9014121750 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 3452209082) 0x7f9004005af0 con 0x7f9014121750 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 >> v1:172.21.15.45:6792/0 conn(0x7f9014121750 legacy=0x7f901409c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.345+0000 7f9009ffb640 1 -- 172.21.15.45:0/777913586 >> v1:172.21.15.45:6792/0 conn(0x7f9014121750 legacy=0x7f901409c290 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.369 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.388 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.388 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:19.388 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.346+0000 7f901bb54640 1 -- 172.21.15.45:0/777913586 >> v1:172.21.15.84:6812/31478 conn(0x7f8ff803aae0 legacy=0x7f8ff803cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.346+0000 7f901bb54640 10 monclient: shutdown 2024-09-18T14:17:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.346+0000 7f901bb54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.346+0000 7f901bb54640 1 -- 172.21.15.45:0/777913586 >> v1:172.21.15.45:6789/0 conn(0x7f901406e810 legacy=0x7f901413d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.346+0000 7f901a0f1640 1 -- 172.21.15.45:0/777913586 reap_dead start 2024-09-18T14:17:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.346+0000 7f901a0f1640 1 -- 172.21.15.45:0/777913586 reap_dead start 2024-09-18T14:17:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.346+0000 7f901bb54640 1 -- 172.21.15.45:0/777913586 shutdown_connections 2024-09-18T14:17:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.346+0000 7f901bb54640 1 -- 172.21.15.45:0/777913586 >> 172.21.15.45:0/777913586 conn(0x7f90140a5bb0 msgr2=0x7f90140a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.346+0000 7f901bb54640 1 -- 172.21.15.45:0/777913586 shutdown_connections 2024-09-18T14:17:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.346+0000 7f901bb54640 1 -- 172.21.15.45:0/777913586 wait complete. 2024-09-18T14:17:19.390 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:19.390 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:17:19.391 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:17:19.463 INFO:tasks.rados.rados.0.smithi084.stdout:3776: done (3 left) 2024-09-18T14:17:19.464 INFO:tasks.rados.rados.0.smithi084.stdout:3777: done (2 left) 2024-09-18T14:17:19.464 INFO:tasks.rados.rados.0.smithi084.stdout:3778: done (1 left) 2024-09-18T14:17:19.464 INFO:tasks.rados.rados.0.smithi084.stdout:3779: done (0 left) 2024-09-18T14:17:19.464 INFO:tasks.rados.rados.0.smithi084.stdout:3780: rollback oid 8 current snap is 421 2024-09-18T14:17:19.464 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 8 to 419 2024-09-18T14:17:19.464 INFO:tasks.rados.rados.0.smithi084.stdout:3781: rollback oid 3 current snap is 421 2024-09-18T14:17:19.464 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 3 to 398 2024-09-18T14:17:19.464 INFO:tasks.rados.rados.0.smithi084.stdout:3782: read oid 41 snap -1 2024-09-18T14:17:19.464 INFO:tasks.rados.rados.0.smithi084.stdout:3782: expect (ObjNum 1227 snap 401 seq_num 1227) 2024-09-18T14:17:19.464 INFO:tasks.rados.rados.0.smithi084.stdout:3783: copy_from oid 19 from oid 21 current snap is 421 2024-09-18T14:17:19.464 INFO:tasks.rados.rados.0.smithi084.stdout:3784: delete oid 13 current snap is 421 2024-09-18T14:17:19.465 INFO:tasks.rados.rados.0.smithi084.stdout:3783: finishing copy_from to smithi08434663-19 2024-09-18T14:17:19.465 INFO:tasks.rados.rados.0.smithi084.stdout:3783: got expected ENOENT (src dne) 2024-09-18T14:17:19.465 INFO:tasks.rados.rados.0.smithi084.stdout:3783: finishing copy_from racing read to smithi08434663-19 2024-09-18T14:17:19.468 INFO:tasks.rados.rados.0.smithi084.stdout:3783: done (4 left) 2024-09-18T14:17:19.468 INFO:tasks.rados.rados.0.smithi084.stdout:3784: done (3 left) 2024-09-18T14:17:19.468 INFO:tasks.rados.rados.0.smithi084.stdout:3785: read oid 38 snap -1 2024-09-18T14:17:19.468 INFO:tasks.rados.rados.0.smithi084.stdout:3785: expect (ObjNum 1171 snap 382 seq_num 1171) 2024-09-18T14:17:19.469 INFO:tasks.rados.rados.0.smithi084.stdout:3786: snap_create 2024-09-18T14:17:19.469 INFO:tasks.rados.rados.0.smithi084.stdout:3781: finishing rollback tid 0 to smithi08434663-3 2024-09-18T14:17:19.469 INFO:tasks.rados.rados.0.smithi084.stdout:3781: finishing rollback tid 1 to smithi08434663-3 2024-09-18T14:17:19.469 INFO:tasks.rados.rados.0.smithi084.stdout:3781: finishing rollback tid 2 to smithi08434663-3 2024-09-18T14:17:19.469 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 1587 (ObjNum 967 snap 308 seq_num 967) dirty exists 2024-09-18T14:17:19.470 INFO:tasks.rados.rados.0.smithi084.stdout:3782: expect (ObjNum 1227 snap 401 seq_num 1227) 2024-09-18T14:17:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.492+0000 7f9f3135d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.492+0000 7f9f3135d640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.492+0000 7f9f3135d640 10 monclient: monmap: 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:19.493241+0000 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:19.493241+0000 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.494 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.492+0000 7f9f3135d640 1 Processor -- start 2024-09-18T14:17:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.492+0000 7f9f3135d640 1 -- start start 2024-09-18T14:17:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f3135d640 10 monclient: init 2024-09-18T14:17:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f3135d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f3135d640 10 monclient: _add_conns ranks=[5,3,2,0,1,7,8,6,4] 2024-09-18T14:17:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f3135d640 10 monclient(hunting): picked mon.noname-h con 0x7f9f2c06e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f3135d640 10 monclient(hunting): picked mon.noname-g con 0x7f9f2c0a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f3135d640 10 monclient(hunting): picked mon.noname-b con 0x7f9f2c0a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f3135d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c05ed70 con 0x7f9f2c0a1080 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f3135d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c097620 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f3135d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c097340 con 0x7f9f2c06e810 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f3135d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f3135d640 10 monclient(hunting): authenticate will time out at 1854.831608s 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f2b7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f9f2c0a1080 0x7f9f2c13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60844/0 (socket says 172.21.15.45:60844) 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f2b7fe640 1 -- 172.21.15.45:0/3980024120 learned_addr learned my addr 172.21.15.45:0/3980024120 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.493+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340772282 0 0) 0x7f9f2c05ed70 con 0x7f9f2c0a1080 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient(hunting): my global_id is 145523 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f180034e0 con 0x7f9f2c0a1080 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941157149 0 0) 0x7f9f2c097340 con 0x7f9f2c06e810 2024-09-18T14:17:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient(hunting): my global_id is 184802 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f2c05ed70 con 0x7f9f2c06e810 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097484332 0 0) 0x7f9f2c097620 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient(hunting): my global_id is 224112 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f2c097340 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2062042420 0 0) 0x7f9f180034e0 con 0x7f9f2c0a1080 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 >> v1:172.21.15.84:6791/0 conn(0x7f9f2c06e810 legacy=0x7f9f2c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 >> v1:172.21.15.84:6790/0 conn(0x7f9f2c0a0cb0 legacy=0x7f9f2c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 monclient: found mon.noname-b 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c0a1660 con 0x7f9f2c0a1080 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f1c001830 con 0x7f9f2c0a1080 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f3135d640 5 monclient: authenticate success, global_id 145523 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f3135d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient: dump: 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f3135d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9f1c001a10 con 0x7f9f2c0a1080 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 1 -- 172.21.15.45:0/3980024120 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f1c004ba0 con 0x7f9f2c0a1080 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f3135d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f3135d640 4 set_mon_vals no callback set 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f29ffb640 10 monclient: dump: 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f3135d640 10 monclient: shutdown 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f3135d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f3135d640 1 -- 172.21.15.45:0/3980024120 >> v1:172.21.15.45:6790/0 conn(0x7f9f2c0a1080 legacy=0x7f9f2c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f3135d640 1 -- 172.21.15.45:0/3980024120 shutdown_connections 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f3135d640 1 -- 172.21.15.45:0/3980024120 >> 172.21.15.45:0/3980024120 conn(0x7f9f2c0a5bb0 msgr2=0x7f9f2c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f3135d640 1 -- 172.21.15.45:0/3980024120 shutdown_connections 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.494+0000 7f9f3135d640 1 -- 172.21.15.45:0/3980024120 wait complete. 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.495+0000 7f9f3135d640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.495+0000 7f9f3135d640 10 monclient: monmap: 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.495+0000 7f9f3135d640 1 Processor -- start 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.495+0000 7f9f3135d640 1 -- start start 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.495+0000 7f9f3135d640 10 monclient: init 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f3135d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f3135d640 10 monclient: _add_conns ranks=[8,5,4,2,1,6,3,0,7] 2024-09-18T14:17:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f3135d640 10 monclient(hunting): picked mon.noname-e con 0x7f9f2c06e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f3135d640 10 monclient(hunting): picked mon.noname-h con 0x7f9f2c0a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f3135d640 10 monclient(hunting): picked mon.noname-c con 0x7f9f2c0a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f3135d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c09fa10 con 0x7f9f2c0a1080 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f3135d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c0a2270 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f3135d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c136a90 con 0x7f9f2c06e810 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f3135d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f3135d640 10 monclient(hunting): authenticate will time out at 1854.834471s 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f2affd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f9f2c06e810 0x7f9f2c1353f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:33010/0 (socket says 172.21.15.45:33010) 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f2affd640 1 -- 172.21.15.45:0/3697803816 learned_addr learned my addr 172.21.15.45:0/3697803816 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054277399 0 0) 0x7f9f2c136a90 con 0x7f9f2c06e810 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 10 monclient(hunting): my global_id is 6074 2024-09-18T14:17:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f0c0034e0 con 0x7f9f2c06e810 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667970131 0 0) 0x7f9f2c09fa10 con 0x7f9f2c0a1080 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 10 monclient(hunting): my global_id is 165145 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f2c136a90 con 0x7f9f2c0a1080 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126996416 0 0) 0x7f9f2c0a2270 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 10 monclient(hunting): my global_id is 184811 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.496+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f2c09fa10 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3520682332 0 0) 0x7f9f2c09fa10 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f2c0a2270 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f140019e0 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient(hunting): dump: 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 764732605 0 0) 0x7f9f2c136a90 con 0x7f9f2c0a1080 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f2c09fa10 con 0x7f9f2c0a1080 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3835538313 0 0) 0x7f9f0c0034e0 con 0x7f9f2c06e810 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f2c136a90 con 0x7f9f2c06e810 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f1c0017f0 con 0x7f9f2c0a1080 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient(hunting): dump: 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f24002f10 con 0x7f9f2c06e810 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient(hunting): dump: 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 159389065 0 0) 0x7f9f2c0a2270 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 >> v1:172.21.15.45:6793/0 conn(0x7f9f2c06e810 legacy=0x7f9f2c1353f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 >> v1:172.21.15.45:6791/0 conn(0x7f9f2c0a1080 legacy=0x7f9f2c149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 monclient: found mon.h 2024-09-18T14:17:19.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c1424a0 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f3135d640 5 monclient: authenticate success, global_id 184811 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f3135d640 10 monclient: _renew_subs 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f3135d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f3135d640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.84:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c136190 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f3135d640 10 monclient: _renew_subs 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f3135d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f3135d640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.84:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c122ad0 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.5 v1:172.21.15.84:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9f14001c80 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.5 v1:172.21.15.84:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f14002170 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f297fa640 4 set_mon_vals no callback set 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.497+0000 7f9f22ffd640 10 monclient: dump: 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f3135d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f3135d640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9f10006580 con 0x7f9f10002830 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.5 v1:172.21.15.84:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f9f140022d0 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.5 v1:172.21.15.84:6791/0 8 ==== osd_map(846..846 src has 529..846) v4 ==== 3955+0+0 (unknown 1173980225 0 0) 0x7f9f140440b0 con 0x7f9f2c0a0cb0 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261417399 0 0) 0x7f9f10006580 con 0x7f9f10002830 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 10 monclient(hunting): my global_id is 6083 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f2c0a2270 con 0x7f9f10002830 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f2b7fe640 10 monclient: get_auth_request con 0x7f9f0c03a9a0 auth_method 0 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1834184677 0 0) 0x7f9f2c0a2270 con 0x7f9f10002830 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f10006580 con 0x7f9f10002830 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f24001af0 con 0x7f9f10002830 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 10 monclient: dump: 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2094746259 0 0) 0x7f9f10006580 con 0x7f9f10002830 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.498+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f1000a0a0 con 0x7f9f10002830 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.499+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9f24005b00 con 0x7f9f10002830 2024-09-18T14:17:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.499+0000 7f9f22ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:19.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.499+0000 7f9f22ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:19.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.499+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 >> v1:172.21.15.45:6793/0 conn(0x7f9f10002830 legacy=0x7f9f10004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.499+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 >> v1:172.21.15.45:6793/0 conn(0x7f9f10002830 legacy=0x7f9f10004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f3135d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:19.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f3135d640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9f10009e70 con 0x7f9f100068f0 2024-09-18T14:17:19.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815474995 0 0) 0x7f9f10009e70 con 0x7f9f100068f0 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 10 monclient(hunting): my global_id is 6092 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f10006580 con 0x7f9f100068f0 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1337992920 0 0) 0x7f9f10006580 con 0x7f9f100068f0 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f10009e70 con 0x7f9f100068f0 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9f240025b0 con 0x7f9f100068f0 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 10 monclient: dump: 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1756470714 0 0) 0x7f9f10009e70 con 0x7f9f100068f0 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9f10008750 con 0x7f9f100068f0 2024-09-18T14:17:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7389 (unknown 0 0 3195501862) 0x7f9f24006100 con 0x7f9f100068f0 2024-09-18T14:17:19.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:19.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:19.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 >> v1:172.21.15.45:6793/0 conn(0x7f9f100068f0 legacy=0x7f9f10006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.505+0000 7f9f22ffd640 1 -- 172.21.15.45:0/3697803816 >> v1:172.21.15.45:6793/0 conn(0x7f9f100068f0 legacy=0x7f9f10006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.527 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:19.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:19.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:19.527 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:17:19.527 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:19.527 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:19.528 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:19.529 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:19.530 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:19.531 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.532 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:19.534 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.535 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:19.537 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:19.538 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:19.541 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.542 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.545 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.546 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.546 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:19.546 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.506+0000 7f9f3135d640 1 -- 172.21.15.45:0/3697803816 >> v1:172.21.15.84:6812/31478 conn(0x7f9f0c03a9a0 legacy=0x7f9f0c03ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.506+0000 7f9f3135d640 10 monclient: shutdown 2024-09-18T14:17:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.506+0000 7f9f3135d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.506+0000 7f9f3135d640 1 -- 172.21.15.45:0/3697803816 >> v1:172.21.15.84:6791/0 conn(0x7f9f2c0a0cb0 legacy=0x7f9f2c1468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.506+0000 7f9f2b7fe640 1 -- 172.21.15.45:0/3697803816 reap_dead start 2024-09-18T14:17:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.506+0000 7f9f2b7fe640 1 -- 172.21.15.45:0/3697803816 reap_dead start 2024-09-18T14:17:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.506+0000 7f9f3135d640 1 -- 172.21.15.45:0/3697803816 shutdown_connections 2024-09-18T14:17:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.506+0000 7f9f3135d640 1 -- 172.21.15.45:0/3697803816 >> 172.21.15.45:0/3697803816 conn(0x7f9f2c0a5bb0 msgr2=0x7f9f2c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.506+0000 7f9f3135d640 1 -- 172.21.15.45:0/3697803816 shutdown_connections 2024-09-18T14:17:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.506+0000 7f9f3135d640 1 -- 172.21.15.45:0/3697803816 wait complete. 2024-09-18T14:17:19.548 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:19.548 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:17:19.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:17:19.619 INFO:tasks.rados.rados.0.smithi084.stdout:3780: finishing rollback tid 0 to smithi08434663-8 2024-09-18T14:17:19.620 INFO:tasks.rados.rados.0.smithi084.stdout:3780: finishing rollback tid 1 to smithi08434663-8 2024-09-18T14:17:19.620 INFO:tasks.rados.rados.0.smithi084.stdout:3780: finishing rollback tid 2 to smithi08434663-8 2024-09-18T14:17:19.620 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 1438 (ObjNum 1253 snap 414 seq_num 1253) dirty exists 2024-09-18T14:17:19.620 INFO:tasks.rados.rados.0.smithi084.stdout:3785: expect (ObjNum 1171 snap 382 seq_num 1171) 2024-09-18T14:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.645+0000 7f8b3e59d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.645+0000 7f8b3e59d640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.645+0000 7f8b3e59d640 10 monclient: monmap: 2024-09-18T14:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:19.645983+0000 2024-09-18T14:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:19.645983+0000 2024-09-18T14:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.647 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.647 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.647 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.645+0000 7f8b3e59d640 1 Processor -- start 2024-09-18T14:17:19.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.645+0000 7f8b3e59d640 1 -- start start 2024-09-18T14:17:19.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.645+0000 7f8b3e59d640 10 monclient: init 2024-09-18T14:17:19.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b3e59d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b3e59d640 10 monclient: _add_conns ranks=[6,7,1,5,8,0,2,4,3] 2024-09-18T14:17:19.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b3e59d640 10 monclient(hunting): picked mon.noname-d con 0x7f8b3806e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:19.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b3e59d640 10 monclient(hunting): picked mon.noname-i con 0x7f8b380a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:17:19.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b3e59d640 10 monclient(hunting): picked mon.noname-f con 0x7f8b380a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:19.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b3e59d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3805ed70 con 0x7f8b380a1080 2024-09-18T14:17:19.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b3e59d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b38097620 con 0x7f8b3806e810 2024-09-18T14:17:19.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b3e59d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b38097340 con 0x7f8b380a0cb0 2024-09-18T14:17:19.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b3e59d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b3e59d640 10 monclient(hunting): authenticate will time out at 1854.984380s 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b37fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f8b3806e810 0x7f8b380a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43100/0 (socket says 172.21.15.45:43100) 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b37fff640 1 -- 172.21.15.45:0/248382795 learned_addr learned my addr 172.21.15.45:0/248382795 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142544102 0 0) 0x7f8b38097620 con 0x7f8b3806e810 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 10 monclient(hunting): my global_id is 184956 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b200034e0 con 0x7f8b3806e810 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230103792 0 0) 0x7f8b38097340 con 0x7f8b380a0cb0 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 10 monclient(hunting): my global_id is 214423 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b38097620 con 0x7f8b380a0cb0 2024-09-18T14:17:19.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966953888 0 0) 0x7f8b3805ed70 con 0x7f8b380a1080 2024-09-18T14:17:19.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 10 monclient(hunting): my global_id is 214390 2024-09-18T14:17:19.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.646+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b38097340 con 0x7f8b380a1080 2024-09-18T14:17:19.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1062138590 0 0) 0x7f8b200034e0 con 0x7f8b3806e810 2024-09-18T14:17:19.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 >> v1:172.21.15.84:6792/0 conn(0x7f8b380a0cb0 legacy=0x7f8b38137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 >> v1:172.21.15.84:6789/0 conn(0x7f8b380a1080 legacy=0x7f8b3813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 1 monclient: found mon.noname-d 2024-09-18T14:17:19.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:17:19.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b380a1660 con 0x7f8b3806e810 2024-09-18T14:17:19.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b28001830 con 0x7f8b3806e810 2024-09-18T14:17:19.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:17:19.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 10 monclient: dump: 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.660 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.661 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.661 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.661 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b3e59d640 5 monclient: authenticate success, global_id 184956 2024-09-18T14:17:19.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b3e59d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:19.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8b28001a10 con 0x7f8b3806e810 2024-09-18T14:17:19.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 1 -- 172.21.15.45:0/248382795 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b28004ba0 con 0x7f8b3806e810 2024-09-18T14:17:19.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b3e59d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:19.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b3e59d640 4 set_mon_vals no callback set 2024-09-18T14:17:19.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b36ffd640 10 monclient: dump: 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.662 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.663 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.663 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.663 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.663 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b3e59d640 10 monclient: shutdown 2024-09-18T14:17:19.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b3e59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b3e59d640 1 -- 172.21.15.45:0/248382795 >> v1:172.21.15.45:6792/0 conn(0x7f8b3806e810 legacy=0x7f8b380a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b3e59d640 1 -- 172.21.15.45:0/248382795 shutdown_connections 2024-09-18T14:17:19.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b3e59d640 1 -- 172.21.15.45:0/248382795 >> 172.21.15.45:0/248382795 conn(0x7f8b380a5bb0 msgr2=0x7f8b380a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b3e59d640 1 -- 172.21.15.45:0/248382795 shutdown_connections 2024-09-18T14:17:19.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.647+0000 7f8b3e59d640 1 -- 172.21.15.45:0/248382795 wait complete. 2024-09-18T14:17:19.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.648+0000 7f8b3e59d640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.648+0000 7f8b3e59d640 10 monclient: monmap: 2024-09-18T14:17:19.664 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.664 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:19.664 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:19.665 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:19.665 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.665 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:19.665 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.665 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.665 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.665 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.665 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.665 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.665 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.648+0000 7f8b3e59d640 1 Processor -- start 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.648+0000 7f8b3e59d640 1 -- start start 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.648+0000 7f8b3e59d640 10 monclient: init 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b3e59d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b3e59d640 10 monclient: _add_conns ranks=[8,2,0,3,4,1,6,5,7] 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b3e59d640 10 monclient(hunting): picked mon.noname-e con 0x7f8b3806e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b3e59d640 10 monclient(hunting): picked mon.noname-b con 0x7f8b380a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b3e59d640 10 monclient(hunting): picked mon.noname-a con 0x7f8b380a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b3e59d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3809d6e0 con 0x7f8b380a1080 2024-09-18T14:17:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b3e59d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3809fa10 con 0x7f8b380a0cb0 2024-09-18T14:17:19.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b3e59d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b380a2270 con 0x7f8b3806e810 2024-09-18T14:17:19.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b3e59d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b3e59d640 10 monclient(hunting): authenticate will time out at 1854.987257s 2024-09-18T14:17:19.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b37fff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f8b3806e810 0x7f8b3809b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:33042/0 (socket says 172.21.15.45:33042) 2024-09-18T14:17:19.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b37fff640 1 -- 172.21.15.45:0/2861451624 learned_addr learned my addr 172.21.15.45:0/2861451624 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931961748 0 0) 0x7f8b3809fa10 con 0x7f8b380a0cb0 2024-09-18T14:17:19.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): my global_id is 145532 2024-09-18T14:17:19.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b240034e0 con 0x7f8b380a0cb0 2024-09-18T14:17:19.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476492076 0 0) 0x7f8b380a2270 con 0x7f8b3806e810 2024-09-18T14:17:19.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): my global_id is 6101 2024-09-18T14:17:19.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b3809fa10 con 0x7f8b3806e810 2024-09-18T14:17:19.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721610987 0 0) 0x7f8b3809d6e0 con 0x7f8b380a1080 2024-09-18T14:17:19.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): my global_id is 6417 2024-09-18T14:17:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b380a2270 con 0x7f8b380a1080 2024-09-18T14:17:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3720188768 0 0) 0x7f8b240034e0 con 0x7f8b380a0cb0 2024-09-18T14:17:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b3809d6e0 con 0x7f8b380a0cb0 2024-09-18T14:17:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 666045589 0 0) 0x7f8b3809fa10 con 0x7f8b3806e810 2024-09-18T14:17:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b240034e0 con 0x7f8b3806e810 2024-09-18T14:17:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b2c0016c0 con 0x7f8b380a0cb0 2024-09-18T14:17:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:17:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): dump: 2024-09-18T14:17:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3817299037 0 0) 0x7f8b380a2270 con 0x7f8b380a1080 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.649+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b3809fa10 con 0x7f8b380a1080 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b280017f0 con 0x7f8b3806e810 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient(hunting): dump: 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b30001b70 con 0x7f8b380a1080 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient(hunting): dump: 2024-09-18T14:17:19.672 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 548992717 0 0) 0x7f8b240034e0 con 0x7f8b3806e810 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 >> v1:172.21.15.45:6790/0 conn(0x7f8b380a0cb0 legacy=0x7f8b3809bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 >> v1:172.21.15.45:6789/0 conn(0x7f8b380a1080 legacy=0x7f8b38149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 1 monclient: found mon.e 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3809d160 con 0x7f8b3806e810 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b3e59d640 5 monclient: authenticate success, global_id 6101 2024-09-18T14:17:19.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b3e59d640 10 monclient: _renew_subs 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b3e59d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b3e59d640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3809ec20 con 0x7f8b3806e810 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b3e59d640 10 monclient: _renew_subs 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b3e59d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b3e59d640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3812dfe0 con 0x7f8b3806e810 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8b28004b50 con 0x7f8b3806e810 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b28005160 con 0x7f8b3806e810 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b367fc640 4 set_mon_vals no callback set 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.650+0000 7f8b1ffff640 10 monclient: dump: 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.675 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b3e59d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b3e59d640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8b18006580 con 0x7f8b18002830 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f8b28010130 con 0x7f8b3806e810 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(846..846 src has 529..846) v4 ==== 3955+0+0 (unknown 1173980225 0 0) 0x7f8b28046ca0 con 0x7f8b3806e810 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763446108 0 0) 0x7f8b18006580 con 0x7f8b18002830 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 10 monclient(hunting): my global_id is 214399 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b240034e0 con 0x7f8b18002830 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b3cb3a640 10 monclient: get_auth_request con 0x7f8b2403aad0 auth_method 0 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3338395893 0 0) 0x7f8b240034e0 con 0x7f8b18002830 2024-09-18T14:17:19.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b18006580 con 0x7f8b18002830 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b2c002570 con 0x7f8b18002830 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 10 monclient: dump: 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 830996634 0 0) 0x7f8b18006580 con 0x7f8b18002830 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.651+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b1800a0a0 con 0x7f8b18002830 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.652+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8b2c004890 con 0x7f8b18002830 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.652+0000 7f8b1ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.652+0000 7f8b1ffff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.652+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 >> v1:172.21.15.84:6789/0 conn(0x7f8b18002830 legacy=0x7f8b18004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.652+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 >> v1:172.21.15.84:6789/0 conn(0x7f8b18002830 legacy=0x7f8b18004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b3e59d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:19.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b3e59d640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8b180074f0 con 0x7f8b180068a0 2024-09-18T14:17:19.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541518412 0 0) 0x7f8b180074f0 con 0x7f8b180068a0 2024-09-18T14:17:19.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 10 monclient(hunting): my global_id is 214408 2024-09-18T14:17:19.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b18006580 con 0x7f8b180068a0 2024-09-18T14:17:19.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 856916267 0 0) 0x7f8b18006580 con 0x7f8b180068a0 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b180074f0 con 0x7f8b180068a0 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b2c003030 con 0x7f8b180068a0 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 10 monclient: dump: 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.682 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3394506993 0 0) 0x7f8b180074f0 con 0x7f8b180068a0 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.658+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8b18008750 con 0x7f8b180068a0 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 3803084788) 0x7f8b2c0044e0 con 0x7f8b180068a0 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b1ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b1ffff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 >> v1:172.21.15.84:6789/0 conn(0x7f8b180068a0 legacy=0x7f8b18006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b1ffff640 1 -- 172.21.15.45:0/2861451624 >> v1:172.21.15.84:6789/0 conn(0x7f8b180068a0 legacy=0x7f8b18006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.684 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:19.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:19.684 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:19.684 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:19.684 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:19.684 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:19.684 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:19.684 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:19.684 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.685 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:19.687 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:19.688 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:19.688 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:19.688 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:19.688 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:19.688 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:19.688 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:19.688 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:19.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:19.688 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:19.688 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:19.689 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:19.689 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.689 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.689 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.689 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.689 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.689 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.689 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.690 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:19.691 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.692 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.694 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:19.695 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.696 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.698 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:19.699 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.700 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.701 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.704 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.704 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.704 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:19.704 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:19.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b3e59d640 1 -- 172.21.15.45:0/2861451624 >> v1:172.21.15.84:6812/31478 conn(0x7f8b2403aad0 legacy=0x7f8b2403cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b3e59d640 10 monclient: shutdown 2024-09-18T14:17:19.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b3e59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b3e59d640 1 -- 172.21.15.45:0/2861451624 >> v1:172.21.15.45:6793/0 conn(0x7f8b3806e810 legacy=0x7f8b3809b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b3cb3a640 1 -- 172.21.15.45:0/2861451624 reap_dead start 2024-09-18T14:17:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b3cb3a640 1 -- 172.21.15.45:0/2861451624 reap_dead start 2024-09-18T14:17:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b3e59d640 1 -- 172.21.15.45:0/2861451624 shutdown_connections 2024-09-18T14:17:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.659+0000 7f8b3e59d640 1 -- 172.21.15.45:0/2861451624 >> 172.21.15.45:0/2861451624 conn(0x7f8b380a5bb0 msgr2=0x7f8b38131950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.660+0000 7f8b3e59d640 1 -- 172.21.15.45:0/2861451624 shutdown_connections 2024-09-18T14:17:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.660+0000 7f8b3e59d640 1 -- 172.21.15.45:0/2861451624 wait complete. 2024-09-18T14:17:19.706 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:19.706 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:17:19.707 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:17:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.803+0000 7f95f3a20640 10 monclient: get_monmap_and_config 2024-09-18T14:17:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.803+0000 7f95f3a20640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.803+0000 7f95f3a20640 10 monclient: monmap: 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:19.804616+0000 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:19.804616+0000 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.806 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 1 Processor -- start 2024-09-18T14:17:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 1 -- start start 2024-09-18T14:17:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 10 monclient: init 2024-09-18T14:17:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 10 monclient: _add_conns ranks=[8,2,0,1,3,4,6,7,5] 2024-09-18T14:17:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 10 monclient(hunting): picked mon.noname-e con 0x7f95ec09f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 10 monclient(hunting): picked mon.noname-b con 0x7f95ec09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 10 monclient(hunting): picked mon.noname-a con 0x7f95ec09ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec05ed70 con 0x7f95ec09ff70 2024-09-18T14:17:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec09dfc0 con 0x7f95ec09fba0 2024-09-18T14:17:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec09dce0 con 0x7f95ec09f7d0 2024-09-18T14:17:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.804+0000 7f95f3a20640 10 monclient(hunting): authenticate will time out at 1855.142956s 2024-09-18T14:17:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95f17bc640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f95ec09f7d0 0x7f95ec132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:33048/0 (socket says 172.21.15.45:33048) 2024-09-18T14:17:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95f17bc640 1 -- 172.21.15.45:0/4257623801 learned_addr learned my addr 172.21.15.45:0/4257623801 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95f0fbb640 1 --1- 172.21.15.45:0/4257623801 >> v1:172.21.15.45:6790/0 conn(0x7f95ec09fba0 0x7f95ec137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60852/0 (socket says 172.21.15.45:60852) 2024-09-18T14:17:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242832977 0 0) 0x7f95ec09dfc0 con 0x7f95ec09fba0 2024-09-18T14:17:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient(hunting): my global_id is 145541 2024-09-18T14:17:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d80034e0 con 0x7f95ec09fba0 2024-09-18T14:17:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878948906 0 0) 0x7f95ec05ed70 con 0x7f95ec09ff70 2024-09-18T14:17:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient(hunting): my global_id is 6426 2024-09-18T14:17:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ec09dfc0 con 0x7f95ec09ff70 2024-09-18T14:17:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101556178 0 0) 0x7f95ec09dce0 con 0x7f95ec09f7d0 2024-09-18T14:17:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient(hunting): my global_id is 6110 2024-09-18T14:17:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ec05ed70 con 0x7f95ec09f7d0 2024-09-18T14:17:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2896039236 0 0) 0x7f95d80034e0 con 0x7f95ec09fba0 2024-09-18T14:17:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 >> v1:172.21.15.45:6793/0 conn(0x7f95ec09f7d0 legacy=0x7f95ec132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 >> v1:172.21.15.45:6789/0 conn(0x7f95ec09ff70 legacy=0x7f95ec13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 monclient: found mon.noname-b 2024-09-18T14:17:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:17:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec0a1c00 con 0x7f95ec09fba0 2024-09-18T14:17:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95f3a20640 5 monclient: authenticate success, global_id 145541 2024-09-18T14:17:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f95dc001830 con 0x7f95ec09fba0 2024-09-18T14:17:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:17:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient: dump: 2024-09-18T14:17:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.820 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95f3a20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f95dc001990 con 0x7f95ec09fba0 2024-09-18T14:17:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95f3a20640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 1 -- 172.21.15.45:0/4257623801 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f95dc001c00 con 0x7f95ec09fba0 2024-09-18T14:17:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95f3a20640 4 set_mon_vals no callback set 2024-09-18T14:17:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.805+0000 7f95e3fff640 10 monclient: dump: 2024-09-18T14:17:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.822 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.806+0000 7f95f3a20640 10 monclient: shutdown 2024-09-18T14:17:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.806+0000 7f95f3a20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.806+0000 7f95f3a20640 1 -- 172.21.15.45:0/4257623801 >> v1:172.21.15.45:6790/0 conn(0x7f95ec09fba0 legacy=0x7f95ec137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.806+0000 7f95f3a20640 1 -- 172.21.15.45:0/4257623801 shutdown_connections 2024-09-18T14:17:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.806+0000 7f95f3a20640 1 -- 172.21.15.45:0/4257623801 >> 172.21.15.45:0/4257623801 conn(0x7f95ec0630b0 msgr2=0x7f95ec0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.806+0000 7f95f3a20640 1 -- 172.21.15.45:0/4257623801 shutdown_connections 2024-09-18T14:17:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.806+0000 7f95f3a20640 1 -- 172.21.15.45:0/4257623801 wait complete. 2024-09-18T14:17:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.806+0000 7f95f3a20640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.806+0000 7f95f3a20640 10 monclient: monmap: 2024-09-18T14:17:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.825 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.807+0000 7f95f3a20640 1 Processor -- start 2024-09-18T14:17:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.807+0000 7f95f3a20640 1 -- start start 2024-09-18T14:17:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.807+0000 7f95f3a20640 10 monclient: init 2024-09-18T14:17:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.807+0000 7f95f3a20640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.807+0000 7f95f3a20640 10 monclient: _add_conns ranks=[2,3,6,5,8,7,4,1,0] 2024-09-18T14:17:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.807+0000 7f95f3a20640 10 monclient(hunting): picked mon.noname-b con 0x7f95ec09f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.807+0000 7f95f3a20640 10 monclient(hunting): picked mon.noname-g con 0x7f95ec09fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.807+0000 7f95f3a20640 10 monclient(hunting): picked mon.noname-d con 0x7f95ec09ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.807+0000 7f95f3a20640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec09d310 con 0x7f95ec09f7d0 2024-09-18T14:17:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.807+0000 7f95f3a20640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec0a2560 con 0x7f95ec09fba0 2024-09-18T14:17:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.807+0000 7f95f3a20640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec099d30 con 0x7f95ec09ff70 2024-09-18T14:17:19.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95f3a20640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95f3a20640 10 monclient(hunting): authenticate will time out at 1855.146021s 2024-09-18T14:17:19.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95f17bc640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f95ec09f7d0 0x7f95ec098470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60864/0 (socket says 172.21.15.45:60864) 2024-09-18T14:17:19.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95f17bc640 1 -- 172.21.15.45:0/1445323214 learned_addr learned my addr 172.21.15.45:0/1445323214 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448243137 0 0) 0x7f95ec09d310 con 0x7f95ec09f7d0 2024-09-18T14:17:19.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): my global_id is 145550 2024-09-18T14:17:19.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d00034e0 con 0x7f95ec09f7d0 2024-09-18T14:17:19.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952612471 0 0) 0x7f95ec099d30 con 0x7f95ec09ff70 2024-09-18T14:17:19.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): my global_id is 184965 2024-09-18T14:17:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ec09d310 con 0x7f95ec09ff70 2024-09-18T14:17:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691071994 0 0) 0x7f95ec0a2560 con 0x7f95ec09fba0 2024-09-18T14:17:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): my global_id is 224121 2024-09-18T14:17:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ec099d30 con 0x7f95ec09fba0 2024-09-18T14:17:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1321933653 0 0) 0x7f95d00034e0 con 0x7f95ec09f7d0 2024-09-18T14:17:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95ec0a2560 con 0x7f95ec09f7d0 2024-09-18T14:17:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f95e8001350 con 0x7f95ec09f7d0 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): dump: 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.829 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3242215476 0 0) 0x7f95ec09d310 con 0x7f95ec09ff70 2024-09-18T14:17:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95d00034e0 con 0x7f95ec09ff70 2024-09-18T14:17:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3571620972 0 0) 0x7f95ec099d30 con 0x7f95ec09fba0 2024-09-18T14:17:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95ec09d310 con 0x7f95ec09fba0 2024-09-18T14:17:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f95e4001c40 con 0x7f95ec09ff70 2024-09-18T14:17:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): dump: 2024-09-18T14:17:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.834 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f95dc001b00 con 0x7f95ec09fba0 2024-09-18T14:17:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient(hunting): dump: 2024-09-18T14:17:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.836 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 166408667 0 0) 0x7f95ec0a2560 con 0x7f95ec09f7d0 2024-09-18T14:17:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 >> v1:172.21.15.45:6792/0 conn(0x7f95ec09ff70 legacy=0x7f95ec149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 >> v1:172.21.15.84:6790/0 conn(0x7f95ec09fba0 legacy=0x7f95ec098b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 monclient: found mon.b 2024-09-18T14:17:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec09b860 con 0x7f95ec09f7d0 2024-09-18T14:17:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95e17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95f3a20640 5 monclient: authenticate success, global_id 145550 2024-09-18T14:17:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95f3a20640 10 monclient: _renew_subs 2024-09-18T14:17:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95f3a20640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95f3a20640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ec12d260 con 0x7f95ec09f7d0 2024-09-18T14:17:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95f3a20640 10 monclient: _renew_subs 2024-09-18T14:17:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95f3a20640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.808+0000 7f95f3a20640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ec099290 con 0x7f95ec09f7d0 2024-09-18T14:17:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.809+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f95e8004a50 con 0x7f95ec09f7d0 2024-09-18T14:17:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.809+0000 7f95e17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.809+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f95e8004f00 con 0x7f95ec09f7d0 2024-09-18T14:17:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.809+0000 7f95e17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.809+0000 7f95e17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.809+0000 7f95e37fe640 4 set_mon_vals no callback set 2024-09-18T14:17:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.809+0000 7f95e17fa640 10 monclient: dump: 2024-09-18T14:17:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.840 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.809+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f95e800fed0 con 0x7f95ec09f7d0 2024-09-18T14:17:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.809+0000 7f95f3a20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.809+0000 7f95f3a20640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f95cc006580 con 0x7f95cc002830 2024-09-18T14:17:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.809+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(846..846 src has 529..846) v4 ==== 3955+0+0 (unknown 1173980225 0 0) 0x7f95e8046a80 con 0x7f95ec09f7d0 2024-09-18T14:17:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95f1fbd640 10 monclient: get_auth_request con 0x7f95d003aa30 auth_method 0 2024-09-18T14:17:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474613150 0 0) 0x7f95cc006580 con 0x7f95cc002830 2024-09-18T14:17:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 10 monclient(hunting): my global_id is 224130 2024-09-18T14:17:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ec0a2560 con 0x7f95cc002830 2024-09-18T14:17:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2096434649 0 0) 0x7f95ec0a2560 con 0x7f95cc002830 2024-09-18T14:17:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95cc006580 con 0x7f95cc002830 2024-09-18T14:17:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f95dc0050d0 con 0x7f95cc002830 2024-09-18T14:17:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 10 monclient: dump: 2024-09-18T14:17:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.843 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 473443387 0 0) 0x7f95cc006580 con 0x7f95cc002830 2024-09-18T14:17:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95cc00a0a0 con 0x7f95cc002830 2024-09-18T14:17:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f95dc0072d0 con 0x7f95cc002830 2024-09-18T14:17:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 >> v1:172.21.15.84:6790/0 conn(0x7f95cc002830 legacy=0x7f95cc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.810+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 >> v1:172.21.15.84:6790/0 conn(0x7f95cc002830 legacy=0x7f95cc004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.816+0000 7f95f3a20640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.816+0000 7f95f3a20640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f95cc007540 con 0x7f95cc0068f0 2024-09-18T14:17:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899177272 0 0) 0x7f95cc007540 con 0x7f95cc0068f0 2024-09-18T14:17:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 10 monclient(hunting): my global_id is 224139 2024-09-18T14:17:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95cc006580 con 0x7f95cc0068f0 2024-09-18T14:17:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3349556427 0 0) 0x7f95cc006580 con 0x7f95cc0068f0 2024-09-18T14:17:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95cc007540 con 0x7f95cc0068f0 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f95dc005aa0 con 0x7f95cc0068f0 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 10 monclient: dump: 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1762901832 0 0) 0x7f95cc007540 con 0x7f95cc0068f0 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f95cc0087a0 con 0x7f95cc0068f0 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 417517212) 0x7f95dc00a2c0 con 0x7f95cc0068f0 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 >> v1:172.21.15.84:6790/0 conn(0x7f95cc0068f0 legacy=0x7f95cc006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.817+0000 7f95e17fa640 1 -- 172.21.15.45:0/1445323214 >> v1:172.21.15.84:6790/0 conn(0x7f95cc0068f0 legacy=0x7f95cc006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:19.848 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:19.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:19.848 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:19.848 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:19.848 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:19.848 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:19.848 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:19.848 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:19.850 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.851 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.851 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.851 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.851 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.851 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.851 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.851 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.851 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.851 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.851 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:19.852 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:19.852 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:19.852 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:19.852 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:19.852 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:19.852 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:19.852 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:19.852 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:19.852 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:19.852 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:19.852 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:19.852 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:19.853 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:19.854 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.854 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:19.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.854 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:19.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.854 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:19.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:19.854 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.854 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.854 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:19.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:19.856 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.856 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.856 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:19.856 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.856 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.856 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.856 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:19.856 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:19.856 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.857 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.857 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.857 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:19.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:19.857 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.857 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.857 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:19.857 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.858 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.858 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.858 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:19.858 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:19.858 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.858 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.858 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:19.859 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:19.861 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.861 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.861 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:19.861 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:19.861 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.861 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.861 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.862 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:19.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:19.862 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.862 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.862 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:19.862 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.862 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.862 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.864 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:19.864 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.864 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.864 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.864 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:19.864 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:19.864 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.864 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.864 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.865 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:19.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:19.865 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.865 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.865 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:19.865 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.865 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.866 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:19.866 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:19.866 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.866 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.866 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.866 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:19.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:19.866 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:19.867 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:19.868 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.868 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.868 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:19.868 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:19.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.868 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.868 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.868 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.869 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:19.869 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:19.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:19.869 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:19.869 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:19.869 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:19.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:19.869 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:19.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:19.869 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:19.869 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:19.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.818+0000 7f95f3a20640 1 -- 172.21.15.45:0/1445323214 >> v1:172.21.15.84:6812/31478 conn(0x7f95d003aa30 legacy=0x7f95d003cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.818+0000 7f95f3a20640 10 monclient: shutdown 2024-09-18T14:17:19.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.818+0000 7f95f3a20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.818+0000 7f95f3a20640 1 -- 172.21.15.45:0/1445323214 >> v1:172.21.15.45:6790/0 conn(0x7f95ec09f7d0 legacy=0x7f95ec098470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.818+0000 7f95f1fbd640 1 -- 172.21.15.45:0/1445323214 reap_dead start 2024-09-18T14:17:19.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.818+0000 7f95f1fbd640 1 -- 172.21.15.45:0/1445323214 reap_dead start 2024-09-18T14:17:19.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.818+0000 7f95f3a20640 1 -- 172.21.15.45:0/1445323214 shutdown_connections 2024-09-18T14:17:19.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.818+0000 7f95f3a20640 1 -- 172.21.15.45:0/1445323214 >> 172.21.15.45:0/1445323214 conn(0x7f95ec0630b0 msgr2=0x7f95ec131f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.818+0000 7f95f3a20640 1 -- 172.21.15.45:0/1445323214 shutdown_connections 2024-09-18T14:17:19.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.818+0000 7f95f3a20640 1 -- 172.21.15.45:0/1445323214 wait complete. 2024-09-18T14:17:19.872 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:19.872 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:17:19.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:17:19.968 INFO:tasks.rados.rados.0.smithi084.stdout:3780: done (4 left) 2024-09-18T14:17:19.968 INFO:tasks.rados.rados.0.smithi084.stdout:3781: done (3 left) 2024-09-18T14:17:19.968 INFO:tasks.rados.rados.0.smithi084.stdout:3782: done (2 left) 2024-09-18T14:17:19.968 INFO:tasks.rados.rados.0.smithi084.stdout:3785: done (1 left) 2024-09-18T14:17:19.968 INFO:tasks.rados.rados.0.smithi084.stdout:3786: done (0 left) 2024-09-18T14:17:19.968 INFO:tasks.rados.rados.0.smithi084.stdout:3787: read oid 40 snap 387 2024-09-18T14:17:19.968 INFO:tasks.rados.rados.0.smithi084.stdout:3787: expect (ObjNum 1072 snap 341 seq_num 1072) 2024-09-18T14:17:19.968 INFO:tasks.rados.rados.0.smithi084.stdout:3788: read oid 7 snap -1 2024-09-18T14:17:19.968 INFO:tasks.rados.rados.0.smithi084.stdout:3788: expect deleted 2024-09-18T14:17:19.969 INFO:tasks.rados.rados.0.smithi084.stdout:3789: copy_from oid 2 from oid 37 current snap is 422 2024-09-18T14:17:19.969 INFO:tasks.rados.rados.0.smithi084.stdout:3790: delete oid 32 current snap is 422 2024-09-18T14:17:19.970 INFO:tasks.rados.rados.0.smithi084.stdout:3789: finishing copy_from to smithi08434663-2 2024-09-18T14:17:19.970 INFO:tasks.rados.rados.0.smithi084.stdout:3789: got expected ENOENT (src dne) 2024-09-18T14:17:19.970 INFO:tasks.rados.rados.0.smithi084.stdout:3789: finishing copy_from racing read to smithi08434663-2 2024-09-18T14:17:19.972 INFO:tasks.rados.rados.0.smithi084.stdout:3788: done (3 left) 2024-09-18T14:17:19.972 INFO:tasks.rados.rados.0.smithi084.stdout:3789: done (2 left) 2024-09-18T14:17:19.972 INFO:tasks.rados.rados.0.smithi084.stdout:3790: done (1 left) 2024-09-18T14:17:19.972 INFO:tasks.rados.rados.0.smithi084.stdout:3791: snap_create 2024-09-18T14:17:19.974 INFO:tasks.rados.rados.0.smithi084.stdout:3787: expect (ObjNum 1072 snap 341 seq_num 1072) 2024-09-18T14:17:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.974+0000 7ff3cfd5d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.974+0000 7ff3cfd5d640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.974+0000 7ff3cfd5d640 10 monclient: monmap: 2024-09-18T14:17:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:19.975445+0000 2024-09-18T14:17:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:19.975445+0000 2024-09-18T14:17:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.976 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.976 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.976 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.976 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.976 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.976 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 1 Processor -- start 2024-09-18T14:17:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 1 -- start start 2024-09-18T14:17:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 10 monclient: init 2024-09-18T14:17:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 10 monclient: _add_conns ranks=[6,1,0,5,8,4,7,2,3] 2024-09-18T14:17:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 10 monclient(hunting): picked mon.noname-d con 0x7ff3c806e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 10 monclient(hunting): picked mon.noname-f con 0x7ff3c80a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 10 monclient(hunting): picked mon.noname-a con 0x7ff3c80a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c805ed70 con 0x7ff3c80a1080 2024-09-18T14:17:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c8097620 con 0x7ff3c80a0cb0 2024-09-18T14:17:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c8097340 con 0x7ff3c806e810 2024-09-18T14:17:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.975+0000 7ff3cfd5d640 10 monclient(hunting): authenticate will time out at 1855.313978s 2024-09-18T14:17:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3cdaf9640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7ff3c806e810 0x7ff3c80a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43110/0 (socket says 172.21.15.45:43110) 2024-09-18T14:17:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3cdaf9640 1 -- 172.21.15.45:0/3445274438 learned_addr learned my addr 172.21.15.45:0/3445274438 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859274268 0 0) 0x7ff3c805ed70 con 0x7ff3c80a1080 2024-09-18T14:17:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient(hunting): my global_id is 6435 2024-09-18T14:17:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3b40034e0 con 0x7ff3c80a1080 2024-09-18T14:17:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067040761 0 0) 0x7ff3c8097340 con 0x7ff3c806e810 2024-09-18T14:17:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient(hunting): my global_id is 184974 2024-09-18T14:17:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c805ed70 con 0x7ff3c806e810 2024-09-18T14:17:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778166299 0 0) 0x7ff3c8097620 con 0x7ff3c80a0cb0 2024-09-18T14:17:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient(hunting): my global_id is 214417 2024-09-18T14:17:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c8097340 con 0x7ff3c80a0cb0 2024-09-18T14:17:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4129818963 0 0) 0x7ff3b40034e0 con 0x7ff3c80a1080 2024-09-18T14:17:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 >> v1:172.21.15.45:6792/0 conn(0x7ff3c806e810 legacy=0x7ff3c80a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 >> v1:172.21.15.84:6789/0 conn(0x7ff3c80a0cb0 legacy=0x7ff3c8137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient: _finish_hunting 0 2024-09-18T14:17:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 monclient: found mon.noname-a 2024-09-18T14:17:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:17:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c80a1660 con 0x7ff3c80a1080 2024-09-18T14:17:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient: _finish_auth 0 2024-09-18T14:17:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3cfd5d640 5 monclient: authenticate success, global_id 6435 2024-09-18T14:17:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3c0001bb0 con 0x7ff3c80a1080 2024-09-18T14:17:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:17:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient: dump: 2024-09-18T14:17:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.991 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3cfd5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff3c0001d90 con 0x7ff3c80a1080 2024-09-18T14:17:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 1 -- 172.21.15.45:0/3445274438 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3c00022c0 con 0x7ff3c80a1080 2024-09-18T14:17:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3ccaf7640 10 monclient: dump: 2024-09-18T14:17:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:19.993 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3cfd5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.976+0000 7ff3cfd5d640 4 set_mon_vals no callback set 2024-09-18T14:17:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.977+0000 7ff3cfd5d640 10 monclient: shutdown 2024-09-18T14:17:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.977+0000 7ff3cfd5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.977+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/3445274438 >> v1:172.21.15.45:6789/0 conn(0x7ff3c80a1080 legacy=0x7ff3c813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.977+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/3445274438 shutdown_connections 2024-09-18T14:17:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.977+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/3445274438 >> 172.21.15.45:0/3445274438 conn(0x7ff3c80a5bb0 msgr2=0x7ff3c80a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.977+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/3445274438 shutdown_connections 2024-09-18T14:17:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.977+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/3445274438 wait complete. 2024-09-18T14:17:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.977+0000 7ff3cfd5d640 10 monclient: build_initial_monmap 2024-09-18T14:17:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.977+0000 7ff3cfd5d640 10 monclient: monmap: 2024-09-18T14:17:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:19.995 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:19.995 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:19.995 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:19.995 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:19.995 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:19.995 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:19.995 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:19.995 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:19.995 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:19.995 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:19.995 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:19.996 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:19.996 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:19.996 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:19.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.978+0000 7ff3cfd5d640 1 Processor -- start 2024-09-18T14:17:19.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.978+0000 7ff3cfd5d640 1 -- start start 2024-09-18T14:17:19.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.978+0000 7ff3cfd5d640 10 monclient: init 2024-09-18T14:17:19.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.978+0000 7ff3cfd5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:19.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.978+0000 7ff3cfd5d640 10 monclient: _add_conns ranks=[4,1,5,3,7,6,2,8,0] 2024-09-18T14:17:19.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.978+0000 7ff3cfd5d640 10 monclient(hunting): picked mon.noname-c con 0x7ff3c806e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:19.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.978+0000 7ff3cfd5d640 10 monclient(hunting): picked mon.noname-f con 0x7ff3c80a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:19.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.978+0000 7ff3cfd5d640 10 monclient(hunting): picked mon.noname-h con 0x7ff3c80a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:19.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3cfd5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c8122be0 con 0x7ff3c80a0cb0 2024-09-18T14:17:19.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3cfd5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c813d9d0 con 0x7ff3c806e810 2024-09-18T14:17:19.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3cfd5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c8122620 con 0x7ff3c80a1080 2024-09-18T14:17:19.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3cfd5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:19.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3cfd5d640 10 monclient(hunting): authenticate will time out at 1855.317088s 2024-09-18T14:17:19.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3cdaf9640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7ff3c806e810 0x7ff3c813d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:40990/0 (socket says 172.21.15.45:40990) 2024-09-18T14:17:19.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3cdaf9640 1 -- 172.21.15.45:0/470833239 learned_addr learned my addr 172.21.15.45:0/470833239 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:19.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3cd2f8640 1 --1- 172.21.15.45:0/470833239 >> v1:172.21.15.84:6789/0 conn(0x7ff3c80a0cb0 0x7ff3c81464f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:42234/0 (socket says 172.21.15.45:42234) 2024-09-18T14:17:19.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013280586 0 0) 0x7ff3c813d9d0 con 0x7ff3c806e810 2024-09-18T14:17:19.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): my global_id is 165154 2024-09-18T14:17:19.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3ac0034e0 con 0x7ff3c806e810 2024-09-18T14:17:19.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284742796 0 0) 0x7ff3c8122be0 con 0x7ff3c80a0cb0 2024-09-18T14:17:19.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): my global_id is 214426 2024-09-18T14:17:19.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c813d9d0 con 0x7ff3c80a0cb0 2024-09-18T14:17:19.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968370228 0 0) 0x7ff3c8122620 con 0x7ff3c80a1080 2024-09-18T14:17:19.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): my global_id is 184820 2024-09-18T14:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c8122be0 con 0x7ff3c80a1080 2024-09-18T14:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 805225928 0 0) 0x7ff3c813d9d0 con 0x7ff3c80a0cb0 2024-09-18T14:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3c8122620 con 0x7ff3c80a0cb0 2024-09-18T14:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3b8001c50 con 0x7ff3c80a0cb0 2024-09-18T14:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): dump: 2024-09-18T14:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.000 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.000 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.000 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.000 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.000 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.000 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.000 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.000 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.000 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.000 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.000 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.000 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 348635825 0 0) 0x7ff3c8122be0 con 0x7ff3c80a1080 2024-09-18T14:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3c813d9d0 con 0x7ff3c80a1080 2024-09-18T14:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2535980695 0 0) 0x7ff3ac0034e0 con 0x7ff3c806e810 2024-09-18T14:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3c8122be0 con 0x7ff3c806e810 2024-09-18T14:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3c00013a0 con 0x7ff3c80a1080 2024-09-18T14:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): dump: 2024-09-18T14:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3c40015d0 con 0x7ff3c806e810 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient(hunting): dump: 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.004 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.004 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.004 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.004 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.004 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.004 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.004 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.004 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.004 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.004 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.004 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.005 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1555270128 0 0) 0x7ff3c8122620 con 0x7ff3c80a0cb0 2024-09-18T14:17:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 >> v1:172.21.15.84:6791/0 conn(0x7ff3c80a1080 legacy=0x7ff3c8149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 >> v1:172.21.15.45:6791/0 conn(0x7ff3c806e810 legacy=0x7ff3c813d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient: _finish_hunting 0 2024-09-18T14:17:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 monclient: found mon.f 2024-09-18T14:17:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c81201c0 con 0x7ff3c80a0cb0 2024-09-18T14:17:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient: _finish_auth 0 2024-09-18T14:17:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3cfd5d640 5 monclient: authenticate success, global_id 214426 2024-09-18T14:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3cfd5d640 10 monclient: _renew_subs 2024-09-18T14:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3cfd5d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c8122340 con 0x7ff3c80a0cb0 2024-09-18T14:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff3b80016a0 con 0x7ff3c80a0cb0 2024-09-18T14:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3b8002420 con 0x7ff3c80a0cb0 2024-09-18T14:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.980+0000 7ff3bdffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.980+0000 7ff3bdffb640 10 monclient: dump: 2024-09-18T14:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.980+0000 7ff3cfd5d640 10 monclient: _renew_subs 2024-09-18T14:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.979+0000 7ff3bffff640 4 set_mon_vals no callback set 2024-09-18T14:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.980+0000 7ff3cfd5d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.980+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c8121470 con 0x7ff3c80a0cb0 2024-09-18T14:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.980+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff3b8002600 con 0x7ff3c80a0cb0 2024-09-18T14:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.980+0000 7ff3cfd5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.980+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff3c804c4a0 con 0x7ff3c813e570 2024-09-18T14:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(847..847 src has 529..847) v4 ==== 3927+0+0 (unknown 3776827267 0 0) 0x7ff3b8046a30 con 0x7ff3c80a0cb0 2024-09-18T14:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3ce2fa640 10 monclient: get_auth_request con 0x7ff3ac03ab20 auth_method 0 2024-09-18T14:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858679963 0 0) 0x7ff3c804c4a0 con 0x7ff3c813e570 2024-09-18T14:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 10 monclient(hunting): my global_id is 184829 2024-09-18T14:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c8122620 con 0x7ff3c813e570 2024-09-18T14:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2923171843 0 0) 0x7ff3c8122620 con 0x7ff3c813e570 2024-09-18T14:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3c804c4a0 con 0x7ff3c813e570 2024-09-18T14:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3c4004cd0 con 0x7ff3c813e570 2024-09-18T14:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 10 monclient: dump: 2024-09-18T14:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.011 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2337549622 0 0) 0x7ff3c804c4a0 con 0x7ff3c813e570 2024-09-18T14:17:20.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.981+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3c804b720 con 0x7ff3c813e570 2024-09-18T14:17:20.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.982+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff3c4006fc0 con 0x7ff3c813e570 2024-09-18T14:17:20.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.982+0000 7ff3bdffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:20.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.982+0000 7ff3bdffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:20.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.982+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 >> v1:172.21.15.84:6791/0 conn(0x7ff3c813e570 legacy=0x7ff3c809ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.982+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 >> v1:172.21.15.84:6791/0 conn(0x7ff3c813e570 legacy=0x7ff3c809ba60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.987+0000 7ff3cfd5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.987+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff3c813dfc0 con 0x7ff3c813dbf0 2024-09-18T14:17:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452203451 0 0) 0x7ff3c813dfc0 con 0x7ff3c813dbf0 2024-09-18T14:17:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 10 monclient(hunting): my global_id is 184838 2024-09-18T14:17:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c804c4a0 con 0x7ff3c813dbf0 2024-09-18T14:17:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2480153743 0 0) 0x7ff3c804c4a0 con 0x7ff3c813dbf0 2024-09-18T14:17:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3c813dfc0 con 0x7ff3c813dbf0 2024-09-18T14:17:20.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff3c4005630 con 0x7ff3c813dbf0 2024-09-18T14:17:20.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:20.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 10 monclient: dump: 2024-09-18T14:17:20.014 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.014 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.014 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.014 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.014 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.014 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.015 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.015 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.015 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.015 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.015 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.015 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.015 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.015 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.015 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.015 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4145848234 0 0) 0x7ff3c813dfc0 con 0x7ff3c813dbf0 2024-09-18T14:17:20.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff3c8062dd0 con 0x7ff3c813dbf0 2024-09-18T14:17:20.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 2358029383) 0x7ff3c4006a00 con 0x7ff3c813dbf0 2024-09-18T14:17:20.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:20.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:20.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 >> v1:172.21.15.84:6791/0 conn(0x7ff3c813dbf0 legacy=0x7ff3c809c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.988+0000 7ff3bdffb640 1 -- 172.21.15.45:0/470833239 >> v1:172.21.15.84:6791/0 conn(0x7ff3c813dbf0 legacy=0x7ff3c809c170 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:20.016 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:20.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:20.017 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:20.017 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:20.017 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:20.017 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:20.017 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:20.017 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:20.017 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:20.017 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:20.017 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:20.017 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:20.018 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:20.018 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:20.018 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:20.018 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:20.018 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:17:20.018 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:20.018 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:20.018 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:20.018 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:20.018 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:20.018 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:20.018 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:20.019 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:20.019 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:20.019 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:20.019 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:20.019 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:20.019 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:20.019 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:20.019 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:20.019 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:20.019 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:20.019 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:20.019 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:20.020 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:20.020 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:20.020 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:20.020 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:20.020 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:20.020 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.020 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:20.020 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:20.020 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:20.021 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:20.021 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:20.021 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:20.021 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:20.021 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:20.021 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:20.021 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:20.021 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:20.021 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:20.021 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:20.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:20.022 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:20.022 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:20.022 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:20.022 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:20.022 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:20.022 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:20.022 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:20.022 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:20.022 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:20.022 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:20.022 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:20.023 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:20.023 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:20.023 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:20.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.023 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:20.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.023 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:20.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:20.023 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.024 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.024 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.024 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:20.024 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.024 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.024 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:20.024 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:20.024 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.025 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.025 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.025 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:20.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:20.025 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.025 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.026 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:20.026 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.026 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.026 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.026 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:20.026 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:20.026 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.026 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.026 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.027 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:20.027 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:20.027 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.027 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.027 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.027 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:20.027 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.027 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.028 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.028 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:20.028 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:20.028 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.028 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.028 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.028 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:20.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:20.029 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.029 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.029 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:20.029 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.029 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.029 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.029 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:20.030 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:20.030 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.030 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.030 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.030 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:20.030 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:20.030 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.030 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.031 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:20.031 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.031 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.031 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.031 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:20.031 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:20.031 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.031 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.031 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.032 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:20.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:20.032 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.032 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.032 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.032 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:20.032 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.032 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.032 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.033 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:20.033 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:20.033 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.033 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.033 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.033 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:20.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:20.033 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.033 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.034 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.034 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:20.034 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.034 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.034 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.034 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:20.034 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:20.034 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.034 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.035 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.037 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:20.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:20.038 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:20.038 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.038 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.038 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:20.038 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:20.039 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.039 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.039 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.039 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:20.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:20.039 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.039 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.040 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:20.040 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.040 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.040 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.040 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:20.040 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:20.040 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.040 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.040 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.040 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.041 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.041 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:20.041 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:20.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.041 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:20.041 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:20.041 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:20.041 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.041 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:20.041 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:20.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.042 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:20.042 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:20.042 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:20.042 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.042 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:20.042 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:17:20.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.042 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:20.042 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:20.042 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:20.042 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.043 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.043 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:20.043 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:20.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.989+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/470833239 >> v1:172.21.15.84:6812/31478 conn(0x7ff3ac03ab20 legacy=0x7ff3ac03cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.989+0000 7ff3cfd5d640 10 monclient: shutdown 2024-09-18T14:17:20.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.989+0000 7ff3cfd5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:20.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.989+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/470833239 >> v1:172.21.15.84:6789/0 conn(0x7ff3c80a0cb0 legacy=0x7ff3c81464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.989+0000 7ff3ce2fa640 1 -- 172.21.15.45:0/470833239 reap_dead start 2024-09-18T14:17:20.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.989+0000 7ff3ce2fa640 1 -- 172.21.15.45:0/470833239 reap_dead start 2024-09-18T14:17:20.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.989+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/470833239 shutdown_connections 2024-09-18T14:17:20.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.989+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/470833239 >> 172.21.15.45:0/470833239 conn(0x7ff3c80a5bb0 msgr2=0x7ff3c80a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:20.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.989+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/470833239 shutdown_connections 2024-09-18T14:17:20.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:19.989+0000 7ff3cfd5d640 1 -- 172.21.15.45:0/470833239 wait complete. 2024-09-18T14:17:20.046 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:20.046 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:17:20.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:17:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.143+0000 7f44d2a68640 10 monclient: get_monmap_and_config 2024-09-18T14:17:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.143+0000 7f44d2a68640 10 monclient: build_initial_monmap 2024-09-18T14:17:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.143+0000 7f44d2a68640 10 monclient: monmap: 2024-09-18T14:17:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:20.144512+0000 2024-09-18T14:17:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:20.144512+0000 2024-09-18T14:17:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:20.145 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:20.145 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.144+0000 7f44d2a68640 1 Processor -- start 2024-09-18T14:17:20.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.144+0000 7f44d2a68640 1 -- start start 2024-09-18T14:17:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.144+0000 7f44d2a68640 10 monclient: init 2024-09-18T14:17:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.144+0000 7f44d2a68640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.144+0000 7f44d2a68640 10 monclient: _add_conns ranks=[6,4,1,2,3,0,5,8,7] 2024-09-18T14:17:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.144+0000 7f44d2a68640 10 monclient(hunting): picked mon.noname-d con 0x7f44cc09f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.144+0000 7f44d2a68640 10 monclient(hunting): picked mon.noname-c con 0x7f44cc09fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.144+0000 7f44d2a68640 10 monclient(hunting): picked mon.noname-f con 0x7f44cc09ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.144+0000 7f44d2a68640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc05ed70 con 0x7f44cc09ff70 2024-09-18T14:17:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44d2a68640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc09dfc0 con 0x7f44cc09fba0 2024-09-18T14:17:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cbfff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f44cc09fba0 0x7f44cc137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:41006/0 (socket says 172.21.15.45:41006) 2024-09-18T14:17:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cbfff640 1 -- 172.21.15.45:0/1451285008 learned_addr learned my addr 172.21.15.45:0/1451285008 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44d2a68640 1 -- 172.21.15.45:0/1451285008 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc09dce0 con 0x7f44cc09f7d0 2024-09-18T14:17:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44d2a68640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44d2a68640 10 monclient(hunting): authenticate will time out at 1855.483274s 2024-09-18T14:17:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418194403 0 0) 0x7f44cc09dfc0 con 0x7f44cc09fba0 2024-09-18T14:17:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient(hunting): my global_id is 165163 2024-09-18T14:17:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44bc0034e0 con 0x7f44cc09fba0 2024-09-18T14:17:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957457127 0 0) 0x7f44cc05ed70 con 0x7f44cc09ff70 2024-09-18T14:17:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient(hunting): my global_id is 214435 2024-09-18T14:17:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44cc09dfc0 con 0x7f44cc09ff70 2024-09-18T14:17:20.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67088591 0 0) 0x7f44cc09dce0 con 0x7f44cc09f7d0 2024-09-18T14:17:20.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient(hunting): my global_id is 184983 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44cc05ed70 con 0x7f44cc09f7d0 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 755596820 0 0) 0x7f44cc09dfc0 con 0x7f44cc09ff70 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 >> v1:172.21.15.45:6792/0 conn(0x7f44cc09f7d0 legacy=0x7f44cc132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 >> v1:172.21.15.45:6791/0 conn(0x7f44cc09fba0 legacy=0x7f44cc137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 monclient: found mon.noname-f 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc0a1c00 con 0x7f44cc09ff70 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient: _finish_auth 0 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:20.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f44c4001b90 con 0x7f44cc09ff70 2024-09-18T14:17:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:17:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient: dump: 2024-09-18T14:17:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44d2a68640 5 monclient: authenticate success, global_id 214435 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44d2a68640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f44c4001cf0 con 0x7f44cc09ff70 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 1 -- 172.21.15.45:0/1451285008 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f44c40022a0 con 0x7f44cc09ff70 2024-09-18T14:17:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44cb7fe640 10 monclient: dump: 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.160 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44d2a68640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.145+0000 7f44d2a68640 4 set_mon_vals no callback set 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.146+0000 7f44d2a68640 10 monclient: shutdown 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.146+0000 7f44d2a68640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.146+0000 7f44d2a68640 1 -- 172.21.15.45:0/1451285008 >> v1:172.21.15.84:6789/0 conn(0x7f44cc09ff70 legacy=0x7f44cc13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.146+0000 7f44d2a68640 1 -- 172.21.15.45:0/1451285008 shutdown_connections 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.146+0000 7f44d2a68640 1 -- 172.21.15.45:0/1451285008 >> 172.21.15.45:0/1451285008 conn(0x7f44cc0630b0 msgr2=0x7f44cc0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.146+0000 7f44d2a68640 1 -- 172.21.15.45:0/1451285008 shutdown_connections 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.146+0000 7f44d2a68640 1 -- 172.21.15.45:0/1451285008 wait complete. 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.146+0000 7f44d2a68640 10 monclient: build_initial_monmap 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.146+0000 7f44d2a68640 10 monclient: monmap: 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:20.163 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.147+0000 7f44d2a68640 1 Processor -- start 2024-09-18T14:17:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.147+0000 7f44d2a68640 1 -- start start 2024-09-18T14:17:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.147+0000 7f44d2a68640 10 monclient: init 2024-09-18T14:17:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.147+0000 7f44d2a68640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.147+0000 7f44d2a68640 10 monclient: _add_conns ranks=[7,2,1,5,8,3,0,4,6] 2024-09-18T14:17:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.147+0000 7f44d2a68640 10 monclient(hunting): picked mon.noname-i con 0x7f44cc09f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:17:20.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 10 monclient(hunting): picked mon.noname-b con 0x7f44cc09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 10 monclient(hunting): picked mon.noname-f con 0x7f44cc09ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc0a2560 con 0x7f44cc09ff70 2024-09-18T14:17:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc121a20 con 0x7f44cc09fba0 2024-09-18T14:17:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc121d00 con 0x7f44cc09f7d0 2024-09-18T14:17:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 10 monclient(hunting): authenticate will time out at 1855.486141s 2024-09-18T14:17:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44cbfff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f44cc09fba0 0x7f44cc146a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60868/0 (socket says 172.21.15.45:60868) 2024-09-18T14:17:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d0804640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f44cc09f7d0 0x7f44cc1315f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:52278/0 (socket says 172.21.15.45:52278) 2024-09-18T14:17:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44cbfff640 1 -- 172.21.15.45:0/3890287313 learned_addr learned my addr 172.21.15.45:0/3890287313 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842963003 0 0) 0x7f44cc121a20 con 0x7f44cc09fba0 2024-09-18T14:17:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): my global_id is 145559 2024-09-18T14:17:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44b00034e0 con 0x7f44cc09fba0 2024-09-18T14:17:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378102171 0 0) 0x7f44cc121d00 con 0x7f44cc09f7d0 2024-09-18T14:17:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): my global_id is 214432 2024-09-18T14:17:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44cc121a20 con 0x7f44cc09f7d0 2024-09-18T14:17:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981305719 0 0) 0x7f44cc0a2560 con 0x7f44cc09ff70 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): my global_id is 214444 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44cc121d00 con 0x7f44cc09ff70 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4156360900 0 0) 0x7f44b00034e0 con 0x7f44cc09fba0 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f44cc0a2560 con 0x7f44cc09fba0 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f44b8001720 con 0x7f44cc09fba0 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): dump: 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.167 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3886030662 0 0) 0x7f44cc121a20 con 0x7f44cc09f7d0 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f44b00034e0 con 0x7f44cc09f7d0 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f44c0001a50 con 0x7f44cc09f7d0 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): dump: 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.169 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1504800343 0 0) 0x7f44cc121d00 con 0x7f44cc09ff70 2024-09-18T14:17:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f44cc121a20 con 0x7f44cc09ff70 2024-09-18T14:17:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f44c40021a0 con 0x7f44cc09ff70 2024-09-18T14:17:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient(hunting): dump: 2024-09-18T14:17:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.170 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1944312059 0 0) 0x7f44cc0a2560 con 0x7f44cc09fba0 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 >> v1:172.21.15.84:6792/0 conn(0x7f44cc09f7d0 legacy=0x7f44cc1315f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 >> v1:172.21.15.84:6789/0 conn(0x7f44cc09ff70 legacy=0x7f44cc14a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:17:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 monclient: found mon.b 2024-09-18T14:17:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc1207e0 con 0x7f44cc09fba0 2024-09-18T14:17:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 10 monclient: _finish_auth 0 2024-09-18T14:17:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44c8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 5 monclient: authenticate success, global_id 145559 2024-09-18T14:17:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 10 monclient: _renew_subs 2024-09-18T14:17:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc133bf0 con 0x7f44cc09fba0 2024-09-18T14:17:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 10 monclient: _renew_subs 2024-09-18T14:17:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.148+0000 7f44d2a68640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f44cc142580 con 0x7f44cc09fba0 2024-09-18T14:17:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.149+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f44b80018e0 con 0x7f44cc09fba0 2024-09-18T14:17:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.149+0000 7f44c8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.149+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f44b8004e10 con 0x7f44cc09fba0 2024-09-18T14:17:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.149+0000 7f44c8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.149+0000 7f44caffd640 4 set_mon_vals no callback set 2024-09-18T14:17:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.149+0000 7f44c8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.149+0000 7f44c8ff9640 10 monclient: dump: 2024-09-18T14:17:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.173 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.149+0000 7f44d2a68640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.149+0000 7f44d2a68640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f44ac006580 con 0x7f44ac002830 2024-09-18T14:17:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.149+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f44b800fde0 con 0x7f44cc09fba0 2024-09-18T14:17:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.149+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(847..847 src has 529..847) v4 ==== 3927+0+0 (unknown 3776827267 0 0) 0x7f44b8046ae0 con 0x7f44cc09fba0 2024-09-18T14:17:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2996091428 0 0) 0x7f44ac006580 con 0x7f44ac002830 2024-09-18T14:17:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 10 monclient(hunting): my global_id is 214441 2024-09-18T14:17:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44cc0a2560 con 0x7f44ac002830 2024-09-18T14:17:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44d1005640 10 monclient: get_auth_request con 0x7f44b003ab40 auth_method 0 2024-09-18T14:17:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 775713374 0 0) 0x7f44cc0a2560 con 0x7f44ac002830 2024-09-18T14:17:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f44ac006580 con 0x7f44ac002830 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f44c0004b40 con 0x7f44ac002830 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 10 monclient: dump: 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.176 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 539241617 0 0) 0x7f44ac006580 con 0x7f44ac002830 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44ac00a0a0 con 0x7f44ac002830 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f44c0006fc0 con 0x7f44ac002830 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:20.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 >> v1:172.21.15.84:6792/0 conn(0x7f44ac002830 legacy=0x7f44ac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.150+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 >> v1:172.21.15.84:6792/0 conn(0x7f44ac002830 legacy=0x7f44ac004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.156+0000 7f44d2a68640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.156+0000 7f44d2a68640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f44ac0095f0 con 0x7f44ac008ff0 2024-09-18T14:17:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335240082 0 0) 0x7f44ac0095f0 con 0x7f44ac008ff0 2024-09-18T14:17:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 10 monclient(hunting): my global_id is 214450 2024-09-18T14:17:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44ac006580 con 0x7f44ac008ff0 2024-09-18T14:17:20.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2504538385 0 0) 0x7f44ac006580 con 0x7f44ac008ff0 2024-09-18T14:17:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f44ac0095f0 con 0x7f44ac008ff0 2024-09-18T14:17:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f44c00055b0 con 0x7f44ac008ff0 2024-09-18T14:17:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:17:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 10 monclient: dump: 2024-09-18T14:17:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2437422706 0 0) 0x7f44ac0095f0 con 0x7f44ac008ff0 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f44ac0098b0 con 0x7f44ac008ff0 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 563025832) 0x7f44c0006a10 con 0x7f44ac008ff0 2024-09-18T14:17:20.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:20.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:20.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 >> v1:172.21.15.84:6792/0 conn(0x7f44ac008ff0 legacy=0x7f44ac008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.157+0000 7f44c8ff9640 1 -- 172.21.15.45:0/3890287313 >> v1:172.21.15.84:6792/0 conn(0x7f44ac008ff0 legacy=0x7f44ac008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:20.182 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:20.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:20.203 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:20.203 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.158+0000 7f44d2a68640 1 -- 172.21.15.45:0/3890287313 >> v1:172.21.15.84:6812/31478 conn(0x7f44b003ab40 legacy=0x7f44b003cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.158+0000 7f44d2a68640 10 monclient: shutdown 2024-09-18T14:17:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.158+0000 7f44d2a68640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.158+0000 7f44d2a68640 1 -- 172.21.15.45:0/3890287313 >> v1:172.21.15.45:6790/0 conn(0x7f44cc09fba0 legacy=0x7f44cc146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.158+0000 7f44d1005640 1 -- 172.21.15.45:0/3890287313 reap_dead start 2024-09-18T14:17:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.158+0000 7f44d1005640 1 -- 172.21.15.45:0/3890287313 reap_dead start 2024-09-18T14:17:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.158+0000 7f44d2a68640 1 -- 172.21.15.45:0/3890287313 shutdown_connections 2024-09-18T14:17:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.158+0000 7f44d2a68640 1 -- 172.21.15.45:0/3890287313 >> 172.21.15.45:0/3890287313 conn(0x7f44cc0630b0 msgr2=0x7f44cc0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.158+0000 7f44d2a68640 1 -- 172.21.15.45:0/3890287313 shutdown_connections 2024-09-18T14:17:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.158+0000 7f44d2a68640 1 -- 172.21.15.45:0/3890287313 wait complete. 2024-09-18T14:17:20.205 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:20.205 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-18T14:17:20.205 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-18T14:17:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.300+0000 7f1e2479d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.300+0000 7f1e2479d640 10 monclient: build_initial_monmap 2024-09-18T14:17:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.300+0000 7f1e2479d640 10 monclient: monmap: 2024-09-18T14:17:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:20.301642+0000 2024-09-18T14:17:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:20.301642+0000 2024-09-18T14:17:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:20.303 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 1 Processor -- start 2024-09-18T14:17:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 1 -- start start 2024-09-18T14:17:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 10 monclient: init 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 10 monclient: _add_conns ranks=[4,2,1,5,6,3,0,7,8] 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 10 monclient(hunting): picked mon.noname-c con 0x7f1e1c09f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 10 monclient(hunting): picked mon.noname-b con 0x7f1e1c09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 10 monclient(hunting): picked mon.noname-f con 0x7f1e1c09ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c05ed70 con 0x7f1e1c09ff70 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c09dfc0 con 0x7f1e1c09fba0 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c09dce0 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.301+0000 7f1e2479d640 10 monclient(hunting): authenticate will time out at 1855.639999s 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21d38640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f1e1c09fba0 0x7f1e1c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60884/0 (socket says 172.21.15.45:60884) 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e22539640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f1e1c09f7d0 0x7f1e1c132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:41020/0 (socket says 172.21.15.45:41020) 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21d38640 1 -- 172.21.15.45:0/2539877922 learned_addr learned my addr 172.21.15.45:0/2539877922 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 1 -- 172.21.15.45:0/2539877922 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472298161 0 0) 0x7f1e1c09dce0 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient(hunting): my global_id is 165172 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 1 -- 172.21.15.45:0/2539877922 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e080034e0 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 1 -- 172.21.15.45:0/2539877922 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275904053 0 0) 0x7f1e1c09dfc0 con 0x7f1e1c09fba0 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient(hunting): my global_id is 145568 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 1 -- 172.21.15.45:0/2539877922 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e1c09dce0 con 0x7f1e1c09fba0 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 1 -- 172.21.15.45:0/2539877922 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2192989104 0 0) 0x7f1e080034e0 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 1 -- 172.21.15.45:0/2539877922 >> v1:172.21.15.45:6790/0 conn(0x7f1e1c09fba0 legacy=0x7f1e1c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 1 -- 172.21.15.45:0/2539877922 >> v1:172.21.15.84:6789/0 conn(0x7f1e1c09ff70 legacy=0x7f1e1c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient: _finish_hunting 0 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 1 monclient: found mon.noname-c 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 1 -- 172.21.15.45:0/2539877922 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c0a1c00 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient: _finish_auth 0 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 1 -- 172.21.15.45:0/2539877922 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1e18001830 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e2479d640 5 monclient: authenticate success, global_id 165172 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e2479d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient: dump: 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.313 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e2479d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 1 -- 172.21.15.45:0/2539877922 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e18001990 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e2479d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 1 -- 172.21.15.45:0/2539877922 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1e18004ae0 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e2479d640 4 set_mon_vals no callback set 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.302+0000 7f1e21537640 10 monclient: dump: 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.315 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.303+0000 7f1e2479d640 10 monclient: shutdown 2024-09-18T14:17:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.303+0000 7f1e2479d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.303+0000 7f1e2479d640 1 -- 172.21.15.45:0/2539877922 >> v1:172.21.15.45:6791/0 conn(0x7f1e1c09f7d0 legacy=0x7f1e1c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.303+0000 7f1e2479d640 1 -- 172.21.15.45:0/2539877922 shutdown_connections 2024-09-18T14:17:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.303+0000 7f1e2479d640 1 -- 172.21.15.45:0/2539877922 >> 172.21.15.45:0/2539877922 conn(0x7f1e1c0630b0 msgr2=0x7f1e1c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.303+0000 7f1e2479d640 1 -- 172.21.15.45:0/2539877922 shutdown_connections 2024-09-18T14:17:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.303+0000 7f1e2479d640 1 -- 172.21.15.45:0/2539877922 wait complete. 2024-09-18T14:17:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.303+0000 7f1e2479d640 10 monclient: build_initial_monmap 2024-09-18T14:17:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.303+0000 7f1e2479d640 10 monclient: monmap: 2024-09-18T14:17:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:20.317 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 1 Processor -- start 2024-09-18T14:17:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 1 -- start start 2024-09-18T14:17:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 10 monclient: init 2024-09-18T14:17:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 10 monclient: _add_conns ranks=[2,6,5,1,4,8,0,3,7] 2024-09-18T14:17:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 10 monclient(hunting): picked mon.noname-b con 0x7f1e1c09f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 10 monclient(hunting): picked mon.noname-d con 0x7f1e1c09fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 10 monclient(hunting): picked mon.noname-h con 0x7f1e1c09ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c11ee20 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c09d310 con 0x7f1e1c09ff70 2024-09-18T14:17:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c0a2560 con 0x7f1e1c09fba0 2024-09-18T14:17:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e2479d640 10 monclient(hunting): authenticate will time out at 1855.642756s 2024-09-18T14:17:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e22539640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f1e1c09f7d0 0x7f1e1c11cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60894/0 (socket says 172.21.15.45:60894) 2024-09-18T14:17:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.304+0000 7f1e22539640 1 -- 172.21.15.45:0/643438107 learned_addr learned my addr 172.21.15.45:0/643438107 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577946379 0 0) 0x7f1e1c11ee20 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): my global_id is 145577 2024-09-18T14:17:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e000034e0 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402206226 0 0) 0x7f1e1c0a2560 con 0x7f1e1c09fba0 2024-09-18T14:17:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): my global_id is 184992 2024-09-18T14:17:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e1c11ee20 con 0x7f1e1c09fba0 2024-09-18T14:17:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987491014 0 0) 0x7f1e1c09d310 con 0x7f1e1c09ff70 2024-09-18T14:17:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): my global_id is 184847 2024-09-18T14:17:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e1c0a2560 con 0x7f1e1c09ff70 2024-09-18T14:17:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 139931273 0 0) 0x7f1e000034e0 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e1c09d310 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1e18001430 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:17:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): dump: 2024-09-18T14:17:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.322 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1048321496 0 0) 0x7f1e1c11ee20 con 0x7f1e1c09fba0 2024-09-18T14:17:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e000034e0 con 0x7f1e1c09fba0 2024-09-18T14:17:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1e0c0014b0 con 0x7f1e1c09fba0 2024-09-18T14:17:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): dump: 2024-09-18T14:17:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.324 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 860640400 0 0) 0x7f1e1c0a2560 con 0x7f1e1c09ff70 2024-09-18T14:17:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e1c11ee20 con 0x7f1e1c09ff70 2024-09-18T14:17:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1e14001b90 con 0x7f1e1c09ff70 2024-09-18T14:17:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient(hunting): dump: 2024-09-18T14:17:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.325 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3942903737 0 0) 0x7f1e1c09d310 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 >> v1:172.21.15.45:6792/0 conn(0x7f1e1c09fba0 legacy=0x7f1e1c11d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 >> v1:172.21.15.84:6791/0 conn(0x7f1e1c09ff70 legacy=0x7f1e1c149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient: _finish_hunting 0 2024-09-18T14:17:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 monclient: found mon.b 2024-09-18T14:17:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c11e8a0 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient: _finish_auth 0 2024-09-18T14:17:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e2479d640 5 monclient: authenticate success, global_id 145577 2024-09-18T14:17:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e2479d640 10 monclient: _renew_subs 2024-09-18T14:17:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e2479d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e2479d640 1 -- 172.21.15.45:0/643438107 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c120360 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e2479d640 10 monclient: _renew_subs 2024-09-18T14:17:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e2479d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e2479d640 1 -- 172.21.15.45:0/643438107 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c11f470 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e18004d60 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1e18005210 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e20d36640 4 set_mon_vals no callback set 2024-09-18T14:17:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.305+0000 7f1e127fc640 10 monclient: dump: 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:20.329 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.306+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1e180101e0 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.306+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(847..847 src has 529..847) v4 ==== 3927+0+0 (unknown 3776827267 0 0) 0x7f1e18046d80 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.306+0000 7f1e2479d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.306+0000 7f1e2479d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.306+0000 7f1e2479d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.306+0000 7f1e2479d640 1 -- 172.21.15.45:0/643438107 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e04005010 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.306+0000 7f1e21d38640 10 monclient: get_auth_request con 0x7f1e0003a9b0 auth_method 0 2024-09-18T14:17:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.311+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f1e18015190 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.311+0000 7f1e127fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.311+0000 7f1e127fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:20.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.538+0000 7f1e2479d640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-18T14:17:20.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.538+0000 7f1e2479d640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-18T14:17:20.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.538+0000 7f1e2479d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:20.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.538+0000 7f1e2479d640 1 -- 172.21.15.45:0/643438107 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f1e04005ad0 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.539+0000 7f1e127fc640 1 -- 172.21.15.45:0/643438107 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f1e18014fb0 con 0x7f1e1c09f7d0 2024-09-18T14:17:20.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.539+0000 7f1e127fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-18T14:17:20.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.539+0000 7f1e127fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:20.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=2101173442,config=3140514009,health=4204155350,kv=594742435,logm=3984480330}) 2024-09-18T14:17:20.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=2,health=11,kv=7,logm=53} crc {auth=3763198310,config=3140514009,health=2253591146,kv=4140378858,logm=3288882976}) 2024-09-18T14:17:20.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=2101173442,config=3140514009,health=4204155350,kv=594742435,logm=3984480330}) 2024-09-18T14:17:20.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=26,config=2,health=10,kv=8,logm=54} crc {auth=531341452,config=3140514009,health=2355373623,kv=594742435,logm=882328392}) 2024-09-18T14:17:20.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.539+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=2101173442,config=3140514009,health=4204155350,kv=594742435,logm=3984480330}) 2024-09-18T14:17:20.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=26,config=2,health=11,kv=7,logm=54} crc {auth=2101173442,config=3140514009,health=2253591146,kv=594742435,logm=1555188882}) 2024-09-18T14:17:20.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=2101173442,config=3140514009,health=4204155350,kv=594742435,logm=3984480330}) 2024-09-18T14:17:20.543 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=25,config=2,health=12,kv=7,logm=54} crc {auth=2471967528,config=3140514009,health=4204155350,kv=11119927,logm=1970537142}) 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=2101173442,config=3140514009,health=4204155350,kv=594742435,logm=3984480330}) 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=23,config=1,health=12,kv=8,logm=56} crc {auth=1331156363,config=1228700967,health=4204155350,kv=681570228,logm=3350519317}) 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=2101173442,config=3140514009,health=4204155350,kv=594742435,logm=3984480330}) 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=1,health=10,kv=7,logm=56} crc {auth=4281549647,config=1228700967,health=2970778479,kv=4171427160,logm=269715853}) 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=2101173442,config=3140514009,health=4204155350,kv=594742435,logm=3984480330}) 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=177480804,config=3140514009,health=3051900221,kv=594742435,logm=1307244109}) 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=2101173442,config=3140514009,health=4204155350,kv=594742435,logm=3984480330}) 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.540+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=4149313649,config=3140514009,health=4204155350,kv=594742435,logm=1043576982}) 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.544 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144639790}) 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=659041518}) 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144639790}) 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1323167654}) 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144639790}) 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2453909017}) 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144639790}) 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2754796229}) 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144639790}) 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2489416444}) 2024-09-18T14:17:20.545 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.546 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144639790}) 2024-09-18T14:17:20.546 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3010617314}) 2024-09-18T14:17:20.546 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.546 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144639790}) 2024-09-18T14:17:20.546 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=325366524}) 2024-09-18T14:17:20.546 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.546 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144639790}) 2024-09-18T14:17:20.546 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.541+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=665321232}) 2024-09-18T14:17:20.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e2479d640 1 -- 172.21.15.45:0/643438107 >> v1:172.21.15.84:6812/31478 conn(0x7f1e0003a9b0 legacy=0x7f1e0003ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e2479d640 10 monclient: shutdown 2024-09-18T14:17:20.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e2479d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:20.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e2479d640 1 -- 172.21.15.45:0/643438107 >> v1:172.21.15.45:6790/0 conn(0x7f1e1c09f7d0 legacy=0x7f1e1c11cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:20.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e2479d640 1 -- 172.21.15.45:0/643438107 shutdown_connections 2024-09-18T14:17:20.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e2479d640 1 -- 172.21.15.45:0/643438107 >> 172.21.15.45:0/643438107 conn(0x7f1e1c0630b0 msgr2=0x7f1e1c0a4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:20.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e2479d640 1 -- 172.21.15.45:0/643438107 shutdown_connections 2024-09-18T14:17:20.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e2479d640 1 -- 172.21.15.45:0/643438107 wait complete. 2024-09-18T14:17:20.559 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.559 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3706774862}) 2024-09-18T14:17:20.559 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2648213016}) 2024-09-18T14:17:20.559 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.559 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3706774862}) 2024-09-18T14:17:20.559 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=984899792}) 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3706774862}) 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=36578476}) 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3706774862}) 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2504096392}) 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3706774862}) 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2314752132}) 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3706774862}) 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3073702547}) 2024-09-18T14:17:20.560 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3706774862}) 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=254819867}) 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3706774862}) 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.542+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2796180293}) 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2249924202}) 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827623570}) 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2249924202}) 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=145061009}) 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.561 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2249924202}) 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=819762810}) 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2249924202}) 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2966308118}) 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2249924202}) 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=607753567}) 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2249924202}) 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540048941}) 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2249924202}) 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3636790932}) 2024-09-18T14:17:20.562 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2249924202}) 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.544+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4042320622}) 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=622356745}) 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=623037216}) 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=622356745}) 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1716266727}) 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=622356745}) 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3430580892}) 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=622356745}) 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2108575268}) 2024-09-18T14:17:20.563 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=622356745}) 2024-09-18T14:17:20.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3389815024}) 2024-09-18T14:17:20.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=622356745}) 2024-09-18T14:17:20.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3090142863}) 2024-09-18T14:17:20.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=622356745}) 2024-09-18T14:17:20.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2792567677}) 2024-09-18T14:17:20.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=622356745}) 2024-09-18T14:17:20.564 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.545+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2203273435}) 2024-09-18T14:17:20.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=560205643}) 2024-09-18T14:17:20.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2245329377}) 2024-09-18T14:17:20.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=560205643}) 2024-09-18T14:17:20.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3413647102}) 2024-09-18T14:17:20.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=560205643}) 2024-09-18T14:17:20.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.546+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2106797136}) 2024-09-18T14:17:20.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.565 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=560205643}) 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4272138505}) 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=560205643}) 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3352178287}) 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=560205643}) 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3293179288}) 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=560205643}) 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3188494475}) 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=560205643}) 2024-09-18T14:17:20.566 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.547+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1856774795}) 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2368112691}) 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1558839777}) 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2368112691}) 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2541424445}) 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2368112691}) 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1598588314}) 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2368112691}) 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1055861456}) 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.567 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2368112691}) 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3763439641}) 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2368112691}) 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3213671719}) 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2368112691}) 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3512169590}) 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2368112691}) 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.548+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4234630208}) 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1832489103}) 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3517395293}) 2024-09-18T14:17:20.568 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1832489103}) 2024-09-18T14:17:20.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1351671972}) 2024-09-18T14:17:20.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1832489103}) 2024-09-18T14:17:20.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1393601469}) 2024-09-18T14:17:20.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1832489103}) 2024-09-18T14:17:20.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=322051449}) 2024-09-18T14:17:20.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1832489103}) 2024-09-18T14:17:20.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4048181526}) 2024-09-18T14:17:20.569 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1832489103}) 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3467155139}) 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1832489103}) 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3444258518}) 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1832489103}) 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.549+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=144861860}) 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1335804718}) 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3475834613}) 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.570 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1335804718}) 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2171780101}) 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1335804718}) 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2029207434}) 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1335804718}) 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=833628964}) 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1335804718}) 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1158967828}) 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1335804718}) 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2702655543}) 2024-09-18T14:17:20.571 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1335804718}) 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1567821197}) 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1335804718}) 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.550+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=367441244}) 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=81395109}) 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1915814570}) 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=81395109}) 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=751599103}) 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=81395109}) 2024-09-18T14:17:20.572 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2069205830}) 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=81395109}) 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2128729806}) 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=81395109}) 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1944374191}) 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=81395109}) 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4146538248}) 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=81395109}) 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3758644376}) 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.573 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=81395109}) 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.551+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=405589033}) 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1733413818}) 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3495691553}) 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1733413818}) 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=694916752}) 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1733413818}) 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=622049820}) 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1733413818}) 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2046441239}) 2024-09-18T14:17:20.574 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1733413818}) 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3453479649}) 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1733413818}) 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3021969978}) 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1733413818}) 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2604889634}) 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.552+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1733413818}) 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.553+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=155585221}) 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.575 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=322996995}) 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=877864913}) 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=322996995}) 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3588508446}) 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=322996995}) 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3693307522}) 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=322996995}) 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1463637099}) 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=322996995}) 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1727467544}) 2024-09-18T14:17:20.576 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=322996995}) 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2618707780}) 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=322996995}) 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3585799218}) 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=322996995}) 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.554+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2592060126}) 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2870918515}) 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2551341516}) 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.577 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2870918515}) 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1431483174}) 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2870918515}) 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3707332083}) 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2870918515}) 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1218059520}) 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2870918515}) 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2605942229}) 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2870918515}) 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3852883681}) 2024-09-18T14:17:20.578 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2870918515}) 2024-09-18T14:17:20.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=355196069}) 2024-09-18T14:17:20.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2870918515}) 2024-09-18T14:17:20.579 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.555+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3465489602}) 2024-09-18T14:17:20.580 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-18T14:17:20.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1860623643}) 2024-09-18T14:17:20.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=13202495}) 2024-09-18T14:17:20.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1860623643}) 2024-09-18T14:17:20.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1263924292}) 2024-09-18T14:17:20.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.715 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1860623643}) 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1874536214}) 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1860623643}) 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3809776485}) 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1860623643}) 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1252168448}) 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1860623643}) 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1309615921}) 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1860623643}) 2024-09-18T14:17:20.716 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2863590997}) 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1860623643}) 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.556+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4068973732}) 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3692207308}) 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1449654038}) 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3692207308}) 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3671493870}) 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3692207308}) 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1780769661}) 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.717 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3692207308}) 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=974700020}) 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3692207308}) 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=303281584}) 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3692207308}) 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=839664470}) 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3692207308}) 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2643677274}) 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3692207308}) 2024-09-18T14:17:20.718 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.557+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4281532836}) 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3192585455}) 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1906836973}) 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3192585455}) 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1892052877}) 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3192585455}) 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2328774689}) 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3192585455}) 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1685620198}) 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.719 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3192585455}) 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=216845082}) 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3192585455}) 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=866253340}) 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3192585455}) 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2224416131}) 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3192585455}) 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.558+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1033682591}) 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3689514658}) 2024-09-18T14:17:20.720 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3962505292}) 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3689514658}) 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4265282275}) 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3689514658}) 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.559+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1328389359}) 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3689514658}) 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3415749903}) 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3689514658}) 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2421490428}) 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.721 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3689514658}) 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1732985305}) 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3689514658}) 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=205209570}) 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3689514658}) 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.560+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2531440513}) 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=492444870}) 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613300296}) 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=492444870}) 2024-09-18T14:17:20.722 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1580626185}) 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=492444870}) 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1798327823}) 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=492444870}) 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3872706655}) 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=492444870}) 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=797581641}) 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=492444870}) 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4122378244}) 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.723 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=492444870}) 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2526553863}) 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=492444870}) 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.561+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3117765348}) 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=284979859}) 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2893495692}) 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=284979859}) 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2865296098}) 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=284979859}) 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1290110613}) 2024-09-18T14:17:20.724 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=284979859}) 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3075820152}) 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=284979859}) 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4110757079}) 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=284979859}) 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2862294908}) 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=284979859}) 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3208212803}) 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.725 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=284979859}) 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.562+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2454929170}) 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2154125163}) 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1428554117}) 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2154125163}) 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1249038472}) 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2154125163}) 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1985338972}) 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2154125163}) 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1603939873}) 2024-09-18T14:17:20.726 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2154125163}) 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4078694402}) 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2154125163}) 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=946056313}) 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2154125163}) 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=176537975}) 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2154125163}) 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.563+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=736661512}) 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2494706336}) 2024-09-18T14:17:20.727 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3581408063}) 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2494706336}) 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2879407149}) 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2494706336}) 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1672681502}) 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2494706336}) 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4013969664}) 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2494706336}) 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=957263391}) 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.728 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2494706336}) 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=104083827}) 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2494706336}) 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3142561076}) 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2494706336}) 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.564+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2781359455}) 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=618958411}) 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2588823629}) 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=618958411}) 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1697174382}) 2024-09-18T14:17:20.729 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=618958411}) 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=359054719}) 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=618958411}) 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=562680098}) 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=618958411}) 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1136972713}) 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=618958411}) 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=693161798}) 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.730 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=618958411}) 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4272417868}) 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=618958411}) 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.565+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=93485765}) 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=702057846}) 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3970216663}) 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=702057846}) 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1531503654}) 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=702057846}) 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1255259552}) 2024-09-18T14:17:20.731 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=702057846}) 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=134350870}) 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=702057846}) 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=338219137}) 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=702057846}) 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=373802}) 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=702057846}) 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1265525994}) 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=702057846}) 2024-09-18T14:17:20.732 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.566+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2240977578}) 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220222419}) 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1905468377}) 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220222419}) 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3275323319}) 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.567+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220222419}) 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=502439433}) 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220222419}) 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2280587694}) 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.733 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220222419}) 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=588973225}) 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220222419}) 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=140502794}) 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220222419}) 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1512313759}) 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220222419}) 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.568+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=350508642}) 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2743166691}) 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1790785347}) 2024-09-18T14:17:20.734 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2743166691}) 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2535376339}) 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2743166691}) 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2918185241}) 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2743166691}) 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2358584294}) 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2743166691}) 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=418813314}) 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.735 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2743166691}) 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1204157816}) 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2743166691}) 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4274839991}) 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2743166691}) 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.569+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3214201422}) 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4018832388}) 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1214133757}) 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4018832388}) 2024-09-18T14:17:20.736 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3897307624}) 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4018832388}) 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3499178563}) 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4018832388}) 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=597778219}) 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4018832388}) 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2945624764}) 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4018832388}) 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2822373503}) 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.737 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4018832388}) 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2242302341}) 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.570+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4018832388}) 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1351826685}) 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2507015790}) 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4179723680}) 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.571+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2507015790}) 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2028670801}) 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2507015790}) 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2765760267}) 2024-09-18T14:17:20.738 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2507015790}) 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2338075180}) 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2507015790}) 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2430094587}) 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2507015790}) 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3772198389}) 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2507015790}) 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1727510703}) 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2507015790}) 2024-09-18T14:17:20.739 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2905071075}) 2024-09-18T14:17:20.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=202053185}) 2024-09-18T14:17:20.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=890543256}) 2024-09-18T14:17:20.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.572+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=202053185}) 2024-09-18T14:17:20.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=930689410}) 2024-09-18T14:17:20.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=202053185}) 2024-09-18T14:17:20.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2967403584}) 2024-09-18T14:17:20.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=202053185}) 2024-09-18T14:17:20.740 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1400443308}) 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=202053185}) 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=870746434}) 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=202053185}) 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675958647}) 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=202053185}) 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=913300131}) 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=202053185}) 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.573+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=960334499}) 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.741 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2082239078}) 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2973506100}) 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2082239078}) 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3706224665}) 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2082239078}) 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1963439841}) 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2082239078}) 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3354609308}) 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2082239078}) 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1032422197}) 2024-09-18T14:17:20.742 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2082239078}) 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=665058571}) 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2082239078}) 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2336603098}) 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2082239078}) 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.574+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1502455641}) 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543653037}) 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1615843237}) 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543653037}) 2024-09-18T14:17:20.743 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4018619656}) 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543653037}) 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=990152020}) 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543653037}) 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2688700239}) 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543653037}) 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1118816500}) 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543653037}) 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4182713504}) 2024-09-18T14:17:20.744 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543653037}) 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3280961882}) 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543653037}) 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.575+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1144418093}) 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3361076090}) 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2456158087}) 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3361076090}) 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=506494169}) 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.745 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3361076090}) 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=103487586}) 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3361076090}) 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1682800496}) 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3361076090}) 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2628202038}) 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3361076090}) 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=980775037}) 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3361076090}) 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3235554181}) 2024-09-18T14:17:20.746 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3361076090}) 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.576+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2484759397}) 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.577+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.577+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197866386}) 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.577+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2786379248}) 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.577+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.577+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197866386}) 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3334998483}) 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197866386}) 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=89795877}) 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197866386}) 2024-09-18T14:17:20.747 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1095364780}) 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197866386}) 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1432270584}) 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197866386}) 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=215247777}) 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197866386}) 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2058669294}) 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197866386}) 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.578+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2332575427}) 2024-09-18T14:17:20.748 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2661393664}) 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=310974739}) 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2661393664}) 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1098481099}) 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2661393664}) 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=899774454}) 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2661393664}) 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=582643943}) 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.749 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2661393664}) 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=287964036}) 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2661393664}) 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2881740858}) 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2661393664}) 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2303333442}) 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2661393664}) 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.579+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3729413473}) 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440280766}) 2024-09-18T14:17:20.750 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4174373831}) 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440280766}) 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2455103959}) 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440280766}) 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2240439460}) 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440280766}) 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=105748096}) 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440280766}) 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3866832546}) 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.751 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440280766}) 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4005950623}) 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440280766}) 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=30878238}) 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440280766}) 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.580+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1899386419}) 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235316810}) 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265950027}) 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235316810}) 2024-09-18T14:17:20.752 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4288665351}) 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235316810}) 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=507944861}) 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235316810}) 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4173591820}) 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235316810}) 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2014107203}) 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235316810}) 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1351826103}) 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.753 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235316810}) 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1909772302}) 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=235316810}) 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.581+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=256574070}) 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3242055484}) 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1694091296}) 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3242055484}) 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3706421068}) 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3242055484}) 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1619683294}) 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.754 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3242055484}) 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=279920105}) 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3242055484}) 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1161366145}) 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3242055484}) 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1068765223}) 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3242055484}) 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1452194359}) 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3242055484}) 2024-09-18T14:17:20.755 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.582+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3897999218}) 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3318349206}) 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4275286690}) 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3318349206}) 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1312934317}) 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3318349206}) 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2101266035}) 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3318349206}) 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1264641355}) 2024-09-18T14:17:20.756 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3318349206}) 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3111232222}) 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3318349206}) 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1461554912}) 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3318349206}) 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1980758404}) 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3318349206}) 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.583+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=524701229}) 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=501978525}) 2024-09-18T14:17:20.757 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=658457433}) 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=501978525}) 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1848222091}) 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=501978525}) 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2235173262}) 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=501978525}) 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3976164472}) 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=501978525}) 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=212142489}) 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.758 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=501978525}) 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=568472809}) 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=501978525}) 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3758416702}) 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=501978525}) 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.585+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3197140133}) 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140486561}) 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3078057126}) 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140486561}) 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3951827603}) 2024-09-18T14:17:20.759 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140486561}) 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2847828556}) 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140486561}) 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1139248759}) 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140486561}) 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2642133991}) 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140486561}) 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3098660799}) 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.760 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140486561}) 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2923609296}) 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140486561}) 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.586+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3702536665}) 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1192632785}) 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3212462995}) 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1192632785}) 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=968479879}) 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1192632785}) 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1413708199}) 2024-09-18T14:17:20.761 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1192632785}) 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1692147427}) 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1192632785}) 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1216758922}) 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1192632785}) 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=871624697}) 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1192632785}) 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3363790189}) 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1192632785}) 2024-09-18T14:17:20.762 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.587+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=327258922}) 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877939202}) 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2045824256}) 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877939202}) 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2393780284}) 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877939202}) 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2741859456}) 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877939202}) 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2285903642}) 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.763 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877939202}) 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2966260538}) 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877939202}) 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=300940108}) 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877939202}) 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3230740711}) 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877939202}) 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.588+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3226558202}) 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3295062336}) 2024-09-18T14:17:20.764 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3027403449}) 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3295062336}) 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2684079252}) 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3295062336}) 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1880931129}) 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3295062336}) 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3690958077}) 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3295062336}) 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2662979938}) 2024-09-18T14:17:20.765 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3295062336}) 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1724424553}) 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3295062336}) 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3960488168}) 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3295062336}) 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.589+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3887807605}) 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1361765868}) 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2580628510}) 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1361765868}) 2024-09-18T14:17:20.766 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4026953162}) 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1361765868}) 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2569745935}) 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1361765868}) 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1484312329}) 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1361765868}) 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2272071338}) 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1361765868}) 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2780365386}) 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.767 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1361765868}) 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=123244255}) 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1361765868}) 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.590+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3776487525}) 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1336180256}) 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3576295245}) 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1336180256}) 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2770921653}) 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1336180256}) 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2255976844}) 2024-09-18T14:17:20.768 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1336180256}) 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=11841687}) 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1336180256}) 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4171262865}) 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1336180256}) 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3322186093}) 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1336180256}) 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3633173797}) 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.769 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1336180256}) 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.591+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2736203416}) 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=346268626}) 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3188380742}) 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=346268626}) 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4034535304}) 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=346268626}) 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3458632827}) 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.592+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=346268626}) 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3297349637}) 2024-09-18T14:17:20.770 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=346268626}) 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3132866150}) 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=346268626}) 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2478580887}) 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=346268626}) 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2749948623}) 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=346268626}) 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.593+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1978205036}) 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716243440}) 2024-09-18T14:17:20.771 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1089574332}) 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716243440}) 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4199640141}) 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716243440}) 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4158587430}) 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716243440}) 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1067666404}) 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716243440}) 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2866723937}) 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.772 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716243440}) 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2100359761}) 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716243440}) 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3285909592}) 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3716243440}) 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.594+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3123120783}) 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=557745453}) 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1797137041}) 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=557745453}) 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1265086724}) 2024-09-18T14:17:20.773 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=557745453}) 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=495139222}) 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=557745453}) 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2493220005}) 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=557745453}) 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3184579827}) 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=557745453}) 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1658218911}) 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=557745453}) 2024-09-18T14:17:20.774 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2821444481}) 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=557745453}) 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.595+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3951213692}) 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3629967006}) 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2441532414}) 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3629967006}) 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=964923583}) 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3629967006}) 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3380561392}) 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.775 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3629967006}) 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=399390597}) 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3629967006}) 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1949705388}) 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3629967006}) 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3660528183}) 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3629967006}) 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2446210043}) 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3629967006}) 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.596+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=271002743}) 2024-09-18T14:17:20.776 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2615545472}) 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4106919599}) 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2615545472}) 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1458491964}) 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2615545472}) 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=158591248}) 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2615545472}) 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4138883271}) 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2615545472}) 2024-09-18T14:17:20.777 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=96601408}) 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2615545472}) 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1663553608}) 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2615545472}) 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2285359317}) 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2615545472}) 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.597+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2325271421}) 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2628774260}) 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=852214665}) 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.778 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2628774260}) 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1856226693}) 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2628774260}) 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4071955556}) 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2628774260}) 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=305209957}) 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2628774260}) 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=351281916}) 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2628774260}) 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1885540598}) 2024-09-18T14:17:20.779 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2628774260}) 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1671593744}) 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2628774260}) 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.598+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1154865132}) 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=705449186}) 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1120718095}) 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=705449186}) 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2646178958}) 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.780 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=705449186}) 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1754604441}) 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=705449186}) 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3583031815}) 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=705449186}) 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3661177603}) 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.599+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=705449186}) 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3538017180}) 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=705449186}) 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176857054}) 2024-09-18T14:17:20.781 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=705449186}) 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1891456180}) 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.600+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=647620506}) 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2522718119}) 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=647620506}) 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3626344754}) 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=647620506}) 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1925822473}) 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=647620506}) 2024-09-18T14:17:20.782 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=609408888}) 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=647620506}) 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=736888264}) 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=647620506}) 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2198667193}) 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=647620506}) 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1947092673}) 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=647620506}) 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.601+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1888765554}) 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1454814887}) 2024-09-18T14:17:20.783 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1944732793}) 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1454814887}) 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3363549995}) 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1454814887}) 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=930757053}) 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1454814887}) 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3204501713}) 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1454814887}) 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1972064507}) 2024-09-18T14:17:20.784 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1454814887}) 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3076635179}) 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1454814887}) 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=104957193}) 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1454814887}) 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.602+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3223642665}) 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280886788}) 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827183576}) 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280886788}) 2024-09-18T14:17:20.785 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1904529359}) 2024-09-18T14:17:20.786 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.786 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280886788}) 2024-09-18T14:17:20.786 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1641448312}) 2024-09-18T14:17:20.786 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.786 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280886788}) 2024-09-18T14:17:20.786 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2096711391}) 2024-09-18T14:17:20.786 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.786 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280886788}) 2024-09-18T14:17:20.786 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1511289059}) 2024-09-18T14:17:20.786 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280886788}) 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=645379602}) 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280886788}) 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4209529875}) 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280886788}) 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.603+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3702381441}) 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3944186627}) 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3529920764}) 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.787 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3944186627}) 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1838797387}) 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3944186627}) 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2981204695}) 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3944186627}) 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2392717794}) 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3944186627}) 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=492976815}) 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3944186627}) 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2744259764}) 2024-09-18T14:17:20.788 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3944186627}) 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2899457790}) 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3944186627}) 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.604+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3826188275}) 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=254405598}) 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3288265127}) 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=254405598}) 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2282720732}) 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.789 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=254405598}) 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2674577290}) 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=254405598}) 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3614162236}) 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=254405598}) 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4077000316}) 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=254405598}) 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2659178399}) 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=254405598}) 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1195321013}) 2024-09-18T14:17:20.790 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=254405598}) 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.605+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3242646954}) 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=922962564}) 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3600488208}) 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=922962564}) 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=426000759}) 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=922962564}) 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=960796248}) 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=922962564}) 2024-09-18T14:17:20.791 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3346239512}) 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=922962564}) 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=968884414}) 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=922962564}) 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2242363909}) 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=922962564}) 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=454064066}) 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=922962564}) 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.606+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4131224498}) 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.792 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100039786}) 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3771644101}) 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100039786}) 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3177934209}) 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100039786}) 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3171866298}) 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100039786}) 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2152933379}) 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100039786}) 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=455858421}) 2024-09-18T14:17:20.793 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100039786}) 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=128358143}) 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.607+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100039786}) 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4145898236}) 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100039786}) 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.608+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=907460379}) 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500613143}) 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2735871861}) 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.794 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500613143}) 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2106106347}) 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500613143}) 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=365411840}) 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500613143}) 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3414309110}) 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500613143}) 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2551643359}) 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500613143}) 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=934826739}) 2024-09-18T14:17:20.795 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500613143}) 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3358429274}) 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500613143}) 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.609+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3453104971}) 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1901086644}) 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2037660768}) 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1901086644}) 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=24332365}) 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1901086644}) 2024-09-18T14:17:20.796 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=428932851}) 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1901086644}) 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3465772061}) 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1901086644}) 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4047203446}) 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1901086644}) 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2484039816}) 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1901086644}) 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2124360166}) 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.610+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.797 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.611+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1901086644}) 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.611+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=748992483}) 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.611+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3850664408}) 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=691109219}) 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3850664408}) 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2281232920}) 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3850664408}) 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1098859153}) 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3850664408}) 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=106845349}) 2024-09-18T14:17:20.798 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3850664408}) 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1518427143}) 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3850664408}) 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1744176505}) 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3850664408}) 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3002272253}) 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3850664408}) 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.612+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1961443918}) 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2256907420}) 2024-09-18T14:17:20.799 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162790887}) 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2256907420}) 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2565470877}) 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2256907420}) 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4952029}) 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2256907420}) 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=233773052}) 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2256907420}) 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3159388330}) 2024-09-18T14:17:20.800 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2256907420}) 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2176529217}) 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2256907420}) 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1351638968}) 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2256907420}) 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.613+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=502998570}) 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4120565183}) 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2861343080}) 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4120565183}) 2024-09-18T14:17:20.801 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3632502744}) 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4120565183}) 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2671735694}) 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4120565183}) 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4270486029}) 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4120565183}) 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4094096634}) 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4120565183}) 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1317017684}) 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4120565183}) 2024-09-18T14:17:20.802 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=276242041}) 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.614+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4120565183}) 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.615+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3789030276}) 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.615+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.615+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631523928}) 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.615+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3147991031}) 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.615+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.615+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631523928}) 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.615+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1004045925}) 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.615+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.615+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631523928}) 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.615+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=204696876}) 2024-09-18T14:17:20.803 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631523928}) 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2141710885}) 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631523928}) 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4088807499}) 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631523928}) 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=783273303}) 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631523928}) 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=931146993}) 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.804 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631523928}) 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.616+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1748588889}) 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3693761093}) 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3738659511}) 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3693761093}) 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=670474556}) 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3693761093}) 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4119618964}) 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3693761093}) 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3531823857}) 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.805 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3693761093}) 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1266377232}) 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3693761093}) 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1182659366}) 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3693761093}) 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=440239884}) 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3693761093}) 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.617+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3973405420}) 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3879392513}) 2024-09-18T14:17:20.806 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=380931228}) 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3879392513}) 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3232656080}) 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3879392513}) 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1445461795}) 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3879392513}) 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=626875509}) 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3879392513}) 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2195000060}) 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3879392513}) 2024-09-18T14:17:20.807 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3094157946}) 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3879392513}) 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2972906240}) 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3879392513}) 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.618+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1603268893}) 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2157015623}) 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=750983653}) 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2157015623}) 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3814054264}) 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.808 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2157015623}) 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1946269553}) 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2157015623}) 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1702469048}) 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2157015623}) 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1713456550}) 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2157015623}) 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=326229387}) 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2157015623}) 2024-09-18T14:17:20.809 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=313600851}) 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2157015623}) 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.620+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1766807758}) 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1735612017}) 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3858809146}) 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1735612017}) 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=143316433}) 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1735612017}) 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3826483056}) 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1735612017}) 2024-09-18T14:17:20.810 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3171564479}) 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1735612017}) 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=76451011}) 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1735612017}) 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2623141028}) 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1735612017}) 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1343527599}) 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1735612017}) 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.621+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2808074746}) 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.811 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753229542}) 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4255961408}) 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753229542}) 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1813720636}) 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753229542}) 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3211870385}) 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753229542}) 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1024245754}) 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753229542}) 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3087420980}) 2024-09-18T14:17:20.812 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753229542}) 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=192978565}) 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753229542}) 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4216596245}) 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753229542}) 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.622+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2387929646}) 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3889615078}) 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1405356017}) 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3889615078}) 2024-09-18T14:17:20.813 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2549144611}) 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3889615078}) 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2825550554}) 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3889615078}) 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2964618859}) 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3889615078}) 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2540577328}) 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3889615078}) 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=917317655}) 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.814 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3889615078}) 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3386165607}) 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.623+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3889615078}) 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.624+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1303001475}) 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2798688851}) 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2216830831}) 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2798688851}) 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2847910108}) 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2798688851}) 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4022503631}) 2024-09-18T14:17:20.815 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2798688851}) 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1896456210}) 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2798688851}) 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3062198560}) 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2798688851}) 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2481881611}) 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2798688851}) 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1237624661}) 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.816 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2798688851}) 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.625+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4146789012}) 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1383692924}) 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3410120224}) 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1383692924}) 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4155130841}) 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1383692924}) 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3744459701}) 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1383692924}) 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2355899007}) 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.817 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1383692924}) 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2735854061}) 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1383692924}) 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2714527693}) 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.626+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1383692924}) 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.627+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1121598297}) 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.627+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.627+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1383692924}) 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.627+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3520580589}) 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.627+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3598380064}) 2024-09-18T14:17:20.818 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1155573664}) 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3598380064}) 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=112653427}) 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3598380064}) 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1722535516}) 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3598380064}) 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3068392326}) 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3598380064}) 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2116797519}) 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.819 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3598380064}) 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2923077882}) 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3598380064}) 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1954976630}) 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3598380064}) 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.628+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3385221479}) 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4202118808}) 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2614286659}) 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4202118808}) 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1067630264}) 2024-09-18T14:17:20.820 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4202118808}) 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2885010397}) 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4202118808}) 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1039128929}) 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4202118808}) 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1320013420}) 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4202118808}) 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2642712655}) 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4202118808}) 2024-09-18T14:17:20.821 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2588416956}) 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4202118808}) 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.629+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4021231739}) 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.630+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.630+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4176373092}) 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.630+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1978296328}) 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.630+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.630+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4176373092}) 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.630+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=633499321}) 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.630+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.630+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4176373092}) 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.630+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1475035528}) 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.630+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.822 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4176373092}) 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3958695349}) 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4176373092}) 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1731281209}) 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4176373092}) 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1156970543}) 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4176373092}) 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=332574579}) 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4176373092}) 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.631+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2183668065}) 2024-09-18T14:17:20.823 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=660737196}) 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1344960839}) 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=660737196}) 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3255304229}) 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=660737196}) 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3284978799}) 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=660737196}) 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2401806921}) 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.824 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=660737196}) 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1596954226}) 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=660737196}) 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3784189542}) 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=660737196}) 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=72941152}) 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=660737196}) 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.632+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1007778479}) 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399663351}) 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4261505342}) 2024-09-18T14:17:20.825 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399663351}) 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=472590068}) 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399663351}) 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1369566126}) 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399663351}) 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3481849312}) 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399663351}) 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=926454348}) 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399663351}) 2024-09-18T14:17:20.826 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=510400758}) 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399663351}) 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=49932132}) 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399663351}) 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.633+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1996218161}) 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996059232}) 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1224603913}) 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996059232}) 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1677632249}) 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.827 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996059232}) 2024-09-18T14:17:20.828 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3977031959}) 2024-09-18T14:17:20.828 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.828 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996059232}) 2024-09-18T14:17:20.828 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2267492963}) 2024-09-18T14:17:20.828 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.828 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996059232}) 2024-09-18T14:17:20.828 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2431295880}) 2024-09-18T14:17:20.828 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.828 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996059232}) 2024-09-18T14:17:20.828 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2225855621}) 2024-09-18T14:17:20.828 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.828 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996059232}) 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2442593389}) 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996059232}) 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.635+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1133331337}) 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3040558289}) 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3764729967}) 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3040558289}) 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=497061038}) 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3040558289}) 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2104082482}) 2024-09-18T14:17:20.829 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3040558289}) 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2437614530}) 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3040558289}) 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=237299978}) 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3040558289}) 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3714510908}) 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3040558289}) 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=655637957}) 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3040558289}) 2024-09-18T14:17:20.830 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.636+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4267151087}) 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.637+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.637+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=799569037}) 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.637+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=661427054}) 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.637+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.637+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=799569037}) 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.637+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=518949403}) 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.637+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.637+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=799569037}) 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2155949757}) 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=799569037}) 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=973363450}) 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.831 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=799569037}) 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1791796180}) 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=799569037}) 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=578833829}) 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=799569037}) 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=122353240}) 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=799569037}) 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.638+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2645691965}) 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3599091777}) 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2901682863}) 2024-09-18T14:17:20.832 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3599091777}) 2024-09-18T14:17:20.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3506472795}) 2024-09-18T14:17:20.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3599091777}) 2024-09-18T14:17:20.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3653258317}) 2024-09-18T14:17:20.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3599091777}) 2024-09-18T14:17:20.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=568923426}) 2024-09-18T14:17:20.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3599091777}) 2024-09-18T14:17:20.833 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1964609680}) 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3599091777}) 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3937657441}) 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3599091777}) 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2785671599}) 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3599091777}) 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.639+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3982999702}) 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4089034886}) 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=190270269}) 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.834 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4089034886}) 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3993845541}) 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4089034886}) 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2475258358}) 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4089034886}) 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2986732253}) 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4089034886}) 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=297979614}) 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4089034886}) 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2240916339}) 2024-09-18T14:17:20.835 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4089034886}) 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=831049844}) 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.640+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.641+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4089034886}) 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.641+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=649481104}) 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.641+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1678719640}) 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1372955133}) 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1678719640}) 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2311224933}) 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1678719640}) 2024-09-18T14:17:20.836 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3068520054}) 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1678719640}) 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1738430743}) 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1678719640}) 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2733563824}) 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1678719640}) 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1911981718}) 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1678719640}) 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=878115445}) 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.837 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1678719640}) 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.642+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3821676113}) 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1844499710}) 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3181432684}) 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1844499710}) 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=203921376}) 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1844499710}) 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2644548536}) 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1844499710}) 2024-09-18T14:17:20.838 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1349302860}) 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1844499710}) 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=840021354}) 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1844499710}) 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=71687999}) 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1844499710}) 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3678720334}) 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1844499710}) 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.643+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=153982059}) 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.839 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4224273023}) 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2620254589}) 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4224273023}) 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4212875192}) 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4224273023}) 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1194837902}) 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4224273023}) 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3799735724}) 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4224273023}) 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3954939726}) 2024-09-18T14:17:20.840 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4224273023}) 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.644+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1397506426}) 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.645+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.645+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4224273023}) 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.645+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3103642989}) 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.645+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.645+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4224273023}) 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.645+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4259871087}) 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.645+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.645+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=596912229}) 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1865266183}) 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=596912229}) 2024-09-18T14:17:20.841 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2240121544}) 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=596912229}) 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1677414730}) 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=596912229}) 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1047578504}) 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=596912229}) 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1161645716}) 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=596912229}) 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3944403719}) 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.842 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=596912229}) 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3010653762}) 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=596912229}) 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.646+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4053875213}) 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3908595521}) 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4196745400}) 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3908595521}) 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3375199826}) 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3908595521}) 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3987817666}) 2024-09-18T14:17:20.843 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3908595521}) 2024-09-18T14:17:20.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=405903007}) 2024-09-18T14:17:20.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3908595521}) 2024-09-18T14:17:20.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2041431173}) 2024-09-18T14:17:20.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3908595521}) 2024-09-18T14:17:20.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=67061584}) 2024-09-18T14:17:20.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3908595521}) 2024-09-18T14:17:20.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2888666458}) 2024-09-18T14:17:20.844 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3908595521}) 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.647+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2333056764}) 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.648+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.648+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=265225333}) 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.648+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2214320136}) 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.648+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.648+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=265225333}) 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.648+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2340086368}) 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.648+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.648+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=265225333}) 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.648+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1954314799}) 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=265225333}) 2024-09-18T14:17:20.845 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1467228057}) 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=265225333}) 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4159380186}) 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=265225333}) 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1134197699}) 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=265225333}) 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4245902919}) 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=265225333}) 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.649+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3877188019}) 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3832237004,mdsmap=1531151339}) 2024-09-18T14:17:20.846 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=96,mdsmap=3,mgr=1} crc {logm=186007201,mdsmap=1531151339,mgr=3533299129}) 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3832237004,mdsmap=1531151339}) 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3895723717,mdsmap=1531151339}) 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3832237004,mdsmap=1531151339}) 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=2947661344,mdsmap=1531151339}) 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3832237004,mdsmap=1531151339}) 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=95,mdsmap=3,mgr=2} crc {logm=178082650,mdsmap=1531151339,mgr=934748636}) 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3832237004,mdsmap=1531151339}) 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=551521707,mdsmap=2826658659}) 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.847 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3832237004,mdsmap=1531151339}) 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=1828877478,mdsmap=3585761405}) 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3832237004,mdsmap=1531151339}) 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=95,mdsmap=3,mgr=2} crc {logm=692008988,mdsmap=1531151339,mgr=934748636}) 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3832237004,mdsmap=1531151339}) 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.650+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=2404063761,mdsmap=3585761405}) 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=67} crc {mgr=2734874852,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=527378448,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=2194805333}) 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgr=8,mgr_metadata=1,mgrstat=7,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=70} crc {mgr=1359932698,mgr_metadata=270970526,mgrstat=1161806717,mon_config_key=370073362,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=3313558131}) 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=67} crc {mgr=2734874852,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=527378448,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=2194805333}) 2024-09-18T14:17:20.848 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=67} crc {mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=143346430,mon_config_key=1810658514,monmap=455718034,osd_metadata=3328057993,osd_pg_creating=2549746244,osd_snap=1287129697}) 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=67} crc {mgr=2734874852,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=527378448,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=2194805333}) 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=4,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=70} crc {mgr=1301466042,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1444552896,mon_config_key=3155120151,monmap=455718034,osd_metadata=3587064157,osd_pg_creating=2549746244,osd_snap=3836528537}) 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=67} crc {mgr=2734874852,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=527378448,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=2194805333}) 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=66} crc {mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=143346430,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=2054189660}) 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=67} crc {mgr=2734874852,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=527378448,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=2194805333}) 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=68} crc {mgr=2394081750,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=143346430,mon_config_key=1810658514,monmap=3140514009,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=3316242329}) 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=67} crc {mgr=2734874852,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=527378448,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=2194805333}) 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=66} crc {mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=143346430,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=1017843658}) 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.849 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=67} crc {mgr=2734874852,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=527378448,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=2194805333}) 2024-09-18T14:17:20.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=68} crc {mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1721381447,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=3281000427}) 2024-09-18T14:17:20.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=67} crc {mgr=2734874852,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=527378448,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=2194805333}) 2024-09-18T14:17:20.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.652+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=66} crc {mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970527,mgrstat=143346430,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=2549746244,osd_snap=3546279069}) 2024-09-18T14:17:20.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=70,osdmap=30} crc {osd_snap=1003622459,osdmap=996970151}) 2024-09-18T14:17:20.850 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=1932630309,osdmap=2662818087}) 2024-09-18T14:17:20.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=70,osdmap=30} crc {osd_snap=1003622459,osdmap=996970151}) 2024-09-18T14:17:20.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=71,osdmap=29} crc {osd_snap=3165301854,osdmap=3685261621}) 2024-09-18T14:17:20.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=70,osdmap=30} crc {osd_snap=1003622459,osdmap=996970151}) 2024-09-18T14:17:20.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=71,osdmap=29} crc {osd_snap=2758023873,osdmap=1428562853}) 2024-09-18T14:17:20.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=70,osdmap=30} crc {osd_snap=1003622459,osdmap=996970151}) 2024-09-18T14:17:20.852 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=69,osdmap=31} crc {osd_snap=4021201248,osdmap=2657489752}) 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=70,osdmap=30} crc {osd_snap=1003622459,osdmap=996970151}) 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=1835930914,osdmap=2508264360}) 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=70,osdmap=30} crc {osd_snap=1003622459,osdmap=996970151}) 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=66,osdmap=34} crc {osd_snap=3736417625,osdmap=836763539}) 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=70,osdmap=30} crc {osd_snap=1003622459,osdmap=996970151}) 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=2554254104,osdmap=538019968}) 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.653+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=70,osdmap=30} crc {osd_snap=1003622459,osdmap=996970151}) 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.654+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=1005005423,osdmap=740400640}) 2024-09-18T14:17:20.853 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.654+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.654+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2097354742}) 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.654+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1360046414}) 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.654+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.654+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2097354742}) 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.654+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3183960879}) 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2097354742}) 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2945974599}) 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2097354742}) 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=193865432}) 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2097354742}) 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2045317553}) 2024-09-18T14:17:20.854 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2097354742}) 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3115675622}) 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2097354742}) 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2788596458}) 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2097354742}) 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.655+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=655280767}) 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3694803299}) 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3066715384}) 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.855 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3694803299}) 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2837698071}) 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3694803299}) 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3263357972}) 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3694803299}) 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3592355480}) 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3694803299}) 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=533562217}) 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3694803299}) 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2879258070}) 2024-09-18T14:17:20.856 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3694803299}) 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2267661954}) 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3694803299}) 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.656+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=4104774106}) 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.657+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.657+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2596402209}) 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.657+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2340080935}) 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.657+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.657+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2596402209}) 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1325608306}) 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2596402209}) 2024-09-18T14:17:20.857 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3812997239}) 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2596402209}) 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4212469324}) 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2596402209}) 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=833440814}) 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2596402209}) 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1562459478}) 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2596402209}) 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=487735348}) 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.858 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2596402209}) 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.658+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=72355061}) 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.659+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.659+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1917888104}) 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.659+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=773131914}) 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.659+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.659+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1917888104}) 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.659+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3000754420}) 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.659+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.659+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1917888104}) 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.659+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2490723995}) 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1917888104}) 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=24432438}) 2024-09-18T14:17:20.859 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1917888104}) 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1125241595}) 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1917888104}) 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3660221248}) 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1917888104}) 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3801165249}) 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1917888104}) 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.660+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1711490816}) 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=45} crc {osdmap=2136784393}) 2024-09-18T14:17:20.860 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=45} crc {osdmap=3623293792}) 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=45} crc {osdmap=2136784393}) 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=46} crc {osdmap=2952288358}) 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=45} crc {osdmap=2136784393}) 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=43} crc {osdmap=578307778}) 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=45} crc {osdmap=2136784393}) 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=42} crc {osdmap=3531328521}) 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=45} crc {osdmap=2136784393}) 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=45} crc {osdmap=1365643299}) 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.861 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=45} crc {osdmap=2136784393}) 2024-09-18T14:17:20.862 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=43} crc {osdmap=1306197476}) 2024-09-18T14:17:20.862 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.862 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=45} crc {osdmap=2136784393}) 2024-09-18T14:17:20.862 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=43} crc {osdmap=3955342827}) 2024-09-18T14:17:20.862 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:17:20.862 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=45} crc {osdmap=2136784393}) 2024-09-18T14:17:20.862 INFO:tasks.ceph.mon.e.smithi045.stderr:2024-09-18T14:17:20.661+0000 7f1e45ff1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=46} crc {osdmap=2665404830}) 2024-09-18T14:17:21.046 INFO:tasks.rados.rados.0.smithi084.stdout:3787: done (1 left) 2024-09-18T14:17:21.047 INFO:tasks.rados.rados.0.smithi084.stdout:3791: done (0 left) 2024-09-18T14:17:21.047 INFO:tasks.rados.rados.0.smithi084.stdout:3792: write oid 6 current snap is 423 2024-09-18T14:17:21.047 INFO:tasks.rados.rados.0.smithi084.stdout:3792: seq_num 1266 ranges {477662=579832,1683380=555768,2966450=311212} 2024-09-18T14:17:21.052 INFO:tasks.rados.rados.0.smithi084.stdout:3792: writing smithi08434663-6 from 477662 to 1057494 tid 1 2024-09-18T14:17:21.059 INFO:tasks.rados.rados.0.smithi084.stdout:3792: writing smithi08434663-6 from 1683380 to 2239148 tid 2 2024-09-18T14:17:21.065 INFO:tasks.rados.rados.0.smithi084.stdout:3792: writing smithi08434663-6 from 2966450 to 3277662 tid 3 2024-09-18T14:17:21.065 INFO:tasks.rados.rados.0.smithi084.stdout:3793: snap_remove snap 413 2024-09-18T14:17:21.581 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:17:21.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:17:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.680+0000 7f0cd9e6f640 10 monclient: get_monmap_and_config 2024-09-18T14:17:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.680+0000 7f0cd9e6f640 10 monclient: build_initial_monmap 2024-09-18T14:17:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.680+0000 7f0cd9e6f640 10 monclient: monmap: 2024-09-18T14:17:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:21.681510+0000 2024-09-18T14:17:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:21.681510+0000 2024-09-18T14:17:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:21.682 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:21.682 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:21.682 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:21.682 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:21.682 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:21.682 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:21.682 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:21.682 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:21.682 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:21.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.681+0000 7f0cd9e6f640 1 Processor -- start 2024-09-18T14:17:21.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.681+0000 7f0cd9e6f640 1 -- start start 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.681+0000 7f0cd9e6f640 10 monclient: init 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.681+0000 7f0cd9e6f640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.681+0000 7f0cd9e6f640 10 monclient: _add_conns ranks=[7,3,2,0,5,4,1,8,6] 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.681+0000 7f0cd9e6f640 10 monclient(hunting): picked mon.noname-i con 0x7f0cd406e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.681+0000 7f0cd9e6f640 10 monclient(hunting): picked mon.noname-g con 0x7f0cd40a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.681+0000 7f0cd9e6f640 10 monclient(hunting): picked mon.noname-b con 0x7f0cd40a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.681+0000 7f0cd9e6f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd405ed70 con 0x7f0cd40a1080 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd9e6f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd4097620 con 0x7f0cd40a0cb0 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd9e6f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd4097340 con 0x7f0cd406e810 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd9e6f640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd9e6f640 10 monclient(hunting): authenticate will time out at 1857.020084s 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd3fff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f0cd40a1080 0x7f0cd413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60922/0 (socket says 172.21.15.45:60922) 2024-09-18T14:17:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd3fff640 1 -- 172.21.15.45:0/3194201009 learned_addr learned my addr 172.21.15.45:0/3194201009 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728758028 0 0) 0x7f0cd405ed70 con 0x7f0cd40a1080 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient(hunting): my global_id is 145586 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cbc0034e0 con 0x7f0cd40a1080 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761786331 0 0) 0x7f0cd4097340 con 0x7f0cd406e810 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient(hunting): my global_id is 214459 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd405ed70 con 0x7f0cd406e810 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140127609 0 0) 0x7f0cd4097620 con 0x7f0cd40a0cb0 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient(hunting): my global_id is 224148 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd4097340 con 0x7f0cd40a0cb0 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2106563006 0 0) 0x7f0cbc0034e0 con 0x7f0cd40a1080 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 >> v1:172.21.15.84:6792/0 conn(0x7f0cd406e810 legacy=0x7f0cd40a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 >> v1:172.21.15.84:6790/0 conn(0x7f0cd40a0cb0 legacy=0x7f0cd4137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient: _finish_hunting 0 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 monclient: found mon.noname-b 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40a1660 con 0x7f0cd40a1080 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient: _finish_auth 0 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ccc001bb0 con 0x7f0cd40a1080 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd9e6f640 5 monclient: authenticate success, global_id 145586 2024-09-18T14:17:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd9e6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient: dump: 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd9e6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0ccc001d90 con 0x7f0cd40a1080 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 1 -- 172.21.15.45:0/3194201009 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ccc004f20 con 0x7f0cd40a1080 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd9e6f640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd9e6f640 4 set_mon_vals no callback set 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.682+0000 7f0cd27fc640 10 monclient: dump: 2024-09-18T14:17:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.683+0000 7f0cd9e6f640 10 monclient: shutdown 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.683+0000 7f0cd9e6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.683+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/3194201009 >> v1:172.21.15.45:6790/0 conn(0x7f0cd40a1080 legacy=0x7f0cd413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.683+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/3194201009 shutdown_connections 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.683+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/3194201009 >> 172.21.15.45:0/3194201009 conn(0x7f0cd40a5bb0 msgr2=0x7f0cd40a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.683+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/3194201009 shutdown_connections 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.683+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/3194201009 wait complete. 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.683+0000 7f0cd9e6f640 10 monclient: build_initial_monmap 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.683+0000 7f0cd9e6f640 10 monclient: monmap: 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 1 Processor -- start 2024-09-18T14:17:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 1 -- start start 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 10 monclient: init 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 10 monclient: _add_conns ranks=[0,3,7,4,6,1,5,8,2] 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 10 monclient(hunting): picked mon.noname-a con 0x7f0cd406e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 10 monclient(hunting): picked mon.noname-g con 0x7f0cd40a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 10 monclient(hunting): picked mon.noname-i con 0x7f0cd40a1080 addr v1:172.21.15.84:6792/0 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd409fa10 con 0x7f0cd406e810 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd40a2270 con 0x7f0cd40a0cb0 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd4120e70 con 0x7f0cd40a1080 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd9e6f640 10 monclient(hunting): authenticate will time out at 1857.022889s 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.684+0000 7f0cd37fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0cd406e810 0x7f0cd4135370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45928/0 (socket says 172.21.15.45:45928) 2024-09-18T14:17:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cd37fe640 1 -- 172.21.15.45:0/2779550932 learned_addr learned my addr 172.21.15.45:0/2779550932 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721582504 0 0) 0x7f0cd409fa10 con 0x7f0cd406e810 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): my global_id is 6444 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cc00034e0 con 0x7f0cd406e810 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266010131 0 0) 0x7f0cd40a2270 con 0x7f0cd40a0cb0 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): my global_id is 224157 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd409fa10 con 0x7f0cd40a0cb0 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010706694 0 0) 0x7f0cd4120e70 con 0x7f0cd40a1080 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): my global_id is 214468 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd40a2270 con 0x7f0cd40a1080 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1875456464 0 0) 0x7f0cc00034e0 con 0x7f0cd406e810 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cd4120e70 con 0x7f0cd406e810 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0cc4002c10 con 0x7f0cd406e810 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): dump: 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1054349293 0 0) 0x7f0cd40a2270 con 0x7f0cd40a1080 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cc00034e0 con 0x7f0cd40a1080 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3119865594 0 0) 0x7f0cd409fa10 con 0x7f0cd40a0cb0 2024-09-18T14:17:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cd40a2270 con 0x7f0cd40a0cb0 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0ccc001b70 con 0x7f0cd40a1080 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): dump: 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0cc8001a60 con 0x7f0cd40a0cb0 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient(hunting): dump: 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1059831068 0 0) 0x7f0cd4120e70 con 0x7f0cd406e810 2024-09-18T14:17:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 >> v1:172.21.15.84:6792/0 conn(0x7f0cd40a1080 legacy=0x7f0cd4149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 >> v1:172.21.15.84:6790/0 conn(0x7f0cd40a0cb0 legacy=0x7f0cd41468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 monclient: found mon.a 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd4122a50 con 0x7f0cd406e810 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient: _finish_auth 0 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cd9e6f640 5 monclient: authenticate success, global_id 6444 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cd9e6f640 10 monclient: _renew_subs 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cd9e6f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/2779550932 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd4122490 con 0x7f0cd406e810 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cd9e6f640 10 monclient: _renew_subs 2024-09-18T14:17:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cd9e6f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/2779550932 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd4121530 con 0x7f0cd406e810 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0cc4001760 con 0x7f0cd406e810 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f0cc4001c90 con 0x7f0cd406e810 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cbb7fe640 10 monclient: dump: 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.685+0000 7f0cd1ffb640 4 set_mon_vals no callback set 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.686+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f0cc4001e70 con 0x7f0cd406e810 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.686+0000 7f0cd9e6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.686+0000 7f0cd9e6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.686+0000 7f0cd9e6f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.686+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/2779550932 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd404c4a0 con 0x7f0cd406e810 2024-09-18T14:17:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.686+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(848..848 src has 637..848) v4 ==== 3927+0+0 (unknown 1863320222 0 0) 0x7f0cc4046c00 con 0x7f0cd406e810 2024-09-18T14:17:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.687+0000 7f0cd2ffd640 10 monclient: get_auth_request con 0x7f0cc003a960 auth_method 0 2024-09-18T14:17:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.691+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f0cc4002120 con 0x7f0cd406e810 2024-09-18T14:17:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.691+0000 7f0cbb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.691+0000 7f0cbb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.919+0000 7f0cd9e6f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:17:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.919+0000 7f0cd9e6f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:17:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.919+0000 7f0cd9e6f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.919+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/2779550932 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0cd404b720 con 0x7f0cd406e810 2024-09-18T14:17:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.920+0000 7f0cbb7fe640 1 -- 172.21.15.45:0/2779550932 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2689 (unknown 254746657 0 100162893) 0x7f0cc4014dd0 con 0x7f0cd406e810 2024-09-18T14:17:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.920+0000 7f0cbb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:17:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.920+0000 7f0cbb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:21.921 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":146,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:17:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.921+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/2779550932 >> v1:172.21.15.84:6812/31478 conn(0x7f0cc003a960 legacy=0x7f0cc003ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.921+0000 7f0cd9e6f640 10 monclient: shutdown 2024-09-18T14:17:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.921+0000 7f0cd9e6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.921+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/2779550932 >> v1:172.21.15.45:6789/0 conn(0x7f0cd406e810 legacy=0x7f0cd4135370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.921+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/2779550932 shutdown_connections 2024-09-18T14:17:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.921+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/2779550932 >> 172.21.15.45:0/2779550932 conn(0x7f0cd40a5bb0 msgr2=0x7f0cd40a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.922+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/2779550932 shutdown_connections 2024-09-18T14:17:21.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:21.922+0000 7f0cd9e6f640 1 -- 172.21.15.45:0/2779550932 wait complete. 2024-09-18T14:17:21.932 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:17:21.932 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-18T14:17:21.932 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:17:21.933 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:17:22.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.031+0000 7f4d84d09640 10 monclient: get_monmap_and_config 2024-09-18T14:17:22.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.031+0000 7f4d84d09640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.031+0000 7f4d84d09640 10 monclient: monmap: 2024-09-18T14:17:22.032 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.032 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.032 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:22.032138+0000 2024-09-18T14:17:22.032 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:22.032138+0000 2024-09-18T14:17:22.032 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.032 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.031+0000 7f4d84d09640 1 Processor -- start 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.031+0000 7f4d84d09640 1 -- start start 2024-09-18T14:17:22.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.031+0000 7f4d84d09640 10 monclient: init 2024-09-18T14:17:22.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d84d09640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d84d09640 10 monclient: _add_conns ranks=[8,5,0,7,4,6,2,3,1] 2024-09-18T14:17:22.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d84d09640 10 monclient(hunting): picked mon.noname-e con 0x7f4d8006e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:22.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d84d09640 10 monclient(hunting): picked mon.noname-h con 0x7f4d800a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d84d09640 10 monclient(hunting): picked mon.noname-a con 0x7f4d800a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d84d09640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8005ed70 con 0x7f4d800a1080 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d84d09640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d80097620 con 0x7f4d800a0cb0 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d84d09640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d80097340 con 0x7f4d8006e810 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d84d09640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d84d09640 10 monclient(hunting): authenticate will time out at 1857.370490s 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7e59c640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f4d8006e810 0x7f4d800a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:33058/0 (socket says 172.21.15.45:33058) 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7e59c640 1 -- 172.21.15.45:0/3026518784 learned_addr learned my addr 172.21.15.45:0/3026518784 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7ed9d640 1 --1- 172.21.15.45:0/3026518784 >> v1:172.21.15.45:6789/0 conn(0x7f4d800a1080 0x7f4d8013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45940/0 (socket says 172.21.15.45:45940) 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686808640 0 0) 0x7f4d80097340 con 0x7f4d8006e810 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7d59a640 10 monclient(hunting): my global_id is 6119 2024-09-18T14:17:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d680034e0 con 0x7f4d8006e810 2024-09-18T14:17:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777556025 0 0) 0x7f4d8005ed70 con 0x7f4d800a1080 2024-09-18T14:17:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7d59a640 10 monclient(hunting): my global_id is 6453 2024-09-18T14:17:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d80097340 con 0x7f4d800a1080 2024-09-18T14:17:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.032+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624560576 0 0) 0x7f4d80097620 con 0x7f4d800a0cb0 2024-09-18T14:17:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient(hunting): my global_id is 184856 2024-09-18T14:17:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d8005ed70 con 0x7f4d800a0cb0 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1636155917 0 0) 0x7f4d680034e0 con 0x7f4d8006e810 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 >> v1:172.21.15.84:6791/0 conn(0x7f4d800a0cb0 legacy=0x7f4d80137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 >> v1:172.21.15.45:6789/0 conn(0x7f4d800a1080 legacy=0x7f4d8013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 1 monclient: found mon.noname-e 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d800a1660 con 0x7f4d8006e810 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d70001bb0 con 0x7f4d8006e810 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d84d09640 5 monclient: authenticate success, global_id 6119 2024-09-18T14:17:22.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d84d09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:17:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient: dump: 2024-09-18T14:17:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.046 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d84d09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4d70001d90 con 0x7f4d8006e810 2024-09-18T14:17:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d84d09640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 1 -- 172.21.15.45:0/3026518784 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d70004f20 con 0x7f4d8006e810 2024-09-18T14:17:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d84d09640 4 set_mon_vals no callback set 2024-09-18T14:17:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:17:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d7d59a640 10 monclient: dump: 2024-09-18T14:17:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.047 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.048 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.049 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d84d09640 10 monclient: shutdown 2024-09-18T14:17:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d84d09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d84d09640 1 -- 172.21.15.45:0/3026518784 >> v1:172.21.15.45:6793/0 conn(0x7f4d8006e810 legacy=0x7f4d800a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d84d09640 1 -- 172.21.15.45:0/3026518784 shutdown_connections 2024-09-18T14:17:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d84d09640 1 -- 172.21.15.45:0/3026518784 >> 172.21.15.45:0/3026518784 conn(0x7f4d800a5bb0 msgr2=0x7f4d800a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d84d09640 1 -- 172.21.15.45:0/3026518784 shutdown_connections 2024-09-18T14:17:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.033+0000 7f4d84d09640 1 -- 172.21.15.45:0/3026518784 wait complete. 2024-09-18T14:17:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.034+0000 7f4d84d09640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.034+0000 7f4d84d09640 10 monclient: monmap: 2024-09-18T14:17:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.034+0000 7f4d84d09640 1 Processor -- start 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.034+0000 7f4d84d09640 1 -- start start 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.034+0000 7f4d84d09640 10 monclient: init 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d84d09640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d84d09640 10 monclient: _add_conns ranks=[6,4,0,8,5,1,7,3,2] 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d84d09640 10 monclient(hunting): picked mon.noname-d con 0x7f4d8006e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d84d09640 10 monclient(hunting): picked mon.noname-c con 0x7f4d800a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:22.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d84d09640 10 monclient(hunting): picked mon.noname-a con 0x7f4d800a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:22.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d84d09640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8009fa10 con 0x7f4d800a1080 2024-09-18T14:17:22.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d84d09640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8013dce0 con 0x7f4d800a0cb0 2024-09-18T14:17:22.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d84d09640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d800a2270 con 0x7f4d8006e810 2024-09-18T14:17:22.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d84d09640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d84d09640 10 monclient(hunting): authenticate will time out at 1857.373250s 2024-09-18T14:17:22.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d7e59c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f4d8006e810 0x7f4d8013d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43146/0 (socket says 172.21.15.45:43146) 2024-09-18T14:17:22.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d7e59c640 1 -- 172.21.15.45:0/708456844 learned_addr learned my addr 172.21.15.45:0/708456844 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676100944 0 0) 0x7f4d8009fa10 con 0x7f4d800a1080 2024-09-18T14:17:22.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): my global_id is 6462 2024-09-18T14:17:22.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d600034e0 con 0x7f4d800a1080 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708820778 0 0) 0x7f4d8013dce0 con 0x7f4d800a0cb0 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): my global_id is 165181 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d8009fa10 con 0x7f4d800a0cb0 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700448721 0 0) 0x7f4d800a2270 con 0x7f4d8006e810 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): my global_id is 185001 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d8013dce0 con 0x7f4d8006e810 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2988117430 0 0) 0x7f4d600034e0 con 0x7f4d800a1080 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d800a2270 con 0x7f4d800a1080 2024-09-18T14:17:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d78001a00 con 0x7f4d800a1080 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): dump: 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.054 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 361127780 0 0) 0x7f4d8013dce0 con 0x7f4d8006e810 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d600034e0 con 0x7f4d8006e810 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4233653013 0 0) 0x7f4d8009fa10 con 0x7f4d800a0cb0 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d8013dce0 con 0x7f4d800a0cb0 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d70001b70 con 0x7f4d8006e810 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:22.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.035+0000 7f4d6e7fc640 10 monclient(hunting): dump: 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d74001a60 con 0x7f4d800a0cb0 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 10 monclient(hunting): dump: 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2916708266 0 0) 0x7f4d800a2270 con 0x7f4d800a1080 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 >> v1:172.21.15.45:6792/0 conn(0x7f4d8006e810 legacy=0x7f4d8013d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 >> v1:172.21.15.45:6791/0 conn(0x7f4d800a0cb0 legacy=0x7f4d801467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 1 monclient: found mon.a 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d80122670 con 0x7f4d800a1080 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d84d09640 5 monclient: authenticate success, global_id 6462 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d84d09640 10 monclient: _renew_subs 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d84d09640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d84d09640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d80122950 con 0x7f4d800a1080 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d84d09640 10 monclient: _renew_subs 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d84d09640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d84d09640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8013eb60 con 0x7f4d800a1080 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4d78001cd0 con 0x7f4d800a1080 2024-09-18T14:17:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d78004d90 con 0x7f4d800a1080 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d7cd99640 4 set_mon_vals no callback set 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.036+0000 7f4d6e7fc640 10 monclient: dump: 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d84d09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d84d09640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4d64006580 con 0x7f4d64002830 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4d7800fd60 con 0x7f4d800a1080 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(848..848 src has 637..848) v4 ==== 3927+0+0 (unknown 1863320222 0 0) 0x7f4d780469f0 con 0x7f4d800a1080 2024-09-18T14:17:22.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40355246 0 0) 0x7f4d64006580 con 0x7f4d64002830 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 10 monclient(hunting): my global_id is 6471 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d800a2270 con 0x7f4d64002830 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d7dd9b640 10 monclient: get_auth_request con 0x7f4d6003a960 auth_method 0 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 701167128 0 0) 0x7f4d800a2270 con 0x7f4d64002830 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d64006580 con 0x7f4d64002830 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d70004f40 con 0x7f4d64002830 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 10 monclient: dump: 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.062 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.063 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.064 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3816793733 0 0) 0x7f4d64006580 con 0x7f4d64002830 2024-09-18T14:17:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.037+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d6400a0a0 con 0x7f4d64002830 2024-09-18T14:17:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.038+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f4d700072a0 con 0x7f4d64002830 2024-09-18T14:17:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.038+0000 7f4d6e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.038+0000 7f4d6e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.038+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 >> v1:172.21.15.45:6789/0 conn(0x7f4d64002830 legacy=0x7f4d64004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.038+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 >> v1:172.21.15.45:6789/0 conn(0x7f4d64002830 legacy=0x7f4d64004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.065 INFO:tasks.rados.rados.0.smithi084.stdout:3793: done (1 left) 2024-09-18T14:17:22.066 INFO:tasks.rados.rados.0.smithi084.stdout:3794: rollback oid 43 current snap is 423 2024-09-18T14:17:22.066 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 411 2024-09-18T14:17:22.066 INFO:tasks.rados.rados.0.smithi084.stdout:3792: finishing write tid 1 to smithi08434663-6 2024-09-18T14:17:22.066 INFO:tasks.rados.rados.0.smithi084.stdout:3792: finishing write tid 2 to smithi08434663-6 2024-09-18T14:17:22.066 INFO:tasks.rados.rados.0.smithi084.stdout:3792: finishing write tid 3 to smithi08434663-6 2024-09-18T14:17:22.066 INFO:tasks.rados.rados.0.smithi084.stdout:3792: finishing write tid 4 to smithi08434663-6 2024-09-18T14:17:22.066 INFO:tasks.rados.rados.0.smithi084.stdout:3792: finishing write tid 5 to smithi08434663-6 2024-09-18T14:17:22.066 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 1556 (ObjNum 1266 snap 423 seq_num 1266) dirty exists 2024-09-18T14:17:22.066 INFO:tasks.rados.rados.0.smithi084.stdout:3792: left oid 6 (ObjNum 1266 snap 423 seq_num 1266) 2024-09-18T14:17:22.066 INFO:tasks.rados.rados.0.smithi084.stdout:3792: done (1 left) 2024-09-18T14:17:22.066 INFO:tasks.rados.rados.0.smithi084.stdout:3795: snap_create 2024-09-18T14:17:22.066 INFO:tasks.rados.rados.0.smithi084.stdout:3794: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:17:22.067 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 2039 (ObjNum 705 snap 0 seq_num 3269435280) dirty dne 2024-09-18T14:17:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.043+0000 7f4d84d09640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.043+0000 7f4d84d09640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4d64007540 con 0x7f4d640068f0 2024-09-18T14:17:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563045278 0 0) 0x7f4d64007540 con 0x7f4d640068f0 2024-09-18T14:17:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 10 monclient(hunting): my global_id is 6480 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d64006580 con 0x7f4d640068f0 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1354669034 0 0) 0x7f4d64006580 con 0x7f4d640068f0 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d64007540 con 0x7f4d640068f0 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4d70005b40 con 0x7f4d640068f0 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 10 monclient: dump: 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.069 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 642376892 0 0) 0x7f4d64007540 con 0x7f4d640068f0 2024-09-18T14:17:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4d640087a0 con 0x7f4d640068f0 2024-09-18T14:17:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 2705145437) 0x7f4d7000a290 con 0x7f4d640068f0 2024-09-18T14:17:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 >> v1:172.21.15.45:6789/0 conn(0x7f4d640068f0 legacy=0x7f4d64006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.044+0000 7f4d6e7fc640 1 -- 172.21.15.45:0/708456844 >> v1:172.21.15.45:6789/0 conn(0x7f4d640068f0 legacy=0x7f4d64006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.070 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:22.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:22.070 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:17:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:22.092 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.045+0000 7f4d84d09640 1 -- 172.21.15.45:0/708456844 >> v1:172.21.15.84:6812/31478 conn(0x7f4d6003a960 legacy=0x7f4d6003ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.045+0000 7f4d84d09640 10 monclient: shutdown 2024-09-18T14:17:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.045+0000 7f4d84d09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.045+0000 7f4d84d09640 1 -- 172.21.15.45:0/708456844 >> v1:172.21.15.45:6789/0 conn(0x7f4d800a1080 legacy=0x7f4d80149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.045+0000 7f4d7ed9d640 1 -- 172.21.15.45:0/708456844 reap_dead start 2024-09-18T14:17:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.045+0000 7f4d7ed9d640 1 -- 172.21.15.45:0/708456844 reap_dead start 2024-09-18T14:17:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.045+0000 7f4d84d09640 1 -- 172.21.15.45:0/708456844 shutdown_connections 2024-09-18T14:17:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.045+0000 7f4d84d09640 1 -- 172.21.15.45:0/708456844 >> 172.21.15.45:0/708456844 conn(0x7f4d800a5bb0 msgr2=0x7f4d800a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.045+0000 7f4d84d09640 1 -- 172.21.15.45:0/708456844 shutdown_connections 2024-09-18T14:17:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.045+0000 7f4d84d09640 1 -- 172.21.15.45:0/708456844 wait complete. 2024-09-18T14:17:22.095 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:22.095 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:17:22.095 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:17:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.191+0000 7fecdd099640 10 monclient: get_monmap_and_config 2024-09-18T14:17:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.191+0000 7fecdd099640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.191+0000 7fecdd099640 10 monclient: monmap: 2024-09-18T14:17:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:22.192447+0000 2024-09-18T14:17:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:22.192447+0000 2024-09-18T14:17:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.193 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 1 Processor -- start 2024-09-18T14:17:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 1 -- start start 2024-09-18T14:17:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 10 monclient: init 2024-09-18T14:17:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 10 monclient: _add_conns ranks=[5,4,6,2,3,0,7,1,8] 2024-09-18T14:17:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 10 monclient(hunting): picked mon.noname-h con 0x7fecd809f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 10 monclient(hunting): picked mon.noname-c con 0x7fecd809fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 10 monclient(hunting): picked mon.noname-d con 0x7fecd809ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd805ed70 con 0x7fecd809fba0 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd809dfc0 con 0x7fecd809f7d0 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd809dce0 con 0x7fecd809ff70 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecdd099640 10 monclient(hunting): authenticate will time out at 1857.530652s 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecd759e640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fecd809ff70 0x7fecd813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43154/0 (socket says 172.21.15.45:43154) 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecd759e640 1 -- 172.21.15.45:0/2548447356 learned_addr learned my addr 172.21.15.45:0/2548447356 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.192+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753908045 0 0) 0x7fecd809dce0 con 0x7fecd809ff70 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient(hunting): my global_id is 185010 2024-09-18T14:17:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecc00034e0 con 0x7fecd809ff70 2024-09-18T14:17:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035780525 0 0) 0x7fecd805ed70 con 0x7fecd809fba0 2024-09-18T14:17:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient(hunting): my global_id is 165190 2024-09-18T14:17:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd809dce0 con 0x7fecd809fba0 2024-09-18T14:17:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860157870 0 0) 0x7fecd809dfc0 con 0x7fecd809f7d0 2024-09-18T14:17:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient(hunting): my global_id is 184865 2024-09-18T14:17:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd805ed70 con 0x7fecd809f7d0 2024-09-18T14:17:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 487038133 0 0) 0x7fecc00034e0 con 0x7fecd809ff70 2024-09-18T14:17:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 >> v1:172.21.15.84:6791/0 conn(0x7fecd809f7d0 legacy=0x7fecd8132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 >> v1:172.21.15.45:6791/0 conn(0x7fecd809fba0 legacy=0x7fecd8137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 monclient: found mon.noname-d 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd80a1c00 con 0x7fecd809ff70 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecdd099640 5 monclient: authenticate success, global_id 185010 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fecd0001bb0 con 0x7fecd809ff70 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecdd099640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient: dump: 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecdd099640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fecd0001d10 con 0x7fecd809ff70 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 1 -- 172.21.15.45:0/2548447356 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fecd0004f20 con 0x7fecd809ff70 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecdd099640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecdd099640 4 set_mon_vals no callback set 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecd5d9b640 10 monclient: dump: 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecdd099640 10 monclient: shutdown 2024-09-18T14:17:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecdd099640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecdd099640 1 -- 172.21.15.45:0/2548447356 >> v1:172.21.15.45:6792/0 conn(0x7fecd809ff70 legacy=0x7fecd813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecdd099640 1 -- 172.21.15.45:0/2548447356 shutdown_connections 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.193+0000 7fecdd099640 1 -- 172.21.15.45:0/2548447356 >> 172.21.15.45:0/2548447356 conn(0x7fecd80630b0 msgr2=0x7fecd80634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.194+0000 7fecdd099640 1 -- 172.21.15.45:0/2548447356 shutdown_connections 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.194+0000 7fecdd099640 1 -- 172.21.15.45:0/2548447356 wait complete. 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.194+0000 7fecdd099640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.194+0000 7fecdd099640 10 monclient: monmap: 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.194+0000 7fecdd099640 1 Processor -- start 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecdd099640 1 -- start start 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecdd099640 10 monclient: init 2024-09-18T14:17:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecdd099640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecdd099640 10 monclient: _add_conns ranks=[5,3,4,8,1,2,0,7,6] 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecdd099640 10 monclient(hunting): picked mon.noname-h con 0x7fecd809f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecdd099640 10 monclient(hunting): picked mon.noname-g con 0x7fecd809fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecdd099640 10 monclient(hunting): picked mon.noname-c con 0x7fecd809ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecdd099640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd81208b0 con 0x7fecd809fba0 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecdd099640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd8121ee0 con 0x7fecd809ff70 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecdd099640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd809d310 con 0x7fecd809f7d0 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecdd099640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecdd099640 10 monclient(hunting): authenticate will time out at 1857.533508s 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecd759e640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fecd809ff70 0x7fecd8149c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:41050/0 (socket says 172.21.15.45:41050) 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecd759e640 1 -- 172.21.15.45:0/511580619 learned_addr learned my addr 172.21.15.45:0/511580619 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230228243 0 0) 0x7fecd8121ee0 con 0x7fecd809ff70 2024-09-18T14:17:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 10 monclient(hunting): my global_id is 165199 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecc40034e0 con 0x7fecd809ff70 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729580297 0 0) 0x7fecd809d310 con 0x7fecd809f7d0 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 10 monclient(hunting): my global_id is 184874 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd8121ee0 con 0x7fecd809f7d0 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377809656 0 0) 0x7fecd81208b0 con 0x7fecd809fba0 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 10 monclient(hunting): my global_id is 224166 2024-09-18T14:17:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.195+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd809d310 con 0x7fecd809fba0 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 46013191 0 0) 0x7fecc40034e0 con 0x7fecd809ff70 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fecd81208b0 con 0x7fecd809ff70 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fecd0001af0 con 0x7fecd809ff70 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient(hunting): dump: 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1741062061 0 0) 0x7fecd8121ee0 con 0x7fecd809f7d0 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fecc40034e0 con 0x7fecd809f7d0 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fecc8001c30 con 0x7fecd809f7d0 2024-09-18T14:17:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient(hunting): dump: 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 414551313 0 0) 0x7fecd809d310 con 0x7fecd809fba0 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fecd8121ee0 con 0x7fecd809fba0 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feccc001a50 con 0x7fecd809fba0 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient(hunting): dump: 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 412960188 0 0) 0x7fecd81208b0 con 0x7fecd809ff70 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 >> v1:172.21.15.84:6791/0 conn(0x7fecd809f7d0 legacy=0x7fecd81418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 >> v1:172.21.15.84:6790/0 conn(0x7fecd809fba0 legacy=0x7fecd8131ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 monclient: found mon.c 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd8142750 con 0x7fecd809ff70 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecdd099640 5 monclient: authenticate success, global_id 165199 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecdd099640 10 monclient: _renew_subs 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecdd099640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecdd099640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd8121c00 con 0x7fecd809ff70 2024-09-18T14:17:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecdd099640 10 monclient: _renew_subs 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecdd099640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecdd099640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd81200d0 con 0x7fecd809ff70 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fecd0004e40 con 0x7fecd809ff70 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fecd00053b0 con 0x7fecd809ff70 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecd559a640 4 set_mon_vals no callback set 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.196+0000 7fecbeffd640 10 monclient: dump: 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecdd099640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecdd099640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fecb8006580 con 0x7fecb8002830 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fecd0010400 con 0x7fecd809ff70 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(849..849 src has 637..849) v4 ==== 3971+0+0 (unknown 3046409346 0 0) 0x7fecd0046e10 con 0x7fecd809ff70 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3738409183 0 0) 0x7fecb8006580 con 0x7fecb8002830 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecbeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecbeffd640 10 monclient(hunting): my global_id is 145595 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd81208b0 con 0x7fecb8002830 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecd659c640 10 monclient: get_auth_request con 0x7fecc403aa10 auth_method 0 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2245496997 0 0) 0x7fecd81208b0 con 0x7fecb8002830 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fecb8006580 con 0x7fecb8002830 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.197+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fecc8002110 con 0x7fecb8002830 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.198+0000 7fecbeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.198+0000 7fecbeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.198+0000 7fecbeffd640 10 monclient: dump: 2024-09-18T14:17:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.222 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.198+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3489275406 0 0) 0x7fecb8006580 con 0x7fecb8002830 2024-09-18T14:17:22.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.198+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fecb800a0a0 con 0x7fecb8002830 2024-09-18T14:17:22.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.198+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fecc80025f0 con 0x7fecb8002830 2024-09-18T14:17:22.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.198+0000 7fecbeffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.198+0000 7fecbeffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:22.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.198+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 >> v1:172.21.15.45:6790/0 conn(0x7fecb8002830 legacy=0x7fecb8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.198+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 >> v1:172.21.15.45:6790/0 conn(0x7fecb8002830 legacy=0x7fecb8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.204+0000 7fecdd099640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.204+0000 7fecdd099640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fecb8009e70 con 0x7fecb80068f0 2024-09-18T14:17:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.204+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934766220 0 0) 0x7fecb8009e70 con 0x7fecb80068f0 2024-09-18T14:17:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.204+0000 7fecbeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.204+0000 7fecbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.204+0000 7fecbeffd640 10 monclient(hunting): my global_id is 145604 2024-09-18T14:17:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.204+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb8006580 con 0x7fecb80068f0 2024-09-18T14:17:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3612664709 0 0) 0x7fecb8006580 con 0x7fecb80068f0 2024-09-18T14:17:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fecb8009e70 con 0x7fecb80068f0 2024-09-18T14:17:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fecc8003000 con 0x7fecb80068f0 2024-09-18T14:17:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 10 monclient: dump: 2024-09-18T14:17:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1181674371 0 0) 0x7fecb8009e70 con 0x7fecb80068f0 2024-09-18T14:17:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fecb8008750 con 0x7fecb80068f0 2024-09-18T14:17:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 3894467336) 0x7fecc80032a0 con 0x7fecb80068f0 2024-09-18T14:17:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 >> v1:172.21.15.45:6790/0 conn(0x7fecb80068f0 legacy=0x7fecb8006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecbeffd640 1 -- 172.21.15.45:0/511580619 >> v1:172.21.15.45:6790/0 conn(0x7fecb80068f0 legacy=0x7fecb8006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.227 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:22.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:22.227 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:22.227 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:22.227 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:22.228 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:22.228 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:22.228 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:22.228 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:22.228 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:22.228 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:22.228 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:22.228 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:22.228 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:22.228 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:22.228 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.228 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.229 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:22.230 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:22.231 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.232 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.233 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.234 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:22.235 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:22.235 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.235 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.235 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.235 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:22.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:22.235 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.236 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.237 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:22.237 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:22.237 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.237 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.237 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:22.237 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.237 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.238 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.238 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:22.238 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:22.238 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.238 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.238 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.238 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:22.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.246 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.247 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.247 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:17:22.247 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.247 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.247 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:22.247 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:22.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecdd099640 1 -- 172.21.15.45:0/511580619 >> v1:172.21.15.84:6812/31478 conn(0x7fecc403aa10 legacy=0x7fecc403cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecdd099640 10 monclient: shutdown 2024-09-18T14:17:22.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecdd099640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecdd099640 1 -- 172.21.15.45:0/511580619 >> v1:172.21.15.45:6791/0 conn(0x7fecd809ff70 legacy=0x7fecd8149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecd759e640 1 -- 172.21.15.45:0/511580619 reap_dead start 2024-09-18T14:17:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecd759e640 1 -- 172.21.15.45:0/511580619 reap_dead start 2024-09-18T14:17:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecdd099640 1 -- 172.21.15.45:0/511580619 shutdown_connections 2024-09-18T14:17:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecdd099640 1 -- 172.21.15.45:0/511580619 >> 172.21.15.45:0/511580619 conn(0x7fecd80630b0 msgr2=0x7fecd80a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecdd099640 1 -- 172.21.15.45:0/511580619 shutdown_connections 2024-09-18T14:17:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.205+0000 7fecdd099640 1 -- 172.21.15.45:0/511580619 wait complete. 2024-09-18T14:17:22.249 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:22.249 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:17:22.250 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:17:22.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.346+0000 7f8262294640 10 monclient: get_monmap_and_config 2024-09-18T14:17:22.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.346+0000 7f8262294640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.346+0000 7f8262294640 10 monclient: monmap: 2024-09-18T14:17:22.347 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.347 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.347 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:22.347277+0000 2024-09-18T14:17:22.347 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:22.347277+0000 2024-09-18T14:17:22.347 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.347 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.347 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.347 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.347 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.348 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.348 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.348 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.348 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.348 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.348 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.348 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 1 Processor -- start 2024-09-18T14:17:22.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 1 -- start start 2024-09-18T14:17:22.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 10 monclient: init 2024-09-18T14:17:22.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 10 monclient: _add_conns ranks=[3,0,2,8,5,7,4,1,6] 2024-09-18T14:17:22.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 10 monclient(hunting): picked mon.noname-g con 0x7f825c09f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:22.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 10 monclient(hunting): picked mon.noname-a con 0x7f825c09fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:22.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 10 monclient(hunting): picked mon.noname-b con 0x7f825c09ff70 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:22.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c05ed70 con 0x7f825c09fba0 2024-09-18T14:17:22.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c09dfc0 con 0x7f825c09ff70 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c09dce0 con 0x7f825c09f7d0 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8262294640 10 monclient(hunting): authenticate will time out at 1857.685726s 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f825b7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f825c09fba0 0x7f825c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45976/0 (socket says 172.21.15.45:45976) 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f8260831640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f825c09ff70 0x7f825c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60968/0 (socket says 172.21.15.45:60968) 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.347+0000 7f825b7fe640 1 -- 172.21.15.45:0/423011275 learned_addr learned my addr 172.21.15.45:0/423011275 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967174390 0 0) 0x7f825c09dfc0 con 0x7f825c09ff70 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient(hunting): my global_id is 145613 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82440034e0 con 0x7f825c09ff70 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106243639 0 0) 0x7f825c05ed70 con 0x7f825c09fba0 2024-09-18T14:17:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient(hunting): my global_id is 6489 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f825c09dfc0 con 0x7f825c09fba0 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623970145 0 0) 0x7f825c09dce0 con 0x7f825c09f7d0 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient(hunting): my global_id is 224175 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f825c05ed70 con 0x7f825c09f7d0 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4140974677 0 0) 0x7f82440034e0 con 0x7f825c09ff70 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 >> v1:172.21.15.84:6790/0 conn(0x7f825c09f7d0 legacy=0x7f825c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 >> v1:172.21.15.45:6789/0 conn(0x7f825c09fba0 legacy=0x7f825c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 monclient: found mon.noname-b 2024-09-18T14:17:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c0a1c00 con 0x7f825c09ff70 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8254001ae0 con 0x7f825c09ff70 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f8262294640 5 monclient: authenticate success, global_id 145613 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f8262294640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient: dump: 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.361 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f8262294640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8254001cc0 con 0x7f825c09ff70 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 1 -- 172.21.15.45:0/423011275 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8254004b20 con 0x7f825c09ff70 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f825affd640 10 monclient: dump: 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.362 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f8262294640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.348+0000 7f8262294640 4 set_mon_vals no callback set 2024-09-18T14:17:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.349+0000 7f8262294640 10 monclient: shutdown 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.349+0000 7f8262294640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.349+0000 7f8262294640 1 -- 172.21.15.45:0/423011275 >> v1:172.21.15.45:6790/0 conn(0x7f825c09ff70 legacy=0x7f825c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.349+0000 7f8262294640 1 -- 172.21.15.45:0/423011275 shutdown_connections 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.349+0000 7f8262294640 1 -- 172.21.15.45:0/423011275 >> 172.21.15.45:0/423011275 conn(0x7f825c0630b0 msgr2=0x7f825c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.349+0000 7f8262294640 1 -- 172.21.15.45:0/423011275 shutdown_connections 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.349+0000 7f8262294640 1 -- 172.21.15.45:0/423011275 wait complete. 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.349+0000 7f8262294640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.349+0000 7f8262294640 10 monclient: monmap: 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.349+0000 7f8262294640 1 Processor -- start 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f8262294640 1 -- start start 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f8262294640 10 monclient: init 2024-09-18T14:17:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f8262294640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f8262294640 10 monclient: _add_conns ranks=[3,4,5,8,1,0,7,6,2] 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f8262294640 10 monclient(hunting): picked mon.noname-g con 0x7f825c09f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f8262294640 10 monclient(hunting): picked mon.noname-c con 0x7f825c09fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f8262294640 10 monclient(hunting): picked mon.noname-h con 0x7f825c09ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f8262294640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c09d310 con 0x7f825c09f7d0 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f8262294640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c11eaf0 con 0x7f825c09fba0 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f8262294640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c0a2560 con 0x7f825c09ff70 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f8262294640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f8262294640 10 monclient(hunting): authenticate will time out at 1857.688709s 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f825b7fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f825c09fba0 0x7f825c146d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:41058/0 (socket says 172.21.15.45:41058) 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.350+0000 7f825b7fe640 1 -- 172.21.15.45:0/4161949812 learned_addr learned my addr 172.21.15.45:0/4161949812 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383059365 0 0) 0x7f825c11eaf0 con 0x7f825c09fba0 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): my global_id is 165208 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82480034e0 con 0x7f825c09fba0 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522499277 0 0) 0x7f825c0a2560 con 0x7f825c09ff70 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): my global_id is 184883 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f825c11eaf0 con 0x7f825c09ff70 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220509731 0 0) 0x7f825c09d310 con 0x7f825c09f7d0 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): my global_id is 224184 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f825c0a2560 con 0x7f825c09f7d0 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1287724539 0 0) 0x7f82480034e0 con 0x7f825c09fba0 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f825c09d310 con 0x7f825c09fba0 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8250001610 con 0x7f825c09fba0 2024-09-18T14:17:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): dump: 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.368 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.369 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.369 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.369 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.369 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:22.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2152214521 0 0) 0x7f825c11eaf0 con 0x7f825c09ff70 2024-09-18T14:17:22.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82480034e0 con 0x7f825c09ff70 2024-09-18T14:17:22.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8254001a20 con 0x7f825c09ff70 2024-09-18T14:17:22.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): dump: 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.370 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.371 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.371 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.371 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1491292944 0 0) 0x7f825c0a2560 con 0x7f825c09f7d0 2024-09-18T14:17:22.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f825c11eaf0 con 0x7f825c09f7d0 2024-09-18T14:17:22.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f824c001aa0 con 0x7f825c09f7d0 2024-09-18T14:17:22.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:22.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient(hunting): dump: 2024-09-18T14:17:22.371 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.371 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.371 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.373 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 849969954 0 0) 0x7f825c09d310 con 0x7f825c09fba0 2024-09-18T14:17:22.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 >> v1:172.21.15.84:6791/0 conn(0x7f825c09ff70 legacy=0x7f825c14a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 >> v1:172.21.15.84:6790/0 conn(0x7f825c09f7d0 legacy=0x7f825c1316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 monclient: found mon.c 2024-09-18T14:17:22.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:17:22.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c142890 con 0x7f825c09fba0 2024-09-18T14:17:22.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8262294640 5 monclient: authenticate success, global_id 165208 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8262294640 10 monclient: _renew_subs 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8262294640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8262294640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825c11d920 con 0x7f825c09fba0 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8262294640 10 monclient: _renew_subs 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8262294640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8262294640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f825c09b9d0 con 0x7f825c09fba0 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8250004a50 con 0x7f825c09fba0 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8250001850 con 0x7f825c09fba0 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f825a7fc640 4 set_mon_vals no callback set 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.351+0000 7f8243fff640 10 monclient: dump: 2024-09-18T14:17:22.374 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.375 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.375 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.375 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.375 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.375 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.375 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.375 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.375 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.375 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.375 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.375 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.376 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.376 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.376 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.376 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.352+0000 7f8262294640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.352+0000 7f8262294640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f823c006580 con 0x7f823c002830 2024-09-18T14:17:22.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.352+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f825000fd70 con 0x7f825c09fba0 2024-09-18T14:17:22.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.352+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(849..849 src has 637..849) v4 ==== 3971+0+0 (unknown 3046409346 0 0) 0x7f8250046ae0 con 0x7f825c09fba0 2024-09-18T14:17:22.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.352+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975138117 0 0) 0x7f823c006580 con 0x7f823c002830 2024-09-18T14:17:22.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.352+0000 7f8243fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.352+0000 7f8243fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.352+0000 7f8243fff640 10 monclient(hunting): my global_id is 165217 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.352+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f825c09d310 con 0x7f823c002830 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8260831640 10 monclient: get_auth_request con 0x7f824803a9e0 auth_method 0 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2749951352 0 0) 0x7f825c09d310 con 0x7f823c002830 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f823c006580 con 0x7f823c002830 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f824c0022b0 con 0x7f823c002830 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 10 monclient: dump: 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.377 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 187123834 0 0) 0x7f823c006580 con 0x7f823c002830 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f823c00a0a0 con 0x7f823c002830 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f824c0027a0 con 0x7f823c002830 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:22.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 >> v1:172.21.15.45:6791/0 conn(0x7f823c002830 legacy=0x7f823c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.353+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 >> v1:172.21.15.45:6791/0 conn(0x7f823c002830 legacy=0x7f823c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f8262294640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:22.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f8262294640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f823c007510 con 0x7f823c0068c0 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f825bfff640 1 -- 172.21.15.45:0/4161949812 >> v1:172.21.15.45:6791/0 conn(0x7f823c0068c0 legacy=0x7f823c006d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f825bfff640 1 -- 172.21.15.45:0/4161949812 >> v1:172.21.15.45:6791/0 conn(0x7f823c0068c0 legacy=0x7f823c006d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f825bfff640 1 --1- 172.21.15.45:0/4161949812 >> v1:172.21.15.45:6791/0 conn(0x7f823c0068c0 0x7f823c006d40 :-1 s=OPENED pgs=432 cs=1 l=1).handle_message read tag failed 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f825bfff640 1 --1- 172.21.15.45:0/4161949812 >> v1:172.21.15.45:6791/0 conn(0x7f823c0068c0 0x7f823c006d40 :-1 s=OPENED pgs=432 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 >> v1:172.21.15.45:6791/0 conn(0x7f823c0068c0 legacy=0x7f823c006d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 >> v1:172.21.15.45:6791/0 conn(0x7f823c0068c0 legacy=0x7f823c006d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f823c006580 con 0x7f8248042d30 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922932702 0 0) 0x7f823c006580 con 0x7f8248042d30 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f8243fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f8243fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f8243fff640 10 monclient(hunting): my global_id is 165226 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.359+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f825c09d310 con 0x7f8248042d30 2024-09-18T14:17:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1158514731 0 0) 0x7f825c09d310 con 0x7f8248042d30 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f823c006580 con 0x7f8248042d30 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f824c002c60 con 0x7f8248042d30 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 10 monclient: dump: 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 849485189 0 0) 0x7f823c006580 con 0x7f8248042d30 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f824c0027a0 con 0x7f8248042d30 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 1712454150) 0x7f824c007d90 con 0x7f8248042d30 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:22.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 >> v1:172.21.15.45:6791/0 conn(0x7f8248042d30 legacy=0x7f8248041810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8243fff640 1 -- 172.21.15.45:0/4161949812 >> v1:172.21.15.45:6791/0 conn(0x7f8248042d30 legacy=0x7f8248041810 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8260831640 1 -- 172.21.15.45:0/4161949812 reap_dead start 2024-09-18T14:17:22.384 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 5 2024-09-18T14:17:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:22.403 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8262294640 1 -- 172.21.15.45:0/4161949812 >> v1:172.21.15.84:6812/31478 conn(0x7f824803a9e0 legacy=0x7f824803ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8262294640 10 monclient: shutdown 2024-09-18T14:17:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8262294640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8262294640 1 -- 172.21.15.45:0/4161949812 >> v1:172.21.15.45:6791/0 conn(0x7f825c09fba0 legacy=0x7f825c146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.360+0000 7f8262294640 1 -- 172.21.15.45:0/4161949812 shutdown_connections 2024-09-18T14:17:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.361+0000 7f8262294640 1 -- 172.21.15.45:0/4161949812 >> 172.21.15.45:0/4161949812 conn(0x7f825c0630b0 msgr2=0x7f825c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.361+0000 7f8262294640 1 -- 172.21.15.45:0/4161949812 shutdown_connections 2024-09-18T14:17:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.361+0000 7f8262294640 1 -- 172.21.15.45:0/4161949812 wait complete. 2024-09-18T14:17:22.405 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:22.405 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:17:22.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:17:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.502+0000 7f9949300640 10 monclient: get_monmap_and_config 2024-09-18T14:17:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.502+0000 7f9949300640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.502+0000 7f9949300640 10 monclient: monmap: 2024-09-18T14:17:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:22.503609+0000 2024-09-18T14:17:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:22.503609+0000 2024-09-18T14:17:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.504 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.504 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.504 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.504 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.504 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.504 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.504 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.504 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 1 Processor -- start 2024-09-18T14:17:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 1 -- start start 2024-09-18T14:17:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 10 monclient: init 2024-09-18T14:17:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 10 monclient: _add_conns ranks=[5,4,3,1,2,8,6,0,7] 2024-09-18T14:17:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 10 monclient(hunting): picked mon.noname-h con 0x7f994406e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 10 monclient(hunting): picked mon.noname-c con 0x7f99440a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 10 monclient(hunting): picked mon.noname-g con 0x7f99440a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994405ed70 con 0x7f99440a1080 2024-09-18T14:17:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9944097620 con 0x7f99440a0cb0 2024-09-18T14:17:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9944097340 con 0x7f994406e810 2024-09-18T14:17:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f9949300640 10 monclient(hunting): authenticate will time out at 1857.841936s 2024-09-18T14:17:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.503+0000 7f99427fc640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f99440a0cb0 0x7f9944137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:41106/0 (socket says 172.21.15.45:41106) 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f99427fc640 1 -- 172.21.15.45:0/113792479 learned_addr learned my addr 172.21.15.45:0/113792479 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074332290 0 0) 0x7f9944097620 con 0x7f99440a0cb0 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient(hunting): my global_id is 165235 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f992c0034e0 con 0x7f99440a0cb0 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079832370 0 0) 0x7f9944097340 con 0x7f994406e810 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient(hunting): my global_id is 184892 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9944097620 con 0x7f994406e810 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578783277 0 0) 0x7f994405ed70 con 0x7f99440a1080 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient(hunting): my global_id is 224193 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9944097340 con 0x7f99440a1080 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3540898067 0 0) 0x7f992c0034e0 con 0x7f99440a0cb0 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 >> v1:172.21.15.84:6791/0 conn(0x7f994406e810 legacy=0x7f99440a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 >> v1:172.21.15.84:6790/0 conn(0x7f99440a1080 legacy=0x7f994413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 monclient: found mon.noname-c 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99440a1660 con 0x7f99440a0cb0 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9938001830 con 0x7f99440a0cb0 2024-09-18T14:17:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9949300640 5 monclient: authenticate success, global_id 165235 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9949300640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient: dump: 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9949300640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9938001a10 con 0x7f99440a0cb0 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 1 -- 172.21.15.45:0/113792479 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9938004ba0 con 0x7f99440a0cb0 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9949300640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9949300640 4 set_mon_vals no callback set 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.504+0000 7f9941ffb640 10 monclient: dump: 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.505+0000 7f9949300640 10 monclient: shutdown 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.505+0000 7f9949300640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.505+0000 7f9949300640 1 -- 172.21.15.45:0/113792479 >> v1:172.21.15.45:6791/0 conn(0x7f99440a0cb0 legacy=0x7f9944137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.505+0000 7f9949300640 1 -- 172.21.15.45:0/113792479 shutdown_connections 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.505+0000 7f9949300640 1 -- 172.21.15.45:0/113792479 >> 172.21.15.45:0/113792479 conn(0x7f99440a5bb0 msgr2=0x7f99440a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.505+0000 7f9949300640 1 -- 172.21.15.45:0/113792479 shutdown_connections 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.505+0000 7f9949300640 1 -- 172.21.15.45:0/113792479 wait complete. 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.505+0000 7f9949300640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.505+0000 7f9949300640 10 monclient: monmap: 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 1 Processor -- start 2024-09-18T14:17:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 1 -- start start 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 10 monclient: init 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 10 monclient: _add_conns ranks=[4,7,0,6,1,8,5,3,2] 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 10 monclient(hunting): picked mon.noname-c con 0x7f994406e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 10 monclient(hunting): picked mon.noname-i con 0x7f99440a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 10 monclient(hunting): picked mon.noname-a con 0x7f99440a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99441361a0 con 0x7f99440a1080 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99440a2270 con 0x7f994406e810 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994409fa10 con 0x7f99440a0cb0 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9949300640 10 monclient(hunting): authenticate will time out at 1857.844766s 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9942ffd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f994406e810 0x7f9944135400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:41112/0 (socket says 172.21.15.45:41112) 2024-09-18T14:17:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.506+0000 7f9942ffd640 1 -- 172.21.15.45:0/2629668080 learned_addr learned my addr 172.21.15.45:0/2629668080 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540834065 0 0) 0x7f99440a2270 con 0x7f994406e810 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): my global_id is 165244 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99300034e0 con 0x7f994406e810 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994991440 0 0) 0x7f99441361a0 con 0x7f99440a1080 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): my global_id is 6498 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99440a2270 con 0x7f99440a1080 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145325430 0 0) 0x7f994409fa10 con 0x7f99440a0cb0 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): my global_id is 214477 2024-09-18T14:17:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99441361a0 con 0x7f99440a0cb0 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1566063039 0 0) 0x7f99300034e0 con 0x7f994406e810 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f994409fa10 con 0x7f994406e810 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9934001be0 con 0x7f994406e810 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): dump: 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2385015425 0 0) 0x7f99440a2270 con 0x7f99440a1080 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f99300034e0 con 0x7f99440a1080 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f993c001d00 con 0x7f99440a1080 2024-09-18T14:17:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): dump: 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3182082438 0 0) 0x7f99441361a0 con 0x7f99440a0cb0 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f99440a2270 con 0x7f99440a0cb0 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f99380017f0 con 0x7f99440a0cb0 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient(hunting): dump: 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 470166395 0 0) 0x7f994409fa10 con 0x7f994406e810 2024-09-18T14:17:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 >> v1:172.21.15.84:6792/0 conn(0x7f99440a0cb0 legacy=0x7f99441468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 >> v1:172.21.15.45:6789/0 conn(0x7f99440a1080 legacy=0x7f9944149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 monclient: found mon.c 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99441209f0 con 0x7f994406e810 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f9949300640 5 monclient: authenticate success, global_id 165244 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f9949300640 10 monclient: _renew_subs 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f9949300640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f9949300640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9944142400 con 0x7f994406e810 2024-09-18T14:17:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f9949300640 10 monclient: _renew_subs 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f9949300640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f9949300640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9944136a90 con 0x7f994406e810 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f99340020f0 con 0x7f994406e810 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f99340025a0 con 0x7f994406e810 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f99417fa640 4 set_mon_vals no callback set 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.507+0000 7f992affd640 10 monclient: dump: 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.508+0000 7f9949300640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.508+0000 7f9949300640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9924006580 con 0x7f9924002830 2024-09-18T14:17:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.508+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f9934002780 con 0x7f994406e810 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.508+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(849..849 src has 637..849) v4 ==== 3971+0+0 (unknown 3046409346 0 0) 0x7f9934046bf0 con 0x7f994406e810 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.508+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248242175 0 0) 0x7f9924006580 con 0x7f9924002830 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.508+0000 7f992affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.508+0000 7f992affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.508+0000 7f992affd640 10 monclient(hunting): my global_id is 185019 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.508+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f994409fa10 con 0x7f9924002830 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.508+0000 7f99437fe640 10 monclient: get_auth_request con 0x7f993003a9f0 auth_method 0 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1013300111 0 0) 0x7f994409fa10 con 0x7f9924002830 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9924006580 con 0x7f9924002830 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f9938004ae0 con 0x7f9924002830 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 10 monclient: dump: 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.533 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2478269793 0 0) 0x7f9924006580 con 0x7f9924002830 2024-09-18T14:17:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f992400a0a0 con 0x7f9924002830 2024-09-18T14:17:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9938005010 con 0x7f9924002830 2024-09-18T14:17:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 >> v1:172.21.15.45:6792/0 conn(0x7f9924002830 legacy=0x7f9924004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.509+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 >> v1:172.21.15.45:6792/0 conn(0x7f9924002830 legacy=0x7f9924004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f9949300640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f9949300640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9924009660 con 0x7f9924009040 2024-09-18T14:17:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198151722 0 0) 0x7f9924009660 con 0x7f9924009040 2024-09-18T14:17:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f992affd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f992affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f992affd640 10 monclient(hunting): my global_id is 185028 2024-09-18T14:17:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9924006580 con 0x7f9924009040 2024-09-18T14:17:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4048747173 0 0) 0x7f9924006580 con 0x7f9924009040 2024-09-18T14:17:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9924009660 con 0x7f9924009040 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f99380057d0 con 0x7f9924009040 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f992affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f992affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.515+0000 7f992affd640 10 monclient: dump: 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3269599341 0 0) 0x7f9924009660 con 0x7f9924009040 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9924009920 con 0x7f9924009040 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 3337179813) 0x7f9938007990 con 0x7f9924009040 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f992affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f992affd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 >> v1:172.21.15.45:6792/0 conn(0x7f9924009040 legacy=0x7f9924008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f992affd640 1 -- 172.21.15.45:0/2629668080 >> v1:172.21.15.45:6792/0 conn(0x7f9924009040 legacy=0x7f9924008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.538 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:22.557 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:22.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f9949300640 1 -- 172.21.15.45:0/2629668080 >> v1:172.21.15.84:6812/31478 conn(0x7f993003a9f0 legacy=0x7f993003cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f9949300640 10 monclient: shutdown 2024-09-18T14:17:22.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f9949300640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f9949300640 1 -- 172.21.15.45:0/2629668080 >> v1:172.21.15.45:6791/0 conn(0x7f994406e810 legacy=0x7f9944135400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f99437fe640 1 -- 172.21.15.45:0/2629668080 reap_dead start 2024-09-18T14:17:22.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f99437fe640 1 -- 172.21.15.45:0/2629668080 reap_dead start 2024-09-18T14:17:22.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f9949300640 1 -- 172.21.15.45:0/2629668080 shutdown_connections 2024-09-18T14:17:22.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f9949300640 1 -- 172.21.15.45:0/2629668080 >> 172.21.15.45:0/2629668080 conn(0x7f99440a5bb0 msgr2=0x7f99440a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f9949300640 1 -- 172.21.15.45:0/2629668080 shutdown_connections 2024-09-18T14:17:22.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.516+0000 7f9949300640 1 -- 172.21.15.45:0/2629668080 wait complete. 2024-09-18T14:17:22.559 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:22.559 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:17:22.559 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:17:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.656+0000 7f12d891d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.656+0000 7f12d891d640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.656+0000 7f12d891d640 10 monclient: monmap: 2024-09-18T14:17:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:22.657099+0000 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:22.657099+0000 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.658 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.658 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.656+0000 7f12d891d640 1 Processor -- start 2024-09-18T14:17:22.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.656+0000 7f12d891d640 1 -- start start 2024-09-18T14:17:22.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.656+0000 7f12d891d640 10 monclient: init 2024-09-18T14:17:22.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d891d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d891d640 10 monclient: _add_conns ranks=[6,2,0,4,8,3,5,7,1] 2024-09-18T14:17:22.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d891d640 10 monclient(hunting): picked mon.noname-d con 0x7f12d406e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:22.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d891d640 10 monclient(hunting): picked mon.noname-b con 0x7f12d40a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:22.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d891d640 10 monclient(hunting): picked mon.noname-a con 0x7f12d40a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:22.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d891d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d405ed70 con 0x7f12d40a1080 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d891d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d4097620 con 0x7f12d40a0cb0 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d891d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d4097340 con 0x7f12d406e810 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d891d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d891d640 10 monclient(hunting): authenticate will time out at 1857.995634s 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d259c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f12d406e810 0x7f12d40a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43186/0 (socket says 172.21.15.45:43186) 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d259c640 1 -- 172.21.15.45:0/1102519361 learned_addr learned my addr 172.21.15.45:0/1102519361 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145103343 0 0) 0x7f12d405ed70 con 0x7f12d40a1080 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d159a640 10 monclient(hunting): my global_id is 6507 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12c00034e0 con 0x7f12d40a1080 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222769864 0 0) 0x7f12d4097620 con 0x7f12d40a0cb0 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d159a640 10 monclient(hunting): my global_id is 145622 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d405ed70 con 0x7f12d40a0cb0 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.657+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611521384 0 0) 0x7f12d4097340 con 0x7f12d406e810 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient(hunting): my global_id is 185037 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d4097620 con 0x7f12d406e810 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1075531366 0 0) 0x7f12c00034e0 con 0x7f12d40a1080 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 >> v1:172.21.15.45:6792/0 conn(0x7f12d406e810 legacy=0x7f12d40a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 >> v1:172.21.15.45:6790/0 conn(0x7f12d40a0cb0 legacy=0x7f12d4137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 1 monclient: found mon.noname-a 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d40a1660 con 0x7f12d40a1080 2024-09-18T14:17:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f12c4001bb0 con 0x7f12d40a1080 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient: dump: 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.668 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d891d640 5 monclient: authenticate success, global_id 6507 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d891d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f12c4001d90 con 0x7f12d40a1080 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 1 -- 172.21.15.45:0/1102519361 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f12c40022c0 con 0x7f12d40a1080 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d159a640 10 monclient: dump: 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d891d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d891d640 4 set_mon_vals no callback set 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d891d640 10 monclient: shutdown 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d891d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d891d640 1 -- 172.21.15.45:0/1102519361 >> v1:172.21.15.45:6789/0 conn(0x7f12d40a1080 legacy=0x7f12d413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d891d640 1 -- 172.21.15.45:0/1102519361 shutdown_connections 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.658+0000 7f12d891d640 1 -- 172.21.15.45:0/1102519361 >> 172.21.15.45:0/1102519361 conn(0x7f12d40a5bb0 msgr2=0x7f12d40a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.659+0000 7f12d891d640 1 -- 172.21.15.45:0/1102519361 shutdown_connections 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.659+0000 7f12d891d640 1 -- 172.21.15.45:0/1102519361 wait complete. 2024-09-18T14:17:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.659+0000 7f12d891d640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.659+0000 7f12d891d640 10 monclient: monmap: 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.659+0000 7f12d891d640 1 Processor -- start 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.659+0000 7f12d891d640 1 -- start start 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.659+0000 7f12d891d640 10 monclient: init 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12d891d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12d891d640 10 monclient: _add_conns ranks=[2,7,4,0,1,8,6,3,5] 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12d891d640 10 monclient(hunting): picked mon.noname-b con 0x7f12d406e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12d891d640 10 monclient(hunting): picked mon.noname-i con 0x7f12d40a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12d891d640 10 monclient(hunting): picked mon.noname-c con 0x7f12d40a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12d891d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d409fa10 con 0x7f12d406e810 2024-09-18T14:17:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12d891d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d413da40 con 0x7f12d40a1080 2024-09-18T14:17:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12d891d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d40a2270 con 0x7f12d40a0cb0 2024-09-18T14:17:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12d891d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12d891d640 10 monclient(hunting): authenticate will time out at 1857.998293s 2024-09-18T14:17:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12d259c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f12d406e810 0x7f12d413d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60974/0 (socket says 172.21.15.45:60974) 2024-09-18T14:17:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12d259c640 1 -- 172.21.15.45:0/3185698596 learned_addr learned my addr 172.21.15.45:0/3185698596 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541946568 0 0) 0x7f12d409fa10 con 0x7f12d406e810 2024-09-18T14:17:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): my global_id is 145631 2024-09-18T14:17:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b40034e0 con 0x7f12d406e810 2024-09-18T14:17:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024248817 0 0) 0x7f12d413da40 con 0x7f12d40a1080 2024-09-18T14:17:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): my global_id is 165253 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d409fa10 con 0x7f12d40a1080 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162947906 0 0) 0x7f12d40a2270 con 0x7f12d40a0cb0 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): my global_id is 214486 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d413da40 con 0x7f12d40a0cb0 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1540561788 0 0) 0x7f12b40034e0 con 0x7f12d406e810 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12d40a2270 con 0x7f12d406e810 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f12cc001460 con 0x7f12d406e810 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:17:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): dump: 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.677 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1341498011 0 0) 0x7f12d409fa10 con 0x7f12d40a1080 2024-09-18T14:17:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12b40034e0 con 0x7f12d40a1080 2024-09-18T14:17:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f12c40013a0 con 0x7f12d40a1080 2024-09-18T14:17:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 10 monclient(hunting): dump: 2024-09-18T14:17:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.678 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 509168968 0 0) 0x7f12d40a2270 con 0x7f12d406e810 2024-09-18T14:17:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.660+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 >> v1:172.21.15.84:6792/0 conn(0x7f12d40a0cb0 legacy=0x7f12d4146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 >> v1:172.21.15.45:6791/0 conn(0x7f12d40a1080 legacy=0x7f12d4149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 1 monclient: found mon.b 2024-09-18T14:17:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d413e850 con 0x7f12d406e810 2024-09-18T14:17:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12d891d640 5 monclient: authenticate success, global_id 145631 2024-09-18T14:17:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12d891d640 10 monclient: _renew_subs 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12d891d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12d891d640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d41223d0 con 0x7f12d406e810 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12d891d640 10 monclient: _renew_subs 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12d891d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12d891d640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d4120070 con 0x7f12d406e810 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f12cc004a50 con 0x7f12d406e810 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f12cc004f00 con 0x7f12d406e810 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12d0d99640 4 set_mon_vals no callback set 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 10 monclient: dump: 2024-09-18T14:17:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12d891d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12d891d640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f12b8006580 con 0x7f12b8002830 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.661+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f12cc00fed0 con 0x7f12d406e810 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(849..849 src has 637..849) v4 ==== 3971+0+0 (unknown 3046409346 0 0) 0x7f12cc046c60 con 0x7f12d406e810 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032987144 0 0) 0x7f12b8006580 con 0x7f12b8002830 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 10 monclient(hunting): my global_id is 6128 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d40a2270 con 0x7f12b8002830 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12d2d9d640 10 monclient: get_auth_request con 0x7f12b403a5a0 auth_method 0 2024-09-18T14:17:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1696583294 0 0) 0x7f12d40a2270 con 0x7f12b8002830 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12b8006580 con 0x7f12b8002830 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f12bc0021f0 con 0x7f12b8002830 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 10 monclient: dump: 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3854818390 0 0) 0x7f12b8006580 con 0x7f12b8002830 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.662+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12b800a0a0 con 0x7f12b8002830 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.663+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f12bc004410 con 0x7f12b8002830 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.663+0000 7f12ca7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.663+0000 7f12ca7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.663+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 >> v1:172.21.15.45:6793/0 conn(0x7f12b8002830 legacy=0x7f12b8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.663+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 >> v1:172.21.15.45:6793/0 conn(0x7f12b8002830 legacy=0x7f12b8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.668+0000 7f12d891d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.668+0000 7f12d891d640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f12b8007540 con 0x7f12b80068f0 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355033660 0 0) 0x7f12b8007540 con 0x7f12b80068f0 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 10 monclient(hunting): my global_id is 6137 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b8006580 con 0x7f12b80068f0 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 737801318 0 0) 0x7f12b8006580 con 0x7f12b80068f0 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12b8007540 con 0x7f12b80068f0 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f12bc002cf0 con 0x7f12b80068f0 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:17:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 10 monclient: dump: 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.688 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1917163062 0 0) 0x7f12b8007540 con 0x7f12b80068f0 2024-09-18T14:17:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f12b80087a0 con 0x7f12b80068f0 2024-09-18T14:17:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7389 (unknown 0 0 463199725) 0x7f12bc004640 con 0x7f12b80068f0 2024-09-18T14:17:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 >> v1:172.21.15.45:6793/0 conn(0x7f12b80068f0 legacy=0x7f12b8006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.669+0000 7f12ca7fc640 1 -- 172.21.15.45:0/3185698596 >> v1:172.21.15.45:6793/0 conn(0x7f12b80068f0 legacy=0x7f12b8006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.688 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:22.707 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:22.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.670+0000 7f12d891d640 1 -- 172.21.15.45:0/3185698596 >> v1:172.21.15.84:6812/31478 conn(0x7f12b403a5a0 legacy=0x7f12b403ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.670+0000 7f12d891d640 10 monclient: shutdown 2024-09-18T14:17:22.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.670+0000 7f12d891d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.670+0000 7f12d891d640 1 -- 172.21.15.45:0/3185698596 >> v1:172.21.15.45:6790/0 conn(0x7f12d406e810 legacy=0x7f12d413d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.670+0000 7f12d2d9d640 1 -- 172.21.15.45:0/3185698596 reap_dead start 2024-09-18T14:17:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.670+0000 7f12d2d9d640 1 -- 172.21.15.45:0/3185698596 reap_dead start 2024-09-18T14:17:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.670+0000 7f12d891d640 1 -- 172.21.15.45:0/3185698596 shutdown_connections 2024-09-18T14:17:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.670+0000 7f12d891d640 1 -- 172.21.15.45:0/3185698596 >> 172.21.15.45:0/3185698596 conn(0x7f12d40a5bb0 msgr2=0x7f12d40a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.670+0000 7f12d891d640 1 -- 172.21.15.45:0/3185698596 shutdown_connections 2024-09-18T14:17:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.670+0000 7f12d891d640 1 -- 172.21.15.45:0/3185698596 wait complete. 2024-09-18T14:17:22.709 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:22.709 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:17:22.710 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:17:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.806+0000 7fa2e7ec9640 10 monclient: get_monmap_and_config 2024-09-18T14:17:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.806+0000 7fa2e7ec9640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.806+0000 7fa2e7ec9640 10 monclient: monmap: 2024-09-18T14:17:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:22.807251+0000 2024-09-18T14:17:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:22.807251+0000 2024-09-18T14:17:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.809 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.806+0000 7fa2e7ec9640 1 Processor -- start 2024-09-18T14:17:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e7ec9640 1 -- start start 2024-09-18T14:17:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e7ec9640 10 monclient: init 2024-09-18T14:17:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e7ec9640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e7ec9640 10 monclient: _add_conns ranks=[8,3,6,1,4,2,0,7,5] 2024-09-18T14:17:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e7ec9640 10 monclient(hunting): picked mon.noname-e con 0x7fa2e006e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e7ec9640 10 monclient(hunting): picked mon.noname-g con 0x7fa2e00a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e7ec9640 10 monclient(hunting): picked mon.noname-d con 0x7fa2e00a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e7ec9640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e005ed70 con 0x7fa2e00a0cb0 2024-09-18T14:17:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e7ec9640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e0097620 con 0x7fa2e00a1080 2024-09-18T14:17:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e7ec9640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e0097340 con 0x7fa2e006e810 2024-09-18T14:17:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e7ec9640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e7ec9640 10 monclient(hunting): authenticate will time out at 1858.145625s 2024-09-18T14:17:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e5c65640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fa2e006e810 0x7fa2e00a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:33090/0 (socket says 172.21.15.45:33090) 2024-09-18T14:17:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e6466640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fa2e00a1080 0x7fa2e013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43202/0 (socket says 172.21.15.45:43202) 2024-09-18T14:17:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.807+0000 7fa2e5c65640 1 -- 172.21.15.45:0/4102450878 learned_addr learned my addr 172.21.15.45:0/4102450878 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897293707 0 0) 0x7fa2e0097620 con 0x7fa2e00a1080 2024-09-18T14:17:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient(hunting): my global_id is 185046 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2cc0034e0 con 0x7fa2e00a1080 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142499021 0 0) 0x7fa2e0097340 con 0x7fa2e006e810 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient(hunting): my global_id is 6146 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e0097620 con 0x7fa2e006e810 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142517550 0 0) 0x7fa2e005ed70 con 0x7fa2e00a0cb0 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient(hunting): my global_id is 224202 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e0097340 con 0x7fa2e00a0cb0 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2041143413 0 0) 0x7fa2cc0034e0 con 0x7fa2e00a1080 2024-09-18T14:17:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 >> v1:172.21.15.45:6793/0 conn(0x7fa2e006e810 legacy=0x7fa2e00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 >> v1:172.21.15.84:6790/0 conn(0x7fa2e00a0cb0 legacy=0x7fa2e0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 monclient: found mon.noname-d 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00a1660 con 0x7fa2e00a1080 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e7ec9640 5 monclient: authenticate success, global_id 185046 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2d8001bb0 con 0x7fa2e00a1080 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e7ec9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:17:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient: dump: 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e7ec9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa2d8001d10 con 0x7fa2e00a1080 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e7ec9640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 1 -- 172.21.15.45:0/4102450878 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2d8004b20 con 0x7fa2e00a1080 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e7ec9640 4 set_mon_vals no callback set 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e4c63640 10 monclient: dump: 2024-09-18T14:17:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.825 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e7ec9640 10 monclient: shutdown 2024-09-18T14:17:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e7ec9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.808+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/4102450878 >> v1:172.21.15.45:6792/0 conn(0x7fa2e00a1080 legacy=0x7fa2e013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.809+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/4102450878 shutdown_connections 2024-09-18T14:17:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.809+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/4102450878 >> 172.21.15.45:0/4102450878 conn(0x7fa2e00a5bb0 msgr2=0x7fa2e00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.809+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/4102450878 shutdown_connections 2024-09-18T14:17:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.809+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/4102450878 wait complete. 2024-09-18T14:17:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.809+0000 7fa2e7ec9640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.809+0000 7fa2e7ec9640 10 monclient: monmap: 2024-09-18T14:17:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.809+0000 7fa2e7ec9640 1 Processor -- start 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.809+0000 7fa2e7ec9640 1 -- start start 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.809+0000 7fa2e7ec9640 10 monclient: init 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2e7ec9640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2e7ec9640 10 monclient: _add_conns ranks=[6,2,0,8,1,7,3,4,5] 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2e7ec9640 10 monclient(hunting): picked mon.noname-d con 0x7fa2e006e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2e7ec9640 10 monclient(hunting): picked mon.noname-b con 0x7fa2e00a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2e7ec9640 10 monclient(hunting): picked mon.noname-a con 0x7fa2e00a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2e7ec9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e009fa10 con 0x7fa2e00a1080 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2e7ec9640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e013dbb0 con 0x7fa2e00a0cb0 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2e7ec9640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e00a2270 con 0x7fa2e006e810 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2e7ec9640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2e7ec9640 10 monclient(hunting): authenticate will time out at 1858.148410s 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2e5c65640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fa2e006e810 0x7fa2e013d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43204/0 (socket says 172.21.15.45:43204) 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2e5c65640 1 -- 172.21.15.45:0/367755921 learned_addr learned my addr 172.21.15.45:0/367755921 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143036238 0 0) 0x7fa2e00a2270 con 0x7fa2e006e810 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 10 monclient(hunting): my global_id is 185055 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c40034e0 con 0x7fa2e006e810 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352306527 0 0) 0x7fa2e013dbb0 con 0x7fa2e00a0cb0 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 10 monclient(hunting): my global_id is 145640 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e00a2270 con 0x7fa2e00a0cb0 2024-09-18T14:17:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254681467 0 0) 0x7fa2e009fa10 con 0x7fa2e00a1080 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 10 monclient(hunting): my global_id is 6516 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e013dbb0 con 0x7fa2e00a1080 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 755816772 0 0) 0x7fa2c40034e0 con 0x7fa2e006e810 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2e009fa10 con 0x7fa2e006e810 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.810+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2dc0014b0 con 0x7fa2e006e810 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient(hunting): dump: 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 944326005 0 0) 0x7fa2e00a2270 con 0x7fa2e00a0cb0 2024-09-18T14:17:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2c40034e0 con 0x7fa2e00a0cb0 2024-09-18T14:17:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3966178100 0 0) 0x7fa2e013dbb0 con 0x7fa2e00a1080 2024-09-18T14:17:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2e00a2270 con 0x7fa2e00a1080 2024-09-18T14:17:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2d0001b20 con 0x7fa2e00a0cb0 2024-09-18T14:17:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient(hunting): dump: 2024-09-18T14:17:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2d8001a70 con 0x7fa2e00a1080 2024-09-18T14:17:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient(hunting): dump: 2024-09-18T14:17:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3505631423 0 0) 0x7fa2e009fa10 con 0x7fa2e006e810 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 >> v1:172.21.15.45:6790/0 conn(0x7fa2e00a0cb0 legacy=0x7fa2e01466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 >> v1:172.21.15.45:6789/0 conn(0x7fa2e00a1080 legacy=0x7fa2e0149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 monclient: found mon.d 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e013e9c0 con 0x7fa2e006e810 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2e7ec9640 5 monclient: authenticate success, global_id 185055 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2e7ec9640 10 monclient: _renew_subs 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2e7ec9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e0122540 con 0x7fa2e006e810 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa2dc0016d0 con 0x7fa2e006e810 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2dc004e10 con 0x7fa2e006e810 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d5ffb640 10 monclient: dump: 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.837 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2e7ec9640 10 monclient: _renew_subs 2024-09-18T14:17:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2e7ec9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2d7fff640 4 set_mon_vals no callback set 2024-09-18T14:17:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.811+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e0120150 con 0x7fa2e006e810 2024-09-18T14:17:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.812+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fa2dc00fde0 con 0x7fa2e006e810 2024-09-18T14:17:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.812+0000 7fa2e7ec9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.812+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa2e004c4a0 con 0x7fa2e0121620 2024-09-18T14:17:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.812+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(849..849 src has 637..849) v4 ==== 3971+0+0 (unknown 3046409346 0 0) 0x7fa2dc0469f0 con 0x7fa2e006e810 2024-09-18T14:17:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.812+0000 7fa2e6466640 10 monclient: get_auth_request con 0x7fa2c403ab30 auth_method 0 2024-09-18T14:17:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885745687 0 0) 0x7fa2e004c4a0 con 0x7fa2e0121620 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 10 monclient(hunting): my global_id is 214462 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e009fa10 con 0x7fa2e0121620 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2133763009 0 0) 0x7fa2e009fa10 con 0x7fa2e0121620 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2e004c4a0 con 0x7fa2e0121620 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2d00026e0 con 0x7fa2e0121620 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 10 monclient: dump: 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2266198120 0 0) 0x7fa2e004c4a0 con 0x7fa2e0121620 2024-09-18T14:17:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2e004b720 con 0x7fa2e0121620 2024-09-18T14:17:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa2d0002ac0 con 0x7fa2e0121620 2024-09-18T14:17:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 >> v1:172.21.15.84:6789/0 conn(0x7fa2e0121620 legacy=0x7fa2e009bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.813+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 >> v1:172.21.15.84:6789/0 conn(0x7fa2e0121620 legacy=0x7fa2e009bba0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.819+0000 7fa2e7ec9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.819+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa2e013e1c0 con 0x7fa2e013ddf0 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507716915 0 0) 0x7fa2e013e1c0 con 0x7fa2e013ddf0 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 10 monclient(hunting): my global_id is 214471 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e004c4a0 con 0x7fa2e013ddf0 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2337927072 0 0) 0x7fa2e004c4a0 con 0x7fa2e013ddf0 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2e013e1c0 con 0x7fa2e013ddf0 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa2d0002d50 con 0x7fa2e013ddf0 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 10 monclient: dump: 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3347765374 0 0) 0x7fa2e013e1c0 con 0x7fa2e013ddf0 2024-09-18T14:17:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa2e0062dd0 con 0x7fa2e013ddf0 2024-09-18T14:17:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 684061032) 0x7fa2d00055d0 con 0x7fa2e013ddf0 2024-09-18T14:17:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 >> v1:172.21.15.84:6789/0 conn(0x7fa2e013ddf0 legacy=0x7fa2e009c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.820+0000 7fa2d5ffb640 1 -- 172.21.15.45:0/367755921 >> v1:172.21.15.84:6789/0 conn(0x7fa2e013ddf0 legacy=0x7fa2e009c2b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.844 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:22.865 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:22.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.821+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/367755921 >> v1:172.21.15.84:6812/31478 conn(0x7fa2c403ab30 legacy=0x7fa2c403cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.821+0000 7fa2e7ec9640 10 monclient: shutdown 2024-09-18T14:17:22.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.821+0000 7fa2e7ec9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.821+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/367755921 >> v1:172.21.15.45:6792/0 conn(0x7fa2e006e810 legacy=0x7fa2e013d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.821+0000 7fa2e6466640 1 -- 172.21.15.45:0/367755921 reap_dead start 2024-09-18T14:17:22.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.821+0000 7fa2e6466640 1 -- 172.21.15.45:0/367755921 reap_dead start 2024-09-18T14:17:22.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.821+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/367755921 shutdown_connections 2024-09-18T14:17:22.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.821+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/367755921 >> 172.21.15.45:0/367755921 conn(0x7fa2e00a5bb0 msgr2=0x7fa2e00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.821+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/367755921 shutdown_connections 2024-09-18T14:17:22.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.821+0000 7fa2e7ec9640 1 -- 172.21.15.45:0/367755921 wait complete. 2024-09-18T14:17:22.867 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:22.867 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:17:22.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:17:22.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.964+0000 7f6c28309640 10 monclient: get_monmap_and_config 2024-09-18T14:17:22.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.964+0000 7f6c28309640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.964+0000 7f6c28309640 10 monclient: monmap: 2024-09-18T14:17:22.965 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.965 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.965 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:22.965530+0000 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:22.965530+0000 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 1 Processor -- start 2024-09-18T14:17:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 1 -- start start 2024-09-18T14:17:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 10 monclient: init 2024-09-18T14:17:22.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 10 monclient: _add_conns ranks=[3,2,6,0,1,4,8,7,5] 2024-09-18T14:17:22.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 10 monclient(hunting): picked mon.noname-g con 0x7f6c2009f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:22.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 10 monclient(hunting): picked mon.noname-b con 0x7f6c2009fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:22.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 10 monclient(hunting): picked mon.noname-d con 0x7f6c2009ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:22.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2005ed70 con 0x7f6c2009fba0 2024-09-18T14:17:22.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2009dfc0 con 0x7f6c2009f7d0 2024-09-18T14:17:22.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2009dce0 con 0x7f6c2009ff70 2024-09-18T14:17:22.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c28309640 10 monclient(hunting): authenticate will time out at 1858.303837s 2024-09-18T14:17:22.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c258a4640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f6c2009fba0 0x7f6c20137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60996/0 (socket says 172.21.15.45:60996) 2024-09-18T14:17:22.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c268a6640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f6c2009ff70 0x7f6c2013b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43212/0 (socket says 172.21.15.45:43212) 2024-09-18T14:17:22.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.965+0000 7f6c258a4640 1 -- 172.21.15.45:0/1907967888 learned_addr learned my addr 172.21.15.45:0/1907967888 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767710008 0 0) 0x7f6c2005ed70 con 0x7f6c2009fba0 2024-09-18T14:17:22.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient(hunting): my global_id is 145649 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c0c0034e0 con 0x7f6c2009fba0 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397591176 0 0) 0x7f6c2009dce0 con 0x7f6c2009ff70 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient(hunting): my global_id is 185064 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c2005ed70 con 0x7f6c2009ff70 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077159573 0 0) 0x7f6c2009dfc0 con 0x7f6c2009f7d0 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient(hunting): my global_id is 224211 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c2009dce0 con 0x7f6c2009f7d0 2024-09-18T14:17:22.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2419114333 0 0) 0x7f6c2005ed70 con 0x7f6c2009ff70 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 >> v1:172.21.15.84:6790/0 conn(0x7f6c2009f7d0 legacy=0x7f6c20132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 >> v1:172.21.15.45:6790/0 conn(0x7f6c2009fba0 legacy=0x7f6c20137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 monclient: found mon.noname-d 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c200a1c00 con 0x7f6c2009ff70 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6c18001b10 con 0x7f6c2009ff70 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c28309640 5 monclient: authenticate success, global_id 185064 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c28309640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient: dump: 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.979 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c28309640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6c18001cf0 con 0x7f6c2009ff70 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c28309640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 1 -- 172.21.15.45:0/1907967888 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6c18004e60 con 0x7f6c2009ff70 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c28309640 4 set_mon_vals no callback set 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.966+0000 7f6c250a3640 10 monclient: dump: 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.980 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.981 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.967+0000 7f6c28309640 10 monclient: shutdown 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.967+0000 7f6c28309640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.967+0000 7f6c28309640 1 -- 172.21.15.45:0/1907967888 >> v1:172.21.15.45:6792/0 conn(0x7f6c2009ff70 legacy=0x7f6c2013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.967+0000 7f6c28309640 1 -- 172.21.15.45:0/1907967888 shutdown_connections 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.967+0000 7f6c28309640 1 -- 172.21.15.45:0/1907967888 >> 172.21.15.45:0/1907967888 conn(0x7f6c200630b0 msgr2=0x7f6c200634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.967+0000 7f6c28309640 1 -- 172.21.15.45:0/1907967888 shutdown_connections 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.967+0000 7f6c28309640 1 -- 172.21.15.45:0/1907967888 wait complete. 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.967+0000 7f6c28309640 10 monclient: build_initial_monmap 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.967+0000 7f6c28309640 10 monclient: monmap: 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:22.982 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 1 Processor -- start 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 1 -- start start 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 10 monclient: init 2024-09-18T14:17:22.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 10 monclient: _add_conns ranks=[8,5,1,0,6,3,2,7,4] 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 10 monclient(hunting): picked mon.noname-e con 0x7f6c2009f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 10 monclient(hunting): picked mon.noname-h con 0x7f6c2009fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 10 monclient(hunting): picked mon.noname-f con 0x7f6c2009ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c201208b0 con 0x7f6c2009ff70 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c20121ee0 con 0x7f6c2009fba0 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2009d310 con 0x7f6c2009f7d0 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c28309640 10 monclient(hunting): authenticate will time out at 1858.306591s 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c260a5640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f6c2009f7d0 0x7f6c201418b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:33100/0 (socket says 172.21.15.45:33100) 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c260a5640 1 -- 172.21.15.45:0/2039184272 learned_addr learned my addr 172.21.15.45:0/2039184272 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649425321 0 0) 0x7f6c2009d310 con 0x7f6c2009f7d0 2024-09-18T14:17:22.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c167fc640 10 monclient(hunting): my global_id is 6155 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.968+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c040034e0 con 0x7f6c2009f7d0 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017807436 0 0) 0x7f6c201208b0 con 0x7f6c2009ff70 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): my global_id is 214480 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c2009d310 con 0x7f6c2009ff70 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571564911 0 0) 0x7f6c20121ee0 con 0x7f6c2009fba0 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): my global_id is 184901 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c201208b0 con 0x7f6c2009fba0 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1117766118 0 0) 0x7f6c040034e0 con 0x7f6c2009f7d0 2024-09-18T14:17:22.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c20121ee0 con 0x7f6c2009f7d0 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6c1c002bf0 con 0x7f6c2009f7d0 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): dump: 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.986 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4013711974 0 0) 0x7f6c2009d310 con 0x7f6c2009ff70 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c040034e0 con 0x7f6c2009ff70 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6c18001a50 con 0x7f6c2009ff70 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): dump: 2024-09-18T14:17:22.987 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.988 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.989 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.989 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 569175368 0 0) 0x7f6c201208b0 con 0x7f6c2009fba0 2024-09-18T14:17:22.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c2009d310 con 0x7f6c2009fba0 2024-09-18T14:17:22.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6c100015b0 con 0x7f6c2009fba0 2024-09-18T14:17:22.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:22.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient(hunting): dump: 2024-09-18T14:17:22.989 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.989 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.989 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.989 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2048709123 0 0) 0x7f6c20121ee0 con 0x7f6c2009f7d0 2024-09-18T14:17:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 >> v1:172.21.15.84:6791/0 conn(0x7f6c2009fba0 legacy=0x7f6c20131ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 >> v1:172.21.15.84:6789/0 conn(0x7f6c2009ff70 legacy=0x7f6c20149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient: _finish_hunting 0 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 monclient: found mon.e 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c20142750 con 0x7f6c2009f7d0 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient: _finish_auth 0 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c28309640 5 monclient: authenticate success, global_id 6155 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c28309640 10 monclient: _renew_subs 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c28309640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c28309640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c20121c00 con 0x7f6c2009f7d0 2024-09-18T14:17:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c28309640 10 monclient: _renew_subs 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c28309640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c28309640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c201200d0 con 0x7f6c2009f7d0 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6c1c001760 con 0x7f6c2009f7d0 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6c1c001c50 con 0x7f6c2009f7d0 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c248a2640 4 set_mon_vals no callback set 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.969+0000 7f6c167fc640 10 monclient: dump: 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.970+0000 7f6c28309640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.970+0000 7f6c28309640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6c08006580 con 0x7f6c08002830 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.970+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f6c1c001db0 con 0x7f6c2009f7d0 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.970+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(849..849 src has 637..849) v4 ==== 3971+0+0 (unknown 3046409346 0 0) 0x7f6c1c046c00 con 0x7f6c2009f7d0 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.970+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876637094 0 0) 0x7f6c08006580 con 0x7f6c08002830 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.970+0000 7f6c167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.970+0000 7f6c167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.970+0000 7f6c167fc640 10 monclient(hunting): my global_id is 224220 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.970+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c20121ee0 con 0x7f6c08002830 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c268a6640 10 monclient: get_auth_request con 0x7f6c0403a910 auth_method 0 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 712435069 0 0) 0x7f6c20121ee0 con 0x7f6c08002830 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c08006580 con 0x7f6c08002830 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6c10004ae0 con 0x7f6c08002830 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 10 monclient: dump: 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:22.996 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1045653183 0 0) 0x7f6c08006580 con 0x7f6c08002830 2024-09-18T14:17:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c0800a0a0 con 0x7f6c08002830 2024-09-18T14:17:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6c10006fc0 con 0x7f6c08002830 2024-09-18T14:17:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 >> v1:172.21.15.84:6790/0 conn(0x7f6c08002830 legacy=0x7f6c08004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.971+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 >> v1:172.21.15.84:6790/0 conn(0x7f6c08002830 legacy=0x7f6c08004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:22.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.977+0000 7f6c28309640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:22.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.977+0000 7f6c28309640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6c08007520 con 0x7f6c080068d0 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.977+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296765888 0 0) 0x7f6c08007520 con 0x7f6c080068d0 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.977+0000 7f6c167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.977+0000 7f6c167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.977+0000 7f6c167fc640 10 monclient(hunting): my global_id is 224229 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.977+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c08006580 con 0x7f6c080068d0 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2604265978 0 0) 0x7f6c08006580 con 0x7f6c080068d0 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c08007520 con 0x7f6c080068d0 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6c100055f0 con 0x7f6c080068d0 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 10 monclient: dump: 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.000 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 603849159 0 0) 0x7f6c08007520 con 0x7f6c080068d0 2024-09-18T14:17:23.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6c08008750 con 0x7f6c080068d0 2024-09-18T14:17:23.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 3171839575) 0x7f6c100069c0 con 0x7f6c080068d0 2024-09-18T14:17:23.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:23.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:23.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 >> v1:172.21.15.84:6790/0 conn(0x7f6c080068d0 legacy=0x7f6c08006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.978+0000 7f6c167fc640 1 -- 172.21.15.45:0/2039184272 >> v1:172.21.15.84:6790/0 conn(0x7f6c080068d0 legacy=0x7f6c08006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:23.000 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:23.019 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:23.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.979+0000 7f6c28309640 1 -- 172.21.15.45:0/2039184272 >> v1:172.21.15.84:6812/31478 conn(0x7f6c0403a910 legacy=0x7f6c0403cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.979+0000 7f6c28309640 10 monclient: shutdown 2024-09-18T14:17:23.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.979+0000 7f6c28309640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:23.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.979+0000 7f6c28309640 1 -- 172.21.15.45:0/2039184272 >> v1:172.21.15.45:6793/0 conn(0x7f6c2009f7d0 legacy=0x7f6c201418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.979+0000 7f6c268a6640 1 -- 172.21.15.45:0/2039184272 reap_dead start 2024-09-18T14:17:23.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.979+0000 7f6c268a6640 1 -- 172.21.15.45:0/2039184272 reap_dead start 2024-09-18T14:17:23.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.979+0000 7f6c28309640 1 -- 172.21.15.45:0/2039184272 shutdown_connections 2024-09-18T14:17:23.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.979+0000 7f6c28309640 1 -- 172.21.15.45:0/2039184272 >> 172.21.15.45:0/2039184272 conn(0x7f6c200630b0 msgr2=0x7f6c200a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:23.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.979+0000 7f6c28309640 1 -- 172.21.15.45:0/2039184272 shutdown_connections 2024-09-18T14:17:23.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:22.979+0000 7f6c28309640 1 -- 172.21.15.45:0/2039184272 wait complete. 2024-09-18T14:17:23.022 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:23.022 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:17:23.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:17:23.068 INFO:tasks.rados.rados.0.smithi084.stdout:3794: done (1 left) 2024-09-18T14:17:23.068 INFO:tasks.rados.rados.0.smithi084.stdout:3795: done (0 left) 2024-09-18T14:17:23.068 INFO:tasks.rados.rados.0.smithi084.stdout:3796: write (excl) oid 47 current snap is 424 2024-09-18T14:17:23.068 INFO:tasks.rados.rados.0.smithi084.stdout:3796: seq_num 1267 ranges {494469=655081,1741862=415779,2710226=584243} 2024-09-18T14:17:23.074 INFO:tasks.rados.rados.0.smithi084.stdout:3796: writing smithi08434663-47 from 494469 to 1149550 tid 1 2024-09-18T14:17:23.080 INFO:tasks.rados.rados.0.smithi084.stdout:3796: writing smithi08434663-47 from 1741862 to 2157641 tid 2 2024-09-18T14:17:23.086 INFO:tasks.rados.rados.0.smithi084.stdout:3796: writing smithi08434663-47 from 2710226 to 3294469 tid 3 2024-09-18T14:17:23.086 INFO:tasks.rados.rados.0.smithi084.stdout:3797: delete oid 48 current snap is 424 2024-09-18T14:17:23.086 INFO:tasks.rados.rados.0.smithi084.stdout:3796: finishing write tid 1 to smithi08434663-47 2024-09-18T14:17:23.087 INFO:tasks.rados.rados.0.smithi084.stdout:3796: finishing write tid 2 to smithi08434663-47 2024-09-18T14:17:23.091 INFO:tasks.rados.rados.0.smithi084.stdout:3796: finishing write tid 3 to smithi08434663-47 2024-09-18T14:17:23.091 INFO:tasks.rados.rados.0.smithi084.stdout:3797: done (1 left) 2024-09-18T14:17:23.091 INFO:tasks.rados.rados.0.smithi084.stdout:3798: write (excl) oid 33 current snap is 424 2024-09-18T14:17:23.092 INFO:tasks.rados.rados.0.smithi084.stdout:3798: seq_num 1269 ranges {528083=405579,1342473=419448,2248719=733949} 2024-09-18T14:17:23.096 INFO:tasks.rados.rados.0.smithi084.stdout:3798: writing smithi08434663-33 from 528083 to 933662 tid 1 2024-09-18T14:17:23.101 INFO:tasks.rados.rados.0.smithi084.stdout:3798: writing smithi08434663-33 from 1342473 to 1761921 tid 2 2024-09-18T14:17:23.108 INFO:tasks.rados.rados.0.smithi084.stdout:3798: writing smithi08434663-33 from 2248719 to 2982668 tid 3 2024-09-18T14:17:23.108 INFO:tasks.rados.rados.0.smithi084.stdout:3799: write (excl) oid 21 current snap is 424 2024-09-18T14:17:23.108 INFO:tasks.rados.rados.0.smithi084.stdout:3796: finishing write tid 4 to smithi08434663-47 2024-09-18T14:17:23.108 INFO:tasks.rados.rados.0.smithi084.stdout:3799: seq_num 1270 ranges {544890=597181,1917308=563106,3076142=268748} 2024-09-18T14:17:23.114 INFO:tasks.rados.rados.0.smithi084.stdout:3799: writing smithi08434663-21 from 544890 to 1142071 tid 1 2024-09-18T14:17:23.121 INFO:tasks.rados.rados.0.smithi084.stdout:3799: writing smithi08434663-21 from 1917308 to 2480414 tid 2 2024-09-18T14:17:23.127 INFO:tasks.rados.rados.0.smithi084.stdout:3799: writing smithi08434663-21 from 3076142 to 3344890 tid 3 2024-09-18T14:17:23.127 INFO:tasks.rados.rados.0.smithi084.stdout:3800: read oid 32 snap -1 2024-09-18T14:17:23.127 INFO:tasks.rados.rados.0.smithi084.stdout:3800: expect deleted 2024-09-18T14:17:23.127 INFO:tasks.rados.rados.0.smithi084.stdout:3801: read oid 19 snap -1 2024-09-18T14:17:23.127 INFO:tasks.rados.rados.0.smithi084.stdout:3801: expect (ObjNum 1156 snap 374 seq_num 1156) 2024-09-18T14:17:23.127 INFO:tasks.rados.rados.0.smithi084.stdout:3802: write oid 23 current snap is 424 2024-09-18T14:17:23.127 INFO:tasks.rados.rados.0.smithi084.stdout:3796: finishing write tid 5 to smithi08434663-47 2024-09-18T14:17:23.127 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 2056 (ObjNum 1267 snap 424 seq_num 1267) dirty exists 2024-09-18T14:17:23.128 INFO:tasks.rados.rados.0.smithi084.stdout:3796: left oid 47 (ObjNum 1267 snap 424 seq_num 1267) 2024-09-18T14:17:23.128 INFO:tasks.rados.rados.0.smithi084.stdout:3798: finishing write tid 1 to smithi08434663-33 2024-09-18T14:17:23.128 INFO:tasks.rados.rados.0.smithi084.stdout:3798: finishing write tid 2 to smithi08434663-33 2024-09-18T14:17:23.128 INFO:tasks.rados.rados.0.smithi084.stdout:3802: seq_num 1271 ranges {561697=672430,1975790=706764,3103565=258132} 2024-09-18T14:17:23.135 INFO:tasks.rados.rados.0.smithi084.stdout:3802: writing smithi08434663-23 from 561697 to 1234127 tid 1 2024-09-18T14:17:23.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.126+0000 7f81c5f5d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.126+0000 7f81c5f5d640 10 monclient: build_initial_monmap 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.126+0000 7f81c5f5d640 10 monclient: monmap: 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:23.127213+0000 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:23.127213+0000 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.126+0000 7f81c5f5d640 1 Processor -- start 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81c5f5d640 1 -- start start 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81c5f5d640 10 monclient: init 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81c5f5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81c5f5d640 10 monclient: _add_conns ranks=[4,7,5,1,8,6,0,2,3] 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81c5f5d640 10 monclient(hunting): picked mon.noname-c con 0x7f81c006e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81c5f5d640 10 monclient(hunting): picked mon.noname-i con 0x7f81c00a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81c5f5d640 10 monclient(hunting): picked mon.noname-h con 0x7f81c00a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81c5f5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c005ed70 con 0x7f81c006e810 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81c5f5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c0097620 con 0x7f81c00a1080 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81c5f5d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c0097340 con 0x7f81c00a0cb0 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81c5f5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81c5f5d640 10 monclient(hunting): authenticate will time out at 1858.465606s 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81bf7fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f81c006e810 0x7f81c00a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:41138/0 (socket says 172.21.15.45:41138) 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81bf7fe640 1 -- 172.21.15.45:0/47807026 learned_addr learned my addr 172.21.15.45:0/47807026 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.127+0000 7f81be7fc640 1 -- 172.21.15.45:0/47807026 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006892312 0 0) 0x7f81c005ed70 con 0x7f81c006e810 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient(hunting): my global_id is 165262 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 1 -- 172.21.15.45:0/47807026 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a80034e0 con 0x7f81c006e810 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 1 -- 172.21.15.45:0/47807026 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599193725 0 0) 0x7f81c0097620 con 0x7f81c00a1080 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient(hunting): my global_id is 184910 2024-09-18T14:17:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 1 -- 172.21.15.45:0/47807026 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81c005ed70 con 0x7f81c00a1080 2024-09-18T14:17:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 1 -- 172.21.15.45:0/47807026 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1484418677 0 0) 0x7f81a80034e0 con 0x7f81c006e810 2024-09-18T14:17:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 1 -- 172.21.15.45:0/47807026 >> v1:172.21.15.84:6792/0 conn(0x7f81c00a0cb0 legacy=0x7f81c0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 1 -- 172.21.15.45:0/47807026 >> v1:172.21.15.84:6791/0 conn(0x7f81c00a1080 legacy=0x7f81c013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:17:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 1 monclient: found mon.noname-c 2024-09-18T14:17:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:17:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 1 -- 172.21.15.45:0/47807026 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c00a1660 con 0x7f81c006e810 2024-09-18T14:17:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient: _finish_auth 0 2024-09-18T14:17:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 1 -- 172.21.15.45:0/47807026 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f81b0001830 con 0x7f81c006e810 2024-09-18T14:17:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81c5f5d640 5 monclient: authenticate success, global_id 165262 2024-09-18T14:17:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81c5f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:17:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient: dump: 2024-09-18T14:17:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81c5f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 1 -- 172.21.15.45:0/47807026 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f81b0001a10 con 0x7f81c006e810 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81c5f5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 1 -- 172.21.15.45:0/47807026 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f81b0001c00 con 0x7f81c006e810 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81c5f5d640 4 set_mon_vals no callback set 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.128+0000 7f81be7fc640 10 monclient: dump: 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.144 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.129+0000 7f81c5f5d640 10 monclient: shutdown 2024-09-18T14:17:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.129+0000 7f81c5f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.129+0000 7f81c5f5d640 1 -- 172.21.15.45:0/47807026 >> v1:172.21.15.45:6791/0 conn(0x7f81c006e810 legacy=0x7f81c00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.129+0000 7f81c5f5d640 1 -- 172.21.15.45:0/47807026 shutdown_connections 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.129+0000 7f81c5f5d640 1 -- 172.21.15.45:0/47807026 >> 172.21.15.45:0/47807026 conn(0x7f81c00a5bb0 msgr2=0x7f81c00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.129+0000 7f81c5f5d640 1 -- 172.21.15.45:0/47807026 shutdown_connections 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.129+0000 7f81c5f5d640 1 -- 172.21.15.45:0/47807026 wait complete. 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.129+0000 7f81c5f5d640 10 monclient: build_initial_monmap 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.129+0000 7f81c5f5d640 10 monclient: monmap: 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 1 Processor -- start 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 1 -- start start 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 10 monclient: init 2024-09-18T14:17:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 10 monclient: _add_conns ranks=[6,2,1,0,3,7,8,5,4] 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 10 monclient(hunting): picked mon.noname-d con 0x7f81c006e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 10 monclient(hunting): picked mon.noname-b con 0x7f81c00a0cb0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 10 monclient(hunting): picked mon.noname-f con 0x7f81c00a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c0122e90 con 0x7f81c00a1080 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c013dc80 con 0x7f81c00a0cb0 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c01228d0 con 0x7f81c006e810 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81c5f5d640 10 monclient(hunting): authenticate will time out at 1858.468794s 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81beffd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f81c00a0cb0 0x7f81c01467a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:32768/0 (socket says 172.21.15.45:32768) 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81bf7fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f81c006e810 0x7f81c013d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43230/0 (socket says 172.21.15.45:43230) 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.130+0000 7f81beffd640 1 -- 172.21.15.45:0/2673483281 learned_addr learned my addr 172.21.15.45:0/2673483281 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747152469 0 0) 0x7f81c01228d0 con 0x7f81c006e810 2024-09-18T14:17:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): my global_id is 185073 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81ac0034e0 con 0x7f81c006e810 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529893424 0 0) 0x7f81c013dc80 con 0x7f81c00a0cb0 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): my global_id is 145658 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81c01228d0 con 0x7f81c00a0cb0 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975956406 0 0) 0x7f81c0122e90 con 0x7f81c00a1080 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): my global_id is 214489 2024-09-18T14:17:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81c013dc80 con 0x7f81c00a1080 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4182569614 0 0) 0x7f81ac0034e0 con 0x7f81c006e810 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81c0122e90 con 0x7f81c006e810 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f81b0001060 con 0x7f81c006e810 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): dump: 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2106288518 0 0) 0x7f81c01228d0 con 0x7f81c00a0cb0 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81ac0034e0 con 0x7f81c00a0cb0 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f81b4001b20 con 0x7f81c00a0cb0 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): dump: 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.152 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.152 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.152 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1924395174 0 0) 0x7f81c013dc80 con 0x7f81c00a1080 2024-09-18T14:17:23.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81c01228d0 con 0x7f81c00a1080 2024-09-18T14:17:23.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f81b8001d80 con 0x7f81c00a1080 2024-09-18T14:17:23.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:23.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient(hunting): dump: 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 571585112 0 0) 0x7f81c0122e90 con 0x7f81c006e810 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 >> v1:172.21.15.45:6790/0 conn(0x7f81c00a0cb0 legacy=0x7f81c01467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 >> v1:172.21.15.84:6789/0 conn(0x7f81c00a1080 legacy=0x7f81c0149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient: _finish_hunting 0 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 monclient: found mon.d 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c0120470 con 0x7f81c006e810 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient: _finish_auth 0 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81c5f5d640 5 monclient: authenticate success, global_id 185073 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81c5f5d640 10 monclient: _renew_subs 2024-09-18T14:17:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81c5f5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81c5f5d640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c01225f0 con 0x7f81c006e810 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81c5f5d640 10 monclient: _renew_subs 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81c5f5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81c5f5d640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c0121720 con 0x7f81c006e810 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f81b0001bc0 con 0x7f81c006e810 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f81b0005110 con 0x7f81c006e810 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81bdffb640 4 set_mon_vals no callback set 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.131+0000 7f81a77fe640 10 monclient: dump: 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.132+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f81b00100e0 con 0x7f81c006e810 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.132+0000 7f81c5f5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.132+0000 7f81c5f5d640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f81a0006580 con 0x7f81a0002830 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.132+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(850..850 src has 637..850) v4 ==== 3943+0+0 (unknown 179577907 0 0) 0x7f81b0046c70 con 0x7f81c006e810 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.132+0000 7f81bffff640 10 monclient: get_auth_request con 0x7f81ac03a8c0 auth_method 0 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.132+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785684363 0 0) 0x7f81a0006580 con 0x7f81a0002830 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.132+0000 7f81a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.132+0000 7f81a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.132+0000 7f81a77fe640 10 monclient(hunting): my global_id is 184919 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.132+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81c0122e90 con 0x7f81a0002830 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2487250035 0 0) 0x7f81c0122e90 con 0x7f81a0002830 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81a0006580 con 0x7f81a0002830 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f81b40025f0 con 0x7f81a0002830 2024-09-18T14:17:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 10 monclient: dump: 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.159 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3954031654 0 0) 0x7f81a0006580 con 0x7f81a0002830 2024-09-18T14:17:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81a000a0a0 con 0x7f81a0002830 2024-09-18T14:17:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f81b4002ab0 con 0x7f81a0002830 2024-09-18T14:17:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 >> v1:172.21.15.84:6791/0 conn(0x7f81a0002830 legacy=0x7f81a0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.133+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 >> v1:172.21.15.84:6791/0 conn(0x7f81a0002830 legacy=0x7f81a0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:23.161 INFO:tasks.rados.rados.0.smithi084.stdout:3802: writing smithi08434663-23 from 1975790 to 2682554 tid 2 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:3802: writing smithi08434663-23 from 3103565 to 3361697 tid 3 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:3796: done (5 left) 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:3803: copy_from oid 48 from oid 9 current snap is 424 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:3798: finishing write tid 3 to smithi08434663-33 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:3798: finishing write tid 4 to smithi08434663-33 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:3798: finishing write tid 5 to smithi08434663-33 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 1749 (ObjNum 1269 snap 424 seq_num 1269) dirty exists 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:3798: left oid 33 (ObjNum 1269 snap 424 seq_num 1269) 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:3799: finishing write tid 1 to smithi08434663-21 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:3799: finishing write tid 2 to smithi08434663-21 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:3798: done (5 left) 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:3804: snap_create 2024-09-18T14:17:23.162 INFO:tasks.rados.rados.0.smithi084.stdout:3799: finishing write tid 3 to smithi08434663-21 2024-09-18T14:17:23.163 INFO:tasks.rados.rados.0.smithi084.stdout:3799: finishing write tid 4 to smithi08434663-21 2024-09-18T14:17:23.163 INFO:tasks.rados.rados.0.smithi084.stdout:3799: finishing write tid 5 to smithi08434663-21 2024-09-18T14:17:23.163 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 21 v 1574 (ObjNum 1270 snap 424 seq_num 1270) dirty exists 2024-09-18T14:17:23.163 INFO:tasks.rados.rados.0.smithi084.stdout:3799: left oid 21 (ObjNum 1270 snap 424 seq_num 1270) 2024-09-18T14:17:23.163 INFO:tasks.rados.rados.0.smithi084.stdout:3801: expect (ObjNum 1156 snap 374 seq_num 1156) 2024-09-18T14:17:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.139+0000 7f81c5f5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.139+0000 7f81c5f5d640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f81a0007540 con 0x7f81a00068f0 2024-09-18T14:17:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950148571 0 0) 0x7f81a0007540 con 0x7f81a00068f0 2024-09-18T14:17:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 10 monclient(hunting): my global_id is 184928 2024-09-18T14:17:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a0006580 con 0x7f81a00068f0 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1869930203 0 0) 0x7f81a0006580 con 0x7f81a00068f0 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81a0007540 con 0x7f81a00068f0 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f81b4002d40 con 0x7f81a00068f0 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 10 monclient: dump: 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.164 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 972258001 0 0) 0x7f81a0007540 con 0x7f81a00068f0 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f81a00087a0 con 0x7f81a00068f0 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 1189554907) 0x7f81b4007d90 con 0x7f81a00068f0 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:23.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:23.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 >> v1:172.21.15.84:6791/0 conn(0x7f81a00068f0 legacy=0x7f81a0006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.140+0000 7f81a77fe640 1 -- 172.21.15.45:0/2673483281 >> v1:172.21.15.84:6791/0 conn(0x7f81a00068f0 legacy=0x7f81a0006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:23.166 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:17:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:23.185 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:23.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.141+0000 7f81c5f5d640 1 -- 172.21.15.45:0/2673483281 >> v1:172.21.15.84:6812/31478 conn(0x7f81ac03a8c0 legacy=0x7f81ac03cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.141+0000 7f81c5f5d640 10 monclient: shutdown 2024-09-18T14:17:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.141+0000 7f81c5f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.141+0000 7f81c5f5d640 1 -- 172.21.15.45:0/2673483281 >> v1:172.21.15.45:6792/0 conn(0x7f81c006e810 legacy=0x7f81c013d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.141+0000 7f81bffff640 1 -- 172.21.15.45:0/2673483281 reap_dead start 2024-09-18T14:17:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.141+0000 7f81bffff640 1 -- 172.21.15.45:0/2673483281 reap_dead start 2024-09-18T14:17:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.141+0000 7f81c5f5d640 1 -- 172.21.15.45:0/2673483281 shutdown_connections 2024-09-18T14:17:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.141+0000 7f81c5f5d640 1 -- 172.21.15.45:0/2673483281 >> 172.21.15.45:0/2673483281 conn(0x7f81c00a5bb0 msgr2=0x7f81c00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.141+0000 7f81c5f5d640 1 -- 172.21.15.45:0/2673483281 shutdown_connections 2024-09-18T14:17:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.141+0000 7f81c5f5d640 1 -- 172.21.15.45:0/2673483281 wait complete. 2024-09-18T14:17:23.188 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:23.188 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:17:23.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:17:23.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.284+0000 7f6880ee8640 10 monclient: get_monmap_and_config 2024-09-18T14:17:23.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.284+0000 7f6880ee8640 10 monclient: build_initial_monmap 2024-09-18T14:17:23.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.284+0000 7f6880ee8640 10 monclient: monmap: 2024-09-18T14:17:23.285 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:23.285 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:23.285518+0000 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:23.285518+0000 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:23.286 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 1 Processor -- start 2024-09-18T14:17:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 1 -- start start 2024-09-18T14:17:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 10 monclient: init 2024-09-18T14:17:23.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:23.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 10 monclient: _add_conns ranks=[0,2,5,3,7,8,4,6,1] 2024-09-18T14:17:23.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 10 monclient(hunting): picked mon.noname-a con 0x7f687c09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:23.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 10 monclient(hunting): picked mon.noname-b con 0x7f687c09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:23.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 10 monclient(hunting): picked mon.noname-h con 0x7f687c09ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:23.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c05ed70 con 0x7f687c09f7d0 2024-09-18T14:17:23.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c09dfc0 con 0x7f687c09fba0 2024-09-18T14:17:23.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c09dce0 con 0x7f687c09ff70 2024-09-18T14:17:23.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:23.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6880ee8640 10 monclient(hunting): authenticate will time out at 1858.623876s 2024-09-18T14:17:23.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6879d9b640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f687c09fba0 0x7f687c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:32784/0 (socket says 172.21.15.45:32784) 2024-09-18T14:17:23.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f6879d9b640 1 -- 172.21.15.45:0/498579907 learned_addr learned my addr 172.21.15.45:0/498579907 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:23.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.285+0000 7f687a59c640 1 --1- 172.21.15.45:0/498579907 >> v1:172.21.15.45:6789/0 conn(0x7f687c09f7d0 0x7f687c132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46004/0 (socket says 172.21.15.45:46004) 2024-09-18T14:17:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416441945 0 0) 0x7f687c09dfc0 con 0x7f687c09fba0 2024-09-18T14:17:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient(hunting): my global_id is 145667 2024-09-18T14:17:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68640034e0 con 0x7f687c09fba0 2024-09-18T14:17:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312499620 0 0) 0x7f687c05ed70 con 0x7f687c09f7d0 2024-09-18T14:17:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient(hunting): my global_id is 6525 2024-09-18T14:17:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f687c09dfc0 con 0x7f687c09f7d0 2024-09-18T14:17:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760416943 0 0) 0x7f687c09dce0 con 0x7f687c09ff70 2024-09-18T14:17:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient(hunting): my global_id is 184937 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f687c05ed70 con 0x7f687c09ff70 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3438589368 0 0) 0x7f68640034e0 con 0x7f687c09fba0 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 >> v1:172.21.15.84:6791/0 conn(0x7f687c09ff70 legacy=0x7f687c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 >> v1:172.21.15.45:6789/0 conn(0x7f687c09f7d0 legacy=0x7f687c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient: _finish_hunting 0 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 monclient: found mon.noname-b 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c0a1c00 con 0x7f687c09fba0 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient: _finish_auth 0 2024-09-18T14:17:23.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6870001830 con 0x7f687c09fba0 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f6880ee8640 5 monclient: authenticate success, global_id 145667 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f6880ee8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient: dump: 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.298 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f6880ee8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6870001990 con 0x7f687c09fba0 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 1 -- 172.21.15.45:0/498579907 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6870004ba0 con 0x7f687c09fba0 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f6880ee8640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:23.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f6880ee8640 4 set_mon_vals no callback set 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.286+0000 7f687959a640 10 monclient: dump: 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.300 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.287+0000 7f6880ee8640 10 monclient: shutdown 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.287+0000 7f6880ee8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.287+0000 7f6880ee8640 1 -- 172.21.15.45:0/498579907 >> v1:172.21.15.45:6790/0 conn(0x7f687c09fba0 legacy=0x7f687c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.287+0000 7f6880ee8640 1 -- 172.21.15.45:0/498579907 shutdown_connections 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.287+0000 7f6880ee8640 1 -- 172.21.15.45:0/498579907 >> 172.21.15.45:0/498579907 conn(0x7f687c0630b0 msgr2=0x7f687c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.287+0000 7f6880ee8640 1 -- 172.21.15.45:0/498579907 shutdown_connections 2024-09-18T14:17:23.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.287+0000 7f6880ee8640 1 -- 172.21.15.45:0/498579907 wait complete. 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.287+0000 7f6880ee8640 10 monclient: build_initial_monmap 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.287+0000 7f6880ee8640 10 monclient: monmap: 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 1 Processor -- start 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 1 -- start start 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 10 monclient: init 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 10 monclient: _add_conns ranks=[4,0,6,5,1,7,2,3,8] 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 10 monclient(hunting): picked mon.noname-c con 0x7f687c09f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 10 monclient(hunting): picked mon.noname-a con 0x7f687c09fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 10 monclient(hunting): picked mon.noname-d con 0x7f687c09ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c09d310 con 0x7f687c09fba0 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c0a2560 con 0x7f687c09f7d0 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c142600 con 0x7f687c09ff70 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6880ee8640 10 monclient(hunting): authenticate will time out at 1858.626645s 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f687a59c640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f687c09f7d0 0x7f687c141730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:41142/0 (socket says 172.21.15.45:41142) 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f687a59c640 1 -- 172.21.15.45:0/3677891059 learned_addr learned my addr 172.21.15.45:0/3677891059 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f6879d9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f687c09fba0 0x7f687c131930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46020/0 (socket says 172.21.15.45:46020) 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834392613 0 0) 0x7f687c0a2560 con 0x7f687c09f7d0 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 10 monclient(hunting): my global_id is 165271 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f685c0034e0 con 0x7f687c09f7d0 2024-09-18T14:17:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575211785 0 0) 0x7f687c142600 con 0x7f687c09ff70 2024-09-18T14:17:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 10 monclient(hunting): my global_id is 185082 2024-09-18T14:17:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f687c0a2560 con 0x7f687c09ff70 2024-09-18T14:17:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289400428 0 0) 0x7f687c09d310 con 0x7f687c09fba0 2024-09-18T14:17:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 10 monclient(hunting): my global_id is 6534 2024-09-18T14:17:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.288+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f687c142600 con 0x7f687c09fba0 2024-09-18T14:17:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1865177744 0 0) 0x7f687c0a2560 con 0x7f687c09ff70 2024-09-18T14:17:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f687c09d310 con 0x7f687c09ff70 2024-09-18T14:17:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6874002cb0 con 0x7f687c09ff70 2024-09-18T14:17:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:17:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient(hunting): dump: 2024-09-18T14:17:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3532966028 0 0) 0x7f685c0034e0 con 0x7f687c09f7d0 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f687c0a2560 con 0x7f687c09f7d0 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1541546471 0 0) 0x7f687c142600 con 0x7f687c09fba0 2024-09-18T14:17:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f685c0034e0 con 0x7f687c09fba0 2024-09-18T14:17:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f686c001590 con 0x7f687c09f7d0 2024-09-18T14:17:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient(hunting): dump: 2024-09-18T14:17:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6870001770 con 0x7f687c09fba0 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient(hunting): dump: 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 531062454 0 0) 0x7f687c09d310 con 0x7f687c09ff70 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 >> v1:172.21.15.45:6791/0 conn(0x7f687c09f7d0 legacy=0x7f687c141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 >> v1:172.21.15.45:6789/0 conn(0x7f687c09fba0 legacy=0x7f687c131930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 monclient: found mon.d 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c120b20 con 0x7f687c09ff70 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f6880ee8640 5 monclient: authenticate success, global_id 185082 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f6880ee8640 10 monclient: _renew_subs 2024-09-18T14:17:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f6880ee8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f6880ee8640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687c121a80 con 0x7f687c09ff70 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f6880ee8640 10 monclient: _renew_subs 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f6880ee8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f6880ee8640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c120650 con 0x7f687c09ff70 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6874001440 con 0x7f687c09ff70 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f68740018f0 con 0x7f687c09ff70 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f6878d99640 4 set_mon_vals no callback set 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.289+0000 7f686a7fc640 10 monclient: dump: 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.312 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.313 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.290+0000 7f6880ee8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.290+0000 7f6880ee8640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6860006580 con 0x7f6860002830 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.290+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f6874001a50 con 0x7f687c09ff70 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.290+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(850..850 src has 637..850) v4 ==== 3943+0+0 (unknown 179577907 0 0) 0x7f6874044440 con 0x7f687c09ff70 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.290+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2483596185 0 0) 0x7f6860006580 con 0x7f6860002830 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.290+0000 7f686a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.290+0000 7f686a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.290+0000 7f686a7fc640 10 monclient(hunting): my global_id is 214504 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.290+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f687c09d310 con 0x7f6860002830 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.290+0000 7f6879d9b640 10 monclient: get_auth_request con 0x7f685c03aa30 auth_method 0 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 490578301 0 0) 0x7f687c09d310 con 0x7f6860002830 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6860006580 con 0x7f6860002830 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f686c004dd0 con 0x7f6860002830 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 10 monclient: dump: 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.316 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.316 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.316 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.316 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 458213032 0 0) 0x7f6860006580 con 0x7f6860002830 2024-09-18T14:17:23.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f686000a0a0 con 0x7f6860002830 2024-09-18T14:17:23.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f686c007090 con 0x7f6860002830 2024-09-18T14:17:23.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:23.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:23.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 >> v1:172.21.15.84:6792/0 conn(0x7f6860002830 legacy=0x7f6860004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.291+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 >> v1:172.21.15.84:6792/0 conn(0x7f6860002830 legacy=0x7f6860004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.297+0000 7f6880ee8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.297+0000 7f6880ee8640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6860007540 con 0x7f68600068f0 2024-09-18T14:17:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.297+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053009215 0 0) 0x7f6860007540 con 0x7f68600068f0 2024-09-18T14:17:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.297+0000 7f686a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.297+0000 7f686a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.297+0000 7f686a7fc640 10 monclient(hunting): my global_id is 214513 2024-09-18T14:17:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.297+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6860006580 con 0x7f68600068f0 2024-09-18T14:17:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2703866101 0 0) 0x7f6860006580 con 0x7f68600068f0 2024-09-18T14:17:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6860007540 con 0x7f68600068f0 2024-09-18T14:17:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f686c005850 con 0x7f68600068f0 2024-09-18T14:17:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:17:23.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 10 monclient: dump: 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.319 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.320 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2388671781 0 0) 0x7f6860007540 con 0x7f68600068f0 2024-09-18T14:17:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f68600087a0 con 0x7f68600068f0 2024-09-18T14:17:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 3957625140) 0x7f686c006cc0 con 0x7f68600068f0 2024-09-18T14:17:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 >> v1:172.21.15.84:6792/0 conn(0x7f68600068f0 legacy=0x7f6860006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f686a7fc640 1 -- 172.21.15.45:0/3677891059 >> v1:172.21.15.84:6792/0 conn(0x7f68600068f0 legacy=0x7f6860006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:23.320 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 146, 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:17:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:23.340 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f6880ee8640 1 -- 172.21.15.45:0/3677891059 >> v1:172.21.15.84:6812/31478 conn(0x7f685c03aa30 legacy=0x7f685c03cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f6880ee8640 10 monclient: shutdown 2024-09-18T14:17:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f6880ee8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f6880ee8640 1 -- 172.21.15.45:0/3677891059 >> v1:172.21.15.45:6792/0 conn(0x7f687c09ff70 legacy=0x7f687c149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f687ad9d640 1 -- 172.21.15.45:0/3677891059 reap_dead start 2024-09-18T14:17:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f687ad9d640 1 -- 172.21.15.45:0/3677891059 reap_dead start 2024-09-18T14:17:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f6880ee8640 1 -- 172.21.15.45:0/3677891059 shutdown_connections 2024-09-18T14:17:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.298+0000 7f6880ee8640 1 -- 172.21.15.45:0/3677891059 >> 172.21.15.45:0/3677891059 conn(0x7f687c0630b0 msgr2=0x7f687c0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.299+0000 7f6880ee8640 1 -- 172.21.15.45:0/3677891059 shutdown_connections 2024-09-18T14:17:23.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.299+0000 7f6880ee8640 1 -- 172.21.15.45:0/3677891059 wait complete. 2024-09-18T14:17:23.343 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:17:23.343 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b', 'e'] 2024-09-18T14:17:23.343 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-18T14:17:23.344 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-18T14:17:23.344 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-18T14:17:23.344 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-18T14:17:23.344 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:17:23.344 INFO:tasks.rados.rados.0.smithi084.stdout:3802: finishing write tid 1 to smithi08434663-23 2024-09-18T14:17:23.344 INFO:tasks.rados.rados.0.smithi084.stdout:3802: finishing write tid 2 to smithi08434663-23 2024-09-18T14:17:23.344 INFO:tasks.rados.rados.0.smithi084.stdout:3802: finishing write tid 3 to smithi08434663-23 2024-09-18T14:17:23.344 INFO:tasks.rados.rados.0.smithi084.stdout:3802: finishing write tid 4 to smithi08434663-23 2024-09-18T14:17:23.345 INFO:tasks.rados.rados.0.smithi084.stdout:3802: finishing write tid 5 to smithi08434663-23 2024-09-18T14:17:23.345 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1560 (ObjNum 1271 snap 424 seq_num 1271) dirty exists 2024-09-18T14:17:23.345 INFO:tasks.rados.rados.0.smithi084.stdout:3802: left oid 23 (ObjNum 1271 snap 424 seq_num 1271) 2024-09-18T14:17:23.345 INFO:tasks.rados.rados.0.smithi084.stdout:3803: finishing copy_from racing read to smithi08434663-48 2024-09-18T14:17:23.345 INFO:tasks.rados.rados.0.smithi084.stdout:3803: finishing copy_from to smithi08434663-48 2024-09-18T14:17:23.345 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 2052 (ObjNum 1251 snap 411 seq_num 1251) dirty exists 2024-09-18T14:17:23.359 INFO:tasks.ceph.mon.b:Stopped 2024-09-18T14:17:23.359 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-09-18T14:17:23.359 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-18T14:17:23.359 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-18T14:17:23.359 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:17:23.417 INFO:tasks.ceph.mon.e:Stopped 2024-09-18T14:17:23.417 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-18T14:17:23.418 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:17:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.552+0000 7f84b1b5d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.552+0000 7f84b1b5d640 10 monclient: build_initial_monmap 2024-09-18T14:17:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.552+0000 7f84b1b5d640 10 monclient: monmap: 2024-09-18T14:17:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:23.553446+0000 2024-09-18T14:17:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:23.553446+0000 2024-09-18T14:17:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:23.554 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 1 Processor -- start 2024-09-18T14:17:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 1 -- start start 2024-09-18T14:17:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 10 monclient: init 2024-09-18T14:17:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 10 monclient: _add_conns ranks=[4,1,5,7,2,0,6,3,8] 2024-09-18T14:17:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 10 monclient(hunting): picked mon.noname-c con 0x7f84ac09f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 10 monclient(hunting): picked mon.noname-f con 0x7f84ac09fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 10 monclient(hunting): picked mon.noname-h con 0x7f84ac09ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ac05ed70 con 0x7f84ac09fba0 2024-09-18T14:17:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ac09dfc0 con 0x7f84ac09f7d0 2024-09-18T14:17:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ac09dce0 con 0x7f84ac09ff70 2024-09-18T14:17:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84b1b5d640 10 monclient(hunting): authenticate will time out at 1858.891797s 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84ab7fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f84ac09f7d0 0x7f84ac132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:41150/0 (socket says 172.21.15.45:41150) 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.553+0000 7f84ab7fe640 1 -- 172.21.15.45:0/2221319575 learned_addr learned my addr 172.21.15.45:0/2221319575 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470155597 0 0) 0x7f84ac09dfc0 con 0x7f84ac09f7d0 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient(hunting): my global_id is 165280 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84940034e0 con 0x7f84ac09f7d0 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661856031 0 0) 0x7f84ac09dce0 con 0x7f84ac09ff70 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient(hunting): my global_id is 184946 2024-09-18T14:17:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84ac09dfc0 con 0x7f84ac09ff70 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362430649 0 0) 0x7f84ac05ed70 con 0x7f84ac09fba0 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient(hunting): my global_id is 214498 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84ac09dce0 con 0x7f84ac09fba0 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3751414990 0 0) 0x7f84ac09dfc0 con 0x7f84ac09ff70 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 >> v1:172.21.15.45:6791/0 conn(0x7f84ac09f7d0 legacy=0x7f84ac132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 >> v1:172.21.15.84:6789/0 conn(0x7f84ac09fba0 legacy=0x7f84ac137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 monclient: found mon.noname-h 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T14:17:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac0a1c00 con 0x7f84ac09ff70 2024-09-18T14:17:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient: _finish_auth 0 2024-09-18T14:17:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84b1b5d640 5 monclient: authenticate success, global_id 184946 2024-09-18T14:17:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f84a4001ac0 con 0x7f84ac09ff70 2024-09-18T14:17:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84b1b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:17:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient: dump: 2024-09-18T14:17:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84b1b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f84a4001c20 con 0x7f84ac09ff70 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 1 -- 172.21.15.45:0/2221319575 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f84a40021d0 con 0x7f84ac09ff70 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84aa7fc640 10 monclient: dump: 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84b1b5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.554+0000 7f84b1b5d640 4 set_mon_vals no callback set 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.555+0000 7f84b1b5d640 10 monclient: shutdown 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.555+0000 7f84b1b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.555+0000 7f84b1b5d640 1 -- 172.21.15.45:0/2221319575 >> v1:172.21.15.84:6791/0 conn(0x7f84ac09ff70 legacy=0x7f84ac13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.555+0000 7f84b1b5d640 1 -- 172.21.15.45:0/2221319575 shutdown_connections 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.555+0000 7f84b1b5d640 1 -- 172.21.15.45:0/2221319575 >> 172.21.15.45:0/2221319575 conn(0x7f84ac0630b0 msgr2=0x7f84ac0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.555+0000 7f84b1b5d640 1 -- 172.21.15.45:0/2221319575 shutdown_connections 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.555+0000 7f84b1b5d640 1 -- 172.21.15.45:0/2221319575 wait complete. 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.555+0000 7f84b1b5d640 10 monclient: build_initial_monmap 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.555+0000 7f84b1b5d640 10 monclient: monmap: 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:23.568 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 1 Processor -- start 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 1 -- start start 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 10 monclient: init 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 10 monclient: _add_conns ranks=[0,8,1,5,3,4,7,6,2] 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 10 monclient(hunting): picked mon.noname-a con 0x7f84ac09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:23.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 10 monclient(hunting): picked mon.noname-e con 0x7f84ac09fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 10 monclient(hunting): picked mon.noname-f con 0x7f84ac09ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ac11f4a0 con 0x7f84ac09f7d0 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ac1207a0 con 0x7f84ac09ff70 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ac11f8b0 con 0x7f84ac09fba0 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84b1b5d640 10 monclient(hunting): authenticate will time out at 1858.894645s 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84aaffd640 1 -- >> v1:172.21.15.45:6793/0 conn(0x7f84ac09fba0 legacy=0x7f84ac146b30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84ab7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f84ac09f7d0 0x7f84ac130650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46024/0 (socket says 172.21.15.45:46024) 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84ab7fe640 1 -- 172.21.15.45:0/1402206258 learned_addr learned my addr 172.21.15.45:0/1402206258 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774655322 0 0) 0x7f84ac11f4a0 con 0x7f84ac09f7d0 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84937fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84937fe640 10 monclient(hunting): my global_id is 6543 2024-09-18T14:17:23.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.556+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84980034e0 con 0x7f84ac09f7d0 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702162542 0 0) 0x7f84ac1207a0 con 0x7f84ac09ff70 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient(hunting): my global_id is 214507 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84ac11f4a0 con 0x7f84ac09ff70 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1298366867 0 0) 0x7f84980034e0 con 0x7f84ac09f7d0 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f84ac1207a0 con 0x7f84ac09f7d0 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f849c001690 con 0x7f84ac09f7d0 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient(hunting): dump: 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.571 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.572 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.572 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.572 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.572 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.572 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.572 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.572 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.572 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.573 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.573 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.573 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.573 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.573 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.573 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:23.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1513507176 0 0) 0x7f84ac11f4a0 con 0x7f84ac09ff70 2024-09-18T14:17:23.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f84980034e0 con 0x7f84ac09ff70 2024-09-18T14:17:23.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f84a40020d0 con 0x7f84ac09ff70 2024-09-18T14:17:23.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:23.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient(hunting): dump: 2024-09-18T14:17:23.574 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.574 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.574 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.574 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2424784380 0 0) 0x7f84ac1207a0 con 0x7f84ac09f7d0 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 >> v1:172.21.15.45:6793/0 conn(0x7f84ac09fba0 legacy=0x7f84ac146b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 >> v1:172.21.15.84:6789/0 conn(0x7f84ac09ff70 legacy=0x7f84ac14a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient: _finish_hunting 0 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 monclient: found mon.a 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac09d310 con 0x7f84ac09f7d0 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient: _finish_auth 0 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84b1b5d640 5 monclient: authenticate success, global_id 6543 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84b1b5d640 10 monclient: _renew_subs 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84b1b5d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:23.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84b1b5d640 1 -- 172.21.15.45:0/1402206258 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ac0a2560 con 0x7f84ac09f7d0 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84b1b5d640 10 monclient: _renew_subs 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84b1b5d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84b1b5d640 1 -- 172.21.15.45:0/1402206258 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ac11ed20 con 0x7f84ac09f7d0 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f849c004a50 con 0x7f84ac09f7d0 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f849c004f00 con 0x7f84ac09f7d0 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84a9ffb640 4 set_mon_vals no callback set 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.557+0000 7f84937fe640 10 monclient: dump: 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:23.577 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:23.579 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.558+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f849c00fed0 con 0x7f84ac09f7d0 2024-09-18T14:17:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.558+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(850..850 src has 637..850) v4 ==== 3943+0+0 (unknown 179577907 0 0) 0x7f849c046cb0 con 0x7f84ac09f7d0 2024-09-18T14:17:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.558+0000 7f84b1b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.558+0000 7f84b1b5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.558+0000 7f84b1b5d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.558+0000 7f84b1b5d640 1 -- 172.21.15.45:0/1402206258 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848c005010 con 0x7f84ac09f7d0 2024-09-18T14:17:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.558+0000 7f84aaffd640 10 monclient: get_auth_request con 0x7f849803a420 auth_method 0 2024-09-18T14:17:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.563+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f849c0119e0 con 0x7f84ac09f7d0 2024-09-18T14:17:23.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.563+0000 7f84937fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:23.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.563+0000 7f84937fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:23.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.789+0000 7f84b1b5d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:17:23.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.789+0000 7f84b1b5d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:17:23.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.789+0000 7f84b1b5d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:23.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.789+0000 7f84b1b5d640 1 -- 172.21.15.45:0/1402206258 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f848c005840 con 0x7f84ac09f7d0 2024-09-18T14:17:23.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.789+0000 7f84937fe640 1 -- 172.21.15.45:0/1402206258 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2689 (unknown 254746657 0 3339958800) 0x7f849c014e10 con 0x7f84ac09f7d0 2024-09-18T14:17:23.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.789+0000 7f84937fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:17:23.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.789+0000 7f84937fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:23.790 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":146,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":5,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:17:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.791+0000 7f84b1b5d640 1 -- 172.21.15.45:0/1402206258 >> v1:172.21.15.84:6812/31478 conn(0x7f849803a420 legacy=0x7f849803c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.791+0000 7f84b1b5d640 10 monclient: shutdown 2024-09-18T14:17:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.791+0000 7f84b1b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.791+0000 7f84b1b5d640 1 -- 172.21.15.45:0/1402206258 >> v1:172.21.15.45:6789/0 conn(0x7f84ac09f7d0 legacy=0x7f84ac130650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.791+0000 7f84b1b5d640 1 -- 172.21.15.45:0/1402206258 shutdown_connections 2024-09-18T14:17:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.791+0000 7f84b1b5d640 1 -- 172.21.15.45:0/1402206258 >> 172.21.15.45:0/1402206258 conn(0x7f84ac0630b0 msgr2=0x7f84ac0a4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.791+0000 7f84b1b5d640 1 -- 172.21.15.45:0/1402206258 shutdown_connections 2024-09-18T14:17:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:23.791+0000 7f84b1b5d640 1 -- 172.21.15.45:0/1402206258 wait complete. 2024-09-18T14:17:26.803 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:17:26.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.910+0000 7f9403e20640 10 monclient: get_monmap_and_config 2024-09-18T14:17:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.910+0000 7f9403e20640 10 monclient: build_initial_monmap 2024-09-18T14:17:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.910+0000 7f9403e20640 10 monclient: monmap: 2024-09-18T14:17:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:26.911854+0000 2024-09-18T14:17:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:26.911854+0000 2024-09-18T14:17:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:26.921 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.911+0000 7f9403e20640 1 Processor -- start 2024-09-18T14:17:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.911+0000 7f9403e20640 1 -- start start 2024-09-18T14:17:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.911+0000 7f9403e20640 10 monclient: init 2024-09-18T14:17:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9403e20640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9403e20640 10 monclient: _add_conns ranks=[0,2,6,4,7,8,1,3,5] 2024-09-18T14:17:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9403e20640 10 monclient(hunting): picked mon.noname-a con 0x7f93fc09f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9403e20640 10 monclient(hunting): picked mon.noname-b con 0x7f93fc09fba0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9403e20640 10 monclient(hunting): picked mon.noname-d con 0x7f93fc09ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9403e20640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc05ed70 con 0x7f93fc09f7d0 2024-09-18T14:17:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9403e20640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc09dfc0 con 0x7f93fc09fba0 2024-09-18T14:17:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9403e20640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc09dce0 con 0x7f93fc09ff70 2024-09-18T14:17:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9403e20640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f94013bb640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7f93fc09fba0 legacy=0x7f93fc137a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:17:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9403e20640 10 monclient(hunting): authenticate will time out at 1862.250466s 2024-09-18T14:17:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9401bbc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f93fc09f7d0 0x7f93fc132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46040/0 (socket says 172.21.15.45:46040) 2024-09-18T14:17:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f94023bd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f93fc09ff70 0x7f93fc13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:43266/0 (socket says 172.21.15.45:43266) 2024-09-18T14:17:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9401bbc640 1 -- 172.21.15.45:0/65137379 learned_addr learned my addr 172.21.15.45:0/65137379 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:26.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9400bba640 1 -- 172.21.15.45:0/65137379 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999797393 0 0) 0x7f93fc05ed70 con 0x7f93fc09f7d0 2024-09-18T14:17:26.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9400bba640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:26.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9400bba640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:26.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9400bba640 10 monclient(hunting): my global_id is 6552 2024-09-18T14:17:26.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9400bba640 1 -- 172.21.15.45:0/65137379 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93e40034e0 con 0x7f93fc09f7d0 2024-09-18T14:17:26.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9400bba640 1 -- 172.21.15.45:0/65137379 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407051832 0 0) 0x7f93fc09dce0 con 0x7f93fc09ff70 2024-09-18T14:17:26.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9400bba640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:26.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9400bba640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:26.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9400bba640 10 monclient(hunting): my global_id is 185091 2024-09-18T14:17:26.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.912+0000 7f9400bba640 1 -- 172.21.15.45:0/65137379 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93fc05ed70 con 0x7f93fc09ff70 2024-09-18T14:17:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 1 -- 172.21.15.45:0/65137379 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3076609774 0 0) 0x7f93e40034e0 con 0x7f93fc09f7d0 2024-09-18T14:17:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 1 -- 172.21.15.45:0/65137379 >> v1:172.21.15.45:6792/0 conn(0x7f93fc09ff70 legacy=0x7f93fc13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 1 -- 172.21.15.45:0/65137379 >> v1:172.21.15.45:6790/0 conn(0x7f93fc09fba0 legacy=0x7f93fc137a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:17:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 10 monclient: _finish_hunting 0 2024-09-18T14:17:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 1 monclient: found mon.noname-a 2024-09-18T14:17:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:17:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 1 -- 172.21.15.45:0/65137379 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc0a1c00 con 0x7f93fc09f7d0 2024-09-18T14:17:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 10 monclient: _finish_auth 0 2024-09-18T14:17:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 1 -- 172.21.15.45:0/65137379 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f93ec001bb0 con 0x7f93fc09f7d0 2024-09-18T14:17:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9403e20640 5 monclient: authenticate success, global_id 6552 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9403e20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 10 monclient: dump: 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9403e20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 1 -- 172.21.15.45:0/65137379 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f93ec001d90 con 0x7f93fc09f7d0 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 1 -- 172.21.15.45:0/65137379 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f93ec004f20 con 0x7f93fc09f7d0 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9400bba640 10 monclient: dump: 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:26.928 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9403e20640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9403e20640 4 set_mon_vals no callback set 2024-09-18T14:17:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9403e20640 10 monclient: shutdown 2024-09-18T14:17:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9403e20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9403e20640 1 -- 172.21.15.45:0/65137379 >> v1:172.21.15.45:6789/0 conn(0x7f93fc09f7d0 legacy=0x7f93fc132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9403e20640 1 -- 172.21.15.45:0/65137379 shutdown_connections 2024-09-18T14:17:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9403e20640 1 -- 172.21.15.45:0/65137379 >> 172.21.15.45:0/65137379 conn(0x7f93fc0630b0 msgr2=0x7f93fc0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.913+0000 7f9403e20640 1 -- 172.21.15.45:0/65137379 shutdown_connections 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.914+0000 7f9403e20640 1 -- 172.21.15.45:0/65137379 wait complete. 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.914+0000 7f9403e20640 10 monclient: build_initial_monmap 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.914+0000 7f9403e20640 10 monclient: monmap: 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:26.930 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.914+0000 7f9403e20640 1 Processor -- start 2024-09-18T14:17:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.914+0000 7f9403e20640 1 -- start start 2024-09-18T14:17:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.914+0000 7f9403e20640 10 monclient: init 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f9403e20640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f9403e20640 10 monclient: _add_conns ranks=[5,8,1,2,7,0,6,4,3] 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f9403e20640 10 monclient(hunting): picked mon.noname-h con 0x7f93fc09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f9403e20640 10 monclient(hunting): picked mon.noname-e con 0x7f93fc09fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f9403e20640 10 monclient(hunting): picked mon.noname-f con 0x7f93fc09ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f9403e20640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc11c650 con 0x7f93fc09ff70 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f94013bb640 1 -- >> v1:172.21.15.45:6793/0 conn(0x7f93fc09fba0 legacy=0x7f93fc142290 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f9403e20640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc0a2560 con 0x7f93fc09f7d0 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f9403e20640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc11d290 con 0x7f93fc09fba0 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f9403e20640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f9403e20640 10 monclient(hunting): authenticate will time out at 1862.253506s 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f94023bd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f93fc09ff70 0x7f93fc149de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:42324/0 (socket says 172.21.15.45:42324) 2024-09-18T14:17:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f9401bbc640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f93fc09f7d0 0x7f93fc141b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:49472/0 (socket says 172.21.15.45:49472) 2024-09-18T14:17:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f94023bd640 1 -- 172.21.15.45:0/909171139 learned_addr learned my addr 172.21.15.45:0/909171139 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.915+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383186368 0 0) 0x7f93fc0a2560 con 0x7f93fc09f7d0 2024-09-18T14:17:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): my global_id is 184955 2024-09-18T14:17:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93e80034e0 con 0x7f93fc09f7d0 2024-09-18T14:17:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675477675 0 0) 0x7f93fc11c650 con 0x7f93fc09ff70 2024-09-18T14:17:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): my global_id is 214516 2024-09-18T14:17:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93fc0a2560 con 0x7f93fc09ff70 2024-09-18T14:17:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2454389053 0 0) 0x7f93e80034e0 con 0x7f93fc09f7d0 2024-09-18T14:17:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f93fc11c650 con 0x7f93fc09f7d0 2024-09-18T14:17:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 246705686 0 0) 0x7f93fc0a2560 con 0x7f93fc09ff70 2024-09-18T14:17:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f93e80034e0 con 0x7f93fc09ff70 2024-09-18T14:17:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f93ec004f60 con 0x7f93fc09f7d0 2024-09-18T14:17:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:17:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): dump: 2024-09-18T14:17:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:26.936 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f93f40019e0 con 0x7f93fc09ff70 2024-09-18T14:17:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient(hunting): dump: 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:26.938 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3018594847 0 0) 0x7f93fc11c650 con 0x7f93fc09f7d0 2024-09-18T14:17:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 >> v1:172.21.15.45:6793/0 conn(0x7f93fc09fba0 legacy=0x7f93fc142290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:17:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 >> v1:172.21.15.84:6789/0 conn(0x7f93fc09ff70 legacy=0x7f93fc149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:17:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 monclient: found mon.h 2024-09-18T14:17:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc09d310 con 0x7f93fc09f7d0 2024-09-18T14:17:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient: _finish_auth 0 2024-09-18T14:17:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f9403e20640 5 monclient: authenticate success, global_id 184955 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f9403e20640 10 monclient: _renew_subs 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f9403e20640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f9403e20640 1 -- 172.21.15.45:0/909171139 --> v1:172.21.15.84:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93fc1429a0 con 0x7f93fc09f7d0 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.5 v1:172.21.15.84:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f93ec0017b0 con 0x7f93fc09f7d0 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.5 v1:172.21.15.84:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f93ec005560 con 0x7f93fc09f7d0 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93f9ffb640 10 monclient: dump: 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.917+0000 7f9403e20640 10 monclient: _renew_subs 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.917+0000 7f9403e20640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.917+0000 7f9403e20640 1 -- 172.21.15.45:0/909171139 --> v1:172.21.15.84:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc09bfe0 con 0x7f93fc09f7d0 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.916+0000 7f93fbfff640 4 set_mon_vals no callback set 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.917+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.5 v1:172.21.15.84:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f93ec0105b0 con 0x7f93fc09f7d0 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.917+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.5 v1:172.21.15.84:6791/0 8 ==== osd_map(850..850 src has 637..850) v4 ==== 3943+0+0 (unknown 179577907 0 0) 0x7f93ec047060 con 0x7f93fc09f7d0 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.918+0000 7f9403e20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.918+0000 7f9403e20640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.918+0000 7f9403e20640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.918+0000 7f9403e20640 1 -- 172.21.15.45:0/909171139 --> v1:172.21.15.84:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc04c4a0 con 0x7f93fc09f7d0 2024-09-18T14:17:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.918+0000 7f94013bb640 10 monclient: get_auth_request con 0x7f93e803a3b0 auth_method 0 2024-09-18T14:17:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.923+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.5 v1:172.21.15.84:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f93ec015460 con 0x7f93fc09f7d0 2024-09-18T14:17:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.923+0000 7f93f9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:26.923+0000 7f93f9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.151+0000 7f9403e20640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:17:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.151+0000 7f9403e20640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:17:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.151+0000 7f9403e20640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.151+0000 7f9403e20640 1 -- 172.21.15.45:0/909171139 --> v1:172.21.15.84:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f93fc04b720 con 0x7f93fc09f7d0 2024-09-18T14:17:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.151+0000 7f93f9ffb640 1 -- 172.21.15.45:0/909171139 <== mon.5 v1:172.21.15.84:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2689 (unknown 254746657 0 430523881) 0x7f93ec015280 con 0x7f93fc09f7d0 2024-09-18T14:17:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.152+0000 7f93f9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:17:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.152+0000 7f93f9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:27.153 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":146,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":8,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:17:27.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.154+0000 7f9403e20640 1 -- 172.21.15.45:0/909171139 >> v1:172.21.15.84:6812/31478 conn(0x7f93e803a3b0 legacy=0x7f93e803c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:27.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.154+0000 7f9403e20640 10 monclient: shutdown 2024-09-18T14:17:27.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.154+0000 7f9403e20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:27.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.154+0000 7f9403e20640 1 -- 172.21.15.45:0/909171139 >> v1:172.21.15.84:6791/0 conn(0x7f93fc09f7d0 legacy=0x7f93fc141b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:27.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.154+0000 7f9403e20640 1 -- 172.21.15.45:0/909171139 shutdown_connections 2024-09-18T14:17:27.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.154+0000 7f9403e20640 1 -- 172.21.15.45:0/909171139 >> 172.21.15.45:0/909171139 conn(0x7f93fc0630b0 msgr2=0x7f93fc130b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:27.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.154+0000 7f9403e20640 1 -- 172.21.15.45:0/909171139 shutdown_connections 2024-09-18T14:17:27.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:27.154+0000 7f9403e20640 1 -- 172.21.15.45:0/909171139 wait complete. 2024-09-18T14:17:30.166 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:17:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.273+0000 7f7b44e6f640 10 monclient: get_monmap_and_config 2024-09-18T14:17:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.273+0000 7f7b44e6f640 10 monclient: build_initial_monmap 2024-09-18T14:17:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.273+0000 7f7b44e6f640 10 monclient: monmap: 2024-09-18T14:17:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:30.274864+0000 2024-09-18T14:17:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:30.274864+0000 2024-09-18T14:17:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:30.276 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.274+0000 7f7b44e6f640 1 Processor -- start 2024-09-18T14:17:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.274+0000 7f7b44e6f640 1 -- start start 2024-09-18T14:17:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.274+0000 7f7b44e6f640 10 monclient: init 2024-09-18T14:17:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.275+0000 7f7b44e6f640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.275+0000 7f7b44e6f640 10 monclient: _add_conns ranks=[4,0,3,6,8,1,7,5,2] 2024-09-18T14:17:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.275+0000 7f7b44e6f640 10 monclient(hunting): picked mon.noname-c con 0x7f7b4009f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.275+0000 7f7b44e6f640 10 monclient(hunting): picked mon.noname-a con 0x7f7b4009fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.275+0000 7f7b44e6f640 10 monclient(hunting): picked mon.noname-g con 0x7f7b4009ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.275+0000 7f7b44e6f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4005ed70 con 0x7f7b4009fba0 2024-09-18T14:17:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.275+0000 7f7b44e6f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4009dfc0 con 0x7f7b4009ff70 2024-09-18T14:17:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.275+0000 7f7b44e6f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4009dce0 con 0x7f7b4009f7d0 2024-09-18T14:17:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.275+0000 7f7b44e6f640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.275+0000 7f7b44e6f640 10 monclient(hunting): authenticate will time out at 1865.613305s 2024-09-18T14:17:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.275+0000 7f7b3e59c640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f7b4009f7d0 0x7f7b40132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:55466/0 (socket says 172.21.15.45:55466) 2024-09-18T14:17:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:30.275+0000 7f7b3e59c640 1 -- 172.21.15.45:0/3875396164 learned_addr learned my addr 172.21.15.45:0/3875396164 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 10 monclient(hunting): tick 2024-09-18T14:17:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 1 monclient(hunting): continuing hunt 2024-09-18T14:17:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6791/0 conn(0x7f7b4009f7d0 legacy=0x7f7b40132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.84:6790/0 conn(0x7f7b4009ff70 legacy=0x7f7b4013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6789/0 conn(0x7f7b4009fba0 legacy=0x7f7b40137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 10 monclient: _add_conns ranks=[1,2,8,7,6,5] 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 10 monclient(hunting): picked mon.noname-f con 0x7f7b200037a0 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 10 monclient(hunting): picked mon.noname-b con 0x7f7b20007380 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 10 monclient(hunting): picked mon.noname-e con 0x7f7b2000b100 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 1 -- 172.21.15.45:0/3875396164 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b400a1c00 con 0x7f7b200037a0 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 1 -- 172.21.15.45:0/3875396164 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2000ff80 con 0x7f7b20007380 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 1 -- 172.21.15.45:0/3875396164 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b200111c0 con 0x7f7b2000b100 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b2ffff640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b3dd9b640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6790/0 conn(0x7f7b20007380 legacy=0x7f7b20009830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:17:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.276+0000 7f7b3ed9d640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6793/0 conn(0x7f7b2000b100 legacy=0x7f7b2000d5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:33.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.477+0000 7f7b3dd9b640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6790/0 conn(0x7f7b20007380 legacy=0x7f7b20009830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:17:33.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.477+0000 7f7b3ed9d640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6793/0 conn(0x7f7b2000b100 legacy=0x7f7b2000d5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.878+0000 7f7b3ed9d640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6793/0 conn(0x7f7b2000b100 legacy=0x7f7b2000d5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:33.878+0000 7f7b3dd9b640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6790/0 conn(0x7f7b20007380 legacy=0x7f7b20009830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:17:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:34.678+0000 7f7b3ed9d640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6793/0 conn(0x7f7b2000b100 legacy=0x7f7b2000d5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:34.678+0000 7f7b3dd9b640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6790/0 conn(0x7f7b20007380 legacy=0x7f7b20009830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 10 monclient(hunting): tick 2024-09-18T14:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 1 monclient(hunting): continuing hunt 2024-09-18T14:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 10 monclient(hunting): _reopen_session rank -1 2024-09-18T14:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6793/0 conn(0x7f7b2000b100 legacy=0x7f7b2000d5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6790/0 conn(0x7f7b20007380 legacy=0x7f7b20009830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.84:6789/0 conn(0x7f7b200037a0 legacy=0x7f7b20005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 10 monclient: _add_conns ranks=[7,6,5] 2024-09-18T14:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b3ed9d640 1 -- 172.21.15.45:0/3875396164 reap_dead start 2024-09-18T14:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 10 monclient(hunting): picked mon.noname-i con 0x7f7b2000f110 addr v1:172.21.15.84:6792/0 2024-09-18T14:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 10 monclient(hunting): picked mon.noname-d con 0x7f7b2000dd60 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 10 monclient(hunting): picked mon.noname-h con 0x7f7b2000e130 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 1 -- 172.21.15.45:0/3875396164 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b20012420 con 0x7f7b2000e130 2024-09-18T14:17:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 1 -- 172.21.15.45:0/3875396164 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2000ff80 con 0x7f7b2000dd60 2024-09-18T14:17:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 1 -- 172.21.15.45:0/3875396164 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b200111c0 con 0x7f7b2000f110 2024-09-18T14:17:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:36.275+0000 7f7b2ffff640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.071+0000 7f7b3d59a640 1 -- 172.21.15.45:0/3875396164 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527099022 0 0) 0x7f7b200111c0 con 0x7f7b2000f110 2024-09-18T14:17:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.071+0000 7f7b3d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.071+0000 7f7b3d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.071+0000 7f7b3d59a640 10 monclient(hunting): my global_id is 214522 2024-09-18T14:17:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.071+0000 7f7b3d59a640 1 -- 172.21.15.45:0/3875396164 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b280034e0 con 0x7f7b2000f110 2024-09-18T14:17:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.071+0000 7f7b3d59a640 1 -- 172.21.15.45:0/3875396164 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699891861 0 0) 0x7f7b2000ff80 con 0x7f7b2000dd60 2024-09-18T14:17:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.071+0000 7f7b3d59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.071+0000 7f7b3d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.071+0000 7f7b3d59a640 10 monclient(hunting): my global_id is 185100 2024-09-18T14:17:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.071+0000 7f7b3d59a640 1 -- 172.21.15.45:0/3875396164 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b200111c0 con 0x7f7b2000dd60 2024-09-18T14:17:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 1 -- 172.21.15.45:0/3875396164 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3561059692 0 0) 0x7f7b200111c0 con 0x7f7b2000dd60 2024-09-18T14:17:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.84:6792/0 conn(0x7f7b2000f110 legacy=0x7f7b2000f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.84:6791/0 conn(0x7f7b2000e130 legacy=0x7f7b20005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 10 monclient: _finish_hunting 0 2024-09-18T14:17:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 1 monclient: found mon.noname-d 2024-09-18T14:17:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:17:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 1 -- 172.21.15.45:0/3875396164 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2000d680 con 0x7f7b2000dd60 2024-09-18T14:17:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 10 monclient: _finish_auth 0 2024-09-18T14:17:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 1 -- 172.21.15.45:0/3875396164 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7b340018f0 con 0x7f7b2000dd60 2024-09-18T14:17:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b44e6f640 5 monclient: authenticate success, global_id 185100 2024-09-18T14:17:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b44e6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:17:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 10 monclient: dump: 2024-09-18T14:17:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b44e6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 1 -- 172.21.15.45:0/3875396164 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7b34001ac0 con 0x7f7b2000dd60 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b44e6f640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 1 -- 172.21.15.45:0/3875396164 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7b34002070 con 0x7f7b2000dd60 2024-09-18T14:17:38.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b44e6f640 4 set_mon_vals no callback set 2024-09-18T14:17:38.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:38.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.072+0000 7f7b3d59a640 10 monclient: dump: 2024-09-18T14:17:38.079 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.079 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.079 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.079 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.079 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.079 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.073+0000 7f7b44e6f640 10 monclient: shutdown 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.073+0000 7f7b44e6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.073+0000 7f7b44e6f640 1 -- 172.21.15.45:0/3875396164 >> v1:172.21.15.45:6792/0 conn(0x7f7b2000dd60 legacy=0x7f7b20009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.073+0000 7f7b44e6f640 1 -- 172.21.15.45:0/3875396164 shutdown_connections 2024-09-18T14:17:38.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.073+0000 7f7b44e6f640 1 -- 172.21.15.45:0/3875396164 >> 172.21.15.45:0/3875396164 conn(0x7f7b400630b0 msgr2=0x7f7b400634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:38.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.073+0000 7f7b44e6f640 1 -- 172.21.15.45:0/3875396164 shutdown_connections 2024-09-18T14:17:38.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.073+0000 7f7b44e6f640 1 -- 172.21.15.45:0/3875396164 wait complete. 2024-09-18T14:17:38.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.074+0000 7f7b44e6f640 10 monclient: build_initial_monmap 2024-09-18T14:17:38.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.074+0000 7f7b44e6f640 10 monclient: monmap: 2024-09-18T14:17:38.084 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.074+0000 7f7b44e6f640 1 Processor -- start 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.074+0000 7f7b44e6f640 1 -- start start 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.074+0000 7f7b44e6f640 10 monclient: init 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b44e6f640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b44e6f640 10 monclient: _add_conns ranks=[7,1,2,8,6,4,0,5,3] 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b44e6f640 10 monclient(hunting): picked mon.noname-i con 0x7f7b2000dd60 addr v1:172.21.15.84:6792/0 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b44e6f640 10 monclient(hunting): picked mon.noname-f con 0x7f7b2000e130 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b44e6f640 10 monclient(hunting): picked mon.noname-b con 0x7f7b2000f110 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b44e6f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b400a2560 con 0x7f7b2000e130 2024-09-18T14:17:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b44e6f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4009bd00 con 0x7f7b2000f110 2024-09-18T14:17:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b44e6f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4011e9f0 con 0x7f7b2000dd60 2024-09-18T14:17:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b3ed9d640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7f7b2000f110 legacy=0x7f7b4014a650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:17:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b44e6f640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b44e6f640 10 monclient(hunting): authenticate will time out at 1873.414457s 2024-09-18T14:17:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b3e59c640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f7b2000dd60 0x7f7b401318d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:47534/0 (socket says 172.21.15.45:47534) 2024-09-18T14:17:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.075+0000 7f7b3e59c640 1 -- 172.21.15.45:0/1811141707 learned_addr learned my addr 172.21.15.45:0/1811141707 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.080+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491998369 0 0) 0x7f7b400a2560 con 0x7f7b2000e130 2024-09-18T14:17:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.080+0000 7f7b2e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.080+0000 7f7b2e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.080+0000 7f7b2e7fc640 10 monclient(hunting): my global_id is 214525 2024-09-18T14:17:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.080+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b2000ee40 con 0x7f7b2000e130 2024-09-18T14:17:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.080+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3359921080 0 0) 0x7f7b2000ee40 con 0x7f7b2000e130 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.080+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7b400a2560 con 0x7f7b2000e130 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.080+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7b34001f70 con 0x7f7b2000e130 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.080+0000 7f7b2e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.080+0000 7f7b2e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.080+0000 7f7b2e7fc640 10 monclient(hunting): dump: 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.088 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.080+0000 7f7b2e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3018875830 0 0) 0x7f7b400a2560 con 0x7f7b2000e130 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 >> v1:172.21.15.84:6792/0 conn(0x7f7b2000dd60 legacy=0x7f7b401318d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 >> v1:172.21.15.45:6790/0 conn(0x7f7b2000f110 legacy=0x7f7b4014a650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:17:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 1 monclient: found mon.f 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4011ecd0 con 0x7f7b2000e130 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b44e6f640 5 monclient: authenticate success, global_id 214525 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b44e6f640 10 monclient: _renew_subs 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b44e6f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b44e6f640 1 -- 172.21.15.45:0/1811141707 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4011da90 con 0x7f7b2000e130 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b44e6f640 10 monclient: _renew_subs 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b44e6f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:38.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b44e6f640 1 -- 172.21.15.45:0/1811141707 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b40131fe0 con 0x7f7b2000e130 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7b34002380 con 0x7f7b2000e130 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7b340028b0 con 0x7f7b2000e130 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b3cd99640 4 set_mon_vals no callback set 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.081+0000 7f7b2e7fc640 10 monclient: dump: 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.092 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.082+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7b34002a90 con 0x7f7b2000e130 2024-09-18T14:17:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.082+0000 7f7b44e6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.082+0000 7f7b44e6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.082+0000 7f7b44e6f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:38.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.082+0000 7f7b44e6f640 1 -- 172.21.15.45:0/1811141707 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b24005010 con 0x7f7b2000e130 2024-09-18T14:17:38.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.082+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(850..850 src has 637..850) v4 ==== 3943+0+0 (unknown 179577907 0 0) 0x7f7b34046c60 con 0x7f7b2000e130 2024-09-18T14:17:38.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.082+0000 7f7b3e59c640 10 monclient: get_auth_request con 0x7f7b2003bf90 auth_method 0 2024-09-18T14:17:38.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.088+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 <== mon.1 v1:172.21.15.84:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f7b34015480 con 0x7f7b2000e130 2024-09-18T14:17:38.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.088+0000 7f7b2e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:38.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.088+0000 7f7b2e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:38.137 INFO:tasks.rados.rados.0.smithi084.stdout:3799: done (5 left) 2024-09-18T14:17:38.137 INFO:tasks.rados.rados.0.smithi084.stdout:3800: done (4 left) 2024-09-18T14:17:38.137 INFO:tasks.rados.rados.0.smithi084.stdout:3801: done (3 left) 2024-09-18T14:17:38.137 INFO:tasks.rados.rados.0.smithi084.stdout:3802: done (2 left) 2024-09-18T14:17:38.137 INFO:tasks.rados.rados.0.smithi084.stdout:3803: done (1 left) 2024-09-18T14:17:38.137 INFO:tasks.rados.rados.0.smithi084.stdout:3804: done (0 left) 2024-09-18T14:17:38.137 INFO:tasks.rados.rados.0.smithi084.stdout:3805: snap_create 2024-09-18T14:17:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.322+0000 7f7b44e6f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:17:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.322+0000 7f7b44e6f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:17:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.322+0000 7f7b44e6f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.322+0000 7f7b44e6f640 1 -- 172.21.15.45:0/1811141707 --> v1:172.21.15.84:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7b24005840 con 0x7f7b2000e130 2024-09-18T14:17:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.324+0000 7f7b2e7fc640 1 -- 172.21.15.45:0/1811141707 <== mon.1 v1:172.21.15.84:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2677 (unknown 254746657 0 203262924) 0x7f7b34014e80 con 0x7f7b2000e130 2024-09-18T14:17:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.324+0000 7f7b2e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:17:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.324+0000 7f7b2e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:38.325 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":152,"quorum":[0,1,3,4,5,6,7],"quorum_names":["a","f","g","c","h","d","i"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:17:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.325+0000 7f7b44e6f640 1 -- 172.21.15.45:0/1811141707 >> v1:172.21.15.84:6812/31478 conn(0x7f7b2003bf90 legacy=0x7f7b2003e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.325+0000 7f7b44e6f640 10 monclient: shutdown 2024-09-18T14:17:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.325+0000 7f7b44e6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.325+0000 7f7b44e6f640 1 -- 172.21.15.45:0/1811141707 >> v1:172.21.15.84:6789/0 conn(0x7f7b2000e130 legacy=0x7f7b40146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.325+0000 7f7b44e6f640 1 -- 172.21.15.45:0/1811141707 shutdown_connections 2024-09-18T14:17:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.325+0000 7f7b44e6f640 1 -- 172.21.15.45:0/1811141707 >> 172.21.15.45:0/1811141707 conn(0x7f7b400630b0 msgr2=0x7f7b400a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.325+0000 7f7b44e6f640 1 -- 172.21.15.45:0/1811141707 shutdown_connections 2024-09-18T14:17:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.325+0000 7f7b44e6f640 1 -- 172.21.15.45:0/1811141707 wait complete. 2024-09-18T14:17:38.337 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-18T14:17:38.337 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-09-18T14:17:38.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:17:38.369 INFO:tasks.rados.rados.0.smithi084.stdout:3805: done (0 left) 2024-09-18T14:17:38.369 INFO:tasks.rados.rados.0.smithi084.stdout:3806: delete oid 48 current snap is 426 2024-09-18T14:17:38.376 INFO:tasks.rados.rados.0.smithi084.stdout:3806: done (0 left) 2024-09-18T14:17:38.376 INFO:tasks.rados.rados.0.smithi084.stdout:3807: delete oid 26 current snap is 426 2024-09-18T14:17:38.378 INFO:tasks.rados.rados.0.smithi084.stdout:3807: done (0 left) 2024-09-18T14:17:38.378 INFO:tasks.rados.rados.0.smithi084.stdout:3808: write oid 39 current snap is 426 2024-09-18T14:17:38.379 INFO:tasks.rados.rados.0.smithi084.stdout:3808: seq_num 1274 ranges {612118=498177,1634883=570444,2669481=706662} 2024-09-18T14:17:38.384 INFO:tasks.rados.rados.0.smithi084.stdout:3808: writing smithi08434663-39 from 612118 to 1110295 tid 1 2024-09-18T14:17:38.390 INFO:tasks.rados.rados.0.smithi084.stdout:3808: writing smithi08434663-39 from 1634883 to 2205327 tid 2 2024-09-18T14:17:38.397 INFO:tasks.rados.rados.0.smithi084.stdout:3808: writing smithi08434663-39 from 2669481 to 3376143 tid 3 2024-09-18T14:17:38.397 INFO:tasks.rados.rados.0.smithi084.stdout:3809: write oid 5 current snap is 426 2024-09-18T14:17:38.397 INFO:tasks.rados.rados.0.smithi084.stdout:3808: finishing write tid 1 to smithi08434663-39 2024-09-18T14:17:38.397 INFO:tasks.rados.rados.0.smithi084.stdout:3809: seq_num 1275 ranges {628925=573426,1693365=714102,3096904=332021} 2024-09-18T14:17:38.403 INFO:tasks.rados.rados.0.smithi084.stdout:3809: writing smithi08434663-5 from 628925 to 1202351 tid 1 2024-09-18T14:17:38.410 INFO:tasks.rados.rados.0.smithi084.stdout:3809: writing smithi08434663-5 from 1693365 to 2407467 tid 2 2024-09-18T14:17:38.416 INFO:tasks.rados.rados.0.smithi084.stdout:3809: writing smithi08434663-5 from 3096904 to 3428925 tid 3 2024-09-18T14:17:38.416 INFO:tasks.rados.rados.0.smithi084.stdout:3810: snap_remove snap 397 2024-09-18T14:17:38.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.436+0000 7f6fc2d5d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:38.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.436+0000 7f6fc2d5d640 10 monclient: build_initial_monmap 2024-09-18T14:17:38.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.436+0000 7f6fc2d5d640 10 monclient: monmap: 2024-09-18T14:17:38.438 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:38.438 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.438 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:38.438572+0000 2024-09-18T14:17:38.438 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:38.438572+0000 2024-09-18T14:17:38.438 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:38.438 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.438 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:38.438 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:38.439 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:38.439 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:38.439 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:38.439 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:38.439 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:38.439 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:38.439 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:38.439 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 1 Processor -- start 2024-09-18T14:17:38.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 1 -- start start 2024-09-18T14:17:38.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 10 monclient: init 2024-09-18T14:17:38.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:38.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 10 monclient: _add_conns ranks=[1,3,8,0,2,4,7,5,6] 2024-09-18T14:17:38.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 10 monclient(hunting): picked mon.noname-f con 0x7f6fbc0a2610 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:38.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 10 monclient(hunting): picked mon.noname-g con 0x7f6fbc1330b0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:38.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 10 monclient(hunting): picked mon.noname-e con 0x7f6fbc096a20 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc05ed70 con 0x7f6fbc0a2610 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc0a1740 con 0x7f6fbc1330b0 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc0a29e0 con 0x7f6fbc096a20 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc2d5d640 10 monclient(hunting): authenticate will time out at 1873.776942s 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.437+0000 7f6fc12fa640 1 -- >> v1:172.21.15.45:6793/0 conn(0x7f6fbc096a20 legacy=0x7f6fbc13af40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb3fff640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7f6fbc1330b0 0x7f6fbc137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:55916/0 (socket says 172.21.15.45:55916) 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb3fff640 1 -- 172.21.15.45:0/1268273364 learned_addr learned my addr 172.21.15.45:0/1268273364 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 1 -- 172.21.15.45:0/1268273364 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377875441 0 0) 0x7f6fbc0a1740 con 0x7f6fbc1330b0 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient(hunting): my global_id is 224238 2024-09-18T14:17:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 1 -- 172.21.15.45:0/1268273364 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa40034e0 con 0x7f6fbc1330b0 2024-09-18T14:17:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 1 -- 172.21.15.45:0/1268273364 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376105525 0 0) 0x7f6fbc05ed70 con 0x7f6fbc0a2610 2024-09-18T14:17:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient(hunting): my global_id is 214534 2024-09-18T14:17:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 1 -- 172.21.15.45:0/1268273364 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fbc0a1740 con 0x7f6fbc0a2610 2024-09-18T14:17:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 1 -- 172.21.15.45:0/1268273364 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2217333138 0 0) 0x7f6fa40034e0 con 0x7f6fbc1330b0 2024-09-18T14:17:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 1 -- 172.21.15.45:0/1268273364 >> v1:172.21.15.45:6793/0 conn(0x7f6fbc096a20 legacy=0x7f6fbc13af40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:17:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 1 -- 172.21.15.45:0/1268273364 >> v1:172.21.15.84:6789/0 conn(0x7f6fbc0a2610 legacy=0x7f6fbc1329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient: _finish_hunting 0 2024-09-18T14:17:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 1 monclient: found mon.noname-g 2024-09-18T14:17:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 1 -- 172.21.15.45:0/1268273364 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0970a0 con 0x7f6fbc1330b0 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient: _finish_auth 0 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 1 -- 172.21.15.45:0/1268273364 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fac001830 con 0x7f6fbc1330b0 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fc2d5d640 5 monclient: authenticate success, global_id 224238 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fc2d5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient: dump: 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fc2d5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 1 -- 172.21.15.45:0/1268273364 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6fac001990 con 0x7f6fbc1330b0 2024-09-18T14:17:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 1 -- 172.21.15.45:0/1268273364 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fac004ba0 con 0x7f6fbc1330b0 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fb37fe640 10 monclient: dump: 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fc2d5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.438+0000 7f6fc2d5d640 4 set_mon_vals no callback set 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.439+0000 7f6fc2d5d640 10 monclient: shutdown 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.439+0000 7f6fc2d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.439+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/1268273364 >> v1:172.21.15.84:6790/0 conn(0x7f6fbc1330b0 legacy=0x7f6fbc137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.439+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/1268273364 shutdown_connections 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.439+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/1268273364 >> 172.21.15.45:0/1268273364 conn(0x7f6fbc0630f0 msgr2=0x7f6fbc0634f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.439+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/1268273364 shutdown_connections 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.439+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/1268273364 wait complete. 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.439+0000 7f6fc2d5d640 10 monclient: build_initial_monmap 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.439+0000 7f6fc2d5d640 10 monclient: monmap: 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.440+0000 7f6fc2d5d640 1 Processor -- start 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.440+0000 7f6fc2d5d640 1 -- start start 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.440+0000 7f6fc2d5d640 10 monclient: init 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.440+0000 7f6fc2d5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.440+0000 7f6fc2d5d640 10 monclient: _add_conns ranks=[7,6,8,0,3,1,4,2,5] 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.440+0000 7f6fc2d5d640 10 monclient(hunting): picked mon.noname-i con 0x7f6fbc096a20 addr v1:172.21.15.84:6792/0 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.440+0000 7f6fc2d5d640 10 monclient(hunting): picked mon.noname-d con 0x7f6fbc0a2610 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 10 monclient(hunting): picked mon.noname-e con 0x7f6fbc1330b0 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc0a2350 con 0x7f6fbc0a2610 2024-09-18T14:17:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc09e240 con 0x7f6fbc096a20 2024-09-18T14:17:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc12fa640 1 -- >> v1:172.21.15.45:6793/0 conn(0x7f6fbc1330b0 legacy=0x7f6fbc1498b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc09df60 con 0x7f6fbc1330b0 2024-09-18T14:17:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 10 monclient(hunting): authenticate will time out at 1873.780082s 2024-09-18T14:17:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb3fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f6fbc0a2610 0x7f6fbc09b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:35340/0 (socket says 172.21.15.45:35340) 2024-09-18T14:17:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc0af9640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f6fbc096a20 0x7f6fbc09a940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:47542/0 (socket says 172.21.15.45:47542) 2024-09-18T14:17:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb3fff640 1 -- 172.21.15.45:0/2749950379 learned_addr learned my addr 172.21.15.45:0/2749950379 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777596724 0 0) 0x7f6fbc0a2350 con 0x7f6fbc0a2610 2024-09-18T14:17:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): my global_id is 185109 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa80034e0 con 0x7f6fbc0a2610 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342383645 0 0) 0x7f6fbc09e240 con 0x7f6fbc096a20 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): my global_id is 214540 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fbc0a2350 con 0x7f6fbc096a20 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3300747321 0 0) 0x7f6fa80034e0 con 0x7f6fbc0a2610 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fbc09e240 con 0x7f6fbc0a2610 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fac0047a0 con 0x7f6fbc0a2610 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): dump: 2024-09-18T14:17:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.459 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.459 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1969689841 0 0) 0x7f6fbc0a2350 con 0x7f6fbc096a20 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fa80034e0 con 0x7f6fbc096a20 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fb8001a50 con 0x7f6fbc096a20 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient(hunting): dump: 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.462 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1264266096 0 0) 0x7f6fbc09e240 con 0x7f6fbc0a2610 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 >> v1:172.21.15.45:6793/0 conn(0x7f6fbc1330b0 legacy=0x7f6fbc1498b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 >> v1:172.21.15.84:6792/0 conn(0x7f6fbc096a20 legacy=0x7f6fbc09a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 monclient: found mon.d 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc09cd20 con 0x7f6fbc0a2610 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 10 monclient: _finish_auth 0 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 5 monclient: authenticate success, global_id 185109 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 10 monclient: _renew_subs 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc09dc80 con 0x7f6fbc0a2610 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 10 monclient: _renew_subs 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.441+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc09f9d0 con 0x7f6fbc0a2610 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.442+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6fac004cb0 con 0x7f6fbc0a2610 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.442+0000 7f6fb0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.442+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fac0051a0 con 0x7f6fbc0a2610 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.442+0000 7f6fb0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.442+0000 7f6fb0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.442+0000 7f6fb0ff9640 10 monclient: dump: 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.442+0000 7f6fb2ffd640 4 set_mon_vals no callback set 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.442+0000 7f6fc2d5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.442+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6fbc04c4a0 con 0x7f6fbc09b760 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.442+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f6fac0063d0 con 0x7f6fbc0a2610 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(852..852 src has 637..852) v4 ==== 3943+0+0 (unknown 2896255082 0 0) 0x7f6fac046d60 con 0x7f6fbc0a2610 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909315269 0 0) 0x7f6fbc04c4a0 con 0x7f6fbc09b760 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 10 monclient(hunting): my global_id is 6561 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fbc09e240 con 0x7f6fbc09b760 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fc12fa640 10 monclient: get_auth_request con 0x7f6fa803a3d0 auth_method 0 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3964128216 0 0) 0x7f6fbc09e240 con 0x7f6fbc09b760 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fbc04c4a0 con 0x7f6fbc09b760 2024-09-18T14:17:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fb8002160 con 0x7f6fbc09b760 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 10 monclient: dump: 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.468 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 190199367 0 0) 0x7f6fbc04c4a0 con 0x7f6fbc09b760 2024-09-18T14:17:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6fbc09bb30 con 0x7f6fbc09b760 2024-09-18T14:17:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6fb80025e0 con 0x7f6fbc09b760 2024-09-18T14:17:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 >> v1:172.21.15.45:6789/0 conn(0x7f6fbc09b760 legacy=0x7f6fbc097e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.443+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 >> v1:172.21.15.45:6789/0 conn(0x7f6fbc09b760 legacy=0x7f6fbc097e40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.449+0000 7f6fc2d5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.449+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6fbc04b720 con 0x7f6fbc02a8d0 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403084022 0 0) 0x7f6fbc04b720 con 0x7f6fbc02a8d0 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 10 monclient(hunting): my global_id is 6570 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fbc04c4a0 con 0x7f6fbc02a8d0 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2789178776 0 0) 0x7f6fbc04c4a0 con 0x7f6fbc02a8d0 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fbc04b720 con 0x7f6fbc02a8d0 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6fb80055f0 con 0x7f6fbc02a8d0 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 10 monclient: dump: 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1715084184 0 0) 0x7f6fbc04b720 con 0x7f6fbc02a8d0 2024-09-18T14:17:38.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6fbc12c670 con 0x7f6fbc02a8d0 2024-09-18T14:17:38.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7367 (unknown 0 0 2998179454) 0x7f6fb8007a50 con 0x7f6fbc02a8d0 2024-09-18T14:17:38.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:38.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:38.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 >> v1:172.21.15.45:6789/0 conn(0x7f6fbc02a8d0 legacy=0x7f6fbc098550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fb0ff9640 1 -- 172.21.15.45:0/2749950379 >> v1:172.21.15.45:6789/0 conn(0x7f6fbc02a8d0 legacy=0x7f6fbc098550 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:38.472 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:38.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:38.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 152, 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: 7 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:38.473 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:38.475 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:38.476 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.477 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:38.478 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:38.479 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.479 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.479 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.479 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:38.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:38.479 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.479 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.479 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:38.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.481 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.484 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.485 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:38.487 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.488 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.489 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:17:38.490 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.491 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.491 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:38.491 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:38.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/2749950379 >> v1:172.21.15.84:6812/31478 conn(0x7f6fa803a3d0 legacy=0x7f6fa803c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fc2d5d640 10 monclient: shutdown 2024-09-18T14:17:38.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fc2d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:38.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.450+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/2749950379 >> v1:172.21.15.45:6792/0 conn(0x7f6fbc0a2610 legacy=0x7f6fbc09b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.451+0000 7f6fc12fa640 1 -- 172.21.15.45:0/2749950379 reap_dead start 2024-09-18T14:17:38.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.451+0000 7f6fc12fa640 1 -- 172.21.15.45:0/2749950379 reap_dead start 2024-09-18T14:17:38.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.451+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/2749950379 shutdown_connections 2024-09-18T14:17:38.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.451+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/2749950379 >> 172.21.15.45:0/2749950379 conn(0x7f6fbc0630f0 msgr2=0x7f6fbc133c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:38.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.451+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/2749950379 shutdown_connections 2024-09-18T14:17:38.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.451+0000 7f6fc2d5d640 1 -- 172.21.15.45:0/2749950379 wait complete. 2024-09-18T14:17:38.493 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:17:38.493 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-09-18T14:17:38.493 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:17:38.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.589+0000 7fdf0135d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:38.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.589+0000 7fdf0135d640 10 monclient: build_initial_monmap 2024-09-18T14:17:38.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.589+0000 7fdf0135d640 10 monclient: monmap: 2024-09-18T14:17:38.590 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:38.590 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:38.590971+0000 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:38.590971+0000 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:38.591 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.589+0000 7fdf0135d640 1 Processor -- start 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.589+0000 7fdf0135d640 1 -- start start 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.589+0000 7fdf0135d640 10 monclient: init 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 10 monclient: _add_conns ranks=[2,1,8,4,3,5,0,7,6] 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 10 monclient(hunting): picked mon.noname-b con 0x7fdefc09f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 10 monclient(hunting): picked mon.noname-f con 0x7fdefc09fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 10 monclient(hunting): picked mon.noname-e con 0x7fdefc09ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc05ed70 con 0x7fdefc09fba0 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc09dfc0 con 0x7fdefc09f7d0 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc09dce0 con 0x7fdefc09ff70 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 10 monclient(hunting): authenticate will time out at 1873.929152s 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdefaffd640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7fdefc09f7d0 legacy=0x7fdefc132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:17:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdefb7fe640 1 -- >> v1:172.21.15.45:6793/0 conn(0x7fdefc09ff70 legacy=0x7fdefc13b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdefa7fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdefc09fba0 0x7fdefc137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:48610/0 (socket says 172.21.15.45:48610) 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdefa7fc640 1 -- 172.21.15.45:0/3780844446 learned_addr learned my addr 172.21.15.45:0/3780844446 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 1 -- 172.21.15.45:0/3780844446 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245313634 0 0) 0x7fdefc05ed70 con 0x7fdefc09fba0 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient(hunting): my global_id is 214543 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 1 -- 172.21.15.45:0/3780844446 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdee40034e0 con 0x7fdefc09fba0 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 1 -- 172.21.15.45:0/3780844446 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2703576456 0 0) 0x7fdee40034e0 con 0x7fdefc09fba0 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 1 -- 172.21.15.45:0/3780844446 >> v1:172.21.15.45:6793/0 conn(0x7fdefc09ff70 legacy=0x7fdefc13b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 1 -- 172.21.15.45:0/3780844446 >> v1:172.21.15.45:6790/0 conn(0x7fdefc09f7d0 legacy=0x7fdefc132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 1 monclient: found mon.noname-f 2024-09-18T14:17:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 1 -- 172.21.15.45:0/3780844446 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc0a1c00 con 0x7fdefc09fba0 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient: _finish_auth 0 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 1 -- 172.21.15.45:0/3780844446 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdeec001830 con 0x7fdefc09fba0 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 5 monclient: authenticate success, global_id 214543 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient: dump: 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 1 -- 172.21.15.45:0/3780844446 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdeec001a10 con 0x7fdefc09fba0 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 1 -- 172.21.15.45:0/3780844446 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdeec004ba0 con 0x7fdefc09fba0 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdf0135d640 4 set_mon_vals no callback set 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.590+0000 7fdef9ffb640 10 monclient: dump: 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.591+0000 7fdf0135d640 10 monclient: shutdown 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.591+0000 7fdf0135d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.591+0000 7fdf0135d640 1 -- 172.21.15.45:0/3780844446 >> v1:172.21.15.84:6789/0 conn(0x7fdefc09fba0 legacy=0x7fdefc137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.591+0000 7fdf0135d640 1 -- 172.21.15.45:0/3780844446 shutdown_connections 2024-09-18T14:17:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.591+0000 7fdf0135d640 1 -- 172.21.15.45:0/3780844446 >> 172.21.15.45:0/3780844446 conn(0x7fdefc0630b0 msgr2=0x7fdefc0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.591+0000 7fdf0135d640 1 -- 172.21.15.45:0/3780844446 shutdown_connections 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.591+0000 7fdf0135d640 1 -- 172.21.15.45:0/3780844446 wait complete. 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 10 monclient: build_initial_monmap 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 10 monclient: monmap: 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:38.605 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 1 Processor -- start 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 1 -- start start 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 10 monclient: init 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 10 monclient: _add_conns ranks=[5,8,1,2,6,7,3,4,0] 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 10 monclient(hunting): picked mon.noname-h con 0x7fdefc09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 10 monclient(hunting): picked mon.noname-e con 0x7fdefc09fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 10 monclient(hunting): picked mon.noname-f con 0x7fdefc09ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc09d310 con 0x7fdefc09ff70 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc0a2560 con 0x7fdefc09f7d0 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc099000 con 0x7fdefc09fba0 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdf0135d640 10 monclient(hunting): authenticate will time out at 1873.931921s 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.592+0000 7fdefa7fc640 1 -- >> v1:172.21.15.45:6793/0 conn(0x7fdefc09fba0 legacy=0x7fdefc0988f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdefb7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdefc09ff70 0x7fdefc149bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:48620/0 (socket says 172.21.15.45:48620) 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdefaffd640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7fdefc09f7d0 0x7fdefc0981e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:44096/0 (socket says 172.21.15.45:44096) 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdefb7fe640 1 -- 172.21.15.45:0/1025366243 learned_addr learned my addr 172.21.15.45:0/1025366243 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854631160 0 0) 0x7fdefc09d310 con 0x7fdefc09ff70 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): my global_id is 214552 2024-09-18T14:17:38.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdee80034e0 con 0x7fdefc09ff70 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802785395 0 0) 0x7fdefc0a2560 con 0x7fdefc09f7d0 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): my global_id is 184964 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdefc09d310 con 0x7fdefc09f7d0 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1982723828 0 0) 0x7fdee80034e0 con 0x7fdefc09ff70 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdefc0a2560 con 0x7fdefc09ff70 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdef40045a0 con 0x7fdefc09ff70 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): dump: 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.608 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 800383737 0 0) 0x7fdefc09d310 con 0x7fdefc09f7d0 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdee80034e0 con 0x7fdefc09f7d0 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdef00041f0 con 0x7fdefc09f7d0 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient(hunting): dump: 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3011600990 0 0) 0x7fdefc0a2560 con 0x7fdefc09ff70 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 >> v1:172.21.15.45:6793/0 conn(0x7fdefc09fba0 legacy=0x7fdefc0988f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 >> v1:172.21.15.84:6791/0 conn(0x7fdefc09f7d0 legacy=0x7fdefc0981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 monclient: found mon.f 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc09b7a0 con 0x7fdefc09ff70 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient: _finish_auth 0 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdf0135d640 5 monclient: authenticate success, global_id 214552 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdf0135d640 10 monclient: _renew_subs 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdf0135d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdf0135d640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc099ce0 con 0x7fdefc09ff70 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdf0135d640 10 monclient: _renew_subs 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdf0135d640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdf0135d640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc12d240 con 0x7fdefc09ff70 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdef4004780 con 0x7fdefc09ff70 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdef4004cb0 con 0x7fdefc09ff70 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdef97fa640 4 set_mon_vals no callback set 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.593+0000 7fdee2ffd640 10 monclient: dump: 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.613 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.594+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fdef4004e90 con 0x7fdefc09ff70 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.594+0000 7fdf0135d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.594+0000 7fdf0135d640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdedc006580 con 0x7fdedc002830 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.594+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(852..852 src has 637..852) v4 ==== 3943+0+0 (unknown 2896255082 0 0) 0x7fdef4046ec0 con 0x7fdefc09ff70 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.594+0000 7fdefa7fc640 10 monclient: get_auth_request con 0x7fdee803a3a0 auth_method 0 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863268276 0 0) 0x7fdedc006580 con 0x7fdedc002830 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 10 monclient(hunting): my global_id is 165289 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdefc0a2560 con 0x7fdedc002830 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 431448178 0 0) 0x7fdefc0a2560 con 0x7fdedc002830 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdedc006580 con 0x7fdedc002830 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdef0004cb0 con 0x7fdedc002830 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 10 monclient: dump: 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1926515995 0 0) 0x7fdedc006580 con 0x7fdedc002830 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdedc00a0a0 con 0x7fdedc002830 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fdef0005260 con 0x7fdedc002830 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 >> v1:172.21.15.45:6791/0 conn(0x7fdedc002830 legacy=0x7fdedc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.595+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 >> v1:172.21.15.45:6791/0 conn(0x7fdedc002830 legacy=0x7fdedc004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.601+0000 7fdf0135d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.601+0000 7fdf0135d640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdedc007540 con 0x7fdedc0068f0 2024-09-18T14:17:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.601+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420410811 0 0) 0x7fdedc007540 con 0x7fdedc0068f0 2024-09-18T14:17:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.601+0000 7fdee2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.601+0000 7fdee2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.601+0000 7fdee2ffd640 10 monclient(hunting): my global_id is 165298 2024-09-18T14:17:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.601+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdedc006580 con 0x7fdedc0068f0 2024-09-18T14:17:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.601+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4122318153 0 0) 0x7fdedc006580 con 0x7fdedc0068f0 2024-09-18T14:17:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdedc007540 con 0x7fdedc0068f0 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdef0005a40 con 0x7fdedc0068f0 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdee2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdee2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdee2ffd640 10 monclient: dump: 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3251304191 0 0) 0x7fdedc007540 con 0x7fdedc0068f0 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdedc0087a0 con 0x7fdedc0068f0 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 717963993) 0x7fdef0007d80 con 0x7fdedc0068f0 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdee2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdee2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 >> v1:172.21.15.45:6791/0 conn(0x7fdedc0068f0 legacy=0x7fdedc006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdee2ffd640 1 -- 172.21.15.45:0/1025366243 >> v1:172.21.15.45:6791/0 conn(0x7fdedc0068f0 legacy=0x7fdedc006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:38.621 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:38.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:38.621 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:38.621 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:38.621 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 152, 2024-09-18T14:17:38.621 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:38.621 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:38.621 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:38.621 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:38.621 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: 7 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:38.622 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:38.623 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:38.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:38.625 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.627 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:38.628 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.629 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:38.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.632 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.634 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:38.635 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.636 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:38.637 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.638 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4 2024-09-18T14:17:38.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.640 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.640 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:38.640 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:38.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdf0135d640 1 -- 172.21.15.45:0/1025366243 >> v1:172.21.15.84:6812/31478 conn(0x7fdee803a3a0 legacy=0x7fdee803c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdf0135d640 10 monclient: shutdown 2024-09-18T14:17:38.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdf0135d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:38.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdf0135d640 1 -- 172.21.15.45:0/1025366243 >> v1:172.21.15.84:6789/0 conn(0x7fdefc09ff70 legacy=0x7fdefc149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdefb7fe640 1 -- 172.21.15.45:0/1025366243 reap_dead start 2024-09-18T14:17:38.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdefb7fe640 1 -- 172.21.15.45:0/1025366243 reap_dead start 2024-09-18T14:17:38.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdf0135d640 1 -- 172.21.15.45:0/1025366243 shutdown_connections 2024-09-18T14:17:38.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdf0135d640 1 -- 172.21.15.45:0/1025366243 >> 172.21.15.45:0/1025366243 conn(0x7fdefc0630b0 msgr2=0x7fdefc130c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:38.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdf0135d640 1 -- 172.21.15.45:0/1025366243 shutdown_connections 2024-09-18T14:17:38.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.602+0000 7fdf0135d640 1 -- 172.21.15.45:0/1025366243 wait complete. 2024-09-18T14:17:38.642 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:17:38.642 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-18T14:17:38.642 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:17:38.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:38.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 10 monclient: build_initial_monmap 2024-09-18T14:17:38.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 10 monclient: monmap: 2024-09-18T14:17:38.739 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:38.739 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:38.739810+0000 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:38.739810+0000 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:38.740 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 1 Processor -- start 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 1 -- start start 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 10 monclient: init 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 10 monclient: _add_conns ranks=[3,6,0,8,2,7,5,4,1] 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 10 monclient(hunting): picked mon.noname-g con 0x7f920409f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 10 monclient(hunting): picked mon.noname-d con 0x7f920409fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 10 monclient(hunting): picked mon.noname-a con 0x7f920409ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920405ed70 con 0x7f920409ff70 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920409dfc0 con 0x7f920409f7d0 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920409dce0 con 0x7f920409fba0 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.738+0000 7f9208b1d640 10 monclient(hunting): authenticate will time out at 1874.077990s 2024-09-18T14:17:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f91f9d9b640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f920409fba0 0x7f9204137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:35342/0 (socket says 172.21.15.45:35342) 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9202d9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f920409ff70 0x7f920413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43090/0 (socket says 172.21.15.45:43090) 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f91f9d9b640 1 -- 172.21.15.45:0/410597768 learned_addr learned my addr 172.21.15.45:0/410597768 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030594710 0 0) 0x7f920405ed70 con 0x7f920409ff70 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient(hunting): my global_id is 6579 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f00034e0 con 0x7f920409ff70 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203927871 0 0) 0x7f920409dce0 con 0x7f920409fba0 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient(hunting): my global_id is 185118 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f920405ed70 con 0x7f920409fba0 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598121718 0 0) 0x7f920409dfc0 con 0x7f920409f7d0 2024-09-18T14:17:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient(hunting): my global_id is 224247 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f920409dce0 con 0x7f920409f7d0 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 688551915 0 0) 0x7f91f00034e0 con 0x7f920409ff70 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 >> v1:172.21.15.45:6792/0 conn(0x7f920409fba0 legacy=0x7f9204137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 >> v1:172.21.15.84:6790/0 conn(0x7f920409f7d0 legacy=0x7f9204132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient: _finish_hunting 0 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 monclient: found mon.noname-a 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92040a1c00 con 0x7f920409ff70 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient: _finish_auth 0 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91fc001ae0 con 0x7f920409ff70 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9208b1d640 5 monclient: authenticate success, global_id 6579 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9208b1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient: dump: 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.752 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9208b1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f91fc001c40 con 0x7f920409ff70 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 1 -- 172.21.15.45:0/410597768 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91fc004b20 con 0x7f920409ff70 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:38.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9201d9b640 10 monclient: dump: 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.754 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9208b1d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.739+0000 7f9208b1d640 4 set_mon_vals no callback set 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.740+0000 7f9208b1d640 10 monclient: shutdown 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.740+0000 7f9208b1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.740+0000 7f9208b1d640 1 -- 172.21.15.45:0/410597768 >> v1:172.21.15.45:6789/0 conn(0x7f920409ff70 legacy=0x7f920413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.740+0000 7f9208b1d640 1 -- 172.21.15.45:0/410597768 shutdown_connections 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.740+0000 7f9208b1d640 1 -- 172.21.15.45:0/410597768 >> 172.21.15.45:0/410597768 conn(0x7f92040630b0 msgr2=0x7f92040634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.740+0000 7f9208b1d640 1 -- 172.21.15.45:0/410597768 shutdown_connections 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.740+0000 7f9208b1d640 1 -- 172.21.15.45:0/410597768 wait complete. 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.741+0000 7f9208b1d640 10 monclient: build_initial_monmap 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.741+0000 7f9208b1d640 10 monclient: monmap: 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:38.756 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.741+0000 7f9208b1d640 1 Processor -- start 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.741+0000 7f9208b1d640 1 -- start start 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.741+0000 7f9208b1d640 10 monclient: init 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.741+0000 7f9208b1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.741+0000 7f9208b1d640 10 monclient: _add_conns ranks=[2,4,5,7,0,3,6,1,8] 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.741+0000 7f9208b1d640 10 monclient(hunting): picked mon.noname-b con 0x7f920409f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.741+0000 7f9208b1d640 10 monclient(hunting): picked mon.noname-c con 0x7f920409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.741+0000 7f9208b1d640 10 monclient(hunting): picked mon.noname-h con 0x7f920409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.741+0000 7f920259c640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7f920409f7d0 legacy=0x7f92041317f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.741+0000 7f9208b1d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920411ebf0 con 0x7f920409f7d0 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f9208b1d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920411d5f0 con 0x7f920409fba0 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f9208b1d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920409bbf0 con 0x7f920409ff70 2024-09-18T14:17:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f9208b1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f9208b1d640 10 monclient(hunting): authenticate will time out at 1874.081065s 2024-09-18T14:17:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91f9d9b640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f920409fba0 0x7f9204146e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58568/0 (socket says 172.21.15.45:58568) 2024-09-18T14:17:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91f9d9b640 1 -- 172.21.15.45:0/8965516 learned_addr learned my addr 172.21.15.45:0/8965516 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417790094 0 0) 0x7f920411d5f0 con 0x7f920409fba0 2024-09-18T14:17:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): my global_id is 165307 2024-09-18T14:17:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e40034e0 con 0x7f920409fba0 2024-09-18T14:17:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195525849 0 0) 0x7f920409bbf0 con 0x7f920409ff70 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): my global_id is 184973 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f920411d5f0 con 0x7f920409ff70 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2164405714 0 0) 0x7f91e40034e0 con 0x7f920409fba0 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f920409bbf0 con 0x7f920409fba0 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91ec001ba0 con 0x7f920409fba0 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): dump: 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 385546512 0 0) 0x7f920411d5f0 con 0x7f920409ff70 2024-09-18T14:17:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91e40034e0 con 0x7f920409ff70 2024-09-18T14:17:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91fc001a20 con 0x7f920409ff70 2024-09-18T14:17:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient(hunting): dump: 2024-09-18T14:17:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.762 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2067102563 0 0) 0x7f920409bbf0 con 0x7f920409fba0 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 >> v1:172.21.15.84:6791/0 conn(0x7f920409ff70 legacy=0x7f920414a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 >> v1:172.21.15.45:6790/0 conn(0x7f920409f7d0 legacy=0x7f92041317f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient: _finish_hunting 0 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 monclient: found mon.c 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920411e910 con 0x7f920409fba0 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient: _finish_auth 0 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f9208b1d640 5 monclient: authenticate success, global_id 165307 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f9208b1d640 10 monclient: _renew_subs 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f9208b1d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:17:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f9208b1d640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9204131f00 con 0x7f920409fba0 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f9208b1d640 10 monclient: _renew_subs 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f9208b1d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f9208b1d640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f920411cb00 con 0x7f920409fba0 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f91ec004a50 con 0x7f920409fba0 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91ec004f00 con 0x7f920409fba0 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f920159a640 4 set_mon_vals no callback set 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.742+0000 7f91faffd640 10 monclient: dump: 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.743+0000 7f9208b1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.743+0000 7f9208b1d640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f91e8006580 con 0x7f91e8002830 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.743+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f91ec00fed0 con 0x7f920409fba0 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(852..852 src has 637..852) v4 ==== 3943+0+0 (unknown 2896255082 0 0) 0x7f91ec046c30 con 0x7f920409fba0 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705321651 0 0) 0x7f91e8006580 con 0x7f91e8002830 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 10 monclient(hunting): my global_id is 185127 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f920409bbf0 con 0x7f91e8002830 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f9202d9d640 10 monclient: get_auth_request con 0x7f91e403a360 auth_method 0 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 588485327 0 0) 0x7f920409bbf0 con 0x7f91e8002830 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91e8006580 con 0x7f91e8002830 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91f4002bf0 con 0x7f91e8002830 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 10 monclient: dump: 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.768 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2944146775 0 0) 0x7f91e8006580 con 0x7f91e8002830 2024-09-18T14:17:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91e800a0a0 con 0x7f91e8002830 2024-09-18T14:17:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f91f4006470 con 0x7f91e8002830 2024-09-18T14:17:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 >> v1:172.21.15.45:6792/0 conn(0x7f91e8002830 legacy=0x7f91e8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.744+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 >> v1:172.21.15.45:6792/0 conn(0x7f91e8002830 legacy=0x7f91e8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f9208b1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f9208b1d640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f91e8009660 con 0x7f91e8009040 2024-09-18T14:17:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724190656 0 0) 0x7f91e8009660 con 0x7f91e8009040 2024-09-18T14:17:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 10 monclient(hunting): my global_id is 185136 2024-09-18T14:17:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e8006580 con 0x7f91e8009040 2024-09-18T14:17:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2160894296 0 0) 0x7f91e8006580 con 0x7f91e8009040 2024-09-18T14:17:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91e8009660 con 0x7f91e8009040 2024-09-18T14:17:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f91f4004d10 con 0x7f91e8009040 2024-09-18T14:17:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 10 monclient: dump: 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.772 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.772 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.772 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.772 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 469376279 0 0) 0x7f91e8009660 con 0x7f91e8009040 2024-09-18T14:17:38.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f91e8009920 con 0x7f91e8009040 2024-09-18T14:17:38.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 489093520) 0x7f91f4006780 con 0x7f91e8009040 2024-09-18T14:17:38.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:38.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:38.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 >> v1:172.21.15.45:6792/0 conn(0x7f91e8009040 legacy=0x7f91e8008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f91faffd640 1 -- 172.21.15.45:0/8965516 >> v1:172.21.15.45:6792/0 conn(0x7f91e8009040 legacy=0x7f91e8008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 152, 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:38.773 INFO:teuthology.orchestra.run.smithi045.stdout: 7 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:38.774 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:38.775 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:38.776 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:38.777 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.778 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.780 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.782 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.786 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.787 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:38.790 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.791 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.791 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:38.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.791 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:38.791 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:38.791 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:38.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.791 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.791 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:38.791 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:38.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f9208b1d640 1 -- 172.21.15.45:0/8965516 >> v1:172.21.15.84:6812/31478 conn(0x7f91e403a360 legacy=0x7f91e403c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f9208b1d640 10 monclient: shutdown 2024-09-18T14:17:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f9208b1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f9208b1d640 1 -- 172.21.15.45:0/8965516 >> v1:172.21.15.45:6791/0 conn(0x7f920409fba0 legacy=0x7f9204146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f9202d9d640 1 -- 172.21.15.45:0/8965516 reap_dead start 2024-09-18T14:17:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f9202d9d640 1 -- 172.21.15.45:0/8965516 reap_dead start 2024-09-18T14:17:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f9208b1d640 1 -- 172.21.15.45:0/8965516 shutdown_connections 2024-09-18T14:17:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f9208b1d640 1 -- 172.21.15.45:0/8965516 >> 172.21.15.45:0/8965516 conn(0x7f92040630b0 msgr2=0x7f92040a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f9208b1d640 1 -- 172.21.15.45:0/8965516 shutdown_connections 2024-09-18T14:17:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.751+0000 7f9208b1d640 1 -- 172.21.15.45:0/8965516 wait complete. 2024-09-18T14:17:38.793 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:17:38.794 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-09-18T14:17:38.794 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:17:38.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 10 monclient: build_initial_monmap 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 10 monclient: monmap: 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:38.891304+0000 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:38.891304+0000 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:38.892 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:38.892 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:38.892 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:38.892 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:38.892 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 1 Processor -- start 2024-09-18T14:17:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 1 -- start start 2024-09-18T14:17:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 10 monclient: init 2024-09-18T14:17:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 10 monclient: _add_conns ranks=[4,0,1,3,2,8,5,6,7] 2024-09-18T14:17:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 10 monclient(hunting): picked mon.noname-c con 0x7ff1e009f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 10 monclient(hunting): picked mon.noname-a con 0x7ff1e009fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 10 monclient(hunting): picked mon.noname-f con 0x7ff1e009ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e005ed70 con 0x7ff1e009fba0 2024-09-18T14:17:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e009dfc0 con 0x7ff1e009ff70 2024-09-18T14:17:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e009dce0 con 0x7ff1e009f7d0 2024-09-18T14:17:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1e595d640 10 monclient(hunting): authenticate will time out at 1874.229698s 2024-09-18T14:17:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1de7fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff1e009fba0 0x7ff1e0137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43092/0 (socket says 172.21.15.45:43092) 2024-09-18T14:17:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1deffd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7ff1e009f7d0 0x7ff1e0132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:58580/0 (socket says 172.21.15.45:58580) 2024-09-18T14:17:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.890+0000 7ff1de7fc640 1 -- 172.21.15.45:0/2920436600 learned_addr learned my addr 172.21.15.45:0/2920436600 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288918310 0 0) 0x7ff1e005ed70 con 0x7ff1e009fba0 2024-09-18T14:17:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient(hunting): my global_id is 6588 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c80034e0 con 0x7ff1e009fba0 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349065974 0 0) 0x7ff1e009dce0 con 0x7ff1e009f7d0 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient(hunting): my global_id is 165316 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e005ed70 con 0x7ff1e009f7d0 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161108985 0 0) 0x7ff1e009dfc0 con 0x7ff1e009ff70 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient(hunting): my global_id is 214561 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e009dce0 con 0x7ff1e009ff70 2024-09-18T14:17:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3227396719 0 0) 0x7ff1c80034e0 con 0x7ff1e009fba0 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 >> v1:172.21.15.45:6791/0 conn(0x7ff1e009f7d0 legacy=0x7ff1e0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 >> v1:172.21.15.84:6789/0 conn(0x7ff1e009ff70 legacy=0x7ff1e013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient: _finish_hunting 0 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 monclient: found mon.noname-a 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00a1c00 con 0x7ff1e009fba0 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient: _finish_auth 0 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1d4001830 con 0x7ff1e009fba0 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1e595d640 5 monclient: authenticate success, global_id 6588 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1e595d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient: dump: 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1e595d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff1d4001990 con 0x7ff1e009fba0 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1e595d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 1 -- 172.21.15.45:0/2920436600 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1d4004ba0 con 0x7ff1e009fba0 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1e595d640 4 set_mon_vals no callback set 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1ddffb640 10 monclient: dump: 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1e595d640 10 monclient: shutdown 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1e595d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.891+0000 7ff1e595d640 1 -- 172.21.15.45:0/2920436600 >> v1:172.21.15.45:6789/0 conn(0x7ff1e009fba0 legacy=0x7ff1e0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.892+0000 7ff1e595d640 1 -- 172.21.15.45:0/2920436600 shutdown_connections 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.892+0000 7ff1e595d640 1 -- 172.21.15.45:0/2920436600 >> 172.21.15.45:0/2920436600 conn(0x7ff1e00630b0 msgr2=0x7ff1e00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.892+0000 7ff1e595d640 1 -- 172.21.15.45:0/2920436600 shutdown_connections 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.892+0000 7ff1e595d640 1 -- 172.21.15.45:0/2920436600 wait complete. 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.892+0000 7ff1e595d640 10 monclient: build_initial_monmap 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.892+0000 7ff1e595d640 10 monclient: monmap: 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.892+0000 7ff1e595d640 1 Processor -- start 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1e595d640 1 -- start start 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1e595d640 10 monclient: init 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1e595d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1e595d640 10 monclient: _add_conns ranks=[1,3,8,2,6,5,4,0,7] 2024-09-18T14:17:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1e595d640 10 monclient(hunting): picked mon.noname-f con 0x7ff1e009f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1e595d640 10 monclient(hunting): picked mon.noname-g con 0x7ff1e009fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1e595d640 10 monclient(hunting): picked mon.noname-e con 0x7ff1e009ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1e595d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e009bf70 con 0x7ff1e009f7d0 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1e595d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e009ba20 con 0x7ff1e009fba0 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1e595d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e0099e80 con 0x7ff1e009ff70 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1e595d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1df7fe640 1 -- >> v1:172.21.15.45:6793/0 conn(0x7ff1e009ff70 legacy=0x7ff1e0149bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1e595d640 10 monclient(hunting): authenticate will time out at 1874.232575s 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1de7fc640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7ff1e009fba0 0x7ff1e0098c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:55932/0 (socket says 172.21.15.45:55932) 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1deffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff1e009f7d0 0x7ff1e0098560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:48630/0 (socket says 172.21.15.45:48630) 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1de7fc640 1 -- 172.21.15.45:0/2485176277 learned_addr learned my addr 172.21.15.45:0/2485176277 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799994039 0 0) 0x7ff1e009ba20 con 0x7ff1e009fba0 2024-09-18T14:17:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1c6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1c6ffd640 10 monclient(hunting): my global_id is 224256 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1cc0034e0 con 0x7ff1e009fba0 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784267613 0 0) 0x7ff1e009bf70 con 0x7ff1e009f7d0 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1c6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1c6ffd640 10 monclient(hunting): my global_id is 214570 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.893+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e009ba20 con 0x7ff1e009f7d0 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3523268716 0 0) 0x7ff1cc0034e0 con 0x7ff1e009fba0 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1e009bf70 con 0x7ff1e009fba0 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1d4001830 con 0x7ff1e009fba0 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:17:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient(hunting): dump: 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 20849732 0 0) 0x7ff1e009ba20 con 0x7ff1e009f7d0 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1cc0034e0 con 0x7ff1e009f7d0 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1d0001990 con 0x7ff1e009f7d0 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient(hunting): dump: 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.911 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3339133598 0 0) 0x7ff1e009bf70 con 0x7ff1e009fba0 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 >> v1:172.21.15.45:6793/0 conn(0x7ff1e009ff70 legacy=0x7ff1e0149bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 >> v1:172.21.15.84:6789/0 conn(0x7ff1e009f7d0 legacy=0x7ff1e0098560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 monclient: found mon.g 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e009bc90 con 0x7ff1e009fba0 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient: _finish_auth 0 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1e595d640 5 monclient: authenticate success, global_id 224256 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1e595d640 10 monclient: _renew_subs 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1e595d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1e595d640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e009c250 con 0x7ff1e009fba0 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1e595d640 10 monclient: _renew_subs 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1e595d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1e595d640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e009c670 con 0x7ff1e009fba0 2024-09-18T14:17:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff1d4001230 con 0x7ff1e009fba0 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1d4005160 con 0x7ff1e009fba0 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1dd7fa640 4 set_mon_vals no callback set 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.894+0000 7ff1c6ffd640 10 monclient: dump: 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1e595d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1e595d640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff1c0006580 con 0x7ff1c0002830 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7ff1d4010130 con 0x7ff1e009fba0 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(852..852 src has 637..852) v4 ==== 3943+0+0 (unknown 2896255082 0 0) 0x7ff1d4046e20 con 0x7ff1e009fba0 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1df7fe640 10 monclient: get_auth_request con 0x7ff1cc03a370 auth_method 0 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611194712 0 0) 0x7ff1c0006580 con 0x7ff1c0002830 2024-09-18T14:17:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 10 monclient(hunting): my global_id is 214579 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e009bf70 con 0x7ff1c0002830 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 181129589 0 0) 0x7ff1e009bf70 con 0x7ff1c0002830 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1c0006580 con 0x7ff1c0002830 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1d0004ea0 con 0x7ff1c0002830 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.895+0000 7ff1c6ffd640 10 monclient: dump: 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.896+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2359263828 0 0) 0x7ff1c0006580 con 0x7ff1c0002830 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.896+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1c000a0a0 con 0x7ff1c0002830 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.896+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff1d0007160 con 0x7ff1c0002830 2024-09-18T14:17:38.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.896+0000 7ff1c6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:38.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.896+0000 7ff1c6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:38.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.896+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 >> v1:172.21.15.84:6789/0 conn(0x7ff1c0002830 legacy=0x7ff1c0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.896+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 >> v1:172.21.15.84:6789/0 conn(0x7ff1c0002830 legacy=0x7ff1c0004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:38.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1e595d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:38.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1e595d640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff1c0009610 con 0x7ff1c0008ff0 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417870289 0 0) 0x7ff1c0009610 con 0x7ff1c0008ff0 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 10 monclient(hunting): my global_id is 214588 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c0006580 con 0x7ff1c0008ff0 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 526669180 0 0) 0x7ff1c0006580 con 0x7ff1c0008ff0 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1c0009610 con 0x7ff1c0008ff0 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7ff1d0005920 con 0x7ff1c0008ff0 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 10 monclient: dump: 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3526935503 0 0) 0x7ff1c0009610 con 0x7ff1c0008ff0 2024-09-18T14:17:38.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.902+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff1c00098d0 con 0x7ff1c0008ff0 2024-09-18T14:17:38.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 3297983503) 0x7ff1d0006d50 con 0x7ff1c0008ff0 2024-09-18T14:17:38.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1c6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:38.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1c6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:38.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 >> v1:172.21.15.84:6789/0 conn(0x7ff1c0008ff0 legacy=0x7ff1c0008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1c6ffd640 1 -- 172.21.15.45:0/2485176277 >> v1:172.21.15.84:6789/0 conn(0x7ff1c0008ff0 legacy=0x7ff1c0008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:38.922 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:38.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:38.922 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:38.922 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 152, 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: 7 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:38.925 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:38.926 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:38.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.928 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.929 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:38.931 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.932 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.933 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.934 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:38.935 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.936 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.937 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:38.938 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:38.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.940 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:38.940 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:38.940 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:38.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.940 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.940 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:38.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.940 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:38.940 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:38.941 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:38.941 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.941 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:38.941 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:38.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:38.941 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:38.942 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:38.942 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:38.942 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:38.942 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:38.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:38.942 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:38.942 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1e595d640 1 -- 172.21.15.45:0/2485176277 >> v1:172.21.15.84:6812/31478 conn(0x7ff1cc03a370 legacy=0x7ff1cc03c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1e595d640 10 monclient: shutdown 2024-09-18T14:17:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1e595d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1e595d640 1 -- 172.21.15.45:0/2485176277 >> v1:172.21.15.84:6790/0 conn(0x7ff1e009fba0 legacy=0x7ff1e0098c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1df7fe640 1 -- 172.21.15.45:0/2485176277 reap_dead start 2024-09-18T14:17:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1df7fe640 1 -- 172.21.15.45:0/2485176277 reap_dead start 2024-09-18T14:17:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1e595d640 1 -- 172.21.15.45:0/2485176277 shutdown_connections 2024-09-18T14:17:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1e595d640 1 -- 172.21.15.45:0/2485176277 >> 172.21.15.45:0/2485176277 conn(0x7ff1e00630b0 msgr2=0x7ff1e0130ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1e595d640 1 -- 172.21.15.45:0/2485176277 shutdown_connections 2024-09-18T14:17:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:38.903+0000 7ff1e595d640 1 -- 172.21.15.45:0/2485176277 wait complete. 2024-09-18T14:17:38.944 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:17:38.945 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-09-18T14:17:38.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:17:39.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.040+0000 7fb603f9d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:39.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.040+0000 7fb603f9d640 10 monclient: build_initial_monmap 2024-09-18T14:17:39.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.040+0000 7fb603f9d640 10 monclient: monmap: 2024-09-18T14:17:39.042 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:39.042 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.042 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:39.042622+0000 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:39.042622+0000 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:39.043 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 1 Processor -- start 2024-09-18T14:17:39.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 1 -- start start 2024-09-18T14:17:39.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 10 monclient: init 2024-09-18T14:17:39.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:39.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 10 monclient: _add_conns ranks=[8,3,1,2,4,7,0,5,6] 2024-09-18T14:17:39.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 10 monclient(hunting): picked mon.noname-e con 0x7fb5fc09f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:39.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 10 monclient(hunting): picked mon.noname-g con 0x7fb5fc09fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:39.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 10 monclient(hunting): picked mon.noname-f con 0x7fb5fc09ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc05ed70 con 0x7fb5fc09ff70 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc09dfc0 con 0x7fb5fc09fba0 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc09dce0 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.041+0000 7fb603f9d640 10 monclient(hunting): authenticate will time out at 1874.380980s 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb601d39640 1 -- >> v1:172.21.15.45:6793/0 conn(0x7fb5fc09f7d0 legacy=0x7fb5fc132b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb601538640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7fb5fc09fba0 0x7fb5fc137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:55938/0 (socket says 172.21.15.45:55938) 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb601538640 1 -- 172.21.15.45:0/2606768925 learned_addr learned my addr 172.21.15.45:0/2606768925 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 1 -- 172.21.15.45:0/2606768925 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194418368 0 0) 0x7fb5fc09dfc0 con 0x7fb5fc09fba0 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient(hunting): my global_id is 224265 2024-09-18T14:17:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 1 -- 172.21.15.45:0/2606768925 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5e80034e0 con 0x7fb5fc09fba0 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 1 -- 172.21.15.45:0/2606768925 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119179752 0 0) 0x7fb5fc05ed70 con 0x7fb5fc09ff70 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient(hunting): my global_id is 214597 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 1 -- 172.21.15.45:0/2606768925 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5fc09dfc0 con 0x7fb5fc09ff70 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 1 -- 172.21.15.45:0/2606768925 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3022456648 0 0) 0x7fb5e80034e0 con 0x7fb5fc09fba0 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 1 -- 172.21.15.45:0/2606768925 >> v1:172.21.15.45:6793/0 conn(0x7fb5fc09f7d0 legacy=0x7fb5fc132b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 1 -- 172.21.15.45:0/2606768925 >> v1:172.21.15.84:6789/0 conn(0x7fb5fc09ff70 legacy=0x7fb5fc13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient: _finish_hunting 0 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 1 monclient: found mon.noname-g 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T14:17:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 1 -- 172.21.15.45:0/2606768925 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc0a1c00 con 0x7fb5fc09fba0 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient: _finish_auth 0 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 1 -- 172.21.15.45:0/2606768925 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb5f8001830 con 0x7fb5fc09fba0 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient: dump: 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb603f9d640 5 monclient: authenticate success, global_id 224265 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb603f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 1 -- 172.21.15.45:0/2606768925 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb5f8001990 con 0x7fb5fc09fba0 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb603f9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 1 -- 172.21.15.45:0/2606768925 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb5f8004ba0 con 0x7fb5fc09fba0 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb603f9d640 4 set_mon_vals no callback set 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.042+0000 7fb600d37640 10 monclient: dump: 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.043+0000 7fb603f9d640 10 monclient: shutdown 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.043+0000 7fb603f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.043+0000 7fb603f9d640 1 -- 172.21.15.45:0/2606768925 >> v1:172.21.15.84:6790/0 conn(0x7fb5fc09fba0 legacy=0x7fb5fc137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.043+0000 7fb603f9d640 1 -- 172.21.15.45:0/2606768925 shutdown_connections 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.043+0000 7fb603f9d640 1 -- 172.21.15.45:0/2606768925 >> 172.21.15.45:0/2606768925 conn(0x7fb5fc0630b0 msgr2=0x7fb5fc0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.043+0000 7fb603f9d640 1 -- 172.21.15.45:0/2606768925 shutdown_connections 2024-09-18T14:17:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.043+0000 7fb603f9d640 1 -- 172.21.15.45:0/2606768925 wait complete. 2024-09-18T14:17:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 10 monclient: build_initial_monmap 2024-09-18T14:17:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 10 monclient: monmap: 2024-09-18T14:17:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 1 Processor -- start 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 1 -- start start 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 10 monclient: init 2024-09-18T14:17:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 10 monclient: _add_conns ranks=[5,0,4,3,7,1,2,6,8] 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 10 monclient(hunting): picked mon.noname-h con 0x7fb5fc09f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 10 monclient(hunting): picked mon.noname-a con 0x7fb5fc09fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 10 monclient(hunting): picked mon.noname-c con 0x7fb5fc09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc09d310 con 0x7fb5fc09fba0 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc0a2560 con 0x7fb5fc09ff70 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc11d690 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.044+0000 7fb603f9d640 10 monclient(hunting): authenticate will time out at 1874.383955s 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb601d39640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7fb5fc09f7d0 0x7fb5fc09a340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:44132/0 (socket says 172.21.15.45:44132) 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb601d39640 1 -- 172.21.15.45:0/1998887069 learned_addr learned my addr 172.21.15.45:0/1998887069 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb601538640 1 --1- 172.21.15.45:0/1998887069 >> v1:172.21.15.45:6789/0 conn(0x7fb5fc09fba0 0x7fb5fc09aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43106/0 (socket says 172.21.15.45:43106) 2024-09-18T14:17:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025965791 0 0) 0x7fb5fc11d690 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): my global_id is 184982 2024-09-18T14:17:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5e00034e0 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327581163 0 0) 0x7fb5fc09d310 con 0x7fb5fc09fba0 2024-09-18T14:17:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): my global_id is 6597 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5fc11d690 con 0x7fb5fc09fba0 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864409455 0 0) 0x7fb5fc0a2560 con 0x7fb5fc09ff70 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): my global_id is 165325 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5fc09d310 con 0x7fb5fc09ff70 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2408667874 0 0) 0x7fb5e00034e0 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5fc0a2560 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb5f0004340 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): dump: 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.064 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 120884405 0 0) 0x7fb5fc09d310 con 0x7fb5fc09ff70 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5e00034e0 con 0x7fb5fc09ff70 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1053722581 0 0) 0x7fb5fc11d690 con 0x7fb5fc09fba0 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5fc09d310 con 0x7fb5fc09fba0 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb5f4001c40 con 0x7fb5fc09ff70 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): dump: 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb5f8001770 con 0x7fb5fc09fba0 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient(hunting): dump: 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 825883950 0 0) 0x7fb5fc0a2560 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 >> v1:172.21.15.45:6791/0 conn(0x7fb5fc09ff70 legacy=0x7fb5fc09b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 >> v1:172.21.15.45:6789/0 conn(0x7fb5fc09fba0 legacy=0x7fb5fc09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient: _finish_hunting 0 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 monclient: found mon.h 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc11d3b0 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient: _finish_auth 0 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb603f9d640 5 monclient: authenticate success, global_id 184982 2024-09-18T14:17:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb603f9d640 10 monclient: _renew_subs 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb603f9d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb603f9d640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc11c490 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.5 v1:172.21.15.84:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb5f0004520 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.5 v1:172.21.15.84:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb5f0004a50 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5edffb640 10 monclient: dump: 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.046+0000 7fb603f9d640 10 monclient: _renew_subs 2024-09-18T14:17:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.045+0000 7fb5effff640 4 set_mon_vals no callback set 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.046+0000 7fb603f9d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.046+0000 7fb603f9d640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc14adf0 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.046+0000 7fb603f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.046+0000 7fb603f9d640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb5fc04c4a0 con 0x7fb5fc0a44a0 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.046+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.5 v1:172.21.15.84:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fb5f0004c30 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.5 v1:172.21.15.84:6791/0 8 ==== osd_map(852..852 src has 637..852) v4 ==== 3943+0+0 (unknown 2896255082 0 0) 0x7fb5f0046bb0 con 0x7fb5fc09f7d0 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596904740 0 0) 0x7fb5fc04c4a0 con 0x7fb5fc0a44a0 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 10 monclient(hunting): my global_id is 224274 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5fc0a2560 con 0x7fb5fc0a44a0 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb60253a640 10 monclient: get_auth_request con 0x7fb5e003ab10 auth_method 0 2024-09-18T14:17:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 701640921 0 0) 0x7fb5fc0a2560 con 0x7fb5fc0a44a0 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5fc04c4a0 con 0x7fb5fc0a44a0 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb5f8004b30 con 0x7fb5fc0a44a0 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 10 monclient: dump: 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 511546170 0 0) 0x7fb5fc04c4a0 con 0x7fb5fc0a44a0 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.047+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5fc09bf20 con 0x7fb5fc0a44a0 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.048+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb5f8005160 con 0x7fb5fc0a44a0 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.048+0000 7fb5edffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.048+0000 7fb5edffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.048+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 >> v1:172.21.15.84:6790/0 conn(0x7fb5fc0a44a0 legacy=0x7fb5fc0a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.048+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 >> v1:172.21.15.84:6790/0 conn(0x7fb5fc0a44a0 legacy=0x7fb5fc0a4870 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb603f9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb603f9d640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb5fc11d850 con 0x7fb5fc0a4f80 2024-09-18T14:17:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901631645 0 0) 0x7fb5fc11d850 con 0x7fb5fc0a4f80 2024-09-18T14:17:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 10 monclient(hunting): my global_id is 224283 2024-09-18T14:17:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5fc04c4a0 con 0x7fb5fc0a4f80 2024-09-18T14:17:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 892119409 0 0) 0x7fb5fc04c4a0 con 0x7fb5fc0a4f80 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5fc11d850 con 0x7fb5fc0a4f80 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb5f8005a00 con 0x7fb5fc0a4f80 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 10 monclient: dump: 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 175186700 0 0) 0x7fb5fc11d850 con 0x7fb5fc0a4f80 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.054+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb5fc04b720 con 0x7fb5fc0a4f80 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7365 (unknown 0 0 2403350410) 0x7fb5f8008150 con 0x7fb5fc0a4f80 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb5edffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb5edffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 >> v1:172.21.15.84:6790/0 conn(0x7fb5fc0a4f80 legacy=0x7fb5fc14efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb5edffb640 1 -- 172.21.15.45:0/1998887069 >> v1:172.21.15.84:6790/0 conn(0x7fb5fc0a4f80 legacy=0x7fb5fc14efa0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:39.078 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:39.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:39.078 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:39.078 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:39.078 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 152, 2024-09-18T14:17:39.078 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:39.078 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:39.078 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:39.078 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:39.078 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:39.078 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:39.078 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: 7 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 0, 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:39.079 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:39.080 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:39.081 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:39.082 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.083 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:39.086 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:39.087 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.088 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:39.091 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:39.092 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:39.095 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:39.096 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb603f9d640 1 -- 172.21.15.45:0/1998887069 >> v1:172.21.15.84:6812/31478 conn(0x7fb5e003ab10 legacy=0x7fb5e003cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb603f9d640 10 monclient: shutdown 2024-09-18T14:17:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb603f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb603f9d640 1 -- 172.21.15.45:0/1998887069 >> v1:172.21.15.84:6791/0 conn(0x7fb5fc09f7d0 legacy=0x7fb5fc09a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb60253a640 1 -- 172.21.15.45:0/1998887069 reap_dead start 2024-09-18T14:17:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb60253a640 1 -- 172.21.15.45:0/1998887069 reap_dead start 2024-09-18T14:17:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb603f9d640 1 -- 172.21.15.45:0/1998887069 shutdown_connections 2024-09-18T14:17:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb603f9d640 1 -- 172.21.15.45:0/1998887069 >> 172.21.15.45:0/1998887069 conn(0x7fb5fc0630b0 msgr2=0x7fb5fc130b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb603f9d640 1 -- 172.21.15.45:0/1998887069 shutdown_connections 2024-09-18T14:17:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.056+0000 7fb603f9d640 1 -- 172.21.15.45:0/1998887069 wait complete. 2024-09-18T14:17:39.099 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:17:39.099 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-18T14:17:39.099 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:17:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.199+0000 7fac1639d640 10 monclient: get_monmap_and_config 2024-09-18T14:17:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.199+0000 7fac1639d640 10 monclient: build_initial_monmap 2024-09-18T14:17:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.199+0000 7fac1639d640 10 monclient: monmap: 2024-09-18T14:17:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:39.200944+0000 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:39.200944+0000 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:39.202 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.199+0000 7fac1639d640 1 Processor -- start 2024-09-18T14:17:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.199+0000 7fac1639d640 1 -- start start 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.199+0000 7fac1639d640 10 monclient: init 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac1639d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac1639d640 10 monclient: _add_conns ranks=[5,1,8,2,4,6,0,3,7] 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac1639d640 10 monclient(hunting): picked mon.noname-h con 0x7fac1009f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac1639d640 10 monclient(hunting): picked mon.noname-f con 0x7fac1009fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac1639d640 10 monclient(hunting): picked mon.noname-e con 0x7fac1009ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac1639d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1005ed70 con 0x7fac1009fba0 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac1639d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1009dfc0 con 0x7fac1009f7d0 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac1639d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1009dce0 con 0x7fac1009ff70 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac1639d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac1639d640 10 monclient(hunting): authenticate will time out at 1874.539405s 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac1493a640 1 -- >> v1:172.21.15.45:6793/0 conn(0x7fac1009ff70 legacy=0x7fac1013b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac0ffff640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7fac1009f7d0 0x7fac10132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:44144/0 (socket says 172.21.15.45:44144) 2024-09-18T14:17:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac07fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fac1009fba0 0x7fac10137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:48650/0 (socket says 172.21.15.45:48650) 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac0ffff640 1 -- 172.21.15.45:0/3721930174 learned_addr learned my addr 172.21.15.45:0/3721930174 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac0f7fe640 1 -- 172.21.15.45:0/3721930174 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341320330 0 0) 0x7fac1009dfc0 con 0x7fac1009f7d0 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac0f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac0f7fe640 10 monclient(hunting): my global_id is 184991 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac0f7fe640 1 -- 172.21.15.45:0/3721930174 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabfc0034e0 con 0x7fac1009f7d0 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac0f7fe640 1 -- 172.21.15.45:0/3721930174 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799029012 0 0) 0x7fac1005ed70 con 0x7fac1009fba0 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac0f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac0f7fe640 10 monclient(hunting): my global_id is 214606 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.200+0000 7fac0f7fe640 1 -- 172.21.15.45:0/3721930174 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac1009dfc0 con 0x7fac1009fba0 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 1 -- 172.21.15.45:0/3721930174 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 543106390 0 0) 0x7fac1009dfc0 con 0x7fac1009fba0 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 1 -- 172.21.15.45:0/3721930174 >> v1:172.21.15.45:6793/0 conn(0x7fac1009ff70 legacy=0x7fac1013b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:17:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 1 -- 172.21.15.45:0/3721930174 >> v1:172.21.15.84:6791/0 conn(0x7fac1009f7d0 legacy=0x7fac10132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 1 monclient: found mon.noname-f 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 1 -- 172.21.15.45:0/3721930174 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac100a1c00 con 0x7fac1009fba0 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 10 monclient: _finish_auth 0 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 1 -- 172.21.15.45:0/3721930174 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fabf80016c0 con 0x7fac1009fba0 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 10 monclient: dump: 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac1639d640 5 monclient: authenticate success, global_id 214606 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac1639d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 1 -- 172.21.15.45:0/3721930174 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fabf8001820 con 0x7fac1009fba0 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac1639d640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac1639d640 4 set_mon_vals no callback set 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 1 -- 172.21.15.45:0/3721930174 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fabf80047a0 con 0x7fac1009fba0 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac0f7fe640 10 monclient: dump: 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac1639d640 10 monclient: shutdown 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac1639d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac1639d640 1 -- 172.21.15.45:0/3721930174 >> v1:172.21.15.84:6789/0 conn(0x7fac1009fba0 legacy=0x7fac10137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac1639d640 1 -- 172.21.15.45:0/3721930174 shutdown_connections 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac1639d640 1 -- 172.21.15.45:0/3721930174 >> 172.21.15.45:0/3721930174 conn(0x7fac100630b0 msgr2=0x7fac100634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac1639d640 1 -- 172.21.15.45:0/3721930174 shutdown_connections 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.201+0000 7fac1639d640 1 -- 172.21.15.45:0/3721930174 wait complete. 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.202+0000 7fac1639d640 10 monclient: build_initial_monmap 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.202+0000 7fac1639d640 10 monclient: monmap: 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.202+0000 7fac1639d640 1 Processor -- start 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.202+0000 7fac1639d640 1 -- start start 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.202+0000 7fac1639d640 10 monclient: init 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac1639d640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac1639d640 10 monclient: _add_conns ranks=[1,5,7,6,3,0,2,8,4] 2024-09-18T14:17:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac1639d640 10 monclient(hunting): picked mon.noname-f con 0x7fac1009f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac1639d640 10 monclient(hunting): picked mon.noname-h con 0x7fac1009fba0 addr v1:172.21.15.84:6791/0 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac1639d640 10 monclient(hunting): picked mon.noname-i con 0x7fac1009ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac1639d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1009d310 con 0x7fac1009f7d0 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac1639d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac100a2560 con 0x7fac1009fba0 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac1639d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac10141eb0 con 0x7fac1009ff70 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac1639d640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac1639d640 10 monclient(hunting): authenticate will time out at 1874.542350s 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac07fff640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7fac1009fba0 0x7fac10130a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:44146/0 (socket says 172.21.15.45:44146) 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fac1009f7d0 0x7fac101417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:48666/0 (socket says 172.21.15.45:48666) 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac07fff640 1 -- 172.21.15.45:0/3772340342 learned_addr learned my addr 172.21.15.45:0/3772340342 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243540242 0 0) 0x7fac100a2560 con 0x7fac1009fba0 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): my global_id is 185000 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabf00034e0 con 0x7fac1009fba0 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886532640 0 0) 0x7fac1009d310 con 0x7fac1009f7d0 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): my global_id is 214615 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac100a2560 con 0x7fac1009f7d0 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 909969918 0 0) 0x7fac10141eb0 con 0x7fac1009ff70 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): my global_id is 214549 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac1009d310 con 0x7fac1009ff70 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 95783478 0 0) 0x7fabf00034e0 con 0x7fac1009fba0 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fac10141eb0 con 0x7fac1009fba0 2024-09-18T14:17:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fabf8001cd0 con 0x7fac1009fba0 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): dump: 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.203+0000 7fac0cff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1696479376 0 0) 0x7fac100a2560 con 0x7fac1009f7d0 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fabf00034e0 con 0x7fac1009f7d0 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fac000015b0 con 0x7fac1009f7d0 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient(hunting): dump: 2024-09-18T14:17:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 64206189 0 0) 0x7fac1009d310 con 0x7fac1009ff70 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fac100a2560 con 0x7fac1009ff70 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fac08004460 con 0x7fac1009ff70 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient(hunting): dump: 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 419288495 0 0) 0x7fac10141eb0 con 0x7fac1009fba0 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 >> v1:172.21.15.84:6792/0 conn(0x7fac1009ff70 legacy=0x7fac10149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 >> v1:172.21.15.84:6789/0 conn(0x7fac1009f7d0 legacy=0x7fac101417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient: _finish_hunting 0 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 monclient: found mon.h 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1011fc30 con 0x7fac1009fba0 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient: _finish_auth 0 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac1639d640 5 monclient: authenticate success, global_id 185000 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac1639d640 10 monclient: _renew_subs 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac1639d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac1639d640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac10120b90 con 0x7fac1009fba0 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac1639d640 10 monclient: _renew_subs 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac1639d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac1639d640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fac10131150 con 0x7fac1009fba0 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fabf8000fb0 con 0x7fac1009fba0 2024-09-18T14:17:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fabf8005010 con 0x7fac1009fba0 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0effd640 4 set_mon_vals no callback set 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac0cff9640 10 monclient: dump: 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac1639d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.204+0000 7fac1639d640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fabf4006580 con 0x7fabf4002830 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fabf8010060 con 0x7fac1009fba0 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 8 ==== osd_map(852..852 src has 637..852) v4 ==== 3943+0+0 (unknown 2896255082 0 0) 0x7fabf8046c50 con 0x7fac1009fba0 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167193540 0 0) 0x7fabf4006580 con 0x7fabf4002830 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 10 monclient(hunting): my global_id is 185009 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac10141eb0 con 0x7fabf4002830 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac1493a640 10 monclient: get_auth_request con 0x7fabf003aa10 auth_method 0 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 486663819 0 0) 0x7fac10141eb0 con 0x7fabf4002830 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fabf4006580 con 0x7fabf4002830 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fac00004b60 con 0x7fabf4002830 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 10 monclient: dump: 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2863412433 0 0) 0x7fabf4006580 con 0x7fabf4002830 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.205+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fabf400a0a0 con 0x7fabf4002830 2024-09-18T14:17:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.206+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fac00005110 con 0x7fabf4002830 2024-09-18T14:17:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.206+0000 7fac0cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.206+0000 7fac0cff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.206+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 >> v1:172.21.15.84:6791/0 conn(0x7fabf4002830 legacy=0x7fabf4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.206+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 >> v1:172.21.15.84:6791/0 conn(0x7fabf4002830 legacy=0x7fabf4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:39.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.211+0000 7fac1639d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:39.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.211+0000 7fac1639d640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fabf4007540 con 0x7fabf40068f0 2024-09-18T14:17:39.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406939563 0 0) 0x7fabf4007540 con 0x7fabf40068f0 2024-09-18T14:17:39.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 10 monclient(hunting): my global_id is 185018 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabf4006580 con 0x7fabf40068f0 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1127381958 0 0) 0x7fabf4006580 con 0x7fabf40068f0 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fabf4007540 con 0x7fabf40068f0 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fac000058f0 con 0x7fabf40068f0 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 10 monclient: dump: 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3414456402 0 0) 0x7fabf4007540 con 0x7fabf40068f0 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.212+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fabf40087a0 con 0x7fabf40068f0 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7705 (unknown 0 0 2401085105) 0x7fac00005c90 con 0x7fabf40068f0 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac0cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac0cff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:39.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 >> v1:172.21.15.84:6791/0 conn(0x7fabf40068f0 legacy=0x7fabf4006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac0cff9640 1 -- 172.21.15.45:0/3772340342 >> v1:172.21.15.84:6791/0 conn(0x7fabf40068f0 legacy=0x7fabf4006d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:39.234 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:39.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:39.234 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:39.234 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:39.234 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 152, 2024-09-18T14:17:39.234 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:39.234 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:39.234 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:39.234 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: 7 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:39.235 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:39.236 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:39.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:39.253 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.253 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.253 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:39.253 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:39.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac1639d640 1 -- 172.21.15.45:0/3772340342 >> v1:172.21.15.84:6812/31478 conn(0x7fabf003aa10 legacy=0x7fabf003cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac1639d640 10 monclient: shutdown 2024-09-18T14:17:39.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac1639d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:39.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac1639d640 1 -- 172.21.15.45:0/3772340342 >> v1:172.21.15.84:6791/0 conn(0x7fac1009fba0 legacy=0x7fac10130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac1493a640 1 -- 172.21.15.45:0/3772340342 reap_dead start 2024-09-18T14:17:39.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac1493a640 1 -- 172.21.15.45:0/3772340342 reap_dead start 2024-09-18T14:17:39.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac1639d640 1 -- 172.21.15.45:0/3772340342 shutdown_connections 2024-09-18T14:17:39.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac1639d640 1 -- 172.21.15.45:0/3772340342 >> 172.21.15.45:0/3772340342 conn(0x7fac100630b0 msgr2=0x7fac100a6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:39.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac1639d640 1 -- 172.21.15.45:0/3772340342 shutdown_connections 2024-09-18T14:17:39.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.213+0000 7fac1639d640 1 -- 172.21.15.45:0/3772340342 wait complete. 2024-09-18T14:17:39.255 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:17:39.255 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-09-18T14:17:39.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:17:39.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 10 monclient: get_monmap_and_config 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 10 monclient: build_initial_monmap 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 10 monclient: monmap: 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:17:39.353290+0000 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:17:39.353290+0000 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:39.354 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 1 Processor -- start 2024-09-18T14:17:39.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 1 -- start start 2024-09-18T14:17:39.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 10 monclient: init 2024-09-18T14:17:39.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:39.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 10 monclient: _add_conns ranks=[3,1,8,7,6,0,4,2,5] 2024-09-18T14:17:39.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 10 monclient(hunting): picked mon.noname-g con 0x7f3cf409f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 10 monclient(hunting): picked mon.noname-f con 0x7f3cf409fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:17:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 10 monclient(hunting): picked mon.noname-e con 0x7f3cf409ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:17:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf405ed70 con 0x7f3cf409fba0 2024-09-18T14:17:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf409dfc0 con 0x7f3cf409f7d0 2024-09-18T14:17:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf409dce0 con 0x7f3cf409ff70 2024-09-18T14:17:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cfb100640 10 monclient(hunting): authenticate will time out at 1874.691640s 2024-09-18T14:17:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cf969d640 1 -- >> v1:172.21.15.45:6793/0 conn(0x7f3cf409ff70 legacy=0x7f3cf413b190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:17:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cf8e9c640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7f3cf409f7d0 0x7f3cf4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:55970/0 (socket says 172.21.15.45:55970) 2024-09-18T14:17:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cf3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3cf409fba0 0x7f3cf4137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:48676/0 (socket says 172.21.15.45:48676) 2024-09-18T14:17:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.352+0000 7f3cf8e9c640 1 -- 172.21.15.45:0/3718960746 learned_addr learned my addr 172.21.15.45:0/3718960746 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 1 -- 172.21.15.45:0/3718960746 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1282830710 0 0) 0x7f3cf405ed70 con 0x7f3cf409fba0 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient(hunting): my global_id is 214624 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 1 -- 172.21.15.45:0/3718960746 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ce40034e0 con 0x7f3cf409fba0 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 1 -- 172.21.15.45:0/3718960746 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870833358 0 0) 0x7f3cf409dfc0 con 0x7f3cf409f7d0 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient(hunting): my global_id is 224292 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 1 -- 172.21.15.45:0/3718960746 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf405ed70 con 0x7f3cf409f7d0 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 1 -- 172.21.15.45:0/3718960746 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 34053343 0 0) 0x7f3ce40034e0 con 0x7f3cf409fba0 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 1 -- 172.21.15.45:0/3718960746 >> v1:172.21.15.45:6793/0 conn(0x7f3cf409ff70 legacy=0x7f3cf413b190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 1 -- 172.21.15.45:0/3718960746 >> v1:172.21.15.84:6790/0 conn(0x7f3cf409f7d0 legacy=0x7f3cf4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient: _finish_hunting 0 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 1 monclient: found mon.noname-f 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 1 -- 172.21.15.45:0/3718960746 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf40a1c00 con 0x7f3cf409fba0 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient: _finish_auth 0 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 1 -- 172.21.15.45:0/3718960746 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3ce0001760 con 0x7f3cf409fba0 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cfb100640 5 monclient: authenticate success, global_id 214624 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cfb100640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient: dump: 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cfb100640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 1 -- 172.21.15.45:0/3718960746 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3ce0001940 con 0x7f3cf409fba0 2024-09-18T14:17:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cfb100640 10 monclient: get_monmap_and_config success 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cfb100640 4 set_mon_vals no callback set 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 1 -- 172.21.15.45:0/3718960746 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3ce0004ae0 con 0x7f3cf409fba0 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cf37fe640 10 monclient: dump: 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cfb100640 10 monclient: shutdown 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cfb100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cfb100640 1 -- 172.21.15.45:0/3718960746 >> v1:172.21.15.84:6789/0 conn(0x7f3cf409fba0 legacy=0x7f3cf4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cfb100640 1 -- 172.21.15.45:0/3718960746 shutdown_connections 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.353+0000 7f3cfb100640 1 -- 172.21.15.45:0/3718960746 >> 172.21.15.45:0/3718960746 conn(0x7f3cf40630b0 msgr2=0x7f3cf40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.354+0000 7f3cfb100640 1 -- 172.21.15.45:0/3718960746 shutdown_connections 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.354+0000 7f3cfb100640 1 -- 172.21.15.45:0/3718960746 wait complete. 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.354+0000 7f3cfb100640 10 monclient: build_initial_monmap 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.354+0000 7f3cfb100640 10 monclient: monmap: 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:17:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.354+0000 7f3cfb100640 1 Processor -- start 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.354+0000 7f3cfb100640 1 -- start start 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.354+0000 7f3cfb100640 10 monclient: init 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cfb100640 10 monclient: _reopen_session rank -1 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cfb100640 10 monclient: _add_conns ranks=[6,0,3,8,2,1,7,4,5] 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cfb100640 10 monclient(hunting): picked mon.noname-d con 0x7f3cf409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cfb100640 10 monclient(hunting): picked mon.noname-a con 0x7f3cf409fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:17:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cfb100640 10 monclient(hunting): picked mon.noname-g con 0x7f3cf409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cfb100640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf409be30 con 0x7f3cf409fba0 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cfb100640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf411e420 con 0x7f3cf409ff70 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cfb100640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf411d720 con 0x7f3cf409f7d0 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cfb100640 10 monclient(hunting): _renew_subs 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cfb100640 10 monclient(hunting): authenticate will time out at 1874.694485s 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf8e9c640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f3cf409f7d0 0x7f3cf4131560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:35374/0 (socket says 172.21.15.45:35374) 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf8e9c640 1 -- 172.21.15.45:0/3007234289 learned_addr learned my addr 172.21.15.45:0/3007234289 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015857767 0 0) 0x7f3cf409be30 con 0x7f3cf409fba0 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 10 monclient(hunting): my global_id is 6606 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cd80034e0 con 0x7f3cf409fba0 2024-09-18T14:17:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117349023 0 0) 0x7f3cf411d720 con 0x7f3cf409f7d0 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 10 monclient(hunting): my global_id is 185145 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf409be30 con 0x7f3cf409f7d0 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803043917 0 0) 0x7f3cf411e420 con 0x7f3cf409ff70 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 10 monclient(hunting): my global_id is 224301 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.355+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf411d720 con 0x7f3cf409ff70 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4141930399 0 0) 0x7f3cd80034e0 con 0x7f3cf409fba0 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3cf411e420 con 0x7f3cf409fba0 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3ce0001360 con 0x7f3cf409fba0 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient(hunting): dump: 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1286413127 0 0) 0x7f3cf409be30 con 0x7f3cf409f7d0 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3cd80034e0 con 0x7f3cf409f7d0 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3ce8001a60 con 0x7f3cf409f7d0 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient(hunting): dump: 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4010771536 0 0) 0x7f3cf411d720 con 0x7f3cf409ff70 2024-09-18T14:17:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3cf409be30 con 0x7f3cf409ff70 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3cec004460 con 0x7f3cf409ff70 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient(hunting): dump: 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2597002894 0 0) 0x7f3cf411e420 con 0x7f3cf409fba0 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 >> v1:172.21.15.45:6792/0 conn(0x7f3cf409f7d0 legacy=0x7f3cf4131560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 >> v1:172.21.15.84:6790/0 conn(0x7f3cf409ff70 legacy=0x7f3cf414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 monclient: found mon.a 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf411e680 con 0x7f3cf409fba0 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient: _finish_auth 0 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cfb100640 5 monclient: authenticate success, global_id 6606 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cfb100640 10 monclient: _renew_subs 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cfb100640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cfb100640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf411e960 con 0x7f3cf409fba0 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cfb100640 10 monclient: _renew_subs 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cfb100640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cfb100640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf4136ea0 con 0x7f3cf409fba0 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3ce0004d60 con 0x7f3cf409fba0 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3ce0005210 con 0x7f3cf409fba0 2024-09-18T14:17:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf2ffd640 4 set_mon_vals no callback set 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.356+0000 7f3cf0ff9640 10 monclient: dump: 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f3ce00101e0 con 0x7f3cf409fba0 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cfb100640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cfb100640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3cf404c4a0 con 0x7f3cf409ac30 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(852..852 src has 637..852) v4 ==== 3943+0+0 (unknown 2896255082 0 0) 0x7f3ce0046c30 con 0x7f3cf409fba0 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf969d640 10 monclient: get_auth_request con 0x7f3cd803aa40 auth_method 0 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690348373 0 0) 0x7f3cf404c4a0 con 0x7f3cf409ac30 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 10 monclient(hunting): my global_id is 214558 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf411e420 con 0x7f3cf409ac30 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1985280506 0 0) 0x7f3cf411e420 con 0x7f3cf409ac30 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3cf404c4a0 con 0x7f3cf409ac30 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3ce8004e10 con 0x7f3cf409ac30 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.357+0000 7f3cf0ff9640 10 monclient: dump: 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.358+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 881971423 0 0) 0x7f3cf404c4a0 con 0x7f3cf409ac30 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.358+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3cf404b720 con 0x7f3cf409ac30 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.358+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3ce8005270 con 0x7f3cf409ac30 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.358+0000 7f3cf0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:17:39.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.358+0000 7f3cf0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:17:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.358+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 >> v1:172.21.15.84:6792/0 conn(0x7f3cf409ac30 legacy=0x7f3cf4096a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.358+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 >> v1:172.21.15.84:6792/0 conn(0x7f3cf409ac30 legacy=0x7f3cf4096a20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:39.385 INFO:tasks.rados.rados.0.smithi084.stdout:3810: done (2 left) 2024-09-18T14:17:39.385 INFO:tasks.rados.rados.0.smithi084.stdout:3811: snap_create 2024-09-18T14:17:39.385 INFO:tasks.rados.rados.0.smithi084.stdout:3808: finishing write tid 2 to smithi08434663-39 2024-09-18T14:17:39.385 INFO:tasks.rados.rados.0.smithi084.stdout:3808: finishing write tid 3 to smithi08434663-39 2024-09-18T14:17:39.385 INFO:tasks.rados.rados.0.smithi084.stdout:3808: finishing write tid 4 to smithi08434663-39 2024-09-18T14:17:39.385 INFO:tasks.rados.rados.0.smithi084.stdout:3808: finishing write tid 5 to smithi08434663-39 2024-09-18T14:17:39.385 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 2060 (ObjNum 1274 snap 426 seq_num 1274) dirty exists 2024-09-18T14:17:39.385 INFO:tasks.rados.rados.0.smithi084.stdout:3808: left oid 39 (ObjNum 1274 snap 426 seq_num 1274) 2024-09-18T14:17:39.385 INFO:tasks.rados.rados.0.smithi084.stdout:3809: finishing write tid 1 to smithi08434663-5 2024-09-18T14:17:39.386 INFO:tasks.rados.rados.0.smithi084.stdout:3809: finishing write tid 2 to smithi08434663-5 2024-09-18T14:17:39.386 INFO:tasks.rados.rados.0.smithi084.stdout:3809: finishing write tid 3 to smithi08434663-5 2024-09-18T14:17:39.386 INFO:tasks.rados.rados.0.smithi084.stdout:3809: finishing write tid 4 to smithi08434663-5 2024-09-18T14:17:39.386 INFO:tasks.rados.rados.0.smithi084.stdout:3809: finishing write tid 5 to smithi08434663-5 2024-09-18T14:17:39.386 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 2050 (ObjNum 1275 snap 426 seq_num 1275) dirty exists 2024-09-18T14:17:39.386 INFO:tasks.rados.rados.0.smithi084.stdout:3809: left oid 5 (ObjNum 1275 snap 426 seq_num 1275) 2024-09-18T14:17:39.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.364+0000 7f3cfb100640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:17:39.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.364+0000 7f3cfb100640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3cf4071550 con 0x7f3cf402a8d0 2024-09-18T14:17:39.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.372+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564561146 0 0) 0x7f3cf4071550 con 0x7f3cf402a8d0 2024-09-18T14:17:39.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.372+0000 7f3cf0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:17:39.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.372+0000 7f3cf0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:17:39.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.372+0000 7f3cf0ff9640 10 monclient(hunting): my global_id is 214567 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.372+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf404c4a0 con 0x7f3cf402a8d0 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.373+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1112803396 0 0) 0x7f3cf404c4a0 con 0x7f3cf402a8d0 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.373+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3cf4071550 con 0x7f3cf402a8d0 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.373+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3ce8005bd0 con 0x7f3cf402a8d0 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.373+0000 7f3cf0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.373+0000 7f3cf0ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.373+0000 7f3cf0ff9640 10 monclient: dump: 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:17:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.374+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 75502043 0 0) 0x7f3cf4071550 con 0x7f3cf402a8d0 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.374+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3cf409b000 con 0x7f3cf402a8d0 2024-09-18T14:17:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.374+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 2430193523) 0x7f3ce8007d80 con 0x7f3cf402a8d0 2024-09-18T14:17:39.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.374+0000 7f3cf0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:17:39.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.374+0000 7f3cf0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:17:39.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.374+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 >> v1:172.21.15.84:6792/0 conn(0x7f3cf402a8d0 legacy=0x7f3cf4097130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.374+0000 7f3cf0ff9640 1 -- 172.21.15.45:0/3007234289 >> v1:172.21.15.84:6792/0 conn(0x7f3cf402a8d0 legacy=0x7f3cf4097130 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:17:39.390 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:17:39.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:39.390 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:39.390 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:17:39.390 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 152, 2024-09-18T14:17:39.390 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:17:39.390 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: 7 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 1, 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:39.391 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:39.392 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:17:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:17:39.394 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:17:39.395 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:17:39.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.399 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.401 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.402 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:17:39.404 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.405 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:39.406 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:17:39.407 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:17:39.408 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:17:39.408 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:17:39.408 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:17:39.408 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:17:39.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:17:39.408 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:17:39.408 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:17:39.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.376+0000 7f3cfb100640 1 -- 172.21.15.45:0/3007234289 >> v1:172.21.15.84:6812/31478 conn(0x7f3cd803aa40 legacy=0x7f3cd803cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.376+0000 7f3cfb100640 10 monclient: shutdown 2024-09-18T14:17:39.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.376+0000 7f3cfb100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:17:39.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.376+0000 7f3cfb100640 1 -- 172.21.15.45:0/3007234289 >> v1:172.21.15.45:6789/0 conn(0x7f3cf409fba0 legacy=0x7f3cf4146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:17:39.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.376+0000 7f3cf969d640 1 -- 172.21.15.45:0/3007234289 reap_dead start 2024-09-18T14:17:39.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.376+0000 7f3cf969d640 1 -- 172.21.15.45:0/3007234289 reap_dead start 2024-09-18T14:17:39.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.376+0000 7f3cfb100640 1 -- 172.21.15.45:0/3007234289 shutdown_connections 2024-09-18T14:17:39.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.377+0000 7f3cfb100640 1 -- 172.21.15.45:0/3007234289 >> 172.21.15.45:0/3007234289 conn(0x7f3cf40630b0 msgr2=0x7f3cf40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:17:39.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.377+0000 7f3cfb100640 1 -- 172.21.15.45:0/3007234289 shutdown_connections 2024-09-18T14:17:39.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:17:39.377+0000 7f3cfb100640 1 -- 172.21.15.45:0/3007234289 wait complete. 2024-09-18T14:17:39.411 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-18T14:17:39.411 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-18T14:17:40.378 INFO:tasks.rados.rados.0.smithi084.stdout:3808: done (2 left) 2024-09-18T14:17:40.378 INFO:tasks.rados.rados.0.smithi084.stdout:3809: done (1 left) 2024-09-18T14:17:40.378 INFO:tasks.rados.rados.0.smithi084.stdout:3811: done (0 left) 2024-09-18T14:17:40.379 INFO:tasks.rados.rados.0.smithi084.stdout:3812: write oid 2 current snap is 427 2024-09-18T14:17:40.379 INFO:tasks.rados.rados.0.smithi084.stdout:3812: seq_num 1276 ranges {645732=648675,1751847=574113,3124327=321405} 2024-09-18T14:17:40.385 INFO:tasks.rados.rados.0.smithi084.stdout:3812: writing smithi08434663-2 from 645732 to 1294407 tid 1 2024-09-18T14:17:40.391 INFO:tasks.rados.rados.0.smithi084.stdout:3812: writing smithi08434663-2 from 1751847 to 2325960 tid 2 2024-09-18T14:17:40.397 INFO:tasks.rados.rados.0.smithi084.stdout:3812: writing smithi08434663-2 from 3124327 to 3445732 tid 3 2024-09-18T14:17:40.397 INFO:tasks.rados.rados.0.smithi084.stdout:3813: read oid 14 snap 425 2024-09-18T14:17:40.397 INFO:tasks.rados.rados.0.smithi084.stdout:3813: expect (ObjNum 1239 snap 409 seq_num 1239) 2024-09-18T14:17:40.397 INFO:tasks.rados.rados.0.smithi084.stdout:3814: rollback oid 44 current snap is 427 2024-09-18T14:17:40.398 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 44 to 415 2024-09-18T14:17:40.398 INFO:tasks.rados.rados.0.smithi084.stdout:3812: finishing write tid 1 to smithi08434663-2 2024-09-18T14:17:40.398 INFO:tasks.rados.rados.0.smithi084.stdout:3812: finishing write tid 2 to smithi08434663-2 2024-09-18T14:17:40.398 INFO:tasks.rados.rados.0.smithi084.stdout:3815: write oid 5 current snap is 427 2024-09-18T14:17:40.398 INFO:tasks.rados.rados.0.smithi084.stdout:3815: seq_num 1277 ranges {662539=723924,2093976=717771,3435397=27142} 2024-09-18T14:17:40.405 INFO:tasks.rados.rados.0.smithi084.stdout:3815: writing smithi08434663-5 from 662539 to 1386463 tid 1 2024-09-18T14:17:40.413 INFO:tasks.rados.rados.0.smithi084.stdout:3815: writing smithi08434663-5 from 2093976 to 2811747 tid 2 2024-09-18T14:17:40.417 INFO:tasks.rados.rados.0.smithi084.stdout:3815: writing smithi08434663-5 from 3435397 to 3462539 tid 3 2024-09-18T14:17:40.417 INFO:tasks.rados.rados.0.smithi084.stdout:3816: write (excl) oid 25 current snap is 427 2024-09-18T14:17:40.417 INFO:tasks.rados.rados.0.smithi084.stdout:3816: seq_num 1278 ranges {679346=515526,1868811=577782,3179173=300173} 2024-09-18T14:17:40.423 INFO:tasks.rados.rados.0.smithi084.stdout:3816: writing smithi08434663-25 from 679346 to 1194872 tid 1 2024-09-18T14:17:40.430 INFO:tasks.rados.rados.0.smithi084.stdout:3816: writing smithi08434663-25 from 1868811 to 2446593 tid 2 2024-09-18T14:17:40.436 INFO:tasks.rados.rados.0.smithi084.stdout:3816: writing smithi08434663-25 from 3179173 to 3479346 tid 3 2024-09-18T14:17:40.436 INFO:tasks.rados.rados.0.smithi084.stdout:3817: rollback oid 29 current snap is 427 2024-09-18T14:17:40.436 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 29 to 411 2024-09-18T14:17:40.436 INFO:tasks.rados.rados.0.smithi084.stdout:3812: finishing write tid 3 to smithi08434663-2 2024-09-18T14:17:40.436 INFO:tasks.rados.rados.0.smithi084.stdout:3814: finishing rollback tid 0 to smithi08434663-44 2024-09-18T14:17:40.436 INFO:tasks.rados.rados.0.smithi084.stdout:3818: read oid 20 snap -1 2024-09-18T14:17:40.437 INFO:tasks.rados.rados.0.smithi084.stdout:3818: expect (ObjNum 1171 snap 382 seq_num 1171) 2024-09-18T14:17:40.437 INFO:tasks.rados.rados.0.smithi084.stdout:3819: snap_create 2024-09-18T14:17:40.437 INFO:tasks.rados.rados.0.smithi084.stdout:3813: expect (ObjNum 1239 snap 409 seq_num 1239) 2024-09-18T14:17:40.529 INFO:tasks.rados.rados.0.smithi084.stdout:3812: finishing write tid 4 to smithi08434663-2 2024-09-18T14:17:40.530 INFO:tasks.rados.rados.0.smithi084.stdout:3812: finishing write tid 5 to smithi08434663-2 2024-09-18T14:17:40.530 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 2219 (ObjNum 1276 snap 427 seq_num 1276) dirty exists 2024-09-18T14:17:40.530 INFO:tasks.rados.rados.0.smithi084.stdout:3812: left oid 2 (ObjNum 1276 snap 427 seq_num 1276) 2024-09-18T14:17:40.530 INFO:tasks.rados.rados.0.smithi084.stdout:3814: finishing rollback tid 1 to smithi08434663-44 2024-09-18T14:17:40.530 INFO:tasks.rados.rados.0.smithi084.stdout:3814: finishing rollback tid 2 to smithi08434663-44 2024-09-18T14:17:40.530 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 2053 (ObjNum 991 snap 321 seq_num 991) dirty exists 2024-09-18T14:17:40.530 INFO:tasks.rados.rados.0.smithi084.stdout:3815: finishing write tid 1 to smithi08434663-5 2024-09-18T14:17:40.530 INFO:tasks.rados.rados.0.smithi084.stdout:3815: finishing write tid 2 to smithi08434663-5 2024-09-18T14:17:40.530 INFO:tasks.rados.rados.0.smithi084.stdout:3815: finishing write tid 3 to smithi08434663-5 2024-09-18T14:17:40.530 INFO:tasks.rados.rados.0.smithi084.stdout:3815: finishing write tid 4 to smithi08434663-5 2024-09-18T14:17:40.530 INFO:tasks.rados.rados.0.smithi084.stdout:3815: finishing write tid 5 to smithi08434663-5 2024-09-18T14:17:40.531 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 2057 (ObjNum 1277 snap 427 seq_num 1277) dirty exists 2024-09-18T14:17:40.531 INFO:tasks.rados.rados.0.smithi084.stdout:3815: left oid 5 (ObjNum 1277 snap 427 seq_num 1277) 2024-09-18T14:17:40.531 INFO:tasks.rados.rados.0.smithi084.stdout:3816: finishing write tid 1 to smithi08434663-25 2024-09-18T14:17:40.531 INFO:tasks.rados.rados.0.smithi084.stdout:3816: finishing write tid 2 to smithi08434663-25 2024-09-18T14:17:40.531 INFO:tasks.rados.rados.0.smithi084.stdout:3817: finishing rollback tid 1 to smithi08434663-29 2024-09-18T14:17:40.531 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 1580 (ObjNum 129 snap 0 seq_num 2494744072) dirty dne 2024-09-18T14:17:40.531 INFO:tasks.rados.rados.0.smithi084.stdout:3816: finishing write tid 3 to smithi08434663-25 2024-09-18T14:17:40.531 INFO:tasks.rados.rados.0.smithi084.stdout:3816: finishing write tid 4 to smithi08434663-25 2024-09-18T14:17:40.531 INFO:tasks.rados.rados.0.smithi084.stdout:3816: finishing write tid 5 to smithi08434663-25 2024-09-18T14:17:40.531 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 2057 (ObjNum 1278 snap 427 seq_num 1278) dirty exists 2024-09-18T14:17:40.531 INFO:tasks.rados.rados.0.smithi084.stdout:3816: left oid 25 (ObjNum 1278 snap 427 seq_num 1278) 2024-09-18T14:17:40.531 INFO:tasks.rados.rados.0.smithi084.stdout:3818: expect (ObjNum 1171 snap 382 seq_num 1171) 2024-09-18T14:17:41.380 INFO:tasks.rados.rados.0.smithi084.stdout:3812: done (7 left) 2024-09-18T14:17:41.381 INFO:tasks.rados.rados.0.smithi084.stdout:3813: done (6 left) 2024-09-18T14:17:41.381 INFO:tasks.rados.rados.0.smithi084.stdout:3814: done (5 left) 2024-09-18T14:17:41.381 INFO:tasks.rados.rados.0.smithi084.stdout:3815: done (4 left) 2024-09-18T14:17:41.381 INFO:tasks.rados.rados.0.smithi084.stdout:3816: done (3 left) 2024-09-18T14:17:41.381 INFO:tasks.rados.rados.0.smithi084.stdout:3817: done (2 left) 2024-09-18T14:17:41.381 INFO:tasks.rados.rados.0.smithi084.stdout:3818: done (1 left) 2024-09-18T14:17:41.382 INFO:tasks.rados.rados.0.smithi084.stdout:3819: done (0 left) 2024-09-18T14:17:41.382 INFO:tasks.rados.rados.0.smithi084.stdout:3820: snap_create 2024-09-18T14:17:42.398 INFO:tasks.rados.rados.0.smithi084.stdout:3820: done (0 left) 2024-09-18T14:17:42.398 INFO:tasks.rados.rados.0.smithi084.stdout:3821: write (excl) oid 38 current snap is 429 2024-09-18T14:17:42.398 INFO:tasks.rados.rados.0.smithi084.stdout:3821: seq_num 1279 ranges {696153=590775,1927293=721440,3206596=289557} 2024-09-18T14:17:42.405 INFO:tasks.rados.rados.0.smithi084.stdout:3821: writing smithi08434663-38 from 696153 to 1286928 tid 1 2024-09-18T14:17:42.412 INFO:tasks.rados.rados.0.smithi084.stdout:3821: writing smithi08434663-38 from 1927293 to 2648733 tid 2 2024-09-18T14:17:42.417 INFO:tasks.rados.rados.0.smithi084.stdout:3821: writing smithi08434663-38 from 3206596 to 3496153 tid 3 2024-09-18T14:17:42.417 INFO:tasks.rados.rados.0.smithi084.stdout:3822: snap_remove snap 415 2024-09-18T14:17:43.371 INFO:tasks.rados.rados.0.smithi084.stdout:3822: done (1 left) 2024-09-18T14:17:43.371 INFO:tasks.rados.rados.0.smithi084.stdout:3823: delete oid 48 current snap is 429 2024-09-18T14:17:43.371 INFO:tasks.rados.rados.0.smithi084.stdout:3821: finishing write tid 1 to smithi08434663-38 2024-09-18T14:17:43.372 INFO:tasks.rados.rados.0.smithi084.stdout:3821: finishing write tid 2 to smithi08434663-38 2024-09-18T14:17:43.372 INFO:tasks.rados.rados.0.smithi084.stdout:3821: finishing write tid 3 to smithi08434663-38 2024-09-18T14:17:43.372 INFO:tasks.rados.rados.0.smithi084.stdout:3821: finishing write tid 4 to smithi08434663-38 2024-09-18T14:17:43.372 INFO:tasks.rados.rados.0.smithi084.stdout:3821: finishing write tid 5 to smithi08434663-38 2024-09-18T14:17:43.372 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 38 v 2043 (ObjNum 1279 snap 429 seq_num 1279) dirty exists 2024-09-18T14:17:43.372 INFO:tasks.rados.rados.0.smithi084.stdout:3821: left oid 38 (ObjNum 1279 snap 429 seq_num 1279) 2024-09-18T14:17:43.385 INFO:tasks.rados.rados.0.smithi084.stdout:3821: done (1 left) 2024-09-18T14:17:43.385 INFO:tasks.rados.rados.0.smithi084.stdout:3823: done (0 left) 2024-09-18T14:17:43.386 INFO:tasks.rados.rados.0.smithi084.stdout:3824: snap_remove snap 418 2024-09-18T14:17:44.387 INFO:tasks.rados.rados.0.smithi084.stdout:3824: done (0 left) 2024-09-18T14:17:44.387 INFO:tasks.rados.rados.0.smithi084.stdout:3825: snap_create 2024-09-18T14:17:45.386 INFO:tasks.rados.rados.0.smithi084.stdout:3825: done (0 left) 2024-09-18T14:17:45.387 INFO:tasks.rados.rados.0.smithi084.stdout:3826: rollback oid 22 current snap is 430 2024-09-18T14:17:45.387 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 22 to 428 2024-09-18T14:17:45.387 INFO:tasks.rados.rados.0.smithi084.stdout:3827: copy_from oid 21 from oid 7 current snap is 430 2024-09-18T14:17:45.387 INFO:tasks.rados.rados.0.smithi084.stdout:3828: snap_remove snap 414 2024-09-18T14:17:46.391 INFO:tasks.rados.rados.0.smithi084.stdout:3828: done (2 left) 2024-09-18T14:17:46.391 INFO:tasks.rados.rados.0.smithi084.stdout:3829: read oid 42 snap -1 2024-09-18T14:17:46.392 INFO:tasks.rados.rados.0.smithi084.stdout:3829: expect (ObjNum 1054 snap 336 seq_num 1054) 2024-09-18T14:17:46.392 INFO:tasks.rados.rados.0.smithi084.stdout:3830: write (excl) oid 31 current snap is 430 2024-09-18T14:17:46.392 INFO:tasks.rados.rados.0.smithi084.stdout:3827: finishing copy_from to smithi08434663-21 2024-09-18T14:17:46.392 INFO:tasks.rados.rados.0.smithi084.stdout:3827: got expected ENOENT (src dne) 2024-09-18T14:17:46.392 INFO:tasks.rados.rados.0.smithi084.stdout:3827: finishing copy_from racing read to smithi08434663-21 2024-09-18T14:17:46.392 INFO:tasks.rados.rados.0.smithi084.stdout:3826: finishing rollback tid 0 to smithi08434663-22 2024-09-18T14:17:46.392 INFO:tasks.rados.rados.0.smithi084.stdout:3826: finishing rollback tid 1 to smithi08434663-22 2024-09-18T14:17:46.392 INFO:tasks.rados.rados.0.smithi084.stdout:3826: finishing rollback tid 2 to smithi08434663-22 2024-09-18T14:17:46.392 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 22 v 1590 (ObjNum 1252 snap 413 seq_num 1252) dirty exists 2024-09-18T14:17:46.392 INFO:tasks.rados.rados.0.smithi084.stdout:3830: seq_num 1281 ranges {729767=741273,1927904=725109,3145089=384678} 2024-09-18T14:17:46.399 INFO:tasks.rados.rados.0.smithi084.stdout:3830: writing smithi08434663-31 from 729767 to 1471040 tid 1 2024-09-18T14:17:46.407 INFO:tasks.rados.rados.0.smithi084.stdout:3830: writing smithi08434663-31 from 1927904 to 2653013 tid 2 2024-09-18T14:17:46.412 INFO:tasks.rados.rados.0.smithi084.stdout:3830: writing smithi08434663-31 from 3145089 to 3529767 tid 3 2024-09-18T14:17:46.412 INFO:tasks.rados.rados.0.smithi084.stdout:3826: done (3 left) 2024-09-18T14:17:46.412 INFO:tasks.rados.rados.0.smithi084.stdout:3827: done (2 left) 2024-09-18T14:17:46.412 INFO:tasks.rados.rados.0.smithi084.stdout:3831: delete oid 19 current snap is 430 2024-09-18T14:17:46.412 INFO:tasks.rados.rados.0.smithi084.stdout:3830: finishing write tid 1 to smithi08434663-31 2024-09-18T14:17:46.413 INFO:tasks.rados.rados.0.smithi084.stdout:3829: expect (ObjNum 1054 snap 336 seq_num 1054) 2024-09-18T14:17:46.714 INFO:tasks.rados.rados.0.smithi084.stdout:3830: finishing write tid 2 to smithi08434663-31 2024-09-18T14:17:46.714 INFO:tasks.rados.rados.0.smithi084.stdout:3830: finishing write tid 3 to smithi08434663-31 2024-09-18T14:17:46.714 INFO:tasks.rados.rados.0.smithi084.stdout:3830: finishing write tid 4 to smithi08434663-31 2024-09-18T14:17:46.715 INFO:tasks.rados.rados.0.smithi084.stdout:3830: finishing write tid 5 to smithi08434663-31 2024-09-18T14:17:46.715 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 2130 (ObjNum 1281 snap 430 seq_num 1281) dirty exists 2024-09-18T14:17:46.715 INFO:tasks.rados.rados.0.smithi084.stdout:3830: left oid 31 (ObjNum 1281 snap 430 seq_num 1281) 2024-09-18T14:17:46.715 INFO:tasks.rados.rados.0.smithi084.stdout:3829: done (2 left) 2024-09-18T14:17:46.715 INFO:tasks.rados.rados.0.smithi084.stdout:3830: done (1 left) 2024-09-18T14:17:46.715 INFO:tasks.rados.rados.0.smithi084.stdout:3831: done (0 left) 2024-09-18T14:17:46.715 INFO:tasks.rados.rados.0.smithi084.stdout:3832: snap_create 2024-09-18T14:17:47.396 INFO:tasks.rados.rados.0.smithi084.stdout:3832: done (0 left) 2024-09-18T14:17:47.396 INFO:tasks.rados.rados.0.smithi084.stdout:3833: read oid 19 snap -1 2024-09-18T14:17:47.396 INFO:tasks.rados.rados.0.smithi084.stdout:3833: expect deleted 2024-09-18T14:17:47.396 INFO:tasks.rados.rados.0.smithi084.stdout:3834: write oid 15 current snap is 431 2024-09-18T14:17:47.397 INFO:tasks.rados.rados.0.smithi084.stdout:3834: seq_num 1283 ranges {763381=491771,2044868=728778,3199935=363446} 2024-09-18T14:17:47.402 INFO:tasks.rados.rados.0.smithi084.stdout:3834: writing smithi08434663-15 from 763381 to 1255152 tid 1 2024-09-18T14:17:47.411 INFO:tasks.rados.rados.0.smithi084.stdout:3834: writing smithi08434663-15 from 2044868 to 2773646 tid 2 2024-09-18T14:17:47.416 INFO:tasks.rados.rados.0.smithi084.stdout:3834: writing smithi08434663-15 from 3199935 to 3563381 tid 3 2024-09-18T14:17:47.416 INFO:tasks.rados.rados.0.smithi084.stdout:3835: read oid 30 snap -1 2024-09-18T14:17:47.416 INFO:tasks.rados.rados.0.smithi084.stdout:3835: expect deleted 2024-09-18T14:17:47.416 INFO:tasks.rados.rados.0.smithi084.stdout:3836: write oid 50 current snap is 431 2024-09-18T14:17:47.416 INFO:tasks.rados.rados.0.smithi084.stdout:3836: seq_num 1284 ranges {780188=567020,2103350=472436,3227358=352830} 2024-09-18T14:17:47.423 INFO:tasks.rados.rados.0.smithi084.stdout:3836: writing smithi08434663-50 from 780188 to 1347208 tid 1 2024-09-18T14:17:47.429 INFO:tasks.rados.rados.0.smithi084.stdout:3836: writing smithi08434663-50 from 2103350 to 2575786 tid 2 2024-09-18T14:17:47.435 INFO:tasks.rados.rados.0.smithi084.stdout:3836: writing smithi08434663-50 from 3227358 to 3580188 tid 3 2024-09-18T14:17:47.435 INFO:tasks.rados.rados.0.smithi084.stdout:3837: write (excl) oid 33 current snap is 431 2024-09-18T14:17:47.435 INFO:tasks.rados.rados.0.smithi084.stdout:3837: seq_num 1285 ranges {796995=758622,2161832=732447} 2024-09-18T14:17:47.444 INFO:tasks.rados.rados.0.smithi084.stdout:3837: writing smithi08434663-33 from 796995 to 1555617 tid 1 2024-09-18T14:17:47.451 INFO:tasks.rados.rados.0.smithi084.stdout:3837: writing smithi08434663-33 from 2161832 to 2894279 tid 2 2024-09-18T14:17:47.451 INFO:tasks.rados.rados.0.smithi084.stdout:3838: snap_remove snap 387 2024-09-18T14:17:48.405 INFO:tasks.rados.rados.0.smithi084.stdout:3838: done (5 left) 2024-09-18T14:17:48.405 INFO:tasks.rados.rados.0.smithi084.stdout:3839: copy_from oid 1 from oid 8 current snap is 431 2024-09-18T14:17:48.405 INFO:tasks.rados.rados.0.smithi084.stdout:3834: finishing write tid 1 to smithi08434663-15 2024-09-18T14:17:48.405 INFO:tasks.rados.rados.0.smithi084.stdout:3834: finishing write tid 2 to smithi08434663-15 2024-09-18T14:17:48.405 INFO:tasks.rados.rados.0.smithi084.stdout:3834: finishing write tid 3 to smithi08434663-15 2024-09-18T14:17:48.405 INFO:tasks.rados.rados.0.smithi084.stdout:3834: finishing write tid 4 to smithi08434663-15 2024-09-18T14:17:48.405 INFO:tasks.rados.rados.0.smithi084.stdout:3834: finishing write tid 5 to smithi08434663-15 2024-09-18T14:17:48.405 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 2056 (ObjNum 1283 snap 431 seq_num 1283) dirty exists 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:3834: left oid 15 (ObjNum 1283 snap 431 seq_num 1283) 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:3836: finishing write tid 1 to smithi08434663-50 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:3836: finishing write tid 2 to smithi08434663-50 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:3836: finishing write tid 3 to smithi08434663-50 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:3836: finishing write tid 4 to smithi08434663-50 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:3836: finishing write tid 5 to smithi08434663-50 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 2231 (ObjNum 1284 snap 431 seq_num 1284) dirty exists 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:3836: left oid 50 (ObjNum 1284 snap 431 seq_num 1284) 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:3837: finishing write tid 1 to smithi08434663-33 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:3837: finishing write tid 2 to smithi08434663-33 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:3837: finishing write tid 3 to smithi08434663-33 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:3837: finishing write tid 4 to smithi08434663-33 2024-09-18T14:17:48.406 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 1752 (ObjNum 1285 snap 431 seq_num 1285) dirty exists 2024-09-18T14:17:48.407 INFO:tasks.rados.rados.0.smithi084.stdout:3837: left oid 33 (ObjNum 1285 snap 431 seq_num 1285) 2024-09-18T14:17:48.407 INFO:tasks.rados.rados.0.smithi084.stdout:3833: done (5 left) 2024-09-18T14:17:48.407 INFO:tasks.rados.rados.0.smithi084.stdout:3834: done (4 left) 2024-09-18T14:17:48.407 INFO:tasks.rados.rados.0.smithi084.stdout:3835: done (3 left) 2024-09-18T14:17:48.407 INFO:tasks.rados.rados.0.smithi084.stdout:3836: done (2 left) 2024-09-18T14:17:48.407 INFO:tasks.rados.rados.0.smithi084.stdout:3837: done (1 left) 2024-09-18T14:17:48.407 INFO:tasks.rados.rados.0.smithi084.stdout:3840: delete oid 26 current snap is 431 2024-09-18T14:17:48.439 INFO:tasks.rados.rados.0.smithi084.stdout:3839: finishing copy_from racing read to smithi08434663-1 2024-09-18T14:17:48.445 INFO:tasks.rados.rados.0.smithi084.stdout:3840: done (1 left) 2024-09-18T14:17:48.445 INFO:tasks.rados.rados.0.smithi084.stdout:3841: write (excl) oid 14 current snap is 431 2024-09-18T14:17:48.445 INFO:tasks.rados.rados.0.smithi084.stdout:3841: seq_num 1287 ranges {430609=509120,1478796=736116,2909627=602257} 2024-09-18T14:17:48.450 INFO:tasks.rados.rados.0.smithi084.stdout:3841: writing smithi08434663-14 from 430609 to 939729 tid 1 2024-09-18T14:17:48.457 INFO:tasks.rados.rados.0.smithi084.stdout:3841: writing smithi08434663-14 from 1478796 to 2214912 tid 2 2024-09-18T14:17:48.465 INFO:tasks.rados.rados.0.smithi084.stdout:3841: writing smithi08434663-14 from 2909627 to 3511884 tid 3 2024-09-18T14:17:48.465 INFO:tasks.rados.rados.0.smithi084.stdout:3842: write (excl) oid 19 current snap is 431 2024-09-18T14:17:48.465 INFO:tasks.rados.rados.0.smithi084.stdout:3839: finishing copy_from to smithi08434663-1 2024-09-18T14:17:48.465 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 1438 (ObjNum 1253 snap 414 seq_num 1253) dirty exists 2024-09-18T14:17:48.465 INFO:tasks.rados.rados.0.smithi084.stdout:3841: finishing write tid 1 to smithi08434663-14 2024-09-18T14:17:48.465 INFO:tasks.rados.rados.0.smithi084.stdout:3841: finishing write tid 2 to smithi08434663-14 2024-09-18T14:17:48.465 INFO:tasks.rados.rados.0.smithi084.stdout:3842: seq_num 1288 ranges {447416=584369,1537278=479774,2537050=529882,3524784=122632} 2024-09-18T14:17:48.470 INFO:tasks.rados.rados.0.smithi084.stdout:3842: writing smithi08434663-19 from 447416 to 1031785 tid 1 2024-09-18T14:17:48.476 INFO:tasks.rados.rados.0.smithi084.stdout:3842: writing smithi08434663-19 from 1537278 to 2017052 tid 2 2024-09-18T14:17:48.482 INFO:tasks.rados.rados.0.smithi084.stdout:3842: writing smithi08434663-19 from 2537050 to 3066932 tid 3 2024-09-18T14:17:48.485 INFO:tasks.rados.rados.0.smithi084.stdout:3842: writing smithi08434663-19 from 3524784 to 3647416 tid 4 2024-09-18T14:17:48.485 INFO:tasks.rados.rados.0.smithi084.stdout:3839: done (2 left) 2024-09-18T14:17:48.486 INFO:tasks.rados.rados.0.smithi084.stdout:3843: read oid 23 snap -1 2024-09-18T14:17:48.486 INFO:tasks.rados.rados.0.smithi084.stdout:3843: expect (ObjNum 1271 snap 424 seq_num 1271) 2024-09-18T14:17:48.486 INFO:tasks.rados.rados.0.smithi084.stdout:3844: snap_create 2024-09-18T14:17:48.486 INFO:tasks.rados.rados.0.smithi084.stdout:3841: finishing write tid 3 to smithi08434663-14 2024-09-18T14:17:48.486 INFO:tasks.rados.rados.0.smithi084.stdout:3841: finishing write tid 4 to smithi08434663-14 2024-09-18T14:17:48.486 INFO:tasks.rados.rados.0.smithi084.stdout:3841: finishing write tid 5 to smithi08434663-14 2024-09-18T14:17:48.486 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 2244 (ObjNum 1287 snap 431 seq_num 1287) dirty exists 2024-09-18T14:17:48.486 INFO:tasks.rados.rados.0.smithi084.stdout:3841: left oid 14 (ObjNum 1287 snap 431 seq_num 1287) 2024-09-18T14:17:48.486 INFO:tasks.rados.rados.0.smithi084.stdout:3842: finishing write tid 1 to smithi08434663-19 2024-09-18T14:17:48.486 INFO:tasks.rados.rados.0.smithi084.stdout:3842: finishing write tid 2 to smithi08434663-19 2024-09-18T14:17:48.486 INFO:tasks.rados.rados.0.smithi084.stdout:3842: finishing write tid 3 to smithi08434663-19 2024-09-18T14:17:48.488 INFO:tasks.rados.rados.0.smithi084.stdout:3842: finishing write tid 4 to smithi08434663-19 2024-09-18T14:17:48.488 INFO:tasks.rados.rados.0.smithi084.stdout:3842: finishing write tid 5 to smithi08434663-19 2024-09-18T14:17:48.488 INFO:tasks.rados.rados.0.smithi084.stdout:3842: finishing write tid 6 to smithi08434663-19 2024-09-18T14:17:48.488 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 19 v 2249 (ObjNum 1288 snap 431 seq_num 1288) dirty exists 2024-09-18T14:17:48.488 INFO:tasks.rados.rados.0.smithi084.stdout:3842: left oid 19 (ObjNum 1288 snap 431 seq_num 1288) 2024-09-18T14:17:48.494 INFO:tasks.rados.rados.0.smithi084.stdout:3843: expect (ObjNum 1271 snap 424 seq_num 1271) 2024-09-18T14:17:49.404 INFO:tasks.rados.rados.0.smithi084.stdout:3841: done (3 left) 2024-09-18T14:17:49.404 INFO:tasks.rados.rados.0.smithi084.stdout:3842: done (2 left) 2024-09-18T14:17:49.404 INFO:tasks.rados.rados.0.smithi084.stdout:3843: done (1 left) 2024-09-18T14:17:49.404 INFO:tasks.rados.rados.0.smithi084.stdout:3844: done (0 left) 2024-09-18T14:17:49.405 INFO:tasks.rados.rados.0.smithi084.stdout:3845: copy_from oid 34 from oid 13 current snap is 432 2024-09-18T14:17:49.405 INFO:tasks.rados.rados.0.smithi084.stdout:3846: rollback oid 18 current snap is 432 2024-09-18T14:17:49.405 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 18 to 360 2024-09-18T14:17:49.405 INFO:tasks.rados.rados.0.smithi084.stdout:3847: snap_create 2024-09-18T14:17:49.407 INFO:tasks.rados.rados.0.smithi084.stdout:3845: finishing copy_from to smithi08434663-34 2024-09-18T14:17:49.407 INFO:tasks.rados.rados.0.smithi084.stdout:3845: got expected ENOENT (src dne) 2024-09-18T14:17:49.407 INFO:tasks.rados.rados.0.smithi084.stdout:3845: finishing copy_from racing read to smithi08434663-34 2024-09-18T14:17:49.410 INFO:tasks.rados.rados.0.smithi084.stdout:3846: finishing rollback tid 0 to smithi08434663-18 2024-09-18T14:17:49.410 INFO:tasks.rados.rados.0.smithi084.stdout:3846: finishing rollback tid 1 to smithi08434663-18 2024-09-18T14:17:49.412 INFO:tasks.rados.rados.0.smithi084.stdout:3846: finishing rollback tid 2 to smithi08434663-18 2024-09-18T14:17:49.412 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 1593 (ObjNum 937 snap 299 seq_num 937) dirty exists 2024-09-18T14:17:50.416 INFO:tasks.rados.rados.0.smithi084.stdout:3845: done (2 left) 2024-09-18T14:17:50.416 INFO:tasks.rados.rados.0.smithi084.stdout:3846: done (1 left) 2024-09-18T14:17:50.416 INFO:tasks.rados.rados.0.smithi084.stdout:3847: done (0 left) 2024-09-18T14:17:50.416 INFO:tasks.rados.rados.0.smithi084.stdout:3848: copy_from oid 4 from oid 13 current snap is 433 2024-09-18T14:17:50.416 INFO:tasks.rados.rados.0.smithi084.stdout:3849: snap_create 2024-09-18T14:17:50.418 INFO:tasks.rados.rados.0.smithi084.stdout:3848: finishing copy_from to smithi08434663-4 2024-09-18T14:17:50.418 INFO:tasks.rados.rados.0.smithi084.stdout:3848: got expected ENOENT (src dne) 2024-09-18T14:17:50.418 INFO:tasks.rados.rados.0.smithi084.stdout:3848: finishing copy_from racing read to smithi08434663-4 2024-09-18T14:17:51.421 INFO:tasks.rados.rados.0.smithi084.stdout:3848: done (1 left) 2024-09-18T14:17:51.421 INFO:tasks.rados.rados.0.smithi084.stdout:3849: done (0 left) 2024-09-18T14:17:51.421 INFO:tasks.rados.rados.0.smithi084.stdout:3850: copy_from oid 50 from oid 38 current snap is 434 2024-09-18T14:17:51.421 INFO:tasks.rados.rados.0.smithi084.stdout:3851: rollback oid 47 current snap is 434 2024-09-18T14:17:51.421 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 47 to 396 2024-09-18T14:17:51.421 INFO:tasks.rados.rados.0.smithi084.stdout:3852: copy_from oid 3 from oid 44 current snap is 434 2024-09-18T14:17:51.422 INFO:tasks.rados.rados.0.smithi084.stdout:3853: snap_remove snap 419 2024-09-18T14:17:52.428 INFO:tasks.rados.rados.0.smithi084.stdout:3853: done (3 left) 2024-09-18T14:17:52.428 INFO:tasks.rados.rados.0.smithi084.stdout:3854: snap_create 2024-09-18T14:17:52.428 INFO:tasks.rados.rados.0.smithi084.stdout:3851: finishing rollback tid 0 to smithi08434663-47 2024-09-18T14:17:52.428 INFO:tasks.rados.rados.0.smithi084.stdout:3851: finishing rollback tid 1 to smithi08434663-47 2024-09-18T14:17:52.428 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 2062 (ObjNum 129 snap 0 seq_num 2495315104) dirty dne 2024-09-18T14:17:52.428 INFO:tasks.rados.rados.0.smithi084.stdout:3850: finishing copy_from racing read to smithi08434663-50 2024-09-18T14:17:52.428 INFO:tasks.rados.rados.0.smithi084.stdout:3852: finishing copy_from racing read to smithi08434663-3 2024-09-18T14:17:52.428 INFO:tasks.rados.rados.0.smithi084.stdout:3852: finishing copy_from to smithi08434663-3 2024-09-18T14:17:52.428 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 2053 (ObjNum 991 snap 321 seq_num 991) dirty exists 2024-09-18T14:17:52.428 INFO:tasks.rados.rados.0.smithi084.stdout:3850: finishing copy_from to smithi08434663-50 2024-09-18T14:17:52.429 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 2043 (ObjNum 1279 snap 429 seq_num 1279) dirty exists 2024-09-18T14:17:53.374 INFO:tasks.rados.rados.0.smithi084.stdout:3850: done (3 left) 2024-09-18T14:17:53.375 INFO:tasks.rados.rados.0.smithi084.stdout:3851: done (2 left) 2024-09-18T14:17:53.375 INFO:tasks.rados.rados.0.smithi084.stdout:3852: done (1 left) 2024-09-18T14:17:53.375 INFO:tasks.rados.rados.0.smithi084.stdout:3854: done (0 left) 2024-09-18T14:17:53.375 INFO:tasks.rados.rados.0.smithi084.stdout:3855: snap_create 2024-09-18T14:17:54.384 INFO:tasks.rados.rados.0.smithi084.stdout:3855: done (0 left) 2024-09-18T14:17:54.384 INFO:tasks.rados.rados.0.smithi084.stdout:3856: rollback oid 23 current snap is 436 2024-09-18T14:17:54.384 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 23 to 411 2024-09-18T14:17:54.384 INFO:tasks.rados.rados.0.smithi084.stdout:3857: snap_create 2024-09-18T14:17:54.389 INFO:tasks.rados.rados.0.smithi084.stdout:3856: finishing rollback tid 0 to smithi08434663-23 2024-09-18T14:17:54.389 INFO:tasks.rados.rados.0.smithi084.stdout:3856: finishing rollback tid 1 to smithi08434663-23 2024-09-18T14:17:54.389 INFO:tasks.rados.rados.0.smithi084.stdout:3856: finishing rollback tid 2 to smithi08434663-23 2024-09-18T14:17:54.389 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1563 (ObjNum 1241 snap 409 seq_num 1241) dirty exists 2024-09-18T14:17:55.386 INFO:tasks.rados.rados.0.smithi084.stdout:3856: done (1 left) 2024-09-18T14:17:55.386 INFO:tasks.rados.rados.0.smithi084.stdout:3857: done (0 left) 2024-09-18T14:17:55.386 INFO:tasks.rados.rados.0.smithi084.stdout:3858: rollback oid 49 current snap is 437 2024-09-18T14:17:55.386 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 49 to 407 2024-09-18T14:17:55.386 INFO:tasks.rados.rados.0.smithi084.stdout:3859: rollback oid 17 current snap is 437 2024-09-18T14:17:55.387 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 17 to 417 2024-09-18T14:17:55.387 INFO:tasks.rados.rados.0.smithi084.stdout:3860: delete oid 15 current snap is 437 2024-09-18T14:17:55.392 INFO:tasks.rados.rados.0.smithi084.stdout:3860: done (2 left) 2024-09-18T14:17:55.392 INFO:tasks.rados.rados.0.smithi084.stdout:3861: copy_from oid 41 from oid 46 current snap is 437 2024-09-18T14:17:55.392 INFO:tasks.rados.rados.0.smithi084.stdout:3862: snap_create 2024-09-18T14:17:55.393 INFO:tasks.rados.rados.0.smithi084.stdout:3858: finishing rollback tid 0 to smithi08434663-49 2024-09-18T14:17:55.394 INFO:tasks.rados.rados.0.smithi084.stdout:3858: finishing rollback tid 1 to smithi08434663-49 2024-09-18T14:17:55.394 INFO:tasks.rados.rados.0.smithi084.stdout:3861: finishing copy_from to smithi08434663-41 2024-09-18T14:17:55.394 INFO:tasks.rados.rados.0.smithi084.stdout:3861: got expected ENOENT (src dne) 2024-09-18T14:17:55.394 INFO:tasks.rados.rados.0.smithi084.stdout:3858: finishing rollback tid 2 to smithi08434663-49 2024-09-18T14:17:55.394 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 49 v 1755 (ObjNum 1163 snap 379 seq_num 1163) dirty exists 2024-09-18T14:17:55.394 INFO:tasks.rados.rados.0.smithi084.stdout:3859: finishing rollback tid 0 to smithi08434663-17 2024-09-18T14:17:55.394 INFO:tasks.rados.rados.0.smithi084.stdout:3861: finishing copy_from racing read to smithi08434663-41 2024-09-18T14:17:55.394 INFO:tasks.rados.rados.0.smithi084.stdout:3859: finishing rollback tid 1 to smithi08434663-17 2024-09-18T14:17:55.394 INFO:tasks.rados.rados.0.smithi084.stdout:3859: finishing rollback tid 2 to smithi08434663-17 2024-09-18T14:17:55.395 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1885 (ObjNum 1242 snap 409 seq_num 1242) dirty exists 2024-09-18T14:17:56.392 INFO:tasks.rados.rados.0.smithi084.stdout:3858: done (3 left) 2024-09-18T14:17:56.392 INFO:tasks.rados.rados.0.smithi084.stdout:3859: done (2 left) 2024-09-18T14:17:56.392 INFO:tasks.rados.rados.0.smithi084.stdout:3861: done (1 left) 2024-09-18T14:17:56.392 INFO:tasks.rados.rados.0.smithi084.stdout:3862: done (0 left) 2024-09-18T14:17:56.392 INFO:tasks.rados.rados.0.smithi084.stdout:3863: read oid 44 snap 411 2024-09-18T14:17:56.392 INFO:tasks.rados.rados.0.smithi084.stdout:3863: expect (ObjNum 1145 snap 369 seq_num 1145) 2024-09-18T14:17:56.392 INFO:tasks.rados.rados.0.smithi084.stdout:3864: read oid 11 snap -1 2024-09-18T14:17:56.392 INFO:tasks.rados.rados.0.smithi084.stdout:3864: expect (ObjNum 1251 snap 411 seq_num 1251) 2024-09-18T14:17:56.393 INFO:tasks.rados.rados.0.smithi084.stdout:3865: copy_from oid 17 from oid 32 current snap is 438 2024-09-18T14:17:56.393 INFO:tasks.rados.rados.0.smithi084.stdout:3866: read oid 13 snap -1 2024-09-18T14:17:56.393 INFO:tasks.rados.rados.0.smithi084.stdout:3866: expect deleted 2024-09-18T14:17:56.393 INFO:tasks.rados.rados.0.smithi084.stdout:3867: copy_from oid 12 from oid 14 current snap is 438 2024-09-18T14:17:56.393 INFO:tasks.rados.rados.0.smithi084.stdout:3868: write oid 42 current snap is 438 2024-09-18T14:17:56.393 INFO:tasks.rados.rados.0.smithi084.stdout:3868: seq_num 1290 ranges {481030=734867,1937889=483443,2875543=552426} 2024-09-18T14:17:56.399 INFO:tasks.rados.rados.0.smithi084.stdout:3868: writing smithi08434663-42 from 481030 to 1215897 tid 1 2024-09-18T14:17:56.406 INFO:tasks.rados.rados.0.smithi084.stdout:3868: writing smithi08434663-42 from 1937889 to 2421332 tid 2 2024-09-18T14:17:56.411 INFO:tasks.rados.rados.0.smithi084.stdout:3868: writing smithi08434663-42 from 2875543 to 3427969 tid 3 2024-09-18T14:17:56.411 INFO:tasks.rados.rados.0.smithi084.stdout:3869: snap_remove snap 437 2024-09-18T14:17:57.398 INFO:tasks.rados.rados.0.smithi084.stdout:3869: done (6 left) 2024-09-18T14:17:57.398 INFO:tasks.rados.rados.0.smithi084.stdout:3870: rollback oid 46 current snap is 438 2024-09-18T14:17:57.398 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 46 to 410 2024-09-18T14:17:57.398 INFO:tasks.rados.rados.0.smithi084.stdout:3865: finishing copy_from to smithi08434663-17 2024-09-18T14:17:57.398 INFO:tasks.rados.rados.0.smithi084.stdout:3865: got expected ENOENT (src dne) 2024-09-18T14:17:57.398 INFO:tasks.rados.rados.0.smithi084.stdout:3865: finishing copy_from racing read to smithi08434663-17 2024-09-18T14:17:57.398 INFO:tasks.rados.rados.0.smithi084.stdout:3864: expect (ObjNum 1251 snap 411 seq_num 1251) 2024-09-18T14:17:57.498 INFO:tasks.rados.rados.0.smithi084.stdout:3863: expect (ObjNum 1145 snap 369 seq_num 1145) 2024-09-18T14:17:57.597 INFO:tasks.rados.rados.0.smithi084.stdout:3867: finishing copy_from racing read to smithi08434663-12 2024-09-18T14:17:57.598 INFO:tasks.rados.rados.0.smithi084.stdout:3868: finishing write tid 1 to smithi08434663-42 2024-09-18T14:17:57.598 INFO:tasks.rados.rados.0.smithi084.stdout:3868: finishing write tid 2 to smithi08434663-42 2024-09-18T14:17:57.598 INFO:tasks.rados.rados.0.smithi084.stdout:3867: finishing copy_from to smithi08434663-12 2024-09-18T14:17:57.598 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 2244 (ObjNum 1287 snap 431 seq_num 1287) dirty exists 2024-09-18T14:17:57.598 INFO:tasks.rados.rados.0.smithi084.stdout:3868: finishing write tid 3 to smithi08434663-42 2024-09-18T14:17:57.598 INFO:tasks.rados.rados.0.smithi084.stdout:3868: finishing write tid 4 to smithi08434663-42 2024-09-18T14:17:57.598 INFO:tasks.rados.rados.0.smithi084.stdout:3868: finishing write tid 5 to smithi08434663-42 2024-09-18T14:17:57.598 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 42 v 2032 (ObjNum 1290 snap 438 seq_num 1290) dirty exists 2024-09-18T14:17:57.598 INFO:tasks.rados.rados.0.smithi084.stdout:3868: left oid 42 (ObjNum 1290 snap 438 seq_num 1290) 2024-09-18T14:17:57.598 INFO:tasks.rados.rados.0.smithi084.stdout:3863: done (6 left) 2024-09-18T14:17:57.599 INFO:tasks.rados.rados.0.smithi084.stdout:3864: done (5 left) 2024-09-18T14:17:57.599 INFO:tasks.rados.rados.0.smithi084.stdout:3865: done (4 left) 2024-09-18T14:17:57.599 INFO:tasks.rados.rados.0.smithi084.stdout:3866: done (3 left) 2024-09-18T14:17:57.599 INFO:tasks.rados.rados.0.smithi084.stdout:3867: done (2 left) 2024-09-18T14:17:57.599 INFO:tasks.rados.rados.0.smithi084.stdout:3868: done (1 left) 2024-09-18T14:17:57.599 INFO:tasks.rados.rados.0.smithi084.stdout:3871: write (excl) oid 25 current snap is 438 2024-09-18T14:17:57.599 INFO:tasks.rados.rados.0.smithi084.stdout:3870: finishing rollback tid 1 to smithi08434663-46 2024-09-18T14:17:57.599 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 46 v 2041 (ObjNum 33 snap 0 seq_num 3) dirty dne 2024-09-18T14:17:57.599 INFO:tasks.rados.rados.0.smithi084.stdout:3871: seq_num 1291 ranges {497837=410116,1596371=743454,2902966=763698} 2024-09-18T14:17:57.602 INFO:tasks.rados.rados.0.smithi084.stdout:3871: writing smithi08434663-25 from 497837 to 907953 tid 1 2024-09-18T14:17:57.610 INFO:tasks.rados.rados.0.smithi084.stdout:3871: writing smithi08434663-25 from 1596371 to 2339825 tid 2 2024-09-18T14:17:57.618 INFO:tasks.rados.rados.0.smithi084.stdout:3871: writing smithi08434663-25 from 2902966 to 3666664 tid 3 2024-09-18T14:17:57.618 INFO:tasks.rados.rados.0.smithi084.stdout:3870: done (1 left) 2024-09-18T14:17:57.618 INFO:tasks.rados.rados.0.smithi084.stdout:3872: write oid 50 current snap is 438 2024-09-18T14:17:57.618 INFO:tasks.rados.rados.0.smithi084.stdout:3872: seq_num 1292 ranges {514644=485365,1654853=487112,2930389=574970} 2024-09-18T14:17:57.623 INFO:tasks.rados.rados.0.smithi084.stdout:3872: writing smithi08434663-50 from 514644 to 1000009 tid 1 2024-09-18T14:17:57.629 INFO:tasks.rados.rados.0.smithi084.stdout:3872: writing smithi08434663-50 from 1654853 to 2141965 tid 2 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:3872: writing smithi08434663-50 from 2930389 to 3505359 tid 3 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:3873: read oid 42 snap 421 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:3873: expect (ObjNum 1054 snap 336 seq_num 1054) 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:3874: read oid 31 snap 432 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:3874: expect (ObjNum 1281 snap 430 seq_num 1281) 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:3875: delete oid 6 current snap is 438 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:3871: finishing write tid 1 to smithi08434663-25 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:3871: finishing write tid 2 to smithi08434663-25 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:3871: finishing write tid 3 to smithi08434663-25 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:3871: finishing write tid 4 to smithi08434663-25 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:3871: finishing write tid 5 to smithi08434663-25 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 25 v 2061 (ObjNum 1291 snap 438 seq_num 1291) dirty exists 2024-09-18T14:17:57.638 INFO:tasks.rados.rados.0.smithi084.stdout:3871: left oid 25 (ObjNum 1291 snap 438 seq_num 1291) 2024-09-18T14:17:57.639 INFO:tasks.rados.rados.0.smithi084.stdout:3872: finishing write tid 1 to smithi08434663-50 2024-09-18T14:17:57.639 INFO:tasks.rados.rados.0.smithi084.stdout:3872: finishing write tid 2 to smithi08434663-50 2024-09-18T14:17:57.642 INFO:tasks.rados.rados.0.smithi084.stdout:3871: done (4 left) 2024-09-18T14:17:57.642 INFO:tasks.rados.rados.0.smithi084.stdout:3875: done (3 left) 2024-09-18T14:17:57.643 INFO:tasks.rados.rados.0.smithi084.stdout:3876: delete oid 40 current snap is 438 2024-09-18T14:17:57.644 INFO:tasks.rados.rados.0.smithi084.stdout:3872: finishing write tid 3 to smithi08434663-50 2024-09-18T14:17:57.644 INFO:tasks.rados.rados.0.smithi084.stdout:3872: finishing write tid 4 to smithi08434663-50 2024-09-18T14:17:57.644 INFO:tasks.rados.rados.0.smithi084.stdout:3872: finishing write tid 5 to smithi08434663-50 2024-09-18T14:17:57.644 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 50 v 2264 (ObjNum 1292 snap 438 seq_num 1292) dirty exists 2024-09-18T14:17:57.644 INFO:tasks.rados.rados.0.smithi084.stdout:3872: left oid 50 (ObjNum 1292 snap 438 seq_num 1292) 2024-09-18T14:17:57.646 INFO:tasks.rados.rados.0.smithi084.stdout:3872: done (3 left) 2024-09-18T14:17:57.646 INFO:tasks.rados.rados.0.smithi084.stdout:3876: done (2 left) 2024-09-18T14:17:57.646 INFO:tasks.rados.rados.0.smithi084.stdout:3877: write oid 35 current snap is 438 2024-09-18T14:17:57.646 INFO:tasks.rados.rados.0.smithi084.stdout:3877: seq_num 1295 ranges {565065=427465,1430299=634439,2496305=525139,3472104=292961} 2024-09-18T14:17:57.651 INFO:tasks.rados.rados.0.smithi084.stdout:3877: writing smithi08434663-35 from 565065 to 992530 tid 1 2024-09-18T14:17:57.657 INFO:tasks.rados.rados.0.smithi084.stdout:3877: writing smithi08434663-35 from 1430299 to 2064738 tid 2 2024-09-18T14:17:57.662 INFO:tasks.rados.rados.0.smithi084.stdout:3877: writing smithi08434663-35 from 2496305 to 3021444 tid 3 2024-09-18T14:17:57.667 INFO:tasks.rados.rados.0.smithi084.stdout:3877: writing smithi08434663-35 from 3472104 to 3765065 tid 4 2024-09-18T14:17:57.667 INFO:tasks.rados.rados.0.smithi084.stdout:3878: snap_create 2024-09-18T14:17:57.667 INFO:tasks.rados.rados.0.smithi084.stdout:3873: expect (ObjNum 1054 snap 336 seq_num 1054) 2024-09-18T14:17:57.966 INFO:tasks.rados.rados.0.smithi084.stdout:3877: finishing write tid 1 to smithi08434663-35 2024-09-18T14:17:57.966 INFO:tasks.rados.rados.0.smithi084.stdout:3877: finishing write tid 2 to smithi08434663-35 2024-09-18T14:17:57.967 INFO:tasks.rados.rados.0.smithi084.stdout:3874: expect (ObjNum 1281 snap 430 seq_num 1281) 2024-09-18T14:17:58.197 INFO:tasks.rados.rados.0.smithi084.stdout:3877: finishing write tid 3 to smithi08434663-35 2024-09-18T14:17:58.198 INFO:tasks.rados.rados.0.smithi084.stdout:3877: finishing write tid 4 to smithi08434663-35 2024-09-18T14:17:58.198 INFO:tasks.rados.rados.0.smithi084.stdout:3877: finishing write tid 5 to smithi08434663-35 2024-09-18T14:17:58.198 INFO:tasks.rados.rados.0.smithi084.stdout:3877: finishing write tid 6 to smithi08434663-35 2024-09-18T14:17:58.198 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 35 v 2062 (ObjNum 1295 snap 438 seq_num 1295) dirty exists 2024-09-18T14:17:58.198 INFO:tasks.rados.rados.0.smithi084.stdout:3877: left oid 35 (ObjNum 1295 snap 438 seq_num 1295) 2024-09-18T14:17:58.404 INFO:tasks.rados.rados.0.smithi084.stdout:3873: done (3 left) 2024-09-18T14:17:58.404 INFO:tasks.rados.rados.0.smithi084.stdout:3874: done (2 left) 2024-09-18T14:17:58.404 INFO:tasks.rados.rados.0.smithi084.stdout:3877: done (1 left) 2024-09-18T14:17:58.404 INFO:tasks.rados.rados.0.smithi084.stdout:3878: done (0 left) 2024-09-18T14:17:58.404 INFO:tasks.rados.rados.0.smithi084.stdout:3879: snap_remove snap 428 2024-09-18T14:17:59.409 INFO:tasks.rados.rados.0.smithi084.stdout:3879: done (0 left) 2024-09-18T14:17:59.410 INFO:tasks.rados.rados.0.smithi084.stdout:3880: copy_from oid 2 from oid 8 current snap is 439 2024-09-18T14:17:59.410 INFO:tasks.rados.rados.0.smithi084.stdout:3881: snap_create 2024-09-18T14:17:59.424 INFO:tasks.rados.rados.0.smithi084.stdout:3880: finishing copy_from racing read to smithi08434663-2 2024-09-18T14:17:59.426 INFO:tasks.rados.rados.0.smithi084.stdout:3880: finishing copy_from to smithi08434663-2 2024-09-18T14:17:59.426 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 1438 (ObjNum 1253 snap 414 seq_num 1253) dirty exists 2024-09-18T14:18:00.415 INFO:tasks.rados.rados.0.smithi084.stdout:3880: done (1 left) 2024-09-18T14:18:00.415 INFO:tasks.rados.rados.0.smithi084.stdout:3881: done (0 left) 2024-09-18T14:18:00.415 INFO:tasks.rados.rados.0.smithi084.stdout:3882: read oid 13 snap 335 2024-09-18T14:18:00.415 INFO:tasks.rados.rados.0.smithi084.stdout:3882: expect deleted 2024-09-18T14:18:00.415 INFO:tasks.rados.rados.0.smithi084.stdout:3883: write (excl) oid 1 current snap is 440 2024-09-18T14:18:00.416 INFO:tasks.rados.rados.0.smithi084.stdout:3883: seq_num 1296 ranges {581872=502714,1488781=494450,2640081=736411} 2024-09-18T14:18:00.420 INFO:tasks.rados.rados.0.smithi084.stdout:3883: writing smithi08434663-1 from 581872 to 1084586 tid 1 2024-09-18T14:18:00.425 INFO:tasks.rados.rados.0.smithi084.stdout:3883: writing smithi08434663-1 from 1488781 to 1983231 tid 2 2024-09-18T14:18:00.433 INFO:tasks.rados.rados.0.smithi084.stdout:3883: writing smithi08434663-1 from 2640081 to 3376492 tid 3 2024-09-18T14:18:00.433 INFO:tasks.rados.rados.0.smithi084.stdout:3884: delete oid 32 current snap is 440 2024-09-18T14:18:00.434 INFO:tasks.rados.rados.0.smithi084.stdout:3883: finishing write tid 1 to smithi08434663-1 2024-09-18T14:18:00.434 INFO:tasks.rados.rados.0.smithi084.stdout:3883: finishing write tid 2 to smithi08434663-1 2024-09-18T14:18:00.438 INFO:tasks.rados.rados.0.smithi084.stdout:3883: finishing write tid 3 to smithi08434663-1 2024-09-18T14:18:00.439 INFO:tasks.rados.rados.0.smithi084.stdout:3883: finishing write tid 4 to smithi08434663-1 2024-09-18T14:18:00.439 INFO:tasks.rados.rados.0.smithi084.stdout:3883: finishing write tid 5 to smithi08434663-1 2024-09-18T14:18:00.439 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 1794 (ObjNum 1296 snap 440 seq_num 1296) dirty exists 2024-09-18T14:18:00.439 INFO:tasks.rados.rados.0.smithi084.stdout:3883: left oid 1 (ObjNum 1296 snap 440 seq_num 1296) 2024-09-18T14:18:00.439 INFO:tasks.rados.rados.0.smithi084.stdout:3882: done (2 left) 2024-09-18T14:18:00.439 INFO:tasks.rados.rados.0.smithi084.stdout:3883: done (1 left) 2024-09-18T14:18:00.439 INFO:tasks.rados.rados.0.smithi084.stdout:3884: done (0 left) 2024-09-18T14:18:00.439 INFO:tasks.rados.rados.0.smithi084.stdout:3885: rollback oid 3 current snap is 440 2024-09-18T14:18:00.439 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 3 to 335 2024-09-18T14:18:00.439 INFO:tasks.rados.rados.0.smithi084.stdout:3886: delete oid 42 current snap is 440 2024-09-18T14:18:00.443 INFO:tasks.rados.rados.0.smithi084.stdout:3885: finishing rollback tid 0 to smithi08434663-3 2024-09-18T14:18:00.444 INFO:tasks.rados.rados.0.smithi084.stdout:3885: finishing rollback tid 1 to smithi08434663-3 2024-09-18T14:18:00.444 INFO:tasks.rados.rados.0.smithi084.stdout:3885: finishing rollback tid 2 to smithi08434663-3 2024-09-18T14:18:00.444 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 2056 (ObjNum 967 snap 308 seq_num 967) dirty exists 2024-09-18T14:18:00.445 INFO:tasks.rados.rados.0.smithi084.stdout:3885: done (1 left) 2024-09-18T14:18:00.445 INFO:tasks.rados.rados.0.smithi084.stdout:3886: done (0 left) 2024-09-18T14:18:00.445 INFO:tasks.rados.rados.0.smithi084.stdout:3887: snap_remove snap 436 2024-09-18T14:18:01.419 INFO:tasks.rados.rados.0.smithi084.stdout:3887: done (0 left) 2024-09-18T14:18:01.419 INFO:tasks.rados.rados.0.smithi084.stdout:3888: write (excl) oid 48 current snap is 440 2024-09-18T14:18:01.419 INFO:tasks.rados.rados.0.smithi084.stdout:3888: seq_num 1299 ranges {632293=728461,1947874=641777,3005997=686580} 2024-09-18T14:18:01.427 INFO:tasks.rados.rados.0.smithi084.stdout:3888: writing smithi08434663-48 from 632293 to 1360754 tid 1 2024-09-18T14:18:01.433 INFO:tasks.rados.rados.0.smithi084.stdout:3888: writing smithi08434663-48 from 1947874 to 2589651 tid 2 2024-09-18T14:18:01.440 INFO:tasks.rados.rados.0.smithi084.stdout:3888: writing smithi08434663-48 from 3005997 to 3692577 tid 3 2024-09-18T14:18:01.440 INFO:tasks.rados.rados.0.smithi084.stdout:3889: write oid 23 current snap is 440 2024-09-18T14:18:01.440 INFO:tasks.rados.rados.0.smithi084.stdout:3888: finishing write tid 1 to smithi08434663-48 2024-09-18T14:18:01.440 INFO:tasks.rados.rados.0.smithi084.stdout:3889: seq_num 1300 ranges {649100=403710,1606356=501788,2633420=497852,3636005=213095} 2024-09-18T14:18:01.445 INFO:tasks.rados.rados.0.smithi084.stdout:3889: writing smithi08434663-23 from 649100 to 1052810 tid 1 2024-09-18T14:18:01.451 INFO:tasks.rados.rados.0.smithi084.stdout:3889: writing smithi08434663-23 from 1606356 to 2108144 tid 2 2024-09-18T14:18:01.456 INFO:tasks.rados.rados.0.smithi084.stdout:3889: writing smithi08434663-23 from 2633420 to 3131272 tid 3 2024-09-18T14:18:01.461 INFO:tasks.rados.rados.0.smithi084.stdout:3889: writing smithi08434663-23 from 3636005 to 3849100 tid 4 2024-09-18T14:18:01.461 INFO:tasks.rados.rados.0.smithi084.stdout:3890: read oid 45 snap -1 2024-09-18T14:18:01.461 INFO:tasks.rados.rados.0.smithi084.stdout:3890: expect deleted 2024-09-18T14:18:01.461 INFO:tasks.rados.rados.0.smithi084.stdout:3891: write (excl) oid 26 current snap is 440 2024-09-18T14:18:01.461 INFO:tasks.rados.rados.0.smithi084.stdout:3888: finishing write tid 2 to smithi08434663-48 2024-09-18T14:18:01.461 INFO:tasks.rados.rados.0.smithi084.stdout:3888: finishing write tid 3 to smithi08434663-48 2024-09-18T14:18:01.461 INFO:tasks.rados.rados.0.smithi084.stdout:3888: finishing write tid 4 to smithi08434663-48 2024-09-18T14:18:01.462 INFO:tasks.rados.rados.0.smithi084.stdout:3891: seq_num 1301 ranges {665907=595312,1781191=645446,3177196=688711} 2024-09-18T14:18:01.467 INFO:tasks.rados.rados.0.smithi084.stdout:3891: writing smithi08434663-26 from 665907 to 1261219 tid 1 2024-09-18T14:18:01.474 INFO:tasks.rados.rados.0.smithi084.stdout:3891: writing smithi08434663-26 from 1781191 to 2426637 tid 2 2024-09-18T14:18:01.482 INFO:tasks.rados.rados.0.smithi084.stdout:3891: writing smithi08434663-26 from 3177196 to 3865907 tid 3 2024-09-18T14:18:01.482 INFO:tasks.rados.rados.0.smithi084.stdout:3892: snap_create 2024-09-18T14:18:01.482 INFO:tasks.rados.rados.0.smithi084.stdout:3888: finishing write tid 5 to smithi08434663-48 2024-09-18T14:18:01.482 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 2065 (ObjNum 1299 snap 440 seq_num 1299) dirty exists 2024-09-18T14:18:01.482 INFO:tasks.rados.rados.0.smithi084.stdout:3888: left oid 48 (ObjNum 1299 snap 440 seq_num 1299) 2024-09-18T14:18:01.482 INFO:tasks.rados.rados.0.smithi084.stdout:3889: finishing write tid 1 to smithi08434663-23 2024-09-18T14:18:01.483 INFO:tasks.rados.rados.0.smithi084.stdout:3889: finishing write tid 2 to smithi08434663-23 2024-09-18T14:18:01.483 INFO:tasks.rados.rados.0.smithi084.stdout:3889: finishing write tid 3 to smithi08434663-23 2024-09-18T14:18:01.483 INFO:tasks.rados.rados.0.smithi084.stdout:3889: finishing write tid 4 to smithi08434663-23 2024-09-18T14:18:01.483 INFO:tasks.rados.rados.0.smithi084.stdout:3889: finishing write tid 5 to smithi08434663-23 2024-09-18T14:18:01.483 INFO:tasks.rados.rados.0.smithi084.stdout:3889: finishing write tid 6 to smithi08434663-23 2024-09-18T14:18:01.483 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1569 (ObjNum 1300 snap 440 seq_num 1300) dirty exists 2024-09-18T14:18:01.483 INFO:tasks.rados.rados.0.smithi084.stdout:3889: left oid 23 (ObjNum 1300 snap 440 seq_num 1300) 2024-09-18T14:18:01.483 INFO:tasks.rados.rados.0.smithi084.stdout:3891: finishing write tid 1 to smithi08434663-26 2024-09-18T14:18:01.483 INFO:tasks.rados.rados.0.smithi084.stdout:3891: finishing write tid 2 to smithi08434663-26 2024-09-18T14:18:01.487 INFO:tasks.rados.rados.0.smithi084.stdout:3891: finishing write tid 3 to smithi08434663-26 2024-09-18T14:18:01.487 INFO:tasks.rados.rados.0.smithi084.stdout:3891: finishing write tid 4 to smithi08434663-26 2024-09-18T14:18:01.487 INFO:tasks.rados.rados.0.smithi084.stdout:3891: finishing write tid 5 to smithi08434663-26 2024-09-18T14:18:01.487 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 26 v 1798 (ObjNum 1301 snap 440 seq_num 1301) dirty exists 2024-09-18T14:18:01.487 INFO:tasks.rados.rados.0.smithi084.stdout:3891: left oid 26 (ObjNum 1301 snap 440 seq_num 1301) 2024-09-18T14:18:02.426 INFO:tasks.rados.rados.0.smithi084.stdout:3888: done (4 left) 2024-09-18T14:18:02.427 INFO:tasks.rados.rados.0.smithi084.stdout:3889: done (3 left) 2024-09-18T14:18:02.427 INFO:tasks.rados.rados.0.smithi084.stdout:3890: done (2 left) 2024-09-18T14:18:02.427 INFO:tasks.rados.rados.0.smithi084.stdout:3891: done (1 left) 2024-09-18T14:18:02.427 INFO:tasks.rados.rados.0.smithi084.stdout:3892: done (0 left) 2024-09-18T14:18:02.427 INFO:tasks.rados.rados.0.smithi084.stdout:3893: read oid 19 snap -1 2024-09-18T14:18:02.427 INFO:tasks.rados.rados.0.smithi084.stdout:3893: expect (ObjNum 1288 snap 431 seq_num 1288) 2024-09-18T14:18:02.427 INFO:tasks.rados.rados.0.smithi084.stdout:3894: read oid 25 snap -1 2024-09-18T14:18:02.427 INFO:tasks.rados.rados.0.smithi084.stdout:3894: expect (ObjNum 1291 snap 438 seq_num 1291) 2024-09-18T14:18:02.427 INFO:tasks.rados.rados.0.smithi084.stdout:3895: snap_create 2024-09-18T14:18:02.434 INFO:tasks.rados.rados.0.smithi084.stdout:3894: expect (ObjNum 1291 snap 438 seq_num 1291) 2024-09-18T14:18:02.601 INFO:tasks.rados.rados.0.smithi084.stdout:3893: expect (ObjNum 1288 snap 431 seq_num 1288) 2024-09-18T14:18:03.376 INFO:tasks.rados.rados.0.smithi084.stdout:3893: done (2 left) 2024-09-18T14:18:03.377 INFO:tasks.rados.rados.0.smithi084.stdout:3894: done (1 left) 2024-09-18T14:18:03.377 INFO:tasks.rados.rados.0.smithi084.stdout:3895: done (0 left) 2024-09-18T14:18:03.377 INFO:tasks.rados.rados.0.smithi084.stdout:3896: rollback oid 47 current snap is 442 2024-09-18T14:18:03.377 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 47 to 404 2024-09-18T14:18:03.377 INFO:tasks.rados.rados.0.smithi084.stdout:3897: copy_from oid 30 from oid 19 current snap is 442 2024-09-18T14:18:03.377 INFO:tasks.rados.rados.0.smithi084.stdout:3898: snap_create 2024-09-18T14:18:03.381 INFO:tasks.rados.rados.0.smithi084.stdout:3896: finishing rollback tid 1 to smithi08434663-47 2024-09-18T14:18:03.381 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 2062 (ObjNum 65 snap 0 seq_num 2494867120) dirty dne 2024-09-18T14:18:03.388 INFO:tasks.rados.rados.0.smithi084.stdout:3897: finishing copy_from racing read to smithi08434663-30 2024-09-18T14:18:03.392 INFO:tasks.rados.rados.0.smithi084.stdout:3897: finishing copy_from to smithi08434663-30 2024-09-18T14:18:03.392 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 2249 (ObjNum 1288 snap 431 seq_num 1288) dirty exists 2024-09-18T14:18:04.382 INFO:tasks.rados.rados.0.smithi084.stdout:3896: done (2 left) 2024-09-18T14:18:04.382 INFO:tasks.rados.rados.0.smithi084.stdout:3897: done (1 left) 2024-09-18T14:18:04.382 INFO:tasks.rados.rados.0.smithi084.stdout:3898: done (0 left) 2024-09-18T14:18:04.382 INFO:tasks.rados.rados.0.smithi084.stdout:3899: copy_from oid 4 from oid 13 current snap is 443 2024-09-18T14:18:04.383 INFO:tasks.rados.rados.0.smithi084.stdout:3900: write (excl) oid 14 current snap is 443 2024-09-18T14:18:04.383 INFO:tasks.rados.rados.0.smithi084.stdout:3900: seq_num 1302 ranges {682714=670561,2123320=505457,3088266=636749} 2024-09-18T14:18:04.389 INFO:tasks.rados.rados.0.smithi084.stdout:3900: writing smithi08434663-14 from 682714 to 1353275 tid 1 2024-09-18T14:18:04.396 INFO:tasks.rados.rados.0.smithi084.stdout:3900: writing smithi08434663-14 from 2123320 to 2628777 tid 2 2024-09-18T14:18:04.403 INFO:tasks.rados.rados.0.smithi084.stdout:3900: writing smithi08434663-14 from 3088266 to 3725015 tid 3 2024-09-18T14:18:04.403 INFO:tasks.rados.rados.0.smithi084.stdout:3901: copy_from oid 48 from oid 15 current snap is 443 2024-09-18T14:18:04.403 INFO:tasks.rados.rados.0.smithi084.stdout:3899: finishing copy_from to smithi08434663-4 2024-09-18T14:18:04.403 INFO:tasks.rados.rados.0.smithi084.stdout:3899: got expected ENOENT (src dne) 2024-09-18T14:18:04.403 INFO:tasks.rados.rados.0.smithi084.stdout:3899: finishing copy_from racing read to smithi08434663-4 2024-09-18T14:18:04.403 INFO:tasks.rados.rados.0.smithi084.stdout:3900: finishing write tid 1 to smithi08434663-14 2024-09-18T14:18:04.403 INFO:tasks.rados.rados.0.smithi084.stdout:3900: finishing write tid 2 to smithi08434663-14 2024-09-18T14:18:04.403 INFO:tasks.rados.rados.0.smithi084.stdout:3899: done (2 left) 2024-09-18T14:18:04.403 INFO:tasks.rados.rados.0.smithi084.stdout:3902: copy_from oid 29 from oid 34 current snap is 443 2024-09-18T14:18:04.404 INFO:tasks.rados.rados.0.smithi084.stdout:3903: rollback oid 28 current snap is 443 2024-09-18T14:18:04.404 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 28 to 410 2024-09-18T14:18:04.404 INFO:tasks.rados.rados.0.smithi084.stdout:3904: copy_from oid 33 from oid 4 current snap is 443 2024-09-18T14:18:04.404 INFO:tasks.rados.rados.0.smithi084.stdout:3905: read oid 35 snap -1 2024-09-18T14:18:04.404 INFO:tasks.rados.rados.0.smithi084.stdout:3905: expect (ObjNum 1295 snap 438 seq_num 1295) 2024-09-18T14:18:04.404 INFO:tasks.rados.rados.0.smithi084.stdout:3906: snap_create 2024-09-18T14:18:04.410 INFO:tasks.rados.rados.0.smithi084.stdout:3901: finishing copy_from to smithi08434663-48 2024-09-18T14:18:04.410 INFO:tasks.rados.rados.0.smithi084.stdout:3901: got expected ENOENT (src dne) 2024-09-18T14:18:04.410 INFO:tasks.rados.rados.0.smithi084.stdout:3901: finishing copy_from racing read to smithi08434663-48 2024-09-18T14:18:04.411 INFO:tasks.rados.rados.0.smithi084.stdout:3905: expect (ObjNum 1295 snap 438 seq_num 1295) 2024-09-18T14:18:04.605 INFO:tasks.rados.rados.0.smithi084.stdout:3900: finishing write tid 3 to smithi08434663-14 2024-09-18T14:18:04.605 INFO:tasks.rados.rados.0.smithi084.stdout:3902: finishing copy_from racing read to smithi08434663-29 2024-09-18T14:18:04.605 INFO:tasks.rados.rados.0.smithi084.stdout:3902: finishing copy_from to smithi08434663-29 2024-09-18T14:18:04.605 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 29 v 1870 (ObjNum 1207 snap 396 seq_num 1207) dirty exists 2024-09-18T14:18:04.605 INFO:tasks.rados.rados.0.smithi084.stdout:3900: finishing write tid 4 to smithi08434663-14 2024-09-18T14:18:04.605 INFO:tasks.rados.rados.0.smithi084.stdout:3900: finishing write tid 5 to smithi08434663-14 2024-09-18T14:18:04.605 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 14 v 2284 (ObjNum 1302 snap 443 seq_num 1302) dirty exists 2024-09-18T14:18:04.605 INFO:tasks.rados.rados.0.smithi084.stdout:3900: left oid 14 (ObjNum 1302 snap 443 seq_num 1302) 2024-09-18T14:18:04.605 INFO:tasks.rados.rados.0.smithi084.stdout:3903: finishing rollback tid 0 to smithi08434663-28 2024-09-18T14:18:04.605 INFO:tasks.rados.rados.0.smithi084.stdout:3903: finishing rollback tid 1 to smithi08434663-28 2024-09-18T14:18:04.605 INFO:tasks.rados.rados.0.smithi084.stdout:3903: finishing rollback tid 2 to smithi08434663-28 2024-09-18T14:18:04.606 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 28 v 2288 (ObjNum 1072 snap 341 seq_num 1072) dirty exists 2024-09-18T14:18:04.606 INFO:tasks.rados.rados.0.smithi084.stdout:3904: finishing copy_from racing read to smithi08434663-33 2024-09-18T14:18:04.606 INFO:tasks.rados.rados.0.smithi084.stdout:3904: finishing copy_from to smithi08434663-33 2024-09-18T14:18:04.606 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 33 v 1965 (ObjNum 1244 snap 409 seq_num 1244) dirty exists 2024-09-18T14:18:05.384 INFO:tasks.rados.rados.0.smithi084.stdout:3900: done (6 left) 2024-09-18T14:18:05.384 INFO:tasks.rados.rados.0.smithi084.stdout:3901: done (5 left) 2024-09-18T14:18:05.384 INFO:tasks.rados.rados.0.smithi084.stdout:3902: done (4 left) 2024-09-18T14:18:05.384 INFO:tasks.rados.rados.0.smithi084.stdout:3903: done (3 left) 2024-09-18T14:18:05.384 INFO:tasks.rados.rados.0.smithi084.stdout:3904: done (2 left) 2024-09-18T14:18:05.384 INFO:tasks.rados.rados.0.smithi084.stdout:3905: done (1 left) 2024-09-18T14:18:05.385 INFO:tasks.rados.rados.0.smithi084.stdout:3906: done (0 left) 2024-09-18T14:18:05.385 INFO:tasks.rados.rados.0.smithi084.stdout:3907: read oid 35 snap -1 2024-09-18T14:18:05.385 INFO:tasks.rados.rados.0.smithi084.stdout:3907: expect (ObjNum 1295 snap 438 seq_num 1295) 2024-09-18T14:18:05.385 INFO:tasks.rados.rados.0.smithi084.stdout:3908: copy_from oid 36 from oid 30 current snap is 444 2024-09-18T14:18:05.385 INFO:tasks.rados.rados.0.smithi084.stdout:3909: delete oid 28 current snap is 444 2024-09-18T14:18:05.390 INFO:tasks.rados.rados.0.smithi084.stdout:3907: expect (ObjNum 1295 snap 438 seq_num 1295) 2024-09-18T14:18:05.583 INFO:tasks.rados.rados.0.smithi084.stdout:3908: finishing copy_from racing read to smithi08434663-36 2024-09-18T14:18:05.583 INFO:tasks.rados.rados.0.smithi084.stdout:3908: finishing copy_from to smithi08434663-36 2024-09-18T14:18:05.583 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 2249 (ObjNum 1288 snap 431 seq_num 1288) dirty exists 2024-09-18T14:18:05.583 INFO:tasks.rados.rados.0.smithi084.stdout:3907: done (2 left) 2024-09-18T14:18:05.583 INFO:tasks.rados.rados.0.smithi084.stdout:3908: done (1 left) 2024-09-18T14:18:05.583 INFO:tasks.rados.rados.0.smithi084.stdout:3909: done (0 left) 2024-09-18T14:18:05.583 INFO:tasks.rados.rados.0.smithi084.stdout:3910: delete oid 10 current snap is 444 2024-09-18T14:18:05.586 INFO:tasks.rados.rados.0.smithi084.stdout:3910: done (0 left) 2024-09-18T14:18:05.586 INFO:tasks.rados.rados.0.smithi084.stdout:3911: write oid 1 current snap is 444 2024-09-18T14:18:05.586 INFO:tasks.rados.rados.0.smithi084.stdout:3911: seq_num 1305 ranges {733135=496308,1898766=652784,3170535=470565} 2024-09-18T14:18:05.592 INFO:tasks.rados.rados.0.smithi084.stdout:3911: writing smithi08434663-1 from 733135 to 1229443 tid 1 2024-09-18T14:18:05.600 INFO:tasks.rados.rados.0.smithi084.stdout:3911: writing smithi08434663-1 from 1898766 to 2551550 tid 2 2024-09-18T14:18:05.606 INFO:tasks.rados.rados.0.smithi084.stdout:3911: writing smithi08434663-1 from 3170535 to 3641100 tid 3 2024-09-18T14:18:05.606 INFO:tasks.rados.rados.0.smithi084.stdout:3912: copy_from oid 26 from oid 37 current snap is 444 2024-09-18T14:18:05.606 INFO:tasks.rados.rados.0.smithi084.stdout:3911: finishing write tid 1 to smithi08434663-1 2024-09-18T14:18:05.606 INFO:tasks.rados.rados.0.smithi084.stdout:3911: finishing write tid 2 to smithi08434663-1 2024-09-18T14:18:05.606 INFO:tasks.rados.rados.0.smithi084.stdout:3913: write oid 44 current snap is 444 2024-09-18T14:18:05.606 INFO:tasks.rados.rados.0.smithi084.stdout:3913: seq_num 1306 ranges {749942=571557,1840895=796442,3365252=584690} 2024-09-18T14:18:05.613 INFO:tasks.rados.rados.0.smithi084.stdout:3913: writing smithi08434663-44 from 749942 to 1321499 tid 1 2024-09-18T14:18:05.620 INFO:tasks.rados.rados.0.smithi084.stdout:3913: writing smithi08434663-44 from 1840895 to 2637337 tid 2 2024-09-18T14:18:05.628 INFO:tasks.rados.rados.0.smithi084.stdout:3913: writing smithi08434663-44 from 3365252 to 3949942 tid 3 2024-09-18T14:18:05.628 INFO:tasks.rados.rados.0.smithi084.stdout:3914: read oid 17 snap -1 2024-09-18T14:18:05.628 INFO:tasks.rados.rados.0.smithi084.stdout:3914: expect (ObjNum 1242 snap 409 seq_num 1242) 2024-09-18T14:18:05.628 INFO:tasks.rados.rados.0.smithi084.stdout:3915: delete oid 13 current snap is 444 2024-09-18T14:18:05.628 INFO:tasks.rados.rados.0.smithi084.stdout:3911: finishing write tid 3 to smithi08434663-1 2024-09-18T14:18:05.628 INFO:tasks.rados.rados.0.smithi084.stdout:3912: finishing copy_from to smithi08434663-26 2024-09-18T14:18:05.628 INFO:tasks.rados.rados.0.smithi084.stdout:3912: got expected ENOENT (src dne) 2024-09-18T14:18:05.628 INFO:tasks.rados.rados.0.smithi084.stdout:3912: finishing copy_from racing read to smithi08434663-26 2024-09-18T14:18:05.628 INFO:tasks.rados.rados.0.smithi084.stdout:3911: finishing write tid 4 to smithi08434663-1 2024-09-18T14:18:05.628 INFO:tasks.rados.rados.0.smithi084.stdout:3911: finishing write tid 5 to smithi08434663-1 2024-09-18T14:18:05.628 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 1803 (ObjNum 1305 snap 444 seq_num 1305) dirty exists 2024-09-18T14:18:05.628 INFO:tasks.rados.rados.0.smithi084.stdout:3911: left oid 1 (ObjNum 1305 snap 444 seq_num 1305) 2024-09-18T14:18:05.629 INFO:tasks.rados.rados.0.smithi084.stdout:3913: finishing write tid 1 to smithi08434663-44 2024-09-18T14:18:05.629 INFO:tasks.rados.rados.0.smithi084.stdout:3913: finishing write tid 2 to smithi08434663-44 2024-09-18T14:18:05.630 INFO:tasks.rados.rados.0.smithi084.stdout:3913: finishing write tid 3 to smithi08434663-44 2024-09-18T14:18:05.630 INFO:tasks.rados.rados.0.smithi084.stdout:3913: finishing write tid 4 to smithi08434663-44 2024-09-18T14:18:05.630 INFO:tasks.rados.rados.0.smithi084.stdout:3913: finishing write tid 5 to smithi08434663-44 2024-09-18T14:18:05.630 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 44 v 2061 (ObjNum 1306 snap 444 seq_num 1306) dirty exists 2024-09-18T14:18:05.630 INFO:tasks.rados.rados.0.smithi084.stdout:3913: left oid 44 (ObjNum 1306 snap 444 seq_num 1306) 2024-09-18T14:18:05.637 INFO:tasks.rados.rados.0.smithi084.stdout:3914: expect (ObjNum 1242 snap 409 seq_num 1242) 2024-09-18T14:18:05.805 INFO:tasks.rados.rados.0.smithi084.stdout:3911: done (4 left) 2024-09-18T14:18:05.806 INFO:tasks.rados.rados.0.smithi084.stdout:3912: done (3 left) 2024-09-18T14:18:05.806 INFO:tasks.rados.rados.0.smithi084.stdout:3913: done (2 left) 2024-09-18T14:18:05.806 INFO:tasks.rados.rados.0.smithi084.stdout:3914: done (1 left) 2024-09-18T14:18:05.806 INFO:tasks.rados.rados.0.smithi084.stdout:3915: done (0 left) 2024-09-18T14:18:05.806 INFO:tasks.rados.rados.0.smithi084.stdout:3916: copy_from oid 1 from oid 10 current snap is 444 2024-09-18T14:18:05.806 INFO:tasks.rados.rados.0.smithi084.stdout:3917: rollback oid 15 current snap is 444 2024-09-18T14:18:05.806 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 15 to 404 2024-09-18T14:18:05.806 INFO:tasks.rados.rados.0.smithi084.stdout:3918: rollback oid 18 current snap is 444 2024-09-18T14:18:05.806 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 18 to 432 2024-09-18T14:18:05.806 INFO:tasks.rados.rados.0.smithi084.stdout:3919: write (excl) oid 7 current snap is 444 2024-09-18T14:18:05.806 INFO:tasks.rados.rados.0.smithi084.stdout:3919: seq_num 1308 ranges {783556=438408,1674212=400111,2736451=420734,3882270=101286} 2024-09-18T14:18:05.812 INFO:tasks.rados.rados.0.smithi084.stdout:3919: writing smithi08434663-7 from 783556 to 1221964 tid 1 2024-09-18T14:18:05.816 INFO:tasks.rados.rados.0.smithi084.stdout:3919: writing smithi08434663-7 from 1674212 to 2074323 tid 2 2024-09-18T14:18:05.822 INFO:tasks.rados.rados.0.smithi084.stdout:3919: writing smithi08434663-7 from 2736451 to 3157185 tid 3 2024-09-18T14:18:05.827 INFO:tasks.rados.rados.0.smithi084.stdout:3919: writing smithi08434663-7 from 3882270 to 3983556 tid 4 2024-09-18T14:18:05.827 INFO:tasks.rados.rados.0.smithi084.stdout:3920: write oid 43 current snap is 444 2024-09-18T14:18:05.827 INFO:tasks.rados.rados.0.smithi084.stdout:3920: seq_num 1309 ranges {400363=513657,1332694=660122} 2024-09-18T14:18:05.831 INFO:tasks.rados.rados.0.smithi084.stdout:3920: writing smithi08434663-43 from 400363 to 914020 tid 1 2024-09-18T14:18:05.838 INFO:tasks.rados.rados.0.smithi084.stdout:3920: writing smithi08434663-43 from 1332694 to 1992816 tid 2 2024-09-18T14:18:05.838 INFO:tasks.rados.rados.0.smithi084.stdout:3921: rollback oid 4 current snap is 444 2024-09-18T14:18:05.838 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 4 to 420 2024-09-18T14:18:05.838 INFO:tasks.rados.rados.0.smithi084.stdout:3916: finishing copy_from to smithi08434663-1 2024-09-18T14:18:05.838 INFO:tasks.rados.rados.0.smithi084.stdout:3916: got expected ENOENT (src dne) 2024-09-18T14:18:05.838 INFO:tasks.rados.rados.0.smithi084.stdout:3916: finishing copy_from racing read to smithi08434663-1 2024-09-18T14:18:05.838 INFO:tasks.rados.rados.0.smithi084.stdout:3917: finishing rollback tid 1 to smithi08434663-15 2024-09-18T14:18:05.838 INFO:tasks.rados.rados.0.smithi084.stdout:3917: finishing rollback tid 2 to smithi08434663-15 2024-09-18T14:18:05.838 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 15 v 2064 (ObjNum 1176 snap 387 seq_num 1176) dirty exists 2024-09-18T14:18:05.838 INFO:tasks.rados.rados.0.smithi084.stdout:3918: finishing rollback tid 0 to smithi08434663-18 2024-09-18T14:18:05.838 INFO:tasks.rados.rados.0.smithi084.stdout:3918: finishing rollback tid 1 to smithi08434663-18 2024-09-18T14:18:05.838 INFO:tasks.rados.rados.0.smithi084.stdout:3918: finishing rollback tid 2 to smithi08434663-18 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 2059 (ObjNum 937 snap 299 seq_num 937) dirty exists 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:3919: finishing write tid 1 to smithi08434663-7 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:3916: done (5 left) 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:3917: done (4 left) 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:3918: done (3 left) 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:3922: snap_create 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:3919: finishing write tid 2 to smithi08434663-7 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:3919: finishing write tid 3 to smithi08434663-7 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:3919: finishing write tid 4 to smithi08434663-7 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:3919: finishing write tid 5 to smithi08434663-7 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:3919: finishing write tid 6 to smithi08434663-7 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 7 v 1574 (ObjNum 1308 snap 444 seq_num 1308) dirty exists 2024-09-18T14:18:05.839 INFO:tasks.rados.rados.0.smithi084.stdout:3919: left oid 7 (ObjNum 1308 snap 444 seq_num 1308) 2024-09-18T14:18:05.840 INFO:tasks.rados.rados.0.smithi084.stdout:3920: finishing write tid 1 to smithi08434663-43 2024-09-18T14:18:05.841 INFO:tasks.rados.rados.0.smithi084.stdout:3920: finishing write tid 2 to smithi08434663-43 2024-09-18T14:18:05.842 INFO:tasks.rados.rados.0.smithi084.stdout:3920: finishing write tid 3 to smithi08434663-43 2024-09-18T14:18:05.842 INFO:tasks.rados.rados.0.smithi084.stdout:3920: finishing write tid 4 to smithi08434663-43 2024-09-18T14:18:05.842 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 2046 (ObjNum 1309 snap 444 seq_num 1309) dirty exists 2024-09-18T14:18:05.842 INFO:tasks.rados.rados.0.smithi084.stdout:3920: left oid 43 (ObjNum 1309 snap 444 seq_num 1309) 2024-09-18T14:18:05.845 INFO:tasks.rados.rados.0.smithi084.stdout:3921: finishing rollback tid 0 to smithi08434663-4 2024-09-18T14:18:05.845 INFO:tasks.rados.rados.0.smithi084.stdout:3921: finishing rollback tid 1 to smithi08434663-4 2024-09-18T14:18:05.845 INFO:tasks.rados.rados.0.smithi084.stdout:3921: finishing rollback tid 2 to smithi08434663-4 2024-09-18T14:18:05.845 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 4 v 2044 (ObjNum 1244 snap 409 seq_num 1244) dirty exists 2024-09-18T14:18:06.398 INFO:tasks.rados.rados.0.smithi084.stdout:3919: done (3 left) 2024-09-18T14:18:06.398 INFO:tasks.rados.rados.0.smithi084.stdout:3920: done (2 left) 2024-09-18T14:18:06.398 INFO:tasks.rados.rados.0.smithi084.stdout:3921: done (1 left) 2024-09-18T14:18:06.398 INFO:tasks.rados.rados.0.smithi084.stdout:3922: done (0 left) 2024-09-18T14:18:06.398 INFO:tasks.rados.rados.0.smithi084.stdout:3923: snap_create 2024-09-18T14:18:07.403 INFO:tasks.rados.rados.0.smithi084.stdout:3923: done (0 left) 2024-09-18T14:18:07.403 INFO:tasks.rados.rados.0.smithi084.stdout:3924: read oid 28 snap -1 2024-09-18T14:18:07.403 INFO:tasks.rados.rados.0.smithi084.stdout:3924: expect deleted 2024-09-18T14:18:07.403 INFO:tasks.rados.rados.0.smithi084.stdout:3925: write oid 48 current snap is 446 2024-09-18T14:18:07.403 INFO:tasks.rados.rados.0.smithi084.stdout:3925: seq_num 1310 ranges {417170=588906,1674823=342347} 2024-09-18T14:18:07.409 INFO:tasks.rados.rados.0.smithi084.stdout:3925: writing smithi08434663-48 from 417170 to 1006076 tid 1 2024-09-18T14:18:07.415 INFO:tasks.rados.rados.0.smithi084.stdout:3925: writing smithi08434663-48 from 1674823 to 2017170 tid 2 2024-09-18T14:18:07.415 INFO:tasks.rados.rados.0.smithi084.stdout:3926: snap_remove snap 335 2024-09-18T14:18:08.380 INFO:tasks.rados.rados.0.smithi084.stdout:3926: done (2 left) 2024-09-18T14:18:08.380 INFO:tasks.rados.rados.0.smithi084.stdout:3927: snap_remove snap 412 2024-09-18T14:18:09.391 INFO:tasks.rados.rados.0.smithi084.stdout:3927: done (2 left) 2024-09-18T14:18:09.392 INFO:tasks.rados.rados.0.smithi084.stdout:3928: copy_from oid 31 from oid 29 current snap is 446 2024-09-18T14:18:09.392 INFO:tasks.rados.rados.0.smithi084.stdout:3925: finishing write tid 1 to smithi08434663-48 2024-09-18T14:18:09.392 INFO:tasks.rados.rados.0.smithi084.stdout:3925: finishing write tid 2 to smithi08434663-48 2024-09-18T14:18:09.392 INFO:tasks.rados.rados.0.smithi084.stdout:3925: finishing write tid 3 to smithi08434663-48 2024-09-18T14:18:09.392 INFO:tasks.rados.rados.0.smithi084.stdout:3925: finishing write tid 4 to smithi08434663-48 2024-09-18T14:18:09.392 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 48 v 2068 (ObjNum 1310 snap 446 seq_num 1310) dirty exists 2024-09-18T14:18:09.392 INFO:tasks.rados.rados.0.smithi084.stdout:3925: left oid 48 (ObjNum 1310 snap 446 seq_num 1310) 2024-09-18T14:18:09.392 INFO:tasks.rados.rados.0.smithi084.stdout:3924: done (2 left) 2024-09-18T14:18:09.393 INFO:tasks.rados.rados.0.smithi084.stdout:3925: done (1 left) 2024-09-18T14:18:09.393 INFO:tasks.rados.rados.0.smithi084.stdout:3929: snap_remove snap 429 2024-09-18T14:18:10.400 INFO:tasks.rados.rados.0.smithi084.stdout:3929: done (1 left) 2024-09-18T14:18:10.400 INFO:tasks.rados.rados.0.smithi084.stdout:3930: delete oid 18 current snap is 446 2024-09-18T14:18:10.401 INFO:tasks.rados.rados.0.smithi084.stdout:3928: finishing copy_from racing read to smithi08434663-31 2024-09-18T14:18:10.401 INFO:tasks.rados.rados.0.smithi084.stdout:3928: finishing copy_from to smithi08434663-31 2024-09-18T14:18:10.401 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 1870 (ObjNum 1207 snap 396 seq_num 1207) dirty exists 2024-09-18T14:18:10.410 INFO:tasks.rados.rados.0.smithi084.stdout:3928: done (1 left) 2024-09-18T14:18:10.410 INFO:tasks.rados.rados.0.smithi084.stdout:3930: done (0 left) 2024-09-18T14:18:10.410 INFO:tasks.rados.rados.0.smithi084.stdout:3931: copy_from oid 39 from oid 32 current snap is 446 2024-09-18T14:18:10.410 INFO:tasks.rados.rados.0.smithi084.stdout:3932: read oid 47 snap 445 2024-09-18T14:18:10.410 INFO:tasks.rados.rados.0.smithi084.stdout:3932: expect deleted 2024-09-18T14:18:10.410 INFO:tasks.rados.rados.0.smithi084.stdout:3933: read oid 28 snap -1 2024-09-18T14:18:10.410 INFO:tasks.rados.rados.0.smithi084.stdout:3933: expect deleted 2024-09-18T14:18:10.410 INFO:tasks.rados.rados.0.smithi084.stdout:3934: delete oid 9 current snap is 446 2024-09-18T14:18:10.411 INFO:tasks.rados.rados.0.smithi084.stdout:3931: finishing copy_from to smithi08434663-39 2024-09-18T14:18:10.411 INFO:tasks.rados.rados.0.smithi084.stdout:3931: got expected ENOENT (src dne) 2024-09-18T14:18:10.411 INFO:tasks.rados.rados.0.smithi084.stdout:3931: finishing copy_from racing read to smithi08434663-39 2024-09-18T14:18:10.416 INFO:tasks.rados.rados.0.smithi084.stdout:3931: done (3 left) 2024-09-18T14:18:10.416 INFO:tasks.rados.rados.0.smithi084.stdout:3932: done (2 left) 2024-09-18T14:18:10.416 INFO:tasks.rados.rados.0.smithi084.stdout:3933: done (1 left) 2024-09-18T14:18:10.416 INFO:tasks.rados.rados.0.smithi084.stdout:3934: done (0 left) 2024-09-18T14:18:10.416 INFO:tasks.rados.rados.0.smithi084.stdout:3935: snap_remove snap 402 2024-09-18T14:18:11.399 INFO:tasks.rados.rados.0.smithi084.stdout:3935: done (0 left) 2024-09-18T14:18:11.399 INFO:tasks.rados.rados.0.smithi084.stdout:3936: write (excl) oid 34 current snap is 446 2024-09-18T14:18:11.399 INFO:tasks.rados.rados.0.smithi084.stdout:3936: seq_num 1313 ranges {467591=414653,1450269=617322} 2024-09-18T14:18:11.404 INFO:tasks.rados.rados.0.smithi084.stdout:3936: writing smithi08434663-34 from 467591 to 882244 tid 1 2024-09-18T14:18:11.411 INFO:tasks.rados.rados.0.smithi084.stdout:3936: writing smithi08434663-34 from 1450269 to 2067591 tid 2 2024-09-18T14:18:11.412 INFO:tasks.rados.rados.0.smithi084.stdout:3937: write oid 6 current snap is 446 2024-09-18T14:18:11.412 INFO:tasks.rados.rados.0.smithi084.stdout:3937: seq_num 1314 ranges {484398=489902,1392398=411118} 2024-09-18T14:18:11.418 INFO:tasks.rados.rados.0.smithi084.stdout:3937: writing smithi08434663-6 from 484398 to 974300 tid 1 2024-09-18T14:18:11.423 INFO:tasks.rados.rados.0.smithi084.stdout:3937: writing smithi08434663-6 from 1392398 to 1803516 tid 2 2024-09-18T14:18:11.423 INFO:tasks.rados.rados.0.smithi084.stdout:3938: rollback oid 23 current snap is 446 2024-09-18T14:18:11.423 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 23 to 420 2024-09-18T14:18:11.423 INFO:tasks.rados.rados.0.smithi084.stdout:3936: finishing write tid 1 to smithi08434663-34 2024-09-18T14:18:11.424 INFO:tasks.rados.rados.0.smithi084.stdout:3936: finishing write tid 2 to smithi08434663-34 2024-09-18T14:18:11.424 INFO:tasks.rados.rados.0.smithi084.stdout:3936: finishing write tid 3 to smithi08434663-34 2024-09-18T14:18:11.424 INFO:tasks.rados.rados.0.smithi084.stdout:3936: finishing write tid 4 to smithi08434663-34 2024-09-18T14:18:11.424 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 34 v 2068 (ObjNum 1313 snap 446 seq_num 1313) dirty exists 2024-09-18T14:18:11.424 INFO:tasks.rados.rados.0.smithi084.stdout:3936: left oid 34 (ObjNum 1313 snap 446 seq_num 1313) 2024-09-18T14:18:11.424 INFO:tasks.rados.rados.0.smithi084.stdout:3937: finishing write tid 1 to smithi08434663-6 2024-09-18T14:18:11.424 INFO:tasks.rados.rados.0.smithi084.stdout:3936: done (2 left) 2024-09-18T14:18:11.424 INFO:tasks.rados.rados.0.smithi084.stdout:3939: write oid 1 current snap is 446 2024-09-18T14:18:11.424 INFO:tasks.rados.rados.0.smithi084.stdout:3939: seq_num 1315 ranges {501205=565151,1850880=250325} 2024-09-18T14:18:11.428 INFO:tasks.rados.rados.0.smithi084.stdout:3939: writing smithi08434663-1 from 501205 to 1066356 tid 1 2024-09-18T14:18:11.435 INFO:tasks.rados.rados.0.smithi084.stdout:3939: writing smithi08434663-1 from 1850880 to 2101205 tid 2 2024-09-18T14:18:11.435 INFO:tasks.rados.rados.0.smithi084.stdout:3940: delete oid 24 current snap is 446 2024-09-18T14:18:11.435 INFO:tasks.rados.rados.0.smithi084.stdout:3937: finishing write tid 2 to smithi08434663-6 2024-09-18T14:18:11.435 INFO:tasks.rados.rados.0.smithi084.stdout:3937: finishing write tid 3 to smithi08434663-6 2024-09-18T14:18:11.435 INFO:tasks.rados.rados.0.smithi084.stdout:3937: finishing write tid 4 to smithi08434663-6 2024-09-18T14:18:11.435 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 1577 (ObjNum 1314 snap 446 seq_num 1314) dirty exists 2024-09-18T14:18:11.435 INFO:tasks.rados.rados.0.smithi084.stdout:3937: left oid 6 (ObjNum 1314 snap 446 seq_num 1314) 2024-09-18T14:18:11.435 INFO:tasks.rados.rados.0.smithi084.stdout:3938: finishing rollback tid 0 to smithi08434663-23 2024-09-18T14:18:11.435 INFO:tasks.rados.rados.0.smithi084.stdout:3938: finishing rollback tid 1 to smithi08434663-23 2024-09-18T14:18:11.435 INFO:tasks.rados.rados.0.smithi084.stdout:3938: finishing rollback tid 2 to smithi08434663-23 2024-09-18T14:18:11.435 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 23 v 1580 (ObjNum 1254 snap 420 seq_num 1254) dirty exists 2024-09-18T14:18:11.437 INFO:tasks.rados.rados.0.smithi084.stdout:3937: done (3 left) 2024-09-18T14:18:11.437 INFO:tasks.rados.rados.0.smithi084.stdout:3938: done (2 left) 2024-09-18T14:18:11.437 INFO:tasks.rados.rados.0.smithi084.stdout:3940: done (1 left) 2024-09-18T14:18:11.437 INFO:tasks.rados.rados.0.smithi084.stdout:3941: delete oid 28 current snap is 446 2024-09-18T14:18:11.440 INFO:tasks.rados.rados.0.smithi084.stdout:3939: finishing write tid 1 to smithi08434663-1 2024-09-18T14:18:11.440 INFO:tasks.rados.rados.0.smithi084.stdout:3939: finishing write tid 2 to smithi08434663-1 2024-09-18T14:18:11.441 INFO:tasks.rados.rados.0.smithi084.stdout:3941: done (1 left) 2024-09-18T14:18:11.441 INFO:tasks.rados.rados.0.smithi084.stdout:3942: copy_from oid 43 from oid 42 current snap is 446 2024-09-18T14:18:11.441 INFO:tasks.rados.rados.0.smithi084.stdout:3943: write oid 37 current snap is 446 2024-09-18T14:18:11.441 INFO:tasks.rados.rados.0.smithi084.stdout:3943: seq_num 1318 ranges {551626=507251,1626326=418456} 2024-09-18T14:18:11.446 INFO:tasks.rados.rados.0.smithi084.stdout:3943: writing smithi08434663-37 from 551626 to 1058877 tid 1 2024-09-18T14:18:11.452 INFO:tasks.rados.rados.0.smithi084.stdout:3943: writing smithi08434663-37 from 1626326 to 2044782 tid 2 2024-09-18T14:18:11.453 INFO:tasks.rados.rados.0.smithi084.stdout:3944: read oid 46 snap -1 2024-09-18T14:18:11.453 INFO:tasks.rados.rados.0.smithi084.stdout:3944: expect deleted 2024-09-18T14:18:11.453 INFO:tasks.rados.rados.0.smithi084.stdout:3945: copy_from oid 6 from oid 48 current snap is 446 2024-09-18T14:18:11.453 INFO:tasks.rados.rados.0.smithi084.stdout:3942: finishing copy_from to smithi08434663-43 2024-09-18T14:18:11.453 INFO:tasks.rados.rados.0.smithi084.stdout:3942: got expected ENOENT (src dne) 2024-09-18T14:18:11.453 INFO:tasks.rados.rados.0.smithi084.stdout:3942: finishing copy_from racing read to smithi08434663-43 2024-09-18T14:18:11.453 INFO:tasks.rados.rados.0.smithi084.stdout:3939: finishing write tid 3 to smithi08434663-1 2024-09-18T14:18:11.453 INFO:tasks.rados.rados.0.smithi084.stdout:3939: finishing write tid 4 to smithi08434663-1 2024-09-18T14:18:11.453 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 1 v 1806 (ObjNum 1315 snap 446 seq_num 1315) dirty exists 2024-09-18T14:18:11.453 INFO:tasks.rados.rados.0.smithi084.stdout:3939: left oid 1 (ObjNum 1315 snap 446 seq_num 1315) 2024-09-18T14:18:11.453 INFO:tasks.rados.rados.0.smithi084.stdout:3943: finishing write tid 1 to smithi08434663-37 2024-09-18T14:18:11.454 INFO:tasks.rados.rados.0.smithi084.stdout:3939: done (4 left) 2024-09-18T14:18:11.454 INFO:tasks.rados.rados.0.smithi084.stdout:3942: done (3 left) 2024-09-18T14:18:11.454 INFO:tasks.rados.rados.0.smithi084.stdout:3946: delete oid 41 current snap is 446 2024-09-18T14:18:11.455 INFO:tasks.rados.rados.0.smithi084.stdout:3943: finishing write tid 2 to smithi08434663-37 2024-09-18T14:18:11.456 INFO:tasks.rados.rados.0.smithi084.stdout:3943: finishing write tid 3 to smithi08434663-37 2024-09-18T14:18:11.456 INFO:tasks.rados.rados.0.smithi084.stdout:3943: finishing write tid 4 to smithi08434663-37 2024-09-18T14:18:11.456 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 37 v 2252 (ObjNum 1318 snap 446 seq_num 1318) dirty exists 2024-09-18T14:18:11.456 INFO:tasks.rados.rados.0.smithi084.stdout:3943: left oid 37 (ObjNum 1318 snap 446 seq_num 1318) 2024-09-18T14:18:11.456 INFO:tasks.rados.rados.0.smithi084.stdout:3943: done (3 left) 2024-09-18T14:18:11.457 INFO:tasks.rados.rados.0.smithi084.stdout:3944: done (2 left) 2024-09-18T14:18:11.457 INFO:tasks.rados.rados.0.smithi084.stdout:3946: done (1 left) 2024-09-18T14:18:11.457 INFO:tasks.rados.rados.0.smithi084.stdout:3947: snap_remove snap 444 2024-09-18T14:18:12.419 INFO:tasks.rados.rados.0.smithi084.stdout:3947: done (1 left) 2024-09-18T14:18:12.419 INFO:tasks.rados.rados.0.smithi084.stdout:3948: snap_remove snap 426 2024-09-18T14:18:13.380 INFO:tasks.rados.rados.0.smithi084.stdout:3948: done (1 left) 2024-09-18T14:18:13.380 INFO:tasks.rados.rados.0.smithi084.stdout:3949: rollback oid 3 current snap is 446 2024-09-18T14:18:13.380 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 3 to 421 2024-09-18T14:18:13.381 INFO:tasks.rados.rados.0.smithi084.stdout:3945: finishing copy_from racing read to smithi08434663-6 2024-09-18T14:18:13.381 INFO:tasks.rados.rados.0.smithi084.stdout:3945: finishing copy_from to smithi08434663-6 2024-09-18T14:18:13.381 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 6 v 2068 (ObjNum 1310 snap 446 seq_num 1310) dirty exists 2024-09-18T14:18:13.381 INFO:tasks.rados.rados.0.smithi084.stdout:3945: done (1 left) 2024-09-18T14:18:13.381 INFO:tasks.rados.rados.0.smithi084.stdout:3950: read oid 40 snap -1 2024-09-18T14:18:13.381 INFO:tasks.rados.rados.0.smithi084.stdout:3950: expect deleted 2024-09-18T14:18:13.381 INFO:tasks.rados.rados.0.smithi084.stdout:3951: read oid 42 snap -1 2024-09-18T14:18:13.381 INFO:tasks.rados.rados.0.smithi084.stdout:3951: expect deleted 2024-09-18T14:18:13.381 INFO:tasks.rados.rados.0.smithi084.stdout:3952: rollback oid 12 current snap is 446 2024-09-18T14:18:13.381 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 12 to 442 2024-09-18T14:18:13.381 INFO:tasks.rados.rados.0.smithi084.stdout:3953: write oid 36 current snap is 446 2024-09-18T14:18:13.381 INFO:tasks.rados.rados.0.smithi084.stdout:3953: seq_num 1320 ranges {585240=657749,1743290=422125} 2024-09-18T14:18:13.390 INFO:tasks.rados.rados.0.smithi084.stdout:3953: writing smithi08434663-36 from 585240 to 1242989 tid 1 2024-09-18T14:18:13.395 INFO:tasks.rados.rados.0.smithi084.stdout:3953: writing smithi08434663-36 from 1743290 to 2165415 tid 2 2024-09-18T14:18:13.396 INFO:tasks.rados.rados.0.smithi084.stdout:3949: finishing rollback tid 0 to smithi08434663-3 2024-09-18T14:18:13.397 INFO:tasks.rados.rados.0.smithi084.stdout:3949: finishing rollback tid 1 to smithi08434663-3 2024-09-18T14:18:13.397 INFO:tasks.rados.rados.0.smithi084.stdout:3950: done (4 left) 2024-09-18T14:18:13.397 INFO:tasks.rados.rados.0.smithi084.stdout:3951: done (3 left) 2024-09-18T14:18:13.397 INFO:tasks.rados.rados.0.smithi084.stdout:3954: read oid 48 snap -1 2024-09-18T14:18:13.397 INFO:tasks.rados.rados.0.smithi084.stdout:3954: expect (ObjNum 1310 snap 446 seq_num 1310) 2024-09-18T14:18:13.397 INFO:tasks.rados.rados.0.smithi084.stdout:3955: write oid 11 current snap is 446 2024-09-18T14:18:13.397 INFO:tasks.rados.rados.0.smithi084.stdout:3955: seq_num 1321 ranges {602047=732998,1801772=400275} 2024-09-18T14:18:13.405 INFO:tasks.rados.rados.0.smithi084.stdout:3955: writing smithi08434663-11 from 602047 to 1335045 tid 1 2024-09-18T14:18:13.410 INFO:tasks.rados.rados.0.smithi084.stdout:3955: writing smithi08434663-11 from 1801772 to 2202047 tid 2 2024-09-18T14:18:13.410 INFO:tasks.rados.rados.0.smithi084.stdout:3956: write (excl) oid 39 current snap is 446 2024-09-18T14:18:13.410 INFO:tasks.rados.rados.0.smithi084.stdout:3956: seq_num 1322 ranges {618854=408247,1743901=425794} 2024-09-18T14:18:13.416 INFO:tasks.rados.rados.0.smithi084.stdout:3956: writing smithi08434663-39 from 618854 to 1027101 tid 1 2024-09-18T14:18:13.423 INFO:tasks.rados.rados.0.smithi084.stdout:3956: writing smithi08434663-39 from 1743901 to 2169695 tid 2 2024-09-18T14:18:13.423 INFO:tasks.rados.rados.0.smithi084.stdout:3957: delete oid 33 current snap is 446 2024-09-18T14:18:13.423 INFO:tasks.rados.rados.0.smithi084.stdout:3949: finishing rollback tid 2 to smithi08434663-3 2024-09-18T14:18:13.423 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 2063 (ObjNum 967 snap 308 seq_num 967) dirty exists 2024-09-18T14:18:13.423 INFO:tasks.rados.rados.0.smithi084.stdout:3954: expect (ObjNum 1310 snap 446 seq_num 1310) 2024-09-18T14:18:13.508 INFO:tasks.rados.rados.0.smithi084.stdout:3953: finishing write tid 1 to smithi08434663-36 2024-09-18T14:18:13.508 INFO:tasks.rados.rados.0.smithi084.stdout:3952: finishing rollback tid 0 to smithi08434663-12 2024-09-18T14:18:13.508 INFO:tasks.rados.rados.0.smithi084.stdout:3952: finishing rollback tid 1 to smithi08434663-12 2024-09-18T14:18:13.508 INFO:tasks.rados.rados.0.smithi084.stdout:3952: finishing rollback tid 2 to smithi08434663-12 2024-09-18T14:18:13.508 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 12 v 2329 (ObjNum 1287 snap 431 seq_num 1287) dirty exists 2024-09-18T14:18:13.508 INFO:tasks.rados.rados.0.smithi084.stdout:3953: finishing write tid 2 to smithi08434663-36 2024-09-18T14:18:13.508 INFO:tasks.rados.rados.0.smithi084.stdout:3953: finishing write tid 3 to smithi08434663-36 2024-09-18T14:18:13.508 INFO:tasks.rados.rados.0.smithi084.stdout:3953: finishing write tid 4 to smithi08434663-36 2024-09-18T14:18:13.508 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 2255 (ObjNum 1320 snap 446 seq_num 1320) dirty exists 2024-09-18T14:18:13.508 INFO:tasks.rados.rados.0.smithi084.stdout:3953: left oid 36 (ObjNum 1320 snap 446 seq_num 1320) 2024-09-18T14:18:13.508 INFO:tasks.rados.rados.0.smithi084.stdout:3955: finishing write tid 1 to smithi08434663-11 2024-09-18T14:18:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3955: finishing write tid 2 to smithi08434663-11 2024-09-18T14:18:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3955: finishing write tid 3 to smithi08434663-11 2024-09-18T14:18:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3955: finishing write tid 4 to smithi08434663-11 2024-09-18T14:18:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 11 v 2065 (ObjNum 1321 snap 446 seq_num 1321) dirty exists 2024-09-18T14:18:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3955: left oid 11 (ObjNum 1321 snap 446 seq_num 1321) 2024-09-18T14:18:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3956: finishing write tid 1 to smithi08434663-39 2024-09-18T14:18:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3956: finishing write tid 2 to smithi08434663-39 2024-09-18T14:18:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3956: finishing write tid 3 to smithi08434663-39 2024-09-18T14:18:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3956: finishing write tid 4 to smithi08434663-39 2024-09-18T14:18:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 39 v 2068 (ObjNum 1322 snap 446 seq_num 1322) dirty exists 2024-09-18T14:18:13.509 INFO:tasks.rados.rados.0.smithi084.stdout:3956: left oid 39 (ObjNum 1322 snap 446 seq_num 1322) 2024-09-18T14:18:13.511 INFO:tasks.rados.rados.0.smithi084.stdout:3949: done (6 left) 2024-09-18T14:18:13.511 INFO:tasks.rados.rados.0.smithi084.stdout:3952: done (5 left) 2024-09-18T14:18:13.511 INFO:tasks.rados.rados.0.smithi084.stdout:3953: done (4 left) 2024-09-18T14:18:13.511 INFO:tasks.rados.rados.0.smithi084.stdout:3954: done (3 left) 2024-09-18T14:18:13.511 INFO:tasks.rados.rados.0.smithi084.stdout:3955: done (2 left) 2024-09-18T14:18:13.511 INFO:tasks.rados.rados.0.smithi084.stdout:3956: done (1 left) 2024-09-18T14:18:13.511 INFO:tasks.rados.rados.0.smithi084.stdout:3957: done (0 left) 2024-09-18T14:18:13.511 INFO:tasks.rados.rados.0.smithi084.stdout:3958: read oid 15 snap -1 2024-09-18T14:18:13.511 INFO:tasks.rados.rados.0.smithi084.stdout:3958: expect (ObjNum 1176 snap 387 seq_num 1176) 2024-09-18T14:18:13.512 INFO:tasks.rados.rados.0.smithi084.stdout:3959: delete oid 32 current snap is 446 2024-09-18T14:18:13.513 INFO:tasks.rados.rados.0.smithi084.stdout:3959: done (1 left) 2024-09-18T14:18:13.513 INFO:tasks.rados.rados.0.smithi084.stdout:3960: copy_from oid 47 from oid 44 current snap is 446 2024-09-18T14:18:13.513 INFO:tasks.rados.rados.0.smithi084.stdout:3961: write oid 3 current snap is 446 2024-09-18T14:18:13.513 INFO:tasks.rados.rados.0.smithi084.stdout:3961: seq_num 1325 ranges {669275=750347,2035700=233575} 2024-09-18T14:18:13.521 INFO:tasks.rados.rados.0.smithi084.stdout:3961: writing smithi08434663-3 from 669275 to 1419622 tid 1 2024-09-18T14:18:13.526 INFO:tasks.rados.rados.0.smithi084.stdout:3961: writing smithi08434663-3 from 2035700 to 2269275 tid 2 2024-09-18T14:18:13.526 INFO:tasks.rados.rados.0.smithi084.stdout:3962: write oid 43 current snap is 446 2024-09-18T14:18:13.526 INFO:tasks.rados.rados.0.smithi084.stdout:3962: seq_num 1326 ranges {686082=425596,1577829=433132} 2024-09-18T14:18:13.532 INFO:tasks.rados.rados.0.smithi084.stdout:3962: writing smithi08434663-43 from 686082 to 1111678 tid 1 2024-09-18T14:18:13.537 INFO:tasks.rados.rados.0.smithi084.stdout:3962: writing smithi08434663-43 from 1577829 to 2010961 tid 2 2024-09-18T14:18:13.537 INFO:tasks.rados.rados.0.smithi084.stdout:3963: rollback oid 8 current snap is 446 2024-09-18T14:18:13.537 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 8 to 421 2024-09-18T14:18:13.537 INFO:tasks.rados.rados.0.smithi084.stdout:3958: expect (ObjNum 1176 snap 387 seq_num 1176) 2024-09-18T14:18:13.658 INFO:tasks.rados.rados.0.smithi084.stdout:3960: finishing copy_from racing read to smithi08434663-47 2024-09-18T14:18:13.658 INFO:tasks.rados.rados.0.smithi084.stdout:3960: finishing copy_from to smithi08434663-47 2024-09-18T14:18:13.658 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 47 v 2061 (ObjNum 1306 snap 444 seq_num 1306) dirty exists 2024-09-18T14:18:13.658 INFO:tasks.rados.rados.0.smithi084.stdout:3961: finishing write tid 1 to smithi08434663-3 2024-09-18T14:18:13.658 INFO:tasks.rados.rados.0.smithi084.stdout:3961: finishing write tid 2 to smithi08434663-3 2024-09-18T14:18:13.658 INFO:tasks.rados.rados.0.smithi084.stdout:3961: finishing write tid 3 to smithi08434663-3 2024-09-18T14:18:13.658 INFO:tasks.rados.rados.0.smithi084.stdout:3961: finishing write tid 4 to smithi08434663-3 2024-09-18T14:18:13.658 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 2066 (ObjNum 1325 snap 446 seq_num 1325) dirty exists 2024-09-18T14:18:13.658 INFO:tasks.rados.rados.0.smithi084.stdout:3961: left oid 3 (ObjNum 1325 snap 446 seq_num 1325) 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:3962: finishing write tid 1 to smithi08434663-43 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:3958: done (4 left) 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:3960: done (3 left) 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:3961: done (2 left) 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:3964: snap_create 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:3962: finishing write tid 2 to smithi08434663-43 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:3963: finishing rollback tid 0 to smithi08434663-8 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:3962: finishing write tid 3 to smithi08434663-43 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:3962: finishing write tid 4 to smithi08434663-43 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 2049 (ObjNum 1326 snap 446 seq_num 1326) dirty exists 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:3962: left oid 43 (ObjNum 1326 snap 446 seq_num 1326) 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:3963: finishing rollback tid 1 to smithi08434663-8 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:3963: finishing rollback tid 2 to smithi08434663-8 2024-09-18T14:18:13.659 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 8 v 1441 (ObjNum 1253 snap 414 seq_num 1253) dirty exists 2024-09-18T14:18:14.430 INFO:tasks.rados.rados.0.smithi084.stdout:3962: done (2 left) 2024-09-18T14:18:14.431 INFO:tasks.rados.rados.0.smithi084.stdout:3963: done (1 left) 2024-09-18T14:18:14.431 INFO:tasks.rados.rados.0.smithi084.stdout:3964: done (0 left) 2024-09-18T14:18:14.431 INFO:tasks.rados.rados.0.smithi084.stdout:3965: write (excl) oid 17 current snap is 447 2024-09-18T14:18:14.431 INFO:tasks.rados.rados.0.smithi084.stdout:3965: seq_num 1327 ranges {702889=500845,1636311=576790} 2024-09-18T14:18:14.437 INFO:tasks.rados.rados.0.smithi084.stdout:3965: writing smithi08434663-17 from 702889 to 1203734 tid 1 2024-09-18T14:18:14.442 INFO:tasks.rados.rados.0.smithi084.stdout:3965: writing smithi08434663-17 from 1636311 to 2213101 tid 2 2024-09-18T14:18:14.443 INFO:tasks.rados.rados.0.smithi084.stdout:3966: rollback oid 43 current snap is 447 2024-09-18T14:18:14.443 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 43 to 425 2024-09-18T14:18:14.443 INFO:tasks.rados.rados.0.smithi084.stdout:3965: finishing write tid 1 to smithi08434663-17 2024-09-18T14:18:14.443 INFO:tasks.rados.rados.0.smithi084.stdout:3967: delete oid 40 current snap is 447 2024-09-18T14:18:14.447 INFO:tasks.rados.rados.0.smithi084.stdout:3966: finishing rollback tid 0 to smithi08434663-43 2024-09-18T14:18:14.447 INFO:tasks.rados.rados.0.smithi084.stdout:3966: finishing rollback tid 1 to smithi08434663-43 2024-09-18T14:18:14.448 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 2051 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-09-18T14:18:14.449 INFO:tasks.rados.rados.0.smithi084.stdout:3965: finishing write tid 2 to smithi08434663-17 2024-09-18T14:18:14.449 INFO:tasks.rados.rados.0.smithi084.stdout:3965: finishing write tid 3 to smithi08434663-17 2024-09-18T14:18:14.449 INFO:tasks.rados.rados.0.smithi084.stdout:3965: finishing write tid 4 to smithi08434663-17 2024-09-18T14:18:14.449 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 17 v 1889 (ObjNum 1327 snap 447 seq_num 1327) dirty exists 2024-09-18T14:18:14.449 INFO:tasks.rados.rados.0.smithi084.stdout:3965: left oid 17 (ObjNum 1327 snap 447 seq_num 1327) 2024-09-18T14:18:14.449 INFO:tasks.rados.rados.0.smithi084.stdout:3965: done (2 left) 2024-09-18T14:18:14.450 INFO:tasks.rados.rados.0.smithi084.stdout:3966: done (1 left) 2024-09-18T14:18:14.450 INFO:tasks.rados.rados.0.smithi084.stdout:3967: done (0 left) 2024-09-18T14:18:14.450 INFO:tasks.rados.rados.0.smithi084.stdout:3968: copy_from oid 46 from oid 32 current snap is 447 2024-09-18T14:18:14.450 INFO:tasks.rados.rados.0.smithi084.stdout:3969: delete oid 2 current snap is 447 2024-09-18T14:18:14.452 INFO:tasks.rados.rados.0.smithi084.stdout:3968: finishing copy_from to smithi08434663-46 2024-09-18T14:18:14.452 INFO:tasks.rados.rados.0.smithi084.stdout:3968: got expected ENOENT (src dne) 2024-09-18T14:18:14.452 INFO:tasks.rados.rados.0.smithi084.stdout:3968: finishing copy_from racing read to smithi08434663-46 2024-09-18T14:18:14.453 INFO:tasks.rados.rados.0.smithi084.stdout:3968: done (1 left) 2024-09-18T14:18:14.453 INFO:tasks.rados.rados.0.smithi084.stdout:3969: done (0 left) 2024-09-18T14:18:14.453 INFO:tasks.rados.rados.0.smithi084.stdout:3970: write oid 36 current snap is 447 2024-09-18T14:18:14.453 INFO:tasks.rados.rados.0.smithi084.stdout:3970: seq_num 1330 ranges {753310=726592,2095404=257906} 2024-09-18T14:18:14.461 INFO:tasks.rados.rados.0.smithi084.stdout:3970: writing smithi08434663-36 from 753310 to 1479902 tid 1 2024-09-18T14:18:14.466 INFO:tasks.rados.rados.0.smithi084.stdout:3970: writing smithi08434663-36 from 2095404 to 2353310 tid 2 2024-09-18T14:18:14.466 INFO:tasks.rados.rados.0.smithi084.stdout:3971: snap_create 2024-09-18T14:18:14.469 INFO:tasks.rados.rados.0.smithi084.stdout:3970: finishing write tid 1 to smithi08434663-36 2024-09-18T14:18:14.469 INFO:tasks.rados.rados.0.smithi084.stdout:3970: finishing write tid 2 to smithi08434663-36 2024-09-18T14:18:14.469 INFO:tasks.rados.rados.0.smithi084.stdout:3970: finishing write tid 3 to smithi08434663-36 2024-09-18T14:18:14.469 INFO:tasks.rados.rados.0.smithi084.stdout:3970: finishing write tid 4 to smithi08434663-36 2024-09-18T14:18:14.470 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 36 v 2259 (ObjNum 1330 snap 447 seq_num 1330) dirty exists 2024-09-18T14:18:14.470 INFO:tasks.rados.rados.0.smithi084.stdout:3970: left oid 36 (ObjNum 1330 snap 447 seq_num 1330) 2024-09-18T14:18:15.433 INFO:tasks.rados.rados.0.smithi084.stdout:3970: done (1 left) 2024-09-18T14:18:15.433 INFO:tasks.rados.rados.0.smithi084.stdout:3971: done (0 left) 2024-09-18T14:18:15.433 INFO:tasks.rados.rados.0.smithi084.stdout:3972: copy_from oid 29 from oid 32 current snap is 448 2024-09-18T14:18:15.433 INFO:tasks.rados.rados.0.smithi084.stdout:3973: read oid 39 snap -1 2024-09-18T14:18:15.433 INFO:tasks.rados.rados.0.smithi084.stdout:3973: expect (ObjNum 1322 snap 446 seq_num 1322) 2024-09-18T14:18:15.433 INFO:tasks.rados.rados.0.smithi084.stdout:3974: write (excl) oid 13 current snap is 448 2024-09-18T14:18:15.433 INFO:tasks.rados.rados.0.smithi084.stdout:3974: seq_num 1331 ranges {770117=518194,1870239=499878} 2024-09-18T14:18:15.439 INFO:tasks.rados.rados.0.smithi084.stdout:3974: writing smithi08434663-13 from 770117 to 1288311 tid 1 2024-09-18T14:18:15.445 INFO:tasks.rados.rados.0.smithi084.stdout:3974: writing smithi08434663-13 from 1870239 to 2370117 tid 2 2024-09-18T14:18:15.445 INFO:tasks.rados.rados.0.smithi084.stdout:3975: snap_remove snap 446 2024-09-18T14:18:16.444 INFO:tasks.rados.rados.0.smithi084.stdout:3975: done (3 left) 2024-09-18T14:18:16.445 INFO:tasks.rados.rados.0.smithi084.stdout:3976: snap_create 2024-09-18T14:18:16.445 INFO:tasks.rados.rados.0.smithi084.stdout:3972: finishing copy_from to smithi08434663-29 2024-09-18T14:18:16.445 INFO:tasks.rados.rados.0.smithi084.stdout:3972: got expected ENOENT (src dne) 2024-09-18T14:18:16.445 INFO:tasks.rados.rados.0.smithi084.stdout:3972: finishing copy_from racing read to smithi08434663-29 2024-09-18T14:18:16.445 INFO:tasks.rados.rados.0.smithi084.stdout:3973: expect (ObjNum 1322 snap 446 seq_num 1322) 2024-09-18T14:18:16.583 INFO:tasks.rados.rados.0.smithi084.stdout:3974: finishing write tid 1 to smithi08434663-13 2024-09-18T14:18:16.583 INFO:tasks.rados.rados.0.smithi084.stdout:3974: finishing write tid 2 to smithi08434663-13 2024-09-18T14:18:16.583 INFO:tasks.rados.rados.0.smithi084.stdout:3974: finishing write tid 3 to smithi08434663-13 2024-09-18T14:18:16.583 INFO:tasks.rados.rados.0.smithi084.stdout:3974: finishing write tid 4 to smithi08434663-13 2024-09-18T14:18:16.583 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 13 v 1892 (ObjNum 1331 snap 448 seq_num 1331) dirty exists 2024-09-18T14:18:16.583 INFO:tasks.rados.rados.0.smithi084.stdout:3974: left oid 13 (ObjNum 1331 snap 448 seq_num 1331) 2024-09-18T14:18:17.444 INFO:tasks.rados.rados.0.smithi084.stdout:3972: done (3 left) 2024-09-18T14:18:17.444 INFO:tasks.rados.rados.0.smithi084.stdout:3973: done (2 left) 2024-09-18T14:18:17.444 INFO:tasks.rados.rados.0.smithi084.stdout:3974: done (1 left) 2024-09-18T14:18:17.444 INFO:tasks.rados.rados.0.smithi084.stdout:3976: done (0 left) 2024-09-18T14:18:17.444 INFO:tasks.rados.rados.0.smithi084.stdout:3977: write (excl) oid 2 current snap is 449 2024-09-18T14:18:17.444 INFO:tasks.rados.rados.0.smithi084.stdout:3977: seq_num 1332 ranges {786924=593443,1928721=458203} 2024-09-18T14:18:17.451 INFO:tasks.rados.rados.0.smithi084.stdout:3977: writing smithi08434663-2 from 786924 to 1380367 tid 1 2024-09-18T14:18:17.457 INFO:tasks.rados.rados.0.smithi084.stdout:3977: writing smithi08434663-2 from 1928721 to 2386924 tid 2 2024-09-18T14:18:17.458 INFO:tasks.rados.rados.0.smithi084.stdout:3978: read oid 17 snap -1 2024-09-18T14:18:17.458 INFO:tasks.rados.rados.0.smithi084.stdout:3978: expect (ObjNum 1327 snap 447 seq_num 1327) 2024-09-18T14:18:17.458 INFO:tasks.rados.rados.0.smithi084.stdout:3979: rollback oid 45 current snap is 449 2024-09-18T14:18:17.458 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 45 to 422 2024-09-18T14:18:17.458 INFO:tasks.rados.rados.0.smithi084.stdout:3980: copy_from oid 5 from oid 33 current snap is 449 2024-09-18T14:18:17.458 INFO:tasks.rados.rados.0.smithi084.stdout:3981: snap_create 2024-09-18T14:18:17.458 INFO:tasks.rados.rados.0.smithi084.stdout:3979: finishing rollback tid 1 to smithi08434663-45 2024-09-18T14:18:17.458 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 45 v 0 (ObjNum 129 snap 0 seq_num 2494840984) dirty dne 2024-09-18T14:18:17.458 INFO:tasks.rados.rados.0.smithi084.stdout:3980: finishing copy_from to smithi08434663-5 2024-09-18T14:18:17.459 INFO:tasks.rados.rados.0.smithi084.stdout:3980: got expected ENOENT (src dne) 2024-09-18T14:18:17.459 INFO:tasks.rados.rados.0.smithi084.stdout:3980: finishing copy_from racing read to smithi08434663-5 2024-09-18T14:18:17.460 INFO:tasks.rados.rados.0.smithi084.stdout:3977: finishing write tid 1 to smithi08434663-2 2024-09-18T14:18:17.461 INFO:tasks.rados.rados.0.smithi084.stdout:3977: finishing write tid 2 to smithi08434663-2 2024-09-18T14:18:17.462 INFO:tasks.rados.rados.0.smithi084.stdout:3977: finishing write tid 3 to smithi08434663-2 2024-09-18T14:18:17.462 INFO:tasks.rados.rados.0.smithi084.stdout:3977: finishing write tid 4 to smithi08434663-2 2024-09-18T14:18:17.462 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 2 v 2341 (ObjNum 1332 snap 449 seq_num 1332) dirty exists 2024-09-18T14:18:17.462 INFO:tasks.rados.rados.0.smithi084.stdout:3977: left oid 2 (ObjNum 1332 snap 449 seq_num 1332) 2024-09-18T14:18:17.465 INFO:tasks.rados.rados.0.smithi084.stdout:3978: expect (ObjNum 1327 snap 447 seq_num 1327) 2024-09-18T14:18:18.382 INFO:tasks.rados.rados.0.smithi084.stdout:3977: done (4 left) 2024-09-18T14:18:18.382 INFO:tasks.rados.rados.0.smithi084.stdout:3978: done (3 left) 2024-09-18T14:18:18.383 INFO:tasks.rados.rados.0.smithi084.stdout:3979: done (2 left) 2024-09-18T14:18:18.383 INFO:tasks.rados.rados.0.smithi084.stdout:3980: done (1 left) 2024-09-18T14:18:18.383 INFO:tasks.rados.rados.0.smithi084.stdout:3981: done (0 left) 2024-09-18T14:18:18.383 INFO:tasks.rados.rados.0.smithi084.stdout:3982: write oid 30 current snap is 450 2024-09-18T14:18:18.383 INFO:tasks.rados.rados.0.smithi084.stdout:3982: seq_num 1333 ranges {403731=668692,1587203=587797} 2024-09-18T14:18:18.388 INFO:tasks.rados.rados.0.smithi084.stdout:3982: writing smithi08434663-30 from 403731 to 1072423 tid 1 2024-09-18T14:18:18.395 INFO:tasks.rados.rados.0.smithi084.stdout:3982: writing smithi08434663-30 from 1587203 to 2175000 tid 2 2024-09-18T14:18:18.395 INFO:tasks.rados.rados.0.smithi084.stdout:3983: snap_create 2024-09-18T14:18:18.405 INFO:tasks.rados.rados.0.smithi084.stdout:3982: finishing write tid 1 to smithi08434663-30 2024-09-18T14:18:18.421 INFO:tasks.rados.rados.0.smithi084.stdout:3982: finishing write tid 2 to smithi08434663-30 2024-09-18T14:18:18.421 INFO:tasks.rados.rados.0.smithi084.stdout:3982: finishing write tid 3 to smithi08434663-30 2024-09-18T14:18:18.421 INFO:tasks.rados.rados.0.smithi084.stdout:3982: finishing write tid 4 to smithi08434663-30 2024-09-18T14:18:18.421 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 30 v 2252 (ObjNum 1333 snap 450 seq_num 1333) dirty exists 2024-09-18T14:18:18.421 INFO:tasks.rados.rados.0.smithi084.stdout:3982: left oid 30 (ObjNum 1333 snap 450 seq_num 1333) 2024-09-18T14:18:19.395 INFO:tasks.rados.rados.0.smithi084.stdout:3982: done (1 left) 2024-09-18T14:18:19.396 INFO:tasks.rados.rados.0.smithi084.stdout:3983: done (0 left) 2024-09-18T14:18:19.396 INFO:tasks.rados.rados.0.smithi084.stdout:3984: rollback oid 5 current snap is 451 2024-09-18T14:18:19.396 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 5 to 410 2024-09-18T14:18:19.396 INFO:tasks.rados.rados.0.smithi084.stdout:3985: write (excl) oid 16 current snap is 451 2024-09-18T14:18:19.396 INFO:tasks.rados.rados.0.smithi084.stdout:3985: seq_num 1334 ranges {420538=743941,1929332=491206} 2024-09-18T14:18:19.402 INFO:tasks.rados.rados.0.smithi084.stdout:3985: writing smithi08434663-16 from 420538 to 1164479 tid 1 2024-09-18T14:18:19.409 INFO:tasks.rados.rados.0.smithi084.stdout:3985: writing smithi08434663-16 from 1929332 to 2420538 tid 2 2024-09-18T14:18:19.409 INFO:tasks.rados.rados.0.smithi084.stdout:3986: snap_remove snap 440 2024-09-18T14:18:20.400 INFO:tasks.rados.rados.0.smithi084.stdout:3986: done (2 left) 2024-09-18T14:18:20.400 INFO:tasks.rados.rados.0.smithi084.stdout:3987: rollback oid 18 current snap is 451 2024-09-18T14:18:20.400 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 18 to 447 2024-09-18T14:18:20.400 INFO:tasks.rados.rados.0.smithi084.stdout:3984: finishing rollback tid 0 to smithi08434663-5 2024-09-18T14:18:20.400 INFO:tasks.rados.rados.0.smithi084.stdout:3984: finishing rollback tid 1 to smithi08434663-5 2024-09-18T14:18:20.400 INFO:tasks.rados.rados.0.smithi084.stdout:3984: finishing rollback tid 2 to smithi08434663-5 2024-09-18T14:18:20.401 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 5 v 2064 (ObjNum 1232 snap 406 seq_num 1232) dirty exists 2024-09-18T14:18:20.401 INFO:tasks.rados.rados.0.smithi084.stdout:3985: finishing write tid 1 to smithi08434663-16 2024-09-18T14:18:20.401 INFO:tasks.rados.rados.0.smithi084.stdout:3985: finishing write tid 2 to smithi08434663-16 2024-09-18T14:18:20.401 INFO:tasks.rados.rados.0.smithi084.stdout:3985: finishing write tid 3 to smithi08434663-16 2024-09-18T14:18:20.401 INFO:tasks.rados.rados.0.smithi084.stdout:3985: finishing write tid 4 to smithi08434663-16 2024-09-18T14:18:20.401 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 16 v 2068 (ObjNum 1334 snap 451 seq_num 1334) dirty exists 2024-09-18T14:18:20.401 INFO:tasks.rados.rados.0.smithi084.stdout:3985: left oid 16 (ObjNum 1334 snap 451 seq_num 1334) 2024-09-18T14:18:20.401 INFO:tasks.rados.rados.0.smithi084.stdout:3984: done (2 left) 2024-09-18T14:18:20.401 INFO:tasks.rados.rados.0.smithi084.stdout:3985: done (1 left) 2024-09-18T14:18:20.401 INFO:tasks.rados.rados.0.smithi084.stdout:3988: snap_create 2024-09-18T14:18:20.409 INFO:tasks.rados.rados.0.smithi084.stdout:3987: finishing rollback tid 1 to smithi08434663-18 2024-09-18T14:18:20.409 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 18 v 2060 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-09-18T14:18:21.407 INFO:tasks.rados.rados.0.smithi084.stdout:3987: done (1 left) 2024-09-18T14:18:21.407 INFO:tasks.rados.rados.0.smithi084.stdout:3988: done (0 left) 2024-09-18T14:18:21.407 INFO:tasks.rados.rados.0.smithi084.stdout:3989: write (excl) oid 24 current snap is 452 2024-09-18T14:18:21.407 INFO:tasks.rados.rados.0.smithi084.stdout:3989: seq_num 1335 ranges {437345=419190,1587814=591466} 2024-09-18T14:18:21.412 INFO:tasks.rados.rados.0.smithi084.stdout:3989: writing smithi08434663-24 from 437345 to 856535 tid 1 2024-09-18T14:18:21.419 INFO:tasks.rados.rados.0.smithi084.stdout:3989: writing smithi08434663-24 from 1587814 to 2179280 tid 2 2024-09-18T14:18:21.419 INFO:tasks.rados.rados.0.smithi084.stdout:3990: rollback oid 3 current snap is 452 2024-09-18T14:18:21.419 INFO:tasks.rados.rados.0.smithi084.stdout:rollback oid 3 to 442 2024-09-18T14:18:21.419 INFO:tasks.rados.rados.0.smithi084.stdout:3989: finishing write tid 1 to smithi08434663-24 2024-09-18T14:18:21.420 INFO:tasks.rados.rados.0.smithi084.stdout:3991: snap_create 2024-09-18T14:18:21.423 INFO:tasks.rados.rados.0.smithi084.stdout:3989: finishing write tid 2 to smithi08434663-24 2024-09-18T14:18:21.423 INFO:tasks.rados.rados.0.smithi084.stdout:3989: finishing write tid 3 to smithi08434663-24 2024-09-18T14:18:21.423 INFO:tasks.rados.rados.0.smithi084.stdout:3989: finishing write tid 4 to smithi08434663-24 2024-09-18T14:18:21.424 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 24 v 2134 (ObjNum 1335 snap 452 seq_num 1335) dirty exists 2024-09-18T14:18:21.424 INFO:tasks.rados.rados.0.smithi084.stdout:3989: left oid 24 (ObjNum 1335 snap 452 seq_num 1335) 2024-09-18T14:18:21.424 INFO:tasks.rados.rados.0.smithi084.stdout:3990: finishing rollback tid 0 to smithi08434663-3 2024-09-18T14:18:21.424 INFO:tasks.rados.rados.0.smithi084.stdout:3990: finishing rollback tid 1 to smithi08434663-3 2024-09-18T14:18:21.427 INFO:tasks.rados.rados.0.smithi084.stdout:3990: finishing rollback tid 2 to smithi08434663-3 2024-09-18T14:18:21.427 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 3 v 2069 (ObjNum 967 snap 308 seq_num 967) dirty exists 2024-09-18T14:18:22.418 INFO:tasks.rados.rados.0.smithi084.stdout:3989: done (2 left) 2024-09-18T14:18:22.418 INFO:tasks.rados.rados.0.smithi084.stdout:3990: done (1 left) 2024-09-18T14:18:22.418 INFO:tasks.rados.rados.0.smithi084.stdout:3991: done (0 left) 2024-09-18T14:18:22.418 INFO:tasks.rados.rados.0.smithi084.stdout:3992: read oid 15 snap -1 2024-09-18T14:18:22.418 INFO:tasks.rados.rados.0.smithi084.stdout:3992: expect (ObjNum 1176 snap 387 seq_num 1176) 2024-09-18T14:18:22.418 INFO:tasks.rados.rados.0.smithi084.stdout:3993: write oid 31 current snap is 453 2024-09-18T14:18:22.419 INFO:tasks.rados.rados.0.smithi084.stdout:3993: seq_num 1336 ranges {454152=494439,1646296=735124} 2024-09-18T14:18:22.423 INFO:tasks.rados.rados.0.smithi084.stdout:3993: writing smithi08434663-31 from 454152 to 948591 tid 1 2024-09-18T14:18:22.431 INFO:tasks.rados.rados.0.smithi084.stdout:3993: writing smithi08434663-31 from 1646296 to 2381420 tid 2 2024-09-18T14:18:22.432 INFO:tasks.rados.rados.0.smithi084.stdout:3994: delete oid 28 current snap is 453 2024-09-18T14:18:22.432 INFO:tasks.rados.rados.0.smithi084.stdout:3993: finishing write tid 1 to smithi08434663-31 2024-09-18T14:18:22.432 INFO:tasks.rados.rados.0.smithi084.stdout:3992: expect (ObjNum 1176 snap 387 seq_num 1176) 2024-09-18T14:18:22.554 INFO:tasks.rados.rados.0.smithi084.stdout:3993: finishing write tid 2 to smithi08434663-31 2024-09-18T14:18:22.554 INFO:tasks.rados.rados.0.smithi084.stdout:3993: finishing write tid 3 to smithi08434663-31 2024-09-18T14:18:22.554 INFO:tasks.rados.rados.0.smithi084.stdout:3993: finishing write tid 4 to smithi08434663-31 2024-09-18T14:18:22.554 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 31 v 2137 (ObjNum 1336 snap 453 seq_num 1336) dirty exists 2024-09-18T14:18:22.554 INFO:tasks.rados.rados.0.smithi084.stdout:3993: left oid 31 (ObjNum 1336 snap 453 seq_num 1336) 2024-09-18T14:18:22.555 INFO:tasks.rados.rados.0.smithi084.stdout:3992: done (2 left) 2024-09-18T14:18:22.555 INFO:tasks.rados.rados.0.smithi084.stdout:3993: done (1 left) 2024-09-18T14:18:22.555 INFO:tasks.rados.rados.0.smithi084.stdout:3994: done (0 left) 2024-09-18T14:18:22.555 INFO:tasks.rados.rados.0.smithi084.stdout:3995: read oid 12 snap -1 2024-09-18T14:18:22.555 INFO:tasks.rados.rados.0.smithi084.stdout:3995: expect (ObjNum 1287 snap 431 seq_num 1287) 2024-09-18T14:18:22.555 INFO:tasks.rados.rados.0.smithi084.stdout:3996: write oid 43 current snap is 453 2024-09-18T14:18:22.555 INFO:tasks.rados.rados.0.smithi084.stdout:3996: seq_num 1338 ranges {487766=644937,1646907=738793} 2024-09-18T14:18:22.560 INFO:tasks.rados.rados.0.smithi084.stdout:3996: writing smithi08434663-43 from 487766 to 1132703 tid 1 2024-09-18T14:18:22.567 INFO:tasks.rados.rados.0.smithi084.stdout:3996: writing smithi08434663-43 from 1646907 to 2385700 tid 2 2024-09-18T14:18:22.567 INFO:tasks.rados.rados.0.smithi084.stdout:3997: delete oid 24 current snap is 453 2024-09-18T14:18:22.567 INFO:tasks.rados.rados.0.smithi084.stdout:3995: expect (ObjNum 1287 snap 431 seq_num 1287) 2024-09-18T14:18:22.733 INFO:tasks.rados.rados.0.smithi084.stdout:3996: finishing write tid 1 to smithi08434663-43 2024-09-18T14:18:22.733 INFO:tasks.rados.rados.0.smithi084.stdout:3996: finishing write tid 2 to smithi08434663-43 2024-09-18T14:18:22.733 INFO:tasks.rados.rados.0.smithi084.stdout:3996: finishing write tid 3 to smithi08434663-43 2024-09-18T14:18:22.733 INFO:tasks.rados.rados.0.smithi084.stdout:3996: finishing write tid 4 to smithi08434663-43 2024-09-18T14:18:22.733 INFO:tasks.rados.rados.0.smithi084.stdout:update_object_version oid 43 v 2054 (ObjNum 1338 snap 453 seq_num 1338) dirty exists 2024-09-18T14:18:22.733 INFO:tasks.rados.rados.0.smithi084.stdout:3996: left oid 43 (ObjNum 1338 snap 453 seq_num 1338) 2024-09-18T14:18:22.733 INFO:tasks.rados.rados.0.smithi084.stdout:3995: done (2 left) 2024-09-18T14:18:22.733 INFO:tasks.rados.rados.0.smithi084.stdout:3996: done (1 left) 2024-09-18T14:18:22.733 INFO:tasks.rados.rados.0.smithi084.stdout:3997: done (0 left) 2024-09-18T14:18:22.733 INFO:tasks.rados.rados.0.smithi084.stdout:3998: snap_create 2024-09-18T14:18:23.382 INFO:tasks.rados.rados.0.smithi084.stdout:3998: done (0 left) 2024-09-18T14:18:23.383 INFO:tasks.rados.rados.0.smithi084.stdout:3999: read oid 19 snap -1 2024-09-18T14:18:23.383 INFO:tasks.rados.rados.0.smithi084.stdout:3999: expect (ObjNum 1288 snap 431 seq_num 1288) 2024-09-18T14:18:23.383 INFO:tasks.rados.rados.0.smithi084.stdout: waiting on 1 2024-09-18T14:18:23.397 INFO:tasks.rados.rados.0.smithi084.stdout:3999: expect (ObjNum 1288 snap 431 seq_num 1288) 2024-09-18T14:18:23.516 INFO:tasks.rados.rados.0.smithi084.stdout:3999: done (0 left) 2024-09-18T14:18:23.517 INFO:tasks.rados.rados.0.smithi084.stderr:0 errors. 2024-09-18T14:18:23.518 INFO:tasks.rados.rados.0.smithi084.stderr: 2024-09-18T14:18:23.520 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T14:18:23.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.624+0000 7f1e51d9d640 10 monclient: get_monmap_and_config 2024-09-18T14:18:23.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.624+0000 7f1e51d9d640 10 monclient: build_initial_monmap 2024-09-18T14:18:23.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.624+0000 7f1e51d9d640 10 monclient: monmap: 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:18:23.625015+0000 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:18:23.625015+0000 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:18:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:18:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:18:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:18:23.626 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.624+0000 7f1e51d9d640 1 Processor -- start 2024-09-18T14:18:23.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.624+0000 7f1e51d9d640 1 -- start start 2024-09-18T14:18:23.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.624+0000 7f1e51d9d640 10 monclient: init 2024-09-18T14:18:23.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.625+0000 7f1e51d9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:18:23.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.625+0000 7f1e51d9d640 10 monclient: _add_conns ranks=[5,1,7,6,2,4,8,0,3] 2024-09-18T14:18:23.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.625+0000 7f1e51d9d640 10 monclient(hunting): picked mon.noname-h con 0x7f1e4c06e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:18:23.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.625+0000 7f1e51d9d640 10 monclient(hunting): picked mon.noname-f con 0x7f1e4c0a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:18:23.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.625+0000 7f1e51d9d640 10 monclient(hunting): picked mon.noname-i con 0x7f1e4c0a1080 addr v1:172.21.15.84:6792/0 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.625+0000 7f1e51d9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4c05ed70 con 0x7f1e4c0a0cb0 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.625+0000 7f1e51d9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4c097620 con 0x7f1e4c06e810 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.625+0000 7f1e51d9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4c097340 con 0x7f1e4c0a1080 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.625+0000 7f1e51d9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.625+0000 7f1e51d9d640 10 monclient(hunting): authenticate will time out at 1918.963398s 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.625+0000 7f1e4b7fe640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f1e4c06e810 0x7f1e4c0a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:44956/0 (socket says 172.21.15.45:44956) 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.625+0000 7f1e4b7fe640 1 -- 172.21.15.45:0/2290144772 learned_addr learned my addr 172.21.15.45:0/2290144772 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 598367261 0 0) 0x7f1e4c097620 con 0x7f1e4c06e810 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient(hunting): my global_id is 185027 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e340034e0 con 0x7f1e4c06e810 2024-09-18T14:18:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134527940 0 0) 0x7f1e4c05ed70 con 0x7f1e4c0a0cb0 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient(hunting): my global_id is 214633 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e4c097620 con 0x7f1e4c0a0cb0 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393488144 0 0) 0x7f1e4c097340 con 0x7f1e4c0a1080 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient(hunting): my global_id is 214576 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e4c05ed70 con 0x7f1e4c0a1080 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434321413 0 0) 0x7f1e340034e0 con 0x7f1e4c06e810 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 >> v1:172.21.15.84:6792/0 conn(0x7f1e4c0a1080 legacy=0x7f1e4c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 >> v1:172.21.15.84:6789/0 conn(0x7f1e4c0a0cb0 legacy=0x7f1e4c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 monclient: found mon.noname-h 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c0a1660 con 0x7f1e4c06e810 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1e3c001830 con 0x7f1e4c06e810 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e51d9d640 5 monclient: authenticate success, global_id 185027 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient: dump: 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:23.637 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e51d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e3c001a10 con 0x7f1e4c06e810 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e51d9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e51d9d640 4 set_mon_vals no callback set 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 1 -- 172.21.15.45:0/2290144772 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1e3c004ba0 con 0x7f1e4c06e810 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e4a7fc640 10 monclient: dump: 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e51d9d640 10 monclient: shutdown 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e51d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e51d9d640 1 -- 172.21.15.45:0/2290144772 >> v1:172.21.15.84:6791/0 conn(0x7f1e4c06e810 legacy=0x7f1e4c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e51d9d640 1 -- 172.21.15.45:0/2290144772 shutdown_connections 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.626+0000 7f1e51d9d640 1 -- 172.21.15.45:0/2290144772 >> 172.21.15.45:0/2290144772 conn(0x7f1e4c0a5bb0 msgr2=0x7f1e4c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.627+0000 7f1e51d9d640 1 -- 172.21.15.45:0/2290144772 shutdown_connections 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.627+0000 7f1e51d9d640 1 -- 172.21.15.45:0/2290144772 wait complete. 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.627+0000 7f1e51d9d640 10 monclient: build_initial_monmap 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.627+0000 7f1e51d9d640 10 monclient: monmap: 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:18:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 1 Processor -- start 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 1 -- start start 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 10 monclient: init 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:18:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 10 monclient: _add_conns ranks=[5,6,8,1,7,0,2,4,3] 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 10 monclient(hunting): picked mon.noname-h con 0x7f1e4c06e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 10 monclient(hunting): picked mon.noname-d con 0x7f1e4c0a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 10 monclient(hunting): picked mon.noname-e con 0x7f1e4c0a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4c122e30 con 0x7f1e4c06e810 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4c13dc20 con 0x7f1e4c0a0cb0 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4c122870 con 0x7f1e4c0a1080 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e51d9d640 10 monclient(hunting): authenticate will time out at 1918.966626s 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e4bfff640 1 -- >> v1:172.21.15.45:6793/0 conn(0x7f1e4c0a1080 legacy=0x7f1e4c149e70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e4affd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f1e4c0a0cb0 0x7f1e4c146740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56072/0 (socket says 172.21.15.45:56072) 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e4affd640 1 -- 172.21.15.45:0/3885056200 learned_addr learned my addr 172.21.15.45:0/3885056200 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620310478 0 0) 0x7f1e4c13dc20 con 0x7f1e4c0a0cb0 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e337fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e337fe640 10 monclient(hunting): my global_id is 185154 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e380034e0 con 0x7f1e4c0a0cb0 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722066245 0 0) 0x7f1e4c122e30 con 0x7f1e4c06e810 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e337fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e337fe640 10 monclient(hunting): my global_id is 185036 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.628+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e4c13dc20 con 0x7f1e4c06e810 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1410015651 0 0) 0x7f1e4c13dc20 con 0x7f1e4c06e810 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e4c122e30 con 0x7f1e4c06e810 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1e3c001830 con 0x7f1e4c06e810 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient(hunting): dump: 2024-09-18T14:18:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3268184232 0 0) 0x7f1e380034e0 con 0x7f1e4c0a0cb0 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e4c13dc20 con 0x7f1e4c0a0cb0 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1e40001d90 con 0x7f1e4c0a0cb0 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient(hunting): dump: 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3914814471 0 0) 0x7f1e4c122e30 con 0x7f1e4c06e810 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 >> v1:172.21.15.45:6793/0 conn(0x7f1e4c0a1080 legacy=0x7f1e4c149e70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 >> v1:172.21.15.45:6792/0 conn(0x7f1e4c0a0cb0 legacy=0x7f1e4c146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient: _finish_hunting 0 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 monclient: found mon.h 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c122590 con 0x7f1e4c06e810 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient: _finish_auth 0 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e51d9d640 5 monclient: authenticate success, global_id 185036 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e51d9d640 10 monclient: _renew_subs 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e51d9d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e51d9d640 1 -- 172.21.15.45:0/3885056200 --> v1:172.21.15.84:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c122b50 con 0x7f1e4c06e810 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e51d9d640 10 monclient: _renew_subs 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e51d9d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e51d9d640 1 -- 172.21.15.45:0/3885056200 --> v1:172.21.15.84:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4c1216c0 con 0x7f1e4c06e810 2024-09-18T14:18:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.5 v1:172.21.15.84:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e3c004b00 con 0x7f1e4c06e810 2024-09-18T14:18:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:18:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.5 v1:172.21.15.84:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1e3c005160 con 0x7f1e4c06e810 2024-09-18T14:18:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:18:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e337fe640 10 monclient: dump: 2024-09-18T14:18:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.629+0000 7f1e49ffb640 4 set_mon_vals no callback set 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.630+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.5 v1:172.21.15.84:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1e3c010130 con 0x7f1e4c06e810 2024-09-18T14:18:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.630+0000 7f1e51d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:18:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.630+0000 7f1e51d9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:18:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.630+0000 7f1e51d9d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:18:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.630+0000 7f1e51d9d640 1 -- 172.21.15.45:0/3885056200 --> v1:172.21.15.84:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4c04c4a0 con 0x7f1e4c06e810 2024-09-18T14:18:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.630+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.5 v1:172.21.15.84:6791/0 8 ==== osd_map(897..897 src has 637..897) v4 ==== 3943+0+0 (unknown 1321444408 0 0) 0x7f1e3c046c10 con 0x7f1e4c06e810 2024-09-18T14:18:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.630+0000 7f1e4affd640 10 monclient: get_auth_request con 0x7f1e3803a440 auth_method 0 2024-09-18T14:18:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.635+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.5 v1:172.21.15.84:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f1e3c0153d0 con 0x7f1e4c06e810 2024-09-18T14:18:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.635+0000 7f1e337fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:18:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.635+0000 7f1e337fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:18:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.797+0000 7f1e51d9d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-18T14:18:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.797+0000 7f1e51d9d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T14:18:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.797+0000 7f1e51d9d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.84:6791/0 2024-09-18T14:18:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.797+0000 7f1e51d9d640 1 -- 172.21.15.45:0/3885056200 --> v1:172.21.15.84:6791/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e4c04b720 con 0x7f1e4c06e810 2024-09-18T14:18:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.798+0000 7f1e337fe640 1 -- 172.21.15.45:0/3885056200 <== mon.5 v1:172.21.15.84:6791/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v897) v1 ==== 74+0+11656 (unknown 436117740 0 3386085349) 0x7f1e3c014dd0 con 0x7f1e4c06e810 2024-09-18T14:18:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.798+0000 7f1e337fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T14:18:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.798+0000 7f1e337fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:18:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T14:18:23.799 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":897,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","created":"2024-09-18T13:58:52.168851+0000","modified":"2024-09-18T14:18:23.376379+0000","last_up_change":"2024-09-18T13:59:00.910314+0000","last_in_change":"2024-09-18T13:58:53.737759+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T13:59:01.925362+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-18T13:59:01.954982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-09-18T13:59:10.966639+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"897","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":875,"snap_epoch":897,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"8c3a8909-07ce-4141-98dd-131951f18ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6804","nonce":32276}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6805","nonce":32276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6807","nonce":32276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6806","nonce":32276}]},"public_addr":"172.21.15.45:6804/32276","cluster_addr":"172.21.15.45:6805/32276","heartbeat_back_addr":"172.21.15.45:6807/32276","heartbeat_front_addr":"172.21.15.45:6806/32276","state":["exists","up"]},{"osd":1,"uuid":"a12b3bc3-1e03-47dc-98c1-c8bd199551a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6800","nonce":32277}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6801","nonce":32277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6803","nonce":32277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6802","nonce":32277}]},"public_addr":"172.21.15.45:6800/32277","cluster_addr":"172.21.15.45:6801/32277","heartbeat_back_addr":"172.21.15.45:6803/32277","heartbeat_front_addr":"172.21.15.45:6802/32277","state":["exists","up"]},{"osd":2,"uuid":"216b8b62-e41c-4bc5-9395-f2d56d402b1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6808","nonce":32278}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6809","nonce":32278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6811","nonce":32278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6810","nonce":32278}]},"public_addr":"172.21.15.45:6808/32278","cluster_addr":"172.21.15.45:6809/32278","heartbeat_back_addr":"172.21.15.45:6811/32278","heartbeat_front_addr":"172.21.15.45:6810/32278","state":["exists","up"]},{"osd":3,"uuid":"c750b0e6-9143-489d-9be0-eb085178ff2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6802","nonce":32024}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6804","nonce":32024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6807","nonce":32024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6806","nonce":32024}]},"public_addr":"172.21.15.84:6802/32024","cluster_addr":"172.21.15.84:6804/32024","heartbeat_back_addr":"172.21.15.84:6807/32024","heartbeat_front_addr":"172.21.15.84:6806/32024","state":["exists","up"]},{"osd":4,"uuid":"9680908d-dc81-44a0-8603-93711579d42e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6800","nonce":32025}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6801","nonce":32025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6805","nonce":32025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6803","nonce":32025}]},"public_addr":"172.21.15.84:6800/32025","cluster_addr":"172.21.15.84:6801/32025","heartbeat_back_addr":"172.21.15.84:6805/32025","heartbeat_front_addr":"172.21.15.84:6803/32025","state":["exists","up"]},{"osd":5,"uuid":"b3b63eef-c8fc-495f-960f-7436315119ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6808","nonce":32048}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6809","nonce":32048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6811","nonce":32048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6810","nonce":32048}]},"public_addr":"172.21.15.84:6808/32048","cluster_addr":"172.21.15.84:6809/32048","heartbeat_back_addr":"172.21.15.84:6811/32048","heartbeat_front_addr":"172.21.15.84:6810/32048","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.823226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.883512+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.904155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.801736+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.835588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.910413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":832,"length":1},{"begin":854,"length":1},{"begin":858,"length":1},{"begin":866,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T14:18:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.800+0000 7f1e51d9d640 1 -- 172.21.15.45:0/3885056200 >> v1:172.21.15.84:6812/31478 conn(0x7f1e3803a440 legacy=0x7f1e3803c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.800+0000 7f1e51d9d640 10 monclient: shutdown 2024-09-18T14:18:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.800+0000 7f1e51d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:18:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.800+0000 7f1e51d9d640 1 -- 172.21.15.45:0/3885056200 >> v1:172.21.15.84:6791/0 conn(0x7f1e4c06e810 legacy=0x7f1e4c13d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.800+0000 7f1e51d9d640 1 -- 172.21.15.45:0/3885056200 shutdown_connections 2024-09-18T14:18:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.800+0000 7f1e51d9d640 1 -- 172.21.15.45:0/3885056200 >> 172.21.15.45:0/3885056200 conn(0x7f1e4c0a5bb0 msgr2=0x7f1e4c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:18:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.800+0000 7f1e51d9d640 1 -- 172.21.15.45:0/3885056200 shutdown_connections 2024-09-18T14:18:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.800+0000 7f1e51d9d640 1 -- 172.21.15.45:0/3885056200 wait complete. 2024-09-18T14:18:23.812 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T14:18:23.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.907+0000 7f8b1599d640 10 monclient: get_monmap_and_config 2024-09-18T14:18:23.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.907+0000 7f8b1599d640 10 monclient: build_initial_monmap 2024-09-18T14:18:23.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.907+0000 7f8b1599d640 10 monclient: monmap: 2024-09-18T14:18:23.908 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:18:23.908 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:23.908 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:18:23.908726+0000 2024-09-18T14:18:23.908 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:18:23.908726+0000 2024-09-18T14:18:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:18:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:18:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:18:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:18:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:18:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:18:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:18:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:18:23.912 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:18:23.912 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:18:23.912 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.908+0000 7f8b1599d640 1 Processor -- start 2024-09-18T14:18:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.908+0000 7f8b1599d640 1 -- start start 2024-09-18T14:18:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.908+0000 7f8b1599d640 10 monclient: init 2024-09-18T14:18:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.908+0000 7f8b1599d640 10 monclient: _reopen_session rank -1 2024-09-18T14:18:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.908+0000 7f8b1599d640 10 monclient: _add_conns ranks=[1,3,0,7,8,5,4,2,6] 2024-09-18T14:18:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.908+0000 7f8b1599d640 10 monclient(hunting): picked mon.noname-f con 0x7f8b1006e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:18:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.908+0000 7f8b1599d640 10 monclient(hunting): picked mon.noname-g con 0x7f8b100a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:18:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.908+0000 7f8b1599d640 10 monclient(hunting): picked mon.noname-a con 0x7f8b100a1080 addr v1:172.21.15.45:6789/0 2024-09-18T14:18:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.908+0000 7f8b1599d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1005ed70 con 0x7f8b100a1080 2024-09-18T14:18:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.908+0000 7f8b1599d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b10097620 con 0x7f8b1006e810 2024-09-18T14:18:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b1599d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b10097340 con 0x7f8b100a0cb0 2024-09-18T14:18:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b1599d640 10 monclient(hunting): _renew_subs 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b1599d640 10 monclient(hunting): authenticate will time out at 1919.247023s 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0f7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8b100a1080 0x7f8b1013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59848/0 (socket says 172.21.15.45:59848) 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0f7fe640 1 -- 172.21.15.45:0/3994993184 learned_addr learned my addr 172.21.15.45:0/3994993184 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384605375 0 0) 0x7f8b1005ed70 con 0x7f8b100a1080 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient(hunting): my global_id is 6615 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8af80034e0 con 0x7f8b100a1080 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466302180 0 0) 0x7f8b10097340 con 0x7f8b100a0cb0 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient(hunting): my global_id is 224310 2024-09-18T14:18:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b1005ed70 con 0x7f8b100a0cb0 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106806460 0 0) 0x7f8b10097620 con 0x7f8b1006e810 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient(hunting): my global_id is 214642 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b10097340 con 0x7f8b1006e810 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1430599983 0 0) 0x7f8af80034e0 con 0x7f8b100a1080 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 >> v1:172.21.15.84:6790/0 conn(0x7f8b100a0cb0 legacy=0x7f8b10137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 >> v1:172.21.15.84:6789/0 conn(0x7f8b1006e810 legacy=0x7f8b100a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient: _finish_hunting 0 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 monclient: found mon.noname-a 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:18:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b100a1660 con 0x7f8b100a1080 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient: _finish_auth 0 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b08001bb0 con 0x7f8b100a1080 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b1599d640 5 monclient: authenticate success, global_id 6615 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b1599d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient: dump: 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b1599d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:18:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8b08001d90 con 0x7f8b100a1080 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 1 -- 172.21.15.45:0/3994993184 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b08004f20 con 0x7f8b100a1080 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b1599d640 10 monclient: get_monmap_and_config success 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b1599d640 4 set_mon_vals no callback set 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.909+0000 7f8b0dffb640 10 monclient: dump: 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:23.928 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.910+0000 7f8b1599d640 10 monclient: shutdown 2024-09-18T14:18:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.910+0000 7f8b1599d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:18:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.910+0000 7f8b1599d640 1 -- 172.21.15.45:0/3994993184 >> v1:172.21.15.45:6789/0 conn(0x7f8b100a1080 legacy=0x7f8b1013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.910+0000 7f8b1599d640 1 -- 172.21.15.45:0/3994993184 shutdown_connections 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.910+0000 7f8b1599d640 1 -- 172.21.15.45:0/3994993184 >> 172.21.15.45:0/3994993184 conn(0x7f8b100a5bb0 msgr2=0x7f8b100a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.910+0000 7f8b1599d640 1 -- 172.21.15.45:0/3994993184 shutdown_connections 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.910+0000 7f8b1599d640 1 -- 172.21.15.45:0/3994993184 wait complete. 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.910+0000 7f8b1599d640 10 monclient: build_initial_monmap 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.910+0000 7f8b1599d640 10 monclient: monmap: 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:18:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 1 Processor -- start 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 1 -- start start 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 10 monclient: init 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 10 monclient: _reopen_session rank -1 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 10 monclient: _add_conns ranks=[0,3,6,1,8,4,5,2,7] 2024-09-18T14:18:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 10 monclient(hunting): picked mon.noname-a con 0x7f8b1006e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 10 monclient(hunting): picked mon.noname-g con 0x7f8b100a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 10 monclient(hunting): picked mon.noname-d con 0x7f8b100a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1009fa10 con 0x7f8b1006e810 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b100a2270 con 0x7f8b100a0cb0 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b10120e70 con 0x7f8b100a1080 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 10 monclient(hunting): _renew_subs 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.911+0000 7f8b1599d640 10 monclient(hunting): authenticate will time out at 1919.249975s 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b0effd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8b1006e810 0x7f8b10135370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59860/0 (socket says 172.21.15.45:59860) 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b0effd640 1 -- 172.21.15.45:0/1351200643 learned_addr learned my addr 172.21.15.45:0/1351200643 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891031813 0 0) 0x7f8b1009fa10 con 0x7f8b1006e810 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): my global_id is 6624 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8afc0034e0 con 0x7f8b1006e810 2024-09-18T14:18:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144449207 0 0) 0x7f8b10120e70 con 0x7f8b100a1080 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): my global_id is 185163 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b1009fa10 con 0x7f8b100a1080 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127609294 0 0) 0x7f8b100a2270 con 0x7f8b100a0cb0 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): my global_id is 224319 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b10120e70 con 0x7f8b100a0cb0 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1593120480 0 0) 0x7f8afc0034e0 con 0x7f8b1006e810 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b0effd640 1 -- 172.21.15.45:0/1351200643 >> v1:172.21.15.45:6789/0 conn(0x7f8b1006e810 legacy=0x7f8b10135370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b0effd640 1 -- 172.21.15.45:0/1351200643 >> v1:172.21.15.45:6789/0 conn(0x7f8b1006e810 legacy=0x7f8b10135370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:18:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b100a2270 con 0x7f8b1006e810 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b0effd640 1 --1- 172.21.15.45:0/1351200643 >> v1:172.21.15.45:6789/0 conn(0x7f8b1006e810 0x7f8b10135370 :-1 s=OPENED pgs=738 cs=1 l=1).handle_message read tag failed 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b0effd640 1 --1- 172.21.15.45:0/1351200643 >> v1:172.21.15.45:6789/0 conn(0x7f8b1006e810 0x7f8b10135370 :-1 s=OPENED pgs=738 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3118433261 0 0) 0x7f8b1009fa10 con 0x7f8b100a1080 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8afc0034e0 con 0x7f8b100a1080 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.45:6789/0 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b08001b70 con 0x7f8b100a1080 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): dump: 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.933 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:18:23.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1945961162 0 0) 0x7f8afc0034e0 con 0x7f8b100a1080 2024-09-18T14:18:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 >> v1:172.21.15.84:6790/0 conn(0x7f8b100a0cb0 legacy=0x7f8b101468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 >> v1:172.21.15.45:6789/0 conn(0x7f8b1006e810 legacy=0x7f8b10135370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:18:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:18:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 monclient: found mon.d 2024-09-18T14:18:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:18:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:18:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b10122a50 con 0x7f8b100a1080 2024-09-18T14:18:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient: _finish_auth 0 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b1599d640 5 monclient: authenticate success, global_id 185163 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b1599d640 10 monclient: _renew_subs 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b1599d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b1599d640 1 -- 172.21.15.45:0/1351200643 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b10122490 con 0x7f8b100a1080 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b1599d640 10 monclient: _renew_subs 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b1599d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b1599d640 1 -- 172.21.15.45:0/1351200643 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b10121530 con 0x7f8b100a1080 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8b08004dd0 con 0x7f8b100a1080 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f8b080015d0 con 0x7f8b100a1080 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8b0d7fa640 4 set_mon_vals no callback set 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.912+0000 7f8af6ffd640 10 monclient: dump: 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.913+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f8b08010170 con 0x7f8b100a1080 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.913+0000 7f8b1599d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.913+0000 7f8b1599d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.913+0000 7f8b1599d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.913+0000 7f8b1599d640 1 -- 172.21.15.45:0/1351200643 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af0005010 con 0x7f8b100a1080 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.913+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(897..897 src has 637..897) v4 ==== 3943+0+0 (unknown 1321444408 0 0) 0x7f8b08046db0 con 0x7f8b100a1080 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.914+0000 7f8b0effd640 10 monclient: get_auth_request con 0x7f8afc03a120 auth_method 0 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.918+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f8b08015640 con 0x7f8b100a1080 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.918+0000 7f8af6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:18:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:23.918+0000 7f8af6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:18:24.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.078+0000 7f8b1599d640 1 -- 172.21.15.45:0/1351200643 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8af0000b90 con 0x7f8afc03a120 2024-09-18T14:18:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.082+0000 7f8af6ffd640 1 -- 172.21.15.45:0/1351200643 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+97833 (unknown 2965378022 0 2124417310) 0x7f8af0000b90 con 0x7f8afc03a120 2024-09-18T14:18:24.083 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T14:18:24.083 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-18T14:18:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.084+0000 7f8b1599d640 1 -- 172.21.15.45:0/1351200643 >> v1:172.21.15.84:6812/31478 conn(0x7f8afc03a120 legacy=0x7f8afc03c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.084+0000 7f8b1599d640 10 monclient: shutdown 2024-09-18T14:18:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.084+0000 7f8b1599d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:18:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.084+0000 7f8b1599d640 1 -- 172.21.15.45:0/1351200643 >> v1:172.21.15.45:6792/0 conn(0x7f8b100a1080 legacy=0x7f8b10149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:24.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.084+0000 7f8b1599d640 1 -- 172.21.15.45:0/1351200643 shutdown_connections 2024-09-18T14:18:24.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.084+0000 7f8b1599d640 1 -- 172.21.15.45:0/1351200643 >> 172.21.15.45:0/1351200643 conn(0x7f8b100a5bb0 msgr2=0x7f8b100a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:18:24.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.084+0000 7f8b1599d640 1 -- 172.21.15.45:0/1351200643 shutdown_connections 2024-09-18T14:18:24.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.084+0000 7f8b1599d640 1 -- 172.21.15.45:0/1351200643 wait complete. 2024-09-18T14:18:24.095 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":1473,"stamp":"2024-09-18T14:18:23.384166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":437611493,"num_objects":205,"num_object_clones":152,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":205,"num_whiteouts":12,"num_read":7580,"num_read_kb":7385360,"num_write":5302,"num_write_kb":2033726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18391,"ondisk_log_size":18391,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":2350028,"kb_used_data":540912,"kb_used_omap":9,"kb_used_meta":1809014,"kb_avail":563881012,"statfs":{"total":579820584960,"available":577414156288,"internally_reserved":0,"allocated":553893888,"data_stored":716478575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1852430352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8341636,"num_objects":3,"num_object_clones":3,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":-1,"num_read":13,"num_read_kb":5554,"num_write":16,"num_write_kb":3464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.991185"},"pg_stats":[{"pgid":"3.f","version":"894'1605","reported_seq":4287,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.409263+0000","last_change":"2024-09-18T14:18:20.409435+0000","last_active":"2024-09-18T14:18:20.409263+0000","last_peered":"2024-09-18T14:18:20.409263+0000","last_clean":"2024-09-18T14:18:20.409263+0000","last_became_active":"2024-09-18T13:59:12.095873+0000","last_became_peered":"2024-09-18T13:59:12.095873+0000","last_unstale":"2024-09-18T14:18:20.409263+0000","last_undegraded":"2024-09-18T14:18:20.409263+0000","last_fullsized":"2024-09-18T14:18:20.409263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":1605,"log_dups_size":0,"ondisk_log_size":1605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:40:39.010005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072348009999999999,"stat_sum":{"num_bytes":32144807,"num_objects":14,"num_object_clones":10,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":726,"num_read_kb":716171,"num_write":418,"num_write_kb":181211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"3.e","version":"894'1075","reported_seq":3774,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.409173+0000","last_change":"2024-09-18T14:18:20.409385+0000","last_active":"2024-09-18T14:18:20.409173+0000","last_peered":"2024-09-18T14:18:20.409173+0000","last_clean":"2024-09-18T14:18:20.409173+0000","last_became_active":"2024-09-18T13:59:12.093191+0000","last_became_peered":"2024-09-18T13:59:12.093191+0000","last_unstale":"2024-09-18T14:18:20.409173+0000","last_undegraded":"2024-09-18T14:18:20.409173+0000","last_fullsized":"2024-09-18T14:18:20.409173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:41:55.592912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072728469999999998,"stat_sum":{"num_bytes":25942543,"num_objects":10,"num_object_clones":7,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":458,"num_read_kb":456372,"num_write":308,"num_write_kb":125151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"3.d","version":"894'1873","reported_seq":4520,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.416707+0000","last_change":"2024-09-18T14:18:20.416931+0000","last_active":"2024-09-18T14:18:20.416707+0000","last_peered":"2024-09-18T14:18:20.416707+0000","last_clean":"2024-09-18T14:18:20.416707+0000","last_became_active":"2024-09-18T13:59:12.084519+0000","last_became_peered":"2024-09-18T13:59:12.084519+0000","last_unstale":"2024-09-18T14:18:20.416707+0000","last_undegraded":"2024-09-18T14:18:20.416707+0000","last_fullsized":"2024-09-18T14:18:20.416707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":1873,"log_dups_size":0,"ondisk_log_size":1873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T01:00:56.398496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010838347,"stat_sum":{"num_bytes":43045674,"num_objects":18,"num_object_clones":13,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":678,"num_read_kb":678686,"num_write":553,"num_write_kb":221062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":2654,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.402299+0000","last_change":"2024-09-18T14:18:20.402927+0000","last_active":"2024-09-18T14:18:20.402299+0000","last_peered":"2024-09-18T14:18:20.402299+0000","last_clean":"2024-09-18T14:18:20.402299+0000","last_became_active":"2024-09-18T13:59:12.093132+0000","last_became_peered":"2024-09-18T13:59:12.093132+0000","last_unstale":"2024-09-18T14:18:20.402299+0000","last_undegraded":"2024-09-18T14:18:20.402299+0000","last_fullsized":"2024-09-18T14:18:20.402299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T00:35:56.526224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087634400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"3.b","version":"894'1132","reported_seq":3814,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.404949+0000","last_change":"2024-09-18T14:18:20.404949+0000","last_active":"2024-09-18T14:18:20.404949+0000","last_peered":"2024-09-18T14:18:20.404949+0000","last_clean":"2024-09-18T14:18:20.404949+0000","last_became_active":"2024-09-18T13:59:12.081066+0000","last_became_peered":"2024-09-18T13:59:12.081066+0000","last_unstale":"2024-09-18T14:18:20.404949+0000","last_undegraded":"2024-09-18T14:18:20.404949+0000","last_fullsized":"2024-09-18T14:18:20.404949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":1132,"log_dups_size":0,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T22:44:34.300869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057258040000000001,"stat_sum":{"num_bytes":37337246,"num_objects":16,"num_object_clones":13,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":410,"num_read_kb":330926,"num_write":356,"num_write_kb":134852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"3.a","version":"894'688","reported_seq":3438,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.417140+0000","last_change":"2024-09-18T14:18:20.417311+0000","last_active":"2024-09-18T14:18:20.417140+0000","last_peered":"2024-09-18T14:18:20.417140+0000","last_clean":"2024-09-18T14:18:20.417140+0000","last_became_active":"2024-09-18T13:59:12.090343+0000","last_became_peered":"2024-09-18T13:59:12.090343+0000","last_unstale":"2024-09-18T14:18:20.417140+0000","last_undegraded":"2024-09-18T14:18:20.417140+0000","last_fullsized":"2024-09-18T14:18:20.417140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:51:01.409248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011031396000000001,"stat_sum":{"num_bytes":27569851,"num_objects":12,"num_object_clones":10,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":356,"num_read_kb":379368,"num_write":226,"num_write_kb":82415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"3.9","version":"894'2349","reported_seq":5022,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.420329+0000","last_change":"2024-09-18T14:18:20.420503+0000","last_active":"2024-09-18T14:18:20.420329+0000","last_peered":"2024-09-18T14:18:20.420329+0000","last_clean":"2024-09-18T14:18:20.420329+0000","last_became_active":"2024-09-18T13:59:12.084513+0000","last_became_peered":"2024-09-18T13:59:12.084513+0000","last_unstale":"2024-09-18T14:18:20.420329+0000","last_undegraded":"2024-09-18T14:18:20.420329+0000","last_fullsized":"2024-09-18T14:18:20.420329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":2349,"log_dups_size":0,"ondisk_log_size":2349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:16:09.816428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014488535,"stat_sum":{"num_bytes":52551991,"num_objects":26,"num_object_clones":19,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":2,"num_read":1014,"num_read_kb":954914,"num_write":658,"num_write_kb":255001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"3.8","version":"894'2088","reported_seq":4706,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.410210+0000","last_change":"2024-09-18T14:18:20.410401+0000","last_active":"2024-09-18T14:18:20.410210+0000","last_peered":"2024-09-18T14:18:20.410210+0000","last_clean":"2024-09-18T14:18:20.410210+0000","last_became_active":"2024-09-18T13:59:12.085005+0000","last_became_peered":"2024-09-18T13:59:12.085005+0000","last_unstale":"2024-09-18T14:18:20.410210+0000","last_undegraded":"2024-09-18T14:18:20.410210+0000","last_fullsized":"2024-09-18T14:18:20.410210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":2088,"log_dups_size":0,"ondisk_log_size":2088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T23:25:12.247272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010113782999999999,"stat_sum":{"num_bytes":33339701,"num_objects":17,"num_object_clones":13,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":2,"num_read":742,"num_read_kb":687611,"num_write":606,"num_write_kb":212249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":881,"reported_epoch":893,"state":"active+clean","last_fresh":"2024-09-18T14:18:19.395501+0000","last_change":"2024-09-18T13:59:05.035941+0000","last_active":"2024-09-18T14:18:19.395501+0000","last_peered":"2024-09-18T14:18:19.395501+0000","last_clean":"2024-09-18T14:18:19.395501+0000","last_became_active":"2024-09-18T13:59:03.705607+0000","last_became_peered":"2024-09-18T13:59:03.705607+0000","last_unstale":"2024-09-18T14:18:19.395501+0000","last_undegraded":"2024-09-18T14:18:19.395501+0000","last_fullsized":"2024-09-18T14:18:19.395501+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:25:38.392196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000366475,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"895'1217","reported_seq":3886,"reported_epoch":895,"state":"active+clean","last_fresh":"2024-09-18T14:18:21.424868+0000","last_change":"2024-09-18T14:18:20.405324+0000","last_active":"2024-09-18T14:18:21.424868+0000","last_peered":"2024-09-18T14:18:21.424868+0000","last_clean":"2024-09-18T14:18:21.424868+0000","last_became_active":"2024-09-18T13:59:12.093546+0000","last_became_peered":"2024-09-18T13:59:12.093546+0000","last_unstale":"2024-09-18T14:18:21.424868+0000","last_undegraded":"2024-09-18T14:18:21.424868+0000","last_fullsized":"2024-09-18T14:18:21.424868+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:54:25.909556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061350889999999998,"stat_sum":{"num_bytes":32644185,"num_objects":17,"num_object_clones":14,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":487,"num_read_kb":501200,"num_write":350,"num_write_kb":129347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":870,"reported_epoch":893,"state":"active+clean","last_fresh":"2024-09-18T14:18:19.395175+0000","last_change":"2024-09-18T13:59:05.875836+0000","last_active":"2024-09-18T14:18:19.395175+0000","last_peered":"2024-09-18T14:18:19.395175+0000","last_clean":"2024-09-18T14:18:19.395175+0000","last_became_active":"2024-09-18T13:59:03.705710+0000","last_became_peered":"2024-09-18T13:59:03.705710+0000","last_unstale":"2024-09-18T14:18:19.395175+0000","last_undegraded":"2024-09-18T14:18:19.395175+0000","last_fullsized":"2024-09-18T14:18:19.395175+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:12:55.202686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029541399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"894'1201","reported_seq":3890,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.411246+0000","last_change":"2024-09-18T14:18:20.414343+0000","last_active":"2024-09-18T14:18:20.411246+0000","last_peered":"2024-09-18T14:18:20.411246+0000","last_clean":"2024-09-18T14:18:20.411246+0000","last_became_active":"2024-09-18T13:59:12.085037+0000","last_became_peered":"2024-09-18T13:59:12.085037+0000","last_unstale":"2024-09-18T14:18:20.411246+0000","last_undegraded":"2024-09-18T14:18:20.411246+0000","last_fullsized":"2024-09-18T14:18:20.411246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":1201,"log_dups_size":0,"ondisk_log_size":1201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:24:39.733895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087170819999999993,"stat_sum":{"num_bytes":31807082,"num_objects":11,"num_object_clones":8,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":488,"num_read_kb":479824,"num_write":333,"num_write_kb":142318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"1.2","version":"16'2","reported_seq":884,"reported_epoch":893,"state":"active+clean","last_fresh":"2024-09-18T14:18:19.398857+0000","last_change":"2024-09-18T13:59:05.036440+0000","last_active":"2024-09-18T14:18:19.398857+0000","last_peered":"2024-09-18T14:18:19.398857+0000","last_clean":"2024-09-18T14:18:19.398857+0000","last_became_active":"2024-09-18T13:59:03.705659+0000","last_became_peered":"2024-09-18T13:59:03.705659+0000","last_unstale":"2024-09-18T14:18:19.398857+0000","last_undegraded":"2024-09-18T14:18:19.398857+0000","last_fullsized":"2024-09-18T14:18:19.398857+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:49:16.930535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037781799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"894'1466","reported_seq":4139,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.409880+0000","last_change":"2024-09-18T14:18:20.409880+0000","last_active":"2024-09-18T14:18:20.409880+0000","last_peered":"2024-09-18T14:18:20.409880+0000","last_clean":"2024-09-18T14:18:20.409880+0000","last_became_active":"2024-09-18T13:59:12.084175+0000","last_became_peered":"2024-09-18T13:59:12.084175+0000","last_unstale":"2024-09-18T14:18:20.409880+0000","last_undegraded":"2024-09-18T14:18:20.409880+0000","last_fullsized":"2024-09-18T14:18:20.409880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":1466,"log_dups_size":0,"ondisk_log_size":1466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:57:14.482283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010251397000000001,"stat_sum":{"num_bytes":27159630,"num_objects":14,"num_object_clones":10,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":580,"num_read_kb":529094,"num_write":467,"num_write_kb":162991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"1.3","version":"14'1","reported_seq":875,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.399241+0000","last_change":"2024-09-18T13:59:05.036690+0000","last_active":"2024-09-18T14:18:20.399241+0000","last_peered":"2024-09-18T14:18:20.399241+0000","last_clean":"2024-09-18T14:18:20.399241+0000","last_became_active":"2024-09-18T13:59:02.977312+0000","last_became_peered":"2024-09-18T13:59:02.977312+0000","last_unstale":"2024-09-18T14:18:20.399241+0000","last_undegraded":"2024-09-18T14:18:20.399241+0000","last_fullsized":"2024-09-18T14:18:20.399241+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:03:06.377092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072047099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'72","reported_seq":966,"reported_epoch":893,"state":"active+clean","last_fresh":"2024-09-18T14:18:19.395447+0000","last_change":"2024-09-18T13:59:02.975019+0000","last_active":"2024-09-18T14:18:19.395447+0000","last_peered":"2024-09-18T14:18:19.395447+0000","last_clean":"2024-09-18T14:18:19.395447+0000","last_became_active":"2024-09-18T13:59:02.974164+0000","last_became_peered":"2024-09-18T13:59:02.974164+0000","last_unstale":"2024-09-18T14:18:19.395447+0000","last_undegraded":"2024-09-18T14:18:19.395447+0000","last_fullsized":"2024-09-18T14:18:19.395447+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T22:10:24.958774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"894'1351","reported_seq":4033,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.415450+0000","last_change":"2024-09-18T14:18:20.415651+0000","last_active":"2024-09-18T14:18:20.415450+0000","last_peered":"2024-09-18T14:18:20.415450+0000","last_clean":"2024-09-18T14:18:20.415450+0000","last_became_active":"2024-09-18T13:59:12.083263+0000","last_became_peered":"2024-09-18T13:59:12.083263+0000","last_unstale":"2024-09-18T14:18:20.415450+0000","last_undegraded":"2024-09-18T14:18:20.415450+0000","last_fullsized":"2024-09-18T14:18:20.415450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":1351,"log_dups_size":0,"ondisk_log_size":1351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T00:08:24.134039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012159392999999999,"stat_sum":{"num_bytes":29143723,"num_objects":14,"num_object_clones":10,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":539,"num_read_kb":578685,"num_write":331,"num_write_kb":140933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"3.6","version":"894'655","reported_seq":3375,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.408227+0000","last_change":"2024-09-18T14:18:20.408662+0000","last_active":"2024-09-18T14:18:20.408227+0000","last_peered":"2024-09-18T14:18:20.408227+0000","last_clean":"2024-09-18T14:18:20.408227+0000","last_became_active":"2024-09-18T13:59:12.080624+0000","last_became_peered":"2024-09-18T13:59:12.080624+0000","last_unstale":"2024-09-18T14:18:20.408227+0000","last_undegraded":"2024-09-18T14:18:20.408227+0000","last_fullsized":"2024-09-18T14:18:20.408227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:24:39.733895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056516229999999997,"stat_sum":{"num_bytes":16470429,"num_objects":9,"num_object_clones":7,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":309,"num_read_kb":311311,"num_write":186,"num_write_kb":77430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":874,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.399488+0000","last_change":"2024-09-18T13:59:05.036936+0000","last_active":"2024-09-18T14:18:20.399488+0000","last_peered":"2024-09-18T14:18:20.399488+0000","last_clean":"2024-09-18T14:18:20.399488+0000","last_became_active":"2024-09-18T13:59:02.977186+0000","last_became_peered":"2024-09-18T13:59:02.977186+0000","last_unstale":"2024-09-18T14:18:20.399488+0000","last_undegraded":"2024-09-18T14:18:20.399488+0000","last_fullsized":"2024-09-18T14:18:20.399488+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:01:46.516709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065897000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"894'592","reported_seq":3307,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.405589+0000","last_change":"2024-09-18T14:18:20.405822+0000","last_active":"2024-09-18T14:18:20.405589+0000","last_peered":"2024-09-18T14:18:20.405589+0000","last_clean":"2024-09-18T14:18:20.405589+0000","last_became_active":"2024-09-18T13:59:12.083487+0000","last_became_peered":"2024-09-18T13:59:12.083487+0000","last_unstale":"2024-09-18T14:18:20.405589+0000","last_undegraded":"2024-09-18T14:18:20.405589+0000","last_fullsized":"2024-09-18T14:18:20.405589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T01:03:59.141782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006352964,"stat_sum":{"num_bytes":18874932,"num_objects":10,"num_object_clones":8,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":278,"num_read_kb":342898,"num_write":142,"num_write_kb":60460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":882,"reported_epoch":893,"state":"active+clean","last_fresh":"2024-09-18T14:18:19.398327+0000","last_change":"2024-09-18T13:59:05.036455+0000","last_active":"2024-09-18T14:18:19.398327+0000","last_peered":"2024-09-18T14:18:19.398327+0000","last_clean":"2024-09-18T14:18:19.398327+0000","last_became_active":"2024-09-18T13:59:02.974240+0000","last_became_peered":"2024-09-18T13:59:02.974240+0000","last_unstale":"2024-09-18T14:18:19.398327+0000","last_undegraded":"2024-09-18T14:18:19.398327+0000","last_fullsized":"2024-09-18T14:18:19.398327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:13:40.312100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038595099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"890'686","reported_seq":3360,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.405634+0000","last_change":"2024-09-18T14:18:20.405634+0000","last_active":"2024-09-18T14:18:20.405634+0000","last_peered":"2024-09-18T14:18:20.405634+0000","last_clean":"2024-09-18T14:18:20.405634+0000","last_became_active":"2024-09-18T13:59:12.083315+0000","last_became_peered":"2024-09-18T13:59:12.083315+0000","last_unstale":"2024-09-18T14:18:20.405634+0000","last_undegraded":"2024-09-18T14:18:20.405634+0000","last_fullsized":"2024-09-18T14:18:20.405634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T22:10:52.731849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062531430000000001,"stat_sum":{"num_bytes":17099443,"num_objects":8,"num_object_clones":6,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":276,"num_read_kb":262439,"num_write":168,"num_write_kb":65612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":881,"reported_epoch":893,"state":"active+clean","last_fresh":"2024-09-18T14:18:19.395715+0000","last_change":"2024-09-18T13:59:05.035951+0000","last_active":"2024-09-18T14:18:19.395715+0000","last_peered":"2024-09-18T14:18:19.395715+0000","last_clean":"2024-09-18T14:18:19.395715+0000","last_became_active":"2024-09-18T13:59:03.705691+0000","last_became_peered":"2024-09-18T13:59:03.705691+0000","last_unstale":"2024-09-18T14:18:19.395715+0000","last_undegraded":"2024-09-18T14:18:19.395715+0000","last_fullsized":"2024-09-18T14:18:19.395715+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:45:16.896857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033710499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"892'338","reported_seq":3046,"reported_epoch":894,"state":"active+clean","last_fresh":"2024-09-18T14:18:20.402073+0000","last_change":"2024-09-18T14:18:20.402948+0000","last_active":"2024-09-18T14:18:20.402073+0000","last_peered":"2024-09-18T14:18:20.402073+0000","last_clean":"2024-09-18T14:18:20.402073+0000","last_became_active":"2024-09-18T13:59:12.093114+0000","last_became_peered":"2024-09-18T13:59:12.093114+0000","last_unstale":"2024-09-18T14:18:20.402073+0000","last_undegraded":"2024-09-18T14:18:20.402073+0000","last_fullsized":"2024-09-18T14:18:20.402073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:11.063489+0000","last_clean_scrub_stamp":"2024-09-18T13:59:11.063489+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:01:21.222558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010007620000000001,"stat_sum":{"num_bytes":11889869,"num_objects":5,"num_object_clones":4,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":173,"num_read_kb":175806,"num_write":97,"num_write_kb":41064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"340","length":"1"},{"start":"356","length":"1"},{"start":"35a","length":"1"},{"start":"362","length":"1"}]},{"pgid":"1.7","version":"0'0","reported_seq":870,"reported_epoch":893,"state":"active+clean","last_fresh":"2024-09-18T14:18:19.395322+0000","last_change":"2024-09-18T13:59:05.875874+0000","last_active":"2024-09-18T14:18:19.395322+0000","last_peered":"2024-09-18T14:18:19.395322+0000","last_clean":"2024-09-18T14:18:19.395322+0000","last_became_active":"2024-09-18T13:59:02.976936+0000","last_became_peered":"2024-09-18T13:59:02.976936+0000","last_unstale":"2024-09-18T14:18:19.395322+0000","last_undegraded":"2024-09-18T14:18:19.395322+0000","last_fullsized":"2024-09-18T14:18:19.395322+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:20:37.890959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036210499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":437021106,"num_objects":201,"num_object_clones":152,"num_object_copies":402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":201,"num_whiteouts":12,"num_read":7514,"num_read_kb":7385305,"num_write":5199,"num_write_kb":2032096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":541122560,"data_stored":711319985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18316,"ondisk_log_size":18316,"up":32,"acting":32,"num_store_stats":6},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607787,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418528,"kb_used_data":108680,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93953312,"statfs":{"total":96636764160,"available":96208191488,"internally_reserved":0,"allocated":111288320,"data_stored":158407917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:17:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.34899999999999998},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.19},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.91700000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.36599999999999999},"min":{"1min":0.28299999999999997,"5min":0.23999999999999999,"15min":0.219},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":1.0740000000000001},"last":0.375}]},{"osd":2,"last update":"Wed Sep 18 14:17:45 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34300000000000003,"15min":0.36599999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":1.133},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.372,"15min":0.38600000000000001},"min":{"1min":0.26700000000000002,"5min":0.26100000000000001,"15min":0.20999999999999999},"max":{"1min":0.52300000000000002,"5min":0.52800000000000002,"15min":0.71899999999999997},"last":0.51400000000000001}]},{"osd":3,"last update":"Wed Sep 18 14:17:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.54300000000000004},"min":{"1min":0.39800000000000002,"5min":0.33100000000000002,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.747,"15min":0.871},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.503,"15min":0.52600000000000002},"min":{"1min":0.38300000000000001,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.034},"last":0.623}]},{"osd":4,"last update":"Wed Sep 18 14:17:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.39900000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.45100000000000001,"5min":0.35499999999999998,"15min":0.28299999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":1.323},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed Sep 18 14:17:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53600000000000003,"15min":0.55800000000000005},"min":{"1min":0.39600000000000002,"5min":0.378,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.87},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.315,"15min":0.28000000000000003},"max":{"1min":0.68600000000000005,"5min":0.77200000000000002,"15min":1.2669999999999999},"last":0.52900000000000003}]}]},{"osd":5,"up_from":11,"seq":47244640490,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":359468,"kb_used_data":62108,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94012372,"statfs":{"total":96636764160,"available":96268668928,"internally_reserved":0,"allocated":63598592,"data_stored":81451590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:17:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.53400000000000003},"min":{"1min":0.38,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":2.6800000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":3.206},"last":0.65700000000000003}]},{"osd":1,"last update":"Wed Sep 18 14:17:44 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48299999999999998,"15min":0.48899999999999999},"min":{"1min":0.32100000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.624,"5min":1.335,"15min":1.335},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.48299999999999998,"15min":0.49399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.68400000000000005,"5min":0.69399999999999995,"15min":2.3610000000000002},"last":0.55900000000000005}]},{"osd":2,"last update":"Wed Sep 18 14:17:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.495,"15min":0.50800000000000001},"min":{"1min":0.33100000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.59799999999999998,"5min":1.25,"15min":2.976},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47999999999999998,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.64100000000000001,"5min":0.83999999999999997,"15min":2.8719999999999999},"last":0.51600000000000001}]},{"osd":3,"last update":"Wed Sep 18 14:17:44 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35499999999999998,"15min":0.35399999999999998},"min":{"1min":0.25800000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.64700000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.377,"15min":0.376},"min":{"1min":0.28799999999999998,"5min":0.224,"15min":0.21199999999999999},"max":{"1min":0.56699999999999995,"5min":0.629,"15min":0.69099999999999995},"last":0.5}]},{"osd":4,"last update":"Wed Sep 18 14:17:44 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.373,"15min":0.375},"min":{"1min":0.25700000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.58699999999999997,"5min":1.194,"15min":1.194},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.219},"max":{"1min":0.63700000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.45000000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640490,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412360,"kb_used_data":106680,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93959480,"statfs":{"total":96636764160,"available":96214507520,"internally_reserved":0,"allocated":109240320,"data_stored":133875220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.183},"max":{"1min":0.65700000000000003,"5min":0.745,"15min":0.81799999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.42499999999999999,"5min":0.32900000000000001,"15min":0.26300000000000001},"max":{"1min":0.68400000000000005,"5min":0.80300000000000005,"15min":0.83299999999999996},"last":0.621}]},{"osd":1,"last update":"Wed Sep 18 14:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.318,"5min":0.29799999999999999,"15min":0.26500000000000001},"max":{"1min":0.55800000000000005,"5min":0.77300000000000002,"15min":0.79200000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.46999999999999997,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64500000000000002,"5min":0.69599999999999995,"15min":0.71399999999999997},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Sep 18 14:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47899999999999998,"15min":0.48899999999999999},"min":{"1min":0.39000000000000001,"5min":0.252,"15min":0.246},"max":{"1min":0.59499999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46899999999999997,"15min":0.47999999999999998},"min":{"1min":0.36099999999999999,"5min":0.30399999999999999,"15min":0.28799999999999998},"max":{"1min":0.60099999999999998,"5min":0.68100000000000005,"15min":0.84599999999999997},"last":0.57899999999999996}]},{"osd":3,"last update":"Wed Sep 18 14:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.34699999999999998,"15min":0.35099999999999998},"min":{"1min":0.23999999999999999,"5min":0.22500000000000001,"15min":0.20499999999999999},"max":{"1min":0.52200000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.26500000000000001,"5min":0.23599999999999999,"15min":0.22},"max":{"1min":0.501,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.372}]},{"osd":5,"last update":"Wed Sep 18 14:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35799999999999998,"15min":0.36199999999999999},"min":{"1min":0.253,"5min":0.20899999999999999,"15min":0.20499999999999999},"max":{"1min":0.48299999999999998,"5min":0.64600000000000002,"15min":0.753},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.28100000000000003,"5min":0.252,"15min":0.20899999999999999},"max":{"1min":0.51900000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.42299999999999999}]}]},{"osd":3,"up_from":11,"seq":47244640490,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361148,"kb_used_data":63792,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94010692,"statfs":{"total":96636764160,"available":96266948608,"internally_reserved":0,"allocated":65323008,"data_stored":79172107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.5,"15min":0.48299999999999998},"min":{"1min":0.27800000000000002,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.59299999999999997,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.26200000000000001},"max":{"1min":0.61799999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.56000000000000005}]},{"osd":1,"last update":"Wed Sep 18 14:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.28599999999999998},"max":{"1min":0.60699999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.51800000000000002,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.28100000000000003,"15min":0.27500000000000002},"max":{"1min":0.66200000000000003,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Sep 18 14:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54600000000000004,"15min":0.53600000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.64500000000000002,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56699999999999995,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.30099999999999999},"max":{"1min":0.64400000000000002,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.58999999999999997}]},{"osd":4,"last update":"Wed Sep 18 14:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36099999999999999,"15min":0.35999999999999999},"min":{"1min":0.246,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52100000000000002,"5min":0.61899999999999999,"15min":0.68899999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.27900000000000003,"5min":0.22900000000000001,"15min":0.20399999999999999},"max":{"1min":0.52900000000000003,"5min":0.59799999999999998,"15min":0.63800000000000001},"last":0.28899999999999998}]},{"osd":5,"last update":"Wed Sep 18 14:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.27700000000000002,"5min":0.24299999999999999,"15min":0.215},"max":{"1min":0.53200000000000003,"5min":0.59399999999999997,"15min":0.70499999999999996},"last":0.314},{"interface":"front","average":{"1min":0.438,"5min":0.41299999999999998,"15min":0.40799999999999997},"min":{"1min":0.307,"5min":0.248,"15min":0.19700000000000001},"max":{"1min":0.55500000000000005,"5min":0.63,"15min":0.748},"last":0.35999999999999999}]}]},{"osd":2,"up_from":11,"seq":47244640490,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424436,"kb_used_data":127080,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93947404,"statfs":{"total":96636764160,"available":96202141696,"internally_reserved":0,"allocated":130129920,"data_stored":176035715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.318,"15min":0.33300000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20300000000000001},"max":{"1min":0.44900000000000001,"5min":0.53200000000000003,"15min":0.875},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.34799999999999998,"15min":0.36399999999999999},"min":{"1min":0.24299999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.55600000000000005,"5min":1.1100000000000001,"15min":1.357},"last":0.37}]},{"osd":1,"last update":"Wed Sep 18 14:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.39900000000000002,"15min":0.41599999999999998},"min":{"1min":0.30599999999999999,"5min":0.28100000000000003,"15min":0.25},"max":{"1min":0.496,"5min":0.76700000000000002,"15min":4.117},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.374,"15min":0.40000000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.217},"max":{"1min":0.48499999999999999,"5min":0.629,"15min":4.0529999999999999},"last":0.46500000000000002}]},{"osd":3,"last update":"Wed Sep 18 14:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.502,"15min":0.51900000000000002},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.68700000000000006,"5min":0.77200000000000002,"15min":1.3049999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.501,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35599999999999998,"15min":0.29599999999999999},"max":{"1min":0.627,"5min":1.026,"15min":1.506},"last":0.42099999999999999}]},{"osd":4,"last update":"Wed Sep 18 14:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56200000000000006,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.69099999999999995,"5min":0.94399999999999995,"15min":1.2789999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56299999999999994,"15min":0.57899999999999996},"min":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.30499999999999999},"max":{"1min":0.67200000000000004,"5min":1.2669999999999999,"15min":2.3159999999999998},"last":0.502}]},{"osd":5,"last update":"Wed Sep 18 14:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.65800000000000003,"5min":1.155,"15min":1.224},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53100000000000003,"15min":0.54700000000000004},"min":{"1min":0.37,"5min":0.35999999999999999,"15min":0.34599999999999997},"max":{"1min":0.64600000000000002,"5min":1.0409999999999999,"15min":1.2629999999999999},"last":0.51700000000000002}]}]},{"osd":0,"up_from":11,"seq":47244640490,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374088,"kb_used_data":72572,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93997752,"statfs":{"total":96636764160,"available":96253698048,"internally_reserved":0,"allocated":74313728,"data_stored":87536026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 14:17:29 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33000000000000002,"15min":0.34300000000000003},"min":{"1min":0.187,"5min":0.187,"15min":0.17199999999999999},"max":{"1min":0.45900000000000002,"5min":0.628,"15min":0.628},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.41099999999999998},"min":{"1min":0.27700000000000002,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.72299999999999998},"last":0.45000000000000001}]},{"osd":2,"last update":"Wed Sep 18 14:17:29 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33800000000000002,"15min":0.36199999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":0.44,"5min":0.51000000000000001,"15min":0.629},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.36699999999999999,"15min":0.378},"min":{"1min":0.25,"5min":0.25,"15min":0.224},"max":{"1min":0.50800000000000001,"5min":0.52100000000000002,"15min":0.71499999999999997},"last":0.40899999999999997}]},{"osd":3,"last update":"Wed Sep 18 14:17:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.28599999999999998},"max":{"1min":0.66700000000000004,"5min":0.73799999999999999,"15min":0.88200000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.54800000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.29899999999999999},"max":{"1min":0.75600000000000001,"5min":0.76500000000000001,"15min":0.88900000000000001},"last":0.64300000000000002}]},{"osd":4,"last update":"Wed Sep 18 14:17:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.56100000000000005},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.77400000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.28000000000000003},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.91700000000000004},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed Sep 18 14:17:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.54800000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.28699999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.84899999999999998},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55000000000000004,"15min":0.56100000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.71599999999999997,"5min":0.84599999999999997,"15min":1.1479999999999999},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72196096,"data_stored":86378032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109658112,"data_stored":159586292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128008192,"data_stored":174873209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":62668800,"data_stored":77480673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":107114496,"data_stored":132712695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61476864,"data_stored":80289084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T14:18:24.097 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-18T14:18:24.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-18T14:18:24.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.198+0000 7f439055d640 10 monclient: get_monmap_and_config 2024-09-18T14:18:24.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.198+0000 7f439055d640 10 monclient: build_initial_monmap 2024-09-18T14:18:24.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.198+0000 7f439055d640 10 monclient: monmap: 2024-09-18T14:18:24.199 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:18:24.199 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:18:24.199806+0000 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:18:24.199806+0000 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:18:24.200 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.199+0000 7f439055d640 1 Processor -- start 2024-09-18T14:18:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.199+0000 7f439055d640 1 -- start start 2024-09-18T14:18:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.199+0000 7f439055d640 10 monclient: init 2024-09-18T14:18:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 10 monclient: _reopen_session rank -1 2024-09-18T14:18:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 10 monclient: _add_conns ranks=[5,8,2,0,1,3,7,4,6] 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 10 monclient(hunting): picked mon.noname-h con 0x7f43880a25c0 addr v1:172.21.15.84:6791/0 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 10 monclient(hunting): picked mon.noname-e con 0x7f438809ee00 addr v1:172.21.15.45:6793/0 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 10 monclient(hunting): picked mon.noname-b con 0x7f438809f1d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438805ed70 con 0x7f438809f1d0 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43880a1a50 con 0x7f43880a25c0 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43880972b0 con 0x7f438809ee00 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 10 monclient(hunting): _renew_subs 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 10 monclient(hunting): authenticate will time out at 1919.538204s 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438eafa640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7f438809f1d0 legacy=0x7f438813ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438daf8640 1 -- >> v1:172.21.15.45:6793/0 conn(0x7f438809ee00 legacy=0x7f4388137710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.45:6793/0 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438e2f9640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f43880a25c0 0x7f43881323e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:44982/0 (socket says 172.21.15.45:44982) 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438e2f9640 1 -- 172.21.15.45:0/2411506061 learned_addr learned my addr 172.21.15.45:0/2411506061 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:18:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 1 -- 172.21.15.45:0/2411506061 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781778399 0 0) 0x7f43880a1a50 con 0x7f43880a25c0 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 10 monclient(hunting): my global_id is 185045 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 1 -- 172.21.15.45:0/2411506061 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43740034e0 con 0x7f43880a25c0 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 1 -- 172.21.15.45:0/2411506061 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3371797303 0 0) 0x7f43740034e0 con 0x7f43880a25c0 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 1 -- 172.21.15.45:0/2411506061 >> v1:172.21.15.45:6793/0 conn(0x7f438809ee00 legacy=0x7f4388137710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 1 -- 172.21.15.45:0/2411506061 >> v1:172.21.15.45:6790/0 conn(0x7f438809f1d0 legacy=0x7f438813ae40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 10 monclient: _finish_hunting 0 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 1 monclient: found mon.noname-h 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 1 -- 172.21.15.45:0/2411506061 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4388133290 con 0x7f43880a25c0 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 10 monclient: _finish_auth 0 2024-09-18T14:18:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 1 -- 172.21.15.45:0/2411506061 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4384001830 con 0x7f43880a25c0 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 5 monclient: authenticate success, global_id 185045 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f438d2f7640 10 monclient: dump: 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.200+0000 7f439055d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f438d2f7640 1 -- 172.21.15.45:0/2411506061 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4384001a10 con 0x7f43880a25c0 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f438d2f7640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f438d2f7640 1 -- 172.21.15.45:0/2411506061 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4384004ba0 con 0x7f43880a25c0 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f439055d640 10 monclient: get_monmap_and_config success 2024-09-18T14:18:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f439055d640 4 set_mon_vals no callback set 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f438d2f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f438d2f7640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f438d2f7640 10 monclient: dump: 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:24.212 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f439055d640 10 monclient: shutdown 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f439055d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f439055d640 1 -- 172.21.15.45:0/2411506061 >> v1:172.21.15.84:6791/0 conn(0x7f43880a25c0 legacy=0x7f43881323e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f439055d640 1 -- 172.21.15.45:0/2411506061 shutdown_connections 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f439055d640 1 -- 172.21.15.45:0/2411506061 >> 172.21.15.45:0/2411506061 conn(0x7f43880698b0 msgr2=0x7f4388063030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f439055d640 1 -- 172.21.15.45:0/2411506061 shutdown_connections 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.201+0000 7f439055d640 1 -- 172.21.15.45:0/2411506061 wait complete. 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 10 monclient: build_initial_monmap 2024-09-18T14:18:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 10 monclient: monmap: 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:18:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 1 Processor -- start 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 1 -- start start 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 10 monclient: init 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 10 monclient: _reopen_session rank -1 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 10 monclient: _add_conns ranks=[6,2,1,4,5,7,0,3,8] 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 10 monclient(hunting): picked mon.noname-d con 0x7f438809ee00 addr v1:172.21.15.45:6792/0 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 10 monclient(hunting): picked mon.noname-b con 0x7f438809f1d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 10 monclient(hunting): picked mon.noname-f con 0x7f43880a25c0 addr v1:172.21.15.84:6789/0 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438809f910 con 0x7f43880a25c0 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43880a2190 con 0x7f438809f1d0 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438811d770 con 0x7f438809ee00 2024-09-18T14:18:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 10 monclient(hunting): _renew_subs 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f439055d640 10 monclient(hunting): authenticate will time out at 1919.540895s 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f438daf8640 1 -- >> v1:172.21.15.45:6790/0 conn(0x7f438809f1d0 legacy=0x7f438809d170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6790/0 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f438e2f9640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f438809ee00 0x7f438809ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56084/0 (socket says 172.21.15.45:56084) 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.202+0000 7f438e2f9640 1 -- 172.21.15.45:0/3620592605 learned_addr learned my addr 172.21.15.45:0/3620592605 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065153470 0 0) 0x7f438811d770 con 0x7f438809ee00 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): my global_id is 185172 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f436c0034e0 con 0x7f438809ee00 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987260226 0 0) 0x7f438809f910 con 0x7f43880a25c0 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:18:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): my global_id is 214651 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f438811d770 con 0x7f43880a25c0 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1378699524 0 0) 0x7f436c0034e0 con 0x7f438809ee00 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f438809f910 con 0x7f438809ee00 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4384001c40 con 0x7f438809ee00 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): dump: 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3074567831 0 0) 0x7f438811d770 con 0x7f43880a25c0 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f436c0034e0 con 0x7f43880a25c0 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f43800044a0 con 0x7f43880a25c0 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:18:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient(hunting): dump: 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1257484029 0 0) 0x7f438809f910 con 0x7f438809ee00 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 >> v1:172.21.15.45:6790/0 conn(0x7f438809f1d0 legacy=0x7f438809d170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 >> v1:172.21.15.84:6789/0 conn(0x7f43880a25c0 legacy=0x7f43881498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 monclient: found mon.d 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438811cf10 con 0x7f438809ee00 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:18:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f439055d640 5 monclient: authenticate success, global_id 185172 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f439055d640 10 monclient: _renew_subs 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f439055d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f439055d640 1 -- 172.21.15.45:0/3620592605 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438811c250 con 0x7f438809ee00 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f439055d640 10 monclient: _renew_subs 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f439055d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f439055d640 1 -- 172.21.15.45:0/3620592605 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f438809e2e0 con 0x7f438809ee00 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4384004d90 con 0x7f438809ee00 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4384005240 con 0x7f438809ee00 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f438caf6640 4 set_mon_vals no callback set 2024-09-18T14:18:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.203+0000 7f437e7fc640 10 monclient: dump: 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:18:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:18:24.223 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:18:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.204+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4384010210 con 0x7f438809ee00 2024-09-18T14:18:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.204+0000 7f439055d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:18:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.204+0000 7f439055d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:18:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.204+0000 7f439055d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:18:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.204+0000 7f439055d640 1 -- 172.21.15.45:0/3620592605 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4370005010 con 0x7f438809ee00 2024-09-18T14:18:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.204+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(897..897 src has 637..897) v4 ==== 3943+0+0 (unknown 1321444408 0 0) 0x7f4384046d70 con 0x7f438809ee00 2024-09-18T14:18:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.205+0000 7f438daf8640 10 monclient: get_auth_request con 0x7f436c03a3f0 auth_method 0 2024-09-18T14:18:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.209+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f4384015530 con 0x7f438809ee00 2024-09-18T14:18:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.209+0000 7f437e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:18:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.209+0000 7f437e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:18:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.371+0000 7f439055d640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2024-09-18T14:18:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.371+0000 7f439055d640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2024-09-18T14:18:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.371+0000 7f439055d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:18:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.371+0000 7f439055d640 1 -- 172.21.15.45:0/3620592605 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f43700052c0 con 0x7f438809ee00 2024-09-18T14:18:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.441+0000 7f437e7fc640 1 -- 172.21.15.45:0/3620592605 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]=0 pool 'unique_pool_0' removed v898) v1 ==== 175+0+0 (unknown 1826623748 0 0) 0x7f4384014f30 con 0x7f438809ee00 2024-09-18T14:18:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.441+0000 7f437e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2024-09-18T14:18:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.441+0000 7f437e7fc640 10 monclient: _finish_command 2 = system:0 pool 'unique_pool_0' removed 2024-09-18T14:18:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'unique_pool_0' removed 2024-09-18T14:18:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.450+0000 7f439055d640 1 -- 172.21.15.45:0/3620592605 >> v1:172.21.15.84:6812/31478 conn(0x7f436c03a3f0 legacy=0x7f436c03c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.450+0000 7f439055d640 10 monclient: shutdown 2024-09-18T14:18:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.450+0000 7f439055d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:18:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.450+0000 7f439055d640 1 -- 172.21.15.45:0/3620592605 >> v1:172.21.15.45:6792/0 conn(0x7f438809ee00 legacy=0x7f438809ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:18:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.450+0000 7f439055d640 1 -- 172.21.15.45:0/3620592605 shutdown_connections 2024-09-18T14:18:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.450+0000 7f439055d640 1 -- 172.21.15.45:0/3620592605 >> 172.21.15.45:0/3620592605 conn(0x7f43880698b0 msgr2=0x7f43880a0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:18:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.451+0000 7f439055d640 1 -- 172.21.15.45:0/3620592605 shutdown_connections 2024-09-18T14:18:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:18:24.451+0000 7f439055d640 1 -- 172.21.15.45:0/3620592605 wait complete. 2024-09-18T14:18:24.470 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-09-18T14:18:24.484 INFO:tasks.mon_thrash:joining mon_thrasher 2024-09-18T14:19:09.411 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-18T14:19:09.412 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-18T14:19:09.412 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-18T14:19:09.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-18T14:19:09.417 INFO:tasks.ceph.mon.b:Started 2024-09-18T14:19:09.417 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-18T14:19:09.417 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-09-18T14:19:09.417 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-18T14:19:09.417 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-18T14:19:09.419 INFO:tasks.ceph.mon.e:Started 2024-09-18T14:19:09.419 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:19:09.420 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:19:09.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.535+0000 7fc04ee80640 10 monclient: get_monmap_and_config 2024-09-18T14:19:09.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.535+0000 7fc04ee80640 10 monclient: build_initial_monmap 2024-09-18T14:19:09.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.535+0000 7fc04ee80640 10 monclient: monmap: 2024-09-18T14:19:09.536 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:09.536 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:09.536 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:09.536274+0000 2024-09-18T14:19:09.536 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:09.536274+0000 2024-09-18T14:19:09.536 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:09.537 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:09.537 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:09.537 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:09.537 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:09.537 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:09.537 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:09.537 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:09.537 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:09.537 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:09.537 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:09.537 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 1 Processor -- start 2024-09-18T14:19:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 1 -- start start 2024-09-18T14:19:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 10 monclient: init 2024-09-18T14:19:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 10 monclient: _add_conns ranks=[2,7,5,3,4,0,6,1,8] 2024-09-18T14:19:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 10 monclient(hunting): picked mon.noname-b con 0x7fc04806e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:19:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 10 monclient(hunting): picked mon.noname-i con 0x7fc0480a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 10 monclient(hunting): picked mon.noname-h con 0x7fc0480a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04805ed70 con 0x7fc04806e810 2024-09-18T14:19:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc048097620 con 0x7fc0480a1080 2024-09-18T14:19:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc048097340 con 0x7fc0480a0cb0 2024-09-18T14:19:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04ee80640 10 monclient(hunting): authenticate will time out at 1964.874932s 2024-09-18T14:19:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.536+0000 7fc04cc1c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fc04806e810 0x7fc0480a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:53522/0 (socket says 172.21.15.45:53522) 2024-09-18T14:19:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc04cc1c640 1 -- 172.21.15.45:0/1670707949 learned_addr learned my addr 172.21.15.45:0/1670707949 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 1 -- 172.21.15.45:0/1670707949 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692742295 0 0) 0x7fc048097620 con 0x7fc0480a1080 2024-09-18T14:19:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient(hunting): my global_id is 185054 2024-09-18T14:19:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 1 -- 172.21.15.45:0/1670707949 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0340034e0 con 0x7fc0480a1080 2024-09-18T14:19:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 1 -- 172.21.15.45:0/1670707949 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900125310 0 0) 0x7fc048097340 con 0x7fc0480a0cb0 2024-09-18T14:19:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient(hunting): my global_id is 214585 2024-09-18T14:19:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 1 -- 172.21.15.45:0/1670707949 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc048097620 con 0x7fc0480a0cb0 2024-09-18T14:19:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 1 -- 172.21.15.45:0/1670707949 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3764314058 0 0) 0x7fc0340034e0 con 0x7fc0480a1080 2024-09-18T14:19:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 1 -- 172.21.15.45:0/1670707949 >> v1:172.21.15.84:6792/0 conn(0x7fc0480a0cb0 legacy=0x7fc048137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 1 -- 172.21.15.45:0/1670707949 >> v1:172.21.15.45:6790/0 conn(0x7fc04806e810 legacy=0x7fc0480a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient: _finish_hunting 0 2024-09-18T14:19:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 1 monclient: found mon.noname-h 2024-09-18T14:19:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.84:6791/0 2024-09-18T14:19:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 1 -- 172.21.15.45:0/1670707949 --> v1:172.21.15.84:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0480a1660 con 0x7fc0480a1080 2024-09-18T14:19:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient: _finish_auth 0 2024-09-18T14:19:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 1 -- 172.21.15.45:0/1670707949 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc040001830 con 0x7fc0480a1080 2024-09-18T14:19:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-18T14:19:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient: dump: 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc04ee80640 5 monclient: authenticate success, global_id 185054 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc04ee80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 1 -- 172.21.15.45:0/1670707949 <== mon.5 v1:172.21.15.84:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc040001a10 con 0x7fc0480a1080 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc04ee80640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 1 -- 172.21.15.45:0/1670707949 <== mon.5 v1:172.21.15.84:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc040002240 con 0x7fc0480a1080 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc04ee80640 4 set_mon_vals no callback set 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.537+0000 7fc0477fe640 10 monclient: dump: 2024-09-18T14:19:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:09.554 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.538+0000 7fc04ee80640 10 monclient: shutdown 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.538+0000 7fc04ee80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.538+0000 7fc04ee80640 1 -- 172.21.15.45:0/1670707949 >> v1:172.21.15.84:6791/0 conn(0x7fc0480a1080 legacy=0x7fc04813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.538+0000 7fc04ee80640 1 -- 172.21.15.45:0/1670707949 shutdown_connections 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.538+0000 7fc04ee80640 1 -- 172.21.15.45:0/1670707949 >> 172.21.15.45:0/1670707949 conn(0x7fc0480a5bb0 msgr2=0x7fc0480a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.538+0000 7fc04ee80640 1 -- 172.21.15.45:0/1670707949 shutdown_connections 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.538+0000 7fc04ee80640 1 -- 172.21.15.45:0/1670707949 wait complete. 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.539+0000 7fc04ee80640 10 monclient: build_initial_monmap 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.539+0000 7fc04ee80640 10 monclient: monmap: 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.539+0000 7fc04ee80640 1 Processor -- start 2024-09-18T14:19:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.539+0000 7fc04ee80640 1 -- start start 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.539+0000 7fc04ee80640 10 monclient: init 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc04ee80640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc04ee80640 10 monclient: _add_conns ranks=[0,6,3,4,7,5,1,8,2] 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc04ee80640 10 monclient(hunting): picked mon.noname-a con 0x7fc04806e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc04ee80640 10 monclient(hunting): picked mon.noname-d con 0x7fc0480a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc04ee80640 10 monclient(hunting): picked mon.noname-g con 0x7fc0480a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc04ee80640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc048122e80 con 0x7fc04806e810 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc04ee80640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04813dc70 con 0x7fc0480a1080 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc04ee80640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0481228c0 con 0x7fc0480a0cb0 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc04ee80640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc04ee80640 10 monclient(hunting): authenticate will time out at 1964.878345s 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc047fff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fc0480a0cb0 0x7fc048146790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:52018/0 (socket says 172.21.15.45:52018) 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc047fff640 1 -- 172.21.15.45:0/1163710483 learned_addr learned my addr 172.21.15.45:0/1163710483 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176544462 0 0) 0x7fc048122e80 con 0x7fc04806e810 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc044ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc044ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc044ff9640 10 monclient(hunting): my global_id is 6633 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0280034e0 con 0x7fc04806e810 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993605172 0 0) 0x7fc0481228c0 con 0x7fc0480a0cb0 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc044ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc044ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc044ff9640 10 monclient(hunting): my global_id is 185181 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.540+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc048122e80 con 0x7fc0480a0cb0 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555396008 0 0) 0x7fc04813dc70 con 0x7fc0480a1080 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): my global_id is 224328 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0481228c0 con 0x7fc0480a1080 2024-09-18T14:19:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1788862375 0 0) 0x7fc0280034e0 con 0x7fc04806e810 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc04813dc70 con 0x7fc04806e810 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc038001990 con 0x7fc04806e810 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): dump: 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2724652215 0 0) 0x7fc048122e80 con 0x7fc0480a0cb0 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0280034e0 con 0x7fc0480a0cb0 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc030001de0 con 0x7fc0480a0cb0 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): dump: 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3038255880 0 0) 0x7fc0481228c0 con 0x7fc0480a1080 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc048122e80 con 0x7fc0480a1080 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc0400018a0 con 0x7fc0480a1080 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient(hunting): dump: 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1639747203 0 0) 0x7fc04813dc70 con 0x7fc04806e810 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 >> v1:172.21.15.45:6792/0 conn(0x7fc0480a0cb0 legacy=0x7fc048146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 >> v1:172.21.15.84:6790/0 conn(0x7fc0480a1080 legacy=0x7fc048149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 monclient: found mon.a 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc048120410 con 0x7fc04806e810 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 10 monclient: _finish_auth 0 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc044ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc04ee80640 5 monclient: authenticate success, global_id 6633 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc04ee80640 10 monclient: _renew_subs 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc04ee80640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc04ee80640 1 -- 172.21.15.45:0/1163710483 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0481225e0 con 0x7fc04806e810 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc04ee80640 10 monclient: _renew_subs 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc04ee80640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.541+0000 7fc04ee80640 1 -- 172.21.15.45:0/1163710483 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc048121710 con 0x7fc04806e810 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.542+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc0380047a0 con 0x7fc04806e810 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.542+0000 7fc044ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.542+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc038004d10 con 0x7fc04806e810 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.542+0000 7fc044ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.542+0000 7fc044ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.542+0000 7fc046ffd640 4 set_mon_vals no callback set 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.542+0000 7fc044ff9640 10 monclient: dump: 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:09.566 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:09.566 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:09.566 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:09.566 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:09.566 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:09.566 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:09.566 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:09.566 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:09.566 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:09.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.542+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc038004f30 con 0x7fc04806e810 2024-09-18T14:19:09.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.542+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7fc038046750 con 0x7fc04806e810 2024-09-18T14:19:09.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.542+0000 7fc04ee80640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:09.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.543+0000 7fc04ee80640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:09.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.543+0000 7fc04ee80640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:09.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.543+0000 7fc04ee80640 1 -- 172.21.15.45:0/1163710483 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02c005010 con 0x7fc04806e810 2024-09-18T14:19:09.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.543+0000 7fc047fff640 10 monclient: get_auth_request con 0x7fc02803aa40 auth_method 0 2024-09-18T14:19:09.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.549+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fc038015120 con 0x7fc04806e810 2024-09-18T14:19:09.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.549+0000 7fc044ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:09.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.549+0000 7fc044ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:09.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.786+0000 7fc04ee80640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:19:09.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.786+0000 7fc04ee80640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:19:09.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.786+0000 7fc04ee80640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:09.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:09.786+0000 7fc04ee80640 1 -- 172.21.15.45:0/1163710483 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc02c005840 con 0x7fc04806e810 2024-09-18T14:19:12.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:12.540+0000 7fc03f7fe640 10 monclient: tick 2024-09-18T14:19:12.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:12.540+0000 7fc03f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:12.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:12.540+0000 7fc03f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:13.541+0000 7fc03f7fe640 10 monclient: tick 2024-09-18T14:19:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:13.541+0000 7fc03f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:14.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.540+0000 7fc03f7fe640 10 monclient: tick 2024-09-18T14:19:14.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.540+0000 7fc03f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:14.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.588+0000 7fc044ff9640 1 -- 172.21.15.45:0/1163710483 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2683 (unknown 254746657 0 1864327281) 0x7fc038014b20 con 0x7fc04806e810 2024-09-18T14:19:14.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.588+0000 7fc044ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:19:14.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.588+0000 7fc044ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:14.590 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":154,"quorum":[0,1,3,4,5,6,7,8],"quorum_names":["a","f","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:19:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.593+0000 7fc04ee80640 1 -- 172.21.15.45:0/1163710483 >> v1:172.21.15.84:6812/31478 conn(0x7fc02803aa40 legacy=0x7fc02803cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.593+0000 7fc04ee80640 10 monclient: shutdown 2024-09-18T14:19:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.593+0000 7fc04ee80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.593+0000 7fc04ee80640 1 -- 172.21.15.45:0/1163710483 >> v1:172.21.15.45:6789/0 conn(0x7fc04806e810 legacy=0x7fc04813d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.593+0000 7fc04ee80640 1 -- 172.21.15.45:0/1163710483 shutdown_connections 2024-09-18T14:19:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.593+0000 7fc04ee80640 1 -- 172.21.15.45:0/1163710483 >> 172.21.15.45:0/1163710483 conn(0x7fc0480a5bb0 msgr2=0x7fc0480a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.593+0000 7fc04ee80640 1 -- 172.21.15.45:0/1163710483 shutdown_connections 2024-09-18T14:19:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:14.593+0000 7fc04ee80640 1 -- 172.21.15.45:0/1163710483 wait complete. 2024-09-18T14:19:17.612 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:19:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.720+0000 7f7975a3f640 10 monclient: get_monmap_and_config 2024-09-18T14:19:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.720+0000 7f7975a3f640 10 monclient: build_initial_monmap 2024-09-18T14:19:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.720+0000 7f7975a3f640 10 monclient: monmap: 2024-09-18T14:19:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:17.721147+0000 2024-09-18T14:19:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:17.721147+0000 2024-09-18T14:19:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:17.722 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:17.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.720+0000 7f7975a3f640 1 Processor -- start 2024-09-18T14:19:17.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f7975a3f640 1 -- start start 2024-09-18T14:19:17.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f7975a3f640 10 monclient: init 2024-09-18T14:19:17.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f7975a3f640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:17.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f7975a3f640 10 monclient: _add_conns ranks=[7,3,2,5,1,8,6,0,4] 2024-09-18T14:19:17.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f7975a3f640 10 monclient(hunting): picked mon.noname-i con 0x7f797006e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f7975a3f640 10 monclient(hunting): picked mon.noname-g con 0x7f79700a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f7975a3f640 10 monclient(hunting): picked mon.noname-b con 0x7f79700a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:19:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f7975a3f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797005ed70 con 0x7f79700a1080 2024-09-18T14:19:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f7975a3f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7970097620 con 0x7f79700a0cb0 2024-09-18T14:19:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f7975a3f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7970097340 con 0x7f797006e810 2024-09-18T14:19:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f7975a3f640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f7975a3f640 10 monclient(hunting): authenticate will time out at 1973.059673s 2024-09-18T14:19:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f796f7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f79700a1080 0x7f797013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39724/0 (socket says 172.21.15.45:39724) 2024-09-18T14:19:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.721+0000 7f796f7fe640 1 -- 172.21.15.45:0/3036453404 learned_addr learned my addr 172.21.15.45:0/3036453404 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 1 -- 172.21.15.45:0/3036453404 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701255885 0 0) 0x7f7970097340 con 0x7f797006e810 2024-09-18T14:19:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient(hunting): my global_id is 214594 2024-09-18T14:19:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 1 -- 172.21.15.45:0/3036453404 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f795c0034e0 con 0x7f797006e810 2024-09-18T14:19:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 1 -- 172.21.15.45:0/3036453404 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355953104 0 0) 0x7f7970097620 con 0x7f79700a0cb0 2024-09-18T14:19:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient(hunting): my global_id is 224337 2024-09-18T14:19:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 1 -- 172.21.15.45:0/3036453404 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7970097340 con 0x7f79700a0cb0 2024-09-18T14:19:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 1 -- 172.21.15.45:0/3036453404 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 572621597 0 0) 0x7f7970097340 con 0x7f79700a0cb0 2024-09-18T14:19:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 1 -- 172.21.15.45:0/3036453404 >> v1:172.21.15.84:6792/0 conn(0x7f797006e810 legacy=0x7f79700a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 1 -- 172.21.15.45:0/3036453404 >> v1:172.21.15.45:6790/0 conn(0x7f79700a1080 legacy=0x7f797013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:17.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 1 monclient: found mon.noname-g 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.84:6790/0 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 1 -- 172.21.15.45:0/3036453404 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79700a1660 con 0x7f79700a0cb0 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 1 -- 172.21.15.45:0/3036453404 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7958001810 con 0x7f79700a0cb0 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f7975a3f640 5 monclient: authenticate success, global_id 224337 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f7975a3f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient: dump: 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:17.733 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:17.734 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f7975a3f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 1 -- 172.21.15.45:0/3036453404 <== mon.3 v1:172.21.15.84:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f79580019f0 con 0x7f79700a0cb0 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 1 -- 172.21.15.45:0/3036453404 <== mon.3 v1:172.21.15.84:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7958001f20 con 0x7f79700a0cb0 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f796e7fc640 10 monclient: dump: 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f7975a3f640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.722+0000 7f7975a3f640 4 set_mon_vals no callback set 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.723+0000 7f7975a3f640 10 monclient: shutdown 2024-09-18T14:19:17.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.723+0000 7f7975a3f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.723+0000 7f7975a3f640 1 -- 172.21.15.45:0/3036453404 >> v1:172.21.15.84:6790/0 conn(0x7f79700a0cb0 legacy=0x7f7970137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.723+0000 7f7975a3f640 1 -- 172.21.15.45:0/3036453404 shutdown_connections 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.723+0000 7f7975a3f640 1 -- 172.21.15.45:0/3036453404 >> 172.21.15.45:0/3036453404 conn(0x7f79700a5bb0 msgr2=0x7f79700a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.723+0000 7f7975a3f640 1 -- 172.21.15.45:0/3036453404 shutdown_connections 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.724+0000 7f7975a3f640 1 -- 172.21.15.45:0/3036453404 wait complete. 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.724+0000 7f7975a3f640 10 monclient: build_initial_monmap 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.724+0000 7f7975a3f640 10 monclient: monmap: 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.724+0000 7f7975a3f640 1 Processor -- start 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f7975a3f640 1 -- start start 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f7975a3f640 10 monclient: init 2024-09-18T14:19:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f7975a3f640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f7975a3f640 10 monclient: _add_conns ranks=[5,7,3,8,1,4,0,6,2] 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f7975a3f640 10 monclient(hunting): picked mon.noname-h con 0x7f797006e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f7975a3f640 10 monclient(hunting): picked mon.noname-i con 0x7f79700a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f7975a3f640 10 monclient(hunting): picked mon.noname-g con 0x7f79700a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f7975a3f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7970122e90 con 0x7f79700a1080 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f7975a3f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797013dc80 con 0x7f797006e810 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f7975a3f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79701228d0 con 0x7f79700a0cb0 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f7975a3f640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f7975a3f640 10 monclient(hunting): authenticate will time out at 1973.063711s 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f796effd640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f797006e810 0x7f797013d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:60530/0 (socket says 172.21.15.45:60530) 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.725+0000 7f796effd640 1 -- 172.21.15.45:0/3419570345 learned_addr learned my addr 172.21.15.45:0/3419570345 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235547484 0 0) 0x7f79701228d0 con 0x7f79700a0cb0 2024-09-18T14:19:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): my global_id is 214603 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79500034e0 con 0x7f79700a0cb0 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701604236 0 0) 0x7f7970122e90 con 0x7f79700a1080 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): my global_id is 224346 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79701228d0 con 0x7f79700a1080 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430609990 0 0) 0x7f797013dc80 con 0x7f797006e810 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): my global_id is 185063 2024-09-18T14:19:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7970122e90 con 0x7f797006e810 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1564957429 0 0) 0x7f79500034e0 con 0x7f79700a0cb0 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f797013dc80 con 0x7f79700a0cb0 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7958001ea0 con 0x7f79700a0cb0 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): dump: 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1110085140 0 0) 0x7f79701228d0 con 0x7f79700a1080 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f79500034e0 con 0x7f79700a1080 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7968001c60 con 0x7f79700a1080 2024-09-18T14:19:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): dump: 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2416669055 0 0) 0x7f7970122e90 con 0x7f797006e810 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f79701228d0 con 0x7f797006e810 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7960001db0 con 0x7f797006e810 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.726+0000 7f7966ffd640 10 monclient(hunting): dump: 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:17.744 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:17.745 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:17.746 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:17.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 251081568 0 0) 0x7f797013dc80 con 0x7f79700a0cb0 2024-09-18T14:19:17.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 >> v1:172.21.15.84:6791/0 conn(0x7f797006e810 legacy=0x7f797013d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:17.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 >> v1:172.21.15.84:6790/0 conn(0x7f79700a1080 legacy=0x7f7970149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:17.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:19:17.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 1 monclient: found mon.i 2024-09-18T14:19:17.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:17.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:19:17.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79701203e0 con 0x7f79700a0cb0 2024-09-18T14:19:17.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 10 monclient: _finish_auth 0 2024-09-18T14:19:17.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:17.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7975a3f640 5 monclient: authenticate success, global_id 214603 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7975a3f640 10 monclient: _renew_subs 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7975a3f640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7975a3f640 1 -- 172.21.15.45:0/3419570345 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79701225f0 con 0x7f79700a0cb0 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7975a3f640 10 monclient: _renew_subs 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7975a3f640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7975a3f640 1 -- 172.21.15.45:0/3419570345 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7970121720 con 0x7f79700a0cb0 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7958002490 con 0x7f79700a0cb0 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7958002940 con 0x7f79700a0cb0 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f796dffb640 4 set_mon_vals no callback set 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 10 monclient: dump: 2024-09-18T14:19:17.747 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:17.748 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:17.749 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:17.749 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:17.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.727+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f795800d910 con 0x7f79700a0cb0 2024-09-18T14:19:17.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.728+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f7958044290 con 0x7f79700a0cb0 2024-09-18T14:19:17.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.728+0000 7f7975a3f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:17.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.728+0000 7f7975a3f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:17.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.728+0000 7f7975a3f640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:19:17.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.728+0000 7f7975a3f640 1 -- 172.21.15.45:0/3419570345 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7954005010 con 0x7f79700a0cb0 2024-09-18T14:19:17.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.728+0000 7f796effd640 10 monclient: get_auth_request con 0x7f795003a950 auth_method 0 2024-09-18T14:19:17.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.733+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.7 v1:172.21.15.84:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f7958012840 con 0x7f79700a0cb0 2024-09-18T14:19:17.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.733+0000 7f7966ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:17.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.733+0000 7f7966ffd640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.963+0000 7f7975a3f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:19:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.963+0000 7f7975a3f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:19:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.963+0000 7f7975a3f640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:19:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.963+0000 7f7975a3f640 1 -- 172.21.15.45:0/3419570345 --> v1:172.21.15.84:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7954005840 con 0x7f79700a0cb0 2024-09-18T14:19:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.964+0000 7f7966ffd640 1 -- 172.21.15.45:0/3419570345 <== mon.7 v1:172.21.15.84:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2689 (unknown 254746657 0 1989508231) 0x7f7958012660 con 0x7f79700a0cb0 2024-09-18T14:19:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.964+0000 7f7966ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:19:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.964+0000 7f7966ffd640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:17.965 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":156,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:19:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.966+0000 7f7975a3f640 1 -- 172.21.15.45:0/3419570345 >> v1:172.21.15.84:6812/31478 conn(0x7f795003a950 legacy=0x7f795003ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.966+0000 7f7975a3f640 10 monclient: shutdown 2024-09-18T14:19:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.966+0000 7f7975a3f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.966+0000 7f7975a3f640 1 -- 172.21.15.45:0/3419570345 >> v1:172.21.15.84:6792/0 conn(0x7f79700a0cb0 legacy=0x7f79701467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.966+0000 7f7975a3f640 1 -- 172.21.15.45:0/3419570345 shutdown_connections 2024-09-18T14:19:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.966+0000 7f7975a3f640 1 -- 172.21.15.45:0/3419570345 >> 172.21.15.45:0/3419570345 conn(0x7f79700a5bb0 msgr2=0x7f79700a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.966+0000 7f7975a3f640 1 -- 172.21.15.45:0/3419570345 shutdown_connections 2024-09-18T14:19:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:17.966+0000 7f7975a3f640 1 -- 172.21.15.45:0/3419570345 wait complete. 2024-09-18T14:19:17.977 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:19:17.977 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-18T14:19:17.977 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-18T14:19:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.074+0000 7f2cfba57640 10 monclient: get_monmap_and_config 2024-09-18T14:19:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.074+0000 7f2cfba57640 10 monclient: build_initial_monmap 2024-09-18T14:19:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.074+0000 7f2cfba57640 10 monclient: monmap: 2024-09-18T14:19:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:18.075363+0000 2024-09-18T14:19:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:18.075363+0000 2024-09-18T14:19:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:18.077 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 1 Processor -- start 2024-09-18T14:19:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 1 -- start start 2024-09-18T14:19:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 10 monclient: init 2024-09-18T14:19:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 10 monclient: _add_conns ranks=[6,1,0,5,4,7,3,8,2] 2024-09-18T14:19:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 10 monclient(hunting): picked mon.noname-d con 0x7f2cf409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 10 monclient(hunting): picked mon.noname-f con 0x7f2cf409fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 10 monclient(hunting): picked mon.noname-a con 0x7f2cf409ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf405ed70 con 0x7f2cf409ff70 2024-09-18T14:19:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf409dfc0 con 0x7f2cf409fba0 2024-09-18T14:19:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf409dce0 con 0x7f2cf409f7d0 2024-09-18T14:19:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cfba57640 10 monclient(hunting): authenticate will time out at 1973.413907s 2024-09-18T14:19:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cf97f3640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f2cf409f7d0 0x7f2cf4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56308/0 (socket says 172.21.15.45:56308) 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.075+0000 7f2cf97f3640 1 -- 172.21.15.45:0/1469964144 learned_addr learned my addr 172.21.15.45:0/1469964144 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288198939 0 0) 0x7f2cf405ed70 con 0x7f2cf409ff70 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient(hunting): my global_id is 6642 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ce00034e0 con 0x7f2cf409ff70 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322878036 0 0) 0x7f2cf409dce0 con 0x7f2cf409f7d0 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient(hunting): my global_id is 185190 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf405ed70 con 0x7f2cf409f7d0 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169572901 0 0) 0x7f2cf409dfc0 con 0x7f2cf409fba0 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient(hunting): my global_id is 214660 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf409dce0 con 0x7f2cf409fba0 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 328327783 0 0) 0x7f2ce00034e0 con 0x7f2cf409ff70 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 >> v1:172.21.15.45:6792/0 conn(0x7f2cf409f7d0 legacy=0x7f2cf4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 >> v1:172.21.15.84:6789/0 conn(0x7f2cf409fba0 legacy=0x7f2cf4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 monclient: found mon.noname-a 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40a1c00 con 0x7f2cf409ff70 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient: _finish_auth 0 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cfba57640 5 monclient: authenticate success, global_id 6642 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2cec001ae0 con 0x7f2cf409ff70 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cfba57640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient: dump: 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cfba57640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2cec001cc0 con 0x7f2cf409ff70 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 1 -- 172.21.15.45:0/1469964144 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2cec002270 con 0x7f2cf409ff70 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cfba57640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cfba57640 4 set_mon_vals no callback set 2024-09-18T14:19:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.076+0000 7f2cebfff640 10 monclient: dump: 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.077+0000 7f2cfba57640 10 monclient: shutdown 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.077+0000 7f2cfba57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.077+0000 7f2cfba57640 1 -- 172.21.15.45:0/1469964144 >> v1:172.21.15.45:6789/0 conn(0x7f2cf409ff70 legacy=0x7f2cf413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.077+0000 7f2cfba57640 1 -- 172.21.15.45:0/1469964144 shutdown_connections 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.077+0000 7f2cfba57640 1 -- 172.21.15.45:0/1469964144 >> 172.21.15.45:0/1469964144 conn(0x7f2cf40630b0 msgr2=0x7f2cf40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.077+0000 7f2cfba57640 1 -- 172.21.15.45:0/1469964144 shutdown_connections 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.077+0000 7f2cfba57640 1 -- 172.21.15.45:0/1469964144 wait complete. 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.077+0000 7f2cfba57640 10 monclient: build_initial_monmap 2024-09-18T14:19:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.077+0000 7f2cfba57640 10 monclient: monmap: 2024-09-18T14:19:18.092 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:18.092 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:18.092 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:18.092 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:18.092 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:18.092 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:18.092 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:18.092 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:18.092 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:18.092 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:18.092 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 1 Processor -- start 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 1 -- start start 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 10 monclient: init 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 10 monclient: _add_conns ranks=[2,3,7,1,0,4,6,8,5] 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 10 monclient(hunting): picked mon.noname-b con 0x7f2cf409f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 10 monclient(hunting): picked mon.noname-g con 0x7f2cf409fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 10 monclient(hunting): picked mon.noname-i con 0x7f2cf409ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf41428e0 con 0x7f2cf409f7d0 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf40a2560 con 0x7f2cf409fba0 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf409b800 con 0x7f2cf409ff70 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cfba57640 10 monclient(hunting): authenticate will time out at 1973.416857s 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cf97f3640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f2cf409f7d0 0x7f2cf4131670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39734/0 (socket says 172.21.15.45:39734) 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.078+0000 7f2cf97f3640 1 -- 172.21.15.45:0/2809308881 learned_addr learned my addr 172.21.15.45:0/2809308881 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529772062 0 0) 0x7f2cf41428e0 con 0x7f2cf409f7d0 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): my global_id is 234101 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd80034e0 con 0x7f2cf409f7d0 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283156865 0 0) 0x7f2cf409b800 con 0x7f2cf409ff70 2024-09-18T14:19:18.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): my global_id is 214612 2024-09-18T14:19:18.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf41428e0 con 0x7f2cf409ff70 2024-09-18T14:19:18.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2709847320 0 0) 0x7f2cd80034e0 con 0x7f2cf409f7d0 2024-09-18T14:19:18.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2cf409b800 con 0x7f2cf409f7d0 2024-09-18T14:19:18.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2cf0001610 con 0x7f2cf409f7d0 2024-09-18T14:19:18.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:19:18.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): dump: 2024-09-18T14:19:18.095 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774506830 0 0) 0x7f2cf40a2560 con 0x7f2cf409fba0 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient(hunting): my global_id is 224355 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd80034e0 con 0x7f2cf409fba0 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3576596543 0 0) 0x7f2cf409b800 con 0x7f2cf409f7d0 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 >> v1:172.21.15.84:6792/0 conn(0x7f2cf409ff70 legacy=0x7f2cf414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 >> v1:172.21.15.84:6790/0 conn(0x7f2cf409fba0 legacy=0x7f2cf4146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient: _finish_hunting 0 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 monclient: found mon.b 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:19:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf411e790 con 0x7f2cf409f7d0 2024-09-18T14:19:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient: _finish_auth 0 2024-09-18T14:19:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2cfba57640 5 monclient: authenticate success, global_id 234101 2024-09-18T14:19:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2cfba57640 10 monclient: _renew_subs 2024-09-18T14:19:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2cfba57640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:19:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2cfba57640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf411ea70 con 0x7f2cf409f7d0 2024-09-18T14:19:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2cfba57640 10 monclient: _renew_subs 2024-09-18T14:19:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2cfba57640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:19:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2cfba57640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf411d4a0 con 0x7f2cf409f7d0 2024-09-18T14:19:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2cf0004a50 con 0x7f2cf409f7d0 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2cf0004f00 con 0x7f2cf409f7d0 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ceb7fe640 4 set_mon_vals no callback set 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.079+0000 7f2ce97fa640 10 monclient: dump: 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.080+0000 7f2cfba57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.080+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2cf000fed0 con 0x7f2cf409f7d0 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.080+0000 7f2cfba57640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2cd4006580 con 0x7f2cd4002830 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.080+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f2cf0046960 con 0x7f2cf409f7d0 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.080+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809954346 0 0) 0x7f2cd4006580 con 0x7f2cd4002830 2024-09-18T14:19:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.080+0000 7f2ce97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.080+0000 7f2ce97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.080+0000 7f2ce97fa640 10 monclient(hunting): my global_id is 6651 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf409b800 con 0x7f2cd4002830 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2cf9ff4640 10 monclient: get_auth_request con 0x7f2cd8039ee0 auth_method 0 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 120371535 0 0) 0x7f2cf409b800 con 0x7f2cd4002830 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2cd4006580 con 0x7f2cd4002830 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2ce40019a0 con 0x7f2cd4002830 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 10 monclient: dump: 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1200450827 0 0) 0x7f2cd4006580 con 0x7f2cd4002830 2024-09-18T14:19:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2cd400a0a0 con 0x7f2cd4002830 2024-09-18T14:19:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2ce4001f50 con 0x7f2cd4002830 2024-09-18T14:19:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 >> v1:172.21.15.45:6789/0 conn(0x7f2cd4002830 legacy=0x7f2cd4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.081+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 >> v1:172.21.15.45:6789/0 conn(0x7f2cd4002830 legacy=0x7f2cd4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2cfba57640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:19:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2cfba57640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2cd40074f0 con 0x7f2cd40068a0 2024-09-18T14:19:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939030925 0 0) 0x7f2cd40074f0 con 0x7f2cd40068a0 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2ce97fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2ce97fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2ce97fa640 10 monclient(hunting): my global_id is 6660 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd4006580 con 0x7f2cd40068a0 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2938598818 0 0) 0x7f2cd4006580 con 0x7f2cd40068a0 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2cd40074f0 con 0x7f2cd40068a0 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2ce4002630 con 0x7f2cd40068a0 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2ce97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2ce97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.087+0000 7f2ce97fa640 10 monclient: dump: 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2020265788 0 0) 0x7f2cd40074f0 con 0x7f2cd40068a0 2024-09-18T14:19:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 --> v1:172.21.15.45:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2cd4008750 con 0x7f2cd40068a0 2024-09-18T14:19:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 <== mon.0 v1:172.21.15.45:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7389 (unknown 0 0 3959078133) 0x7f2ce4003a80 con 0x7f2cd40068a0 2024-09-18T14:19:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2ce97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:19:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2ce97fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 >> v1:172.21.15.45:6789/0 conn(0x7f2cd40068a0 legacy=0x7f2cd4006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2ce97fa640 1 -- 172.21.15.45:0/2809308881 >> v1:172.21.15.45:6789/0 conn(0x7f2cd40068a0 legacy=0x7f2cd4006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.107 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:18.107 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:19:18.107 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:19:18.107 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "leader", 2024-09-18T14:19:18.107 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 156, 2024-09-18T14:19:18.107 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:18.108 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:19:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:19:18.110 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:19:18.111 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:19:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:19:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:19:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:19:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.116 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:19:18.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:19:18.116 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.116 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.116 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:19:18.116 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.116 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.116 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.117 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:19:18.117 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:19:18.117 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.117 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.117 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.117 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:19:18.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:19:18.117 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.117 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.118 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:19:18.118 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.118 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.118 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:19:18.118 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:19:18.118 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.118 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.118 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.119 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:19:18.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:19:18.119 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.119 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.119 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:19:18.119 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.119 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.119 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.122 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:19:18.124 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:19:18.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.126 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:19:18.127 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:18.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2cfba57640 1 -- 172.21.15.45:0/2809308881 >> v1:172.21.15.84:6812/31478 conn(0x7f2cd8039ee0 legacy=0x7f2cd803c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2cfba57640 10 monclient: shutdown 2024-09-18T14:19:18.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2cfba57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:18.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2cfba57640 1 -- 172.21.15.45:0/2809308881 >> v1:172.21.15.45:6790/0 conn(0x7f2cf409f7d0 legacy=0x7f2cf4131670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2cf9ff4640 1 -- 172.21.15.45:0/2809308881 reap_dead start 2024-09-18T14:19:18.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2cf9ff4640 1 -- 172.21.15.45:0/2809308881 reap_dead start 2024-09-18T14:19:18.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2cfba57640 1 -- 172.21.15.45:0/2809308881 shutdown_connections 2024-09-18T14:19:18.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2cfba57640 1 -- 172.21.15.45:0/2809308881 >> 172.21.15.45:0/2809308881 conn(0x7f2cf40630b0 msgr2=0x7f2cf40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:18.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2cfba57640 1 -- 172.21.15.45:0/2809308881 shutdown_connections 2024-09-18T14:19:18.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.088+0000 7f2cfba57640 1 -- 172.21.15.45:0/2809308881 wait complete. 2024-09-18T14:19:18.129 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:19:18.129 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-18T14:19:18.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-18T14:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.225+0000 7fbd69d1d640 10 monclient: get_monmap_and_config 2024-09-18T14:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.225+0000 7fbd69d1d640 10 monclient: build_initial_monmap 2024-09-18T14:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.225+0000 7fbd69d1d640 10 monclient: monmap: 2024-09-18T14:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:18.226614+0000 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:18.226614+0000 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:18.227 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.226+0000 7fbd69d1d640 1 Processor -- start 2024-09-18T14:19:18.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.226+0000 7fbd69d1d640 1 -- start start 2024-09-18T14:19:18.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.226+0000 7fbd69d1d640 10 monclient: init 2024-09-18T14:19:18.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.226+0000 7fbd69d1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:18.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.226+0000 7fbd69d1d640 10 monclient: _add_conns ranks=[4,8,5,3,1,6,2,7,0] 2024-09-18T14:19:18.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.226+0000 7fbd69d1d640 10 monclient(hunting): picked mon.noname-c con 0x7fbd640a2520 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:18.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.226+0000 7fbd69d1d640 10 monclient(hunting): picked mon.noname-e con 0x7fbd64096a20 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:18.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.226+0000 7fbd69d1d640 10 monclient(hunting): picked mon.noname-h con 0x7fbd64132fc0 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:18.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.226+0000 7fbd69d1d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6405ed70 con 0x7fbd640a2520 2024-09-18T14:19:18.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd69d1d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd64097040 con 0x7fbd64132fc0 2024-09-18T14:19:18.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd69d1d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640a28f0 con 0x7fbd64096a20 2024-09-18T14:19:18.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd69d1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:18.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd69d1d640 10 monclient(hunting): authenticate will time out at 1973.565057s 2024-09-18T14:19:18.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd637fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fbd640a2520 0x7fbd641328b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36870/0 (socket says 172.21.15.45:36870) 2024-09-18T14:19:18.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd62ffd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fbd64096a20 0x7fbd64137750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34720/0 (socket says 172.21.15.45:34720) 2024-09-18T14:19:18.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd637fe640 1 -- 172.21.15.45:0/748083622 learned_addr learned my addr 172.21.15.45:0/748083622 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:18.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 1 -- 172.21.15.45:0/748083622 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52606272 0 0) 0x7fbd640a28f0 con 0x7fbd64096a20 2024-09-18T14:19:18.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient(hunting): my global_id is 234107 2024-09-18T14:19:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 1 -- 172.21.15.45:0/748083622 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd4c0034e0 con 0x7fbd64096a20 2024-09-18T14:19:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 1 -- 172.21.15.45:0/748083622 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538965784 0 0) 0x7fbd6405ed70 con 0x7fbd640a2520 2024-09-18T14:19:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient(hunting): my global_id is 165334 2024-09-18T14:19:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 1 -- 172.21.15.45:0/748083622 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd640a28f0 con 0x7fbd640a2520 2024-09-18T14:19:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd63fff640 1 -- 172.21.15.45:0/748083622 >> v1:172.21.15.84:6791/0 conn(0x7fbd64132fc0 legacy=0x7fbd6413ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-09-18T14:19:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd63fff640 1 -- 172.21.15.45:0/748083622 >> v1:172.21.15.84:6791/0 conn(0x7fbd64132fc0 legacy=0x7fbd6413ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:19:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd63fff640 1 --1- 172.21.15.45:0/748083622 >> v1:172.21.15.84:6791/0 conn(0x7fbd64132fc0 0x7fbd6413ae80 :-1 s=OPENED pgs=603 cs=1 l=1).handle_message read tag failed 2024-09-18T14:19:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd63fff640 1 --1- 172.21.15.45:0/748083622 >> v1:172.21.15.84:6791/0 conn(0x7fbd64132fc0 0x7fbd6413ae80 :-1 s=OPENED pgs=603 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:19:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.84:6791/0 2024-09-18T14:19:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 1 -- 172.21.15.45:0/748083622 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2471801990 0 0) 0x7fbd4c0034e0 con 0x7fbd64096a20 2024-09-18T14:19:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 1 -- 172.21.15.45:0/748083622 >> v1:172.21.15.84:6791/0 conn(0x7fbd64132fc0 legacy=0x7fbd6413ae80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 1 -- 172.21.15.45:0/748083622 >> v1:172.21.15.45:6791/0 conn(0x7fbd640a2520 legacy=0x7fbd641328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 1 monclient: found mon.noname-e 2024-09-18T14:19:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:19:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 1 -- 172.21.15.45:0/748083622 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6409f4b0 con 0x7fbd64096a20 2024-09-18T14:19:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 1 -- 172.21.15.45:0/748083622 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd58001760 con 0x7fbd64096a20 2024-09-18T14:19:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient: dump: 2024-09-18T14:19:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.242 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd69d1d640 5 monclient: authenticate success, global_id 234107 2024-09-18T14:19:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd69d1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 1 -- 172.21.15.45:0/748083622 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbd58001940 con 0x7fbd64096a20 2024-09-18T14:19:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 1 -- 172.21.15.45:0/748083622 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd580047a0 con 0x7fbd64096a20 2024-09-18T14:19:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd69d1d640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd69d1d640 4 set_mon_vals no callback set 2024-09-18T14:19:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.227+0000 7fbd627fc640 10 monclient: dump: 2024-09-18T14:19:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.244 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.228+0000 7fbd69d1d640 10 monclient: shutdown 2024-09-18T14:19:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.228+0000 7fbd69d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.228+0000 7fbd69d1d640 1 -- 172.21.15.45:0/748083622 >> v1:172.21.15.45:6793/0 conn(0x7fbd64096a20 legacy=0x7fbd64137750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.228+0000 7fbd69d1d640 1 -- 172.21.15.45:0/748083622 shutdown_connections 2024-09-18T14:19:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.228+0000 7fbd69d1d640 1 -- 172.21.15.45:0/748083622 >> 172.21.15.45:0/748083622 conn(0x7fbd64063130 msgr2=0x7fbd64063500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.228+0000 7fbd69d1d640 1 -- 172.21.15.45:0/748083622 shutdown_connections 2024-09-18T14:19:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.228+0000 7fbd69d1d640 1 -- 172.21.15.45:0/748083622 wait complete. 2024-09-18T14:19:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.228+0000 7fbd69d1d640 10 monclient: build_initial_monmap 2024-09-18T14:19:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.228+0000 7fbd69d1d640 10 monclient: monmap: 2024-09-18T14:19:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:18.246 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.229+0000 7fbd69d1d640 1 Processor -- start 2024-09-18T14:19:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.229+0000 7fbd69d1d640 1 -- start start 2024-09-18T14:19:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.229+0000 7fbd69d1d640 10 monclient: init 2024-09-18T14:19:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.229+0000 7fbd69d1d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.229+0000 7fbd69d1d640 10 monclient: _add_conns ranks=[4,6,3,7,8,2,1,0,5] 2024-09-18T14:19:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.229+0000 7fbd69d1d640 10 monclient(hunting): picked mon.noname-c con 0x7fbd64096a20 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.229+0000 7fbd69d1d640 10 monclient(hunting): picked mon.noname-d con 0x7fbd640a2520 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.229+0000 7fbd69d1d640 10 monclient(hunting): picked mon.noname-g con 0x7fbd64132fc0 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.229+0000 7fbd69d1d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6409f8e0 con 0x7fbd64132fc0 2024-09-18T14:19:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.229+0000 7fbd69d1d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640a2270 con 0x7fbd64096a20 2024-09-18T14:19:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd69d1d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd64142320 con 0x7fbd640a2520 2024-09-18T14:19:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd69d1d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd69d1d640 10 monclient(hunting): authenticate will time out at 1973.568045s 2024-09-18T14:19:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd637fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fbd64096a20 0x7fbd641313c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36880/0 (socket says 172.21.15.45:36880) 2024-09-18T14:19:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd637fe640 1 -- 172.21.15.45:0/1495734203 learned_addr learned my addr 172.21.15.45:0/1495734203 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212596244 0 0) 0x7fbd640a2270 con 0x7fbd64096a20 2024-09-18T14:19:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): my global_id is 165343 2024-09-18T14:19:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd500034e0 con 0x7fbd64096a20 2024-09-18T14:19:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454601342 0 0) 0x7fbd64142320 con 0x7fbd640a2520 2024-09-18T14:19:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): my global_id is 185199 2024-09-18T14:19:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd640a2270 con 0x7fbd640a2520 2024-09-18T14:19:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790739869 0 0) 0x7fbd6409f8e0 con 0x7fbd64132fc0 2024-09-18T14:19:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): my global_id is 224364 2024-09-18T14:19:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd64142320 con 0x7fbd64132fc0 2024-09-18T14:19:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3295151421 0 0) 0x7fbd500034e0 con 0x7fbd64096a20 2024-09-18T14:19:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd6409f8e0 con 0x7fbd64096a20 2024-09-18T14:19:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 657151617 0 0) 0x7fbd640a2270 con 0x7fbd640a2520 2024-09-18T14:19:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd500034e0 con 0x7fbd640a2520 2024-09-18T14:19:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd54001570 con 0x7fbd64096a20 2024-09-18T14:19:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:19:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): dump: 2024-09-18T14:19:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.251 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd580016a0 con 0x7fbd640a2520 2024-09-18T14:19:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): dump: 2024-09-18T14:19:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.253 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3237323570 0 0) 0x7fbd64142320 con 0x7fbd64132fc0 2024-09-18T14:19:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd640a2270 con 0x7fbd64132fc0 2024-09-18T14:19:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd5c001d10 con 0x7fbd64132fc0 2024-09-18T14:19:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient(hunting): dump: 2024-09-18T14:19:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.255 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3803333442 0 0) 0x7fbd6409f8e0 con 0x7fbd64096a20 2024-09-18T14:19:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 >> v1:172.21.15.45:6792/0 conn(0x7fbd640a2520 legacy=0x7fbd64146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 >> v1:172.21.15.84:6790/0 conn(0x7fbd64132fc0 legacy=0x7fbd64149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:19:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 monclient: found mon.c 2024-09-18T14:19:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd641201a0 con 0x7fbd64096a20 2024-09-18T14:19:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 10 monclient: _finish_auth 0 2024-09-18T14:19:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd69d1d640 5 monclient: authenticate success, global_id 165343 2024-09-18T14:19:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd69d1d640 10 monclient: _renew_subs 2024-09-18T14:19:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd69d1d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd69d1d640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd641205b0 con 0x7fbd64096a20 2024-09-18T14:19:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd69d1d640 10 monclient: _renew_subs 2024-09-18T14:19:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd69d1d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.230+0000 7fbd69d1d640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6411f9e0 con 0x7fbd64096a20 2024-09-18T14:19:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.231+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbd54004a50 con 0x7fbd64096a20 2024-09-18T14:19:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.231+0000 7fbd4b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.231+0000 7fbd61ffb640 4 set_mon_vals no callback set 2024-09-18T14:19:18.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.231+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd54004f00 con 0x7fbd64096a20 2024-09-18T14:19:18.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.231+0000 7fbd4b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.231+0000 7fbd4b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:18.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.231+0000 7fbd4b7fe640 10 monclient: dump: 2024-09-18T14:19:18.257 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.257 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.257 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.257 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.257 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.257 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.257 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.258 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.258 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.258 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.258 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.258 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.258 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.258 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.258 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.258 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.231+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fbd5400fed0 con 0x7fbd64096a20 2024-09-18T14:19:18.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.231+0000 7fbd69d1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:18.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.231+0000 7fbd69d1d640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbd44006580 con 0x7fbd44002830 2024-09-18T14:19:18.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7fbd54046a20 con 0x7fbd64096a20 2024-09-18T14:19:18.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd63fff640 10 monclient: get_auth_request con 0x7fbd5003a9b0 auth_method 0 2024-09-18T14:19:18.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033084639 0 0) 0x7fbd44006580 con 0x7fbd44002830 2024-09-18T14:19:18.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 10 monclient(hunting): my global_id is 234110 2024-09-18T14:19:18.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd6409f8e0 con 0x7fbd44002830 2024-09-18T14:19:18.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2118252864 0 0) 0x7fbd6409f8e0 con 0x7fbd44002830 2024-09-18T14:19:18.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd44006580 con 0x7fbd44002830 2024-09-18T14:19:18.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd58005050 con 0x7fbd44002830 2024-09-18T14:19:18.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:19:18.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 10 monclient: dump: 2024-09-18T14:19:18.260 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.260 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.260 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.260 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.260 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.260 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.260 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.260 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.260 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.260 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.260 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.261 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.261 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.261 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.261 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.261 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1324877638 0 0) 0x7fbd44006580 con 0x7fbd44002830 2024-09-18T14:19:18.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd4400a0a0 con 0x7fbd44002830 2024-09-18T14:19:18.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fbd58005360 con 0x7fbd44002830 2024-09-18T14:19:18.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:18.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:18.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 >> v1:172.21.15.45:6790/0 conn(0x7fbd44002830 legacy=0x7fbd44004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.232+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 >> v1:172.21.15.45:6790/0 conn(0x7fbd44002830 legacy=0x7fbd44004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.238+0000 7fbd69d1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:19:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.238+0000 7fbd69d1d640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbd44009660 con 0x7fbd44009040 2024-09-18T14:19:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.238+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472488886 0 0) 0x7fbd44009660 con 0x7fbd44009040 2024-09-18T14:19:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 10 monclient(hunting): my global_id is 234119 2024-09-18T14:19:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd44006580 con 0x7fbd44009040 2024-09-18T14:19:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3443096491 0 0) 0x7fbd44006580 con 0x7fbd44009040 2024-09-18T14:19:18.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd44009660 con 0x7fbd44009040 2024-09-18T14:19:18.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fbd58005c60 con 0x7fbd44009040 2024-09-18T14:19:18.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:19:18.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 10 monclient: dump: 2024-09-18T14:19:18.264 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.264 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.264 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.264 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.264 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.264 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.265 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2465668234 0 0) 0x7fbd44009660 con 0x7fbd44009040 2024-09-18T14:19:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 --> v1:172.21.15.45:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbd44009920 con 0x7fbd44009040 2024-09-18T14:19:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 <== mon.2 v1:172.21.15.45:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 3054473918) 0x7fbd58008350 con 0x7fbd44009040 2024-09-18T14:19:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:19:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 >> v1:172.21.15.45:6790/0 conn(0x7fbd44009040 legacy=0x7fbd44008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd4b7fe640 1 -- 172.21.15.45:0/1495734203 >> v1:172.21.15.45:6790/0 conn(0x7fbd44009040 legacy=0x7fbd44008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.266 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:19:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:19:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:19:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 156, 2024-09-18T14:19:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:19:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:19:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:19:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:19:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:19:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:19:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:19:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:19:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:19:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:19:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:19:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:19:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:19:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:19:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:19:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:19:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:19:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:19:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:19:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:19:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:19:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:19:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:19:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:19:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:19:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:19:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:19:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:19:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:19:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:19:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:19:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:19:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:19:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:19:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:19:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:19:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:19:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:19:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:19:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:19:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:19:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:19:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:19:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:19:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:19:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:19:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:19:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:19:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:19:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:19:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:19:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:19:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:19:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:19:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:19:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:19:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:19:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:19:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:19:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:19:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:19:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:19:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:19:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:19:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:19:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:19:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:19:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:19:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:19:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:19:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:19:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:19:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:19:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:19:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:19:18.289 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:18.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd69d1d640 1 -- 172.21.15.45:0/1495734203 >> v1:172.21.15.84:6812/31478 conn(0x7fbd5003a9b0 legacy=0x7fbd5003ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd69d1d640 10 monclient: shutdown 2024-09-18T14:19:18.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd69d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:18.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd69d1d640 1 -- 172.21.15.45:0/1495734203 >> v1:172.21.15.45:6791/0 conn(0x7fbd64096a20 legacy=0x7fbd641313c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.239+0000 7fbd63fff640 1 -- 172.21.15.45:0/1495734203 reap_dead start 2024-09-18T14:19:18.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.240+0000 7fbd63fff640 1 -- 172.21.15.45:0/1495734203 reap_dead start 2024-09-18T14:19:18.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.240+0000 7fbd69d1d640 1 -- 172.21.15.45:0/1495734203 shutdown_connections 2024-09-18T14:19:18.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.240+0000 7fbd69d1d640 1 -- 172.21.15.45:0/1495734203 >> 172.21.15.45:0/1495734203 conn(0x7fbd64063130 msgr2=0x7fbd640a3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:18.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.240+0000 7fbd69d1d640 1 -- 172.21.15.45:0/1495734203 shutdown_connections 2024-09-18T14:19:18.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.240+0000 7fbd69d1d640 1 -- 172.21.15.45:0/1495734203 wait complete. 2024-09-18T14:19:18.292 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:19:18.292 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-18T14:19:18.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-18T14:19:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.389+0000 7fd74e434640 10 monclient: get_monmap_and_config 2024-09-18T14:19:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.389+0000 7fd74e434640 10 monclient: build_initial_monmap 2024-09-18T14:19:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.389+0000 7fd74e434640 10 monclient: monmap: 2024-09-18T14:19:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:18.390399+0000 2024-09-18T14:19:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:18.390399+0000 2024-09-18T14:19:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:18.393 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.391+0000 7fd74e434640 1 Processor -- start 2024-09-18T14:19:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.391+0000 7fd74e434640 1 -- start start 2024-09-18T14:19:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.391+0000 7fd74e434640 10 monclient: init 2024-09-18T14:19:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.392+0000 7fd74e434640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.392+0000 7fd74e434640 10 monclient: _add_conns ranks=[5,1,0,7,3,8,4,2,6] 2024-09-18T14:19:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.392+0000 7fd74e434640 10 monclient(hunting): picked mon.noname-h con 0x7fd74812e5a0 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.392+0000 7fd74e434640 10 monclient(hunting): picked mon.noname-f con 0x7fd74806a300 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.392+0000 7fd74e434640 10 monclient(hunting): picked mon.noname-a con 0x7fd74806eb60 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.392+0000 7fd74e434640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74805ed70 con 0x7fd74806eb60 2024-09-18T14:19:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.392+0000 7fd74e434640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74806b950 con 0x7fd74806a300 2024-09-18T14:19:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.392+0000 7fd74e434640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74806b670 con 0x7fd74812e5a0 2024-09-18T14:19:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.392+0000 7fd74e434640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.392+0000 7fd74e434640 10 monclient(hunting): authenticate will time out at 1973.730722s 2024-09-18T14:19:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.392+0000 7fd74c9d1640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd74806eb60 0x7fd7481366f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58878/0 (socket says 172.21.15.45:58878) 2024-09-18T14:19:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.392+0000 7fd74c9d1640 1 -- 172.21.15.45:0/2948972378 learned_addr learned my addr 172.21.15.45:0/2948972378 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 1 -- 172.21.15.45:0/2948972378 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231406096 0 0) 0x7fd74805ed70 con 0x7fd74806eb60 2024-09-18T14:19:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 10 monclient(hunting): my global_id is 6669 2024-09-18T14:19:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 1 -- 172.21.15.45:0/2948972378 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7300034e0 con 0x7fd74806eb60 2024-09-18T14:19:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 1 -- 172.21.15.45:0/2948972378 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4205694473 0 0) 0x7fd7300034e0 con 0x7fd74806eb60 2024-09-18T14:19:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 1 -- 172.21.15.45:0/2948972378 >> v1:172.21.15.84:6791/0 conn(0x7fd74812e5a0 legacy=0x7fd74812e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-18T14:19:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 1 -- 172.21.15.45:0/2948972378 >> v1:172.21.15.84:6789/0 conn(0x7fd74806a300 legacy=0x7fd74806e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:19:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 1 monclient: found mon.noname-a 2024-09-18T14:19:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:19:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 1 -- 172.21.15.45:0/2948972378 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7480a22c0 con 0x7fd74806eb60 2024-09-18T14:19:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 10 monclient: _finish_auth 0 2024-09-18T14:19:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 1 -- 172.21.15.45:0/2948972378 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd740001bb0 con 0x7fd74806eb60 2024-09-18T14:19:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:19:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd746ffd640 10 monclient: dump: 2024-09-18T14:19:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.401 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd74e434640 5 monclient: authenticate success, global_id 6669 2024-09-18T14:19:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.393+0000 7fd74e434640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd746ffd640 1 -- 172.21.15.45:0/2948972378 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd740001e40 con 0x7fd74806eb60 2024-09-18T14:19:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd746ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd74e434640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd746ffd640 1 -- 172.21.15.45:0/2948972378 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd7400020b0 con 0x7fd74806eb60 2024-09-18T14:19:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd746ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd74e434640 4 set_mon_vals no callback set 2024-09-18T14:19:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd746ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd746ffd640 10 monclient: dump: 2024-09-18T14:19:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.403 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd74e434640 10 monclient: shutdown 2024-09-18T14:19:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd74e434640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd74e434640 1 -- 172.21.15.45:0/2948972378 >> v1:172.21.15.45:6789/0 conn(0x7fd74806eb60 legacy=0x7fd7481366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd74e434640 1 -- 172.21.15.45:0/2948972378 shutdown_connections 2024-09-18T14:19:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd74e434640 1 -- 172.21.15.45:0/2948972378 >> 172.21.15.45:0/2948972378 conn(0x7fd748067d00 msgr2=0x7fd7480a19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.394+0000 7fd74e434640 1 -- 172.21.15.45:0/2948972378 shutdown_connections 2024-09-18T14:19:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.395+0000 7fd74e434640 1 -- 172.21.15.45:0/2948972378 wait complete. 2024-09-18T14:19:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.395+0000 7fd74e434640 10 monclient: build_initial_monmap 2024-09-18T14:19:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.395+0000 7fd74e434640 10 monclient: monmap: 2024-09-18T14:19:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:18.405 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.396+0000 7fd74e434640 1 Processor -- start 2024-09-18T14:19:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.396+0000 7fd74e434640 1 -- start start 2024-09-18T14:19:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.396+0000 7fd74e434640 10 monclient: init 2024-09-18T14:19:18.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd74e434640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:18.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd74e434640 10 monclient: _add_conns ranks=[7,4,0,5,2,3,8,6,1] 2024-09-18T14:19:18.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd74e434640 10 monclient(hunting): picked mon.noname-i con 0x7fd74806a300 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd74e434640 10 monclient(hunting): picked mon.noname-c con 0x7fd74806eb60 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd74e434640 10 monclient(hunting): picked mon.noname-a con 0x7fd74812e5a0 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd74e434640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74809bca0 con 0x7fd74812e5a0 2024-09-18T14:19:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd74e434640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74813db90 con 0x7fd74806eb60 2024-09-18T14:19:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd74e434640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74812b800 con 0x7fd74806a300 2024-09-18T14:19:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd74e434640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd74e434640 10 monclient(hunting): authenticate will time out at 1973.735379s 2024-09-18T14:19:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd7477fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fd74806eb60 0x7fd74812d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36884/0 (socket says 172.21.15.45:36884) 2024-09-18T14:19:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd7477fe640 1 -- 172.21.15.45:0/2768775117 learned_addr learned my addr 172.21.15.45:0/2768775117 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd7477fe640 1 -- 172.21.15.45:0/2768775117 >> v1:172.21.15.45:6791/0 conn(0x7fd74806eb60 legacy=0x7fd74812d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-18T14:19:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd7477fe640 1 -- 172.21.15.45:0/2768775117 >> v1:172.21.15.45:6791/0 conn(0x7fd74806eb60 legacy=0x7fd74812d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:19:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd7477fe640 1 --1- 172.21.15.45:0/2768775117 >> v1:172.21.15.45:6791/0 conn(0x7fd74806eb60 0x7fd74812d5d0 :-1 s=OPENED pgs=493 cs=1 l=1).handle_message read tag failed 2024-09-18T14:19:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd7477fe640 1 --1- 172.21.15.45:0/2768775117 >> v1:172.21.15.45:6791/0 conn(0x7fd74806eb60 0x7fd74812d5d0 :-1 s=OPENED pgs=493 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:19:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd72ffff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.45:6791/0 2024-09-18T14:19:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614168548 0 0) 0x7fd74812b800 con 0x7fd74806a300 2024-09-18T14:19:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd72ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd72ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.397+0000 7fd72ffff640 10 monclient(hunting): my global_id is 214621 2024-09-18T14:19:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7340034e0 con 0x7fd74806a300 2024-09-18T14:19:18.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287009978 0 0) 0x7fd74809bca0 con 0x7fd74812e5a0 2024-09-18T14:19:18.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient(hunting): my global_id is 6678 2024-09-18T14:19:18.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd74812b800 con 0x7fd74812e5a0 2024-09-18T14:19:18.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 961330084 0 0) 0x7fd7340034e0 con 0x7fd74806a300 2024-09-18T14:19:18.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd74809bca0 con 0x7fd74806a300 2024-09-18T14:19:18.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3817129465 0 0) 0x7fd74812b800 con 0x7fd74812e5a0 2024-09-18T14:19:18.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd7340034e0 con 0x7fd74812e5a0 2024-09-18T14:19:18.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd738002970 con 0x7fd74806a300 2024-09-18T14:19:18.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:19:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient(hunting): dump: 2024-09-18T14:19:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.417 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd740001b70 con 0x7fd74812e5a0 2024-09-18T14:19:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:18.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient(hunting): dump: 2024-09-18T14:19:18.418 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.418 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.418 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.418 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.418 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.418 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.418 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.418 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.418 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.418 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.419 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1879537877 0 0) 0x7fd7340034e0 con 0x7fd74812e5a0 2024-09-18T14:19:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 >> v1:172.21.15.84:6792/0 conn(0x7fd74806a300 legacy=0x7fd74813d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 >> v1:172.21.15.45:6791/0 conn(0x7fd74806eb60 legacy=0x7fd74812d5d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 monclient: found mon.a 2024-09-18T14:19:18.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:18.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:18.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74812bae0 con 0x7fd74812e5a0 2024-09-18T14:19:18.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 10 monclient: _finish_auth 0 2024-09-18T14:19:18.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd72ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:18.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd74e434640 5 monclient: authenticate success, global_id 6678 2024-09-18T14:19:18.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd74e434640 10 monclient: _renew_subs 2024-09-18T14:19:18.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd74e434640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:18.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd74e434640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74813d860 con 0x7fd74812e5a0 2024-09-18T14:19:18.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd74e434640 10 monclient: _renew_subs 2024-09-18T14:19:18.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd74e434640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.398+0000 7fd74e434640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74812de80 con 0x7fd74812e5a0 2024-09-18T14:19:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.400+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd740001360 con 0x7fd74812e5a0 2024-09-18T14:19:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.400+0000 7fd72ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.400+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd740005380 con 0x7fd74812e5a0 2024-09-18T14:19:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.400+0000 7fd72ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.400+0000 7fd72ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.400+0000 7fd7467fc640 4 set_mon_vals no callback set 2024-09-18T14:19:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.400+0000 7fd72ffff640 10 monclient: dump: 2024-09-18T14:19:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.422 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.423 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.423 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.400+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd740010350 con 0x7fd74812e5a0 2024-09-18T14:19:18.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.400+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7fd740046d30 con 0x7fd74812e5a0 2024-09-18T14:19:18.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.401+0000 7fd747fff640 10 monclient: get_auth_request con 0x7fd73403e7b0 auth_method 0 2024-09-18T14:19:18.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.402+0000 7fd74e434640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:18.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.402+0000 7fd74e434640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd728006580 con 0x7fd728002830 2024-09-18T14:19:18.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869806131 0 0) 0x7fd728006580 con 0x7fd728002830 2024-09-18T14:19:18.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 10 monclient(hunting): my global_id is 165352 2024-09-18T14:19:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7340034e0 con 0x7fd728002830 2024-09-18T14:19:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 300260542 0 0) 0x7fd7340034e0 con 0x7fd728002830 2024-09-18T14:19:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd728006580 con 0x7fd728002830 2024-09-18T14:19:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd73c001a70 con 0x7fd728002830 2024-09-18T14:19:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 10 monclient: dump: 2024-09-18T14:19:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.424 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.425 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 393702548 0 0) 0x7fd728006580 con 0x7fd728002830 2024-09-18T14:19:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd72800a0a0 con 0x7fd728002830 2024-09-18T14:19:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd73c005780 con 0x7fd728002830 2024-09-18T14:19:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 >> v1:172.21.15.45:6791/0 conn(0x7fd728002830 legacy=0x7fd728004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.403+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 >> v1:172.21.15.45:6791/0 conn(0x7fd728002830 legacy=0x7fd728004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.410+0000 7fd74e434640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:19:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.410+0000 7fd74e434640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd728009660 con 0x7fd728009040 2024-09-18T14:19:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028468251 0 0) 0x7fd728009660 con 0x7fd728009040 2024-09-18T14:19:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 10 monclient(hunting): my global_id is 165361 2024-09-18T14:19:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd728006580 con 0x7fd728009040 2024-09-18T14:19:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3325580632 0 0) 0x7fd728006580 con 0x7fd728009040 2024-09-18T14:19:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd728009660 con 0x7fd728009040 2024-09-18T14:19:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd73c002300 con 0x7fd728009040 2024-09-18T14:19:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 10 monclient: dump: 2024-09-18T14:19:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.428 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 149498755 0 0) 0x7fd728009660 con 0x7fd728009040 2024-09-18T14:19:18.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 --> v1:172.21.15.45:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd728009920 con 0x7fd728009040 2024-09-18T14:19:18.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 <== mon.4 v1:172.21.15.45:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 1151048672) 0x7fd73c005a30 con 0x7fd728009040 2024-09-18T14:19:18.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:19:18.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:18.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 >> v1:172.21.15.45:6791/0 conn(0x7fd728009040 legacy=0x7fd728008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.411+0000 7fd72ffff640 1 -- 172.21.15.45:0/2768775117 >> v1:172.21.15.45:6791/0 conn(0x7fd728009040 legacy=0x7fd728008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.429 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:18.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:19:18.430 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:19:18.430 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:19:18.430 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 156, 2024-09-18T14:19:18.430 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:19:18.430 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:19:18.430 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:19:18.430 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:19:18.430 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:19:18.430 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:19:18.430 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:19:18.431 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:19:18.431 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:19:18.431 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:19:18.431 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.431 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:19:18.431 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:18.431 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:19:18.431 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:19:18.431 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.431 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.431 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.432 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.432 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.432 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.432 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.432 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.432 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.432 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:19:18.432 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:19:18.432 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.432 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.433 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.433 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.433 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.433 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.433 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.433 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.433 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:19:18.433 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:19:18.433 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:19:18.433 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:19:18.434 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:19:18.434 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:19:18.434 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:18.434 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:18.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:19:18.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:19:18.434 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:19:18.434 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:19:18.434 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:19:18.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:19:18.434 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:19:18.435 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:18.435 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:19:18.435 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.435 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.435 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.435 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.435 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.435 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.435 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.435 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.436 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.436 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:19:18.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:19:18.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:19:18.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:19:18.436 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.436 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.437 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:19:18.437 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.437 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.437 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:19:18.437 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:19:18.437 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.437 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.437 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:19:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:19:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:19:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.439 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:19:18.439 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:19:18.439 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.439 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.439 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:19:18.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:19:18.439 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.439 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.440 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:19:18.440 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.440 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.440 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.440 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:19:18.440 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:19:18.440 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.440 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.441 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:19:18.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:19:18.441 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.441 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.441 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:19:18.441 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.442 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.442 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.442 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:19:18.442 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:19:18.442 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.442 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.442 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:19:18.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:19:18.443 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.443 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.443 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:19:18.443 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.443 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.443 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:19:18.444 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:19:18.444 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.444 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.444 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:19:18.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:19:18.444 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.444 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:19:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:19:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:19:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:19:18.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:19:18.446 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.446 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.446 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:19:18.446 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.446 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.446 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.447 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:19:18.447 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:19:18.447 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.447 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.447 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:19:18.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:19:18.447 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.448 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.448 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:19:18.448 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.448 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.448 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:19:18.448 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:19:18.448 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.449 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.449 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:19:18.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:19:18.449 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:19:18.450 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.450 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:19:18.450 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:19:18.450 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.450 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.450 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.451 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:19:18.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:19:18.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.451 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.451 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.451 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:18.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.451 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:19:18.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.452 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.452 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.452 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:19:18.452 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.452 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.452 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:19:18.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.452 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.452 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.452 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 3 2024-09-18T14:19:18.452 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.453 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.453 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:19:18.453 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:18.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.412+0000 7fd74e434640 1 -- 172.21.15.45:0/2768775117 >> v1:172.21.15.84:6812/31478 conn(0x7fd73403e7b0 legacy=0x7fd734040c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.412+0000 7fd74e434640 10 monclient: shutdown 2024-09-18T14:19:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.412+0000 7fd74e434640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.412+0000 7fd74e434640 1 -- 172.21.15.45:0/2768775117 >> v1:172.21.15.45:6789/0 conn(0x7fd74812e5a0 legacy=0x7fd748145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.412+0000 7fd74c9d1640 1 -- 172.21.15.45:0/2768775117 reap_dead start 2024-09-18T14:19:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.412+0000 7fd74c9d1640 1 -- 172.21.15.45:0/2768775117 reap_dead start 2024-09-18T14:19:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.412+0000 7fd74e434640 1 -- 172.21.15.45:0/2768775117 shutdown_connections 2024-09-18T14:19:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.412+0000 7fd74e434640 1 -- 172.21.15.45:0/2768775117 >> 172.21.15.45:0/2768775117 conn(0x7fd748067d00 msgr2=0x7fd74809fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.412+0000 7fd74e434640 1 -- 172.21.15.45:0/2768775117 shutdown_connections 2024-09-18T14:19:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.412+0000 7fd74e434640 1 -- 172.21.15.45:0/2768775117 wait complete. 2024-09-18T14:19:18.455 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:19:18.456 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-18T14:19:18.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-18T14:19:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.553+0000 7feac24dd640 10 monclient: get_monmap_and_config 2024-09-18T14:19:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.553+0000 7feac24dd640 10 monclient: build_initial_monmap 2024-09-18T14:19:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.553+0000 7feac24dd640 10 monclient: monmap: 2024-09-18T14:19:18.554 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:18.554913+0000 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:18.554913+0000 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:18.556 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.554+0000 7feac24dd640 1 Processor -- start 2024-09-18T14:19:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.554+0000 7feac24dd640 1 -- start start 2024-09-18T14:19:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.554+0000 7feac24dd640 10 monclient: init 2024-09-18T14:19:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feac24dd640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feac24dd640 10 monclient: _add_conns ranks=[4,6,7,1,2,0,5,3,8] 2024-09-18T14:19:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feac24dd640 10 monclient(hunting): picked mon.noname-c con 0x7feabc09f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feac24dd640 10 monclient(hunting): picked mon.noname-d con 0x7feabc09fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feac24dd640 10 monclient(hunting): picked mon.noname-i con 0x7feabc09ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feac24dd640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feabc05ed70 con 0x7feabc09f7d0 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feac24dd640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feabc09dfc0 con 0x7feabc09fba0 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feac24dd640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feabc09dce0 con 0x7feabc09ff70 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feac24dd640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feac24dd640 10 monclient(hunting): authenticate will time out at 1973.893274s 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabbfff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7feabc09f7d0 0x7feabc132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36908/0 (socket says 172.21.15.45:36908) 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabb7fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7feabc09fba0 0x7feabc137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56330/0 (socket says 172.21.15.45:56330) 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabbfff640 1 -- 172.21.15.45:0/934331552 learned_addr learned my addr 172.21.15.45:0/934331552 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854246110 0 0) 0x7feabc09dfc0 con 0x7feabc09fba0 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient(hunting): my global_id is 185208 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa40034e0 con 0x7feabc09fba0 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002005415 0 0) 0x7feabc05ed70 con 0x7feabc09f7d0 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient(hunting): my global_id is 165370 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feabc09dfc0 con 0x7feabc09f7d0 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198854897 0 0) 0x7feabc09dce0 con 0x7feabc09ff70 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient(hunting): my global_id is 214630 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feabc05ed70 con 0x7feabc09ff70 2024-09-18T14:19:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 138179526 0 0) 0x7feaa40034e0 con 0x7feabc09fba0 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 >> v1:172.21.15.84:6792/0 conn(0x7feabc09ff70 legacy=0x7feabc13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 >> v1:172.21.15.45:6791/0 conn(0x7feabc09f7d0 legacy=0x7feabc132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient: _finish_hunting 0 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 1 monclient: found mon.noname-d 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc0a1c00 con 0x7feabc09fba0 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient: _finish_auth 0 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feac24dd640 5 monclient: authenticate success, global_id 185208 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feab0001830 con 0x7feabc09fba0 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:18.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.555+0000 7feabaffd640 10 monclient: dump: 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.569 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feac24dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feab0001a10 con 0x7feabc09fba0 2024-09-18T14:19:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feabaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feac24dd640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feabaffd640 1 -- 172.21.15.45:0/934331552 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feab0004ae0 con 0x7feabc09fba0 2024-09-18T14:19:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feabaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feac24dd640 4 set_mon_vals no callback set 2024-09-18T14:19:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feabaffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feabaffd640 10 monclient: dump: 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feac24dd640 10 monclient: shutdown 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feac24dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feac24dd640 1 -- 172.21.15.45:0/934331552 >> v1:172.21.15.45:6792/0 conn(0x7feabc09fba0 legacy=0x7feabc137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feac24dd640 1 -- 172.21.15.45:0/934331552 shutdown_connections 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feac24dd640 1 -- 172.21.15.45:0/934331552 >> 172.21.15.45:0/934331552 conn(0x7feabc0630b0 msgr2=0x7feabc0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feac24dd640 1 -- 172.21.15.45:0/934331552 shutdown_connections 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.556+0000 7feac24dd640 1 -- 172.21.15.45:0/934331552 wait complete. 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.557+0000 7feac24dd640 10 monclient: build_initial_monmap 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.557+0000 7feac24dd640 10 monclient: monmap: 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:18.572 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.557+0000 7feac24dd640 1 Processor -- start 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.557+0000 7feac24dd640 1 -- start start 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.557+0000 7feac24dd640 10 monclient: init 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.557+0000 7feac24dd640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.557+0000 7feac24dd640 10 monclient: _add_conns ranks=[2,8,0,4,7,1,6,5,3] 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.557+0000 7feac24dd640 10 monclient(hunting): picked mon.noname-b con 0x7feabc09f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feac24dd640 10 monclient(hunting): picked mon.noname-e con 0x7feabc09fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feac24dd640 10 monclient(hunting): picked mon.noname-a con 0x7feabc09ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feac24dd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feabc09d310 con 0x7feabc09ff70 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feac24dd640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feabc0a2560 con 0x7feabc09f7d0 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feac24dd640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feabc142600 con 0x7feabc09fba0 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feac24dd640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feac24dd640 10 monclient(hunting): authenticate will time out at 1973.896068s 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feabbfff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7feabc09f7d0 0x7feabc141730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39758/0 (socket says 172.21.15.45:39758) 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feabbfff640 1 -- 172.21.15.45:0/1878478679 learned_addr learned my addr 172.21.15.45:0/1878478679 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feabb7fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7feabc09fba0 0x7feabc131930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34734/0 (socket says 172.21.15.45:34734) 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feac0a7a640 1 --1- 172.21.15.45:0/1878478679 >> v1:172.21.15.45:6789/0 conn(0x7feabc09ff70 0x7feabc149a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58890/0 (socket says 172.21.15.45:58890) 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511854631 0 0) 0x7feabc0a2560 con 0x7feabc09f7d0 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): my global_id is 234128 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa80034e0 con 0x7feabc09f7d0 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743777362 0 0) 0x7feabc142600 con 0x7feabc09fba0 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): my global_id is 234116 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feabc0a2560 con 0x7feabc09fba0 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966074194 0 0) 0x7feabc09d310 con 0x7feabc09ff70 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): my global_id is 6687 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feabc142600 con 0x7feabc09ff70 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2936176703 0 0) 0x7feaa80034e0 con 0x7feabc09f7d0 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feabc09d310 con 0x7feabc09f7d0 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feaac001610 con 0x7feabc09f7d0 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): dump: 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 834121871 0 0) 0x7feabc0a2560 con 0x7feabc09fba0 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feaa80034e0 con 0x7feabc09fba0 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feab0004ae0 con 0x7feabc09fba0 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): dump: 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3671079725 0 0) 0x7feabc142600 con 0x7feabc09ff70 2024-09-18T14:19:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feabc0a2560 con 0x7feabc09ff70 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feab4001d00 con 0x7feabc09ff70 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient(hunting): dump: 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3174594069 0 0) 0x7feabc09d310 con 0x7feabc09f7d0 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 >> v1:172.21.15.45:6793/0 conn(0x7feabc09fba0 legacy=0x7feabc131930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 >> v1:172.21.15.45:6789/0 conn(0x7feabc09ff70 legacy=0x7feabc149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 monclient: found mon.b 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:18.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc120b20 con 0x7feabc09f7d0 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 10 monclient: _finish_auth 0 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.558+0000 7feaa3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feac24dd640 5 monclient: authenticate success, global_id 234128 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feac24dd640 10 monclient: _renew_subs 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feac24dd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feac24dd640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feabc121a80 con 0x7feabc09f7d0 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feac24dd640 10 monclient: _renew_subs 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feac24dd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feac24dd640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc120650 con 0x7feabc09f7d0 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feaac004a50 con 0x7feabc09f7d0 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feaa3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feaac004f00 con 0x7feabc09f7d0 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feaa3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feaba7fc640 4 set_mon_vals no callback set 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feaa3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feaa3fff640 10 monclient: dump: 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feac24dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feac24dd640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fea9c006580 con 0x7fea9c002830 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.559+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7feaac00fed0 con 0x7feabc09f7d0 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7feaac046ae0 con 0x7feabc09f7d0 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706289441 0 0) 0x7fea9c006580 con 0x7fea9c002830 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 10 monclient(hunting): my global_id is 185217 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feabc09d310 con 0x7fea9c002830 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feac0a7a640 10 monclient: get_auth_request con 0x7feaa803abb0 auth_method 0 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3870677426 0 0) 0x7feabc09d310 con 0x7fea9c002830 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fea9c006580 con 0x7fea9c002830 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feab00011d0 con 0x7fea9c002830 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 10 monclient: dump: 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.585 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 496351732 0 0) 0x7fea9c006580 con 0x7fea9c002830 2024-09-18T14:19:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea9c00a0a0 con 0x7fea9c002830 2024-09-18T14:19:18.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7feab00018a0 con 0x7fea9c002830 2024-09-18T14:19:18.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:18.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:18.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 >> v1:172.21.15.45:6792/0 conn(0x7fea9c002830 legacy=0x7fea9c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.560+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 >> v1:172.21.15.45:6792/0 conn(0x7fea9c002830 legacy=0x7fea9c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.566+0000 7feac24dd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:19:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.566+0000 7feac24dd640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fea9c0074f0 con 0x7fea9c0068a0 2024-09-18T14:19:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.566+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981905393 0 0) 0x7fea9c0074f0 con 0x7fea9c0068a0 2024-09-18T14:19:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 10 monclient(hunting): my global_id is 185226 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea9c006580 con 0x7fea9c0068a0 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1243857473 0 0) 0x7fea9c006580 con 0x7fea9c0068a0 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fea9c0074f0 con 0x7fea9c0068a0 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7feab00057f0 con 0x7fea9c0068a0 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 10 monclient: dump: 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 947049881 0 0) 0x7fea9c0074f0 con 0x7fea9c0068a0 2024-09-18T14:19:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 --> v1:172.21.15.45:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fea9c008750 con 0x7fea9c0068a0 2024-09-18T14:19:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 <== mon.6 v1:172.21.15.45:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 275002466) 0x7feab0008090 con 0x7fea9c0068a0 2024-09-18T14:19:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:19:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 >> v1:172.21.15.45:6792/0 conn(0x7fea9c0068a0 legacy=0x7fea9c006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.567+0000 7feaa3fff640 1 -- 172.21.15.45:0/1878478679 >> v1:172.21.15.45:6792/0 conn(0x7fea9c0068a0 legacy=0x7fea9c006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.592 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:18.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 156, 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:19:18.593 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 3, 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.594 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.595 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.596 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:19:18.596 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:19:18.596 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:19:18.596 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:19:18.596 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:19:18.596 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:19:18.596 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:18.596 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:18.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:19:18.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:19:18.596 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.597 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.598 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.599 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.601 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.601 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:19:18.601 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:19:18.601 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.601 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.601 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:19:18.601 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.601 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.602 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.604 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:19:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:19:18.606 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.607 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.609 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.610 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:18.611 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.612 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.612 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:19:18.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.612 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.612 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.612 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:18.612 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.612 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.612 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:19:18.612 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.568+0000 7feac24dd640 1 -- 172.21.15.45:0/1878478679 >> v1:172.21.15.84:6812/31478 conn(0x7feaa803abb0 legacy=0x7feaa803d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.568+0000 7feac24dd640 10 monclient: shutdown 2024-09-18T14:19:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.568+0000 7feac24dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.568+0000 7feac24dd640 1 -- 172.21.15.45:0/1878478679 >> v1:172.21.15.45:6790/0 conn(0x7feabc09f7d0 legacy=0x7feabc141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.568+0000 7feac0a7a640 1 -- 172.21.15.45:0/1878478679 reap_dead start 2024-09-18T14:19:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.568+0000 7feac0a7a640 1 -- 172.21.15.45:0/1878478679 reap_dead start 2024-09-18T14:19:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.568+0000 7feac24dd640 1 -- 172.21.15.45:0/1878478679 shutdown_connections 2024-09-18T14:19:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.568+0000 7feac24dd640 1 -- 172.21.15.45:0/1878478679 >> 172.21.15.45:0/1878478679 conn(0x7feabc0630b0 msgr2=0x7feabc0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.568+0000 7feac24dd640 1 -- 172.21.15.45:0/1878478679 shutdown_connections 2024-09-18T14:19:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.568+0000 7feac24dd640 1 -- 172.21.15.45:0/1878478679 wait complete. 2024-09-18T14:19:18.615 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:19:18.615 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-18T14:19:18.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-18T14:19:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.711+0000 7fdccab9d640 10 monclient: get_monmap_and_config 2024-09-18T14:19:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.711+0000 7fdccab9d640 10 monclient: build_initial_monmap 2024-09-18T14:19:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.712+0000 7fdccab9d640 10 monclient: monmap: 2024-09-18T14:19:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:18.712950+0000 2024-09-18T14:19:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:18.712950+0000 2024-09-18T14:19:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:18.714 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.712+0000 7fdccab9d640 1 Processor -- start 2024-09-18T14:19:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.712+0000 7fdccab9d640 1 -- start start 2024-09-18T14:19:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.712+0000 7fdccab9d640 10 monclient: init 2024-09-18T14:19:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdccab9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdccab9d640 10 monclient: _add_conns ranks=[6,8,5,4,0,1,7,3,2] 2024-09-18T14:19:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdccab9d640 10 monclient(hunting): picked mon.noname-d con 0x7fdcc4098300 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdccab9d640 10 monclient(hunting): picked mon.noname-e con 0x7fdcc40986d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdccab9d640 10 monclient(hunting): picked mon.noname-h con 0x7fdcc4098b00 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdccab9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc405ed70 con 0x7fdcc4098b00 2024-09-18T14:19:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdccab9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc4133160 con 0x7fdcc4098300 2024-09-18T14:19:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdccab9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc40965e0 con 0x7fdcc40986d0 2024-09-18T14:19:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdccab9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdccab9d640 10 monclient(hunting): authenticate will time out at 1974.051383s 2024-09-18T14:19:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc8939640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fdcc4098300 0x7fdcc4132a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56362/0 (socket says 172.21.15.45:56362) 2024-09-18T14:19:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc8939640 1 -- 172.21.15.45:0/2494137051 learned_addr learned my addr 172.21.15.45:0/2494137051 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc3fff640 1 --1- 172.21.15.45:0/2494137051 >> v1:172.21.15.45:6793/0 conn(0x7fdcc40986d0 0x7fdcc4137770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34746/0 (socket says 172.21.15.45:34746) 2024-09-18T14:19:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358791202 0 0) 0x7fdcc40965e0 con 0x7fdcc40986d0 2024-09-18T14:19:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 10 monclient(hunting): my global_id is 234125 2024-09-18T14:19:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcac0034e0 con 0x7fdcc40986d0 2024-09-18T14:19:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337355650 0 0) 0x7fdcc4133160 con 0x7fdcc4098300 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 10 monclient(hunting): my global_id is 185235 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc40965e0 con 0x7fdcc4098300 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471334514 0 0) 0x7fdcc405ed70 con 0x7fdcc4098b00 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 10 monclient(hunting): my global_id is 185072 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc4133160 con 0x7fdcc4098b00 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.713+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3276243601 0 0) 0x7fdcac0034e0 con 0x7fdcc40986d0 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 >> v1:172.21.15.45:6792/0 conn(0x7fdcc4098300 legacy=0x7fdcc4132a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 >> v1:172.21.15.84:6791/0 conn(0x7fdcc4098b00 legacy=0x7fdcc413aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 10 monclient: _finish_hunting 0 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 1 monclient: found mon.noname-e 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc40968c0 con 0x7fdcc40986d0 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 10 monclient: _finish_auth 0 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdccab9d640 5 monclient: authenticate success, global_id 234125 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdcb8001830 con 0x7fdcc40986d0 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdccab9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 10 monclient: dump: 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdccab9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdcb8001a10 con 0x7fdcc40986d0 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdccab9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 1 -- 172.21.15.45:0/2494137051 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdcb8004ae0 con 0x7fdcc40986d0 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdccab9d640 4 set_mon_vals no callback set 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdcc37fe640 10 monclient: dump: 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdccab9d640 10 monclient: shutdown 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdccab9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdccab9d640 1 -- 172.21.15.45:0/2494137051 >> v1:172.21.15.45:6793/0 conn(0x7fdcc40986d0 legacy=0x7fdcc4137770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdccab9d640 1 -- 172.21.15.45:0/2494137051 shutdown_connections 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdccab9d640 1 -- 172.21.15.45:0/2494137051 >> 172.21.15.45:0/2494137051 conn(0x7fdcc4067b20 msgr2=0x7fdcc40a2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdccab9d640 1 -- 172.21.15.45:0/2494137051 shutdown_connections 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.714+0000 7fdccab9d640 1 -- 172.21.15.45:0/2494137051 wait complete. 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.715+0000 7fdccab9d640 10 monclient: build_initial_monmap 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.715+0000 7fdccab9d640 10 monclient: monmap: 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.715+0000 7fdccab9d640 1 Processor -- start 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.715+0000 7fdccab9d640 1 -- start start 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.715+0000 7fdccab9d640 10 monclient: init 2024-09-18T14:19:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 10 monclient: _add_conns ranks=[6,2,5,7,0,3,4,1,8] 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 10 monclient(hunting): picked mon.noname-d con 0x7fdcc4098300 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 10 monclient(hunting): picked mon.noname-b con 0x7fdcc40986d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 10 monclient(hunting): picked mon.noname-h con 0x7fdcc4098b00 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc412f7f0 con 0x7fdcc40986d0 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc412fdb0 con 0x7fdcc4098b00 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc412fad0 con 0x7fdcc4098300 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 10 monclient(hunting): authenticate will time out at 1974.054148s 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc8939640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fdcc4098300 0x7fdcc4141800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56372/0 (socket says 172.21.15.45:56372) 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc8939640 1 -- 172.21.15.45:0/3142567020 learned_addr learned my addr 172.21.15.45:0/3142567020 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424599105 0 0) 0x7fdcc412fad0 con 0x7fdcc4098300 2024-09-18T14:19:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): my global_id is 185244 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb00034e0 con 0x7fdcc4098300 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516446956 0 0) 0x7fdcc412f7f0 con 0x7fdcc40986d0 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): my global_id is 234137 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc412fad0 con 0x7fdcc40986d0 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687969591 0 0) 0x7fdcc412fdb0 con 0x7fdcc4098b00 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): my global_id is 185081 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc412f7f0 con 0x7fdcc4098b00 2024-09-18T14:19:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3208045791 0 0) 0x7fdcb00034e0 con 0x7fdcc4098300 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdcc412fdb0 con 0x7fdcc4098300 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdcb4001ce0 con 0x7fdcc4098300 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): dump: 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.734 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3619899378 0 0) 0x7fdcc412f7f0 con 0x7fdcc4098b00 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdcb00034e0 con 0x7fdcc4098b00 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdcbc001d80 con 0x7fdcc4098b00 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:19:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient(hunting): dump: 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 14265679 0 0) 0x7fdcc412fad0 con 0x7fdcc40986d0 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdcc412f7f0 con 0x7fdcc40986d0 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1966080352 0 0) 0x7fdcc412fdb0 con 0x7fdcc4098300 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 >> v1:172.21.15.84:6791/0 conn(0x7fdcc4098b00 legacy=0x7fdcc414a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 >> v1:172.21.15.45:6790/0 conn(0x7fdcc40986d0 legacy=0x7fdcc41469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient: _finish_hunting 0 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 monclient: found mon.d 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc4130a10 con 0x7fdcc4098300 2024-09-18T14:19:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 10 monclient: _finish_auth 0 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdcc0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 5 monclient: authenticate success, global_id 185244 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 10 monclient: _renew_subs 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.716+0000 7fdccab9d640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc412e210 con 0x7fdcc4098300 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdccab9d640 10 monclient: _renew_subs 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdccab9d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdccab9d640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc412d460 con 0x7fdcc4098300 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdcb4004a50 con 0x7fdcc4098300 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdcc0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdcb4004f00 con 0x7fdcc4098300 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdcc0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdcc0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:18.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdcc2ffd640 4 set_mon_vals no callback set 2024-09-18T14:19:18.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdcc0ff9640 10 monclient: dump: 2024-09-18T14:19:18.739 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.739 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.739 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.739 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.739 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.739 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.739 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.739 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.739 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fdcb400fed0 con 0x7fdcc4098300 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdccab9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.717+0000 7fdccab9d640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdca4006580 con 0x7fdca4002830 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7fdcb4046940 con 0x7fdcc4098300 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc3fff640 10 monclient: get_auth_request con 0x7fdcb003a990 auth_method 0 2024-09-18T14:19:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155818507 0 0) 0x7fdca4006580 con 0x7fdca4002830 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 10 monclient(hunting): my global_id is 234134 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc412fdb0 con 0x7fdca4002830 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 778184412 0 0) 0x7fdcc412fdb0 con 0x7fdca4002830 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdca4006580 con 0x7fdca4002830 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdcbc002510 con 0x7fdca4002830 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 10 monclient: dump: 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 379433219 0 0) 0x7fdca4006580 con 0x7fdca4002830 2024-09-18T14:19:18.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdca400a0a0 con 0x7fdca4002830 2024-09-18T14:19:18.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fdcbc0028b0 con 0x7fdca4002830 2024-09-18T14:19:18.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:18.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:18.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 >> v1:172.21.15.45:6793/0 conn(0x7fdca4002830 legacy=0x7fdca4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.718+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 >> v1:172.21.15.45:6793/0 conn(0x7fdca4002830 legacy=0x7fdca4004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.724+0000 7fdccab9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:19:18.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.725+0000 7fdccab9d640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdca4009660 con 0x7fdca4009040 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.729+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919846881 0 0) 0x7fdca4009660 con 0x7fdca4009040 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.730+0000 7fdcc0ff9640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.730+0000 7fdcc0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.730+0000 7fdcc0ff9640 10 monclient(hunting): my global_id is 234143 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.730+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdca4006580 con 0x7fdca4009040 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.730+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1504861013 0 0) 0x7fdca4006580 con 0x7fdca4009040 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.730+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdca4009660 con 0x7fdca4009040 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.730+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fdcbc002fd0 con 0x7fdca4009040 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.730+0000 7fdcc0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.730+0000 7fdcc0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.730+0000 7fdcc0ff9640 10 monclient: dump: 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.746 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.736+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 758985064 0 0) 0x7fdca4009660 con 0x7fdca4009040 2024-09-18T14:19:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.736+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 --> v1:172.21.15.45:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdca4009920 con 0x7fdca4009040 2024-09-18T14:19:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.736+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 <== mon.8 v1:172.21.15.45:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 963104058) 0x7fdcbc0062a0 con 0x7fdca4009040 2024-09-18T14:19:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.736+0000 7fdcc0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:19:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.736+0000 7fdcc0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.736+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 >> v1:172.21.15.45:6793/0 conn(0x7fdca4009040 legacy=0x7fdca4008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.737+0000 7fdcc0ff9640 1 -- 172.21.15.45:0/3142567020 >> v1:172.21.15.45:6793/0 conn(0x7fdca4009040 legacy=0x7fdca4008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.747 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:19:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 156, 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:19:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:19:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:19:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:19:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:19:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:19:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:19:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:19:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:19:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:19:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:19:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:19:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:19:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:19:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:19:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:19:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:19:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:19:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.767 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:19:18.767 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.737+0000 7fdccab9d640 1 -- 172.21.15.45:0/3142567020 >> v1:172.21.15.84:6812/31478 conn(0x7fdcb003a990 legacy=0x7fdcb003ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.737+0000 7fdccab9d640 10 monclient: shutdown 2024-09-18T14:19:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.737+0000 7fdccab9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.737+0000 7fdccab9d640 1 -- 172.21.15.45:0/3142567020 >> v1:172.21.15.45:6792/0 conn(0x7fdcc4098300 legacy=0x7fdcc4141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.737+0000 7fdcc913a640 1 -- 172.21.15.45:0/3142567020 reap_dead start 2024-09-18T14:19:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.737+0000 7fdcc913a640 1 -- 172.21.15.45:0/3142567020 reap_dead start 2024-09-18T14:19:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.737+0000 7fdccab9d640 1 -- 172.21.15.45:0/3142567020 shutdown_connections 2024-09-18T14:19:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.737+0000 7fdccab9d640 1 -- 172.21.15.45:0/3142567020 >> 172.21.15.45:0/3142567020 conn(0x7fdcc4067b20 msgr2=0x7fdcc4096e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.737+0000 7fdccab9d640 1 -- 172.21.15.45:0/3142567020 shutdown_connections 2024-09-18T14:19:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.737+0000 7fdccab9d640 1 -- 172.21.15.45:0/3142567020 wait complete. 2024-09-18T14:19:18.769 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:19:18.769 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-18T14:19:18.769 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-18T14:19:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.865+0000 7fc61e154640 10 monclient: get_monmap_and_config 2024-09-18T14:19:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.866+0000 7fc61e154640 10 monclient: build_initial_monmap 2024-09-18T14:19:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.866+0000 7fc61e154640 10 monclient: monmap: 2024-09-18T14:19:18.866 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:18.866969+0000 2024-09-18T14:19:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:18.866969+0000 2024-09-18T14:19:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:18.867 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:18.868 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.866+0000 7fc61e154640 1 Processor -- start 2024-09-18T14:19:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.866+0000 7fc61e154640 1 -- start start 2024-09-18T14:19:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.866+0000 7fc61e154640 10 monclient: init 2024-09-18T14:19:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc61e154640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc61e154640 10 monclient: _add_conns ranks=[2,8,6,1,0,4,3,5,7] 2024-09-18T14:19:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc61e154640 10 monclient(hunting): picked mon.noname-b con 0x7fc61809f7d0 addr v1:172.21.15.45:6790/0 2024-09-18T14:19:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc61e154640 10 monclient(hunting): picked mon.noname-e con 0x7fc61809fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc61e154640 10 monclient(hunting): picked mon.noname-d con 0x7fc61809ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc61e154640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61805ed70 con 0x7fc61809f7d0 2024-09-18T14:19:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc61e154640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61809dfc0 con 0x7fc61809ff70 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc61e154640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61809dce0 con 0x7fc61809fba0 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc61e154640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc61e154640 10 monclient(hunting): authenticate will time out at 1974.205319s 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc60effd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fc61809fba0 0x7fc618137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34770/0 (socket says 172.21.15.45:34770) 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc6177fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fc61809f7d0 0x7fc618132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39772/0 (socket says 172.21.15.45:39772) 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc60effd640 1 -- 172.21.15.45:0/2139757672 learned_addr learned my addr 172.21.15.45:0/2139757672 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc617fff640 1 --1- 172.21.15.45:0/2139757672 >> v1:172.21.15.45:6792/0 conn(0x7fc61809ff70 0x7fc61813b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56374/0 (socket says 172.21.15.45:56374) 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873697952 0 0) 0x7fc61805ed70 con 0x7fc61809f7d0 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 10 monclient(hunting): my global_id is 234146 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6040034e0 con 0x7fc61809f7d0 2024-09-18T14:19:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925776124 0 0) 0x7fc61809dfc0 con 0x7fc61809ff70 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 10 monclient(hunting): my global_id is 185253 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc61805ed70 con 0x7fc61809ff70 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339801443 0 0) 0x7fc61809dce0 con 0x7fc61809fba0 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 10 monclient(hunting): my global_id is 234152 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc61809dfc0 con 0x7fc61809fba0 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.867+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 754400189 0 0) 0x7fc6040034e0 con 0x7fc61809f7d0 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 >> v1:172.21.15.45:6793/0 conn(0x7fc61809fba0 legacy=0x7fc618137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 >> v1:172.21.15.45:6792/0 conn(0x7fc61809ff70 legacy=0x7fc61813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 1 monclient: found mon.noname-b 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6180a1c00 con 0x7fc61809f7d0 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 10 monclient: _finish_auth 0 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc608001830 con 0x7fc61809f7d0 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc61e154640 5 monclient: authenticate success, global_id 234146 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc61e154640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 10 monclient: dump: 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc61e154640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc608001990 con 0x7fc61809f7d0 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 1 -- 172.21.15.45:0/2139757672 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc608004ae0 con 0x7fc61809f7d0 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc61e154640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc61e154640 4 set_mon_vals no callback set 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc616ffd640 10 monclient: dump: 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc61e154640 10 monclient: shutdown 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc61e154640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc61e154640 1 -- 172.21.15.45:0/2139757672 >> v1:172.21.15.45:6790/0 conn(0x7fc61809f7d0 legacy=0x7fc618132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc61e154640 1 -- 172.21.15.45:0/2139757672 shutdown_connections 2024-09-18T14:19:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc61e154640 1 -- 172.21.15.45:0/2139757672 >> 172.21.15.45:0/2139757672 conn(0x7fc6180630b0 msgr2=0x7fc6180634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc61e154640 1 -- 172.21.15.45:0/2139757672 shutdown_connections 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.868+0000 7fc61e154640 1 -- 172.21.15.45:0/2139757672 wait complete. 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.869+0000 7fc61e154640 10 monclient: build_initial_monmap 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.869+0000 7fc61e154640 10 monclient: monmap: 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.869+0000 7fc61e154640 1 Processor -- start 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.869+0000 7fc61e154640 1 -- start start 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.869+0000 7fc61e154640 10 monclient: init 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc61e154640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc61e154640 10 monclient: _add_conns ranks=[1,7,3,8,5,2,4,6,0] 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc61e154640 10 monclient(hunting): picked mon.noname-f con 0x7fc61809f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc61e154640 10 monclient(hunting): picked mon.noname-i con 0x7fc61809fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc61e154640 10 monclient(hunting): picked mon.noname-g con 0x7fc61809ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc61e154640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61809d310 con 0x7fc61809f7d0 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc61e154640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6180a2560 con 0x7fc61809ff70 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc61e154640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc618120610 con 0x7fc61809fba0 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc61e154640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc61e154640 10 monclient(hunting): authenticate will time out at 1974.208287s 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc617fff640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7fc61809ff70 0x7fc618149c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:43536/0 (socket says 172.21.15.45:43536) 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc6177fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc61809f7d0 0x7fc61811d1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:40508/0 (socket says 172.21.15.45:40508) 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60effd640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7fc61809fba0 0x7fc61811d8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:59876/0 (socket says 172.21.15.45:59876) 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc617fff640 1 -- 172.21.15.45:0/1845108692 learned_addr learned my addr 172.21.15.45:0/1845108692 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465551344 0 0) 0x7fc61809d310 con 0x7fc61809f7d0 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 10 monclient(hunting): my global_id is 214678 2024-09-18T14:19:18.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5f80034e0 con 0x7fc61809f7d0 2024-09-18T14:19:18.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305708802 0 0) 0x7fc618120610 con 0x7fc61809fba0 2024-09-18T14:19:18.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 10 monclient(hunting): my global_id is 214639 2024-09-18T14:19:18.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc61809d310 con 0x7fc61809fba0 2024-09-18T14:19:18.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371634584 0 0) 0x7fc6180a2560 con 0x7fc61809ff70 2024-09-18T14:19:18.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 10 monclient(hunting): my global_id is 224373 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.870+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc618120610 con 0x7fc61809ff70 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3052836572 0 0) 0x7fc5f80034e0 con 0x7fc61809f7d0 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc6180a2560 con 0x7fc61809f7d0 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc608004b20 con 0x7fc61809f7d0 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient(hunting): dump: 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.887 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.888 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2200081717 0 0) 0x7fc61809d310 con 0x7fc61809fba0 2024-09-18T14:19:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5f80034e0 con 0x7fc61809fba0 2024-09-18T14:19:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3773708530 0 0) 0x7fc618120610 con 0x7fc61809ff70 2024-09-18T14:19:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc61809d310 con 0x7fc61809ff70 2024-09-18T14:19:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc600001320 con 0x7fc61809fba0 2024-09-18T14:19:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:19:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient(hunting): dump: 2024-09-18T14:19:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.889 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc610001d00 con 0x7fc61809ff70 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient(hunting): dump: 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.891 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.892 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.892 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.892 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.892 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.892 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3764280798 0 0) 0x7fc6180a2560 con 0x7fc61809f7d0 2024-09-18T14:19:18.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 >> v1:172.21.15.84:6792/0 conn(0x7fc61809fba0 legacy=0x7fc61811d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 >> v1:172.21.15.84:6790/0 conn(0x7fc61809ff70 legacy=0x7fc618149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:18.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 monclient: found mon.f 2024-09-18T14:19:18.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:19:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc618120330 con 0x7fc61809f7d0 2024-09-18T14:19:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient: _finish_auth 0 2024-09-18T14:19:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc61e154640 5 monclient: authenticate success, global_id 214678 2024-09-18T14:19:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc61e154640 10 monclient: _renew_subs 2024-09-18T14:19:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc61e154640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:19:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc61e154640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61811e860 con 0x7fc61809f7d0 2024-09-18T14:19:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc61e154640 10 monclient: _renew_subs 2024-09-18T14:19:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc61e154640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.84:6789/0 2024-09-18T14:19:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc61e154640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61811f3d0 con 0x7fc61809f7d0 2024-09-18T14:19:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc608004d60 con 0x7fc61809f7d0 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc608001250 con 0x7fc61809f7d0 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc6167fc640 4 set_mon_vals no callback set 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.871+0000 7fc60ffff640 10 monclient: dump: 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc61e154640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc61e154640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc5fc006580 con 0x7fc5fc002830 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc608010080 con 0x7fc61809f7d0 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7fc608046b40 con 0x7fc61809f7d0 2024-09-18T14:19:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095491487 0 0) 0x7fc5fc006580 con 0x7fc5fc002830 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 10 monclient(hunting): my global_id is 214687 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6180a2560 con 0x7fc5fc002830 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc617fff640 10 monclient: get_auth_request con 0x7fc5f803ab70 auth_method 0 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2813973548 0 0) 0x7fc6180a2560 con 0x7fc5fc002830 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5fc006580 con 0x7fc5fc002830 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc600004ae0 con 0x7fc5fc002830 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.872+0000 7fc60ffff640 10 monclient: dump: 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.873+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3305533577 0 0) 0x7fc5fc006580 con 0x7fc5fc002830 2024-09-18T14:19:18.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.873+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5fc00a0a0 con 0x7fc5fc002830 2024-09-18T14:19:18.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.873+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc600006fc0 con 0x7fc5fc002830 2024-09-18T14:19:18.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.873+0000 7fc60ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:18.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.873+0000 7fc60ffff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:18.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.873+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 >> v1:172.21.15.84:6789/0 conn(0x7fc5fc002830 legacy=0x7fc5fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.873+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 >> v1:172.21.15.84:6789/0 conn(0x7fc5fc002830 legacy=0x7fc5fc004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc61e154640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc61e154640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc5fc0095f0 con 0x7fc5fc008ff0 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443469826 0 0) 0x7fc5fc0095f0 con 0x7fc5fc008ff0 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc60ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc60ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc60ffff640 10 monclient(hunting): my global_id is 214696 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5fc006580 con 0x7fc5fc008ff0 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1042783826 0 0) 0x7fc5fc006580 con 0x7fc5fc008ff0 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5fc0095f0 con 0x7fc5fc008ff0 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc600005630 con 0x7fc5fc008ff0 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc60ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc60ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:19:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.879+0000 7fc60ffff640 10 monclient: dump: 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:18.901 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:18.902 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2094294657 0 0) 0x7fc5fc0095f0 con 0x7fc5fc008ff0 2024-09-18T14:19:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 --> v1:172.21.15.84:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc5fc0098b0 con 0x7fc5fc008ff0 2024-09-18T14:19:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 <== mon.1 v1:172.21.15.84:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 4022724833) 0x7fc600006a20 con 0x7fc5fc008ff0 2024-09-18T14:19:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc60ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:19:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc60ffff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 >> v1:172.21.15.84:6789/0 conn(0x7fc5fc008ff0 legacy=0x7fc5fc008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc60ffff640 1 -- 172.21.15.45:0/1845108692 >> v1:172.21.15.84:6789/0 conn(0x7fc5fc008ff0 legacy=0x7fc5fc008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:18.902 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 156, 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:19:18.903 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.904 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.905 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:19:18.906 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:18.907 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:18.908 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:18.908 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.908 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:19:18.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.908 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:19:18.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.908 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:19:18.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:19:18.908 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.908 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:19:18.909 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.910 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.911 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.911 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:19:18.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:19:18.911 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.911 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.911 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:19:18.911 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.912 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:19:18.914 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:19:18.915 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:19:18.916 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:19:18.918 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:19:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:19:18.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:19:18.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:18.922 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:18.922 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:18.922 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:19:18.922 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:18.922 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:18.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:18.922 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:19:18.922 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:18.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc61e154640 1 -- 172.21.15.45:0/1845108692 >> v1:172.21.15.84:6812/31478 conn(0x7fc5f803ab70 legacy=0x7fc5f803d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc61e154640 10 monclient: shutdown 2024-09-18T14:19:18.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc61e154640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:18.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc61e154640 1 -- 172.21.15.45:0/1845108692 >> v1:172.21.15.84:6789/0 conn(0x7fc61809f7d0 legacy=0x7fc61811d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:18.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc617fff640 1 -- 172.21.15.45:0/1845108692 reap_dead start 2024-09-18T14:19:18.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc617fff640 1 -- 172.21.15.45:0/1845108692 reap_dead start 2024-09-18T14:19:18.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc61e154640 1 -- 172.21.15.45:0/1845108692 shutdown_connections 2024-09-18T14:19:18.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc61e154640 1 -- 172.21.15.45:0/1845108692 >> 172.21.15.45:0/1845108692 conn(0x7fc6180630b0 msgr2=0x7fc6180a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:18.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc61e154640 1 -- 172.21.15.45:0/1845108692 shutdown_connections 2024-09-18T14:19:18.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:18.880+0000 7fc61e154640 1 -- 172.21.15.45:0/1845108692 wait complete. 2024-09-18T14:19:18.924 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:19:18.925 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-18T14:19:18.925 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-18T14:19:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.022+0000 7f86ea40f640 10 monclient: get_monmap_and_config 2024-09-18T14:19:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.022+0000 7f86ea40f640 10 monclient: build_initial_monmap 2024-09-18T14:19:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.022+0000 7f86ea40f640 10 monclient: monmap: 2024-09-18T14:19:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:19.023412+0000 2024-09-18T14:19:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:19.023412+0000 2024-09-18T14:19:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:19.024 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:19.024 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 1 Processor -- start 2024-09-18T14:19:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 1 -- start start 2024-09-18T14:19:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 10 monclient: init 2024-09-18T14:19:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 10 monclient: _add_conns ranks=[7,8,0,6,1,3,4,2,5] 2024-09-18T14:19:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 10 monclient(hunting): picked mon.noname-i con 0x7f86e409f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 10 monclient(hunting): picked mon.noname-e con 0x7f86e409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 10 monclient(hunting): picked mon.noname-a con 0x7f86e409ff70 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e405ed70 con 0x7f86e409ff70 2024-09-18T14:19:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e409dfc0 con 0x7f86e409f7d0 2024-09-18T14:19:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e409dce0 con 0x7f86e409fba0 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86ea40f640 10 monclient(hunting): authenticate will time out at 1974.361921s 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86e89ac640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f86e409ff70 0x7f86e413b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58894/0 (socket says 172.21.15.45:58894) 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86e89ac640 1 -- 172.21.15.45:0/4115262623 learned_addr learned my addr 172.21.15.45:0/4115262623 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.023+0000 7f86e37fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f86e409fba0 0x7f86e4137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34784/0 (socket says 172.21.15.45:34784) 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955846611 0 0) 0x7f86e409dce0 con 0x7f86e409fba0 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient(hunting): my global_id is 234161 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d00034e0 con 0x7f86e409fba0 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973324433 0 0) 0x7f86e405ed70 con 0x7f86e409ff70 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient(hunting): my global_id is 6696 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e409dce0 con 0x7f86e409ff70 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724752003 0 0) 0x7f86e409dfc0 con 0x7f86e409f7d0 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient(hunting): my global_id is 214648 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e405ed70 con 0x7f86e409f7d0 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2532871965 0 0) 0x7f86d00034e0 con 0x7f86e409fba0 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 >> v1:172.21.15.84:6792/0 conn(0x7f86e409f7d0 legacy=0x7f86e4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 >> v1:172.21.15.45:6789/0 conn(0x7f86e409ff70 legacy=0x7f86e413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 monclient: found mon.noname-e 2024-09-18T14:19:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e40a1c00 con 0x7f86e409fba0 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient: _finish_auth 0 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86ea40f640 5 monclient: authenticate success, global_id 234161 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86ea40f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86cc001830 con 0x7f86e409fba0 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient: dump: 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86ea40f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:19.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f86cc001990 con 0x7f86e409fba0 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 1 -- 172.21.15.45:0/4115262623 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86cc004ae0 con 0x7f86e409fba0 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86ea40f640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86ea40f640 4 set_mon_vals no callback set 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.024+0000 7f86e2ffd640 10 monclient: dump: 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.038 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.025+0000 7f86ea40f640 10 monclient: shutdown 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.025+0000 7f86ea40f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.025+0000 7f86ea40f640 1 -- 172.21.15.45:0/4115262623 >> v1:172.21.15.45:6793/0 conn(0x7f86e409fba0 legacy=0x7f86e4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.025+0000 7f86ea40f640 1 -- 172.21.15.45:0/4115262623 shutdown_connections 2024-09-18T14:19:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.025+0000 7f86ea40f640 1 -- 172.21.15.45:0/4115262623 >> 172.21.15.45:0/4115262623 conn(0x7f86e40630b0 msgr2=0x7f86e40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.025+0000 7f86ea40f640 1 -- 172.21.15.45:0/4115262623 shutdown_connections 2024-09-18T14:19:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.025+0000 7f86ea40f640 1 -- 172.21.15.45:0/4115262623 wait complete. 2024-09-18T14:19:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.025+0000 7f86ea40f640 10 monclient: build_initial_monmap 2024-09-18T14:19:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.025+0000 7f86ea40f640 10 monclient: monmap: 2024-09-18T14:19:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:19.040 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 1 Processor -- start 2024-09-18T14:19:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 1 -- start start 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 10 monclient: init 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 10 monclient: _add_conns ranks=[5,4,6,3,1,2,7,0,8] 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 10 monclient(hunting): picked mon.noname-h con 0x7f86e409f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 10 monclient(hunting): picked mon.noname-c con 0x7f86e409fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 10 monclient(hunting): picked mon.noname-d con 0x7f86e409ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e409d310 con 0x7f86e409fba0 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e40a2560 con 0x7f86e409f7d0 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e4122520 con 0x7f86e409ff70 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86ea40f640 10 monclient(hunting): authenticate will time out at 1974.364809s 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86e37fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f86e409fba0 0x7f86e411f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36922/0 (socket says 172.21.15.45:36922) 2024-09-18T14:19:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86e37fe640 1 -- 172.21.15.45:0/3560274306 learned_addr learned my addr 172.21.15.45:0/3560274306 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.026+0000 7f86e89ac640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f86e409ff70 0x7f86e414a620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56376/0 (socket says 172.21.15.45:56376) 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710546761 0 0) 0x7f86e409d310 con 0x7f86e409fba0 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): my global_id is 165379 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c40034e0 con 0x7f86e409fba0 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341452024 0 0) 0x7f86e4122520 con 0x7f86e409ff70 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): my global_id is 185262 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e409d310 con 0x7f86e409ff70 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 133002441 0 0) 0x7f86e409d310 con 0x7f86e409ff70 2024-09-18T14:19:19.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86e4122520 con 0x7f86e409ff70 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726089838 0 0) 0x7f86e40a2560 con 0x7f86e409f7d0 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): my global_id is 185090 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e409d310 con 0x7f86e409f7d0 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4122400797 0 0) 0x7f86c40034e0 con 0x7f86e409fba0 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86e40a2560 con 0x7f86e409fba0 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86dc001c30 con 0x7f86e409ff70 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): dump: 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.044 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.045 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86cc004ae0 con 0x7f86e409fba0 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient(hunting): dump: 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2310247562 0 0) 0x7f86e4122520 con 0x7f86e409ff70 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 >> v1:172.21.15.84:6791/0 conn(0x7f86e409f7d0 legacy=0x7f86e411e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 >> v1:172.21.15.45:6791/0 conn(0x7f86e409fba0 legacy=0x7f86e411f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 monclient: found mon.d 2024-09-18T14:19:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e411f7f0 con 0x7f86e409ff70 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient: _finish_auth 0 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86ea40f640 5 monclient: authenticate success, global_id 185262 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86ea40f640 10 monclient: _renew_subs 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86ea40f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86ea40f640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e4120430 con 0x7f86e409ff70 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86ea40f640 10 monclient: _renew_subs 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86ea40f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86ea40f640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e4131400 con 0x7f86e409ff70 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f86dc001d90 con 0x7f86e409ff70 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86dc004f60 con 0x7f86e409ff70 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86e27fc640 4 set_mon_vals no callback set 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.027+0000 7f86dbfff640 10 monclient: dump: 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.028+0000 7f86ea40f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.028+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f86dc00ffb0 con 0x7f86e409ff70 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.028+0000 7f86ea40f640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f86c8006580 con 0x7f86c8002830 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.028+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f86dc046ad0 con 0x7f86e409ff70 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86e37fe640 10 monclient: get_auth_request con 0x7f86c403a5a0 auth_method 0 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140791555 0 0) 0x7f86c8006580 con 0x7f86c8002830 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 10 monclient(hunting): my global_id is 224382 2024-09-18T14:19:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e4122520 con 0x7f86c8002830 2024-09-18T14:19:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2147344969 0 0) 0x7f86e4122520 con 0x7f86c8002830 2024-09-18T14:19:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86c8006580 con 0x7f86c8002830 2024-09-18T14:19:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86d40019a0 con 0x7f86c8002830 2024-09-18T14:19:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 10 monclient: dump: 2024-09-18T14:19:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.053 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 731791142 0 0) 0x7f86c8006580 con 0x7f86c8002830 2024-09-18T14:19:19.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.84:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86c800a0a0 con 0x7f86c8002830 2024-09-18T14:19:19.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f86d4001f50 con 0x7f86c8002830 2024-09-18T14:19:19.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:19.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:19.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 >> v1:172.21.15.84:6790/0 conn(0x7f86c8002830 legacy=0x7f86c8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.029+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 >> v1:172.21.15.84:6790/0 conn(0x7f86c8002830 legacy=0x7f86c8004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.035+0000 7f86ea40f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:19:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.035+0000 7f86ea40f640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f86c80074f0 con 0x7f86c80068a0 2024-09-18T14:19:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402819897 0 0) 0x7f86c80074f0 con 0x7f86c80068a0 2024-09-18T14:19:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 10 monclient(hunting): my global_id is 224391 2024-09-18T14:19:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c8006580 con 0x7f86c80068a0 2024-09-18T14:19:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3703151707 0 0) 0x7f86c8006580 con 0x7f86c80068a0 2024-09-18T14:19:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86c80074f0 con 0x7f86c80068a0 2024-09-18T14:19:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86d40027b0 con 0x7f86c80068a0 2024-09-18T14:19:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 10 monclient: dump: 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.057 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2483179786 0 0) 0x7f86c80074f0 con 0x7f86c80068a0 2024-09-18T14:19:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 --> v1:172.21.15.84:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f86c8008750 con 0x7f86c80068a0 2024-09-18T14:19:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 <== mon.3 v1:172.21.15.84:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 2325860392) 0x7f86d4002990 con 0x7f86c80068a0 2024-09-18T14:19:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:19:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 >> v1:172.21.15.84:6790/0 conn(0x7f86c80068a0 legacy=0x7f86c8006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.036+0000 7f86dbfff640 1 -- 172.21.15.45:0/3560274306 >> v1:172.21.15.84:6790/0 conn(0x7f86c80068a0 legacy=0x7f86c8006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:19.058 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:19:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:19:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:19:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 156, 2024-09-18T14:19:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:19:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:19:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:19:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:19:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:19:19.058 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:19.059 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:19.060 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:19:19.061 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:19.062 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:19:19.063 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.064 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.065 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.067 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:19:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:19:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:19:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:19:19.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.071 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.072 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.074 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.076 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.078 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:19:19.078 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:19.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.037+0000 7f86ea40f640 1 -- 172.21.15.45:0/3560274306 >> v1:172.21.15.84:6812/31478 conn(0x7f86c403a5a0 legacy=0x7f86c403ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.037+0000 7f86ea40f640 10 monclient: shutdown 2024-09-18T14:19:19.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.037+0000 7f86ea40f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:19.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.037+0000 7f86ea40f640 1 -- 172.21.15.45:0/3560274306 >> v1:172.21.15.45:6792/0 conn(0x7f86e409ff70 legacy=0x7f86e414a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.037+0000 7f86e89ac640 1 -- 172.21.15.45:0/3560274306 reap_dead start 2024-09-18T14:19:19.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.037+0000 7f86e89ac640 1 -- 172.21.15.45:0/3560274306 reap_dead start 2024-09-18T14:19:19.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.037+0000 7f86ea40f640 1 -- 172.21.15.45:0/3560274306 shutdown_connections 2024-09-18T14:19:19.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.037+0000 7f86ea40f640 1 -- 172.21.15.45:0/3560274306 >> 172.21.15.45:0/3560274306 conn(0x7f86e40630b0 msgr2=0x7f86e40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.037+0000 7f86ea40f640 1 -- 172.21.15.45:0/3560274306 shutdown_connections 2024-09-18T14:19:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.037+0000 7f86ea40f640 1 -- 172.21.15.45:0/3560274306 wait complete. 2024-09-18T14:19:19.080 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:19:19.080 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-18T14:19:19.080 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-18T14:19:19.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.177+0000 7f64358e8640 10 monclient: get_monmap_and_config 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.177+0000 7f64358e8640 10 monclient: build_initial_monmap 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.177+0000 7f64358e8640 10 monclient: monmap: 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:19.178161+0000 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:19.178161+0000 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:19.179 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.177+0000 7f64358e8640 1 Processor -- start 2024-09-18T14:19:19.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.177+0000 7f64358e8640 1 -- start start 2024-09-18T14:19:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.177+0000 7f64358e8640 10 monclient: init 2024-09-18T14:19:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f64358e8640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f64358e8640 10 monclient: _add_conns ranks=[2,4,8,7,6,5,1,0,3] 2024-09-18T14:19:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f64358e8640 10 monclient(hunting): picked mon.noname-b con 0x7f643006e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:19:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f64358e8640 10 monclient(hunting): picked mon.noname-c con 0x7f64300a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f64358e8640 10 monclient(hunting): picked mon.noname-e con 0x7f64300a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f64358e8640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643005ed70 con 0x7f643006e810 2024-09-18T14:19:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f64358e8640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6430097620 con 0x7f64300a0cb0 2024-09-18T14:19:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f64358e8640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6430097340 con 0x7f64300a1080 2024-09-18T14:19:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f64358e8640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f64358e8640 10 monclient(hunting): authenticate will time out at 1974.516385s 2024-09-18T14:19:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642effd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f643006e810 0x7f64300a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39782/0 (socket says 172.21.15.45:39782) 2024-09-18T14:19:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f64267fc640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f64300a0cb0 0x7f6430137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36928/0 (socket says 172.21.15.45:36928) 2024-09-18T14:19:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642effd640 1 -- 172.21.15.45:0/1700942541 learned_addr learned my addr 172.21.15.45:0/1700942541 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642f7fe640 1 --1- 172.21.15.45:0/1700942541 >> v1:172.21.15.45:6793/0 conn(0x7f64300a1080 0x7f643013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34800/0 (socket says 172.21.15.45:34800) 2024-09-18T14:19:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973094859 0 0) 0x7f6430097340 con 0x7f64300a1080 2024-09-18T14:19:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 10 monclient(hunting): my global_id is 234170 2024-09-18T14:19:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f641c0034e0 con 0x7f64300a1080 2024-09-18T14:19:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154158666 0 0) 0x7f643005ed70 con 0x7f643006e810 2024-09-18T14:19:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 10 monclient(hunting): my global_id is 234155 2024-09-18T14:19:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6430097340 con 0x7f643006e810 2024-09-18T14:19:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887537831 0 0) 0x7f6430097620 con 0x7f64300a0cb0 2024-09-18T14:19:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 10 monclient(hunting): my global_id is 165388 2024-09-18T14:19:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.178+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643005ed70 con 0x7f64300a0cb0 2024-09-18T14:19:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4021859773 0 0) 0x7f641c0034e0 con 0x7f64300a1080 2024-09-18T14:19:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 >> v1:172.21.15.45:6791/0 conn(0x7f64300a0cb0 legacy=0x7f6430137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 >> v1:172.21.15.45:6790/0 conn(0x7f643006e810 legacy=0x7f64300a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 1 monclient: found mon.noname-e 2024-09-18T14:19:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:19:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300a1660 con 0x7f64300a1080 2024-09-18T14:19:19.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6428001bb0 con 0x7f64300a1080 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 10 monclient: dump: 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f64358e8640 5 monclient: authenticate success, global_id 234170 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f64358e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6428001d10 con 0x7f64300a1080 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 1 -- 172.21.15.45:0/1700942541 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6428004b20 con 0x7f64300a1080 2024-09-18T14:19:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f64358e8640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f64358e8640 4 set_mon_vals no callback set 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f642e7fc640 10 monclient: dump: 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f64358e8640 10 monclient: shutdown 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f64358e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f64358e8640 1 -- 172.21.15.45:0/1700942541 >> v1:172.21.15.45:6793/0 conn(0x7f64300a1080 legacy=0x7f643013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f64358e8640 1 -- 172.21.15.45:0/1700942541 shutdown_connections 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f64358e8640 1 -- 172.21.15.45:0/1700942541 >> 172.21.15.45:0/1700942541 conn(0x7f64300a5bb0 msgr2=0x7f64300a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f64358e8640 1 -- 172.21.15.45:0/1700942541 shutdown_connections 2024-09-18T14:19:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.179+0000 7f64358e8640 1 -- 172.21.15.45:0/1700942541 wait complete. 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.180+0000 7f64358e8640 10 monclient: build_initial_monmap 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.180+0000 7f64358e8640 10 monclient: monmap: 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.180+0000 7f64358e8640 1 Processor -- start 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.180+0000 7f64358e8640 1 -- start start 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.180+0000 7f64358e8640 10 monclient: init 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64358e8640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64358e8640 10 monclient: _add_conns ranks=[3,0,2,7,6,5,4,8,1] 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64358e8640 10 monclient(hunting): picked mon.noname-g con 0x7f643006e810 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64358e8640 10 monclient(hunting): picked mon.noname-a con 0x7f64300a0cb0 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64358e8640 10 monclient(hunting): picked mon.noname-b con 0x7f64300a1080 addr v1:172.21.15.45:6790/0 2024-09-18T14:19:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64358e8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6430122c70 con 0x7f64300a0cb0 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64358e8640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643013da60 con 0x7f64300a1080 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64358e8640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64301226b0 con 0x7f643006e810 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64358e8640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64358e8640 10 monclient(hunting): authenticate will time out at 1974.519337s 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f642f7fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f64300a1080 0x7f6430149cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39796/0 (socket says 172.21.15.45:39796) 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f642f7fe640 1 -- 172.21.15.45:0/416421961 learned_addr learned my addr 172.21.15.45:0/416421961 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373472037 0 0) 0x7f643013da60 con 0x7f64300a1080 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): my global_id is 234164 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64100034e0 con 0x7f64300a1080 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268046957 0 0) 0x7f6430122c70 con 0x7f64300a0cb0 2024-09-18T14:19:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): my global_id is 6705 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643013da60 con 0x7f64300a0cb0 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 906937046 0 0) 0x7f643013da60 con 0x7f64300a0cb0 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6430122c70 con 0x7f64300a0cb0 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f64180013e0 con 0x7f64300a0cb0 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): dump: 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2866547535 0 0) 0x7f64100034e0 con 0x7f64300a1080 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f643013da60 con 0x7f64300a1080 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397865181 0 0) 0x7f64301226b0 con 0x7f643006e810 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): my global_id is 224400 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64100034e0 con 0x7f643006e810 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6428001a70 con 0x7f64300a1080 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.181+0000 7f64277fe640 10 monclient(hunting): dump: 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.203 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1570008010 0 0) 0x7f6430122c70 con 0x7f64300a0cb0 2024-09-18T14:19:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 >> v1:172.21.15.84:6790/0 conn(0x7f643006e810 legacy=0x7f643013d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 >> v1:172.21.15.45:6790/0 conn(0x7f64300a1080 legacy=0x7f6430149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 10 monclient: _finish_hunting 0 2024-09-18T14:19:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 1 monclient: found mon.a 2024-09-18T14:19:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64301201c0 con 0x7f64300a0cb0 2024-09-18T14:19:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 10 monclient: _finish_auth 0 2024-09-18T14:19:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64358e8640 5 monclient: authenticate success, global_id 6705 2024-09-18T14:19:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64358e8640 10 monclient: _renew_subs 2024-09-18T14:19:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64358e8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64358e8640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64301223d0 con 0x7f64300a0cb0 2024-09-18T14:19:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64358e8640 10 monclient: _renew_subs 2024-09-18T14:19:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64358e8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64358e8640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6430121500 con 0x7f64300a0cb0 2024-09-18T14:19:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f64180016e0 con 0x7f64300a0cb0 2024-09-18T14:19:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6418004e10 con 0x7f64300a0cb0 2024-09-18T14:19:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f642dffb640 4 set_mon_vals no callback set 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 10 monclient: dump: 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.206 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.182+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f641800fe60 con 0x7f64300a0cb0 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64358e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64358e8640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6414006580 con 0x7f6414002830 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f6418046970 con 0x7f64300a0cb0 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f642f7fe640 10 monclient: get_auth_request con 0x7f641003a5d0 auth_method 0 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372158471 0 0) 0x7f6414006580 con 0x7f6414002830 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64277fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64277fe640 10 monclient(hunting): my global_id is 185099 2024-09-18T14:19:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6430122c70 con 0x7f6414002830 2024-09-18T14:19:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1835822071 0 0) 0x7f6430122c70 con 0x7f6414002830 2024-09-18T14:19:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6414006580 con 0x7f6414002830 2024-09-18T14:19:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6420001550 con 0x7f6414002830 2024-09-18T14:19:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64277fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:19:19.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.183+0000 7f64277fe640 10 monclient: dump: 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.210 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.210 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.210 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.184+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 248705401 0 0) 0x7f6414006580 con 0x7f6414002830 2024-09-18T14:19:19.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.184+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.84:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f641400a0a0 con 0x7f6414002830 2024-09-18T14:19:19.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.184+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f64200047a0 con 0x7f6414002830 2024-09-18T14:19:19.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.184+0000 7f64277fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:19.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.184+0000 7f64277fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:19.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.184+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 >> v1:172.21.15.84:6791/0 conn(0x7f6414002830 legacy=0x7f6414004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.184+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 >> v1:172.21.15.84:6791/0 conn(0x7f6414002830 legacy=0x7f6414004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:19.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64358e8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:19:19.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64358e8640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6414009660 con 0x7f6414009040 2024-09-18T14:19:19.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1546742703 0 0) 0x7f6414009660 con 0x7f6414009040 2024-09-18T14:19:19.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64277fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64277fe640 10 monclient(hunting): my global_id is 185108 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6414006580 con 0x7f6414009040 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3267966966 0 0) 0x7f6414006580 con 0x7f6414009040 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6414009660 con 0x7f6414009040 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6420004f60 con 0x7f6414009040 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64277fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.190+0000 7f64277fe640 10 monclient: dump: 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.5 v1:172.21.15.84:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2989466096 0 0) 0x7f6414009660 con 0x7f6414009040 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 --> v1:172.21.15.84:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6414009920 con 0x7f6414009040 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 <== mon.5 v1:172.21.15.84:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 3714128759) 0x7f6420005200 con 0x7f6414009040 2024-09-18T14:19:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64277fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:19:19.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64277fe640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:19.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 >> v1:172.21.15.84:6791/0 conn(0x7f6414009040 legacy=0x7f6414008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64277fe640 1 -- 172.21.15.45:0/416421961 >> v1:172.21.15.84:6791/0 conn(0x7f6414009040 legacy=0x7f6414008720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:19.215 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:19:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:19:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:19:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 156, 2024-09-18T14:19:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:19:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:19:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:19:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:19:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:19:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:19:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:19:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:19:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": [ 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:19.235 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:19.235 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.235 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.235 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:19:19.235 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64358e8640 1 -- 172.21.15.45:0/416421961 >> v1:172.21.15.84:6812/31478 conn(0x7f641003a5d0 legacy=0x7f641003ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64358e8640 10 monclient: shutdown 2024-09-18T14:19:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64358e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64358e8640 1 -- 172.21.15.45:0/416421961 >> v1:172.21.15.45:6789/0 conn(0x7f64300a0cb0 legacy=0x7f6430146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f642f7fe640 1 -- 172.21.15.45:0/416421961 reap_dead start 2024-09-18T14:19:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f642f7fe640 1 -- 172.21.15.45:0/416421961 reap_dead start 2024-09-18T14:19:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64358e8640 1 -- 172.21.15.45:0/416421961 shutdown_connections 2024-09-18T14:19:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64358e8640 1 -- 172.21.15.45:0/416421961 >> 172.21.15.45:0/416421961 conn(0x7f64300a5bb0 msgr2=0x7f64300a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64358e8640 1 -- 172.21.15.45:0/416421961 shutdown_connections 2024-09-18T14:19:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.191+0000 7f64358e8640 1 -- 172.21.15.45:0/416421961 wait complete. 2024-09-18T14:19:19.237 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:19:19.237 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-18T14:19:19.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-18T14:19:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.334+0000 7f2f4399d640 10 monclient: get_monmap_and_config 2024-09-18T14:19:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.334+0000 7f2f4399d640 10 monclient: build_initial_monmap 2024-09-18T14:19:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.334+0000 7f2f4399d640 10 monclient: monmap: 2024-09-18T14:19:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:19.335034+0000 2024-09-18T14:19:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:19.335034+0000 2024-09-18T14:19:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.335 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:19.336 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.334+0000 7f2f4399d640 1 Processor -- start 2024-09-18T14:19:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.334+0000 7f2f4399d640 1 -- start start 2024-09-18T14:19:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.334+0000 7f2f4399d640 10 monclient: init 2024-09-18T14:19:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f4399d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f4399d640 10 monclient: _add_conns ranks=[3,8,6,7,2,4,0,1,5] 2024-09-18T14:19:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f4399d640 10 monclient(hunting): picked mon.noname-g con 0x7f2f3c06e810 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f4399d640 10 monclient(hunting): picked mon.noname-e con 0x7f2f3c0a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f4399d640 10 monclient(hunting): picked mon.noname-d con 0x7f2f3c0a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f4399d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c05ed70 con 0x7f2f3c06e810 2024-09-18T14:19:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f4399d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c097620 con 0x7f2f3c0a1080 2024-09-18T14:19:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f4399d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c097340 con 0x7f2f3c0a0cb0 2024-09-18T14:19:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f4399d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f4399d640 10 monclient(hunting): authenticate will time out at 1974.673376s 2024-09-18T14:19:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f41f3a640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f2f3c0a1080 0x7f2f3c13af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56390/0 (socket says 172.21.15.45:56390) 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f40f38640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f2f3c0a0cb0 0x7f2f3c137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34810/0 (socket says 172.21.15.45:34810) 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f41f3a640 1 -- 172.21.15.45:0/1726927057 learned_addr learned my addr 172.21.15.45:0/1726927057 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022924152 0 0) 0x7f2f3c097620 con 0x7f2f3c0a1080 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 10 monclient(hunting): my global_id is 185271 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f280034e0 con 0x7f2f3c0a1080 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452589317 0 0) 0x7f2f3c097340 con 0x7f2f3c0a0cb0 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 10 monclient(hunting): my global_id is 234179 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f3c097620 con 0x7f2f3c0a0cb0 2024-09-18T14:19:19.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519363346 0 0) 0x7f2f3c05ed70 con 0x7f2f3c06e810 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 10 monclient(hunting): my global_id is 224409 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.335+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f3c097340 con 0x7f2f3c06e810 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 576703595 0 0) 0x7f2f280034e0 con 0x7f2f3c0a1080 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 >> v1:172.21.15.45:6793/0 conn(0x7f2f3c0a0cb0 legacy=0x7f2f3c137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 >> v1:172.21.15.84:6790/0 conn(0x7f2f3c06e810 legacy=0x7f2f3c0a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 1 monclient: found mon.noname-d 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c0a1660 con 0x7f2f3c0a1080 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 10 monclient: _finish_auth 0 2024-09-18T14:19:19.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f34001bb0 con 0x7f2f3c0a1080 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f4399d640 5 monclient: authenticate success, global_id 185271 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f4399d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 10 monclient: dump: 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f4399d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f34001d90 con 0x7f2f3c0a1080 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f4399d640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 1 -- 172.21.15.45:0/1726927057 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f34004f20 con 0x7f2f3c0a1080 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f4399d640 4 set_mon_vals no callback set 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f2ffff640 10 monclient: dump: 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f4399d640 10 monclient: shutdown 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f4399d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f4399d640 1 -- 172.21.15.45:0/1726927057 >> v1:172.21.15.45:6792/0 conn(0x7f2f3c0a1080 legacy=0x7f2f3c13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f4399d640 1 -- 172.21.15.45:0/1726927057 shutdown_connections 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f4399d640 1 -- 172.21.15.45:0/1726927057 >> 172.21.15.45:0/1726927057 conn(0x7f2f3c0a5bb0 msgr2=0x7f2f3c0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f4399d640 1 -- 172.21.15.45:0/1726927057 shutdown_connections 2024-09-18T14:19:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.336+0000 7f2f4399d640 1 -- 172.21.15.45:0/1726927057 wait complete. 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.337+0000 7f2f4399d640 10 monclient: build_initial_monmap 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.337+0000 7f2f4399d640 10 monclient: monmap: 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:19.351 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.337+0000 7f2f4399d640 1 Processor -- start 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.337+0000 7f2f4399d640 1 -- start start 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.337+0000 7f2f4399d640 10 monclient: init 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f4399d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f4399d640 10 monclient: _add_conns ranks=[8,6,3,4,5,1,2,7,0] 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f4399d640 10 monclient(hunting): picked mon.noname-e con 0x7f2f3c06e810 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f4399d640 10 monclient(hunting): picked mon.noname-d con 0x7f2f3c0a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f4399d640 10 monclient(hunting): picked mon.noname-g con 0x7f2f3c0a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f4399d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c122da0 con 0x7f2f3c0a1080 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f4399d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c13db90 con 0x7f2f3c0a0cb0 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f4399d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c1227e0 con 0x7f2f3c06e810 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f4399d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f4399d640 10 monclient(hunting): authenticate will time out at 1974.676310s 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f41739640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f2f3c06e810 0x7f2f3c13d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34824/0 (socket says 172.21.15.45:34824) 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f41739640 1 -- 172.21.15.45:0/2244097972 learned_addr learned my addr 172.21.15.45:0/2244097972 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689801530 0 0) 0x7f2f3c1227e0 con 0x7f2f3c06e810 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): my global_id is 234188 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f200034e0 con 0x7f2f3c06e810 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180130678 0 0) 0x7f2f3c13db90 con 0x7f2f3c0a0cb0 2024-09-18T14:19:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): my global_id is 185280 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f3c1227e0 con 0x7f2f3c0a0cb0 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334408691 0 0) 0x7f2f3c122da0 con 0x7f2f3c0a1080 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): my global_id is 224418 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f3c13db90 con 0x7f2f3c0a1080 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3071012517 0 0) 0x7f2f200034e0 con 0x7f2f3c06e810 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f3c122da0 con 0x7f2f3c06e810 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f38002bf0 con 0x7f2f3c06e810 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): dump: 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3081618822 0 0) 0x7f2f3c1227e0 con 0x7f2f3c0a0cb0 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.338+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f200034e0 con 0x7f2f3c0a0cb0 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f30001550 con 0x7f2f3c0a0cb0 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 10 monclient(hunting): dump: 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1777205233 0 0) 0x7f2f3c122da0 con 0x7f2f3c06e810 2024-09-18T14:19:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 >> v1:172.21.15.45:6792/0 conn(0x7f2f3c0a0cb0 legacy=0x7f2f3c1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 >> v1:172.21.15.84:6790/0 conn(0x7f2f3c0a1080 legacy=0x7f2f3c149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 10 monclient: _finish_hunting 0 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 1 monclient: found mon.e 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c120330 con 0x7f2f3c06e810 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 10 monclient: _finish_auth 0 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f4399d640 5 monclient: authenticate success, global_id 234188 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f4399d640 10 monclient: _renew_subs 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f4399d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f4399d640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c122500 con 0x7f2f3c06e810 2024-09-18T14:19:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f4399d640 10 monclient: _renew_subs 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f4399d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f4399d640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c121630 con 0x7f2f3c06e810 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f38001760 con 0x7f2f3c06e810 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f38001c50 con 0x7f2f3c06e810 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2d7fa640 10 monclient: dump: 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.339+0000 7f2f2f7fe640 4 set_mon_vals no callback set 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f4399d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2f38001db0 con 0x7f2f3c06e810 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f4399d640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2f1c006580 con 0x7f2f1c002830 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f2f38046950 con 0x7f2f3c06e810 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3753693517 0 0) 0x7f2f1c006580 con 0x7f2f1c002830 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f41f3a640 10 monclient: get_auth_request con 0x7f2f2003a5a0 auth_method 0 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 10 monclient(hunting): my global_id is 214657 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f3c122da0 con 0x7f2f1c002830 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1933783122 0 0) 0x7f2f3c122da0 con 0x7f2f1c002830 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f1c006580 con 0x7f2f1c002830 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f30004ae0 con 0x7f2f1c002830 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 10 monclient: dump: 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.363 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2654438500 0 0) 0x7f2f1c006580 con 0x7f2f1c002830 2024-09-18T14:19:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.340+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.84:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f1c00a0a0 con 0x7f2f1c002830 2024-09-18T14:19:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.341+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2f30005090 con 0x7f2f1c002830 2024-09-18T14:19:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.341+0000 7f2f2d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.341+0000 7f2f2d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.341+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 >> v1:172.21.15.84:6792/0 conn(0x7f2f1c002830 legacy=0x7f2f1c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.341+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 >> v1:172.21.15.84:6792/0 conn(0x7f2f1c002830 legacy=0x7f2f1c004cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:19.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f4399d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-18T14:19:19.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f4399d640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2f1c0074f0 con 0x7f2f1c0068a0 2024-09-18T14:19:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492775712 0 0) 0x7f2f1c0074f0 con 0x7f2f1c0068a0 2024-09-18T14:19:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 10 monclient(hunting): my global_id is 214666 2024-09-18T14:19:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f1c006580 con 0x7f2f1c0068a0 2024-09-18T14:19:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 176722437 0 0) 0x7f2f1c006580 con 0x7f2f1c0068a0 2024-09-18T14:19:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f1c0074f0 con 0x7f2f1c0068a0 2024-09-18T14:19:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2f30005970 con 0x7f2f1c0068a0 2024-09-18T14:19:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:19:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 10 monclient: dump: 2024-09-18T14:19:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.367 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3291411205 0 0) 0x7f2f1c0074f0 con 0x7f2f1c0068a0 2024-09-18T14:19:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.347+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 --> v1:172.21.15.84:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2f1c008750 con 0x7f2f1c0068a0 2024-09-18T14:19:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 <== mon.7 v1:172.21.15.84:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 1885511832) 0x7f2f30005b50 con 0x7f2f1c0068a0 2024-09-18T14:19:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f2d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-18T14:19:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f2d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 >> v1:172.21.15.84:6792/0 conn(0x7f2f1c0068a0 legacy=0x7f2f1c006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f2d7fa640 1 -- 172.21.15.45:0/2244097972 >> v1:172.21.15.84:6792/0 conn(0x7f2f1c0068a0 legacy=0x7f2f1c006d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:19.367 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:19.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "peon", 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: "election_epoch": 156, 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum": [ 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: 1, 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: 3, 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: 4, 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: 6, 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: 7, 2024-09-18T14:19:19.368 INFO:teuthology.orchestra.run.smithi045.stdout: 8 2024-09-18T14:19:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_age": 4, 2024-09-18T14:19:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "required_con": "2449958755906961412", 2024-09-18T14:19:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "required_mon": [ 2024-09-18T14:19:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:19.369 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_con": "4540138320759226367", 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "quorum_mon": [ 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:19.370 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "outside_quorum": [], 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "extra_probe_peers": [], 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "sync_provider": [], 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": { 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 1, 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "0f5ca4bb-86c8-4c54-b818-324eb902ab17", 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-18T13:58:22.742395Z", 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release": 17, 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min_mon_release_name": "quincy", 2024-09-18T14:19:19.371 INFO:teuthology.orchestra.run.smithi045.stdout: "election_strategy": 3, 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "disallowed_leaders: ": "", 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false, 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tiebreaker_mon": "", 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_ranks: ": "", 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "features": { 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "persistent": [ 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "kraken", 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "luminous", 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mimic", 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap-prune", 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "nautilus", 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus", 2024-09-18T14:19:19.372 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific", 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "elector-pinging", 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy" 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "optional": [] 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mons": [ 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 0, 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "a", 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.373 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789", 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6789/0", 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6789/0", 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 1, 2024-09-18T14:19:19.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "f", 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789", 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6789/0", 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6789/0", 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.375 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 2, 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "b", 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790", 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.376 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6790/0", 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6790/0", 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 3, 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "g", 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790", 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6790/0", 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6790/0", 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 4, 2024-09-18T14:19:19.378 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "c", 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791", 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6791/0", 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6791/0", 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.379 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 5, 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "h", 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791", 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.380 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6791/0", 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6791/0", 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 6, 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "d", 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792", 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6792/0", 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6792/0", 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.382 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 7, 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "i", 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792", 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.84:6792/0", 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.84:6792/0", 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.383 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "rank": 8, 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "e", 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793", 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 0 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6793/0", 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6793/0", 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "priority": 0, 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "{}" 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "feature_map": { 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": [ 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:19.385 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": [ 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 1 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "client": [ 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-18T14:19:19.386 INFO:teuthology.orchestra.run.smithi045.stdout: "release": "luminous", 2024-09-18T14:19:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 2 2024-09-18T14:19:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-18T14:19:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-18T14:19:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-18T14:19:19.387 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": false 2024-09-18T14:19:19.387 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f4399d640 1 -- 172.21.15.45:0/2244097972 >> v1:172.21.15.84:6812/31478 conn(0x7f2f2003a5a0 legacy=0x7f2f2003ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f4399d640 10 monclient: shutdown 2024-09-18T14:19:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f4399d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f4399d640 1 -- 172.21.15.45:0/2244097972 >> v1:172.21.15.45:6793/0 conn(0x7f2f3c06e810 legacy=0x7f2f3c13d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f41f3a640 1 -- 172.21.15.45:0/2244097972 reap_dead start 2024-09-18T14:19:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f41f3a640 1 -- 172.21.15.45:0/2244097972 reap_dead start 2024-09-18T14:19:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f4399d640 1 -- 172.21.15.45:0/2244097972 shutdown_connections 2024-09-18T14:19:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f4399d640 1 -- 172.21.15.45:0/2244097972 >> 172.21.15.45:0/2244097972 conn(0x7f2f3c0a5bb0 msgr2=0x7f2f3c0a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f4399d640 1 -- 172.21.15.45:0/2244097972 shutdown_connections 2024-09-18T14:19:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.348+0000 7f2f4399d640 1 -- 172.21.15.45:0/2244097972 wait complete. 2024-09-18T14:19:19.389 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-18T14:19:19.389 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-18T14:19:19.390 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-18T14:19:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.485+0000 7f6cf8300640 10 monclient: get_monmap_and_config 2024-09-18T14:19:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.485+0000 7f6cf8300640 10 monclient: build_initial_monmap 2024-09-18T14:19:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.485+0000 7f6cf8300640 10 monclient: monmap: 2024-09-18T14:19:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:19.486405+0000 2024-09-18T14:19:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:19.486405+0000 2024-09-18T14:19:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:19.487 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 1 Processor -- start 2024-09-18T14:19:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 1 -- start start 2024-09-18T14:19:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 10 monclient: init 2024-09-18T14:19:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 10 monclient: _add_conns ranks=[5,4,1,3,7,2,8,6,0] 2024-09-18T14:19:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 10 monclient(hunting): picked mon.noname-h con 0x7f6cf009f7d0 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 10 monclient(hunting): picked mon.noname-c con 0x7f6cf009fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 10 monclient(hunting): picked mon.noname-f con 0x7f6cf009ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf005ed70 con 0x7f6cf009ff70 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf009dfc0 con 0x7f6cf009fba0 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf009dce0 con 0x7f6cf009f7d0 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf8300640 10 monclient(hunting): authenticate will time out at 1974.824929s 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.486+0000 7f6cf589b640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f6cf009fba0 0x7f6cf0137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36944/0 (socket says 172.21.15.45:36944) 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf589b640 1 -- 172.21.15.45:0/1385907868 learned_addr learned my addr 172.21.15.45:0/1385907868 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667853790 0 0) 0x7f6cf009dfc0 con 0x7f6cf009fba0 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient(hunting): my global_id is 165397 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cdc0034e0 con 0x7f6cf009fba0 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94927174 0 0) 0x7f6cf005ed70 con 0x7f6cf009ff70 2024-09-18T14:19:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient(hunting): my global_id is 214705 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cf009dfc0 con 0x7f6cf009ff70 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062872303 0 0) 0x7f6cf009dce0 con 0x7f6cf009f7d0 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient(hunting): my global_id is 185117 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cf005ed70 con 0x7f6cf009f7d0 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1513991339 0 0) 0x7f6cdc0034e0 con 0x7f6cf009fba0 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 >> v1:172.21.15.84:6791/0 conn(0x7f6cf009f7d0 legacy=0x7f6cf0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 >> v1:172.21.15.84:6789/0 conn(0x7f6cf009ff70 legacy=0x7f6cf013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient: _finish_hunting 0 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 monclient: found mon.noname-c 2024-09-18T14:19:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf00a1c00 con 0x7f6cf009fba0 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient: _finish_auth 0 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6ce00017b0 con 0x7f6cf009fba0 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf8300640 5 monclient: authenticate success, global_id 165397 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf8300640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient: dump: 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf8300640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6ce0001910 con 0x7f6cf009fba0 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 1 -- 172.21.15.45:0/1385907868 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6ce0004ba0 con 0x7f6cf009fba0 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf509a640 10 monclient: dump: 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.501 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf8300640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.487+0000 7f6cf8300640 4 set_mon_vals no callback set 2024-09-18T14:19:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.488+0000 7f6cf8300640 10 monclient: shutdown 2024-09-18T14:19:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.488+0000 7f6cf8300640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.488+0000 7f6cf8300640 1 -- 172.21.15.45:0/1385907868 >> v1:172.21.15.45:6791/0 conn(0x7f6cf009fba0 legacy=0x7f6cf0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.488+0000 7f6cf8300640 1 -- 172.21.15.45:0/1385907868 shutdown_connections 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.488+0000 7f6cf8300640 1 -- 172.21.15.45:0/1385907868 >> 172.21.15.45:0/1385907868 conn(0x7f6cf00630b0 msgr2=0x7f6cf00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.488+0000 7f6cf8300640 1 -- 172.21.15.45:0/1385907868 shutdown_connections 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.488+0000 7f6cf8300640 1 -- 172.21.15.45:0/1385907868 wait complete. 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.488+0000 7f6cf8300640 10 monclient: build_initial_monmap 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.488+0000 7f6cf8300640 10 monclient: monmap: 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.488+0000 7f6cf8300640 1 Processor -- start 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf8300640 1 -- start start 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf8300640 10 monclient: init 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf8300640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf8300640 10 monclient: _add_conns ranks=[3,1,8,4,5,2,0,6,7] 2024-09-18T14:19:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf8300640 10 monclient(hunting): picked mon.noname-g con 0x7f6cf009f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf8300640 10 monclient(hunting): picked mon.noname-f con 0x7f6cf009fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf8300640 10 monclient(hunting): picked mon.noname-e con 0x7f6cf009ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf8300640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf009d310 con 0x7f6cf009fba0 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf8300640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf00a2560 con 0x7f6cf009f7d0 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf8300640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf011dba0 con 0x7f6cf009ff70 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf8300640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf8300640 10 monclient(hunting): authenticate will time out at 1974.827514s 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf689d640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f6cf009ff70 0x7f6cf014a6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34832/0 (socket says 172.21.15.45:34832) 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf689d640 1 -- 172.21.15.45:0/4007109181 learned_addr learned my addr 172.21.15.45:0/4007109181 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6cf609c640 1 --1- 172.21.15.45:0/4007109181 >> v1:172.21.15.84:6790/0 conn(0x7f6cf009f7d0 0x7f6cf01319e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:43576/0 (socket says 172.21.15.45:43576) 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414085821 0 0) 0x7f6cf011dba0 con 0x7f6cf009ff70 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 10 monclient(hunting): my global_id is 234197 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cd40034e0 con 0x7f6cf009ff70 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625380814 0 0) 0x7f6cf00a2560 con 0x7f6cf009f7d0 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 10 monclient(hunting): my global_id is 224427 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cf011dba0 con 0x7f6cf009f7d0 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980235596 0 0) 0x7f6cf009d310 con 0x7f6cf009fba0 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 10 monclient(hunting): my global_id is 214714 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.489+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cf00a2560 con 0x7f6cf009fba0 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 9378500 0 0) 0x7f6cd40034e0 con 0x7f6cf009ff70 2024-09-18T14:19:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6cf009d310 con 0x7f6cf009ff70 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6ce8002f00 con 0x7f6cf009ff70 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient(hunting): dump: 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.506 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3661912276 0 0) 0x7f6cf011dba0 con 0x7f6cf009f7d0 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6cd40034e0 con 0x7f6cf009f7d0 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6cec0019d0 con 0x7f6cf009f7d0 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient(hunting): dump: 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.507 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1347751332 0 0) 0x7f6cf00a2560 con 0x7f6cf009fba0 2024-09-18T14:19:19.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6cf011dba0 con 0x7f6cf009fba0 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2650522360 0 0) 0x7f6cf009d310 con 0x7f6cf009ff70 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 >> v1:172.21.15.84:6790/0 conn(0x7f6cf009f7d0 legacy=0x7f6cf01319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 >> v1:172.21.15.84:6789/0 conn(0x7f6cf009fba0 legacy=0x7f6cf0146f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 monclient: found mon.e 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf011eb00 con 0x7f6cf009ff70 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6cf8300640 5 monclient: authenticate success, global_id 234197 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6cf8300640 10 monclient: _renew_subs 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6cf8300640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6cf8300640 1 -- 172.21.15.45:0/4007109181 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf011ede0 con 0x7f6cf009ff70 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6cf8300640 10 monclient: _renew_subs 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6cf8300640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6cf8300640 1 -- 172.21.15.45:0/4007109181 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf011c960 con 0x7f6cf009ff70 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6ce8001990 con 0x7f6cf009ff70 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6cf4899640 4 set_mon_vals no callback set 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6ce8001f40 con 0x7f6cf009ff70 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.490+0000 7f6ce67fc640 10 monclient: dump: 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.491+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f6ce8002120 con 0x7f6cf009ff70 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.491+0000 7f6cf8300640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.491+0000 7f6cf8300640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.491+0000 7f6cf8300640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.491+0000 7f6cf8300640 1 -- 172.21.15.45:0/4007109181 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf004c4a0 con 0x7f6cf009ff70 2024-09-18T14:19:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.491+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f6ce8046d20 con 0x7f6cf009ff70 2024-09-18T14:19:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.491+0000 7f6cf609c640 10 monclient: get_auth_request con 0x7f6cd403a950 auth_method 0 2024-09-18T14:19:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.495+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f6ce80023d0 con 0x7f6cf009ff70 2024-09-18T14:19:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.495+0000 7f6ce67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.495+0000 7f6ce67fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.728+0000 7f6cf8300640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-18T14:19:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.728+0000 7f6cf8300640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-18T14:19:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.728+0000 7f6cf8300640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.728+0000 7f6cf8300640 1 -- 172.21.15.45:0/4007109181 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f6cf004b720 con 0x7f6cf009ff70 2024-09-18T14:19:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.730+0000 7f6ce67fc640 1 -- 172.21.15.45:0/4007109181 <== mon.8 v1:172.21.15.45:6793/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f6ce8015030 con 0x7f6cf009ff70 2024-09-18T14:19:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.730+0000 7f6ce67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-18T14:19:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.730+0000 7f6ce67fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:19.733 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.733 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=711970895,config=3140514009,health=103428445,kv=594742435,logm=1958402335}) 2024-09-18T14:19:19.733 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=29,config=2,health=12,kv=8,logm=49} crc {auth=1186938810,config=3140514009,health=1168478589,kv=594742435,logm=2185651726}) 2024-09-18T14:19:19.733 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.733 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=711970895,config=3140514009,health=103428445,kv=594742435,logm=1958402335}) 2024-09-18T14:19:19.733 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=29,config=2,health=11,kv=8,logm=50} crc {auth=1186938810,config=3140514009,health=3098005747,kv=594742435,logm=1773980283}) 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=711970895,config=3140514009,health=103428445,kv=594742435,logm=1958402335}) 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=28,config=2,health=12,kv=7,logm=51} crc {auth=3074433267,config=3140514009,health=2755639913,kv=210776046,logm=3193075495}) 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=711970895,config=3140514009,health=103428445,kv=594742435,logm=1958402335}) 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=28,health=11,kv=7,logm=54} crc {auth=2209914168,health=3159613034,kv=2464314623,logm=1641341330}) 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=711970895,config=3140514009,health=103428445,kv=594742435,logm=1958402335}) 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=29,config=2,health=10,kv=8,logm=51} crc {auth=1186938810,config=3140514009,health=3622972507,kv=594742435,logm=1217357817}) 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=711970895,config=3140514009,health=103428445,kv=594742435,logm=1958402335}) 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=28,config=1,health=12,kv=8,logm=51} crc {auth=2759168563,config=1228700967,health=103428445,kv=594742435,logm=1492287681}) 2024-09-18T14:19:19.734 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=711970895,config=3140514009,health=103428445,kv=594742435,logm=1958402335}) 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=26,config=2,health=12,kv=7,logm=53} crc {auth=1538989648,config=3140514009,health=103428445,kv=1989676319,logm=814029038}) 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=711970895,config=3140514009,health=103428445,kv=594742435,logm=1958402335}) 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.730+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=28,config=1,health=12,kv=7,logm=52} crc {auth=3810364372,config=1228700967,health=439555430,kv=2464314623,logm=102820215}) 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=780161099}) 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=981648046}) 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=780161099}) 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4003627907}) 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=780161099}) 2024-09-18T14:19:19.735 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1036252806}) 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=780161099}) 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1711224059}) 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=780161099}) 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1716090014}) 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=780161099}) 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1216736584}) 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=780161099}) 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3961129315}) 2024-09-18T14:19:19.736 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.737 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=780161099}) 2024-09-18T14:19:19.737 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.732+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1615891094}) 2024-09-18T14:19:19.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.733+0000 7f6cf8300640 1 -- 172.21.15.45:0/4007109181 >> v1:172.21.15.84:6812/31478 conn(0x7f6cd403a950 legacy=0x7f6cd403ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.733+0000 7f6cf8300640 10 monclient: shutdown 2024-09-18T14:19:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.733+0000 7f6cf8300640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.733+0000 7f6cf8300640 1 -- 172.21.15.45:0/4007109181 >> v1:172.21.15.45:6793/0 conn(0x7f6cf009ff70 legacy=0x7f6cf014a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.733+0000 7f6cf8300640 1 -- 172.21.15.45:0/4007109181 shutdown_connections 2024-09-18T14:19:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.733+0000 7f6cf8300640 1 -- 172.21.15.45:0/4007109181 >> 172.21.15.45:0/4007109181 conn(0x7f6cf00630b0 msgr2=0x7f6cf00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.733+0000 7f6cf8300640 1 -- 172.21.15.45:0/4007109181 shutdown_connections 2024-09-18T14:19:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:19.734+0000 7f6cf8300640 1 -- 172.21.15.45:0/4007109181 wait complete. 2024-09-18T14:19:19.740 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.740 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=737467138}) 2024-09-18T14:19:19.740 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043709932}) 2024-09-18T14:19:19.740 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.740 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=737467138}) 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4026207099}) 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=737467138}) 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2003590015}) 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=737467138}) 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2469057242}) 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.734+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.735+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=737467138}) 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.735+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=615301207}) 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.735+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.735+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=737467138}) 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.735+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=70194900}) 2024-09-18T14:19:19.741 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.735+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.735+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=737467138}) 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.735+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2829715404}) 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.735+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.735+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=737467138}) 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.735+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4104068043}) 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3874961758}) 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3487303811}) 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3874961758}) 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2742792802}) 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3874961758}) 2024-09-18T14:19:19.742 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2135020621}) 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3874961758}) 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3271466960}) 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3874961758}) 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=983971316}) 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3874961758}) 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2072361414}) 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3874961758}) 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3286922168}) 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3874961758}) 2024-09-18T14:19:19.743 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.736+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3315565566}) 2024-09-18T14:19:19.746 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.745+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.746 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.745+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729660524}) 2024-09-18T14:19:19.746 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.745+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3021525858}) 2024-09-18T14:19:19.746 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.746 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729660524}) 2024-09-18T14:19:19.746 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2943264767}) 2024-09-18T14:19:19.747 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.747 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729660524}) 2024-09-18T14:19:19.747 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3044106542}) 2024-09-18T14:19:19.747 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.747 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729660524}) 2024-09-18T14:19:19.747 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=438649964}) 2024-09-18T14:19:19.747 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.747 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729660524}) 2024-09-18T14:19:19.747 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1064527820}) 2024-09-18T14:19:19.747 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.748 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729660524}) 2024-09-18T14:19:19.748 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4170626899}) 2024-09-18T14:19:19.748 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.748 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729660524}) 2024-09-18T14:19:19.748 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3394987211}) 2024-09-18T14:19:19.748 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.748 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729660524}) 2024-09-18T14:19:19.748 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.746+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2529836214}) 2024-09-18T14:19:19.753 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-18T14:19:19.756 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.756 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024313961}) 2024-09-18T14:19:19.756 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2408443623}) 2024-09-18T14:19:19.756 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.756 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024313961}) 2024-09-18T14:19:19.756 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=11189577}) 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024313961}) 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2590248782}) 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024313961}) 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4001790150}) 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024313961}) 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1346119319}) 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024313961}) 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1930402713}) 2024-09-18T14:19:19.757 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.758 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024313961}) 2024-09-18T14:19:19.758 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1634205367}) 2024-09-18T14:19:19.758 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.758 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024313961}) 2024-09-18T14:19:19.758 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.752+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2319374689}) 2024-09-18T14:19:19.761 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.761 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123429005}) 2024-09-18T14:19:19.761 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3430353225}) 2024-09-18T14:19:19.761 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.761 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123429005}) 2024-09-18T14:19:19.761 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=669577617}) 2024-09-18T14:19:19.761 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.761 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123429005}) 2024-09-18T14:19:19.761 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1332216828}) 2024-09-18T14:19:19.761 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123429005}) 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1625588183}) 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123429005}) 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3632872538}) 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123429005}) 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2884919633}) 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123429005}) 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3181569352}) 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.762 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123429005}) 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.755+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4205332803}) 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=168375643}) 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4206591977}) 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=168375643}) 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3372185370}) 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=168375643}) 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=546437287}) 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=168375643}) 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1584628788}) 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.763 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=168375643}) 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273605317}) 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=168375643}) 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=264652908}) 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=168375643}) 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1278584630}) 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=168375643}) 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.758+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3914769638}) 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3258599455}) 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2549499415}) 2024-09-18T14:19:19.764 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.765 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3258599455}) 2024-09-18T14:19:19.765 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=58679508}) 2024-09-18T14:19:19.765 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.765 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3258599455}) 2024-09-18T14:19:19.765 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2135693319}) 2024-09-18T14:19:19.765 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.765 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3258599455}) 2024-09-18T14:19:19.965 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2167689398}) 2024-09-18T14:19:19.965 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.965 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3258599455}) 2024-09-18T14:19:19.966 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2265045946}) 2024-09-18T14:19:19.966 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.966 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3258599455}) 2024-09-18T14:19:19.966 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=284445453}) 2024-09-18T14:19:19.966 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.966 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3258599455}) 2024-09-18T14:19:19.966 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2716649865}) 2024-09-18T14:19:19.966 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.966 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3258599455}) 2024-09-18T14:19:19.966 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.760+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2718004384}) 2024-09-18T14:19:19.966 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.966 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3042528335}) 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=120548197}) 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3042528335}) 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3344429009}) 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3042528335}) 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3311428798}) 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3042528335}) 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3727495068}) 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.762+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3042528335}) 2024-09-18T14:19:19.967 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.763+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2096375761}) 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.763+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.763+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3042528335}) 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.763+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3920230519}) 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.763+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.763+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3042528335}) 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.763+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2744801311}) 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.763+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.763+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3042528335}) 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.763+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=802944703}) 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3230148465}) 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4237964706}) 2024-09-18T14:19:19.968 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3230148465}) 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2462599834}) 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3230148465}) 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1900921104}) 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3230148465}) 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3712771166}) 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3230148465}) 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=321682938}) 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.969 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3230148465}) 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1603478126}) 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3230148465}) 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3870198846}) 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3230148465}) 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.764+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=292454555}) 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.765+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.765+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986910743}) 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.765+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3797414152}) 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.765+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.765+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986910743}) 2024-09-18T14:19:19.970 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1982844991}) 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986910743}) 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2814854840}) 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986910743}) 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1239615232}) 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986910743}) 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2340995196}) 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986910743}) 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1423970139}) 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986910743}) 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3661369602}) 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986910743}) 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.766+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3346334643}) 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1039055674}) 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1322284900}) 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1039055674}) 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=496707384}) 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1039055674}) 2024-09-18T14:19:19.972 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1779873024}) 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1039055674}) 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=487057897}) 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1039055674}) 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3337631787}) 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1039055674}) 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4217186861}) 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1039055674}) 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3561873565}) 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.973 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1039055674}) 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.767+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2812346389}) 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=285773226}) 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3795741534}) 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=285773226}) 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1612069821}) 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=285773226}) 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1148657251}) 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=285773226}) 2024-09-18T14:19:19.974 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4097633856}) 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=285773226}) 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3467565417}) 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=285773226}) 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2521894760}) 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=285773226}) 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1132405416}) 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=285773226}) 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.768+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2972893470}) 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.975 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3154786000}) 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1010805434}) 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3154786000}) 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1858411432}) 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3154786000}) 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3559270351}) 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3154786000}) 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=687235690}) 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3154786000}) 2024-09-18T14:19:19.976 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=773538958}) 2024-09-18T14:19:19.977 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.977 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3154786000}) 2024-09-18T14:19:19.977 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4126182752}) 2024-09-18T14:19:19.977 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.977 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3154786000}) 2024-09-18T14:19:19.977 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3364226135}) 2024-09-18T14:19:19.977 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.977 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3154786000}) 2024-09-18T14:19:19.977 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.770+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3795312538}) 2024-09-18T14:19:19.977 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.977 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1069973175}) 2024-09-18T14:19:19.977 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1187446118}) 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1069973175}) 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1039055813}) 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1069973175}) 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3495642890}) 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1069973175}) 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2312983074}) 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1069973175}) 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3872120892}) 2024-09-18T14:19:19.978 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1069973175}) 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1958524816}) 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.771+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.772+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1069973175}) 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.772+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=346489063}) 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.772+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.772+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1069973175}) 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.772+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=768854649}) 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2299895283}) 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3378204308}) 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.979 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2299895283}) 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2745971304}) 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2299895283}) 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=397901237}) 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2299895283}) 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3043713736}) 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2299895283}) 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1502989178}) 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2299895283}) 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4080992008}) 2024-09-18T14:19:19.980 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.981 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2299895283}) 2024-09-18T14:19:19.981 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2598772756}) 2024-09-18T14:19:19.981 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.981 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2299895283}) 2024-09-18T14:19:19.981 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.773+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3010053748}) 2024-09-18T14:19:19.981 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.981 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197322337}) 2024-09-18T14:19:19.981 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=9011876}) 2024-09-18T14:19:19.981 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.981 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197322337}) 2024-09-18T14:19:19.981 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=483364633}) 2024-09-18T14:19:19.981 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197322337}) 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2926588548}) 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197322337}) 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1623009708}) 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197322337}) 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2793167368}) 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197322337}) 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1137757290}) 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.982 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197322337}) 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2191211251}) 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197322337}) 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.774+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2045135502}) 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2279241356}) 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4237024409}) 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2279241356}) 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=7705940}) 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2279241356}) 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4103886157}) 2024-09-18T14:19:19.983 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2279241356}) 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4119888476}) 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2279241356}) 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=484113256}) 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2279241356}) 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=716947693}) 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2279241356}) 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=139377121}) 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2279241356}) 2024-09-18T14:19:19.984 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.775+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4117081356}) 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=200852986}) 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2736894256}) 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=200852986}) 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3078678164}) 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=200852986}) 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=123156154}) 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=200852986}) 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=47428379}) 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=200852986}) 2024-09-18T14:19:19.985 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1096984125}) 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=200852986}) 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=951538709}) 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=200852986}) 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3006142077}) 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=200852986}) 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.776+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3228887386}) 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=444309550}) 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2695265824}) 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.986 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=444309550}) 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2904751584}) 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=444309550}) 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2978527633}) 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=444309550}) 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=197080259}) 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=444309550}) 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1063843027}) 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=444309550}) 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1761242146}) 2024-09-18T14:19:19.987 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=444309550}) 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=556629318}) 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=444309550}) 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.777+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3192603954}) 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=402728937}) 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1778753250}) 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=402728937}) 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3336090360}) 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.988 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=402728937}) 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=346132152}) 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=402728937}) 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1494993183}) 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=402728937}) 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4070238290}) 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=402728937}) 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2562252774}) 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=402728937}) 2024-09-18T14:19:19.989 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=163175552}) 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.778+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.779+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=402728937}) 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.779+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2820114823}) 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.779+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3221015389}) 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3825988121}) 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3221015389}) 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1838223302}) 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3221015389}) 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1356976740}) 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.990 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3221015389}) 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1757043889}) 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3221015389}) 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4066898329}) 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3221015389}) 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=14879052}) 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3221015389}) 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=739443415}) 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3221015389}) 2024-09-18T14:19:19.991 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.780+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2816519219}) 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2918367937}) 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=595942289}) 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2918367937}) 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3676626776}) 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2918367937}) 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2337996921}) 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2918367937}) 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2463938768}) 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.992 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2918367937}) 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1020922506}) 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2918367937}) 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1637237837}) 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2918367937}) 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2323595697}) 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2918367937}) 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.781+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3542648178}) 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=253208341}) 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=807918819}) 2024-09-18T14:19:19.993 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=253208341}) 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=767668430}) 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=253208341}) 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3604926806}) 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=253208341}) 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=386255877}) 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=253208341}) 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2887754782}) 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=253208341}) 2024-09-18T14:19:19.994 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1508529335}) 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=253208341}) 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2442225770}) 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=253208341}) 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.782+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2503183512}) 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3940367126}) 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3375841556}) 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3940367126}) 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1111636519}) 2024-09-18T14:19:19.995 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3940367126}) 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3265169462}) 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3940367126}) 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2823407516}) 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3940367126}) 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2329995967}) 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3940367126}) 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2202032828}) 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.996 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3940367126}) 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3591590639}) 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3940367126}) 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.783+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1215634585}) 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.784+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489291918}) 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1323410989}) 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489291918}) 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2655266348}) 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489291918}) 2024-09-18T14:19:19.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2285167897}) 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489291918}) 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4026571193}) 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489291918}) 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2306988056}) 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489291918}) 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1937514314}) 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489291918}) 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1203548850}) 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.998 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489291918}) 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.785+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2614081678}) 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=984831717}) 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3532391315}) 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=984831717}) 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2540151863}) 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=984831717}) 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=58717105}) 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=984831717}) 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1665832437}) 2024-09-18T14:19:19.999 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=984831717}) 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3055855354}) 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=984831717}) 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2388803190}) 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=984831717}) 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=692006275}) 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=984831717}) 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.786+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3438786632}) 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.000 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=624983503}) 2024-09-18T14:19:20.001 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2796317603}) 2024-09-18T14:19:20.001 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.001 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=624983503}) 2024-09-18T14:19:20.001 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=581239341}) 2024-09-18T14:19:20.001 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.001 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=624983503}) 2024-09-18T14:19:20.001 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1344703515}) 2024-09-18T14:19:20.001 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.001 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=624983503}) 2024-09-18T14:19:20.001 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2139945342}) 2024-09-18T14:19:20.001 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.001 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=624983503}) 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2906218808}) 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=624983503}) 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2836352716}) 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=624983503}) 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=99612358}) 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=624983503}) 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.787+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=921843767}) 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.788+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.788+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=85308865}) 2024-09-18T14:19:20.002 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.788+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2185287240}) 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.788+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.788+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=85308865}) 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.788+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2855797139}) 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.788+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.788+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=85308865}) 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3801411026}) 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=85308865}) 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4078147472}) 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=85308865}) 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1556265840}) 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.003 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=85308865}) 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2249225946}) 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=85308865}) 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3793882914}) 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=85308865}) 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.789+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=460540112}) 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3833247516}) 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4210650383}) 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3833247516}) 2024-09-18T14:19:20.004 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1080798368}) 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3833247516}) 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2246241584}) 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3833247516}) 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3834225469}) 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3833247516}) 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=235793990}) 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3833247516}) 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2767246321}) 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3833247516}) 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2825663907}) 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3833247516}) 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.790+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1752132090}) 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3494478421}) 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1260283555}) 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3494478421}) 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2591163705}) 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3494478421}) 2024-09-18T14:19:20.006 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3199150709}) 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3494478421}) 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3311965190}) 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3494478421}) 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2510248805}) 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3494478421}) 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3243873412}) 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3494478421}) 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1979682899}) 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3494478421}) 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.791+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=284981436}) 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.792+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.792+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722887622}) 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.792+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2648881580}) 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.792+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.792+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722887622}) 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.792+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1365245215}) 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722887622}) 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2082718566}) 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722887622}) 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3210021959}) 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722887622}) 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2650727114}) 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722887622}) 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4137150765}) 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722887622}) 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1475923441}) 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722887622}) 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.793+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3840183112}) 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1456756655}) 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1689875721}) 2024-09-18T14:19:20.009 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1456756655}) 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=394183210}) 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1456756655}) 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2292987171}) 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1456756655}) 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2454184173}) 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1456756655}) 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1756680318}) 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.010 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1456756655}) 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=756828023}) 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1456756655}) 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2502539890}) 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1456756655}) 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.794+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=125193200}) 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.795+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.795+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=561962673}) 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.795+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2316632111}) 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.795+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.795+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=561962673}) 2024-09-18T14:19:20.011 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.795+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3700183973}) 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.795+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.795+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=561962673}) 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.795+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3730306036}) 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.795+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.795+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=561962673}) 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2736640345}) 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=561962673}) 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=268359186}) 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=561962673}) 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1316844229}) 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.012 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=561962673}) 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=872020916}) 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=561962673}) 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.796+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2400366689}) 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2950647783}) 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1745884284}) 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2950647783}) 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=832945773}) 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2950647783}) 2024-09-18T14:19:20.013 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2588177220}) 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2950647783}) 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1483212999}) 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2950647783}) 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=263105880}) 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2950647783}) 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1668508373}) 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2950647783}) 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4022271089}) 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2950647783}) 2024-09-18T14:19:20.014 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.797+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1785692952}) 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3136544806}) 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3133222621}) 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3136544806}) 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2726665922}) 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3136544806}) 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2750563507}) 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3136544806}) 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=305142503}) 2024-09-18T14:19:20.015 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3136544806}) 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=290232412}) 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.798+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.799+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3136544806}) 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.799+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3375844477}) 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.799+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.799+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3136544806}) 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.799+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1604896041}) 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.799+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.799+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3136544806}) 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.799+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4110228832}) 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1923947114}) 2024-09-18T14:19:20.016 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1133881185}) 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1923947114}) 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1097622697}) 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1923947114}) 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3516886970}) 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1923947114}) 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=914342377}) 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1923947114}) 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1876317477}) 2024-09-18T14:19:20.017 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1923947114}) 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2355127564}) 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1923947114}) 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2373450987}) 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1923947114}) 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.800+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1446497228}) 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863714803}) 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=697148304}) 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863714803}) 2024-09-18T14:19:20.018 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1121992965}) 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863714803}) 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1026383075}) 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863714803}) 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=874907298}) 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863714803}) 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4013302088}) 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863714803}) 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3488199189}) 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.019 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863714803}) 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.801+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=723334023}) 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.802+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.802+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863714803}) 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.802+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=943100066}) 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.802+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.802+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1373082283}) 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1277204881}) 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1373082283}) 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1094139742}) 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1373082283}) 2024-09-18T14:19:20.020 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2471847026}) 2024-09-18T14:19:20.021 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.021 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1373082283}) 2024-09-18T14:19:20.021 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1213055745}) 2024-09-18T14:19:20.021 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.021 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1373082283}) 2024-09-18T14:19:20.021 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=576366820}) 2024-09-18T14:19:20.021 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.021 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1373082283}) 2024-09-18T14:19:20.021 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1874456587}) 2024-09-18T14:19:20.021 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.021 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1373082283}) 2024-09-18T14:19:20.021 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3038229596}) 2024-09-18T14:19:20.022 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.022 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1373082283}) 2024-09-18T14:19:20.022 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.803+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4036681893}) 2024-09-18T14:19:20.022 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.022 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3396340354}) 2024-09-18T14:19:20.022 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3155100187}) 2024-09-18T14:19:20.022 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.022 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3396340354}) 2024-09-18T14:19:20.022 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1585028434}) 2024-09-18T14:19:20.022 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.022 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3396340354}) 2024-09-18T14:19:20.022 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2195915712}) 2024-09-18T14:19:20.023 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.023 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3396340354}) 2024-09-18T14:19:20.023 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2871715864}) 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3396340354}) 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4260155943}) 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3396340354}) 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=935700495}) 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3396340354}) 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=701534205}) 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3396340354}) 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.804+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=507312616}) 2024-09-18T14:19:20.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748318698}) 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1047605266}) 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748318698}) 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3559816698}) 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748318698}) 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1627113523}) 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748318698}) 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2157649693}) 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.805+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748318698}) 2024-09-18T14:19:20.025 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.806+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677556450}) 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.806+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.806+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748318698}) 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.806+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1749486907}) 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.806+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.806+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748318698}) 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.806+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3060400305}) 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.806+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.806+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748318698}) 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.806+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3005573070}) 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280914819}) 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=135071422}) 2024-09-18T14:19:20.026 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280914819}) 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3277245358}) 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280914819}) 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2768488550}) 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280914819}) 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4123237714}) 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280914819}) 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2386993639}) 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.027 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280914819}) 2024-09-18T14:19:20.028 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=415199437}) 2024-09-18T14:19:20.028 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.028 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280914819}) 2024-09-18T14:19:20.028 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=738167591}) 2024-09-18T14:19:20.028 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.028 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280914819}) 2024-09-18T14:19:20.028 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.807+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1167560729}) 2024-09-18T14:19:20.028 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.028 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1822465464}) 2024-09-18T14:19:20.028 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=200708801}) 2024-09-18T14:19:20.028 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.028 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1822465464}) 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=688074758}) 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1822465464}) 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3706528716}) 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1822465464}) 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3246169055}) 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1822465464}) 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1491214259}) 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1822465464}) 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=502805059}) 2024-09-18T14:19:20.029 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.808+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.809+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1822465464}) 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.809+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=792227883}) 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.809+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.809+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1822465464}) 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.809+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2228140455}) 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2752527495}) 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4278387755}) 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2752527495}) 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2431486207}) 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2752527495}) 2024-09-18T14:19:20.030 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1260907933}) 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2752527495}) 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2439557611}) 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2752527495}) 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=248438282}) 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2752527495}) 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3803058168}) 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2752527495}) 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3087556548}) 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.031 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2752527495}) 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.810+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2816440515}) 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1854092728}) 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3135616490}) 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1854092728}) 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1305460634}) 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1854092728}) 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3739699980}) 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1854092728}) 2024-09-18T14:19:20.032 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1641917522}) 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1854092728}) 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4163791847}) 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1854092728}) 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=618695294}) 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1854092728}) 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2694366352}) 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1854092728}) 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.811+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3105150403}) 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.033 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308923102}) 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1204875729}) 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308923102}) 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3171955416}) 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308923102}) 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2160138377}) 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308923102}) 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3708824147}) 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.812+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.813+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308923102}) 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.813+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2522055289}) 2024-09-18T14:19:20.034 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.813+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.813+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308923102}) 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.813+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=271867723}) 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.813+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.813+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308923102}) 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.813+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1834682293}) 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.813+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.813+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308923102}) 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.813+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3837328153}) 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3652154409}) 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2462455563}) 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3652154409}) 2024-09-18T14:19:20.035 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=624561077}) 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3652154409}) 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1646273030}) 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3652154409}) 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3065807351}) 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3652154409}) 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622490192}) 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3652154409}) 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=408048195}) 2024-09-18T14:19:20.036 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3652154409}) 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1838478458}) 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3652154409}) 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.814+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=273737836}) 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2700039222}) 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=386805818}) 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2700039222}) 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3461819625}) 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.037 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2700039222}) 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3209596577}) 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2700039222}) 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1783880355}) 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2700039222}) 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1194080644}) 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2700039222}) 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1130186464}) 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2700039222}) 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.815+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3162056587}) 2024-09-18T14:19:20.038 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.816+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.816+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2700039222}) 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.816+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=877338492}) 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.816+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187528422}) 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1460560056}) 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187528422}) 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=208925371}) 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187528422}) 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3628884303}) 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187528422}) 2024-09-18T14:19:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=648641770}) 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187528422}) 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1210530112}) 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187528422}) 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4144721204}) 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187528422}) 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=541383131}) 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187528422}) 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.817+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=567591147}) 2024-09-18T14:19:20.040 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3595585583}) 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4118196788}) 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3595585583}) 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1222469638}) 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3595585583}) 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4290047935}) 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3595585583}) 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1291454282}) 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.041 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3595585583}) 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2890174750}) 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3595585583}) 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3979091577}) 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3595585583}) 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1163757338}) 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3595585583}) 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.818+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1789219196}) 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.819+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.819+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1648453800}) 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.819+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=387467817}) 2024-09-18T14:19:20.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.819+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.819+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1648453800}) 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=927603909}) 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1648453800}) 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3374952566}) 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1648453800}) 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1762589303}) 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1648453800}) 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1803248934}) 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1648453800}) 2024-09-18T14:19:20.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2609000882}) 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1648453800}) 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2764787962}) 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1648453800}) 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.820+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=264140504}) 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1907144889}) 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1682624226}) 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1907144889}) 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3766175618}) 2024-09-18T14:19:20.044 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1907144889}) 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=928604259}) 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1907144889}) 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3482925611}) 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1907144889}) 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1276386220}) 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1907144889}) 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3634368994}) 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.821+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.822+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1907144889}) 2024-09-18T14:19:20.045 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.822+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=223604091}) 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.822+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.822+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1907144889}) 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.822+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3551695998}) 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297994507}) 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=917744346}) 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297994507}) 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3019455522}) 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297994507}) 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2362045651}) 2024-09-18T14:19:20.046 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297994507}) 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3187717091}) 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297994507}) 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=829662077}) 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297994507}) 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1319554660}) 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297994507}) 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2409976341}) 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.047 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297994507}) 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.823+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4109701333}) 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3865926739}) 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1091122360}) 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3865926739}) 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1333370640}) 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3865926739}) 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=91599152}) 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3865926739}) 2024-09-18T14:19:20.048 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=247077293}) 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3865926739}) 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3679412516}) 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3865926739}) 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2253853739}) 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3865926739}) 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3775659531}) 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3865926739}) 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.824+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2551381080}) 2024-09-18T14:19:20.049 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.825+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.825+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4220765006}) 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.825+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2700320276}) 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.825+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.825+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4220765006}) 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.825+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3097142287}) 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.825+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.825+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4220765006}) 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.825+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3916318672}) 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.825+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.825+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4220765006}) 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2150961285}) 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.050 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4220765006}) 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2907903656}) 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4220765006}) 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=805922480}) 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4220765006}) 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2183110596}) 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4220765006}) 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.826+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2407406859}) 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=535952081}) 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2559356386}) 2024-09-18T14:19:20.051 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=535952081}) 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=553371589}) 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=535952081}) 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3460736295}) 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=535952081}) 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=612701073}) 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=535952081}) 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3190699098}) 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.052 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=535952081}) 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2854741789}) 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=535952081}) 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=371144384}) 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=535952081}) 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.827+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=853941144}) 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3077742735}) 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3243875056}) 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3077742735}) 2024-09-18T14:19:20.053 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=67396140}) 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3077742735}) 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2018830794}) 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3077742735}) 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2676236046}) 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.828+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.829+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3077742735}) 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.829+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3238532452}) 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.829+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.829+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3077742735}) 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.829+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1193581860}) 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.829+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.054 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.829+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3077742735}) 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.829+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=949281459}) 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.829+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.829+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3077742735}) 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.829+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2822515573}) 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1300348848}) 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2356230393}) 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1300348848}) 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3243743333}) 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1300348848}) 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=628871269}) 2024-09-18T14:19:20.055 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1300348848}) 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2283151426}) 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1300348848}) 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3774513608}) 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1300348848}) 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=944579119}) 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1300348848}) 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2678746463}) 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.056 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1300348848}) 2024-09-18T14:19:20.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.830+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1230067722}) 2024-09-18T14:19:20.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.831+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.831+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1985908118}) 2024-09-18T14:19:20.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.831+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2980343059}) 2024-09-18T14:19:20.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.831+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.831+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1985908118}) 2024-09-18T14:19:20.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.831+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1570299464}) 2024-09-18T14:19:20.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.831+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.831+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1985908118}) 2024-09-18T14:19:20.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.831+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=165927516}) 2024-09-18T14:19:20.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.831+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.831+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1985908118}) 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4160496758}) 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1985908118}) 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3316475430}) 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1985908118}) 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=953706954}) 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1985908118}) 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2390836406}) 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1985908118}) 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.832+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=838948344}) 2024-09-18T14:19:20.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1884588058}) 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3877311233}) 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1884588058}) 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4256754957}) 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1884588058}) 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=823865261}) 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1884588058}) 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4229240225}) 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1884588058}) 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1770615281}) 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1884588058}) 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2844371107}) 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1884588058}) 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3437904300}) 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1884588058}) 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.833+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=237060332}) 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.834+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.834+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2474612164}) 2024-09-18T14:19:20.060 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.834+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2252185598}) 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.834+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.834+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2474612164}) 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.834+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4038311837}) 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.834+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2474612164}) 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4081568433}) 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2474612164}) 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1291689166}) 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2474612164}) 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2935232632}) 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.061 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2474612164}) 2024-09-18T14:19:20.062 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1122014882}) 2024-09-18T14:19:20.062 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.062 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2474612164}) 2024-09-18T14:19:20.062 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=159681539}) 2024-09-18T14:19:20.062 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.062 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2474612164}) 2024-09-18T14:19:20.062 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.835+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1398272980}) 2024-09-18T14:19:20.062 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.062 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3273279662}) 2024-09-18T14:19:20.062 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1705388825}) 2024-09-18T14:19:20.062 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.062 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3273279662}) 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3850946867}) 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3273279662}) 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2150007272}) 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3273279662}) 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1982604278}) 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3273279662}) 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3460998325}) 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3273279662}) 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2167638217}) 2024-09-18T14:19:20.063 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3273279662}) 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=717075259}) 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3273279662}) 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.836+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3443634581}) 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.837+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.837+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=719108010}) 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.837+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1088330422}) 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.837+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.837+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=719108010}) 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.837+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3889239645}) 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.064 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=719108010}) 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=201075308}) 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=719108010}) 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4232833760}) 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=719108010}) 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3829639305}) 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=719108010}) 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1787930730}) 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=719108010}) 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=750854617}) 2024-09-18T14:19:20.065 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=719108010}) 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.838+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=902900351}) 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=75715588}) 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=363320921}) 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=75715588}) 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=666703896}) 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=75715588}) 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=573083802}) 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.066 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=75715588}) 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=772362727}) 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=75715588}) 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3182263382}) 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=75715588}) 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1978435258}) 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=75715588}) 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=678232493}) 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=75715588}) 2024-09-18T14:19:20.067 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.839+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=479825391}) 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.840+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.840+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2056032413}) 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.840+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2257503116}) 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.840+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2056032413}) 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3074753955}) 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2056032413}) 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1923138091}) 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2056032413}) 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=976393593}) 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.068 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2056032413}) 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2582134122}) 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2056032413}) 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2606956476}) 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2056032413}) 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3494479868}) 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2056032413}) 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.841+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1626130540}) 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=911570050}) 2024-09-18T14:19:20.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=544624498}) 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=911570050}) 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4044510165}) 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=911570050}) 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=84396570}) 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=911570050}) 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3731492120}) 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=911570050}) 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3537664122}) 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.070 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=911570050}) 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2340524650}) 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=911570050}) 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2055822055}) 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=911570050}) 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.842+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1312892518}) 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.843+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.843+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3660650425}) 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.843+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=217280499}) 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.843+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.843+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3660650425}) 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.843+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2899278546}) 2024-09-18T14:19:20.071 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.843+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.843+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3660650425}) 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3023734426}) 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3660650425}) 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=614959496}) 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3660650425}) 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1180485564}) 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3660650425}) 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4197986561}) 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.072 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3660650425}) 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1034908308}) 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3660650425}) 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.844+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2010292673}) 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176968632}) 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3475203886}) 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176968632}) 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1184752694}) 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176968632}) 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=837713295}) 2024-09-18T14:19:20.073 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176968632}) 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1689445965}) 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176968632}) 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801751083}) 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176968632}) 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3610986081}) 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176968632}) 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2002608834}) 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176968632}) 2024-09-18T14:19:20.074 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.845+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3892581091}) 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=713873692}) 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1666632523}) 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=713873692}) 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=412140896}) 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=713873692}) 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2908126625}) 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=713873692}) 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=642858375}) 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.075 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=713873692}) 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2428724367}) 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=713873692}) 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2003517972}) 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=713873692}) 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3970141005}) 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=713873692}) 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.846+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4145088107}) 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.847+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.847+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=646275594}) 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.847+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=357215953}) 2024-09-18T14:19:20.076 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.847+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.077 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.847+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=646275594}) 2024-09-18T14:19:20.077 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.847+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1208522510}) 2024-09-18T14:19:20.077 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.077 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=646275594}) 2024-09-18T14:19:20.077 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=987793234}) 2024-09-18T14:19:20.077 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.077 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=646275594}) 2024-09-18T14:19:20.077 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3433311203}) 2024-09-18T14:19:20.077 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.077 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=646275594}) 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4253205549}) 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=646275594}) 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3685566006}) 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=646275594}) 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3578543045}) 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=646275594}) 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.848+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2403142718}) 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2784888414}) 2024-09-18T14:19:20.078 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1229253531}) 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2784888414}) 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1020646374}) 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2784888414}) 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3451775702}) 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2784888414}) 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2999799738}) 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2784888414}) 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=868654854}) 2024-09-18T14:19:20.079 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2784888414}) 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=783184237}) 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2784888414}) 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4158725279}) 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2784888414}) 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.849+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=24994658}) 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=950315861}) 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3488403564}) 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.080 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=950315861}) 2024-09-18T14:19:20.081 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=468708470}) 2024-09-18T14:19:20.081 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.081 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=950315861}) 2024-09-18T14:19:20.081 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=228698987}) 2024-09-18T14:19:20.081 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.081 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=950315861}) 2024-09-18T14:19:20.081 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2368427283}) 2024-09-18T14:19:20.081 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.081 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=950315861}) 2024-09-18T14:19:20.081 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4100896672}) 2024-09-18T14:19:20.081 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=950315861}) 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2992732666}) 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=950315861}) 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3497504134}) 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=950315861}) 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.850+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2610975127}) 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435854828}) 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3910732821}) 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.082 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435854828}) 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2312131176}) 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435854828}) 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2983927651}) 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435854828}) 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1470085810}) 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435854828}) 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1900906738}) 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435854828}) 2024-09-18T14:19:20.083 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2366538081}) 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435854828}) 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3518241022}) 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435854828}) 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.851+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1396662705}) 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.852+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.852+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=705035675}) 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.852+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2967776369}) 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.852+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.852+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=705035675}) 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.852+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3077844382}) 2024-09-18T14:19:20.084 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.852+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.852+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=705035675}) 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.852+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2124082088}) 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.852+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.852+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=705035675}) 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3459232099}) 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=705035675}) 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3921950161}) 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=705035675}) 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1482063799}) 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.085 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=705035675}) 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=322877430}) 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=705035675}) 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.853+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=878714443}) 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465851489}) 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4131655159}) 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465851489}) 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3026653537}) 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465851489}) 2024-09-18T14:19:20.086 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=962958899}) 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465851489}) 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3019732381}) 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465851489}) 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3913386547}) 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465851489}) 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2210163914}) 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465851489}) 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1714485262}) 2024-09-18T14:19:20.087 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465851489}) 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.854+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1406643113}) 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=180922613}) 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2277542016}) 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=180922613}) 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3582677815}) 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=180922613}) 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1073403892}) 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.088 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=180922613}) 2024-09-18T14:19:20.089 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=827485194}) 2024-09-18T14:19:20.089 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.089 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=180922613}) 2024-09-18T14:19:20.089 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2005373592}) 2024-09-18T14:19:20.089 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.089 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=180922613}) 2024-09-18T14:19:20.089 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=865139862}) 2024-09-18T14:19:20.089 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.089 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=180922613}) 2024-09-18T14:19:20.089 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=488767912}) 2024-09-18T14:19:20.089 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.089 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=180922613}) 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.855+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1521097919}) 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3917702279}) 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=341897275}) 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3917702279}) 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4093291801}) 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3917702279}) 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3394406384}) 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3917702279}) 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2279249392}) 2024-09-18T14:19:20.090 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.856+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.857+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3917702279}) 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.857+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1975448360}) 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.857+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.857+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3917702279}) 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.857+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4250303401}) 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.857+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.857+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3917702279}) 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.857+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4132041056}) 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.857+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.857+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3917702279}) 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.857+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=359754487}) 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.091 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3148235403}) 2024-09-18T14:19:20.092 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659147898}) 2024-09-18T14:19:20.092 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.092 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3148235403}) 2024-09-18T14:19:20.092 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1950675334}) 2024-09-18T14:19:20.092 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.092 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3148235403}) 2024-09-18T14:19:20.092 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=820995860}) 2024-09-18T14:19:20.092 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.092 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3148235403}) 2024-09-18T14:19:20.092 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1842862563}) 2024-09-18T14:19:20.092 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.092 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3148235403}) 2024-09-18T14:19:20.093 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.858+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2501503470}) 2024-09-18T14:19:20.093 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.859+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.093 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.859+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3148235403}) 2024-09-18T14:19:20.093 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.859+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2333603698}) 2024-09-18T14:19:20.093 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.859+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.093 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.859+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3148235403}) 2024-09-18T14:19:20.093 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.859+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=888565924}) 2024-09-18T14:19:20.093 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.859+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.093 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.859+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3148235403}) 2024-09-18T14:19:20.093 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.859+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=214938006}) 2024-09-18T14:19:20.093 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.093 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=636724606}) 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1190569499}) 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=636724606}) 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1752647796}) 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=636724606}) 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=973674321}) 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=636724606}) 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3535053512}) 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=636724606}) 2024-09-18T14:19:20.094 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4289318017}) 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=636724606}) 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3071019571}) 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=636724606}) 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4208507462}) 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=636724606}) 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.860+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1848189157}) 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1213235199}) 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2553932128}) 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1213235199}) 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1425372189}) 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1213235199}) 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3238852337}) 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1213235199}) 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2645309357}) 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1213235199}) 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3314194544}) 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1213235199}) 2024-09-18T14:19:20.096 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=324135965}) 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1213235199}) 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2604907590}) 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1213235199}) 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.861+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2823382901}) 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2478389383}) 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1836221205}) 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2478389383}) 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2896777977}) 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.097 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2478389383}) 2024-09-18T14:19:20.098 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=796549793}) 2024-09-18T14:19:20.098 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.098 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2478389383}) 2024-09-18T14:19:20.098 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2700160235}) 2024-09-18T14:19:20.098 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.862+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.098 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.863+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2478389383}) 2024-09-18T14:19:20.098 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.863+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1361222854}) 2024-09-18T14:19:20.098 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.863+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.098 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.863+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2478389383}) 2024-09-18T14:19:20.098 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.863+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1827927027}) 2024-09-18T14:19:20.098 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.863+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.098 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.863+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2478389383}) 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.863+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=626529991}) 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.863+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.863+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2478389383}) 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.863+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3359934269}) 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=996368061}) 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=807044595}) 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=996368061}) 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4135044665}) 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=996368061}) 2024-09-18T14:19:20.099 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3128792521}) 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=996368061}) 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1661871466}) 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=996368061}) 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1895627583}) 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=996368061}) 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2206625082}) 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=996368061}) 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3151885625}) 2024-09-18T14:19:20.100 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=996368061}) 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.864+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=51232505}) 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.865+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.865+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=86091628}) 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.865+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2895633219}) 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.865+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.865+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=86091628}) 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.865+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4290536569}) 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=86091628}) 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=651282529}) 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=86091628}) 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2187536210}) 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=86091628}) 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1905356782}) 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=86091628}) 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2018121612}) 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=86091628}) 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=517771126}) 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=86091628}) 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.866+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4267989953}) 2024-09-18T14:19:20.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292876718}) 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2341325863}) 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292876718}) 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=282693439}) 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292876718}) 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1660139651}) 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292876718}) 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1013004279}) 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292876718}) 2024-09-18T14:19:20.103 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.867+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2085525254}) 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.868+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.868+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292876718}) 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.868+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2689717520}) 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.868+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.868+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292876718}) 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.868+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2989002848}) 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.868+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.868+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292876718}) 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.868+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2425367613}) 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1391211581}) 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2501221682}) 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.104 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1391211581}) 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1665753137}) 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1391211581}) 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3471114484}) 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1391211581}) 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1411166071}) 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1391211581}) 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4191049308}) 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1391211581}) 2024-09-18T14:19:20.105 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3244327019}) 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1391211581}) 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3038019645}) 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1391211581}) 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.869+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2333670948}) 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730326977}) 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2160039630}) 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730326977}) 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2152522078}) 2024-09-18T14:19:20.106 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730326977}) 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=992527246}) 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730326977}) 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=622203188}) 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730326977}) 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.870+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=837048806}) 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.871+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.871+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730326977}) 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.871+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1324195217}) 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.871+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.871+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730326977}) 2024-09-18T14:19:20.107 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.871+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4189623342}) 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.871+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.871+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730326977}) 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.871+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1708196618}) 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4067503893}) 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2663306581}) 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4067503893}) 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2995512528}) 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4067503893}) 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3445485894}) 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.108 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4067503893}) 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3100005647}) 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4067503893}) 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=662657887}) 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4067503893}) 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=67099079}) 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4067503893}) 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3219611969}) 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4067503893}) 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.872+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3892400164}) 2024-09-18T14:19:20.109 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043436916}) 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2192902881}) 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043436916}) 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4230698267}) 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043436916}) 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1647172084}) 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043436916}) 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3163412576}) 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.873+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.110 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.874+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043436916}) 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.874+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4118224338}) 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.874+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.874+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043436916}) 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.874+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1282191050}) 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.874+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.874+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043436916}) 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.874+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3189629086}) 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.874+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.874+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043436916}) 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.874+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=282694121}) 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1271755402}) 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4199909820}) 2024-09-18T14:19:20.111 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1271755402}) 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3886879806}) 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1271755402}) 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=306099774}) 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1271755402}) 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2802734583}) 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1271755402}) 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1555931155}) 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1271755402}) 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4226422222}) 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1271755402}) 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4094149415}) 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1271755402}) 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.875+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=419120639}) 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.876+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.876+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914646024}) 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.876+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1079784357}) 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.876+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.876+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914646024}) 2024-09-18T14:19:20.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.876+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=773412973}) 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.876+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.876+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914646024}) 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.876+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4170848193}) 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.876+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.876+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914646024}) 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=750953055}) 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914646024}) 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2442630745}) 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914646024}) 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2406135812}) 2024-09-18T14:19:20.114 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914646024}) 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3574945520}) 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914646024}) 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.877+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1362043033}) 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608409192}) 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1061346104}) 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608409192}) 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=611861169}) 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608409192}) 2024-09-18T14:19:20.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1343425398}) 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608409192}) 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1439947318}) 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608409192}) 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=235185379}) 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608409192}) 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3810057725}) 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608409192}) 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3842643643}) 2024-09-18T14:19:20.116 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608409192}) 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.878+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1139453520}) 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510732961}) 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1625401164}) 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510732961}) 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1408704747}) 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510732961}) 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3881043873}) 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.117 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510732961}) 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2359870235}) 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510732961}) 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4247783534}) 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510732961}) 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=356341751}) 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510732961}) 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=346026417}) 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510732961}) 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.879+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3366322351}) 2024-09-18T14:19:20.118 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707019443}) 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3878703696}) 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707019443}) 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=110944522}) 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707019443}) 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3149822191}) 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707019443}) 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=723873242}) 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707019443}) 2024-09-18T14:19:20.119 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3968429142}) 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707019443}) 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=381831887}) 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707019443}) 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2460078573}) 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.880+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.881+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707019443}) 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.881+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1611079033}) 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.881+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.881+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3306590294}) 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.881+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3867916532}) 2024-09-18T14:19:20.120 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.881+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.881+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3306590294}) 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.881+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=917057641}) 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.881+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.881+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3306590294}) 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1301202857}) 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3306590294}) 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=32533820}) 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3306590294}) 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2165188060}) 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3306590294}) 2024-09-18T14:19:20.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3006742920}) 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3306590294}) 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3846616430}) 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3306590294}) 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.882+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1297369217}) 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557573099}) 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1741075165}) 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557573099}) 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2861878030}) 2024-09-18T14:19:20.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557573099}) 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3646053460}) 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557573099}) 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=89487197}) 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557573099}) 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3722132716}) 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557573099}) 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3370092592}) 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.123 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557573099}) 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1357463911}) 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557573099}) 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.883+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3431130973}) 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3731966259}) 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2453810106}) 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3731966259}) 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2746718139}) 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3731966259}) 2024-09-18T14:19:20.124 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=755316705}) 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3731966259}) 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3513167524}) 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3731966259}) 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1386231311}) 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3731966259}) 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3003707662}) 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3731966259}) 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=942086419}) 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.125 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3731966259}) 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.884+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1500410424}) 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=122739510}) 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3253747535}) 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=122739510}) 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3593017159}) 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=122739510}) 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3835810353}) 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=122739510}) 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3045981957}) 2024-09-18T14:19:20.126 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=122739510}) 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2207981051}) 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=122739510}) 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2291453578}) 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=122739510}) 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=625564928}) 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=122739510}) 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.885+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1053286402}) 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.886+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.886+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639360299}) 2024-09-18T14:19:20.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.886+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4147619309}) 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.886+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.886+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639360299}) 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.886+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3424066436}) 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.886+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.886+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639360299}) 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.886+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1214503417}) 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.886+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.886+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639360299}) 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.886+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4002276337}) 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.887+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.887+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639360299}) 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.887+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2587490211}) 2024-09-18T14:19:20.128 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.887+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.887+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639360299}) 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.887+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=485290666}) 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.887+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.887+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639360299}) 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.887+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2502631026}) 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.887+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.887+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639360299}) 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.887+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2971482387}) 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785526359}) 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4065156868}) 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785526359}) 2024-09-18T14:19:20.129 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1381748136}) 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785526359}) 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1037681307}) 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785526359}) 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2136673802}) 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785526359}) 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2619795806}) 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785526359}) 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=613722514}) 2024-09-18T14:19:20.130 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785526359}) 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1196870988}) 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785526359}) 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.888+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1444284013}) 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.889+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.889+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=302155912}) 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.889+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3267081276}) 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.889+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.889+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=302155912}) 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.889+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3927510262}) 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.889+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.889+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=302155912}) 2024-09-18T14:19:20.131 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.889+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3886425024}) 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.889+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.889+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=302155912}) 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3443279034}) 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=302155912}) 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=586319302}) 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=302155912}) 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1277689582}) 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=302155912}) 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=393962038}) 2024-09-18T14:19:20.132 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=302155912}) 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.890+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2591757217}) 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1921144821}) 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4137885494}) 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1921144821}) 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2172773542}) 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1921144821}) 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=338654604}) 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.133 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1921144821}) 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2934474976}) 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1921144821}) 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3198799578}) 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1921144821}) 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=248592746}) 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1921144821}) 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3370318286}) 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1921144821}) 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.891+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2276595335}) 2024-09-18T14:19:20.134 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4038244253}) 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2548433139}) 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4038244253}) 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3452165881}) 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4038244253}) 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=979631461}) 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4038244253}) 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=971519765}) 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4038244253}) 2024-09-18T14:19:20.135 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=274315681}) 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4038244253}) 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3217847735}) 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4038244253}) 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2341852823}) 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4038244253}) 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.892+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2364606896}) 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2389056684}) 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1237948909}) 2024-09-18T14:19:20.136 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2389056684}) 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3006905754}) 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2389056684}) 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=91358272}) 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2389056684}) 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2603788413}) 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2389056684}) 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=922610289}) 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2389056684}) 2024-09-18T14:19:20.137 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4204932760}) 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2389056684}) 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=16228146}) 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2389056684}) 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.893+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2880165083}) 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.894+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.894+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2568865661}) 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.894+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=925656085}) 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.894+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2568865661}) 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=861002593}) 2024-09-18T14:19:20.138 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2568865661}) 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2713071102}) 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2568865661}) 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3144244469}) 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2568865661}) 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1987299835}) 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2568865661}) 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1274826939}) 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.139 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2568865661}) 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3198342014}) 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2568865661}) 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.895+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1943042563}) 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898309343}) 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1302315082}) 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898309343}) 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2130046006}) 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898309343}) 2024-09-18T14:19:20.140 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1963876308}) 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898309343}) 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2470718281}) 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898309343}) 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4205333149}) 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898309343}) 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2192457487}) 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898309343}) 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=189299340}) 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.141 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898309343}) 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.896+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3447096890}) 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338889296}) 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1904365545}) 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338889296}) 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1232281165}) 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338889296}) 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1843629063}) 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338889296}) 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3741102991}) 2024-09-18T14:19:20.142 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338889296}) 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2515195090}) 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338889296}) 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=113727348}) 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338889296}) 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3776100381}) 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338889296}) 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.897+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2601686193}) 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.143 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=49,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=80707846,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1459946096,mon_config_key=1810658514,monmap=3782259094,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=1078222138}) 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=48,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=17} crc {logm=1119819402,mdsmap=3585761405,mgr=1259143633,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1459946096,mon_config_key=1810658514,monmap=455718035,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=1287177570}) 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=49,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=80707846,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1459946096,mon_config_key=1810658514,monmap=3782259094,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=1078222138}) 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=50,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=16} crc {logm=2395061221,mdsmap=3585761405,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3920230569,mon_config_key=2098055267,monmap=3140514009,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=3870023309}) 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=49,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=80707846,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1459946096,mon_config_key=1810658514,monmap=3782259094,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=1078222138}) 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=47,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=17} crc {logm=2916002649,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2427825275,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=2888100995}) 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=49,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=80707846,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1459946096,mon_config_key=1810658514,monmap=3782259094,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=1078222138}) 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=48,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=4,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=18} crc {logm=3288628524,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=3945061516,mon_config_key=3155120151,monmap=455718034,osd_metadata=3702629295,osd_pg_creating=920254315,osd_snap=277283546}) 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=49,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=80707846,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1459946096,mon_config_key=1810658514,monmap=3782259094,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=1078222138}) 2024-09-18T14:19:20.144 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=46,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=17} crc {logm=3486361306,mdsmap=1531151339,mgr=2965137648,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1459946096,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=1532849726}) 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=49,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=80707846,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1459946096,mon_config_key=1810658514,monmap=3782259094,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=1078222138}) 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=49,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16} crc {logm=2803278544,mdsmap=1531151339,mgr=405966909,mgr_command_descs=3545737492,mgrstat=1642844290,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=1827520569}) 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=49,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=80707846,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1459946096,mon_config_key=1810658514,monmap=3782259094,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=1078222138}) 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=49,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=15} crc {logm=1705928118,mdsmap=2394902515,mgr=3670438315,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1459946096,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=538628745}) 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=49,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=80707846,mdsmap=1531151339,mgr=1359932698,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=1459946096,mon_config_key=1810658514,monmap=3782259094,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=1078222138}) 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.899+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=44,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=23} crc {logm=3384918590,mdsmap=3140514009,mgr=1821329879,mgr_command_descs=3545737492,mgr_metadata=270970526,mgrstat=2974141689,mon_config_key=1810658514,monmap=455718034,osd_metadata=2800182873,osd_pg_creating=920254315,osd_snap=1812015661}) 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=1950487561}) 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=100} crc {osd_snap=1341778374}) 2024-09-18T14:19:20.145 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=1950487561}) 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=100} crc {osd_snap=2803022383}) 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=1950487561}) 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=100} crc {osd_snap=2966892094}) 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=1950487561}) 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=100} crc {osd_snap=2622162831}) 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=1950487561}) 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=100} crc {osd_snap=2255321546}) 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=1950487561}) 2024-09-18T14:19:20.146 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=100} crc {osd_snap=2401957961}) 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=1950487561}) 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=100} crc {osd_snap=3482925154}) 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=1950487561}) 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.900+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=100} crc {osd_snap=696660911}) 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=45,osdmap=55} crc {osd_snap=501793006,osdmap=335768170}) 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=47,osdmap=53} crc {osd_snap=659087175,osdmap=1407155008}) 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=45,osdmap=55} crc {osd_snap=501793006,osdmap=335768170}) 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=43,osdmap=57} crc {osd_snap=3619194003,osdmap=1447479649}) 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.147 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=45,osdmap=55} crc {osd_snap=501793006,osdmap=335768170}) 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=46,osdmap=54} crc {osd_snap=673998580,osdmap=2657286896}) 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=45,osdmap=55} crc {osd_snap=501793006,osdmap=335768170}) 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=41,osdmap=59} crc {osd_snap=1876515582,osdmap=1729529896}) 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=45,osdmap=55} crc {osd_snap=501793006,osdmap=335768170}) 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.901+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=46,osdmap=54} crc {osd_snap=1407393229,osdmap=2355456972}) 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.902+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.902+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=45,osdmap=55} crc {osd_snap=501793006,osdmap=335768170}) 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.902+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=45,osdmap=55} crc {osd_snap=3579409577,osdmap=174687915}) 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.902+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.902+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=45,osdmap=55} crc {osd_snap=501793006,osdmap=335768170}) 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.902+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=46,osdmap=54} crc {osd_snap=1611001810,osdmap=2746391870}) 2024-09-18T14:19:20.148 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.902+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.902+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=45,osdmap=55} crc {osd_snap=501793006,osdmap=335768170}) 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.902+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=42,osdmap=58} crc {osd_snap=4169123194,osdmap=1848868582}) 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3920812237}) 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1350280107}) 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3920812237}) 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4130110929}) 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3920812237}) 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3839360313}) 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3920812237}) 2024-09-18T14:19:20.149 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3850327115}) 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3920812237}) 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3929802820}) 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3920812237}) 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=15004944}) 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3920812237}) 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=604760525}) 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3920812237}) 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.903+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3052602518}) 2024-09-18T14:19:20.150 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=82276922}) 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3479493219}) 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=82276922}) 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4128957311}) 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=82276922}) 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1643692892}) 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=82276922}) 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1160804968}) 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=82276922}) 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1449991079}) 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=82276922}) 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3357652480}) 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=82276922}) 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3298157123}) 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=82276922}) 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.904+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=781245290}) 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2220753260}) 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2914312895}) 2024-09-18T14:19:20.152 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2220753260}) 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2899548643}) 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2220753260}) 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2878835176}) 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2220753260}) 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3058291030}) 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2220753260}) 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=397781655}) 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.153 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2220753260}) 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=882963761}) 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2220753260}) 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=105766884}) 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.906+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2220753260}) 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.907+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3628437745}) 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.907+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.907+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=41} crc {osdmap=1656096370}) 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.907+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=40} crc {osdmap=2123850399}) 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.907+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.907+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=41} crc {osdmap=1656096370}) 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.907+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=41} crc {osdmap=3267476012}) 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.907+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.154 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=41} crc {osdmap=1656096370}) 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=40} crc {osdmap=1663629616}) 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=41} crc {osdmap=1656096370}) 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=40} crc {osdmap=3255169678}) 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=41} crc {osdmap=1656096370}) 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=40} crc {osdmap=2566517067}) 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=41} crc {osdmap=1656096370}) 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=42} crc {osdmap=1999631724}) 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=41} crc {osdmap=1656096370}) 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=42} crc {osdmap=3991632837}) 2024-09-18T14:19:20.155 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-18T14:19:20.156 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=41} crc {osdmap=1656096370}) 2024-09-18T14:19:20.156 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-18T14:19:19.908+0000 7fe3de9f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=42} crc {osdmap=3991632837}) 2024-09-18T14:19:20.754 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-18T14:19:20.755 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-18T14:19:20.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.888+0000 7f86f4d9d640 10 monclient: get_monmap_and_config 2024-09-18T14:19:20.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.888+0000 7f86f4d9d640 10 monclient: build_initial_monmap 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.888+0000 7f86f4d9d640 10 monclient: monmap: 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:20.888997+0000 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:20.888997+0000 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:20.890 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:20.890 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:20.890 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:20.890 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:20.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.888+0000 7f86f4d9d640 1 Processor -- start 2024-09-18T14:19:20.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.888+0000 7f86f4d9d640 1 -- start start 2024-09-18T14:19:20.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.888+0000 7f86f4d9d640 10 monclient: init 2024-09-18T14:19:20.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86f4d9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:20.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86f4d9d640 10 monclient: _add_conns ranks=[4,5,8,0,7,3,6,2,1] 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86f4d9d640 10 monclient(hunting): picked mon.noname-c con 0x7f86f006e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86f4d9d640 10 monclient(hunting): picked mon.noname-h con 0x7f86f00a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86f4d9d640 10 monclient(hunting): picked mon.noname-e con 0x7f86f00a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86f4d9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f005ed70 con 0x7f86f006e810 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86f4d9d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f0097620 con 0x7f86f00a0cb0 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86f4d9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f0097340 con 0x7f86f00a1080 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86f4d9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86f4d9d640 10 monclient(hunting): authenticate will time out at 1976.227485s 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ee59c640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f86f006e810 0x7f86f00a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36960/0 (socket says 172.21.15.45:36960) 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ee59c640 1 -- 172.21.15.45:0/1673673572 learned_addr learned my addr 172.21.15.45:0/1673673572 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515157818 0 0) 0x7f86f005ed70 con 0x7f86f006e810 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:20.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 10 monclient(hunting): my global_id is 165406 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d80034e0 con 0x7f86f006e810 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508953681 0 0) 0x7f86f0097620 con 0x7f86f00a0cb0 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 10 monclient(hunting): my global_id is 185126 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f005ed70 con 0x7f86f00a0cb0 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400323935 0 0) 0x7f86f0097340 con 0x7f86f00a1080 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 10 monclient(hunting): my global_id is 234206 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.889+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f0097620 con 0x7f86f00a1080 2024-09-18T14:19:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4270762437 0 0) 0x7f86d80034e0 con 0x7f86f006e810 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 >> v1:172.21.15.45:6793/0 conn(0x7f86f00a1080 legacy=0x7f86f013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 >> v1:172.21.15.84:6791/0 conn(0x7f86f00a0cb0 legacy=0x7f86f0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 10 monclient: _finish_hunting 0 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 1 monclient: found mon.noname-c 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f00a1660 con 0x7f86f006e810 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 10 monclient: _finish_auth 0 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86e0001bb0 con 0x7f86f006e810 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86f4d9d640 5 monclient: authenticate success, global_id 165406 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86f4d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 10 monclient: dump: 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86f4d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f86e0001d10 con 0x7f86f006e810 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 1 -- 172.21.15.45:0/1673673572 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86e0004be0 con 0x7f86f006e810 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86f4d9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86f4d9d640 4 set_mon_vals no callback set 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86ed59a640 10 monclient: dump: 2024-09-18T14:19:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86f4d9d640 10 monclient: shutdown 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86f4d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86f4d9d640 1 -- 172.21.15.45:0/1673673572 >> v1:172.21.15.45:6791/0 conn(0x7f86f006e810 legacy=0x7f86f00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86f4d9d640 1 -- 172.21.15.45:0/1673673572 shutdown_connections 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86f4d9d640 1 -- 172.21.15.45:0/1673673572 >> 172.21.15.45:0/1673673572 conn(0x7f86f00a5bb0 msgr2=0x7f86f00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86f4d9d640 1 -- 172.21.15.45:0/1673673572 shutdown_connections 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.890+0000 7f86f4d9d640 1 -- 172.21.15.45:0/1673673572 wait complete. 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.891+0000 7f86f4d9d640 10 monclient: build_initial_monmap 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.891+0000 7f86f4d9d640 10 monclient: monmap: 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.891+0000 7f86f4d9d640 1 Processor -- start 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.891+0000 7f86f4d9d640 1 -- start start 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.891+0000 7f86f4d9d640 10 monclient: init 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86f4d9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86f4d9d640 10 monclient: _add_conns ranks=[0,6,1,2,4,7,8,5,3] 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86f4d9d640 10 monclient(hunting): picked mon.noname-a con 0x7f86f006e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86f4d9d640 10 monclient(hunting): picked mon.noname-d con 0x7f86f00a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86f4d9d640 10 monclient(hunting): picked mon.noname-f con 0x7f86f00a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86f4d9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f0122d50 con 0x7f86f006e810 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86f4d9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f013db40 con 0x7f86f00a1080 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86f4d9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f0122790 con 0x7f86f00a0cb0 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86f4d9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86f4d9d640 10 monclient(hunting): authenticate will time out at 1976.230203s 2024-09-18T14:19:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86ee59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f86f006e810 0x7f86f013d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58902/0 (socket says 172.21.15.45:58902) 2024-09-18T14:19:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86ee59c640 1 -- 172.21.15.45:0/759256111 learned_addr learned my addr 172.21.15.45:0/759256111 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86edd9b640 1 --1- 172.21.15.45:0/759256111 >> v1:172.21.15.45:6792/0 conn(0x7f86f00a0cb0 0x7f86f0146660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56410/0 (socket says 172.21.15.45:56410) 2024-09-18T14:19:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765713846 0 0) 0x7f86f0122d50 con 0x7f86f006e810 2024-09-18T14:19:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): my global_id is 6714 2024-09-18T14:19:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d00034e0 con 0x7f86f006e810 2024-09-18T14:19:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84152835 0 0) 0x7f86f0122790 con 0x7f86f00a0cb0 2024-09-18T14:19:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): my global_id is 185289 2024-09-18T14:19:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f0122d50 con 0x7f86f00a0cb0 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689760975 0 0) 0x7f86f013db40 con 0x7f86f00a1080 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): my global_id is 214723 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f0122790 con 0x7f86f00a1080 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4259743984 0 0) 0x7f86d00034e0 con 0x7f86f006e810 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86f013db40 con 0x7f86f006e810 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86e0001a70 con 0x7f86f006e810 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): dump: 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:20.909 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:20.910 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2948440481 0 0) 0x7f86f0122d50 con 0x7f86f00a0cb0 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86d00034e0 con 0x7f86f00a0cb0 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 852455292 0 0) 0x7f86f0122790 con 0x7f86f00a1080 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86f0122d50 con 0x7f86f00a1080 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86e4001d90 con 0x7f86f00a0cb0 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.892+0000 7f86de7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 10 monclient(hunting): dump: 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86e8002870 con 0x7f86f00a1080 2024-09-18T14:19:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:20.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:19:20.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 10 monclient(hunting): dump: 2024-09-18T14:19:20.913 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:20.913 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:20.913 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:20.913 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:20.913 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:20.913 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:20.913 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3016066125 0 0) 0x7f86f013db40 con 0x7f86f006e810 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 >> v1:172.21.15.45:6792/0 conn(0x7f86f00a0cb0 legacy=0x7f86f0146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 >> v1:172.21.15.84:6789/0 conn(0x7f86f00a1080 legacy=0x7f86f0149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 1 monclient: found mon.a 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f0120250 con 0x7f86f006e810 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86f4d9d640 5 monclient: authenticate success, global_id 6714 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86f4d9d640 10 monclient: _renew_subs 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86f4d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86f4d9d640 1 -- 172.21.15.45:0/759256111 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f01224b0 con 0x7f86f006e810 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86f4d9d640 10 monclient: _renew_subs 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86f4d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86f4d9d640 1 -- 172.21.15.45:0/759256111 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f01215e0 con 0x7f86f006e810 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f86e0005110 con 0x7f86f006e810 2024-09-18T14:19:20.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f86e00055c0 con 0x7f86f006e810 2024-09-18T14:19:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86ecd99640 4 set_mon_vals no callback set 2024-09-18T14:19:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.893+0000 7f86de7fc640 10 monclient: dump: 2024-09-18T14:19:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.894+0000 7f86f4d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.894+0000 7f86f4d9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.894+0000 7f86f4d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.894+0000 7f86f4d9d640 1 -- 172.21.15.45:0/759256111 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d4005010 con 0x7f86f006e810 2024-09-18T14:19:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.894+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f86e0010590 con 0x7f86f006e810 2024-09-18T14:19:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.894+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f86e0046f40 con 0x7f86f006e810 2024-09-18T14:19:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.894+0000 7f86edd9b640 10 monclient: get_auth_request con 0x7f86d003a9e0 auth_method 0 2024-09-18T14:19:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.898+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f86e0011f60 con 0x7f86f006e810 2024-09-18T14:19:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.898+0000 7f86de7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:20.898+0000 7f86de7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:21.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.128+0000 7f86f4d9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-18T14:19:21.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.128+0000 7f86f4d9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-18T14:19:21.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.128+0000 7f86f4d9d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:21.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.128+0000 7f86f4d9d640 1 -- 172.21.15.45:0/759256111 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f86d4005840 con 0x7f86f006e810 2024-09-18T14:19:21.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.128+0000 7f86de7fc640 1 -- 172.21.15.45:0/759256111 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2689 (unknown 254746657 0 1476219660) 0x7f86e0015370 con 0x7f86f006e810 2024-09-18T14:19:21.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.128+0000 7f86de7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-18T14:19:21.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.128+0000 7f86de7fc640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:21.129 INFO:teuthology.orchestra.run.smithi045.stdout:{"election_epoch":156,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","modified":"2024-09-18T13:58:22.742395Z","created":"2024-09-18T13:58:22.742395Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6790","nonce":0}]},"addr":"172.21.15.84:6790/0","public_addr":"172.21.15.84:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6791","nonce":0}]},"addr":"172.21.15.45:6791/0","public_addr":"172.21.15.45:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6791","nonce":0}]},"addr":"172.21.15.84:6791/0","public_addr":"172.21.15.84:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6792","nonce":0}]},"addr":"172.21.15.45:6792/0","public_addr":"172.21.15.45:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6792","nonce":0}]},"addr":"172.21.15.84:6792/0","public_addr":"172.21.15.84:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6793","nonce":0}]},"addr":"172.21.15.45:6793/0","public_addr":"172.21.15.45:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-18T14:19:21.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.130+0000 7f86f4d9d640 1 -- 172.21.15.45:0/759256111 >> v1:172.21.15.84:6812/31478 conn(0x7f86d003a9e0 legacy=0x7f86d003ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.130+0000 7f86f4d9d640 10 monclient: shutdown 2024-09-18T14:19:21.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.130+0000 7f86f4d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:21.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.130+0000 7f86f4d9d640 1 -- 172.21.15.45:0/759256111 >> v1:172.21.15.45:6789/0 conn(0x7f86f006e810 legacy=0x7f86f013d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.130+0000 7f86f4d9d640 1 -- 172.21.15.45:0/759256111 shutdown_connections 2024-09-18T14:19:21.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.130+0000 7f86f4d9d640 1 -- 172.21.15.45:0/759256111 >> 172.21.15.45:0/759256111 conn(0x7f86f00a5bb0 msgr2=0x7f86f00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:21.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.130+0000 7f86f4d9d640 1 -- 172.21.15.45:0/759256111 shutdown_connections 2024-09-18T14:19:21.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.130+0000 7f86f4d9d640 1 -- 172.21.15.45:0/759256111 wait complete. 2024-09-18T14:19:21.141 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-18T14:19:21.141 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-18T14:19:21.150 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-18T14:19:21.151 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T14:19:21.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.246+0000 7f09c691d640 10 monclient: get_monmap_and_config 2024-09-18T14:19:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.246+0000 7f09c691d640 10 monclient: build_initial_monmap 2024-09-18T14:19:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.246+0000 7f09c691d640 10 monclient: monmap: 2024-09-18T14:19:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:21.247118+0000 2024-09-18T14:19:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:21.247118+0000 2024-09-18T14:19:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:21.248 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:21.248 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:21.248 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:21.248 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:21.248 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:21.248 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.246+0000 7f09c691d640 1 Processor -- start 2024-09-18T14:19:21.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.246+0000 7f09c691d640 1 -- start start 2024-09-18T14:19:21.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.246+0000 7f09c691d640 10 monclient: init 2024-09-18T14:19:21.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09c691d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:21.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09c691d640 10 monclient: _add_conns ranks=[7,5,8,1,4,3,6,0,2] 2024-09-18T14:19:21.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09c691d640 10 monclient(hunting): picked mon.noname-i con 0x7f09c006e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:21.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09c691d640 10 monclient(hunting): picked mon.noname-h con 0x7f09c00a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09c691d640 10 monclient(hunting): picked mon.noname-e con 0x7f09c00a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09c691d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c005ed70 con 0x7f09c00a0cb0 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09c691d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c0097620 con 0x7f09c006e810 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09c691d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c0097340 con 0x7f09c00a1080 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09c691d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09c691d640 10 monclient(hunting): authenticate will time out at 1976.585559s 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09c4eba640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f09c00a1080 0x7f09c013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34846/0 (socket says 172.21.15.45:34846) 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09c4eba640 1 -- 172.21.15.45:0/2650294081 learned_addr learned my addr 172.21.15.45:0/2650294081 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788650440 0 0) 0x7f09c0097340 con 0x7f09c00a1080 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09beffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09beffd640 10 monclient(hunting): my global_id is 234215 2024-09-18T14:19:21.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09a80034e0 con 0x7f09c00a1080 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426625746 0 0) 0x7f09c0097620 con 0x7f09c006e810 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09beffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09beffd640 10 monclient(hunting): my global_id is 214675 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c0097340 con 0x7f09c006e810 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.247+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068754157 0 0) 0x7f09c005ed70 con 0x7f09c00a0cb0 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient(hunting): my global_id is 185135 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c0097620 con 0x7f09c00a0cb0 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4201893850 0 0) 0x7f09a80034e0 con 0x7f09c00a1080 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 >> v1:172.21.15.84:6792/0 conn(0x7f09c006e810 legacy=0x7f09c00a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 >> v1:172.21.15.84:6791/0 conn(0x7f09c00a0cb0 legacy=0x7f09c0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient: _finish_hunting 0 2024-09-18T14:19:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 1 monclient: found mon.noname-e 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00a1660 con 0x7f09c00a1080 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient: _finish_auth 0 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f09b8001bb0 con 0x7f09c00a1080 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09c691d640 5 monclient: authenticate success, global_id 234215 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09c691d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient: dump: 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09c691d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f09b8001d90 con 0x7f09c00a1080 2024-09-18T14:19:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09c691d640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 1 -- 172.21.15.45:0/2650294081 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f09b8004f20 con 0x7f09c00a1080 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09c691d640 4 set_mon_vals no callback set 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09beffd640 10 monclient: dump: 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09c691d640 10 monclient: shutdown 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09c691d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09c691d640 1 -- 172.21.15.45:0/2650294081 >> v1:172.21.15.45:6793/0 conn(0x7f09c00a1080 legacy=0x7f09c013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09c691d640 1 -- 172.21.15.45:0/2650294081 shutdown_connections 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09c691d640 1 -- 172.21.15.45:0/2650294081 >> 172.21.15.45:0/2650294081 conn(0x7f09c00a5bb0 msgr2=0x7f09c00a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09c691d640 1 -- 172.21.15.45:0/2650294081 shutdown_connections 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.248+0000 7f09c691d640 1 -- 172.21.15.45:0/2650294081 wait complete. 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.249+0000 7f09c691d640 10 monclient: build_initial_monmap 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.249+0000 7f09c691d640 10 monclient: monmap: 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.249+0000 7f09c691d640 1 Processor -- start 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.249+0000 7f09c691d640 1 -- start start 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.249+0000 7f09c691d640 10 monclient: init 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09c691d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09c691d640 10 monclient: _add_conns ranks=[3,4,8,6,0,5,2,1,7] 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09c691d640 10 monclient(hunting): picked mon.noname-g con 0x7f09c006e810 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09c691d640 10 monclient(hunting): picked mon.noname-c con 0x7f09c00a0cb0 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09c691d640 10 monclient(hunting): picked mon.noname-e con 0x7f09c00a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09c691d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c0122df0 con 0x7f09c006e810 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09c691d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c013dbe0 con 0x7f09c00a0cb0 2024-09-18T14:19:21.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09c691d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c0122830 con 0x7f09c00a1080 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09c691d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09c691d640 10 monclient(hunting): authenticate will time out at 1976.588281s 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09bf7fe640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f09c00a0cb0 0x7f09c0146700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36968/0 (socket says 172.21.15.45:36968) 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09bf7fe640 1 -- 172.21.15.45:0/922432849 learned_addr learned my addr 172.21.15.45:0/922432849 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09c4eba640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f09c00a1080 0x7f09c0149e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34848/0 (socket says 172.21.15.45:34848) 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698587637 0 0) 0x7f09c0122830 con 0x7f09c00a1080 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): my global_id is 234224 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09ac0034e0 con 0x7f09c00a1080 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147195820 0 0) 0x7f09c0122df0 con 0x7f09c006e810 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): my global_id is 224436 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c0122830 con 0x7f09c006e810 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193012894 0 0) 0x7f09c013dbe0 con 0x7f09c00a0cb0 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): my global_id is 165415 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c0122df0 con 0x7f09c00a0cb0 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1983349050 0 0) 0x7f09c0122830 con 0x7f09c006e810 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09c013dbe0 con 0x7f09c006e810 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 252516472 0 0) 0x7f09ac0034e0 con 0x7f09c00a1080 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09c0122830 con 0x7f09c00a1080 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f09b00016c0 con 0x7f09c006e810 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): dump: 2024-09-18T14:19:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.267 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4242752121 0 0) 0x7f09c0122df0 con 0x7f09c00a0cb0 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.250+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09ac0034e0 con 0x7f09c00a0cb0 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f09b8001b70 con 0x7f09c00a1080 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient(hunting): dump: 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f09b4001970 con 0x7f09c00a0cb0 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient(hunting): dump: 2024-09-18T14:19:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.3 v1:172.21.15.84:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2217858642 0 0) 0x7f09c013dbe0 con 0x7f09c006e810 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 >> v1:172.21.15.45:6793/0 conn(0x7f09c00a1080 legacy=0x7f09c0149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 >> v1:172.21.15.45:6791/0 conn(0x7f09c00a0cb0 legacy=0x7f09c0146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 1 monclient: found mon.g 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 --> v1:172.21.15.84:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0120340 con 0x7f09c006e810 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient: _finish_auth 0 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09c691d640 5 monclient: authenticate success, global_id 224436 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09c691d640 10 monclient: _renew_subs 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09c691d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:19:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09c691d640 1 -- 172.21.15.45:0/922432849 --> v1:172.21.15.84:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c0122550 con 0x7f09c006e810 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09c691d640 10 monclient: _renew_subs 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09c691d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09c691d640 1 -- 172.21.15.45:0/922432849 --> v1:172.21.15.84:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c0121680 con 0x7f09c006e810 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.3 v1:172.21.15.84:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f09b0002040 con 0x7f09c006e810 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.3 v1:172.21.15.84:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f09b00024f0 con 0x7f09c006e810 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09a7fff640 10 monclient: dump: 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.251+0000 7f09be7fc640 4 set_mon_vals no callback set 2024-09-18T14:19:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.252+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.3 v1:172.21.15.84:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f09b000d4c0 con 0x7f09c006e810 2024-09-18T14:19:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.252+0000 7f09c691d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.252+0000 7f09c691d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.252+0000 7f09c691d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:19:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.252+0000 7f09c691d640 1 -- 172.21.15.45:0/922432849 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c004c4a0 con 0x7f09c006e810 2024-09-18T14:19:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.252+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.3 v1:172.21.15.84:6790/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f09b0043ef0 con 0x7f09c006e810 2024-09-18T14:19:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.252+0000 7f09bf7fe640 10 monclient: get_auth_request con 0x7f09ac03a9e0 auth_method 0 2024-09-18T14:19:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.257+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.3 v1:172.21.15.84:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f09b0012860 con 0x7f09c006e810 2024-09-18T14:19:21.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.257+0000 7f09a7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:21.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.257+0000 7f09a7fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.421+0000 7f09c691d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-18T14:19:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.421+0000 7f09c691d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T14:19:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.421+0000 7f09c691d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.84:6790/0 2024-09-18T14:19:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.421+0000 7f09c691d640 1 -- 172.21.15.45:0/922432849 --> v1:172.21.15.84:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09c004b720 con 0x7f09c006e810 2024-09-18T14:19:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.422+0000 7f09a7fff640 1 -- 172.21.15.45:0/922432849 <== mon.3 v1:172.21.15.84:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v899) v1 ==== 74+0+9977 (unknown 3789296979 0 2988026738) 0x7f09b0012260 con 0x7f09c006e810 2024-09-18T14:19:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.422+0000 7f09a7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T14:19:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.422+0000 7f09a7fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:21.423 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T14:19:21.423 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":899,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","created":"2024-09-18T13:58:52.168851+0000","modified":"2024-09-18T14:18:28.377860+0000","last_up_change":"2024-09-18T13:59:00.910314+0000","last_in_change":"2024-09-18T13:58:53.737759+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T13:59:01.925362+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-18T13:59:01.954982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"8c3a8909-07ce-4141-98dd-131951f18ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6804","nonce":32276}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6805","nonce":32276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6807","nonce":32276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6806","nonce":32276}]},"public_addr":"172.21.15.45:6804/32276","cluster_addr":"172.21.15.45:6805/32276","heartbeat_back_addr":"172.21.15.45:6807/32276","heartbeat_front_addr":"172.21.15.45:6806/32276","state":["exists","up"]},{"osd":1,"uuid":"a12b3bc3-1e03-47dc-98c1-c8bd199551a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6800","nonce":32277}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6801","nonce":32277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6803","nonce":32277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6802","nonce":32277}]},"public_addr":"172.21.15.45:6800/32277","cluster_addr":"172.21.15.45:6801/32277","heartbeat_back_addr":"172.21.15.45:6803/32277","heartbeat_front_addr":"172.21.15.45:6802/32277","state":["exists","up"]},{"osd":2,"uuid":"216b8b62-e41c-4bc5-9395-f2d56d402b1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6808","nonce":32278}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6809","nonce":32278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6811","nonce":32278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6810","nonce":32278}]},"public_addr":"172.21.15.45:6808/32278","cluster_addr":"172.21.15.45:6809/32278","heartbeat_back_addr":"172.21.15.45:6811/32278","heartbeat_front_addr":"172.21.15.45:6810/32278","state":["exists","up"]},{"osd":3,"uuid":"c750b0e6-9143-489d-9be0-eb085178ff2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6802","nonce":32024}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6804","nonce":32024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6807","nonce":32024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6806","nonce":32024}]},"public_addr":"172.21.15.84:6802/32024","cluster_addr":"172.21.15.84:6804/32024","heartbeat_back_addr":"172.21.15.84:6807/32024","heartbeat_front_addr":"172.21.15.84:6806/32024","state":["exists","up"]},{"osd":4,"uuid":"9680908d-dc81-44a0-8603-93711579d42e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6800","nonce":32025}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6801","nonce":32025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6805","nonce":32025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6803","nonce":32025}]},"public_addr":"172.21.15.84:6800/32025","cluster_addr":"172.21.15.84:6801/32025","heartbeat_back_addr":"172.21.15.84:6805/32025","heartbeat_front_addr":"172.21.15.84:6803/32025","state":["exists","up"]},{"osd":5,"uuid":"b3b63eef-c8fc-495f-960f-7436315119ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6808","nonce":32048}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6809","nonce":32048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6811","nonce":32048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6810","nonce":32048}]},"public_addr":"172.21.15.84:6808/32048","cluster_addr":"172.21.15.84:6809/32048","heartbeat_back_addr":"172.21.15.84:6811/32048","heartbeat_front_addr":"172.21.15.84:6810/32048","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.823226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.883512+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.904155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.801736+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.835588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.910413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T14:19:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.424+0000 7f09c691d640 1 -- 172.21.15.45:0/922432849 >> v1:172.21.15.84:6812/31478 conn(0x7f09ac03a9e0 legacy=0x7f09ac03ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.424+0000 7f09c691d640 10 monclient: shutdown 2024-09-18T14:19:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.424+0000 7f09c691d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.424+0000 7f09c691d640 1 -- 172.21.15.45:0/922432849 >> v1:172.21.15.84:6790/0 conn(0x7f09c006e810 legacy=0x7f09c013d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.424+0000 7f09c691d640 1 -- 172.21.15.45:0/922432849 shutdown_connections 2024-09-18T14:19:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.424+0000 7f09c691d640 1 -- 172.21.15.45:0/922432849 >> 172.21.15.45:0/922432849 conn(0x7f09c00a5bb0 msgr2=0x7f09c00a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.424+0000 7f09c691d640 1 -- 172.21.15.45:0/922432849 shutdown_connections 2024-09-18T14:19:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.424+0000 7f09c691d640 1 -- 172.21.15.45:0/922432849 wait complete. 2024-09-18T14:19:21.435 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T14:19:21.435 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T14:19:21.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.530+0000 7fb135909640 10 monclient: get_monmap_and_config 2024-09-18T14:19:21.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.530+0000 7fb135909640 10 monclient: build_initial_monmap 2024-09-18T14:19:21.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.530+0000 7fb135909640 10 monclient: monmap: 2024-09-18T14:19:21.531 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:21.531875+0000 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:21.531875+0000 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:21.533 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:21.533 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.531+0000 7fb135909640 1 Processor -- start 2024-09-18T14:19:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.531+0000 7fb135909640 1 -- start start 2024-09-18T14:19:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.531+0000 7fb135909640 10 monclient: init 2024-09-18T14:19:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.531+0000 7fb135909640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb135909640 10 monclient: _add_conns ranks=[7,3,1,4,5,6,8,2,0] 2024-09-18T14:19:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb135909640 10 monclient(hunting): picked mon.noname-i con 0x7fb13009f7d0 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb135909640 10 monclient(hunting): picked mon.noname-g con 0x7fb13009fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb135909640 10 monclient(hunting): picked mon.noname-f con 0x7fb13009ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb135909640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13005ed70 con 0x7fb13009ff70 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb135909640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13009dfc0 con 0x7fb13009fba0 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb135909640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13009dce0 con 0x7fb13009f7d0 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb135909640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb135909640 10 monclient(hunting): authenticate will time out at 1976.870096s 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb1267fc640 1 --1- >> v1:172.21.15.84:6790/0 conn(0x7fb13009fba0 0x7fb130137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6790/0 says I am v1:172.21.15.45:43590/0 (socket says 172.21.15.45:43590) 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb1267fc640 1 -- 172.21.15.45:0/885211274 learned_addr learned my addr 172.21.15.45:0/885211274 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909217445 0 0) 0x7fb13009dce0 con 0x7fb13009f7d0 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient(hunting): my global_id is 214684 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb11c0034e0 con 0x7fb13009f7d0 2024-09-18T14:19:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71157702 0 0) 0x7fb13009dfc0 con 0x7fb13009fba0 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient(hunting): my global_id is 224445 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb13009dce0 con 0x7fb13009fba0 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362631334 0 0) 0x7fb13005ed70 con 0x7fb13009ff70 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient(hunting): my global_id is 214732 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb13009dfc0 con 0x7fb13009ff70 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 127147263 0 0) 0x7fb11c0034e0 con 0x7fb13009f7d0 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 >> v1:172.21.15.84:6790/0 conn(0x7fb13009fba0 legacy=0x7fb130137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 >> v1:172.21.15.84:6789/0 conn(0x7fb13009ff70 legacy=0x7fb13013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 1 monclient: found mon.noname-i 2024-09-18T14:19:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.84:6792/0 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1300a1c00 con 0x7fb13009f7d0 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb120001830 con 0x7fb13009f7d0 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb135909640 5 monclient: authenticate success, global_id 214684 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb135909640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb12e7fc640 10 monclient: dump: 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.532+0000 7fb135909640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 <== mon.7 v1:172.21.15.84:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb120001a10 con 0x7fb13009f7d0 2024-09-18T14:19:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb12e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb12e7fc640 1 -- 172.21.15.45:0/885211274 <== mon.7 v1:172.21.15.84:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb120001fc0 con 0x7fb13009f7d0 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb135909640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb135909640 4 set_mon_vals no callback set 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb12e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb12e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb12e7fc640 10 monclient: dump: 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb135909640 10 monclient: shutdown 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb135909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb135909640 1 -- 172.21.15.45:0/885211274 >> v1:172.21.15.84:6792/0 conn(0x7fb13009f7d0 legacy=0x7fb130132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb135909640 1 -- 172.21.15.45:0/885211274 shutdown_connections 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb135909640 1 -- 172.21.15.45:0/885211274 >> 172.21.15.45:0/885211274 conn(0x7fb1300630b0 msgr2=0x7fb1300634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb135909640 1 -- 172.21.15.45:0/885211274 shutdown_connections 2024-09-18T14:19:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.533+0000 7fb135909640 1 -- 172.21.15.45:0/885211274 wait complete. 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 10 monclient: build_initial_monmap 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 10 monclient: monmap: 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 1 Processor -- start 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 1 -- start start 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 10 monclient: init 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 10 monclient: _add_conns ranks=[4,6,8,2,7,5,3,1,0] 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 10 monclient(hunting): picked mon.noname-c con 0x7fb13009f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 10 monclient(hunting): picked mon.noname-d con 0x7fb13009fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 10 monclient(hunting): picked mon.noname-e con 0x7fb13009ff70 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13009a370 con 0x7fb13009f7d0 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1300a2560 con 0x7fb13009fba0 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13009d310 con 0x7fb13009ff70 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.534+0000 7fb135909640 10 monclient(hunting): authenticate will time out at 1976.872995s 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1267fc640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fb13009fba0 0x7fb130098c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56418/0 (socket says 172.21.15.45:56418) 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb12effd640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fb13009f7d0 0x7fb130098570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36972/0 (socket says 172.21.15.45:36972) 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1267fc640 1 -- 172.21.15.45:0/2815889206 learned_addr learned my addr 172.21.15.45:0/2815889206 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb12f7fe640 1 --1- 172.21.15.45:0/2815889206 >> v1:172.21.15.45:6793/0 conn(0x7fb13009ff70 0x7fb130149bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34860/0 (socket says 172.21.15.45:34860) 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1282065771 0 0) 0x7fb1300a2560 con 0x7fb13009fba0 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): my global_id is 185298 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1100034e0 con 0x7fb13009fba0 2024-09-18T14:19:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771408582 0 0) 0x7fb13009a370 con 0x7fb13009f7d0 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): my global_id is 165424 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1300a2560 con 0x7fb13009f7d0 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831886889 0 0) 0x7fb13009d310 con 0x7fb13009ff70 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): my global_id is 234233 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb13009a370 con 0x7fb13009ff70 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1770884883 0 0) 0x7fb1100034e0 con 0x7fb13009fba0 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb13009d310 con 0x7fb13009fba0 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2800288049 0 0) 0x7fb13009a370 con 0x7fb13009ff70 2024-09-18T14:19:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb1100034e0 con 0x7fb13009ff70 2024-09-18T14:19:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 368448744 0 0) 0x7fb1300a2560 con 0x7fb13009f7d0 2024-09-18T14:19:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb13009a370 con 0x7fb13009f7d0 2024-09-18T14:19:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb118001650 con 0x7fb13009fba0 2024-09-18T14:19:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): dump: 2024-09-18T14:19:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb128001af0 con 0x7fb13009ff70 2024-09-18T14:19:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): dump: 2024-09-18T14:19:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb120001ec0 con 0x7fb13009f7d0 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 10 monclient(hunting): dump: 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.535+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2872387996 0 0) 0x7fb1100034e0 con 0x7fb13009ff70 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 >> v1:172.21.15.45:6792/0 conn(0x7fb13009fba0 legacy=0x7fb130098c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 >> v1:172.21.15.45:6791/0 conn(0x7fb13009f7d0 legacy=0x7fb130098570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 10 monclient: _finish_hunting 0 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 1 monclient: found mon.e 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13009be70 con 0x7fb13009ff70 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 10 monclient: _finish_auth 0 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb135909640 5 monclient: authenticate success, global_id 234233 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb135909640 10 monclient: _renew_subs 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb135909640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb135909640 1 -- 172.21.15.45:0/2815889206 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13012d360 con 0x7fb13009ff70 2024-09-18T14:19:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb135909640 10 monclient: _renew_subs 2024-09-18T14:19:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb135909640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb135909640 1 -- 172.21.15.45:0/2815889206 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13009c570 con 0x7fb13009ff70 2024-09-18T14:19:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb128001c50 con 0x7fb13009ff70 2024-09-18T14:19:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fb128004e20 con 0x7fb13009ff70 2024-09-18T14:19:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb12dffb640 4 set_mon_vals no callback set 2024-09-18T14:19:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 10 monclient: dump: 2024-09-18T14:19:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.536+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fb12800fdf0 con 0x7fb13009ff70 2024-09-18T14:19:21.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.537+0000 7fb135909640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:21.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.537+0000 7fb135909640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:21.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.537+0000 7fb135909640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:21.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.537+0000 7fb135909640 1 -- 172.21.15.45:0/2815889206 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb114005010 con 0x7fb13009ff70 2024-09-18T14:19:21.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.537+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7fb1280469e0 con 0x7fb13009ff70 2024-09-18T14:19:21.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.537+0000 7fb12effd640 10 monclient: get_auth_request con 0x7fb11003ab40 auth_method 0 2024-09-18T14:19:21.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.541+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fb128014f30 con 0x7fb13009ff70 2024-09-18T14:19:21.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.541+0000 7fb1277fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:21.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.541+0000 7fb1277fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.702+0000 7fb135909640 1 -- 172.21.15.45:0/2815889206 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb114000b90 con 0x7fb11003ab40 2024-09-18T14:19:21.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.705+0000 7fb1277fe640 1 -- 172.21.15.45:0/2815889206 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+53779 (unknown 2965378022 0 807510626) 0x7fb114000b90 con 0x7fb11003ab40 2024-09-18T14:19:21.705 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T14:19:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-18T14:19:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.706+0000 7fb135909640 1 -- 172.21.15.45:0/2815889206 >> v1:172.21.15.84:6812/31478 conn(0x7fb11003ab40 legacy=0x7fb11003cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.706+0000 7fb135909640 10 monclient: shutdown 2024-09-18T14:19:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.706+0000 7fb135909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.706+0000 7fb135909640 1 -- 172.21.15.45:0/2815889206 >> v1:172.21.15.45:6793/0 conn(0x7fb13009ff70 legacy=0x7fb130149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.706+0000 7fb135909640 1 -- 172.21.15.45:0/2815889206 shutdown_connections 2024-09-18T14:19:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.706+0000 7fb135909640 1 -- 172.21.15.45:0/2815889206 >> 172.21.15.45:0/2815889206 conn(0x7fb1300630b0 msgr2=0x7fb130130c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.707+0000 7fb135909640 1 -- 172.21.15.45:0/2815889206 shutdown_connections 2024-09-18T14:19:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.707+0000 7fb135909640 1 -- 172.21.15.45:0/2815889206 wait complete. 2024-09-18T14:19:21.717 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":1504,"stamp":"2024-09-18T14:19:20.143725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1822840,"kb_used_data":13744,"kb_used_omap":9,"kb_used_meta":1809014,"kb_avail":564408200,"statfs":{"total":579820584960,"available":577953996800,"internally_reserved":0,"allocated":14073856,"data_stored":8160502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1852430352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002586"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":886,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.440540+0000","last_change":"2024-09-18T13:59:05.035941+0000","last_active":"2024-09-18T14:18:24.440540+0000","last_peered":"2024-09-18T14:18:24.440540+0000","last_clean":"2024-09-18T14:18:24.440540+0000","last_became_active":"2024-09-18T13:59:03.705607+0000","last_became_peered":"2024-09-18T13:59:03.705607+0000","last_unstale":"2024-09-18T14:18:24.440540+0000","last_undegraded":"2024-09-18T14:18:24.440540+0000","last_fullsized":"2024-09-18T14:18:24.440540+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:25:38.392196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000366475,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":875,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.441537+0000","last_change":"2024-09-18T13:59:05.875836+0000","last_active":"2024-09-18T14:18:24.441537+0000","last_peered":"2024-09-18T14:18:24.441537+0000","last_clean":"2024-09-18T14:18:24.441537+0000","last_became_active":"2024-09-18T13:59:03.705710+0000","last_became_peered":"2024-09-18T13:59:03.705710+0000","last_unstale":"2024-09-18T14:18:24.441537+0000","last_undegraded":"2024-09-18T14:18:24.441537+0000","last_fullsized":"2024-09-18T14:18:24.441537+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:12:55.202686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029541399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":889,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.748247+0000","last_change":"2024-09-18T13:59:05.036440+0000","last_active":"2024-09-18T14:18:24.748247+0000","last_peered":"2024-09-18T14:18:24.748247+0000","last_clean":"2024-09-18T14:18:24.748247+0000","last_became_active":"2024-09-18T13:59:03.705659+0000","last_became_peered":"2024-09-18T13:59:03.705659+0000","last_unstale":"2024-09-18T14:18:24.748247+0000","last_undegraded":"2024-09-18T14:18:24.748247+0000","last_fullsized":"2024-09-18T14:18:24.748247+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:49:16.930535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037781799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":879,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.441619+0000","last_change":"2024-09-18T13:59:05.036690+0000","last_active":"2024-09-18T14:18:24.441619+0000","last_peered":"2024-09-18T14:18:24.441619+0000","last_clean":"2024-09-18T14:18:24.441619+0000","last_became_active":"2024-09-18T13:59:02.977312+0000","last_became_peered":"2024-09-18T13:59:02.977312+0000","last_unstale":"2024-09-18T14:18:24.441619+0000","last_undegraded":"2024-09-18T14:18:24.441619+0000","last_fullsized":"2024-09-18T14:18:24.441619+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:03:06.377092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072047099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'72","reported_seq":971,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.440454+0000","last_change":"2024-09-18T13:59:02.975019+0000","last_active":"2024-09-18T14:18:24.440454+0000","last_peered":"2024-09-18T14:18:24.440454+0000","last_clean":"2024-09-18T14:18:24.440454+0000","last_became_active":"2024-09-18T13:59:02.974164+0000","last_became_peered":"2024-09-18T13:59:02.974164+0000","last_unstale":"2024-09-18T14:18:24.440454+0000","last_undegraded":"2024-09-18T14:18:24.440454+0000","last_fullsized":"2024-09-18T14:18:24.440454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T22:10:24.958774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":878,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.441141+0000","last_change":"2024-09-18T13:59:05.036936+0000","last_active":"2024-09-18T14:18:24.441141+0000","last_peered":"2024-09-18T14:18:24.441141+0000","last_clean":"2024-09-18T14:18:24.441141+0000","last_became_active":"2024-09-18T13:59:02.977186+0000","last_became_peered":"2024-09-18T13:59:02.977186+0000","last_unstale":"2024-09-18T14:18:24.441141+0000","last_undegraded":"2024-09-18T14:18:24.441141+0000","last_fullsized":"2024-09-18T14:18:24.441141+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:01:46.516709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065897000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":887,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.748247+0000","last_change":"2024-09-18T13:59:05.036455+0000","last_active":"2024-09-18T14:18:24.748247+0000","last_peered":"2024-09-18T14:18:24.748247+0000","last_clean":"2024-09-18T14:18:24.748247+0000","last_became_active":"2024-09-18T13:59:02.974240+0000","last_became_peered":"2024-09-18T13:59:02.974240+0000","last_unstale":"2024-09-18T14:18:24.748247+0000","last_undegraded":"2024-09-18T14:18:24.748247+0000","last_fullsized":"2024-09-18T14:18:24.748247+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:13:40.312100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038595099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":886,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.440557+0000","last_change":"2024-09-18T13:59:05.035951+0000","last_active":"2024-09-18T14:18:24.440557+0000","last_peered":"2024-09-18T14:18:24.440557+0000","last_clean":"2024-09-18T14:18:24.440557+0000","last_became_active":"2024-09-18T13:59:03.705691+0000","last_became_peered":"2024-09-18T13:59:03.705691+0000","last_unstale":"2024-09-18T14:18:24.440557+0000","last_undegraded":"2024-09-18T14:18:24.440557+0000","last_fullsized":"2024-09-18T14:18:24.440557+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:45:16.896857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033710499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":875,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.443578+0000","last_change":"2024-09-18T13:59:05.875874+0000","last_active":"2024-09-18T14:18:24.443578+0000","last_peered":"2024-09-18T14:18:24.443578+0000","last_clean":"2024-09-18T14:18:24.443578+0000","last_became_active":"2024-09-18T13:59:02.976936+0000","last_became_peered":"2024-09-18T13:59:02.976936+0000","last_unstale":"2024-09-18T14:18:24.443578+0000","last_undegraded":"2024-09-18T14:18:24.443578+0000","last_fullsized":"2024-09-18T14:18:24.443578+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:20:37.890959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036210499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607798,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312472,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059368,"statfs":{"total":96636764160,"available":96316792832,"internally_reserved":0,"allocated":2699264,"data_stored":1712696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.35099999999999998,"15min":0.34999999999999998},"min":{"1min":0.28100000000000003,"5min":0.23300000000000001,"15min":0.19},"max":{"1min":0.441,"5min":0.54000000000000004,"15min":0.91700000000000004},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.35399999999999998,"15min":0.36499999999999999},"min":{"1min":0.29899999999999999,"5min":0.23999999999999999,"15min":0.219},"max":{"1min":0.46700000000000003,"5min":0.55700000000000005,"15min":1.0740000000000001},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.48699999999999999,"5min":0.53200000000000003,"15min":1.133},"last":0.371},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.71899999999999997},"last":0.33100000000000002}]},{"osd":3,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52900000000000003,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.747,"15min":0.871},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.64200000000000002,"5min":0.74299999999999999,"15min":1.034},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.50800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73799999999999999,"5min":0.76400000000000001,"15min":0.995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.58299999999999996},"min":{"1min":0.48899999999999999,"5min":0.35499999999999998,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.80000000000000004,"15min":1.323},"last":0.50900000000000001}]},{"osd":5,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.55800000000000005},"min":{"1min":0.44,"5min":0.378,"15min":0.33200000000000002},"max":{"1min":0.67500000000000004,"5min":0.77700000000000002,"15min":0.87},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.54300000000000004},"min":{"1min":0.41899999999999998,"5min":0.315,"15min":0.28000000000000003},"max":{"1min":0.75700000000000001,"5min":0.77200000000000002,"15min":1.2669999999999999},"last":0.42999999999999999}]}]},{"osd":5,"up_from":11,"seq":47244640501,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299472,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072368,"statfs":{"total":96636764160,"available":96330104832,"internally_reserved":0,"allocated":2166784,"data_stored":1183768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.53400000000000003},"min":{"1min":0.318,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":2.6800000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52300000000000002,"15min":0.54600000000000004},"min":{"1min":0.39900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":3.206},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47899999999999998,"15min":0.48799999999999999},"min":{"1min":0.36299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64800000000000002,"5min":1.335,"15min":1.335},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65300000000000002,"5min":0.69399999999999995,"15min":2.3610000000000002},"last":0.58499999999999996}]},{"osd":2,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48999999999999999,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65700000000000003,"5min":1.25,"15min":2.976},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":2.8719999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.35399999999999998},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.375},"min":{"1min":0.27300000000000002,"5min":0.224,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.629,"15min":0.69099999999999995},"last":0.375}]},{"osd":4,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.29399999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.68999999999999995,"5min":1.194,"15min":1.194},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.219},"max":{"1min":0.56200000000000006,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.32900000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640501,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307796,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064044,"statfs":{"total":96636764160,"available":96321581056,"internally_reserved":0,"allocated":2170880,"data_stored":1183787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.52400000000000002},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.183},"max":{"1min":0.66000000000000003,"5min":0.745,"15min":0.81799999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.26300000000000001},"max":{"1min":0.67200000000000004,"5min":0.80300000000000005,"15min":0.83299999999999996},"last":0.59099999999999997}]},{"osd":1,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46700000000000003,"15min":0.47499999999999998},"min":{"1min":0.33400000000000002,"5min":0.314,"15min":0.26500000000000001},"max":{"1min":0.59299999999999997,"5min":0.77300000000000002,"15min":0.79200000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.47799999999999998,"15min":0.48499999999999999},"min":{"1min":0.377,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.71399999999999997},"last":0.54500000000000004}]},{"osd":2,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.252,"15min":0.246},"max":{"1min":0.59899999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.32800000000000001,"5min":0.30399999999999999,"15min":0.28799999999999998},"max":{"1min":0.57899999999999996,"5min":0.68100000000000005,"15min":0.84599999999999997},"last":0.64200000000000002}]},{"osd":3,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.28100000000000003,"5min":0.22500000000000001,"15min":0.20499999999999999},"max":{"1min":0.58399999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.23699999999999999,"5min":0.23599999999999999,"15min":0.22},"max":{"1min":0.58799999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.36699999999999999}]},{"osd":5,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26000000000000001,"5min":0.20899999999999999,"15min":0.20499999999999999},"max":{"1min":0.50600000000000001,"5min":0.64600000000000002,"15min":0.753},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.38800000000000001}]}]},{"osd":3,"up_from":11,"seq":47244640501,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299992,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071848,"statfs":{"total":96636764160,"available":96329572352,"internally_reserved":0,"allocated":2699264,"data_stored":1712696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.31,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.624,"5min":0.66900000000000004,"15min":1.9530000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48999999999999999,"15min":0.47899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.26200000000000001},"max":{"1min":0.68000000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47899999999999998,"15min":0.47099999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.28100000000000003,"15min":0.27500000000000002},"max":{"1min":0.65500000000000003,"5min":0.66200000000000003,"15min":2.5190000000000001},"last":0.44}]},{"osd":2,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52100000000000002,"15min":0.52900000000000003},"min":{"1min":0.376,"5min":0.317,"15min":0.317},"max":{"1min":0.63,"5min":0.72999999999999998,"15min":2.3660000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.30099999999999999},"max":{"1min":0.61599999999999999,"5min":0.71899999999999997,"15min":2.3919999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35799999999999998},"min":{"1min":0.224,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.68899999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.372},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.20399999999999999},"max":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.63800000000000001},"last":0.497}]},{"osd":5,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.379,"15min":0.379},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.218},"max":{"1min":0.54500000000000004,"5min":0.58399999999999996,"15min":0.70499999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.435,"5min":0.41599999999999998,"15min":0.40799999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.748},"last":0.59299999999999997}]}]},{"osd":2,"up_from":11,"seq":47244640501,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299472,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072368,"statfs":{"total":96636764160,"available":96330104832,"internally_reserved":0,"allocated":2166784,"data_stored":1183768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32000000000000001,"15min":0.33100000000000002},"min":{"1min":0.22600000000000001,"5min":0.20399999999999999,"15min":0.20300000000000001},"max":{"1min":0.40899999999999997,"5min":0.53200000000000003,"15min":0.875},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.25,"5min":0.19,"15min":0.19},"max":{"1min":0.47899999999999998,"5min":1.1100000000000001,"15min":1.357},"last":0.316}]},{"osd":1,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.28100000000000003,"15min":0.25},"max":{"1min":0.56499999999999995,"5min":0.76700000000000002,"15min":4.117},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.379,"15min":0.39900000000000002},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.217},"max":{"1min":0.46500000000000002,"5min":0.629,"15min":4.0529999999999999},"last":0.33200000000000002}]},{"osd":3,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51100000000000001,"15min":0.52200000000000002},"min":{"1min":0.38300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.749,"5min":0.77200000000000002,"15min":1.3049999999999999},"last":0.502},{"interface":"front","average":{"1min":0.503,"5min":0.502,"15min":0.52500000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.29599999999999999},"max":{"1min":0.69799999999999995,"5min":1.026,"15min":1.506},"last":0.436}]},{"osd":4,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42199999999999999,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.86299999999999999,"5min":0.94399999999999995,"15min":1.2789999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.30499999999999999},"max":{"1min":0.77400000000000002,"5min":1.2669999999999999,"15min":2.3159999999999998},"last":0.52300000000000002}]},{"osd":5,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53000000000000003,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.35599999999999998,"15min":0.34200000000000003},"max":{"1min":0.92700000000000005,"5min":1.155,"15min":1.224},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.40100000000000002,"5min":0.35999999999999999,"15min":0.34599999999999997},"max":{"1min":0.69799999999999995,"5min":1.0409999999999999,"15min":1.2629999999999999},"last":0.56000000000000005}]}]},{"osd":0,"up_from":11,"seq":47244640501,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303636,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068204,"statfs":{"total":96636764160,"available":96325840896,"internally_reserved":0,"allocated":2170880,"data_stored":1183787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.34399999999999997},"min":{"1min":0.219,"5min":0.187,"15min":0.17199999999999999},"max":{"1min":0.495,"5min":0.53100000000000003,"15min":0.628},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.29799999999999999,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.49299999999999999,"5min":0.57699999999999996,"15min":0.72299999999999998},"last":0.41999999999999998}]},{"osd":2,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34100000000000003,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.629},"last":0.438},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.26300000000000001,"5min":0.25,"15min":0.224},"max":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.71499999999999997},"last":0.38800000000000001}]},{"osd":3,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.55300000000000005},"min":{"1min":0.41099999999999998,"5min":0.29599999999999999,"15min":0.28599999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.88200000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.54900000000000004},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.29899999999999999},"max":{"1min":0.64300000000000002,"5min":0.75600000000000001,"15min":0.88900000000000001},"last":0.66400000000000003}]},{"osd":4,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.39100000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.71299999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.58399999999999996},"min":{"1min":0.47499999999999998,"5min":0.36299999999999999,"15min":0.28000000000000003},"max":{"1min":0.747,"5min":0.79400000000000004,"15min":0.91700000000000004},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.29799999999999999,"15min":0.28699999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.84899999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.73199999999999998,"5min":0.84599999999999997,"15min":1.1479999999999999},"last":0.68600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T14:19:21.719 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T14:19:21.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.813+0000 7f2159d5d640 10 monclient: get_monmap_and_config 2024-09-18T14:19:21.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.814+0000 7f2159d5d640 10 monclient: build_initial_monmap 2024-09-18T14:19:21.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.814+0000 7f2159d5d640 10 monclient: monmap: 2024-09-18T14:19:21.814 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:21.814967+0000 2024-09-18T14:19:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:21.814967+0000 2024-09-18T14:19:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:21.816 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.814+0000 7f2159d5d640 1 Processor -- start 2024-09-18T14:19:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.814+0000 7f2159d5d640 1 -- start start 2024-09-18T14:19:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.814+0000 7f2159d5d640 10 monclient: init 2024-09-18T14:19:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 10 monclient: _add_conns ranks=[3,0,1,5,7,2,6,8,4] 2024-09-18T14:19:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 10 monclient(hunting): picked mon.noname-g con 0x7f215409f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 10 monclient(hunting): picked mon.noname-a con 0x7f215409fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 10 monclient(hunting): picked mon.noname-f con 0x7f215409ff70 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f215405ed70 con 0x7f215409fba0 2024-09-18T14:19:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f215409dfc0 con 0x7f215409ff70 2024-09-18T14:19:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f215409dce0 con 0x7f215409f7d0 2024-09-18T14:19:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 10 monclient(hunting): authenticate will time out at 1977.153342s 2024-09-18T14:19:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f214affd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f215409fba0 0x7f2154137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58906/0 (socket says 172.21.15.45:58906) 2024-09-18T14:19:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f214affd640 1 -- 172.21.15.45:0/559830671 learned_addr learned my addr 172.21.15.45:0/559830671 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 1 -- 172.21.15.45:0/559830671 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567519727 0 0) 0x7f215405ed70 con 0x7f215409fba0 2024-09-18T14:19:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 10 monclient(hunting): my global_id is 6723 2024-09-18T14:19:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 1 -- 172.21.15.45:0/559830671 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21400034e0 con 0x7f215409fba0 2024-09-18T14:19:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 1 -- 172.21.15.45:0/559830671 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3142477530 0 0) 0x7f21400034e0 con 0x7f215409fba0 2024-09-18T14:19:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 1 -- 172.21.15.45:0/559830671 >> v1:172.21.15.84:6790/0 conn(0x7f215409f7d0 legacy=0x7f2154132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 1 -- 172.21.15.45:0/559830671 >> v1:172.21.15.84:6789/0 conn(0x7f215409ff70 legacy=0x7f215413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 1 monclient: found mon.noname-a 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 1 -- 172.21.15.45:0/559830671 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21540a1c00 con 0x7f215409fba0 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 10 monclient: _finish_auth 0 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 1 -- 172.21.15.45:0/559830671 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f213c001830 con 0x7f215409fba0 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 5 monclient: authenticate success, global_id 6723 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:19:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2152ffd640 10 monclient: dump: 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.815+0000 7f2159d5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2152ffd640 1 -- 172.21.15.45:0/559830671 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f213c001a10 con 0x7f215409fba0 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2152ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2152ffd640 1 -- 172.21.15.45:0/559830671 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f213c004ba0 con 0x7f215409fba0 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2159d5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2159d5d640 4 set_mon_vals no callback set 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2152ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2152ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2152ffd640 10 monclient: dump: 2024-09-18T14:19:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2159d5d640 10 monclient: shutdown 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2159d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2159d5d640 1 -- 172.21.15.45:0/559830671 >> v1:172.21.15.45:6789/0 conn(0x7f215409fba0 legacy=0x7f2154137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2159d5d640 1 -- 172.21.15.45:0/559830671 shutdown_connections 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2159d5d640 1 -- 172.21.15.45:0/559830671 >> 172.21.15.45:0/559830671 conn(0x7f21540630b0 msgr2=0x7f21540634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2159d5d640 1 -- 172.21.15.45:0/559830671 shutdown_connections 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2159d5d640 1 -- 172.21.15.45:0/559830671 wait complete. 2024-09-18T14:19:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.816+0000 7f2159d5d640 10 monclient: build_initial_monmap 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 10 monclient: monmap: 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 1 Processor -- start 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 1 -- start start 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 10 monclient: init 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 10 monclient: _add_conns ranks=[6,8,3,2,4,1,0,5,7] 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 10 monclient(hunting): picked mon.noname-d con 0x7f215409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 10 monclient(hunting): picked mon.noname-e con 0x7f215409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 10 monclient(hunting): picked mon.noname-g con 0x7f215409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21540a2560 con 0x7f215409ff70 2024-09-18T14:19:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2154121780 con 0x7f215409f7d0 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2154120ab0 con 0x7f215409fba0 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.817+0000 7f2159d5d640 10 monclient(hunting): authenticate will time out at 1977.155965s 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f21537fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f215409f7d0 0x7f215411d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56424/0 (socket says 172.21.15.45:56424) 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f21537fe640 1 -- 172.21.15.45:0/4169847442 learned_addr learned my addr 172.21.15.45:0/4169847442 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758324602 0 0) 0x7f2154121780 con 0x7f215409f7d0 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): my global_id is 185307 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21340034e0 con 0x7f215409f7d0 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242840340 0 0) 0x7f2154120ab0 con 0x7f215409fba0 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): my global_id is 234242 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2154121780 con 0x7f215409fba0 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217428313 0 0) 0x7f21540a2560 con 0x7f215409ff70 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): my global_id is 224463 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2154120ab0 con 0x7f215409ff70 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3490968044 0 0) 0x7f21340034e0 con 0x7f215409f7d0 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21540a2560 con 0x7f215409f7d0 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2144001ba0 con 0x7f215409f7d0 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): dump: 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.835 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 310655909 0 0) 0x7f2154121780 con 0x7f215409fba0 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21340034e0 con 0x7f215409fba0 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2785432210 0 0) 0x7f2154120ab0 con 0x7f215409ff70 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2154121780 con 0x7f215409ff70 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f213c001770 con 0x7f215409fba0 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): dump: 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f214c001ba0 con 0x7f215409ff70 2024-09-18T14:19:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient(hunting): dump: 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2456011194 0 0) 0x7f21540a2560 con 0x7f215409f7d0 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 >> v1:172.21.15.45:6793/0 conn(0x7f215409fba0 legacy=0x7f215411d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 >> v1:172.21.15.84:6790/0 conn(0x7f215409ff70 legacy=0x7f2154149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 monclient: found mon.d 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215411eed0 con 0x7f215409f7d0 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 10 monclient: _finish_auth 0 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f214bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f2159d5d640 5 monclient: authenticate success, global_id 185307 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f2159d5d640 10 monclient: _renew_subs 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f2159d5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f2159d5d640 1 -- 172.21.15.45:0/4169847442 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215411f590 con 0x7f215409f7d0 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f2159d5d640 10 monclient: _renew_subs 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f2159d5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.818+0000 7f2159d5d640 1 -- 172.21.15.45:0/4169847442 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f215411e9c0 con 0x7f215409f7d0 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.819+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f21440021b0 con 0x7f215409f7d0 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.819+0000 7f214bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.819+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f21440026a0 con 0x7f215409f7d0 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.819+0000 7f214bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.819+0000 7f21527fc640 4 set_mon_vals no callback set 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.819+0000 7f214bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.819+0000 7f214bfff640 10 monclient: dump: 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:21.842 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.819+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f2144002880 con 0x7f215409f7d0 2024-09-18T14:19:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.820+0000 7f2159d5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.820+0000 7f2159d5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.820+0000 7f2159d5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.820+0000 7f2159d5d640 1 -- 172.21.15.45:0/4169847442 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2138005010 con 0x7f215409f7d0 2024-09-18T14:19:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.820+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f2144046a40 con 0x7f215409f7d0 2024-09-18T14:19:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.820+0000 7f214affd640 10 monclient: get_auth_request con 0x7f213403ab30 auth_method 0 2024-09-18T14:19:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.824+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f2144014f30 con 0x7f215409f7d0 2024-09-18T14:19:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.824+0000 7f214bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.824+0000 7f214bfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:21.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.988+0000 7f2159d5d640 1 -- 172.21.15.45:0/4169847442 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2138000b90 con 0x7f213403ab30 2024-09-18T14:19:21.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.990+0000 7f214bfff640 1 -- 172.21.15.45:0/4169847442 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+53779 (unknown 2965378022 0 807510626) 0x7f213c007e90 con 0x7f213403ab30 2024-09-18T14:19:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T14:19:21.992 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-18T14:19:21.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.992+0000 7f2159d5d640 1 -- 172.21.15.45:0/4169847442 >> v1:172.21.15.84:6812/31478 conn(0x7f213403ab30 legacy=0x7f213403cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.992+0000 7f2159d5d640 10 monclient: shutdown 2024-09-18T14:19:21.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.992+0000 7f2159d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:21.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.992+0000 7f2159d5d640 1 -- 172.21.15.45:0/4169847442 >> v1:172.21.15.45:6792/0 conn(0x7f215409f7d0 legacy=0x7f215411d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:21.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.992+0000 7f2159d5d640 1 -- 172.21.15.45:0/4169847442 shutdown_connections 2024-09-18T14:19:21.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.992+0000 7f2159d5d640 1 -- 172.21.15.45:0/4169847442 >> 172.21.15.45:0/4169847442 conn(0x7f21540630b0 msgr2=0x7f21540a4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:21.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.992+0000 7f2159d5d640 1 -- 172.21.15.45:0/4169847442 shutdown_connections 2024-09-18T14:19:21.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:21.992+0000 7f2159d5d640 1 -- 172.21.15.45:0/4169847442 wait complete. 2024-09-18T14:19:22.003 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":1504,"stamp":"2024-09-18T14:19:20.143725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1822840,"kb_used_data":13744,"kb_used_omap":9,"kb_used_meta":1809014,"kb_avail":564408200,"statfs":{"total":579820584960,"available":577953996800,"internally_reserved":0,"allocated":14073856,"data_stored":8160502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1852430352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002586"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":886,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.440540+0000","last_change":"2024-09-18T13:59:05.035941+0000","last_active":"2024-09-18T14:18:24.440540+0000","last_peered":"2024-09-18T14:18:24.440540+0000","last_clean":"2024-09-18T14:18:24.440540+0000","last_became_active":"2024-09-18T13:59:03.705607+0000","last_became_peered":"2024-09-18T13:59:03.705607+0000","last_unstale":"2024-09-18T14:18:24.440540+0000","last_undegraded":"2024-09-18T14:18:24.440540+0000","last_fullsized":"2024-09-18T14:18:24.440540+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:25:38.392196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000366475,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":875,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.441537+0000","last_change":"2024-09-18T13:59:05.875836+0000","last_active":"2024-09-18T14:18:24.441537+0000","last_peered":"2024-09-18T14:18:24.441537+0000","last_clean":"2024-09-18T14:18:24.441537+0000","last_became_active":"2024-09-18T13:59:03.705710+0000","last_became_peered":"2024-09-18T13:59:03.705710+0000","last_unstale":"2024-09-18T14:18:24.441537+0000","last_undegraded":"2024-09-18T14:18:24.441537+0000","last_fullsized":"2024-09-18T14:18:24.441537+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:12:55.202686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029541399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":889,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.748247+0000","last_change":"2024-09-18T13:59:05.036440+0000","last_active":"2024-09-18T14:18:24.748247+0000","last_peered":"2024-09-18T14:18:24.748247+0000","last_clean":"2024-09-18T14:18:24.748247+0000","last_became_active":"2024-09-18T13:59:03.705659+0000","last_became_peered":"2024-09-18T13:59:03.705659+0000","last_unstale":"2024-09-18T14:18:24.748247+0000","last_undegraded":"2024-09-18T14:18:24.748247+0000","last_fullsized":"2024-09-18T14:18:24.748247+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:49:16.930535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037781799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":879,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.441619+0000","last_change":"2024-09-18T13:59:05.036690+0000","last_active":"2024-09-18T14:18:24.441619+0000","last_peered":"2024-09-18T14:18:24.441619+0000","last_clean":"2024-09-18T14:18:24.441619+0000","last_became_active":"2024-09-18T13:59:02.977312+0000","last_became_peered":"2024-09-18T13:59:02.977312+0000","last_unstale":"2024-09-18T14:18:24.441619+0000","last_undegraded":"2024-09-18T14:18:24.441619+0000","last_fullsized":"2024-09-18T14:18:24.441619+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:03:06.377092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072047099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'72","reported_seq":971,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.440454+0000","last_change":"2024-09-18T13:59:02.975019+0000","last_active":"2024-09-18T14:18:24.440454+0000","last_peered":"2024-09-18T14:18:24.440454+0000","last_clean":"2024-09-18T14:18:24.440454+0000","last_became_active":"2024-09-18T13:59:02.974164+0000","last_became_peered":"2024-09-18T13:59:02.974164+0000","last_unstale":"2024-09-18T14:18:24.440454+0000","last_undegraded":"2024-09-18T14:18:24.440454+0000","last_fullsized":"2024-09-18T14:18:24.440454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T22:10:24.958774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":878,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.441141+0000","last_change":"2024-09-18T13:59:05.036936+0000","last_active":"2024-09-18T14:18:24.441141+0000","last_peered":"2024-09-18T14:18:24.441141+0000","last_clean":"2024-09-18T14:18:24.441141+0000","last_became_active":"2024-09-18T13:59:02.977186+0000","last_became_peered":"2024-09-18T13:59:02.977186+0000","last_unstale":"2024-09-18T14:18:24.441141+0000","last_undegraded":"2024-09-18T14:18:24.441141+0000","last_fullsized":"2024-09-18T14:18:24.441141+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:01:46.516709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065897000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":887,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.748247+0000","last_change":"2024-09-18T13:59:05.036455+0000","last_active":"2024-09-18T14:18:24.748247+0000","last_peered":"2024-09-18T14:18:24.748247+0000","last_clean":"2024-09-18T14:18:24.748247+0000","last_became_active":"2024-09-18T13:59:02.974240+0000","last_became_peered":"2024-09-18T13:59:02.974240+0000","last_unstale":"2024-09-18T14:18:24.748247+0000","last_undegraded":"2024-09-18T14:18:24.748247+0000","last_fullsized":"2024-09-18T14:18:24.748247+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:13:40.312100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038595099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":886,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.440557+0000","last_change":"2024-09-18T13:59:05.035951+0000","last_active":"2024-09-18T14:18:24.440557+0000","last_peered":"2024-09-18T14:18:24.440557+0000","last_clean":"2024-09-18T14:18:24.440557+0000","last_became_active":"2024-09-18T13:59:03.705691+0000","last_became_peered":"2024-09-18T13:59:03.705691+0000","last_unstale":"2024-09-18T14:18:24.440557+0000","last_undegraded":"2024-09-18T14:18:24.440557+0000","last_fullsized":"2024-09-18T14:18:24.440557+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:45:16.896857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033710499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":875,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.443578+0000","last_change":"2024-09-18T13:59:05.875874+0000","last_active":"2024-09-18T14:18:24.443578+0000","last_peered":"2024-09-18T14:18:24.443578+0000","last_clean":"2024-09-18T14:18:24.443578+0000","last_became_active":"2024-09-18T13:59:02.976936+0000","last_became_peered":"2024-09-18T13:59:02.976936+0000","last_unstale":"2024-09-18T14:18:24.443578+0000","last_undegraded":"2024-09-18T14:18:24.443578+0000","last_fullsized":"2024-09-18T14:18:24.443578+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:20:37.890959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036210499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607798,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312472,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059368,"statfs":{"total":96636764160,"available":96316792832,"internally_reserved":0,"allocated":2699264,"data_stored":1712696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.35099999999999998,"15min":0.34999999999999998},"min":{"1min":0.28100000000000003,"5min":0.23300000000000001,"15min":0.19},"max":{"1min":0.441,"5min":0.54000000000000004,"15min":0.91700000000000004},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.35399999999999998,"15min":0.36499999999999999},"min":{"1min":0.29899999999999999,"5min":0.23999999999999999,"15min":0.219},"max":{"1min":0.46700000000000003,"5min":0.55700000000000005,"15min":1.0740000000000001},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.48699999999999999,"5min":0.53200000000000003,"15min":1.133},"last":0.371},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.71899999999999997},"last":0.33100000000000002}]},{"osd":3,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52900000000000003,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.747,"15min":0.871},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.64200000000000002,"5min":0.74299999999999999,"15min":1.034},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.50800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73799999999999999,"5min":0.76400000000000001,"15min":0.995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.58299999999999996},"min":{"1min":0.48899999999999999,"5min":0.35499999999999998,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.80000000000000004,"15min":1.323},"last":0.50900000000000001}]},{"osd":5,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.55800000000000005},"min":{"1min":0.44,"5min":0.378,"15min":0.33200000000000002},"max":{"1min":0.67500000000000004,"5min":0.77700000000000002,"15min":0.87},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.54300000000000004},"min":{"1min":0.41899999999999998,"5min":0.315,"15min":0.28000000000000003},"max":{"1min":0.75700000000000001,"5min":0.77200000000000002,"15min":1.2669999999999999},"last":0.42999999999999999}]}]},{"osd":5,"up_from":11,"seq":47244640501,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299472,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072368,"statfs":{"total":96636764160,"available":96330104832,"internally_reserved":0,"allocated":2166784,"data_stored":1183768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.53400000000000003},"min":{"1min":0.318,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":2.6800000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52300000000000002,"15min":0.54600000000000004},"min":{"1min":0.39900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":3.206},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47899999999999998,"15min":0.48799999999999999},"min":{"1min":0.36299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64800000000000002,"5min":1.335,"15min":1.335},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65300000000000002,"5min":0.69399999999999995,"15min":2.3610000000000002},"last":0.58499999999999996}]},{"osd":2,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48999999999999999,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65700000000000003,"5min":1.25,"15min":2.976},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":2.8719999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.35399999999999998},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.375},"min":{"1min":0.27300000000000002,"5min":0.224,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.629,"15min":0.69099999999999995},"last":0.375}]},{"osd":4,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.29399999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.68999999999999995,"5min":1.194,"15min":1.194},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.219},"max":{"1min":0.56200000000000006,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.32900000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640501,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307796,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064044,"statfs":{"total":96636764160,"available":96321581056,"internally_reserved":0,"allocated":2170880,"data_stored":1183787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.52400000000000002},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.183},"max":{"1min":0.66000000000000003,"5min":0.745,"15min":0.81799999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.26300000000000001},"max":{"1min":0.67200000000000004,"5min":0.80300000000000005,"15min":0.83299999999999996},"last":0.59099999999999997}]},{"osd":1,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46700000000000003,"15min":0.47499999999999998},"min":{"1min":0.33400000000000002,"5min":0.314,"15min":0.26500000000000001},"max":{"1min":0.59299999999999997,"5min":0.77300000000000002,"15min":0.79200000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.47799999999999998,"15min":0.48499999999999999},"min":{"1min":0.377,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.71399999999999997},"last":0.54500000000000004}]},{"osd":2,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.252,"15min":0.246},"max":{"1min":0.59899999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.32800000000000001,"5min":0.30399999999999999,"15min":0.28799999999999998},"max":{"1min":0.57899999999999996,"5min":0.68100000000000005,"15min":0.84599999999999997},"last":0.64200000000000002}]},{"osd":3,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.28100000000000003,"5min":0.22500000000000001,"15min":0.20499999999999999},"max":{"1min":0.58399999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.23699999999999999,"5min":0.23599999999999999,"15min":0.22},"max":{"1min":0.58799999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.36699999999999999}]},{"osd":5,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26000000000000001,"5min":0.20899999999999999,"15min":0.20499999999999999},"max":{"1min":0.50600000000000001,"5min":0.64600000000000002,"15min":0.753},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.38800000000000001}]}]},{"osd":3,"up_from":11,"seq":47244640501,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299992,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071848,"statfs":{"total":96636764160,"available":96329572352,"internally_reserved":0,"allocated":2699264,"data_stored":1712696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.31,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.624,"5min":0.66900000000000004,"15min":1.9530000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48999999999999999,"15min":0.47899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.26200000000000001},"max":{"1min":0.68000000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47899999999999998,"15min":0.47099999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.28100000000000003,"15min":0.27500000000000002},"max":{"1min":0.65500000000000003,"5min":0.66200000000000003,"15min":2.5190000000000001},"last":0.44}]},{"osd":2,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52100000000000002,"15min":0.52900000000000003},"min":{"1min":0.376,"5min":0.317,"15min":0.317},"max":{"1min":0.63,"5min":0.72999999999999998,"15min":2.3660000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.30099999999999999},"max":{"1min":0.61599999999999999,"5min":0.71899999999999997,"15min":2.3919999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35799999999999998},"min":{"1min":0.224,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.68899999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.372},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.20399999999999999},"max":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.63800000000000001},"last":0.497}]},{"osd":5,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.379,"15min":0.379},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.218},"max":{"1min":0.54500000000000004,"5min":0.58399999999999996,"15min":0.70499999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.435,"5min":0.41599999999999998,"15min":0.40799999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.748},"last":0.59299999999999997}]}]},{"osd":2,"up_from":11,"seq":47244640501,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299472,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072368,"statfs":{"total":96636764160,"available":96330104832,"internally_reserved":0,"allocated":2166784,"data_stored":1183768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32000000000000001,"15min":0.33100000000000002},"min":{"1min":0.22600000000000001,"5min":0.20399999999999999,"15min":0.20300000000000001},"max":{"1min":0.40899999999999997,"5min":0.53200000000000003,"15min":0.875},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.25,"5min":0.19,"15min":0.19},"max":{"1min":0.47899999999999998,"5min":1.1100000000000001,"15min":1.357},"last":0.316}]},{"osd":1,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.28100000000000003,"15min":0.25},"max":{"1min":0.56499999999999995,"5min":0.76700000000000002,"15min":4.117},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.379,"15min":0.39900000000000002},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.217},"max":{"1min":0.46500000000000002,"5min":0.629,"15min":4.0529999999999999},"last":0.33200000000000002}]},{"osd":3,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51100000000000001,"15min":0.52200000000000002},"min":{"1min":0.38300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.749,"5min":0.77200000000000002,"15min":1.3049999999999999},"last":0.502},{"interface":"front","average":{"1min":0.503,"5min":0.502,"15min":0.52500000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.29599999999999999},"max":{"1min":0.69799999999999995,"5min":1.026,"15min":1.506},"last":0.436}]},{"osd":4,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42199999999999999,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.86299999999999999,"5min":0.94399999999999995,"15min":1.2789999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.30499999999999999},"max":{"1min":0.77400000000000002,"5min":1.2669999999999999,"15min":2.3159999999999998},"last":0.52300000000000002}]},{"osd":5,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53000000000000003,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.35599999999999998,"15min":0.34200000000000003},"max":{"1min":0.92700000000000005,"5min":1.155,"15min":1.224},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.40100000000000002,"5min":0.35999999999999999,"15min":0.34599999999999997},"max":{"1min":0.69799999999999995,"5min":1.0409999999999999,"15min":1.2629999999999999},"last":0.56000000000000005}]}]},{"osd":0,"up_from":11,"seq":47244640501,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303636,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068204,"statfs":{"total":96636764160,"available":96325840896,"internally_reserved":0,"allocated":2170880,"data_stored":1183787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.34399999999999997},"min":{"1min":0.219,"5min":0.187,"15min":0.17199999999999999},"max":{"1min":0.495,"5min":0.53100000000000003,"15min":0.628},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.29799999999999999,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.49299999999999999,"5min":0.57699999999999996,"15min":0.72299999999999998},"last":0.41999999999999998}]},{"osd":2,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34100000000000003,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.629},"last":0.438},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.26300000000000001,"5min":0.25,"15min":0.224},"max":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.71499999999999997},"last":0.38800000000000001}]},{"osd":3,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.55300000000000005},"min":{"1min":0.41099999999999998,"5min":0.29599999999999999,"15min":0.28599999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.88200000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.54900000000000004},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.29899999999999999},"max":{"1min":0.64300000000000002,"5min":0.75600000000000001,"15min":0.88900000000000001},"last":0.66400000000000003}]},{"osd":4,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.39100000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.71299999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.58399999999999996},"min":{"1min":0.47499999999999998,"5min":0.36299999999999999,"15min":0.28000000000000003},"max":{"1min":0.747,"5min":0.79400000000000004,"15min":0.91700000000000004},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.29799999999999999,"15min":0.28699999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.84899999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.73199999999999998,"5min":0.84599999999999997,"15min":1.1479999999999999},"last":0.68600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T14:19:22.004 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T14:19:22.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T14:19:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.099+0000 7f27da4c0640 10 monclient: get_monmap_and_config 2024-09-18T14:19:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.099+0000 7f27da4c0640 10 monclient: build_initial_monmap 2024-09-18T14:19:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.099+0000 7f27da4c0640 10 monclient: monmap: 2024-09-18T14:19:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:22.100592+0000 2024-09-18T14:19:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:22.100592+0000 2024-09-18T14:19:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.100+0000 7f27da4c0640 1 Processor -- start 2024-09-18T14:19:22.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.100+0000 7f27da4c0640 1 -- start start 2024-09-18T14:19:22.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.100+0000 7f27da4c0640 10 monclient: init 2024-09-18T14:19:22.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.100+0000 7f27da4c0640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.100+0000 7f27da4c0640 10 monclient: _add_conns ranks=[4,6,3,8,5,2,0,7,1] 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.100+0000 7f27da4c0640 10 monclient(hunting): picked mon.noname-c con 0x7f27d409f7d0 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.100+0000 7f27da4c0640 10 monclient(hunting): picked mon.noname-d con 0x7f27d409fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.100+0000 7f27da4c0640 10 monclient(hunting): picked mon.noname-g con 0x7f27d409ff70 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27da4c0640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d405ed70 con 0x7f27d409ff70 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27da4c0640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d409dfc0 con 0x7f27d409f7d0 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27da4c0640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d409dce0 con 0x7f27d409fba0 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27da4c0640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27da4c0640 10 monclient(hunting): authenticate will time out at 1977.439059s 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d37fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f27d409fba0 0x7f27d4137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56430/0 (socket says 172.21.15.45:56430) 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d3fff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f27d409f7d0 0x7f27d4132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36978/0 (socket says 172.21.15.45:36978) 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d37fe640 1 -- 172.21.15.45:0/2479226675 learned_addr learned my addr 172.21.15.45:0/2479226675 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214330986 0 0) 0x7f27d409dfc0 con 0x7f27d409f7d0 2024-09-18T14:19:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient(hunting): my global_id is 165433 2024-09-18T14:19:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27bc0034e0 con 0x7f27d409f7d0 2024-09-18T14:19:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507139518 0 0) 0x7f27d409dce0 con 0x7f27d409fba0 2024-09-18T14:19:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient(hunting): my global_id is 185316 2024-09-18T14:19:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27d409dfc0 con 0x7f27d409fba0 2024-09-18T14:19:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89085268 0 0) 0x7f27d405ed70 con 0x7f27d409ff70 2024-09-18T14:19:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient(hunting): my global_id is 224472 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27d409dce0 con 0x7f27d409ff70 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2575543630 0 0) 0x7f27bc0034e0 con 0x7f27d409f7d0 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 >> v1:172.21.15.45:6792/0 conn(0x7f27d409fba0 legacy=0x7f27d4137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 >> v1:172.21.15.84:6790/0 conn(0x7f27d409ff70 legacy=0x7f27d413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 monclient: found mon.noname-c 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d40a1c00 con 0x7f27d409f7d0 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient: _finish_auth 0 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f27c4001830 con 0x7f27d409f7d0 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient: dump: 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27da4c0640 5 monclient: authenticate success, global_id 165433 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27da4c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f27c4001990 con 0x7f27d409f7d0 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 1 -- 172.21.15.45:0/2479226675 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f27c4004ae0 con 0x7f27d409f7d0 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27da4c0640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27da4c0640 4 set_mon_vals no callback set 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.101+0000 7f27d2ffd640 10 monclient: dump: 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.102+0000 7f27da4c0640 10 monclient: shutdown 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.102+0000 7f27da4c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.102+0000 7f27da4c0640 1 -- 172.21.15.45:0/2479226675 >> v1:172.21.15.45:6791/0 conn(0x7f27d409f7d0 legacy=0x7f27d4132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.102+0000 7f27da4c0640 1 -- 172.21.15.45:0/2479226675 shutdown_connections 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.102+0000 7f27da4c0640 1 -- 172.21.15.45:0/2479226675 >> 172.21.15.45:0/2479226675 conn(0x7f27d40630b0 msgr2=0x7f27d40634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.102+0000 7f27da4c0640 1 -- 172.21.15.45:0/2479226675 shutdown_connections 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.102+0000 7f27da4c0640 1 -- 172.21.15.45:0/2479226675 wait complete. 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.102+0000 7f27da4c0640 10 monclient: build_initial_monmap 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.102+0000 7f27da4c0640 10 monclient: monmap: 2024-09-18T14:19:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:22.118 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 1 Processor -- start 2024-09-18T14:19:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 1 -- start start 2024-09-18T14:19:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 10 monclient: init 2024-09-18T14:19:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 10 monclient: _add_conns ranks=[8,3,5,7,6,1,2,4,0] 2024-09-18T14:19:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 10 monclient(hunting): picked mon.noname-e con 0x7f27d409f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 10 monclient(hunting): picked mon.noname-g con 0x7f27d409fba0 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 10 monclient(hunting): picked mon.noname-h con 0x7f27d409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d40a2560 con 0x7f27d409fba0 2024-09-18T14:19:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d4120830 con 0x7f27d409ff70 2024-09-18T14:19:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d4121df0 con 0x7f27d409f7d0 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27da4c0640 10 monclient(hunting): authenticate will time out at 1977.441939s 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.103+0000 7f27d3fff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f27d409f7d0 0x7f27d41417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34882/0 (socket says 172.21.15.45:34882) 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27d3fff640 1 -- 172.21.15.45:0/625089461 learned_addr learned my addr 172.21.15.45:0/625089461 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107355293 0 0) 0x7f27d4121df0 con 0x7f27d409f7d0 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): my global_id is 234251 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c00034e0 con 0x7f27d409f7d0 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641282230 0 0) 0x7f27d40a2560 con 0x7f27d409fba0 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): my global_id is 224481 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27d4121df0 con 0x7f27d409fba0 2024-09-18T14:19:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826752431 0 0) 0x7f27d4120830 con 0x7f27d409ff70 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): my global_id is 185144 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27d40a2560 con 0x7f27d409ff70 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 573985087 0 0) 0x7f27c00034e0 con 0x7f27d409f7d0 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27d4120830 con 0x7f27d409f7d0 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f27c4001430 con 0x7f27d409f7d0 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): dump: 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1921604970 0 0) 0x7f27d4121df0 con 0x7f27d409fba0 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27c00034e0 con 0x7f27d409fba0 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f27c80013a0 con 0x7f27d409fba0 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): dump: 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 939439434 0 0) 0x7f27d40a2560 con 0x7f27d409ff70 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27d4121df0 con 0x7f27d409ff70 2024-09-18T14:19:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f27cc002c70 con 0x7f27d409ff70 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient(hunting): dump: 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2526854297 0 0) 0x7f27d4120830 con 0x7f27d409f7d0 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 >> v1:172.21.15.84:6791/0 conn(0x7f27d409ff70 legacy=0x7f27d4149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 >> v1:172.21.15.84:6790/0 conn(0x7f27d409fba0 legacy=0x7f27d41319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 monclient: found mon.e 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d4142170 con 0x7f27d409f7d0 2024-09-18T14:19:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient: _finish_auth 0 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27da4c0640 5 monclient: authenticate success, global_id 234251 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27da4c0640 10 monclient: _renew_subs 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27da4c0640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27da4c0640 1 -- 172.21.15.45:0/625089461 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d4141eb0 con 0x7f27d409f7d0 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27da4c0640 10 monclient: _renew_subs 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27da4c0640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27da4c0640 1 -- 172.21.15.45:0/625089461 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d4121b10 con 0x7f27d409f7d0 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f27c4004d60 con 0x7f27d409f7d0 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f27c4005210 con 0x7f27d409f7d0 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27d27fc640 4 set_mon_vals no callback set 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.104+0000 7f27bbfff640 10 monclient: dump: 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:22.128 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.105+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f27c40101e0 con 0x7f27d409f7d0 2024-09-18T14:19:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.105+0000 7f27da4c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.105+0000 7f27da4c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.105+0000 7f27da4c0640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.105+0000 7f27da4c0640 1 -- 172.21.15.45:0/625089461 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b4005010 con 0x7f27d409f7d0 2024-09-18T14:19:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.105+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f27c40469e0 con 0x7f27d409f7d0 2024-09-18T14:19:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.106+0000 7f27d37fe640 10 monclient: get_auth_request con 0x7f27c003aab0 auth_method 0 2024-09-18T14:19:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.110+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f27c4014f90 con 0x7f27d409f7d0 2024-09-18T14:19:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.110+0000 7f27bbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.110+0000 7f27bbfff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.270+0000 7f27da4c0640 1 -- 172.21.15.45:0/625089461 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f27b4000b90 con 0x7f27c003aab0 2024-09-18T14:19:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.272+0000 7f27bbfff640 1 -- 172.21.15.45:0/625089461 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+53724 (unknown 2965378022 0 423014698) 0x7f27b4000b90 con 0x7f27c003aab0 2024-09-18T14:19:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T14:19:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-18T14:19:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.274+0000 7f27da4c0640 1 -- 172.21.15.45:0/625089461 >> v1:172.21.15.84:6812/31478 conn(0x7f27c003aab0 legacy=0x7f27c003cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.274+0000 7f27da4c0640 10 monclient: shutdown 2024-09-18T14:19:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.274+0000 7f27da4c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.274+0000 7f27da4c0640 1 -- 172.21.15.45:0/625089461 >> v1:172.21.15.45:6793/0 conn(0x7f27d409f7d0 legacy=0x7f27d41417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.274+0000 7f27da4c0640 1 -- 172.21.15.45:0/625089461 shutdown_connections 2024-09-18T14:19:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.274+0000 7f27da4c0640 1 -- 172.21.15.45:0/625089461 >> 172.21.15.45:0/625089461 conn(0x7f27d40630b0 msgr2=0x7f27d40a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.274+0000 7f27da4c0640 1 -- 172.21.15.45:0/625089461 shutdown_connections 2024-09-18T14:19:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.274+0000 7f27da4c0640 1 -- 172.21.15.45:0/625089461 wait complete. 2024-09-18T14:19:22.284 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":1505,"stamp":"2024-09-18T14:19:22.144020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1822840,"kb_used_data":13744,"kb_used_omap":9,"kb_used_meta":1809014,"kb_avail":564408200,"statfs":{"total":579820584960,"available":577953996800,"internally_reserved":0,"allocated":14073856,"data_stored":8160502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1852430352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002529"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":886,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.440540+0000","last_change":"2024-09-18T13:59:05.035941+0000","last_active":"2024-09-18T14:18:24.440540+0000","last_peered":"2024-09-18T14:18:24.440540+0000","last_clean":"2024-09-18T14:18:24.440540+0000","last_became_active":"2024-09-18T13:59:03.705607+0000","last_became_peered":"2024-09-18T13:59:03.705607+0000","last_unstale":"2024-09-18T14:18:24.440540+0000","last_undegraded":"2024-09-18T14:18:24.440540+0000","last_fullsized":"2024-09-18T14:18:24.440540+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:25:38.392196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000366475,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":875,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.441537+0000","last_change":"2024-09-18T13:59:05.875836+0000","last_active":"2024-09-18T14:18:24.441537+0000","last_peered":"2024-09-18T14:18:24.441537+0000","last_clean":"2024-09-18T14:18:24.441537+0000","last_became_active":"2024-09-18T13:59:03.705710+0000","last_became_peered":"2024-09-18T13:59:03.705710+0000","last_unstale":"2024-09-18T14:18:24.441537+0000","last_undegraded":"2024-09-18T14:18:24.441537+0000","last_fullsized":"2024-09-18T14:18:24.441537+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:12:55.202686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029541399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":889,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.748247+0000","last_change":"2024-09-18T13:59:05.036440+0000","last_active":"2024-09-18T14:18:24.748247+0000","last_peered":"2024-09-18T14:18:24.748247+0000","last_clean":"2024-09-18T14:18:24.748247+0000","last_became_active":"2024-09-18T13:59:03.705659+0000","last_became_peered":"2024-09-18T13:59:03.705659+0000","last_unstale":"2024-09-18T14:18:24.748247+0000","last_undegraded":"2024-09-18T14:18:24.748247+0000","last_fullsized":"2024-09-18T14:18:24.748247+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:49:16.930535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037781799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":879,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.441619+0000","last_change":"2024-09-18T13:59:05.036690+0000","last_active":"2024-09-18T14:18:24.441619+0000","last_peered":"2024-09-18T14:18:24.441619+0000","last_clean":"2024-09-18T14:18:24.441619+0000","last_became_active":"2024-09-18T13:59:02.977312+0000","last_became_peered":"2024-09-18T13:59:02.977312+0000","last_unstale":"2024-09-18T14:18:24.441619+0000","last_undegraded":"2024-09-18T14:18:24.441619+0000","last_fullsized":"2024-09-18T14:18:24.441619+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:03:06.377092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072047099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'72","reported_seq":971,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.440454+0000","last_change":"2024-09-18T13:59:02.975019+0000","last_active":"2024-09-18T14:18:24.440454+0000","last_peered":"2024-09-18T14:18:24.440454+0000","last_clean":"2024-09-18T14:18:24.440454+0000","last_became_active":"2024-09-18T13:59:02.974164+0000","last_became_peered":"2024-09-18T13:59:02.974164+0000","last_unstale":"2024-09-18T14:18:24.440454+0000","last_undegraded":"2024-09-18T14:18:24.440454+0000","last_fullsized":"2024-09-18T14:18:24.440454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T22:10:24.958774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":878,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.441141+0000","last_change":"2024-09-18T13:59:05.036936+0000","last_active":"2024-09-18T14:18:24.441141+0000","last_peered":"2024-09-18T14:18:24.441141+0000","last_clean":"2024-09-18T14:18:24.441141+0000","last_became_active":"2024-09-18T13:59:02.977186+0000","last_became_peered":"2024-09-18T13:59:02.977186+0000","last_unstale":"2024-09-18T14:18:24.441141+0000","last_undegraded":"2024-09-18T14:18:24.441141+0000","last_fullsized":"2024-09-18T14:18:24.441141+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:01:46.516709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065897000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":887,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.748247+0000","last_change":"2024-09-18T13:59:05.036455+0000","last_active":"2024-09-18T14:18:24.748247+0000","last_peered":"2024-09-18T14:18:24.748247+0000","last_clean":"2024-09-18T14:18:24.748247+0000","last_became_active":"2024-09-18T13:59:02.974240+0000","last_became_peered":"2024-09-18T13:59:02.974240+0000","last_unstale":"2024-09-18T14:18:24.748247+0000","last_undegraded":"2024-09-18T14:18:24.748247+0000","last_fullsized":"2024-09-18T14:18:24.748247+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:13:40.312100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038595099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":886,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.440557+0000","last_change":"2024-09-18T13:59:05.035951+0000","last_active":"2024-09-18T14:18:24.440557+0000","last_peered":"2024-09-18T14:18:24.440557+0000","last_clean":"2024-09-18T14:18:24.440557+0000","last_became_active":"2024-09-18T13:59:03.705691+0000","last_became_peered":"2024-09-18T13:59:03.705691+0000","last_unstale":"2024-09-18T14:18:24.440557+0000","last_undegraded":"2024-09-18T14:18:24.440557+0000","last_fullsized":"2024-09-18T14:18:24.440557+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:45:16.896857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033710499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":875,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.443578+0000","last_change":"2024-09-18T13:59:05.875874+0000","last_active":"2024-09-18T14:18:24.443578+0000","last_peered":"2024-09-18T14:18:24.443578+0000","last_clean":"2024-09-18T14:18:24.443578+0000","last_became_active":"2024-09-18T13:59:02.976936+0000","last_became_peered":"2024-09-18T13:59:02.976936+0000","last_unstale":"2024-09-18T14:18:24.443578+0000","last_undegraded":"2024-09-18T14:18:24.443578+0000","last_fullsized":"2024-09-18T14:18:24.443578+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:20:37.890959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036210499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607799,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312472,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059368,"statfs":{"total":96636764160,"available":96316792832,"internally_reserved":0,"allocated":2699264,"data_stored":1712696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.35099999999999998,"15min":0.34999999999999998},"min":{"1min":0.28100000000000003,"5min":0.23300000000000001,"15min":0.19},"max":{"1min":0.441,"5min":0.54000000000000004,"15min":0.91700000000000004},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.35399999999999998,"15min":0.36499999999999999},"min":{"1min":0.29899999999999999,"5min":0.23999999999999999,"15min":0.219},"max":{"1min":0.46700000000000003,"5min":0.55700000000000005,"15min":1.0740000000000001},"last":0.28799999999999998}]},{"osd":2,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.48699999999999999,"5min":0.53200000000000003,"15min":1.133},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.71899999999999997},"last":0.30399999999999999}]},{"osd":3,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52900000000000003,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.747,"15min":0.871},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.64200000000000002,"5min":0.74299999999999999,"15min":1.034},"last":0.375}]},{"osd":4,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.50800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73799999999999999,"5min":0.76400000000000001,"15min":0.995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.58299999999999996},"min":{"1min":0.48899999999999999,"5min":0.35499999999999998,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.80000000000000004,"15min":1.323},"last":0.42099999999999999}]},{"osd":5,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.55800000000000005},"min":{"1min":0.44,"5min":0.378,"15min":0.33200000000000002},"max":{"1min":0.67500000000000004,"5min":0.77700000000000002,"15min":0.87},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.54300000000000004},"min":{"1min":0.41899999999999998,"5min":0.315,"15min":0.28000000000000003},"max":{"1min":0.75700000000000001,"5min":0.77200000000000002,"15min":1.2669999999999999},"last":0.44400000000000001}]}]},{"osd":5,"up_from":11,"seq":47244640502,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299472,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072368,"statfs":{"total":96636764160,"available":96330104832,"internally_reserved":0,"allocated":2166784,"data_stored":1183768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.53400000000000003},"min":{"1min":0.318,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":2.6800000000000002},"last":0.505},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52300000000000002,"15min":0.54600000000000004},"min":{"1min":0.39900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":3.206},"last":0.58099999999999996}]},{"osd":1,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47899999999999998,"15min":0.48799999999999999},"min":{"1min":0.36299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64800000000000002,"5min":1.335,"15min":1.335},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65300000000000002,"5min":0.69399999999999995,"15min":2.3610000000000002},"last":0.55900000000000005}]},{"osd":2,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48999999999999999,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65700000000000003,"5min":1.25,"15min":2.976},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":2.8719999999999999},"last":0.54400000000000004}]},{"osd":3,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.35399999999999998},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.375},"min":{"1min":0.27300000000000002,"5min":0.224,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.629,"15min":0.69099999999999995},"last":0.46899999999999997}]},{"osd":4,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.29399999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.68999999999999995,"5min":1.194,"15min":1.194},"last":0.434},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.219},"max":{"1min":0.56200000000000006,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.53000000000000003}]}]},{"osd":4,"up_from":11,"seq":47244640502,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307796,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064044,"statfs":{"total":96636764160,"available":96321581056,"internally_reserved":0,"allocated":2170880,"data_stored":1183787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.52400000000000002},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.183},"max":{"1min":0.66000000000000003,"5min":0.745,"15min":0.81799999999999995},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.26300000000000001},"max":{"1min":0.67200000000000004,"5min":0.80300000000000005,"15min":0.83299999999999996},"last":0.54200000000000004}]},{"osd":1,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46700000000000003,"15min":0.47499999999999998},"min":{"1min":0.33400000000000002,"5min":0.314,"15min":0.26500000000000001},"max":{"1min":0.59299999999999997,"5min":0.77300000000000002,"15min":0.79200000000000004},"last":0.443},{"interface":"front","average":{"1min":0.498,"5min":0.47799999999999998,"15min":0.48499999999999999},"min":{"1min":0.377,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.71399999999999997},"last":0.50900000000000001}]},{"osd":2,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.252,"15min":0.246},"max":{"1min":0.59899999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.32800000000000001,"5min":0.30399999999999999,"15min":0.28799999999999998},"max":{"1min":0.57899999999999996,"5min":0.68100000000000005,"15min":0.84599999999999997},"last":0.42299999999999999}]},{"osd":3,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.28100000000000003,"5min":0.22500000000000001,"15min":0.20499999999999999},"max":{"1min":0.58399999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.23699999999999999,"5min":0.23599999999999999,"15min":0.22},"max":{"1min":0.58799999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39200000000000002}]},{"osd":5,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26000000000000001,"5min":0.20899999999999999,"15min":0.20499999999999999},"max":{"1min":0.50600000000000001,"5min":0.64600000000000002,"15min":0.753},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.52900000000000003}]}]},{"osd":3,"up_from":11,"seq":47244640502,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299992,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071848,"statfs":{"total":96636764160,"available":96329572352,"internally_reserved":0,"allocated":2699264,"data_stored":1712696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.31,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.624,"5min":0.66900000000000004,"15min":1.9530000000000001},"last":0.434},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48999999999999999,"15min":0.47899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.26200000000000001},"max":{"1min":0.68000000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.309}]},{"osd":1,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47899999999999998,"15min":0.47099999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.376},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.28100000000000003,"15min":0.27500000000000002},"max":{"1min":0.65500000000000003,"5min":0.66200000000000003,"15min":2.5190000000000001},"last":0.39500000000000002}]},{"osd":2,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52100000000000002,"15min":0.52900000000000003},"min":{"1min":0.376,"5min":0.317,"15min":0.317},"max":{"1min":0.63,"5min":0.72999999999999998,"15min":2.3660000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.30099999999999999},"max":{"1min":0.61599999999999999,"5min":0.71899999999999997,"15min":2.3919999999999999},"last":0.32400000000000001}]},{"osd":4,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35799999999999998},"min":{"1min":0.224,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.68899999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.372},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.20399999999999999},"max":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.63800000000000001},"last":0.46600000000000003}]},{"osd":5,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.379,"15min":0.379},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.218},"max":{"1min":0.54500000000000004,"5min":0.58399999999999996,"15min":0.70499999999999996},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.41599999999999998,"15min":0.40799999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.748},"last":0.501}]}]},{"osd":2,"up_from":11,"seq":47244640502,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299472,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072368,"statfs":{"total":96636764160,"available":96330104832,"internally_reserved":0,"allocated":2166784,"data_stored":1183768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32000000000000001,"15min":0.33100000000000002},"min":{"1min":0.22600000000000001,"5min":0.20399999999999999,"15min":0.20300000000000001},"max":{"1min":0.40899999999999997,"5min":0.53200000000000003,"15min":0.875},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.25,"5min":0.19,"15min":0.19},"max":{"1min":0.47899999999999998,"5min":1.1100000000000001,"15min":1.357},"last":0.33500000000000002}]},{"osd":1,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.28100000000000003,"15min":0.25},"max":{"1min":0.56499999999999995,"5min":0.76700000000000002,"15min":4.117},"last":0.379},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.379,"15min":0.39900000000000002},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.217},"max":{"1min":0.46500000000000002,"5min":0.629,"15min":4.0529999999999999},"last":0.35699999999999998}]},{"osd":3,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51100000000000001,"15min":0.52200000000000002},"min":{"1min":0.38300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.749,"5min":0.77200000000000002,"15min":1.3049999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.502,"15min":0.52500000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.29599999999999999},"max":{"1min":0.69799999999999995,"5min":1.026,"15min":1.506},"last":0.52600000000000002}]},{"osd":4,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42199999999999999,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.86299999999999999,"5min":0.94399999999999995,"15min":1.2789999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.30499999999999999},"max":{"1min":0.77400000000000002,"5min":1.2669999999999999,"15min":2.3159999999999998},"last":0.56699999999999995}]},{"osd":5,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53000000000000003,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.35599999999999998,"15min":0.34200000000000003},"max":{"1min":0.92700000000000005,"5min":1.155,"15min":1.224},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.40100000000000002,"5min":0.35999999999999999,"15min":0.34599999999999997},"max":{"1min":0.69799999999999995,"5min":1.0409999999999999,"15min":1.2629999999999999},"last":0.51100000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640502,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303636,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068204,"statfs":{"total":96636764160,"available":96325840896,"internally_reserved":0,"allocated":2170880,"data_stored":1183787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.34399999999999997},"min":{"1min":0.219,"5min":0.187,"15min":0.17199999999999999},"max":{"1min":0.495,"5min":0.53100000000000003,"15min":0.628},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.29799999999999999,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.49299999999999999,"5min":0.57699999999999996,"15min":0.72299999999999998},"last":0.30199999999999999}]},{"osd":2,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34100000000000003,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.629},"last":0.246},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.26300000000000001,"5min":0.25,"15min":0.224},"max":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.71499999999999997},"last":0.26800000000000002}]},{"osd":3,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.55300000000000005},"min":{"1min":0.41099999999999998,"5min":0.29599999999999999,"15min":0.28599999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.88200000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.54900000000000004},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.29899999999999999},"max":{"1min":0.64300000000000002,"5min":0.75600000000000001,"15min":0.88900000000000001},"last":0.35799999999999998}]},{"osd":4,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.39100000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.71299999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.58399999999999996},"min":{"1min":0.47499999999999998,"5min":0.36299999999999999,"15min":0.28000000000000003},"max":{"1min":0.747,"5min":0.79400000000000004,"15min":0.91700000000000004},"last":0.38400000000000001}]},{"osd":5,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.29799999999999999,"15min":0.28699999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.84899999999999998},"last":0.375},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.73199999999999998,"5min":0.84599999999999997,"15min":1.1479999999999999},"last":0.42099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T14:19:22.286 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T14:19:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.381+0000 7f4bcfcf8640 10 monclient: get_monmap_and_config 2024-09-18T14:19:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.381+0000 7f4bcfcf8640 10 monclient: build_initial_monmap 2024-09-18T14:19:22.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.381+0000 7f4bcfcf8640 10 monclient: monmap: 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:22.382002+0000 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:22.382002+0000 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:22.382 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:22.383 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:22.383 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.381+0000 7f4bcfcf8640 1 Processor -- start 2024-09-18T14:19:22.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.381+0000 7f4bcfcf8640 1 -- start start 2024-09-18T14:19:22.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.381+0000 7f4bcfcf8640 10 monclient: init 2024-09-18T14:19:22.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcfcf8640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:22.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcfcf8640 10 monclient: _add_conns ranks=[1,7,5,2,6,3,0,4,8] 2024-09-18T14:19:22.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcfcf8640 10 monclient(hunting): picked mon.noname-f con 0x7f4bc806e810 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcfcf8640 10 monclient(hunting): picked mon.noname-i con 0x7f4bc80a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcfcf8640 10 monclient(hunting): picked mon.noname-h con 0x7f4bc80a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcfcf8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc805ed70 con 0x7f4bc806e810 2024-09-18T14:19:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcfcf8640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc8097620 con 0x7f4bc80a1080 2024-09-18T14:19:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcfcf8640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc8097340 con 0x7f4bc80a0cb0 2024-09-18T14:19:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcfcf8640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcfcf8640 10 monclient(hunting): authenticate will time out at 1977.720351s 2024-09-18T14:19:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcd293640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7f4bc80a0cb0 0x7f4bc8137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:59926/0 (socket says 172.21.15.45:59926) 2024-09-18T14:19:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bce295640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f4bc80a1080 0x7f4bc813af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:60640/0 (socket says 172.21.15.45:60640) 2024-09-18T14:19:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcd293640 1 -- 172.21.15.45:0/779529719 learned_addr learned my addr 172.21.15.45:0/779529719 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcda94640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4bc806e810 0x7f4bc80a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.45:40564/0 (socket says 172.21.15.45:40564) 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.382+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261797878 0 0) 0x7f4bc805ed70 con 0x7f4bc806e810 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient(hunting): my global_id is 214750 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bb40034e0 con 0x7f4bc806e810 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214088946 0 0) 0x7f4bc8097340 con 0x7f4bc80a0cb0 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient(hunting): my global_id is 214693 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc805ed70 con 0x7f4bc80a0cb0 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826931701 0 0) 0x7f4bc8097620 con 0x7f4bc80a1080 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient(hunting): my global_id is 185153 2024-09-18T14:19:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc8097340 con 0x7f4bc80a1080 2024-09-18T14:19:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3439666851 0 0) 0x7f4bb40034e0 con 0x7f4bc806e810 2024-09-18T14:19:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 >> v1:172.21.15.84:6792/0 conn(0x7f4bc80a0cb0 legacy=0x7f4bc8137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 >> v1:172.21.15.84:6791/0 conn(0x7f4bc80a1080 legacy=0x7f4bc813af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient: _finish_hunting 0 2024-09-18T14:19:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 monclient: found mon.noname-f 2024-09-18T14:19:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.84:6789/0 2024-09-18T14:19:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc80a1660 con 0x7f4bc806e810 2024-09-18T14:19:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient: _finish_auth 0 2024-09-18T14:19:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:22.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcfcf8640 5 monclient: authenticate success, global_id 214750 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcfcf8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4bc4001760 con 0x7f4bc806e810 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient: dump: 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:22.396 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcfcf8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 <== mon.1 v1:172.21.15.84:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4bc4001940 con 0x7f4bc806e810 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcfcf8640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 1 -- 172.21.15.45:0/779529719 <== mon.1 v1:172.21.15.84:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4bc40047a0 con 0x7f4bc806e810 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcfcf8640 4 set_mon_vals no callback set 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcca92640 10 monclient: dump: 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcfcf8640 10 monclient: shutdown 2024-09-18T14:19:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcfcf8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/779529719 >> v1:172.21.15.84:6789/0 conn(0x7f4bc806e810 legacy=0x7f4bc80a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/779529719 shutdown_connections 2024-09-18T14:19:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.383+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/779529719 >> 172.21.15.45:0/779529719 conn(0x7f4bc80a5bb0 msgr2=0x7f4bc80a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.384+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/779529719 shutdown_connections 2024-09-18T14:19:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.384+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/779529719 wait complete. 2024-09-18T14:19:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.384+0000 7f4bcfcf8640 10 monclient: build_initial_monmap 2024-09-18T14:19:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.384+0000 7f4bcfcf8640 10 monclient: monmap: 2024-09-18T14:19:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:22.400 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.384+0000 7f4bcfcf8640 1 Processor -- start 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.384+0000 7f4bcfcf8640 1 -- start start 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.384+0000 7f4bcfcf8640 10 monclient: init 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bcfcf8640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bcfcf8640 10 monclient: _add_conns ranks=[6,1,3,0,7,4,2,5,8] 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bcfcf8640 10 monclient(hunting): picked mon.noname-d con 0x7f4bc806e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bcfcf8640 10 monclient(hunting): picked mon.noname-f con 0x7f4bc80a0cb0 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bcfcf8640 10 monclient(hunting): picked mon.noname-g con 0x7f4bc80a1080 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bcfcf8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc8122d00 con 0x7f4bc80a0cb0 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bcfcf8640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc813daf0 con 0x7f4bc80a1080 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bcfcf8640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc8122740 con 0x7f4bc806e810 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bcfcf8640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bcfcf8640 10 monclient(hunting): authenticate will time out at 1977.723359s 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bcda94640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f4bc806e810 0x7f4bc813d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56432/0 (socket says 172.21.15.45:56432) 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bcda94640 1 -- 172.21.15.45:0/2472867979 learned_addr learned my addr 172.21.15.45:0/2472867979 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341293623 0 0) 0x7f4bc8122740 con 0x7f4bc806e810 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): my global_id is 185325 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bac0034e0 con 0x7f4bc806e810 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135913350 0 0) 0x7f4bc8122d00 con 0x7f4bc80a0cb0 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): my global_id is 214759 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc8122740 con 0x7f4bc80a0cb0 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300668655 0 0) 0x7f4bc813daf0 con 0x7f4bc80a1080 2024-09-18T14:19:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): my global_id is 224490 2024-09-18T14:19:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc8122d00 con 0x7f4bc80a1080 2024-09-18T14:19:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 704991808 0 0) 0x7f4bac0034e0 con 0x7f4bc806e810 2024-09-18T14:19:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4bc813daf0 con 0x7f4bc806e810 2024-09-18T14:19:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4bc4001720 con 0x7f4bc806e810 2024-09-18T14:19:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): dump: 2024-09-18T14:19:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.385+0000 7f4bbdffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1596189209 0 0) 0x7f4bc8122740 con 0x7f4bc80a0cb0 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4bac0034e0 con 0x7f4bc80a0cb0 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4bb8001550 con 0x7f4bc80a0cb0 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient(hunting): dump: 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3649952908 0 0) 0x7f4bc8122d00 con 0x7f4bc80a1080 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4bc8122740 con 0x7f4bc80a1080 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4bc0001b10 con 0x7f4bc80a1080 2024-09-18T14:19:22.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient(hunting): dump: 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3670509849 0 0) 0x7f4bc813daf0 con 0x7f4bc806e810 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 >> v1:172.21.15.84:6790/0 conn(0x7f4bc80a1080 legacy=0x7f4bc8149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 >> v1:172.21.15.84:6789/0 conn(0x7f4bc80a0cb0 legacy=0x7f4bc8146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient: _finish_hunting 0 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 monclient: found mon.d 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8120200 con 0x7f4bc806e810 2024-09-18T14:19:22.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient: _finish_auth 0 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bcfcf8640 5 monclient: authenticate success, global_id 185325 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bcfcf8640 10 monclient: _renew_subs 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bcfcf8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/2472867979 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc8122460 con 0x7f4bc806e810 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4bc4004cd0 con 0x7f4bc806e810 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4bc4005180 con 0x7f4bc806e810 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbdffb640 10 monclient: dump: 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:22.409 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:22.410 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:22.411 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:22.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bbffff640 4 set_mon_vals no callback set 2024-09-18T14:19:22.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bcfcf8640 10 monclient: _renew_subs 2024-09-18T14:19:22.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bcfcf8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:22.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.386+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/2472867979 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc8121590 con 0x7f4bc806e810 2024-09-18T14:19:22.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.387+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4bc40101d0 con 0x7f4bc806e810 2024-09-18T14:19:22.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.387+0000 7f4bcfcf8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:22.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.387+0000 7f4bcfcf8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:22.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.387+0000 7f4bcfcf8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:22.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.387+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/2472867979 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc804c4a0 con 0x7f4bc806e810 2024-09-18T14:19:22.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.387+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f4bc4046b30 con 0x7f4bc806e810 2024-09-18T14:19:22.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.387+0000 7f4bcd293640 10 monclient: get_auth_request con 0x7f4bac03aa30 auth_method 0 2024-09-18T14:19:22.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.392+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f4bc4015560 con 0x7f4bc806e810 2024-09-18T14:19:22.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.392+0000 7f4bbdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:22.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.392+0000 7f4bbdffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:22.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.556+0000 7f4bcfcf8640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-18T14:19:22.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.557+0000 7f4bcfcf8640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T14:19:22.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.557+0000 7f4bcfcf8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:22.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.557+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/2472867979 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bc804b720 con 0x7f4bc806e810 2024-09-18T14:19:22.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.557+0000 7f4bbdffb640 1 -- 172.21.15.45:0/2472867979 <== mon.6 v1:172.21.15.45:6792/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v899) v1 ==== 74+0+9977 (unknown 3789296979 0 2988026738) 0x7f4bc4014f60 con 0x7f4bc806e810 2024-09-18T14:19:22.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.557+0000 7f4bbdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-18T14:19:22.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.557+0000 7f4bbdffb640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T14:19:22.559 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":899,"fsid":"0f5ca4bb-86c8-4c54-b818-324eb902ab17","created":"2024-09-18T13:58:52.168851+0000","modified":"2024-09-18T14:18:28.377860+0000","last_up_change":"2024-09-18T13:59:00.910314+0000","last_in_change":"2024-09-18T13:58:53.737759+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T13:59:01.925362+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-18T13:59:01.954982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"8c3a8909-07ce-4141-98dd-131951f18ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6804","nonce":32276}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6805","nonce":32276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6807","nonce":32276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6806","nonce":32276}]},"public_addr":"172.21.15.45:6804/32276","cluster_addr":"172.21.15.45:6805/32276","heartbeat_back_addr":"172.21.15.45:6807/32276","heartbeat_front_addr":"172.21.15.45:6806/32276","state":["exists","up"]},{"osd":1,"uuid":"a12b3bc3-1e03-47dc-98c1-c8bd199551a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6800","nonce":32277}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6801","nonce":32277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6803","nonce":32277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6802","nonce":32277}]},"public_addr":"172.21.15.45:6800/32277","cluster_addr":"172.21.15.45:6801/32277","heartbeat_back_addr":"172.21.15.45:6803/32277","heartbeat_front_addr":"172.21.15.45:6802/32277","state":["exists","up"]},{"osd":2,"uuid":"216b8b62-e41c-4bc5-9395-f2d56d402b1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6808","nonce":32278}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6809","nonce":32278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6811","nonce":32278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6810","nonce":32278}]},"public_addr":"172.21.15.45:6808/32278","cluster_addr":"172.21.15.45:6809/32278","heartbeat_back_addr":"172.21.15.45:6811/32278","heartbeat_front_addr":"172.21.15.45:6810/32278","state":["exists","up"]},{"osd":3,"uuid":"c750b0e6-9143-489d-9be0-eb085178ff2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6802","nonce":32024}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6804","nonce":32024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6807","nonce":32024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6806","nonce":32024}]},"public_addr":"172.21.15.84:6802/32024","cluster_addr":"172.21.15.84:6804/32024","heartbeat_back_addr":"172.21.15.84:6807/32024","heartbeat_front_addr":"172.21.15.84:6806/32024","state":["exists","up"]},{"osd":4,"uuid":"9680908d-dc81-44a0-8603-93711579d42e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6800","nonce":32025}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6801","nonce":32025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6805","nonce":32025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6803","nonce":32025}]},"public_addr":"172.21.15.84:6800/32025","cluster_addr":"172.21.15.84:6801/32025","heartbeat_back_addr":"172.21.15.84:6805/32025","heartbeat_front_addr":"172.21.15.84:6803/32025","state":["exists","up"]},{"osd":5,"uuid":"b3b63eef-c8fc-495f-960f-7436315119ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6808","nonce":32048}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6809","nonce":32048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6811","nonce":32048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.84:6810","nonce":32048}]},"public_addr":"172.21.15.84:6808/32048","cluster_addr":"172.21.15.84:6809/32048","heartbeat_back_addr":"172.21.15.84:6811/32048","heartbeat_front_addr":"172.21.15.84:6810/32048","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.823226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.883512+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.904155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.801736+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.835588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T13:58:58.910413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T14:19:22.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.559+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/2472867979 >> v1:172.21.15.84:6812/31478 conn(0x7f4bac03aa30 legacy=0x7f4bac03cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.559+0000 7f4bcfcf8640 10 monclient: shutdown 2024-09-18T14:19:22.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.559+0000 7f4bcfcf8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:22.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.559+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/2472867979 >> v1:172.21.15.45:6792/0 conn(0x7f4bc806e810 legacy=0x7f4bc813d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:22.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.559+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/2472867979 shutdown_connections 2024-09-18T14:19:22.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.559+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/2472867979 >> 172.21.15.45:0/2472867979 conn(0x7f4bc80a5bb0 msgr2=0x7f4bc80a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:22.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.559+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/2472867979 shutdown_connections 2024-09-18T14:19:22.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:22.559+0000 7f4bcfcf8640 1 -- 172.21.15.45:0/2472867979 wait complete. 2024-09-18T14:19:23.570 INFO:tasks.ceph:Scrubbing osd.0 2024-09-18T14:19:23.571 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-18T14:19:23.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.675+0000 7f13202ef640 10 monclient: get_monmap_and_config 2024-09-18T14:19:23.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.675+0000 7f13202ef640 10 monclient: build_initial_monmap 2024-09-18T14:19:23.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.675+0000 7f13202ef640 10 monclient: monmap: 2024-09-18T14:19:23.676 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:23.676 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.676 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:23.676154+0000 2024-09-18T14:19:23.677 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:23.676154+0000 2024-09-18T14:19:23.677 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:23.677 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.677 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:23.677 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:23.677 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:23.677 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:23.677 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:23.677 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:23.677 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:23.677 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:23.678 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:23.678 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.675+0000 7f13202ef640 1 Processor -- start 2024-09-18T14:19:23.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.675+0000 7f13202ef640 1 -- start start 2024-09-18T14:19:23.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.675+0000 7f13202ef640 10 monclient: init 2024-09-18T14:19:23.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f13202ef640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:23.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f13202ef640 10 monclient: _add_conns ranks=[7,4,5,3,2,6,8,1,0] 2024-09-18T14:19:23.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f13202ef640 10 monclient(hunting): picked mon.noname-i con 0x7f131806e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:23.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f13202ef640 10 monclient(hunting): picked mon.noname-c con 0x7f1318096a20 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:23.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f13202ef640 10 monclient(hunting): picked mon.noname-h con 0x7f1318138f10 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f13202ef640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131805ed70 con 0x7f1318096a20 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f13202ef640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13180a1b50 con 0x7f1318138f10 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f13202ef640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131806ebe0 con 0x7f131806e810 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f13202ef640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f13202ef640 10 monclient(hunting): authenticate will time out at 1979.014524s 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f131d88a640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f1318096a20 0x7f13181377f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36988/0 (socket says 172.21.15.45:36988) 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f131d88a640 1 -- 172.21.15.45:0/53404710 learned_addr learned my addr 172.21.15.45:0/53404710 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827694325 0 0) 0x7f131805ed70 con 0x7f1318096a20 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f131d089640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f131d089640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f131d089640 10 monclient(hunting): my global_id is 165442 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.676+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13000034e0 con 0x7f1318096a20 2024-09-18T14:19:23.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975852026 0 0) 0x7f13180a1b50 con 0x7f1318138f10 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient(hunting): my global_id is 185162 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f131805ed70 con 0x7f1318138f10 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576819115 0 0) 0x7f131806ebe0 con 0x7f131806e810 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient(hunting): my global_id is 214702 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13180a1b50 con 0x7f131806e810 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1642854925 0 0) 0x7f13000034e0 con 0x7f1318096a20 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 >> v1:172.21.15.84:6792/0 conn(0x7f131806e810 legacy=0x7f131809ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 >> v1:172.21.15.84:6791/0 conn(0x7f1318138f10 legacy=0x7f131813b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient: _finish_hunting 0 2024-09-18T14:19:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 1 monclient: found mon.noname-c 2024-09-18T14:19:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:19:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318132990 con 0x7f1318096a20 2024-09-18T14:19:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient: _finish_auth 0 2024-09-18T14:19:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f130c001830 con 0x7f1318096a20 2024-09-18T14:19:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f13202ef640 5 monclient: authenticate success, global_id 165442 2024-09-18T14:19:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f13202ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:19:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient: dump: 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:23.691 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:23.691 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f13202ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:23.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f130c001a10 con 0x7f1318096a20 2024-09-18T14:19:23.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:23.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 1 -- 172.21.15.45:0/53404710 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f130c004ba0 con 0x7f1318096a20 2024-09-18T14:19:23.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f13202ef640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:23.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f13202ef640 4 set_mon_vals no callback set 2024-09-18T14:19:23.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:23.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f131d089640 10 monclient: dump: 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:23.692 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:23.693 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:23.693 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:23.693 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:23.693 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f13202ef640 10 monclient: shutdown 2024-09-18T14:19:23.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f13202ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:23.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f13202ef640 1 -- 172.21.15.45:0/53404710 >> v1:172.21.15.45:6791/0 conn(0x7f1318096a20 legacy=0x7f13181377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f13202ef640 1 -- 172.21.15.45:0/53404710 shutdown_connections 2024-09-18T14:19:23.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f13202ef640 1 -- 172.21.15.45:0/53404710 >> 172.21.15.45:0/53404710 conn(0x7f13180a5c20 msgr2=0x7f13180a6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:23.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.677+0000 7f13202ef640 1 -- 172.21.15.45:0/53404710 shutdown_connections 2024-09-18T14:19:23.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.678+0000 7f13202ef640 1 -- 172.21.15.45:0/53404710 wait complete. 2024-09-18T14:19:23.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.678+0000 7f13202ef640 10 monclient: build_initial_monmap 2024-09-18T14:19:23.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.678+0000 7f13202ef640 10 monclient: monmap: 2024-09-18T14:19:23.694 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:23.694 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:23.694 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:23.694 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:23.694 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:23.694 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:23.694 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:23.694 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:23.694 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:23.694 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.678+0000 7f13202ef640 1 Processor -- start 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.678+0000 7f13202ef640 1 -- start start 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.678+0000 7f13202ef640 10 monclient: init 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13202ef640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13202ef640 10 monclient: _add_conns ranks=[4,3,8,7,0,6,2,1,5] 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13202ef640 10 monclient(hunting): picked mon.noname-c con 0x7f131806e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:23.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13202ef640 10 monclient(hunting): picked mon.noname-g con 0x7f1318096a20 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:23.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13202ef640 10 monclient(hunting): picked mon.noname-e con 0x7f1318138f10 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:23.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13202ef640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131809a530 con 0x7f1318096a20 2024-09-18T14:19:23.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13202ef640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13180995d0 con 0x7f131806e810 2024-09-18T14:19:23.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13202ef640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131809a810 con 0x7f1318138f10 2024-09-18T14:19:23.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13202ef640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:23.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13202ef640 10 monclient(hunting): authenticate will time out at 1979.017336s 2024-09-18T14:19:23.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f131e08b640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f131806e810 0x7f131809e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:36992/0 (socket says 172.21.15.45:36992) 2024-09-18T14:19:23.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f131e08b640 1 -- 172.21.15.45:0/2029075877 learned_addr learned my addr 172.21.15.45:0/2029075877 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:23.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761006327 0 0) 0x7f13180995d0 con 0x7f131806e810 2024-09-18T14:19:23.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:23.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): my global_id is 165451 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13040034e0 con 0x7f131806e810 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237375908 0 0) 0x7f131809a530 con 0x7f1318096a20 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): my global_id is 224499 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13180995d0 con 0x7f1318096a20 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001924044 0 0) 0x7f131809a810 con 0x7f1318138f10 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): my global_id is 234260 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f131809a530 con 0x7f1318138f10 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2037931990 0 0) 0x7f13040034e0 con 0x7f131806e810 2024-09-18T14:19:23.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f131809a810 con 0x7f131806e810 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1308001ef0 con 0x7f131806e810 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): dump: 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:23.698 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1358859126 0 0) 0x7f13180995d0 con 0x7f1318096a20 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13040034e0 con 0x7f1318096a20 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f130c004ba0 con 0x7f1318096a20 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:23.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 10 monclient(hunting): dump: 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:23.700 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3926717987 0 0) 0x7f131809a530 con 0x7f1318138f10 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.679+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13180995d0 con 0x7f1318138f10 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f13100018c0 con 0x7f1318138f10 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 10 monclient(hunting): dump: 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.701 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:23.703 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 990300139 0 0) 0x7f131809a810 con 0x7f131806e810 2024-09-18T14:19:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 >> v1:172.21.15.45:6793/0 conn(0x7f1318138f10 legacy=0x7f131814a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 >> v1:172.21.15.84:6790/0 conn(0x7f1318096a20 legacy=0x7f1318146980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 1 monclient: found mon.c 2024-09-18T14:19:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131809aaf0 con 0x7f131806e810 2024-09-18T14:19:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13202ef640 5 monclient: authenticate success, global_id 165451 2024-09-18T14:19:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13202ef640 10 monclient: _renew_subs 2024-09-18T14:19:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13202ef640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13202ef640 1 -- 172.21.15.45:0/2029075877 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131809ea30 con 0x7f131806e810 2024-09-18T14:19:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13202ef640 10 monclient: _renew_subs 2024-09-18T14:19:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13202ef640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13202ef640 1 -- 172.21.15.45:0/2029075877 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1318141cc0 con 0x7f131806e810 2024-09-18T14:19:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1308002380 con 0x7f131806e810 2024-09-18T14:19:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f13080028b0 con 0x7f131806e810 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f131c888640 4 set_mon_vals no callback set 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 10 monclient: dump: 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.680+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1308002a90 con 0x7f131806e810 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.681+0000 7f13202ef640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.681+0000 7f13202ef640 1 -- 172.21.15.45:0/2029075877 --> v1:172.21.15.45:6791/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f12f8000f20 con 0x7f131806e810 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.681+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f1308046d60 con 0x7f131806e810 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.681+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 --> v1:172.21.15.45:6804/32276 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13040450c0 con 0x7f1304041460 2024-09-18T14:19:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.681+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== mon.4 v1:172.21.15.45:6791/0 9 ==== mon_get_version_reply(handle=1 version=899) v2 ==== 24+0+0 (unknown 3649037721 0 0) 0x7f13080156d0 con 0x7f131806e810 2024-09-18T14:19:23.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.681+0000 7f13167fc640 10 monclient: handle_get_version_reply finishing 1 version 899 2024-09-18T14:19:23.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.681+0000 7f131d88a640 10 monclient: get_auth_request con 0x7f130403a920 auth_method 0 2024-09-18T14:19:23.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.681+0000 7f131e88c640 10 monclient: get_auth_request con 0x7f1304041460 auth_method 0 2024-09-18T14:19:23.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.682+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== osd.0 v1:172.21.15.45:6804/32276 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f13040450c0 con 0x7f1304041460 2024-09-18T14:19:23.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.693+0000 7f13202ef640 1 -- 172.21.15.45:0/2029075877 --> v1:172.21.15.45:6804/32276 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f12f8002b60 con 0x7f1304041460 2024-09-18T14:19:23.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.693+0000 7f13167fc640 1 -- 172.21.15.45:0/2029075877 <== osd.0 v1:172.21.15.45:6804/32276 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f12f8002b60 con 0x7f1304041460 2024-09-18T14:19:23.708 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:23.708 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T14:19:23.708 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:23.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.694+0000 7f13202ef640 1 -- 172.21.15.45:0/2029075877 >> v1:172.21.15.45:6804/32276 conn(0x7f1304041460 legacy=0x7f1304043870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.694+0000 7f13202ef640 1 -- 172.21.15.45:0/2029075877 >> v1:172.21.15.84:6812/31478 conn(0x7f130403a920 legacy=0x7f130403cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.694+0000 7f13202ef640 10 monclient: shutdown 2024-09-18T14:19:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.694+0000 7f13202ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.694+0000 7f13202ef640 1 -- 172.21.15.45:0/2029075877 >> v1:172.21.15.45:6791/0 conn(0x7f131806e810 legacy=0x7f131809e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.694+0000 7f131e88c640 1 -- 172.21.15.45:0/2029075877 reap_dead start 2024-09-18T14:19:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.694+0000 7f13202ef640 1 -- 172.21.15.45:0/2029075877 shutdown_connections 2024-09-18T14:19:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.694+0000 7f13202ef640 1 -- 172.21.15.45:0/2029075877 >> 172.21.15.45:0/2029075877 conn(0x7f13180a5c20 msgr2=0x7f1318062fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.694+0000 7f13202ef640 1 -- 172.21.15.45:0/2029075877 shutdown_connections 2024-09-18T14:19:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.694+0000 7f13202ef640 1 -- 172.21.15.45:0/2029075877 wait complete. 2024-09-18T14:19:23.712 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-18T14:19:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.808+0000 7fd9d7c8b640 10 monclient: get_monmap_and_config 2024-09-18T14:19:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.808+0000 7fd9d7c8b640 10 monclient: build_initial_monmap 2024-09-18T14:19:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.808+0000 7fd9d7c8b640 10 monclient: monmap: 2024-09-18T14:19:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:23.809638+0000 2024-09-18T14:19:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:23.809638+0000 2024-09-18T14:19:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.810 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:23.810 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:23.810 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:23.810 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:23.810 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:23.810 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:23.810 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:23.810 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:23.810 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:23.810 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 1 Processor -- start 2024-09-18T14:19:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 1 -- start start 2024-09-18T14:19:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 10 monclient: init 2024-09-18T14:19:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 10 monclient: _add_conns ranks=[3,6,7,4,0,1,5,2,8] 2024-09-18T14:19:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 10 monclient(hunting): picked mon.noname-g con 0x7fd9d009f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 10 monclient(hunting): picked mon.noname-d con 0x7fd9d009fba0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 10 monclient(hunting): picked mon.noname-i con 0x7fd9d009ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d005ed70 con 0x7fd9d009f7d0 2024-09-18T14:19:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d009dfc0 con 0x7fd9d009fba0 2024-09-18T14:19:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d009dce0 con 0x7fd9d009ff70 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.809+0000 7fd9d7c8b640 10 monclient(hunting): authenticate will time out at 1979.147960s 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d5226640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fd9d009fba0 0x7fd9d0137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56444/0 (socket says 172.21.15.45:56444) 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d5226640 1 -- 172.21.15.45:0/4113027468 learned_addr learned my addr 172.21.15.45:0/4113027468 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1187687776 0 0) 0x7fd9d009dfc0 con 0x7fd9d009fba0 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient(hunting): my global_id is 185334 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9bc0034e0 con 0x7fd9d009fba0 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017779766 0 0) 0x7fd9d005ed70 con 0x7fd9d009f7d0 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient(hunting): my global_id is 224508 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d009dfc0 con 0x7fd9d009f7d0 2024-09-18T14:19:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034086775 0 0) 0x7fd9d009dce0 con 0x7fd9d009ff70 2024-09-18T14:19:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient(hunting): my global_id is 214711 2024-09-18T14:19:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d005ed70 con 0x7fd9d009ff70 2024-09-18T14:19:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1697082306 0 0) 0x7fd9bc0034e0 con 0x7fd9d009fba0 2024-09-18T14:19:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 >> v1:172.21.15.84:6792/0 conn(0x7fd9d009ff70 legacy=0x7fd9d013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 >> v1:172.21.15.84:6790/0 conn(0x7fd9d009f7d0 legacy=0x7fd9d0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient: _finish_hunting 0 2024-09-18T14:19:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 monclient: found mon.noname-d 2024-09-18T14:19:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:19:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d00a1c00 con 0x7fd9d009fba0 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient: _finish_auth 0 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd9c0001830 con 0x7fd9d009fba0 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient: dump: 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d7c8b640 5 monclient: authenticate success, global_id 185334 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d7c8b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd9c0001a10 con 0x7fd9d009fba0 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d7c8b640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 1 -- 172.21.15.45:0/4113027468 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd9c0004ba0 con 0x7fd9d009fba0 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d7c8b640 4 set_mon_vals no callback set 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.810+0000 7fd9d4a25640 10 monclient: dump: 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.811+0000 7fd9d7c8b640 10 monclient: shutdown 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.811+0000 7fd9d7c8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.811+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/4113027468 >> v1:172.21.15.45:6792/0 conn(0x7fd9d009fba0 legacy=0x7fd9d0137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.811+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/4113027468 shutdown_connections 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.811+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/4113027468 >> 172.21.15.45:0/4113027468 conn(0x7fd9d00630b0 msgr2=0x7fd9d00634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.811+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/4113027468 shutdown_connections 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.811+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/4113027468 wait complete. 2024-09-18T14:19:23.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.811+0000 7fd9d7c8b640 10 monclient: build_initial_monmap 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.811+0000 7fd9d7c8b640 10 monclient: monmap: 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:23.824 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:23.825 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 1 Processor -- start 2024-09-18T14:19:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 1 -- start start 2024-09-18T14:19:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 10 monclient: init 2024-09-18T14:19:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 10 monclient: _add_conns ranks=[8,0,6,1,5,2,7,3,4] 2024-09-18T14:19:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 10 monclient(hunting): picked mon.noname-e con 0x7fd9d009f7d0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 10 monclient(hunting): picked mon.noname-a con 0x7fd9d009fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 10 monclient(hunting): picked mon.noname-d con 0x7fd9d009ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d009d310 con 0x7fd9d009fba0 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d00a2560 con 0x7fd9d009ff70 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d0142600 con 0x7fd9d009f7d0 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d7c8b640 10 monclient(hunting): authenticate will time out at 1979.150711s 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d6228640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7fd9d009ff70 0x7fd9d0149a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56454/0 (socket says 172.21.15.45:56454) 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d5226640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd9d009fba0 0x7fd9d0131930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58918/0 (socket says 172.21.15.45:58918) 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.812+0000 7fd9d6228640 1 -- 172.21.15.45:0/417060316 learned_addr learned my addr 172.21.15.45:0/417060316 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294655425 0 0) 0x7fd9d00a2560 con 0x7fd9d009ff70 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): my global_id is 185343 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9b40034e0 con 0x7fd9d009ff70 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837444921 0 0) 0x7fd9d0142600 con 0x7fd9d009f7d0 2024-09-18T14:19:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): my global_id is 234269 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d00a2560 con 0x7fd9d009f7d0 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988680674 0 0) 0x7fd9d009d310 con 0x7fd9d009fba0 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): my global_id is 6732 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d0142600 con 0x7fd9d009fba0 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2794883372 0 0) 0x7fd9b40034e0 con 0x7fd9d009ff70 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd9d009d310 con 0x7fd9d009ff70 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd9c8002e50 con 0x7fd9d009ff70 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): dump: 2024-09-18T14:19:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 798080139 0 0) 0x7fd9d00a2560 con 0x7fd9d009f7d0 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd9b40034e0 con 0x7fd9d009f7d0 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 36865584 0 0) 0x7fd9d0142600 con 0x7fd9d009fba0 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd9d00a2560 con 0x7fd9d009fba0 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd9cc001700 con 0x7fd9d009f7d0 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): dump: 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:23.830 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd9c0001770 con 0x7fd9d009fba0 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient(hunting): dump: 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3934968416 0 0) 0x7fd9d009d310 con 0x7fd9d009ff70 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 >> v1:172.21.15.45:6793/0 conn(0x7fd9d009f7d0 legacy=0x7fd9d0141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 >> v1:172.21.15.45:6789/0 conn(0x7fd9d009fba0 legacy=0x7fd9d0131930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 monclient: found mon.d 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d0120b20 con 0x7fd9d009ff70 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient: _finish_auth 0 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9d7c8b640 5 monclient: authenticate success, global_id 185343 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9d7c8b640 10 monclient: _renew_subs 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9d7c8b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/417060316 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d0121a80 con 0x7fd9d009ff70 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd9c8003110 con 0x7fd9d009ff70 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd9c8001d00 con 0x7fd9d009ff70 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c5ffb640 10 monclient: dump: 2024-09-18T14:19:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:23.835 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9c7fff640 4 set_mon_vals no callback set 2024-09-18T14:19:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9d7c8b640 10 monclient: _renew_subs 2024-09-18T14:19:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9d7c8b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.813+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/417060316 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d0120650 con 0x7fd9d009ff70 2024-09-18T14:19:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.814+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd9c8001e60 con 0x7fd9d009ff70 2024-09-18T14:19:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.814+0000 7fd9d7c8b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.814+0000 7fd9d7c8b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.814+0000 7fd9d7c8b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.814+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7fd9c8046c20 con 0x7fd9d009ff70 2024-09-18T14:19:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.814+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/417060316 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d004c4a0 con 0x7fd9d009ff70 2024-09-18T14:19:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.815+0000 7fd9d5a27640 10 monclient: get_auth_request con 0x7fd9b403ab10 auth_method 0 2024-09-18T14:19:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.819+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fd9c8015170 con 0x7fd9d009ff70 2024-09-18T14:19:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.819+0000 7fd9c5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.819+0000 7fd9c5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:23.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.980+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/417060316 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9d004b720 con 0x7fd9b403ab10 2024-09-18T14:19:23.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.981+0000 7fd9c5ffb640 1 -- 172.21.15.45:0/417060316 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (unknown 2057319476 0 0) 0x7fd9d004b720 con 0x7fd9b403ab10 2024-09-18T14:19:23.981 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 0 to deep-scrub 2024-09-18T14:19:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.982+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/417060316 >> v1:172.21.15.84:6812/31478 conn(0x7fd9b403ab10 legacy=0x7fd9b403cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.982+0000 7fd9d7c8b640 10 monclient: shutdown 2024-09-18T14:19:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.982+0000 7fd9d7c8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.982+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/417060316 >> v1:172.21.15.45:6792/0 conn(0x7fd9d009ff70 legacy=0x7fd9d0149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.983+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/417060316 shutdown_connections 2024-09-18T14:19:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.983+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/417060316 >> 172.21.15.45:0/417060316 conn(0x7fd9d00630b0 msgr2=0x7fd9d00a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.983+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/417060316 shutdown_connections 2024-09-18T14:19:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:23.983+0000 7fd9d7c8b640 1 -- 172.21.15.45:0/417060316 wait complete. 2024-09-18T14:19:23.993 INFO:tasks.ceph:Scrubbing osd.1 2024-09-18T14:19:23.993 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-18T14:19:24.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.092+0000 7fd6ae220640 10 monclient: get_monmap_and_config 2024-09-18T14:19:24.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.092+0000 7fd6ae220640 10 monclient: build_initial_monmap 2024-09-18T14:19:24.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.092+0000 7fd6ae220640 10 monclient: monmap: 2024-09-18T14:19:24.093 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:24.093 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.093 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:24.093880+0000 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:24.093880+0000 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:24.094 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.093+0000 7fd6ae220640 1 Processor -- start 2024-09-18T14:19:24.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.093+0000 7fd6ae220640 1 -- start start 2024-09-18T14:19:24.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.093+0000 7fd6ae220640 10 monclient: init 2024-09-18T14:19:24.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 10 monclient: _add_conns ranks=[8,1,2,6,5,0,4,3,7] 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 10 monclient(hunting): picked mon.noname-e con 0x7fd6a8132f80 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 10 monclient(hunting): picked mon.noname-f con 0x7fd6a8096a20 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 10 monclient(hunting): picked mon.noname-b con 0x7fd6a8138f70 addr v1:172.21.15.45:6790/0 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a805ed70 con 0x7fd6a8096a20 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a80973f0 con 0x7fd6a8138f70 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a809f5e0 con 0x7fd6a8132f80 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 10 monclient(hunting): authenticate will time out at 1979.432244s 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a77fe640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fd6a8132f80 0x7fd6a809eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34914/0 (socket says 172.21.15.45:34914) 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a7fff640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fd6a8138f70 0x7fd6a813b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39802/0 (socket says 172.21.15.45:39802) 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a77fe640 1 -- 172.21.15.45:0/1116131133 learned_addr learned my addr 172.21.15.45:0/1116131133 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 1 -- 172.21.15.45:0/1116131133 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218077081 0 0) 0x7fd6a809f5e0 con 0x7fd6a8132f80 2024-09-18T14:19:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 10 monclient(hunting): my global_id is 234278 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 1 -- 172.21.15.45:0/1116131133 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6900034e0 con 0x7fd6a8132f80 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 1 -- 172.21.15.45:0/1116131133 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636082700 0 0) 0x7fd6a80973f0 con 0x7fd6a8138f70 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 10 monclient(hunting): my global_id is 234173 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 1 -- 172.21.15.45:0/1116131133 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6a809f5e0 con 0x7fd6a8138f70 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 1 -- 172.21.15.45:0/1116131133 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3250636541 0 0) 0x7fd6900034e0 con 0x7fd6a8132f80 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 1 -- 172.21.15.45:0/1116131133 >> v1:172.21.15.45:6790/0 conn(0x7fd6a8138f70 legacy=0x7fd6a813b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 1 -- 172.21.15.45:0/1116131133 >> v1:172.21.15.84:6789/0 conn(0x7fd6a8096a20 legacy=0x7fd6a8137850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 1 monclient: found mon.noname-e 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:19:24.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 1 -- 172.21.15.45:0/1116131133 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a8097660 con 0x7fd6a8132f80 2024-09-18T14:19:24.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:24.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:24.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 5 monclient: authenticate success, global_id 234278 2024-09-18T14:19:24.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:24.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 1 -- 172.21.15.45:0/1116131133 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd698001830 con 0x7fd6a8132f80 2024-09-18T14:19:24.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:24.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6a67fc640 10 monclient: dump: 2024-09-18T14:19:24.104 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.104 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.105 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.094+0000 7fd6ae220640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6a67fc640 1 -- 172.21.15.45:0/1116131133 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd698001990 con 0x7fd6a8132f80 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6a67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6ae220640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6a67fc640 1 -- 172.21.15.45:0/1116131133 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd698004ba0 con 0x7fd6a8132f80 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6ae220640 4 set_mon_vals no callback set 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6a67fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6a67fc640 10 monclient: dump: 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6ae220640 10 monclient: shutdown 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6ae220640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6ae220640 1 -- 172.21.15.45:0/1116131133 >> v1:172.21.15.45:6793/0 conn(0x7fd6a8132f80 legacy=0x7fd6a809eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6ae220640 1 -- 172.21.15.45:0/1116131133 shutdown_connections 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6ae220640 1 -- 172.21.15.45:0/1116131133 >> 172.21.15.45:0/1116131133 conn(0x7fd6a80630b0 msgr2=0x7fd6a8063480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6ae220640 1 -- 172.21.15.45:0/1116131133 shutdown_connections 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.095+0000 7fd6ae220640 1 -- 172.21.15.45:0/1116131133 wait complete. 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.096+0000 7fd6ae220640 10 monclient: build_initial_monmap 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.096+0000 7fd6ae220640 10 monclient: monmap: 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.096+0000 7fd6ae220640 1 Processor -- start 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.096+0000 7fd6ae220640 1 -- start start 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.096+0000 7fd6ae220640 10 monclient: init 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.096+0000 7fd6ae220640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.096+0000 7fd6ae220640 10 monclient: _add_conns ranks=[7,1,3,5,6,0,4,2,8] 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.096+0000 7fd6ae220640 10 monclient(hunting): picked mon.noname-i con 0x7fd6a8096a20 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd6ae220640 10 monclient(hunting): picked mon.noname-f con 0x7fd6a8132f80 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd6ae220640 10 monclient(hunting): picked mon.noname-g con 0x7fd6a8138f70 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd6ae220640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a809f9f0 con 0x7fd6a8132f80 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd6ae220640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a80a23f0 con 0x7fd6a8138f70 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd6ae220640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a81423f0 con 0x7fd6a8096a20 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd6ae220640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd6ae220640 10 monclient(hunting): authenticate will time out at 1979.435075s 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd6a77fe640 1 --1- >> v1:172.21.15.84:6792/0 conn(0x7fd6a8096a20 0x7fd6a8141530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6792/0 says I am v1:172.21.15.45:59954/0 (socket says 172.21.15.45:59954) 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd6a77fe640 1 -- 172.21.15.45:0/3975339779 learned_addr learned my addr 172.21.15.45:0/3975339779 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649705904 0 0) 0x7fd6a81423f0 con 0x7fd6a8096a20 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): my global_id is 214720 2024-09-18T14:19:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6940034e0 con 0x7fd6a8096a20 2024-09-18T14:19:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655215085 0 0) 0x7fd6a809f9f0 con 0x7fd6a8132f80 2024-09-18T14:19:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): my global_id is 214777 2024-09-18T14:19:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6a81423f0 con 0x7fd6a8132f80 2024-09-18T14:19:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718352028 0 0) 0x7fd6a80a23f0 con 0x7fd6a8138f70 2024-09-18T14:19:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): my global_id is 224517 2024-09-18T14:19:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6a809f9f0 con 0x7fd6a8138f70 2024-09-18T14:19:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3138162275 0 0) 0x7fd6940034e0 con 0x7fd6a8096a20 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6a80a23f0 con 0x7fd6a8096a20 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd6980047a0 con 0x7fd6a8096a20 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): dump: 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1254879769 0 0) 0x7fd6a809f9f0 con 0x7fd6a8138f70 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6940034e0 con 0x7fd6a8138f70 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2639890446 0 0) 0x7fd6a81423f0 con 0x7fd6a8132f80 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6a809f9f0 con 0x7fd6a8132f80 2024-09-18T14:19:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd6a0001c30 con 0x7fd6a8138f70 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): dump: 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd69c001950 con 0x7fd6a8132f80 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.097+0000 7fd68f7fe640 10 monclient(hunting): dump: 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.7 v1:172.21.15.84:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3865375212 0 0) 0x7fd6a80a23f0 con 0x7fd6a8096a20 2024-09-18T14:19:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 >> v1:172.21.15.84:6790/0 conn(0x7fd6a8138f70 legacy=0x7fd6a8149880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 >> v1:172.21.15.84:6789/0 conn(0x7fd6a8132f80 legacy=0x7fd6a809e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 10 monclient: _finish_hunting 0 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 1 monclient: found mon.i 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 --> v1:172.21.15.84:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a8099840 con 0x7fd6a8096a20 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 10 monclient: _finish_auth 0 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd6ae220640 5 monclient: authenticate success, global_id 214720 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd6ae220640 10 monclient: _renew_subs 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd6ae220640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd6ae220640 1 -- 172.21.15.45:0/3975339779 --> v1:172.21.15.84:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a8099cf0 con 0x7fd6a8096a20 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd6ae220640 10 monclient: _renew_subs 2024-09-18T14:19:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd6ae220640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd6ae220640 1 -- 172.21.15.45:0/3975339779 --> v1:172.21.15.84:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a809e860 con 0x7fd6a8096a20 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.7 v1:172.21.15.84:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd698004e20 con 0x7fd6a8096a20 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.7 v1:172.21.15.84:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd6980052d0 con 0x7fd6a8096a20 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd6a5ffb640 4 set_mon_vals no callback set 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd68f7fe640 10 monclient: dump: 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd6ae220640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.84:6792/0 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.098+0000 7fd6ae220640 1 -- 172.21.15.45:0/3975339779 --> v1:172.21.15.84:6792/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd688000f20 con 0x7fd6a8096a20 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.099+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.7 v1:172.21.15.84:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd6980102a0 con 0x7fd6a8096a20 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.099+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.7 v1:172.21.15.84:6792/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7fd698046bc0 con 0x7fd6a8096a20 2024-09-18T14:19:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.099+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 --> v1:172.21.15.45:6800/32277 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6940453c0 con 0x7fd694041760 2024-09-18T14:19:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.099+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== mon.7 v1:172.21.15.84:6792/0 9 ==== mon_get_version_reply(handle=1 version=899) v2 ==== 24+0+0 (unknown 3649037721 0 0) 0x7fd698046f90 con 0x7fd6a8096a20 2024-09-18T14:19:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.099+0000 7fd68f7fe640 10 monclient: handle_get_version_reply finishing 1 version 899 2024-09-18T14:19:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.099+0000 7fd6a6ffd640 10 monclient: get_auth_request con 0x7fd69403ab50 auth_method 0 2024-09-18T14:19:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.099+0000 7fd6a7fff640 10 monclient: get_auth_request con 0x7fd694041760 auth_method 0 2024-09-18T14:19:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.100+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== osd.1 v1:172.21.15.45:6800/32277 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fd6940453c0 con 0x7fd694041760 2024-09-18T14:19:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.111+0000 7fd6ae220640 1 -- 172.21.15.45:0/3975339779 --> v1:172.21.15.45:6800/32277 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd688002b60 con 0x7fd694041760 2024-09-18T14:19:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.111+0000 7fd68f7fe640 1 -- 172.21.15.45:0/3975339779 <== osd.1 v1:172.21.15.45:6800/32277 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7fd688002b60 con 0x7fd694041760 2024-09-18T14:19:24.121 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:24.121 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T14:19:24.121 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.112+0000 7fd6ae220640 1 -- 172.21.15.45:0/3975339779 >> v1:172.21.15.45:6800/32277 conn(0x7fd694041760 legacy=0x7fd694043b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.112+0000 7fd6ae220640 1 -- 172.21.15.45:0/3975339779 >> v1:172.21.15.84:6812/31478 conn(0x7fd69403ab50 legacy=0x7fd69403d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.112+0000 7fd6ae220640 10 monclient: shutdown 2024-09-18T14:19:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.112+0000 7fd6ae220640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.112+0000 7fd6ae220640 1 -- 172.21.15.45:0/3975339779 >> v1:172.21.15.84:6792/0 conn(0x7fd6a8096a20 legacy=0x7fd6a8141530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.112+0000 7fd6a7fff640 1 -- 172.21.15.45:0/3975339779 reap_dead start 2024-09-18T14:19:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.112+0000 7fd6ae220640 1 -- 172.21.15.45:0/3975339779 shutdown_connections 2024-09-18T14:19:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.112+0000 7fd6ae220640 1 -- 172.21.15.45:0/3975339779 >> 172.21.15.45:0/3975339779 conn(0x7fd6a80630b0 msgr2=0x7fd6a80a4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.112+0000 7fd6ae220640 1 -- 172.21.15.45:0/3975339779 shutdown_connections 2024-09-18T14:19:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.112+0000 7fd6ae220640 1 -- 172.21.15.45:0/3975339779 wait complete. 2024-09-18T14:19:24.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-18T14:19:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.219+0000 7f4f91b5d640 10 monclient: get_monmap_and_config 2024-09-18T14:19:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.219+0000 7f4f91b5d640 10 monclient: build_initial_monmap 2024-09-18T14:19:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.219+0000 7f4f91b5d640 10 monclient: monmap: 2024-09-18T14:19:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:24.220893+0000 2024-09-18T14:19:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:24.220893+0000 2024-09-18T14:19:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:24.222 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.220+0000 7f4f91b5d640 1 Processor -- start 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.220+0000 7f4f91b5d640 1 -- start start 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.220+0000 7f4f91b5d640 10 monclient: init 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f91b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f91b5d640 10 monclient: _add_conns ranks=[6,8,4,7,3,5,1,2,0] 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f91b5d640 10 monclient(hunting): picked mon.noname-d con 0x7f4f8c09f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f91b5d640 10 monclient(hunting): picked mon.noname-e con 0x7f4f8c09fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f91b5d640 10 monclient(hunting): picked mon.noname-c con 0x7f4f8c09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f91b5d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8c05ed70 con 0x7f4f8c09ff70 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f91b5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8c09dfc0 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f91b5d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8c09dce0 con 0x7f4f8c09fba0 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f91b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f91b5d640 10 monclient(hunting): authenticate will time out at 1979.559222s 2024-09-18T14:19:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8b7fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f4f8c09f7d0 0x7f4f8c132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56470/0 (socket says 172.21.15.45:56470) 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8bfff640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f4f8c09ff70 0x7f4f8c13b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:37000/0 (socket says 172.21.15.45:37000) 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8affd640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f4f8c09fba0 0x7f4f8c137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34930/0 (socket says 172.21.15.45:34930) 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8bfff640 1 -- 172.21.15.45:0/881680645 learned_addr learned my addr 172.21.15.45:0/881680645 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926022150 0 0) 0x7f4f8c09dfc0 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient(hunting): my global_id is 185352 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f740034e0 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37603017 0 0) 0x7f4f8c09dce0 con 0x7f4f8c09fba0 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient(hunting): my global_id is 234287 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f8c09dfc0 con 0x7f4f8c09fba0 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460681272 0 0) 0x7f4f8c05ed70 con 0x7f4f8c09ff70 2024-09-18T14:19:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient(hunting): my global_id is 165460 2024-09-18T14:19:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f8c09dce0 con 0x7f4f8c09ff70 2024-09-18T14:19:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2990343473 0 0) 0x7f4f740034e0 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 >> v1:172.21.15.45:6793/0 conn(0x7f4f8c09fba0 legacy=0x7f4f8c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 >> v1:172.21.15.45:6791/0 conn(0x7f4f8c09ff70 legacy=0x7f4f8c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 1 monclient: found mon.noname-d 2024-09-18T14:19:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:19:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c0a1c00 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f91b5d640 5 monclient: authenticate success, global_id 185352 2024-09-18T14:19:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f7c001760 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f91b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.221+0000 7f4f8a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f8a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f8a7fc640 10 monclient: dump: 2024-09-18T14:19:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f91b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4f7c0018c0 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f8a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f91b5d640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f8a7fc640 1 -- 172.21.15.45:0/881680645 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f7c0047a0 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f91b5d640 4 set_mon_vals no callback set 2024-09-18T14:19:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f8a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f8a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f8a7fc640 10 monclient: dump: 2024-09-18T14:19:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.237 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f91b5d640 10 monclient: shutdown 2024-09-18T14:19:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f91b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f91b5d640 1 -- 172.21.15.45:0/881680645 >> v1:172.21.15.45:6792/0 conn(0x7f4f8c09f7d0 legacy=0x7f4f8c132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f91b5d640 1 -- 172.21.15.45:0/881680645 shutdown_connections 2024-09-18T14:19:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f91b5d640 1 -- 172.21.15.45:0/881680645 >> 172.21.15.45:0/881680645 conn(0x7f4f8c0630b0 msgr2=0x7f4f8c0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f91b5d640 1 -- 172.21.15.45:0/881680645 shutdown_connections 2024-09-18T14:19:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.222+0000 7f4f91b5d640 1 -- 172.21.15.45:0/881680645 wait complete. 2024-09-18T14:19:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.223+0000 7f4f91b5d640 10 monclient: build_initial_monmap 2024-09-18T14:19:24.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.223+0000 7f4f91b5d640 10 monclient: monmap: 2024-09-18T14:19:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.223+0000 7f4f91b5d640 1 Processor -- start 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.223+0000 7f4f91b5d640 1 -- start start 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.223+0000 7f4f91b5d640 10 monclient: init 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 10 monclient: _add_conns ranks=[1,0,6,4,7,8,5,3,2] 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 10 monclient(hunting): picked mon.noname-f con 0x7f4f8c09f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 10 monclient(hunting): picked mon.noname-a con 0x7f4f8c09fba0 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 10 monclient(hunting): picked mon.noname-d con 0x7f4f8c09ff70 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8c09d310 con 0x7f4f8c09fba0 2024-09-18T14:19:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8c0a2560 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8c120330 con 0x7f4f8c09ff70 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 10 monclient(hunting): authenticate will time out at 1979.562124s 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f8bfff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f4f8c09ff70 0x7f4f8c149cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56482/0 (socket says 172.21.15.45:56482) 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f8bfff640 1 -- 172.21.15.45:0/81393522 learned_addr learned my addr 172.21.15.45:0/81393522 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f8affd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4f8c09fba0 0x7f4f8c11d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58934/0 (socket says 172.21.15.45:58934) 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608367938 0 0) 0x7f4f8c120330 con 0x7f4f8c09ff70 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): my global_id is 185361 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f780034e0 con 0x7f4f8c09ff70 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647639165 0 0) 0x7f4f8c09d310 con 0x7f4f8c09fba0 2024-09-18T14:19:24.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): my global_id is 6741 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f8c120330 con 0x7f4f8c09fba0 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863004428 0 0) 0x7f4f8c0a2560 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): my global_id is 214786 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f8c09d310 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2377801905 0 0) 0x7f4f780034e0 con 0x7f4f8c09ff70 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f8c0a2560 con 0x7f4f8c09ff70 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 388138171 0 0) 0x7f4f8c120330 con 0x7f4f8c09fba0 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f780034e0 con 0x7f4f8c09fba0 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f84001d40 con 0x7f4f8c09ff70 2024-09-18T14:19:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): dump: 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.242 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f80001790 con 0x7f4f8c09fba0 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): dump: 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.243 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.244 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.244 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.244 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.244 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.244 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.244 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.244 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.244 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.244 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.244 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.244 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1376274810 0 0) 0x7f4f8c09d310 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f8c120330 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f7c0016a0 con 0x7f4f8c09f7d0 2024-09-18T14:19:24.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:19:24.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient(hunting): dump: 2024-09-18T14:19:24.245 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.245 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.245 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.245 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.245 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.245 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2422361107 0 0) 0x7f4f8c0a2560 con 0x7f4f8c09ff70 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 >> v1:172.21.15.84:6789/0 conn(0x7f4f8c09f7d0 legacy=0x7f4f8c11d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 >> v1:172.21.15.45:6789/0 conn(0x7f4f8c09fba0 legacy=0x7f4f8c11d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient: _finish_hunting 0 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 monclient: found mon.d 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c11e800 con 0x7f4f8c09ff70 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 10 monclient: _finish_auth 0 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 5 monclient: authenticate success, global_id 185361 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 10 monclient: _renew_subs 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.224+0000 7f4f91b5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.225+0000 7f4f91b5d640 1 -- 172.21.15.45:0/81393522 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8c132370 con 0x7f4f8c09ff70 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.225+0000 7f4f91b5d640 10 monclient: _renew_subs 2024-09-18T14:19:24.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.225+0000 7f4f91b5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.225+0000 7f4f91b5d640 1 -- 172.21.15.45:0/81393522 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8c11de30 con 0x7f4f8c09ff70 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.225+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4f84001ea0 con 0x7f4f8c09ff70 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.225+0000 7f4f737fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.225+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f4f84002080 con 0x7f4f8c09ff70 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.225+0000 7f4f737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.225+0000 7f4f89ffb640 4 set_mon_vals no callback set 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.225+0000 7f4f737fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.225+0000 7f4f737fe640 10 monclient: dump: 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.248 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.225+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f4f8400ff80 con 0x7f4f8c09ff70 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.226+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f4f84046b50 con 0x7f4f8c09ff70 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.226+0000 7f4f8b7fe640 10 monclient: get_auth_request con 0x7f4f7803a9f0 auth_method 0 2024-09-18T14:19:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.226+0000 7f4f91b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.226+0000 7f4f91b5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.226+0000 7f4f91b5d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.226+0000 7f4f91b5d640 1 -- 172.21.15.45:0/81393522 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c005010 con 0x7f4f8c09ff70 2024-09-18T14:19:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.230+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f4f84015520 con 0x7f4f8c09ff70 2024-09-18T14:19:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.230+0000 7f4f737fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.230+0000 7f4f737fe640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.399+0000 7f4f91b5d640 1 -- 172.21.15.45:0/81393522 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f6c000b90 con 0x7f4f7803a9f0 2024-09-18T14:19:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.399+0000 7f4f737fe640 1 -- 172.21.15.45:0/81393522 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (unknown 489605429 0 0) 0x7f4f6c000b90 con 0x7f4f7803a9f0 2024-09-18T14:19:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 1 to deep-scrub 2024-09-18T14:19:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.402+0000 7f4f91b5d640 1 -- 172.21.15.45:0/81393522 >> v1:172.21.15.84:6812/31478 conn(0x7f4f7803a9f0 legacy=0x7f4f7803cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.402+0000 7f4f91b5d640 10 monclient: shutdown 2024-09-18T14:19:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.402+0000 7f4f91b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.402+0000 7f4f91b5d640 1 -- 172.21.15.45:0/81393522 >> v1:172.21.15.45:6792/0 conn(0x7f4f8c09ff70 legacy=0x7f4f8c149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.402+0000 7f4f91b5d640 1 -- 172.21.15.45:0/81393522 shutdown_connections 2024-09-18T14:19:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.402+0000 7f4f91b5d640 1 -- 172.21.15.45:0/81393522 >> 172.21.15.45:0/81393522 conn(0x7f4f8c0630b0 msgr2=0x7f4f8c0a4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.402+0000 7f4f91b5d640 1 -- 172.21.15.45:0/81393522 shutdown_connections 2024-09-18T14:19:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.402+0000 7f4f91b5d640 1 -- 172.21.15.45:0/81393522 wait complete. 2024-09-18T14:19:24.414 INFO:tasks.ceph:Scrubbing osd.2 2024-09-18T14:19:24.414 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-18T14:19:24.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.514+0000 7f307362b640 10 monclient: get_monmap_and_config 2024-09-18T14:19:24.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.514+0000 7f307362b640 10 monclient: build_initial_monmap 2024-09-18T14:19:24.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.514+0000 7f307362b640 10 monclient: monmap: 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:24.515027+0000 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:24.515027+0000 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:24.516 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:24.516 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.514+0000 7f307362b640 1 Processor -- start 2024-09-18T14:19:24.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.514+0000 7f307362b640 1 -- start start 2024-09-18T14:19:24.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.514+0000 7f307362b640 10 monclient: init 2024-09-18T14:19:24.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f307362b640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:24.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f307362b640 10 monclient: _add_conns ranks=[4,7,0,2,3,8,5,1,6] 2024-09-18T14:19:24.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f307362b640 10 monclient(hunting): picked mon.noname-c con 0x7f306c06e810 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:24.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f307362b640 10 monclient(hunting): picked mon.noname-i con 0x7f306c096a20 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:24.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f307362b640 10 monclient(hunting): picked mon.noname-a con 0x7f306c138f10 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:24.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f307362b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c05ed70 con 0x7f306c138f10 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f307362b640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c0a1b50 con 0x7f306c06e810 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f307362b640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c06ebe0 con 0x7f306c096a20 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f307362b640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f307362b640 10 monclient(hunting): authenticate will time out at 1979.853376s 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f30713c7640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f306c06e810 0x7f306c09ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:37014/0 (socket says 172.21.15.45:37014) 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f30713c7640 1 -- 172.21.15.45:0/3683358650 learned_addr learned my addr 172.21.15.45:0/3683358650 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3071bc8640 1 --1- 172.21.15.45:0/3683358650 >> v1:172.21.15.45:6789/0 conn(0x7f306c138f10 0x7f306c13b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58950/0 (socket says 172.21.15.45:58950) 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222776126 0 0) 0x7f306c05ed70 con 0x7f306c138f10 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 10 monclient(hunting): my global_id is 6750 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30580034e0 con 0x7f306c138f10 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368911845 0 0) 0x7f306c0a1b50 con 0x7f306c06e810 2024-09-18T14:19:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 10 monclient(hunting): my global_id is 165469 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f306c05ed70 con 0x7f306c06e810 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572526104 0 0) 0x7f306c06ebe0 con 0x7f306c096a20 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.515+0000 7f3063fff640 10 monclient(hunting): my global_id is 214729 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f306c0a1b50 con 0x7f306c096a20 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 361786313 0 0) 0x7f30580034e0 con 0x7f306c138f10 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 >> v1:172.21.15.84:6792/0 conn(0x7f306c096a20 legacy=0x7f306c1377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 >> v1:172.21.15.45:6791/0 conn(0x7f306c06e810 legacy=0x7f306c09ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 1 monclient: found mon.noname-a 2024-09-18T14:19:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c132990 con 0x7f306c138f10 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 10 monclient: _finish_auth 0 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3064001bb0 con 0x7f306c138f10 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f307362b640 5 monclient: authenticate success, global_id 6750 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f307362b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 10 monclient: dump: 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f307362b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3064001d90 con 0x7f306c138f10 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 1 -- 172.21.15.45:0/3683358650 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3064004e60 con 0x7f306c138f10 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f307362b640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f307362b640 4 set_mon_vals no callback set 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f3063fff640 10 monclient: dump: 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.529 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.529 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.529 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.529 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.529 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.529 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f307362b640 10 monclient: shutdown 2024-09-18T14:19:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f307362b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f307362b640 1 -- 172.21.15.45:0/3683358650 >> v1:172.21.15.45:6789/0 conn(0x7f306c138f10 legacy=0x7f306c13b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f307362b640 1 -- 172.21.15.45:0/3683358650 shutdown_connections 2024-09-18T14:19:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f307362b640 1 -- 172.21.15.45:0/3683358650 >> 172.21.15.45:0/3683358650 conn(0x7f306c0a5c20 msgr2=0x7f306c0a6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.516+0000 7f307362b640 1 -- 172.21.15.45:0/3683358650 shutdown_connections 2024-09-18T14:19:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.517+0000 7f307362b640 1 -- 172.21.15.45:0/3683358650 wait complete. 2024-09-18T14:19:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.517+0000 7f307362b640 10 monclient: build_initial_monmap 2024-09-18T14:19:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.517+0000 7f307362b640 10 monclient: monmap: 2024-09-18T14:19:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:24.533 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.517+0000 7f307362b640 1 Processor -- start 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.517+0000 7f307362b640 1 -- start start 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.517+0000 7f307362b640 10 monclient: init 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f307362b640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f307362b640 10 monclient: _add_conns ranks=[6,4,0,2,1,7,3,8,5] 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f307362b640 10 monclient(hunting): picked mon.noname-d con 0x7f306c06e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f307362b640 10 monclient(hunting): picked mon.noname-c con 0x7f306c096a20 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f307362b640 10 monclient(hunting): picked mon.noname-a con 0x7f306c138f10 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f307362b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c09f9a0 con 0x7f306c138f10 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f307362b640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c0a2270 con 0x7f306c096a20 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f307362b640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c141be0 con 0x7f306c06e810 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f307362b640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f307362b640 10 monclient(hunting): authenticate will time out at 1979.856428s 2024-09-18T14:19:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f3071bc8640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f306c138f10 0x7f306c149820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58958/0 (socket says 172.21.15.45:58958) 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f3071bc8640 1 -- 172.21.15.45:0/962652600 learned_addr learned my addr 172.21.15.45:0/962652600 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30713c7640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f306c06e810 0x7f306c1414d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56494/0 (socket says 172.21.15.45:56494) 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f3070bc6640 1 --1- 172.21.15.45:0/962652600 >> v1:172.21.15.45:6791/0 conn(0x7f306c096a20 0x7f306c09e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:37030/0 (socket says 172.21.15.45:37030) 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043698982 0 0) 0x7f306c09f9a0 con 0x7f306c138f10 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 10 monclient(hunting): my global_id is 6759 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30500034e0 con 0x7f306c138f10 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196764311 0 0) 0x7f306c0a2270 con 0x7f306c096a20 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 10 monclient(hunting): my global_id is 165478 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f306c09f9a0 con 0x7f306c096a20 2024-09-18T14:19:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230797216 0 0) 0x7f306c141be0 con 0x7f306c06e810 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 10 monclient(hunting): my global_id is 185370 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.518+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f306c0a2270 con 0x7f306c06e810 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 623848528 0 0) 0x7f30500034e0 con 0x7f306c138f10 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f306c141be0 con 0x7f306c138f10 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3064001b30 con 0x7f306c138f10 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient(hunting): dump: 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 536410276 0 0) 0x7f306c0a2270 con 0x7f306c06e810 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f30500034e0 con 0x7f306c06e810 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3648531710 0 0) 0x7f306c09f9a0 con 0x7f306c096a20 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f306c0a2270 con 0x7f306c096a20 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3068001470 con 0x7f306c06e810 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient(hunting): dump: 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f305c001a20 con 0x7f306c096a20 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient(hunting): dump: 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2221046842 0 0) 0x7f306c141be0 con 0x7f306c138f10 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 >> v1:172.21.15.45:6792/0 conn(0x7f306c06e810 legacy=0x7f306c1414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 >> v1:172.21.15.45:6791/0 conn(0x7f306c096a20 legacy=0x7f306c09e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient: _finish_hunting 0 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 monclient: found mon.a 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c142390 con 0x7f306c138f10 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient: _finish_auth 0 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f307362b640 5 monclient: authenticate success, global_id 6759 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f307362b640 10 monclient: _renew_subs 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f307362b640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f307362b640 1 -- 172.21.15.45:0/962652600 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c09a430 con 0x7f306c138f10 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f307362b640 10 monclient: _renew_subs 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f307362b640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f307362b640 1 -- 172.21.15.45:0/962652600 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c099860 con 0x7f306c138f10 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3064004ed0 con 0x7f306c138f10 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3064005400 con 0x7f306c138f10 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f3062ffd640 4 set_mon_vals no callback set 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.519+0000 7f30617fa640 10 monclient: dump: 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.520+0000 7f307362b640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.520+0000 7f307362b640 1 -- 172.21.15.45:0/962652600 --> v1:172.21.15.45:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f304c000f20 con 0x7f306c138f10 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.520+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f30640055e0 con 0x7f306c138f10 2024-09-18T14:19:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.520+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f3064046e30 con 0x7f306c138f10 2024-09-18T14:19:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.520+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 --> v1:172.21.15.45:6808/32278 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30500452a0 con 0x7f3050041640 2024-09-18T14:19:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.520+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_get_version_reply(handle=1 version=899) v2 ==== 24+0+0 (unknown 3649037721 0 0) 0x7f3064047260 con 0x7f306c138f10 2024-09-18T14:19:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.520+0000 7f30617fa640 10 monclient: handle_get_version_reply finishing 1 version 899 2024-09-18T14:19:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.520+0000 7f30713c7640 10 monclient: get_auth_request con 0x7f305003aa60 auth_method 0 2024-09-18T14:19:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.520+0000 7f3070bc6640 10 monclient: get_auth_request con 0x7f3050041640 auth_method 0 2024-09-18T14:19:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.521+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== osd.2 v1:172.21.15.45:6808/32278 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f30500452a0 con 0x7f3050041640 2024-09-18T14:19:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.533+0000 7f307362b640 1 -- 172.21.15.45:0/962652600 --> v1:172.21.15.45:6808/32278 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f304c002b60 con 0x7f3050041640 2024-09-18T14:19:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.534+0000 7f30617fa640 1 -- 172.21.15.45:0/962652600 <== osd.2 v1:172.21.15.45:6808/32278 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f304c002b60 con 0x7f3050041640 2024-09-18T14:19:24.551 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:24.551 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T14:19:24.551 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.534+0000 7f307362b640 1 -- 172.21.15.45:0/962652600 >> v1:172.21.15.45:6808/32278 conn(0x7f3050041640 legacy=0x7f3050043a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.534+0000 7f307362b640 1 -- 172.21.15.45:0/962652600 >> v1:172.21.15.84:6812/31478 conn(0x7f305003aa60 legacy=0x7f305003cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.534+0000 7f307362b640 10 monclient: shutdown 2024-09-18T14:19:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.534+0000 7f307362b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.534+0000 7f307362b640 1 -- 172.21.15.45:0/962652600 >> v1:172.21.15.45:6789/0 conn(0x7f306c138f10 legacy=0x7f306c149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.534+0000 7f3071bc8640 1 -- 172.21.15.45:0/962652600 reap_dead start 2024-09-18T14:19:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.534+0000 7f307362b640 1 -- 172.21.15.45:0/962652600 shutdown_connections 2024-09-18T14:19:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.534+0000 7f307362b640 1 -- 172.21.15.45:0/962652600 >> 172.21.15.45:0/962652600 conn(0x7f306c0a5c20 msgr2=0x7f306c062fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.534+0000 7f307362b640 1 -- 172.21.15.45:0/962652600 shutdown_connections 2024-09-18T14:19:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.534+0000 7f307362b640 1 -- 172.21.15.45:0/962652600 wait complete. 2024-09-18T14:19:24.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-18T14:19:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.650+0000 7efe9b2d4640 10 monclient: get_monmap_and_config 2024-09-18T14:19:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.650+0000 7efe9b2d4640 10 monclient: build_initial_monmap 2024-09-18T14:19:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.650+0000 7efe9b2d4640 10 monclient: monmap: 2024-09-18T14:19:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:24.651419+0000 2024-09-18T14:19:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:24.651419+0000 2024-09-18T14:19:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:24.652 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 1 Processor -- start 2024-09-18T14:19:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 1 -- start start 2024-09-18T14:19:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 10 monclient: init 2024-09-18T14:19:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 10 monclient: _add_conns ranks=[6,1,5,4,0,2,7,3,8] 2024-09-18T14:19:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 10 monclient(hunting): picked mon.noname-d con 0x7efe9409f7d0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 10 monclient(hunting): picked mon.noname-f con 0x7efe9409fba0 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 10 monclient(hunting): picked mon.noname-h con 0x7efe9409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9405ed70 con 0x7efe9409fba0 2024-09-18T14:19:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9409dfc0 con 0x7efe9409ff70 2024-09-18T14:19:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9409dce0 con 0x7efe9409f7d0 2024-09-18T14:19:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe9b2d4640 10 monclient(hunting): authenticate will time out at 1979.989867s 2024-09-18T14:19:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe99070640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7efe9409f7d0 0x7efe94132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56508/0 (socket says 172.21.15.45:56508) 2024-09-18T14:19:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.651+0000 7efe99070640 1 -- 172.21.15.45:0/1470472137 learned_addr learned my addr 172.21.15.45:0/1470472137 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647741152 0 0) 0x7efe9409dce0 con 0x7efe9409f7d0 2024-09-18T14:19:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient(hunting): my global_id is 185379 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe800034e0 con 0x7efe9409f7d0 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267443401 0 0) 0x7efe9409dfc0 con 0x7efe9409ff70 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient(hunting): my global_id is 185171 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe9409dce0 con 0x7efe9409ff70 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371436067 0 0) 0x7efe9405ed70 con 0x7efe9409fba0 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient(hunting): my global_id is 214795 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe9409dfc0 con 0x7efe9409fba0 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 876636686 0 0) 0x7efe800034e0 con 0x7efe9409f7d0 2024-09-18T14:19:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 >> v1:172.21.15.84:6791/0 conn(0x7efe9409ff70 legacy=0x7efe9413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 >> v1:172.21.15.84:6789/0 conn(0x7efe9409fba0 legacy=0x7efe94137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 monclient: found mon.noname-d 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940a1c00 con 0x7efe9409f7d0 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient: _finish_auth 0 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe9b2d4640 5 monclient: authenticate success, global_id 185379 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe900017b0 con 0x7efe9409f7d0 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe9b2d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient: dump: 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe9b2d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efe90001910 con 0x7efe9409f7d0 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 1 -- 172.21.15.45:0/1470472137 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe90004ba0 con 0x7efe9409f7d0 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe9b2d4640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe9b2d4640 4 set_mon_vals no callback set 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.652+0000 7efe8bfff640 10 monclient: dump: 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.665 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.653+0000 7efe9b2d4640 10 monclient: shutdown 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.653+0000 7efe9b2d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.653+0000 7efe9b2d4640 1 -- 172.21.15.45:0/1470472137 >> v1:172.21.15.45:6792/0 conn(0x7efe9409f7d0 legacy=0x7efe94132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.653+0000 7efe9b2d4640 1 -- 172.21.15.45:0/1470472137 shutdown_connections 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.653+0000 7efe9b2d4640 1 -- 172.21.15.45:0/1470472137 >> 172.21.15.45:0/1470472137 conn(0x7efe940630b0 msgr2=0x7efe940634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.653+0000 7efe9b2d4640 1 -- 172.21.15.45:0/1470472137 shutdown_connections 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.653+0000 7efe9b2d4640 1 -- 172.21.15.45:0/1470472137 wait complete. 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.653+0000 7efe9b2d4640 10 monclient: build_initial_monmap 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.653+0000 7efe9b2d4640 10 monclient: monmap: 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 1 Processor -- start 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 1 -- start start 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 10 monclient: init 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 10 monclient: _add_conns ranks=[0,8,5,4,2,3,7,6,1] 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 10 monclient(hunting): picked mon.noname-a con 0x7efe9409f7d0 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 10 monclient(hunting): picked mon.noname-e con 0x7efe9409fba0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 10 monclient(hunting): picked mon.noname-h con 0x7efe9409ff70 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9409d310 con 0x7efe9409f7d0 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe940a2560 con 0x7efe9409ff70 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe941428a0 con 0x7efe9409fba0 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9b2d4640 10 monclient(hunting): authenticate will time out at 1979.992702s 2024-09-18T14:19:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9886f640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7efe9409fba0 0x7efe94146d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34944/0 (socket says 172.21.15.45:34944) 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.654+0000 7efe9886f640 1 -- 172.21.15.45:0/3317959947 learned_addr learned my addr 172.21.15.45:0/3317959947 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828189223 0 0) 0x7efe941428a0 con 0x7efe9409fba0 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): my global_id is 234296 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe780034e0 con 0x7efe9409fba0 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704436411 0 0) 0x7efe9409d310 con 0x7efe9409f7d0 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): my global_id is 6768 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe941428a0 con 0x7efe9409f7d0 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960657496 0 0) 0x7efe940a2560 con 0x7efe9409ff70 2024-09-18T14:19:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): my global_id is 185180 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe9409d310 con 0x7efe9409ff70 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 484654715 0 0) 0x7efe780034e0 con 0x7efe9409fba0 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe940a2560 con 0x7efe9409fba0 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe84002c00 con 0x7efe9409fba0 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): dump: 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1213848772 0 0) 0x7efe941428a0 con 0x7efe9409f7d0 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe780034e0 con 0x7efe9409f7d0 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe900016f0 con 0x7efe9409f7d0 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): dump: 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2740477271 0 0) 0x7efe9409d310 con 0x7efe9409ff70 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe941428a0 con 0x7efe9409ff70 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe8c002bf0 con 0x7efe9409ff70 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient(hunting): dump: 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.674 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3936744512 0 0) 0x7efe940a2560 con 0x7efe9409fba0 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 >> v1:172.21.15.84:6791/0 conn(0x7efe9409ff70 legacy=0x7efe9414a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 >> v1:172.21.15.45:6789/0 conn(0x7efe9409f7d0 legacy=0x7efe941317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient: _finish_hunting 0 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 monclient: found mon.e 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9411e8c0 con 0x7efe9409fba0 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient: _finish_auth 0 2024-09-18T14:19:24.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe9b2d4640 5 monclient: authenticate success, global_id 234296 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe9b2d4640 10 monclient: _renew_subs 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe9b2d4640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe9b2d4640 1 -- 172.21.15.45:0/3317959947 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9409ba10 con 0x7efe9409fba0 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe9b2d4640 10 monclient: _renew_subs 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe9b2d4640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe9b2d4640 1 -- 172.21.15.45:0/3317959947 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9411d960 con 0x7efe9409fba0 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efe84001560 con 0x7efe9409fba0 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7efe84001a50 con 0x7efe9409fba0 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe8b7fe640 4 set_mon_vals no callback set 2024-09-18T14:19:24.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.655+0000 7efe897fa640 10 monclient: dump: 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.678 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.656+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7efe84001bb0 con 0x7efe9409fba0 2024-09-18T14:19:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.656+0000 7efe9b2d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.656+0000 7efe9b2d4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.656+0000 7efe9b2d4640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.656+0000 7efe9b2d4640 1 -- 172.21.15.45:0/3317959947 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe74005010 con 0x7efe9409fba0 2024-09-18T14:19:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.656+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7efe84046980 con 0x7efe9409fba0 2024-09-18T14:19:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.656+0000 7efe99070640 10 monclient: get_auth_request con 0x7efe7803a950 auth_method 0 2024-09-18T14:19:24.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.660+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7efe840152f0 con 0x7efe9409fba0 2024-09-18T14:19:24.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.660+0000 7efe897fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:24.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.660+0000 7efe897fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:24.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.825+0000 7efe9b2d4640 1 -- 172.21.15.45:0/3317959947 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7efe74000b90 con 0x7efe7803a950 2024-09-18T14:19:24.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.826+0000 7efe897fa640 1 -- 172.21.15.45:0/3317959947 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (unknown 3049115190 0 0) 0x7efe74000b90 con 0x7efe7803a950 2024-09-18T14:19:24.827 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 2 to deep-scrub 2024-09-18T14:19:24.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.827+0000 7efe9b2d4640 1 -- 172.21.15.45:0/3317959947 >> v1:172.21.15.84:6812/31478 conn(0x7efe7803a950 legacy=0x7efe7803ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.827+0000 7efe9b2d4640 10 monclient: shutdown 2024-09-18T14:19:24.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.827+0000 7efe9b2d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:24.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.827+0000 7efe9b2d4640 1 -- 172.21.15.45:0/3317959947 >> v1:172.21.15.45:6793/0 conn(0x7efe9409fba0 legacy=0x7efe94146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.828+0000 7efe9b2d4640 1 -- 172.21.15.45:0/3317959947 shutdown_connections 2024-09-18T14:19:24.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.828+0000 7efe9b2d4640 1 -- 172.21.15.45:0/3317959947 >> 172.21.15.45:0/3317959947 conn(0x7efe940630b0 msgr2=0x7efe940a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:24.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.828+0000 7efe9b2d4640 1 -- 172.21.15.45:0/3317959947 shutdown_connections 2024-09-18T14:19:24.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.828+0000 7efe9b2d4640 1 -- 172.21.15.45:0/3317959947 wait complete. 2024-09-18T14:19:24.839 INFO:tasks.ceph:Scrubbing osd.3 2024-09-18T14:19:24.839 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-18T14:19:24.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.938+0000 7fa0f459d640 10 monclient: get_monmap_and_config 2024-09-18T14:19:24.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.938+0000 7fa0f459d640 10 monclient: build_initial_monmap 2024-09-18T14:19:24.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.938+0000 7fa0f459d640 10 monclient: monmap: 2024-09-18T14:19:24.946 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:24.946 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.946 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:24.939130+0000 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:24.939130+0000 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.938+0000 7fa0f459d640 1 Processor -- start 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.938+0000 7fa0f459d640 1 -- start start 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.938+0000 7fa0f459d640 10 monclient: init 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f459d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f459d640 10 monclient: _add_conns ranks=[2,7,3,5,0,8,1,6,4] 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f459d640 10 monclient(hunting): picked mon.noname-b con 0x7fa0ec06e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f459d640 10 monclient(hunting): picked mon.noname-i con 0x7fa0ec096a20 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f459d640 10 monclient(hunting): picked mon.noname-g con 0x7fa0ec138f10 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f459d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec05ed70 con 0x7fa0ec06e810 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f459d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec0a1b50 con 0x7fa0ec138f10 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f459d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec06ebe0 con 0x7fa0ec096a20 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f459d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f459d640 10 monclient(hunting): authenticate will time out at 1980.277493s 2024-09-18T14:19:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f2339640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fa0ec06e810 0x7fa0ec09ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39804/0 (socket says 172.21.15.45:39804) 2024-09-18T14:19:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f2339640 1 -- 172.21.15.45:0/4079451127 learned_addr learned my addr 172.21.15.45:0/4079451127 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f1337640 1 -- 172.21.15.45:0/4079451127 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318964326 0 0) 0x7fa0ec05ed70 con 0x7fa0ec06e810 2024-09-18T14:19:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f1337640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f1337640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f1337640 10 monclient(hunting): my global_id is 234182 2024-09-18T14:19:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.939+0000 7fa0f1337640 1 -- 172.21.15.45:0/4079451127 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0d80034e0 con 0x7fa0ec06e810 2024-09-18T14:19:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 1 -- 172.21.15.45:0/4079451127 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1975060642 0 0) 0x7fa0d80034e0 con 0x7fa0ec06e810 2024-09-18T14:19:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 1 -- 172.21.15.45:0/4079451127 >> v1:172.21.15.84:6792/0 conn(0x7fa0ec096a20 legacy=0x7fa0ec1377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 1 -- 172.21.15.45:0/4079451127 >> v1:172.21.15.84:6790/0 conn(0x7fa0ec138f10 legacy=0x7fa0ec13b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 10 monclient: _finish_hunting 0 2024-09-18T14:19:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 1 monclient: found mon.noname-b 2024-09-18T14:19:24.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.45:6790/0 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 1 -- 172.21.15.45:0/4079451127 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec132990 con 0x7fa0ec06e810 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 10 monclient: _finish_auth 0 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f459d640 5 monclient: authenticate success, global_id 234182 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 1 -- 172.21.15.45:0/4079451127 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa0e8001830 con 0x7fa0ec06e810 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f459d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 10 monclient: dump: 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f459d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 1 -- 172.21.15.45:0/4079451127 <== mon.2 v1:172.21.15.45:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa0e8001a10 con 0x7fa0ec06e810 2024-09-18T14:19:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f459d640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 1 -- 172.21.15.45:0/4079451127 <== mon.2 v1:172.21.15.45:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa0e8004ba0 con 0x7fa0ec06e810 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f459d640 4 set_mon_vals no callback set 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f1337640 10 monclient: dump: 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f459d640 10 monclient: shutdown 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f459d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f459d640 1 -- 172.21.15.45:0/4079451127 >> v1:172.21.15.45:6790/0 conn(0x7fa0ec06e810 legacy=0x7fa0ec09ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f459d640 1 -- 172.21.15.45:0/4079451127 shutdown_connections 2024-09-18T14:19:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f459d640 1 -- 172.21.15.45:0/4079451127 >> 172.21.15.45:0/4079451127 conn(0x7fa0ec0a5c20 msgr2=0x7fa0ec0a6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f459d640 1 -- 172.21.15.45:0/4079451127 shutdown_connections 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.940+0000 7fa0f459d640 1 -- 172.21.15.45:0/4079451127 wait complete. 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.941+0000 7fa0f459d640 10 monclient: build_initial_monmap 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.941+0000 7fa0f459d640 10 monclient: monmap: 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.941+0000 7fa0f459d640 1 Processor -- start 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.941+0000 7fa0f459d640 1 -- start start 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.941+0000 7fa0f459d640 10 monclient: init 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0f459d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0f459d640 10 monclient: _add_conns ranks=[2,0,4,3,8,7,6,1,5] 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0f459d640 10 monclient(hunting): picked mon.noname-b con 0x7fa0ec06e810 addr v1:172.21.15.45:6790/0 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0f459d640 10 monclient(hunting): picked mon.noname-a con 0x7fa0ec096a20 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0f459d640 10 monclient(hunting): picked mon.noname-c con 0x7fa0ec138f10 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0f459d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec0a2270 con 0x7fa0ec096a20 2024-09-18T14:19:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0f459d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec099610 con 0x7fa0ec06e810 2024-09-18T14:19:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0f459d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec09a570 con 0x7fa0ec138f10 2024-09-18T14:19:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0f459d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0f459d640 10 monclient(hunting): authenticate will time out at 1980.280135s 2024-09-18T14:19:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0f2339640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fa0ec06e810 0x7fa0ec09e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39810/0 (socket says 172.21.15.45:39810) 2024-09-18T14:19:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0f2339640 1 -- 172.21.15.45:0/773909361 learned_addr learned my addr 172.21.15.45:0/773909361 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721198414 0 0) 0x7fa0ec09a570 con 0x7fa0ec138f10 2024-09-18T14:19:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): my global_id is 165487 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0d00034e0 con 0x7fa0ec138f10 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365990947 0 0) 0x7fa0ec0a2270 con 0x7fa0ec096a20 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): my global_id is 6777 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0ec09a570 con 0x7fa0ec096a20 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968063495 0 0) 0x7fa0ec099610 con 0x7fa0ec06e810 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): my global_id is 234191 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0ec0a2270 con 0x7fa0ec06e810 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3925180789 0 0) 0x7fa0d00034e0 con 0x7fa0ec138f10 2024-09-18T14:19:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0ec099610 con 0x7fa0ec138f10 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 460992605 0 0) 0x7fa0ec09a570 con 0x7fa0ec096a20 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0d00034e0 con 0x7fa0ec096a20 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa0e4001d20 con 0x7fa0ec138f10 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): dump: 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4224875284 0 0) 0x7fa0ec0a2270 con 0x7fa0ec06e810 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0ec09a570 con 0x7fa0ec06e810 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa0dc0019e0 con 0x7fa0ec096a20 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): dump: 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.960 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa0e8004ae0 con 0x7fa0ec06e810 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.942+0000 7fa0e27fc640 10 monclient(hunting): dump: 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1742840495 0 0) 0x7fa0ec099610 con 0x7fa0ec138f10 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 >> v1:172.21.15.45:6790/0 conn(0x7fa0ec06e810 legacy=0x7fa0ec09e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 >> v1:172.21.15.45:6789/0 conn(0x7fa0ec096a20 legacy=0x7fa0ec1469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 1 monclient: found mon.c 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec09a850 con 0x7fa0ec138f10 2024-09-18T14:19:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0f459d640 5 monclient: authenticate success, global_id 165487 2024-09-18T14:19:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0f459d640 10 monclient: _renew_subs 2024-09-18T14:19:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0f459d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0f459d640 1 -- 172.21.15.45:0/773909361 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ec09ea70 con 0x7fa0ec138f10 2024-09-18T14:19:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0f459d640 10 monclient: _renew_subs 2024-09-18T14:19:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0f459d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0f459d640 1 -- 172.21.15.45:0/773909361 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ec141cc0 con 0x7fa0ec138f10 2024-09-18T14:19:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa0e4003050 con 0x7fa0ec138f10 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fa0e40025b0 con 0x7fa0ec138f10 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0f0b36640 4 set_mon_vals no callback set 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 10 monclient: dump: 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:24.965 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.943+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fa0e4002790 con 0x7fa0ec138f10 2024-09-18T14:19:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.944+0000 7fa0f459d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.944+0000 7fa0f459d640 1 -- 172.21.15.45:0/773909361 --> v1:172.21.15.45:6791/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0d4000f20 con 0x7fa0ec138f10 2024-09-18T14:19:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.944+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7fa0e4044310 con 0x7fa0ec138f10 2024-09-18T14:19:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.944+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 --> v1:172.21.15.84:6802/32024 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0d00451d0 con 0x7fa0d0041570 2024-09-18T14:19:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.944+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== mon.4 v1:172.21.15.45:6791/0 9 ==== mon_get_version_reply(handle=1 version=899) v2 ==== 24+0+0 (unknown 3649037721 0 0) 0x7fa0e4044740 con 0x7fa0ec138f10 2024-09-18T14:19:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.944+0000 7fa0e27fc640 10 monclient: handle_get_version_reply finishing 1 version 899 2024-09-18T14:19:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.944+0000 7fa0f2339640 10 monclient: get_auth_request con 0x7fa0d003a9e0 auth_method 0 2024-09-18T14:19:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.944+0000 7fa0f1b38640 10 monclient: get_auth_request con 0x7fa0d0041570 auth_method 0 2024-09-18T14:19:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.945+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== osd.3 v1:172.21.15.84:6802/32024 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fa0d00451d0 con 0x7fa0d0041570 2024-09-18T14:19:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.956+0000 7fa0f459d640 1 -- 172.21.15.45:0/773909361 --> v1:172.21.15.84:6802/32024 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa0d4002b60 con 0x7fa0d0041570 2024-09-18T14:19:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.957+0000 7fa0e27fc640 1 -- 172.21.15.45:0/773909361 <== osd.3 v1:172.21.15.84:6802/32024 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7fa0d4002b60 con 0x7fa0d0041570 2024-09-18T14:19:24.967 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:24.967 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T14:19:24.967 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.957+0000 7fa0f459d640 1 -- 172.21.15.45:0/773909361 >> v1:172.21.15.84:6802/32024 conn(0x7fa0d0041570 legacy=0x7fa0d0043980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.957+0000 7fa0f459d640 1 -- 172.21.15.45:0/773909361 >> v1:172.21.15.84:6812/31478 conn(0x7fa0d003a9e0 legacy=0x7fa0d003ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.957+0000 7fa0f459d640 10 monclient: shutdown 2024-09-18T14:19:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.957+0000 7fa0f459d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.957+0000 7fa0f459d640 1 -- 172.21.15.45:0/773909361 >> v1:172.21.15.45:6791/0 conn(0x7fa0ec138f10 legacy=0x7fa0ec14a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.957+0000 7fa0f2b3a640 1 -- 172.21.15.45:0/773909361 reap_dead start 2024-09-18T14:19:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.957+0000 7fa0f459d640 1 -- 172.21.15.45:0/773909361 shutdown_connections 2024-09-18T14:19:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.957+0000 7fa0f459d640 1 -- 172.21.15.45:0/773909361 >> 172.21.15.45:0/773909361 conn(0x7fa0ec0a5c20 msgr2=0x7fa0ec0a5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.957+0000 7fa0f459d640 1 -- 172.21.15.45:0/773909361 shutdown_connections 2024-09-18T14:19:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:24.957+0000 7fa0f459d640 1 -- 172.21.15.45:0/773909361 wait complete. 2024-09-18T14:19:24.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-18T14:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.066+0000 7fc504880640 10 monclient: get_monmap_and_config 2024-09-18T14:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.066+0000 7fc504880640 10 monclient: build_initial_monmap 2024-09-18T14:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.066+0000 7fc504880640 10 monclient: monmap: 2024-09-18T14:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:25.067668+0000 2024-09-18T14:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:25.067668+0000 2024-09-18T14:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.067+0000 7fc504880640 1 Processor -- start 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.067+0000 7fc504880640 1 -- start start 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.067+0000 7fc504880640 10 monclient: init 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.067+0000 7fc504880640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.067+0000 7fc504880640 10 monclient: _add_conns ranks=[5,8,6,7,4,2,0,1,3] 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.067+0000 7fc504880640 10 monclient(hunting): picked mon.noname-h con 0x7fc50006e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.067+0000 7fc504880640 10 monclient(hunting): picked mon.noname-e con 0x7fc5000a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc504880640 10 monclient(hunting): picked mon.noname-d con 0x7fc5000a1080 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc504880640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50005ed70 con 0x7fc50006e810 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc504880640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc500097620 con 0x7fc5000a1080 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc504880640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc500097340 con 0x7fc5000a0cb0 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc504880640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc504880640 10 monclient(hunting): authenticate will time out at 1980.406081s 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fdd9b640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fc5000a0cb0 0x7fc500137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34958/0 (socket says 172.21.15.45:34958) 2024-09-18T14:19:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fdd9b640 1 -- 172.21.15.45:0/319432393 learned_addr learned my addr 172.21.15.45:0/319432393 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fed9d640 1 --1- 172.21.15.45:0/319432393 >> v1:172.21.15.45:6792/0 conn(0x7fc5000a1080 0x7fc50013af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56522/0 (socket says 172.21.15.45:56522) 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465589413 0 0) 0x7fc500097340 con 0x7fc5000a0cb0 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient(hunting): my global_id is 234305 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e80034e0 con 0x7fc5000a0cb0 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092671316 0 0) 0x7fc500097620 con 0x7fc5000a1080 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient(hunting): my global_id is 185388 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc500097340 con 0x7fc5000a1080 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195167341 0 0) 0x7fc50005ed70 con 0x7fc50006e810 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient(hunting): my global_id is 185189 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc500097620 con 0x7fc50006e810 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1402564577 0 0) 0x7fc500097340 con 0x7fc5000a1080 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 >> v1:172.21.15.45:6793/0 conn(0x7fc5000a0cb0 legacy=0x7fc500137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 >> v1:172.21.15.84:6791/0 conn(0x7fc50006e810 legacy=0x7fc5000a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient: _finish_hunting 0 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 monclient: found mon.noname-d 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5000a1660 con 0x7fc5000a1080 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient: _finish_auth 0 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4f8001b90 con 0x7fc5000a1080 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc504880640 5 monclient: authenticate success, global_id 185388 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc504880640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient: dump: 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc504880640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc4f8001cf0 con 0x7fc5000a1080 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc504880640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 1 -- 172.21.15.45:0/319432393 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4f8004b20 con 0x7fc5000a1080 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc504880640 4 set_mon_vals no callback set 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.068+0000 7fc4fd59a640 10 monclient: dump: 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.069+0000 7fc504880640 10 monclient: shutdown 2024-09-18T14:19:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.069+0000 7fc504880640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.069+0000 7fc504880640 1 -- 172.21.15.45:0/319432393 >> v1:172.21.15.45:6792/0 conn(0x7fc5000a1080 legacy=0x7fc50013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.069+0000 7fc504880640 1 -- 172.21.15.45:0/319432393 shutdown_connections 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.069+0000 7fc504880640 1 -- 172.21.15.45:0/319432393 >> 172.21.15.45:0/319432393 conn(0x7fc5000a5bb0 msgr2=0x7fc5000a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.069+0000 7fc504880640 1 -- 172.21.15.45:0/319432393 shutdown_connections 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.069+0000 7fc504880640 1 -- 172.21.15.45:0/319432393 wait complete. 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.069+0000 7fc504880640 10 monclient: build_initial_monmap 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.069+0000 7fc504880640 10 monclient: monmap: 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 1 Processor -- start 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 1 -- start start 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 10 monclient: init 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 10 monclient: _add_conns ranks=[5,8,1,6,2,0,7,3,4] 2024-09-18T14:19:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 10 monclient(hunting): picked mon.noname-h con 0x7fc50006e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 10 monclient(hunting): picked mon.noname-e con 0x7fc5000a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 10 monclient(hunting): picked mon.noname-f con 0x7fc5000a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc500122c40 con 0x7fc5000a1080 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50013da30 con 0x7fc50006e810 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc500122680 con 0x7fc5000a0cb0 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc504880640 10 monclient(hunting): authenticate will time out at 1980.408753s 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc4fdd9b640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7fc5000a0cb0 0x7fc500146550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34972/0 (socket says 172.21.15.45:34972) 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.070+0000 7fc4fdd9b640 1 -- 172.21.15.45:0/879236263 learned_addr learned my addr 172.21.15.45:0/879236263 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968674251 0 0) 0x7fc500122680 con 0x7fc5000a0cb0 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): my global_id is 234314 2024-09-18T14:19:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e00034e0 con 0x7fc5000a0cb0 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222802360 0 0) 0x7fc50013da30 con 0x7fc50006e810 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): my global_id is 185198 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc500122680 con 0x7fc50006e810 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 811712899 0 0) 0x7fc4e00034e0 con 0x7fc5000a0cb0 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc50013da30 con 0x7fc5000a0cb0 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473325386 0 0) 0x7fc500122c40 con 0x7fc5000a1080 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): my global_id is 214804 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e00034e0 con 0x7fc5000a1080 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4f4001690 con 0x7fc5000a0cb0 2024-09-18T14:19:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): dump: 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2579812606 0 0) 0x7fc500122680 con 0x7fc50006e810 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc500122c40 con 0x7fc50006e810 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4f0001570 con 0x7fc50006e810 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient(hunting): dump: 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3320091363 0 0) 0x7fc50013da30 con 0x7fc5000a0cb0 2024-09-18T14:19:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 >> v1:172.21.15.84:6791/0 conn(0x7fc50006e810 legacy=0x7fc50013d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 >> v1:172.21.15.84:6789/0 conn(0x7fc5000a1080 legacy=0x7fc500149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 monclient: found mon.e 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc500120140 con 0x7fc5000a0cb0 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc504880640 5 monclient: authenticate success, global_id 234314 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc504880640 10 monclient: _renew_subs 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc504880640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc504880640 1 -- 172.21.15.45:0/879236263 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5001223a0 con 0x7fc5000a0cb0 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc504880640 10 monclient: _renew_subs 2024-09-18T14:19:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc504880640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc504880640 1 -- 172.21.15.45:0/879236263 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5001214d0 con 0x7fc5000a0cb0 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc4f4004a50 con 0x7fc5000a0cb0 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fc4f4004f00 con 0x7fc5000a0cb0 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4fcd99640 4 set_mon_vals no callback set 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.071+0000 7fc4ee7fc640 10 monclient: dump: 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.090 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.072+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fc4f400fed0 con 0x7fc5000a0cb0 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.072+0000 7fc504880640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.072+0000 7fc504880640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.072+0000 7fc504880640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.072+0000 7fc504880640 1 -- 172.21.15.45:0/879236263 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e4005010 con 0x7fc5000a0cb0 2024-09-18T14:19:25.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.072+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7fc4f4046950 con 0x7fc5000a0cb0 2024-09-18T14:19:25.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.072+0000 7fc4fe59c640 10 monclient: get_auth_request con 0x7fc4e003a650 auth_method 0 2024-09-18T14:19:25.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.076+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fc4f4011b40 con 0x7fc5000a0cb0 2024-09-18T14:19:25.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.076+0000 7fc4ee7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:25.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.076+0000 7fc4ee7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:25.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.239+0000 7fc504880640 1 -- 172.21.15.45:0/879236263 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4e4000b90 con 0x7fc4e003a650 2024-09-18T14:19:25.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.239+0000 7fc4ee7fc640 1 -- 172.21.15.45:0/879236263 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (unknown 3526570807 0 0) 0x7fc4e4000b90 con 0x7fc4e003a650 2024-09-18T14:19:25.240 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 3 to deep-scrub 2024-09-18T14:19:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.241+0000 7fc504880640 1 -- 172.21.15.45:0/879236263 >> v1:172.21.15.84:6812/31478 conn(0x7fc4e003a650 legacy=0x7fc4e003cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.241+0000 7fc504880640 10 monclient: shutdown 2024-09-18T14:19:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.241+0000 7fc504880640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.241+0000 7fc504880640 1 -- 172.21.15.45:0/879236263 >> v1:172.21.15.45:6793/0 conn(0x7fc5000a0cb0 legacy=0x7fc500146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.241+0000 7fc504880640 1 -- 172.21.15.45:0/879236263 shutdown_connections 2024-09-18T14:19:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.241+0000 7fc504880640 1 -- 172.21.15.45:0/879236263 >> 172.21.15.45:0/879236263 conn(0x7fc5000a5bb0 msgr2=0x7fc5000a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.241+0000 7fc504880640 1 -- 172.21.15.45:0/879236263 shutdown_connections 2024-09-18T14:19:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.241+0000 7fc504880640 1 -- 172.21.15.45:0/879236263 wait complete. 2024-09-18T14:19:25.252 INFO:tasks.ceph:Scrubbing osd.4 2024-09-18T14:19:25.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-18T14:19:25.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.352+0000 7f3f31f9d640 10 monclient: get_monmap_and_config 2024-09-18T14:19:25.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.352+0000 7f3f31f9d640 10 monclient: build_initial_monmap 2024-09-18T14:19:25.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.352+0000 7f3f31f9d640 10 monclient: monmap: 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:25.353010+0000 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:25.353010+0000 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:25.354 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.352+0000 7f3f31f9d640 1 Processor -- start 2024-09-18T14:19:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.352+0000 7f3f31f9d640 1 -- start start 2024-09-18T14:19:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.352+0000 7f3f31f9d640 10 monclient: init 2024-09-18T14:19:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f31f9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f31f9d640 10 monclient: _add_conns ranks=[6,1,8,4,5,0,7,2,3] 2024-09-18T14:19:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f31f9d640 10 monclient(hunting): picked mon.noname-d con 0x7f3f2c06e810 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f31f9d640 10 monclient(hunting): picked mon.noname-f con 0x7f3f2c096a20 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f31f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f3f2c138f10 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f31f9d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c05ed70 con 0x7f3f2c096a20 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f31f9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c0a1b50 con 0x7f3f2c06e810 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f31f9d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c06ebe0 con 0x7f3f2c138f10 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f31f9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f31f9d640 10 monclient(hunting): authenticate will time out at 1980.691358s 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2b7fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f3f2c06e810 0x7f3f2c09ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56528/0 (socket says 172.21.15.45:56528) 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2bfff640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f3f2c138f10 0x7f3f2c13b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34988/0 (socket says 172.21.15.45:34988) 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2b7fe640 1 -- 172.21.15.45:0/3408675483 learned_addr learned my addr 172.21.15.45:0/3408675483 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378477692 0 0) 0x7f3f2c0a1b50 con 0x7f3f2c06e810 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 10 monclient(hunting): my global_id is 185397 2024-09-18T14:19:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f140034e0 con 0x7f3f2c06e810 2024-09-18T14:19:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598349320 0 0) 0x7f3f2c06ebe0 con 0x7f3f2c138f10 2024-09-18T14:19:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 10 monclient(hunting): my global_id is 234323 2024-09-18T14:19:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2c0a1b50 con 0x7f3f2c138f10 2024-09-18T14:19:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841238831 0 0) 0x7f3f2c05ed70 con 0x7f3f2c096a20 2024-09-18T14:19:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 10 monclient(hunting): my global_id is 214813 2024-09-18T14:19:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.353+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2c06ebe0 con 0x7f3f2c096a20 2024-09-18T14:19:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1654658957 0 0) 0x7f3f140034e0 con 0x7f3f2c06e810 2024-09-18T14:19:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 >> v1:172.21.15.45:6793/0 conn(0x7f3f2c138f10 legacy=0x7f3f2c13b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 >> v1:172.21.15.84:6789/0 conn(0x7f3f2c096a20 legacy=0x7f3f2c1377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 1 monclient: found mon.noname-d 2024-09-18T14:19:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:19:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c132990 con 0x7f3f2c06e810 2024-09-18T14:19:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3f1c001830 con 0x7f3f2c06e810 2024-09-18T14:19:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f31f9d640 5 monclient: authenticate success, global_id 185397 2024-09-18T14:19:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f31f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 10 monclient: dump: 2024-09-18T14:19:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.366 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f31f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3f1c001a10 con 0x7f3f2c06e810 2024-09-18T14:19:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 1 -- 172.21.15.45:0/3408675483 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3f1c004ba0 con 0x7f3f2c06e810 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f31f9d640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f31f9d640 4 set_mon_vals no callback set 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f2a7fc640 10 monclient: dump: 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f31f9d640 10 monclient: shutdown 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f31f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f31f9d640 1 -- 172.21.15.45:0/3408675483 >> v1:172.21.15.45:6792/0 conn(0x7f3f2c06e810 legacy=0x7f3f2c09ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f31f9d640 1 -- 172.21.15.45:0/3408675483 shutdown_connections 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f31f9d640 1 -- 172.21.15.45:0/3408675483 >> 172.21.15.45:0/3408675483 conn(0x7f3f2c0a5c20 msgr2=0x7f3f2c0a6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f31f9d640 1 -- 172.21.15.45:0/3408675483 shutdown_connections 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.354+0000 7f3f31f9d640 1 -- 172.21.15.45:0/3408675483 wait complete. 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.355+0000 7f3f31f9d640 10 monclient: build_initial_monmap 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.355+0000 7f3f31f9d640 10 monclient: monmap: 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.355+0000 7f3f31f9d640 1 Processor -- start 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.355+0000 7f3f31f9d640 1 -- start start 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.355+0000 7f3f31f9d640 10 monclient: init 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f31f9d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f31f9d640 10 monclient: _add_conns ranks=[7,4,6,1,3,8,2,5,0] 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f31f9d640 10 monclient(hunting): picked mon.noname-i con 0x7f3f2c06e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f31f9d640 10 monclient(hunting): picked mon.noname-c con 0x7f3f2c096a20 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f31f9d640 10 monclient(hunting): picked mon.noname-d con 0x7f3f2c138f10 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f31f9d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c09b550 con 0x7f3f2c096a20 2024-09-18T14:19:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f31f9d640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c141ae0 con 0x7f3f2c138f10 2024-09-18T14:19:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f31f9d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c142250 con 0x7f3f2c06e810 2024-09-18T14:19:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f31f9d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f31f9d640 10 monclient(hunting): authenticate will time out at 1980.694259s 2024-09-18T14:19:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f2bfff640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f3f2c138f10 0x7f3f2c149720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56538/0 (socket says 172.21.15.45:56538) 2024-09-18T14:19:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f2bfff640 1 -- 172.21.15.45:0/2794495245 learned_addr learned my addr 172.21.15.45:0/2794495245 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259015513 0 0) 0x7f3f2c141ae0 con 0x7f3f2c138f10 2024-09-18T14:19:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): my global_id is 185406 2024-09-18T14:19:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f180034e0 con 0x7f3f2c138f10 2024-09-18T14:19:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250023640 0 0) 0x7f3f2c09b550 con 0x7f3f2c096a20 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): my global_id is 165496 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2c141ae0 con 0x7f3f2c096a20 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931158989 0 0) 0x7f3f2c142250 con 0x7f3f2c06e810 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): my global_id is 214747 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2c09b550 con 0x7f3f2c06e810 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 468230908 0 0) 0x7f3f180034e0 con 0x7f3f2c138f10 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3f2c142250 con 0x7f3f2c138f10 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3f24001d60 con 0x7f3f2c138f10 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): dump: 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 862109584 0 0) 0x7f3f2c141ae0 con 0x7f3f2c096a20 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3f180034e0 con 0x7f3f2c096a20 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3f20001ae0 con 0x7f3f2c096a20 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): dump: 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4106673329 0 0) 0x7f3f2c09b550 con 0x7f3f2c06e810 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3f2c141ae0 con 0x7f3f2c06e810 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3f1c0017b0 con 0x7f3f2c06e810 2024-09-18T14:19:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.356+0000 7f3f137fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 10 monclient(hunting): dump: 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.376 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4053541613 0 0) 0x7f3f2c142250 con 0x7f3f2c138f10 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 >> v1:172.21.15.84:6792/0 conn(0x7f3f2c06e810 legacy=0x7f3f2c1413d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 >> v1:172.21.15.45:6791/0 conn(0x7f3f2c096a20 legacy=0x7f3f2c09e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 10 monclient: _finish_hunting 0 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 1 monclient: found mon.d 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c0a2270 con 0x7f3f2c138f10 2024-09-18T14:19:25.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 10 monclient: _finish_auth 0 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f31f9d640 5 monclient: authenticate success, global_id 185406 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f31f9d640 10 monclient: _renew_subs 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f31f9d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f31f9d640 1 -- 172.21.15.45:0/2794495245 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2c09b270 con 0x7f3f2c138f10 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f31f9d640 10 monclient: _renew_subs 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f31f9d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f31f9d640 1 -- 172.21.15.45:0/2794495245 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2c09e900 con 0x7f3f2c138f10 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3f24001fe0 con 0x7f3f2c138f10 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f3f24005110 con 0x7f3f2c138f10 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f29ffb640 4 set_mon_vals no callback set 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 10 monclient: dump: 2024-09-18T14:19:25.379 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.379 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.379 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.379 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.379 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.379 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.379 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.379 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.379 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.379 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.379 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f31f9d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f31f9d640 1 -- 172.21.15.45:0/2794495245 --> v1:172.21.15.45:6792/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3f0c000f20 con 0x7f3f2c138f10 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.357+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f3f240100e0 con 0x7f3f2c138f10 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.358+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f3f24046bb0 con 0x7f3f2c138f10 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.358+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 --> v1:172.21.15.84:6800/32025 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f18045270 con 0x7f3f18041610 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.358+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_get_version_reply(handle=1 version=899) v2 ==== 24+0+0 (unknown 3649037721 0 0) 0x7f3f24046fe0 con 0x7f3f2c138f10 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.358+0000 7f3f137fe640 10 monclient: handle_get_version_reply finishing 1 version 899 2024-09-18T14:19:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.358+0000 7f3f2b7fe640 10 monclient: get_auth_request con 0x7f3f1803aa80 auth_method 0 2024-09-18T14:19:25.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.358+0000 7f3f2affd640 10 monclient: get_auth_request con 0x7f3f18041610 auth_method 0 2024-09-18T14:19:25.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.359+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== osd.4 v1:172.21.15.84:6800/32025 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f3f18045270 con 0x7f3f18041610 2024-09-18T14:19:25.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.370+0000 7f3f31f9d640 1 -- 172.21.15.45:0/2794495245 --> v1:172.21.15.84:6800/32025 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3f0c002b60 con 0x7f3f18041610 2024-09-18T14:19:25.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.371+0000 7f3f137fe640 1 -- 172.21.15.45:0/2794495245 <== osd.4 v1:172.21.15.84:6800/32025 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f3f0c002b60 con 0x7f3f18041610 2024-09-18T14:19:25.382 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T14:19:25.382 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:25.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.371+0000 7f3f31f9d640 1 -- 172.21.15.45:0/2794495245 >> v1:172.21.15.84:6800/32025 conn(0x7f3f18041610 legacy=0x7f3f18043a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.371+0000 7f3f31f9d640 1 -- 172.21.15.45:0/2794495245 >> v1:172.21.15.84:6812/31478 conn(0x7f3f1803aa80 legacy=0x7f3f1803cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.371+0000 7f3f31f9d640 10 monclient: shutdown 2024-09-18T14:19:25.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.371+0000 7f3f31f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:25.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.371+0000 7f3f31f9d640 1 -- 172.21.15.45:0/2794495245 >> v1:172.21.15.45:6792/0 conn(0x7f3f2c138f10 legacy=0x7f3f2c149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.371+0000 7f3f2bfff640 1 -- 172.21.15.45:0/2794495245 reap_dead start 2024-09-18T14:19:25.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.371+0000 7f3f31f9d640 1 -- 172.21.15.45:0/2794495245 shutdown_connections 2024-09-18T14:19:25.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.371+0000 7f3f31f9d640 1 -- 172.21.15.45:0/2794495245 >> 172.21.15.45:0/2794495245 conn(0x7f3f2c0a5c20 msgr2=0x7f3f2c0a4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:25.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.371+0000 7f3f31f9d640 1 -- 172.21.15.45:0/2794495245 shutdown_connections 2024-09-18T14:19:25.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.371+0000 7f3f31f9d640 1 -- 172.21.15.45:0/2794495245 wait complete. 2024-09-18T14:19:25.384 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-18T14:19:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.483+0000 7fd0e340f640 10 monclient: get_monmap_and_config 2024-09-18T14:19:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.483+0000 7fd0e340f640 10 monclient: build_initial_monmap 2024-09-18T14:19:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.483+0000 7fd0e340f640 10 monclient: monmap: 2024-09-18T14:19:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:25.484136+0000 2024-09-18T14:19:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:25.484136+0000 2024-09-18T14:19:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:25.485 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.483+0000 7fd0e340f640 1 Processor -- start 2024-09-18T14:19:25.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.483+0000 7fd0e340f640 1 -- start start 2024-09-18T14:19:25.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e340f640 10 monclient: init 2024-09-18T14:19:25.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e340f640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:25.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e340f640 10 monclient: _add_conns ranks=[3,4,7,8,6,5,2,1,0] 2024-09-18T14:19:25.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e340f640 10 monclient(hunting): picked mon.noname-g con 0x7fd0dc09f7d0 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:25.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e340f640 10 monclient(hunting): picked mon.noname-c con 0x7fd0dc09fba0 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:25.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e340f640 10 monclient(hunting): picked mon.noname-i con 0x7fd0dc09ff70 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:25.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e340f640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc05ed70 con 0x7fd0dc09f7d0 2024-09-18T14:19:25.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e340f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc09dfc0 con 0x7fd0dc09fba0 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e340f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc09dce0 con 0x7fd0dc09ff70 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e340f640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e340f640 10 monclient(hunting): authenticate will time out at 1980.822499s 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e09aa640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fd0dc09fba0 0x7fd0dc137a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:37066/0 (socket says 172.21.15.45:37066) 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0e09aa640 1 -- 172.21.15.45:0/735390369 learned_addr learned my addr 172.21.15.45:0/735390369 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705848569 0 0) 0x7fd0dc09dfc0 con 0x7fd0dc09fba0 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0d3fff640 10 monclient(hunting): my global_id is 165505 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.484+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0c80034e0 con 0x7fd0dc09fba0 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824927705 0 0) 0x7fd0dc09dce0 con 0x7fd0dc09ff70 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient(hunting): my global_id is 214756 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0dc09dfc0 con 0x7fd0dc09ff70 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794084849 0 0) 0x7fd0dc05ed70 con 0x7fd0dc09f7d0 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient(hunting): my global_id is 224535 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0dc09dce0 con 0x7fd0dc09f7d0 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3166942909 0 0) 0x7fd0c80034e0 con 0x7fd0dc09fba0 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 >> v1:172.21.15.84:6792/0 conn(0x7fd0dc09ff70 legacy=0x7fd0dc13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 >> v1:172.21.15.84:6790/0 conn(0x7fd0dc09f7d0 legacy=0x7fd0dc132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 1 monclient: found mon.noname-c 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc0a1c00 con 0x7fd0dc09fba0 2024-09-18T14:19:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient: _finish_auth 0 2024-09-18T14:19:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd0cc0017b0 con 0x7fd0dc09fba0 2024-09-18T14:19:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0e340f640 5 monclient: authenticate success, global_id 165505 2024-09-18T14:19:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0e340f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:19:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient: dump: 2024-09-18T14:19:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0e340f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd0cc001910 con 0x7fd0dc09fba0 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 1 -- 172.21.15.45:0/735390369 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd0cc004ba0 con 0x7fd0dc09fba0 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0d3fff640 10 monclient: dump: 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.499 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0e340f640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0e340f640 4 set_mon_vals no callback set 2024-09-18T14:19:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0e340f640 10 monclient: shutdown 2024-09-18T14:19:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0e340f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0e340f640 1 -- 172.21.15.45:0/735390369 >> v1:172.21.15.45:6791/0 conn(0x7fd0dc09fba0 legacy=0x7fd0dc137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0e340f640 1 -- 172.21.15.45:0/735390369 shutdown_connections 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0e340f640 1 -- 172.21.15.45:0/735390369 >> 172.21.15.45:0/735390369 conn(0x7fd0dc0630b0 msgr2=0x7fd0dc0634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0e340f640 1 -- 172.21.15.45:0/735390369 shutdown_connections 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.485+0000 7fd0e340f640 1 -- 172.21.15.45:0/735390369 wait complete. 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.486+0000 7fd0e340f640 10 monclient: build_initial_monmap 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.486+0000 7fd0e340f640 10 monclient: monmap: 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.486+0000 7fd0e340f640 1 Processor -- start 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.486+0000 7fd0e340f640 1 -- start start 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.486+0000 7fd0e340f640 10 monclient: init 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0e340f640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0e340f640 10 monclient: _add_conns ranks=[1,7,4,2,8,0,6,3,5] 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0e340f640 10 monclient(hunting): picked mon.noname-f con 0x7fd0dc09f7d0 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0e340f640 10 monclient(hunting): picked mon.noname-i con 0x7fd0dc09fba0 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0e340f640 10 monclient(hunting): picked mon.noname-c con 0x7fd0dc09ff70 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0e340f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc09d310 con 0x7fd0dc09f7d0 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0e340f640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc0a2560 con 0x7fd0dc09ff70 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0e340f640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc142890 con 0x7fd0dc09fba0 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0e340f640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0e340f640 10 monclient(hunting): authenticate will time out at 1980.825550s 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0e19ac640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7fd0dc09ff70 0x7fd0dc14a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:37082/0 (socket says 172.21.15.45:37082) 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0e19ac640 1 -- 172.21.15.45:0/849782552 learned_addr learned my addr 172.21.15.45:0/849782552 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764681381 0 0) 0x7fd0dc0a2560 con 0x7fd0dc09ff70 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0d17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0d17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0d17fa640 10 monclient(hunting): my global_id is 165514 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.487+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0c00034e0 con 0x7fd0dc09ff70 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339616191 0 0) 0x7fd0dc09d310 con 0x7fd0dc09f7d0 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient(hunting): my global_id is 214822 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0dc0a2560 con 0x7fd0dc09f7d0 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137553151 0 0) 0x7fd0dc142890 con 0x7fd0dc09fba0 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient(hunting): my global_id is 214765 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0dc09d310 con 0x7fd0dc09fba0 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 855862332 0 0) 0x7fd0c00034e0 con 0x7fd0dc09ff70 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 --> v1:172.21.15.45:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd0dc142890 con 0x7fd0dc09ff70 2024-09-18T14:19:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd0d4002e00 con 0x7fd0dc09ff70 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient(hunting): dump: 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 <== mon.4 v1:172.21.15.45:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 135632481 0 0) 0x7fd0dc142890 con 0x7fd0dc09ff70 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 >> v1:172.21.15.84:6792/0 conn(0x7fd0dc09fba0 legacy=0x7fd0dc146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 >> v1:172.21.15.84:6789/0 conn(0x7fd0dc09f7d0 legacy=0x7fd0dc1317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient: _finish_hunting 0 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 monclient: found mon.c 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc11d9d0 con 0x7fd0dc09ff70 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient: _finish_auth 0 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0e340f640 5 monclient: authenticate success, global_id 165514 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0e340f640 10 monclient: _renew_subs 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0e340f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0e340f640 1 -- 172.21.15.45:0/849782552 --> v1:172.21.15.45:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0dc11e8c0 con 0x7fd0dc09ff70 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0e340f640 10 monclient: _renew_subs 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0e340f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0e340f640 1 -- 172.21.15.45:0/849782552 --> v1:172.21.15.45:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0dc136ea0 con 0x7fd0dc09ff70 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 <== mon.4 v1:172.21.15.45:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd0d4003040 con 0x7fd0dc09ff70 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 <== mon.4 v1:172.21.15.45:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7fd0d4001d30 con 0x7fd0dc09ff70 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d37fe640 4 set_mon_vals no callback set 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.488+0000 7fd0d17fa640 10 monclient: dump: 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.489+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 <== mon.4 v1:172.21.15.45:6791/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7fd0d4001e90 con 0x7fd0dc09ff70 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.489+0000 7fd0e340f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.489+0000 7fd0e340f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.489+0000 7fd0e340f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.45:6791/0 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.489+0000 7fd0e340f640 1 -- 172.21.15.45:0/849782552 --> v1:172.21.15.45:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0bc005010 con 0x7fd0dc09ff70 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.489+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 <== mon.4 v1:172.21.15.45:6791/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7fd0d4046bf0 con 0x7fd0dc09ff70 2024-09-18T14:19:25.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.489+0000 7fd0e11ab640 10 monclient: get_auth_request con 0x7fd0c0039f00 auth_method 0 2024-09-18T14:19:25.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.493+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 <== mon.4 v1:172.21.15.45:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7fd0d4015140 con 0x7fd0dc09ff70 2024-09-18T14:19:25.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.493+0000 7fd0d17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:25.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.493+0000 7fd0d17fa640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:25.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.659+0000 7fd0e340f640 1 -- 172.21.15.45:0/849782552 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0bc000b90 con 0x7fd0c0039f00 2024-09-18T14:19:25.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.660+0000 7fd0d17fa640 1 -- 172.21.15.45:0/849782552 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (unknown 3782719169 0 0) 0x7fd0bc000b90 con 0x7fd0c0039f00 2024-09-18T14:19:25.661 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 4 to deep-scrub 2024-09-18T14:19:25.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.661+0000 7fd0e340f640 1 -- 172.21.15.45:0/849782552 >> v1:172.21.15.84:6812/31478 conn(0x7fd0c0039f00 legacy=0x7fd0c003c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.661+0000 7fd0e340f640 10 monclient: shutdown 2024-09-18T14:19:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.661+0000 7fd0e340f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.661+0000 7fd0e340f640 1 -- 172.21.15.45:0/849782552 >> v1:172.21.15.45:6791/0 conn(0x7fd0dc09ff70 legacy=0x7fd0dc14a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.662+0000 7fd0e340f640 1 -- 172.21.15.45:0/849782552 shutdown_connections 2024-09-18T14:19:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.662+0000 7fd0e340f640 1 -- 172.21.15.45:0/849782552 >> 172.21.15.45:0/849782552 conn(0x7fd0dc0630b0 msgr2=0x7fd0dc0a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.662+0000 7fd0e340f640 1 -- 172.21.15.45:0/849782552 shutdown_connections 2024-09-18T14:19:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.662+0000 7fd0e340f640 1 -- 172.21.15.45:0/849782552 wait complete. 2024-09-18T14:19:25.673 INFO:tasks.ceph:Scrubbing osd.5 2024-09-18T14:19:25.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-18T14:19:25.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.772+0000 7f2819ad4640 10 monclient: get_monmap_and_config 2024-09-18T14:19:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.772+0000 7f2819ad4640 10 monclient: build_initial_monmap 2024-09-18T14:19:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.772+0000 7f2819ad4640 10 monclient: monmap: 2024-09-18T14:19:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:25.773178+0000 2024-09-18T14:19:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:25.773178+0000 2024-09-18T14:19:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:25.774 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:25.774 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:25.774 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:25.774 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:25.774 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:25.774 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:25.774 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.772+0000 7f2819ad4640 1 Processor -- start 2024-09-18T14:19:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2819ad4640 1 -- start start 2024-09-18T14:19:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2819ad4640 10 monclient: init 2024-09-18T14:19:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2819ad4640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2819ad4640 10 monclient: _add_conns ranks=[7,6,3,2,8,1,0,4,5] 2024-09-18T14:19:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2819ad4640 10 monclient(hunting): picked mon.noname-i con 0x7f281406e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2819ad4640 10 monclient(hunting): picked mon.noname-d con 0x7f2814096a20 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2819ad4640 10 monclient(hunting): picked mon.noname-g con 0x7f2814138f10 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2819ad4640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281405ed70 con 0x7f2814138f10 2024-09-18T14:19:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2819ad4640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28140a1b50 con 0x7f2814096a20 2024-09-18T14:19:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2819ad4640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281406ebe0 con 0x7f281406e810 2024-09-18T14:19:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2819ad4640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2819ad4640 10 monclient(hunting): authenticate will time out at 1981.111607s 2024-09-18T14:19:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f280affd640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f2814096a20 0x7f28141377f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56550/0 (socket says 172.21.15.45:56550) 2024-09-18T14:19:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f280affd640 1 -- 172.21.15.45:0/2672683656 learned_addr learned my addr 172.21.15.45:0/2672683656 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2812ffd640 1 -- 172.21.15.45:0/2672683656 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523359392 0 0) 0x7f28140a1b50 con 0x7f2814096a20 2024-09-18T14:19:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2812ffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2812ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2812ffd640 10 monclient(hunting): my global_id is 185415 2024-09-18T14:19:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.773+0000 7f2812ffd640 1 -- 172.21.15.45:0/2672683656 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28000034e0 con 0x7f2814096a20 2024-09-18T14:19:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 1 -- 172.21.15.45:0/2672683656 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 312915889 0 0) 0x7f28000034e0 con 0x7f2814096a20 2024-09-18T14:19:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 1 -- 172.21.15.45:0/2672683656 >> v1:172.21.15.84:6792/0 conn(0x7f281406e810 legacy=0x7f281409ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 1 -- 172.21.15.45:0/2672683656 >> v1:172.21.15.84:6790/0 conn(0x7f2814138f10 legacy=0x7f281413b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 10 monclient: _finish_hunting 0 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 1 monclient: found mon.noname-d 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.45:6792/0 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 1 -- 172.21.15.45:0/2672683656 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2814132990 con 0x7f2814096a20 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 10 monclient: _finish_auth 0 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 1 -- 172.21.15.45:0/2672683656 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f27fc001830 con 0x7f2814096a20 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 10 monclient: dump: 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.784 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2819ad4640 5 monclient: authenticate success, global_id 185415 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2819ad4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 1 -- 172.21.15.45:0/2672683656 <== mon.6 v1:172.21.15.45:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f27fc001a10 con 0x7f2814096a20 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2819ad4640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 1 -- 172.21.15.45:0/2672683656 <== mon.6 v1:172.21.15.45:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f27fc004ba0 con 0x7f2814096a20 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2819ad4640 4 set_mon_vals no callback set 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2812ffd640 10 monclient: dump: 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2819ad4640 10 monclient: shutdown 2024-09-18T14:19:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2819ad4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2819ad4640 1 -- 172.21.15.45:0/2672683656 >> v1:172.21.15.45:6792/0 conn(0x7f2814096a20 legacy=0x7f28141377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2819ad4640 1 -- 172.21.15.45:0/2672683656 shutdown_connections 2024-09-18T14:19:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2819ad4640 1 -- 172.21.15.45:0/2672683656 >> 172.21.15.45:0/2672683656 conn(0x7f28140a5c20 msgr2=0x7f28140a6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2819ad4640 1 -- 172.21.15.45:0/2672683656 shutdown_connections 2024-09-18T14:19:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.774+0000 7f2819ad4640 1 -- 172.21.15.45:0/2672683656 wait complete. 2024-09-18T14:19:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.775+0000 7f2819ad4640 10 monclient: build_initial_monmap 2024-09-18T14:19:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.775+0000 7f2819ad4640 10 monclient: monmap: 2024-09-18T14:19:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.775+0000 7f2819ad4640 1 Processor -- start 2024-09-18T14:19:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.775+0000 7f2819ad4640 1 -- start start 2024-09-18T14:19:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.775+0000 7f2819ad4640 10 monclient: init 2024-09-18T14:19:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f2819ad4640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f2819ad4640 10 monclient: _add_conns ranks=[5,2,3,6,0,7,1,8,4] 2024-09-18T14:19:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f2819ad4640 10 monclient(hunting): picked mon.noname-h con 0x7f281406e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f2819ad4640 10 monclient(hunting): picked mon.noname-b con 0x7f2814096a20 addr v1:172.21.15.45:6790/0 2024-09-18T14:19:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f2819ad4640 10 monclient(hunting): picked mon.noname-g con 0x7f2814138f10 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f2819ad4640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281409f9a0 con 0x7f2814096a20 2024-09-18T14:19:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f2819ad4640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28140a2270 con 0x7f2814138f10 2024-09-18T14:19:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f2819ad4640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2814131bc0 con 0x7f281406e810 2024-09-18T14:19:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f2819ad4640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f2819ad4640 10 monclient(hunting): authenticate will time out at 1981.114244s 2024-09-18T14:19:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280affd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f2814096a20 0x7f28141465a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:39816/0 (socket says 172.21.15.45:39816) 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280affd640 1 -- 172.21.15.45:0/3759091495 learned_addr learned my addr 172.21.15.45:0/3759091495 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818687250 0 0) 0x7f281409f9a0 con 0x7f2814096a20 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): my global_id is 234200 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27f40034e0 con 0x7f2814096a20 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466723421 0 0) 0x7f28140a2270 con 0x7f2814138f10 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): my global_id is 224553 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f281409f9a0 con 0x7f2814138f10 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702822343 0 0) 0x7f2814131bc0 con 0x7f281406e810 2024-09-18T14:19:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): my global_id is 185207 2024-09-18T14:19:25.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28140a2270 con 0x7f281406e810 2024-09-18T14:19:25.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1123899846 0 0) 0x7f27f40034e0 con 0x7f2814096a20 2024-09-18T14:19:25.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2814131bc0 con 0x7f2814096a20 2024-09-18T14:19:25.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f27fc004be0 con 0x7f2814096a20 2024-09-18T14:19:25.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-18T14:19:25.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): dump: 2024-09-18T14:19:25.791 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.792 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 304580435 0 0) 0x7f281409f9a0 con 0x7f2814138f10 2024-09-18T14:19:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27f40034e0 con 0x7f2814138f10 2024-09-18T14:19:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1587164755 0 0) 0x7f28140a2270 con 0x7f281406e810 2024-09-18T14:19:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f281409f9a0 con 0x7f281406e810 2024-09-18T14:19:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 659467415 0 0) 0x7f2814131bc0 con 0x7f2814096a20 2024-09-18T14:19:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.776+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 >> v1:172.21.15.84:6791/0 conn(0x7f281406e810 legacy=0x7f28141314b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 >> v1:172.21.15.84:6790/0 conn(0x7f2814138f10 legacy=0x7f2814149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 1 monclient: found mon.b 2024-09-18T14:19:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:25.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281409d4d0 con 0x7f2814096a20 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 10 monclient: _finish_auth 0 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f2819ad4640 5 monclient: authenticate success, global_id 234200 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f2819ad4640 10 monclient: _renew_subs 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f2819ad4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f2819ad4640 1 -- 172.21.15.45:0/3759091495 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281409ea90 con 0x7f2814096a20 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f2819ad4640 10 monclient: _renew_subs 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f2819ad4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f2819ad4640 1 -- 172.21.15.45:0/3759091495 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f281409c6c0 con 0x7f2814096a20 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f27fc0011f0 con 0x7f2814096a20 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f27fc001820 con 0x7f2814096a20 2024-09-18T14:19:25.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f28127fc640 4 set_mon_vals no callback set 2024-09-18T14:19:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-18T14:19:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 10 monclient: dump: 2024-09-18T14:19:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f27fc00ffd0 con 0x7f2814096a20 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f2819ad4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.45:6790/0 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.777+0000 7f2819ad4640 1 -- 172.21.15.45:0/3759091495 --> v1:172.21.15.45:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f27f8000f20 con 0x7f2814096a20 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.778+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f27fc046a30 con 0x7f2814096a20 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.778+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 --> v1:172.21.15.84:6808/32048 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27f4044c70 con 0x7f27f4041010 2024-09-18T14:19:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.778+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_get_version_reply(handle=1 version=899) v2 ==== 24+0+0 (unknown 3649037721 0 0) 0x7f27fc046e60 con 0x7f2814096a20 2024-09-18T14:19:25.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.778+0000 7f280bfff640 10 monclient: handle_get_version_reply finishing 1 version 899 2024-09-18T14:19:25.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.778+0000 7f28137fe640 10 monclient: get_auth_request con 0x7f27f403a520 auth_method 0 2024-09-18T14:19:25.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.778+0000 7f2813fff640 10 monclient: get_auth_request con 0x7f27f4041010 auth_method 0 2024-09-18T14:19:25.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.779+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== osd.5 v1:172.21.15.84:6808/32048 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f27f4044c70 con 0x7f27f4041010 2024-09-18T14:19:25.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.790+0000 7f2819ad4640 1 -- 172.21.15.45:0/3759091495 --> v1:172.21.15.84:6808/32048 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f27f8002b60 con 0x7f27f4041010 2024-09-18T14:19:25.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.791+0000 7f280bfff640 1 -- 172.21.15.45:0/3759091495 <== osd.5 v1:172.21.15.84:6808/32048 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f27f8002b60 con 0x7f27f4041010 2024-09-18T14:19:25.798 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-18T14:19:25.798 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T14:19:25.798 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-18T14:19:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.791+0000 7f2819ad4640 1 -- 172.21.15.45:0/3759091495 >> v1:172.21.15.84:6808/32048 conn(0x7f27f4041010 legacy=0x7f27f4043420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.791+0000 7f2819ad4640 1 -- 172.21.15.45:0/3759091495 >> v1:172.21.15.84:6812/31478 conn(0x7f27f403a520 legacy=0x7f27f403c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.791+0000 7f2819ad4640 10 monclient: shutdown 2024-09-18T14:19:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.791+0000 7f2819ad4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.791+0000 7f2819ad4640 1 -- 172.21.15.45:0/3759091495 >> v1:172.21.15.45:6790/0 conn(0x7f2814096a20 legacy=0x7f28141465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.791+0000 7f2813fff640 1 -- 172.21.15.45:0/3759091495 reap_dead start 2024-09-18T14:19:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.791+0000 7f2819ad4640 1 -- 172.21.15.45:0/3759091495 shutdown_connections 2024-09-18T14:19:25.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.791+0000 7f2819ad4640 1 -- 172.21.15.45:0/3759091495 >> 172.21.15.45:0/3759091495 conn(0x7f28140a5c20 msgr2=0x7f2814135250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:25.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.791+0000 7f2819ad4640 1 -- 172.21.15.45:0/3759091495 shutdown_connections 2024-09-18T14:19:25.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.791+0000 7f2819ad4640 1 -- 172.21.15.45:0/3759091495 wait complete. 2024-09-18T14:19:25.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-18T14:19:25.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.898+0000 7f6e6c4af640 10 monclient: get_monmap_and_config 2024-09-18T14:19:25.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.898+0000 7f6e6c4af640 10 monclient: build_initial_monmap 2024-09-18T14:19:25.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.898+0000 7f6e6c4af640 10 monclient: monmap: 2024-09-18T14:19:25.899 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:25.899 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.899 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:25.899108+0000 2024-09-18T14:19:25.899 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:25.899108+0000 2024-09-18T14:19:25.899 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:25.899 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:25.900 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.898+0000 7f6e6c4af640 1 Processor -- start 2024-09-18T14:19:25.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.898+0000 7f6e6c4af640 1 -- start start 2024-09-18T14:19:25.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.898+0000 7f6e6c4af640 10 monclient: init 2024-09-18T14:19:25.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e6c4af640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:25.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e6c4af640 10 monclient: _add_conns ranks=[5,8,1,3,6,2,4,0,7] 2024-09-18T14:19:25.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e6c4af640 10 monclient(hunting): picked mon.noname-h con 0x7f6e6406e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:25.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e6c4af640 10 monclient(hunting): picked mon.noname-e con 0x7f6e640a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:25.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e6c4af640 10 monclient(hunting): picked mon.noname-f con 0x7f6e640a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:25.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e6c4af640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6405ed70 con 0x7f6e640a1080 2024-09-18T14:19:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e6c4af640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e64097620 con 0x7f6e6406e810 2024-09-18T14:19:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e6c4af640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e64097340 con 0x7f6e640a0cb0 2024-09-18T14:19:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e6c4af640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e6c4af640 10 monclient(hunting): authenticate will time out at 1981.237451s 2024-09-18T14:19:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e69a4a640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f6e640a0cb0 0x7f6e64137810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:34990/0 (socket says 172.21.15.45:34990) 2024-09-18T14:19:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e69a4a640 1 -- 172.21.15.45:0/1103098968 learned_addr learned my addr 172.21.15.45:0/1103098968 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256486002 0 0) 0x7f6e64097340 con 0x7f6e640a0cb0 2024-09-18T14:19:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e69249640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e69249640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e69249640 10 monclient(hunting): my global_id is 234332 2024-09-18T14:19:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.899+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e500034e0 con 0x7f6e640a0cb0 2024-09-18T14:19:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158716515 0 0) 0x7f6e6405ed70 con 0x7f6e640a1080 2024-09-18T14:19:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient(hunting): my global_id is 214831 2024-09-18T14:19:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e64097340 con 0x7f6e640a1080 2024-09-18T14:19:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289236439 0 0) 0x7f6e64097620 con 0x7f6e6406e810 2024-09-18T14:19:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient(hunting): my global_id is 185216 2024-09-18T14:19:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e6405ed70 con 0x7f6e6406e810 2024-09-18T14:19:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 844827887 0 0) 0x7f6e500034e0 con 0x7f6e640a0cb0 2024-09-18T14:19:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 >> v1:172.21.15.84:6791/0 conn(0x7f6e6406e810 legacy=0x7f6e640a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 >> v1:172.21.15.84:6789/0 conn(0x7f6e640a1080 legacy=0x7f6e6413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient: _finish_hunting 0 2024-09-18T14:19:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 1 monclient: found mon.noname-e 2024-09-18T14:19:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:19:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e640a1660 con 0x7f6e640a0cb0 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient: _finish_auth 0 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e6c4af640 5 monclient: authenticate success, global_id 234332 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6e54001830 con 0x7f6e640a0cb0 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e6c4af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient: dump: 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.913 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e6c4af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6e54001a10 con 0x7f6e640a0cb0 2024-09-18T14:19:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e6c4af640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 1 -- 172.21.15.45:0/1103098968 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6e54004ba0 con 0x7f6e640a0cb0 2024-09-18T14:19:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e6c4af640 4 set_mon_vals no callback set 2024-09-18T14:19:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e69249640 10 monclient: dump: 2024-09-18T14:19:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e6c4af640 10 monclient: shutdown 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e6c4af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e6c4af640 1 -- 172.21.15.45:0/1103098968 >> v1:172.21.15.45:6793/0 conn(0x7f6e640a0cb0 legacy=0x7f6e64137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e6c4af640 1 -- 172.21.15.45:0/1103098968 shutdown_connections 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e6c4af640 1 -- 172.21.15.45:0/1103098968 >> 172.21.15.45:0/1103098968 conn(0x7f6e640a5bb0 msgr2=0x7f6e640a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e6c4af640 1 -- 172.21.15.45:0/1103098968 shutdown_connections 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.900+0000 7f6e6c4af640 1 -- 172.21.15.45:0/1103098968 wait complete. 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.901+0000 7f6e6c4af640 10 monclient: build_initial_monmap 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.901+0000 7f6e6c4af640 10 monclient: monmap: 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:25.916 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.901+0000 7f6e6c4af640 1 Processor -- start 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.901+0000 7f6e6c4af640 1 -- start start 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.901+0000 7f6e6c4af640 10 monclient: init 2024-09-18T14:19:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 10 monclient: _add_conns ranks=[5,6,1,7,3,2,4,8,0] 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 10 monclient(hunting): picked mon.noname-h con 0x7f6e6406e810 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 10 monclient(hunting): picked mon.noname-d con 0x7f6e640a0cb0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 10 monclient(hunting): picked mon.noname-f con 0x7f6e640a1080 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e641230e0 con 0x7f6e640a1080 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6413ded0 con 0x7f6e6406e810 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e64122b20 con 0x7f6e640a0cb0 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 10 monclient(hunting): authenticate will time out at 1981.240176s 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6a24b640 1 --1- >> v1:172.21.15.84:6791/0 conn(0x7f6e6406e810 0x7f6e6413d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.84:6791/0 says I am v1:172.21.15.45:60702/0 (socket says 172.21.15.45:60702) 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6a24b640 1 -- 172.21.15.45:0/3377122970 learned_addr learned my addr 172.21.15.45:0/3377122970 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e69a4a640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f6e640a0cb0 0x7f6e641469f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:56554/0 (socket says 172.21.15.45:56554) 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893239963 0 0) 0x7f6e64122b20 con 0x7f6e640a0cb0 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): my global_id is 185424 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e480034e0 con 0x7f6e640a0cb0 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084776450 0 0) 0x7f6e6413ded0 con 0x7f6e6406e810 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): my global_id is 185225 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e64122b20 con 0x7f6e6406e810 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210072199 0 0) 0x7f6e641230e0 con 0x7f6e640a1080 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): my global_id is 214840 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e6413ded0 con 0x7f6e640a1080 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6a24b640 1 -- 172.21.15.45:0/3377122970 >> v1:172.21.15.84:6791/0 conn(0x7f6e6406e810 legacy=0x7f6e6413d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-18T14:19:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6a24b640 1 -- 172.21.15.45:0/3377122970 >> v1:172.21.15.84:6791/0 conn(0x7f6e6406e810 legacy=0x7f6e6413d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6a24b640 1 --1- 172.21.15.45:0/3377122970 >> v1:172.21.15.84:6791/0 conn(0x7f6e6406e810 0x7f6e6413d7a0 :-1 s=OPENED pgs=625 cs=1 l=1).handle_message read tag failed 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6a24b640 1 --1- 172.21.15.45:0/3377122970 >> v1:172.21.15.84:6791/0 conn(0x7f6e6406e810 0x7f6e6413d7a0 :-1 s=OPENED pgs=625 cs=1 l=1).fault on lossy channel, failing 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.84:6791/0 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2605074801 0 0) 0x7f6e480034e0 con 0x7f6e640a0cb0 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6e641230e0 con 0x7f6e640a0cb0 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6e540017f0 con 0x7f6e640a0cb0 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): dump: 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2706091250 0 0) 0x7f6e6413ded0 con 0x7f6e640a1080 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6e480034e0 con 0x7f6e640a1080 2024-09-18T14:19:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6e5c001c30 con 0x7f6e640a1080 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient(hunting): dump: 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1398139810 0 0) 0x7f6e641230e0 con 0x7f6e640a0cb0 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 >> v1:172.21.15.84:6791/0 conn(0x7f6e6406e810 legacy=0x7f6e6413d7a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 >> v1:172.21.15.84:6789/0 conn(0x7f6e640a1080 legacy=0x7f6e6414a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 monclient: found mon.d 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e641206c0 con 0x7f6e640a0cb0 2024-09-18T14:19:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e5a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 5 monclient: authenticate success, global_id 185424 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 10 monclient: _renew_subs 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.902+0000 7f6e6c4af640 1 -- 172.21.15.45:0/3377122970 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e64122840 con 0x7f6e640a0cb0 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e6c4af640 10 monclient: _renew_subs 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e6c4af640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e6c4af640 1 -- 172.21.15.45:0/3377122970 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e64121970 con 0x7f6e640a0cb0 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6e54004c10 con 0x7f6e640a0cb0 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e5a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f6e540051a0 con 0x7f6e640a0cb0 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e5a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:25.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e68a48640 4 set_mon_vals no callback set 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e5a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e5a7fc640 10 monclient: dump: 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:25.925 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:25.926 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f6e54010170 con 0x7f6e640a0cb0 2024-09-18T14:19:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e6c4af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e6c4af640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e6c4af640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.903+0000 7f6e6c4af640 1 -- 172.21.15.45:0/3377122970 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4c005010 con 0x7f6e640a0cb0 2024-09-18T14:19:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.904+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f6e54046b50 con 0x7f6e640a0cb0 2024-09-18T14:19:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.904+0000 7f6e6a24b640 10 monclient: get_auth_request con 0x7f6e4803a5c0 auth_method 0 2024-09-18T14:19:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.908+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f6e54015590 con 0x7f6e640a0cb0 2024-09-18T14:19:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.908+0000 7f6e5a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:25.908+0000 7f6e5a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:26.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.072+0000 7f6e6c4af640 1 -- 172.21.15.45:0/3377122970 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e4c000b90 con 0x7f6e4803a5c0 2024-09-18T14:19:26.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.073+0000 7f6e5a7fc640 1 -- 172.21.15.45:0/3377122970 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (unknown 2264484800 0 0) 0x7f6e4c000b90 con 0x7f6e4803a5c0 2024-09-18T14:19:26.074 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 5 to deep-scrub 2024-09-18T14:19:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.075+0000 7f6e6c4af640 1 -- 172.21.15.45:0/3377122970 >> v1:172.21.15.84:6812/31478 conn(0x7f6e4803a5c0 legacy=0x7f6e4803ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.075+0000 7f6e6c4af640 10 monclient: shutdown 2024-09-18T14:19:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.075+0000 7f6e6c4af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.075+0000 7f6e6c4af640 1 -- 172.21.15.45:0/3377122970 >> v1:172.21.15.45:6792/0 conn(0x7f6e640a0cb0 legacy=0x7f6e641469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.075+0000 7f6e6c4af640 1 -- 172.21.15.45:0/3377122970 shutdown_connections 2024-09-18T14:19:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.075+0000 7f6e6c4af640 1 -- 172.21.15.45:0/3377122970 >> 172.21.15.45:0/3377122970 conn(0x7f6e640a5bb0 msgr2=0x7f6e640a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.075+0000 7f6e6c4af640 1 -- 172.21.15.45:0/3377122970 shutdown_connections 2024-09-18T14:19:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.075+0000 7f6e6c4af640 1 -- 172.21.15.45:0/3377122970 wait complete. 2024-09-18T14:19:26.086 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T14:19:26.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.182+0000 7f1adbc34640 10 monclient: get_monmap_and_config 2024-09-18T14:19:26.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.182+0000 7f1adbc34640 10 monclient: build_initial_monmap 2024-09-18T14:19:26.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.182+0000 7f1adbc34640 10 monclient: monmap: 2024-09-18T14:19:26.183 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:26.183 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:26.183 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:26.183010+0000 2024-09-18T14:19:26.183 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:26.183010+0000 2024-09-18T14:19:26.183 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:26.183 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:26.183 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:26.183 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:26.183 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:26.183 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:26.184 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:26.184 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:26.184 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:26.184 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:26.184 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:26.184 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:26.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.182+0000 7f1adbc34640 1 Processor -- start 2024-09-18T14:19:26.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.182+0000 7f1adbc34640 1 -- start start 2024-09-18T14:19:26.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.182+0000 7f1adbc34640 10 monclient: init 2024-09-18T14:19:26.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1adbc34640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:26.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1adbc34640 10 monclient: _add_conns ranks=[7,3,8,2,4,1,5,0,6] 2024-09-18T14:19:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1adbc34640 10 monclient(hunting): picked mon.noname-i con 0x7f1ad406e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1adbc34640 10 monclient(hunting): picked mon.noname-g con 0x7f1ad40a0cb0 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1adbc34640 10 monclient(hunting): picked mon.noname-e con 0x7f1ad40a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1adbc34640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad405ed70 con 0x7f1ad40a0cb0 2024-09-18T14:19:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1adbc34640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad4097620 con 0x7f1ad406e810 2024-09-18T14:19:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1adbc34640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad4097340 con 0x7f1ad40a1080 2024-09-18T14:19:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1adbc34640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1adbc34640 10 monclient(hunting): authenticate will time out at 1981.521548s 2024-09-18T14:19:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1ada1d1640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f1ad40a1080 0x7f1ad413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:35004/0 (socket says 172.21.15.45:35004) 2024-09-18T14:19:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1ada1d1640 1 -- 172.21.15.45:0/3142158505 learned_addr learned my addr 172.21.15.45:0/3142158505 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616532348 0 0) 0x7f1ad4097340 con 0x7f1ad40a1080 2024-09-18T14:19:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1ad89ce640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1ad89ce640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1ad89ce640 10 monclient(hunting): my global_id is 234341 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.183+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ac00034e0 con 0x7f1ad40a1080 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473914309 0 0) 0x7f1ad4097620 con 0x7f1ad406e810 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient(hunting): my global_id is 214783 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad4097340 con 0x7f1ad406e810 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716024730 0 0) 0x7f1ad405ed70 con 0x7f1ad40a0cb0 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient(hunting): my global_id is 224562 2024-09-18T14:19:26.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad4097620 con 0x7f1ad40a0cb0 2024-09-18T14:19:26.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3669820924 0 0) 0x7f1ac00034e0 con 0x7f1ad40a1080 2024-09-18T14:19:26.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 >> v1:172.21.15.84:6792/0 conn(0x7f1ad406e810 legacy=0x7f1ad40a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:26.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 >> v1:172.21.15.84:6790/0 conn(0x7f1ad40a0cb0 legacy=0x7f1ad4137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:26.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient: _finish_hunting 0 2024-09-18T14:19:26.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 1 monclient: found mon.noname-e 2024-09-18T14:19:26.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:26.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.45:6793/0 2024-09-18T14:19:26.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad40a1660 con 0x7f1ad40a1080 2024-09-18T14:19:26.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient: _finish_auth 0 2024-09-18T14:19:26.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:26.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1adbc34640 5 monclient: authenticate success, global_id 234341 2024-09-18T14:19:26.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1acc001bb0 con 0x7f1ad40a1080 2024-09-18T14:19:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient: dump: 2024-09-18T14:19:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1adbc34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 <== mon.8 v1:172.21.15.45:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1acc001d10 con 0x7f1ad40a1080 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1adbc34640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 1 -- 172.21.15.45:0/3142158505 <== mon.8 v1:172.21.15.45:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1acc004be0 con 0x7f1ad40a1080 2024-09-18T14:19:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1adbc34640 4 set_mon_vals no callback set 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1ad89ce640 10 monclient: dump: 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1adbc34640 10 monclient: shutdown 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1adbc34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1adbc34640 1 -- 172.21.15.45:0/3142158505 >> v1:172.21.15.45:6793/0 conn(0x7f1ad40a1080 legacy=0x7f1ad413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1adbc34640 1 -- 172.21.15.45:0/3142158505 shutdown_connections 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1adbc34640 1 -- 172.21.15.45:0/3142158505 >> 172.21.15.45:0/3142158505 conn(0x7f1ad40a5bb0 msgr2=0x7f1ad40a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.184+0000 7f1adbc34640 1 -- 172.21.15.45:0/3142158505 shutdown_connections 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.185+0000 7f1adbc34640 1 -- 172.21.15.45:0/3142158505 wait complete. 2024-09-18T14:19:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.185+0000 7f1adbc34640 10 monclient: build_initial_monmap 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.185+0000 7f1adbc34640 10 monclient: monmap: 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.185+0000 7f1adbc34640 1 Processor -- start 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.185+0000 7f1adbc34640 1 -- start start 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.185+0000 7f1adbc34640 10 monclient: init 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1adbc34640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1adbc34640 10 monclient: _add_conns ranks=[0,7,5,6,4,2,1,8,3] 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1adbc34640 10 monclient(hunting): picked mon.noname-a con 0x7f1ad406e810 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1adbc34640 10 monclient(hunting): picked mon.noname-i con 0x7f1ad40a0cb0 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1adbc34640 10 monclient(hunting): picked mon.noname-h con 0x7f1ad40a1080 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1adbc34640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad40a2270 con 0x7f1ad406e810 2024-09-18T14:19:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1adbc34640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad41212d0 con 0x7f1ad40a1080 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1adbc34640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad4142760 con 0x7f1ad40a0cb0 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1adbc34640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1adbc34640 10 monclient(hunting): authenticate will time out at 1981.524257s 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ad99d0640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1ad406e810 0x7f1ad41418f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58984/0 (socket says 172.21.15.45:58984) 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ad99d0640 1 -- 172.21.15.45:0/529849460 learned_addr learned my addr 172.21.15.45:0/529849460 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271485513 0 0) 0x7f1ad40a2270 con 0x7f1ad406e810 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): my global_id is 6786 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ab80034e0 con 0x7f1ad406e810 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048451927 0 0) 0x7f1ad41212d0 con 0x7f1ad40a1080 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): my global_id is 185234 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad40a2270 con 0x7f1ad40a1080 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458322369 0 0) 0x7f1ad4142760 con 0x7f1ad40a0cb0 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): my global_id is 214792 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad41212d0 con 0x7f1ad40a0cb0 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2374411402 0 0) 0x7f1ab80034e0 con 0x7f1ad406e810 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ad4142760 con 0x7f1ad406e810 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ad0002c10 con 0x7f1ad406e810 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): dump: 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.7 v1:172.21.15.84:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3746675836 0 0) 0x7f1ad41212d0 con 0x7f1ad40a0cb0 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 --> v1:172.21.15.84:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ab80034e0 con 0x7f1ad40a0cb0 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.7 v1:172.21.15.84:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ac4001b20 con 0x7f1ad40a0cb0 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.186+0000 7f1ac9ffb640 10 monclient(hunting): dump: 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 953921611 0 0) 0x7f1ad40a2270 con 0x7f1ad40a1080 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ad41212d0 con 0x7f1ad40a1080 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2918012198 0 0) 0x7f1ad4142760 con 0x7f1ad406e810 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 >> v1:172.21.15.84:6792/0 conn(0x7f1ad40a0cb0 legacy=0x7f1ad4135760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 >> v1:172.21.15.84:6791/0 conn(0x7f1ad40a1080 legacy=0x7f1ad4149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 10 monclient: _finish_hunting 0 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 1 monclient: found mon.a 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad409fa10 con 0x7f1ad406e810 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 10 monclient: _finish_auth 0 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1adbc34640 5 monclient: authenticate success, global_id 6786 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1adbc34640 10 monclient: _renew_subs 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1adbc34640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1adbc34640 1 -- 172.21.15.45:0/529849460 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad4121990 con 0x7f1ad406e810 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1ad00016a0 con 0x7f1ad406e810 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f1ad0001b90 con 0x7f1ad406e810 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1ac9ffb640 10 monclient: dump: 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:26.208 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:26.209 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1acbfff640 4 set_mon_vals no callback set 2024-09-18T14:19:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1adbc34640 10 monclient: _renew_subs 2024-09-18T14:19:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1adbc34640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.187+0000 7f1adbc34640 1 -- 172.21.15.45:0/529849460 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad4135e70 con 0x7f1ad406e810 2024-09-18T14:19:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.188+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f1ad0001d70 con 0x7f1ad406e810 2024-09-18T14:19:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.188+0000 7f1adbc34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.188+0000 7f1adbc34640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.188+0000 7f1adbc34640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.45:6789/0 2024-09-18T14:19:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.188+0000 7f1adbc34640 1 -- 172.21.15.45:0/529849460 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad404c4a0 con 0x7f1ad406e810 2024-09-18T14:19:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.188+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f1ad0046a00 con 0x7f1ad406e810 2024-09-18T14:19:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.188+0000 7f1ad91cf640 10 monclient: get_auth_request con 0x7f1ab803a8a0 auth_method 0 2024-09-18T14:19:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.192+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f1ad0011980 con 0x7f1ad406e810 2024-09-18T14:19:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.192+0000 7f1ac9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.192+0000 7f1ac9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:26.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.352+0000 7f1adbc34640 1 -- 172.21.15.45:0/529849460 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ad4122880 con 0x7f1ab803a8a0 2024-09-18T14:19:26.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.354+0000 7f1ac9ffb640 1 -- 172.21.15.45:0/529849460 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+53564 (unknown 2965378022 0 53571831) 0x7f1ad4122880 con 0x7f1ab803a8a0 2024-09-18T14:19:26.355 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T14:19:26.356 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-18T14:19:26.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.356+0000 7f1adbc34640 1 -- 172.21.15.45:0/529849460 >> v1:172.21.15.84:6812/31478 conn(0x7f1ab803a8a0 legacy=0x7f1ab803cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:26.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.356+0000 7f1adbc34640 10 monclient: shutdown 2024-09-18T14:19:26.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.356+0000 7f1adbc34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:26.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.356+0000 7f1adbc34640 1 -- 172.21.15.45:0/529849460 >> v1:172.21.15.45:6789/0 conn(0x7f1ad406e810 legacy=0x7f1ad41418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:26.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.356+0000 7f1adbc34640 1 -- 172.21.15.45:0/529849460 shutdown_connections 2024-09-18T14:19:26.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.356+0000 7f1adbc34640 1 -- 172.21.15.45:0/529849460 >> 172.21.15.45:0/529849460 conn(0x7f1ad40a5bb0 msgr2=0x7f1ad40a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:26.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.356+0000 7f1adbc34640 1 -- 172.21.15.45:0/529849460 shutdown_connections 2024-09-18T14:19:26.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:26.356+0000 7f1adbc34640 1 -- 172.21.15.45:0/529849460 wait complete. 2024-09-18T14:19:26.367 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":1507,"stamp":"2024-09-18T14:19:26.144772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1822840,"kb_used_data":13744,"kb_used_omap":9,"kb_used_meta":1809014,"kb_avail":564408200,"statfs":{"total":579820584960,"available":577953996800,"internally_reserved":0,"allocated":14073856,"data_stored":8160502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1852430352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002376"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":887,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:18:28.385274+0000","last_change":"2024-09-18T13:59:05.035941+0000","last_active":"2024-09-18T14:18:28.385274+0000","last_peered":"2024-09-18T14:18:28.385274+0000","last_clean":"2024-09-18T14:18:28.385274+0000","last_became_active":"2024-09-18T13:59:03.705607+0000","last_became_peered":"2024-09-18T13:59:03.705607+0000","last_unstale":"2024-09-18T14:18:28.385274+0000","last_undegraded":"2024-09-18T14:18:28.385274+0000","last_fullsized":"2024-09-18T14:18:28.385274+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000366475,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":875,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.441537+0000","last_change":"2024-09-18T13:59:05.875836+0000","last_active":"2024-09-18T14:18:24.441537+0000","last_peered":"2024-09-18T14:18:24.441537+0000","last_clean":"2024-09-18T14:18:24.441537+0000","last_became_active":"2024-09-18T13:59:03.705710+0000","last_became_peered":"2024-09-18T13:59:03.705710+0000","last_unstale":"2024-09-18T14:18:24.441537+0000","last_undegraded":"2024-09-18T14:18:24.441537+0000","last_fullsized":"2024-09-18T14:18:24.441537+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:12:55.202686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029541399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":890,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:18:29.214897+0000","last_change":"2024-09-18T13:59:05.036440+0000","last_active":"2024-09-18T14:18:29.214897+0000","last_peered":"2024-09-18T14:18:29.214897+0000","last_clean":"2024-09-18T14:18:29.214897+0000","last_became_active":"2024-09-18T13:59:03.705659+0000","last_became_peered":"2024-09-18T13:59:03.705659+0000","last_unstale":"2024-09-18T14:18:29.214897+0000","last_undegraded":"2024-09-18T14:18:29.214897+0000","last_fullsized":"2024-09-18T14:18:29.214897+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037781799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":879,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.441619+0000","last_change":"2024-09-18T13:59:05.036690+0000","last_active":"2024-09-18T14:18:24.441619+0000","last_peered":"2024-09-18T14:18:24.441619+0000","last_clean":"2024-09-18T14:18:24.441619+0000","last_became_active":"2024-09-18T13:59:02.977312+0000","last_became_peered":"2024-09-18T13:59:02.977312+0000","last_unstale":"2024-09-18T14:18:24.441619+0000","last_undegraded":"2024-09-18T14:18:24.441619+0000","last_fullsized":"2024-09-18T14:18:24.441619+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:03:06.377092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072047099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'72","reported_seq":972,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:18:28.385193+0000","last_change":"2024-09-18T13:59:02.975019+0000","last_active":"2024-09-18T14:18:28.385193+0000","last_peered":"2024-09-18T14:18:28.385193+0000","last_clean":"2024-09-18T14:18:28.385193+0000","last_became_active":"2024-09-18T13:59:02.974164+0000","last_became_peered":"2024-09-18T13:59:02.974164+0000","last_unstale":"2024-09-18T14:18:28.385193+0000","last_undegraded":"2024-09-18T14:18:28.385193+0000","last_fullsized":"2024-09-18T14:18:28.385193+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":878,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.441141+0000","last_change":"2024-09-18T13:59:05.036936+0000","last_active":"2024-09-18T14:18:24.441141+0000","last_peered":"2024-09-18T14:18:24.441141+0000","last_clean":"2024-09-18T14:18:24.441141+0000","last_became_active":"2024-09-18T13:59:02.977186+0000","last_became_peered":"2024-09-18T13:59:02.977186+0000","last_unstale":"2024-09-18T14:18:24.441141+0000","last_undegraded":"2024-09-18T14:18:24.441141+0000","last_fullsized":"2024-09-18T14:18:24.441141+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:01:46.516709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065897000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":888,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:18:29.214896+0000","last_change":"2024-09-18T13:59:05.036455+0000","last_active":"2024-09-18T14:18:29.214896+0000","last_peered":"2024-09-18T14:18:29.214896+0000","last_clean":"2024-09-18T14:18:29.214896+0000","last_became_active":"2024-09-18T13:59:02.974240+0000","last_became_peered":"2024-09-18T13:59:02.974240+0000","last_unstale":"2024-09-18T14:18:29.214896+0000","last_undegraded":"2024-09-18T14:18:29.214896+0000","last_fullsized":"2024-09-18T14:18:29.214896+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038595099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":887,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:18:28.385210+0000","last_change":"2024-09-18T13:59:05.035951+0000","last_active":"2024-09-18T14:18:28.385210+0000","last_peered":"2024-09-18T14:18:28.385210+0000","last_clean":"2024-09-18T14:18:28.385210+0000","last_became_active":"2024-09-18T13:59:03.705691+0000","last_became_peered":"2024-09-18T13:59:03.705691+0000","last_unstale":"2024-09-18T14:18:28.385210+0000","last_undegraded":"2024-09-18T14:18:28.385210+0000","last_fullsized":"2024-09-18T14:18:28.385210+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033710499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":875,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-18T14:18:24.443578+0000","last_change":"2024-09-18T13:59:05.875874+0000","last_active":"2024-09-18T14:18:24.443578+0000","last_peered":"2024-09-18T14:18:24.443578+0000","last_clean":"2024-09-18T14:18:24.443578+0000","last_became_active":"2024-09-18T13:59:02.976936+0000","last_became_peered":"2024-09-18T13:59:02.976936+0000","last_unstale":"2024-09-18T14:18:24.443578+0000","last_undegraded":"2024-09-18T14:18:24.443578+0000","last_fullsized":"2024-09-18T14:18:24.443578+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T13:59:01.955109+0000","last_clean_scrub_stamp":"2024-09-18T13:59:01.955109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T20:20:37.890959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036210499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607799,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312472,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059368,"statfs":{"total":96636764160,"available":96316792832,"internally_reserved":0,"allocated":2699264,"data_stored":1712696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.35099999999999998,"15min":0.34999999999999998},"min":{"1min":0.28100000000000003,"5min":0.23300000000000001,"15min":0.19},"max":{"1min":0.441,"5min":0.54000000000000004,"15min":0.91700000000000004},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.35399999999999998,"15min":0.36499999999999999},"min":{"1min":0.29899999999999999,"5min":0.23999999999999999,"15min":0.219},"max":{"1min":0.46700000000000003,"5min":0.55700000000000005,"15min":1.0740000000000001},"last":0.28799999999999998}]},{"osd":2,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.48699999999999999,"5min":0.53200000000000003,"15min":1.133},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.71899999999999997},"last":0.30399999999999999}]},{"osd":3,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52900000000000003,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.747,"15min":0.871},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.64200000000000002,"5min":0.74299999999999999,"15min":1.034},"last":0.375}]},{"osd":4,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.50800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73799999999999999,"5min":0.76400000000000001,"15min":0.995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.58299999999999996},"min":{"1min":0.48899999999999999,"5min":0.35499999999999998,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.80000000000000004,"15min":1.323},"last":0.42099999999999999}]},{"osd":5,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.55800000000000005},"min":{"1min":0.44,"5min":0.378,"15min":0.33200000000000002},"max":{"1min":0.67500000000000004,"5min":0.77700000000000002,"15min":0.87},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.54300000000000004},"min":{"1min":0.41899999999999998,"5min":0.315,"15min":0.28000000000000003},"max":{"1min":0.75700000000000001,"5min":0.77200000000000002,"15min":1.2669999999999999},"last":0.44400000000000001}]}]},{"osd":5,"up_from":11,"seq":47244640503,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299472,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072368,"statfs":{"total":96636764160,"available":96330104832,"internally_reserved":0,"allocated":2166784,"data_stored":1183768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.53400000000000003},"min":{"1min":0.318,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":2.6800000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52300000000000002,"15min":0.54600000000000004},"min":{"1min":0.39900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":3.206},"last":0.45500000000000002}]},{"osd":1,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47899999999999998,"15min":0.48799999999999999},"min":{"1min":0.36299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64800000000000002,"5min":1.335,"15min":1.335},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65300000000000002,"5min":0.69399999999999995,"15min":2.3610000000000002},"last":0.33400000000000002}]},{"osd":2,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48999999999999999,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65700000000000003,"5min":1.25,"15min":2.976},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":2.8719999999999999},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.35399999999999998},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.372},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.375},"min":{"1min":0.27300000000000002,"5min":0.224,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.629,"15min":0.69099999999999995},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.29399999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.68999999999999995,"5min":1.194,"15min":1.194},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.219},"max":{"1min":0.56200000000000006,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.42799999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640503,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307796,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064044,"statfs":{"total":96636764160,"available":96321581056,"internally_reserved":0,"allocated":2170880,"data_stored":1183787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.52400000000000002},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.183},"max":{"1min":0.66000000000000003,"5min":0.745,"15min":0.81799999999999995},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.26300000000000001},"max":{"1min":0.67200000000000004,"5min":0.80300000000000005,"15min":0.83299999999999996},"last":0.35699999999999998}]},{"osd":1,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46700000000000003,"15min":0.47499999999999998},"min":{"1min":0.33400000000000002,"5min":0.314,"15min":0.26500000000000001},"max":{"1min":0.59299999999999997,"5min":0.77300000000000002,"15min":0.79200000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.47799999999999998,"15min":0.48499999999999999},"min":{"1min":0.377,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.71399999999999997},"last":0.45300000000000001}]},{"osd":2,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.252,"15min":0.246},"max":{"1min":0.59899999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.437},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.32800000000000001,"5min":0.30399999999999999,"15min":0.28799999999999998},"max":{"1min":0.57899999999999996,"5min":0.68100000000000005,"15min":0.84599999999999997},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.28100000000000003,"5min":0.22500000000000001,"15min":0.20499999999999999},"max":{"1min":0.58399999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.23699999999999999,"5min":0.23599999999999999,"15min":0.22},"max":{"1min":0.58799999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41099999999999998}]},{"osd":5,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26000000000000001,"5min":0.20899999999999999,"15min":0.20499999999999999},"max":{"1min":0.50600000000000001,"5min":0.64600000000000002,"15min":0.753},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.44400000000000001}]}]},{"osd":3,"up_from":11,"seq":47244640503,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299992,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071848,"statfs":{"total":96636764160,"available":96329572352,"internally_reserved":0,"allocated":2699264,"data_stored":1712696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.31,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.624,"5min":0.66900000000000004,"15min":1.9530000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48999999999999999,"15min":0.47899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.26200000000000001},"max":{"1min":0.68000000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.42399999999999999}]},{"osd":1,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47899999999999998,"15min":0.47099999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.28100000000000003,"15min":0.27500000000000002},"max":{"1min":0.65500000000000003,"5min":0.66200000000000003,"15min":2.5190000000000001},"last":0.307}]},{"osd":2,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52100000000000002,"15min":0.52900000000000003},"min":{"1min":0.376,"5min":0.317,"15min":0.317},"max":{"1min":0.63,"5min":0.72999999999999998,"15min":2.3660000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.30099999999999999},"max":{"1min":0.61599999999999999,"5min":0.71899999999999997,"15min":2.3919999999999999},"last":0.34799999999999998}]},{"osd":4,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35799999999999998},"min":{"1min":0.224,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.68899999999999995},"last":0.44},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.372},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.20399999999999999},"max":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.63800000000000001},"last":0.36499999999999999}]},{"osd":5,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.379,"15min":0.379},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.218},"max":{"1min":0.54500000000000004,"5min":0.58399999999999996,"15min":0.70499999999999996},"last":0.38},{"interface":"front","average":{"1min":0.435,"5min":0.41599999999999998,"15min":0.40799999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.748},"last":0.40999999999999998}]}]},{"osd":2,"up_from":11,"seq":47244640503,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299472,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072368,"statfs":{"total":96636764160,"available":96330104832,"internally_reserved":0,"allocated":2166784,"data_stored":1183768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32000000000000001,"15min":0.33100000000000002},"min":{"1min":0.22600000000000001,"5min":0.20399999999999999,"15min":0.20300000000000001},"max":{"1min":0.40899999999999997,"5min":0.53200000000000003,"15min":0.875},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.25,"5min":0.19,"15min":0.19},"max":{"1min":0.47899999999999998,"5min":1.1100000000000001,"15min":1.357},"last":0.24099999999999999}]},{"osd":1,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.28100000000000003,"15min":0.25},"max":{"1min":0.56499999999999995,"5min":0.76700000000000002,"15min":4.117},"last":0.307},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.379,"15min":0.39900000000000002},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.217},"max":{"1min":0.46500000000000002,"5min":0.629,"15min":4.0529999999999999},"last":0.31900000000000001}]},{"osd":3,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51100000000000001,"15min":0.52200000000000002},"min":{"1min":0.38300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.749,"5min":0.77200000000000002,"15min":1.3049999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.502,"15min":0.52500000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.29599999999999999},"max":{"1min":0.69799999999999995,"5min":1.026,"15min":1.506},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42199999999999999,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.86299999999999999,"5min":0.94399999999999995,"15min":1.2789999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.30499999999999999},"max":{"1min":0.77400000000000002,"5min":1.2669999999999999,"15min":2.3159999999999998},"last":0.48799999999999999}]},{"osd":5,"last update":"Wed Sep 18 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53000000000000003,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.35599999999999998,"15min":0.34200000000000003},"max":{"1min":0.92700000000000005,"5min":1.155,"15min":1.224},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.40100000000000002,"5min":0.35999999999999999,"15min":0.34599999999999997},"max":{"1min":0.69799999999999995,"5min":1.0409999999999999,"15min":1.2629999999999999},"last":0.40999999999999998}]}]},{"osd":0,"up_from":11,"seq":47244640503,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303636,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068204,"statfs":{"total":96636764160,"available":96325840896,"internally_reserved":0,"allocated":2170880,"data_stored":1183787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.34399999999999997},"min":{"1min":0.219,"5min":0.187,"15min":0.17199999999999999},"max":{"1min":0.495,"5min":0.53100000000000003,"15min":0.628},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.29799999999999999,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.49299999999999999,"5min":0.57699999999999996,"15min":0.72299999999999998},"last":0.28799999999999998}]},{"osd":2,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34100000000000003,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.629},"last":0.216},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.26300000000000001,"5min":0.25,"15min":0.224},"max":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.71499999999999997},"last":0.248}]},{"osd":3,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.55300000000000005},"min":{"1min":0.41099999999999998,"5min":0.29599999999999999,"15min":0.28599999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.88200000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.54900000000000004},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.29899999999999999},"max":{"1min":0.64300000000000002,"5min":0.75600000000000001,"15min":0.88900000000000001},"last":0.53600000000000003}]},{"osd":4,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.39100000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.71299999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.63},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.58399999999999996},"min":{"1min":0.47499999999999998,"5min":0.36299999999999999,"15min":0.28000000000000003},"max":{"1min":0.747,"5min":0.79400000000000004,"15min":0.91700000000000004},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Sep 18 14:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.29799999999999999,"15min":0.28699999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.84899999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.73199999999999998,"5min":0.84599999999999997,"15min":1.1479999999999999},"last":0.55200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T14:19:26.372 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-18T13:59:01.955109+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=13, tm_min=59, tm_sec=1, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=14, tm_min=19, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T14:19:26.372 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-18T13:59:01.955109+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=13, tm_min=59, tm_sec=1, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=14, tm_min=19, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T14:19:26.373 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-18T13:59:01.955109+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=13, tm_min=59, tm_sec=1, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=14, tm_min=19, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T14:19:26.373 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-18T13:59:01.955109+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=13, tm_min=59, tm_sec=1, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=14, tm_min=19, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T14:19:26.373 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-18T13:59:01.955109+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=13, tm_min=59, tm_sec=1, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=14, tm_min=19, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T14:19:26.373 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-18T13:59:01.955109+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=13, tm_min=59, tm_sec=1, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=14, tm_min=19, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T14:19:26.373 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-18T13:59:01.955109+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=13, tm_min=59, tm_sec=1, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=14, tm_min=19, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T14:19:26.373 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-18T13:59:01.955109+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=13, tm_min=59, tm_sec=1, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=14, tm_min=19, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T14:19:26.373 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-18T13:59:01.955109+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=13, tm_min=59, tm_sec=1, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=14, tm_min=19, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T14:19:26.373 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-18T14:19:46.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T14:19:46.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.484+0000 7f289899d640 10 monclient: get_monmap_and_config 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.484+0000 7f289899d640 10 monclient: build_initial_monmap 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.484+0000 7f289899d640 10 monclient: monmap: 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:46.486094+0000 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:46.486094+0000 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:46.487 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.484+0000 7f289899d640 1 Processor -- start 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.484+0000 7f289899d640 1 -- start start 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.484+0000 7f289899d640 10 monclient: init 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289899d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289899d640 10 monclient: _add_conns ranks=[7,8,4,3,0,1,6,2,5] 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289899d640 10 monclient(hunting): picked mon.noname-i con 0x7f289406e810 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289899d640 10 monclient(hunting): picked mon.noname-e con 0x7f28940a0cb0 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289899d640 10 monclient(hunting): picked mon.noname-c con 0x7f28940a1080 addr v1:172.21.15.45:6791/0 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289899d640 1 -- --> v1:172.21.15.45:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289405ed70 con 0x7f28940a1080 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289899d640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2894097620 con 0x7f289406e810 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289899d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2894097340 con 0x7f28940a0cb0 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289899d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289899d640 10 monclient(hunting): authenticate will time out at 2001.824529s 2024-09-18T14:19:46.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f2892d9d640 1 --1- >> v1:172.21.15.45:6791/0 conn(0x7f28940a1080 0x7f289413af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6791/0 says I am v1:172.21.15.45:39028/0 (socket says 172.21.15.45:39028) 2024-09-18T14:19:46.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f2892d9d640 1 -- 172.21.15.45:0/1434419509 learned_addr learned my addr 172.21.15.45:0/1434419509 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:46.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 <== mon.4 v1:172.21.15.45:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850928652 0 0) 0x7f289405ed70 con 0x7f28940a1080 2024-09-18T14:19:46.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:46.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:46.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289159a640 10 monclient(hunting): my global_id is 165523 2024-09-18T14:19:46.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 --> v1:172.21.15.45:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f287c0034e0 con 0x7f28940a1080 2024-09-18T14:19:46.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383657362 0 0) 0x7f2894097340 con 0x7f28940a0cb0 2024-09-18T14:19:46.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:46.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.485+0000 7f289159a640 10 monclient(hunting): my global_id is 234350 2024-09-18T14:19:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f289405ed70 con 0x7f28940a0cb0 2024-09-18T14:19:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889148004 0 0) 0x7f2894097620 con 0x7f289406e810 2024-09-18T14:19:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient(hunting): my global_id is 214801 2024-09-18T14:19:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2894097340 con 0x7f289406e810 2024-09-18T14:19:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 <== mon.4 v1:172.21.15.45:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2950328124 0 0) 0x7f287c0034e0 con 0x7f28940a1080 2024-09-18T14:19:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 >> v1:172.21.15.45:6793/0 conn(0x7f28940a0cb0 legacy=0x7f2894137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 >> v1:172.21.15.84:6792/0 conn(0x7f289406e810 legacy=0x7f28940a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient: _finish_hunting 0 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 1 monclient: found mon.noname-c 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.45:6791/0 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 --> v1:172.21.15.45:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940a1660 con 0x7f28940a1080 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient: _finish_auth 0 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 <== mon.4 v1:172.21.15.45:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f288c001bb0 con 0x7f28940a1080 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient: dump: 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289899d640 5 monclient: authenticate success, global_id 165523 2024-09-18T14:19:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289899d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 <== mon.4 v1:172.21.15.45:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f288c001d90 con 0x7f28940a1080 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 1 -- 172.21.15.45:0/1434419509 <== mon.4 v1:172.21.15.45:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f288c004f20 con 0x7f28940a1080 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289159a640 10 monclient: dump: 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289899d640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289899d640 4 set_mon_vals no callback set 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289899d640 10 monclient: shutdown 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289899d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.486+0000 7f289899d640 1 -- 172.21.15.45:0/1434419509 >> v1:172.21.15.45:6791/0 conn(0x7f28940a1080 legacy=0x7f289413af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.487+0000 7f289899d640 1 -- 172.21.15.45:0/1434419509 shutdown_connections 2024-09-18T14:19:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.487+0000 7f289899d640 1 -- 172.21.15.45:0/1434419509 >> 172.21.15.45:0/1434419509 conn(0x7f28940a5bb0 msgr2=0x7f28940a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.487+0000 7f289899d640 1 -- 172.21.15.45:0/1434419509 shutdown_connections 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.487+0000 7f289899d640 1 -- 172.21.15.45:0/1434419509 wait complete. 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.487+0000 7f289899d640 10 monclient: build_initial_monmap 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.487+0000 7f289899d640 10 monclient: monmap: 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.487+0000 7f289899d640 1 Processor -- start 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.487+0000 7f289899d640 1 -- start start 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.487+0000 7f289899d640 10 monclient: init 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f289899d640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f289899d640 10 monclient: _add_conns ranks=[3,5,8,1,6,4,2,0,7] 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f289899d640 10 monclient(hunting): picked mon.noname-g con 0x7f289406e810 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f289899d640 10 monclient(hunting): picked mon.noname-h con 0x7f28940a0cb0 addr v1:172.21.15.84:6791/0 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f289899d640 10 monclient(hunting): picked mon.noname-e con 0x7f28940a1080 addr v1:172.21.15.45:6793/0 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f289899d640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2894122d50 con 0x7f289406e810 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f289899d640 1 -- --> v1:172.21.15.84:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289413db40 con 0x7f28940a0cb0 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f289899d640 1 -- --> v1:172.21.15.45:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2894122790 con 0x7f28940a1080 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f289899d640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f289899d640 10 monclient(hunting): authenticate will time out at 2001.827449s 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f2892d9d640 1 --1- >> v1:172.21.15.45:6793/0 conn(0x7f28940a1080 0x7f2894149d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6793/0 says I am v1:172.21.15.45:57242/0 (socket says 172.21.15.45:57242) 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f2892d9d640 1 -- 172.21.15.45:0/796028268 learned_addr learned my addr 172.21.15.45:0/796028268 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.8 v1:172.21.15.45:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542064493 0 0) 0x7f2894122790 con 0x7f28940a1080 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 10 monclient(hunting): my global_id is 234359 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 --> v1:172.21.15.45:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28740034e0 con 0x7f28940a1080 2024-09-18T14:19:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075098003 0 0) 0x7f2894122d50 con 0x7f289406e810 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 10 monclient(hunting): my global_id is 224571 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2894122790 con 0x7f289406e810 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.5 v1:172.21.15.84:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933662682 0 0) 0x7f289413db40 con 0x7f28940a0cb0 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 10 monclient(hunting): my global_id is 185243 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.488+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 --> v1:172.21.15.84:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2894122d50 con 0x7f28940a0cb0 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.8 v1:172.21.15.45:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1929879141 0 0) 0x7f28740034e0 con 0x7f28940a1080 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 --> v1:172.21.15.45:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f289413db40 con 0x7f28940a1080 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.8 v1:172.21.15.45:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f288c001b70 con 0x7f28940a1080 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient(hunting): dump: 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1054126572 0 0) 0x7f2894122790 con 0x7f289406e810 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f28740034e0 con 0x7f289406e810 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f2884001a60 con 0x7f289406e810 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient(hunting): dump: 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.5 v1:172.21.15.84:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2788759299 0 0) 0x7f2894122d50 con 0x7f28940a0cb0 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 --> v1:172.21.15.84:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2894122790 con 0x7f28940a0cb0 2024-09-18T14:19:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.5 v1:172.21.15.84:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f28880016c0 con 0x7f28940a0cb0 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient(hunting): dump: 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.8 v1:172.21.15.45:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2807279957 0 0) 0x7f289413db40 con 0x7f28940a1080 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 >> v1:172.21.15.84:6791/0 conn(0x7f28940a0cb0 legacy=0x7f2894146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 >> v1:172.21.15.84:6790/0 conn(0x7f289406e810 legacy=0x7f289413d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient: _finish_hunting 0 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 monclient: found mon.e 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 --> v1:172.21.15.45:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28941202a0 con 0x7f28940a1080 2024-09-18T14:19:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient: _finish_auth 0 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f289899d640 5 monclient: authenticate success, global_id 234359 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f289899d640 10 monclient: _renew_subs 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f289899d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f289899d640 1 -- 172.21.15.45:0/796028268 --> v1:172.21.15.45:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28941224b0 con 0x7f28940a1080 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f289899d640 10 monclient: _renew_subs 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f289899d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f289899d640 1 -- 172.21.15.45:0/796028268 --> v1:172.21.15.45:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f28941215e0 con 0x7f28940a1080 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.8 v1:172.21.15.45:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f288c0015b0 con 0x7f28940a1080 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.8 v1:172.21.15.45:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f288c005150 con 0x7f28940a1080 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f2890d99640 4 set_mon_vals no callback set 2024-09-18T14:19:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.489+0000 7f28827fc640 10 monclient: dump: 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:46.513 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.490+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.8 v1:172.21.15.45:6793/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f288c0052b0 con 0x7f28940a1080 2024-09-18T14:19:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.490+0000 7f289899d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.490+0000 7f289899d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.490+0000 7f289899d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.45:6793/0 2024-09-18T14:19:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.490+0000 7f289899d640 1 -- 172.21.15.45:0/796028268 --> v1:172.21.15.45:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2878005010 con 0x7f28940a1080 2024-09-18T14:19:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.490+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.8 v1:172.21.15.45:6793/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f288c046c70 con 0x7f28940a1080 2024-09-18T14:19:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.490+0000 7f289259c640 10 monclient: get_auth_request con 0x7f287403a950 auth_method 0 2024-09-18T14:19:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.495+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mon.8 v1:172.21.15.45:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f288c0068d0 con 0x7f28940a1080 2024-09-18T14:19:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.495+0000 7f28827fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.495+0000 7f28827fc640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:46.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.659+0000 7f289899d640 1 -- 172.21.15.45:0/796028268 --> v1:172.21.15.84:6812/31478 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2878000b90 con 0x7f287403a950 2024-09-18T14:19:46.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.662+0000 7f28827fc640 1 -- 172.21.15.45:0/796028268 <== mgr.4116 v1:172.21.15.84:6812/31478 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+54014 (unknown 2965378022 0 356241353) 0x7f2878000b90 con 0x7f287403a950 2024-09-18T14:19:46.663 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-18T14:19:46.664 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-18T14:19:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.663+0000 7f289899d640 1 -- 172.21.15.45:0/796028268 >> v1:172.21.15.84:6812/31478 conn(0x7f287403a950 legacy=0x7f287403ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.663+0000 7f289899d640 10 monclient: shutdown 2024-09-18T14:19:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.663+0000 7f289899d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.663+0000 7f289899d640 1 -- 172.21.15.45:0/796028268 >> v1:172.21.15.45:6793/0 conn(0x7f28940a1080 legacy=0x7f2894149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.663+0000 7f289899d640 1 -- 172.21.15.45:0/796028268 shutdown_connections 2024-09-18T14:19:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.664+0000 7f289899d640 1 -- 172.21.15.45:0/796028268 >> 172.21.15.45:0/796028268 conn(0x7f28940a5bb0 msgr2=0x7f28940a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.664+0000 7f289899d640 1 -- 172.21.15.45:0/796028268 shutdown_connections 2024-09-18T14:19:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.664+0000 7f289899d640 1 -- 172.21.15.45:0/796028268 wait complete. 2024-09-18T14:19:46.676 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":1517,"stamp":"2024-09-18T14:19:46.148778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1826976,"kb_used_data":13744,"kb_used_omap":9,"kb_used_meta":1813174,"kb_avail":564404064,"statfs":{"total":579820584960,"available":577949761536,"internally_reserved":0,"allocated":14073856,"data_stored":8160502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1856690192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002306"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":897,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:19:25.983026+0000","last_change":"2024-09-18T14:19:25.982976+0000","last_active":"2024-09-18T14:19:25.983026+0000","last_peered":"2024-09-18T14:19:25.983026+0000","last_clean":"2024-09-18T14:19:25.983026+0000","last_became_active":"2024-09-18T13:59:03.705607+0000","last_became_peered":"2024-09-18T13:59:03.705607+0000","last_unstale":"2024-09-18T14:19:25.983026+0000","last_undegraded":"2024-09-18T14:19:25.983026+0000","last_fullsized":"2024-09-18T14:19:25.983026+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T14:19:25.982880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T14:19:25.982880+0000","last_clean_scrub_stamp":"2024-09-18T14:19:25.982880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T23:11:43.389045+0000","scrub_duration":0.0014419280000000001,"objects_trimmed":0,"snaptrim_duration":0.000366475,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":886,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:19:29.898355+0000","last_change":"2024-09-18T14:19:29.898314+0000","last_active":"2024-09-18T14:19:29.898355+0000","last_peered":"2024-09-18T14:19:29.898355+0000","last_clean":"2024-09-18T14:19:29.898355+0000","last_became_active":"2024-09-18T13:59:03.705710+0000","last_became_peered":"2024-09-18T13:59:03.705710+0000","last_unstale":"2024-09-18T14:19:29.898355+0000","last_undegraded":"2024-09-18T14:19:29.898355+0000","last_fullsized":"2024-09-18T14:19:29.898355+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T14:19:29.898221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T14:19:29.898221+0000","last_clean_scrub_stamp":"2024-09-18T14:19:29.898221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T02:05:29.235490+0000","scrub_duration":0.0019873730000000002,"objects_trimmed":0,"snaptrim_duration":0.00029541399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"899'3","reported_seq":901,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:19:28.287194+0000","last_change":"2024-09-18T14:19:28.287151+0000","last_active":"2024-09-18T14:19:28.287194+0000","last_peered":"2024-09-18T14:19:28.287194+0000","last_clean":"2024-09-18T14:19:28.287194+0000","last_became_active":"2024-09-18T13:59:03.705659+0000","last_became_peered":"2024-09-18T13:59:03.705659+0000","last_unstale":"2024-09-18T14:19:28.287194+0000","last_undegraded":"2024-09-18T14:19:28.287194+0000","last_fullsized":"2024-09-18T14:19:28.287194+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"899'3","last_scrub_stamp":"2024-09-18T14:19:28.287066+0000","last_deep_scrub":"899'3","last_deep_scrub_stamp":"2024-09-18T14:19:28.287066+0000","last_clean_scrub_stamp":"2024-09-18T14:19:28.287066+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:19:06.988994+0000","scrub_duration":0.0034523980000000002,"objects_trimmed":0,"snaptrim_duration":0.00037781799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"899'2","reported_seq":891,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:19:30.131957+0000","last_change":"2024-09-18T14:19:30.131930+0000","last_active":"2024-09-18T14:19:30.131957+0000","last_peered":"2024-09-18T14:19:30.131957+0000","last_clean":"2024-09-18T14:19:30.131957+0000","last_became_active":"2024-09-18T13:59:02.977312+0000","last_became_peered":"2024-09-18T13:59:02.977312+0000","last_unstale":"2024-09-18T14:19:30.131957+0000","last_undegraded":"2024-09-18T14:19:30.131957+0000","last_fullsized":"2024-09-18T14:19:30.131957+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"899'2","last_scrub_stamp":"2024-09-18T14:19:30.131829+0000","last_deep_scrub":"899'2","last_deep_scrub_stamp":"2024-09-18T14:19:30.131829+0000","last_clean_scrub_stamp":"2024-09-18T14:19:30.131829+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:13:07.871844+0000","scrub_duration":0.0030756170000000001,"objects_trimmed":0,"snaptrim_duration":0.00072047099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"899'74","reported_seq":984,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:19:40.153030+0000","last_change":"2024-09-18T14:19:40.152984+0000","last_active":"2024-09-18T14:19:40.153030+0000","last_peered":"2024-09-18T14:19:40.153030+0000","last_clean":"2024-09-18T14:19:40.153030+0000","last_became_active":"2024-09-18T13:59:02.974164+0000","last_became_peered":"2024-09-18T13:59:02.974164+0000","last_unstale":"2024-09-18T14:19:40.153030+0000","last_undegraded":"2024-09-18T14:19:40.153030+0000","last_fullsized":"2024-09-18T14:19:40.153030+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"899'74","last_scrub_stamp":"2024-09-18T14:19:40.152861+0000","last_deep_scrub":"899'74","last_deep_scrub_stamp":"2024-09-18T14:19:40.152861+0000","last_clean_scrub_stamp":"2024-09-18T14:19:40.152861+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:59:07.686371+0000","scrub_duration":0.0064503240000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":889,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:19:29.151129+0000","last_change":"2024-09-18T14:19:29.151102+0000","last_active":"2024-09-18T14:19:29.151129+0000","last_peered":"2024-09-18T14:19:29.151129+0000","last_clean":"2024-09-18T14:19:29.151129+0000","last_became_active":"2024-09-18T13:59:02.977186+0000","last_became_peered":"2024-09-18T13:59:02.977186+0000","last_unstale":"2024-09-18T14:19:29.151129+0000","last_undegraded":"2024-09-18T14:19:29.151129+0000","last_fullsized":"2024-09-18T14:19:29.151129+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T14:19:29.151014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T14:19:29.151014+0000","last_clean_scrub_stamp":"2024-09-18T14:19:29.151014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T21:31:03.041073+0000","scrub_duration":0.001901395,"objects_trimmed":0,"snaptrim_duration":0.00065897000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":898,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:19:32.296670+0000","last_change":"2024-09-18T14:19:32.296641+0000","last_active":"2024-09-18T14:19:32.296670+0000","last_peered":"2024-09-18T14:19:32.296670+0000","last_clean":"2024-09-18T14:19:32.296670+0000","last_became_active":"2024-09-18T13:59:02.974240+0000","last_became_peered":"2024-09-18T13:59:02.974240+0000","last_unstale":"2024-09-18T14:19:32.296670+0000","last_undegraded":"2024-09-18T14:19:32.296670+0000","last_fullsized":"2024-09-18T14:19:32.296670+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T14:19:32.296549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T14:19:32.296549+0000","last_clean_scrub_stamp":"2024-09-18T14:19:32.296549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T23:47:29.395555+0000","scrub_duration":0.001851461,"objects_trimmed":0,"snaptrim_duration":0.00038595099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":897,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:19:36.118363+0000","last_change":"2024-09-18T14:19:36.118329+0000","last_active":"2024-09-18T14:19:36.118363+0000","last_peered":"2024-09-18T14:19:36.118363+0000","last_clean":"2024-09-18T14:19:36.118363+0000","last_became_active":"2024-09-18T13:59:03.705691+0000","last_became_peered":"2024-09-18T13:59:03.705691+0000","last_unstale":"2024-09-18T14:19:36.118363+0000","last_undegraded":"2024-09-18T14:19:36.118363+0000","last_fullsized":"2024-09-18T14:19:36.118363+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T14:19:36.118247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T14:19:36.118247+0000","last_clean_scrub_stamp":"2024-09-18T14:19:36.118247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:02:20.221646+0000","scrub_duration":0.0021267790000000001,"objects_trimmed":0,"snaptrim_duration":0.00033710499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":886,"reported_epoch":899,"state":"active+clean","last_fresh":"2024-09-18T14:19:35.891041+0000","last_change":"2024-09-18T14:19:35.891015+0000","last_active":"2024-09-18T14:19:35.891041+0000","last_peered":"2024-09-18T14:19:35.891041+0000","last_clean":"2024-09-18T14:19:35.891041+0000","last_became_active":"2024-09-18T13:59:02.976936+0000","last_became_peered":"2024-09-18T13:59:02.976936+0000","last_unstale":"2024-09-18T14:19:35.891041+0000","last_undegraded":"2024-09-18T14:19:35.891041+0000","last_fullsized":"2024-09-18T14:19:35.891041+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T14:19:35.890926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T14:19:35.890926+0000","last_clean_scrub_stamp":"2024-09-18T14:19:35.890926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T22:00:11.810704+0000","scrub_duration":0.0019393889999999999,"objects_trimmed":0,"snaptrim_duration":0.00036210499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607803,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312468,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059372,"statfs":{"total":96636764160,"available":96316796928,"internally_reserved":0,"allocated":2699264,"data_stored":1712696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.35099999999999998,"15min":0.34999999999999998},"min":{"1min":0.28100000000000003,"5min":0.23300000000000001,"15min":0.19},"max":{"1min":0.441,"5min":0.54000000000000004,"15min":0.91700000000000004},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.35399999999999998,"15min":0.36499999999999999},"min":{"1min":0.29899999999999999,"5min":0.23999999999999999,"15min":0.219},"max":{"1min":0.46700000000000003,"5min":0.55700000000000005,"15min":1.0740000000000001},"last":0.32500000000000001}]},{"osd":2,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.48699999999999999,"5min":0.53200000000000003,"15min":1.133},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.71899999999999997},"last":0.34300000000000003}]},{"osd":3,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52900000000000003,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.747,"15min":0.871},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.64200000000000002,"5min":0.74299999999999999,"15min":1.034},"last":0.52600000000000002}]},{"osd":4,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.50800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73799999999999999,"5min":0.76400000000000001,"15min":0.995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.58299999999999996},"min":{"1min":0.48899999999999999,"5min":0.35499999999999998,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.80000000000000004,"15min":1.323},"last":0.56200000000000006}]},{"osd":5,"last update":"Wed Sep 18 14:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.55800000000000005},"min":{"1min":0.44,"5min":0.378,"15min":0.33200000000000002},"max":{"1min":0.67500000000000004,"5min":0.77700000000000002,"15min":0.87},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.54300000000000004},"min":{"1min":0.41899999999999998,"5min":0.315,"15min":0.28000000000000003},"max":{"1min":0.75700000000000001,"5min":0.77200000000000002,"15min":1.2669999999999999},"last":0.57699999999999996}]}]},{"osd":5,"up_from":11,"seq":47244640507,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303628,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068212,"statfs":{"total":96636764160,"available":96325849088,"internally_reserved":0,"allocated":2166784,"data_stored":1183768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.53400000000000003},"min":{"1min":0.318,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":2.6800000000000002},"last":0.438},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52300000000000002,"15min":0.54600000000000004},"min":{"1min":0.39900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":3.206},"last":0.42399999999999999}]},{"osd":1,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47899999999999998,"15min":0.48799999999999999},"min":{"1min":0.36299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64800000000000002,"5min":1.335,"15min":1.335},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65300000000000002,"5min":0.69399999999999995,"15min":2.3610000000000002},"last":0.378}]},{"osd":2,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48999999999999999,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65700000000000003,"5min":1.25,"15min":2.976},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":2.8719999999999999},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.35399999999999998},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.375},"min":{"1min":0.27300000000000002,"5min":0.224,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.629,"15min":0.69099999999999995},"last":0.34200000000000003}]},{"osd":4,"last update":"Wed Sep 18 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.29399999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.68999999999999995,"5min":1.194,"15min":1.194},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.219},"max":{"1min":0.56200000000000006,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.26100000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640507,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307792,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064048,"statfs":{"total":96636764160,"available":96321585152,"internally_reserved":0,"allocated":2170880,"data_stored":1183787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.52400000000000002},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.183},"max":{"1min":0.66000000000000003,"5min":0.745,"15min":0.81799999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.26300000000000001},"max":{"1min":0.67200000000000004,"5min":0.80300000000000005,"15min":0.83299999999999996},"last":0.57699999999999996}]},{"osd":1,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46700000000000003,"15min":0.47499999999999998},"min":{"1min":0.33400000000000002,"5min":0.314,"15min":0.26500000000000001},"max":{"1min":0.59299999999999997,"5min":0.77300000000000002,"15min":0.79200000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.47799999999999998,"15min":0.48499999999999999},"min":{"1min":0.377,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.71399999999999997},"last":0.52000000000000002}]},{"osd":2,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.252,"15min":0.246},"max":{"1min":0.59899999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.32800000000000001,"5min":0.30399999999999999,"15min":0.28799999999999998},"max":{"1min":0.57899999999999996,"5min":0.68100000000000005,"15min":0.84599999999999997},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.28100000000000003,"5min":0.22500000000000001,"15min":0.20499999999999999},"max":{"1min":0.58399999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.23699999999999999,"5min":0.23599999999999999,"15min":0.22},"max":{"1min":0.58799999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.38500000000000001}]},{"osd":5,"last update":"Wed Sep 18 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26000000000000001,"5min":0.20899999999999999,"15min":0.20499999999999999},"max":{"1min":0.50600000000000001,"5min":0.64600000000000002,"15min":0.753},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.40000000000000002}]}]},{"osd":3,"up_from":11,"seq":47244640507,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299988,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071852,"statfs":{"total":96636764160,"available":96329576448,"internally_reserved":0,"allocated":2699264,"data_stored":1712696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.31,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.624,"5min":0.66900000000000004,"15min":1.9530000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48999999999999999,"15min":0.47899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.26200000000000001},"max":{"1min":0.68000000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47899999999999998,"15min":0.47099999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.28100000000000003,"15min":0.27500000000000002},"max":{"1min":0.65500000000000003,"5min":0.66200000000000003,"15min":2.5190000000000001},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52100000000000002,"15min":0.52900000000000003},"min":{"1min":0.376,"5min":0.317,"15min":0.317},"max":{"1min":0.63,"5min":0.72999999999999998,"15min":2.3660000000000001},"last":0.623},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.30099999999999999},"max":{"1min":0.61599999999999999,"5min":0.71899999999999997,"15min":2.3919999999999999},"last":0.52400000000000002}]},{"osd":4,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35799999999999998},"min":{"1min":0.224,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.68899999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.372},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.20399999999999999},"max":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.63800000000000001},"last":0.36799999999999999}]},{"osd":5,"last update":"Wed Sep 18 14:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.379,"15min":0.379},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.218},"max":{"1min":0.54500000000000004,"5min":0.58399999999999996,"15min":0.70499999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.41599999999999998,"15min":0.40799999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.748},"last":0.44}]}]},{"osd":2,"up_from":11,"seq":47244640507,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299468,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072372,"statfs":{"total":96636764160,"available":96330108928,"internally_reserved":0,"allocated":2166784,"data_stored":1183768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 14:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32000000000000001,"15min":0.33100000000000002},"min":{"1min":0.24099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.45400000000000001,"5min":0.52900000000000003,"15min":0.875},"last":0.375},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34100000000000003,"15min":0.36099999999999999},"min":{"1min":0.20899999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.51400000000000001,"5min":0.60299999999999998,"15min":1.357},"last":0.29899999999999999}]},{"osd":1,"last update":"Wed Sep 18 14:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40000000000000002,"15min":0.41799999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.25},"max":{"1min":0.55200000000000005,"5min":0.67400000000000004,"15min":4.117},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.29099999999999998,"5min":0.23200000000000001,"15min":0.217},"max":{"1min":0.50700000000000001,"5min":0.629,"15min":4.0529999999999999},"last":0.32300000000000001}]},{"osd":3,"last update":"Wed Sep 18 14:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.307},"max":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":1.3049999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.51500000000000001,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.29599999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":1.506},"last":0.63}]},{"osd":4,"last update":"Wed Sep 18 14:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.30499999999999999},"max":{"1min":0.79700000000000004,"5min":0.94399999999999995,"15min":1.2789999999999999},"last":0.621},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.31},"max":{"1min":0.81100000000000005,"5min":0.96099999999999997,"15min":2.3159999999999998},"last":0.66600000000000004}]},{"osd":5,"last update":"Wed Sep 18 14:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75600000000000001,"5min":0.92700000000000005,"15min":1.224},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54000000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.35999999999999999,"15min":0.34599999999999997},"max":{"1min":0.76900000000000002,"5min":1.0409999999999999,"15min":1.2629999999999999},"last":0.53600000000000003}]}]},{"osd":0,"up_from":11,"seq":47244640507,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303632,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068208,"statfs":{"total":96636764160,"available":96325844992,"internally_reserved":0,"allocated":2170880,"data_stored":1183787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 14:19:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.34599999999999997},"min":{"1min":0.22500000000000001,"5min":0.187,"15min":0.17199999999999999},"max":{"1min":0.50600000000000001,"5min":0.53100000000000003,"15min":0.628},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.40400000000000003,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.495,"5min":0.57699999999999996,"15min":0.72299999999999998},"last":0.435}]},{"osd":2,"last update":"Wed Sep 18 14:19:34 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.34200000000000003,"15min":0.35599999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.19900000000000001},"max":{"1min":0.438,"5min":0.53500000000000003,"15min":0.629},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.36899999999999999,"15min":0.377},"min":{"1min":0.248,"5min":0.248,"15min":0.224},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.71499999999999997},"last":0.47299999999999998}]},{"osd":3,"last update":"Wed Sep 18 14:19:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.29599999999999999,"15min":0.28599999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.88200000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.35799999999999998,"5min":0.33200000000000002,"15min":0.29899999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.88900000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed Sep 18 14:19:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.42899999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60199999999999998,"15min":0.58699999999999997},"min":{"1min":0.38400000000000001,"5min":0.36299999999999999,"15min":0.28000000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.91700000000000004},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed Sep 18 14:19:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.375,"5min":0.29799999999999999,"15min":0.28699999999999998},"max":{"1min":0.73899999999999999,"5min":0.75600000000000001,"15min":0.84699999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.56200000000000006},"min":{"1min":0.42099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":1.1479999999999999},"last":0.60999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T14:19:46.678 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-18T14:19:46.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.771+0000 7f7ab6899640 10 monclient: get_monmap_and_config 2024-09-18T14:19:46.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.771+0000 7f7ab6899640 10 monclient: build_initial_monmap 2024-09-18T14:19:46.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.771+0000 7f7ab6899640 10 monclient: monmap: 2024-09-18T14:19:46.774 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:46.774 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.774 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T14:19:46.773763+0000 2024-09-18T14:19:46.774 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T14:19:46.773763+0000 2024-09-18T14:19:46.774 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:46.774 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.774 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:46.774 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:46.774 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:46.775 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:46.775 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:46.775 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:46.775 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:46.775 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:46.775 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:46.775 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.772+0000 7f7ab6899640 1 Processor -- start 2024-09-18T14:19:46.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.772+0000 7f7ab6899640 1 -- start start 2024-09-18T14:19:46.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.772+0000 7f7ab6899640 10 monclient: init 2024-09-18T14:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 10 monclient: _add_conns ranks=[0,7,3,4,1,5,6,2,8] 2024-09-18T14:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 10 monclient(hunting): picked mon.noname-a con 0x7f7ab00a2630 addr v1:172.21.15.45:6789/0 2024-09-18T14:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 10 monclient(hunting): picked mon.noname-i con 0x7f7ab009ee90 addr v1:172.21.15.84:6792/0 2024-09-18T14:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 10 monclient(hunting): picked mon.noname-g con 0x7f7ab009f2a0 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab005ed70 con 0x7f7ab00a2630 2024-09-18T14:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00970d0 con 0x7f7ab009f2a0 2024-09-18T14:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 1 -- --> v1:172.21.15.84:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00a0ef0 con 0x7f7ab009ee90 2024-09-18T14:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 10 monclient(hunting): authenticate will time out at 2002.112100s 2024-09-18T14:19:46.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7ab00a2630 0x7f7ab00a07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48006/0 (socket says 172.21.15.45:48006) 2024-09-18T14:19:46.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaffff640 1 -- 172.21.15.45:0/1049200083 learned_addr learned my addr 172.21.15.45:0/1049200083 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:46.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516458276 0 0) 0x7f7ab005ed70 con 0x7f7ab00a2630 2024-09-18T14:19:46.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:46.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:46.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient(hunting): my global_id is 6795 2024-09-18T14:19:46.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a980034e0 con 0x7f7ab00a2630 2024-09-18T14:19:46.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 <== mon.7 v1:172.21.15.84:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743159591 0 0) 0x7f7ab00a0ef0 con 0x7f7ab009ee90 2024-09-18T14:19:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient(hunting): my global_id is 214810 2024-09-18T14:19:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 --> v1:172.21.15.84:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab005ed70 con 0x7f7ab009ee90 2024-09-18T14:19:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300200286 0 0) 0x7f7ab00970d0 con 0x7f7ab009f2a0 2024-09-18T14:19:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient(hunting): my global_id is 224580 2024-09-18T14:19:46.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab00a0ef0 con 0x7f7ab009f2a0 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3716178225 0 0) 0x7f7a980034e0 con 0x7f7ab00a2630 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 >> v1:172.21.15.84:6792/0 conn(0x7f7ab009ee90 legacy=0x7f7ab0137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 >> v1:172.21.15.84:6790/0 conn(0x7f7ab009f2a0 legacy=0x7f7ab013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient: _finish_hunting 0 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 monclient: found mon.noname-a 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.45:6789/0 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab00a2a00 con 0x7f7ab00a2630 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient: _finish_auth 0 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7aa0001830 con 0x7f7ab00a2630 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 5 monclient: authenticate success, global_id 6795 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:46.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-18T14:19:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient: dump: 2024-09-18T14:19:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:46.791 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-18T14:19:46.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7aa0001990 con 0x7f7ab00a2630 2024-09-18T14:19:46.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:46.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 1 -- 172.21.15.45:0/1049200083 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7aa0004ba0 con 0x7f7ab00a2630 2024-09-18T14:19:46.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:46.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-18T14:19:46.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7aaeffd640 10 monclient: dump: 2024-09-18T14:19:46.791 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:46.792 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:46.793 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:46.793 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 10 monclient: get_monmap_and_config success 2024-09-18T14:19:46.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.773+0000 7f7ab6899640 4 set_mon_vals no callback set 2024-09-18T14:19:46.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.774+0000 7f7ab6899640 10 monclient: shutdown 2024-09-18T14:19:46.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.774+0000 7f7ab6899640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:46.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.774+0000 7f7ab6899640 1 -- 172.21.15.45:0/1049200083 >> v1:172.21.15.45:6789/0 conn(0x7f7ab00a2630 legacy=0x7f7ab00a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.774+0000 7f7ab6899640 1 -- 172.21.15.45:0/1049200083 shutdown_connections 2024-09-18T14:19:46.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.774+0000 7f7ab6899640 1 -- 172.21.15.45:0/1049200083 >> 172.21.15.45:0/1049200083 conn(0x7f7ab0062fd0 msgr2=0x7f7ab00633a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:46.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.774+0000 7f7ab6899640 1 -- 172.21.15.45:0/1049200083 shutdown_connections 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.774+0000 7f7ab6899640 1 -- 172.21.15.45:0/1049200083 wait complete. 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.774+0000 7f7ab6899640 10 monclient: build_initial_monmap 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.774+0000 7f7ab6899640 10 monclient: monmap: 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 0 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 0.000000 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:created 0.000000 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 0 (unknown) 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 1 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.noname-a 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.noname-f 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.noname-b 2024-09-18T14:19:46.794 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.noname-g 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.noname-c 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.noname-h 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.noname-d 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.noname-i 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.noname-e 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 1 Processor -- start 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 1 -- start start 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 10 monclient: init 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 10 monclient: _reopen_session rank -1 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 10 monclient: _add_conns ranks=[1,6,3,0,7,4,2,5,8] 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 10 monclient(hunting): picked mon.noname-f con 0x7f7ab009ee90 addr v1:172.21.15.84:6789/0 2024-09-18T14:19:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 10 monclient(hunting): picked mon.noname-d con 0x7f7ab009f2a0 addr v1:172.21.15.45:6792/0 2024-09-18T14:19:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 10 monclient(hunting): picked mon.noname-g con 0x7f7ab00a2630 addr v1:172.21.15.84:6790/0 2024-09-18T14:19:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00a2230 con 0x7f7ab009ee90 2024-09-18T14:19:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 1 -- --> v1:172.21.15.84:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab0142470 con 0x7f7ab00a2630 2024-09-18T14:19:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 1 -- --> v1:172.21.15.45:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab0120a40 con 0x7f7ab009f2a0 2024-09-18T14:19:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 10 monclient(hunting): _renew_subs 2024-09-18T14:19:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7ab6899640 10 monclient(hunting): authenticate will time out at 2002.114821s 2024-09-18T14:19:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7aaf7fe640 1 --1- >> v1:172.21.15.45:6792/0 conn(0x7f7ab009f2a0 0x7f7ab01468b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6792/0 says I am v1:172.21.15.45:40530/0 (socket says 172.21.15.45:40530) 2024-09-18T14:19:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.775+0000 7f7aaf7fe640 1 -- 172.21.15.45:0/967682031 learned_addr learned my addr 172.21.15.45:0/967682031 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-18T14:19:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.6 v1:172.21.15.45:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641342411 0 0) 0x7f7ab0120a40 con 0x7f7ab009f2a0 2024-09-18T14:19:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): my global_id is 185433 2024-09-18T14:19:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 --> v1:172.21.15.45:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a9c0034e0 con 0x7f7ab009f2a0 2024-09-18T14:19:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.3 v1:172.21.15.84:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4261470786 0 0) 0x7f7ab0142470 con 0x7f7ab00a2630 2024-09-18T14:19:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): my global_id is 224589 2024-09-18T14:19:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 --> v1:172.21.15.84:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab0120a40 con 0x7f7ab00a2630 2024-09-18T14:19:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882104948 0 0) 0x7f7ab00a2230 con 0x7f7ab009ee90 2024-09-18T14:19:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): _init_auth method 2 2024-09-18T14:19:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): my global_id is 214849 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab0142470 con 0x7f7ab009ee90 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.6 v1:172.21.15.45:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2663251344 0 0) 0x7f7a9c0034e0 con 0x7f7ab009f2a0 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 --> v1:172.21.15.45:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ab00a2230 con 0x7f7ab009f2a0 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.6 v1:172.21.15.45:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7aa4001cb0 con 0x7f7ab009f2a0 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): dump: 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): _finish_auth 0 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.3 v1:172.21.15.84:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 619975646 0 0) 0x7f7ab0120a40 con 0x7f7ab00a2630 2024-09-18T14:19:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 --> v1:172.21.15.84:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a9c0034e0 con 0x7f7ab00a2630 2024-09-18T14:19:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.3 v1:172.21.15.84:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7aa8001d10 con 0x7f7ab00a2630 2024-09-18T14:19:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-18T14:19:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): dump: 2024-09-18T14:19:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:46.802 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3800369402 0 0) 0x7f7ab0142470 con 0x7f7ab009ee90 2024-09-18T14:19:46.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 --> v1:172.21.15.84:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ab0120a40 con 0x7f7ab009ee90 2024-09-18T14:19:46.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7aa0001770 con 0x7f7ab009ee90 2024-09-18T14:19:46.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:46.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-18T14:19:46.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient(hunting): dump: 2024-09-18T14:19:46.804 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:46.804 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.804 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.804 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.804 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:46.804 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.805 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:46.805 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:46.805 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:46.805 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:46.805 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:46.805 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:46.805 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:46.806 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:46.806 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:46.806 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.6 v1:172.21.15.45:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 756684346 0 0) 0x7f7ab00a2230 con 0x7f7ab009f2a0 2024-09-18T14:19:46.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 >> v1:172.21.15.84:6790/0 conn(0x7f7ab00a2630 legacy=0x7f7ab0149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 >> v1:172.21.15.84:6789/0 conn(0x7f7ab009ee90 legacy=0x7f7ab01353e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient: _finish_hunting 0 2024-09-18T14:19:46.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 monclient: found mon.d 2024-09-18T14:19:46.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-18T14:19:46.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:46.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 --> v1:172.21.15.45:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab0122500 con 0x7f7ab009f2a0 2024-09-18T14:19:46.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient: _finish_auth 0 2024-09-18T14:19:46.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-18T14:19:46.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7ab6899640 5 monclient: authenticate success, global_id 185433 2024-09-18T14:19:46.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7ab6899640 10 monclient: _renew_subs 2024-09-18T14:19:46.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7ab6899640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:46.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7ab6899640 1 -- 172.21.15.45:0/967682031 --> v1:172.21.15.45:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab01227e0 con 0x7f7ab009f2a0 2024-09-18T14:19:46.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7ab6899640 10 monclient: _renew_subs 2024-09-18T14:19:46.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7ab6899640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:46.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7ab6899640 1 -- 172.21.15.45:0/967682031 --> v1:172.21.15.45:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab0135af0 con 0x7f7ab009f2a0 2024-09-18T14:19:46.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.6 v1:172.21.15.45:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7aa40021c0 con 0x7f7ab009f2a0 2024-09-18T14:19:46.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient: handle_config config(0 keys) v1 2024-09-18T14:19:46.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.6 v1:172.21.15.45:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 2860779331 0 0) 0x7f7aa4002670 con 0x7f7ab009f2a0 2024-09-18T14:19:46.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-18T14:19:46.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-18T14:19:46.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7a97fff640 10 monclient: dump: 2024-09-18T14:19:46.810 INFO:teuthology.orchestra.run.smithi045.stderr:epoch 1 2024-09-18T14:19:46.810 INFO:teuthology.orchestra.run.smithi045.stderr:fsid 0f5ca4bb-86c8-4c54-b818-324eb902ab17 2024-09-18T14:19:46.810 INFO:teuthology.orchestra.run.smithi045.stderr:last_changed 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.810 INFO:teuthology.orchestra.run.smithi045.stderr:created 2024-09-18T13:58:22.742395+0000 2024-09-18T14:19:46.810 INFO:teuthology.orchestra.run.smithi045.stderr:min_mon_release 17 (quincy) 2024-09-18T14:19:46.810 INFO:teuthology.orchestra.run.smithi045.stderr:election_strategy: 3 2024-09-18T14:19:46.811 INFO:teuthology.orchestra.run.smithi045.stderr:0: v1:172.21.15.45:6789/0 mon.a 2024-09-18T14:19:46.811 INFO:teuthology.orchestra.run.smithi045.stderr:1: v1:172.21.15.84:6789/0 mon.f 2024-09-18T14:19:46.811 INFO:teuthology.orchestra.run.smithi045.stderr:2: v1:172.21.15.45:6790/0 mon.b 2024-09-18T14:19:46.811 INFO:teuthology.orchestra.run.smithi045.stderr:3: v1:172.21.15.84:6790/0 mon.g 2024-09-18T14:19:46.811 INFO:teuthology.orchestra.run.smithi045.stderr:4: v1:172.21.15.45:6791/0 mon.c 2024-09-18T14:19:46.811 INFO:teuthology.orchestra.run.smithi045.stderr:5: v1:172.21.15.84:6791/0 mon.h 2024-09-18T14:19:46.811 INFO:teuthology.orchestra.run.smithi045.stderr:6: v1:172.21.15.45:6792/0 mon.d 2024-09-18T14:19:46.811 INFO:teuthology.orchestra.run.smithi045.stderr:7: v1:172.21.15.84:6792/0 mon.i 2024-09-18T14:19:46.812 INFO:teuthology.orchestra.run.smithi045.stderr:8: v1:172.21.15.45:6793/0 mon.e 2024-09-18T14:19:46.812 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:46.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.776+0000 7f7aae7fc640 4 set_mon_vals no callback set 2024-09-18T14:19:46.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.777+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.6 v1:172.21.15.45:6792/0 7 ==== mgrmap(e 5) v1 ==== 44646+0+0 (unknown 4279578558 0 0) 0x7f7aa4002850 con 0x7f7ab009f2a0 2024-09-18T14:19:46.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.777+0000 7f7ab6899640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:46.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.777+0000 7f7ab6899640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:46.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.777+0000 7f7ab6899640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:46.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.777+0000 7f7ab6899640 1 -- 172.21.15.45:0/967682031 --> v1:172.21.15.45:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a90005010 con 0x7f7ab009f2a0 2024-09-18T14:19:46.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.778+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.6 v1:172.21.15.45:6792/0 8 ==== osd_map(899..899 src has 637..899) v4 ==== 3502+0+0 (unknown 4111324233 0 0) 0x7f7aa4046a90 con 0x7f7ab009f2a0 2024-09-18T14:19:46.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.778+0000 7f7aaffff640 10 monclient: get_auth_request con 0x7f7a9c03aa40 auth_method 0 2024-09-18T14:19:46.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.782+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.6 v1:172.21.15.45:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176153 (unknown 1092875540 0 2027069090) 0x7f7aa40153a0 con 0x7f7ab009f2a0 2024-09-18T14:19:46.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.782+0000 7f7a97fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-18T14:19:46.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.782+0000 7f7a97fff640 10 monclient: _finish_command 1 = system:0 2024-09-18T14:19:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.945+0000 7f7ab6899640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-18T14:19:46.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.945+0000 7f7ab6899640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-18T14:19:46.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.945+0000 7f7ab6899640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.45:6792/0 2024-09-18T14:19:46.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.945+0000 7f7ab6899640 1 -- 172.21.15.45:0/967682031 --> v1:172.21.15.45:6792/0 -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f7a90005840 con 0x7f7ab009f2a0 2024-09-18T14:19:46.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.952+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.6 v1:172.21.15.45:6792/0 10 ==== config(1 keys) v1 ==== 35+0+0 (unknown 2555425225 0 0) 0x7f7aa40153a0 con 0x7f7ab009f2a0 2024-09-18T14:19:46.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.952+0000 7f7a97fff640 10 monclient: handle_config config(1 keys) v1 2024-09-18T14:19:46.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.952+0000 7f7aae7fc640 4 set_mon_vals no callback set 2024-09-18T14:19:46.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.952+0000 7f7aae7fc640 10 set_mon_vals mon_health_to_clog = false 2024-09-18T14:19:46.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.953+0000 7f7a97fff640 1 -- 172.21.15.45:0/967682031 <== mon.6 v1:172.21.15.45:6792/0 11 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) v1 ==== 123+0+0 (unknown 691139944 0 0) 0x7f7aa4014da0 con 0x7f7ab009f2a0 2024-09-18T14:19:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.953+0000 7f7a97fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-18T14:19:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.953+0000 7f7a97fff640 10 monclient: _finish_command 2 = system:0 2024-09-18T14:19:46.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.955+0000 7f7ab6899640 1 -- 172.21.15.45:0/967682031 >> v1:172.21.15.84:6812/31478 conn(0x7f7a9c03aa40 legacy=0x7f7a9c03cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.955+0000 7f7ab6899640 10 monclient: shutdown 2024-09-18T14:19:46.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.955+0000 7f7ab6899640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-18T14:19:46.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.955+0000 7f7ab6899640 1 -- 172.21.15.45:0/967682031 >> v1:172.21.15.45:6792/0 conn(0x7f7ab009f2a0 legacy=0x7f7ab01468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T14:19:46.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.955+0000 7f7ab6899640 1 -- 172.21.15.45:0/967682031 shutdown_connections 2024-09-18T14:19:46.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.955+0000 7f7ab6899640 1 -- 172.21.15.45:0/967682031 >> 172.21.15.45:0/967682031 conn(0x7f7ab0062fd0 msgr2=0x7f7ab00a4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T14:19:46.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.955+0000 7f7ab6899640 1 -- 172.21.15.45:0/967682031 shutdown_connections 2024-09-18T14:19:46.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-18T14:19:46.955+0000 7f7ab6899640 1 -- 172.21.15.45:0/967682031 wait complete. 2024-09-18T14:19:46.967 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-18T14:19:46.967 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-18T14:19:46.967 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-18T14:19:46.967 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.011 INFO:tasks.ceph.osd.0:Stopped 2024-09-18T14:19:47.011 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-18T14:19:47.011 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.047 INFO:tasks.ceph.osd.1:Stopped 2024-09-18T14:19:47.048 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-18T14:19:47.048 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.122 INFO:tasks.ceph.osd.2:Stopped 2024-09-18T14:19:47.123 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-18T14:19:47.123 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.161 INFO:tasks.ceph.osd.3:Stopped 2024-09-18T14:19:47.161 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-18T14:19:47.161 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.241 INFO:tasks.ceph.osd.4:Stopped 2024-09-18T14:19:47.241 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-18T14:19:47.241 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.320 INFO:tasks.ceph.osd.5:Stopped 2024-09-18T14:19:47.320 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-18T14:19:47.320 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-18T14:19:47.320 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.389 INFO:tasks.ceph.mgr.x:Stopped 2024-09-18T14:19:47.389 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-18T14:19:47.389 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-18T14:19:47.389 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.407 INFO:tasks.ceph.mon.a:Stopped 2024-09-18T14:19:47.407 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-18T14:19:47.407 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.459 INFO:tasks.ceph.mon.b:Stopped 2024-09-18T14:19:47.460 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T14:19:47.460 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.514 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T14:19:47.515 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-09-18T14:19:47.515 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.569 INFO:tasks.ceph.mon.d:Stopped 2024-09-18T14:19:47.569 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-18T14:19:47.569 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.620 INFO:tasks.ceph.mon.e:Stopped 2024-09-18T14:19:47.620 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-18T14:19:47.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.634 INFO:tasks.ceph.mon.f:Stopped 2024-09-18T14:19:47.635 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-18T14:19:47.635 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.688 INFO:tasks.ceph.mon.g:Stopped 2024-09-18T14:19:47.688 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-09-18T14:19:47.688 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.744 INFO:tasks.ceph.mon.h:Stopped 2024-09-18T14:19:47.745 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-18T14:19:47.745 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T14:19:47.798 INFO:tasks.ceph.mon.i:Stopped 2024-09-18T14:19:47.798 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-18T14:19:47.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v ScrubResult | egrep -v 'scrub mismatch' | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-18T14:19:47.844 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-18T14:19:47.844 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-18T14:19:48.046 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-18T14:19:48.046 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-18T14:19:48.120 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-18T14:19:48.121 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-18T14:19:48.187 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-18T14:19:48.187 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-18T14:19:48.336 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-18T14:19:48.336 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-18T14:19:48.440 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-18T14:19:48.440 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-18T14:19:48.499 INFO:tasks.ceph:Archiving mon data... 2024-09-18T14:19:48.500 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/data/mon.a.tgz 2024-09-18T14:19:48.501 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-18T14:19:48.517 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.7j6aWPddTj 2024-09-18T14:19:48.517 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.7j6aWPddTj 2024-09-18T14:19:48.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.7j6aWPddTj 2024-09-18T14:19:48.750 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.7j6aWPddTj is 479KB 2024-09-18T14:19:48.827 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.7j6aWPddTj 2024-09-18T14:19:48.843 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/data/mon.b.tgz 2024-09-18T14:19:48.844 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-18T14:19:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.84u5LurkBb 2024-09-18T14:19:48.899 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.84u5LurkBb 2024-09-18T14:19:49.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.84u5LurkBb 2024-09-18T14:19:49.111 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.84u5LurkBb is 459KB 2024-09-18T14:19:49.208 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.84u5LurkBb 2024-09-18T14:19:49.225 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/data/mon.c.tgz 2024-09-18T14:19:49.225 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-18T14:19:49.280 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.circoi3A86 2024-09-18T14:19:49.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.circoi3A86 2024-09-18T14:19:49.410 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.circoi3A86 2024-09-18T14:19:49.495 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.circoi3A86 is 472KB 2024-09-18T14:19:49.591 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.circoi3A86 2024-09-18T14:19:49.607 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-d to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/data/mon.d.tgz 2024-09-18T14:19:49.607 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-18T14:19:49.661 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.cqkcXQXLvN 2024-09-18T14:19:49.662 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-d -- . > /tmp/tmp.cqkcXQXLvN 2024-09-18T14:19:49.791 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.cqkcXQXLvN 2024-09-18T14:19:49.875 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.cqkcXQXLvN is 470KB 2024-09-18T14:19:49.973 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.cqkcXQXLvN 2024-09-18T14:19:49.989 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-e to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/data/mon.e.tgz 2024-09-18T14:19:49.989 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-18T14:19:50.044 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.t0uG8UhiAs 2024-09-18T14:19:50.045 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-e -- . > /tmp/tmp.t0uG8UhiAs 2024-09-18T14:19:50.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.t0uG8UhiAs 2024-09-18T14:19:50.258 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.t0uG8UhiAs is 459KB 2024-09-18T14:19:50.361 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.t0uG8UhiAs 2024-09-18T14:19:50.377 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/mon/ceph-f to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/data/mon.f.tgz 2024-09-18T14:19:50.377 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2024-09-18T14:19:50.396 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.tiQLuoz25C 2024-09-18T14:19:50.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-f -- . > /tmp/tmp.tiQLuoz25C 2024-09-18T14:19:50.600 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.tiQLuoz25C 2024-09-18T14:19:50.686 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.tiQLuoz25C is 466KB 2024-09-18T14:19:50.783 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.tiQLuoz25C 2024-09-18T14:19:50.799 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/mon/ceph-g to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/data/mon.g.tgz 2024-09-18T14:19:50.799 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2024-09-18T14:19:50.854 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.ZO408Nsg0T 2024-09-18T14:19:50.854 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-g -- . > /tmp/tmp.ZO408Nsg0T 2024-09-18T14:19:50.981 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.ZO408Nsg0T 2024-09-18T14:19:51.065 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.ZO408Nsg0T is 462KB 2024-09-18T14:19:51.161 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.ZO408Nsg0T 2024-09-18T14:19:51.181 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/mon/ceph-h to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/data/mon.h.tgz 2024-09-18T14:19:51.181 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2024-09-18T14:19:51.235 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.i2iiWtOsmx 2024-09-18T14:19:51.235 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-h -- . > /tmp/tmp.i2iiWtOsmx 2024-09-18T14:19:51.360 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.i2iiWtOsmx 2024-09-18T14:19:51.443 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.i2iiWtOsmx is 470KB 2024-09-18T14:19:51.541 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.i2iiWtOsmx 2024-09-18T14:19:51.556 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/mon/ceph-i to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/data/mon.i.tgz 2024-09-18T14:19:51.556 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2024-09-18T14:19:51.610 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.fCA8CupWEH 2024-09-18T14:19:51.610 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-i -- . > /tmp/tmp.fCA8CupWEH 2024-09-18T14:19:51.736 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.fCA8CupWEH 2024-09-18T14:19:51.818 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.fCA8CupWEH is 464KB 2024-09-18T14:19:51.915 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.fCA8CupWEH 2024-09-18T14:19:51.930 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-18T14:19:51.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T14:19:51.933 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T14:19:51.995 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-18T14:19:51.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-18T14:19:52.004 DEBUG:teuthology.orchestra.run.smithi084:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-18T14:19:52.035 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-18T14:19:52.061 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-18T14:19:52.065 INFO:tasks.ceph:Archiving crash dumps... 2024-09-18T14:19:52.067 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/remote/smithi045/crash 2024-09-18T14:19:52.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T14:19:52.110 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/remote/smithi084/crash 2024-09-18T14:19:52.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T14:19:52.134 INFO:tasks.ceph:Compressing logs... 2024-09-18T14:19:52.135 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T14:19:52.154 DEBUG:teuthology.orchestra.run.smithi084:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T14:19:52.179 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-18T14:19:52.179 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38267.log 2024-09-18T14:19:52.180 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42779.log 2024-09-18T14:19:52.180 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49331.log 2024-09-18T14:19:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45426.log 2024-09-18T14:19:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40469.log 2024-09-18T14:19:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42779.log: /var/log/ceph/ceph-client.admin.49331.log: /var/log/ceph/ceph-client.admin.45426.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38267.log.gz 2024-09-18T14:19:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36266.log 2024-09-18T14:19:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40469.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49331.log.gz 2024-09-18T14:19:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43745.log 2024-09-18T14:19:52.187 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42779.log.gz 2024-09-18T14:19:52.187 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45426.log.gz 2024-09-18T14:19:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36266.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40469.log.gz 2024-09-18T14:19:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43451.log 2024-09-18T14:19:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43745.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36266.log.gz 2024-09-18T14:19:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38099.log 2024-09-18T14:19:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49373.log 2024-09-18T14:19:52.187 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43745.log.gz 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38099.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43451.log.gz 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41057.log 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36415.log 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38099.log.gz 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41057.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49373.log.gz 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36669.log 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41057.log.gz 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5/var/log/ceph/ceph-client.admin.36415.log: --verbose -- /var/log/ceph/ceph-client.admin.35112.log 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49499.log 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36415.log.gz 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47027.log 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36669.log.gz 2024-09-18T14:19:52.188 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49499.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35112.log.gz 2024-09-18T14:19:52.189 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35505.log 2024-09-18T14:19:52.189 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49499.log.gz 2024-09-18T14:19:52.189 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.47027.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36036.log 2024-09-18T14:19:52.189 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46943.log 2024-09-18T14:19:52.189 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47027.log.gz 2024-09-18T14:19:52.189 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36036.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35505.log.gz 2024-09-18T14:19:52.189 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40049.log 2024-09-18T14:19:52.189 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38225.log 2024-09-18T14:19:52.189 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36036.log.gz 2024-09-18T14:19:52.189 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37763.log 2024-09-18T14:19:52.189 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46943.log.gz 2024-09-18T14:19:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.g.log 2024-09-18T14:19:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31853.log 2024-09-18T14:19:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mon.g.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31809.log 2024-09-18T14:19:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.31853.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-18T14:19:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.31809.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T14:19:52.214 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31853.log.gz 2024-09-18T14:19:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31893.log 2024-09-18T14:19:52.214 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31809.log.gz 2024-09-18T14:19:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.h.log 2024-09-18T14:19:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.31893.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.i.log 2024-09-18T14:19:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mon.h.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31893.log.gz 2024-09-18T14:19:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31937.log 2024-09-18T14:19:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31729.log 2024-09-18T14:19:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mon.i.log: /var/log/ceph/ceph-client.admin.31937.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-18T14:19:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.31729.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31937.log.gz 2024-09-18T14:19:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T14:19:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31729.log.gz 2024-09-18T14:19:52.215 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-18T14:19:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34663.log 2024-09-18T14:19:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mgr.x.log: 96.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T14:19:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.f.log 2024-09-18T14:19:52.231 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.34663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31769.log 2024-09-18T14:19:52.244 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mon.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-18T14:19:52.245 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.31769.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31769.log.gz 2024-09-18T14:19:52.275 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40049.log.gz 2024-09-18T14:19:52.275 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47153.log 2024-09-18T14:19:52.275 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35251.log 2024-09-18T14:19:52.275 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38225.log.gz 2024-09-18T14:19:52.275 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44257.log 2024-09-18T14:19:52.275 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37763.log.gz 2024-09-18T14:19:52.275 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47153.log.gz 2024-09-18T14:19:52.275 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47447.log 2024-09-18T14:19:52.276 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44257.log: gzip -5 87.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.35251.log.gz 2024-09-18T14:19:52.276 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.37553.log 2024-09-18T14:19:52.276 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48575.log 2024-09-18T14:19:52.276 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44257.log.gz 2024-09-18T14:19:52.276 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37553.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47447.log.gz 2024-09-18T14:19:52.276 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47995.log 2024-09-18T14:19:52.276 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46603.log 2024-09-18T14:19:52.276 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37553.log.gz 2024-09-18T14:19:52.276 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35794.log 2024-09-18T14:19:52.276 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48575.log.gz 2024-09-18T14:19:52.276 INFO:teuthology.orchestra.run.smithi045.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47995.log.gz 2024-09-18T14:19:52.276 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44971.log 2024-09-18T14:19:52.277 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36753.log 2024-09-18T14:19:52.277 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46603.log.gz 2024-09-18T14:19:52.277 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44971.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35794.log.gz 2024-09-18T14:19:52.277 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44593.log 2024-09-18T14:19:52.277 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36753.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44971.log.gz 2024-09-18T14:19:52.277 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47111.log 2024-09-18T14:19:52.277 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36753.log.gz 2024-09-18T14:19:52.277 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43535.log 2024-09-18T14:19:52.277 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45055.log 2024-09-18T14:19:52.277 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44593.log.gz 2024-09-18T14:19:52.277 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43535.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47111.log.gz 2024-09-18T14:19:52.277 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41760.log 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44635.log 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43535.log.gz 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38627.log 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45055.log.gz 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41760.log.gz 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43619.log 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37805.log 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44635.log.gz 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43619.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38627.log.gz 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37805.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43619.log.gz 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37511.log 2024-09-18T14:19:52.278 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38711.log 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37805.log.gz 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46985.log 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38711.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37511.log.gz 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39593.log 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38711.log.gz 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43325.log 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40301.log 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39593.log: /var/log/ceph/ceph-client.admin.43325.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46985.log.gz 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46183.log 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39593.log.gz 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40301.log: /var/log/ceph/ceph-client.admin.45973.log 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43325.log.gz 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35795.log 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40301.log.gz 2024-09-18T14:19:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.45973.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46183.log.gz 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35910.log 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47909.log 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45973.log.gz 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35795.log.gz 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47953.log 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47909.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35910.log.gz 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44089.log 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47909.log.gz 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44131.log 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47953.log: gzip -5/var/log/ceph/ceph-client.admin.44089.log: --verbose -- /var/log/ceph/ceph-client.admin.46099.log 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47953.log.gz 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44341.log 2024-09-18T14:19:52.280 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44089.log.gz 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48701.log 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44131.log.gz 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39005.log 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46099.log.gz 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40721.log 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48701.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44341.log.gz 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44887.log 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48701.log.gz 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41340.log 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39005.log.gz 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44887.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40721.log.gz 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36542.log 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44887.log.gz 2024-09-18T14:19:52.281 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46733.log 2024-09-18T14:19:52.282 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41340.log: /var/log/ceph/ceph-client.admin.36542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38351.log 2024-09-18T14:19:52.282 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41340.log.gz 2024-09-18T14:19:52.282 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46733.log: gzip -5 89.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.36542.log.gz -- 2024-09-18T14:19:52.282 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.45384.log 2024-09-18T14:19:52.282 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43921.log 2024-09-18T14:19:52.282 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38351.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46733.log.gz 2024-09-18T14:19:52.282 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49037.log 2024-09-18T14:19:52.282 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38351.log.gz 2024-09-18T14:19:52.282 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41382.log 2024-09-18T14:19:52.282 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43921.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45384.log.gz 2024-09-18T14:19:52.282 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35250.log 2024-09-18T14:19:52.282 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43921.log.gz 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42569.log 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49037.log.gz 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46561.log 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41382.log.gz 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43157.log 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.35250.log.gz 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48995.log 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42569.log.gz 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48323.log 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46561.log.gz 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47615.log 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43157.log.gz 2024-09-18T14:19:52.283 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0%/var/log/ceph/ceph-client.admin.48323.log: -- replaced with /var/log/ceph/ceph-client.admin.48995.log.gz 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41928.log 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37343.log 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47615.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48323.log.gz 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43837.log 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47615.log.gz 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36796.log 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41928.log.gz 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38795.log 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37343.log.gz 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37931.log 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43837.log.gz 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36796.log: /var/log/ceph/ceph-client.admin.38795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37721.log 2024-09-18T14:19:52.284 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4%/var/log/ceph/ceph-client.admin.37931.log: -- replaced with /var/log/ceph/ceph-client.admin.36796.log.gz 2024-09-18T14:19:52.285 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38795.log.gz 2024-09-18T14:19:52.285 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39089.log 2024-09-18T14:19:52.285 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40007.log 2024-09-18T14:19:52.285 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37721.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37931.log.gz 2024-09-18T14:19:52.285 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45300.log 2024-09-18T14:19:52.285 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37721.log.gz 2024-09-18T14:19:52.285 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39089.log.gz 2024-09-18T14:19:52.285 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40007.log: -- /var/log/ceph/ceph-client.admin.36922.log 2024-09-18T14:19:52.285 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43661.log 2024-09-18T14:19:52.285 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40007.log.gz 2024-09-18T14:19:52.285 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36922.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45300.log.gz 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39677.log 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35500.log 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43661.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36922.log.gz 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48117.log 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43661.log.gz/var/log/ceph/ceph-client.admin.35500.log: 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39257.log 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39677.log.gz 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35500.log.gz 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47489.log 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose 88.1% -- -- replaced with /var/log/ceph/ceph-client.admin.48117.log.gz /var/log/ceph/ceph-client.admin.48239.log 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42905.log 2024-09-18T14:19:52.286 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47489.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39257.log.gz 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42306.log 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47489.log.gz 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35028.log 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48239.log.gz 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42180.log 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42905.log.gz 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35994.log 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35028.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42306.log.gz 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38015.log 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35028.log.gz 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45888.log 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42180.log.gz 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38015.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35994.log.gz 2024-09-18T14:19:52.287 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39467.log 2024-09-18T14:19:52.288 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45888.log: -- /var/log/ceph/ceph-client.admin.46309.log 2024-09-18T14:19:52.288 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38015.log.gz 2024-09-18T14:19:52.288 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45216.log 2024-09-18T14:19:52.288 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45888.log.gz 2024-09-18T14:19:52.288 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46309.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39467.log.gz 2024-09-18T14:19:52.288 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40175.log 2024-09-18T14:19:52.288 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45468.log 2024-09-18T14:19:52.288 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45216.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46309.log.gz 2024-09-18T14:19:52.288 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35725.log 2024-09-18T14:19:52.288 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45216.log.gz 2024-09-18T14:19:52.288 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44383.log 2024-09-18T14:19:52.288 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40175.log.gz 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35725.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45468.log.gz 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40637.log 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35725.log.gz 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40511.log 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44383.log: /var/log/ceph/ceph-client.admin.40637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42264.log 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40511.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44383.log.gz 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42390.log 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40637.log.gz 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47783.log 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40511.log.gz 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47741.log 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42264.log.gz 2024-09-18T14:19:52.289 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40427.log 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47783.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42390.log.gz 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40259.log 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47783.log.gz 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47741.log.gz 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43879.log 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47195.log 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40259.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40427.log.gz 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43879.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40259.log.gz 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46141.log 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43879.log.gz 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40973.log 2024-09-18T14:19:52.290 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47195.log.gz 2024-09-18T14:19:52.291 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40679.log 2024-09-18T14:19:52.291 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40973.log: gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.46141.log.gz 2024-09-18T14:19:52.291 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37679.log 2024-09-18T14:19:52.291 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41802.log 2024-09-18T14:19:52.291 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40679.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40973.log.gz 2024-09-18T14:19:52.291 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42863.log 2024-09-18T14:19:52.291 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40679.log.gz 2024-09-18T14:19:52.291 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41802.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37679.log.gz 2024-09-18T14:19:52.291 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41508.log 2024-09-18T14:19:52.291 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41802.log.gz 2024-09-18T14:19:52.291 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44425.log 2024-09-18T14:19:52.291 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38921.log 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41508.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42863.log.gz 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37469.log 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41508.log.gz 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38921.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44425.log.gz 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39215.log 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38921.log.gz 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42222.log 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38435.log 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39215.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37469.log.gz 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37216.log 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39215.log.gz 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44173.log 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42222.log.gz 2024-09-18T14:19:52.292 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39803.log 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr: 96.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38435.log.gz 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49415.log 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37216.log.gz 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34886.log 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39803.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44173.log.gz 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45762.log 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34886.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39803.log.gz 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49415.log.gz 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40553.log 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34886.log.gz 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44677.log 2024-09-18T14:19:52.293 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45762.log: /var/log/ceph/ceph-client.admin.40553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38183.log 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45762.log.gz 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44677.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40553.log.gz 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39719.log 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36120.log 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38183.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44677.log.gz 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47867.log 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38183.log.gz 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36120.log: gzip 89.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.39719.log.gz -- 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.47321.log 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45342.log 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36120.log.gz 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48869.log 2024-09-18T14:19:52.294 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47867.log.gz 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36458.log 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45342.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47321.log.gz 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40091.log 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45342.log.gz 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49163.log 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48869.log.gz 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40091.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36458.log.gz 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45258.log 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47573.log 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49163.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40091.log.gz 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48827.log 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47573.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49163.log.gz 2024-09-18T14:19:52.295 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45258.log.gz 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48449.log 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47573.log.gz 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36078.log 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48827.log: /var/log/ceph/ceph-client.admin.48449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46649.log 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48827.log.gz 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40385.log 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48449.log.gz 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48911.log 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36078.log.gz 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35502.log 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46649.log.gz 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48911.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40385.log.gz 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35249.log 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36584.log 2024-09-18T14:19:52.296 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35502.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48911.log.gz 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40133.log 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35502.log.gz 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38669.log 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35249.log.gz 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48743.log 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36584.log.gz 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40133.log.gz 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35798.log 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45097.log 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48743.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38669.log.gz 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38753.log 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48743.log.gz 2024-09-18T14:19:52.297 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35798.log.gz 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45097.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44467.log 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45097.log.gz 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47363.log 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38753.log.gz 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44047.log 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47363.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44467.log.gz 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45636.log 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37006.log 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44047.log: 89.4%/var/log/ceph/ceph-client.admin.45636.log: -- replaced with /var/log/ceph/ceph-client.admin.47363.log.gz 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48953.log 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44047.log.gz 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37006.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45636.log.gz 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35952.log 2024-09-18T14:19:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41466.log 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48953.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37006.log.gz 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45804.log 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48953.log.gz 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47405.log 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35952.log.gz 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45804.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41466.log.gz 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38879.log 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45804.log.gz 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36626.log 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47405.log: /var/log/ceph/ceph-client.admin.38879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39047.log 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47405.log.gz 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36626.log: gzip 89.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.38879.log.gz --verbose 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr: -- /var/log/ceph/ceph-client.admin.47657.log 2024-09-18T14:19:52.299 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36162.log 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36626.log.gz 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39761.log 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39047.log.gz 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36162.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47657.log.gz 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37090.log 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36162.log.gz 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47279.log 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39761.log: /var/log/ceph/ceph-client.admin.37090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41141.log 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39761.log.gz 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47279.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37090.log.gz 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38309.log 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41141.log: gzip -5 --verbose 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47279.log.gz 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr: -- /var/log/ceph/ceph-client.admin.31926.log 2024-09-18T14:19:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38309.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41141.log.gz 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46519.log 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.31926.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38309.log.gz 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45720.log 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38393.log 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46519.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31926.log.gz 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47699.log 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46519.log.gz 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38393.log: gzip -5 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45720.log.gz 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48281.log 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47699.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46225.log 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38393.log.gz 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48281.log: /var/log/ceph/ceph-client.admin.37595.log 2024-09-18T14:19:52.301 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47699.log.gz 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46225.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48281.log.gz 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44845.log 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46225.log.gz 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40931.log 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37595.log.gz 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.44845.log: -5 --verbose -- /var/log/ceph/ceph-mon.e.log 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40931.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36500.log 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44845.log.gz 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40931.log.gz 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mon.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47531.log 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45552.log 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43199.log 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36500.log.gz 2024-09-18T14:19:52.302 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45552.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47531.log.gz 2024-09-18T14:19:52.303 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45594.log 2024-09-18T14:19:52.303 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43199.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45552.log.gz 2024-09-18T14:19:52.303 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36265.log 2024-09-18T14:19:52.303 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41550.log 2024-09-18T14:19:52.303 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43199.log.gz 2024-09-18T14:19:52.303 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45594.log: /var/log/ceph/ceph-client.admin.36265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34913.log 2024-09-18T14:19:52.303 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41550.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45594.log.gz 2024-09-18T14:19:52.303 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36265.log.gz 2024-09-18T14:19:52.303 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41550.log.gz 2024-09-18T14:19:52.303 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36711.log 2024-09-18T14:19:52.303 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39965.log 2024-09-18T14:19:52.303 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38057.log 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34913.log.gz 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42737.log 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1%/var/log/ceph/ceph-client.admin.39965.log: -- replaced with /var/log/ceph/ceph-client.admin.36711.log.gz 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41015.log 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39965.log.gz 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42737.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38057.log.gz 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42989.log 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42737.log.gz 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35070.log 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42989.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41015.log.gz 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43493.log 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41592.log 2024-09-18T14:19:52.304 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42989.log.gz 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35070.log: /var/log/ceph/ceph-client.admin.43493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44005.log 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41592.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35070.log.gz 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43493.log.gz 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41592.log.gz 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45013.log 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39635.log 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45013.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44005.log.gz 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39635.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45013.log.gz 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42611.log 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1%/var/log/ceph/ceph-osd.2.log: -- replaced with /var/log/ceph/ceph-client.admin.39635.log.gz 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38963.log 2024-09-18T14:19:52.305 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42611.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42611.log.gz 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44215.log 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42096.log 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44215.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38963.log.gz 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40217.log 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42096.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44215.log.gz 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42096.log.gz 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43115.log 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46435.log 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43115.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40217.log.gz 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42012.log 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43115.log.gz 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44719.log 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42012.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46435.log.gz 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42012.log.gz 2024-09-18T14:19:52.306 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42947.log 2024-09-18T14:19:52.307 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48785.log 2024-09-18T14:19:52.307 INFO:teuthology.orchestra.run.smithi045.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44719.log.gz/var/log/ceph/ceph-client.admin.42947.log: 2024-09-18T14:19:52.307 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42947.log.gz 2024-09-18T14:19:52.307 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42348.log 2024-09-18T14:19:52.307 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47825.log 2024-09-18T14:19:52.307 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48785.log: /var/log/ceph/ceph-client.admin.42348.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48785.log.gz 2024-09-18T14:19:52.307 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42348.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37427.log 2024-09-18T14:19:52.307 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:52.307 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47825.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47825.log.gz 2024-09-18T14:19:52.308 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-osd.3.log: 90.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T14:19:52.321 INFO:teuthology.orchestra.run.smithi084.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.0.34663.log.gz 2024-09-18T14:19:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43367.log 2024-09-18T14:19:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43577.log 2024-09-18T14:19:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37427.log: /var/log/ceph/ceph-client.admin.43367.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37427.log.gz 2024-09-18T14:19:52.324 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36964.log 2024-09-18T14:19:52.324 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43367.log.gz 2024-09-18T14:19:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43577.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43577.log.gz 2024-09-18T14:19:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T14:19:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35501.log 2024-09-18T14:19:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36964.log: /var/log/ceph/ceph.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36964.log.gz 2024-09-18T14:19:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41215.log 2024-09-18T14:19:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35501.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35501.log.gz 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41970.log 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41215.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41215.log.gz 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41099.log 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41970.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41970.log.gz 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42486.log 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41099.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41099.log.gz 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35252.log 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42486.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42486.log.gz 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42054.log 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35252.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35252.log.gz 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45930.log 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42054.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42054.log.gz 2024-09-18T14:19:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42653.log 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45930.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45930.log.gz 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46775.log 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42653.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42653.log.gz 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43073.log 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46775.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46775.log.gz 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46901.log 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43073.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43073.log.gz 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46057.log 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46901.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46901.log.gz 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44761.log 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46057.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46057.log.gz 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36838.log 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44761.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44761.log.gz 2024-09-18T14:19:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40595.log 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36838.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36838.log.gz 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42695.log 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40595.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40595.log.gz 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46267.log 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42695.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42695.log.gz 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45846.log 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46267.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46267.log.gz 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37301.log 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45846.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45846.log.gz 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41886.log 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37301.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37301.log.gz 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44929.log 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41886.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41886.log.gz 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40763.log 2024-09-18T14:19:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44929.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44929.log.gz 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44509.log 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40763.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40763.log.gz 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48037.log 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44509.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44509.log.gz 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39509.log 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48037.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48037.log.gz 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44551.log 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39509.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39509.log.gz 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32358.log 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44551.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44551.log.gz 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35658.log 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32358.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32358.log.gz 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39551.log 2024-09-18T14:19:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35658.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35658.log.gz 2024-09-18T14:19:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36331.log 2024-09-18T14:19:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39551.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39551.log.gz 2024-09-18T14:19:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39923.log 2024-09-18T14:19:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36331.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36331.log.gz 2024-09-18T14:19:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39341.log 2024-09-18T14:19:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39923.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39923.log.gz 2024-09-18T14:19:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48407.log 2024-09-18T14:19:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39341.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39341.log.gz 2024-09-18T14:19:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.d.log 2024-09-18T14:19:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48407.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48407.log.gz 2024-09-18T14:19:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43031.log 2024-09-18T14:19:52.335 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mon.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39383.log 2024-09-18T14:19:52.336 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43031.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43031.log.gz 2024-09-18T14:19:52.349 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45174.log 2024-09-18T14:19:52.350 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39383.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39383.log.gz 2024-09-18T14:19:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35506.log 2024-09-18T14:19:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45174.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45174.log.gz 2024-09-18T14:19:52.364 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39425.log 2024-09-18T14:19:52.365 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35506.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35506.log.gz 2024-09-18T14:19:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41718.log 2024-09-18T14:19:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39425.log: 90.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T14:19:52.386 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39425.log.gz 2024-09-18T14:19:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46393.log 2024-09-18T14:19:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41718.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41718.log.gz 2024-09-18T14:19:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37847.log 2024-09-18T14:19:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46393.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46393.log.gz 2024-09-18T14:19:52.390 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45678.log 2024-09-18T14:19:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37847.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37847.log.gz 2024-09-18T14:19:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43241.log 2024-09-18T14:19:52.393 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45678.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45678.log.gz 2024-09-18T14:19:52.393 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36372.log 2024-09-18T14:19:52.395 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43241.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43241.log.gz 2024-09-18T14:19:52.395 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35253.log 2024-09-18T14:19:52.396 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36372.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36372.log.gz 2024-09-18T14:19:52.396 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41634.log 2024-09-18T14:19:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35253.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.35253.log.gz 2024-09-18T14:19:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-18T14:19:52.399 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41634.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41634.log.gz 2024-09-18T14:19:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46015.log 2024-09-18T14:19:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49079.log 2024-09-18T14:19:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46015.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46015.log.gz 2024-09-18T14:19:52.427 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36204.log 2024-09-18T14:19:52.427 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49079.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49079.log.gz 2024-09-18T14:19:52.428 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27250.log 2024-09-18T14:19:52.442 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36204.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36204.log.gz 2024-09-18T14:19:52.442 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37048.log 2024-09-18T14:19:52.442 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.tmp-client.admin.27250.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27250.log.gz 2024-09-18T14:19:52.451 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48491.log 2024-09-18T14:19:52.453 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37048.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37048.log.gz 2024-09-18T14:19:52.461 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48659.log 2024-09-18T14:19:52.462 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48491.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48491.log.gz 2024-09-18T14:19:52.475 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41676.log 2024-09-18T14:19:52.476 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48659.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48659.log.gz 2024-09-18T14:19:52.484 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40343.log 2024-09-18T14:19:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46817.log 2024-09-18T14:19:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40343.log: 88.4% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41676.log.gz 2024-09-18T14:19:52.498 INFO:teuthology.orchestra.run.smithi084.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-18T14:19:52.502 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49539.log 2024-09-18T14:19:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46817.log: -- replaced with /var/log/ceph/ceph-client.admin.40343.log.gz 2024-09-18T14:19:52.514 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48365.log 2024-09-18T14:19:52.516 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49539.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49539.log.gz 2024-09-18T14:19:52.516 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46817.log.gz 2024-09-18T14:19:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41424.log 2024-09-18T14:19:52.529 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48365.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48365.log.gz 2024-09-18T14:19:52.541 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46477.log 2024-09-18T14:19:52.542 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41424.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41424.log.gz 2024-09-18T14:19:52.550 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37259.log 2024-09-18T14:19:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49205.log 2024-09-18T14:19:52.564 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37259.log: 88.2% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46477.log.gz 2024-09-18T14:19:52.568 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45510.log 2024-09-18T14:19:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49205.log: -- replaced with /var/log/ceph/ceph-client.admin.37259.log.gz 2024-09-18T14:19:52.582 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49205.log.gz 2024-09-18T14:19:52.586 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2024-09-18T14:19:52.586 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45510.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45510.log.gz 2024-09-18T14:19:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47069.log 2024-09-18T14:19:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35254.log 2024-09-18T14:19:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47069.log: 88.4% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2024-09-18T14:19:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47237.log 2024-09-18T14:19:52.617 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35254.log: -- replaced with /var/log/ceph/ceph-client.admin.47069.log.gz 2024-09-18T14:19:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46859.log 2024-09-18T14:19:52.626 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47237.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35254.log.gz 2024-09-18T14:19:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44803.log 2024-09-18T14:19:52.635 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46859.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47237.log.gz 2024-09-18T14:19:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35678.log 2024-09-18T14:19:52.644 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44803.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46859.log.gz 2024-09-18T14:19:52.648 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48617.log 2024-09-18T14:19:52.653 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35678.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44803.log.gz 2024-09-18T14:19:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46351.log 2024-09-18T14:19:52.662 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48617.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35678.log.gz 2024-09-18T14:19:52.671 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48617.log.gz 2024-09-18T14:19:52.674 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38837.log 2024-09-18T14:19:52.676 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46351.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46351.log.gz 2024-09-18T14:19:52.683 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36880.log 2024-09-18T14:19:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39299.log 2024-09-18T14:19:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36880.log: 88.7% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38837.log.gz 2024-09-18T14:19:52.706 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36880.log.gz 2024-09-18T14:19:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43795.log 2024-09-18T14:19:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39299.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39299.log.gz 2024-09-18T14:19:52.717 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34830.log 2024-09-18T14:19:52.726 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43703.log 2024-09-18T14:19:52.730 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34830.log: 88.2% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43795.log.gz 2024-09-18T14:19:52.730 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34830.log.gz 2024-09-18T14:19:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42821.log 2024-09-18T14:19:52.742 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43703.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43703.log.gz 2024-09-18T14:19:52.750 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37637.log 2024-09-18T14:19:52.760 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32275.log 2024-09-18T14:19:52.764 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37637.log: 89.3% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42821.log.gz 2024-09-18T14:19:52.768 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43283.log 2024-09-18T14:19:52.773 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32275.log: -- replaced with /var/log/ceph/ceph-client.admin.37637.log.gz 2024-09-18T14:19:52.777 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39173.log 2024-09-18T14:19:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43283.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32275.log.gz 2024-09-18T14:19:52.786 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46691.log 2024-09-18T14:19:52.791 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39173.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43283.log.gz 2024-09-18T14:19:52.797 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43963.log 2024-09-18T14:19:52.797 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46691.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39173.log.gz 2024-09-18T14:19:52.797 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46691.log.gz 2024-09-18T14:19:52.812 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49457.log 2024-09-18T14:19:52.813 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43963.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43963.log.gz 2024-09-18T14:19:52.821 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40889.log 2024-09-18T14:19:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35504.log 2024-09-18T14:19:52.835 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49457.log.gz 2024-09-18T14:19:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40889.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40889.log.gz 2024-09-18T14:19:52.844 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48533.log 2024-09-18T14:19:52.853 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40847.log 2024-09-18T14:19:52.855 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48533.log: 89.0% 86.8% -- replaced with /var/log/ceph/ceph-client.admin.35504.log.gz 2024-09-18T14:19:52.865 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43409.log 2024-09-18T14:19:52.866 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40847.log: -- replaced with /var/log/ceph/ceph-client.admin.48533.log.gz 2024-09-18T14:19:52.866 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40847.log.gz 2024-09-18T14:19:52.878 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38141.log 2024-09-18T14:19:52.880 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43409.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43409.log.gz 2024-09-18T14:19:52.886 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37889.log 2024-09-18T14:19:52.895 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38141.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49247.log 2024-09-18T14:19:52.899 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37889.log: -- replaced with /var/log/ceph/ceph-client.admin.38141.log.gz 2024-09-18T14:19:52.901 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37889.log.gz 2024-09-18T14:19:52.910 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40805.log 2024-09-18T14:19:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49247.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49247.log.gz 2024-09-18T14:19:52.919 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44299.log 2024-09-18T14:19:52.929 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49289.log 2024-09-18T14:19:52.933 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44299.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40805.log.gz 2024-09-18T14:19:52.938 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41844.log 2024-09-18T14:19:52.942 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49289.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44299.log.gz 2024-09-18T14:19:52.948 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49121.log 2024-09-18T14:19:52.949 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41844.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41844.log.gz 2024-09-18T14:19:52.951 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49289.log.gz 2024-09-18T14:19:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39131.log 2024-09-18T14:19:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49121.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49121.log.gz 2024-09-18T14:19:52.966 INFO:teuthology.orchestra.run.smithi084.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.g.log.gz 2024-09-18T14:19:52.973 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37973.log 2024-09-18T14:19:52.983 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42138.log 2024-09-18T14:19:52.987 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37973.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.39131.log.gz 2024-09-18T14:19:52.991 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37385.log 2024-09-18T14:19:52.996 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42138.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37973.log.gz 2024-09-18T14:19:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37385.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42138.log.gz 2024-09-18T14:19:53.008 INFO:teuthology.orchestra.run.smithi084.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.f.log.gz 2024-09-18T14:19:53.014 INFO:teuthology.orchestra.run.smithi045.stderr: 90.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-18T14:19:53.014 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37385.log.gz 2024-09-18T14:19:53.051 INFO:teuthology.orchestra.run.smithi084.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.i.log.gz 2024-09-18T14:19:53.190 INFO:teuthology.orchestra.run.smithi084.stderr: 89.9% -- replaced with /var/log/ceph/ceph-mon.h.log.gz 2024-09-18T14:19:53.302 INFO:teuthology.orchestra.run.smithi045.stderr: 90.0% -- replaced with /var/log/ceph/ceph-mon.d.log.gz 2024-09-18T14:19:53.378 INFO:teuthology.orchestra.run.smithi045.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-18T14:19:53.616 INFO:teuthology.orchestra.run.smithi045.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-18T14:19:54.482 INFO:teuthology.orchestra.run.smithi045.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.e.log.gz 2024-09-18T14:19:55.277 INFO:teuthology.orchestra.run.smithi084.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-18T14:19:55.291 INFO:teuthology.orchestra.run.smithi084.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-18T14:19:55.470 INFO:teuthology.orchestra.run.smithi045.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-18T14:19:56.164 INFO:teuthology.orchestra.run.smithi084.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-18T14:19:56.166 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-18T14:19:56.166 INFO:teuthology.orchestra.run.smithi084.stderr:real 0m3.975s 2024-09-18T14:19:56.166 INFO:teuthology.orchestra.run.smithi084.stderr:user 0m13.384s 2024-09-18T14:19:56.166 INFO:teuthology.orchestra.run.smithi084.stderr:sys 0m0.438s 2024-09-18T14:19:57.080 INFO:teuthology.orchestra.run.smithi045.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-18T14:19:57.291 INFO:teuthology.orchestra.run.smithi045.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-18T14:19:57.293 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-18T14:19:57.293 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m5.125s 2024-09-18T14:19:57.293 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m19.214s 2024-09-18T14:19:57.293 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m0.740s 2024-09-18T14:19:57.293 INFO:tasks.ceph:Archiving logs... 2024-09-18T14:19:57.294 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/remote/smithi045/log 2024-09-18T14:19:57.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T14:20:00.470 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/remote/smithi084/log 2024-09-18T14:20:00.471 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T14:20:02.593 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-18T14:20:02.602 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-18T14:20:02.603 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T14:20:02.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T14:20:02.683 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-18T14:20:02.692 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-18T14:20:02.692 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T14:20:02.695 DEBUG:teuthology.orchestra.run.smithi084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T14:20:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-09-18T14:20:02.711 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-09-18T14:20:02.714 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T14:20:02.714 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-09-18T14:20:02.714 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 +809us[ +690us] +/- 50ms 2024-09-18T14:20:02.714 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -1961us[-2080us] +/- 48ms 2024-09-18T14:20:02.714 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +1560us[+1440us] +/- 84ms 2024-09-18T14:20:02.714 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T14:20:02.715 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T14:20:02.715 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-09-18T14:20:02.716 INFO:teuthology.orchestra.run.smithi084.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 +830us[ +738us] +/- 50ms 2024-09-18T14:20:02.716 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -1933us[-2025us] +/- 49ms 2024-09-18T14:20:02.716 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +1586us[+1586us] +/- 84ms 2024-09-18T14:20:02.716 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T14:20:02.716 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-18T14:20:02.725 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-18T14:20:02.725 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-18T14:20:02.733 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T14:20:02.832 DEBUG:teuthology.orchestra.run.smithi084:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T14:20:02.902 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-18T14:20:02.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T14:20:02.933 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-18T14:20:02.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T14:20:02.963 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-18T14:20:02.972 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-18T14:20:02.981 INFO:teuthology.task.internal:Duration was 1661.380768 seconds 2024-09-18T14:20:02.981 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-18T14:20:02.990 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-18T14:20:02.990 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T14:20:02.993 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T14:20:03.032 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T14:20:03.044 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T14:20:03.339 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-18T14:20:03.339 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-09-18T14:20:03.339 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T14:20:03.361 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi084.front.sepia.ceph.com 2024-09-18T14:20:03.361 DEBUG:teuthology.orchestra.run.smithi084:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T14:20:03.405 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-18T14:20:03.405 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T14:20:03.408 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T14:20:03.474 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-18T14:20:03.475 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T14:20:03.477 DEBUG:teuthology.orchestra.run.smithi084:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T14:20:03.617 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-18T14:20:03.628 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-18T14:20:03.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T14:20:03.654 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T14:20:03.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-18T14:20:03.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T14:20:03.697 DEBUG:teuthology.orchestra.run.smithi084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T14:20:03.718 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-09-18T14:20:03.742 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = core 2024-09-18T14:20:03.758 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T14:20:03.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T14:20:03.791 DEBUG:teuthology.orchestra.run.smithi084:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T14:20:03.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T14:20:03.812 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-18T14:20:03.822 INFO:teuthology.task.internal:Transferring archived files... 2024-09-18T14:20:03.823 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/remote/smithi045 2024-09-18T14:20:03.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T14:20:03.866 DEBUG:teuthology.misc:Transferring archived files from smithi084:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7911070/remote/smithi084 2024-09-18T14:20:03.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T14:20:03.898 INFO:teuthology.task.internal:Removing archive directory... 2024-09-18T14:20:03.899 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T14:20:03.903 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T14:20:03.948 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-18T14:20:03.959 INFO:teuthology.task.internal:Not uploading archives. 2024-09-18T14:20:03.959 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-18T14:20:03.969 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-18T14:20:03.969 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T14:20:03.971 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T14:20:03.984 INFO:teuthology.orchestra.run.smithi045.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 14:20 /home/ubuntu/cephtest 2024-09-18T14:20:04.003 INFO:teuthology.orchestra.run.smithi084.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 14:20 /home/ubuntu/cephtest 2024-09-18T14:20:04.004 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-18T14:20:04.015 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-18T14:20:04.037 INFO:teuthology.run:Summary data: description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/snaps-few-objects} duration: 1661.3807678222656 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-18T14:20:04.037 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T14:20:04.126 INFO:teuthology.run:pass